--- title: "User Guide: 1 Radiation" subtitle: "Package 'photobiology' `r packageVersion('photobiology')` " author: "Pedro J. Aphalo" date: "`r packageDate('photobiology')`" output: rmarkdown::html_vignette: toc: yes vignette: > %\VignetteEncoding{UTF-8} %\VignetteIndexEntry{User Guide: 1 Radiation} %\VignetteEngine{knitr::rmarkdown} editor_options: markdown: wrap: 72 --- ## Radiation, astronomy and atmosphere The User Guide was earlier divided into two parts: *1. Radiation* and *2. Astronomy*. The astronomy related functions and the [former part 2 of this User Guide](https://docs.r4photobiology.info/SunCalcMeeus/articles/userguide-1-astronomy.html) are now in package 'SunCalcMeeus'. This User Guide is the updated version of former part 1. ## Naming conventions I have developed package '**photobiology**' over several years, and even though I have tried to remain consistent, some inconsistencies have crept in. I have attempted to correct inconsistencies without breaking old code by providing synonyms for some functions. In general I have used names in *snake case* for methods and functions, such as `source_spct()` and *dot case* for their arguments. There are two exceptions to this: the `as.` and `is.` methods which for consistency with base R, retain the dot: `as.source_spct()` and `ìs.source_spct()`. There are still some functions and methods whose names are in *camel case*: these are mostly accessors and setters of metadata attributes such as `setWhatMeasured()` and `getWhatMeasured()`, in which case synonyms `what_measured<-()` and `what_measured()` are now provided with a syntax that parallels that of R's `comment<-()`, `comment()`, and similar methods for other attributes. Names of attributes themselves use dots, while classes use snake case as functions do, allowing consistency between class names and their constructor functions. There is an additional twist: functions named like `clip_wl()` are "verbs" that act on the wavelengths ("wl") modifying them, while those named like `wl_range()` are "nouns" for the properties of the wavelengths. In examples we use variable names ending in `.spct` for spectra, ending in `.mspct` for collections of spectra and ending in `.tb` for tibbles. In most cases the root of the variable names use "snake case" and may have a tag at the end separated with a dot. In the case of variable names used in examples I have been less strict about conventions than for the objects exported by the package. None of these naming conventions are enforced for user defined objects and classes; these conventions are simply those I have tried to follow while developing this package as well as other packages in the r4photobiology suite. ## High and low level functions The package exports two sets of functions for many operations: low-level functions programmed following a functional paradigm, and higher-level functions using an object-oriented paradigm. The former functions take as arguments numeric vectors and are sometimes faster. The later ones take *spectra* objects as arguments, are easier to use, and at least at the moment, to some extent slower. For everyday use, methods for *spectra* objects are recommended, but when maximum performance or flexibility in scripts is desired, the use of the functions taking numeric vectors as arguments may allow optimizations that are not possible with the object-oriented functions. The differences in performance becomes relevant only in extreme cases such as processing in a single script tenths of thousands of spectra. In the vignettes we emphasize the use of the object-oriented classes and methods. ## Example data Data for several spectra are included for use in examples and vignettes, and in testing (Tables 1 and 2). Other packages in the '**r4photobiology suite**' provide many additional data sets. ------------------------------------------------------------------------ **Table 1. Data sets included in the package: spectra.** The CIE standard illuminant data in this package are normalized to one at $\lambda = 560\,$nm, while in the CIE standard they are normalized to 100 at the same wavelength. See documentation for each data object for details and data sources. | Object | class | units | data description | |:------------------------|:-----------------|:-----------------------------|:---------------------------------| | `sun.spct` | `source_spct` | $W\,m^{-2}\,nm^{-1}$ | solar spectral irradiance | | `sun_evening.spct` | `source_spct` | $W\,m^{-2}\,nm^{-1}$ | solar spectral irradiance | | `sun_evening.mspct` | `source_mspct` | $W\,m^{-2}\,nm^{-1}$ | solar spectral irradiance | | `sun_daily.spct` | `source_spct` | $J\,m^{-2}\,d^{-1}\,nm^{-1}$ | solar spectral exposure | | `sun.data` | `data.frame` | $W\,m^{-2}\,nm^{-1}$ | solar spectral irradiance | | `sun_daily.data` | `data.frame` | $J\,m^{-2}\,d^{-1}\,nm^{-1}$ | solar spectral exposure | | `D65.illuminant.spct` | `source_spct` | (norm. 560 nm) | CIE standard | | `A.illuminant.spct` | `source_spct` | (norm. 560 nm) | CIE standard | | `clear.spct` | `filter_spct` | fraction | ideal transparent filter | | `opaque.spct` | `filter_spct` | fraction | ideal opaque filter | | `polyester.spct` | `filter_spct` | fraction | plastic film | | `yellow_gel.spct` | `filter_spct` | fraction | theatrical *gel* filter | | `two_filters.spct` | `filter_spct` | fraction | theatrical *gel* filter | | `two_filters.mspct` | `filter_mspct` | fraction | theatrical *gel* filter | | `green_leaf.spct` | `reflector_spct` | fraction | a birch leaf | | `clear_body.spct` | `object_spct` | fraction | ideal transparent body | | `black_body.spct` | `object_spct` | fraction | ideal black body | | `white_body.spct` | `object_spct` | fraction | ideal white body | | `Ler_leaf.spct` | `object_spct` | fraction | Arabidopsis leaf | | `Ler_leaf_trns.spct` | `filter_spct` | fraction | Arabidopsis leaf (T total) | | `Ler_leaf_trns_i.spct` | `filter_spct` | fraction | Arabidopsis leaf (T internal) | | `Ler_leaf_rflt.spct` | `reflector_spct` | fraction | Arabidopsis leaf (R total) | | `water.spct` | `solute_spct` | $m^{-1}$ | pure water molar att. coef. | | `phenylalanine` | `solute_spct` | $m^{-1}$ | phenylalanine molar att. coef. | | `photodiode.spct` | `response_spct` | $A / W$ | typical Si photodiode | | `ccd.spct` | `response_spct` | $A / W$ | typical CCD array | | `two_sensors.spct` | `response_spct` | $A / W$ | typical CCD array | | `two_sensors.mspct` | `response_mspct` | $A / W$ | typical CCD array | | `white_led.raw_spct` | `raw_spct` | counts | example raw detector counts data | | `white_led.cps_spct` | `cps_spct` | counts / s | example detector counts data | | `white_led.source_spct` | `source_spct` | $W\,m^{-2}\,nm^{-1}$ | spectral irradiance | | `filter_cps.mspct` | `cps_mspct` | $\mathrm{counts} / s$ | example detector counts data | ------------------------------------------------------------------------ ------------------------------------------------------------------------ **Table 2. Data sets included in the package: chromaticity data.** See documentation for each data object for details and data sources. | Object | class | data description | |:--------------|:--------------|:-----------------------------------------| | `ciexyzCC2.spct` | `chroma_spct` | human chromaticity coordinates $2^\circ$ | | `ciexyzCC10.spct` | `chroma_spct` | human chromaticity coordinates $10^\circ$ | | `ciexyzCMF2.spct` | `chroma_spct` | human colour matching function $2^\circ$ | | `ciexyzCMF10.spct` | `chroma_spct` | human colour matching function $10^\circ$ | | `ciev2.spct` | `chroma_spct` | human luminous efficiency $2^\circ$ | | `beesxyzCMF.spct` | `chroma_spct` | bee colour matching function | ------------------------------------------------------------------------ ```{r, include=FALSE, echo=FALSE} knitr::opts_chunk$set(fig.width=7.2, fig.height=4.3) ``` ```{r, printing-spectra, eval=TRUE, include=FALSE} # library(tibble) options(tibble.print_max = 6, tibble.print_min = 4) ``` ## Spectra Package '**photobiology**' defines a family of classes based on the `tibble` class, mostly compatible with R's data frames. The present package by imposing some restrictions on the naming of the member vectors, allows methods to *find* the data when passed one of these objects as argument. In addition, as the data are checked when the object is built or modified, there is no need to test for their validity each time a calculation is carried out. Another advantage of using spectrum objects, is that specialized versions of generic functions like `print` and operators like `+` are defined for spectra. Objects of all the `..._spct` classes are also `tibble` and `data.frame` objects, as a result of how classes have been derived. In this package we define a *generic* or *base* spectrum class, derived from `tibble`, which in turn has been derived from `data.frame`. Classes for specialized types of spectra are derived from `generic_spct`. This *parenthood* hierarchy means that spectral objects can be used almost anywhere where a `data.frame` is expected (and also converted into data frames with R's function `as.data.frame()`). Specializations of many methods including extraction (indexing) methods and partial assignment methods are defined to ensure that the expectations on the variables contained in objects of these classes remain valid in most situations. Other specializations of methods and functions are related to achieving a convenient and concise syntax tailored to spectral data as in the case of mathematical operators and functions. Another important aspect is that when spectral data are stored in objects of these classes, the physical quantities and units of expression are also stored as metadata. Furthermore, attributes are used to keep track of both metadata related to the origin of the data and of later transformations that affect their interpretation, such as normalization or re-scaling. **Although sanity tests are applied at the time of object creation, to a large extent the responsibility of ensuring that the numbers provided as argument to object constructors comply with expectations remains with the users of the package.** In addition to the classes for storing individual spectra or multiple spectra in long form, classes for storing collections of these spectral objects are defined. These classes are derived from class `list` and can contain member spectra of different lengths and measured at different wavelength values. We give in this vignette brief descriptions and examples of the use of different classes, methods, functions and operators. We start with the simplest and most frequently used methods. ### Getting started We load two packages in addition to '**photobiology**', '**lubridate**' and '**dplyr**', as they will be used in the examples. ```{r, pkg-load, eval=TRUE, message = FALSE} library(photobiology) library(lubridate) library(dplyr) ``` ### Classes Package '**photobiology**' defines several classes intended to be used to store different types of spectral data. They are all derived from `generic_spct`, which in turn is derived from `tibble::tibble`. Table 1 lists these classes. Attributes are used to store *metadata* such as information about units of expression. ------------------------------------------------------------------------ **Table 1. Classes for spectral data.** In addition to the required variables listed in the table, additional arbitrary variables are partly supported---some operations will not include them in returned values to avoid ambiguity and other possible conflicts. In addition to the attributes listed in the table, all spectral objects support attributes `multiple.wl`, `idfactor`, `normalized`, `scaled`, `when.measured`, `where.measured`, `what.measured`, `how.measured` plus the normal attributes of `tibble` (and `data.frame`) objects including `comment`. All these attributes plus attributes `instrument.descriptor` and `instrument.settings` are retained across operations on spectra as long as they remain valid. | Class name | Required variables | Attributes | |:----------------|:------------------|:-----------------------------------| | `generic_spct` | `w.length` | | | `calib ration_spct` | `w.length`, `irrad.mult` | `instr.desc` | | `raw_spct` | `w.length`, `counts` | `instr.desc`, `instr.settings`, `linearized` | | `cps_spct` | `w.length`, `cps` | `instr.desc`, `instr.settings`, `linearized` | | `source_spct` | `w.length`, `s.e.irrad` | `instr.desc`, `instr.settings`, `time.unit`, `bswf` | | | `w.length`, `s.q.irrad` | `time.unit`, `bswf` | | `filter_spct` | `w.length`, `Tfr` | `Tfr.type`, `filter.properties` | | | `w.length`, `A` | `Tfr.type`, `filter.roperties` | | `reflector_spct` | `w.length`, `Rfr` | `Rfr.type` | | `object_spct` | `w.length`, `Tfr`, `Rfr` | `Tfr.type`, `Rfr.type` | | `response_spct` | `w.length`, `s.e.response` | `time.unit` | | | `w.length`, `s.q.response` | `time.unit` | | `chroma_spct` | `w.length`, `x`, `y`, `z` | | | `solute_spct` | `w.length`, `K.mole` | `K.type`, `solute.properties` | | | `w.length`, `K.mass` | `K.type`, `solute.properties` | ------------------------------------------------------------------------ The *design* imposes the restriction that data from different observations are never present as different *data columns*, if present, additional data columns represent different properties from the same observation event. In other words, the storage format is *tidy* as nicknamed by Hadley Wickham. In most cases, one spectral object correspond to one spectral observation, but most functions and methods are compatible or can be used to create spectral objects where the spectral data from different observations are stored "longitudinally" and "tagged" with a factor with a level for each observation event. These observations must use consistent units of expression. This long format is useful, for example, when producing plots with package 'ggplot2' or when acquiring or storing time series of spectra. If spectra are stored in *long form*, e.g., for plotting with 'ggplot2', attributes are stored as named lists or enforced to be the same across spectra. This allows to reconstruct a collection of spectra from a long-form spectral object and *vice versa* preserving nearly all of the metadata. In 'photobiology' (<= 0.11.0) the support for these conversions was limited and spectral objects with multiple spectra in long form created with these versions lack complete metadata or have the metadata stored in a way that cannot be easily retrieved, leading to metadata loss on conversion to collections of spectra. ------------------------------------------------------------------------ Initially 'photobiology' supported only the storage of a single spectrum per object. Later the possibility of storing multiple spectra in a list-like collection was implemented, and more recently storage of multiple spectra in long form was made possible. This progression was the result of a shift in the number of spectra being analysed, starting from individual spectra early on to hundreds of thousands or even millions of spectra nowadays. Each format has advantages and disadvantages. Spectra in long-form are a recent addition and some methods rely on conversion into collections of spectra. This approach can be slow when many spectra are involved, as memory allocation and copy is done during the conversion. However, some of the most frequently used operations have now been optimised for performance on spectra in long form by avoiding the conversion. **Rule of thumb** Collections of spectra are the only option when different members belong to different classes. Collections of spectra are the best approach when spectra are to be extracted from the collection or operated upon or plotted individually. Storage in long form can consistently save some storage space but can either decrease or increase execution time depending on the operation (and package version). Such advantages for storage in long form towards performance and memory use are relevant only when many spectra are to be operated upon consistently, say compute the same irradiance summaries from a time series of many thousands of spectra. ------------------------------------------------------------------------ ### Data assumptions A key assumption of the package is that wavelengths are always expressed in manometers ($1~\mathrm{nm} = 1 \cdot 10^{-9}\,\mathrm{m}$). If the data to be analysed use different units for wavelengths, e.g. Ångstrom ($1~\textrm{Å} = 1 \cdot 10^{-10}\,\mathrm{m}$), the values need to be re-expressed before creating objects of the spectral classes. The same applies to all spectral quantities, as there is an expectation in every case, of using base SI units for expression. Table 2 lists the units of expression for the different variables and the metadata attributes that may determine variations in the expression of the quantities. ------------------------------------------------------------------------ **Table 2. Variables used for spectral data and their units of expression.** **A:** as stored in objects of the spectral classes, **B:** also recognized by the `set` family of functions for spectra and automatically converted. `time.unit` accepts in addition to the character strings listed in the table, objects of classes `lubridate::duration` and `period`, in addition `numeric` values are interpreted as seconds. `exposure.time` accepts these same values, but not the character strings. | Variables | Unit of expression | Attribute value | |:---------------------|:------------------------------|:------------------| | **A: stored** | | | | w.length | nm | | | counts | $n$ | | | cps | $n\,s^{-1}$ | | | irrad.mult | $J\,m^{-2}\,nm^{-1}\,n^{-1}$ | | | s.e.irrad | $W\,m^{-2}\,nm^{-1}$ | time.unit = "second" | | s.e.irrad | $J\,m^{-2}\,d^{-1}\,nm^{-1}$ | time.unit = "day" | | s.e.irrad | $J\,m^{-2}\,nm^{-1}$ | time.unit = "exposure" | | s.e.irrad | varies | time.unit = *duration* | | s.q.irrad | $mol\,m^{-2}\,s^{-1}\,nm^{-1}$ | time.unit = "second" | | s.q.irrad | $mol\,m^{-2}\,d^{-1}\,nm^{-1}$ | time.unit = "day" | | s.q.irrad | $mol\,m^{-2}\,nm^{-1}$ | time.unit = "exposure" | | s.q.irrad | varies | time.unit = *duration* | | Tfr | [0..1] | Tfr.type = "total" | | Tfr | [0..1] | Tfr.type = "internal" | | A | a.u. | | | Afr | [0..1] | | | Rfr | [0..1] | Rfr.type = "total" | | Rfr | [0..1] | Rfr.type = "specular" | | s.e.response | $\mathit{x}\,J^{-1}\,s^{-1}\,nm^{-1}$ | time.unit = "second" | | s.e.response | $\mathit{x}\,mol^{-1}\,d^{-1}\,nm^{-1}$ | time.unit = "day" | | s.e.response | $\mathit{x}\,J^{-1}\,nm^{-1}$ | time.unit = "exposure" | | s.e.response | varies | time.unit = *duration* | | s.q.response | $\mathit{x}\,mol^{-1}\,s^{-1}\,nm^{-1}$ | time.unit = "second" | | s.q.response | $\mathit{x}\,mol^{-1}\,d^{-1}\,nm^{-1}$ | time.unit = "day" | | s.q.response | $\mathit{x}\,mol^{-1}\,nm^{-1}$ | time.unit = "exposure" | | s.q.response | varies | time.unit = *duration* | | x, y, z | [0,1] | | | K.mole | $m^{2}\,mol^{-1}, \log_{10}$ based | K.type = "attenuation" | | K.mole | $m^{2}\,mol^{-1}, \log_{10}$ based | K.type = "absorption" | | K.mole | $m^{2}\,mol^{-1}, \log_{10}$ based | K.type = "scattering" | | K.mass | $m^{2}\,g^{-1}, \log_{10}$ based | K.type = "attenuation" | | K.mass | $m^{2}\,g^{-1}, \log_{10}$ based | K.type = "absorption" | | K.mass | $m^{2}\,g^{-1}, \log_{10}$ based | K.type = "scattering" | | **B: converted** | | | | wl $\to$ w.length | nm | | | wavelength $\to$ w.length | nm | | | Tpc $\to$ Tfr | [0..100] | Tfr.type = "total" | | Tpc $\to$ Tfr | [0..100] | Tfr.type = "internal" | | Rpc $\to$ Rfr | [0..100] | Rfr.type = "total" | | Rpc $\to$ Rfr | [0..100] | Rfr.type = "specular" | | counts.per.second $\to$ cps | $n\,s^{-1}$ | | | K.mole $log_e \to log_{10}$ | $m^{-1}\,mol^{-1}, \log_{e}$ based | K.type = "attenuation" | | K.mole $log_e \to log_{10}$ | $m^{-1}\,mol^{-1}, \log_{e}$ based | K.type = "absorption" | | K.mole $log_e \to log_{10}$ | $m^{-1}\,mol^{-1}, \log_{e}$ based | K.type = "scattering" | | K.mass $log_e \to log_{10}$ | $m^{-1}\,g^{-1}, \log_{e}$ based | K.type = "attenuation" | | K.mass $log_e \to log_{10}$ | $m^{-1}\,g^{-1}, \log_{e}$ based | K.type = "absorption" | | K.mass $log_e \to log_{10}$ | $m^{-1}\,g^{-1}, \log_{e}$ based | K.type = "scattering" | ------------------------------------------------------------------------ Energy irradiance is assumed to be expressed in $W\,m^{-2}$ and photon irradiance in $mol^{-1}\,m^{-2}\,s^{-1}$, that is to say using second as unit for time and no SI scale factors for the measured variables. With respect to time, second is the default, but it is possible to set the unit for time to an arbitrary time duration such as day. Obviously, this applies only to rate variables like irradiance and response rates, but not to time-invariant intensive properties like transmittance of filters. The default time unit used is *second*, but *minute*, *hour*, *day* and *exposure* can be used by supplying as arguments `"minute"`, `"hour"`, `"day"`, or `"exposure"`---The meaning of `"exposure"` is the total exposure time, in other words, fluence instead of irradiance---to the constructor `source_spct()`. In addition to these character constants objects of class `lubridate:duration` are also accepted. In other words irradiance is a flux rate while exposure is a flux, as both are expressed per unit of illuminated area. In the case of spectra of the molar attenuation coefficient we use $\log_{10}$ as a base for the stored values but the constructor accepts data expressed in other log bases. The expected units are always in metres ($m^{-1}$) rather than the frequently used centimetres ($cm^{-1}$). Transmittance, reflectance and absorptance are stored as fractions of one. The constructors also accept these quantities expressed as percentages, and convert them. Most attributes are set when a spectral object is created, either using default values or with values supplied as arguments to the constructor. In many cases the default is `NA` used to indicate missing information. Methods for querying and setting these attributes in already constructed objects are also available. ------------------------------------------------------------------------ Not respecting the expectations about data inputs or setting erroneous values in the metadata attributes will yield completely wrong results if calculations are attempted! It is extremely important to make sure that the wavelengths are in nanometres as this is what all functions expect. If wavelength values are in the wrong units, the action-spectra weights and quantum to/from energy units conversions will be wrongly calculated, and the values returned by most functions wrong, without warning. Errors in some cases will be triggered at the time of object creation as the data input to constructors is tested to be within the expected range of values, which in the case of some quantities frequently allows detection of mistakes in the use unit scaling factors. ------------------------------------------------------------------------ If spectral irradiance data is in $W\,m^{-2}\,nm^{-1}$, and the wavelength in $nm$, as is the case for many Macam spectroradiometers, the data can be used directly and functions in the package will return irradiances in $W\,m^{-2}$. If, for example, the spectral irradiance data output by a spectroradiometer is expressed in $mW\,m^{-2}\,nm^{-1}$, and the wavelengths are in Ångstrom then to obtain correct results when using any of the packages in the suite, we need to rescale the data when creating a new object. ```{r, example-1, eval=FALSE} # not run my.spct <- source_spct(w.length = wavelength/10, s.e.irrad = irrad/1000) ``` In the example above, we take advantage of the behaviour of the R language: an operation between a scalar and a vector, is equivalent to applying this operation to each element of the vector. Consequently, in the code above, each value from the vector of wavelengths is divided by 10, and each value in the vector of spectral irradiances is divided by 1000. ### Querying the class Before giving examples of how to construct objects to store spectral data we show how to query the class of an object, and how to query the class of a spectrum. Consistently with R, the package provides *is* functions for querying the type of spectra objects. The only *unusual* function name, defined as a *synonym* for `is.generic_spct`: `is.any_spct()`. ```{r, query-class-1} is.any_spct(sun.spct) is.generic_spct(sun.spct) is.source_spct(sun.spct) is.data.frame(sun.spct) ``` In addition function `class_spct()` returns directly the spectrum-related class attributes---i.e. it filters out from the output of `class()` the underlying inherited classes. ```{r, query-class-2} class_spct(sun.spct) class(sun.spct) ``` ### Construction There are two different approaches to the creation of spectral objects by users. The first group are constructors similar to the `data.frame()` constructor, which take vectors as arguments. The second group are constructors that convert `list` objects, and because of class derivation also `data.frame` objects, into spectral objects. Constructors in this second group are similar to `as.data.frame` from base R. In contrast to the data frame constructors, spectral object constructors require the variables or the vector arguments to be suitably named so that they can be recognized. Here we briefly describe the *as* constructor functions for spectra. In the first example we create an object to store spectral irradiance data for a *fictitious light source*, by first creating a data frame, and creating the spectral object as a copy of it. In the example below we supply a single value, 1, for the spectral irradiance. This value gets recycled as is normal in R, but of course in real use it is more usual to supply a vector of the same length as the `w.length` vector. ```{r, construction-1} my.df <- data.frame(w.length = 400:410, s.e.irrad = 1) my.spct <- as.source_spct(my.df) class(my.spct) class(my.df) my.spct ``` We can copy and convert any spectrum object into any of its base clases, such as `generic_spct` or `data.frame`. This can result in the immediate and surely in the delayed loss of metadata when modified. In addition, objects *downgraded* to a base class support fewer operations and those still available may behave differently. ```{r, construction-2} my.g.spct <- as.generic_spct(my.spct) class(my.g.spct) ``` When constructing spectral objects from numeric vectors the names of the arguments are meaningful and convey information on the nature of the spectral data and basis of expression. ```{r, construction-3} source_spct(w.length = 300:305, s.e.irrad = 1) ``` ```{r, construction-4} z <- 300:305 y <- 2 source_spct(w.length = z, s.e.irrad = y) ``` When argument names are not supplied explicitly as above, the names of the variables are used to identify the data vectors. ```{r, construction-5} w.length <- 300:305 s.e.irrad <- 1 source_spct(w.length, s.e.irrad) ``` The different constructors have additional arguments to be used in setting non-default values for the attributes. These arguments have the same name as the attributes. Here we used the data frame created in the first code chunk of this section. ```{r, construction-6} my.d.spct <- as.source_spct(my.df, time.unit = "day") ``` Argument `strict.range` can be used to override or make more strict the validation of the data values. ```{r, construction-7} source_spct(w.length = 300:305, s.e.irrad = -1) source_spct(w.length = 300:305, s.e.irrad = -1, strict.range = NULL) ``` Argument `comment` can be used to add a comment to the data at the time of construction. ```{r, construction-8} my.cm.spct <- source_spct(w.length = 300:305, s.e.irrad = 1, comment = "This is a comment") comment(my.cm.spct) ``` The constructors treat unrecognized named arguments as data to create aditional columns in the spectral objects. ### Metadata attributes Metadata attributes are used in the spectral objects to store metadata in a consistently. These metadata are in some cases required for conversion among related physical quantities, while in other cases allow printing of ancillary information needed for interpretation, like units of expression. These metadata are also used in other packages in the suite, for example in 'ggspectra' to automatically produce axis labels, titles and annotations. A few attributes are simply a way or organizing the storage of information which is not used in any calculations, functioning as a kind of specialized comments. The metadata described in this section are stored in spectral objects using attributes, which are a normal feature of the R language. An example, is attribute `time.unit` used to indicate if spectral irradiance is expressed per second or integrated over some other time duration. #### Attributes supported by all spectral objects Some attributes are meaningful for all the classes of spectra defined in the package, while most others a specific to individual classes (Table 1). Those that apply to all spectral objects and their summaries are *time of measurement* using attribute `"when.measured"`, *place of measurement* using attribute `"where.measured"`, a *user supplied label* using attribute `"what.measured"`, a label describing origin of the data using attribute `"how.measured"` and free-text *comments*. One can set and get comments stored in spectra by means of base R's `comment()` and `comment() <-` functions and the other attributes listed above with functions following the same syntax and named after the attributes but replacing any `.` by `_`. Functions in this package may set additional attributes to keep track of the actions. For example when a spectrum is normalized or scaled a record of these action is kept in attributes. When spectra are are operated upon the metadata that is not invalidated will be merged when possible---e.g., comments of operands are concatenated comments and set as comment to the returned object. Functions `when_measured()` and `when_measured<-()` are used for retrieving and setting the `"when.measured"` attribute to a date supplied as a `POSIXct` value. Note: we use `POSIXct` objects which describes instants in time in absolute terms as they include time zone information. Package `lubridate` makes entering and operating on `POSIXct` objects rather easy. ```{r, attr-1} my.spct <- sun.spct when_measured(my.spct) <- NULL when_measured(my.spct) when_measured(my.spct) <- lubridate::ymd_hms("2015-10-31 22:55:00", tz = "Europe/Helsinki") when_measured(my.spct) ``` Functions `where_measured()` and `where_measured<-()` are used for retrieving and setting a geocode stored in a `data.frame`. This format is compatible with function `geocode()` from package `ggmap`. We pass latitude and longitude coordinates, as shown below. The returned value is always a data frame with columns `"lon"`, `"lat"` and `"address"`. ```{r, attr-2} where_measured(my.spct) <- NULL where_measured(my.spct) where_measured(my.spct) <- data.frame(lat = 60, lon = -10) where_measured(my.spct) where_measured(my.spct) <- data.frame(lat = 60, lon = -10, address = "Somewhere") where_measured(my.spct) my.spct ``` Functions `what_measured()` and `where_measured<-()`, and `how_measured()` and `how_measured<-()` are used for retrieving or setting a text value containing information about what was measured to obtain the data. ```{r, attr-2a} what_measured(my.spct) <- "something" what_measured(my.spct) my.spct ``` Functions using a different syntax are also available, for these and other attributes which are not likely to be set by users. Many of these additional attributes are meaningful only for some types of spectra. #### Attributes supported by `source_spct` objects One example is the time unit used to express spectral irradiance. Functions are available for querying and setting the state if these attributes. `is_` functions return a logical value, and `get` functions return the values of the attributes themselves. In addition `set` functions can be used to set the value stored in the attributes. Several of the `set` functions are very rarely needed in user code, as these attributes are set during construction or as a side effect of applying other functions and/or operators to the objects. Function `setBSWFUsed()` and other *set* functions are mainly useful to programmers extending the package, but only exceptionally to users. One exception is the case when a wrong value has been assigned by mistake and needs to be overwritten. We can see in the printout that the time unit is reported in the header. ```{r} sun.spct ``` For example function `is_effective()` returns `TRUE` if the spectral data has been weighted with a BSWF. The corresponding `getBSWFUsed()` function can be used, in this case to retrieve the name of the BSWF that was used. Here we demonstrate with one example, where we use a `waveband` object---constructed on-the-fly with a constructor function---, defining a range of wavelengths. ```{r, attr-3} is_effective(sun.spct) is_effective(sun.spct * waveband(c(400, 700))) ``` Sometimes it may be desired to change the time unit used for expressing spectral irradiance or spectral response, and this can be achieved with the *conversion* function `convertTimeUnit`. This function both converts spectral data to the new unit of expression and sets the `time.unit` attribute, preserving the validity of the data object. ```{r, attr-4} ten.minutes.spct <- convertTimeUnit(sun.spct, time.unit = duration(10, "minutes")) ten.minutes.spct getTimeUnit(ten.minutes.spct) ``` #### Attributes supported by `filter_spct` objects A crucial information is whether transmittance (`Tfr`) is expressed as internal or total, stored in attribute `Tfr.type`, as this affects how absorptances and absorbances are computed. The key step is when a `filter_spct` object is created, when the user has to be careful to set this attribute correctly. The objects from which transmittance can differ in additional properties that affect possible calculations. Attribute `filter.properties` is used to store these in an object with fields `Rfr.constant`, `thickness` and `attenuation.mode`. When these metadata are available, in many cases we can use function `convertTfrType()` to convert internal transmittance into total transmittance and *vice versa* and function `convertThickness()` to compute the spectral transmittance of a filter of the same material but different thickness. These attributes are also allowed in the case of `object_spct` and `reflector_spct` and retained during class conversions between them and `filter_spct` objects. They are included in the printout of `filter.spct` objects. ```{r} polyester.spct ``` We estimate the spectral transmittance of 2 mm-thick PET film. ```{r} convertThickness(polyester.spct, thickness = 2e-3) ``` ------------------------------------------------------------------------ Attributes `instr_desc` and `instr_settings` are used to store measurement-related metadata, describing the instrument used and its settings. These attributes are lists, with a few default fields and possibly unlimited *special* attributes. The present package provides functions for operating on them and print-outs include some of the default fields if the attribute is set. The expectation is that these attributes are set by other packages such as 'ooacquire' used for direct data acquisition or raw instrument data import. These instrument-specific attributes can contain lots of information and bloat the size of spectral objects. Two methods, `trimInstrDesc()` and `trimInstrSettings()` can be used to discard parts of these metadata, such as instrument calibration information, once the raw-counts data have been converted into physical units. ------------------------------------------------------------------------ ------------------------------------------------------------------------ Spectral objects created with earlier versions (from before the first CRAN release in 2016) of this package are missing some metadata attributes. For this reason *summary* methods from this package and `autoplot()` and `ggplot()` methods from package 'ggspectra' may print warnings. These *very old* objects can be updated by adding the missing attributes using functions `setTimeUnit`, `setBSWFUsed`, `setTfrType` and `setRfrType`. However, in many cases function `update_spct` can be used to set the missing attributes to default values, or user scripts re-run to rebuild the data objects from raw data. ------------------------------------------------------------------------ ## Collections of spectra ### Classes The package defines several classes intended to be used to store *collections* of different types of spectral data. They are all derived from `generic_mspct`, which in turn is derived from `list`. Table 3 lists them. ------------------------------------------------------------------------ **Table 3. Classes for collections of spectral objects.** Objects of class `generic_mspct` can have member objects of any class derived from `generic_spct` and can be heterogeneous, while the other classes support homogeneous collections of spectral objects. Attributes of these objects can be queried and set with the normal R methods `attr` and `attributes` as well as with functions defined in this package. See table 1 for the attributes used in individual member spectra of collections. | Class name | Class of member objects | Attributes | |:--------------------|:------------------------|---------------------------| | `generic_mspct` | `generic_spct` | `names`, `dim`, `comment` | | `calibration_mspct` | `calibration_spct` | `names`, `dim`, `comment` | | `raw_mspct` | `raw_spct` | `names`, `dim`, `comment` | | `cps_mspct` | `cps_spct` | `names`, `dim`, `comment` | | `source_mspct` | `source_spct` | `names`, `dim`, `comment` | | `filter_mspct` | `filter_spct` | `names`, `dim`, `comment` | | `reflector_mspct` | `reflector_spct` | `names`, `dim`, `comment` | | `object_mspct` | `object_spct` | `names`, `dim`, `comment` | | `response_mspct` | `response_spct` | `names`, `dim`, `comment` | | `chroma_mspct` | `chroma_spct` | `names`, `dim`, `comment` | | `solute_mspct` | `solute_spct` | `names`, `dim`, `comment` | ------------------------------------------------------------------------ Objects of these classes, except for class `generic_mspct`, can only contain members belonging the matching class of spectra. As all other spectral object classes are derived from `generic_spct`, `generic_mspct` objects can contain heterogeneous collections of spectra. In all cases, there are no restrictions on the lengths, wavelength range and/or wavelength step size, or attributes other than `class` of the contained spectra. Mimicking R's arrays and matrices, a `dim` attribute is always present and `dim` methods are provided. This approach is expected to allow the storage of time series of spectral data, or (hyper)spectral image data, or even higher dimensional spectral data. The handling of 1D and 2D spectral collections is already implemented in the summary methods. Handling of 3D and higher dimensional data can be implemented in the future without changing the class definitions. By having implemented `dim`, also methods `ncol` and `nrow` are available as they use `dim` internally. Array-like subscripting collections of spectra is **not** implemented. ### Construction #### Constructors We can construct a collection using a list of spectral objects as a starting point, in this case the spectral irradiance for sunlight. ```{r, col-construction-1} two_suns.mspct <- source_mspct(list(sun1 = sun.spct, sun2 = sun.spct)) two_suns.mspct ``` We can also create heterogeneous collections, but this drastically limits the methods that are applicable to the resulting collection. ```{r, col-construction-2} mixed.mspct <- generic_mspct(list(filter = polyester.spct, source = sun.spct)) class(mixed.mspct) lapply(mixed.mspct, class_spct) ``` #### Using *as.* conversion functions The `as.` coercion methods for collections of spectra, not only change the class of the collection object, but can also optionally apply the corresponding `as.` functions to the member objects. A copy of the original object is made and then class-converted and returned. ```{r, col-construction-3} two_gen.mspct <- as.generic_mspct(two_suns.mspct) class(two_gen.mspct) str(two_gen.mspct, max.level = 1, give.attr = FALSE) ``` In addition to coercion methods for lists of spectra objects, coercion methods are available for lists of data frames (or tibbles) and from matrix objects. One additional feature is that if a single spectrum object or data frame are coerced into a collection of spectra, the behaviour is equivalent to having passed as argument a list containing such object as its only member. ```{r, col-construction-4} one_sun.mspct <- as.source_mspct(sun.spct) class(one_sun.mspct) str(one_sun.mspct, max.level = 1, give.attr = FALSE) ``` Sometimes spectral data stored in a matrix need to be coerced into a collection of spectra. Coercion methods are defined also for this cases. Several spectra may be stored in a matrix either by row or by column, but this can be deduced automatically in the case of rectangular matrices. Wavelengths values are not expected to be part of the matrix, and need to be supplied as a separate numeric vector sorted in ascending order. As in a spectrum wavelength values never repeat, the vector of wavelengths is never recycled and uniqueness of values is enforced. We here use artificial data, in this first example with spectra saved by column. We assume that the values in the matrix are spectral transmittance stored as percentages (hence "Tpc"). ```{r, col-construction-5} x <- matrix(1:100, ncol = 2) wl <- 501:550 # wavelengths in nanometres as.filter_mspct(x, wl, "Tpc") ``` In a second example we supply explicit names for the spectra. ```{r, col-construction-6} as.filter_mspct(x, wl, "Tpc", spct.names = c("A", "B")) ``` There is no change in the call for data stored by row. We create a new matrix, with the same data as above, but stored by row, as some R packages do. ```{r, col-construction-7} xrow <- matrix(1:100, nrow = 2, byrow = TRUE) as.filter_mspct(xrow, wl, "Tpc") ``` There is only one case when an explicit argument for `byrow` is needed: square matrices (same number of spectra as of wavelength values in each spectrum). When coercing collections of spectra into matrices, the metadata contained in the individual spectral objects is discarded, and only the `"comment"` attribute of the collection of spectra copied to the returned object. The wavelength values are preserved in an attribute named "w.length", but are not included as part of the matrix. ```{r, col-construction-8} two_suns.mat <- as.matrix(two_suns.mspct, "s.e.irrad") class(two_suns.mat) dim(two_suns.mat) head(dimnames(two_suns.mat)$spct) head(dimnames(two_suns.mat)$w.length) head(attr(two_suns.mat, "w.length")) ``` The argument `byrow` in the coercion into matrix methods has the same meaning as in the `matrix` constructor function. ```{r, col-construction-9} two_suns.row_mat <- as.matrix(two_suns.mat, "s.e.irrad", byrow = TRUE) class(two_suns.row_mat) dim(two_suns.row_mat) head(dimnames(two_suns.row_mat)$spct) head(attr(two_suns.row_mat, "w.length")) ``` Collections of spectra have a `dim` attribute but it is discarded as it describes dimensions that could require spectral data to be stored in a three dimensional array and cannot be mapped to the two dimensions of a matrix. These functions are not fully automatic, the user needs to provide the name of the variable to extract from each spectrum. If the wavelength values are not consistent among spectra, only those with the same values as the first spectrum in the collection are retained and the remaining ones dropped with a warning. #### Converting *tidy* data Spectral objects containing multiple spectra identified by a factor can be created by row-binding compatible spectral objects. For for binding to succeed, the list of spectra passed as argument must be homogeneous with respect to member class, as well as for certain attributes such as `time.unit`. Other metadata attributes are retained as named lists or multi-row data frames. We use a time series of sunlight spectra. The default name `spct.idx` is used for the `idfactor` unless an argument is passed to parameter `idfactor`. ```{r, bind-1} sun_evening_12.spct <- rbindspct(sun_evening.mspct[1:2]) sun_evening_12.spct ``` The reverse operation, separating the individual spectra from a spectrum object containing multiple spectra in *long form* and storing them as a collection, is implemented in method `subset2mspct`. In this case, metadata is set for the individual spectra if it is available in the object being subset. What metadata is automatically stored depends on the version of this package used when creating the long form spectrum object by means of `rbindspct()`. Versions 0.9.14 and later preserve most of the metadata. ```{r, bind-1a} subset2mspct(sun_evening_12.spct) ``` If multiple spectra are stored in long form (as *tidy* data) in an ordinary `data.frame` or a `tibble` object with columns with suitable names, the same `subset2mspct` method can be used. In these case, arguments indicating the target class for conversion and supplying the name of the index variable encoding the grouping into multiple spectra need to be supplied. A call as in the example below, adds only default metadata to spectral objects. ```{r, bind-2} test1.df <- data.frame(w.length = rep(200:210, 2), s.e.irrad = rep(c(1, 2), c(11, 11)), spectrum = factor(rep(c("A", "B"), c(11,11)))) subset2mspct(test1.df, member.class = "source_spct", idx.var = "spectrum") ``` If all member spectra share the same metadata, and the constructor has a parameter allowing it to be set, it can be passed as a named argument. Here we set the time base of expression of the spectral data to a whole day instead of the default of using seconds as is usual for irradiance. ```{r, bind-3} subset2mspct(test1.df, member.class = "source_spct", idx.var = "spectrum", time.unit = "day") ``` To directly convert a *tidy* data frame into a long form spectral object we need to pass the number of spectra through parameter `multiple.wl` to change the target value of the check for unique wavelength values (default is `multiple.wl = 1L`). The data should be anyway ordered by increasing wavelength and the name of the factor distinguishing the spectra also passed as an argument. ```{r, set-class-1} test2.df <- test1.df setSourceSpct(test2.df, multiple.wl = 2L, idfactor = "spectrum") getMultipleWl(test2.df) getIdFactor(test2.df) ``` If `multiple.wl = NULL` a suitable value is guessed from the data frame passed as first argument. This should work in most cases, but is more time consuming. Using this approach in addition disables the check for a fixed number of spectra. ```{r, set-class-2} test3.df <- test1.df setSourceSpct(test3.df, multiple.wl = NULL, idfactor = "spectrum") getMultipleWl(test3.df) getIdFactor(test3.df) ``` #### Converting *untidy* data frames In a *broad form* (or *untidy*) data.frame the spectral values for different spectra are stored side-by-side as columns, and a single additional column used to store the shared wavelength values. To create a collection of spectral objects, with each member containing a single spectrum, we use function `split2source_mspct` and its equivalents for the remaining classes of spectral objects (class is determined by the function used, and columns which are not `numeric` are skipped). The column containing wavelength values in nanometres is recognized by means of its name. The names used for the spectra in the collection are derived from the names of the columns containing numeric (assumed spectral data) values. Using defaults when variables have the names expected by default. ```{r, split-1a} test2.df <- data.frame(w.length = 200:210, A = 1, B = 2) split2source_mspct(x = test2.df) ``` If variable names differ from the default expectation, they can be named in the call. ```{r, split-1b} test3.df <- data.frame(w = 200:210, A = 1, B = 2, z = "Z") split2source_mspct(x = test3.df, w.length.var = "w", idx.var = "z") ``` The interpretation of the data in the numeric columns other than the wavelengths can be changed. ```{r} split2source_mspct(test2.df, spct.data.var = "s.q.irrad") ``` Also in this case, it is possible to pass additional named arguments to the constructor of spectral objects. ```{r, split-2} split2source_mspct(test2.df, spct.data.var = "s.q.irrad", time.unit = "day") ``` An additional way of storing spectral data is with one spectrum per row and one wavelength per coulmn. In such a case, we can transpose the data frame with function `t()` before pasing it as argument to `split2source_mspct()`, taking care to copy wavelength values into a column and that the column names of the transposed data frame match the expected ones. ### Conversion into a "wide" data frame Sometimes, when exporting spectral data we may need to convert a collection of spectra into an *untidy* or *wide form* data frame. In the example below we use a collection with only two member spectra, but `join_mspct()` can handle collections of any length. ```{r, join-mspct-01} my.df <- join_mspct(sun_evening.mspct) head(my.df) ``` ### Querying the class `is.` functions are defined for all the new classes. R's `class` method can also be used. ```{r, col-query-class-1} is.source_mspct(sun_evening.mspct) class(sun_evening.mspct) ``` In addition to using `class` to query the class of the collection, we can use base R's `lapply` together with `class` or `class_spct` to query the class of each of the members of the collection. ```{r, col-query-class-2} is.filter_mspct(sun_evening.mspct) is.any_mspct(sun_evening.mspct) class(sun_evening.mspct) lapply(sun_evening.mspct, class_spct) lapply(sun_evening.mspct, class) ``` ### Extract, replace and combine R's extraction and replacement methods have specializations for collections of spectra and can be used with the same syntax and functionality as for R lists. However they test the class and validity of the returned objects and replacement members. ------------------------------------------------------------------------ Methods `[`, and `[<-`, extract and replace *parts* of the collection, respectively. Even when only one member is extracted, the returned value is a collection of spectra. The expected replacement value is also, always a collection of spectra. ------------------------------------------------------------------------ ```{r, extract-1} sun_evening.mspct[1] ``` ```{r, extract-1a} sun_evening.mspct[1:3] ``` **Code like that in the chunk below, may be counterintuitive. Collections of spectra are named lists, consequently assigning by position as shown here swaps the values without swapping the names of the slots!** Using summary to check the names and time stamps we can demonstrate this behaviour. ```{r, extract-2} # warning: this does not swap the names, even if it swaps the spectra my.mspct <- sun_evening.mspct summary(my.mspct, which.metadata = "when.measured")$summary[ , -(2:6)] # member spectr swapped positions, but not the slot names my.mspct[1:2] <- my.mspct[2:1] summary(my.mspct, which.metadata = "when.measured")$summary[ , -(2:6)] # of course, we can also swap the names if needed names(my.mspct)[1:2] <- names(my.mspct)[2:1] summary(my.mspct, which.metadata = "when.measured")$summary[ , -(2:6)] ``` ------------------------------------------------------------------------ Methods `[[`, `$` and `[[<-`, extract and replace individual members of the collection, respectively. They always return or expect objects of one of the spectral classes. ------------------------------------------------------------------------ ```{r, extract-3} sun_evening.mspct[[1]] sun_evening.mspct$time.01 sun_evening.mspct[["time.01"]] ``` ```{r, extract-4} # local copy my.mspct <- sun_evening.mspct names(my.mspct) # add computed member my.mspct[["time.01x2"]] <- my.mspct[["time.01"]] * 2 names(my.mspct) # delete a member my.mspct[["time.01x2"]] <- NULL names(my.mspct) ``` As with R's `list` objects, can combine collections of spectra with method `c()` but we can not create new collections from individual spectra using this method. Extraction plus concatenation, moves names together with spectra. ```{r, extract-5} new.spct <- c(my.mspct[5:4], my.mspct[3]) summary(new.spct, which.metadata = "when.measured")$summary[ , -(2:6)] ``` ### Random sampling With large collections of spectra it can be useful to extract a random subset of spectra. Method `pull_sample()` make this easy for multiple spectra, stored either in long form or in collections of spectra. (We can set the seed of the pseudo-random number generator to ensure repeatability). ```{r} set.seed(1234564) sampled.mspct <- pull_sample(sun_evening.mspct, size = 2) summary(sampled.mspct, which.metadata = "when.measured")$summary[ , -(2:6)] ``` ```{r} set.seed(1234564) sampled.spct <- pull_sample(sun_evening.spct, size = 2) summary(sampled.spct) ``` ### Transform or *apply* functions For our *apply* functions we follow the naming convention used in package `plyr`, but using `ms` as prefix for `_mspct` objects. The *apply* functions implemented in the '**photobiology**' package are `msmsply`, `msdply`, `mslply` and `msaply` which both accept a collection of spectra as first argument and return a collection of spectra, a data frame, a list, or an array respectively (see Table 4). ------------------------------------------------------------------------ **Table 4. Apply functions for collections of spectra.** Key: v., value returned by *apply* function; f.v., value returned by the applied function (argument `.fun`). In the table `generic_mspct` and `generic_spct` indicate objects of these classes or any class derived from them. The exact class of the collection of spectra object returned will be determined by the class(es) of the values returned by the applied function. | *apply* function | first arg. class | v\. class | f.v. class | f.v. length | f.v. dims | |:-----------|:-----------|:-----------|:-----------|:-----------|:-----------| | `msmsply` | `generic_mspct` | `generic_mspct` | `generic_spct` | 1 | any | | `msdply` | `generic_mspct` | `data.frame` | `numeric` | $1\ldots n$ | 1 | | `mslply` | `generic_mspct` | `list` | any | any | any | | `msaply` | `generic_mspct` | `vector` | any simple | 1 | 0 | | `msaply` | `generic_mspct` | `matrix` | any simple | $2\ldots n$ | $2\ldots n$ | | `concolve_each` | `generic_mspct` | `generic_mspct` | `generic_spct` | 1 | any | ------------------------------------------------------------------------ Functions `msmsply()`, `msdply` and `mslply` can be used to apply a function to each member spectrum in a collection. The *apply* function to use depends on the return value of the applied function. In the case of `msmsply()` the applied function is expected to return a *transformed* spectrum as another object of class `generic_spct` or a class derived from it. The value returned by `msmsply` is a collection of spectra, of a type determined by the class(es) of the member spectra in the new collection. We start with a simple example in which we add a constant to each spectrum in the collection ```{r, apply-1} two.mspct <- sun_evening.mspct[1:2] msmsply(two.mspct, `+`, 0.1) ``` and continue with a more complex example in which we trim each spectrum, and fill the added values with `NA` values. ```{r, apply-2} msmsply(two.mspct, trim_wl, range = c(285, 500), fill = NA) ``` In the second example we pass two arguments by name to the applied function. The number of arguments is not fixed, but the spectrum will be always passed as the first argument to the function. In the case of `msdply()` the applied function is expected to return an R object of the same length for each of the member spectra. ```{r, apply-3} msdply(two.mspct, wl_max) ``` ```{r, apply-4} wl_ranges.df <- msdply(two.mspct, wl_range) wl_ranges.df cat(comment(wl_ranges.df)) ``` ```{r, apply-5} msdply(two.mspct, wl_range, na.rm = TRUE) ``` In the case of `mslply()` the applied function is expected to return an R object of any length, possibly variable among members. ```{r, apply-6} str(mslply(two.mspct, colnames)) ``` In the case of `msaply()` the applied function is expected to return an R object of length 1, although a list with dimensions will be returned for longer return values. ```{r, apply-7} str(msaply(two.mspct, wl_max)) ``` ```{r, apply-8} msaply(two.mspct, wl_range) ``` For the most common cases for which one would use the *apply* functions described in the previous section methods and functions for operations on collections of spectra are defined in the package. ```{r} wl_range(two.mspct) ``` These methods are described in a later section and listed in Table 9. ### Summary spectra Starting from a collection of spectra, or multiple spectra stored in long form, we can obtain a single spectrum containing a "parallel" summary across them. The current implementation of the methods described in this section expects that all spectra in a collection are of the same class, and data is available for each of them at exactly the same set of wavelengths. In the table below, `source_spct/mspct` is an abbreviation for `source_spct` or `source_mspct`. | functions | class of argument | class of returned spectrum/a | variables in spectrum | |:------------------|:---------------|:---------------|:---------------------| | `s_sum` | `source_spct/mspct` | `source_spct` | same as input | | `s_sum` | `response_spct/mspct` | `response_spct` | same as input | | `s_sum` | `filter_spct/mspct` | `generic_spct` | input tagged `.sum` | | `s_sum` | `reflector_spct/mspct` | `generic_spct` | input tagged `.sum` | | `s_sum` | `calibration_spct/mspct` | `generic_spct` | input tagged `.sum` | | `s_prod` | `source_spct/mspct` | `source_spct` | input tagged `.prod` | | `s_prod` | `response_spct/mspct` | `response_spct` | input tagged `.prod` | | `s_prod` | `filter_spct/mspct` | `filter_spct` | same as input | | `s_prod` | `reflector_spct/mspct` | `reflector_spct` | same as input | | `s_prod` | `calibration_spct/mspct` | `generic_spct` | input tagged `.prod` | | `s_mean` or `s_median` | `source_spct/mspct` | `source_spct` | same as input | | `s_mean` or `s_median` | `response_spct/mspct` | `response_spct` | same as input | | `s_mean` or `s_median` | `filter_spct/mspct` | `filter_spct` | same as input | | `s_mean` or `s_median` | `reflector_spct/mspct` | `reflector_spct` | same as input | | `s_mean` or `s_median` | `calibration_spct/mspct` | `calibration_spct` | same as input | | `s_var`, `s_se` or `s_sd` | `source_spct/mspct` | `generic_spct` | input tagged `.var`, `.se` or `.sd` | | `s_var`, `s_se` or `s_sd` | `response_spct/mspct` | `generic_spct` | input tagged `.var`, `.se` or `.sd` | | `s_var`, `s_se` or `s_sd` | `filter_spct/mspct` | `generic_spct` | input tagged `.var`, `.se` or `.sd` | | `s_var`, `s_se` or `s_sd` | `reflector_spct/mspct` | `generic_spct` | input tagged `.var`, `.se` or `.sd` | | `s_var`, `s_se` or `s_sd` | `calibration_spct/mspct` | `generic_spct` | input tagged `.var`, `.se` or `.sd` | | `s_mean_se` | `source_spct/mspct` | `source_spct` | same as input plus col. tagged `.se` | | `s_mean_se` | `response_spct/mspct` | `response_spct` | same as input plus col. tagged `.se` | | `s_mean_se` | `filter_spct/mspct` | `filter_spct` | same as input plus col. tagged `.se` | | `s_mean_se` | `reflector_spct/mspct` | `reflector_spct` | same as input plus col. tagged `.se` | | `s_mean_se` | `calibration_spct/mspct` | `calibration_spct` | same as input plus col. tagged `.se` | | `s_range` | `source_spct/mspct` | `generic_spct` | input tagged `.min` and `.max` | | `s_range` | `response_spct/mspct` | `generic_spct` | input tagged `.min` and `.max` | | `s_range` | `filter_spct/mspct` | `generic_spct` | input tagged `.min` and `.max` | | `s_range` | `reflector_spct/mspct` | `generic_spct` | input tagged `.min` and `.max` | | `s_range` | `calibration_spct/mspct` | `generic_spct` | input tagged `.min` and `.max` | | `s_quantile` | `source_spct/mspct` | `source_mspct/spct` | same as input | | `s_quantile` | `response_spct/mspct` | `response_mspct/spct` | same as input | | `s_quantile` | `filter_spct/mspct` | `filter_mspct/spct` | same as input | | `s_quantile` | `reflector_spct/mspct` | `reflector_mspct/spct` | same as input | | `s_quantile` | `calibration_spct/mspct` | `calibration_mspct/spct` | same as input | The calculations are done wavelength by wavelength yielding a spectrum at the same wavelengths than those in the input. Removal of `NA` values is done wavelength by wavelength. As shown in the table above, depending on the operation the returned quantity may be equivalent or not to the quantity in the input. When the quantity changes, even if units of expression remain the same, the name of the variable and class of the returned spectrum are different to those in the input. Methods in this "family" are fast and are specially useful for large collections of spectra containing even thousands of individual spectra. Nonetheless, for the example below we use a collection of two spectra. As computing the mean returns values of the same quantity as in the input the returned value is a `source_spct` object, in other words of the same class as the members of `two.mspct`. ```{r} s_mean(sun_evening.mspct) ``` When we compute the standard error, the quantity returned is not the same. Consequently, the object returned is in this case a `generic_spct` instead of `source_spc` as it does not contain spectral irradiance data as the input did. The variable name is in addition tagged with `.se` to *advertise* the change in the quantity. ```{r} s_se(sun_evening.mspct) ``` We can also compute both the mean and standard error simultaneously. ```{r} s_mean_se(sun_evening.mspct) ``` The other functions in this family work in a similar way to those described here. ### Convolution By convolution we normally mean the multiplication value by value at matching wavelengths of two spectra. The function described in this section facilitates this and similar operations among collections of spectra. An example use case could be the convolution of spectral irradiance by spectral transmittance for all combinations of light sources and filters in a collection of source spectra and a collection of filter spectra. Default operator (or function) is that for multiplication, either one or both of the two first arguments must be a collection of spectra. When only one argument is a collection of spectra, the other one can be a spectrum, or even a numeric vector. For multiplication the order of the operands does not affect the returned value. With operators or functions for non-transitive operations the order does, obviously, matter. Convolution can be used, for example to simulate the effect of an optical filter on the irradiance from a light source. ```{r, convolve-1} convolve_each(two.mspct, yellow_gel.spct) ``` ```{r, convolve-2} convolve_each(yellow_gel.spct, two.mspct) ``` ```{r, convolve-3} another_two.mspct <- two.mspct names(another_two.mspct) <- c("a", "b") convolve_each(another_two.mspct, two.mspct) ``` The function `convolve_each` will use other operators or functions and even pass additional named arguments when these are supplied as arguments. ```{r} convolve_each(two.mspct, sun.spct, oper = `+`) ``` ------------------------------------------------------------------------ There are cases where functions `convolve_each()` and `msmsply()` can be both used, but there are also cases where their differences matter. An example is convolving two collections of spectra, a case where only `convolve_each()` can be used. In contrast, when one of the arguments is not a spectrum or a collection of spectra, `msmsply()` should be used instead. ------------------------------------------------------------------------ ### Metadata attributes Some of the `set` and `get` functions used with attributes have method definitions for collections of spectra. Some examples follow. ```{r, col-attr-1} when_measured(two.mspct) when_measured(two.mspct, simplify = TRUE) when_measured(two.mspct) <- ymd("2015-10-31", tz = "Europe/Helsinki") when_measured(two.mspct) when_measured(two.mspct, simplify = TRUE) when_measured(two.mspct) <- list(ymd_hm("2015-10-31 10:00", tz = "Europe/Helsinki"), ymd_hm("2015-10-31 11:00", tz = "Europe/Helsinki")) when_measured(two.mspct) # UTC shown! when_measured(two.mspct, simplify = TRUE) two.mspct ``` Other methods available are `getWhereMeasured` and `setWhereMeasured`, and `getWhatMeasured` and `setWhatMeasured`. Functions `when.measured2tb()`, `geocode2tb()`, `lon2tb()`, `lat2tb()` and `what.measured2tb()` extract these same attributes from collection members into a tibble or data frame. In contrast to the "get" methods described above, if an existing data frame or tibble with a matching number of rows is passed as second argument, the values for the attribute are saved into a new column appended at the right edge of the tibble or data frame. ```{r} when_measured2tb(sun_evening.mspct) ``` By default the new column is named after the name of the attribute, but this default can be overridden by the user. ```{r} when_measured2tb(sun_evening.mspct, col.names = c(when.measured = "acquisition.time")) ``` Function `spct_metadata()` returns the metadata in a tibble. Unless the user supplies a list of names of attributes, all applicable metadata values are returned. A single spectrum or a collection of spectra can be passed. By default columns containing only missing data are dropped. ```{r} spct_metadata(two.mspct) ``` ```{r} spct_metadata(two.mspct, col.names = c("when.measured" = "time", "where.measured" = "geocode"), unnest = FALSE) ``` Function `add_attr2tb()` adds one or more columns with attributes to a tibble or data frame. We here also demonstrate that because the data frame object is passed to the first positional argument, we can use a *pipe*. This difference in interface is the main difference between this function and `spct_metadata()`. ```{r} q_irrad(two.mspct) %>% add_attr2tb(two.mspct, col.names = c("lon", "lat", "when.measured")) ``` We can set the desired names for the columns if we wish them to be something else than the default. ```{r} q_irrad(two.mspct) %>% add_attr2tb(two.mspct, col.names = c(lon = "longitude", lat = "latitude", when.measured = "time")) ``` ## Wavebands When a range of wavelengths or a range of wavelengths plus a spectral weighting function (SWF) is needed for radiation summaries or transformations, methods, operators and functions defined in package '**photobiology**' use `waveband` objects to store these data. A few other bits of information can be included to fine-tune calculations. The waveband definitions do NOT describe whether input spectral irradiances are photon or energy based, nor whether the output irradiance will be based on photon or energy units. All waveband objects belong to the S3 class `waveband`. A waveband stores the boundaries of a range of wavelengths, $\lambda_1$ and $\lambda_2$, so that wavelengths within $\lambda_1 \ge \lambda > \lambda_2$ belong to the waveband. If the waveband describes a spectral weighting function, in addition to the range, two formulations of the function used to compute the weights as a function of wavelength are also stored, $w_E(\lambda)$ and $w_Q(\lambda)$ to be used with energy- or photon based irradiances. Spectral weighting functions are normally used to compute effective irradiances. In the current implementation their use is limited to this case, while for other summaries only the wavelength range is made use of, even when weighting functions are available. ### Construction To create a waveband object we use constructor function `waveband`, and optionally giving a name to it. **We will use these objects in many of the examples below, so you will need to run the code chunk bellow to be able to reproduce those examples.** It should be noted that waveband constructors for the most frequently used wavelength-range definitions are provided by package '**photobiologyWavebands**'. ```{r, wb-1} PAR.wb <- waveband(c(400, 700), wb.name = "PAR") UVA.wb <- waveband(c(315, 400), wb.name = "UVA") UVB.wb <- waveband(c(280, 315), wb.name = "UVB") UVC.wb <- waveband(c(100, 280), wb.name = "UVC") UV.wb <- waveband(c(100, 400), wb.name = "UV") UV_bands.lst <- list(UVC.wb, UVB.wb, UVA.wb) ``` When including a BSWF, we can supply, one or two versions of functions returning the weights as a function of wavelength. Several such functions are defined in package '**photobiologyWavebands**' as well as waveband constructors using them. Here we show how a waveband can be defined based on a SWF, using the CIE definition for the erythemal spectral weighting function. Although the constructor is smart enough to derive the missing function when only one function is supplied, performance may suffer unless two performance-optimized function are provided, one for energy-based effect and a second one for photon-based effect. ```{r, wb-2} CIE_e_fun <- function(w.length){ CIE.energy <- numeric(length(w.length)) CIE.energy[w.length <= 298] <- 1 CIE.energy[(w.length > 298) & (w.length <= 328)] <- 10^(0.094*(298-w.length[(w.length > 298) & (w.length <= 328)])) CIE.energy[(w.length > 328) & (w.length <= 400)] <- 10^(0.015*(139-w.length[(w.length > 328) & (w.length <= 400)])) CIE.energy[w.length > 400] <- 0 return(CIE.energy) } ``` ```{r, wb-3} CIE.wb <- waveband(c(250, 400), weight = "SWF", SWF.e.fun = CIE_e_fun, SWF.norm = 298) ``` Another case where you might want to enter the same function twice, is if you are using an absorptance spectrum as SWF, as the percent of radiation absorbed will be independent of whether photon or energy units are used for the spectral irradiance. The first argument to `waveband()` does not need to be a numeric vector of length two. Any R object of a class that supplies a `range()` method definition that can be interpreted as a range of wavelengths in nanometres can be used. As a consequence, when wanting to construct a waveband covering the whole range of a spectrum one can simply supply the spectrum as argument, or to construct an non-weighed waveband which covers exactly the same range of wavelengths as an existing effective (weighted) waveband, one can supply a waveband object as an argument. ```{r, wb-4} waveband(sun.spct) ``` An "empty" waveband is returned by some functions as a "null" value. ```{r} waveband() ``` ### Querying the class The function `is.waveband` can the used to query any R object. This function returns a logical value. ```{r, wb-5} is.waveband(PAR.wb) ``` Above, we demonstrate that `PAR.wb` is a waveband object, the function `photobiologyWavebands::PAR()` is a waveband constructor returning a waveband object. See package '**photobiologyWavebands**' for details on pre-defined waveband constructors for frequently used wavelength ranges and biological spectral weighting functions (BSWFs). ### Retrieving properties The function `is_effective` can the used to query any R object. ```{r, wb-6} is_effective(waveband(c(400,500))) ``` ## Collections of wavebands In the current implementation there is no special class used for storing collections of `waveband` objects. We simply use base R's `list` class. ### Construction #### List constructor Just base R's functions used to create a list object. ```{r, wb-list-1} wavebands <- list(waveband(c(300,400)), waveband(c(400,500))) wavebands ``` #### Special constructor The function `split_bands` can be used to generate lists of non-weighed wavebands in two different ways: a) it can be used to split a range of wavelengths given by an R object into a series of adjacent wavebands, or b) with a list of objects returning ranges, it can be used to create non-adjacent and even overlapping wavebands. The code chunk bellow shows an example of two variations of case a). With the default value for `length.out` of `NULL` each numerical value in the input is taken as a wavelength (nm) at the boundary between adjacent wavebands. If a numerical value is supplied to `length.out`, then the whole wavelength range of the input is split into this number of equally spaced adjacent wavebands. ```{r, wb-split-1} split_bands(c(200, 225, 300)) split_bands(c(200, 225, 300), length.out = 2) ``` In both examples above, the output is a list of two wavebands, but the *split* boundaries are at a different wavelength. The chunk bellow gives a few more examples of the use of case a). ```{r, wb-split-2} split_bands(sun.spct, length.out = 2) split_bands(PAR.wb, length.out = 2) split_bands(c(200, 800), length.out = 3) ``` Now we demonstrate case b). This case is handled by recursion, so each list element can be anything that is a valid input to the function, including a nested list. However, the returned value is always a flat list of wavebands. ```{r, wb-split-3} split_bands(list(A = c(200, 300), B = c(400, 500), C = c(250, 350))) split_bands(list(c(100, 150, 200), c(800, 825))) ``` In case b) if we supply a numeric value to `length.out`, this value is used recursively for each element of the list. ```{r, wb-split-4} split_bands(UV_bands.lst, length.out = 2) split_bands(list(c(100, 150, 200), c(800, 825)), length.out = 1) ``` ## Object *inspection* methods ### Printing The `print()` method for spectra is based on the method defined in package '**tibble**', consequently, it is possible to use the options from this package to control printing. In the code chunk below, `tibble.print_max`, the number of rows in the spectral object above which only `tibble.print_min` rows are printed, are both set to 5, instead of the default 20 and 10, to avoid excessive clutter in our examples. ```{r, set-up-printing, eval=FALSE} options(tibble.print_max = 4) options(tibble.print_min = 4) ``` For explicit calls to `print()` its argument `n` can be used to control the number of lines printed. If `n` is set to `Inf` the whole spectrum is always printed. The output differs from that of the `print()` method from package 'dplyr' in that additional metadata specific to each type of spectra are shown. ```{r, print-1} print(sun.spct, n = 3) ``` Specialized `print()` methods for collections of spectra and for `waveband` objects are also defined. ### Summary The `summary()` method for spectra is based on base R's `summary()` method for data frames, and accepts the same arguments. The main difference is that the attributes containing metadata and dimensions of the original spectrum object are copied to the summary object. ```{r, print-2} str(summary(sun.spct)) ``` Specialized `print()` methods for summaries of spectra are defined. The output differs from that of the `print()` method from base R in that additional metadata specific to spectra are shown. ```{r, print-2a} summary(sun.spct) ``` In the case of collections of spectra and multiple spectra in long form additional parameters accept arguments that control the type of summary computed. For collections the default is to return a data frame lisitng the member spectra and their properties. ```{r, print-2b} summary(sun_evening.mspct) ``` Passing `expand = "each"` summarises each member of the collection individually, returning a list of summaries. _The long output from the chunk below is not shown._ ```{r, print-2c, eval = FALSE} summary(sun_evening.mspct, expand = "each") ``` The default for multiple spectra in long form is a joint summary, similar to that of the underlying data frame but with the additional metadata as attributes. ```{r, print-2d} summary(sun_evening.spct) ``` To obtain a summary matching that of a collection we can passed `expand = "collection"`. ```{r, print-2e} summary(sun_evening.spct, expand = "collection") ``` ### Handling `NA`s Functions `na.omit()` and `na.exclude` are implemented for all spectral classes. These methods test for `NA`s only the spectral data, not wavelength. They set the `"na.action"` attribute in the same way as the corresponding methods for data frames. In the case of `na.fail()`, `na.pass()` and `na.action()`, the methods from base R, can be used with spectra. ```{r} na.omit(sun.spct) na.exclude(sun.spct) ``` ## Transformations: using operators ### Binary operators All of R's built-in maths operators have definitions for spectra. It is possible to sum, subtract, multiply and divide spectra. These operators can be used even if the spectral data is on different arbitrary sets of wavelengths. Operators by default return values expressed in energy units. Only certain operations are meaningful for a given combination of objects belonging to different classes, and meaningless combinations return `NA` also issuing a warning (see Table 5). By default operations are carried out on spectral energy irradiance for `source_spct` objects and transmittance for `filter_spct` objects. ------------------------------------------------------------------------ **Table 5. Binary operators and their operands.** Validity and class of result. All operations marked `\Y' are allowed, those marked`\N' are forbidden and return `NA` and issue a warning. Operators `%/%` and `%%` follow `/`. | e1 | `+` | `-` | `*` | `/` | `^` | e2 | value | |:--------|:-------:|:-------:|:-------:|:-------:|:-------:|:--------|:----------| | `raw_spct` | Y | Y | Y | Y | Y | `raw_spct` | `raw_spct` | | `cps_spct` | Y | Y | Y | Y | Y | `cps_spct` | `cps_spct` | | `source_spct` | Y | Y | Y | Y | Y | `source_spct` | `source_spct` | | `filter_spct` (T) | N | N | Y | Y | N | `filter_spct` | `filter_spct` | | `filter_spct` (A) | Y | Y | N | N | N | `filter_spct` | `filter_spct` | | `reflector_spct` | N | N | Y | Y | N | `reflector_spct` | `reflector_spct` | | `object_spct` | N | N | N | N | N | `object_spct` | -- | | `response_spct` | Y | Y | Y | Y | N | `response_spct` | `response_spct` | | `chroma_spct` | Y | Y | Y | Y | Y | `chroma_spct` | `chroma_spct` | | `raw_spct` | Y | Y | Y | Y | Y | `numeric` | `raw_spct` | | `cps_spct` | Y | Y | Y | Y | Y | `numeric` | `cps_spct` | | `calibration_spct` | Y | Y | Y | Y | Y | `numeric` | `calibration_spct` | | `source_spct` | Y | Y | Y | Y | Y | `numeric` | `source_spct` | | `filter_spct` | Y | Y | Y | Y | Y | `numeric` | `filter_spct` | | `reflector_spct` | Y | Y | Y | Y | Y | `numeric` | `reflector_spct` | | `object_spct` | N | N | N | N | N | `numeric` | -- | | `response_spct` | Y | Y | Y | Y | Y | `numeric` | `response_spct` | | `chroma_spct` | Y | Y | Y | Y | Y | `numeric` | `chroma_spct` | | `cps_spct` | N | N | Y | N | N | `calibration_spct` | `source_spct` | | `source_spct` | N | N | Y | Y | N | `response_spct` | `response_spct` | | `source_spct` | N | N | Y | Y | N | `filter_spct` (T) | `source_spct` | | `source_spct` | N | N | Y | Y | N | `filter_spct` (A) | `source_spct` | | `source_spct` | N | N | Y | Y | N | `reflector_spct` | `source_spct` | | `source_spct` | N | N | N | N | N | `object_spct` | -- | | `source_spct` | N | N | Y | N | N | `waveband` (no BSWF) | `source_spct` | | `source_spct` | N | N | Y | N | N | `waveband` (BSWF) | `source_spct` (effective) | ------------------------------------------------------------------------ ```{r, bin-oper-1} sun.spct * sun.spct ``` When meaningful, operations between different spectra are also allowed. For example, it is possible to simulate the effect of a filter on a light source by multiplying (or convolving) the two spectra. ```{r, bin-oper-2} sun.spct * polyester.spct ``` If we have two layers of the filter, this can be approximated using either of these two statements. ```{r, bin-oper-3} sun.spct * polyester.spct * polyester.spct sun.spct * polyester.spct^2 ``` Operators are also defined for operations between a spectrum and a numeric vector (with normal recycling). ```{r, bin-oper-4} sun.spct * 2 2 * sun.spct sun.spct * c(0,1) ``` There is one special case, for `chroma_spct`: if the numeric operand has length three, containing three *named* values *x*, *y* and *z*, the corresponding value is used for each of the chromaticity *columns* in the `chroma_spct`. Un-named values or differently named values are not treated specially. Operators are also defined for operations between an spectrum and a `waveband` object. The next to code chunks demonstrate how the class of the result depends on whether the `waveband` object describes a range of wavelengths or a range of wavelengths plus a BSWF. ```{r, bin-oper-5} sun.spct * UVB.wb ``` ```{r, bin-oper-6, eval=FALSE} sun.spct * CIE.wb ``` And of course these operations can be combined into more complex statements, including parentheses, when needed. The example below estimates the difference in effective spectral irradiance according to the CIE98 definition, between sunlight and sunlight filtered with a polyester film. Of course, the result is valid only for the solar spectral data used, which corresponds to Southern Finland. ```{r, bin-oper-7, warning=TRUE, purl=FALSE} sun.spct * CIE.wb - sun.spct * polyester.spct * CIE.wb ``` ### Unary operators and maths functions Many common maths functions, as well as unary minus and plus, are implemented for spectral objects (see Table 6). ------------------------------------------------------------------------ **Table 6. Unary operators and maths functions for spectra.** Classes for which they are implemented and class of the result. All operations marked Y are allowed, those marked N are not implemented and return `NA` and issue a warning. Additional supported functions: `log2, log10, sin, cos, tan, asin, acos, atan, sinpi, cospi, tanpi, signif, floor, ceiling, trunc, sign, abs`. | e1 | `+, -` | `log, exp` | trig. | `round` | `sqrt` | value | |:----------|:---------:|:---------:|:---------:|:---------:|:---------:|:----------| | `raw_spct` | Y | Y | Y | Y | Y | `raw_spct` | | `cps_spct` | Y | Y | Y | Y | Y | `cps_spct` | | `calibration_spct` | Y | Y | Y | Y | Y | `calibration_spct` | | `source_spct` | Y | Y | Y | Y | Y | `source_spct` | | `filter_spct` | Y | Y | Y | Y | Y | `filter_spct` | | `reflector_spct` | Y | Y | Y | Y | Y | `reflector_spct` | | `object_spct` | N | N | N | N | N | -- | | `response_spct` | Y | Y | Y | Y | Y | `response_spct` | | `chroma_spct` | Y | Y | Y | Y | Y | `chroma_spct` | ------------------------------------------------------------------------ ```{r, unary-oper-1} -sun.spct sqrt(sun.spct) ``` ### Options Table 7 lists all the recognized options affecting maths operators and functions, and their default values. Within the suite all functions have a default value which is used when the options are undefined. Options are set using base R's function `options`, and queried with functions `options` and `getOption`. ------------------------------------------------------------------------ **Table 7. Options used in the 'r4photobiology suite' and recognized by methods, operators and functions in the 'photobiology' package.** Option names, accepted and default values, and the purpose of the options are given. Option `photobiology.verbose` is set to the value of R's own `verbose` option at the time the 'photobiology' package is attached to the session. | Option | `values`, **default** | purpose, *unit* | *convenience function* | |:--------------|:---------------|:---------------------------|---------------| | **Base R** | | | | | `digits` | 7 | $d - 3$ used by `summary` | | | **tibble** | | | | | tibble.print_max | **20** | Maximum number of rows printed | | | tibble.print_min | **10** | Number of rows printed if row number threshold is exceeded | | | tibble.width | **NULL** | Output width | | | **r4photobiology** | | | | | photobiology.radiation.unit | **"energy"** | $W\,m^{-2}\,nm^{-1}$ | `using_energy()`, `energy_as_default()` | | | `"photon"` | $\mu mol\,m^{-2}\,nm^{-1}$ | `using_photon()`, `photon_as_default()` | | photobiology.filter.qty | **"transmittance"** | $/1$ | `using_Tfr()`, `Tfr_as_default()` | | | `"absorptance"` | $/1$ | `using_Afr()`, `Afr_as_default()` | | | `"absorbance"` | a.u. $\log_{10}$ base | `using_A()`, `A_as_default()` | | photobiology.strict.range | `NA` | skip range test | `strict_range_as_default(NA)` | | | `TRUE` | trigger an error | `strict_range_as_default(TRUE)` | | | **FALSE** | trigger a warning | `strict_range_as_default(FALSE)` | | photobiology.check.spct | **TRUE** | enable `check_spct()` | `enable_check_spct()` | | | `FALSE` | disable `check_spct()` | `disable_check_spct()` | | photobiology.waveband.trim | `FALSE` | exclude | `wb_trim_as_default(FALSE)` | | | **TRUE** | trim or exclude | `wb_trim_as_default(TRUE)` | | photobiology.use.cached.mult | **FALSE** | do not cache intermediate results | `use_cached_mult_as_default(FALSE)` | | | `TRUE` | cache intermediate results | `use_cached_mult_as_default(TRUE)` | | photobiology.verbose | `FALSE` | minimal warnings and messages | `verbose_as_default(FALSE)` | | | `TRUE` | all warnings and messages | `verbose_as_default(TRUE)` | The behaviour of the operators defined in this package depends on the value of two global options. For example, if we would like the operators to operate on spectral photon irradiance and return spectral photon irradiance instead of spectral energy irradiance, this behaviour can be set, and will remain active until unset or reset. It is possible to change options using base R's function `options()` as shown in the next code chunk. The other options listed in the Table above can be set similarly, while to unset any option, they can be given a `NULL` value. ```{r, options-1} options(photobiology.radiation.unit = "photon") sun.spct * UVB.wb options(photobiology.radiation.unit = "energy") sun.spct * UVB.wb ``` However using convenience functions is easier. The chunk above can be rewritten as below. ```{r, options-2} photon_as_default() sun.spct * UVB.wb energy_as_default() sun.spct * UVB.wb unset_radiation_unit_default() ``` Furthermore, it is possible to temporarily change the options for the evaluation of a single, possibly compound, R expression using a different syntax, reminiscent of that of R's `with()`. ```{r, options-3} using_photon(sun.spct * UVB.wb) using_energy(sun.spct * UVB.wb) ``` ## Transformations: methods and functions In this section we describe methods and functions that take one or more spectral objects, and in some cases also waveband objects, as arguments and return another spectral object (see Tables 8 and 9) or that take a collection of spectral objects, and in some cases also waveband objects, as arguments and return a collection of spectral objects. ------------------------------------------------------------------------ **Table 8. Transformation methods for spectra.** Key: + available, -- not available. | methods | raw/cps | source | response | filter | reflector | object | chroma | |:---------|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | merge | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | rbindspct | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | e2q, q2e | -- | \+ | \+ | -- | -- | -- | -- | | A2T, T2A | -- | -- | -- | \+ | -- | -- | -- | | Afr2T, T2Afr | -- | -- | -- | \+ | -- | -- | -- | | any2T, any2A, any2Afr | -- | -- | -- | \+ | -- | -- | -- | | convertTfrType | -- | -- | -- | \+ | -- | -- | -- | | convertThickness | -- | -- | -- | \+ | -- | -- | -- | | subset | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | clip_wl | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | trim_wl | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | (trim_spct) | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | thin_wl | \+ | \+ | \+ | \+ | \+ | -- | -- | | interpolate_wl | -- | \+ | \+ | \+ | \+ | \+ | \+ | | (interpolate_spct) | -- | \+ | \+ | \+ | \+ | \+ | \+ | | fscale | \+ | \+ | \+ | \+ | \+ | -- | -- | | fshift | \+ | \+ | \+ | \+ | \+ | -- | -- | | normalize | \+ | \+ | \+ | \+ | \+ | -- | -- | | clean | -- | \+ | \+ | \+ | \+ | -- | -- | | despike | -- | \+ | \+ | \+ | \+ | -- | -- | | *maths operators* | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | *maths functions* | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | tag | -- | \+ | \+ | \+ | \+ | \+ | \+ | | untag | -- | \+ | \+ | \+ | \+ | \+ | \+ | ------------------------------------------------------------------------ ------------------------------------------------------------------------ **Table 9. Transformation methods for collections of spectra.** Key: + available, -- not available, **ms** use `msmsply()` or `convolve_each()` to apply function or operator to collection members. | methods | raw/cps | source | response | filter | reflector | object | chroma | |:---------|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | convolve_each | -- | \+ | \+ | \+ | \+ | \+ | \+ | | msmsply | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | msdply | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | mslply | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | msaply | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | rbindspct | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | c | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | *maths operators* | ms | ms | ms | ms | ms | ms | ms | | *maths functions* | ms | ms | ms | ms | ms | ms | ms | | e2q, q2e | -- | \+ | \+ | -- | -- | -- | -- | | A2T, T2A | -- | -- | -- | \+ | -- | -- | -- | | Afr2T, T2Afr | -- | -- | -- | \+ | -- | -- | -- | | any2T, any2A, any2Afr | -- | -- | -- | \+ | -- | -- | -- | | convertTfrType | -- | -- | -- | \+ | -- | -- | -- | | convertThickness | -- | -- | -- | \+ | -- | -- | -- | | clip_wl | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | trim_wl | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | trim2overlap | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | extend2extremes | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | (trim_mspct) | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | thin_wl | \+ | \+ | \+ | \+ | \+ | -- | -- | | interpolate_wl | -- | \+ | \+ | \+ | \+ | \+ | \+ | | (interpolate_mspct) | -- | \+ | \+ | \+ | \+ | \+ | \+ | | fscale | \+ | \+ | \+ | \+ | \+ | -- | -- | | fshift | \+ | \+ | \+ | \+ | \+ | -- | -- | | normalize | \+ | \+ | \+ | \+ | \+ | -- | -- | | clean | -- | \+ | \+ | \+ | \+ | -- | -- | | despike | -- | \+ | \+ | \+ | \+ | -- | -- | | tag | -- | \+ | \+ | \+ | \+ | \+ | \+ | | untag | -- | \+ | \+ | \+ | \+ | \+ | \+ | ------------------------------------------------------------------------ ### Manipulating spectra Sometimes, especially for plotting, we may want to row-bind spectra. When the aim is that the returned object retains its class and other attributes like the time unit. Package '**photobiology**' provides function `rbinspct` for row-binding spectra, with the necessary checks for consistency of the bound spectra. ```{r, manip-1} # STOPGAP shade.spct <- sun.spct ``` By default an ID factor named `spct.idx` is added allow to identify the source of the observations after the binding. If the supplied list has named members, then these names are used as factor levels. If a character value is supplied to as `idfactor` argument, this is used as name for the factor. ```{r, manip-2} rbindspct(list(sun.spct, shade.spct)) rbindspct(list(A = sun.spct, B = shade.spct), idfactor = "site") ``` The name of the ID factor is stored as metadata in attribute `"idfactor"` of the spectral data object. Special *Extract* methods for spectral objects have been implemented. These are used by default and preserve the attributes used by this package, except when the returned value is a single column from the spectral object. ```{r, manip-3} sun.spct[1:10, ] sun.spct[1:10, 1] sun.spct[1:10, 1, drop = TRUE] sun.spct[1:10, "w.length", drop = TRUE] ``` In contrast to `trim_spct`, `subset` never interpolates or inserts *hinges*. On the other hand, the `subset` argument accepts any logical expression and can be consequently used to do subsetting, for example, based on factors. Both `subset()` and `trim()` methods preserve attributes. ```{r, manip-4} subset(sun.spct, s.e.irrad > 0.2) subset(sun.spct, w.length > 600) subset(sun.spct, c(TRUE, rep(FALSE, 99))) ``` R's Extract methods `$` and `[[` can be used to extract whole columns. Replace methods `$<-` and `[<-` have definitions for spectral objects, which allow their safe use. They work identically to those for data frames but check the validity of the spectra after the replacement. ### Conversions between radiation units The functions `e2q` and `q2e` can be used on source spectra to convert spectral energy irradiance into spectral photon irradiance and vice versa. They can be also used with response spectra. A second optional argument sets the action with `"add"` and `"replace"` as possible values. By default these functions use normal copy semantics but also support reference semantics. ```{r, manip-5} e2q(sun.spct, "add") e2q(sun.spct, "replace") ``` When a normalized spectrum is converted, the normalization _criteria_ are applied to the converted data after reversing the pre-existing normalization. ### Conversions among transmission quantities For `filter_spct` objects functions `any2T()`, `any2A()`, and `any2Afr()` allow conversion among spectral transmittance, spectral absorptance and spectral absorbance. Although some conversions require a known reflectance, either as an `Rfr` column in the data or an `Rfr.constant` in the attribute `filter.properties`. ```{r} polyester.spct ``` ```{r} any2Afr(polyester.spct, "add") ``` ```{r} any2Afr(polyester.spct, "replace") ``` When a normalized spectrum is converted, the normalization _criteria_ are applied to the converted data after reversing the pre-existing normalization. ### Normalizing a spectrum Normalization is an approach to rescaling a spectrum so that the spectral quantity takes a fixed value, in most cases exactly 1, at a specific wavelength. Frequently this wavelength corresponds to the maximum value of the spectral quantity. $$Q_\mathrm{norm}(\lambda) = \frac{Q(\lambda)}{\max{Q(\lambda)}}$$ When normalizing to a specific wavelength, say 300 nm, the equation becomes $$Q_\mathrm{norm}(\lambda) = \frac{Q(\lambda)}{{Q(\lambda = 300)}}$$ Function `normalize` allows normalizing a spectrum to a value of one at an arbitrary wavelength (nm) or to the wavelength of either the maximum or the minimum spectral value. It supports all the different spectral classes. In the example we use a `source_spct` object. (The functions can be called with names spelled with "s" or "z".) ```{r, manip-6} normalize(sun.spct) ``` Which is equivalent to supplying `"max"` as argument to `norm`, it is also possible to give a range within which the maximum should be searched. ```{r, manip-7} normalize(sun.spct, range = PAR.wb, norm = "max") ``` It is also possible to normalize to an arbitrary wavelength within the range of the data, even if it is not one of the wavelength values present in the spectral object, as interpolation is used when needed. ```{r, manip-8} normalize(sun.spct, norm = 600.3) ``` The normalization status of a spectral object can be tested with method `is_normalized()` and the normalization used can be recalled with method `getNormalized()`. Function `str()` is used for an output more compact than with the default `print()`. ```{r} norm_sun.spct <- normalize(sun.spct) is_normalized(norm_sun.spct) str(getNormalization(norm_sun.spct)) ``` A normalized spectrum can be restored to its original scaling. ```{r} restored_sun.spct <- normalize(norm_sun.spct, norm = "undo") ``` Once a spectrum is normalized, summary methods that return values in absolute units such as `irrad()`, will trigger an error if applied. Ratios and similar summaries that are invariant with respect to normalization can be calculated. Already normalized spectra can be normalized again. Applying method `fscale()` removes the normalization and clears the corresponding attribute where the normalization information is stored. This attribute also can be cleared with method `setNormalized()`, but this is rarely valid or of any use. ### Rescaling a spectrum While normalization is done with respect to a single wavelength, rescaling as implemented in this package is more flexible. The scaling factor is computed as a summary over a range of wavelengths. This allows us, among other things to rescale spectral irradiance so that irradiance in a specific waveband is equal to one, or to some other user supplied target value. $$Q^\prime(\lambda) = \frac{Q(\lambda)}{f(Q(\lambda))_{\ \mathrm{for}\ \ \lambda_1\geq\lambda>\lambda_2}}\times \alpha$$ where $\alpha$ is the target value for $f(Q(\lambda))$. In other words, function `fscale()` rescales a spectrum by dividing each spectral data value by a value calculated with a function (f) selected by a character string ("total" or "mean"), or an actual R function which can accept the spectrum object supplied as its first argument. Additional named arguments can be also passed. How metadata is set in the returned object is controlled by a logical argument to parameter `set.scaled`. If `TRUE`, the data are labelled as being expressed in relative units and if `FALSE` this attribute is not set. The default argument depends on the value passed as argument to `target`. If this value is one, then the data are marked as no longer being expressed in absolute units. If the `target` is any other numeric value then it is assumed that the intention is to re-scale the data and that absolute physical units remain meaningful. It is important that the metadata matches the actual use as printing of summaries and plot labelling depends on them. ```{r, manip-9} fscale(sun.spct) fscale(sun.spct, set.scaled = FALSE) fscale(sun.spct, target = 100) fscale(sun.spct, target = 100, set.scaled = TRUE) ``` The default for `f` can be overridden. R functions passed as argument should be suitable for summarizing spectral objects as they will receive as first argument a spectrum rather than a numeric vector. Behind this requirement is the need to take into account wavelength steps for integration to be meaningful. In the example below we compute energy and photon irradiances. (The character arguments `"mean"` and `"total"` call functions specific to spectral data.) ```{r, manip-9a} fscale(sun.spct, f = "integral") fscale(sun.spct, range = PAR.wb, f = e_irrad) fscale(sun.spct, range = PAR.wb, f = q_irrad, target = 800e-6) ``` In the third example, the spectral data is rescaled so that the corresponding photosynthetically-active irradiance is equal to one. The normalization status of a spectral object can be tested with method `is_normalized()` and the normalization used can be recalled with method `getNormalized()`. ```{r, manip-9b} my.spct <- fscale(sun.spct) is_scaled(my.spct) getScaled(my.spct) ``` Once a spectrum is scaled, summary methods that return values in absolute units such as `irrad()`, will trigger a warning if applied. Ratios and similar summaries that are invariant with respect to normalization do not trigger warnings. Applying method `normalize()` removes the scaling and clears the corresponding attribute where the scaling information is stored. This attribute also can be cleared with method `setScaled()`, and this can be useful is some cases such as when irradiance has been measured separately from the emission spectrum. ### Shifting the zero of the spectral data scale While re-scaling relies on a multiplicative factor shifting of the scale is done by addition (or subtraction) or constant. This allows us, among other things to remove a baseline computed from a region known to be equal to zero. $$Q^\prime(\lambda) = Q(\lambda) - f(Q(\lambda))_{\ \mathrm{for}\ \ \lambda_1\geq\lambda>\lambda_2}$$ Function `fshift()` shifts the zero of the scale of a spectrum by subtracting from each spectral data value a value calculated with a function (f) selected by a character string ("mean", "min" or "max"), or an actual R function which can accept the spectrum object supplied as its first argument. The range argument selects a region of the spectrum to be used as *reference* in the calculation of the summary. ```{r, manip-10} fshift(white_led.source_spct, range = UVB.wb, f = "mean") fshift(sun.spct, range = c(280,290), f = "min") ``` In the first example, the spectral data shifted so that the mean spectral irradiance becomes zero for the UV-B region. In the second example the minimum value in the range of wavelengths between 280\~nm and 290\~nm is used as zero reference for the scale. ### Replacing off-range spectral data values Method `clean()` should be used with care as off-range values stem almost always from calibration errors or measuring noise. This function allows one to replace such values, but in many cases a zero shift or rescaling could be a better option. Even when the off-range values are the result of random noise, replacing them with the boundary values can cause bias, by censoring the data. In such cases smoothing may be applied first to reduce the possible bias caused by `clean()`. Here we create *artificial* off-range values by subtracting a constant from each spectrum. ```{r, manip-11} clean(polyester.spct - 0.053) ``` It is possible to restrict the *cleaning* to a range of wavelengths and to provide a value to be used as replacement for the off-range data. ```{r, manip-11a} clean(sun.spct - 0.01, range = c(280.5, 282), fill = NA) ``` ### Removing spikes Method `despike()` replaces spikes (very narrow peaks, or valleys) by values estimated from neighbouring pixels. This method should be also used carefully and the results inspected as it can remove features of interest from the data. Usually one knows from theory or experience if spikes can be real features or not. ```{r, manip-12} spikes(sun.spct) ``` ```{r, manip-12a} my_sun.spct <- despike(sun.spct) spikes(my_sun.spct) ``` ### Smoothing of spectra We can use smoothing both to average out random variation among nearby pixels caused by measuring noise, or to filter out the real fine structure of a spectrum to better assess the larger features. The methods described here work by averaging the spectral data from neighbouring wavelengths. The simplest methods are running medians and running means (or "boxcar smoothing" in the terminology used in some software). There are several methods for smoothing available in R based on different algorithms. Some of these methods also automatically set the degree of smoothing based on the data being smoothed. Package 'photobiology' defines function `smooth_spct`, which at the moment supports three different methods. Two are wrappers on R's own methods and a third one, `"custom"`, is designed to use stronger smoothing for values close to zero, where noise in spectral irradiance measurements is proportionally more. The strength of the smoothing and the range of wavelengths operated upon can be adjusted through arguments to parameters. The `comment` attribute is updated. ```{r} smooth_spct(sun.spct) ``` ```{r} smooth_spct(polyester.spct, method = "supsmu", strength = 2) ``` ### Wavelength interpolation Converting spectra available at a given set of wavelengths values to a different one, is frequently needed when operating with several spectra of different origin. One can increase the *apparent* resolution by interpolation, and reduce it by local averaging or smoothing and resampling. The same function works on all `spct` objects, interpolating every numeric column except `w.length` which is set to the new wavelength values supplied as argument. The optional argument `fill.value` controls what value is assigned to the "interpolated" data columns at wavelengths in the new data that are outside the range of wavelengths in the original spectrum being interpolated. ```{r, manip-13} interpolate_wl(sun.spct, seq(400, 500, by = 0.1)) ``` ### Trimming, clipping and thining #### Method `clip_wl()` Sometimes it is desirable to change the range of wavelengths included in a spectrum. If we are interested in a given part of the spectrum, there is no need to do calculations or plotting the whole spectrum. To select part of a spectrum based on a range of wavelengths we may use the `clip_wl()` method. Method `clip_wl()` simply selects a range from the existing spectrum, and unless the range exactly matches the wavelength values present in the spectrum, the range of wavelengths in the returned clipped spectrum will be slightly narrower than the requested by `range`. The range of wavelengths expressed in nanometres can be given as numeric vector of length two. ```{r, trim-1} clip_wl(sun.spct, range = c(400, 402)) clip_wl(sun.spct, range = c(400, NA)) ``` As for other methods in the package, the range can be also supplied as a `waveband` object, or any other object for which `range()` returns a numeric range. Even a different spectrum object is acceptable, and in many cases useful. ```{r, trim-2} clip_wl(sun.spct, range = UVA.wb) ``` The result can be a spectrum of length zero. ```{r, trim-3} clip_wl(sun.spct, range = c(100, 200)) ``` #### Method `trim_wl()` Sometimes, we need more flexibility. We may want to replace the observed values outside a certain range or expand the range of wavelengths, filling the expansion of all other variables with a certain value (i.e. a number, or NA.). In contrast to *clipping* (or functionally equivalent, indexing, or subsetting), *trimming* ensures that there will be spectral data returned at the boundaries of the trimmed region. These values are obtained by interpolation when they are not already present in the data. More flexibility is available in method `trim_wl()`, to which we can supply arguments `range`, `use.hinges`, and `fill`. By default interpolation is used at the boundaries of the `range`, in which case the range of wavelengths in the returned spectrum is that passed as argument to `range`. ```{r, trim-4} trim_wl(sun.spct, c(282.5, NA)) clip_wl(sun.spct, c(282.5, NA)) ``` As for `clip_wl()` the range can be also supplied as a `waveband` object, or any other object for which `range()` returns a numeric range. Even a different spectrum object is acceptable. ```{r, trim-5} trim_wl(sun.spct, PAR.wb) ``` The default for `fill` is `NULL` which results in deletion values outside the trimmed region. However, it is possible to supply a different argument, to be used to replace the off-range data values. ```{r, trim-6} trim_wl(sun.spct, c(281.5, NA), fill = NA) ``` Furthermore, when fill is not `NULL`, expansion is possible. ```{r, trim-7} trim_wl(sun.spct, c(275, NA), fill = 0) ``` By default interpolation at the boundaries is used, but setting `use.hinges` to `FALSE` results in clipping, a behaviour similar to that of `clip_wl` only if `fill == NULL`. ```{r, trim-8} trim_wl(sun.spct, c(281.5, NA), fill = NA) trim_wl(sun.spct, c(281.5, NA), fill = NA, use.hinges = FALSE) ``` When `use.hinges == TRUE` and expansion or replacement is done, two observations are inserted at each boundary, differing in wavelength by $1 \times 10^{-12}$,nm to prevent rounding errors in later calculations. #### Functions `trim2ovelap()` and `extend2extremes()` Functions `trim2ovelap()` and `extend2extremes()` are defined only for collections of spectra. They are convenience functions, as they handle special use cases of `trim_wl()` with a simplified syntax. They can be used to make the wavelength range of the different members of a collection of spectra consistent, either by trimming all spectra to the range of overlapping wavelengths, or by extending the wavelength ranges as needed and filling the added spectral values with a constant value. ```{r} trim2overlap(two.mspct) ``` ```{r} extend2extremes(two.mspct, fill = 0) ``` #### Method `thin_wl()` One possible way of decreasing the storage space occupied by spectral data is to vary the density of wavelength values stored based on the local change in slope (second derivative) of the plot a spectrum. Method `thin_wl()` can be used to remove or "thin" down the wavelength values stored in those regions of the spectrum where this would result in minimal loss of information. The algorithm currently in use is suboptimal in the removal of wavelength values but fast and easy to implement. **Which wavelength values are retained or not could change with future implementations.** ```{r} nrow(yellow_gel.spct) wl_stepsize(yellow_gel.spct) thinned.spct <- thin_wl(yellow_gel.spct) nrow(thinned.spct) wl_stepsize(thinned.spct) ``` The strength of the thinning can be adjusted by passing arguments to parameters `max.wl.step` and `max.slope.delta`. This method is implemented for objects of all spectral classes that a single default data column in addition to wavelengths and also for collections of those objects. **NOTE:** Thinning, obviously, almost always changes spectra originally expressed at the same set of wavelength values into spectra expressed at two different sets of wavelengths values. This forbid some operations among them, like _parallel_ operations in collections of spectra or combining them into a wide data frame. Most of these restrictions have been removed by implementing on-the-fly interpolation in version 0.10.15. ### Convolving weights It is very instructive to look at weighted spectral data to understand how effective irradiances are calculated. Plotting effective spectral irradiance data can be very informative when analysing interactions among photoreceptors and ambient radiation. It can also illustrate the large effect that small measuring errors can have on the estimated effective irradiances or exposures when SWFs have a steep slope. For biologically effective energy irradiance $E_\mathrm{BE}$ the following equation describes the convolution, wavelength by wavelength, between spectral energy irradiance and a biological spectral weighting function $$E_\mathrm{BE}(\lambda) = E(\lambda) \times w_E(\lambda)$$ or for biologically effective photon irradiance $Q_\mathrm{BE}$ $$Q_\mathrm{BE}(\lambda) = Q(\lambda) \times w_Q(\lambda)$$ For the convolution to be possible, both spectral irradiance and spectral weights should be available for the same basis of expression and at the same discrete wavelength values. #### Individual spectra The multiplication operator is defined for operations between a `source_spct` and a `waveband`, so this is the easiest way of doing the calculations. ```{r, weights-1} sun.spct * CIE.wb ``` ### Tagging with bands and colours We call tagging, to the process of adding reference information to spectral data. For example we can add a factor indicating regions or bands in the spectrum. We can add also information on the colour, as seen by humans, for each observed value, or for individual regions or bands of the spectrum. In most cases this additional information is used for annotations when plotting the spectral data. #### Individual spectra The function `tag` can be used to tag different parts of a spectrum according to wavebands. ```{r, tag-1} tag(sun.spct, PAR.wb) tag(sun.spct, UV_bands.lst) ``` The added factor and colour data can be used for further processing or for plotting. Information about the tagging and wavebands is stored in an attribute `tag.attr` in every tagged spectrum, this yields a more compact output and keeps a *trace* of the tagging. ```{r, tag-2} tg.sun.spct <- tag(sun.spct, PAR.wb) attr(tg.sun.spct, "spct.tags") ``` Additional functions are available which return a tagged spectrum and take as input a list of wavebands, but no spectral data. They *build* a spectrum from the data in the wavebands, and are useful for plotting the boundaries of wavebands. ```{r, tag-3} wb2tagged_spct(UV_bands.lst) wb2rect_spct(UV_bands.lst) ``` Function `wb2tagged_spct` returns a tagged spectrum, with two rows for each waveband, corresponding to the low and high wavelength boundaries, while function `wb2rect_spct` returns a spectrum with only one row per waveband, with `w.length` set to its midpoint but with additional columns `xmin` and `xmax` corresponding to the low and high wavelength boundaries of the wavebands. Function `is_tagged` can be used to query if an spectrum is tagged or not, and function `untag` removes the tags. ```{r, tag-4} tg.sun.spct is_tagged(tg.sun.spct) untg.sun.spct <- untag(tg.sun.spct) is_tagged(untg.sun.spct) ``` In the chuck above, we can see how this works, using in this case the default `byref = FALSE` which adds the tags to a copy of the spectrum object. In contrast, setting `byref = TRUE` adds the tags in place, or "by reference"", to the spct object supplied as argument. Passing arguments by reference is unusual for R and is best avoided. ```{r} is_tagged(untg.sun.spct) untag(tg.sun.spct, byref = TRUE) is_tagged(untg.sun.spct) ``` ## Summaries Summaries can be calculated both from individual spectral objects (Table 10) and from collections of spectral objects (Table 11). They return a *simpler* object than the spectral data in their arguments. For example a vector of numeric values, possibly of length one, in the case of individual spectra, or a data frame containing one row of summary data for each spectrum in a collection of spectra. ------------------------------------------------------------------------ **Table 10. Summary methods for spectra.** Key: + available, - not available, (+) object_spct objects can be converted into filter_spct or reflector_spct objects before applying these methods. | methods | raw/cps | source | response | filter | reflector | object | chroma | |:---------------|:-------:|:------:|:--------:|:------:|:---------:|:------:|:------:| | irrad | -- | \+ | -- | -- | -- | -- | -- | | e_irrad | -- | \+ | -- | -- | -- | -- | -- | | q_irrad | -- | \+ | -- | -- | -- | -- | -- | | fluence | -- | \+ | -- | -- | -- | -- | -- | | e_fluence | -- | \+ | -- | -- | -- | -- | -- | | q_fluence | -- | \+ | -- | -- | -- | -- | -- | | ratio | -- | \+ | -- | -- | -- | -- | -- | | e_ratio | -- | \+ | -- | -- | -- | -- | -- | | q_ratio | -- | \+ | -- | -- | -- | -- | -- | | qe_ratio | -- | \+ | -- | -- | -- | -- | -- | | eq_ratio | -- | \+ | -- | -- | -- | -- | -- | | response | -- | -- | \+ | -- | -- | -- | -- | | e_response | -- | -- | \+ | -- | -- | -- | -- | | q_response | -- | -- | \+ | -- | -- | -- | -- | | transmittance | -- | -- | -- | \+ | -- | \+ | -- | | absorptance | -- | -- | -- | \+ | -- | \+ | -- | | absorbance | -- | -- | -- | \+ | -- | \+ | -- | | reflectance | -- | -- | -- | -- | \+ | \+ | -- | | wl_range | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_min | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_max | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_stepsize | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_expanse | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_midpoint | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | labels | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | summary | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | peaks | -- | \+ | \+ | \+ | \+ | (+) | \- | | spikes | -- | \+ | \+ | \+ | \+ | (+) | \- | | valleys | -- | \+ | \+ | \+ | \+ | (+) | \- | | wls_at_target | -- | \+ | \+ | \+ | \+ | (+) | \- | | integrate_spct | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | average_spct | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | color_of | -- | \+ | -- | -- | -- | -- | -- | ------------------------------------------------------------------------ As mentioned above, summary methods for collections of spectra return data frame objects. In many cases preserving the attributes from the different members of the collection in the returned value is important, and can be achieved easily by passing a suitable character vector as argument to parameter `attr2tb`, using the same syntax as described for function `add_attr2tb`. ------------------------------------------------------------------------ **Table 11. Summary methods for collections of spectra.** Key: + available, \* `attr2tb` supported, -- not available, **ms** use `msmsply()` to apply function to collection members, **d** use `msdply()`, **l** use `mslply` to apply function to collection members, **a** use `msaply` to apply function to collection members. | methods | raw/cps | source | response | filter | reflector | object | chroma | |:--------|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | irrad | -- | \* | -- | -- | -- | -- | -- | | e_irrad | -- | \* | -- | -- | -- | -- | -- | | q_irrad | -- | \* | -- | -- | -- | -- | -- | | fluence | -- | \* | -- | -- | -- | -- | -- | | e_fluence | -- | \* | -- | -- | -- | -- | -- | | q_fluence | -- | \* | -- | -- | -- | -- | -- | | ratio | -- | \* | -- | -- | -- | -- | -- | | e_ratio | -- | \* | -- | -- | -- | -- | -- | | q_ratio | -- | \* | -- | -- | -- | -- | -- | | qe_ratio | -- | \* | -- | -- | -- | -- | -- | | eq_ratio | -- | \* | -- | -- | -- | -- | -- | | response | -- | -- | \* | -- | -- | -- | -- | | e_response | -- | -- | \* | -- | -- | -- | -- | | q_response | -- | -- | \* | -- | -- | -- | -- | | transmittance | -- | -- | -- | \* | -- | \* | -- | | absorptance | -- | -- | -- | \* | -- | \* | -- | | absorbance | -- | -- | -- | \* | -- | \* | -- | | reflectance | -- | -- | -- | -- | \* | \* | -- | | color_of | -- | \+ | -- | -- | -- | -- | -- | | wl_range | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_min | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_max | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_stepsize | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_spread | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | wl_midpoint | \+ | \+ | \+ | \+ | \+ | \+ | \+ | | labels | **l** | **l** | **l** | **l** | **l** | **l** | **l** | | summary | **l** | **l** | **l** | **l** | **l** | **l** | **l** | | peaks | -- | \+ | \+ | \+ | \+ | (+) | \- | | valleys | -- | \+ | \+ | \+ | \+ | (+) | \- | | spikes | -- | \+ | \+ | \+ | \+ | (+) | \- | | wls_at_target | -- | \+ | \+ | \+ | \+ | (+) | \- | | integrate_spct | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | | average_spct | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | **a, d, l** | ------------------------------------------------------------------------ ### Summary Specialized definitions of `summary` and the corresponding `print` methods are available for spectral objects. Attributes `"what.measured"`, `"when.measured"` and `"where.measured"` are included in the summary print out only if set in the spectral object summarized. ```{r, summary-1} summary(sun.spct) ``` In the case of collections of spectra, by default a conscise summary is printed. ```{r, summary-2} summary(two_filters.mspct) ``` However, the spectra can be also summarized individually. ```{r, summary-2-each} summary(two_filters.mspct, expand = "each") ``` In the case of multiple spectra in long form, by default a single summary is printed. ```{r, summary-2a} summary(two_filters.spct) ``` Passing `expand = "collection"` we obtain the same condensed summary as shown above as the default for collections of spectra. (Output not shown here.) ```{r, summary-2a-collection, eval=FALSE} summary(two_filters.spct, expand = "collection") ``` Passing `expand = "each"` we obtain the same expanded summary as shown above for `expand = "each"` for collections of spectra. (Output not shown here.) ```{r, summary-2a-each, eval=FALSE} summary(two_filters.spct, expand = "each") ``` ### Wavelength #### Individual spectra The *usual* and three new summary methods are available for spectra, but redefined to return _wavelength_-based summaries in nanometres (nm) directly from spectra. For clarity, synonyms with names starting with `wl_` are provided. The three new generic methods `midpoint()`, `expanse()` and `stepsize()` are also defined for `numeric`. ```{r, summary-3} wl_range(sun.spct) wl_min(sun.spct) wl_max(sun.spct) wl_midpoint(sun.spct) wl_expanse(sun.spct) wl_stepsize(sun.spct) ``` #### Collections of spectra ------------------------------------------------------------------------ Most frequently used summary methods are implemented for collections of spectra. See the Table 11 for those methods that need to be applied with functions `msaply`, `msdply` or `mslply` to members in a collection returning the results in an array (vector, or matrix), a data frame or a list object. In many cases depending of the class desired for the result, one can chose a suitable *apply* function, and sometimes it is best to use such a function, even when the corresponding method is implemented for collections of spectra. ------------------------------------------------------------------------ Collections of spectra can be useful not only for time-series of spectra or spectral images, but also when dealing with a small group of related spectra. In the example below we show how to use a collection of spectra for calculating summaries. The spectra in a collection do **not** need to have been measured at the same wavelength values, or have the same number of rows or even of columns. Consequently, in many cases applying the wavelength summary functions described above to collections of spectra can be useful. The value returned is a data frame, with a number of data columns equal to the length of the returned value by the corresponding method for individual spectra. ```{r, summary-4} filters.mspct <- filter_mspct(list(none = clear.spct, pet = polyester.spct, yellow = yellow_gel.spct)) wl_range(filters.mspct) ``` ### Peaks, valleys and spikes Peaks and valleys in spectra are usually features of interest, while in most cases spikes (very narrow peaks or valleys) are undesired "noise". Of course, there are exceptions and for this reason methods used to extract any of these features have a similar interface. Peaks and valleys can be identified simply as local maxima and minima, or their actual location better estimated by fitted a model to the `local maximum or minimum and its neighbours. It is possible to be `strict` when searching and discard non-unique maxima and minima or altrenatively accept them. In practice, a frequent difficulty is separating true local features of a spectrum from measuring noise, or from less prominent peaks and valleys that can be considering as not interesting. One approach is to use global thresholds for the height of peaks and depth of valleys. Locally maxima and minima are searched within a moving window, and their heigh or depth can be compared to a local threshold, relative to a computed within-moving-window baseline such as the local median, minimum or MAD. #### Individual spectra Methods `peaks` and `valleys` take spectra as first argument and return a subset of the spectral object data corresponding to local maxima and local minima of the measured variable. `span` defines the width of the *window* used as a number of observations. The default is `span = 5` which finds many peaks and valleys in the solar spectrum. Of `r nrow(sun.spct)` in the input `r `nrow(peaks(sun.spct))` rows are returned by `peaks(sun.spct)` and `r nrow(valleys(sun.spct))` rows are returned by `valleys(sun.spct)` ```{r, summary-5} nrow(sun.spct) nrow(peaks(sun.spct)) nrow(valleys(sun.spct)) ``` With a wider window, here 51 consecutive wavelengths, fewer peaks and valleys are returned. ```{r, summary-5a} peaks(sun.spct, span = 51) valleys(sun.spct, span = 51) ``` With a value of `NULL` or an `integer` larger than the number of wavelength values passed as argument to `span` method `peaks` returns the row corresponding to the global maximum, the highest vakue in the whole spectrum. ```{r, summary-8} peaks(sun.spct, span = NULL) ``` By passing `refine.wl = TRUE` we request that location of the peaks is *refined* by fitting a model to describe the peak. **The interpolated `w.length` estimate can be usually trusted as good if the data are not noisy, while the `s.e.irrad` and `s.q.irrad` estimates should be always checked as their values are frequently suspect.** ```{r, summary-6a} peaks(sun.spct, span = NULL, refine.wl = TRUE) ``` In the example above, a single peak was fitted, but it is also possible to fit multiple peaks, i.e., when `span` is passed as argument an odd `integer` vale smaller the the number of rows in the input. In the case of `source_spct` and `response_spct` methods `unit.out` can be used to force peaks to be searched using either energy or photon based spectral irradiance. The default is energy, or the option `"photobiology.radiation.unit"` if set. In most cases, the wavelength at the maximum differs when the data are expressed as a flux of energy or as a flux of photons. ```{r, summary-6} peaks(sun.spct, span = NULL, unit.out = "energy") peaks(sun.spct, span = NULL, unit.out = "photon") ``` As `span` is given as a number of successive observations, if the wavelength step varies significantly along the spectrum, the algorithm used, based only on the sequence of values fails, or at least requires careful interpretation of the returned value, except for the global maximum. When the wavelength step size varies strongly a warning is issued. Given that trimming the ends of a spectrum can result in abnormally narrow steps, steps narrower than 1 nm are not taken into account when assessing the variation in step size. Frequently, we are not interested in all local maxima, but instead only in those that exceed a certain heigh globally or locally (within the window of width `span`). Arguments passed to additional parameters make this possible as described in detail in the help page of `peaks`and `valleys` methods. Method `spikes` differs from `peaks` in that it uses a different algorithm that detects only narrow peaks, usually called by the name of spikes. The value returned is as in the case of `peaks` a subset of the original spectrum, however, each spike can be composed of multiple successive observations. ```{r, summary-10} spikes(sun.spct) ``` The parameters controlling the sensitivity also differ as the algorithm is based on a local estimate of variance, constrained to a given maximum width. With `z.threshold = 6` instead of the default `z.threshold = 9` more spikes are detected. ```{r, summary-10a} spikes(sun.spct, z.threshold = 6) ``` Low level functions `find_peaks`, `find_valleys`, `fit_peaks`, `fit_valleys`, and `find_spikes` take numeric vectors as first argument and return a `logical` vector that can be used as an index to extract the rows from a vector, data frame, or spectral object. #### Collections of spectra The methods are also implemented for collections of spectra and multiple spectra stored in long form. ```{r, col-summary-1} peaks(sun_evening.mspct, span = NULL) ``` ```{r, col-summary-1a} peaks(sun_evening.spct, span = NULL) ``` ### Wavelengths at target value Method `wls_at_target` takes a spectrum as first argument and returns a subset of the spectral object data or a new object corresponding to wavelengths at which the spectrum is at the target value. Geometrically is equivalent to finding the wavelengths at which a horizontal *target* line intercepts the curve depicting the spectrum. ```{r, find-wls-1} wls_at_target(Ler_leaf_trns.spct, target = "half.maximum") wls_at_target(Ler_leaf_trns.spct, target = "half.maximum", interpolate = TRUE) ``` This function can be used with all spectral classes defined in the package. In the case of `source_spct` and `response_spct` methods parameter `unit.out` allows to switch between energy and photon units for the search and returned value, while in the case of `filter_spct` methods filter.qty\` allows to switch between transmittance, absorptance and absorbance. #### Collections of spectra The method is also implemented for collections of spectra. ```{r, find-wls-2} wls_at_target(filters.mspct, target = "half.maximum") ``` ### Irradiance Energy irradiance is the integral over wavelengths of spectral energy irradiance $$E = \int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda)\ \mathrm{d}\lambda$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines a waveband, or range of wavelengths. Photon (=quantum) irradiance is the integral over wavelengths of spectral photon irradiance. $$Q = \int_{\lambda = \lambda_1}^{\lambda = \lambda_2} Q(\lambda)\ \mathrm{d}\lambda$$ Spectrally weighted (or biologically effective) energy irradiance is the integral of the result of the convolution of spectral energy irradiance by a spectral weighting function $w(\lambda)$. $$E_{BE} = \int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda) \cdot w_E(\lambda)\ \mathrm{d}\lambda$$ Spectrally weighted (or effective) photon irradiance can be computed similarly with the corresponding weighting function $w_Q(\lambda)$. #### Individual spectra The code using `spct` objects is simple; to integrate the whole spectrum we can use `irrad()` that returns by default energy irradiance, unless an R option is set to make photon-based units the default. When this flexibility is not needed, `e_irrad()` and `q_irrad()` should be preferred. For most examples we use energy-based units and `e_irrad()`, but they also apply unchanged to `q_irrad()` and photon-based units, and *vice-versa* for those examples using `q_irrad()`. The abbreviations *E* and *Q* are used to denote these two quantities, usually with the waveband as a subscript. In what follows "Total" as subscript denotes the whole range of wavelengths in the spectrum. This is the default. ```{r, irrad-1} e_irrad(sun.spct) ``` To integrate one restricted range of wavelengths from a spectrum, we can provide a wavelength definition. In this example, waveband `PAR.wb` giving the definition of *photosynthetically active radiation*. (`PAR.wb` was defined above.) ```{r, irrad-2} e_irrad(sun.spct, PAR.wb) ``` It is also valid to pass as argument for `w.band` a numeric range representing wavelengths in manometers (nm), which is converted into a waveband definition on-the-fly. Note, however, that the automatically assigned name gives only the range. ```{r, irrad-3} e_irrad(sun.spct, c(400, 700)) ``` The 'photobiology' package uses base SI units, so by default photon irradiance (= quantum irradiance) is expressed in $mol\,s^{-1}\,m^{-2}$. We can pass a scaling factor as shown below if needed. ```{r, irrad-4a} q_irrad(sun.spct, PAR.wb, scale.factor = 1e6) # umol s-1 m-2 ``` It is possible to supply a time unit to use, instead of the default of seconds, as basis of expression for the returned value. However, be aware that conversion into a different time unit than that used during measurement, is only valid for sources like lamps, which have an output the remains constant in time. ```{r, irrad-5} q_irrad(white_led.source_spct, PAR.wb, time.unit = "hour") ``` Using a shorter time unit than the original, yields an average value re-expressed on a new time unit base. ```{r, irrad-6} e_irrad(sun_daily.spct, PAR.wb, time.unit = "second") ``` Lists of wavebands are also accepted as argument for parameter `w.band` in which case a named numeric vector of summary values is returned by default. ```{r, irrad-7} e_irrad(sun.spct, UV_bands.lst) # W m-2 ``` ```{r, irrad-7a} q_irrad(sun.spct, UV_bands.lst) # mol s-1 m-2 ``` ```{r, irrad-7b} q_irrad(sun.spct, UV_bands.lst, scale.factor = 1e6) # umol s-1 m-2 ``` These functions have an additional argument `quantity`, with default `"total"`, which can take values controlling the output. The value "total" yields **irradiance** in $W\,m^{-2}$, integrated over wavelengths for each waveband, while "average" yields the mean **spectral irradiance** within each waveband in $W\,m^{-2}\,nm^{-1}$. ```{r, irrad-8a} e_irrad(sun.spct, UV_bands.lst, quantity = "total") # watt m-2 ``` ```{r, irrad-8b} e_irrad(sun.spct, UV_bands.lst, quantity = "average") # watt m-2 nm-1 ``` When `quantity = "contribution"` irradiances for individual wavebands are expressed relative to the irradiance computed for the whole spectrum, while for `quantity = "relative"` they are expressed relative to the sum of the irradiances for all the wavebands. In both cases values are expressed as fractions of one. ```{r, irrad-8c} e_irrad(sun.spct, UV_bands.lst, quantity = "contribution") ``` ```{r, irrad-8cc} e_irrad(sun.spct, UV_bands.lst, quantity = "relative") ``` When setting `"contribution.pc"` or `"relative.pc"` as `quantity` the same values as in the examples above, are expressed as percentages instead of fractions. ```{r, irrad-8d} e_irrad(sun.spct, UV_bands.lst, quantity = "contribution.pc") ``` ```{r, irrad-8dd} e_irrad(sun.spct, UV_bands.lst, quantity = "relative.pc") ``` The total radiation received on a surface during an exposure event can be also calculated with methods `irrad()`, `e_irrad()` and `q_irrad()`, but the values returned are irradiances expressed on a very unusual time basis. These are no longer W m-2 (J s-1 m-2), but instead J per 8 hours per square meter. ```{r} e_irrad(sun.spct, PAR.wb, time.unit = duration(8, "hours")) ``` When the intention is to calculate a total fluence or exposure for an event, function `fluence()`, `e_fluence()` or `q_fluence()` should be used as this will add the correct metadata attributes to the returned value: expressed as energy or photons per unit area per event. See later sections for details. ```{r} e_fluence(sun.spct, PAR.wb, exposure.time = duration(8, "hours")) ``` In all earlier examples in this section, the default naming of returned values was active. The default naming of the values is an abbreviation of the physical quantity plus the name of the wavelength. Alternatively `"short"` naming uses only the name of the wavebands as shown below, obtained from the label stored in the waveband definitions. ```{r} q_irrad(sun.spct, UV_bands.lst, naming = "short") ``` Naming can also be completely suppressed. ```{r} q_irrad(sun.spct, UV_bands.lst, naming = "none") ``` Names of members of a list of wavebands, override the labels in waveband definitions. ```{r} names(UV_bands.lst) <- c("UV-C", "UV-B", "UV-A") q_irrad(sun.spct, UV_bands.lst, naming = "short", scale.factor = 1e6) ``` #### Collections of spectra Collections of spectra can be useful not only for time-series of spectra or spectral images, but also when dealing with a small group of related spectra. In the example below we show how to use a collection of spectra to estimate irradiances under different filters set up in sunlight. Methods `irrad()`, `e_irrad()` and `q_irrad()` can be used for collections of spectra exactly as for a single spectrum (as described above). We here only show the additional features that apply to collections. In this first example, we can see that the returned object is a data frame instead of a named numeric vector. ```{r} e_irrad(sun_evening.mspct, w.band = PAR.wb) ``` Spectral objects can contain metadata as attributes. When summarizing a collection of spectra it is frequently very useful to copy some of these metadata, extracted from each member of the collection, to columns in the returned data frame. This can be done as follows. ```{r} q_irrad(sun_evening.mspct, w.band = PAR.wb, scale.factor = 1e6, # umol m-2 s-1 attr2tb = c(when.measured = "time", lon = "lon", lat = "lat")) ``` For a more advanced example, we reuse the collection of filter spectra `filters.mspct` from an earlier section. We convolve each filter's spectral transmittance by the spectral irradiance of the light source so as to predict the irradiances under the filters. We specify the name of the column where the names of the spectra (their index) is stored. ```{r, col-convolve-1} filtered_sun <- convolve_each(filters.mspct, sun.spct) q_irrad(filtered_sun, list(UVA.wb, PAR.wb), scale.factor = 1e6, idx = "Filter") ``` The code above example can also be written as a single statement. Here we also tweak column names, delete one column, and swap the position of the remaining columns. ```{r, col-convolve-2} q_irrad(convolve_each(filters.mspct, sun.spct), list("UV-A" = UVA.wb, PAR.wb), scale.factor = 1e6, # umol m-2 s-1 naming = "short", attr2tb = c(what.measured = "Filter type"))[ , c(4, 2, 3)] ``` It is also possible to use an *apply* function. See sections *apply* functions and `convolve` for more details, as in certain cases only one or the other can be used. ### Fluence Energy fluence is energy irradiance integrated over a lapse of time or event, and is expressed as energy per area. Energy fluence is the integral over wavelengths and time of spectral energy irradiance $$F_E = \int_{\lambda = \lambda_1}^{\lambda = \lambda_2} \int_{t = t_1}^{t = t_2} E(\lambda)\ \mathrm{d}\lambda\,\mathrm{d}t$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines a waveband, or range of wavelengths, and $t_1 \geq t > t_2$. However, if we assume that irradiance does not vary in time it simplifies to $$F_E = \Delta t\times\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda)\ \mathrm{d}\lambda$$ where $\Delta t = t_2 - t_1$. The equivalent equation for photon fluence is $$F_Q = \Delta t\times\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} Q(\lambda)\ \mathrm{d}\lambda$$ Spectrally weighted (or effective) energy fluence is the integral over wavelengths and time of the result of the convolution of spectral energy irradiance by a spectral weight function $w(\lambda)$. Assuming that spectral irradiance does not vary with time the corresponding equation is $$F_{E_{BE}} = \Delta t\times\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda) \times w_E(\lambda)\ \mathrm{d}\lambda$$ Spectrally weighted (or effective) photon fluence can be computed similarly with the corresponding weighting function $w_Q(\lambda)$. #### Individual spectra The calculation of fluence values (time-integrated irradiance) is identical to that for irradiance, except that a `exposure.time` argument needs to be supplied. The exposure time must be a `lubridate::duration`, but any argument accepted by `as.duration` can also be used. Functions `fluence`, `e_fluence` and `q_fluence` correspond to `irrad`, `e_irrad` and `q_irrad`, ```{r, fluence-1} fluence(sun.spct, exposure.time = duration(1, "hours")) ``` or ```{r, fluence-1a} fluence(sun.spct, exposure.time = 3600) # seconds ``` and, to obtain the photon fluence for a range of wavelengths, in the example, photosynthetically active radiation, we use the `PAR.wb` waveband object earlier defined, and integrate for 25 minutes of exposure. ```{r, fluence-2} q_fluence(sun.spct, PAR.wb, exposure.time = duration(25, "minutes")) ``` ### Photon and energy ratios A way to summarize the "colour" of light is to compute a ratio between irradiances computed for two different wavebands of the same spectrum. They can be computed either on an energy or photon (quantum) basis, and although not a requirement, use of two wavelength bands with the same expanse is usually most informative. Photon ratios are usually preferred compared to energy ratios in work related to photobiology and photochemistry. $$Q_1 : Q_2 = \frac{\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} Q(\lambda)\ \mathrm{d}\lambda} {\int_{\lambda = \lambda_3}^{\lambda = \lambda_4} Q(\lambda)\ \mathrm{d}\lambda}$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines the waveband for the numerator and $\lambda_3 \geq \lambda > \lambda_4$ defines the waveband for the denominator, i.e., two ranges of wavelengths. Energy ratios are computed similarly $$E_1 : E_2 = \frac{\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda)\ \mathrm{d}\lambda} {\int_{\lambda = \lambda_3}^{\lambda = \lambda_4} E(\lambda)\ \mathrm{d}\lambda}$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines the waveband for the numerator and $\lambda_3 \geq \lambda > \lambda_4$ defines the waveband for the denominator, i.e., two ranges of wavelengths. An energy irradiance to photon irradiance ratio describes the average energy per mol of photon for a waveband or range of wavelengths. $$E : Q = \frac{\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} E(\lambda)\ \mathrm{d}\lambda} {\int_{\lambda = \lambda_1}^{\lambda = \lambda_2} Q(\lambda)\ \mathrm{d}\lambda}$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines a waveband, or range of wavelengths. We can compute the photon to energy ratio ($Q : E$) similarly. Although all these ratios are usually computed without use of spectral weighting functions, their computation can be easily done by including in the equations above a weighting functions ($w(\lambda)$) as described in the section on irradiance. #### Individual spectra The functions described here, in there simplest use, calculate a ratio between two wavebands. The function `q_ratio` returning photon ratios. However both waveband parameters can take lists of wavebands as arguments, with normal recycling rules in effect. The corresponding function `e_ratio` returns energy ratios. A single ratio. ```{r} q_ratio(sun.spct, UVB.wb, PAR.wb) ``` If no waveband is passed as numerator, the whole spectrum is used. The waveband fully outside the wavelength range of the spectrum is dropped silently. ```{r} q_ratio(sun.spct, list(UVC.wb, UVB.wb, UVA.wb)) ``` Three denominators (one skipped) and a single denominator. ```{r} q_ratio(sun.spct, list(UVC.wb, UVB.wb, UVA.wb), PAR.wb) ``` Function `qe_ratio`, has only one waveband parameter, and returns the *photon* to *energy* ratio, while its complement `eq_ratio` returns the *energy* to *photon* ratio. Here we show how parameters `scale.factor` and `name.tag` make scaling the returned value easy. ```{r, ratios-2} qe_ratio(sun.spct, list("UV-B" = UVB.wb, PAR.wb), scale.factor = 1e6, name.tag = " (umol/J)") ``` #### Collections of spectra As other summary methods, `q_ratio()`, `e_ratio()`, `qe_ratio()` and `eq_ratio()` when applied to a collection of spectra, they return a data frame. ```{r, ratios-3} q_ratio(filtered_sun, list(UVB.wb, UVA.wb), PAR.wb) ``` Additional parameters allow the scaling and customized column names possible. ```{r, ratios-3a} q_ratio(filtered_sun, list(UVB.wb, UVA.wb), PAR.wb, scale.factor = 100, name.tag = " (% photons)", idx = "Filter") ``` ### Transmittance, reflectance, absorptance and absorbance Given the laws of conservation of energy, transmittance, reflectance and absorptance, expressed as fractions, always add up to one. $$1 = R_\mathrm{fr,total} + A_\mathrm{fr,total} + T_\mathrm{fr,total}$$ Transmittance can be defined either as the fraction of the radiation "entering" an object that is transmitted, called internal transmittance ($T_\mathrm{fr,internal}$) or as the fraction of the ration incident on an object that is transmitted ($T_\mathrm{fr,total}$). $$1 \times R_\mathrm{fr,total} = A_\mathrm{fr,internal} + T_\mathrm{fr,internal}$$ giving $$T_\mathrm{fr,internal} = T_\mathrm{fr,total} / R_\mathrm{fr,total}$$ and $$A_\mathrm{fr,internal} = T_\mathrm{fr,total} / R_\mathrm{fr,total}$$ In the case of reflectance, "total" has a different meaning, and refers to whether all reflected light or that an specific angle in considered. In practice, both definitions of transmittance and absorptance are in use, while as it is easier to measure total- than internal transmittance, internal transmittance allows the computation of transmittance for different thickness of a material. Consequently, absorptance can be computed as: $$A_\mathrm{fr,internal} = 1 - T_\mathrm{fr,internal}$$ or as $$A_\mathrm{fr,total} = 1 - T_\mathrm{fr,total} - R_\mathrm{fr,total}$$ Absorbance, is a log transformed quantity that tends to be used when the range of values spans several orders of magnitude. It is frequently based on internal absorptance, but can be also based on total absorptance. $$A_\mathrm{internal} = -\log_{10}(1 -A_\mathrm{fr,internal}) = -\log_{10}(T_\mathrm{fr,internal}) $$ and $$A_\mathrm{total} = -\log_{10}(1 - A_\mathrm{fr,total}) = -\log_{10}(T_\mathrm{fr,total} + R_\mathrm{fr,total})$$ When using a spectrophotometer, if our reference is a cuvette with pure solvent and the sample a solution of a compound in the same solvent we measure *internal* transmittance, absorptance or absorbance. If we instead measure the transmittance of a piece of glass or plastic with air as reference, we measure *total* transmittance, absorptance or absorbance. This is so because in the first case the reference is the light transmitted after the reflectance on the cuvette walls has taken place while in the second case the reflections at the glass-air interfaces are missing from the reference. As a result of these different definitions, the implementation of the class `filter_spct()` and `object_spct` and their methods, need to keep track of whether quantities are expressed as internal or total. As in many glasses reflectance varies very weakly with wavelength, storing a constant value for $R_\mathrm{fr,total}$ is also supported. The refractive index of a material can also be used to estimate $R_\mathrm{fr,total}$ for a polished surface. Lack of a value for $R_\mathrm{fr,total}$ can make some conversions and computations impossible. #### Individual spectra The functions `transmittance`, `absorptance` and `absorbance` take `filter_spct` as argument, while function `reflectance` takes `reflector_spct` objects as argument. Functions `transmittance`, `reflectance` and `absorptance` are also implemented for `object_spct`. These functions return as default an average value for these quantities **assuming** a light source with a flat spectral energy output, but this can be changed as described above for `irrad()`. ```{r} transmittance(polyester.spct, list(UVB.wb, UVA.wb, PAR.wb)) ``` We can obtain numerical values without names if needed, ```{r} transmittance(polyester.spct, list(UVB.wb, UVA.wb, PAR.wb), naming = "none") ``` or named only according to the wavebands. ```{r} transmittance(polyester.spct, list(UVB.wb, UVA.wb, PAR.wb), naming = "short") ``` The reflectance of a leaf. ```{r} reflectance(green_leaf.spct, waveband(c(600, 700))) ``` #### Collections of spectra Here we calculate the transmittance of a collection of spectra for three filters in two wavebands, obtaining the results as a data frame, with one row per filter, and one column per waveband. We reuse once more `filters.mspct` from an earlier section. Column names formed from quantity abbreviation and label in waveband definitions. ```{r} transmittance(filters.mspct, w.band = list(UVA.wb, PAR.wb)) ``` With `naming = "short"` column names are the labels of waveband definitions. ```{r} transmittance(filters.mspct, w.band = list(UVA.wb, PAR.wb), naming = "short") ``` If list members are named, column names are formed from quantity abbreviation and name of list members, overriding the labels in the waveband definitions. ```{r} transmittance(filters.mspct, w.band = list("UV-A" = UVA.wb, PAR = PAR.wb)) ``` If list members are named, with `naming = "short"` column names are the names of list members. ```{r} transmittance(filters.mspct, w.band = list(UVA = UVA.wb, PAR = PAR.wb), naming = "short") ``` We can add metadata attributes as columns, changing the name if desired. ```{r} transmittance(filters.mspct, w.band = UVA.wb, naming = "short", attr2tb = c("what.measured" = "Filter type")) ``` We can add metadata attributes as a column, and select and rearrange the columns. ```{r} transmittance(filters.mspct, w.band = UVA.wb, attr2tb = "what.measured")[ , c(3, 2)] ``` ### Normalized difference indexes Like the ratios discussed above normalized differences are a way of summarizing or describing the colour of light sources or objects independently of the overall irradiance. As their name tells, they are defined as differences relative to a sum. Normalized difference indexes are used to summarize the spectral characteristics of light or objects. They are most frequently used in the analysis of spectral reflectance data in remote sensing. $$NDI_{a, b} = \frac{R_a - R_b}{R_a + R_b}$$ where $R_a$ and $R_b$ are reflectances computed from the same spectrum using two different wavebands. The most frequently used NDI is the NDVI or normalized difference vegetation index, which compares reflectance in the red and far-red regions as a way of assessing the presence of vegetation vs. bare land in remote sensing imagery. However, they can be computed based on energy or photon irradiances for two wavebands as shown here, or for any other spectral quantity. $$NDI_{a, b} = \frac{Q_a - Q_b}{Q_a + Q_b}$$ where $Q_a$ and $Q_b$ are irradiances computed from the same spectrum using two different wavebands. Although we here describe how to compute normalized difference indexes from spectral data, they are most frequently generated from data from broad-band sensors in satellite, air-borne or terrestrial imagers. To reconstruct from spectral data indexes computed from these instruments we need to use waveband definitions that mimic the spectral response of the imager of interest. Package 'photobiologyWavebands' provides pre-defined wavebands for several different satellite imagers as well as for wavelength ranges in common use. #### Individual spectra Here we give an example of a possible definition of NDVI using function `normalized_diff_ind()` can be used to calculate, or any index with a similar formulation structure. ```{r} normalized_diff_ind(Ler_leaf_rflt.spct, waveband(c(740, 840)), waveband(c(590, 690)), reflectance) ``` Here we give an *unusual* example to demonstrate that function `normalized_diff_ind()` can be used to calculate any index with a similar formulation structure and using other spectral quantities and summary functions. ```{r} normalized_diff_ind(sun.spct, waveband(c(600, 700)), waveband(c(400, 500)), q_irrad) ``` ### Integrated response As with other spectral quantities, we can integrate over wavelengths spectral responsiveness. In response and action spectra the quantity is a response measured in some arbitrary unit expressed per unit wavelengths and per unit energy or quantum. $$X_E = \int_{\lambda = \lambda_1}^{\lambda = \lambda_2} X_E(\lambda)\ \mathrm{d}\lambda$$ where $\lambda_1 \geq \lambda > \lambda_2$ defines a waveband, or range of wavelengths. The conversion between energy and photon (= quantum) basis of expression is possible as it is for radiation spectra. Depending on the base of expression the returned integrated value is computed **assuming** either a light source with a flat spectral energy irradiance of 1 or spectral photon irradiance of 1, respectively. #### Individual spectra The functions `response`, `e_response` and `q_response` take `response_spct` objects as arguments. If no waveband is supplied as argument, the whole spectrum is integrated. ```{r} response(photodiode.spct) ``` When a waveband, or list of wavebands, is supplied the response is calculated for the wavebands. ```{r} e_response(photodiode.spct, list(UVB.wb, UVA.wb)) ``` This function has an additional argument `quantity`, with default `"total"`, as described for `irrad()`. #### Collections of spectra ```{r} sensors <- response_mspct(list(GaAsP = photodiode.spct, CCD = ccd.spct)) response(sensors, list(UVB.wb, UVA.wb, PAR.wb), quantity = "contribution") ``` ### Integration over wavelengths When we need to integrate some \emph{non-standard} `numeric` variable stored in a spectral object we can use functions `integrate_spct` or `average_spct`. #### Calculation from individual spectra We can integrate the values of arbitrary `numeric` columns other than `w.length` in an spectral object. All spectral classes are derived from `generic_spct`, so the examples in this section apply to objects of any of the derived spectral classes as well. ```{r} integrate_spct(sun.spct) ``` The function `average_spct` integrates every column holding numeric values from a spectrum object, except for `w.length`, and divides the result by the *spread* or width of the wavelength range integrated, returning a value expressed in the same units as the spectral data. ```{r} average_spct(sun.spct) ``` ## Comparison of spectra Comparison of spectra for which data is expressed at different discrete wavelength values can be easily affected by bias if interpolation is used. Function `compare_spct()` does a coarse grained comparison by first summarizing each spectrum over consecutive ranges of wavelengths, and then applying a comparison function to these summary values. The function can be used to compare pairs of spectra, stored as a collection. In the next example we compare two `source_spct` objects using defaults for most arguments. ```{r} compare_spct(source_mspct(list(sun1 = sun.spct, sun2 = sun.spct * 2))) ``` The value returned by default is a `generic_spct` object containing the computed summaries for each waveband plus the result of the comparison between the summaries. The first three columns contain the wavelength at the midpoint of the wavelength range of waveband plus its extremes. In this example we compare two `filter_spct` objects, using summaries over 50-nm-wide bands, and using an operator returning a logical value for the comparison instead of the default division operator. ```{r} compare_spct(filter_mspct(list(pet = polyester.spct, yllw = yellow_gel.spct)), w.band = 50, .comparison.fun = `<`) ``` ## Illumination and light as seen by humans ### Illuminance Illuminance, or luminous flux incident on a surface, is relevant to human vision. It is an approximate quantity as the spectral response function of human vision varies among individuals, with their age and the conditions during measurement. The SI unit for illuminance is lux (lx) or lumnes per square meter. When function `illuminance()` is applied to spectral (energy) irradiance or spectral photon irradiance, illuminance can be directly computed using the same time base of 1 s. ```{r} illuminance(sun.spct) ``` The response of the human eye depends on the size of the target, and response spectra have been defined by CIE for both 2 degrees and 10 degrees targets. ```{r} illuminance(sun.spct, std = "CIE10deg") ``` If one attempts to compute illuminance from spectral fluence or radiation spectra expressed as a flux of energy or photons over a period of time different to 1 s, the average illuminance is returned, in lux. ```{r} illuminance(sun_daily.spct) ``` ### CCT and CRI Colour temperature and colour rendition index can be computed from spectral irradiance. Package 'colorSpec' implement these and other calcualtions related to colour. Package 'photobiologyInOut' provides wrappers to facilitate the computation of CCT and CRI directly from `source_spct` objects by on-the-fly conversion and calling functions from 'colorSpec'. ### RGB colours Different `color_of()` methods allow calculation of RGB colour values for light sources or objects. The returned values are R colour definitions. Method `color_of()` works rather differently depending on the object. The method for `numeric` vectors assumes the numbers are wavelengths in nanometres, and returns a vector of colour definitions of the same length assuming monochromatic light. ```{r} color_of(550) # green color_of(630) # red color_of(c(550, 630, 380, 750)) # vectorized ``` The method for `source_spct` objects returns a single colour definition corresponding to the whole spectrum of a light sources. ```{r} color_of(sun.spct) color_of(sun.spct * yellow_gel.spct) ``` There are no methods for `filter_spct` and `reflector_spct` objects so as shown above we need to convolve them with spectral irradiance from a `source_spct` object. The method for `waveband` objects returns one colour definition per waveband, corresponding to their central wavelength. ```{r} color_of(waveband(c(400, 500), wb.name = "my_BL")) ``` By default CIE coordinates for *typical* human vision are used, but the functions have a parameter that can be used for supplying a different chromaticity definition as a `chroma_spct` object. ```{r} color_of(sun.spct, chroma.type = "CC") color_of(sun.spct, chroma.type = "CMF") color_of(sun.spct, chroma.type = beesxyzCMF.spct) ``` In the case of bees, the RGB values represent a shift towards longer wavelengths compared to the true sensitivity. In other words, they are translated into colours that humans can see and monitors and printers can generate.