Assessing Rapid Variability in Atmospheric Apparent Optical Depth with an Array Spectrometer System

Abstract

A method for determining rapid variations in atmospheric optical depth is proposed. The method is based upon computation of the ratio between close-time spectral measurements of solar direct flux. Use of the ratio avoids the need for absolute calibration of the instruments and minimizes the effects of changes in instrumental conditions (such as temperature or mechanical adjustments) and in air mass. The technique has been applied to some campaigns of measurement for sky conditions ranging from clear skies to scattered-to-broken cloudiness, performed at high frequency (~1Hz) with a system of three array spectrometers, capable of performing very rapid spectral acquisitions, in the 400 to 1700 nm band, thus covering the visible and extending to the near-infrared spectral ranges. Results demonstrate the capacity of this instrumentation and method to detect rapid variation of optical depth, as well as rapid changes in its spectral pattern. The optical depth variability depends on the particular state of the sky and is connected to particle condensation and evaporation processes and to the changes in water vapor content in the transition region between cloud-free and cloudy regions. Thus, the method is suitable for analyzing rapid processes involving particles, either aerosol or cloud droplets, and water vapor, in the cloud boundaries.

1. Introduction

Techniques based on the measurement of electromagnetic radiation in different spectral bands constitute the main way of monitoring atmosphere conditions and components not only from satellites, but also from the ground. Particularly shortwave radiation coming from the solar flux in the spectral range from the ultraviolet to the near-infrared is affected by the atmospheric gases and particles suspended in the air, in the form of aerosol or clouds, and can thus be used to derive atmospheric optical and physical properties of these atmospheric components.

Regarding cloud detection and characterization, extensive literature is devoted to instrumental methods (see the review in [1], the work in [2], and references therein) and, recently, to the discussion on what may be considered a cloud and on the effects of clouds in their vicinity [3,4,5,6,7,8,9,10,11]. These issues may have important consequences when dealing with climate effects, and also in satellite and ground-based cloud characterization [5].

Array-based spectrometers have, over scanning instruments, the major advantage (besides reduced costs) of relatively rapid acquisition of radiation spectra, as the whole spectrum is acquired in one single step. Thus, these instruments show great potential to perform measurements in rapidly changing conditions, for instance, when a scattered-to-broken field of clouds occurs in the atmosphere or in the aerosol-droplet continuum in cloud boundaries. Consequently, swift radiative changes that cannot be tracked by scanning spectroradiometers are recently being studied with the use of array spectrometers (see, for example, [10]). The portability, mechanical stability, and reliability (the result of having no internal moving parts) constitute other significant advantages of array spectrometers over scanning instruments [12]. Array spectrometers suffer, however, from temperature dependence, dark current signal, and, especially in the UV-B region, from stray light effects [13].

Changes in suspended particle amounts and physical characteristics modulate the radiation incident at surface in time scales that can be much shorter than the typical time response of most radiative sensors. Clouds usually evolve on time scales ranging from hours to seconds, but cloud edges can produce radical changes in the direct radiation component at even shorter time intervals. Rapid changes in cloud edges simultaneously affect aerosol hydration, size and concentration of droplets and concentration of water vapor. Study of phenomena involving aerosol and cloud droplets therefore requires monitoring of radiative fluxes at high resolution times. Spectrally resolved measurements, extended to bands affected by the most significant atmospheric components, such as aerosol, clouds, and water vapor [14], further benefit the study of these phenomena. In this work, we have used a system of three array spectrometers covering the visible and part of the near-infrared spectral range to follow rapid variations in atmospheric optical depth. A simple method to derive the optical depth variability from spectra acquired at high temporal resolution is presented and discussed. Special attention has been devoted to assessing the uncertainties in the measurements and to describing some limitations of the method proposed.

2. Array Spectrometer System

The system of three fiber optic spectrometers (hereafter SPC1, SPC2, and SPC3) from Avantes (Apeldoorn, The Netherlands) has been used to perform field measurements of direct solar radiation. The measurement periods analyzed correspond to several campaigns carried out at the meteorological and radiometric station (41.962° N, 2.833° E, 110 m asl), belonging to the Environmental Physics Group (GFA) of the University of Girona (Universitat de Girona, UdG). Non-calibrated spectra were acquired at a rate of about 1 Hz during some periods showing sky conditions that ranged from very clear to scattered and broken cloudiness, including transitions between cloudy and cloud-free air in the system’s field of view.

Some specifications of the spectrometers are listed in

Table 1. Specifications of the spectrometers and characteristics of the foreoptics.

Spectrometer	SPC1	SPC2	SPC3
Avaspec Model	2048-SPU2	ULS2048L-USB2	NIR256-1.7TEC
Detector	CCD	CCD	InGaAs
Cooling	No	No	TEC
Order Sorting Filters 1	350 and 590 nm	600 nm	1000 nm
Optical bench	Czerney-Turner	Czerney-Turner	Czerney-Turner
Focal length	75 mm	75 mm	50 mm
Grating (lines/mm)	600	600	200
Available band (nm)	230–780	590–1100	1034–1700
Slit width (μm)	50	50	50
Fiber diameter (μm)	50	200	200
Opening angle (°)	1.4	1.4	1.4
Limit angle (°)	2	2	2

¹ Long-pass filters.

. Each detector is coupled to a 16-bit AD-converter, and dedicated software allows instruments’ control and spectra acquisition by a personal computer, through a set of USB connectors. The foreoptics comprises a light collimating system and a set of optical fibers to guide light into the spectrometers. Three optically separated compartments in a box mounted on an equatorial sun tracker, with several baffles to reduce internal reflection effects, act as collimator tubes. The opening angle (i.e., half the field of view) and the limit angle are 1.4° and 2.0°, respectively, for the three collimators. Each collimated beam reaches a diffuser (3.9 mm diameter), and light is transmitted to a fiber and finally to the corresponding spectrometer entrance.

Figure 1 shows three spectra taken when pointing the spectrometer system to the sun in a summer day (28 June 2017) showing clear conditions. The stored spectra are averages of 50 spectra (to further enhance the signal-to-noise ratio) acquired with an integration time of 20 ms, thus extending the actual acquisition period to about 1 s. In fact, all measurements analyzed in this work were performed with an integration time fixed at 20 ms, which guarantees a high signal but avoids saturation when monitoring solar direct radiation under low extinction conditions, even in the channels where the signal count reaches the maximum (about 475 nm, 675 and 1550 nm for SPC1, SPC2 and SPC3, respectively). Moreover, a fixed integration time circumvents the linearity problems that can arise if varying integration times were used. Furthermore, all stored spectra were averages of 50 consecutive acquisitions. Maintaining these adjustments along all measurement periods helps to minimize the effect of changes in instrument behavior and has reduced the characterization efforts and tests performed (both in field and in laboratory) to determine the measurement uncertainties (see below).

To reduce stray light effects, the spectrometers rely on a Symmetrical Czerny–Turner design and on some order-sorting coatings on the sensor (Table 1). Furthermore, for SPC1, which suffers the more noticeable stray light effects (see Figure 1b) we checked the stray light levels at some wavelengths by comparing solar direct radiation passing through several selected filters in the foreoptics with the corresponding unfiltered radiation (for sufficiently stable sky conditions). When we used a long-pass filter with cut-on wavelength at 695 nm (colored glass filter 51340 from former Oriel Corporation manufacturer), the level at 350 nm was around 0.2% of the level for unfiltered radiation at the same wavelength, whereas at 500 nm, filtered radiation was reduced to less than 0.02%. For a long-pass filter with cut-on wavelength at 455 nm (filter 51284 from Oriel Corporation), the level of filtered radiation at 350 nm was lower than 1.5% of the unfiltered radiation, whereas it was approximately 0.3% at 400 nm. In the present note, analysis of field measurements is limited to wavelengths above 400 nm, where stray light has very low effects.

Dark current, i.e., electronic flux that generates a signal even when the sensor is not illuminated, is another issue to be addressed when measuring with array spectrometers [12]. Indeed, dark current is known to present a very strong dependence on sensor temperature. To reduce dark current contributions, the sensor of the AvaSpec 2048 SPU2 possesses some blind pixels that are used to dynamically correct the acquired spectra. Moreover, dark current was determined at the start of each measurement period, and continuously subtracted from the signal (in the so-called differential mode of acquisition). In addition, dark spectra were obtained in each experiment at intervals of few minutes. The changes in dark current with respect to the initial conditions, which can mainly be attributed to changes in temperature, were always at the level of the noise in the dark measurements. The dark signal values found in several laboratory tests, at a somewhat lower ambient temperature, were very similar to the levels in field measurements.

Besides the stray light and the dark current issues, optical stability in portable instruments can be affected by a variety of factors, for example, when the foreoptics, such as a light transmission fiber, is removed from or connected to the spectrometer. Other handicaps are the ageing of the fibers affecting their transmittance, the higher detection threshold of diode-based sensors compared with photomultipliers typically used in scanning spectroradiometers, and the temperature dependence of wavelength accuracy, dark current and responsivity. Uncertainties in absolute calibration are like those for scanning instruments [15,16], as the same methods can be applied, but the factors above specified add important difficulties in the characterization.

In order to check and characterize sky conditions during campaigns several instruments routinely measure meteorological and radiative variables and perform some observations at the GFA station. Global, diffuse and direct solar broadband radiation are measured with pyranometers (CM11, from Kipp and Zonen, Delft, The Netherlands) and a pyrheliometer (CH-1, also from Kipp and Zonen), respectively; backscattering profiles are obtained with a Vaisala CL31 ceilometer, and sky images are taken with two hemispherical cameras (Whole Sky Camera, WSC, an own manufactured system, and a SONA system, from Sieltec Canarias, SL), among other variables [17]. Ceilometer observations and whole sky images were used to check sky conditions during the campaign periods of measurement with the array spectrometer system. Optical depth at some defined channels is available at 15 sec resolution from a Multifilter Rotating Shadowband Radiometer (MFR7 model from Yankee Environmental Systems), following the treatments in [18,19].

3. Variations in Apparent Optical Depth

To overcome some of the above described weaknesses of the array spectrometers, concretely those related with instrument changes and absolute calibration, the ratio between consecutive close-time raw measurements (i.e., counts at each wavelength) was used. Working with the ratio of the raw measurements instead of calibrated retrievals avoids absolute calibration of the instrument and its contribution to overall uncertainty. Moreover, the effects of any long-term change in the instrument’s optical behavior are compensated in the ratio between consecutive signals. In addition, using relative quantities is perfectly suitable when the interest is centered on temporal variations in the magnitudes.

We intend to apply the method to incoming radiative flux from selected narrow angular sky regions where changing conditions, as cloud transitions, are occurring. Observations in the zenith direction are particularly interesting and have been chosen by other authors (e.g., [9,20]). Another possibility, adopted herein, involves taking flux measurements centered in the direction of the sun disc where, except when thick clouds intercept the solar ray, shortwave radiance is highly concentrated. Radiance in the sun direction can be several orders of magnitude greater than in the zenith direction, thus benefiting the signal-to-noise ratio in the measurements and providing the possibility to use shorter integration times. This is particularly interesting if very small variations in short times are to be detected, like those occurring when very thin clouds are passing in the field of view of the instruments. Furthermore, measurement of the direct component of the solar flux enables short-time changes in atmospheric extinction to be studied in a relatively straightforward manner, by means of the apparent optical depth concept (see below).

Changes in atmospheric conditions lead to variations in direct solar radiation at the surface that can be limited to some absorption bands (e.g., when changing the concentration of some gases) or extended to the broadband (e.g., when changes correspond to cloud thickness or to aerosol load). For a radiometer having a narrow acceptance angle and pointing towards the sun, the resulting signal S_λ(t) for a specific wavelength λ for a given instant t can be expressed with the use of the Beer–Lambert–Bouguer for the extinction of monochromatic radiation:

$$S_{\lambda }\left(t\right)=R_{\lambda }\left(t\right)E_{\lambda }^{ext}\left(t\right)e^{-O{D}_{\lambda }\left(t\right)AM\left(t\right)}$$

(1)

where R_λ(t) is the spectral responsivity of the measurement system, $E_{\lambda }^{ext}\left(t\right)$ is direct normal spectral irradiance at the top of the atmosphere, OD_λ(t) is total optical depth of the atmosphere, and AM(t) is relative air mass (which has been simplified to be the same for all atmospheric constituents). The ratio between two spectra taken at close times t₁ and t₂ can be expressed as:

$$\frac{S_{\lambda }\left(t_2\right)}{S_{\lambda }\left(t_1\right)}=e^{-\delta O{D}_{\lambda }\left(t_1,t_2\right)AM}$$

(2)

where $\delta O{D}_{\lambda }\left(t_1,t_2\right)\equiv OD\left(t_2\right)-O{D}_{\lambda }\left(t_1\right)$ is the change in total optical depth. We have assumed that the two spectra compared in Equation (2) have been acquired at very close times, that is that t₂ − t₁ is much shorter than the typical time required for $E_{\lambda }^{ext}$, AM and S_λ to change noticeably: $E_{\lambda }^{ext}\left(t_1\right)\approx E_{\lambda }^{ext}\left(t_2\right)$, $AM\left(t_1\right)\approx AM\left(t_2\right)$, $R_{\lambda }\left(t_1\right)\approx R_{\lambda }\left(t_2\right)$. Changes in system performance (for example, due to changing temperature affecting dark current and responsivity) or in air mass prevent the use of Equation (2) for comparisons between measurements separated in time.

From Equation (2) we obtain:

$$\delta O{D}_{\lambda }=-\frac{1}{AM}ln\frac{S_{\lambda }\left(t_2\right)}{S_{\lambda }\left(t_1\right)}$$

(3)

where for simplicity the explicit reference to the measurement times has been removed in δOD_λ. We can distribute the change in optical depth in the visible range in:

$$\delta O{D}_{\lambda }=\delta RO{D}_{\lambda }+\delta GO{D}_{\lambda }+\delta AO{D}_{\lambda }+\delta CO{D}_{\lambda }$$

(4)

where δROD, δGOD, δAOD and δCOD stand for the changes in optical depth of Rayleigh molecular scattering by permanent gases, absorption by atmospheric gases (including ozone and other gases), and extinction caused by aerosol and cloud, respectively.

A well-known experimental problem when measuring direct solar fluxes involves the contribution by scattered radiation (as a result of either single or multiple scattering processes) entering the field of view, which can be of several degrees for most instruments currently used due to the need for a certain tolerance in the pointing direction when following the sun trajectory. The contribution by scattered radiation has been the object of study due to its important implications in diverse fields, e.g., when comparing modeled and measured beam radiation, when deriving optical depths from beam radiation measurements, or when assessing the direct solar resource for energy profit. Since the efforts to geometrically characterize the entrance of radiometers in [21], among others, much work has been done with regard to quantify the circumsolar component either from measurements [22], or by means of modeling [23,24], to assess the errors caused in optical depth retrievals [25], and to estimate and correct optical depths for aerosol and for cirrus clouds [26,27].

Other issues when deriving optical depth involve its representativeness when conditions are not uniform in the instrument’s field of view, as well as when a cloud edge crosses the line of sight, and when measurement is performed in a non-vertical direction. These considerations led us to recover the term apparent optical depth [26] as the result of applying the Beer–Lambert–Bouguer law (and Equations (1)–(4)) to measurements taken by any instrument with a finite field of view. Further studies (beyond the scope of this note) should be devoted to addressing the effects of the circumsolar contribution and of the other issues affecting the measurements and the derived variations in optical depth.

4. Rapid Changes in Spectral Structure of Apparent Optical Depth

In this section we will show some features of the spectra acquired by the spectrometer system, and of the derived δOD_λ. As the measurements were performed at a rate of 1 Hz, the latter quantity is expressed in optical depth units per second (from now on, OD units s⁻¹). Figure 2a shows, as an example, the evolution of the signal at 500 nm (therefore taken with SPC1) along a series of acquisitions performed during the pass of some cloud patches across the line of sight of the measurement system, the 28 June 2017, starting at about 9:35 UTC. The shown period covers about 150 s of measurements. Furthermore, the computed δOD at 500 nm is shown in Figure 2b. Along the morning of that day the sky showed scattered-to-broken cloudiness, reported by direct observation and confirmed a posteriori by both the hemispherical images acquired by the camera SONA and the observations from the CL31 ceilometer in the vertical direction. These conditions resulted in the alternance between clear sky and the pass of cloud patches through the line of sight of the spectrometer system. For the period shown in Figure 2, a particular cloud patch crossed the field of view of the spectrometers (for about 1 min) at an altitude of 2000 m. The ceilometer also detected for a few moments a second cloud layer (in the vertical direction) at about 400 m above the first. Figure 3 shows two images, acquired by the SONA camera, respectively, before and during the pass of the cloud patch (at 9:36 and 9:37 UTC).

Just before and after the pass of the cloud patch the sky conditions in the line of sight were clear, which is evident in Figure 2a as periods of relatively stable signal, with very low variations in apparent optical depth, of the order of thousandths or even lower (Figure 2b). It can be observed that before the pass of the cloud patch, the signal at 500 nm shows a slightly negative trend. Later, during the pass of the cloud patch, the spectrometer measurement and the apparent optical depth suffered important variations.

A set of subperiods, denoted as P1 to P6 in Figure 2b and Figure 3, have been selected just to describe the dependency of the computed δOD_λ on the conditions along the sight of view, and to check some spectral features. It can be observed that, despite the measurements have been performed by different uncalibrated spectrometers in different spectral bands, the composed spectra present a good continuity in the wavelengths were the spectra are connected (700 and 1034 nm). The δOD_λ obtained from the SPC1 measurements show a higher noise than those from SPC2 and SPC3, which could be reduced with a further spectral smoothing. However, we have decided not to perform such smoothing here, as the shown noise does not prevent observing the most interesting features in the visible band.

Along the period P1 (Figure 2b and Figure 3) the air conditions are very stable and the variations in the apparent optical depth δOD_λ are very low, of the order of the thousandths (10⁻³), as it has been commented above. Variations in the apparent optical depth, and their time variability, increase for low wavelengths, and on the other hand they are observable for the water vapor absorption bands around 940 and 1100 nm, and especially remarkable for the stronger absorption band at 1380 nm. It should be noted that, for the period P1, despite the variations in the apparent optical depth are in general positive in the whole band (consistently with the fact that the SPC1 signal at 500 nm is slowly diminishing) the variations in the 1380 nm band show variable sign, implying rapid successive increases and decreases of the water vapor content along the solar ray path.

The period P2, closer to the pass of the cloud patch through the line of sight, shows similar δOD_λ oscillations in the whole band, but the variability in the 1380 nm band is higher than in the period P1. In that band the variations in the apparent optical depth reach more than 0.01 OD units s⁻¹ (either positive or negative, see spectra in blue and red, respectively, for the period P2 in Figure 3). The increase in the variability in that band could be attributed to the dominant conditions nearby the cloud, including the water vapor saturation, and the water condensation and evaporation processes that are supposed to take place in these boundary regions. Of course, this can be only stated here as a hypothesis, as the conditions along the whole line of sight (and not only in the vicinity of the described cloud patch) are unavoidably affecting the measurement.

During the period P3, a slight oscillation in the radiation flux at 500 nm occurs, presumably caused by the transition to cloud conditions. Indeed, Figure 3 (Plot P3) displays a set of consecutive spectra showing a steady increase in δOD_λ, much more remarkable for the shorter wavelengths than for the strong water vapor absorption band at 1380 nm. These large variations in time (but smooth along the spectral range) should be attributed to changes in the amount and characteristics of cloud droplets as a result of water condensation in the vicinity of the cloud. In the latter band, some unexpected effects appear, giving negative variations in the apparent optical depth, whereas they are clearly positive for the rest of the measured wavelengths. These effects could be first associated to the combination of the increase of the apparent optical depth in the whole band with a decrease in the absorption band, reinforced by the high nonlinearity of absorption by water vapor in that band. Other instrumental causes could play a role and should be analyzed: for example, each represented spectrum is in fact a combination of 50 individual spectra taken along the 1 sec measurement period, and each measured spectrum is actually affected by the atmospheric conditions along the whole solar ray path towards the spectrometer system.

Plots P4, P5 and P6 in the Figure 3 show δOD_λ spectra for three periods (see also Figure 2) during the pass of the cloud patch. The period P4 corresponds to the entering of the cloud patch into the spectrometer system field of view. It can be observed that during this period the variations in the apparent optical depth do not show a marked spectral structure (in comparison with the range of values for the whole spectrum), and are slightly more remarkable for longer wavelengths, in the near-infrared, than for the visible spectral range. This feature can also be seen in periods P5 and P6, which correspond to the pass of the opaquer part of the cloud patch. Variations in the water vapor absorption band at 1380 nm are still visible in combination with the variations caused by the changes in the amount and characteristics of water droplets.

5. Uncertainty and Detection Limits

Very stable sky conditions lead to very small variability in the direct solar flux entering the spectrometer system, corresponding to very small changes in the apparent optical depth. Under such conditions, changes in the signal can be hidden by noise in the measurement. One may wonder which is the minimum change in apparent optical depth that would be detectable and should be assigned to true changes in the physical conditions in the measurement system line of sight. On the other hand, low signal conditions may lead to high uncertainties when the apparent optical depth is derived.

In this section, we assess the uncertainty of the measurements and evaluate the effects on the detection limits for the variations in apparent optical depth under different sky conditions. With this goal, we present some results of the application of the method described in Section 3 to spectra acquired in a campaign performed during 10 days along the spring of 2015. This campaign was carried out using only the SPC1 instrument (SPC2 and SPC3 were added to complete the system in 2017). Only measurements at a discrete set of wavelengths (every 50 nm from 350 to 650 nm) were stored for various episodes (500−6000 s long) showing different sky conditions: cloudless, high-level cloudiness (cirrus) and scattered-to-broken sky cover and different aerosol loads. Figure 4 shows some images taken with the WSC camera to illustrate the dominant conditions along each measurement period in that campaign.

5.1. Measurement Uncertainty

To assign an uncertainty to each stored measurement (in each channel; it should be noted that in the following paragraphs and equations, all quantities also depend upon wavelength, but we avoid using the subscript λ for simplicity), we performed several tests consisting of (i) series of measurements under very stable conditions, and (ii) series of dark measurements [12]. Like the actual field measurements, all test measurements were performed in differential mode, that is, the dark current S_d (which is determined at the start of each series) is automatically subtracted from the raw measurement signal St, so total noise N in the difference S_t–S_d comes from noise (herein expressed as standard deviations) in both quantities:

$$N=\sqrt{{\sigma }^2\left(S_t\right)+{\sigma }^2\left(S_d\right)}$$

(5)

which, according to [15], can be computed as:

$$N=\sqrt{\varphi Q\Delta t+D{N}^2}$$

(6)

where ϕ is the flux (photons per second) entering the channel, Q is the quantum efficiency (electrons per incident photon), Δt is the integration time and DN is the noise in the observed dark current. Herein, we substitute the flux in Equation (6) by the signal, by assuming that both quantities are proportional (and that the dark noise levels remain unchanged), to estimate the noise in a measurement for any flux level:

$$N=\sqrt{fS+D{N}^2}$$

(7)

The factor f in Equation (7) was determined from the signal and its observed noise in a set of laboratory tests. A very stable deuterium lamp (Avalight-DH-CAL from Avantes) was used to evaluate measurement noise (at radiation levels that were of the order of those reached in field measurements). As an example, at 500 nm (measured by SPC1) f and DN take the values 2.3 and 0.033, respectively. We finally identify N, calculated with Equation (7), as the standard (1-fold) uncertainty u_S of the measurement S in each channel.

A simple propagation of the uncertainties, u_S1 and u_S2, in the input quantities, S(t₁) and S(t₂) in the definition of δOD_λ (Equation (3)), considering that there is no uncertainty in the determination of the air mass, leads (for each channel) to the combined standard uncertainty in the variation in apparent optical depth:

$$u_{\delta OD}=\frac{1}{AM}\sqrt{{\left(\frac{u_{S1}}{S\left(t_1\right)}\right)}^2+{\left(\frac{u_{S2}}{S\left(t_2\right)}\right)}^2}$$

(8)

5.2. Detectable Changes in Apparent Optical Depth

Now we limit the analysis of the variability in apparent optical depth to 500 nm, a convenient wavelength for quantifying the effects of clouds and aerosols in atmospheric optics (hereinafter, the subscript ‘λ’ is removed in δOD when we refer to this wavelength). This wavelength, which is also close to the maximum in the solar spectral irradiance, is fairly representative of the whole visible range. Figure 5 shows the evolution of the signal at 500 nm (with its uncertainty, computed from Equation (7)) for a short period of measurements on June 2, 2015, showing very clear conditions. The signal is very high and the homogeneous conditions inside and near the field of view of the spectrometer system lead to very small variations in the signal. However, about half of the changes between consecutive measurements are higher than the sum of the corresponding standard uncertainties, i.e., these changes accomplish

$$ABS\left[S\left(t_2\right)-S\left(t_1\right)\right]\ge u_{S2}+u_{S1}$$

(9)

Other measurement periods with similar conditions (cloudless, blue and spatially homogeneous sky) in the line of sight were also analyzed: June 1, June 2, and cloudless periods on April 30, May 26 and June 3, 2015. For these periods, the fraction of measurements meeting the condition established by Equation (9) ranges between 40% and 55%. Hereinafter we will use this very simple condition (other conditions involving expanded and/or combined uncertainties could have been applied) as a filter to determine what changes in the signal at 500 nm (and consequently in apparent OD at 500 nm) can be considered as corresponding to real changes in atmospheric conditions.

For the period shown in Figure 5, the absolute value of the derived (significant) δOD is in the range of 0.0005−0.002 OD units s⁻¹. The other measurement periods presenting similar sky conditions show the same range for δOD. Thus, the value 0.0005 OD units s⁻¹ can be considered as the minimum change in apparent OD detectable with reliability for conditions of high signal. On the other hand, the maximum value (0.002) of δOD observed in Figure 5 (and in other clear periods) can be taken as an approximate upper boundary to Rayleigh contribution (that is the molecular contribution to variations in optical depth, that would correspond to changes in atmospheric pressure), and possibly to ozone and well-mixed aerosol contributions to short time variations in OD (see Equation (4)).

5.3. Uncertainties in the δOD Retrieval

The absolute uncertainties in the signal appearing in Equation (9) depend upon the signal levels, and thus on the sky conditions. An initial consequence is that the detection limit for δOD increases with a decrease in the signal. Additionally, the uncertainties (both absolute and relative) in δOD, estimated with Equation (8), depend on sky conditions.

The method described in Section 3 for deriving significant δOD can be applied to a broad variety of situations. Figure 6 shows the absolute and relative uncertainties in δOD, for the whole dataset of the 2015 campaign, represented against the absolute value of δOD, and distinguishing significant changes according to Equation (9). We have obtained that for the measurement periods in 2015 δOD at 500 nm is in the range 0.00001−1 OD units s⁻¹. As expected and explained in the previous section, for clear sky periods low values of δOD are systematically found. Higher variations in OD are obtained when high thin clouds are present, and the highest values for δOD are found when conditions in the line of sight are cloudy or switch between cloudy and cloud-free. The maximum retrieved values in δOD are of the order of 1 OD units s⁻¹ for these strong transition periods.

Whereas the fraction of significant variations in signal and apparent OD is approximately one half for clear sky conditions (Section 5.2), it increases considerably when clouds are present: high thin clouds produce more than 70% of significant variations, and thicker clouds lead to over 90% of significant variations (100% for some periods in the database used). Nonetheless, two conditions, associated with high and low signals, can lead to highly uncertain retrievals of δOD, thus somewhat limiting the applicability of the method. On one hand, high signals appear to be associated with clear conditions in the line of sight (Section 5.2). For these periods the absolute uncertainty in the retrieved δOD is low (Figure 6a), but this does not guarantee a low relative uncertainty (Figure 6b), because δOD is also very low. Subsequently, huge relative uncertainties in δOD (above 10 times the value of δOD) can be found for high signal conditions. On the contrary, conditions with a very low signal, as for some periods on 28 April 2015, give rise to both high absolute and relative uncertainties (Figure 6a,b).

The heterogeneities in the optical thickness of high thin clouds crossing the field of view of the spectrometer system (4, 7 and 13 May, 2015) generate short-time variability in the apparent optical depth of that region, leading to short-time changes in the signal of the CCD; δOD is in the range of 0.001−0.02 OD units s⁻¹. Relative uncertainties are generally low (in the 10−50% range) but may sporadically reach very high values (well above 100%). The lowest uncertainties in the retrieved apparent δOD (below 10%, but often as low as 1%) correspond to transitions between cloud and clear sky in the line of sight. The effect of applying the condition in Equation (9) is also appreciable in Figure 6b: δOD values are seen to be significant mainly when their relative uncertainty is lower than approximately 70%.

5.4. Distributions of Rapid Variations in Apparent Optical Depth

To give more detail on how sky conditions affect rapid variations in apparent optical depth, Figure 7 shows the frequency distribution of δOD retrieved throughout some representative campaign days, grouped according to the dominant sky conditions: clear sky, high thin clouds (cirrus) and periods with mainly scattered and broken clouds (cumulus). If the condition in Equation (9) had not been applied, many more cases would have appeared in the bin centered at 0.001 OD units s⁻¹ in Figure 7, but little differences would appear for higher δOD bins.

Clear conditions dominated in 1 and 2 June 2015. This produced relatively constant and low optical depth (about 0.2 at 500 nm), as determined from the MFR7 measurements. The homogeneous conditions across the system’s field of view revealed as low short-time variability in the optical depth. The normalized frequency histograms for these days, along with other cloudless periods (30 April and 26 May 2015) show accumulation in the extreme of lowest variability in apparent optical depth. For these conditions, δOD maintains values of several thousandths of OD units s⁻¹, which might correspond to variations in air pressure (i.e., to Rayleigh optical depth) and (potentially) in ozone (and other gases as NO2 if there had been high pollution) and aerosol load. During 1 and 2 June the sky was very clear, and very similar distributions were obtained, but for 30 April, the distribution shows a slight skewness towards higher values of δOD. For this day, a cloud was close to the field of view of the system during the measurement period (see image in Figure 4b), and a very thin (sub-visible) cloud layer might have been present in the sky. Indeed, the sequential inspection of the images corresponding to that period shows a very thin veil passing across the sun, which could have contributed to the skewness of the δOD distribution to slightly higher values than for very clear conditions.

When high thin cloudiness is present, optical depth takes higher values (as retrieved from MFR7 measurements). In addition, OD values exhibit noticeable changes (high relative dispersion in OD) along the measurement period (see days 4, 7 and 13 May 2015 in Figure 7). These are quite similar, with maxima in the range 0.002 to 0.01 OD units s⁻¹, clearly higher than for the cloudless periods. Higher rates of variation are linked to thicker clouds: indeed, high clouds observed during 7 May appear in the WSC images (Figure 4) as being clearly thicker than those observed on 4 May.

For the measurement period of 13 May, both a thick aerosol layer and high clouds were present and optical depth was relatively high (above 1 at 500 nm, as retrieved from MFR7), but showed greater stability when compared to conditions clearly dominated by high clouds, as on 4 and 7 May. Despite this long-term relative stability in particle OD, the short-time variability δOD was relatively high (Figure 7). Thus, for the measurement period on 13 May, clouds are the main contributors to δOD, regardless of the presence of a high aerosol load. This is an indication that aerosol particles were in a well-mixed (horizontally homogeneous) layer.

For the periods on 15 and 26 May and 3 June, showing sky conditions from cloudless to scattered-to-broken clouds, both OD and δOD lie in wide ranges, from low values corresponding to cloudless periods, to values as high as 2 and 1 OD units s⁻¹ for OD and δOD, respectively. Only cloudless conditions in the line of sight produce low short-term variability in OD. When clouds present a scattered distribution, showing intermittent and relatively high values of δOD (as for some cloudy periods for 26 May), the frequency distribution of δOD shows a bi-modal shape, with a maximum for low values (around 0.001 OD units s⁻¹) and a second mode showing δOD extending to values well above 0.1 OD units s⁻¹. On the contrary, the cloudy period on 3 June contains less clear sky patches and therefore misses its bi-modal feature, with a somewhat uniform distribution up to δOD of approximately 0.1 OD units s⁻¹. Of course, the relative importance of the modes depends on the exact weight of clear and cloudy conditions during the selected period. Finally, Figure 7 shows the frequency distributions for periods with broken cloudiness, that is, mostly cloudy periods with few clear-sky events in the line of sight (15 May and the first cloudy period on 26 May). Indeed, high values of δOD are evident, and the histograms present their maxima for high values of δOD (around 0.1 OD units s⁻¹).

6. Conclusions

We have presented and tested a method for retrieving rapid variations in apparent atmospheric optical depth based on the ratio between consecutive spectral radiation measurements. The use of relative quantities generally avoids the need for absolute calibration of radiometers and removes the effects of mechanical and optical variations in the instrument. In addition, comparison between close-time measurements contributes to reducing effects of shifts in responsivity and in other instrumental characteristics.

The method was applied to spectral measurements of direct solar flux in the visible to near-infrared range taken by a set of three array spectrometers. We conducted some measurement campaigns at a rate of 1 Hz for several periods covering different sky conditions, from clear sky to scattered-to-broken cloudiness, with the final aim of characterizing variations in optical depth and its spectral dependency. Measurement uncertainties were expressly addressed, as well as limitations when the atmosphere (and thus the signal levels) is highly stable and homogeneous, or when the signal is too low, so relative uncertainties increase.

Rates of variation in apparent optical depth extend over several orders of magnitude, depending on sky conditions. Stable clear conditions systematically give δOD lower than 0.002 OD units s⁻¹ at 500 nm. The presence of high clouds in the line of view of the spectrometer system produces δOD mostly in the 0.002 to 0.02 OD units s⁻¹ range. Thus, the value of 0.002 OD units s⁻¹ could be considered to constitute a threshold between clear and high (and optically thin) cloud conditions, whereas 0.02 OD units s-1 would be the boundary between high thin clouds and low thick clouds. Specifically, when scattered-to-broken conditions are present, typically with low clouds, δOD takes either low values or high values, greater than 0.02 and up to some tenths of OD units s⁻¹, corresponding, respectively, to clear patches between clouds and to clear-cloud transitions along the line of sight. Changes in apparent optical depth through a thick cloud are expected to be very high, but the signal is often too low (and the corresponding relative uncertainty too high) to obtain reliable estimations of these variations with the configuration that we have applied in the current campaigns. We have seen that different cloudiness conditions can be characterized by different normalized frequency histograms of δOD. Standardizing these distributions as a family of Probability Density Functions could be useful as a complementary information for other methods with lower time resolution, when cloudiness and aerosol conditions distinction is needed.

Further, we consider the short-time variability in optical depth to be related to the spatial heterogeneity of conditions in the sky region covered by the field of view (and its vicinities) and it should be possible to quantitatively relate both characteristics. Other research [6,8,20,28] has focused on the spatial variability of the sky conditions. However, this is beyond the scope of the present note, as we have concentrated only upon the description and interpretation of temporal variability.

In summary, temporal variability could be linked to spatial (in)homogeneity, used to detect very thin clouds, which can produce variability in apparent optical depth greater than that of clear sky, or applied to the study of rapid processes, such as cloud edges or in aerosol plumes.

It is expected that the study of the rapid variations in spectral optical depth, combined with radiative simulations, would allow to deepening the study of cloud-aerosol transition processes. Some features of the spectral dependence of δOD from the visible to the near-infrared (including the strong water vapor absorption band at around 1380 nm) have been shown. Rapid variations in the whole spectral range treated (400 to 1700 nm) must be related to the processes of cloud droplet growing or evaporation, whereas rapid variations in the water vapor content have been also observed in a variety of conditions in the field of view of the spectrometer system: for apparently stable clear conditions, within a cloud patch and in the transition region between the clear sky and the cloud. If the observed variations in the water vapor absorption band are linked to particle condensation or evaporation near the cloud patch or contrarily correspond to variability of conditions along the whole solar ray path cannot be strictly elucidated with the proposed method. However, it is reasonable to expect that the stronger detected changes in optical depth are mainly connected to where physical conditions are significantly changing. The use of techniques such as spectral data compression [29], principal component analysis [30] or optimal spectral sampling [31] could be useful in the systematic treatment of the information retrieved with the method here proposed. The exploitation of the qualitative spectral features shown above and/or the retrievals from that treatment techniques should be complemented with the knowledge of the in situ conditions in the vicinity and within the cloud. This presents many difficulties in feasibility, as measurements with high resolution in time and space require considerable efforts (and are notably costly). The radiative simulation can be convenient alternative and may assist in taking advantage of the methods based in spectral measurements. It should be studied, among other issues, how the variations in the size distribution of the cloud droplets could affect the spectral structure of the apparent optical depth in the visible and the near-infrared.