A new approach for simultaneously retrieving cloud albedo and cloud fraction from surface-based shortwave radiation measurements

Surface-based measurements of shortwave (SW) radiative fluxes contain valuable information on cloud properties, but have not been fully used to infer those properties. Here a new analytical approach is presented that simultaneously infers cloud albedo and cloud fraction from surface-based measurements of total and direct radiative fluxes. An inspection of the analytical formulation reveals that cloud fraction is primarily determined by the relative cloud radiative forcing for the direct radiation, defined as the difference between the clear-sky and all-sky direct downwelling radiative fluxes normalized by the clear-sky direct downwelling radiative fluxes, while cloud albedo is primarily determined by the ratio of the relative cloud radiative forcing for the total downwelling radiation to the relative cloud radiative forcing for the direct radiation. The new analytical approach is validated using synthetic measurements generated by the rapid radiative transfer model (RRTM) algorithm with known cloud inputs and some surface- and satellite-based measurements. The effect of cloud absorption is further corrected based on a suite of numerical experiments. The new approach demonstrates the utility of partitioning total radiation into direct and diffuse radiation, and eliminates the potential contamination of errors in existing approaches that retrieve cloud fraction and cloud albedo separately.


Introduction
For decades, intensive research has been dedicated to quantifying the impact of clouds on the Earth's radiation budget (Charlock and Ramanathan 1985, Ramanathan 1987, Raschke et al 2005, Schneide 1972). Cloud radiative forcing Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
(CRF), defined as the change in the net radiation budget due to clouds, is one of the quantities that has been commonly used to gauge the radiative impact of clouds (Cess and Potter 1987, Charlock and Ramanathan 1985, Ramanathan 1987. CRF is a simple but effective means of characterizing the effect of cloud on the radiative budget since it can be readily obtained from global climate models (GCMs) or measured by satellites. Comparison of model-simulated CRF against satellite observations at the top of the atmosphere is instrumental in diagnosing problems in GCMs and the parameterizations and identification of cloud feedbacks as the key factors contributing to the large uncertainty in GCM simulations of atmospheric processes Cess 2004, Stephens 2005). The concept of CRF was later applied to surface-based radiation measurements (Dong et al 2002, Mace andBenson 2008).
Despite its utility, CRF suffers from the drawback of being affected by factors other than clouds, including the solar zenith angle, the definition of what constitutes a clear-sky reference, and the specification of surface albedo. Efforts have been devoted to minimizing the effects of these non-cloud factors (Betts andViterbo 2005, Vavrus 2006), among which effective cloud albedo, defined by Betts and his co-workers as the CRF normalized by the corresponding clear-sky radiative flux, is probably the best. Similar quantities were also referred to as the cloudiness index (Dong et al 2006, O'Malley andDuchon 1996). Liu et al (2011) showed theoretically that the so-called cloud effective albedo is actually a function of cloud fraction and cloud albedo, and suggested using the terminology of relative cloud radiative forcing (RCRF) to avoid the potential confusion or misunderstanding that the normalized CRF is related more to either cloud albedo or cloud fraction. Although it has been long recognized that CRF or RCRF is related intimately to cloud fraction and cloud albedo, and some efforts have been devoted to exploring their relationships (Betts andViterbo 2005, Charlock andRamanathan 1985), our understanding had been largely qualitative until recently, when Liu et al (2011), hereafter Liu2011, presented a theoretical relationship between RCRF, cloud fraction and cloud albedo.
It is noteworthy that the roles of cloud fraction and cloud albedo in shaping the Earth's climate have been investigated since the 1970s (Arakawa 1975, Schneide 1972), yet continue to defy satisfactory understanding and representation in climate models (Bender et al 2006, Bony andDufresne 2005). Besides providing a theoretical understanding of the relationship between CRF, cloud fraction and cloud albedo, the Liu2011 expression can also be used to infer cloud fraction or cloud albedo from measurements of radiative fluxes and the other quantity; Liu2011 used it to infer cloud albedo from surface-based radiation measurements and cloud fraction collected since 1997 by the US Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Program (Ackerman andStokes 2003, Stokes andSchwartz 1994). However, Liu2011 estimated cloud albedo using the cloud fraction data separately estimated with a different algorithm by Long et al (2006), which thus likely suffers from mutual contamination of errors in the retrievals of cloud fraction. The primary objective of this study is to extend Liu2011 and present a new approach that can be used to simultaneously infer cloud albedo and cloud fraction from surface radiation measurements, and thus eliminate the potential mutual error contamination. The new approach extends Liu2011 by utilizing the additional information carried by partitioning of the direct radiative flux. The analytical formulation is evaluated by applying it to the synthetic measurements derived from rigorous radiation transfer calculations and by comparing the results with the other surface-and satellite-based measurements.

Analytical formulation
The Liu2011 formulation is based only on the equations describing the total radiative fluxes measured at the surface. It has been known that the total radiation can be partitioned into direct and diffuse radiation fluxes. The partitioned radiative fluxes carry additional information on scattering media such as clouds, and have been measured at many surface-based sites, such as the ARM sites (Long et al 2006, McFarlane et al 2013. Note that the Long approach for estimating cloud fraction is based on an empirical relationship between the measurements of the normalized diffuse cloud effect and sky fraction observed with a total sky imager. Here we first recap the Liu2011 derivation, in a slightly different way, then present the equations describing the direct radiation, and finally show how to use the system of equations describing the total and direct radiation to simultaneously infer cloud albedo and cloud fraction.

Equations for total radiative flux
The commonly used single cloud layer model (Coakley et al 2005, Ramanathan 1987) is used as a first-order approximation in the analytical formulation described below. For this single cloud layer atmosphere, the all-sky downwelling SW radiative flux at the surface is given by where f , α r , α a , and T are cloud fraction, cloud albedo, cloud absorptance, and the transmittance of the atmosphere under the cloud for diffuse radiation. The land surface albedo is defined as The effect of surface albedo is considered using an approach similar to Wiscombe (1975). The first-order downwelling flux at surface F 1 is given by where F dn cld is the downwelling flux for a single-layer cloud with a surface albedo of zero, and is given by The upwelling all-sky flux at the surface is A combination of equations (1a), (2) and (3) yields where Substituting equation (5) into (4) gives Note that equation (6) reduces to equation (5b) of Liu2011 when surface albedo, and thus the total upwelling flux, is zero.

Equations for direct radiative flux
In a scattering atmosphere, the total radiation reaching the surface can be partitioned into direct and diffuse radiative fluxes. For the direct radiation, we have where the second subscript 'd' indicates that the corresponding quantities are for direct radiation. Note that only the first-order radiation needs to be considered because surface-reflected radiation is diffusive by nature. Based on the Beer-Bouguer-Lambert law for extinction (Liou 2002), F dn cld,d is given by where τ is the cloud optical thickness for the SW spectrum, and µ 0 is the cosine of the solar zenith angle. Combination of equations (7) and (8) yields another equation for f :

Further simplification and underlying physics
The cloud optical thickness τ can be further related to α r using a two-stream approximation suggested by Sagan and Pollack (1967) when cloud absorption is neglected (Meador and Weaver 1979): where g is the asymmetry factor of the cloud particles. From equation (10), τ is given as a function of cloud albedo and the asymmetry factor: Similarly, when cloud absorption is ignored, equation (6) becomes A combination of equations (9), (11) and (12) yields the expression that relates cloud albedo to the surface radiative fluxes: Equation (13a) reveals that cloud albedo is essentially a function of the ratio B 1 /B 2 , which can be calculated from measurements. Equation (13a) can be further approximated by piecewise polynomials to relate cloud albedo explicitly to the ratio B 1 /B 2 : The derivation and evaluation of equation (14) is given in the supplementary material (available at stacks.iop.org/ERL/ 8/044023/mmedia). In the calculation, the mean value of the asymmetry factor of 0.86 is used for the SW region; the selection and effect of the asymmetry factor can also be found in the supplementary material. After cloud albedo is obtained, cloud fraction is given by The preceding analyses suggest a two-step retrieval procedure: first calculate B 1 , B 2 , and B 1 /B 2 from the measured total and direct radiative fluxes, and then estimate cloud albedo using equation (13a) or (14), and cloud fraction using equation (15).
The physical meanings of B 1 and B 2 are worth noting. First, B 1 essentially represents the RCRF for the total downwelling SW radiation because the second term of the denominator of equation (13b) is much less than the first term and can be largely neglected. When surface albedo, and thus the total upwelling flux, are zero, equation (13b) reduces to equation (5b) of Liu2011. This point becomes more evident from the following alternative form of equation (13b): where R total denotes the RCRF for the total downwelling radiative flux. Analogous to the definition of RCRF for the total SW radiation, B 2 can be defined as the RCRF for the direct downwelling SW radiation. Thus, the analytical formulation reveals an important result: cloud fraction is primarily determined by the RCRF for direct radiation while cloud albedo is primarily determined by the ratio of the RCRFs for the total and direct radiation. Furthermore, the piecewise polynomials indicate that the relation between cloud albedo and B 1 /B 2 is roughly linear after the latter is greater than 0.078 72, which in turn suggests a linear relationship between cloud fraction and B 2 based on equation (15). The dual near-linear relationships can be seen clearly in figure 1, which shows cloud albedo (figure 1(a)) and cloud fraction (figure 1(d)) as a joint function of B 1 /B 2 and B 2 . The middle panels show the relative differences of cloud albedo and cloud fraction between the more rigorous rapid radiative transfer model (RRTM) calculations and the explicit polynomial approximation. The lower panels are the same as the middle panels, except for the relative difference between the exact results and the analytical expression equation (13). There are virtually no differences between the analytical expression, the polynomial approximation and the exact solution, except for very thin clouds. Note that the effects of surface albedo and cloud absorption are ignored in the results shown in figure 1 to focus on the essential physics and first-order factors. The description of the rigorous RRTM simulations and the effects from the secondary factors are deferred to section 3.
It is also worth noting that instrumental uncertainty is expected to have minimal effect on B 1 and B 2 based on equations (13b) and (13c) because the errors in the corresponding nominator and denominator tend to cancel with each other, and the total and direct fluxes likely suffer from similar measurement uncertainties. For the same reason, the retrieved cloud albedo and cloud fraction are expected to be largely independent of the instrumental uncertainty as well, an additional advantage of the new approach.

Validation and secondary effects
To reveal the first-order physics and have an analytical formulation, several assumptions are made in the discussion of section 2, including the neglect of absorption by clouds and absorbing gases such as ozone and water vapor. This section serves to (1) validate the analytical equations and (2) analyze and dissect the potential impacts of these secondary factors on the retrievals of cloud fraction and cloud albedo.

General description
We first apply the RRTM (Mlawer et al 1997, Oreopoulos andBarker 1999) to a set of inputs spanning a wide range of conditions to generate the synthetic radiative fluxes. The clear atmosphere follows the 1976 US Standard Atmosphere. A plane-parallel cloud layer containing water droplets is assumed at an altitude of 3.5 km, with cloud fraction varying from 0.01 to 1.0 in steps of 0.01, and cloud optical thicknesses of 0.5, 1, 2, . . . , 10, 12, . . . , 20, 24, . . . , 40, 46, . . . , 70, 80, . . . , 120 for the SW spectral region. The surface albedo varies from 0.0 to 0.8. The values of µ 0 are from 0.1 to 1.0.
The RRTM-calculated downwelling and upwelling SW radiative fluxes are substituted into the analytical expressions discussed in section 2 to calculate B 1 , B 2 , and B 1 /B 2 , and infer the corresponding values of cloud albedo and cloud fraction. As the evaluation reference, the 'true' value of cloud albedo related to a given τ is solved using the RRTM-calculated radiative fluxes for the cloud when it is over a black land surface. Briefly, considering the radiative transfer above and under the cloud layer, we have where F A and F D are the downwelling fluxes above and below the cloud, respectively; F B and F C are the upwelling fluxes above and below the cloud, respectively; the superscript 'T' denote the 'true' value directly calculated from the following equations: For each cloud, the 'true' cloud fraction, f T , is the pre-given value as stated above.

Effect of cloud absorption
Figure 2 compares the cloud albedo (a) and cloud fraction (b) estimated from the piecewise polynomial approximation against the RRTM 'true' values calculated as described above, but only for the surface albedo varying from 0.0 to 0.3-typical values observed at the ARM SGP site. The minimum and maximum of the departure from the 'true' values bracket the range of errors of the analytical retrievals. It is seen that the values of retrieved cloud albedo are generally overestimated compared to the 'true' values given by the RRTM, and the overestimation slightly increases with increasing cloud albedo. This overestimation of ∼14.7% on average arises from the neglect of cloud absorption, because both forward and backward radiative fluxes increase when the cloud absorption is neglected. The increase of backward radiative fluxes in the simulation leads to the overestimation of cloud albedo when the incoming SW radiation is not affected. It is noteworthy from figure 2(b) that, unlike cloud albedo, cloud absorption has minimal effects on the retrieved cloud fractions, with more than 90% of the points falling on the 1:1 line. The marginal effect of cloud absorption on the retrieval of cloud fraction is due to the resultant increases of α and surface fluxes canceling each other, as indicated by equation (6). To support our argument, figures 2(c)-(d) show the results under the same conditions, but with cloud absorption ignored in the RRTM simulation of the SW radiative fluxes. The retrieved values evidently agree better with the 'true' values. The slight relative difference of ∼1.5% in figure 2(c) is probably caused by the difference between the Sagan-Pollack approximation used in the analytical formulation and the discrete ordinates radiative transfer (DISORT) model (Stamnes et al 1988) used by the RRTM.
To further investigate the effect of cloud absorption on the retrieved cloud albedo, we examined the relationships between cloud albedo and cloud absorption for different values of τ and µ 0 (see figure 3(a)). In figure 3(a), cloud albedo and absorption both increase with τ for each solar zenith angle. The increase of cloud absorption becomes more pronounced for optically thick clouds. For optically thin clouds, cloud absorption increases with solar zenith angle due to the increased photon path length within the cloud. However, cloud absorption significantly decreases with increasing solar zenith angle when cloud becomes thick, which can be explained by the strong reflectance associated with a large solar zenith angle. The total photon path length decreases with increasing solar zenith angle because the number of photons in the forward direction is significantly reduced. As discussed above, both forward and backward radiation are overestimated when cloud absorption is ignored in the computation of radiative transfer within cloud. Therefore, only a portion of the energy related to cloud absorption causes the overestimate of cloud albedo. To numerically understand the overestimation, we compare the cloud albedo to that  (14) and (15)  computed when cloud absorption is ignored by the RRTM. A slight overestimate of cloud albedo can be found for thin clouds in figure 3(b). For thick clouds, the overestimation becomes pronounced and increases with decreasing solar zenith angle. The least squares fit of the dots for each solar zenith angle is given by Assuming α r is a sum of true albedo and the corresponding cloud absorptance, α a , we have Equation (17b) is similar to that used in Gautier and Landsfeld (1997). Thus, the neglect of cloud absorption is associated with a 5-13% overestimation of cloud albedo. Equation (17a) suggests that retrieval of cloud albedo can be modified such that The averaged relative difference between the modified cloud albedo and the 'true' values is 1.22%. Thus, the modified cloud albedo clearly exhibits better agreement with the 'true' values for all the solar angles ( figure 3(c)).

Effect of surface albedo
From equation (12), cloud albedo can be given by When the effect of cloud absorption is taken into account, cloud albedo is derived from equation (6): where the cloud fraction is given by equation (9). Thus, the computed cloud albedo for equation (18b) is generally smaller than equation (18a) for a given cloud condition. From the comparison between equations (18a) and (18b), it is seen that the second term of equation (18b) is relative to the effect of cloud absorption increasing with land surface albedo.
To examine the effect of surface albedo and multiple reflections, figure 4 compares the modified cloud albedo derived from equation (17c) to the 'true' value given by equation (16c) when α s = 0.0, 0.2, 0.4, 0.6, and 0.8. It can be seen that the modified cloud albedo compares generally well with the 'true' values when α s < 0.6, though the relative difference between the retrieved and true values increases from 1.5% to 25.9% with increasing surface albedo. The overestimation related to the effect of cloud absorption reaches its maximum over snow-or ice-covered land surfaces.

Comparison with other surface and satellite measurements
To further validate the new approach, we apply it to the surface measurements of SW radiation collected in 2010 at the ARM Southern Great Plains (SGP) site, and compare those with the separate retrievals of Liu2011 for cloud albedo and Long et al (2006) for cloud fraction. Also compared are the retrievals based on the geostationary operational environmental satellite (GOES) 8/11 measurements on a 0.5 • × 0.5 • grid over the ARM SGP domain (Minnis et al 2008). The surface-and satellite-based measurements are provided every 15 min and 1 h, respectively; monthly averages are used in the comparison (figure 5). Figure 5(a) compares cloud albedos from the new approach to those from old surface-based and satellite-based retrievals. The monthly averaged cloud albedos associated with the new approach are slightly greater than the old approach reported in Liu2011, mainly because of the correction for the cloud absorption in the new approach. Compared to the GOES-derived cloud albedo, the new approach overestimates by 35.67% on average. The greater cloud albedo from the new approach is probably related to the instruments and platforms used in the surface-and satellite-based measurements. For example, compared to surface-based retrieval of cloud properties, satellite-based retrievals are known to rely strongly on an accurate estimation of land surface albedo (Platnick et al 2001). Satellite measurements are usually more sensitive to optically thin high-level cirrus clouds (Berendes et al 2004) than surface measurements. Reconciling the difference between surfaceand satellite-based results deserves further investigation. Figure 5(b) compares the cloud fractions. The cloud fraction retrieved by Long et al (2006) is 5.75% larger on average than the new approach. Compared to the new approach, the satellite retrieval of cloud fraction is 4.21% smaller. Thus, the new approach has a slightly better agreement with the satellite retrieval of cloud fraction compared to the results in Long et al (2006).

Concluding remarks
By extending Liu2011 to include equations that describe the downwelling direct as well as total radiative fluxes, a new approach is presented to simultaneously retrieve cloud albedo and cloud fraction from the total and direct radiative fluxes. By analogy with the conventional relative cloud radiative forcing for the total radiation, a new quantity, which we call relative cloud radiative forcing for the direct radiation, is introduced. Analysis of the analytical formulation further reveals that cloud fraction is primarily determined by the relative cloud radiative forcing for the direct radiation, and cloud albedo is determined by the ratio between the relative cloud radiative forcing for the total radiation and direct radiation.
The analytical retrieval algorithm is evaluated by comparisons with synthetic radiative fluxes generated using the commonly used RRTM under a wide range of surface albedo, cloud fraction and optical thicknesses. The results show that the retrieved cloud albedo and cloud fraction agree well with the corresponding 'true' values given as inputs to the RRTM simulations. Neglecting cloud absorption can lead to an overestimation of cloud albedo by 5%-13%, depending on the cloud optical thickness and solar zenith angle. A modified algorithm is developed to minimize the effect of the cloud absorption on the estimate of cloud albedo based on the comparative analysis. The effect of the cloud absorption generally increases with the land surface albedo. Significant overestimation of cloud albedo is found for clouds over snow-or ice-covered land surfaces. The analytical retrieval algorithm is further evaluated by comparisons with year-long surface and satellite measurements. The cloud albedo derived by the new approach is slightly greater than the old surface-based approach, and both are significantly greater than the satellite-based retrievals. The cloud fraction from the new approach is slightly smaller than the old surface-based approach but greater than the satellite-based approach.
Several points are noteworthy. First, an obvious advantage of the new approach is its simultaneous retrievals of cloud fraction and cloud albedo, which eliminates the potential error contamination in existing approaches. For example, Liu2011 used the cloud fraction obtained separately from an empirically based fitting equation that relates diffuse ratio to sky-imager measurements (Long et al 2006). In essence the new approach is an extension and improvement of the combined Long approach for cloud fraction retrieval and Liu2011 approach for cloud albedo retrieval. The new approach also has the advantage of minimizing the effect of measurement errors. Furthermore, although this letter uses broadband radiative fluxes to illustrate the approach, the analytical formulation applies to monochromatic measurements as well. The use of a known spectral dependence at different wavelengths may help improve the retrieval accuracy and a comparison with the multi-wavelength approach reported in Min et al (2008) is underway. Second, the analytical formulation is presented here to emphasize the underlying physics. In principle, one can use more rigorous radiation transfer for the retrievals of cloud fraction and cloud optical depth. The key lies in utilizing the additional information carried by the partitioned radiation measurements that have been routinely conducted but underused. Nevertheless, the analytical formulation requires much less computational time compared to more rigorous radiative transfer models. Third, the equations for direct radiation are used to close the system of equations here for its simplicity; the equations for diffuse radiation can be equivalently used, and are given in the supplementary material (available at stacks.iop.org/ERL/8/044023/mmedia). Fourth, in principle, collocated satellite measurements can be used to infer the cloud thermodynamic phase, which will further improve the retrieval of cloud albedo and cloud fraction and tease out cloud absorption, as discussed in Liu2011. Fifth, the formulation assumes a single homogeneous cloud layer. Although the assumption of a pure single-layer cloud can be relaxed as an effective single-layer cloud in the presence of multilayer clouds (Liu et al 2011), the effective cloud albedo and cloud fraction likely depend on the structure of vertical cloud overlap. Other factors to consider are cloud horizontal inhomogeneity, 3D effects and horizontal photon transport; a more rigorous radiative transfer model is needed for this purpose (Marshak and Davis 2005). Finally, one of the greatest challenges in improving global climate models is to evaluate model results against adequate observations to identify and fix deficiencies in cloud-related parameterizations, which demands high-quality and consistent observational data. In view of existing high-resolution surface-based radiation measurements, the analytical formulation here can also serve as a theoretical framework for utilizing diffuse radiation and high-resolution surface-based radiation measurements for model evaluation.