Improved identification of clouds and ice / snow covered surfaces in SCIAMACHY observations

In the ultra-violet, visible and near infra-red wavelength range the presence of clouds can strongly affect the satellite-based passive remote sensing observation of constituents in the troposphere, because clouds effectively shield the lower part of the atmosphere. Therefore, cloud detection algorithms are of crucial importance in satellite remote sensing. However, the detection of clouds over snow/ice surfaces is particularly difficult in the visible wavelengths as both clouds an snow/ice are both white and highly reflective. The SCIAMACHY Polarisation Measurement Devices (PMD) Identification of Clouds and Ice/snow method (SPICI) uses the SCIAMACHY measurements in the wavelength range between 450 nm and 1.6 μm to make a distinction between clouds and ice/snow covered surfaces, specifically developed to identify cloud-free SCIAMACHY observations. For this purpose the on-board SCIAMACHY PMDs are used because they provide higher spatial resolution compared to the main spectrometer measurements. In this paper we expand on the original SPICI algorithm ( Krijger et al., 2005a) to also adequately detect clouds over snow-covered forests which is inherently difficult because of the similar spectral characteristics. Furthermore the SCIAMACHY measurements suffer from degradation with time. This must be corrected for adequate performance of SPICI over the full SCIAMACHY time range. Such a correction is described here. Finally the performance of the new SPICI algorithm is compared with various other datasets, such as from FRESCO, MICROS and AATSR, focusing on the algorithm improvements. Correspondence to: J. M. Krijger (krijger@sron.nl)


Introduction
Satellite-based passive remote sensing is commonly used to derive global information about the composition of the Earth's atmosphere.Information about the total column or even vertical profiles of different gases in the Earth atmosphere can be obtained by measuring the radiance (intensity) spectrum of sunlight reflected by the Earth's atmosphere, since these spectra contain absorption bands of gases present in the atmosphere.In the ultra-violet , visible and near infrared wavelength range the presence of clouds can strongly affect the observation of constituents in the troposphere, because clouds effectively shield the lower part of the atmosphere.When clouds are not properly accounted for, and especially when a significant part of the airmass of interest is below the cloud, (large) errors are introduced.Therefore, cloud detection algorithms are of crucial importance in satellite remote sensing.
The SCanning Imaging Absorption SpectroMeter for Atmospheric CartograpHY (SCIAMACHY) is a joint German/Dutch/Belgian instrument on board the ESA ENVISAT satellite, which was launched on 1 March 2002 and is still operational (Gottwald and Bovensmann, 2011).SCIAMACHY is a grating spectrometer measuring the radiance of reflected and back-scattered sunlight in the 240-2380 nm wavelength range at 0.2-1.5 nm spectral resolution.In order to account for the instrument polarisation sensitivity, SCIAMACHY measures the polarisation of reflected sunlight using seven broadband detectors, referred to as the Polarisation Measurement Devices (PMDs).
Several cloud-detection algorithms were developed for use in SCIAMACHY or GOME, the predecessor of SCIAMACHY, like ICFA (Kuze and Chance, 1994), OCRA (Loyola, 1998), CRAG (von Bargen et al., 2000), CRUSA (Wenig, 2001), FRESCO (Koelemeijer et al., 2001), SACURA (Kokhanovsky et al., 2003;Rozanov and Kokhanovsky, 2004), ROCINN (Loyola et al., 2007(Loyola et al., , 2010)), FRESCO+ (Wang et al., 2008), GOMECAT (Kurosu et al., 1998), HICRU (Grzegorski et al., 2004), SPICS (Lotz et al., 2009) and MICROS (Schlundt et al., 2011).Other algorithms are developed for the more recently launched OMI as the instrument does not measure the O 2 A band nor broadband PMD measurements as in SCIAMACHY, such as Inverse Cloud Model (Accarreta et al., 2004), Rotational Raman scattering cloud pressure (Joiner and Vasilkov, 2006).These methods either use the high spectral resolution measurements from the main spectrometer, the broadband PMD measurements, a combination of both, or external measurements.Because both clouds and ice/snow covered surfaces are highly reflective and white in the visible wavelength range, many of these algorithms are unable to distinguish between clouds or ice/snow covered surfaces in the observation.In principle, cloud-detection methods using the O 2 A band, like FRESCO, SACURA and ROCINN, can determine the pressure and thus discriminate white clouds from a white surface, but this is not part of the current versions of these algorithms and hence snow covered areas are excluded in the processing of their algorithms.Without the ability to distinguish between cloudy and ice/snow covered surfaces, all observations over ice/snow covered surfaces are usually flagged as cloudy and therefore often not used (for more details see also review of Vasilkov et al., 2010).A method to distinguish between clouds and ice/snow covered surfaces is thus of crucial importance to be able to identify cloud-free observations under these conditions.
Here we present an updated version of the SCIA-MACHY PMD Identification of Clouds and Ice/snow method (SPICI, pronounced with hard c) (Krijger et al., 2005a).It uses, among others, the SCIAMACHY PMD measurements in the wavelength range around 1.6 µm, where the reflectivity of ice/snow covered surfaces is significantly reduced while the reflectivity of clouds is still high.Using this clear spectral difference in reflectivity a distinction between clouds and ice/snow covered surfaces in the SCIAMACHY observations can be made.
Extensive use of SPICI revealed large areas in northern America and Asia as always cloudy during the snow season.This is common behaviour for cloud-retrieval algorithms that cannot distinguish between snow and clouds, but SPICI was intended to make this distinction.These areas turn out to be Boreal forests with snow-covered leaves that have a similar reflectance ratio between 850 nm and 1600 nm as water/ice clouds, varying slightly depending on the snow fraction.This means that the criterium used so far by SPICI to distinguish snow/ice from clouds, based on this ratio is not adequate for these conditions.In this paper we expand on the original SPICI algorithm (Krijger et al., 2005a) to also adequately detect clouds over such snow-covered forests.Furthermore the SCIAMACHY measurements suffer from degradation with time due to degradation of SCIAMACHY optical and detector elements.Therefor a correction for adequate performance of SPICI over the full SCIAMACHY time range has been derived by combining various radiance-monitoring methods.Finally the performance of the new SPICI algorithm is compared with various other datasets, from FRESCO, MICROS and AATSR, focusing on the correctness and stability of the new SPICI algorithm improvements The great strength of SPICI remains its simple implementation without the need of external datasets, allowing scientists to implement their own cloud-mask without downloading large datasets.In addition they can optimize the used thresholds based upon their own needs.
The structure of this paper is as follows.In Sect. 2 we describe SCIAMACHY and its Polarisation Measurement Devices.The original SPICI algorithm is summarized in Sect.3, with the new method to detect snow covered forests in Sect. 4 and the degradation correction in Sect. 5. Validation of the improved SPICI algorithm is presented in Sect.6.We finish with a summary and discussion in Sect.7.

SCIAMACHY
SCIAMACHY's primary mission objective is to perform global measurements of trace gases in the troposphere and stratosphere (Bovensmann et al., 1999).The instrument provides column and/or vertical profile information on O 3 , H 2 CO, SO 2 , BrO, OClO, NO 2 , H 2 O, CO, CO 2 , CH 4 , N 2 O, O 2 , (O 2 ) 2 , and on clouds and aerosols as well.SCIA-MACHY measures the radiance of reflected and backscattered sunlight in 8 channels, covering the 240-1750 nm (channels 1-6) and two IR bands 1940-2040 nm and 2265-2380 nm (channels 7 and 8, respectively) at 0.2-1.5 nm spectral resolution.SCIAMACHY alternates between nadir and limb viewing modes for most part of the orbit.The swath of the instrument in nadir mode is 960 km, and the individual main channel measurements have a footprint on Earth ranging from 60 km × 30 km to 240 km × 30 km (across × along track), thereby providing global coverage in a period of six days (Bovensmann et al., 1999).

SCIAMACHY PMD measurements
SCIAMACHY is a highly polarisation-sensitive instrument due to the instrument's gratings and mirrors.Neglect of such polarisation sensitivity can lead to errors in the radiances of several tens of percent at wavelengths where the instrument polarisation sensitivity is highest.In order to correct for this polarisation sensitivity, SCIAMACHY measures the polarisation of reflected sunlight using seven broadband detectors, referred to as Polarisation Measurement Devices (see Table 1), which roughly cover the spectral range of the main spectrometer.Because the PMDs are mainly sensitive to parallel (to the instrument slit) polarised light, while the main channel spectrometer is sensitive to both polarisation components, information on the polarisation of the incoming light is obtained by combining the two measurements (Slijkhuis, 2000).The PMDs are read out at 40 Hz, but are down-sampled to 32 Hz for processing.This still gives a much better spatial resolution (∼7 km × 30 km) than the main spectral channels where the fastest read-out occurs at 8 Hz, and more commonly at 1 Hz.This high PMD spatial resolution allows for easier detection of clouds which is the reason why we use the PMDs in the SPICI algorithm.In this paper we focus on four PMDs (PMD 2 to 5) that cover the visible and near-infrared wavelength range from 450 nm to 1700 nm.
Because we employ only the ratio between different PMD measurements any effect from polarisation on such ratios is due to the difference in polarisation-sensitivity or degree of polarisation.For all PMDs these differences are small, except PMD 1 and 7, which are not used in this study.

SPICI original
In Krijger et al. (2005a) the complete derivation of the SPICI algorithm is described.In brief, the SPICI method allows for fast and simple identification of cloud-free SCIA-MACHY PMD observations.The NIR SCIAMACHY PMD measurements are employed to distinguish between clouds and ice/snow covered surfaces, which is more complicated employing the visible wavelengths only.The method employs the ratios of different SCIAMACHY PMD measurements which makes the approach very robust with respect to e.g.calibration uncertainties.The threshold values were defined using collocated observations with the well known and validated high-spatial resolution MODIS data.
The initial algorithm consists of two steps: in the first step the algorithm only uses PMD 2, 3 and 4 to determine the presence of a white surface in the visible wavelength range.Because at these wavelengths one cannot separate clouds from ice/snow covered surfaces, a second step is needed to finally detect cloud-free observations also over ice/snow covered surface, based upon the different spectral behavior of clouds and snow/ice between 850 nm and 1600 nm, as both have high reflectance around 850 nm, however snow/ice reflectance is much lower than cloud reflectance around 1600 nm.The SPICI method is easily implemented and can be summarised as follows: with instrument signal S i from PMD i (in BU, dark-signalcorrected).The threshold values were tuned so cloudy observations are rarely flagged cloud-free, although some cloudfree observations are mistakenly flagged cloudy.The thresholds can be somewhat relaxed in cases where some cloud contamination is acceptable.Studies which are extremely sensitive to clouds should decrease the saturation threshold (T ) to values as low as 0.1.

Snow and forest
After its publication, SPICI became part of the offical SCIA-MACHY ESA level 2 product.Its extensive use revealed that SPICI declared large areas in northern America and Asia as always cloudy during the snow season.This is common behaviour for cloud-retrieval algorithms that cannot distinguish between snow and clouds, but SPICI was intended to make this distinction.These areas turn out to be Boreal forests with snow-covered leaves that have a similar reflectance ratio between 850 nm and 1600 nm as water/ice clouds, varying slightly depending on the snow fraction.This means that the criterium used so far by SPICI to distinguish snow/ice from clouds and which is based on this ratio was not adequate for these conditions.To overcome this deficiency, we use a normalized difference snow and vegetation indices as used (among others) by MODIS1 , but adapted to SCIAMACHY data.The PMD 2/PMD 5 ratio is a snow index, using the fact that snow has high visible reflectance and low infrared reflectance, while clouds have the same reflectance at both wavelength ranges: if the ratio is high enough, the pixel is snow-covered.However, this snow index can become too small in forested locations with snow, which leads to the original problem of misidentifying snow as cloud.Therefore the snow index is used in combination with a vegetation index: the PMD 4/PMD 3 ratio.The vegetation index is based on the fact that plants absorb red light and reflect near-infrared light, i.e. vegetation has a high index.Even snow-covered forests have a relatively high vegetation index due to the relative thin snow cover in these cases.To mark these areas as cloud-free, without including other land types with clouds, the snow index is allowed to be lower when the vegetation index is higher.Figure 1 illustrates this by showing a density plot with the ratio of the number of cloud-free and clouded observations (according to collocated MODIS observations) as a function of SCIAMACHY PMD 4/PMD 3 and PMD 2/PMD 5.The exact ratio is indicated in the color bar going from cloud-dominated (blue) to cloud-free dominated (red).Note that the upper right region is completely dominated by cloud-free observations.Given the still uncalibrated state of the SCIAMACHY PMDs an empirical fit was made to this area, allowing a maximum of 10 % misidentified scenes (snow-covered while actually cloudy according to MODIS).The obtained fit is shown in Fig. 1 as the solid curve.In the new version of SPICI all observations in this area (which have been deemed cloudy in the first step of the SPICI algorithm) are now considered snow-covered (and thus cloud-free) instead.This can be formulated as follows: ( No fraction W 25 lower than 0.08 is ever observed, thus avoiding the singularity in the function.Figure 2 shows the locations of the observations that went from clouded to cloud-free in the new version of SPICI and which clearly correspond with Boreal forest.used to derive the m-factors are performed at different scanangles from those used during normal nadir scanning.This especially affects the PMDs as polarisation is very dependent on scan-mirror angle (Schutgens and Stammes, 2003;Krijger et al., 2009).Therefore we searched for alternative methods to determine PMD degradation.The SPICI application requires only ratios of PMD signals, therefore we are only interested in relative degradation.For GOME it was shown that the different PMDs degrade initially at the same rate (Krijger et al., 2005b) and therefore the signal ratios remain constant, but eventually the degradation rates differed.Assuming similar behaviour for the identical SCIAMACHY PMDs here also the changes will be different and the SPICI threshold should be adjusted.Here we choose therefore to correct the PMD ratios and not the individual PMDs.
The large wavelength range covered by the PMDs require different approach to assess the amount of degradation.For the visible wavelengths the same assumption can be made as for the original SPICI algorithm, namely that clouds are white and remain so over time.We identify fully clouded pixels according to the FRESCO algorithm.The behaviour in time of these PMD pixels is indicated as the corresponding "FRESCO" method in Fig. 3. SCIAMACHY main science channels show scan-angle dependent degradation (Bramstedt, 2009), therefore we checked for a similar effect in the PMDs by separating the FRESCO cloudy PMD observations into Extreme-East, East, West and Extreme West.Only in the ratio of the PMD 2 and PMD 3 signals a slight scan-angle dependent effect can be observed, but given the observed uncertainties this effect can at this moment be ignored.An alternative and immediate verification of the PMD degradation in the SPICI algorithm is by only monitoring the most colorsaturated PMD observations (T < 0.1).This is shown as the "SPICI" method in Fig. 3 and it is clear that the different methods agree.
For the non-visible wavelengths this approach can not be used, as ice/water clouds appear slightly differently in the infrared.However, the radiance at scenes like the Libyan desert is expected to be rather constant.In Fig. 3 we have plotted all PMD observations over the Libyan desert as a function of time showing the direct relative degradation.No observations are shown for PMD 2/PMD 3 ratio due to the effect of varying atmospheric constituents on the PMD 2 radiance signal levels.
An alternative method to monitor the PMD 4 and PMD 5 degradation uses the fact that in the infrared only a few different dominant albedo regimes are present.When observing the distribution of PMD 5/PMD 4 measurements, it immediately becomes clear that there are three dominant ratios, one from ice/snow/clouds (low values), one from desert areas (high values) and one from ocean combined with other land observations (middle values).Relative PMD degradation will shift the absolute values of these distributions but not the shape of the distribution, except for a squeeze.See Fig. 4 for an example of the distribution of the PMD 5/PMD 4 ratio in January 2003 (black), where no degradation is expected yet and the same month in 2009 (red).Notice the presence of the three maxima, but the shift/squeezing of the shape as a result of relative degradation.We have determined for each 8 day period the distribution shape of the PMD 5/PMD 4 ratios and the squeeze of the shape, thus assuming a multiplicative degradation.The degradation as obtained from this method is in good agreement with the results from other methods in Fig. 3 (bottom right, labeled "Profile").All methods show in varying degree a small amount of seasonal variation due to varying atmospheric conditions, however these variations do not change the general observed trend.
As expected the curves shown in Fig. 3 differ from the PMD m-factors due to the different scan angles.Figure 3 shows that all relative PMD degradation appear linear in time in first approximation.A linear fit was made through the combination of all methods (except the m-factors) for each PMD ratio as a function of modified Julian date (days since 1 January 2000).The results of the fits are shown in Table 2.These fits will allow correction for "relative" PMD degradation in nadir viewing observations.

Validation
Originally SPICI thresholds were derived and compared to specific co-located scenes observed very close in time by both SCIAMACHY and MODIS.As such it is imperative to also compare SPICI cloud detection with other cloud retrieval algorithms for SCIAMACHY, for which we use FRESCO (Wang et al., 2008) and MICROS (Schlundt et al., 2011).Given the new additions to SPICI we also focus specifically on the degradation or stability of the algorithm over time.As the distinction between clouds and snow/ice is one of SPICI large advantages, we focus on this part of the algorithm by comparing snow-covered scenes co-observed with AATSR.The new improvement to SPICI for snowcovered forests is also compared with AATSR, employing a dataset focusing specifically on such regions.

MICROS
Recently Schlundt et al. (2011) developed a new algorithm, called MICROS (MerIs Cloud fRaction fOr Sciamachy), to determine a geometric cloud fraction for SCIAMACHY ground scenes at nadir using MERIS spectral observations with an accuracy higher than from PMD algorithms.The MICROS algorithm is the improved version of MCFA (MERIS Cloud Fraction Algorithm) by Kokhanovsky et al. (2009).While much more accurate than SPICI due to the much higher spatial resolution it has still problems detecting snow/ice and requires the download of large external datasets together with collocation information of the observations.So in practice it is not very easy to determine the cloudiness of SCIAMACHY observations in this manner.However, because of the close collocation in both space and time between MERIS and SCIAMACHY (both on ENVISAT) it is well suited for a comparison with SPICI.We describe the MICROS method briefly here.

MERIS (MEdium Resolution Imaging Spectrometer
) is a multi-spectral instrument, also mounted on the ENVISAT satellite.It measures the solar radiation reflected by the Earth's atmosphere or surface in the visible and near-infrared part of the electromagnetic spectrum between 390 and 1040 nm at a spectral resolution of 1.8 nm (Bezy and Rast, 1999).By looking in the nadir direction, the Earth is imaged with a spatial resolution of 1.2 km × 1.2 km at reduced resolution (RR; operational) and 300 m × 300 m at full resolution (FR; coastal zones and over land).The instrument's field of view is about 68.5 • covering a swath width of 1150 km, which yields a global coverage every three days.
The MERIS RR data are used as sub-pixel information for the relatively large SCIAMACHY pixel which enables the derivation of SCIAMACHY cloud fraction with an accuracy much higher when compared to PMD based cloud fraction.Both instruments observe the same ground scene simultaneously.

Method
The purpose of the MERIS cloud screening part in MICROS is to identify each pixel as either clear land, clear water or cloud, as described in full detail in Schlundt et al. (2011).Difficulties arise particularly with regard to clouds over bright surfaces, such as snow/ice covered regions, deserts or sun glint areas, which are treated separately by the MICROS algorithm.The algorithm, based on the threshold approach of various MERIS channels (from 412.5 nm to 865 nm), is made up of several steps where specific conditions are checked in order to distinguish between sun glint, water, land, bare soil, optically thin and thick clouds.The very last step of the MERIS cloud screening takes cloud border and adjacency effects (e.g.cloud shadowing or increased radiation by actinic flux) into account.This is done by additionally flagging the two adjacent pixels also as cloudy (in all directions), when a clouded MERIS pixels is found.

Results
We have compared the MICROS thick cloud fraction with the SPICI cloud fraction (averaged over the same main science channel footprint) for a total of 90 orbits randomly selected between 2005 and 2009, focusing here on the lower (snowfree) latitudes, both as function of location and time.This allows the validation of the performance of SPICI over time.In Table 3 the results of the comparison between SPICI and MICROS are summarised as a function of time.Looking at the average over the entire dataset in 61 % (±8 % standard deviation between different orbits) of all cases both MI-CROS and SPICI agree on an observation being cloudy, in 27 % (±6 %) both agree on an observation being cloud-free.In 9 % (±4 %) SPICI classifies a pixel as cloudy, while according to MICROS the observation is cloud-free.However again the main issue is how often SPICI identifies a clouded scene (according to MICROS) as cloud-free, and according to this comparison a total of only 3 % (±2 %) of the measurements is misidentified.These are all scenes which according to MICROS only have a very small amount of cloud present (which some trace gas retrievals can cope with) or are at the (always uncertain) edges of clouds, where MICROS also  flags adjacent unclouded MERIS observations as clouded, to take adjacency effects into account.As shown in Table 3 these fractions do not change within the uncertainties over time, indicating that the degradation is accurately corrected and that the dataset is large enough to be representative.

FRESCO
We performed another validation with the FRESCO+ algorithm (Wang et al., 2008), which uses the O 2 A band to detect clouds.The algorithm was originally developed for GOME-1 (Koelemeijer et al., 2001), has been successfully applied to SCIAMACHY and constantly improved since that time.
While it can only detect effective cloud fraction, it does a good job of detecting the presence of clouds, although not yet over snow/ice covered scenes.As such we focus our comparison here with SPICI on snow/ice free scenes.For each FRESCO cloud retrieval we average the collocated SPICI (PMD) measurements.We selected a number of random days, 1 January 2003, 1 September 2008, 1 January 2009 and 1 January 2010 (to also determine degradation effects).
The results are shown in Table 4 which show that in 66 % (±2 %) of all cases both FRESCO and SPICI agree on an observation being cloudy.In 22 % (±1 %) both agree on an observation being cloud-free.In 8 % (±4 %) SPICI classifies a pixel as cloudy, while according to FRESCO the observation is cloud-free.Again the main issue is how often SPICI identifies a clouded scene as cloud-free, and according to this comparison a total of only 4 % (±0.2 %) of the measurements is misidentified, and these are all scenes which according to FRESCO only have a very small amount of cloud present.
The comparison with FRESCO show very similar results as the comparison between MICROS and SPICI which provides good confidence in the analysis presented.

AATSR
As both MERIS MICROS and SCIAMACHY FRESCO are not (well) suited for cloud detection over snow/ice surfaces, while the snow/ice cloud detection of SPICI is one of the main additions of the algorithm compared to other cloud detection algorithms, we employed AATSR to validate our SPICI snow/ice cloud detection including the applied degradation correction.For this the improved AATSR snow detection method first presented in Istomina et al. (2010) were used.
The AATSR (Advanced Along Track Scanning Radiometer) instrument onboard the ENVISAT satellite is a lowresolution conical imaging spectrometer operating in the visible, near-infrared, mid-infrared and thermal spectrum ranges.The prime scientific objective of AATSR is to establish continuity of the ATSR-1 and ATSR-2 data sets of precise sea surface temperature (SST), thereby ensuring the production of a 10 year near-continuous data set for climate research.The (A)ATSR instruments are unique in their use of along track scanning to provide two views of the surface, and thus improve possibilities for atmospheric correction.The surface is first viewed along the direction of the orbit track, at an angle of 55 • (forward view), as the spacecraft flies towards the scene.Then, 150 s later, when the satellite has moved approximately 1000 km along the ground track, a second observation of the same scene is made at the sub-satellite point (nadir view).The nominal pixel size of AATSR is 1 km × 1 km at the center of the nadir swath and 1 km × 2 km at the center of the forward swath.The AATSR field of view comprises two 512 km wide curved swaths, with 555 pixels across the nadir swath and 371 pixels across the forward swath.

Method
The cloud/snow discrimination method described in full detail in Istomina et al. (2010) uses a combination of relative thresholds in seven AATSR channels in order to discriminate the spectral behavior of clear snow scene from that of cloud, land, ocean, etc.These criteria connect nadir top-ofatmosphere reflectance in visible, near-infrared and brightness temperatures in the thermal-infrared.While the visible and near-infrared criteria select scenes with the spectral behaviour similar to a snow spectrum, the thermal infrared (TIR) thresholds distinguish cloud-free areas over surfaces with emissivity close to unity (snow, open ocean).

Snow and clouds
The validation dataset contains 100 daytime AATSR scenes with approximate dimensions 10 • × 15 • each, with a spatial resolution of 1 km × 1 km (nadir view).It covers the timespan of nearly 6 years, starting in 2003 and ending in 2009, with most data from 2008 (42 scenes).The vast majority of scenes has been observed in the Western Hemisphere beyond the Arctic Circle, at particular locations like Northern Alaska, Beaufort Sea, Queen Elisabeth Islands and Greenland.A few scenes have been observed in the Eastern Hemisphere, at Spitsbergen and the Scandinavian Peninsula.All the data have been observed from March to May, to ensure sufficient snow coverage and enough daylight (solar zenith angle from 50 to 80 • ).As a first comparison with the cloud/snow discrimination method we focus on different surface types covered with almost continuous snow coverage: snow-covered sea ice, snow-covered mountains and fjords.Partial snow coverage like e.g. in the case of snowy forests has been avoided in this first comparison and will be addressed separately in the next section.Open ocean and bare soil areas are almost not represented in this dataset.
Figure 5 shows for two example AATSR observations the snow fraction (the number of cloud-free snow-covered scenes observations by AATSR divided by the total number of AATSR scenes inside the SCIAMACHY PMD scene) for each collocated SCIAMACHY PMD footprint.Only SCIA-MACHY measurements with a solar zenith angle smaller than 85 • are shown.Red indicates a large snow/ice fraction, decreasing through yellow to blue, which indicates nonsnow/ice surfaces (sea, clouds, land cover other than snow, etc).Each PMD footprint that according to SPICI is a cloudfree snow covered scene has been outlined in black.Ideally all AATSR observations with snow fraction 1 across a SCIAMACHY PMD pixel is identified by SPICI as cloudfree snow covered.
The comparison results are shown in Table 5.When comparing these results with those of MICROS or FRESCO, much more AATSR-clear scenes are indicated as cloudcovered by SPICI, because SPICI is cloud-conservative, focusing on only marking those scenes as clear that are really clear and in case of doubt, mark them clouded.Distinction between clouds and snow/ice is complicated, such that many algorithms cannot make the distinction.To avoid misidentifications SPICI is very conservative over such scene and hence less clear scenes are flagged when comparing SPICI with MICROS or FRESCO over snow-free areas as in the previous sections.
Focusing on AATSR clear snow-covered scenes, SPICI detects 45 % of all these clear scenes (and thus conservatively marks 55 % of all clear scenes as clouded).Do note that most other cloud algorithms detect no cloud-free scenes in these circumstances, so this is quite an improvement.As intended SPICI rarely misidentifies cloudy scenes: only 1-5 % of all observations are identified as clear where AATSR identifies the observations as cloudy.This happens mostly at the edges of large snow-covered scenes, most likely due to thin transparent clouds over snow-covered scenes.However, as most SCIAMACHY trace-gas observations use an integration time of 0.25 to 1 s (compared to the 0.03125 s of the PMDs), one must combine 8 to 32 PMD measurements to determine an average cloud-fraction.Since most trace-gas observation methods ignore an observation when even a single PMD measurement indicates clouds, such minor edge effects of a single PMD footprint can be ignored as most neighbouring footprints will already indicate a cloud.Also the number of misidentifications over time remains quite constant, showing that the degradation correction works for the snow/ice detection part of SPICI.

Snow-covered forests
We also specifically investigated the performance of the SPICI algorithm over snow covered Boreal forests, introduced in Sect. 4. For this we compare with AATSR over 355 specially selected scenes over partially snow covered Boreal forests between 2003 and 2009, again between March and May.Snow and cloud cover has been determined with the previously described AATSR method.A total of 362 × 10 3 SPICI observations were studied.AATSR (unlike SPICI) cannot distinguish between snow-covered ground or snow-covered forest.However we employed the Global Land Cover Characterization (GLLC -version 2) database2 to determine the amount of forest in each SPICI observation.Focusing now on the forested observations SPICI detects 43 % of all forested (20 % or more forest cover according GLCC) cloud-free (maximum 10 % clouds according AATSR) scenes.This means over 57 % of all cloud-free snow-covered forest are overlooked, but as SPICI is cloudconservative this is expected for such difficult-to-distinguish snow-covered regions.To confirm the cloud-conservative approach of SPICI; In only 6 % of the cases where SPICI indicates a snow-covered forest, AATSR observed a cloud instead.Again these cases mostly happen at the edge of cloud fields.Yet indeed in 94 % of all cases where SPICI indicates a snow-covered forest, AATSR observes snow and no clouds, hence both agree this is a cloud-free scene.This shows that the addition of the snow-covered forest index in SPICI is useful indeed to identify clear scenes above snow-covered forests.

Discussion and summary
We presented an improved version of the SCIAMACHY PMD Identification of Clouds and Ice/snow method (SPICI).The SPICI algorithm uses the SCIAMACHY measurements in the wavelength range between 450 nm and 1.6 µm to make a distinction between clouds and ice/snow covered surfaces, specifically developed to identify cloud-free SCIAMACHY observations.The SCIAMACHY Polarisation Measurement Devices (PMDs) are used because they provide higher spatial resolution compared to the main spectrometer measurements.The improvements (compared to Krijger et al., 2005a) include a snow over vegetation detection using the PMD 2/PMD 5 compared to PMD 4/PMD 3 ratio for the least saturated, T (= cloudy) observations.In addition we provided a correction for SCIAMACHY PMD degradation in time.This correction is obtained by observing clouds, deserts and albedo distribution profiles.This results in different corrections than the traditional m-factors, as expected due to angle dependent scan-mirror contamination, which causes the observed degradation.
Including both the new expansion for snow-covered forest and the PMD degradation correction, the new SPICI algorithm can be summarized as follows: with m the modified Julian date MJD2000.When validated with AATSR's most advanced method of cloud-free snow detection and MERIS's most advanced method of cloud detection, SPICI works extremely well, with less then 5 % misidentifications where clouded observations are identified as cloud-free.
Concluding, the improved SPICI method allows identification of cloud-free snow/ice covered regions including snowcovered forests.Furthermore, the improved algorithm effectively deals with the PMD degradation over time.

Fig. 1 .
Fig. 1.Density plot showing the ratio of the number of cloud-free and clouded observations (according to MODIS) as a function of the SCIAMACHY ratios PMD 4/PMD 3 and PMD 2/PMD 5. Note that dark red indicates no cloudy measurements were present for those particular ratios, while white at the edges of the image indicates no measurements were present at all.The black curve indicates the cloud-free optimised boundary between the cloud-free and clouded regions.

Fig. 2 .
Fig. 2. The locations of measurements that the new snow and forest test indicates as being snow-covered forests instead of clouds during January 2003.All locations are found over boreal forest.

Fig. 3 .
Fig. 3. Degradation for different PMD ratios (indicated at top) and for different methods (colour-coded, detailed description of methods in the main text) as a function of time.Indicated at the bottom of each figure are the components of a linear fit to the degradation, which can be used for degradation correction.Note the use here of PMD 4/PMD 5 instead of PMD 5/PMD 4 for consistent decrease of the ratio over time.

Fig. 4 .
Fig. 4. The top-left panel shows the distribution profile of PMD 5/PMD 4 ratio as original in January 2003 (black), and the profile in January 2009 (red).It is clear that they have a similar profile with several maxima, apart from a shift.In the top-right panel the distribution of the PMD 5/PMD 4 ratio is again shown but now the January 2009 PMD 5/PMD 4 ratio has been multiplied with increasing degradation values, thus shifting the profile distribution.The bottom-left panel shows the correlation between the January 2009 and original January 2003 profile distribution, as a function of the applied degradation factor.From this panel the most likely (best correlated) degradation factor can be determined.The bottom-right panel repeats the upper-left panel, but now adds the degradation corrected profile for January 2009, showing the similarity in profiles.Repeating this method for each month allows determination of the degradation for the PMD 5/PMD 4 ratio.

Fig. 5 .
Fig. 5. Snow fraction, the number of cloud-free snow-covered scenes observations by AATSR divided by the total number of AATSR scenes inside each collocated SCIAMACHY PMD scene, as indicated by the colorbar on the right.Each footprint indicated by SPICI as cloud-free snow covered scene has been outlined in black.Left panel: 6 May 2004, right panel: 5 May 2009.

Table 1 .
Wavelength ranges of SCIAMACHY Polarisation Measurement Devices.

Table 2 .
Linear fit description of the various relative PMD degradation, with m the modified Julian date MJD2000.

Table 3 .
Comparison of scene identifications between SPICI and MICROS over time.A grand total of 364 × 10 3 SPICI observations (after filtering for solar zenith angle and latitude) were compared to co-located averaged MICROS observations.A scene is considered clouded by MICROS when the average MICROS cloudfraction over the scenes is more than 10 %.The table shows the fraction of the total number of SPICI observations which fall under the catagory indicated in the top of the table.Total number count of SPICI obervations employed for the comparison for each year in the right column.

Table 4 .
Comparison of scene identifications between SPICI and FRESCO over time.A total of 260 × 10 3 SPICI observations were compared to co-located averaged FRESCO observations.A scene is considered clouded by FRESCO when the average FRESCO cloudfraction over the scenes is more than 10 %.The table shows the fraction of the total number of SPICI observations which fall under the catagory indicated in the top of the table.Total number count of SPICI obervations employed for the comparison for each year in the right column.

Table 5 .
Comparison of scene identifications between SPICI and AATSR over time.A total of 91 × 10 3 SPICI observations were compared to co-located averaged AATSR observations.A scenes is considered clouded by AATSR when the average AATSR cloudfraction over the scenes is more than 10 %.The table shows the fraction of the total number of SPICI observations which fall under the catagory indicated in the top of the table.Total number count of SPICI obervations employed for the comparison for each year in the right column.