The Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data – HOAPS-3

The availability of microwave instruments on satellite platforms allows the retrieval of essential water cycle components at high quality for improved understanding and evaluation of water processes in climate modelling. HOAPS-3, the latest version of the satellite climatology “Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data” provides fields of turbulent heat fluxes, evaporation, precipitation, freshwater flux and related atmospheric variables over the global ice-free ocean. This paper describes the content, methodology and retrievals of the HOAPS climatology. A sophisticated processing chain, including all available Special Sensor Microwave Imager (SSM /I) instruments aboard the satellites of the Defense Meteorological Satellites Program (DMSP) and careful inter-sensor calibration, ensures a homogeneous time-series with dense data sampling and hence detailed information of the underlying weather situations. The completely reprocessed data set with a continuous time series from 1987 to 2005 contains neural network based algorithms for precipitation and wind speed and Advanced Very High Resolution Radiometer (AVHRR) based SST fields. Additionally, a new 85 GHz synthesis procedure for the defective SSM /I channels on DMSP F08 from 1988 on has been implemented. Freely available monthly and pentad means, twice daily composites and scan-based data make HOAPS-3 a versatile data set for studying ocean-atmosphere interaction on di ffere t temporal and spatial scales. HOAPS-3 data products are available via http://www.hoaps.org .


Introduction
A thorough knowledge of the global water cycle is crucially important for a detailed understanding and successful modelling of the global climate system.Measuring the relevant quantities is however a notoriously difficult task, especially over the global ocean with generally insufficient spatial and temporal coverage by ships and/or buoy observations.The advent of satellite platforms provided the opportunity to retrieve global data sets with substantially improved coverage over sea and over land.Especially since the availability of passive microwave detectors in space, the observation of several essential water cycle components and related parameters became possible with high quality over the global ocean.
Correspondence to: A. Andersson (axel.andersson@zmaw.de)Due to the inherent physical properties of long-wave electromagnetic radiative transfer through the atmosphere, the microwave spectral range is well suited for the retrieval of atmospheric water vapor column content as well as parameters needed to derive the freshwater flux components at the ocean surface.After first experience with microwave sensors, the availability of the Special Sensor Microwave Imager (SSM/I) aboard the satellites of the Defense Meteorological Satellites Program (DMSP) since 1987 motivated several international research groups to derive and provide long-term global fields of water cycle related quantities.Depending on the main application purpose, data from various additional sources are blended with the SSM/I satellite observations in these emerging climatologies.
Generally, these data sets fall into two categories providing on the one hand turbulent surface moisture fluxes and on the other hand precipitation estimates.The international framework of the SEAFLUX project of the World Climate Research Programme (WCRP) and the International Published by Copernicus Publications.Precipitation Working Group (IPWG) foster the development of satellite-based data sets of surface turbulent fluxes (e.g.Bentamy et al., 2003;Chou et al., 2003;Kubota and Tomita, 2007;Yu et al., 2008) and precipitation (e.g.Adler et al., 2003;Hilburn and Wentz, 2008;Hsu et al., 1997;Huffman et al., 2007;Joyce et al., 2004;Kubota et al., 2007;Xie and Arkin, 1997).While these various satellite products are of importance in their own right, ocean and climate modellers are also extremely interested in global fields of ocean surface freshwater flux.This net gain or loss of water through the ocean surface due to evaporation and precipitation is of crucial importance for the coupling between ocean and atmosphere and as a driving force of global ocean circulation.In principle, the mentioned satellite retrieved data sets could be combined to estimate the global ocean freshwater flux.This would be a highly required but difficult task, as different data sources have to be combined while there is no comprehensive in-situ validation data available (Schlosser and Houser, 2007).
HOAPS is hitherto the only generally available compilation of both precipitation and evaporation with the goal of estimating the freshwater flux from one consistently derived global satellite product.The HOAPS retrieval schemes do not make use of numerical weather prediction (NWP) data, i.e. no additional first guess data is used.This allows the use of HOAPS as an independent satellite based validation reference for model evaluation.Furthermore, an inter-sensor calibration has been implemented in order to achieve a homogenized SSM/I brightness temperature record.In combination with multi-satellite averages and an efficient sea ice detection procedure the SSM/I measurements are used for the derivation of all parameters in the HOAPS data base.Since 1987, six SSM/I instruments have been launched into space and turned out to be stable measuring instruments (Hollinger et al., 1990) for a reliable climatological data set.For the sake of long-term homogeneity, the use of data from different satellite instruments has been avoided as much as possible.The only exception is the sea surface temperature (SST), which is obtained from Advanced Very High Resolution Radiometer (AVHRR) measurements.
The present paper describes the improved and extended version three of HOAPS that is publicly available from the web site www.hoaps.org(Andersson et al., 2007a,b,c).This paper focuses on the technical description of the processing of the satellite input data and the different retrieval schemes for geophysical parameters, while the evaluation and application on climatological scale is subject of related publications (e.g.Andersson et al., 2010a,b;Romanova et al., 2010).Further validations of HOAPS-3 wind speed and precipitation retrievals have been published by Winterfeldt et al. (2010) and Klepp et al. (2010).
Section 2 introduces the key features and data subsets of HOAPS-3.In Sect. 3 the HOAPS data sources are described along with technical data handling.The algorithms and parameter descriptions of the HOAPS data set are given in Sect. 4.

The HOAPS data set
The initial version of HOAPS (Graßl et al., 2000;Jost et al., 2002) was mainly based on the algorithms of Bauer and Schlüssel (1993) and was released first in 1998.Fairall et al. (1996Fairall et al. ( , 2003) ) EVAP, LATE latent heat transfer coefficient Fairall et al. (1996Fairall et al. ( , 2003) TRCE sensible heat flux at sea surface Fairall et al. (1996Fairall et al. ( , 2003) ) HEAT longwave net flux at sea surface Schlüssel (1995) FNET vertically integrated liquid water Bauer (1992) LWPA vertically integrated total water Bauer and Schlüssel (1993) TWPA vertically integrated water vapor Schlüssel and Emery (1990)  Comparisons with other ocean surface turbulent flux data sets within the SEAFLUX project (Kubota et al., 2003;Chou et al., 2004;Curry et al., 2004) indicated that the evaporation in the first HOAPS version was substantially low biased in the tropics.The second version, HOAPS II (Fennig et al., 2006a,b), was available since mid 2004.It included major improvements, such as the concurrent use of all available SSM/I instruments up to December 2002 including inter-calibration and improved algorithms to derive sea surface flux parameters.Further comparisons revealed however, that the global mean precipitation in HOAPS II was significantly lower compared to other climatologies, resulting in an implausibly large global net ocean surface freshwater flux into the atmosphere on the climatological scale.This and a few other issues led to the development of the most recent version HOAPS-3 (Andersson et al., 2007a,b,c).The key features and major changes of each version of HOAPS are listed in Table 1.
The HOAPS-3 data sets cover a time period of more than 18 years of data between July 1987 and December 2005.In total 15 parameters are retrieved and stored in the native SSM/I resolution covering the global ice-free ocean.An overview of all parameters is given in Table 2. Several gridded products are available with a spatial resolution up to 0.5 degree.The temporal resolution varies from monthly and pentad averages to twice daily data.Homogeneous timeseries over several generations of space-borne radiometers are required for the derivation of accurate statistics for climatological analyses.Towards this goal, great care was put into instrument stability assessment and inter-sensor calibration (see Sect. 3.3).

HOAPS data products
Three data subsets of HOAPS-3 are supplied for all 15 parameters ranging from instantaneous scan based pixel-level data to gridded data products This allows HOAPS to be used for a wide range of applications.Moreover, all data sets are available in NetCDF format including extensive meta data.The gridded HOAPS data product can be obtained through the CERA data base of the World Data Center for Climate (WDCC) http://cera-www.dkrz.deor via the HOAPS homepage http://www.hoaps.org.

HOAPS-S
The HOAPS-S data subset contains all parameters in the native SSM/I scan-oriented pixel-level resolution for each individual satellite, providing the basis for the gridded data products HOAPS-G and HOAPS-C.

HOAPS-G
HOAPS-G climatological data sets contain globally gridded data with a spatial resolution of 0.5 degree.Two data sets with temporal average periods of 5 days (pentad) and one month are available (Andersson et al., 2007c,a).The mean fields are computed from the HOAPS-S data by aggregating all SSM/I pixels that have their center of the FOV falling in the respective grid box and averaging over the specific time period.The resulting data sets consist of multi-satellite averages that include all SSM/I instruments available at the same time.The data fields are supplemented by basic statistical information about standard deviation and number of observations per grid cell.

HOAPS-C
HOAPS-C contains one-degree twice daily globally gridded multi-satellite composite fields of each parameter (Andersson et al., 2007b  daily temporal resolution.Each grid cell contains the spatial average of data from the specific satellite that passed this grid box closest to 12:00 and 24:00 UTC respectively.Hence, each grid cell contains data from only one satellite pass and there is no average from two or more satellite passes.This method provides higher spatial consistency on the sub-daily time scale than just averaging all available data to twice daily mean fields.E.g. the appearance of back and forth moving fronts of fast moving weather systems is minimized.The fields are archived for 00:00 to 12:00 UTC and 12:00 to 24:00 UTC.Time steps in the data files are 00:00 UTC (00:00-12:00 UTC overpasses) and 12:00 UTC (12:00-24:00 UTC overpasses).

Data sources and processing
The starting point in the data processing chain, as illustrated in Fig. 1, are SSM/I antenna temperatures.These are converted to an internal brightness temperature (TB) data set.This procedure includes several instrument-related corrections, inter-sensor calibration, and the flagging of land and sea ice covered pixels.From the TB data set and the NODC/RSMAS Pathfinder SST, geophysical parameters are calculated on the native SSM/I resolution (HOAPS-S).Fi-nally, the previously described gridded HOAPS products are generated from the scan-based HOAPS-S data.The SSM/I instrument, the handling of SSM/I raw data, and the SST input data set are described in the following, while the specific retrieval algorithms for geophysical parameters will be described in Sect. 4.

3.1
The SSM/I Instrument SSM/I sensors have been carried aboard the DMSP satellite series since 1987.An extensive description of the instrument and satellite characteristics has been published by Hollinger et al. (1990) and Wentz (1991).Hence, only a very short summary of essential information is given here.The DMSP satellites fly in a near-circular, sun-synchronous orbit, with an inclination of 98.8 • at an approximate altitude of 860 km.Each day, 14.1 orbits with a period of about 102 min are performed.The Earth's surface is sampled with a conical scan at a constant local zenith angle of 53.1 • and a 1400 km wide swath.A nearly complete coverage of the Earth by one SSM/I is achieved within two to three days.Due to the orbit inclination and swath width, the regions poleward of 87.5 • are not covered.To date, six SSM/I instruments have been successfully launched aboard the F08, F10, F11, F13, F14 and F15 spacecraft.All satellites have a local equator crossing time between 5 and 10 a.m./p.m. for the descending/ascending node.The F08 had a reversed orbit with the ascending node in the morning.Also, the view direction of the SSM/I on this satellite is, differently from the others, to the aft.Most of the DMSP satellites have a very stable orbit as shown in Fig. 2. The temporal variation of the equator crossing times is less than three hours for all satellites.At the end of the HOAPS-3 time period the orbits of F14 and F15 begin to decay noticeably, but are still within 2-3 h of original time.
The SSM/I is a conically scanning, seven channel radiometer measuring emitted microwave radiation at four frequency intervals centered at 19.35, 22.235, 37.0, and 85.5 GHz.All data are sampled at horizontal and vertical polarization, except for the 22.235 GHz channel, which measures only vertically polarized radiation.The channels will be referred to as 19, 22, 37, and 85 GHz hereafter and the corresponding TBs of each channel and polarization as TB 19v/h , TB 22v , TB 37v/h , and TB 85v/h .
The spatial resolution varies from 69 km by 43 km with a sampling frequency of 25 km for the 19 GHz channel to 15 km by 13 km with 12.5 km sampling frequency for the 85 GHz channel.The 85 GHz channels are sampled for each rotation of the instrument (A and B-scans) with a resolution of 128 uniformly spaced pixels, while the remaining channels are sampled every other scan (A-scans) with a resolution of 64 pixels.A fixed cold space reflector and a reference black body hot load are used for continuous onboard calibration.

Data processing
Antenna Temperature Tapes (ATT) from Remote Sensing Systems (RSS) as well as Temperature Data Records (TDR) from the National Environmental Satellite, Data, and Information System (NESDIS) are used as SSM/I input data for HOAPS.Either data source has been used during several time periods for various logistic reasons.An analysis of SSM/I data from different sources (Ritchie et al., 1998), including NESDIS TDR and RSS ATT data, showed no systematic differences between these data sets.
In a first step the different input data sets are preprocessed into a common internal data format containing navigated and calibrated antenna temperatures.The antenna temperatures are then converted to TBs according to Wentz (1991) using a fixed set of antenna pattern corrections.In addition, this procedure treats several known issues with the radiometer, including corrections for an along-scan bias (Wentz, 1991;Colton and Poe, 1999) and zenith angle variations (Fuhrhop and Simmer, 1996).The latter is however not applicable to the 85 GHz channels due to their higher resolution.
For the application of the retrieval algorithms it is important that about the same area is seen by all channels.Due to the higher frequency, a single field of view (FOV) at 85 GHz covers only about 18% of the area sampled at 37 GHz (cf.e.g.Spencer et al., 1989).Since the 85 Ghz channels are sampled twice as often in each direction compared to all other SSM/I channels, nine neighboring 85 GHz pixels of the A and B-Scans are averaged down to the resolution of the corresponding 37 GHz pixel.The 85 GHz pixels are averaged with their gaussian weighted distance from the center FOV analogue to the 37 GHz antenna pattern.

Inter-sensor calibration
In order to ensure a homogeneous time series of the successive SSM/Is, the slightly varying individual instrument characteristics have to be corrected by an inter-sensor calibration of the radiometers.The calibration procedure is based on the SSM/I on DMSP F11 which is selected as the reference for a relative intercalibration.In tests with different wind speed algorithms against in-situ buoy data this radiometer exhibits a reliable long-term stability.Furthermore it has a temporal overlap with most of the other radiometers.The SSM/Is on F08 and F15, which do not have a temporal overlap with F11, are calibrated to the F10 and F13 radiometers, respectively.
The calibration coefficients are determined by linear regression from match-up data sets of the radiometers that are based on the TBs after along-scan bias and zenith angle correction.The match-up data sets are constructed from rainfree oceanic TBs of each channel that are binned into a global 1 • by 1 • grid and averaged for 10 days.These selection criteria increase the polarization ratio of the channels, filter out highly variable events such as synoptic fronts with precipitation, and minimize the influence of diurnal cycle variations.For each radiometer a match-up data set with the selected reference radiometer is compiled from the collocated gridded 10-day mean TBs for at least one overlapping year.This match-up data set is randomized and histogram-equalized with a histogram bin size of 1 K for the range of 100 K to 300 K.The calibration coefficients are then calculated by a least squares linear regression between the TB pairs of both radiometers.
Due to the accurate and stable in-orbit calibration of the SSM/I, the coefficients are considered to be constant during the lifetime of a radiometer (Colton and Poe, 1999).However, the unstable orbit of the DMSP F10 made a recalibration for the radiometer aboard this satellite necessary.The F10 calibration coefficients were determined for the years 1992 and 1996 and then linearly interpolated for the intermediate years.
The results of the intercalibration procedure are depicted in Figs. 3 to 6.The graphs show the distribution of the differences for each channel relative to the calibration reference, the SSM/I on DMSP F11.The differences have been calculated for collocated 10-day mean rain-free TBs on a 1 • by 1 • grid for the years 1995 (Figs. 3 and 4 6) and categorized in 0.1 K wide bins.Grid cells with less than one third of the maximum number of observations per month have been filtered in order to reduce errors from insufficient sampling.This corresponds to a threshold of about 130 pixels per grid cell.A clear improvement for the bias is achieved for nearly all channels compared to the TBs without intercalibration.Furthermore, the calibration appears to be stable since the bias for the SSM/I on DMSP-F13 does not differ by more than 0.2 K for all channels between 1995 and 1998.However, the 37 h channel exhibits a slightly higher bias for the intercalibrated TBs compared to the uncalibrated state.
As an example for the temporal stability of the intercalibration, Fig. 7 shows a comparison of global mean TB 19v time series for each SSM/I before and after application of the inter-sensor calibration coefficients.Only minor differences in the brightness temperature time series remain after the intercalibration has been applied.These deviations are mainly caused by differences in the sampling and the diurnal cycle of the individual satellites.

Synthesis of 85 GHz channel data
Due to thermal problems, the SSM/I on the DMSP F08 had to be switched off in December 1987.When the radiometer was reactivated, the 85 GHz vertical polarization channel exhibited significantly increased noise levels.Later, similar problems occurred with the 85 GHz horizontal polarization channel (Hollinger et al., 1990;Wentz, 1992;Ferraro, 1997).Therefore, the 85 GHz channels of the SSM/I on DMSP F08 are regarded as defective from April 1988 until the end of the lifetime of the radiometer and the respective data is not used in HOAPS.
This failure affects all HOAPS parameters depending on these channels (i. and total water content, and longwave net flux at sea surface).A computationally efficient way of handling this problem is to synthesize approximate values of the missing 85 GHz information from the remaining channels.This allows the application of the same algorithms for all parameters throughout all analysis steps without individual replacement algorithms for each parameter.It has, however, to be kept in mind that such a procedure cannot fully reconstruct the specific 85 GHz information but is a computationally convenient way of handling the missing data by a plausible replacement.Moreover, this method is only valid over open water.Over land surfaces this general approach is not applicable due to the strongly varying surface emissivity.
The synthesis is based on a neural network approach, which is used to estimate TB 85v/h of the 9-pixel averaged 85 GHz FOV (see Sect. 3.2) from the other SSM/I channels (TB 19v/h , TB 22v , and TB 37v/h ).
The training data set for the neural network was compiled from randomly selected brightness temperatures of SSM/I radiometers on board DMSP F11, F13, and F14 from 1998.In this year the El Niño event generated extreme values, resulting in a wide distribution of TBs.
Pixels containing high near surface wind speeds or strong precipitation generate comparable TB signatures in the 19 and 37 GHz channels.The accuracy of the synthesis procedure is improved when such situations are treated separately.A good relationship to distinguish these scenes from calm and rain-free conditions proved to be:  Based on this relationship, two input data sets were constructed and the algorithm was divided into two parts.In order to ensure a wide distribution of the training data with different situations of precipitation, the input values were binned in a two-dimensional grid with TB 22v and precipitation rate as axis, yielding approximately 30 000 data samples for each training data set.Finally, the input values were normalized to the sample mean and standard deviation.These data sets were used to train two fully connected feed forward networks consisting of six input neurons (TB 19v/h , TB 22v , TB 37v/h , TB 19v -TB 19h ), two hidden layers, and two output neurons (TB 85h and TB 85v ) (Fig. 8).Additionally the TB 19v -TB 19h input neuron is directly connected to the second layer to increase the impact of absorption/emission by liquid hydrometeors in this spectral range.The neurons in the hidden layer are non-linear with the sigmoid function tanh as the unit's activation function.For the output neuron the activation function is linear.
Figure 9 shows the results for the derived synthesis algorithm applied to a random sample of over-ocean TBs for the satellites that were used to train the neural network and the SSM/I on DMSP F08.The F08 data is from the year 1987, the F11 data is from 1998, and the F13 and F14 data is from 2000.The best results are achieved for the SSM/I on DMSP F14 with a bias of less than 0.5 K for horizontal and vertical polarization.For the SSM/I on DMSP F08 the bias values are slightly higher compared to the other satellites with 0.75 K for the vertical polarization and 1.3 K for the horizontal polarization.This could be explained by the thermal problems with the F08 SSM/I that impaired the onboard calibration of the radiometer and caused a larger instrument noise.Furthermore, only limited data from July to November 1987 is available for the comparison prior to the failure of the instrument.Table 3 summarizes the resulting values for daily global mean values for the HOAPS parameters depending on the 85 GHz channels calculated for the year 1998.The results for the precipitation rate, vertically integrated liquid and total water content turn out to be reliably applicable.For the longwave net flux the values calculated with synthesized TBs exhibit a considerable bias of 7.13 W m −2 due to the 4th power relationship with TB 85v in the algorithm.

Land mask and sea ice detection
For SSM/I pixels within 50 km of any coastline and sea ice margin no geophysical parameters are derived in HOAPS.This is to avoid any contamination of the data by the abrupt change in surface emissivity between water and land or sea ice covered areas, respectively.The corresponding areas are flagged during the processing in the pixel-level data (see Fig. 1).
The land-sea mask is derived from the Global Land Onekm Base Elevation data base (GLOBE Task Team, 1999).This data set is further adjusted by first removing small islands and landmasses with a diameter of less than 5 km, treating these areas as open water.In a second step the coastlines of the remaining land areas are expanded 50 km into the sea.
To account for the varying sea ice margins, a daily sea ice mask is generated from the HOAPS SSM/I TBs.These maps are created in two steps.First the total sea ice covered fraction within a single SSM/I FOV is computed using the NASA Team sea ice algorithm of Swift et al. (1985).The resulting sea ice observations from all available SSM/I instruments are then gridded to common daily mean fields on a regular 0.5 • × 0.5 • grid.In order to distinguish between short-lived Earth Syst.Sci.Data, 2, 215-234, 2010 www.earth-syst-sci-data.net/2/215/2010/ strong rain events and persisting sea ice, which are characterized by similar TB signatures, only grid boxes with an average sea ice fraction above 15% for at least 5 consecutive days are flagged as ice covered.Daily sea ice maps are then derived from this reduced data set by re-expanding the reliably identified sea ice areas in time and space and filling remaining data gaps by spatial and temporal interpolation.Finally, the resulting sea ice margin is extended 50 km into the ocean to avoid any spurious sea ice signal in the HOAPS retrievals.

Sea surface temperature
To exclusively rely on satellite data for the computation of latent and sensible heat flux parameters, the NOAA National Oceanographic Data Center (NODC) and Miami's Rosenstiel School of Marine and Atmospheric Science (RSMAS) Oceans Pathfinder Version 5.0 SST (Kilpatrick et al., 2001;Casey, 2004;NODC, 2008) product is used within HOAPS.In the Pathfinder data set a quality flag with a value from 0 to 7 is assigned for each pixel, with 7 being the highest quality observation.For the further processing in HOAPS, only pixels with an overall quality flag of four or higher are accepted.This conforms to the Pathfinder Version 4 "best-SST" product and involves several quality checks including a cloud screening (c.f.Kilpatrick et al. (2001) for details).
The Pathfinder algorithm coefficients are determined by a regression of the measured ocean skin infrared brightness temperatures against buoy data (Kilpatrick et al., 2001), which means the derived SST technically represents a subskin SST with the mean sub-skin minus skin SST difference (cold skin effect) removed.However, the mean daytime Pathfinder SST is found to be roughly 0.1 K colder than in-situ SST and the nighttime SST is about 0.3 K colder (Reynolds et al., 2002).The reason for this cold bias is unclear, but is most likely caused by undetected clouds.For most regions the value of this cold bias has roughly the same magnitude as the cool skin effect.Hence, the Pathfinder SST is treated as a skin SST for the surface flux calculations in HOAPS.
The AVHRR Pathfinder day and night-time observations are averaged to daily mean SST maps with a grid resolution of 0.25 • .Data void regions are filled by spatial and temporal interpolation.Finally, the data is remapped to native SSM/I pixel resolution.The SST value for each SSM/I pixel is selected from the daily map by the center position of the respective FOV.
In the wake of the volcanic eruption of Mt.Pinatubo in June 1991 the high load of volcanic aerosol contaminated the AVHRR radiance measurements.Undetected aerosol leads to a nighttime cold bias in the daily mean SST fields (Reynolds, 1993).During the first phase for about 6 months following the eruption a significant bias is introduced in the SST.This affects the retrievals of latent and sensible heat flux in HOAPS.In 1992 the effect weakens, but may persist until 1993.However, for the decaying phase we consider the signal not to be entirely spurious since an analogous response with a decay time of up to 31 months after the eruption is independently observed in global tropospheric temperatures and water vapor data (Soden et al., 2002;Harries and Futyan, 2006).

HOAPS retrieval schemes and parameterizations
In this section the algorithms and parameterizations to derive HOAPS parameters are described with a focus on previously unpublished algorithms.All parameters shown in Table 2 are derived at the SSM/I pixel level for the individual FOVs.Parameters such as atmospheric water content, precipitation, and wind speed are derived directly from SSM/I TB observations.Some parameters additionally depend on SST data, which is the only ancillary data set used in the retrievals (see Sect. 3.6).The retrievals that make use of the SST data are the latent and sensible heat fluxes which are estimated with a bulk aerodynamic approach after Fairall et al. (1996Fairall et al. ( , 2003) ) (see Sect. 4.3) and the retrieval of the longwave net flux (see Sect. 4.4).

Wind speed
The near surface wind speed cannot be measured directly from space.Over the ocean, the effect of the wind friction on the surface alters the emissivity of the ocean surface in the microwave spectrum.Thus the wind speed is dynamically coupled to gravity and capillary waves and foam coverage of the ocean surface, which in turn influences the surface emissivity and hence the upwelling radiances.At satellite altitude the SSM/I brightness temperatures (especially TB 19h and TB 37h ) increase non-linearly with increasing wind speed (e.g.Webster Jr. et al., 1976). Furthermore, TB 19v  and TB 37v have to be considered in order to remove the atmospheric contribution to the radiometric signal, which would otherwise tamper with the wind induced surface signatures.
Here a neural network approach is used to derive the wind speed directly from the SSM/I brightness temperatures.Compared to linear models (e.g.Goodberlet et al., 1989) this method allows to account for the non-linearity of the problem and additionally to deal with different atmospheric conditions such as clear sky or cloudy regimes.This is particularly important in situations where the atmospheric moisture increases and the relation of the observed brightness temperatures to the wind speed becomes significantly nonlinear (Petty andKatsaros, 1992, 1994).
Following a neural network approach after Krasnopolsky et al. (1995), the wind speed is estimated using a fully connected 3-layer feed forward neural network.The network is composed of one input layer utilizing TB 19v/h , TB 22v , and TB 37v/h , a hidden layer with three neurons and an output layer with one neuron, the wind speed.A diagram of the neural network is shown in Fig. 10.The activation function of all three neurons in the hidden layer is non-linear by using the sigmoid function tanh.The output neuron is linear in order to maximize the networks' extrapolation capabilities.
Two different data sets serve as input for the training data set.One consists of simulated SSM/I brightness temperatures which are derived from radiative transfer simulations based on radiosonde profiles.The second input data set is compiled from collocated SSM/I and buoy observations.This approach ensures the representativeness of the input and output data of the neural network.
The radiosonde data set consists of about 2000 globally distributed atmospheric profiles.These are subsampled in groups of equidistant near surface wind speed steps of 2 m/s within a range from 2 to 30 m/s.The radiation emitted from the ocean surface is calculated with the surface emissivity model of Bauer (2001) and the top of the atmosphere radiance is calculated with the radiative transport scheme of Schlüssel and Emery (1990).
The buoy data set is compiled using match-ups between SSM/I F11 TBs with near-surface wind speed measurements from 20 buoys from the National Data Buoy Center (NDBC) and 59 buoys from the Tropical Atmosphere/Ocean (TAO) array for the years 1997 and 1998 (see http://www.ndbc.noaa.gov/).The temporal and spatial collocation criteria of a maximum difference of 30 min and 50 km between satellite and buoy observations resulted in a match-up data set of about 470 000 samples.The buoy wind speed measurements were individually converted to a height of 10 m wind using a logarithmic wind profile assuming neutral stratification.Both data sets are then combined by collecting all data samples in 2 m/s wide wind speed bins, ensuring that all parts of the wind spectrum are equally weighted.The neural network's training data set is then compiled by taking an equal number of randomized samples from each of the wind speed bins.The resulting data set consists of about 15 000 samples.By taking randomized samples separately for each bin, the input TBs cover the whole possible range of atmospheric conditions including high water vapor observations from the TAO buoys.The smaller wind speed ranges are mixed with data samples from both data sets, while most samples larger than 15 m/s originate from the set of radiative transfer calculations.
Figure 11 shows the performance of the wind speed retrieval compared to the algorithms of Schlüssel and Luthardt (1991) and Stogryn et al. (1994).Applied to the verification data set, the neural network wind speed algorithm exhibits a very low bias of −0.02 m/s, a standard deviation of 1.5 m/s, and a correlation coefficient of r = 0.98.The standard deviation for the HOAPS neural network algorithm is much lower compared to 4.0 m/s and 5.26 m/s for the two other retrievals.
Earth Syst.Sci.Data, 2, 215-234, 2010 www.earth-syst-sci-data.net/2/215/2010/ When the entire buoy data set is used (not shown) the bias is 0.08 m/s, with a standard deviation of 1.65 m/s and r = 0.83.Comparable values were found by Winterfeldt et al. (2010) who compared wind speed from HOAPS, NCEP, and the QuikSCAT/SeaWinds scatterometer with buoy data from the North Sea and and North Atlantic.The comparisons for the HOAPS algorithm showed a mean RMS of 2 m/s for near coastal as well as for open ocean regions.
In contrast to the neural network algorithm the other wind speed retrievals shown in Fig. 11 exhibit a strong tendency to saturate at high wind speeds.This results in an overall underestimation with biases of −2.1 m/s for the Schlüssel algorithm and −4.9 m/s for the Stogryn algorithm.

Humidity parameters
The algorithm for the specific air humidity at the reference level of 10 m above the sea surface is based on a two-step regression method after Schulz et al. (1993) and its refinement by Schlüssel (1996).Bentamy et al. (2003) showed that the chosen linear channel combination is sufficient for the estimation of the near surface specific humidity.By recalculating the regression coefficients with an improved training data set they were able to remove a bias of 2 g/kg in the inner tropics due to high precipitable water values.This updated version of the algorithm is also used in HOAPS.
For the derivation of the evaporation through the bulk formula, the difference in humidity, i.e. sea surface specific humidity minus near surface specific humidity, is calculated.The specific air humidity just above the sea surface is calculated from the saturation humidity at the sea surface temperature using the Magnus formula (Murray, 1967).An approximate salinity correction is applied by scaling the value for pure water with a factor of 0.98.

Turbulent heat flux parameterization and evaporation
HOAPS-3 latent and sensible heat fluxes are parameterized using the Coupled Ocean-Atmosphere Response Experiment (COARE) bulk flux algorithm version 2.6a (Bradley et al., 2000), which is an updated version of the COARE 2.5b algorithm (Fairall et al., 1996).This new version is based on a flux database containing measurements from higher latitudes and under stronger wind conditions compared to the previous version.With minor modifications of physics and parameterizations to the version 2.6a, the algorithm is published as COARE 3.0a by Fairall et al. (2003).These changes were related to the SST cool skin and warm layer part of the algorithm.However, due to the lack of continuos diurnal cycle information on the surface radiation budget from the SSM/I or AVHRR measurements, this part of the COARE algorithm is not implemented in HOAPS.Neither is the precipitation flux calculation implemented.
The COARE algorithm iteratively estimates stability dependent scaling parameters and wind gustiness to account for subscale variability.The latent and sensible heat fluxes Q l and Q s are calculated by: (2) and where ρ is air density, c p is specific heat of the air at constant pressure, u is the wind speed at 10 m height, L E is the latent heat of evaporation (calculated SST-dependent), C T is the Stanton number, C E is the Dalton number, T s is the SST, Θ a is the potential air temperature, q s is the saturation specific humidity at the sea surface, and q a is the specific humidity at the 10 m atmospheric measurement level.Apart from the air temperature and the transfer coefficients C T and C E , all parameters are derived from SSM/I measurements or from the SST as described above.For the implementation of the algorithm in the HOAPS processing chain no modifications were introduced to the default COARE boundary layer height and roughness length parameterizations.
A direct measurement of the near surface air temperature from space is not possible.Hence, it is estimated from the SST and near surface humidity using the mean of two simple bulk approaches: (a) The satellite derived near surface specific humidity is assumed to be at a constant relative humidity of 80% as proposed by Liu et al. (1994).
(b) A constant temperature difference of 1 K between sea surface and air temperature is assumed (Wells and King-Hele, 1990).
On climatological scale these assumptions are valid for the majority of oceanic conditions.However, in regions with strong stable stratification of the atmospheric surface layer, this approach will affect the quality of the sensible heat flux estimate.The impact of these assumptions on the latent heat flux retrieval is smaller since the air temperature has only a secondary effect on the parameterization through the stability of the atmosphere (Liu et al., 1994).
The evaporation in mm/h is calculated from the latent heat flux after Fairall et al. (1996) by: where ρ 0 is the freshwater density as a function of temperature.

Longwave net flux
The longwave net flux at the sea surface is computed following Schlüssel (1995) from the atmospheric back radiation and the SST utilizing the spectrally integrated surface emissivity which is close to 0.89 (Gardashov et al., 1988) and the Stefan-Boltzmann constant.The atmospheric back radiation is retrieved directly from SSM/I measurements with the algorithm of Schlüssel (1995)  The liquid and ice phase in clouds and precipitation strongly change the radiative properties of the atmosphere within the microwave spectrum and directly influence the radiometer signal.To account for different atmospheric conditions, a threshold technique is used to distinguish between cloudfree, cloudy and rainy scenes.A different set of individual algorithms is used to derive liquid and total water paths for each situation.This approach is based on the internally calculated total water path and rain-rates as described in Bauer and Schlüssel (1993).
In HOAPS, the vertically integrated water vapor is derived with the globally applicable retrieval scheme of Schlüssel and Emery (1990).The majority of information on the integrated water vapor is contained in the 22 GHz channel that is located within a water vapor absorption line.This algorithm additionally uses TB 37v to correct for undesirable effects of atmospheric liquid-water content or extreme near surface temperature gradients in TB 22v .

Precipitation
Microwave based retrievals of precipitation are based on the direct interaction of the radiation field and the hydrometeors (water droplets, ice particles).The emission from cloud and liquid hydrometeor particles at small frequencies causes a strong increase of the brightness temperatures and results in a strong contrast to the radiometrically cold sea surface background.In contrast to that, the brightness temperatures at high frequencies decrease with increasing precipitation due to the scattering of radiation by frozen hydrometeors.
The successful usage of a neural network for the nearsurface wind speed retrieval also encouraged a similar approach for the precipitation algorithm.A fully connected 3layer feed forward network was constructed that includes: an input layer with six neurons at TB 19v/h , TB 22v , TB 37v/h , and TB 85v , a hidden layer with three non-linear neurons using the tanh function as the activation function, and an output layer with one linear neuron, the scaled rain rate R * .Additionally, the input neurons of TB 19v and TB 22v are connected linearly to the output neuron.Figure 12 shows a schematic diagram of the neural network.
The training data set for the neural network set is based on radiative transfer calculations as described in Bauer et al. (2006a,b).This data set contains one month (August 2004) of assimilated SSM/I TBs and the corresponding precipitation values of the European Centre for Medium-Range Weather Forecast (ECMWF) model.
Using precipitation values from the operational variational analysis to compile the training data set not only ensures the statistical representativeness of the input data, it also makes use of the advantage to have the background meteorological surface fields and profiles consistent with the measured SSM/I brightness temperatures.Moreover, it allows to con-  This data set covers a wide variety of globally distributed rainfall events including extreme rainfall in hurricanes and snowfall at high latitudes.However, as it consists of more than 2.5 million data samples, it must be filtered in order to ensure good coverage and equal weight of all possible input TB combinations.Hence it was binned in a two-dimensional grid using TB 22v and the polarization difference TB 19v -TB 19h as axis.The final training data set was then compiled by randomly taking an equal number of samples from each bin, which makes about 110 000 data samples in total of which only 50% were actually used for training.In order to avoid an inappropriately high influence of the larger uncertainties at higher precipitation rates, the training values were scaled non-linearly by the following transformation, with R * representing the transformed precipitation rate: A lower threshold value is applied to the algorithm, below which the precipitation signal is considered to be zero.From experience with the formerly used algorithm, a value of 0.3 mm/h turned out to be an appropriate limit for distinguishing between a real precipitation signal and background noise.
The results of this training procedure are depicted in Table 4.The first line shows the quality of the derived algorithm compared to the unused second half of the training data set, the verification data set.Due to the strong peak of the distribution in case of low precipitation rates, the mean precipitation rate is only about 0.25 mm/h.The derived algorithm reproduces the test cases with a correlation of r = 0.95, a very small bias and a RMS of about 0.06 mm/h, which represents the theoretical algorithm accuracy.Testing the algorithm against the complete ECMWF data set yields a lower mean value of 0.16 mm/h, which is due to the larger sample size compared to the verification data, a bias of −0.06 mm/h, and a RMS of 0.16 mm/h.When a lower cutoff of 0.3 mm/h is applied to the ECMWF data set as it is done in the HOAPS retrieval, the mean precipitation rate increases to 0.78 mm/h.Bias and RMS are very low with values of −0.17 mm/d and 0.34 mm/h, respectively.The correlation is in the range of the two previous cases with r = 0.91.
The detection rates of the algorithm compared to the entire ECMWF data set are 93.4% correctness for the raining and rain-free cases, with a probability of rain detection of 70.5% and a false alarm rate of 20.4%.
Figure 13a shows the results of the training procedure as a comparison of the verification data set, which has not been used in the training procedure (see above), with the new the neural network algorithm, the retrieval of Bauer and Schlüssel (1993) (HOAPS II), and the scattering algorithm of Ferraro (1997).As expected, the neural network performs significantly better in reproducing the rain rates of the verification data set compared to the other data sets.The Bauer algorithm exhibits a systematic underestimation of the precipitation.For rain rates exceeding 7 mm/h the neural network algorithm also exhibits an underestimation compared to the verification data set.The few cases with rain rates exceeding 12 mm/h in the verification data set are reproduced by the neural network and the Ferraro scattering algorithm with comparable values around 10 mm/h.The latter performs not well for lower rain rates due to the missing emission signal in the algorithm.
Figure 13b shows accumulated histograms of the rain rates derived with the three different precipitation algorithms on a logarithmic scale.The histogram bins represent the accumulated fraction of the total reference precipitation from the verification data set in the range from 0.2 mm/h up to 30 mm/h.About 90% of the total ECMWF precipitation comes from rain rates up to 1 mm/h.The new neural network algorithm overestimates the rain rate below 0.3 mm/h and then slightly underestimates rain rates above 0.5 mm/h.This leads to a small underestimation of the total precipitation.Applying a lower threshold at 0.3 mm/h results in a total bias of −0.166 mm/h (Table 4).
Further validations for cold season snowfall over the ocean were carried out by Klepp et al. (2010) by co-locating ship based optical disdrometer and satellite data.In contrast to other satellite derived precipitation estimates the HOAPS retrieval was able to detect even light amounts of snowfall with an accuracy of 96%.

Freshwater flux
The difference between the evaporation and precipitation yields the oceanic freshwater flux into the atmosphere.Since   A sophisticated processing chain has been implemented that includes the handling of SSM/I raw data, the retrieval of geophysical parameters, and gridding procedures.The concurrent use of all available SSM/I instruments with an inter-sensor calibration for the brightness temperatures ensures a homogenized time series with dense data sampling and hence detailed information of the underlying weather situations.Additionally, updated AVHRR based SST fields are included and a new 85 GHz synthesis procedure for the defective channels of the SSM/I on DMSP F08 has been implemented.A great improvement over previously used algorithms is achieved for the retrieval of precipitation and wind speed with the neural network based algorithms.The combination of newly derived algorithms with previously published and validated retrievals result in high quality satellite derived atmospheric and ocean surface parameters.
Future tasks will involve a more detailed investigation of retrieval uncertainties and hence the specification of error estimates.Since the last radiometer of the SSM/I series was launched into space with DMSP F-15, a continuation of the HOAPS climatology will require the use of new sensors, preferably the Special Sensor Microwave Imager/Sounder (SSMIS).Corresponding activities regarding the intercalibration of passive microwave sensors are subject of the Global Space-based Inter-Calibration System (GSICS) by the WMO, and the GPM Intersatellite Calibration Working Group (X-CAL).
HOAPS-3 gridded data sets are freely available as pentad and monthly means and as twice daily multi satellite composite fields through the website http://www.hoaps.orgfrom the CERA data base (http://cera-www.dkrz.de).
The production of the HOAPS data set is currently transferred to the EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF) at the Deutscher Wetterdienst (DWD).Further releases will be generated at CM-SAF and available at http://www.cmsaf.eu/.

Figure 1 .
Figure 1.Flow chart for the data processing chain in HOAPS-3 from SSM/I antenna temperatures to geophysical products.

Figure 2 .
Figure 2. Local Equator crossing times of the DMSP satellites.

Figure 3 .
Figure 3. Distribution of 10 day mean TB differences in 0.1 K bins for the year 1995 of different SSM/Is relative to the reference radiometer on DMSP F11.The panels show the comparisons without (dashed lines) and with (solid lines) intercalibration for TB 19v (a), TB 19h (b), and TB 22v (c).

Figure 5 .
Figure 5. Distribution of 10 day mean TB differences in 0.1 K bins for the year 1998 of different SSM/Is relative to the reference radiometer on DMSP F11.The panels show the comparisons without (dashed lines) and with (solid lines) intercalibration for TB 19v (a), TB 19h (b), and TB 22v (c).

Figure 7 .Figure 8 .
Figure 7. SSM/I inter-sensor calibration as applied in the HOAPS processing chain, exemplarily shown for the time series of monthly global mean TB 19v of each SSM/I without (top) and with (bottom) inter-sensor calibration.

Figure 11 .
Figure11.Comparison of the HOAPS-3 neural network wind speed retrieval (black), the algorithms ofSchlüssel and Luthardt (1991) (red) andStogryn et al. (1994) (blue) against the verification data set of the neural network algorithm.The verification data set contains wind speed measurements from buoys and radiosonde measurements.The latter are binned in 2 m/s steps, resulting in discrete intervals for higher wind speeds.

Figure 12 .
Figure 12.Neural network architecture of the precipitation retrieval algorithm.

Figure 13 .
Figure 13.Comparison of different precipitation algorithms.(a) shows the retrieved precipitation rates of the neural network algorithm (black), the Bauer and Schlüssel (1993) algorithm (red) and the Ferraro (1997) algorithm (blue) against the precipitation rates from the verification data set.(b) shows the accumulated histogram of precipitation as the fraction of total precipitation for the different retrievals and the verification data set between 0.2 mm/h and 30 mm/h.

Figure 14 .
Figure 14.Climatological mean field (left) and zonal mean annual cycle (right) of HOAPS-3 evaporation for the years 1988 to 2005.

Figure 15 .
Figure 15.Climatological mean field (left) and zonal mean annual cycle (right) of HOAPS-3 precipitation for the years 1988 to 2005.

Figure 16 .
Figure 16.Climatological mean field (left) and zonal mean annual cycle (right) of the HOAPS-3 freshwater flux for the years 1988 to 2005.

Table 3 .
Global mean error values for parameters calculated with synthesized brightness temperatures.
under clear and cloudy conditions using the TB 22v , TB 37v/h , and TB 85v/h channels.

Table 4 .
Accuracies of the neural net algorithm for precipitation.N = number of samples; mean, bias, and RMS are in mm/h (see text).