1.

Introduction

Data fusion is the process by which synergy is identified and utilized between two unique data sources.¹ In remote sensing, data fusion has been used to construct infrared (IR) imager channels from IR sounder observations to maintain continuity,² to construct true color imagery when the green color channel is not available,³ and to produce high spatial and temporal resolution surface reflectance products.⁴

The synergy utilized here is the complementing vertical, horizontal, and temporal resolutions of IR imagers and the high spectral resolution (or hyperspectral) sounders (Table 1). IR sounder/imager fusion has been shown to be capable of constructing radiances with sounder-like high spectral resolution at geostationary imager-like high temporal and horizontal resolution.⁵ Measured radiances with high spectral resolution enable high vertical resolution in retrieved temperature and moisture profiles. When the fusion technique is applied directly to the hyperspectral sounder retrievals rather than radiances, sounder/imager fusion allows for construction of the aforementioned high vertical resolution, polar-orbiting sounder-like retrievals at geostationary imager temporal and horizontal resolution.⁶ These fusion products have the vertical resolution of an IR sounder ($\sim 1\mathrm{km}$) and the horizontal and temporal resolutions of an IR imager (2 to 4 km and as high as 1 min, respectively). As such, sounder/imager fusion offers improvements over both sounder and imager data alone.

Table 1

Summary of CrIS and ABI attributes. Adapted from Ref. 6. CrIS has a much higher number of channels than ABI, while ABI has a much smaller FOV size at nadir.

Sounder/imager fusion as applied to radiances has been validated against legacy channels available on Moderate Resolution Imaging Spectroradiometer,⁵ leading to the supposition that sounder/imager fusion as applied to retrievals may contain accurate and useful information. This product fusion has been demonstrated with sounder/imager pairs on the same polar-orbiting platform and with polar-orbiting sounders paired with geostationary imagers.⁶ The latter pairing offers the opportunity to add temporal fusion to spatial fusion. In this work, we provide statistical validation of sounder/imager retrieval product spatial and temporal fusion results using radiosonde data provided by the Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) facility⁷ and model data from the National Oceanic and Atmospheric Association (NOAA) National Centers for Environmental Prediction (NCEP) Rapid Refresh (RAP) model.⁸

2.

Instruments

A typical high spectral resolution (or hyperspectral) IR sounder measures the radiance emitted from the Earth system with thousands of very narrow and well-calibrated channels; e.g., the Atmospheric Infrared Sounder (AIRS),⁹ Infrared Atmospheric Sounding Interferometer,¹⁰ and Crosstrack Infrared Sounder (CrIS).¹¹ An example of a high spectral resolution brightness temperature spectrum is shown in Fig. 1. The fine spectral resolution allows for the retrieval of accurate and detailed vertical temperature and moisture profiles, trace gas concentrations, and cloud and surface parameters.^9,12–14 Broad-band sounders and imagers, on the other hand, are not capable of capturing this level of detail in vertical structure because their sensitivity to spectral differences in IR radiation is much lower, as illustrated by the example of the broad spectral response functions (SRFs) in Fig. 1. In this work, high spectral resolution retrieval products are derived using the University of Wisconsin Dual Regression (UW DR) retrieval algorithm.^15,16 Other retrieval algorithms proficient in processing high spectral resolution measurements can be utilized as well, e.g., the operational NOAA Unique Combined Atmospheric Processing System (NUCAPS).¹⁷ There are many types of high spectral resolution retrieval methodologies. Whereas NUCAPS includes microwave and IR data and an optimal estimation step, which improves the retrieval yield and sounding accuracy under certain conditions, the UW DR algorithm is based on linear regression with non-linearities addressed through classified statistics and various decision steps. In addition, the UW DR algorithm provides atmospheric profiles and surface and cloud parameters under clear and cloudy sky conditions at single field of view (FOV) horizontal resolution. The main differences between these two hyperspectral retrieval systems are described in Ref. 18. In general, hyperspectral sounder algorithms are capable of retrieving vertical temperature and humidity profiles with an accuracy of 1 K and 20% humidity, respectively, within a 1-km layer.^17,19,20 The information from these high spectral radiance measurements and the associated accurate vertical profiles are beneficial to forecasting,^12,21 model assimilation,^22,23 and research applications.¹³

Fig. 1

Example CrIS observed brightness temperature spectrum (blue, left axis) and ABI IR channel SRFs (red, right axis) plotted versus wavenumber.

Many sounders, such as CrIS aboard Suomi-National Polar-orbiting Partnership (S-NPP) and the Joint Polar Satellite System (JPSS) series, are in polar orbit.¹¹ This means that, while their radiance data are available for the entire globe, most notably in regions where in-situ observations are sparse, its $\sim 12\mathrm{h}$ temporal resolution is low (Table 1). Additionally, sounders have a large FOV size, typically $\sim 14\mathrm{km}$. This limits the ability of sounders to provide the type of time evolution or high horizontal resolution information often needed in nowcasting settings.

IR imagers have a much lower number of channels compared with sounders. These channels are used to provide information about the earth surface and atmosphere. A typical imager has roughly 10-IR channels, with each channel often corresponding to a particular characteristic such as mid-level water vapor or ozone.²⁴ The low spectral resolution, however, limits imagers ability to create high vertical resolution profiles of atmospheric characteristics. This low spectral resolution is illustrated by the SRFs of 10 broad IR bands plotted in Fig. 1.

In contrast to spectral or vertical resolution, imagers typically have very high horizontal resolution (Table 1)—sometimes on the order of hundreds of meters.^25,26 This allows for the depiction of spatially small features in the atmosphere such as overshooting tops in a developing convective system.²⁷ Imagers, such as the Advanced Baseline Imager (ABI), are often aboard geostationary satellites. While these platforms are limited to a single viewing angle and cannot view the high latitudes, they are able to provide much higher temporal resolution, sometimes as high as 1 min.²⁴ This makes imagers onboard geostationary platforms ideal for providing the high temporal and spatial resolution information used in nowcasting settings, albeit without highly resolved vertical information.

3.

Fusion Process Methods and Results

The fusion process described herein has been utilized to construct both radiances⁵ and retrievals.⁶ As described in Refs. 5 and 6, a multi-dimensional (k-d) tree search algorithm applied to imager radiances and geolocation is used to find a number ($N$) of best matching sounder FOVs for each imager pixel. The sounder and imager data need not be concurrent, though near concurrent is favored. In radiance fusion, the radiances of these $N$ closest matching FOVs are averaged for each pixel to construct fusion radiances. In retrieval product fusion, as utilized here, the retrievals corresponding to the $N$ closest matching sounder FOVs are averaged at every pressure level to construct retrievals corresponding to each imager pixel. The resulting fusion retrievals show the characteristics of the atmosphere at the vertical resolution of the sounder and at the horizontal resolution and observation time of the imager data. All plots and illustrations shown in this section were created using CrIS granules starting at 1748 UTC and 1754 UTC on 19 May 2017 over the Chesapeake Bay, Virginia, region and the Geostationary Operational Environmental Satellite (GOES)-16 ABI images taken from 1800 UTC to 2100 UTC on the same day.

3.1.

Retrieval Product Fusion

In both radiance fusion and retrieval product fusion, a k-d tree search algorithm is utilized to find matches among the vectors of a training dataset to vectors within a query dataset. The k-d tree is a nearest neighbor search that is computationally fast and efficient and can be applied to spaces of arbitrary dimensionality.²⁸

Imager pixel geolocation and radiance data form the query dataset. The imager pixels are geographically collocated to the sounder FOVs and the radiances are averaged within each FOV to form the training dataset. In other words, high and low spatial resolution imager data form the query and training datasets, respectively. The k-d tree search is then used to match each imager pixel from the query dataset with a set of $N$ best matching sounder FOVs represented by the low spatial resolution imager data in the training dataset. This first step is identical between both radiance and product fusion, and in both cases, hyperspectral sounder data (radiance or retrievals) are not yet used. A cloud mask can be applied to the imager data prior to this process; this saves computing time by eliminating pixels from the query dataset that are known to be cloudy. In this case study, a cloud mask has been applied to the ABI data.²⁹

In the second step, the sounder data from the $N$ best matching FOVs for each pixel are averaged. Here, in retrieval product fusion, it is the sounder retrievals that are averaged. A minimum number of non-cloudy neighbors out of $N$ for each pixel at a given pressure level is set, below which the pixel is assigned as cloudy at the pressure level in question. The result is a high vertical resolution retrieval profile for every clear sky imager pixel. These new data retain the high vertical resolution of the sounder derived retrieval along with the high horizontal resolution of the imager.

The combination of parameters taken to be optimized for this application is a best match subset of $N=5$, a minimum clear best match requirement of minimum $N_{\mathrm{clr}}=2$ and the ABI input channel combination of 8, 9, 10, 11, 13, 14, 15, and 16 (corresponding to wavelengths at 6.2, 7, 7.3, 8.5, 10.3, 11.2, 12.3, and $13.3\mu \mathrm{m}$).²⁴ In this section, a summary of the resultant Chesapeake Bay, Virginia, 19 May 2017 at 1800 UTC retrieval product fusion dataset is presented in comparison with the original ABI and CrIS data from the same time. The data here illustrates the synergy of the imager’s high horizontal resolution and the sounder’s high vertical resolution.

The imager inputs into the fusion process are shown in Fig. 2. The high horizontal resolution of the imager data is immediately evident. The horizontal resolution is 2 km at nadir and 3 to 4 km over the region shown due to the higher viewing zenith angle. The goal is to retain this high horizontal resolution in the final result. While these image inputs do not contain highly resolved vertical information, each channel does offer information content with respect to a particular atmospheric characteristic on which the k-d tree search process can train. For instance, inclusion of ABI channel 9, known as the mid-level water vapor channel,²⁴ allows the k-d tree search to be guided by mid-level water vapor when matching imager pixels with sounder FOVs. The sounder inputs (Fig. 3) contain highly resolved vertical information about the atmosphere but do not have the high horizontal resolution of the imager. The CrIS products used here were retrieved using the full spectral resolution (FSR) data¹⁹ (2211 channels). Other combinations of imager/sounder instrument pairs and retrieval methods capable of processing hyperspectral sounder radiances may be used in a similar fusion framework described here, but they are not presented herein. As discussed throughout this work, the intent of this fusion process is to capture the high horizontal resolution of the imager and the high vertical resolution of the sounder.

Fig. 2

ABI brightness temperatures from channels (a) 8; (b) 9; (c) 10; (d) 11; (e) 13; (f) 14; (g) 15; and (h) 16, taken aboard GOES-16 over the Chesapeake Bay region at 1800 UTC on 19 May 2017. These images serve as the imager input into the fusion process example shown here.

Fig. 3

CrIS derived temperature retrievals at (a) 850 hPa; (b) 500 hPa; and (c) 300 hPa and water vapor mixing ratio retrievals at (d) 850 hPa; (e) 500 hPa; and (f) 300 hPa, over the Chesapeake Bay region at 1754 UTC on 19 May 2017 created using the UW DR retrieval algorithm.^15,16 These data serve as the sounder input into the fusion process example shown here.

Retrieval product fusion using the $N=5$, minimum clear $N_{\mathrm{clr}}=2$, and eight-channel search are shown in Fig. 4. The data do exhibit the high horizontal resolution of the ABI imager as intended. Additionally, the products are “sounder-like” in that they have vertical information of temperature and water vapor. In other words, the spatial fusion of ABI/CrIS increases the horizontal resolution of the soundings by roughly an order of magnitude. This work assesses the accuracy of this retrieval product fusion.

Fig. 4

Retrieval product fusion temperature retrievals at (a) 850 hPa, (b) 500 hPa, and (c) 300 hPa and water vapor mixing ratio retrievals at (d) 850 hPa, (e) 500 hPa, and (f) 300 hPa, over the Chesapeake Bay region at 1754 UTC on 19 May 2017 created using ABI and the UW DR retrieval algorithm^15,16 applied to CrIS radiances.

3.2.

Retrieval Product Fusion Time Series

The high temporal resolution of the ABI imager allows for the creation of time series of retrieval product fusion data. GOES ABI images are available at a refresh rate of 1 min inside one of two mesoscale regions over the contiguous United States (CONUS), 5 min over CONUS, and 15 min over the entire disk.²⁴ This temporal fusion begins at “time step 0” with retrieval product fusion as described in Sec. 3.1 between sounder data and imager data that are very near concurrent in time, i.e., the ABI data is that which is closest in time to the CrIS overpass. “Time step 1” occurs when the next ABI image is available for the region in question. At time step 1, a new retrieval product fusion is created as follows. The k-d tree is used to match each imager pixel at time step 1 with the $N$ best matching imager pixels from time step 0. This means that every pixel in the new imager data from time step 1 is matched with a subset of the best match (in radiance and geolocation) imager pixels from time step 0. The corresponding $N$ fusion retrievals from time step 0 (already at high spatial resolution) are then averaged to form the new fusion retrieval for that pixel at time step 1. This process can be repeated as new ABI data continuously becomes available at subsequent time steps with each new imager pixel being matched with $N$ pixels from the previous image, the high spatial resolution retrievals of which are then averaged to form the retrieval product fusion at the new time step. As before, the best match subset is taken to be of size $N=5$, and the minimum number of clear neighbors is taken to be minimum clear $N_{\mathrm{clr}}=2$.

Figures 5 and 6 show an example time series created by utilizing the spatial/temporal fusion approach, i.e., completing the time step 0 spatial fusion at 1800 UTC and using the temporal fusion procedure to create retrieval product fusion at 2000 UTC and 2200 UTC. The retrieval product fusion constructs a more coherent depiction of the atmospheric changes at the overpass time and over the following 4 h. A detailed description of the atmospheric processes observed in the retrieval product fusion time series of this case study, including a comparison with ABI and NCEP RAP model data, can be found in Sec. 3.3 of Ref. 6. After the 4-h period shown in Fig. 5, the time series no longer tracks the convective development cycle very well. More work is necessary to determine for how long after the sounder overpass temporal fusion remains useful. Overall, this case study offered interesting results and speaks to the potential for spatial and temporal product fusion in nowcasting settings. The near-real time retrieval system used operationally in this scenario is NUCAPS. Spatial/temporal product fusion using ABI radiance data can be applied to NUCAPS to enhance spatial resolution from $\sim 45\mathrm{km}$ to 2 to 4 km and temporal resolution from twice daily to every 15 min to further improve nowcasting and possibly forecasting operations.

Fig. 5

Fusion temperature and at (a)–(c) 850 hPa; (d)–(f) 500 hPa; and (g)–(i) 300 hPa at (a), (d), and (g) 1800 UTC; (b), (e), and (h) 2000 UTC; and (c), (f), and (i) 2200 UTC over Chesapeake Bay using the ABI image from 1800 UTC and the CrIS data from the 1748 UTC and 1754 UTC granules on 19 May 2017. White signifies the presence of cloud. Features in the atmosphere, especially convective lift shown by the 500 hPa temperature, can be followed over time.

Fig. 6

Water vapor mixing ratio at (a)–(c) 850 hPa; (d)–(f) 500 hPa; and (g)–(i) 300 hPa at (a), (d), and (g) 1800 UTC; (b), (e), and (h) 2000 UTC; and (c), (f), and (i) 2200 UTC over Chesapeake Bay using the ABI image from 1800 UTC and the CrIS data from the 1748 UTC and 1754 UTC granules on 19 May 2017. White signifies the presence of cloud. Features in the atmosphere, especially convective lift shown by the 500-hPa water vapor mixing ratio, can be followed over time.

4.

Validation

Radiosondes are commonly used as ground truth reference for validating satellite data products.^30,31 DOE ARM Southern Great Plains (SGP) radiosondes were launched to be concurrent with the operational CrIS overpasses;³² initially S-NPP was designated as the operational spacecraft and later NOAA-20 (designated JPSS-1 prior to launch) was. These radiosonde data were used in this work to develop mean and standard deviation error profiles for CrIS/ABI product fusion. S-NPP precedes NOAA-20 by roughly 50 min (1/2 orbit); thus temporal fusion retrievals with time step 0 at S-NPP overpass time and final time step at NOAA-20 overpass time can also be validated. The RAP model and CrIS UW DR error statistic profiles are added to provide additional context.

4.2.

Retrieval Product Fusion Dataset

To create a statistical comparison between retrieval product fusion and the radiosonde dataset, a retrieval product fusion profile was created for every radiosonde profile included in the dataset. The method by which the retrieval product fusion profile was created depends on whether the sounder overpass and fusion time are concurrent or the fusion is created via the time-series method described in Sec. 3.2.

For validation when the sounder, fusion, and radiosonde are concurrent, retrieval product fusion was completed using the CrIS granules from the overpass targeted by the radiosonde launch and the GOES-16 ABI image from nearest in time to the radiosonde launch. The CrIS granules always include the ARM SGP radiosonde launch site. Fusion was completed for the $\sim 30$ ABI pixels that are closest to the radiosonde launch site. The fusion profile from the pixel with the cloud-free profile reaching the lowest into the troposphere was chosen for validation. This fusion profile is then compared with the radiosonde to compute the statistical comparisons shown in Figs. 7 and 8.

Fig. 7

(a) Mean (solid) and standard deviation (dashed) of temperature differences versus pressure level between sonde and fusion (black), model and fusion (blue) and sounder and sonde (orange) from February 2017 to February 2018 over the DOE ARM SGP site. (b) Number of clear samples versus pressure level for the period in question. Temperature bias in the mid troposphere is $\sim -1.5\mathrm{K}$ with a standard deviation of $\sim 2\mathrm{K}$.

Fig. 8

(a) Mean (solid) and standard deviation (dashed) of relative humidity differences versus pressure level between sonde and fusion (blue) and model and fusion (orange) from February 2017 to February 2018 over the DOE ARM SGP Site. (b) Number of clear samples versus pressure level for the period in question. Relative humidity bias in the upper tropopause is $\sim 0$ with a $\sim -10\%$ bias jump at $\sim 750\mathrm{hPa}$. The standard deviation starts at 25% at the surface but decreases to 15% through the troposphere.

For validation of fusion created via the time-series method described in Sec. 3.2, time step 0 was taken to be fusion between the CrIS granules from S-NPP and the GOES-16 ABI data from nearest in time to the S-NPP overpass. Time-series fusion was then created as described in Sec. 3.2 forward in time to the ABI image closest to the corresponding NOAA-20 overpass. The NOAA-20 overpass is $\sim 50\min$ after the S-NPP overpass. The radiosonde launched $\sim 5\min$ before the NOAA-20 overpass was used for validation. The fusion profile at this later time is then compared with the radiosonde to compute statistical comparisons in Figs. 9 and 10 that represent validation of $\sim 50\min$ time-series fusion.

Fig. 9

(a) Mean (solid) and standard deviation (dashed) of temperature differences versus pressure level between sonde and time-series fusion using S-NPP CrIS (black), model and fusion (blue) and sounder and NOAA-20/JPSS-1 CrIS (orange) from February 2018 to March 2019 over the DOE ARM SGP site. (b) Number of clear samples versus pressure level for the period in question. Temperature bias in the mid- to upper-troposphere is $\sim -2\mathrm{K}$ to $\sim -3\mathrm{K}$ with a standard deviation of $\sim 1\mathrm{K}$ to $\sim 3\mathrm{K}$.

Fig. 10

(a) Mean (solid) and standard deviation (dashed) of relative humidity differences versus pressure level between sonde and time-series fusion using S-NPP CrIS (black), model and fusion (blue) and sounder and NOAA-20/JPSS-1 CrIS (orange) from February 2018 to March 2019 over the DOE ARM SGP site. (b) Number of clear samples versus pressure level for the period in question. Relative humidity bias in the tropopause oscillates between 10% and $-10\%$ with a standard deviation decreasing from 25% at the surface to 20% through the upper troposphere. Microwave radiometer-based water vapor scaling was not available for these radiosonde samples.

5.

Summary and Conclusion

Independently, modern IR imagers and IR sounders aboard Earth-orbiting satellites exhibit unique advantages that make them useful and important tools to today’s weather and climate monitoring operations. By identifying and utilizing an opportunity for synergy between the two types of instruments, the data fusion method demonstrated here produces results exhibiting the advantages of both while shedding some of their weaknesses. This fusion method has been previously proven to produce sounder-like, narrow band radiances at imager-like horizontal resolution to supplement existing imager bands.⁵ Here, we build upon a previous demonstration of spatial/temporal retrieval product fusion⁶ by further tuning this method to produce high vertical resolution retrieval products at high imager horizontal and temporal resolution. Validation is also presented through comparison of fusion retrievals to radiosonde profiles.

The fusion method utilizes a k-d tree search²⁸ to find a subset of sounder FOVs that best matches each imager pixel in imager radiance and geolocation space. Hyperspectral sounder profile retrievals corresponding to the subset of FOVs are averaged at every pressure level to construct the fusion retrieval profile. In the case of ABI/CrIS fusion, the resulting retrieval product has an ABI horizontal resolution of $\sim 2$ to 4 km while maintaining the CrIS high vertical resolution. Once this high spatial resolution data is available (at the CrIS measurement time), temporal fusion can be performed as new ABI images become available. In this process, the k-d tree search is applied to imager radiance data from consecutive time steps. This yields fusion retrieval results with high temporal resolution. By constructing a time series of hourly available fusion products, it is now possible to observe a storm system as it moves into pre-convective environments and watch in high resolution as convection moves warmer air aloft. These data in particular provide a glimpse of the utility that a geostationary hyperspectral sounder would provide to the meteorology community.

We utilized the radiosonde launches associated with the CrIS validation campaign at the DOE ARM SGP site⁷ to compare fusion results versus ground truth. Best estimate profiles similar to Ref. 31 and raw radiosonde profiles were used to study the most accurate quantifications of temperature and water vapor profiles possible. When compared with these best estimate profiles, fusion performs well. The nearest CrIS retrieval FOV was also compared with the radiosonde profile in each case and shows similar biases to the fusion results. This means that the fusion process improves the $\sim 14\text{-}\mathrm{km}$ horizontal resolution of the CrIS sounder retrieval to $\sim 2$ to 4 km with similar accuracy when compared with radiosondes.

Data fusion between instrument pairs, as demonstrated here, is a useful tool that can be applied to instrument groups exhibiting synergy to provide observations with characteristics not possible with individual satellites. Particularly interesting is the opportunity for data fusion among smallsats and cubesats as they become more prevalent.