Comparison of Methods for IKONOS Images Pan-sharpening Using Synthetic Sensors

Many methods are present in literature for pan-sharpening of satellite images: they permit to transfer geometric resolution of panchromatic data to multispectral ones, but the results of their application are different. To evaluate the quality of these products, visual analysis is carried out, above all on the RGB composition to detect colour distortion. To quantize the level of similarity of the pan-sharpened images with them that should be achieved with effective more effective sensors, several indices are available such as: RMSE, correlation coefficients, UIQI, RASE. The principal limit of these indices consists in the terms of comparison because they compare the pansharpened images with the original ones that are with lower resolution. To supply the unavailability of the effective dataset with the same pixel dimensions of the pan-sharpened files, synthetic sensors can be introduced with lower resolution than the original ones. The correspondent degraded images can be submitted to pan-sharpening process and the results can be considered performed if similar to the original multispectral dataset. In this study IKONOS synthetic sensors are introduced to compare different methods: transforming the digital numbers into the radiance of the earth surface, original images of Campania Region are degraded and then submitted to some pan-sharpening approaches. The following methods are considered: multiplicative, simple mean, IHS, Fast IHS, Brovey, Weighted Brovey, Gram Schmidt, Zhang. Each resulting dataset is compared with the original multispectral one to evaluate the performance of each method.


INTRODUCTION
As time passes, data fusion techniques become objects of important researches in the field of remote sensing image processing and they are widely implemented.The consequence is a continuous improvement of these methods in the last years.Sunsynchronous satellite sensors acquire concurrently images with different spectral and geometric resolutions: panchromatic (Pan) data have smaller pixel dimensions, but greater band width compared to the corresponding multispectral (MS) ones.Pan-sharpening is a data fusion approach that allows to get over this limit (Maglione et al., 2014).The principal purpose is to improve multispectral data quality merging MS and Pan data, having both of them complementary spatial and spectral resolution (Chavez et al., 1991;Wang et al., 2005;Alparone et al., 2007).
Several methods are available in literature and each of them produces specific images that are not the same originated by another.For this reason, the resulting pansharpened multispectral data must be evaluated: if visual analysis does not permit to detect colour distortions, quantitative evaluation is necessary.
Numerous comparisons between methods have been published (Thomas and Wald, 2005).To establish the best method so to prefer it in every case is very hard: the evaluation to be effective requires the availability of valid terms of comparison such as multispectral images with the same resolution of the pan-sharpened ones.This condition is difficult to achieve, so, according to Parente and Santamaria (2014) who proposed this approach for Landsat 7 ETM+ imagery, synthetic sensors (SynSs) must be considered.
In this study, IKONOS imagery (including pan as well as multispectral data) are considered.An example of the proposed approach for this type of data is also present in Santurri et al. (2010), but the authors in this case introduce the synthetic sensor degrading the matrices of Digital Numbers (DNs) instead of those concerning spectral radiance.In fact in this study we propose to define SynSs products by two steps: calculation of at-sensor spectral (or TOA, Top of Atmosphere) radiance and degradation of pixel sizes (atmospheric corrections are not considered because synthetic and real sensors are operative in the same conditions).The new images are involved in the pansharpening methods and the resulting TOA radiance matrices are converted in DNs matrices: the obtained

DATA AND FIRST ELABORATIONS
Launched into space on September 24 1999, Ikonos is a part of GeoEye's constellation synchronous 681km orbit and both sensors have a swath width of 11 km.There are two types of sensors on board that acquire s in panchromatic band with 0.82 m resolution at nadir and in multispectral ones (Blue, Green, Red, Near Infrared) with 3.28 m resolution at nadir.Panchromatic and multispectral images, all of these with a dynamic range of 11 bit, present cell sizes respectively to 1 m and 4 m for commercial purposes (Digital Globe, 2013).
Ikonos imageries, because of their high level of spatial and spectral information, are useful for several studies: morphological configurations, urban environments (Herold et al., 2002), land cover anduse, forests, waters andother landscape et al., 2003).Map creation and updating (Belfiore and Parente, 2014) as well as the definition of the variation of shorelines in presence of coastal Giannini et al., 2011;Palazzo et al., 2012) are examples of practical implementations of (Table 1).
Fig. 1: The considered Ikonos scene and its location in Campania region and multispectral images, all of these with a dynamic range of 11 bit, present cell sizes down-resampled respectively to 1 m and 4 m for commercial purposes Ikonos imageries, because of their high level of spatial and spectral information, are useful for several studies: morphological configurations, urban , 2002), land cover and use, landscape elements (Gitas , 2003).Map creation and updating (Belfiore and , 2014) as well as the definition of the variation coastal erosion (Basile ., 2012) are examples Ikonos imageries.
For this application an Ikonos scene, (panchromatic and multispectral data) acquired on 10/10/2005 at 09:59 concerning the Campania region, is used: it covers the municipalities of Bellona, Caiazzo, Capua, Caserta, Castel di Sasso, Castel Morrone, Piana di Monte Verna and Ponte Latone (Fig. 1).Conversion of DNs to radiance: mentioned, a transformation from DNsin radiance values is required because they indicate how much of the power emitted by an emitting or reflecting surface will be received by an optical system looking at the surface from some angle of view (Borengasser 2008).
The following formula makes the transformation possible for the IKONOS images so to obtain at spectral radiance (TOA): where, L λ = Radiance for spectral band λ at the sensor's aperture (W/m 2 DN λ = The digital number of the pixel of the image The procedure is applied both to panchromatic and multispectral data: Taylor (2005) has deter thevalues of CalCoef λ and Bandwidth Ikonos scenes acquired after 22/02/2001

Reduction (downgrading) of geometric resolutions:
The images, which are matrices of radiance, are down sampled respectevely from 4 m to 16 m (for multispectral bands) and from 1 m to 4 m (for panchromatic band).In this way it is possible to emulate images acquirement with the same spectral bands of the original

Conversion of DNs to radiance: As previously
DNsin radiance values is required because they indicate how much of the power emitted by an emitting or reflecting surface will be received by an optical system looking at the surface from some angle of view (Borengasser et al., the transformation possible for the IKONOS images so to obtain at-sensor (1) Radiance for spectral band λ at the 2 /µm/sr) The digital number of the pixel of the

The radiometric calibration coefficient
The procedure is applied both to panchromatic and multispectral data: Taylor (2005)

Reduction (downgrading) of geometric resolutions:
The images, which are matrices of radiance, are downsampled respectevely from 4 m to 16 m (for bands) and from 1 m to 4 m (for In this way it is possible to SynS, having original scenes but  (Crippen, 1987).

Simple mean:
It is based on a simple mean between the panchromatic and each multispectral image (ESRI, 2012).The formula to compute the pan image is: sharpening methods are applied to merge panchromatic and multispectral data: They are described in the following subsections.sharpened images are obtained by simple multiplications of each multispectral band for the panchromatic image using the following formula: (3) sharpened ith image The mean of the pixel values of the (2009) highlight the straightforwardness and simpleness of this algorithm.It contains spectral distorsions (Crippen, 1987).
It is based on a simple mean between the panchromatic and each multispectral image (ESRI, 2012).The formula to compute the pan-sharpened where, δ Pan − I (6) To calculate the I component all multispectral IKONOS bands are used: Fast IHS: This method works in the same way of the IHS, but Intensity (I) is computed using weights.Tu et al. (2004) propose to use values of 0.75 and 0.25 for the green and the blue band, respectively: the weighting parameters are tested by the authors on 92 different IKONOS imageries.Similar numbers are obtained if the same approach proposed by Parente and Santamaria (2013) for Quickbird images and based on spectral response analysis is adopted for IKONOS ones; because NIR and Red are fully included in the panchromatic response, their weights are equal to the unit: Brovey: Described both by Pohl and van Genderen (1998), it is a pan-sharpening algorithm that preserves the relative spectral contributions of each pixel, but replaces its overall brightness with the high-resolution PAN image (Kalpoma and Kudoh, 2007).Each pansharpened image is obtained by n multispectral bands using the formula: Weighted brovey: As well as for FAST IHS, also for the Weighted Brovey method the same specific weights are introduced: Zhang method: Introduced by Zhang (2004) and implemented in Geomatica 2015 by PCI Geomatics, this approach, statistics-based fusion, uses the least squares technique to correct grey values of the fused images according to those of the original Pan and MS images.It utilizes a set of statistic approaches to estimate the grey value relationship between all the input PAN and MS bands to automate the fusion process.Zhang (2002) affirms that his algorithm solves two major problems in image fusion: colour distortion and operator dependency.
Gram schmidt: Implemented by Laben and Brower (2000), the Gram Schmidt (GS) method is one of the most popular pan-sharpening technique: it maximizes image sharpness and minimize radiometric distortion better than other algorithms (Maurer, 2003).Averaging the multispectral bands it is possible to simulate a panchromatic band from the lower spatial resolution spectral bands.The second step consists in a Gram Schmidt transformation where both the simulated panchromatic data, employed as the first band and the multispectral bands are used.The first Gram Schmidt band is replaced by the high spatial resolution panchromatic band.In the last step, an inverse Gram Schmidt transformation is performed to create the pansharpened multispectral bands.

RESULTS
Eight fusion methods are tested on the considered down-sampled dataset: it covers different kinds of areas and its multispectral size is 500 by 500 pixels.In this study 32 pan-sharpened images are produced: their RGB composition, that includes necessary only 3 of the 4 multispectral images (NIR is not considered in the visualization), are shown in Fig. 5. Quantitative analysis is performed comparing the values of the following four quality indexes: Root Mean Square Error (RMSE): Debated by Munechika et al. (1993), it calculates the difference of the standard deviation and the mean of the pansharpened and the original image.It estimates the spectral and spatial quality of the high resolution multispectral images (Rajendran and Vaithiyanathan, 2012).A value closer to 0 indicates a good accuracy: where, BIAS xy and σ xy are respectively the difference between the mean and standard deviation values of the input multispectral image (x) and the output SynS image (y).

Correlation coefficient (ρ):
Its value range varies from -1 to 1.A high correlation value means a good correspondence between the original bands and the corresponding fused ones.The formula is:  Relative Average Spectral Error (RASE): Discussed by Wald (2000) its value is expressed in percent and shows the average performance of a pan sharpened method considering all spectral bands: where, µ = The mean value of DNs of the n input images.
Considering indices values for all the fusion algorithms, which are reported in Table 3 to 9, it is possible to have an overview of the final results.(Table 10).
Even if the indices do not generate the same classification, it is evident that Fast IHS, GS, Zhang and Weighted Brovey produce the best performance.Multiplicative and Simple Mean methods are not reliable as confirmed by the results of low quality.The benefits of the introduction of weights are evident also for IHS and Brovey.Zhang is another valid pansharpening approach: adding a histogram matching among the fused images and the corresponding original ones, it improves, forcing a little, the final outcome.

CONCLUSION
The introduction of synthetic sensors to produce degraded images permits to apply to them different pansharpening methods determining if they are reliable or not.The conversion from DNs to at-sensor spectral radiance is needed: In this study it is supposed that the new sensors intercepts the same energy of the real sensor, but with different geometric resolutions.The equation to convert DNs into radiance for IKONOS images is present in literature and easy to apply.
The different pan-sharpening methods tested on an IKONOS multispectral scene acquired over Campania region in Italy present different results: not all of them have shown a good outcome.In literature problems and limitations of each pan-sharpening method aredescribed by several studies (Wald et al., 1997;Du et al., 2007).This study suggests an approach to support performance evaluation.It is proved that, giving weights to each multispectral band to differentiate their rule in pan-sharpening process, possibility to achieve good results increases a lot.
Considering that this approach classifies performances of each method, it can be used to test new data fusion algorithms and procedures.In this study pixel dimensions are quadrupled, but also other transformations of cell sizes can be considered; in this manner is possible to determine the performance of each method in reference to increasing levels of spatial resolution.

Fig. 2 :
Fig. 2: Comparison between the original panchromatic band (left) and the degraded ones (right)

Fig. 4 :
Fig. 4: RGB composition of the multispectral degraded area lower geometric resolutions.During the re blocks of 16 pixels are replaced by one cell: its radiance value is derived by the average of the corresponding group.In Fig. 2 and 3 two details of the considered area are shown: the first relative to the pan images while the second concerning the multispectral data in RGB composition.Calculation of new DNs values:stated previously formula permits to determine matrices of DNs from the re-sampled matrices of radiance:˖˚ { { { #"Using a synthetic sensor, new scenes are obtained: they have identical position and the same acquisition time of IKONOS, but worse geometric resolutionsPAN-SHARPENING METHODSTo apply pan-sharpening techniques, a synthetic scenesis selected.The considered area (Fig.4) extends 8000 m × 8000 m (UTM/WGS84 plane coordinates -33T zone: E 1 = 422640 m; N m; E 2 = 430640 m; N 2 = 4536772 m different elements like vegetation, building Eight pan-sharpening methods are applied to merge panchromatic and multispectral data: They are described in the following subsections Multiplicative: The pan-sharpened images are by simple multiplications of each multispectral band for the panchromatic image using the following formula: C 6 µ 9@ multispectral degraded area geometric resolutions.During the re-sampling, blocks of 16 pixels are replaced by one cell: its radiance value is derived by the average of the corresponding group.In Fig. 2 and 3 two details of the considered area shown: the first relative to the pan images while the second concerning the multispectral data in RGB Calculation of new DNs values: The inverse of the stated previously formula permits to determine matrices matrices of radiance: {{ (2) a synthetic sensor, new scenes are obtained: they have identical position and the same acquisition time of IKONOS, but worse geometric resolutions.SHARPENING METHODS sharpening techniques, a clip of the synthetic scenesis selected.The considered area (Fig. 4) 8000 m (UTM/WGS84 plane = 422640 m; N 1 = 4544772 = 4536772 m): It contains different elements like vegetation, building sand roads.
Introduced byCarper et al. (1990), it is based on RGB to Intensity-Hue-Saturation (IHS) space transformation.The panchromatic image replaces the intensity component because of their similarity: therefore the inverse transformation is applied.R' G' B' components, having the same geometric resolution of panchromatic data, are determined using the succeeding equation:

Fig. 5 :
Fig. 5: RGB composition of the images derived by the following methods; (a): Brovey Brovey; (c): HIS; (d): FAST HIS; (e) (f): Multiplicative; (g): Simple Mean ρ ˲˳ where, x = The input multispectral image y = The output pan sharpened image (SynS) Cov xy= The covariance between x and y images σ x and σ y = The standard deviation of x and y images Universal Image Quality Index (UIQI):Wang and Bovik (2002), it evaluates the performance of the pan sharpened methods considering three factors such as loss of correlation, luminance and contrast distortion.It is obtained as: ˡH˝H

Table 1 :
Band range of Ikonos images Fig.1: The considered Ikonos scene and its

Table 2 :
Adopted values for the conversion from DNs to radiance

Table 2 :
Adopted values for the conversion from DNs to radiance

Table 3 :
Values of quality index for HIS