THE OPTIMIZED BLOCK-REGRESSION-BASED FUSION ALGORITHM FOR PAN-SHARPENING OF VERY HIGH RESOLUTION SATELLITE IMAGERY

Pan-sharpening of very high resolution remotely sensed imagery need enhancing spatial details while preserving spectral characteristics, and adjusting the sharpened results to realize the different emphases between the two abilities. In order to meet the requirements, this paper is aimed at providing an innovative solution. The block-regression-based algorithm (BR), which was previously presented for fusion of SAR and optical imagery, is firstly applied to sharpen the very high resolution satellite imagery, and the important parameter for adjustment of fusion result, i.e., block size, is optimized according to the two experiments for Worldview-2 and QuickBird datasets in which the optimal block size is selected through the quantitative comparison of the fusion results of different block sizes. Compared to five fusion algorithms (i.e., PC, CN, AWT, Ehlers, BDF) in fusion effects by means of quantitative analysis, BR is reliable for different data sources and can maximize enhancement of spatial details at the expense of a minimum spectral distortion. * Corresponding author


INTRODUCTION
Pan-sharpening has a specific interest, i.e., the lower resolution multispectral image's spatial details are enhanced by adopting the higher resolution panchromatic image corresponding to the multispectral image.The key issues concentrate on a maximum enhancement of its spatial details at the expense of a minimum multispectral distortion for the multispectral images.Recent advances for pan-sharpening are briefly reviewed in the literature (Yang et al., 2014;Yang et al., 2010).Some already published studies show that some pan-sharpening algorithms, to some extent, decrease their ability of enhancement of spatial details in order to keep the spectral features in the sharpened image highly consistent with those in multispectral image, especially for very high resolution images (Alparone et al., 2007;Dahiya et al., 2013;Ghosh and Joshi, 2013;Nikolakopoulos, 2008;Witharana et al., 2013;Yuhendra et al., 2012).In other words, it is a trade-off between maintenance of spectral characteristics and enhancement of spatial details for some pan-sharpening algorithms.
In order to address the problem, some researchers presented pan-sharpening methods with the capacity of adjustment (Chen et al., 2013;Choi, 2006;Fasbender et al., 2008;Möller et al., 2012;Te-Ming et al., 2012;Yee et al., 2014).Fusion results can be adapted by adjusting specific parameters to realize the different emphases between preservation of color characteristics and enhancement of spatial details.In order to achieve the adjustment, some of the existing algorithms need amendment of multiple parameters (Fasbender et al., 2008;Möller et al., 2012;Te-Ming et al., 2012).It is difficult to apply these algorithms in actual engineering projects.A block-regression-based algorithm (BR) (Zhang et al., 2010) can achieve the different emphases by tuning one parameter (i.e., block size) and is workable for applications.According to the literature (Zhang et al., 2010), the difference in block size acting as an important parameter in BR can lead to differences in spatial detail extracted from the higher resolution image, and further lead to different fusion effects (Yang and Zhang, 2014).
In existing works, experiments and analysis based on BR primarily concentrate on fusion of SAR and optical imagery (Zhang et al., 2010) and the desirable ability of adjustment provided by BR is not applied to sharpen the very high resolution satellite imagery.In this paper, eight different configurations of the important parameter, i.e., block sizes, are tested.The optimal block size, which can achieve a satisfying trade-off between preservation of spectral characteristic and enhancement of spatial details, is determined by assessment of fusion quality.The optimal selection is followed by the comparative analysis with other algorithms' results.

THE BLOCK-REGRESSION-BASED (BR) FUSION ALGORITHM
BR (Zhang et al., 2010) algorithm adopts block-based processing.In order to generate fusion results, the algorithm derives a synthetic block as a linear function of blocks of multispectral bands that has the maximum correlation with the corresponding block of the panchromatic band for every block of images.The maximum correlation with the block of the panchromatic image results in the maximum enhancement of the spatial details at the expense of the minimum spectral distortion derived from the fusion operations.The blockregression fusion algorithm can be expressed by Equation 1.
In Equation 1, , , and , , are the pixel values before and after fusion, respectively, pan (i, j) is the pixel value of the panchromatic image, k is the band number, (i, j) represents the pixel location, and c k is the linear regression coefficient in the multiple linear regression of the block region containing the pixel located at (i, j).
In order to eliminate the mutation of fusion effects in the connected regions between neighboring blocks, the panchromatic and multispectral image data used for multiple linear regression requires to be extended to its neighboring blocks.For example, it expands a block size on each direction, i.e., a central block together with 3 × 3 neighboring blocks is used to linearly regress.Thus, two thirds of data used to regress between two neighboring blocks are the same, which ensures that mutation in fusion result of the connected region on two blocks does not occur.

QUALITY METRICS FOR ASSESSMENT
The fusion quality is assessed in two aspects, i.e., preservation of spectral characteristics and enhancement of spatial details (Möller et al., 2012;Saeedi and Faez, 2011;Witharana et al., 2013;Zhou et al., 2014).The used metrics for preservation of spectral characteristics are correlation coefficients (CC), root mean squared error (RMSE) (Möller et al., 2012;Saeedi and Faez, 2011;Witharana et al., 2013) and spectral angle mapper (SAM) (Alparone et al., 2007;Möller et al., 2012;Witharana et al., 2013;Zhou et al., 2014) between multispectral and fused images.The formula for calculating RMSE is as follows: where K, I, J are band count, columns and rows of the image, respectively; while the metric, SAM, is calculated through the following equations: , where β (i,j) is the spectral angle of two spectral vectors at the pixel-location represented as (i, j).The spatial details of sharpened images are assessed through a correlation coefficient between the high-pass filtered panchromatic and the high-pass filtered sharpened images, named Laplacian correlation coefficient (LCC) (Möller et al., 2012;Saeedi and Faez, 2011;Zhou et al., 1998).The Laplacian filter is illustrated here: These abovementioned metrics are commonly used in the field of remote sensing image fusion.

THE OPTIMAL SELECTION OF THE BLOCK SIZE FOR VERY HIGH RESOLUTION SATELLITE IMAGE FUSION
Two experimental datasets, one is Worldview-2 panchromatic and multispectral images and the other is QuickBird panchromatic and multispectral images, are used in the following experiments.

The Worldview-2 dataset
Pan-sharpened Worldview-2 images using BR with different block size are shown in Figure 1.The fusion results of 8 different configurations of block sizes not only enhance the spatial details but also preserve the spectral characteristics.It is difficult to find the obvious difference caused by different block sizes through visual comparison.
In order to further compare the fusion results of different block sizes, we calculate the quality metrics for image fusion described in Section 3. The correlation coefficients between Worldview-2 multispectral and sharpened images are indicated in Table 1, while RMSE and SAM between them in Table 2.
Table 3 shows correlation coefficients between the high-pass filtered panchromatic and the high-pass filtered sharpened images, named Laplacian correlation coefficient (LCC) because of Laplacian filter.

The QuickBird dataset
Pan-sharpened QuickBird images using BR with different block size are shown in Figure 2. Like the results for the Wordview-2 dataset, these results of QuickBird images can enhance the spatial details while preserving the spectral characteristics through visual scrutiny.The quality metrics for image fusion are indicated in Table 4 -6.
In terms of correlation coefficient, the difference between those of Wordview-2 and QuickBird images is that the correlation coefficients between QuickBird multispectral and panchromatic images are relatively average (i.e., about 0.6 -0.7) for the four bands while those of Wordview-2 images are relatively low for two bands (bands 7 and 8).As indicated in    6. LCC between QuickBird panchromatic and fused images using BR with different block size At the same time, these two configurations violate the law caused by the increase of block size according to the results in Table 4 and 5. Thus, they are exempted.With an increase in block size from 32 × 32 to 1024 × 1024, the law which is disclosed by the values of quality metrics of the pan-sharpened QuickBird image is the same as that corresponding to Wordview-2 image, i.e., ability of spectral preservation slightly decreases while the ability of enhancement of spatial details slightly increases.Similar to the results of Wordview-2 dataset, the QuickBird results of four configurations of block sizes, i.e., 64 × 64， 128 × 128， 256 × 256, and 512 × 512, are relatively better, and BR with the block size, 128 × 128 (bold in tables), can achieve a satisfactory trade-off between preservation of spectral characteristics and enhancement of spatial details.
According to the assessment of the fusion results of the two very high resolution datasets using different configurations of block sizes, the conclusion can be drawn that BR can adjust the fusion results by tuning a parameter (i.e., block size) to realize the different emphases between preservation of color characteristics and enhancement of spatial details.The block size, 128 × 128, is optimal for BR which can generate a satisfactory fusion results with this size.It means that BR with this size can maximize enhancement of spatial details at the expense of a minimum spectral distortion derived from the fusion operations and achieves a satisfactory balance between preservation of spectral characteristics and enhancement of spatial details.In the following comparisons of different fusion algorithms, the blocks size for BR is 128 × 128.

COMPARING WITH OTHER PAN-SHARPENING ALGORITHMS IN FUSION QUALITY
In this paper, three typical pan-sharpening algorithms from different types of fusion techniques (i.e., PC (Shettigara, 1992), CN Brovey (Vrabel, 2000) and AWT (Núñez et al., 1999) ) and the two state-of-the art algorithms presented in recent years (i.e., Ehlers (Ling et al., 2007) and BDF (Fasbender et al., 2008) ) are selected to be compared with the optimized BR in fusion quality.Among them, the versions of PC and Ehlers implemented in the commercial software ERDAS IMAGINE and the version of CN Brovey implemented in the software ENVI, which only fuses 3 bands of the multispectral and panchromatic images, are exploited.BDF in the open source software OTB is used, and the used version of AWT is implemented by us using the C++ program language.

The Worldview-2 dataset
The fusion results of these algorithms for Wordview-2 are shown in Figure 3, while the corresponding fusion metrics are listed in

The QuickBird dataset
The fusion results of these algorithms for QuickBird are shown in Figure 4, while the corresponding fusion metrics are listed in  According to the above experimental results of different algorithms and discussions, a more general finding can be drawn as follows: it is quite difficult (or almost impossible) to enhance the spatial details and preserve spectral characteristics at the same time, both to the maximum extent; a more practical option is to realize the different emphases in the light of actual requirements.Therefore, it is extremely important for pansharpening algorithms to adjust the fusion results by setting a simple parameter and to achieve a satisfactory trade-off between preservation of spectral characteristics and enhancement of spatial details in practice.

Figure 1 .
Pan-sharpened Worldview-2 images using BR with different block size (color composites of band 5, 3, and 2, true color for display, display scale 1:1, the result of 1024 × 1024 is not shown because of page limit) Pan-sharpened QuickBird images using BR with different block size (color composites of 4, 3, and 2 bands, false infrared (IR) color for display, display scale 1:1, the result of 1024 × 1024 is not shown because of page limit) 2643 0.9630 0.9625 0.9621 0.9621 0.9620 0.9620 0.9619 0.

BR
Pan-sharpened QuickBird images using different pan-sharpening algorithms Table 1 reveal that the correlation coefficients between Worldview-2 multispectral and sharpened images of different block size are higher than those between panchromatic and multispectral images, especially for band 6, 7, and 8.The SAM values for different block size in Table 2 are the same.With an increase in the block size from 8 × 8 to 1024 × 1024, the correlation coefficients for each band slightly decrease and the RMSE values in Table 2 slightly increase, which indicate that the ability of spectral preservation slightly decreases.In contrast, the LCC values between Worldview-2 panchromatic and fused images in Table 3 indicate that the ability of enhancement of spatial details increases with an increase in the block size from 8 × 8 to 1024 × 1024.This is evidenced by the fact that the LCC values slightly increase with the rising block size, except a case where the LCC values of the block size, 1024 × 1024, are 0.0001 less than those of the block size, 512 × 512, for band 2, 3, and 5.Meanwhile, all LCC values

Table 6
, the LCC values of the two configurations of block sizes, 8 × 8 and 16 × 16 are far lower than those of other configurations.They do not realize the expected results.

9617 Table 1 .
CC between Worldview-2 multispectral and fused images using BR with different block size

Table 5 .
RMSE and SAM (radian)between QuickBird multispectral and fused images using BR with different block size Note: NaN indicates the very small numerical value that can not express by computer.
Table 7 -9.The best values are indicated as bold in these tables and the worst values as italic and red.From Figure3, the spectral preservation of the sharpened results of PCA and CN Brovey is relatively bad, while the other algorithms can preserve the color characteristics of multispectral images very well through visual comparison; in terms of the enhancement of spatial details, the result of AWT is bad.The above states are also validated by the results in Table7-9.As indicated in Table7, the correlation coefficients of three bands are lowest for PC, two bands for Ehlers and CN respectively, and one band for BR.The correlation coefficients of four bands are highest for AWT and BDF, and the average value of all bands is highest for AWT, lowest for CN.BR is in the middle of these algorithms.RMSE and SAM values in Table8indicate that AWT has the best ability of spectral preservation followed by BDF and BR, while PC, Ehlers, CN are relatively inferior in spectral preservation.The results in Table9reflect the ability of enhancement of spatial details.The LCC values of three bands are highest for BR, two bands for Ehlers and CN respectively, and one band for AWT.Meanwhile, the LCC values of three bands are lowest for Ehlers, two bands for AWT and BDF respectively, one band for PC.There is no band whose LCC value is lowest for BR and CN.Furthermore, the average values of LCC are highest for CN and BR, and lowest for BDF and PC.According the experimental results, it can be found that BR has the best ability of enhancement of spatial details followed by CN, others are relatively bad because of either low LCC values of multiple bands or low average value.

Table 10 -
12. Through visual comparison, BR is best in spectral preservation followed by Ehlers, PC is the worst, and others are moderate.In terms of enhancement of spatial details, BR and PC are relatively superior, while Ehlers relatively inferior, others are ordinary.The quantitative results in Table10-11 show that AWT and BDF are best in spectral preservation followed by BR, others are relatively bad.In Table12, the LCC values for PC and BDF are relatively high followed by BR and CN, while the values for Ehlers and AWT are low.