Improved relative radiometric normalization method of remote sensing images for change detection

Abstract. Relative radiometric normalization (RRN) of remotely sensed images is often a preprocessing step during time series analysis and change detection. Conventional RRN methods may lessen the radiation difference of changed pixels in images during the RRN process, thus reducing the accuracy of change detection. To solve this problem, we propose a relative radiometric correction method based on wavelet transform and iteratively reweighted multivariate alteration detection (IR-MAD). A wavelet transform is applied to separate high and low frequency components of both the target image and reference image. The high frequency components remain unprocessed to preserve high frequency information. We use the IR-MAD algorithm to normalize the low frequency component of the target image. A reverse wavelet transform reconstructs the radiometrically normalized image. We tested the proposed method with traditional histogram matching, mean variance, the original IR-MAD method, and a method combining wavelet transform and low-pass filtering, and change detection was conducted to evaluate the RRN quality. The experiments show that the proposed method can not only effectively eliminate the overall radiation difference between images but also enable higher accuracy of change detection.


Introduction
Divergences in the reflectance of remote sensing images for an area can indicate land cover change.Thus, the existing change detection methods often determine whether there is change, according to the radiometric differences between the images. 1,2The physically unchanged ground object in two images acquired at different times can lead to variation in spectral values. 3The reason is that acquisition conditions are different, such as status or posture of a sensor, solar illuminance, observation angles, atmospheric scattering, and absorption. 4,5This problem presents challenges in multitemporal image processing and analysis. 6Therefore, in practical remote sensing applications, radiometric normalization is conducted to eliminate the radiometric discrepancy between images caused by acquisition conditions rather than actual changes in ground objects. 7,80][11][12] Absolute radiometric correction based on single image reveals actual surface response by removing the influence of the atmosphere. 5,13][15][16] To the contrary, relative radiometric normalization aims at minimizing the radiometric differences caused by inconsistencies of acquisition conditions between images.It is used for multitemporal images.In a relative radiometric normalization, one of the images is *Address all correspondence to Kaimin Sun, E-mail: sunkm@whu.edu.cnconsidered as the reference image and all other images are normalized in such a way that they become radiometrically similar to the reference image. 12,17Difficulty to collect the necessary ancillary data, along with the lack of historical data, has further reduced the opportunity to perform absolute radiometric correction on multitemporal images.Fortunately, since RRN does not require any ancillary data and is easier to achieve than absolute correction, it has been widely used to normalize images obtained at different times. 18,19In some applications like change detection and classification, it has been demonstrated that some RRN methods performed better than absolute radiometric corrections. 13,18ommonly used RRN methods consist of two types: nonlinear normalization and linear normalization. 20Nonlinear methods include histogram matching (HM). 8This approach can cause gray scale loss and a disordered overall radiation distribution since it achieves correction by matching the histogram of a target image with that of the reference image. 20Linear methods include minimum-maximum (MM), 9 mean-standard (MS) deviation, 9 haze correction (HC), 21 image regression (IR), 22,23 pseudoinvariant feature (PIF), 11,[24][25][26] dark set-bright set (DB), 17 and no-change set (NC) 27 techniques.Most of these methods (HM, MM, MS, HC, and IR) use all of the pixels in the estimation of normalization coefficients. 28Such methods do not often perform as well as methods using invariant pixels 8,28,29 and can lead to low change detection accuracy since the radiometric difference caused by physical ground change is normalized.Those methods using unchanged pixels, however, are time and labor consuming due to the selection of invariant pixels, and furthermore, the quality of the chosen samples directly affects the relative radiometric correction results. 30,313][34] These invariant pixels selection methods include slow feature analysis, 31 Kauth-Thomas transformation, 17 scattergram-controlled regression, 27 temporally invariant cluster, 18 principal component analysis (PCA), 4 multivariate alteration detection (MAD), 35 and iteratively reweighted multivariate alteration detection (IR-MAD). 36These methods can increase the quality and number of invariant pixels, as well as reduce human intervention and subjectivity.Canty et al. 35 applied MAD to define automatically the invariant pixels within multispectral images of the same area collected at two different times.Their results showed that automatically obtained invariant features generated better results than those produced by manual selection.To improve the sensitivity of MAD, Nielsen et al. 36 proposed IR-MAD.Not only does this method automatically select invariant features but also determine an adaptive threshold through an iterative process.As a consequence, IR-MAD is an effective method to select unchanged pixels.Mateos et al. 37 radiometrically normalized multitemporal remote sensing images using the IR-MAD algorithm.Canty and Nielsen 38 applied IR-MAD for the normalization of LANDSAT and ASTER multitemporal images.These RRN methods assume the invariant pixels' values in the target image linearly related to those of the reference image because of simplification of modeling.However, in fact, the relation does not follow a linear model typically.Thus, the linear assumption will negatively affect the normalization results.Furthermore, these methods can result in loss of high frequency details.
Frequency domain transforms, such as the Fourier, 39 wavelet, 40 and contourlet 41 transforms, were applied in relative radiometric normalization to overcome the limitations of conventional methods.Biday and Bhosle 42 used Fourier and wavelet transforms to separate high and low frequency components in images; the effectiveness was validated in comparative experiments with two other relative radiometric normalization methods.Sun et al. 43 proposed an RRN method based on wavelet transform and low pass filter (WLPF) that effectively improved the change detection accuracy.Li et al. 20 presented a method for relative radiometric consistency processing, based on object-oriented smoothing and contourlet transforms, concluding that the proposed method can improve the visual effects of normalized images thus increasing the accuracy of change detection.
To overcome the limitations of existing methods, we propose a relative radiometric normalization method based on wavelet transform and IR-MAD (WIRMAD).Wavelet transform is used to divide images into spatial high frequency and low frequency components.We automatically extract invariant features from the low frequency components, and then a linear regression equation is used to normalize the low frequency component of the target image to the reference image.A reverse wavelet transform is applied to reconstruct the final normalized image.
We compared the proposed method visually and empirically to the traditional HM, MS, IR-MAD, and WLPF methods in experiments with three pairs of images in China at different spatial resolutions.Furthermore, change detection was conducted on these images to evaluate the RRN quality of the proposed method.The remainder of this paper is as follows: Sec. 2 describes the datasets and Sec. 3 details the proposed relative radiometric normalization method based on wavelet transform and IR-MAD, followed by the experimental results and analysis in Sec. 4. Section 5 discusses the application and limitations of the proposed method, and the conclusion is given in Sec. 6.

Datasets
Four pairs of bitemporal images at different spatial resolutions were employed in the experiments, one from Nanjing for normalization and three for change detection from Shenzhen, Wuhan, and Guangxi in China, respectively.The information of the datasets used in this paper is summarized in Table 1.
For each image pairs, we geometrically registered the target image and reference image with error controlled to within one pixel before normalization and change detection.To evaluate the change detection accuracy, true change maps were produced by visual interpretation.Auxiliary images from Google Earth were used to ensure the accuracy of true change maps.

Methodology
In this paper, we present an RRN method combined wavelet transform and IR-MAD algorithm.Figure 1 illustrates the relative radiometric normalization approach used in this study in a general way; we discuss the major steps in detail in Secs.3.1-3.3.

Wavelet Transform
The low frequency of remote sensing image corresponds to the holistic background radiation information, and the high frequency is the foreground target, texture information.Therefore, the purpose of global radiometric correction can be achieved by eliminating the difference between the low frequency components of the target and reference images.Keeping the high frequency components unprocessed can preserve high frequency information.
Wavelet transform is one of the commonly used algorithms in frequency domain processing, developed in signal processing theory to help extract information from many different kinds of data. 44In this paper, wavelet transform is used to separate the spatially low and high frequencies.There are different types of wavelet basic functions, whose qualities vary according to several criteria.In this study, Haar wavelet, the simplest wavelet and one of the first studied, is used to achieve the separation of low and high frequencies.According to the analysis in Li's study, 20 we decompose the target image and reference image using a four-level wavelet transform.

IR-MAD Algorithm
After wavelet transforms, we used IR-MAD to select invariant pixels from the low-frequency components of target and reference images.IR-MAD is an effective method to select pixels with high no-change probability between images. 3or two n-band images, acquired at times t 1 and t 2 , represent them by two vectors F ¼ ðF 1 ; F 2 ; : : : ; F n Þ T and G ¼ ðG 1 ; G 2 ; : : : ; G n Þ T .Two linear combinations are constructed for all spectral bands: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 6 ; 3 2 9 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 2 9 8 where U and V are called the canonical variates; n is the number of bands; a and b are constant vectors, maximizing the variance of U − V.
In this way, the difference image U − V will show maximum change information, referred to as the MAD variates, where i is number of bands: Assuming that no ground reflectance changes have occurred in two images of a scene, in such a case, the sum of the squares of the standardized MAD variates will approximately follows a chi-square distribution with n degrees of freedom (χ 2 ðnÞ): where Z represent the sum of the squares of the standardized MAD variates and σMAD i is the variance of MAD i .The MAD variates associated with change observations, however, will deviate more or less strongly from such a multivariate normal distribution. 38Therefore, to improve the sensitivity of the MAD transformation, Nielsen et al. 36 weight observations by the probability of no change though an iteration scheme: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 1 1 6 ; 6 8 7 Prðno changeÞ ¼ 1 − P χ 2 ;nðZÞ ; (5) where Prðno changeÞ is the weight, representing the probability of no change, and P χ 2 ;nðZÞ is the quantile of chi-square distribution.
Iterations are continued until the largest absolute change in the canonical correlations is smaller than a preset small value, e.g., 10 −6 . 36For radiometric normalization purposes, we can select all pixels that satisfy Prðno changeÞ > t, where t is a decision threshold, typically 95%. 36The invariant pixels will be used to estimate the normalization coefficients though regression fit and then normalize the target image to the reference image.

Relative Radiometric Normalization
In this paper, we apply the IR-MAD algorithm to normalize the low frequency component of the target image after separation of high frequency and low frequency components by using wavelet transform.For a specific band, assume that f1 is the image to be normalized and f2 is the reference image.The specific steps of relative radiometric normalization based on wavelet transforms and IR-MAD are as follows: 1. Apply wavelet transform to the target image f1 and reference image f2 to obtain the low-frequency components of the images, f L 1 and f L 2 , and the high-frequency components of the images, f H 1 and f H 2 .2. Apply IR-MAD algorithm to select the corresponding invariant pixels from the two lowfrequency components f L 1 and f L 2 .3. Perform the orthogonal linear regression on the selected invariant pixels to determine the relative radiometric normalization coefficients (slope a and intercept b).

Normalize the low frequency f L
1 of the target image f1 to the low frequency component f L 2 of the reference image f 2 through the follow equation: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 1 1 6 ; 3 6 0 5. Replace f L 1 with f L 1 0. 6. Apply the wavelet reverse transform to the low and high frequencies of f1 to obtain the radiometrically normalized result.
4 Experiments and Analysis

Relative Radiometric Normalization
In order to verify the proposed method WIRMAD for relative radiometric normalization, it was compared with the HM, MS, IR-MAD, and WLPF methods.The results are shown in Fig. 2. The radiation differences between RRN results, target image, and reference image were calculated, and Fig. 2 depicts the results.Visual inspections of Figs.2-4 show that the HM, MS, IR-MAD, WLPF, and WIRMAD methods significantly reduce the radiation difference between the target and reference images.The overall brightness and color of RRN results are similar with the reference image and the radiometric consistency is significantly improved compared with the target image in Figs.2(a)-2(g).The differences between RRN results and the reference image demonstrate that the results of WLPF and WIRMAD are more consistent with the reference image than that other methods, as shown in Figs.2(h)-2(m).The differences between RRN results and the reference image demonstrate that the results of WLPF and WIRMAD are more consistent with the reference image than that other methods, as shown in Figs.2(h)-2(m).The result of  WIRMAD method, however, was more consistent with the reference image, a result not evident through a visual inspection.
To quantitatively evaluate and compare our WIRMAD method and the HM, MS, IR-MAD, and WLPF methods, the mean, standard deviation, and correlation between the resulting images, the target image and reference image were calculated to assess the performance of these methods, as shown in Table 1.
As shown in Table 2, from the mean value, we can see that the traditional HM (103.0872),MS (102.5441), and IR-MAD (103.1381)methods yielded results closer to the reference image as compared with WLPF (95.8877) and WIRMAD (93.6330).The standard deviation of the results derived by WLPF (14.5668) and WIRMAD (14.4600) methods, however, is closer to the target image than that of HM (19.0636),MS (19.0603), and IR-MAD (18.3545) methods, which indicates that method based on wavelet transform retains more of the texture information in the original image.
In terms of correlation, as compared with HM, MS, and IR-MAD methods, the results obtained by WLPF and WIRMAD methods show lower correlation, 0.8218 and 0.8070, respectively, with the target image and higher correlation, 0.5201 and 0.5849 with the reference image,

Change Detection
Change detection experiments based on three pairs of bitemporal images at high and mid-high resolutions were carried out to assess further the proposed method.Three different change detection methods were used, object oriented and pixel based change vector analysis (CVA), PCA, and the iterated conditional model based on Markov random fields (ICM-MRF), 45,46 to avoid the contingency caused by specific data and change detection methods.
The RRN results of image pairs over Shenzhen are shown in Fig. 5.The pixel-based change detection results using CVA are also displayed in Fig. 5.
It can be seen from Fig. 5, using the same CVA change detection method on pixel level, more accurate change detection results with radiometric correction are obtained as compared to the raw data without normalization.The change detection results from WLPF and WIRMAD normalization were significantly improved over conventional HM, MS, and IR-MAD methods.
The change detection results were analyzed; the results are shown in Table 3. Omission, false alarm, overall accuracy, and kappa parameters were calculated to evaluate the accuracy of change detection based on RRN results derived by our WIRMAD method, HM, MS, IR-MAD, and WLPF methods.
In Table 3, we can see that the change detection results were similar regardless of the method.Change detection accuracy, however, was significantly increased with RRN as compared to the original image without normalization.The omission and false alarm rates decline, the overall accuracy and kappa coefficients rise.It can be concluded that RRN is crucial for change detection.
After RRN using WLPF and our WIRMAD method, the change detection results are similar and effectively improved over those derived by conventional HM, MS, and IR-MAD methods.These results suggest that dividing low and high frequencies using wavelet transform can improve the change detection accuracy.
Among the five RRN methods, our WIRMAD method derives change detection results with the lowest omissions and false alarms, and the highest overall accuracy and kappa coefficient, indicating that the proposed WIRMAD method can avoid reducing information about change and thus increase the accuracy of change detection.
Pixel-based change detection experiments were conducted to assess the results of RRN of images over Wuhan, China.The RRN results using the proposed WIRMAD method and conventional methods are shown in Fig. 6, as well as the change detection results using PCA derived from these RRN results.It can be seen from Fig. 6, change detection on GF-1 WFV images with RRN produces more accurate results than those obtained on an original image without normalization, especially for false negative results [Figs.6(i)-6(n)].The change detection results derived from RRN using the proposed WIRMAD and WLPF methods are closer to the true change map than results obtained after RRN using conventional HM, MS, and IR-MAD methods.
Table 4 shows the evaluation of change detection omission, false alarm, overall accuracy, and kappa parameter results for the two GF-1 WFV images.
As shown in Table 4, change detection results after RRN have much lower false alarm rate and higher coverall accuracy than raw images without radiometrical correction.Although the omission rate is slightly higher, RRN before change detection significantly improves the results.Change detection with RRN using the proposed WIRMAD method and WLPF method produced more accurate results than HM, MS, and IR-MAD methods to normalize images.The WIRMAD method, especially, had the highest overall accuracy (0.9395, 0.9156) and kappa coefficient (0.4244, 0.3718) when detecting change using PCA and ICM-MRF.
Object oriented change detection, including CVA, PCA and ICM-MRF, were applied to assess the RRN results of images over Guangxi, China.Figure 7 displays the RRN results and ICM-MRF change detection results.
The change detection results derived from RRN using the proposed WIRMAD method and WLPF method are closer to the true change map than results obtained after RRN using conventional HM, MS, and IR-MAD methods, as inferred from Fig. 5, since the omission rate is significantly lower.
The change detection results for GF-2 PMS images were analyzed and displayed in Table 5.We calculated omission, false alarm, overall accuracy, and kappa parameters to evaluate the accuracy of change detection by our WIRMAD method, HM, MS, IR-MAD, and WLPF methods, after RRN.
As shown in Table 5, the omission, false alarm, overall accuracy, and kappa for the change detection results using three different methods (CVA, PCA, and ICM-MRF) show the same trend.After RRN, the omission and false alarm rates decreased, and the overall accuracy and kappa coefficient increased as the omission and kappa values improved significantly.This indicates that the radiation difference between multitemporal images must be reduced though RRN, before change detection.Our WIRMAD method produces change detection results at the highest overall accuracy and kappa, with the lowest value for omissions, suggesting that the separation of low and high frequencies of images contribute to increased change detection accuracy.

Discussion
In this paper, we propose an RRN method based on wavelet transform and the IR-MAD algorithm.Wavelet transform separates the high and low frequency components while IR-MAD radiometrically normalizes the low frequency components of a target image.IR-MAD is a linear relative radiometric normalization method.However, even if the multitemporal images are very similar, it is impossible to have a complete linear relationship between images. 28Extracting low frequency components of images by wavelet transform eliminates the effects of nonlinear factors, such as texture and small changes in ground objects, 42 exposing a higher linear correlation and gives full play to IR-MAD.
The IR-MAD algorithm extracts invariant pixels from the low frequency components, linearly correcting the low frequencies of the target image to the low frequencies of the reference image.This protects the radiometric difference of changed objects in the low frequencies, therefore improving change detection results. 8The RRN results obtained by the proposed WIRMAD method were quite similar to those derived by the WLPF method and more consistent with the reference image than conventional RRN methods.The change detection results derived from RRN using the proposed WIRMAD were the closest to the true change with the highest overall accuracy and kappa coefficient, and lowest omission or false alarm rate, among the tested methods.Furthermore, it can be applied for pixel level or object level change detection regardless method.Moreover, except for change detection, it can be also applied for image dodging when mosaic images with overlapping, 29 gap filling and bad line removing based on a referenced image, and time series analysis. 6,19owever, there are limitations of the proposed method.It is not suitable for the multitemporal images with high nonlinear correlation since this method linearly normalize the low frequency of the target image.In addition, if the changed ground objects occupied a large proportion of the image, the normalized result will differ from the reference image visually.

Conclusions
Multitemporal images have radiation differences due to sensor and atmosphere conditions, even over the same area, creating challenges in multitemporal images processing and analysis.Conventional RRN methods, however, often reduce the difference caused by the change of the ground objects in the process of normalization.This negatively affects the change detection results and time series analysis.In order to solve this problem, we propose an RRN method based on the wavelet transform and IR-MAD algorithm.Wavelet transform is applied to separate the high frequency and low frequency components of both the target and reference images.We use Experimental results show that our WIRMAD method can not only achieves radiometric consistency of the target and reference image but also improves the accuracy of change detection.WIRMAD method applies wavelet transform to preserve high frequency information.In addition, low frequency of the target image is normalized using unchanged pixels selected by the IR-MAD algorithm, thereby improving change detection accuracy, making it more suitable than other RRN methods for change detection at pixel or object level.

Fig. 1
Fig.1The flow chart of the proposed method.

Fig. 5
Fig. 5 Results of relative radiometric normalization and change detection using CVA: (a) reference image, (b) target image, (c) HM, (d) MS, (e) IR-MAD, (f) WLPF, (g) WIRMAD, (h) true change, (i) change detection of (a) and (b) using CVA, (j) change detection of (a) and (c) using CVA, (k) change detection of (a) and (d) using CVA, (l) change detection of (a) and (e) using CVA, (m) change detection of (a) and (f) using CVA and (n) change detection of (a) and (g) using CVA.

Fig. 6
Fig. 6 Results of relative radiometric normalization and change detection using PCA: (a) reference image, (b) target image, (c) HM, (d) MS, (e) IR-MAD, (f) WLPF, (g) WIRMAD, (h) true change, (i) change detection of (a) and (b) using PCA, (j) change detection of (a) and (c) using PCA, (k) change detection of (a) and (d) using PCA, (l) change detection of (a) and (e) using PCA, (m) change detection of (a) and (f) using PCA and (n) change detection of (a) and (g) using PCA.

Table 1
Description of data sets using in this paper.

Table 2
Comparison using statistical parameters of RRN results in Fig.2.

Table 3
Evaluation of change detection accuracy.

Table 4
Evaluation of change detection accuracy.

Table 5
Evaluation of change detection accuracy.MAD algorithm to normalize the low frequency component of the target image.Wavelet reverse transform is conducted to reconstruct the radiometrically normalized image.