An NSST-Based Fusion Method for Airborne Dual-Frequency, High-Spatial-Resolution SAR Images

With the continuous development of synthetic aperture radar (SAR) technology, SAR image data are becoming increasingly abundant. For the same scene, dual-frequency (high-frequency and low-frequency) SAR images can present different details and feature information. SAR image fusion of the two frequencies can combine the advantages of both, thus describing targets more comprehensively. Because high-resolution SAR images contain a large amount of detailed information, such as edges and textures, the traditional fusion methods cannot fuse this information better, resulting in a loss of information. To solve the problem, this article proposes a fusion method suitable for airborne dual-frequency, high-resolution SAR images. First, the source SAR images are decomposed to obtain their low-pass bands and high-pass bands by using the nonsubsampled Shearlet transform (NSST). Then, we apply the improved nonnegative matrix factorization to merge the low-pass bands and use the new sum of modified Laplacian to merge the high-pass bands. After that, the fused low-pass bands and high-pass bands are reconstructed by the inverse NSST, to obtain the final fused image. Finally, by processing the airborne SAR data, the effectiveness of the proposed method is verified.

S YNTHETIC aperture radar (SAR) is a microwave-imaging radar system that can achieve high-resolution imaging [1]. As an active means of microwave remote sensing, SAR offers all-time and all-weather reconnaissance and strong surface penetration, which are widely used in environmental protection, military reconnaissance, surface mapping, and other fields [2]. For SAR signal processing, there are mainly digital signal processing, statistical signal processing, and other techniques. Among them, statistical signal processing plays an important role, which is always applied to extract the useful information from the noise background. So, the statistical signal processing methods are widely used in many SAR research fields, including SAR change detection [3], [4], automatic target detection and recognition [5], image registration [6], image fusion [7], etc.
With the increasing requirements of reconnaissance technology in antispoofing, antijamming, anticamouflage, and other applications, it is difficult for a single SAR system to meet the requirements. Multimode, multipolarization, and multiband SAR technology has thus become an important development direction, and the multiband SAR technology has been rapidly developed [8]. In practical applications, the characteristics of transmission and backscattering are different for the electromagnetic waves of different frequencies, which leads to the different limitations and advantages among SAR images with different bands. For the same scene, a high-frequency SAR image can obtain clearer outline of the scene and richer detailed features of the target that are closer to those of the corresponding optical image. However, as the penetration of high-frequency electromagnetic waves is poor, high-frequency SAR images have difficulty representing hidden targets. Compared with highfrequency SAR images, low-frequency SAR images are dim and have difficulty describing the details of targets. However, they have strong penetrating power and can image hidden targets better, such as scenes and hidden targets in the forest or under the ground. Therefore, fusing SAR images from different bands and synthesizing the information from different bands are beneficial for image interpretation, which is of great significance in military reconnaissance, disaster monitoring, etc. [9].
Among the common image fusion technologies, the fusion methods based on multiscale transform are very popular algorithms used in various image fusion scenarios. The multiscale transform theories mainly include Laplacian pyramid (LP), wavelet transform, curvelet transform (CVT), nonsubsampled contourlet transform (NSCT), etc. In 1983, Burt and Adelson [10] proposed the LP. After that, Toet [11] successively proposed many image fusion algorithms based on improved pyramids, such as the ratio of low-pass pyramid, and the contrast enhancement pyramid [12]. The pyramid-based fusion algorithm has the advantage of fast fusion. However, the pyramid transform is a redundant decomposition, resulting in a large correlation of the decomposed data between the layers. Moreover, it is impossible to set the orientation during the decomposition process, which leads to the loss of high-frequency information about the image. In 1993, Ranchin and Wald [13] first applied discrete wavelet transform (DWT) to the fusion of multisource remote sensing images. Then, a variety of new wavelet transforms, such as multiwavelet transform, complex wavelet transforms, and shift-invariant wavelet transform, have been proposed and used in image fusion [14]. However, the wavelet transform can only make a sparse approximation of the image in the horizontal, vertical, and diagonal directions. It results in that the decomposed data are not enough to describe the details, such as edges and textures, contained in the SAR image well. In 2005, Choi et al. [15] first used the CVT to fuse remote sensing images. And then, more image fusion algorithms based on the CVT have been proposed [16], and good fusion results have been achieved. However, the complexity of the digital implementation of the CVT limits its development. Do and Vetterli [17] proposed the contourlet transform in 2005. During the decomposition process, the contourlet transform can make a sparse approximation of the image in multiple directions. After that, Zheng et al. [18] achieved multiband SAR image fusion  [19] proposed the NSCT, which not only has the advantage of the contourlet transform but also has the shift-invariant. Then, many fusion algorithms based on NSCT have been proposed [20], [22]. In 2020, Song et al. [23] proposed a multiband SAR image fusion algorithm that is based on dual features and NSCT. The results show that this method can synthesize the information of SAR images in different bands and keep details well [23]. However, due to the structure of NSCT, the operation is very time-consuming (Table I).
To solve the aforementioned problems, this article proposes a fusion method that is suitable for dual-frequency, highresolution SAR images. First, a nonsubsampled Shearlet transform (NSST) is introduced. It is a multiscale geometric analysis theory, which has a compactly supported frame and has no constraints on the number of directions during the Shearlet approach. And the source images are decomposed to obtain their low-pass bands and high-pass bands by using the NSST. Then, we apply the improved nonnegative matrix factorization (NMF) to merge the low-pass bands, which can effectively remove redundant information while retaining all the features of the low-pass bands. The high-pass bands are merged by using the new sum of modified Laplacian (NSML), and the details, such as edges and textures, in the source images can be preserved as much as possible. After that, the fused low-pass bands and high-pass bands are reconstructed by the inverse NSST, to obtain the final fused image. To prove the effectiveness of the method proposed in this article, we use the airborne SAR data, which are independently measured, to verify the method.
The rest of this article is organized as follows. In Section II, the basic principles of NSST and NMF are introduced. In Section III, the fusion method for dual-frequency, high-resolution SAR images based on NSST is introduced. In Section IV, the experimental results are analyzed to verify the effectiveness of the method. Section V provides discussion. Finally, Section VI concludes this article.

A. Nonsubsampled Shearlet Transform
During the standard Shearlet transform, the filter banks used are obtained by translation of the window function in the pseudopolar coordinates, which results in a down-sampling operation. Therefore, the standard Shearlet transform does not have the shift-invariant, which leads to a pseudo-Gibbs phenomenon when reconstructing the images [24]. NSST is an improvement over the Shearlet transform, in which the Shearlet filter (SF) banks are mapped from the pseudopolar coordinates to the Cartesian coordinates. Then, the next steps can be done based on two-dimensional convolution and the inverse Fourier transform (IFFT), which avoids the down-sampling operation [25], [26]. Thus, NSST has the shift-invariant.
The implementation of NSST is mainly divided into two steps. First, multiscale decomposition of source images is performed based on the nonsubsampled pyramid (NSP). Second, based on improved SF banks, the localization in the directions of the highpass bands is carried out. The flowchart of NSST is shown in Fig. 1, where the decomposition level of NSP is two.
Step 1: NSP decomposition. After the first level of NSP decomposition, the source image is decomposed into a low-pass band f 1 l and a high-pass band f 1 h . Then, the second level of NSP decomposition is performed on the low-pass band f 1 l to obtain the low-pass band f 2 l and the high-pass band f 2 h . After K levels of NSP decomposition, a low-pass band and K high-pass bands can be obtained.
Step 2: Localization in directions. Based on SF, the localization in directions of the K high-pass bands are achieved, and the specific process is as follows. First, the SF is generated by constructing window functions based on the "Meyer" wavelet. Then, the SF is mapped from the pseudopolar coordinates to the Cartesian coordinates. Next, the K high-pass bands are convoluted with the SF to obtain directional subbands, respectively. Finally, the directional subbands with shift-invariant are obtained by using the IFFT.

B. Nonnegative Matrix Factorization
NMF is a matrix analytic method that decomposes a matrix when all entries of the matrix are nonnegative. This method can effectively extract the main features from different images.
When the pixel values of an image are nonnegative, the features of the image can be obtained by applying NMF to the image with the dimension of the feature space setting appropriate [27].
The NMF can be described as follows. Given a nonnegative matrix V M ×N , find nonnegative matrix factors W M ×R and H R×N such that From the definition of NMF, we can find that there exist errors in the NMF. To minimize the errors between V and W H, a cost function must be defined to quantify the quality of the approximation. In general, for the solution of the NMF, the square of the Euclidean distance between V and W H is usually used as the cost function, and the expression is . Therefore, the more the value of the cost function tends to zero, the better the quality of the approximation.
For the optimization problem of the cost function, the commonly used update rules are additive update rule and multiplicative update rule. The additive update rule is more complex and difficult to converge, and the negative elements produced during iteration will be forced to be zero. Particularly, the additive update rule will be converted into the multiplicative update rule during iteration, for some parameter conditions. Thus, Lee and Seung [28] use the multiplicative update rule to optimize the cost function of the NMF. That is, first use W k to calculate H k+1 , then use H k+1 to calculate W k+1 , and iterate until W H converges to V . The multiplicative update rule has low complexity and relatively fast convergence. In addition, the multiplicative update rule states that if the initial values are nonnegative, the results of each update are nonnegative.
The contents of the multiplicative update rule given by Lee and Seung [28] are as follows: E(V |W H) takes its partial derivative for W and H, respectively The corresponding multiplicative update rules are as follows: In image fusion, a series of recorded images are acquired by using the different sensors. Essentially, these images are a superposition of noises and the real image passing through sensors. Therefore, the model when NMF is performed on an image is where V denotes the recorded image, W denotes the real image, H denotes the weight matrix, and ε is the noise and satisfies the Gaussian distribution. After iterations, the noise ε eventually converges. Thus, the essence of using NMF for image fusion is to recover the real image from the recorded images.
In the NMF, the number R of the characteristic matrix (that is the number of columns of W ) is one to be quantified. When R = 1, the unique W can be obtained after iterations. At this moment, W contains all the features of the recorded images involved in the fusion. In this way, W can be seen as an approximate representation of the real image, so that W can be as the fusion image.

A. Framework of Fusion Method
NSST is a new multiscale geometric analysis tool, which has the shift-invariant. Compared with transforms, such as the Contourlet transform, NSST can obtain a compactly supported frame, and there is no limit on the size of supports and the number of directions during the Shearlet process. Due to these advantages, NSST can make a more detailed description of images. Therefore, in this article, the dual-frequency, high-resolution SAR images will be fused based on NSST, and the specific process is as follows.
Step 1: Apply NSST to decompose the source images A and B, which have been accurately registered, and obtain the corresponding low-pass bands f A L , f B L and high-pass bands f A H , f B H .
Step The flowchart is shown in Fig. 2.

B. Fusion Rule for Low-Pass Bands
The obtained low-pass band obtained by using NSST is the approximate feature of the image, which usually corresponds to a nonnegative matrix. Therefore, this article will design the fusion rule for the low-pass bands based on NMF. According to Section II, the model of NMF is based on additive noise. However, for SAR images, the noise model is multiplicative noise, which is where Y (x, y) denotes the observational image, X(x, y) denotes the real image, and N (x, y) is the speckle noise. To conform to the model of NMF, the logarithm of formula (6) is taken to get formula (7), and the multiplicative noise is converted into the additive noise ln(Y (x, y)) = ln(X(x, y)) + ln (N (x, y)).
The fusion rule for the low-pass bands in this article is described as follows. Suppose that there are k observational images, and Y (x, y) is expressed in vectors The logarithm of formula (8) is taken to obtain . Then, the NMF decomposition equation corresponding to (9) is where Let e W X H X = V X , and apply NMF to V X , that is, V X = W H + ε. During NMF, setting R = 1, that is, the number of columns of W is 1, the unique basis matrix W will be calculated after iteration. At this moment, the obtained W contains the complete features of the k observational images involved in the fusion.

C. Fusion Rule for High-Pass Bands
The obtained high-pass bands reflect the degree of the change of the image grayscale, corresponding to the strong edges, textures, and other significant features. Generally, the high-pass bands are merged by taking the larger coefficient values. However, the pixels of an image are usually related to the area around it. Thus, to obtain a fused image with rich details and a better visual effect, it is necessary to consider a single pixel into a certain area. The sum of modified Laplacian (SML) is an algorithm that can appropriately reflect the edges, textures, and orientation information of images [29]. SML is defined as follows: where s denotes the variable step, (2M + 1) where NLP l,k J (i, j) denotes the modified Laplacian energy; NSML l,k J (i, j) is the corresponding sum of the modified Laplacian energy at the scale l, direction k, and position (i, j).
Since NSML can better reflect the characteristic of images, this article will adopt the max-NSML as the fusion rule for highpass bands, which is defined as follows: where Img l,k A (i, j) is the directional subband at the scale l, direction k, and position (i, j) of the source image A; Img l,k B (i, j) is the directional subband at the scale l, direction k, and position (i, j) of the source image B; Img l,k F (i, j) is the merged subband at the scale l, direction k, and position (i, j).

IV. EXPERIMENT AND ANALYSIS
To verify the effectiveness of the proposed method in the fusion of dual-frequency high-resolution SAR images, the proposed method is compared with the DWT-based fusion method [13] and the NSCT-based fusion method [19]. The main process of the DWT-based fusion method is to first decompose the source images by using the DWT to obtain the low-pass bands of different layers and the high-pass bands in horizontal, vertical, and diagonal directions; then select the appropriate fusion rules to obtain the fused low-pass bands and high-pass bands; and finally use the inverse DWT to obtain the fusion image [31]. The main process of the NSCT-based fusion method is the same, except that NSCT is used as the decomposer and the fusion rules may be different.
Two groups of registered SAR images (Group 1 and Group 2, respectively) were processed. The data of Group 1 are the airborne P-band (low-frequency) and X-band (high-frequency) linear SAR (LSAR) images, and the data of Group 2 are the airborne L-band (low-frequency) and Ku-band (high-frequency) circular SAR (CSAR) images [32], [33]. The data were recorded by the National University of Defense Technology using the SAR systems independently developed. For Group 1, the imaging scene is a wooded area, the polarization is HH, and the resolution of both P-band and X-band LSAR images is 1.5 m. Fig. 3(a) shows the P-band LSAR image of this scene, and the region signed with a red oval in Fig. 3(a) shows seven trucks hidden in the woods. The X-band LSAR image is shown in Fig. 3(b), where eight corner reflectors are signed with a red oval. For Group 2, the imaging scene is an urban road, the polarization is HH. Fig. 4(a) shows the L-band CSAR image whose resolution is 0.5 m, and Fig. 4(b) shows the Ku-band CSAR image whose resolution is about 0.3 m.

A. Processing and Analysis of Measured Data
During the processing of the measured data, the number of decomposition layers for all algorithms is 4. For the fusion method based on DWT and the fusion method based on NSST, the fusion rule of low-pass bands adopts the weighted average method, and the fusion rule for high-pass bands is taking the max value. The aforementioned methods and the methods proposed in this article are applied to process the two groups of measured data. The fusion results of Group 1 are shown in Fig. 5, and Fig. 6 shows the fusion results of Group 2. As can be seen from Figs. 5 and 6, each method can merge the different information from high-and low-frequency SAR images into an image to a certain extent.
For Group 1, Fig. 5(a) shows the result of the DWT-based fusion, Fig. 5(b) shows the result of NSCT-based fusion, and  in Fig. 6(a) and (b) shows the NSCT-based result, and Fig. 6(c) is the result of the proposed method. Compared with the result of the proposed method, the block effect and the poor fusion quality of the eight corner reflectors can also be clearly seen in the DWT-based result. As for the NSCT-based fusion result, it is generally dark and slightly blurred, but the others are similar to Fig. 6(c).
To objectively evaluate the quality of the fusion results, this article uses entropy, standard deviation (SD), average gradient (AG), and edge information evaluation factor (Q AB/F ) to quantitatively analyze the fusion results [34]. Entropy is to evaluate the richness of information. The larger the value, the better the fusion quality, and the more information contained in the fusion image. SD reflects the degree of dispersion of the pixel gray value compared to the overall average, which is used to measure the degree of contrast of an image. The larger the SD, the more dispersed the gray value of the image, and the higher the contrast of the image. AG is also known as the definition. The higher its value is, the better the image definition is and the better the fusion effect is. The edge information evaluation factor Q was proposed by Xydeas and Petrovic [35], which is used to evaluate the number of edges transmitted from the source image to the fusion image. The larger the value is, the more edges transmitted. The aforementioned objective evaluation indexes are applied to the fusion images, and the results are shown in Tables II and III.  From Tables II and III, for Group 1, the entropy, SD, AG, and Q AB/F of the fusion images obtained by the proposed method are better than those of the comparison methods. For the data of Group 2, the entropy, SD, and Q AB/F of the fusion images obtained by this method are better than those of other methods, and AG is second only to the DWT-based fusion results.

B. Analysis and Results of Computational Efficiency
According to Section II, during the direction localization, NSST first performs coordinate transformation and then uses IFFT to achieve shift invariant. Moreover, the inverse NSST only needs to sum the SF banks, rather than inversely transform the directional filter banks, which makes NSST have high computational efficiency, and its operation amount is O (N 2 log N ). This experiment is carried out on two groups of measured data on a desktop computer with a 4.20-GHz Intel i7-7700K processor, 32.00 GB of RAM, and a 64-b Window7 operating system. Table IV shows the results of computational time.
The experimental results show that, in realizing the fusion of dual-frequency high-resolution SAR images, compared with the DWT-based fusion method, the fusion effect of the method proposed in this article has been significantly improved in terms of both subjective vision and objective evaluation indexes. Compared with the fusion method based on NSCT, the proposed method has no significant improvement in the fusion effect. However, the computational efficiency is greatly improved, and the computational time is about 1/8 of the NSCT-based method. It shows that, under the premise of ensuring the fusion quality, the proposed method has a higher computational efficiency, which reflects the effectiveness of the fusion rules of this article, and verifies the superiority of the proposed method.

V. DISCUSSION
According to the analysis results in Section VI, we can find that the fused image obtained from the proposed method gave higher fusion quality than the image from the DWT-based method. And compared with the NSCT-based method, the proposed method has not significantly improved the fusion quality, but it has much higher computational efficiency. Compared with the traditional methods, the proposed method not only has better fusion quality but also has higher computational efficiency.
However, when performing the fusion of the high-pass bands (see Section III-C), the used window is a 3 × 3 square, which may lead to a reduction in the quality of the fusion result. This is because the characteristics of different pixels with their surrounding pixels may be different. Therefore, when considering the influence of surrounding pixels on the center pixel, the size and shape of the window should be adapted to the characteristic of the pixel with its surrounding pixels.

VI. CONCLUSION
There are different limitations and advantages among the high-frequency and low-frequency SAR images. Therefore, the fusion of dual-frequency SAR images can synthesize the information from different bands to form complementary advantages. To solve the problem in the fusion of dual-frequency SAR images, this article introduces NSST as the multiscale transform, which solves the problem that the traditional method has constraints on the number of directions. During the fusion of low-pass bands, an improved NMF method is proposed to merge the low-pass bands, which can effectively remove redundant information while retaining all the characteristics. As for highpass bands, NSML is used to merge the high-pass bands, and the details, such as edges and textures, in the source image are preserved as much as possible. The experimental results show that, compared with traditional methods, the proposed method not only has a good fusion effect on textures, edges, and other details of source images but also has a great improvement in the computational efficiency, which greatly reduces the operation time.
The size and shape of the window used in the fusion of the high-pass bands (see Section III-C) may have a serious effect on the final fusion quality, because the characteristics of different pixels with their surrounding pixels may be different. In future research, we will focus work on analyzing this problem, and propose an effective method to obtain the best window or implement an adaptive window, and finally obtain higher a fused quality of the dual-frequency SAR image.