Image Denoising Algorithm Based on Gradient Domain Guided Filtering and NSST

Traditional image denoising methods, which do not depend on data training, have good interpretability. However, traditional image denoising methods hardly achieve the denoising effect of deep learning methods. Based on traditional image processing techniques, this paper proposes a new hybrid image denoising model. The block-batching and 3-D filtering (BM3D) algorithm is used to obtain the first denoised image. The weighted kernel norm minimization (WNNM) and non-subsampled shearlet transform (NSST) algorithms are successively adopted to get the second denoised image. By the gradient domain guided filtering, the texture information of the first denoised image is extracted to enhance the details of the second denoised image. Specially, we propose the adaptive iterative NSST algorithm based on the improved soft thresholding, in order to solve the problems about the discontinuity of the hard thresholding and the constant deviation of the soft thresholding. Our approach can not only attenuate excessive smoothing, but also restore the natural appearance of the image. Experiments are conducted to demonstrate that our proposed method enjoys PSNR and SSIM performance gains over several deep learning denoising methods.


I. INTRODUCTION
In the process of image generation and transmission, the image quality is often degraded due to the influence of various noises, which is an important factor hindering human cognition. Image denoising is not only an important low-level visual task, but also an indispensable part of high-level visual tasks. Removing image noise and preserving image details are the pivotal problems of high-quality image denoising. Deep learning denoising technique [1], [2], [3], [4], relying on its powerful learning ability, has made outstanding achievements in the field of image denoising. Nevertheless, deep learning denoising methods depend on the training of massive data, which induces the denoising model less interpretable. Beyond that, deep learning technique requires high computing power, which makes the cost of the design model very high.
The associate editor coordinating the review of this manuscript and approving it for publication was Ikramullah Lali.
Traditional image denoising methods, starting from the spatial or frequency domain information of a single image, have good interpretability and low requirement on equipment. Some pixel feature denoising methods, such as adaptive median filters, curvature filters, bilateral filters and guided filters, remove image noise by analyzing the direct relation between the center pixel and other adjacent pixels in a certain size window. Although guided filters [5], [6], [7], [8] can not provide excellent capability in removing image noise, they can smooth the image while clearly maintaining the image boundary. Frequency domain denoising methods transform the image from the spatial domain to the frequency domain by wave filters, then separate the clear image and noise in the frequency domain. As an advanced representation of frequency domain denoising methods, NSST denoising method [9] has the translation invariance and multi-scale characterization, which can overcome Pseudo-Gibbs phenomenon. Particularly, image denoising methods based on non-local self-similarity (NSS) have achieved good effect. By stacking similar 2D image blocks into 3D groups, the VOLUME 11, 2023 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ BM3D algorithm [10] combines the features of NSS and frequency domain sparsity to achieve the excellent denoising performance. The WNNM algorithm [11] uses the low rank property of self-similar blocks to restore clear images. Traditional denoising methods have achieved certain results, however, they are difficult to reach the same level of denoising effect as deep learning methods. Therefore, there is still room for improvement in the existing traditional image denoising methods. Inspired by image fusion technology, we propose a hybrid denoising model on the basis of several traditional denoising methods, including BM3D, WNNM and NSST. Our model can outperform ADNet [12], DnCNN [13] and FFDNet [14] on tests of SSIM values. Aiming at the image damaged by Gaussian noise, the gradient domain guided filtering is used to obtain the main information of the denoised image processed by the BM3D algorithm and the texture information of the denoised image processed by combing the WNNM and NSST algorithms. The gradient domain guided filtering fuses these information to get a clear image with preserved edge details, smooth inner and outer regions. Our innovation lies in the reasonable use of traditional algorithms, as well as the design of the improved soft threshold algorithm in NSST domain. Our main contributions are summarized as follows: 1) We propose an advantage fusion denoising method that handles the classic color and grayscale images degraded with varying levels of Gaussian noise. 2) We propose an improved soft thresholding in NSST denoising process to overcome the defects of soft thresholding and hard thresholding.
3) The experimental results show that our method is competitive with traditional methods and deep learning methods on PSNR and SSIM values.
The remainder of this paper is organized as follows: Section II introduces the related work. Section III presents the related methods needed in this paper. Our proposed model is concretely described in Section IV. Extensive experimental results are shown in Section V, followed by an ablation experiment in Section VI. Section VII concludes the results of this paper.

II. RELATED WORK A. TRADITIONAL DENOISING METHODS
Spatial filtering denoising methods obtain the noise-free image by spatially averaging adjacent pixels, which may blur the image edges. Neighborhood filters [15] remove the image noise by calculating the average value of pixel gray values that are close enough to the center of nearby pixels. Bilateral filters [16] consider spatial proximity and pixel value similarity at the same time to achieve edge preserving denoising, which may result in gradient reverse. With advantages of less computation and no gradient reverse, the guided filtering [5] can achieve the effect of smoothing the image flat area while preserving the image boundary. By introducing the edge perception weight, the gradient domain guided filtering [6] incorporates the first-order edge-aware constraints into the guided filtering, which can smooth the flat area better than guided filtering. Zhu et al. [17] used the gradient domain guided filtering to fuse multi-scale infrared and visible light images. By the multi-scale exposure fusion, Fei et al. [18] performed the gradient domain guided filtering to generate highquality images. Considering the case that the input image and the guidance image are identical, Yin et al. [7] established a more general model for a set of guided filters by means of changing the regularization in the classic guided filtering. Wu et al. [8] proposed a deep learning-guided filter layer for joint upsampling, which can be integrated with convolutional neural networks (CNNs) and jointly optimized by an endto-end training. In the field of image hazing, the gradient domain guided filtering is used to refine the dual transmission maps using the dark channel and atmospheric light [19].
In this paper, we utilize the gradient domain guided filtering to combine the advantages of denoised images obtained by different methods. Frequency domain denoising methods, such as wavelet denoising, ridgelet denoising, curvelet denoising, contourlet denoising and shearlet denoising, remove noise by reducing the noise coefficient. However, these methods may introduce Pseudo-Gibbs artifacts into the denoised image. As an improvement of shearlet transform, NSST has translation invariance that can suppress Pseudo-Gibbs artifacts [20]. Easley et al. [21] proposed a hard threshold NSST denoising method. Shahdoosti and Khaya [22] used the variable-splitting sparse unblending method and augmented Lagrangian classifiers for image denoising in the NSST domain. Goyal et al. [23] proposed an image denoising method on the basis of morphological filtering and bitonic filtering in the NSST domain. As a matter of fact, NSST denoising will cause insufficient noise removal while retaining image details. How to improve the PSNR and SSIM values for denoised image is a significant problem about NSST denoising. The threshold functions, such as the hard thresholding and soft thresholding, play a critical role in the area of frequency domain denoising. Hard thresholding can preserve the local features and edges of the image well, but it will produce Pseudo-Gibbs phenomenon and ringing effect [24]. Soft thresholding has a constant bias, which will affect the closeness between the reconstructed image and the real image [25]. Therefore, how to construct a threshold function with neither constant deviation nor oscillation is very critical. This paper proposes an improved soft threshold algorithm in NSST domain to solve these problems.

B. DEEP EARNING DENOISING METHODS
With the rise of deep learning, deep neural networks have achieved great success in image denoising. Jain and Seung [26] proposed the convolutional neural networks (CNN) for natural image denoising. He et al. [27] proposed the residual neural network (Resnet) to solve the problem about the network degradation with the increase of network depth. Zhang et al. [13] proposed a deep convolutional neural network (DnCNN) for image denoising by utilizing residual learning. By combining the model-based method and discriminant learning method, Zhang et al. [28] proposed a fast and efficient CNN denoiser, named y RCNN, which can solve the inverse problem in the low-level vision. Tai et al. [29] proposed a denoising framework for persistent memory networks that integrates short-term and long-term memory to capture different levels of information. Rem et al. [30], proposed a deep residual network embedded depth-wise separable convolution. A residual dense neural network (RDUNet) for image denoising [31] which consists of densely connected convolutional layers, was introduced to avoid the vanishing gradient problem and speed up the learning process.

A. WEIGHTED NUCLEAR NORM MINIMIZATION
The WNNM algorithm exploit the non-local self-similarity to estimate the latent clean image form a noisy image. Firstly, the WNNM algorithm divides the noisy image x into overlapping local patches. For a local patch, the WNNM algorithm searches for its non-local similar patches across the image using the block matching method [10]. By stacking each group of non-local similar patches into a matrix, the image x is divided into a sequence of non-local similar blocks, denoted by Y 1 x, Y 2 x, · · · , Y N x. In order to retain the image structure more reasonably, the WNNM algorithm computes a low-rank approximation matrix L i of each block Y i x by assigning different weights to different singular values in the nuclear norm. Finally, aggregating Y 1 x, Y 2 x, · · · , Y N x forms the latent clean image. In brief, the model of the WNNM algorithm is as follows where ∥L i ∥ ω i * = i ω i,j σ i,j represents the weighted kernel norm, ω i,j is the non-negative weight, σ i,j is the j-th singular value of the matrix L i , and δ i is the penalty factor.
LetS i be a diagonal matrix of the same dimension as S i with the j-th diagonal element as: Then the closed solution of the problem (1) is L i = U iSi V T i . Additionally, the penalty factor δ i is computed by where σ e = √ 2, ε is a small constant to avoid dividing by zero, α i denotes the standard deviation of the diagonal elements of the matrix S i .  filter (SF) to achieve multi-scale decomposition and multidirectional decomposition. NSST decomposes an image into a low frequency subband and a set of high frequency subbands with different scales. Figure 1 shows a schematic diagram of a three-layer NSST.
Aiming at the discontinuity of the hard thresholding and the constant deviation of the soft thresholding, we propose an improved soft threshold algorithm in NSST domain, namely, where T a is the selected threshold coefficient, C and C s are respectively NSST coefficients of the noisy image and the denoised image.
Computing the threshold T a is the key ingredient of the improved soft threshold algorithm. Let σ ′ be the standard deviation estimated by classical Monte Carlo estimation, namely, where median(·) represents the operation of calculating the median. To improve the accuracy of standard deviation σ ′ , we use the empirical equation as follows By the estimated standard deviation σ , we determine the threshold T a by T a = kσ.
where k is the threshold scale factor.

C. GRADIENT DOMAIN GUIDED FILTERING
Assume that there is a local linear relationship between the guidance image and the output image, the guided filtering explicitly calculates the output image by the guidance image. More specifically, let r (p ′ ) be a local window with a radius r at the center p ′ , let X p and G p respectively denote the local input image and guidance image in the window r (p ′ ). By introducing the regularization factor λ, the goal of guided filtering is to compute the linear coefficients a p ′ and b p ′ by minimize the following local loss function Then the coefficients a p ′ and b p ′ yields the local output image Z p by The gradient domain guided filtering incorporates the explicit first-order edge sensing constraints into the guided filtering, which can improve the ability of the guided filtering in the aspect of the edge information preservation. For any pixel p ′ , using its local variances of 3×3 window 1 (p ′ ) and (2r +1)× (2r + 1) window r (p ′ ), the gradient domain guided filtering introduces the edge perception weight G (p ′ ) defined by where N is the number of image pixels, ε is a small σ G,1 (p ′ ) constant. Without loss of generality, we use χ (p ′ ) to explain the definition about the function χ(·). For the guidance image G, let σ G,1 (p ′ ) and σ G,r (p ′ ) respectively denote the standard deviation of the pixel values in the windows 1 (p ′ ) and r (p ′ ), then define Further, the gradient domain guided filtering introduces the parameter γ p ′ as follows where µ χ,∞ is the mean of the data in the set By introducing the regularization factor λ, the gradient domain guided filtering minimizes the following loss function to get the linear coefficients a p ′ and b p ′ , which yields the output image by whereā p ,b p are respectively the mean values of a p ′ and b p ′ in the window r (p ′ ). In order to improve the image visual effect, we compute the enhanced image by fusing the input image and output image, namely, where m is the enhancement coefficient.

IV. PROPOSED MODEL
There are two severe challenges in traditional image denoising methods, namely, preserving detailed information to obtain the excellent visual quality and filtering uniform regions to improve the signal-to-noise ratio. Therefore, distinguishing between edge details and smooth regions is undoubtedly the most important link in the process of image denoising. As we all know, the BM3D algorithm can better preserve some details. However, experimental results show that the BM3D algorithm will still be likely to result in over-smoothness for the denoised image. Compared with the BM3D algorithm, the denoising effect of the NSST algorithm is less obvious. Nevertheless, the NSST algorithm can avoid the appearance of Pseudo-Gibbs phenomenon. Our proposed model makes use of the gradient domain guided filtering to combine the advantages of the BM3D and NSST algorithms. Besides, we present an adaptive iterative NSST algorithm based on the improved soft thresholding to overcome the shortcomings of soft thresholding and hard thresholding. The frame diagram of our proposed model is shown in Figure 3. On the one hand, we use the BM3D algorithm to obtain the first denoised image, which will serve as the input image of the gradient domain guided filtering. On the other hand, we use the WNNM algorithm to deal with the given noisy image. We perform NSST operation for the denoised image obtained by the WNNM algorithm. Hence, we get the corresponding coefficients of the low frequency subband and high frequency subbands of different scales. Concretely, we choose four layer NSST decomposition.
The first layer is the low frequency subband, which was successively handled by the guided filtering and improved soft thresholding with the threshold scale factor k = 0. The  other layers are the high frequency subbands which were conducted by the improved soft thresholding, where the threshold scale factor k is set to be 40 for the second and third layers, k is set to be 60 for the fourth layer.
In the following, we will describe the implementation details about the adaptive iterative NSST algorithm based on the improved soft thresholding. The objective function of the adaptive iterative NSST algorithm is given by where f represents the image polluted by noise, u is the denoised image, ∇u NSST is the gradient of the image u in NSST domain. Table 1 demonstrates the processes of the adaptive iterative NSST algorithm based on the improved soft thresholding. More specifically, the image is decomposed into the low frequency subband and high-frequency subband by NSST. We use the improved soft thresholding to remove the noise in low frequency subband and high-frequency subbands. In addition, we use the guided filtering to enhance the lowfrequency details. We repeat the above operations until the value of objective function (17) has been locked. Table 2 presents a complete description of our proposed algorithm, which is the core content of the whole algorithm.

A. EXPERIMENTAL SETUP
To carry out the experiments, we use MATLAB (2020a version) software on a computer with specifications: 8GB RAM, Windows 11 operating system, and Intel core i7 processor. In order to verify the performance of our proposed model, we compare the performance of our proposed method with several state-of-the-art denoising methods, including NSST, BM3D, DnCNN, FFDNet and ADNet. In the case of the models selected for comparison, the contrast results were evaluated in all datasets using their respective pre-trained models and the source code of their corresponding authors. We select both the peak signal-to-noise ratio (PSNR) and structural similarity ratio (SSIM) as two of the most popular image quality metrics to evaluate the image denoising effect of different algorithms. The highest PSNR and SSIM results for each image with each noise level is highlighted in bold.
The basic parameters about the WNMM denoising are as follows: the size of each patch is set to 7 × 7, the number of nonlocal similar patches in each group is 60.
The parameters associated with the NSST denoising are described briefly below. The number of decomposition layers for NSST is set to 4. The number of the high-frequency images in each layer is set of [4,4,5,5], and the num- ber of directions corresponding to each layer is set to [32,32,16,16].
For the gradient domain guided filtering, the window radius, regularization factor and enhancement coefficient are respectively fixed by r = 1, λ = 0.09, m = 0.6.

B. GRAYSCALE IMAGE DENOISING
Set12 contains 12 grayscale images (see Figure 4), the size of image 1 through image 7 is 256 × 256, and the size of image 8 through image 12 is 512 × 512, which are used to test the images that were added Gaussian white noise with variances 0.01, 0.03 and 0.05. Table 3 records the PSNR values of the competing methods for each image in Set12 dataset. Our proposed model achieves the highest PSNR in most cases. On average, our proposed model respectively outperforms BM3D, NSST, DnCNN, FFDNet and ADNet by 1.44dB-2.10dB, 0.61dB-0.84dB, 0.21dB-0.49dB, 0.05dB-0.42dB and 0.12dB-0.39dB for different noise levels, In Figure 5 and Figure 6, we select the hair above the eye in Parrot image and the wig area of Man image to compare the visual quality. Although FFDNet algorithm can achieve the highest PSNR value for some images,   FFDNet may over-smooth more textures in the hair area of image Parrot and Man, as shown in Figure 5 and Figure 6, and our method can retain the hair texture without blurring.

C. COLOR IMAGE DENOISING
As shown in Figure 7, we select 20 classical color images as the test images. The size of image 1 through image 13 is 500 × 500, and the size of Image 14-20 is 512 × 512. The classic images not only have abundant structure and streak feature, but also possess smooth areas that blend reasonably with image details. Similar to grayscale images, the noisy color images are obtained by adding Gaussian white noise with variances of 0.01, 0.03, and 0.05.
Due to page limit, we only choose Girl image, Hat image and Lena image for comparing the denoising details. From Figure 8, we can see that our method can retain the textures in curly hair over the forehead and the makeup around the eye. Figure 9 illustrates that our method better restores both the letters and background information on the hat. In Figure 10, the denoised image by our method possesses the stripes on the hat while the other denoised images have lost parts of the stripes.

VI. ABLATION STUDY
To verify the effectiveness of our proposed model, we perform the ablation experiment in the following cases.  olding and soft thresholding. Table 7 shows the denoising results of gray images from the Set12 dataset by adding Gaussian noise with variance 0.03.

A. THE EFFECTIVENESS OF THE WNNM ALGORITHM
We conduct the WNNM algorithm before NSST denoising since NSST denoising method can not remove the noise in low frequency, and the WNNM algorithm can compensate the weaknesses of NSST denoising method, which makes Case 0 perform better than Case 1 in PSNR, as shown in Table 7.

B. THE EFFECTIVENESS OF THE GRADIENT DOMAIN GUIDED FILTERING
From Case 2 and Case 3, we know that the results fused by principal component analysis and guided filtering are both weaker than our proposed model because the gradient domain guided filtering, possessing the capacity of first-order edge     image. Although the effect of NSST denoising alone is not obvious, the denoised image by NSST retains a lot of detailed edges.

C. THE EFFECTIVENESS OF THE IMPROVED SOFT THRESHOLDING
From Case 0, Case 5 and Case 6, we can see that the improved soft thresholding in NSST domain can improve the denoising effect compared to the hard thresholding and soft thresholding.

VII. CONCLUSION
This paper has proposed a joint BM3D and NSST image denoising model, which uses the gradient domain guided filtering to improve the denoising effect of traditional denoising algorithms. The improved threshold algorithm we propose can retain the image details while denoising. Our method has been tested on different classical images and compared with state-of-the-art methods. The experimental results show that our method is effective in recovering noise-free images and enhancing the image details. Our method is suitable for high performance denoising for most grayscale and color images. For classic images polluted by Gaussian noise with vary levels, our proposed algorithm achieves gains in PSNR and SSIM over several deep learning denoising methods. Future work lies in the adaptability improvement of our proposed algorithm.