Image denoising via an improved non-local total variation model

: This study presents an improved non-local total variation (NLTV) model by using the block-matching and three- dimensional filtering (BM3D) algorithm for image denoising. First, the preprocessed image is obtained with the BM3D algorithm. Then, taking the place of the noisy image, the preprocessed image is used to construct the fidelity term of the energy functional and calculate the weight function in NLTV regularisation term. Finally, the energy functional is solved by the split Bregman algorithm. Experimental results demonstrate that the proposed model achieves better denoising performance than the original NLTV model in the visual appearance and objective indices, especially for the highly degenerated images. In addition, the proposed model can effectively suppress the appearance of the false information in the flat region, which overcomes the problem faced by the BM3D algorithm.


Introduction
In the field of image processing, image denoising is a very critical preprocessing step for the segmentation, reconstruction, recognition and so on. Recently, the partial differential equation (PDE) based methods [1][2][3] have been widely applied to remove noise. Based on the PDE theory, Rudin et al. [4] proposed a classical total variation (TV) denoising model, which can effectively preserve image edges. However, the TV model easily causes staircase and over-smoothing effects in the denoised image. To suppress staircase effect, Zhang et al. [5] proposed an adaptive TV model based on a new indicator, that is, the local energy derived from the steerable filter; Yuan et al. [6] proposed a regional spatially adaptive TV super-resolution algorithm, which performed the different regularisation strength with the difference curvature in the flat and structural regions of the image. To overcome the oversmoothing effect, Liu et al. [7] proposed a texture-preserved adaptive TV model based on the structure tensor. In addition, the TV regularisation, as a smoothing prior, is also applied to remove multiplicative noise [8].
Making full use of the non-local self-similarity in the image, Buades et al. [9] proposed a non-local means (NLM) denoising algorithm. Then, the grey theory [10], Monte Carlo theory [11], and fuzzy cluster theory [12] are introduced to further improve the denoising performance of the NLM algorithm. The non-local selfsimilarity prior is also applied to the sparse regularisation denoising model [13,14]. The most successful application of the non-local idea is the block-matching and three-dimensional filtering (BM3D) algorithm [15]. This algorithm has the excellent edge-preserving capability. However, it is easy to cause false textures in the denoised image. Meanwhile, the non-local idea has been applied to the PDE to process images. For instance, Yang et al. [16] proposed a non-local means theory-based Perona-Malik model for image denoising and Zhang et al. [17] proposed a class of non-local tensor telegraph-diffusion equations for coherence enhancement.
Combining the TV theory and the NLM theory, Gilbor and Osher [18] proposed non-local operators and a non-local TV (NLTV) denoising model. The non-local operators have been successfully applied to image recovery [19] and deconvolution and sparse reconstruction [20]. The NLTV model can not only inherit the edge-preserving advantage of the NLM algorithm, but also suppress the staircase effect. Liu and Huang [21] also presented a novel NLTV denoising scheme and this scheme was solved by a more stable split Bregman algorithm.
In this paper, we propose an improved NLTV model based on the BM3D algorithm for image denoising, in which the BM3D algorithm is used as a pre-filter to process the noisy image. On the one hand, the preprocessed image is used to construct the fidelity term of the energy functional. On the other hand, the preprocessed image is also used to calculate the weight function in the NLTV regularisation term. These two ways are applied to the NLTV model so as to improve the performance of edge preservation and the capability of noise removal.
The remainder of this paper is organised as follows: In Section 2, we describe the related works about the TV and NLTV denoising models. In Section 3, we introduce the BM3D algorithm, elaborate on the proposed NLTV model and present the corresponding numerical implementation. Experimental results are given in Section 4 and conclusions are presented in Section 5.

TV denoising model
Let u 0 be the observed noisy image and u be the denoised image. The energy minimisation equation of the classical TV denoising model [4] is as follows: where ∇ is the gradient operator, Ω denotes the image domain, and λ is a positive constant. The first term is the regularisation term to smooth noise, and the second term is the fidelity term to determine the approximation degree between u 0 and u. The Euler-Lagrange equation of (1) can be formulated as The gradient descent flow of (2) is as follows: and the corresponding local coordinate expression of (3) is where u ξξ is the second-order derivative of u in the tangent direction to the isophote lines. Equation (4) indicates that the TV model is an anisotropic diffusion equation. In this equation, the diffusion behaviour occurs only along the isophote direction and the diffusion degree is determined by 1/ | ∇u|. Therefore, edge features can be effectively preserved. However, this diffusion behaviour is imperfect for the flat region, which will cause the staircase effect.

NLTV denoising model
Inspired by the graph Laplacian and the NLM idea, Gilboa and Osher [18] defined variational framework-based non-local operators. Let Ω ⊂ ℜ 2 , x ∈ Ω and u(x) be a real function Ω → ℜ.
The non-local gradient ∇ NL u(x, . ) at the point x is defined as and the norm of ∇ NL u(x) is defined as where w(x, y) is the weight function, expressed as where G a denotes the Gaussian kernel with standard deviation α, h acts as a filtering scale parameter, and u 0 (x + ⋅ ) is the image patch centred at the pixel x. w(x, y) is symmetric, i.e. w(x, y) = w(y, x). Then, the non-local divergence of a vector p at x is defined as div NL p(x) = ∫ Ω ( p(x, y) − p(y, x)) w(x, y)dy (8) and the non-local Laplacian can be defined by Based on the above defined non-local operators, Gilboa and Osher [18] proposed the NLTV regularisation-based image denoising model, as follows: where the Lagrange multiplier λ balances the relationship between the regularisation term and the fidelity term. Compared with the classical TV model, the NLTV model can effectively suppress the staircase effect and maintain more image textures. However, the capabilities of edge preservation and noise removal are weak when the NLTV model is faced with the high noise level.

BM3D algorithm
Based on the combination of non-local idea and three-dimensional (3D) transform, Dabov et al. [15] proposed the BM3D algorithm for image denoising. This algorithm is realised by the basic estimate and the final estimate: (a) Basic estimate: First, group similar blocks together in a 3D array. Then, perform cooperative filtering for the formed group by hard-thresholding of the transform coefficients in the 3D transform domain. Last, aggregate the block estimates obtained by inverting the 3D transform to form the basic estimate of the true-image. (b) Final estimate: The process of this procedure is similar to the basic estimate. However, there are two major differences. One is that the basic estimate is used to perform the block-matching in the grouping process; the other is that the wiener filtering instead of the simple hard-thresholding is used to denoise the transform coefficients by using the basic estimate as the pilot signal.
Let BM3D( ⋅ ) denote the BM3D operation, then the denoised image obtained with the BM3D algorithm can be expressed as The BM3D algorithm has a remarkable advantage in feature preservation, which can effectively preserve fine textures and details. However, this algorithm will produce fake features in the flat region of the image.

Proposed NLTV model
The energy functional of the NLTV denoising model is composed of the fidelity term and the NLTV regularisation term. They play different roles in the denoising process. The fidelity term is used to preserve image features and the NLTV regularisation term is mainly used to smooth noise. In this paper, we improve the NLTV denoising model from the fidelity term and the regularisation term.
In the original NLTV model, the fidelity term aims at making the denoised image as close to the noisy image as possible to preserve structure features. However, in the noisy image, features and noise are very similar, which affects the choice of the Lagrange multiplier λ. If the value of λ is too large, the fidelity term will be propitious to preserve more features in the noisy image. At this time, the noise will also be left behind in the denoised image. Contrarily, if the value of λ is too small, the regularisation term will turn to be the dominant role. Then, detail features cannot be effectively preserved, which will result in a blurred denoised image. To overcome this deficiency, using u BM3D instead of u 0 , we construct a new fidelity term, as follows: Because the BM3D algorithm has a remarkable edge-preserving advantage, the image u BM3D contains more structure features derived from the noisy image. What's more, a lot of noise has been removed. Therefore, after applying u BM3D to the fidelity term, a large λ can be chosen to make the denoised image contain the features in u BM3D as much as possible, which is conducive to good edge-preserving. The weight function determines the denoising performance of the NLTV regularisation term. In the original weight function, the noisy image is used to calculate the weight between patches. In order to overcome noise interference, we use the preprocessed image u BM3D instead of the noisy image to calculate the weight, which can more accurately judge the similarity between patches. The more accurate the similarity weight is, the better the denoising capability of the NLTV regularisation term will be. The new weight function is presented as follows: Then, the non-local gradient can be calculated by Using the above non-local gradient, we propose a new regularisation term, as follows: Combining the fidelity term F new and the regularisation term R new , the proposed NLTV denoising model can be written as

Numerical implementation
Referring to the literature [20], the split Bregman algorithm is employed to solve the optimisation problem of the proposed method. First, replacing [∇ NL u] new with a new variable d, the minimisation problem (16) can be transformed into Introducing the Bregman variable b and employing the classical split Bregman iteration algorithm, (17) can be solved by (see (18)) where β is a positive regularisation parameter. The solution of (18) is obtained by performing an alternating minimisation process The optimality condition for u k + 1 can be obtained by using the variational method, as follows: According to the definitions of the non-local divergence and the non-local Laplacian, (22) can be rewritten as This subproblem can be solved with the Gauss-Seidel algorithm, and the corresponding iterative scheme is where the subscripts i and j denote the discretised points x and y, respectively. Thereinto, u k + 1, n = 0 is set to u k .
The other subproblem d k + 1 can be obtained by extending the soft-thresholding formula to the non-local case, as follows: (see (25)) The discretised schemes of d k + 1 and b k + 1 for the pixel i are as follows: For the proposed method, the iteration is terminated when meeting the following stopping criterion: where ε is a fixed threshold. u k and u k + 1 denote the image vectors at the kth and (k + 1)th iterations, respectively. The split Bregman algorithm for the proposed method is summarised in Algorithm 1. Calculate w new using (13).

Experiment data
In the experiments, the Woman image, Lena image, House image and the Cameraman image are applied to evaluate the effectiveness of the proposed method. The size of these images is all 256 × 256. For each image, the noisy images are generated by adding additive white Gaussian noise with different noise standard deviations to the original image. We compared the performance of the proposed method with six methods, including the TV model [4], NLM algorithm [9], NLTV model [18], BM3D algorithm [15], a typical dictionary learning based method named the K-SVD model [22] and a state-of-the-art denoising method named the weighted nuclear norm minimisation (WNNM) algorithm [23].

Parameter setting
In the proposed method, the parameters of the BM3D algorithm are set according to the recommendations in the original reference [15]. The size of the patch and the search window are 5 × 5 and 11 × 11, respectively. The filtering scale parameter h is associated with the noise standard deviation σ, which is empirically set to 0.2σ. The iteration number of the Gauss-Seidel algorithm is set to 2. λ and β are adjusted empirically to achieve a good performance, which are set to 1 and 0.01, respectively. The threshold ε in the stopping criterion is set to 10 −3 .

Objective evaluation index
We use the peak signal-to-noise ratio (PSNR) and the mean structural similarity (MSSIM) to evaluate the quality of the restored image. Assuming that f is the noise-free reference image, u is the restored image, and M × N is the size of f. The PSNR is employed to evaluate the grey value similarity, defined as PSNR = 10log 10 255 2 1 The MSSIM proposed by Wang et al. [24] is employed to evaluate the structural similarity (SSIM) for the whole image, formulated as where x j and y j are the image contents at the jth local window in the reference and the restored images, respectively, J is the number of local windows of the image, and the structural similarity SSIM(x, y) between image signals x and y is given by where μ x and μ y denote the mean of x and y, respectively; σ x and σ y denote the standard deviation of x and y, respectively; σ xy is the covariance of x and y; and c 1 = (k 1 L) 2 , c 2 = (k 2 L) 2 , where L is the dynamic range of the pixel values (255 for 8-bit greyscale images), k 1 = 0.01 and k 2 = 0.03 are constants.

Analysis of improvements
We conduct an experiment to, respectively, analyse the function of the new fidelity term and the new NLTV regularisation term from visual effect. Fig. 1 shows the denoising results of different models, including the classical NLTV model with the original fidelity term and the original NLTV regularisation term, the constructed model with the new fidelity term and the original NLTV regularisation term, and the proposed model with the new fidelity term and the new NLTV regularisation term. We can see that the constructed model can obtain clearer edges than the classical NLTV model, which means that the new fidelity term can improve the edge-preserving capability. We can also see that the proposed model can effectively suppress the noise in the flat region while the constructed model leaves the residual noise in its denoised image. This means that the new NLTV regularisation term is conducive to noise removal.

Experimental results
To visually analyse the advantages of the proposed method, Figs. 2-5 show the denoising results obtained by using different methods for the Woman image when σ = 25, the Lena image when σ = 35, the House image when σ = 50, the Cameraman image when σ = 70, respectively. Rows from top to down and columns from left to right: original image, noisy image, TV model, NLM algorithm, NLTV model, K-SVD model, BM3D algorithm, WNNM algorithm, proposed method.. As can be seen from these figures, the TV model suffers from the staircase effect, leading to the bad visual effect. This phenomenon is more severe for the high noise level. The NLTV model and the NLM algorithm have the similar edgepreserving capability. However, the NLTV model has a better denoising capability than the NLM algorithm. For the TV model, the NLM algorithm and the NLTV model, the capabilities of denoising and edge-preserving decline dramatically with the increase of the noise level. They cannot achieve a good balance between edge preservation and noise removal. The K-SVD model, BM3D algorithm, and WNNM algorithm can effectively preserve fine details. However, the K-SVD model produces blocky artefacts in its denoised image. The BM3D algorithm and WNNM algorithm also produce fake texture artefacts. What's worse, the artefacts are more obvious when the noise level is high. It is clearly found that the edge-preserving capability of the proposed method is similar to that of the BM3D algorithm and WNNM algorithm and is better than that of the other compared methods. More importantly, the proposed method can effectively suppress the appearance of artefacts while removing noise. We can also see that the denoised image obtained with the proposed method is more similar to the original image. The above results demonstrate that the proposed    Tables 1 and 2, respectively, where the bold values are the best in the line. As can be seen from these tables, the BM3D algorithm, WNNM algorithm and the proposed method obtain higher PSNR and MSSIM values than the TV model, NLM algorithm, NLTV model, and the K-SVD model. In most cases, the proposed method can obtain higher PSNR and MSSIM values than the BM3D algorithm. The PSNR and MSSIM values of the proposed method are very close to that of the WNNM algorithm. In some cases, the proposed method is better than the WNNM algorithm. These quantitative results, which are consistent with the visual effect, further prove the advantages of the proposed method.
Furthermore, taking the Cameraman image with the noise standard deviation σ = 70 as an example, we measure the execution time of the proposed method. The proposed method is implemented in MATLAB 7.6.0 on a personal computer equipped with Intel(R) Core(TM) i5-3450 CPU at 3.10 GHz and 4.00 GB RAM. The execution time of the BM3D algorithm, WNNM algorithm, and the proposed method is presented in Table 3. We can see that the proposed method takes more time than the BM3D algorithm. However, it takes less time than the WNNM algorithm. The future work will be concentrated on reducing the execution time of the proposed method by using the graphics processing unit (GPU) technology [25,26] without compromising its denoising performance.

BGA (Ball Grid Array) solder joint X-ray image denoising
In the defect detection of the BGA solder joint, the denoising is the basis of segmentation and recognition. The two BGA solder joint X-ray images are employed to further test the practicability of the proposed method. We compared the proposed method with the BM3D algorithm and WNNM algorithm. Figs. 6 and 7 show the denoising results of two BGA solder joint X-ray images, respectively. We can see that the BM3D algorithm, WNNM algorithm, and the proposed method can preserve important structure features. It is noticed that the evident streak artefacts exist in the denoised images obtained with the BM3D algorithm and WNNM algorithm. However, these artefacts can be effectively smoothed or reduced by the proposed method. This indicates that the proposed method has a better denoising capability than the BM3D algorithm and WNNM algorithm, which can improve the accuracy of defect detection for the BGA solder joint X-ray images.

Conclusions
In this paper, an improved NLTV model has been proposed for image denoising, which can better balance the contradiction between edge preservation and noise removal. In the proposed model, the improved fidelity term and the improved regularisation term are based on the preprocessed image that is obtained by employing the BM3D algorithm to process the noisy image. And the split Bregman algorithm is used to solve the energy functional of the proposed method. Experimental results confirm that the proposed method can not only effectively remove noise, but also preserve image structures. Compared with the relevant and stateof-the-art denoising methods, the advantages of the proposed method are more prominent for the high noise level.