Image denoising with two-dimensional zero attracting LMS algorithm 2 boyutlu sıfıra çeken LMS algoritmasıyla görüntü iyileştirme

Abstract


Introduction
The earliest studies on gradient-based adaptive algorithms may be traced back to more than six decades.In 1950, the leastmean-square (LMS) algorithm was discovered by Widrow and Hoff [1] in their investigation of the adaptive linear element machine.It is a stochastic gradient algorithm in that it iterates each tap weight of the filter [2].Adaptive filters are widely used in many applications including system identification, echo cancellation [3], channel equalization, etc.
In many engineering and mathematics problems, sparsity is a popular topic [4].In system identification and communication applications, the system may be in sparse nature [5] such as acoustic echo cancellation and network cancellation applications.Channel impulse response is frequently sparse due to high-speed data transmission, which will be dominated by a small number of high elements or taps [6], [7].Many sparse LMS-type algorithms have been introduced lately to exploit sparsity.Gui et al. [8] applied the l0-LMS algorithm in order to estimate sparse channels to increase the accuracy of the estimate.In [9] a zero-attracting LMS (ZALMS) algorithm includes  1 -norm as a penalty term.In [10], a zero-point attraction projection uses l1 norm.In [11],   -norm is combined with  1 -norm.Another penalty term is  0 -norm was used [12] and its performance analysis is done [13].A method in [14] is introducing sparse representations in overcomplete transforms, based on minimization of the  0 -norm.
In [15], the first two-dimensional adaptive filtering algorithm which can be applied as an adaptive line enhancer is presented.It is a direct extension of the one-dimensional (1D) LMS algorithm.Two-Dimensional (2D) adaptive filters are applied to the problems of image denoising as in [16].The performance of the algorithms is improved by changing the step-size and updating the filter coefficients partially.In [15], a system identification application has been developed.It has linear adaptive filters, whose coefficients are refreshed in view of the standardized LMS calculation.In [16], another 2-D adaptive filtering application on equalization that employs the optimum (minimum mean-square-error (MSE) was proposed.A similar adaptive filter application algorithm has proposed in [17].A different approach is in [18], two-dimensional adaptive filtering application is based on metaheuristic algorithms named artificial bee colony algorithm.
BM3D is one of the state-of-the art algorithms that have been used in image denoising which is based on the idea of sliding the image windows and searching the blocks includes a similarity with the processed one at that moment in three dimensional transform domain.The algorithm has two basic steps: The first one is about estimating the denoised image by hard thresholding and the second one is based on Wiener filtering where it is used for the original degraded image and the estimated version that handled from the initial step.[19].
Whatever remains of the paper is sorted out as following: Section 2 includes the proposed algorithm and the derivation of the convergence analysis.In Section 3, the data reuse patterns are discussed.In Section 4, simulations about image deconvolution are provided and discussed.At last, conclusions are drawn.

Materials and methods
Before deriving the proposed algorithm, we try to highlight the main role of the zero-attracting term in the one-dimensional 1D-ZALMS algorithm.

Review of the 1D-ZALMS algorithm
In the 1D-ZALMS algorithm, the cost function   is defined by including the instantaneous squared error into the  1 -norm penalty of the weight vector, where   is the instantaneous error given by   =   −     .  is the desired response,  is a small positive parameter.  is the filter weight vector.The update equation of ZALMS filter is defined as: where  = .It controls the zero-attraction term.The sign function  (. ) is defined as: The term sgn[  ] forces zero attraction on zero or close-tozero coefficients.According to this; when the filter weight coefficient is positive, the value will decrease and vice versa.

Extending to the 2D case
The update equation of the 2D-ZALMS adaptive filter weight matrix can be given as: Where   is the updated weight matrix of size N×N.The estimate of the true 2D instantaneous gradient of {  2 }, with respect to   is can be further expended to The ZALMS algorithm estimates the instantaneous gradient; [10] the gradient of the squared error of an iteration.
where r and c denote the row and column indices, respectively.Substituting (7) in (4) and simplifying yields the weight update equation as: where the filter output is defined as: The filter weights   and the input data matrix   can be reshaped into 1D form.During the k th iteration, both column vectors   and   are respectively According to these, (8) can be written as Hence, using ( 10) and ( 11) the 1D ZALMS can be used as a 2D-ZALMS algorithm in (12) and the update equation becomes: where I is an N×N identity matrix.The step-size selection criteria of the proposed algorithm will be same with the standard LMS algorithm.

The data reuse patterns
The update equation of the ZALMS algorithm can be written in 2D form as: Where   ( 1 ,  2 ) is  ×  weights matrix,  1 = 0,1, … ,  − 1 and  2 = 0,1, … ,  − 1.The filter weights and the input data can be reshaped into 1D, respectively, by: For 2D applications, the update of the filter is done along the horizontal and vertical directions.The filtering of the image is implemented as follows: a subimage with the same size as of the filter itself is selected at each step by moving horizontally from left to right one pixel at a time till the last pixel in the respective row.Subsequently, the same process is repeated for each of the following row.This process will be repeated till the last pixel in the last row of the image.Results of the convolution of the filter and the subimage at each step will be calculated and recorded to obtain the filtered image.Some possible ways of data reuse are shown in Figure 1.In the scheme shown in Figure 1(a), all the data should be used as shown.However, with the same considered mask size, another pattern as in Figure 1(b) with almost 63% of the pixels may be used.And in case of sparse images (where most of the pixels are zeros) [19] the performance of this pattern is very comparable to that in Figure 1(a) but with lower computation time.3 Experimental results

Subjective experiments
In all of the conducted experiments in this study, we considered different 8-bit 256×256 grayscale sparse and non-sparse images where the number of pixels with zero/near zero values is changing.Different noise were applied to the images, namely: additive white Gaussian noise (AWGN), salt & pepper and speckle noise [20].Comparisons for image denoising performance of the proposed 2D-ZALMS algorithm against BM3D and the conventional 2D-LMS algorithm were conducted.
In the first experiment, Barbara and Lena images were degraded with additive white Gaussian noise AWGN with zero mean and 0.01 variance in Figure 2 and Figure 3, respectively.Parts (c) to (e) in aforementioned figures show the denoised images using the proposed 2D-ZALMS, BM3D and 2D-LMS algorithms, respectively.The quality of 2D-ZALMS based denoised image is clearly superior to the one obtained using the other two algortihms.
In order to test the performance of the proposed 2D-ZALMS algorithm under different noise types, the first experiment is repeated with the same parameters using two different noise.
Second experiment uses speckle noise with zero mean and 0.01 variance (Figure 4 and Figure 5 ).
Step size was chosen as µ=0.001 for all algorithms.Figure 4 and 5 show that while BM3D algprithm is givining a blurred denoised images, the proposed 2D-ZALMS algorithm still performs the same as the 2D-LMS algorithm but with lower computation time.
In third experiement, Images were degraded by salt & pepper noise with probability of 0.1 and µ=0.001 for all algorithms (Figure 6 and Figure 7).Results are similar to the ones obtained in second experiment.

Objective experiments
To further emphasize the time consumption criteria of the proposed 2D-ZALMS and 2D-LMS, a comparison between the two algorithms is conducted under AWGN, speckle and salt&pepper noise using different images with 2D-ZALMS using axial configuration.The results are shown in Table 1, Table 2 and Table 3, respectively.On average, the proposed 2D-ZALMS algorithm is 15.96, 17.1 and 17.45 times faster than the 2D LMS algorithm under the three noise types, respectively.This can be justified by the use of the axial configuration for the filter in 2D-ZALMS algorithm instead of the rectangular configuration which is used by the filter of the 2D LMS algorithm.Another objective test was conducted using the signal to noise ratio SNR, the peak signal to noise ratio PSNR and the SSIM.
The Structural Similarity (SSIM) Index quality assessment index is based on the computation of luminance, contrast and structural terms.The overall index is a multiplicative combination of these three terms.A simplified version of SSIM is given by (, ) = (2    +  1 )(2  +  2 ) where   ,   ,   ,   , and   are the local means, standard deviations, and cross-covariance for images x and y.C1 and C2 are very small constants which are included to avoid instability when the denominator is very close to zero.
PSNR and SNR are metrics for measuring the denoised/original image quality and usually is shown in the logarithmic decibel scale.
Given a denoised/recovered  ×  8-bit image x and its original version y.The formulae for SNR and PSNR metrics are given, respectively, as where mean square error MSE is calculated by Comparisons between the 2D-LMS, BM3D and the proposed 2D-ZALMS algorithms for different images and AWGN with different noise variances are provided in Table 4. Results on these tables indicate that with the increase in the noise level, the PSNR, SNR and SSIM values between the denoised image and the noisy image is decreasing.This is due to the increase in the MSE value which is inversely proportional to SNR and PSNR values.Generally, the proposed 2D-ZALMS is performing very close to the 2D LMS in terms of PSNR, SNR, SSIM and MSE values.As the higher PSNR value indicates that the algorithm performance is better than the others.However, it is clear from the table that, the PSNR, SNR and SSIM values for BM3D are relatively high compared to other algorithms.

Discussion and conclusion
A new two-dimensional algorithm presented in this work is introduced for improving the 2D-LMS algorithm performance.
The new 2D zero-attracting least mean square (2D-ZALMS) adaptive filter is improving the performance by imposing a sparsity aware penalty term ( 1 -norm) into the cost function of the original 2D-LMS algorithm.Moreover, two data reuse configurations namely; rectangular and axial configurations were explained and utilized to boost the filtering process.
Images corrupted by different noise types with different parameters such as variance were used.Both algorithms showed comparable results both subjectively by human eye inspection and/or objectively using PSNR metric.Still the results showed that the proposed 2D-ZALMS algorithm is faster than the 2D-LMS algorithm by a high gain factor under different noise types and parameters with image size=256×256,  = 0.001,  = 10 −4 , and  = 10.We compared the proposed algorithm with one of the state of the art algorithm BM3D as well.According to the experimental results, it has a blurry effect on the images.and the objective similarity metrics are higher than the 2D-ZALMS.

Table 1 :
Time consumption comparison in seconds between the 2D LMS and proposed 2D-ZALMS algorithms using different images under AWGN ( 2 = 0.01).

Table 2 :
Time consumption comparison in seconds between the 2D LMS and proposed 2D-ZALMS algorithms using different images under Speckle noise ( 2 = 0.1).

Table 3 :
Time consumption comparison in seconds between the 2D-LMS and proposed 2D-ZALMS algorithms using different images under salt&pepper noise ( = 0.1).