An Iterative Weighted-Mean Filter for Removal of High-Density Salt-and-Pepper Noise

: Salt-and-pepper noise, which is often introduced by sharp and sudden disturbances in the image signal, greatly reduces the quality of images. Great progress has been made for the salt-and-pepper noise removal; however, the problem of image blur and distortion still exists, and the e ﬃ ciency of denoising requires improvement. This paper proposes an iterative weighted-mean ﬁlter (IWMF) algorithm in detecting and removing high-density salt-and-pepper noise. Three steps are required to implement this algorithm: First, the noise value and distribution characteristics were used to identify the noise pixels, e ﬀ ectively improving the accuracy of noise detection. Second, a weighted-mean ﬁlter was applied to the noisy pixels. We adopted an un-ﬁxed shape symmetrical window with better detail preservation ability. Third, this method was performed iteratively, avoiding the streak e ﬀ ect and artifacts in high noise density. The experimental results showed that IWMF outperformed other state-of-the-art ﬁlters at various noise densities, both in subjective visualization and objective digital measures. The extremely fast execution speed of this method is quite suitable for real-time processing.


Introduction
Images are often contaminated by salt-and-pepper noise, due to various reasons, including transmitting images in defective channels or taking pictures with a faulty sensor [1]. The salt-and-pepper noise is classified as the impulse noise and comprises a random set of pixels with extreme intensity and usually in significant contrast with the neighboring noise-free pixels [2]. The existence of salt-andpepper noise, even with a low noise density, can seriously affect visual perception and image analysis, and thus, needs to be detected and removed.
By giving higher predefined weights to the selected pixels in the filtering window before calculating the median, the weighted filter can retain more details than MF. Through automatically adjusting the filtering window size based on the local noise content, the adaptive median filter can prevent image From the discussions above, the current filters usually perform well on images with the noise density within a specific range, but some filtering distortion problems, such as streak effects and oversmoothing, are still common, especially under high noise density conditions. In addition, the discussed filters achieve the denoising effect by increasing the complexity of the algorithm, which inevitably decreases the efficiency and increases the time of the process. In real-time applications, the denoising speed of the filters is required to be largely improved.
To overcome such problems, a robust noise removal method, called iterative weighted mean filter (IWMF) is proposed in this paper. IWMF is based on the effective weighted-mean framework and adopts an un-fixed shape symmetrical window, which has better detail protection ability. With the combination of such algorithms, both the denoising effect and real-time performance can be greatly improved.
The remaining parts are as follows: Section 2 introduces the scheme of the iterative weightedmean filter. Section 3 presents the experimental results of the proposed method compared with other state-of-the-art methods. Finally, our conclusions are presented in Section 4.

Scheme of the Iterative Weighted-Mean Filter
Three stages are involved in the proposed model: The noise detection stage, noise removal stage, and iterative denoising stage. Figure 1 is the flow chart for the proposed model, with details about the flow chart. These three stages are described in detail in the following sections.

Stage 1: Noise Detection
At this stage, the pixels are divided into two categories-there are noisy pixels and noise-free pixels. It is reasonable to assume that the pixels with the extreme maximum and the extreme minimum values are noises [2,15,19,20].
For a pixel g in the image I, let I(g) be its pixel value, and R be its noise recognition matrix. The noise recognition matrix R can be written as:

Stage 1: Noise Detection
At this stage, the pixels are divided into two categories-there are noisy pixels and noise-free pixels. It is reasonable to assume that the pixels with the extreme maximum and the extreme minimum values are noises [2,15,19,20].
Symmetry 2020, 12,1990 4 of 12 For a pixel g in the image I, let I(g) be its pixel value, and R be its noise recognition matrix. The noise recognition matrix R can be written as: where R(g) = 0 represents the "noise-free pixel" and R(g) = 1 represents the "noise pixel". This direct noise detection algorithm cannot accurately recognize the noise pixels in certain images, such as images with high-brightness and artificial pictures. Images with such characteristics often have special regions called extreme intensity flat regions, in which all pixels have extreme intensity. Noise-free pixels in these regions will also be mistakenly detected as noise and removed, and then be severely distorted; artifacts and dark patch-like surfaces on the images cannot be avoided. Therefore, a robust detection algorithm is required to improve the denoising performance.
Take a white extreme intensity flat region (WFR) that is corrupted by salt-and-pepper noises as an example. The fundamental characteristics of the pixels include: 1.
All pixels in this region have extreme intensity.

2.
About half of the noise pixels take the intensity 255, so the total number of pixels with an intensity of 255 is greater than the pixels with an intensity of 0. In other words, pixels with an intensity of 255 are the majority.
For a pixel g in the image I, let I(g) be its pixel value and W 5 (g) be its neighborhood window of size 5 × 5. If I(g) = 255 or I(g) = 0, label g as a noise candidate pixel; otherwise, label g as a noise-free pixel. For a noise candidate pixel g candidate , if all pixels in W 5 (g candidate ) with intensity 0 or 255, and N 255 > T, where N 255 is the number of pixels with an intensity of 255, and the optimal value of T is 20, then g candidate is in the white extreme intensity flat regions. If I(g candidate ) = 255, it is considered a noise-free pixel, otherwise it is a noise pixel.
Such a procedure is also applicable to black extreme intensity flat regions. If a pixel g is in a black extreme intensity flat region and I(g) = 255, then it is considered a noise-free pixel; otherwise, it is a noise pixel. Therefore, mathematically, the improved detection algorithm can be given as 0, 0<I(g)<255 0, g is in WFR and I(g) =255 0, g is in BFR and I(g) =0 1, otherwise (2) where WFR is white extreme intensity flat regions, and BFR is black extreme intensity flat regions. R(g) = 0 represents the "noise-free pixel" and R(g) = 1 represents the "noise pixel".

Stage 2: Noise Removal
In this stage, a decision-based weighted mean filter with an adaptive window is used for denoising. The pixels that are labeled noise-free (R(g) = 1) in the noise detection stage remain unchanged, and the noises (R(g) = 0) are replaced by the restored intensity. The steps for noise removal are as follows.

Selection of Filtering Window
Generally, selecting a small filtering window can preserve image details better, while a large filtering window can adapt to a higher noise density [11,22]. To adaptively change the filtering windows according to the noise density, and thus, improve the process efficiency, a symmetrical window with an unfixed shape was adopted in this paper. The size was adaptively changed according to the noise density. This kind of window can better reflect the local correlation between pixels, and has a stronger detail protection ability than the traditional square window.
Let r be the Euclidean distance between g and the other pixels in W 5 (g), and we define five different windows r i (r 1 , r 2 , ... r 5 ) according to the value of r, as shown in Figure 2a. Let W be the filtering window, and we choose W according to the following three rules: 1.
If the number of noise-free pixels in W 5 (g) is greater than 3, then set W = r 1 as the candidate filtering window. If the number of noise-free pixels in W is less than 3, then let W = r 1 + r 2 . By analogy, increase W by r i (r 1 , r 2 , ... r 5 ) until the number of noise-free pixels selected exceeds 2.

2.
If the number of noise-free pixels in W 5 (g) is 1 or 2, then let W = W 5 (g). 3.
If all pixels in W 5 (g) are noise, then a suitable filtering window cannot be obtained. In this case, the pixel needs to be further detected by method 2 in Section 2.2.2.
the noise pixel value with the restored value, and set R(g) = 0.
Here, D is the noise-free pixels group in window W, (i, j) is the coordinate of pixel g, (r, s) is the relative coordinate with (i, j) as the center, and w(r, s) is the weighted function (4), which can be obtained from 2 2 r s r s  ( , )=1/ w (4) In general, the value of a pixel is closer to the value of its neighboring pixels than the values of far pixels. The median filter takes the median value of pixels in the window as the restoration value of noise pixels, and cannot reflect the difference of the spatial position of pixels. To overcome such a defect, a distanced weighted method is proposed for denoising in this paper. The weighted coefficients are formed to be a center symmetric weight matrix whose values gradually decrease with the increase of distance, as shown in Figure 2b. Such a treatment can well reflect the local correlation between pixels, thereby reducing the loss of detail and avoiding image blur. 2. If g is in the extreme intensity flat regions, then the recovery step is performed according to the formula (5) and set R(g) = 0; otherwise, the pixel is processed in Stage 3.

Stage 3: Noise Removal by Iterative Approach
For the images with high-density noises, as the pixels of a small neighborhood may all be destroyed, it is difficult to calculate the restoration value based on noise-free pixels in a small neighborhood, and thus, a wider window of the pixels is required. However, too large of a filtering

Calculation of Noise Pixels Restored Value
The calculation of noise restored value is based on weighted mean filtering and the distribution characteristics of the pixels. For each noise, the pixels g, if the number of noise-free pixels in W 5 (g) is greater than 0, then the method in 1 is used for denoising; otherwise, the method in 2 is used.

1.
In the spatial filtering theory, corrupted pixels can be restored using the normalized weighted mean of all pixels in the neighborhood. The noise restored value can be calculated as (3). Replace the noise pixel value with the restored value, and set R(g) = 0.
Here, D is the noise-free pixels group in window W, (i, j) is the coordinate of pixel g, (r, s) is the relative coordinate with (i, j) as the center, and w(r, s) is the weighted function (4), which can be obtained from In general, the value of a pixel is closer to the value of its neighboring pixels than the values of far pixels. The median filter takes the median value of pixels in the window as the restoration value of noise pixels, and cannot reflect the difference of the spatial position of pixels. To overcome such a defect, a distanced weighted method is proposed for denoising in this paper. The weighted coefficients are formed to be a center symmetric weight matrix whose values gradually decrease with the increase of distance, as shown in Figure 2b. Such a treatment can well reflect the local correlation between pixels, thereby reducing the loss of detail and avoiding image blur.

2.
If g is in the extreme intensity flat regions, then the recovery step is performed according to the formula (5) and set R(g) = 0; otherwise, the pixel is processed in Stage 3.

Stage 3: Noise Removal by Iterative Approach
For the images with high-density noises, as the pixels of a small neighborhood may all be destroyed, it is difficult to calculate the restoration value based on noise-free pixels in a small neighborhood, and thus, a wider window of the pixels is required. However, too large of a filtering window may result in blurring and unnecessary distortion. If the large window does not contain noise-free pixels, some filters replace the intensity of noisy pixels with the previously processed pixel or the mean of processed pixels in the neighborhood, such as References [17,31], which will lead the restoration image to have a streaking effect or artifacts. Such problems can be solved by using an iterative filter to produce higher quality images.
For each pixel g in image I, the iterative procedure can be described as follows.

1.
For each pixel g with R(g) = 1, process g by the method proposed in stage 2.

2.
If R is not a zero matrix, repeat 1 until R becomes a zero matrix, but use the last reconstruct image as the input image. Otherwise, leave it unchanged. If all pixels in the image are noisy pixels, then the procedure should stop.
Taking a noise image, for example, the denoising procedure can be described, as shown in Figure 3. The image segment obtained from "Lena. jpg" is chosen for illustration, and it is corrupted by salt-and-pepper noise with a density of 70%.
Symmetry 2020, 12, x FOR PEER REVIEW 6 of 13 window may result in blurring and unnecessary distortion. If the large window does not contain noise-free pixels, some filters replace the intensity of noisy pixels with the previously processed pixel or the mean of processed pixels in the neighborhood, such as References [17,31], which will lead the restoration image to have a streaking effect or artifacts. Such problems can be solved by using an iterative filter to produce higher quality images.
For each pixel g in image I, the iterative procedure can be described as follows.
1. For each pixel g with R(g) = 1, process g by the method proposed in stage 2.
2. If R is not a zero matrix, repeat 1 until R becomes a zero matrix, but use the last reconstruct image as the input image. Otherwise, leave it unchanged. If all pixels in the image are noisy pixels, then the procedure should stop.
Taking a noise image, for example, the denoising procedure can be described, as shown in Figure  3. The image segment obtained from "Lena. jpg" is chosen for illustration, and it is corrupted by saltand-pepper noise with a density of 70%.

Simulation Results
To evaluate the performance of the proposed algorithm, we used several pictures ( Figure 4) and 100 natural images ( Figure 5) from the UC Berkeley dataset [33]. All experiments were performed using a PC with InteI) I(TM) i5-9893 CPU @ 3.10 GHZ, 16GB RAM. The program codes were written in C++. In experiments, the performance of the proposed filter was compared with several state-of-the-art filters, such as ASWMF [12], AFMF [9], DAMF [10], DBA [19], AWMF [26], TSF [20], FSMMF [31], MDBMF [17], and ERMI [28], in terms of the PSNR (peak signal to noise ratio), the SSIM (structural similarity index), visual perception, and the average processing time. The PSNR and SSIM are defined as follows: Symmetry 2020, 12, 1990 where f is the original image, g is the denoising image, and m and n are the length and width of the image, respectively; u f and u g are the mean of f and g, respectively; σ f and σ g are the standard deviation of f and g, respectively; σ fg is the covariance between u f and u g ; C 1 and C 2 are constants used to maintain stability, where L = 255 is the dynamic range of pixel values, and K 1 = 0.01, K 2 = 0.03.

Evaluate by Visual Perception and Quantitative Measurements
Figures 6-8 show the restored images using IWMF and several filters for the images Lena and Test018 corrupted by noise density 0.9 and Test006 corrupted by noise density 0.8. It can obviously be seen from the reconstructed images that the ASWMF and AFMF failed to remove noises thoroughly. The DAMF demonstrated good noise removal capabilities; however, the restored images still contain some subtle noisy dots. The AWMF, TSF, and FSMMF outperformed the previous filters in removing noise thoroughly, but introduced a blur effect on the restored image. The ERMI over smoothed the images and lost some details for the high-density noise images. The DBA and MDBMF were affected by a streaking effect to an extent, which degraded the quality of the restored images. Comparatively, the IWMF showed superior performance in terms of noise removal and detail preservation.
In Figure 7, due to a certain number of pixels with extreme intensity values in the original image, the results generated by SAMF, ASWMFF, FSMMF, and ERMI were highly unsatisfactory. The MDBMF replaces a pixel with the previous pixel when there is still no noise-free pixel in 5 × 5 neighborhoods, which caused the restored image to suffer from the streaking effect. The TSF caused a large number of pixels with an intensity of 255 to be destroyed. The IWMF performed well in removing the noise in extreme intensity flat regions and was able to generate the highest quality images.

Evaluate by Visual Perception and Quantitative Measurements
Figures 6-8 show the restored images using IWMF and several filters for the images Lena and Test018 corrupted by noise density 0.9 and Test006 corrupted by noise density 0.8. It can obviously be seen from the reconstructed images that the ASWMF and AFMF failed to remove noises thoroughly. The DAMF demonstrated good noise removal capabilities; however, the restored images still contain some subtle noisy dots. The AWMF, TSF, and FSMMF outperformed the previous filters in removing noise thoroughly, but introduced a blur effect on the restored image. The ERMI over smoothed the images and lost some details for the high-density noise images. The DBA and MDBMF were affected by a streaking effect to an extent, which degraded the quality of the restored images. Comparatively, the IWMF showed superior performance in terms of noise removal and detail preservation.
In Figure 7, due to a certain number of pixels with extreme intensity values in the original image, the results generated by SAMF, ASWMFF, FSMMF, and ERMI were highly unsatisfactory. The MDBMF replaces a pixel with the previous pixel when there is still no noise-free pixel in 5 × 5 neighborhoods, which caused the restored image to suffer from the streaking effect. The TSF caused a large number of pixels with an intensity of 255 to be destroyed. The IWMF performed well in removing the noise in extreme intensity flat regions and was able to generate the highest quality images.

Evaluate by Visual Perception and Quantitative Measurements
Figures 6-8 show the restored images using IWMF and several filters for the images Lena and Test018 corrupted by noise density 0.9 and Test006 corrupted by noise density 0.8. It can obviously be seen from the reconstructed images that the ASWMF and AFMF failed to remove noises thoroughly. The DAMF demonstrated good noise removal capabilities; however, the restored images still contain some subtle noisy dots. The AWMF, TSF, and FSMMF outperformed the previous filters in removing noise thoroughly, but introduced a blur effect on the restored image. The ERMI over smoothed the images and lost some details for the high-density noise images. The DBA and MDBMF were affected by a streaking effect to an extent, which degraded the quality of the restored images. Comparatively, the IWMF showed superior performance in terms of noise removal and detail preservation.
In Figure 7, due to a certain number of pixels with extreme intensity values in the original image, the results generated by SAMF, ASWMFF, FSMMF, and ERMI were highly unsatisfactory. The MDBMF replaces a pixel with the previous pixel when there is still no noise-free pixel in 5 × 5 neighborhoods, which caused the restored image to suffer from the streaking effect. The TSF caused a large number of pixels with an intensity of 255 to be destroyed. The IWMF performed well in removing the noise in extreme intensity flat regions and was able to generate the highest quality images.
Tables 1-3 provide the comparative results of the IWMF and other filters in terms of the PSNR and SSIM using the Lena, Test018, Boat, and Training098 images corrupted with the noise densities ranging from 0.1 to 0.9. The proposed IWMF showed the best noise removal performance of all the filters, according to the values of PSNR and SSIM. The FSMMF and DBA were barely satisfactory, and the DBA produced the lowest PSNR and SSIM in most ranges of the noise density. The ERMI Symmetry 2020, 12, 1990 8 of 12 was effective in removing low-density salt-and-pepper noises but was not suitable for high-density noises removal. The values of PSNR and SSIM for the TSF, MDBMF, DAMF, and ERMI filters were close to each other but lower than the values of IWMF. We further inferred from Figure 9 that the IWMF produced the highest values of PSNR and SSIM at noise densities ranging from 0.1 to 0.9, which indicates that the IWMF gave the best performances in preserving image details and generating the highest quality images.
Symmetry 2020, 12, x FOR PEER REVIEW 8 of 13      IWMF produced the highest values of PSNR and SSIM at noise densities ranging from 0.1 to 0.9, which indicates that the IWMF gave the best performances in preserving image details and generating the highest quality images.

Evaluate by Computational Time
The CPU elapsed time is an important index to evaluate the performance of the filter. Table 4 illustrates the average CPU processing time of the discussed seven filters for 100 natural images with size 481 × 321. These images were corrupted by salt-and-pepper noise with densities ranging from 0.1 to 0.9. For each method, the execution time increased with the increasing noise density, except for AWMF. The AFMA had the longest processing time, due to the complexity of denoising strategy. The denoising strategies of the ASWMF and AFMF meant they processed more pixels than the other methods, which prolonged the execution time.
The execution time of the MDBWF and FSMMF were close to each other, and at a medium level. The ERMI and IWMF, by using the mean-based filtering instead of median-based filtering and without sorting pixel values, showed the best performance in the computing and processing time. In addition, the use of small windows also played an important role in improving efficiency.

Evaluate by Computational Time
The CPU elapsed time is an important index to evaluate the performance of the filter. Table 4 illustrates the average CPU processing time of the discussed seven filters for 100 natural images with size 481 × 321. These images were corrupted by salt-and-pepper noise with densities ranging from 0.1 to 0.9. For each method, the execution time increased with the increasing noise density, except for AWMF. The AFMA had the longest processing time, due to the complexity of denoising strategy. The denoising strategies of the ASWMF and AFMF meant they processed more pixels than the other methods, which prolonged the execution time. The execution time of the MDBWF and FSMMF were close to each other, and at a medium level. The ERMI and IWMF, by using the mean-based filtering instead of median-based filtering and without sorting pixel values, showed the best performance in the computing and processing time. In addition, the use of small windows also played an important role in improving efficiency.

Discussion
In this paper, we proposed an efficient salt-and-pepper noise removal filter, the iterative weighted mean filter (IWMF). In the noise detection stage, the extreme value and the distribution characteristics of the noisy image were used to identify the noise pixels, which overcame the problem of the inaccurate noise detection of traditional switching filters. In the noise removal stage, a weighted mean filter was applied to restore the noisy image, and this was performed iteratively. Comparatively, this noise removal technique effectively avoided the problem of detail blur and streaking effects, especially with high-density noise. Extensive simulation results demonstrate that the proposed filter outperformed other existing state-of-the-art filters in visual perception and quantitative measurements. Due to the efficient filter framework and low computational complexity, IWMF is very suitable for real-time implementation.  (13)).