Fuzzy color video filtering technique for sequences corrupted by additive Gaussian noise
Introduction
Image denoising has various applications, such as image processing, computer vision, medicine, and satellite imaging, where the main purpose of the denoising procedure is to filter or remove undesired data (noise) from a color image and/or video sequences. There are many different reasons why a noise appears, such as non-uniform lighting, random fluctuations in an object׳s surface orientation and texture, sensor limitations, non-ideal transmission, and interference. Noise affects not only the performance of an image in a specific problem but also its perceived quality [1], [2], [3], [4], [5], [6].
The most common type of noise encountered in practice is additive noise, which is generally assumed to be a stochastic process with a zero-mean Gaussian distribution and variance and, in most cases, is spatially independent [7], [8], [9], [10], [11], [12]. In the case of additive noise, every pixel of an image is considered corrupted. There are other types of noise, such as speckle noise, common in ultrasonic and SAR imaging, and impulsive noise [5], [13], [14], [15], [16].
Numerous techniques for the filtering of different types of noise have been proposed, among which the most important characteristic consists of finding an adaptive algorithm that considers the local information of texture, edges and other features of an image [17], [18], [19], [20]. Additionally, the output pixels are processed using efficient filtering based on weighting averages of the corrupted image pixels located in the vicinity of a central pixel that should be denoised [21], [22], [23].
An important difference between image and color video sequence filtering is that in video applications, it is possible to take into account information from a number (past and/or future) of frames for better pixel denoising in the current frame [24], [25], [26]. However, the principal obstacle encountered when two or more frames are processed together for noise removal is the possible existence of local motions between different frames, which may introduce motion blur and ghosting artifacts in the presence of a moving sensor or during object motion [27], [28], [29], [30], [31], [32]. However, there exists a high level correlation between the consecutive frames in a video sequence when local temporal motions can be considered very small. This correlation between consecutive frames presents an excellent opportunity to increase the number of pixels that are highly similar, gathering them during the spatio-temporal filtering stage, leading to a better denoising quality for video sequences [33], [34], [35].
There are several approaches for filtering videos and still images, among which are (a) processing based on domain transformation, basically wavelet techniques; (b) fuzzy-based techniques; (c) filtering via sparse representation; and (d) the nonlocal means (NLM) filters. These algorithms have shown sufficiently good performance results in removing additive noise and exhibiting good preservation of edges, textures, sharpness and the chromatic properties of the filtered color image or video sequence. In the next paragraph, we present a short review of the most promising of these techniques.
In the wavelet domain (technique a), the noise is distributed across coefficients; nevertheless, most of the video sequence information is contained in the largest coefficients [27], [36], [37], [38]. The wavelet-based video encoders and filters that include motion estimation techniques perform equally well at reducing the noise level while preserving image features [39].
A motion estimation obtained by a video codec in the proposed WMVCE video denoising filter (used here as comparative filter) was designed by Jovanov et al. [27]. The motion estimation is applied as an input for the coding scheme. This approach is compared with the performance of two other state of-the-art filters, SEQWT and WST, in terms of visual quality as well as PSNR values for approximately 1 dB.
Saeedi et al. [37] report a wavelet shrinkage algorithm based on fuzzy logic for enhancing wavelet coefficient information. This method employs the inter-channel correlation as a fuzzy feature for improving the denoising performance, presenting better results than applying denoising in each channel separately.
Yin et al. [39] report a linear minimum mean squared-error (LLMMSE) spatiotemporal filter (used here as comparative technique) with adaptive motion compensation. The spatiotemporal adjacent homogeneous pixels that correctly match the current pixel are included in the filtering support for the noise reduction capability [40]. The filter achieves higher levels of PSNR criteria with respect to similar techniques, such as 2D-LLMMSE, TA, and 3D-LLMMSE.
An effective color video denoising algorithm CIFIC proposed by Jingjing et al. in [41] exploits both the inter-color and inter-frame correlation in the video to find a common motion field for RGB components and combines the current noisy observation as well as the inter-frame and inter-color predictors to obtain the LMMSE denoised estimate. Experimental results verify the effectiveness of filter in color noise reduction when compared with other state-of-the-art algorithms.
Paras et al. [42] have designed a new effective technique for Gaussian noise reduction while preserving relevant features employing wavelet transforms with a locally adaptive patch-based (LAPB) thresholding, demonstrating superiority of the proposed method, when compared to well-known state-of-the-art denoising methods.
Dong et al. [43] present an effective image denoising method with bivariate shrinkage threshold in a dual contourlet transform (DCT) domain. It has been experimentally confirmed that novel method can preserve better in the internal structure and the edge details, texture information than other methods.
Fuzzy-based techniques (b) mentioned as the second approach, for the reduction of additive Gaussian noise and other types of noises in color video sequences have been successfully applied as well [25], [44], [45], [46], [47], [48]. The advantage of fuzzy filtering techniques is in the efficient preservation of image features (edges, chromaticity characteristics, texture, etc.) as corrupted pixels are being denoised. Fuzzy logic filters are established by Membership Functions and Fuzzy Rules based on human knowledge and the ability to adapt their characteristics to the current image and noise realizations. These types of techniques are biased by the assumptions the human is making about a pixel/noise model, which in some cases cannot be accurate. In other words, they work well only with the problems they have learned or been tested with [49], [50], [51]. Modern theoretical approaches in denoising have been validated using experiments principally based on the possibility of gathering more samples for similar objects, then applying traditional statistical methods, which depend on the image/noise model, perhaps with some modifications [48], [52], [53]. The principal problem here is how to measure and employ the similarity of a group of objects in an image.
One considerable weakness of traditional filter models is that they are not able to preserve edges or image details in an appropriate manner: edges, which are recognized as discontinuities in the image, are blurred. Fuzzy filters on the other hand can handle with edges and image details in a much better way than other models.
A wide variety of filters are only designed to deal with grayscale images, nevertheless, color image filters have not received much attention in the state-of-the-art. Some proposals reported in the recent years have extended a grayscale denoiser to handling color signal applying it independently in each color channel. But, this technique fails to utilize the correlation between different color components. Fuzzy denoising techniques are able to take advantage from the inter-color correlation to improve over the single-component approaches.
Rosales-Silva et al. [48] and Kravchenko et al. [53] report three-dimensional space–time filtering algorithms (used here as comparative filter) based on fuzzy sets theory. This denoising approach uses the gradient pixel values of the R, G, and B channels and the angular differences between pixels in each color channel, finally employing them in the filtering of the consecutive frames on the basis of fuzzy logic rules. Simulation results have confirmed the superiority of these techniques in comparison with other state-of-the-art algorithms, such as VMMKNN and VGVDF_G.
Melange et al. [52] report a fuzzy-logic-based filter (used here as comparative technique) focusing on an additive white Gaussian noise model. The method is first explained in the pixel domain for grayscale video sequences and is later extended to the wavelet domain and to color video sequences. Numerical simulation results show that the noise is efficiently removed by the proposed fuzzy motion and detail adaptive video filter (FMDAF) but it presents worse results against the VBM3D framework.
Nadernejad et al. [54] propose a novel algorithm where both, fuzzy filtering and partial differential equation methods, are combined to reduce the different kinds of noises. The performance of the algorithm is evaluated showing a better performance compared to the existing methods, PDEs based algorithm, wavelet based algorithms and NLM method.
The third approach (c) in searching similar objects uses the sparse color image representation based on the assumption that images admit a sparse decomposition over a redundant dictionary [7], [10], [55], [56], [57], [58], [59].
Mairal et al. [55] introduce a novel algorithm for color image denoising, inpainting, and demosaicing. The framework is based on learning models modifying the grayscale K-SVD for sparse color image representation. This algorithm learns correlations between the different R, G, and B channels and provides better results compared with independent analysis of color channels.
Dong et al. [57] improve the performance of sparse representation based on image restoration, suppressing sparse coding noise. To this end, the authors exploit the image nonlocal self-similarity. The extensive experiments validate the state-of-the-art performance of the proposed nonlocally centralized sparse representation (NCSR) algorithm.
Dabov et al. [58] present a VBM3D (used here as comparative filter) video filtering method created on highly sparse signal representation in the local 3D transform domain. This method is performed via a 3D data array called “group” by stacking together blocks found to be similar to the currently processed block. The grouping is executed as a spatio-temporal predictive-search block matching. Each 3D group is filtered by a 3D transform-domain shrinkage. Better performance is obtained by using a two-step algorithm where an intermediate estimate is produced by grouping and collaborative hard-thresholding. Then, both algorithms are used to improve the grouping and apply the collaborative Wiener filtering. A recently published paper [35] from the same authors goes a step further by proposing the VBM4D framework, which stacks similar 3D spatio-temporal volumes instead of 2D blocks to form 4D data arrays using the same approach.
Pogrebnyak et al. [59] propose a multiscale approach that aggregates the outputs of DCT filters having different overlapped block sizes. Later, a two-stage denoising procedure is performed that presumes the use of the multiscale DCT-based filtering with hard thresholding at the first stage and a multiscale Wiener DCT-based filtering at the second stage.
Finally, the fourth approach (d) uses the Non Local Means (NLM) filtering techniques that are employed in the denoising of images corrupted by different types of noises. The NLM approach uses all the possible self-predictions and self-similarities that an image can provide, determining the pixel weights for denoising the corrupted image. Several promising algorithms based on this technique are reported in the literature [22], [23], [60], [61].
Jin et al. [23] introduce new nonlocal operators to interpret the (NLM) filter as a regularization of the corresponding Dirichlet functional. The experiments show that the new nonlocal operators yield a better interpretation of the nonlocal means filter.
Jingjing et al. [62] design an advanced color image denoising scheme called multichannel NLM fusion where the inter-channel color correlation is used via constructing and fusing multiple NLM spanning all three channels. Simulations verify the improvement of the framework compared to the single-channel NLM filtering.
Buades et al. [63] propose a new measure method (used here as comparative filter) for noise to evaluate and compare the performance of digital image denoising methods. First, the method computes and analyzes a wide class of denoising algorithms, namely, the local smoothing filters. Second, the authors propose a new algorithm from non-local means based on a non-local averaging of all pixels in an image.
In paper [56], Mairal et al. report a new image model combining the NLM and sparse coding approaches to image restoration into a unified framework. Quantitative and qualitative experiments have shown that the proposed algorithm outperforms the state-of-the-art image denoising techniques.
The novel curvelet based NLM method proposed by Wu et al. [64] determines the similarity of pixels in the noisy image based on the various levels of the multi-scale curvelet transform demonstrating better performance of the framework against the state-of-art NLM denoising.
Generally, the principal objective of different approaches to denoising mixed noises for still images (additive and impulsive) consists of the detection and suppression of random spikes followed by filtering out additive noise [7], [10], [12].
Cai et al. [7] propose a filtering method divided into two phases. First, the outlier candidates are identified. In the second phase, the image is deblurred and denoised simultaneously using essentially the outlier-free data.
Huang et al. [10] report a filtering technique where both impulsive noise and mixed impulse plus Gaussian noises affect the image or video sequence. The authors propose the modified total variation minimization scheme to regularize the deblurred image and fill unsuitable values for noisy image pixels while they are detected by median-type filters.
Camarena et al. [12] use a simple method based on fuzzy logic to filter color images corrupted with mixed impulse and Gaussian noise. In a later section, the best of the above-mentioned techniques are compared with the proposed novel filtering approach.
Modern theoretical approaches in denoising that have been validated using experiments are principally based on a possibility to gather more samples for similar parches. Then, the methods use sophisticated statistical methods, which depend on image/noise model, maybe with some modifications. The principal problem here is how to measure and employ the similarity of group of objects in a color image or in several neighboring color frames of video sequence. Classical example is the mentioned VBM3D technique [58]. Proposed in this paper approach exploits the similar ideas in searching similar parches that permit to gather more samples for processing together color channels for a video frame; following, more samples should be found gathering neighboring frames of a video where the local motions in different frames should be adjusted. Principal difference of novel approach with most of the algorithms reviewed above consists in usage the designed Fuzzy Rules in all filtering steps where the similar objects are learned.
The filtering scheme proposed in this paper is based on fuzzy sets. The proposed Fuzzy-Multichannel-Additive-Noise-Suppression (FMANS) filter is designed to suppress additive noise in color video sequences while preserving image features such as edges, chromaticity characteristics, texture, and fine details. The designed technique consists of three principal filtering stages: spatial, spatio-temporal and spatial postprocessing. In contrast to other state-of-the-art algorithms, in the first spatial stage, eight gradient values in different directions for pixels located in the vicinity of a central pixel as well as the interchannel correlation between analogous pixels in the red (R), green (G) and blue (B) bands are used, where the degree of noise contamination is estimated by employing novel fuzzy rules that allow a better preservation of image features (textures, edges, sharpness, etc.). In the second spatio-temporal filtering stage, two consecutive video frames are analyzed together, where the similarity measure between the consecutive frames is calculated, allowing the formation of an interframe sample of pixels. Possible local motions between consecutive frames are estimated using a block matching procedure in eight directions to perform interframe filtering. In the spatial postprocessing filtering stage, the edges and plain areas of a current frame are distinguished using different filtering procedures. In this paper, we have designed two variants of the proposed fuzzy filter named FMANS_1 and FMANS_2, that depend on sliding windows used. Additionally, the novel FMANS_H filter, which uses hybrid processing connecting the “fuzzy ideology” of the FMANS technique and multiscale Wiener DCT-based filtering is also performed.
Numerous simulation results obtained for color video sequences with different texture characteristics, edges, color properties, and local motions have confirmed the superiority of the novel 3D fuzzy frameworks over other filtering techniques, including VBM3D, in terms of objective criteria (PSNR, MAE, NCD, and SSIM) as well as the subjective perception of the human visual system with respect to different color video sequences. Additionally, the novel frameworks use only two frames concurrently that reduce the computational processing time and memory requirements.
The outline of this paper is as follows. In Section 2, the proposed filtering approach is explained. Numerical experiments and graphical results are given in Section 3. Finally, conclusions are presented in Section 4.
Section snippets
Fuzzy color video filtering
Fuzzy filters are based on the observation that noise causes a small fuzzy derivative, whereas a large fuzzy derivative is caused by the presence of fine details or edges. Fuzzy rules are applied in each direction and consider variations that can occur, such as local motions and variations in edges and fine features [16], [25], [45], [46]. Additionally, fuzzy rules are able to distinguish between noisy pixels, edges, fine image features, and plain areas. These distinctions allow for the main
Performance criteria
To evaluate the effectiveness of the proposed filter in the suppression of additive Gaussian noise and image detail preservation, the novel filter was compared with other known techniques. The filtered frames were evaluated according to the following objective criteria [1], [2], [3], [4], [71], [72].
We employed the PSNR (Peak Signal-to-Noise Ratio) to characterize the noise suppression capabilities of the proposed technique and the MAE (Mean Absolute Error), which measures the level of
Conclusions
The designed 3D filtering frameworks FMANS_1, FMANS_2, and the FMANS_H for the denoising of color video sequences corrupted by additive Gaussian noise are presented. The techniques are based on fuzzy logic theory in application to the basic and several related gradient values along different directions and inter-channel correlations, employing the previous and current temporal frames. The denoising approach consists of three principal filtering stages: spatial, spatio-temporal and spatial
Acknowledgments
This work is supported by Instituto Politecnico Nacional (Mexico) and Consejo Nacional de Ciencia y Tecnologia (Mexico).
Volodymyr Ponomaryov received the Ph.D. degree in 1974 and D.Sci. in 1981. His research interests include signal/image/video processing, pattern recognition, and real-time filtering. He has also been a promoter of 38 Ph.D.s. He has published of about 500 international scientific and conference papers, and also 23 patents of ex USSR, Russia and Mexico, and five scientific books.
References (74)
- et al.
A survey of perceptual image processing methods
Signal Process.: Image Commun.
(2013) Restoration of images corrupted by Gaussian and uniform impulsive noise
Pattern Recognit.
(2010)- et al.
Stractured descriptive experiment design regularization based enhancement on fractional SAR imagery
Signal Process.
(2013) - et al.
A design framework for hybrid approaches of image noise estimation and its application to noise reduction
J. Visual Commun. Image Represent.
(2012) - et al.
A mixed noise image filtering method using weighted-linking PCNNs
Neurocomputing
(2008) - et al.
Image/video denoising based on a mixture of Laplace distributions with local parameters in multidimensional complex wavelet domain
Signal Process.
(2008) - et al.
A fuzzy filter for the removal of random impulse noise in image sequences
Image Vis. Comput.
(2011) - et al.
Efficient video denoising based on dynamic nonlocal means
Image Vis. Comput.
(2012) - et al.
Generalized multihypothesis motion compensated filter for grayscale and color video denoising
Signal Process.
(2013) - et al.
Feature-based motion compensated interpolation for frame rate up-conversion
Neurocomputing
(2014)
Efficient modified directional lifting-based discrete wavelet transform for moving object detection
Signal Process.
A comparison of different block matching algorithms for motion estimation
Proc. Technol.
A new wavelet-based fuzzy single and multi-channel image denoising
Image Vis. Comput.
Heuristic wavelet shrinkage for denoising
Appl. Soft Comput.
Fuzzy directional FD filter for impulsive noise reduction in colour video sequences
J. Visual Commun. Image Represent.
Fuzzy reasoning-based directional median filter design
Signal Process.
Two-step fuzzy logic-based method for impulse noise detection in colour images
Pattern Recognit. Lett.
Peer group switching filter for impulse noise reduction in color images
Pattern Recognit. Lett.
Image denoising using new pixon representation based on fuzzy filtering and partial differential equations
Digital Signal Proces.
Curvelet based nonlocal means algorithm for image denoising
AEUE—Int. J. Electron. Commun.
An adaptive bilateral filter based framework for image denoising
Neurocomputing
Perceptual image quality assessment based on structural similarity and visual masking
Signal Process.: Image Commun.
Color image processing and applications
Monograph
Color Image Processing: Methods and Applications
Multidimensional Signal, Image, and Video Processing and Coding
Handbook of Image and Video Processing, Communications, Networking, and Multimedia
Radar Systems Analysis and Design Using MATLAB
Two-phase approach for deblurring images corrupted by impulse plus Gaussian noise
Inverse Probl. Imaging
Image denoising using mixtures of projected Gaussian scale mixtures
IEEE Trans. Image Process.
Fuzzy peer groups for reducing mixed Gaussian-impulse noise from color images
IEEE Trans. Image Process.
Fast image restoration methods for impulse and Gaussian noise removal
IEEE Signal Process. Lett.
A simple fuzzy method to remove mixed Gaussian-impulsive noise from colour images
IEEE Trans. Fuzzy Syst.
An efficient denoising architecture for removal of impulse noise in images
IEEE Trans. Comput.
Efficient adaptive fuzzy-based switching weighted average filter for the restoration of impulse corrupted digital images
IET Image Process.
Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising
IEEE Trans. Image Process.
Automatic estimation and removal of noise from a single image
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (10)
On the reduction of mixed Gaussian and impulsive noise in heavily corrupted color images
2023, Scientific Reports3D Filtering of Images Corrupted by Additive-Multiplicative Noise
2020, Doklady MathematicsA Review On Video Denoising Methods
2019, 2019 International Conference on Recent Advances in Energy-Efficient Computing and Communication, ICRAECC 2019
Volodymyr Ponomaryov received the Ph.D. degree in 1974 and D.Sci. in 1981. His research interests include signal/image/video processing, pattern recognition, and real-time filtering. He has also been a promoter of 38 Ph.D.s. He has published of about 500 international scientific and conference papers, and also 23 patents of ex USSR, Russia and Mexico, and five scientific books.
Hector Montenegro-Monroy received the Ph.D. degree in communication and electronics from the National Polytechnic Institute of Mexico in 2014. His research interests include image/video processing, denoising, and real-time processing.
Francisco J. Gallegos-Funes received the Ph.D. degree in communications and electronics from National Polytechnic Institute of Mexico in 2003. He is currently an Associate Professor in the Mechanical and Electrical Engineering Higher School from the same institute. His areas of scientific interest are signal and image processing, filtering, steganography, pattern recognition, and real-time applications.
Oleksiy B. Pogrebnyak received his Ph.D. degree from Kharkov Aviation Institute (now National Aerospace University), Ukraine, in 1991. Currently, he is with the Center for Computing Research of National Polytechnic Institute of Mexico. His research interests include digital signal/image filtering and compression, and remote sensing.
Sergiy N. Sadovnychiy received his Ph.D. degree from Kharkov Aviation Institute (now National Aerospace University), Ukraine, in 1991. Currently, he is with the National Petroleum Institute of Mexico. His research interests include digital signal/image processing, communication and acoustic diagnostic.