Elsevier

Neurocomputing

Volume 155, 1 May 2015, Pages 225-246
Neurocomputing

Fuzzy color video filtering technique for sequences corrupted by additive Gaussian noise

https://doi.org/10.1016/j.neucom.2014.12.025Get rights and content

Highlights

  • The framework consists of spatial, spatio-temporal and spatial postprocessing filtering stages.

  • The proposed technique is based on fuzzy rules, gradient values, and interchannel correlations.

  • The framework suppresses additive noise over a wide range of intensities preserving edges and fine features.

  • The framework is extremely efficient in reproducing the chromatic characteristics of images.

  • The scheme outperforms existing state-of-the-art algorithms in terms of objective and subjective criteria.

Abstract

In this paper, a novel framework is presented for the denoising of color video sequences corrupted by additive Gaussian noise. The proposed technique consists of three filtering stages: spatial, spatio-temporal, and spatial postprocessing. During the first spatial stage, the gradient values in eight directions for pixels located in the vicinity of a central pixel, as well as the interchannel correlation between the analogous pixels in different color bands (RGB), are taken into account. These gradient values that estimate the level of noise contamination are employed using the designed fuzzy rules to preserve the image features (e.g., textures, edges, sharpness, and chromatic properties). In the spatio-temporal denoising stage, two consecutive video frames are filtered together, thereby yielding more information. Additionally, small local motions between consecutive frames are estimated using block matching procedure in different directions, gathering interframe samples with similar features for efficient denoising. In the final stage, the edge and plain areas in a current frame are separated for different spatial postprocessing denoising. Two variants of proposed fuzzy filter, depending on sliding windows, are proposed. Additionally, a hybrid fuzzy-Wiener denoising technique is performed employing the proposed filtering approach. Numerous simulation results confirm that these novel fuzzy frameworks outperform other state-of-the-art techniques in terms of objective criteria, as well as subjective visual perception in the various color sequences.

Introduction

Image denoising has various applications, such as image processing, computer vision, medicine, and satellite imaging, where the main purpose of the denoising procedure is to filter or remove undesired data (noise) from a color image and/or video sequences. There are many different reasons why a noise appears, such as non-uniform lighting, random fluctuations in an object׳s surface orientation and texture, sensor limitations, non-ideal transmission, and interference. Noise affects not only the performance of an image in a specific problem but also its perceived quality [1], [2], [3], [4], [5], [6].

The most common type of noise encountered in practice is additive noise, which is generally assumed to be a stochastic process with a zero-mean Gaussian distribution and variance and, in most cases, is spatially independent [7], [8], [9], [10], [11], [12]. In the case of additive noise, every pixel of an image is considered corrupted. There are other types of noise, such as speckle noise, common in ultrasonic and SAR imaging, and impulsive noise [5], [13], [14], [15], [16].

Numerous techniques for the filtering of different types of noise have been proposed, among which the most important characteristic consists of finding an adaptive algorithm that considers the local information of texture, edges and other features of an image [17], [18], [19], [20]. Additionally, the output pixels are processed using efficient filtering based on weighting averages of the corrupted image pixels located in the vicinity of a central pixel that should be denoised [21], [22], [23].

An important difference between image and color video sequence filtering is that in video applications, it is possible to take into account information from a number (past and/or future) of frames for better pixel denoising in the current frame [24], [25], [26]. However, the principal obstacle encountered when two or more frames are processed together for noise removal is the possible existence of local motions between different frames, which may introduce motion blur and ghosting artifacts in the presence of a moving sensor or during object motion [27], [28], [29], [30], [31], [32]. However, there exists a high level correlation between the consecutive frames in a video sequence when local temporal motions can be considered very small. This correlation between consecutive frames presents an excellent opportunity to increase the number of pixels that are highly similar, gathering them during the spatio-temporal filtering stage, leading to a better denoising quality for video sequences [33], [34], [35].

There are several approaches for filtering videos and still images, among which are (a) processing based on domain transformation, basically wavelet techniques; (b) fuzzy-based techniques; (c) filtering via sparse representation; and (d) the nonlocal means (NLM) filters. These algorithms have shown sufficiently good performance results in removing additive noise and exhibiting good preservation of edges, textures, sharpness and the chromatic properties of the filtered color image or video sequence. In the next paragraph, we present a short review of the most promising of these techniques.

In the wavelet domain (technique a), the noise is distributed across coefficients; nevertheless, most of the video sequence information is contained in the largest coefficients [27], [36], [37], [38]. The wavelet-based video encoders and filters that include motion estimation techniques perform equally well at reducing the noise level while preserving image features [39].

A motion estimation obtained by a video codec in the proposed WMVCE video denoising filter (used here as comparative filter) was designed by Jovanov et al. [27]. The motion estimation is applied as an input for the coding scheme. This approach is compared with the performance of two other state of-the-art filters, SEQWT and WST, in terms of visual quality as well as PSNR values for approximately 1 dB.

Saeedi et al. [37] report a wavelet shrinkage algorithm based on fuzzy logic for enhancing wavelet coefficient information. This method employs the inter-channel correlation as a fuzzy feature for improving the denoising performance, presenting better results than applying denoising in each channel separately.

Yin et al. [39] report a linear minimum mean squared-error (LLMMSE) spatiotemporal filter (used here as comparative technique) with adaptive motion compensation. The spatiotemporal adjacent homogeneous pixels that correctly match the current pixel are included in the filtering support for the noise reduction capability [40]. The filter achieves higher levels of PSNR criteria with respect to similar techniques, such as 2D-LLMMSE, TA, and 3D-LLMMSE.

An effective color video denoising algorithm CIFIC proposed by Jingjing et al. in [41] exploits both the inter-color and inter-frame correlation in the video to find a common motion field for RGB components and combines the current noisy observation as well as the inter-frame and inter-color predictors to obtain the LMMSE denoised estimate. Experimental results verify the effectiveness of filter in color noise reduction when compared with other state-of-the-art algorithms.

Paras et al. [42] have designed a new effective technique for Gaussian noise reduction while preserving relevant features employing wavelet transforms with a locally adaptive patch-based (LAPB) thresholding, demonstrating superiority of the proposed method, when compared to well-known state-of-the-art denoising methods.

Dong et al. [43] present an effective image denoising method with bivariate shrinkage threshold in a dual contourlet transform (DCT) domain. It has been experimentally confirmed that novel method can preserve better in the internal structure and the edge details, texture information than other methods.

Fuzzy-based techniques (b) mentioned as the second approach, for the reduction of additive Gaussian noise and other types of noises in color video sequences have been successfully applied as well [25], [44], [45], [46], [47], [48]. The advantage of fuzzy filtering techniques is in the efficient preservation of image features (edges, chromaticity characteristics, texture, etc.) as corrupted pixels are being denoised. Fuzzy logic filters are established by Membership Functions and Fuzzy Rules based on human knowledge and the ability to adapt their characteristics to the current image and noise realizations. These types of techniques are biased by the assumptions the human is making about a pixel/noise model, which in some cases cannot be accurate. In other words, they work well only with the problems they have learned or been tested with [49], [50], [51]. Modern theoretical approaches in denoising have been validated using experiments principally based on the possibility of gathering more samples for similar objects, then applying traditional statistical methods, which depend on the image/noise model, perhaps with some modifications [48], [52], [53]. The principal problem here is how to measure and employ the similarity of a group of objects in an image.

One considerable weakness of traditional filter models is that they are not able to preserve edges or image details in an appropriate manner: edges, which are recognized as discontinuities in the image, are blurred. Fuzzy filters on the other hand can handle with edges and image details in a much better way than other models.

A wide variety of filters are only designed to deal with grayscale images, nevertheless, color image filters have not received much attention in the state-of-the-art. Some proposals reported in the recent years have extended a grayscale denoiser to handling color signal applying it independently in each color channel. But, this technique fails to utilize the correlation between different color components. Fuzzy denoising techniques are able to take advantage from the inter-color correlation to improve over the single-component approaches.

Rosales-Silva et al. [48] and Kravchenko et al. [53] report three-dimensional space–time filtering algorithms (used here as comparative filter) based on fuzzy sets theory. This denoising approach uses the gradient pixel values of the R, G, and B channels and the angular differences between pixels in each color channel, finally employing them in the filtering of the consecutive frames on the basis of fuzzy logic rules. Simulation results have confirmed the superiority of these techniques in comparison with other state-of-the-art algorithms, such as VMMKNN and VGVDF_G.

Melange et al. [52] report a fuzzy-logic-based filter (used here as comparative technique) focusing on an additive white Gaussian noise model. The method is first explained in the pixel domain for grayscale video sequences and is later extended to the wavelet domain and to color video sequences. Numerical simulation results show that the noise is efficiently removed by the proposed fuzzy motion and detail adaptive video filter (FMDAF) but it presents worse results against the VBM3D framework.

Nadernejad et al. [54] propose a novel algorithm where both, fuzzy filtering and partial differential equation methods, are combined to reduce the different kinds of noises. The performance of the algorithm is evaluated showing a better performance compared to the existing methods, PDEs based algorithm, wavelet based algorithms and NLM method.

The third approach (c) in searching similar objects uses the sparse color image representation based on the assumption that images admit a sparse decomposition over a redundant dictionary [7], [10], [55], [56], [57], [58], [59].

Mairal et al. [55] introduce a novel algorithm for color image denoising, inpainting, and demosaicing. The framework is based on learning models modifying the grayscale K-SVD for sparse color image representation. This algorithm learns correlations between the different R, G, and B channels and provides better results compared with independent analysis of color channels.

Dong et al. [57] improve the performance of sparse representation based on image restoration, suppressing sparse coding noise. To this end, the authors exploit the image nonlocal self-similarity. The extensive experiments validate the state-of-the-art performance of the proposed nonlocally centralized sparse representation (NCSR) algorithm.

Dabov et al. [58] present a VBM3D (used here as comparative filter) video filtering method created on highly sparse signal representation in the local 3D transform domain. This method is performed via a 3D data array called “group” by stacking together blocks found to be similar to the currently processed block. The grouping is executed as a spatio-temporal predictive-search block matching. Each 3D group is filtered by a 3D transform-domain shrinkage. Better performance is obtained by using a two-step algorithm where an intermediate estimate is produced by grouping and collaborative hard-thresholding. Then, both algorithms are used to improve the grouping and apply the collaborative Wiener filtering. A recently published paper [35] from the same authors goes a step further by proposing the VBM4D framework, which stacks similar 3D spatio-temporal volumes instead of 2D blocks to form 4D data arrays using the same approach.

Pogrebnyak et al. [59] propose a multiscale approach that aggregates the outputs of DCT filters having different overlapped block sizes. Later, a two-stage denoising procedure is performed that presumes the use of the multiscale DCT-based filtering with hard thresholding at the first stage and a multiscale Wiener DCT-based filtering at the second stage.

Finally, the fourth approach (d) uses the Non Local Means (NLM) filtering techniques that are employed in the denoising of images corrupted by different types of noises. The NLM approach uses all the possible self-predictions and self-similarities that an image can provide, determining the pixel weights for denoising the corrupted image. Several promising algorithms based on this technique are reported in the literature [22], [23], [60], [61].

Jin et al. [23] introduce new nonlocal operators to interpret the (NLM) filter as a regularization of the corresponding Dirichlet functional. The experiments show that the new nonlocal operators yield a better interpretation of the nonlocal means filter.

Jingjing et al. [62] design an advanced color image denoising scheme called multichannel NLM fusion where the inter-channel color correlation is used via constructing and fusing multiple NLM spanning all three channels. Simulations verify the improvement of the framework compared to the single-channel NLM filtering.

Buades et al. [63] propose a new measure method (used here as comparative filter) for noise to evaluate and compare the performance of digital image denoising methods. First, the method computes and analyzes a wide class of denoising algorithms, namely, the local smoothing filters. Second, the authors propose a new algorithm from non-local means based on a non-local averaging of all pixels in an image.

In paper [56], Mairal et al. report a new image model combining the NLM and sparse coding approaches to image restoration into a unified framework. Quantitative and qualitative experiments have shown that the proposed algorithm outperforms the state-of-the-art image denoising techniques.

The novel curvelet based NLM method proposed by Wu et al. [64] determines the similarity of pixels in the noisy image based on the various levels of the multi-scale curvelet transform demonstrating better performance of the framework against the state-of-art NLM denoising.

Generally, the principal objective of different approaches to denoising mixed noises for still images (additive and impulsive) consists of the detection and suppression of random spikes followed by filtering out additive noise [7], [10], [12].

Cai et al. [7] propose a filtering method divided into two phases. First, the outlier candidates are identified. In the second phase, the image is deblurred and denoised simultaneously using essentially the outlier-free data.

Huang et al. [10] report a filtering technique where both impulsive noise and mixed impulse plus Gaussian noises affect the image or video sequence. The authors propose the modified total variation minimization scheme to regularize the deblurred image and fill unsuitable values for noisy image pixels while they are detected by median-type filters.

Camarena et al. [12] use a simple method based on fuzzy logic to filter color images corrupted with mixed impulse and Gaussian noise. In a later section, the best of the above-mentioned techniques are compared with the proposed novel filtering approach.

Modern theoretical approaches in denoising that have been validated using experiments are principally based on a possibility to gather more samples for similar parches. Then, the methods use sophisticated statistical methods, which depend on image/noise model, maybe with some modifications. The principal problem here is how to measure and employ the similarity of group of objects in a color image or in several neighboring color frames of video sequence. Classical example is the mentioned VBM3D technique [58]. Proposed in this paper approach exploits the similar ideas in searching similar parches that permit to gather more samples for processing together color channels for a video frame; following, more samples should be found gathering neighboring frames of a video where the local motions in different frames should be adjusted. Principal difference of novel approach with most of the algorithms reviewed above consists in usage the designed Fuzzy Rules in all filtering steps where the similar objects are learned.

The filtering scheme proposed in this paper is based on fuzzy sets. The proposed Fuzzy-Multichannel-Additive-Noise-Suppression (FMANS) filter is designed to suppress additive noise in color video sequences while preserving image features such as edges, chromaticity characteristics, texture, and fine details. The designed technique consists of three principal filtering stages: spatial, spatio-temporal and spatial postprocessing. In contrast to other state-of-the-art algorithms, in the first spatial stage, eight gradient values in different directions for pixels located in the vicinity of a central pixel as well as the interchannel correlation between analogous pixels in the red (R), green (G) and blue (B) bands are used, where the degree of noise contamination is estimated by employing novel fuzzy rules that allow a better preservation of image features (textures, edges, sharpness, etc.). In the second spatio-temporal filtering stage, two consecutive video frames are analyzed together, where the similarity measure between the consecutive frames is calculated, allowing the formation of an interframe sample of pixels. Possible local motions between consecutive frames are estimated using a block matching procedure in eight directions to perform interframe filtering. In the spatial postprocessing filtering stage, the edges and plain areas of a current frame are distinguished using different filtering procedures. In this paper, we have designed two variants of the proposed fuzzy filter named FMANS_1 and FMANS_2, that depend on sliding windows used. Additionally, the novel FMANS_H filter, which uses hybrid processing connecting the “fuzzy ideology” of the FMANS technique and multiscale Wiener DCT-based filtering is also performed.

Numerous simulation results obtained for color video sequences with different texture characteristics, edges, color properties, and local motions have confirmed the superiority of the novel 3D fuzzy frameworks over other filtering techniques, including VBM3D, in terms of objective criteria (PSNR, MAE, NCD, and SSIM) as well as the subjective perception of the human visual system with respect to different color video sequences. Additionally, the novel frameworks use only two frames concurrently that reduce the computational processing time and memory requirements.

The outline of this paper is as follows. In Section 2, the proposed filtering approach is explained. Numerical experiments and graphical results are given in Section 3. Finally, conclusions are presented in Section 4.

Section snippets

Fuzzy color video filtering

Fuzzy filters are based on the observation that noise causes a small fuzzy derivative, whereas a large fuzzy derivative is caused by the presence of fine details or edges. Fuzzy rules are applied in each direction and consider variations that can occur, such as local motions and variations in edges and fine features [16], [25], [45], [46]. Additionally, fuzzy rules are able to distinguish between noisy pixels, edges, fine image features, and plain areas. These distinctions allow for the main

Performance criteria

To evaluate the effectiveness of the proposed filter in the suppression of additive Gaussian noise and image detail preservation, the novel filter was compared with other known techniques. The filtered frames were evaluated according to the following objective criteria [1], [2], [3], [4], [71], [72].

We employed the PSNR (Peak Signal-to-Noise Ratio) to characterize the noise suppression capabilities of the proposed technique and the MAE (Mean Absolute Error), which measures the level of

Conclusions

The designed 3D filtering frameworks FMANS_1, FMANS_2, and the FMANS_H for the denoising of color video sequences corrupted by additive Gaussian noise are presented. The techniques are based on fuzzy logic theory in application to the basic and several related gradient values along different directions and inter-channel correlations, employing the previous and current temporal frames. The denoising approach consists of three principal filtering stages: spatial, spatio-temporal and spatial

Acknowledgments

This work is supported by Instituto Politecnico Nacional (Mexico) and Consejo Nacional de Ciencia y Tecnologia (Mexico).

Volodymyr Ponomaryov received the Ph.D. degree in 1974 and D.Sci. in 1981. His research interests include signal/image/video processing, pattern recognition, and real-time filtering. He has also been a promoter of 38 Ph.D.s. He has published of about 500 international scientific and conference papers, and also 23 patents of ex USSR, Russia and Mexico, and five scientific books.

References (74)

  • C.H. Hsia et al.

    Efficient modified directional lifting-based discrete wavelet transform for moving object detection

    Signal Process.

    (2014)
  • R. Yaakob et al.

    A comparison of different block matching algorithms for motion estimation

    Proc. Technol.

    (2013)
  • J. Saeedi et al.

    A new wavelet-based fuzzy single and multi-channel image denoising

    Image Vis. Comput.

    (2010)
  • C.C. Liu et al.

    Heuristic wavelet shrinkage for denoising

    Appl. Soft Comput.

    (2011)
  • A.J. Rosales-Silva et al.

    Fuzzy directional FD filter for impulsive noise reduction in colour video sequences

    J. Visual Commun. Image Represent.

    (2012)
  • C. Kang et al.

    Fuzzy reasoning-based directional median filter design

    Signal Process.

    (2009)
  • J.G. Camarena et al.

    Two-step fuzzy logic-based method for impulse noise detection in colour images

    Pattern Recognit. Lett.

    (2010)
  • B. Smolka

    Peer group switching filter for impulse noise reduction in color images

    Pattern Recognit. Lett.

    (2010)
  • E. Nadernejad et al.

    Image denoising using new pixon representation based on fuzzy filtering and partial differential equations

    Digital Signal Proces.

    (2012)
  • K. Wu et al.

    Curvelet based nonlocal means algorithm for image denoising

    AEUE—Int. J. Electron. Commun.

    (2014)
  • Y. Zhang et al.

    An adaptive bilateral filter based framework for image denoising

    Neurocomputing

    (2014)
  • X. Fei et al.

    Perceptual image quality assessment based on structural similarity and visual masking

    Signal Process.: Image Commun.

    (2012)
  • K. Plataniotis et al.

    Color image processing and applications

    Monograph

    (2000)
  • R. Lukac et al.

    Color Image Processing: Methods and Applications

    (2006)
  • J.W. Woods

    Multidimensional Signal, Image, and Video Processing and Coding

    (2011)
  • A. Bovik

    Handbook of Image and Video Processing, Communications, Networking, and Multimedia

    (2005)
  • B.R. Mahafza

    Radar Systems Analysis and Design Using MATLAB

    (2013)
  • J. Cai et al.

    Two-phase approach for deblurring images corrupted by impulse plus Gaussian noise

    Inverse Probl. Imaging

    (2008)
  • B. Goossens et al.

    Image denoising using mixtures of projected Gaussian scale mixtures

    IEEE Trans. Image Process.

    (2009)
  • S. Morillas et al.

    Fuzzy peer groups for reducing mixed Gaussian-impulse noise from color images

    IEEE Trans. Image Process.

    (2009)
  • Y. Huang et al.

    Fast image restoration methods for impulse and Gaussian noise removal

    IEEE Signal Process. Lett.

    (2009)
  • J. Camarena et al.

    A simple fuzzy method to remove mixed Gaussian-impulsive noise from colour images

    IEEE Trans. Fuzzy Syst.

    (2013)
  • B. Smolka, Adaptive truncated vector median filter, in: IEEE International Conference on Computer Science and...
  • C. Lien et al.

    An efficient denoising architecture for removal of impulse noise in images

    IEEE Trans. Comput.

    (2013)
  • J. Varghese et al.

    Efficient adaptive fuzzy-based switching weighted average filter for the restoration of impulse corrupted digital images

    IET Image Process.

    (2014)
  • A. Pizurica et al.

    Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising

    IEEE Trans. Image Process.

    (2006)
  • C. Liu et al.

    Automatic estimation and removal of noise from a single image

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2008)
  • Volodymyr Ponomaryov received the Ph.D. degree in 1974 and D.Sci. in 1981. His research interests include signal/image/video processing, pattern recognition, and real-time filtering. He has also been a promoter of 38 Ph.D.s. He has published of about 500 international scientific and conference papers, and also 23 patents of ex USSR, Russia and Mexico, and five scientific books.

    Hector Montenegro-Monroy received the Ph.D. degree in communication and electronics from the National Polytechnic Institute of Mexico in 2014. His research interests include image/video processing, denoising, and real-time processing.

    Francisco J. Gallegos-Funes received the Ph.D. degree in communications and electronics from National Polytechnic Institute of Mexico in 2003. He is currently an Associate Professor in the Mechanical and Electrical Engineering Higher School from the same institute. His areas of scientific interest are signal and image processing, filtering, steganography, pattern recognition, and real-time applications.

    Oleksiy B. Pogrebnyak received his Ph.D. degree from Kharkov Aviation Institute (now National Aerospace University), Ukraine, in 1991. Currently, he is with the Center for Computing Research of National Polytechnic Institute of Mexico. His research interests include digital signal/image filtering and compression, and remote sensing.

    Sergiy N. Sadovnychiy received his Ph.D. degree from Kharkov Aviation Institute (now National Aerospace University), Ukraine, in 1991. Currently, he is with the National Petroleum Institute of Mexico. His research interests include digital signal/image processing, communication and acoustic diagnostic.

    View full text