Elsevier

Applied Soft Computing

Volume 13, Issue 2, February 2013, Pages 817-832
Applied Soft Computing

Neuro fuzzy and punctual kriging based filter for image restoration

https://doi.org/10.1016/j.asoc.2012.10.017Get rights and content

Abstract

In this paper, we present a hybrid, image restoration approach. The proposed approach combines the geostatistical interpolation of punctual kriging, artificial neural networks (ANNs), and fuzzy logic based approaches. Images degraded with Gaussian white noise are restored by first utilizing fuzzy logic for selecting pixels that needs kriging. Three fuzzy systems are employed. Both type-I and type-II fuzzy sets in addition with neuro fuzzy classifier (NFC) have been used for the detection of noisy pixels. To avoid edge pixels, a post processing technique is used to check the edge pixel connectivity up to lag 5. If the pixel under consideration is an edge pixel, it is excluded from the fuzzy map and thus not estimated. The concept of punctual kriging is then used to estimate the intensity of a noisy pixel. ANN is employed to minimize the cost function of the kriging based pixel intensity estimation procedure. ANN, in contrast to analytical methodologies, avoids both matrix inversion failure and negative weights problems. Image restoration performance based comparison has been made against adaptive Weiner filter and existing fuzzy kriging approaches. Experimental results using 450 images are used to validate the effectiveness of the proposed approach. Different image quality measures are used to compare the efficacy of the proposed NFC and fuzzy type-II approaches for detecting noisy pixels in conjunction with ANN and kriging based estimation.

Highlights

► A hybrid image restoration method using punctual kriging, ANN and fuzzy logic. ► Type-I and type-II fuzzy sets and NFC are employed for selecting pixels that need estimation. ► Genetic algorithm is used to select optimum features for classification using NFC. ► ANN is employed to avoid matrix inversion failure and negative weights problems. ► Performance is analyzed using different image quality measures on 450 images.

Introduction

Image restoration has become a widely investigated field of image processing. In spite of the advances made by recent methods, it is still a challenging task as these methods have yet to achieve a desirable level of applicability in many realistic scenarios. Moreover, with the ever increasing production of digital contents such as images and videos acquired with low resolution cameras and in poor conditions. In such cases, the importance of image restoration has significantly increased. One of the primary tasks in developing image restoration techniques is noise removal without destroying edge information. Noise smoothing and edge enhancement are generally considered as conflicting tasks. Since smoothing a region might destroy an edge while sharpening edges might lead to amplification of unnecessary noise [1]. Therefore, we present a new spatial filtering technique; a neural approach based on punctual kriging and fuzzy logic control. This new approach takes into account this conflict and tries to remove noise, while efficiently preserving the image details and edges information.

Punctual kriging, named after its developer, Krige [2] is heavily used in mining and geostatistics based applications. It is an interpolation technique that gives an optimal linear estimate of an unknown parameter at a sampling point in terms of its known values at the surrounding sampling points [3]. The estimation involves calculation of the semi-variances and modeling of semi-variograms from the sampled data. Besides this, kriging has been applied in many other fields as well.

Fuzzy filters have been extensively applied in image processing over the last decade. Young Sik and Krishnapuram [4] devised fuzzy rule based multiple filters, derived from the method of weighted least squares, for noise removal. Some researchers have also investigated the use of fuzzy clustering for the removal of impulsive noise [5], [6], [7]. In [8], Farbiz and Menhaj have introduced an approach of image filtering based on fuzzy logic control. They have shown how to remove impulsive noise and smooth out Gaussian noise while, simultaneously, preserving image details and edges efficiently. Liang and Looney [9] have proposed a competition fuzzy edge detector to distinguish the noisy pixels from the edge pixels. Further, Khriji and Gabbouj [10] have recently proposed a fuzzy transformation based approach for multichannel image processing. Fuzzy spatial filters have been widely explored for restoration of images. However, with the increase of local information, the number of fuzzy rules in these filters also increases accordingly. To reduce the requirement of such complicated rules, fuzzy control is used as a complementary tool along with the existing techniques to develop better and accurate methods. This is one of the major aims of the investigations presented in this paper.

In the most basic image restoration approach using neural networks, noise is removed from the image by simple filtering. Cellular neural networks by Chua and Yang [11], [12] have been proposed for noise suppression. Improvements have been done for training cellular neural networks that make use of genetic algorithms by Zamparelli [13]. Generalized adaptive neural filter [14], [15] is another interesting neural architecture for noise filtering. It consists of a set of neural operators based on stack filters [14] that make use of binary decomposition of gray valued data.

Combination of order statistic filters and Hopfield neural network have also been developed and used by Qian et al. [16] for noise removal and image de-blurring. Suetake and Uchino [17] have proposed a radial basis function network and Wiener hybrid filter to exploit merits of both for removing noise with an arbitrary distribution. Multilevel sigmoidal activation functions [18] are used by Sivakumar et al. to model a blurred and noisy image with many gray levels without any knowledge of the statistics of the additive noise and blurring function. In [19], Widyanto et al. have proposed a method to improve recognition as well as generalization capability of back-propagation neural network as a hidden layer self-organization inspired by immune algorithm. Recently, Palmer et al. [20] have introduced a spatially regularized neural approach that makes use of local image statistics to apply varying regularization to different areas of the image by using a parallel implementation of the Hopfield neural network. Gwanggil et al. [21], have proposed deinterlacing technique based on a type-II fuzzy logic filter and have introduced application of type-II fuzzy sets for interpolation of interlaced fields. Similarly in [1] an image filtering with hybrid impulse detector is proposed.

Several techniques [22], [23], [24], [25] have also been proposed using the wavelet transform for image denoising. These methods perform a combination of statistical modeling and thresholding on wavelet coefficients at different decomposition levels to suppress noise. More recently, curvelets [26] and ridgelets [27] have been employed for line structure preservation while denoising images. Similarly, Portilla at al. [28] have proposed a method based on using scale mixtures of Gaussians in the wavelet domain for removing noise. Their idea is based on modeling the coefficients at adjacent positions and scales as a product of two independent random variables which allows it to account for empirically observed correlation between the coefficient amplitudes. Moreover, a kernel based method has also been proposed recently by Laparra et al. [29] using support vector regression in the wavelet domain. It tries to non-explicitly model the relationship between wavelet coefficients by encoding specific coefficient relations in an anisotropic kernel. Whereby, the kernel is obtained from mutual information measure computed on a database of images. The non-parametric nature of their method allows it to cope with different types of noise sources without any reformulation. Although, wavelet based methods have performed well in denoising images but they use fixed basis, which is often not suitable for images having rich amount of locally varying structural patterns specially, the natural scenes. Thus these methods introduce several visual artifacts in the reconstructed images [30].

To overcome some of the disadvantages of wavelet transform, Lei et al. [30] have recently proposed a two stage principal component analysis (PCA) based denoising method, which uses the local pixel grouping and data selection. The primary idea behind this approach follows from earlier works on image denoising which make use of PCA transformation for filtering out noise by selecting and preserving the most significant principal components [31]. The first stage in their method yields an initial estimate of the image by removing most of the noise and the second stage further refines the output. However PCA based methods often result in reconstruction with missing fine details thus significantly effecting local structural similarity in complex pictures with several edges.

Besides Pham and Wagner [32], [33] have used punctual kriging along with fuzzy sets to enhance images corrupted by Gaussian white noise. They model soft-thresholding by fuzzy sets. In their approach, the pixel intensity in the processed image is a weighted sum of the original (noisy) and the estimated value through kriging. They have evaluated their results qualitatively in comparison with adaptive Wiener filter [34]. However, their study does not provide any quantitative performance analysis of their proposed technique [35], [36]. In addition, they apply kriging to all pixels in the degraded image. Considering 3 × 3 neighborhood, inverse of a kriging matrix of size 9 × 9 is required, that can make the filtering process computationally expensive. In addition, due to a zero diagonal, inverse of the kriging matrix may not always be possible. The filter weights also suffer from the problem of negative values, which leads to an overall poor performance of the filter. It is also reported that separating noise and original signal from a single input image is under constrained, in theory it is very difficult to recover the original signal [37].

In this paper, we thus propose a hybrid technique based on fuzzy inference system, neural net and punctual kriging for image restoration. This paper makes the following contributions: we introduce an effective hybrid neuro-fuzzy based kriging methodology for image denoising. Both type-I and type-II fuzzy sets and NFC have been employed for decision making about the noisy and noise free pixels. We solve both the problems of matrix inversion failure and the negative weights in punctual kriging by exploiting learning capabilities of artificial neural network (ANN). A post processing phase is also employed to improve the noise decision map by reducing the wrong selected edge pixels.

For clarity and understanding, in Table 1, first we present the abbreviations that have been used in the text.

Rest of the paper is structured as follows: Section 2 introduces punctual kriging and variograms, fuzzy inference system, type-I and type-II fuzzy sets. It also presents some review of ANNs used for image restoration and few of the most commonly used image quality measures along with the proposed variogram based quality measure. Section 3 explains the proposed hybrid technique based on punctual kriging and the neuro-fuzzy approach of adaptive learning. Experimental results and discussion is presented in Section 4. Conclusions are made in Section 5.

Section snippets

Punctual kriging and variogram

Punctual kriging provides the best linear unbiased estimate of an unknown point on a surface [38]. The estimate is the weighted sum of the known neighboring values around the unknown point. The weights are determined to minimize the variance of the estimation-error. To achieve this, kriging uses a variogram model (a concept from geostatistics). Based on the variogram model chosen, known values are assigned optimal weights to calculate the unknown value. Variogram presents the variation of

The proposed hybrid approach

The occurrence of singular matrix in kriging is inherently unpredictable as it depends on the variogram for a pixel in the degraded image. The variogram itself depends on neighboring values of a pixel. Such scenarios should be taken care of separately by replacing the processed pixel with a value given by fuzzy ‘averaging’ or ‘median’ filter, which ever makes the error variance ‘small’.

Table 2 shows the statistics about the number of pixels selected for kriging through Fuzzy Decider. It is

Variograms of the original and degraded images

The experimental semi-variograms of three different types of images (Boat, Blood cells and Lena) have been computed and shown in Fig. 6. The shapes of the variograms for all three images near lag zero are continuous. This shows that the pixel values do not change abruptly at lags near zero. However, for Lena and Boat images, fluctuations start appearing for lags greater than 10. This shows that after a lag of 10 pixels, we enter into a new region. Further, in case of Blood cells image, the

Conclusions

An effective hybrid image denoising method based on the concept of punctual kriging is analyzed. Fuzzy IF THEN rules based on region homogeneity and deviations, are used to intelligently decide the importance of a pixel in view of edge preservation. The performance of both type-I and type-II fuzzy sets has been analyzed for this purpose. The method further solves the kriging matrix inversion and negative filter weights problems due to the learning capabilities of the neural net. The overall

Acknowledgements

This work is supported by the Higher Education Commission of Pakistan under the indigenous PhD scholarship program (17-5-4(Ps4-078)/HEC/Sch/2008/) and BK21, program of South Korea.

Asmatullah Chaudhry received his M.Sc. degree in Physics from Islamia University Bahawalpur, Pakistan in 1993 and his M.S. degree in Nuclear Engineering from Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad, Pakistan, in 1998. He received his M.S. and Ph.D. degrees in Computer Systems Engineering from GIK Institute, Topi, Pakistan, in 2003 and 2007, respectively. He has more than 13 years of research experience and is working as Principal Scientist in HRD Division at

References (65)

  • H.H. Tsai et al.

    An image filter with a hybrid impulse detector based on decision tree and particle swarm optimization

  • D. Krige

    A statistical approach to some basic mine valuation problems on the Witwatersrand

    Journal of the Chemical, Metallurgical and Mining Society of South Africa

    (1951)
  • C. Young Sik et al.

    A robust approach to image enhancement based on fuzzy logic

    IEEE Transactions on Image Processing

    (1997)
  • M. Doroodchi et al.

    Implementation of fuzzy cluster filter for nonlinear signal and image processing

  • A. Hussain et al.

    A hybrid image restoration approach: fuzzy logic and directional weighted median based uniform impulse noise removal

    Knowledge and Information Systems

    (2010)
  • A. Majid et al.

    Impulse noise filtering based on noise-free pixels using genetic programming

    Knowledge and Information Systems

    (2011)
  • F. Farbiz et al.

    A fuzzy logic control based approach for image filtering

  • L. Khriji et al.

    Rational-based adaptive fuzzy filters

    International Journal of Computational Cognition

    (2004)
  • L.O. Chua et al.

    Cellular neural networks: theory

    IEEE Transactions on Circuits and Systems

    (1988)
  • L.O. Chua et al.

    Cellular neural networks: applications

    IEEE Transactions on Circuits and Systems

    (1988)
  • N. Ansari et al.

    Generalised adaptive neural filters

    Electronics Letters

    (1993)
  • Z.Z. Zhang et al.

    Structure and properties of generalized adaptive neural filters for signal enhancement

    IEEE Transactions on Neural Networks

    (1996)
  • W. Qian et al.

    Order statistic-neural network hybrid filters for gamma camera-bremsstrahlung image restoration

    IEEE Transactions on Medical Imaging

    (1993)
  • K. Sivakumar et al.

    Image restoration using a multilayer perceptron with a multilevel sigmoidal function

  • A.S. Palmer et al.

    Spatially adaptive image restoration by neural network filtering

  • J. Gwanggil et al.

    Designing of a type-2 fuzzy logic filter for improving edge-preserving restoration of interlaced-to-progressive conversion

    Information Sciences

    (2009)
  • L. Zhang et al.

    Multiscale LMMSE-based image denoising with optimal wavelet selection

    IEEE Transaction on Circuits and Systems for Video Technology

    (2005)
  • S.G. Chang et al.

    Spatially adaptive wavelet thresholding with context modeling for image denoising

    IEEE Transaction on Image Processing

    (2000)
  • H. Zujun

    Adaptive singular value decomposition in wavelet domain for image denoising

    Pattern Recognition

    (2003)
  • S. Jean-Luc et al.

    The curvelet transform for image denoising

    IEEE Transactions on Image Processing

    (2002)
  • J. Portilla et al.

    Image denoising using scale mixtures of Gaussians in the wavelet domain

    IEEE Transactions on Image Processing

    (2003)
  • V. Laparra et al.

    Image denoising with Kernels based on natural image relations

    Journal of Machine Learning Research

    (2010)
  • Cited by (11)

    • T2FCS filter: Type 2 fuzzy and cuckoo search-based filter design for image restoration

      2019, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      That is, the proposed type 2 fuzzy set is used to transform the blurred pattern classification problem into precise manner. Unlike ordinary fuzzy sets, the type 2 fuzzy sets are used to retain the controlled degree of uncertainty [21]. In addition, the noisy pixels of the input images are removed using the proposed cuckoo search algorithm which requires fewer number of parameter to find the optimal solution.

    • A rapid hybrid algorithm for image restoration combining parametric Wiener filtering and wave atom transform

      2016, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      Then an appropriate method is selected to restore the images with the estimated PSF [1,2]. In other works, authors proposed to restore images in the wavelet domain [3–5]. Donoho and Johnstone in [13] proposed an algorithm for solving the inverse problem known as the wavelet-vaguelletes algorithm.

    • Denoising Medical Images Using Hybrid Filter With Firefly Algorithm

      2019, 2019 International Conference on Recent Advances in Energy-Efficient Computing and Communication, ICRAECC 2019
    • Image enhancement based on bee colony algorithm

      2019, Journal of Engineering and Applied Sciences
    View all citing articles on Scopus

    Asmatullah Chaudhry received his M.Sc. degree in Physics from Islamia University Bahawalpur, Pakistan in 1993 and his M.S. degree in Nuclear Engineering from Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad, Pakistan, in 1998. He received his M.S. and Ph.D. degrees in Computer Systems Engineering from GIK Institute, Topi, Pakistan, in 2003 and 2007, respectively. He has more than 13 years of research experience and is working as Principal Scientist in HRD Division at PINSTECH. Currently, he is postdoc fellow at Chonnam National University, Gwangju, South Korea. His research areas include Image Processing, Pattern Recognition, and Machine Learning.

    Asifullah Khan received his M.Sc. degree in Physics from University of Peshawar, Pakistan in 1996 and his M.S. degree in Nuclear Engineering from PIEAS, Islamabad, Pakistan, in 1998. He received his M.S. and Ph.D. degrees in Computer Systems Engineering from GIK Institute, Topi, Pakistan, in 2003 and 2006, respectively. He has carried out two-years postdoc Research at Signal and Image Processing Lab, Department of Mechatronics, GIST, South Korea. He has more than 13 years of research experience and is working as Associate Professor in DCIS at PIEAS. His research areas include Digital Watermarking, Pattern Recognition, Image Processing, Evolutionary Algorithms, Bioinformatics, Machine Learning, and Computational Materials Science.

    Anwar Majid Mirza received his M.Sc. degree in Physics, Quaid-e-Azam University (QAU), Islamabad in 1987 and he received his M.Sc. degree Nuclear Engineering, CNS, QAU, Islamabad in 1989. He received DIC and Ph.D. in Computational Physics, from Imperial College, UK in 1995. He has research interests in the areas of Machine Intelligence, Soft-Computing, Computer Vision and Modeling & Simulation. He has supervised more than 40 MS/MPhil thesis projects. He has been ranked at number 4 in the list of the most productive scientists of Pakistan in engineering sciences category by Pakistan Council for Science & Technology (PCST) in year 2003.

    Asad Ali received the B.E. degree in Software Engineering from Army Public College of Management & Science (APCOMS), Rawalpindi, Pakistan in 2004 and the MS degree in Computer System Engineering from GIK Institute, Topi, Pakistan in 2006. He is currently a Ph.D. candidate at the Sato Lab in University of Tokyo. He was a recipient of Government scholarship for M.S. studies and MEXT scholarship for Ph.D. studies. His areas of interest include Material Recognition, Reflectance Analysis, Invariant Features, Intelligent Transportation Systems, Reversible Watermarking and Machine Cognition.

    Mehdi Hassan received his M.Sc. degree in Computer Science from Gomal University, D.I.Khan in 2004. He is M.S. leading to Ph.D. scholar of Higher Education Commission (HEC), Pakistan. He received his M.S. degree in Computer System Engineering from GIK Institute, Topi, Pakistan, in 2010 and he is currently continuing his Ph.D. studies at PIEAS, Islamabad, Pakistan. His research interests are Image Processing, Data Mining and Machine Learning, Medical Image Processing and Classification.

    Jin Young Kim received his B.S. degree in Electrical Engineering in 1986. He did his M.S. and Ph.D. in Electrical Engineering from same institute in 1988 and 1994 respectively from Department of Electrical Engineering, Seoul National University, S. Korea. Currently he is working as a Professor in Department of Electronics and Computer Engineering, Chonnam National Univerity, South Korea. His research interests are Audio Visual signal processing and embedded systems.

    View full text