Elsevier

Neurocomputing

Volume 283, 29 March 2018, Pages 214-227
Neurocomputing

Model-Based segmentation of image data using spatially constrained mixture models

https://doi.org/10.1016/j.neucom.2017.12.033Get rights and content

Abstract

In this paper, a novel Bayesian statistical approach is proposed to tackle the problem of natural image segmentation. The proposed approach is based on finite Dirichlet mixture models in which contextual proportions (i.e., the probabilities of class labels) are modeled with spatial smoothness constraints. The major merits of our approach are summarized as follows: Firstly, it exploits the Dirichlet mixture model which can obtain a better statistical performance than commonly used mixture models (such as the Gaussian mixture model), especially for proportional data (i.e, normalized histogram). Secondly, it explicitly models the mixing contextual proportions as probability vectors and simultaneously integrate spatial relationship between pixels into the Dirichlet mixture model, which results in a more robust framework for image segmentation. Finally, we develop a variational Bayes learning method to update the parameters in a closed-form expression. The effectiveness of the proposed approach is compared with other mixture modeling-based image segmentation approaches through extensive experiments that involve both simulated and natural color images.

Introduction

To analyze or extract useful contents of an observed image, it is often helpful to convert the original image data into a more explicit representation. The process of partitioning the original image into homogeneous regions is known as image segmentation, which is one of the most challenging tasks in image analysis and processing. The results of image segmentation normally affect the quality of all the subsequent processes of image analysis, such as object representation and description, and even the following higher level tasks such as object recognition or classification. However, it is quite challenging to obtain satisfied segmented results since images are often corrupted by various factors such as noise, intensity inhomogeneity, etc. Over the past few decades, numerous image segmentation approaches have been developed based on different techniques, such as graph theoretic approaches [1], [2], [3], [4], mean shift [5], [6], binary partition tree [7], [8], and rate distortion theory approaches [9], [10].

Except the above approaches, model-based approaches, in particular finite mixture models, have gained much attention to address image segmentation in a probabilistic manner [11], [12]. A finite mixture model is composed of a finite number of basic distributions (such as Gaussian, Student’s t, etc.) known as mixture components. It is an important statistical tool that exploits the statistical prior knowledge to model the probability density. Finite mixture models have been widely and successfully applied in large number of fields ranging from bioinformatics [13], image retrieval [14] image categorization [15], to object recognition [16]. Recently, mixture models have also been successfully applied in image emotion analysis [17], [18], where Gaussian mixture models are used to represent the valence-arousal emotion labels by analyzing personalized emotion perceptions on the Image-Emotion-Social-Net data set. The parameters of the finite mixture model can be estimated using maximum likelihood (ML) or Maximum a posteriori(MAP) through the approach of Expectation-Maximization (EM). However, the EM algorithm is significantly affected by the initial values of parameters and can easily converge to a local maximum with an inappropriate initialization. In addition, when apply finite mixture models to image segmentation, their performance can be easily degraded by high levels of noise. This is mainly due to the fact that in approaches based on conventional mixture models, the spatial dependencies between nearby pixels are not taken into account, and the prior knowledge that adjacent pixels most likely belong to the same cluster is not considered.

To overcome the above shortcomings of mixture modeling-based image segmentation approaches and improve the robustness to noise, an effective approach is to impose spatial smoothness constraints to provide spatial dependencies between neighboring pixels [19]. The applicability of this kind of approach can be found in many image processing tasks, such as defogging [20], denoising [21], restoration [22] and image segmentation [23]. In recent years, several image segmentation approaches based on mixture models and Markov random filed (MRF) have been proposed [24], [25], [26]. According to these approaches, by imposing the smoothness constraints on the contextual mixing proportion (i.e, the probability to assign a given pixel to a particular class), better image segmentation results can be obtained, especially when images are corrupted by high levels of noise. Nevertheless, since contextual mixing proportions are treated as a probabilistic vector and subject to the constraints that they have to be positive and sum to one, it is not straightforward to obtain a closed-form solution in the M-step of the EM algorithm. A solution to this problem is to include a gradient projection step in the M-step as in [24] with the sacrifice of computational efficiency due to additional training complexity. In addition, to make sure that contextual mixing proportions are positive and sum to one, another reparatory projection step has been introduced in the M-step to get a closed-form update solution [25]. However, this approach also requires high computational cost due to the increased complexity of the training data. Another kind of MRF, based on the spatially variant finite Gaussian mixture model was proposed in [26]. In this model, the contextual mixing proportion is explicitly modeled as a probabilistic vector which follows a Dirichlet Compound Multinomial distribution. It guarantees that the contextual mixing proportion is computed subject to the probability constraints (positive and sum to one) without introducing an extra reparatory projection step. Furthermore, spatial smoothness is imposed in this model by applying the Gauss–Markov random field. Unfortunately, the computational cost is still quite high during the process of estimating parameters due to the complex representation of the log-likelihood function.

In this paper, we propose a novel mixture model with spatial constraints for image segmentation, which has the following advantages compared with other mixture modeling-based image segmentation approaches: (1) In preference to commonly used finite mixture models such as Gaussian mixture model (GMM) [11], the proposed model is based on Dirichlet mixture model. Compared with Gaussian distribution that only allows symmetric modes, the Dirichlet distribution, which is the multivariate generalization of the Beta distribution, may have multiple symmetric and asymmetric modes. Thus, motivated by its flexibility and recent promising results in modeling non-Gaussian data [15], [16], we adopt the Dirichlet distribution as the parent distribution to model image pixels directly in this work. (2) The proposed mixture model takes into account the local spatial relationship between neighboring pixels by explicitly introducing a probability prior distribution over contextual mixing proportions. As a result, the requirements that the contextual mixing proportions must be positive and sum to one are explicitly defined in the proposed Bayesian model with Dirichlet distribution without resorting to the reparatory projection step. (3) Most of the mixture modeling based approaches adopt the EM algorithm to find maximum likelihood solutions for image segmentation [24], [25], [26]. Nevertheless, the EM algorithm suffers from the problems of over-fitting and the incapacity of determining model complexity. In this work, we propose a model learning approach based on variational Bayes [27], [28], such that the parameters of the proposed model and its model complexity (i.e., the number of mixture components) can be efficiently optimized in a closed form.

The contributions of this work can be summarized as follows: (1) A novel image segmentation approach is proposed based on spatially constrained Dirichlet mixture models. (2) The spatial smoothness between nearby pixels is imposed by integrating a Dirichlet Compound Multinomial distribution with the Dirichlet mixture model, in order to improve the robustness of segmentation. (3) A closed-form solution to the estimation of model parameters is derived through variational Bayes. The effectiveness of the proposed spatially constrained mixture model for image segmentation is validated through extensive simulations with both synthetic and natural color images.

The rest of this paper is organized as follows. In Section 2, the proposed Dirichlet Mixture Model with spatial constraints that takes the contextual relationship between the nearby pixels is presented. In Section 3, the variational Bayes algorithm for inferencing the parameters of our model is developed with a closed-form solution. In Section 4, experimental results of the presented model with synthetic and natural images are shown. Lastly, conclusion and directions for the future work are given in Section 5.

Section snippets

Finite Dirichlet mixture model

Given a D-dimensional random vector X=(X1,,XD) subject to the constraints: d=1DXd=1 and 0 ≤ Xd ≤ 1, the Dirichlet distribution over X with parameters α=(α1,,αD) is defined by Dir(X|α)=Γ(d=1Dαd)d=1DΓ(αd)d=1DXdαd1,where Γ( · ) is the gamma function defined as Γ(x)=0tx1etdt. The mean and covariance of the Dirichlet distribution are given by E[Xd]=αdd=1Dαl,and Cov[XaXb]=αaαb(d=1Dαd)2(d=1Dαd+1).

It is noteworthy that the Dirichlet distribution is a multivariate distribution which

Variational learning of the model

In this section, a variational Bayes algorithm is developed to learn the proposed spatially constrained Dirichlet mixture model with a closed-form solution. Variational Bayes (also known as variational inference) is a deterministic approximation method used to formulate the computation of the posterior probability in terms of an optimization problem. In variational Bayes, we use a variational distribution q(Θ) to approximate the true posterior distribution p(Θ|X), where Θ={Z,α,ξ} represents

Experimental results

In this section, we evaluate the performance of the proposed spatially constrained Dirichlet mixture model (can be referred to as SC-DMM) through extensive experiments using both simulated and natural color images. In our experiments, we initialize the number of mixture components to 20 with equal mixing coefficients, and this value will be automatically inferred during the learning process. It it noteworthy that since variational Bayes by theory may converge to some local maxima, it is often

Conclusion

We proposed a novel statistical approach for image segmentation. Our approach is based on a spatially constrained Dirichlet mixture model that takes the spatial relationship between neighboring pixels into account and thus can provide segmentation results with higher degree of robustness against noise. The proposed model is learned by developing a variational Bayes algorithm that can provide a closed-form solution. The effectiveness of the proposed approach is validated through extensive

Acknowledgment

The completion of this work was supported by the National Natural Science Foundation of China (61502183, 61673186), and the Scientific Research Funds of Huaqiao University (600005-Z15Y0016).

Can Hu received his B.E. degree from the Department of Computer Science and Technology, Hunan Institute of Science and Technology, Hunan province, China, in 2014. He is currently working for his M.Sc. degree in the Department of Computer Science and Technology, Huaqiao University, Xiamen, China. His research areas include machine learning and pattern recognition, image processing, and bioinformatics.

References (47)

  • T. Cour et al.

    Spectral segmentation with multiscale graph decomposition

    Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR

    (2005)
  • D. Comaniciu et al.

    Mean shift: a robust approach toward feature space analysis

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2002)
  • M.A. Carreira-Perpinan

    Fast nonparametric clustering with Gaussian blurring mean-shift

    Proceedings of the 23rd International Conference on Machine Learning, ICML

    (2006)
  • P. Salembier et al.

    Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval

    IEEE Trans. Image Process.

    (2000)
  • LuH. et al.

    Binary partition tree for semantic object extraction and image segmentation

    IEEE Trans. Circuits Syst. Video Technol.

    (2007)
  • MaY. et al.

    Segmentation of multivariate mixed data via lossy data coding and compression

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2007)
  • G. McLachlan et al.

    Finite Mixture Models

    (2000)
  • D.M. Titterington et al.

    Statistical Analysis of Finite Mixture Distributions

    (1985)
  • K. Blekas et al.

    Greedy mixture learning for multiple motif discovery in biological sequences

    Bioinformatics

    (2003)
  • FanW. et al.

    Variational learning for finite Dirichlet mixture models and applications

    IEEE Trans. Neural Netw. Learn. Syst.

    (2012)
  • ZhaoS. et al.

    Predicting continuous probability distribution of image emotions in valence-arousal space

    Proceedings of the International Conference on Multimedia

    (2015)
  • ZhaoS. et al.

    Continuous probability distribution prediction of image emotions via multi-task shared sparse regression

    IEEE Trans. Multimed.

    (2017)
  • A. Blake et al.

    Visual Reconstruction

    (1987)
  • Cited by (15)

    • A spatially constrained asymmetric Gaussian mixture model for image segmentation

      2021, Information Sciences
      Citation Excerpt :

      In this section, we demonstrate the performance of the proposed method through various experiments. We compare our method with six state-of-the-art algorithms: a spatially constrained generative model (SCGA-EM) [17], a fast and robust spatially constrained Gaussian mixture model (FRSCGMM) [18], a spatially constrained asymmetric Gaussian mixture model(SCAGMM) [19], a spatially constrained generative asymmetric Gaussian mixture model (SCGAGMM) [11], a spatially constrained in finite Dirichlet mixture models(SC-DMM) [4], and a shifted asymmetric Laplace mixture model (SALMM) [12]. In this paper, we use three qualitative indices, including the misclassification ratio (MCR) [18], the Js value [25] and the Probabilistic Rand Index (PRI) [27], to evaluated the performance of segmentation results.

    • Adaboost-like End-to-End multiple lightweight U-nets for road extraction from optical remote sensing images

      2021, International Journal of Applied Earth Observation and Geoinformation
    • A novel statistical approach for clustering positive data based on finite inverted Beta-Liouville mixture models

      2019, Neurocomputing
      Citation Excerpt :

      Data clustering is a common unsupervised learning technology for data analysis via discovering similar statistical characters in a data set. It has been widely applied in many fields such as image processing [1], remote sensing [2], data mining [3]. Thus, there is an urgent need for effective technologies to model and analyze complicated data.

    • Fuzzy clustering based transition region extraction for image segmentation

      2018, Engineering Science and Technology, an International Journal
      Citation Excerpt :

      In the recent era a number of hybrid segmentation techniques have emerged which provide better segmentation results in comparison to that of the traditional methods. These hybrid segmentations are classified as model based segmentation approaches [4], machine learning approaches [5], graph-cut methods [6,7], active contour and level set methods [8] and transition region based approaches [9–16]. In the model based approaches, the image is characterized by a statistical model and the model parameters are used as features for segmentation [17].

    View all citing articles on Scopus

    Can Hu received his B.E. degree from the Department of Computer Science and Technology, Hunan Institute of Science and Technology, Hunan province, China, in 2014. He is currently working for his M.Sc. degree in the Department of Computer Science and Technology, Huaqiao University, Xiamen, China. His research areas include machine learning and pattern recognition, image processing, and bioinformatics.

    Wentao Fan received his M.Sc. and Ph.D. degrees in electrical and computer engineering from Concordia University, Montreal, Quebec, Canada, in 2009 and 2014, respectively. He is currently an associate professor in the Department of Computer Science and Technology, Huaqiao University, Xiamen, China. His research interests include machine learning, computer vision, and pattern recognition.

    Jixiang Du received his M.Sc. degree in vehicle engineering from Hefei University of Technology in 2002 and his Ph.D. in pattern recognition intelligent system from the University of Science and Technology of China. Currently, he is a professor at the College of Computer Science and Technology, Huaqiao University. He is also the associate dean of the College of Computer Science and Technology. His current research mainly concern pattern recognition and machine learning.

    Yuchen Zeng is currently working for his B.Sc. degree in the Department of Computer Science and Technology, Huaqiao University, Xiamen, China. His research areas include machine learning, pattern recognition, and image processing.

    View full text