Segmenting Images Using Hybridization of K-Means and Fuzzy C-Means Algorithms

Image segmentation is an essential technique of image processing for analyzing an image by partitioning it into non-overlapped regions each region referring to a set of pixels. Image segmentation approaches can be divided into four categories. They are thresholding, edge detection, region extraction and clustering. Clustering techniques can be used for partitioning datasets into groups according to the homogeneity of data points. The present research work proposes two algorithms involving hybridization of K-Means ( KM ) and Fuzzy C-Means ( FCM ) techniques as an attempt to achieve better clustering results. Along with the proposed hybrid algorithms, the present work also experiments with the standard K-Means and FCM algorithms. All the algorithms are experimented on four images. CPU Time, clustering fitness and sum of squared errors (SSE) are computed for measuring clustering performance of the algorithms. In all the experiments it is observed that the proposed hybrid algorithm KMandFCM is consistently producing better clustering results.


Introduction
Images are often the most important category among the available digital data. In the recent years, image data is increasing and will continue increase in the near future. Since it is difficult to deal with large amount of image data as the available data increases, it becomes crucial to use the automated tools for various purposes in connection to image data. The image processing provides wide range of techniques to deal with the images. By using the image processing techniques, we can make the work much easier not only for now, but also for the future when there will be more data and more work to do on the images.
Image segmentation is an essential image processing technique that analyzes an image by partitioning it into non-overlapped regions each region referring to a set of pixels. The pixels in a region are similar with respect to some characteristic such as color, intensity, or texture [1]. The pixels significantly differ with those in the other regions with respect to the same characteristic [2][3][4]. Image segmentation plays an important role in a variety of applications such as robot vision, object recognition, medical imaging and etc. [5][6][7]. Image segmentation approaches can be divided into four categories. They are thresholding, edge detection, region The K-Means is a partitional clustering method, which partitions a given dataset into a pre-specified number, k, of clusters [25]. It is a simple iterative method. The algorithm is initialized by randomly choosing k points from a given dataset as the initial cluster centers, i.e., cluster means. The algorithm iterates through two steps till its convergence: 1.Data assignment: this step partitions the data by assigning each data object to its closest cluster center.
2.Updating the cluster centers: update the center of each cluster based on the objects assigned to that cluster.
The algorithm for K-Means is as follows [26]. Here, k represents the number of clusters, d represents the number of dimensions or attributes, X i represents the ith data sample, μ j (j = 1, 2, …, k) represents the mean vector of cluster C j , t is the iteration number. For termination condition the algorithm computes percentage change, Eq. (2). The algorithm terminates when Percentage change < α. Here, α is assumed to be 3 since it is negligible.
KM algorithm 1.Select k vectors randomly from the dataset as the initial cluster centers, μ j (j = 1, 2, …, k). Set the current iteration t = 0.
2.Assign each vector, X i , to its closest cluster center using Euclidean distance, Eq. (1).

4.Compute Percentage change as follows
where Ψ t is the number of vectors assigned to new clusters in tth iteration and Ψ t + 1 is the number of vectors assigned to new clusters in (t + 1)th iteration.
5.Stop the process if Percentage change < α, otherwise set t = t + 1 and repeat the steps 2-4 with the updated parameter.
The K-Means uses Euclidean distance as a proximity measure for determining the closest cluster to which a data object is assigned [13]. The algorithm stops when the assignment of data points to the clusters no longer changes or some other criterion is satisfied. The K-Means is a widely used algorithm for clustering and it requires less CPU time. However, it mainly suffers from detecting the natural clusters that have non-spherical shapes or widely different sizes or densities [25].

The Fuzzy C-Means (FCM) algorithms
Fuzzy-based clustering techniques focus on modeling uncertain and vague information that is found in the real world situations. These techniques deal with the clusters whose boundaries cannot be defined sharply [14,15]. By fuzzy-based clustering, one can know if data objects fully or partially belong to the clusters based on their memberships in different clusters [27]. Among the fuzzy-based clustering methods, Fuzzy C-Means (FCM) is the most well-known algorithm as it has the advantage of robustness for obscure information about the clusters [1,28].
In FCM, a dataset is grouped into k clusters, where every data object may relate to every cluster with some degree of membership to that cluster [16]. The membership of a data object towards a cluster can range between 0 and 1 [29]. The sum of memberships for each data point must be unity.
The FCM iterates through two phases for converging to a solution. First, each data object will be associated with a membership value for each cluster, and second, assigning the data object to the cluster with the highest membership value [2].
The algorithm for FCM is given below [30]. Here, U is the k Â N membership matrix. While computing the cluster centers and updating the membership matrix at each iteration, the FCM uses membership weight, m. For most data 1.5 ≤ m ≤ 3.0 gives good results [29]. In all our experiments, we take m = 1.25.
FCM algorithm 1.Initialize parameters: select k vectors randomly as cluster means; set initial membership matrix U 0 ð Þ k X N , set the current iteration t = 0.
2.Assign each data object X i to clusters using the membership matrix.
3.Compute jth cluster center as follows: 4.Compute new membership matrix using 5.Assign each data object X i to clusters using the membership matrix.
7.Stop the process if the Percentage change is <α. Otherwise, set t = t + 1 and repeat the steps 3-7 with the updated parameters.

Hybridization involving K-Means and FCM techniques
The partitional [11] and fuzzy-based [16] methods are widely applied clustering techniques in several areas. The partitional clustering methods do hard clustering, where the dataset is partitioned into a specified number of mutually exclusive subsets. The K-Means, as a partitional clustering method is found in the research literature as widely applied technique in a variety of experiments. While clustering the data, the K-Means aims at minimizing the local distortion [39,40]. However, K-Means is ideal if the data objects are distributed in well-separated groups.
In fuzzy-based clustering, objects are not forced to fully belong to one cluster. Here, an object may belong to many clusters with varying degrees of membership. This membership can range between 0 and 1 indicating the partial belongingness of objects to the clusters [16]. Fuzzy clustering techniques help in understanding if the data objects fully or partially belong to clusters depending on their memberships [27]. In FCM, each data object belongs to each cluster with some degree of membership that ranges between 0 and 1 [29]. Here, clusters are treated as fuzzy sets. In general, introducing the fuzzy logic in K-Means is the Fuzzy C-Means algorithm [41].
The following sub-section discusses two algorithms that apply hybridization of K-Means (KM) and Fuzzy C-Means (FCM) clustering techniques [42]. These algorithms are KMFCM and KMandFCM. The KMFCM algorithm first performs K-Means on the given dataset and then performs the FCM using the results of K-Means. The KMandFCM algorithm performs K-Means and FCM in the alternative iterations on the given dataset. The detailed discussion of these hybrid algorithms is presented in the following subsections.

The KMFCM algorithm
The proposed hybrid clustering algorithm KMFCM first performs the K-Means (KM) technique completely on the given dataset. Using the resulted cluster centers of KM as cluster seeds, the FCM is performed on the given dataset till termination. Here, to run the first iteration of the FCM, the cluster centers and the membership matrix are calculated based on the results of KM. The remaining iterations continue as in the FCM algorithm.
The algorithm for the KMFCM is given below. Here, KM-Step is the K-Means step and FCM-Step is the Fuzzy C-Means step.
KMFCM algorithm 1.KM-Step: select k vectors randomly from the dataset as the initial cluster centers μ j (j = 1, …, k). Set the current iteration t = 0.
2.Assign each data object X i to its closest cluster center using Eq. (1).
6.FCM-Step: compute the membership matrix U t ð Þ k X N using Eq. (4) based on the results of KM-Step.
7.Assign data objects to clusters using membership matrix.

The KMandFCM algorithm
Clustering in KMandFCM is performed by executing K-Means and the FCM techniques in alternative iterations on the given dataset till termination. The first iteration is performed using K-Means assuming some randomly selected data points as cluster centers. The second iteration is performed using FCM technique. For this iteration the cluster means, covariance matrices and the membership matrix are calculated using the results of first iteration. Third iteration is performed using K-Means technique. This iteration computes cluster means using results obtained from the second iteration. In this way, clustering is performed using K-Means and FCM in the alternative iterations till termination.
The algorithm for the proposed KMandFCM algorithm is given below. Here, KM-Step is the K-Means step and FCM-Step is the Fuzzy C-Means step.
KM and FCM algorithm 1.Select k vectors randomly from the dataset as initial cluster centers μ j (j = 1, …, k). Set the current iteration t = 0.

2.KM-
Step: assign each vector X i to its closest cluster center using Eq. (1).

4.
For each cluster C j , compute the center μ j using Eq. (3) 5.Compute the new membership matrix U t ð Þ k X N using Eq. (4) 6.Assign data objects to clusters using the membership matrix. 10.Assign each vector X i to its closest cluster center using Eq. (1).
12.Stop the process if Percentage change < α. Otherwise, go to step 3.

Performance evaluation measures
For performance evaluation of algorithms, CPU time in seconds, sum of squared error [12] and clustering fitness [43] are taken into consideration and are calculated for all the algorithms.

Sum of squared errors
The objective of clustering is to minimize the within-cluster sum of squared error (SSE). The lesser the SSE, the better the goodness of fit is. The sum of squared error [12] for the results of each clustering algorithm is computed using the Eq. (5) Here, X i is the ith data object in the dataset, μ j (j = 1, …, k) is the center of the cluster C j , and k is the number of clusters.

Clustering fitness
The main objective of any clustering algorithm is to generate clusters with higher intra-cluster similarity and lower inter-cluster similarity. So, it is also important to consider inter-cluster similarity while evaluating the clustering performance. In the present work, clustering fitness is also considered as a performance criterion, which requires the calculation of both intra-cluster similarity and inter-cluster similarity. The computation of clustering fitness also requires the experiential knowledge, λ. The computation of clustering fitness results in higher value when the inter-cluster similarity is low and results in lower value for when the inter-cluster similarity is high. Also that to make the computation of clustering fitness unbiased, the value of λ is taken as 0.5 [43].
(a) Intra-cluster similarity for the cluster C j : it can be quantified via a function of the reciprocals of intra-cluster radii within each of the resulting clusters. The intra-cluster similarity [43] of a cluster C j (1 = j = k), denoted as S tra (C j ), is defined in Eq. (6) Here, n is the number of items in cluster C j , I j (1 = j = n) is the jth item in cluster C j , and dist(I j , Centroid) calculates the distance between I j and the centroid of C j , which is the intra-cluster radius of C j . To smooth the value of S tra (C j ) and allow for possible singleton clusters, 1 is added to the denominator and numerator.
(b) Intra-cluster similarity for one clustering result C: it is denoted as S tra (C).
It is defined in Eq. (7), [43] Here, k is the number of resulting clusters in C and S tra (C j ) is the intra-cluster similarity for the cluster C j .
(c) Inter-cluster similarity: it can be quantified via a function of the reciprocals of inter-cluster radii of the clustering centroids. The inter-cluster similarity [43] for one of the possible clustering results C, denoted as S ter (C j ), is defined as Eq. (8) Here, k is the number of resulting clusters in C, 1 = j = k, Centroid j is the centroid of the jth cluster in C, Centroid 2 is the centroid of all centroids of clusters in C. We compute inter-cluster radius of Centroid j by calculating dist(Centroid j , Centroid 2 ), which is distance between Centroid j , and Centroid 2 . To smooth the value of S ter (C) and allow for possible all-inclusive clustering result, 1 is added to the denominator and the numerator.
(d) Clustering fitness: the clustering fitness [43] for one of the possible clustering results C, denoted as CF, is defined as Eq. (9) Here, λ (0 < λ < 1) is an experiential weight, S tra (C) is the intra-cluster similarity for the clustering result C and S ter (C) is the inter-cluster similarity for the clustering result C. To avoid biasedness in our experiments, λ is assumed to be 0.5.

Experiments and results
Experimental work has been carried out on the system with Intel(R) Core(TM) i3-5005U CPU@2.00GHz processor speed, 4GB RAM, Windows 7 OS (64-bit) and using JDK1.7.0_45. Separate modules are written for each of the above discussed methods to observe the CPU time for clustering any dataset by keeping the cluster seeds same for all methods. I/O operations are eliminated and the CPU time observed is strictly for clustering of the data.
Along with the newly developed hybrid algorithms, experiments are also conducted with the algorithms for standard K-Means (KM) and Fuzzy C-Means (FCM) for performance comparison. All the algorithms are executed using datasets that are related to four images. The details of these images are available in Table 1.
The medical images used in the present experiment are heart image [44] and kidneys image [45] (Figures 1 and 2). The experiments are also carried out using two benchmark images. They are Baboon and Lena images [46] (Figures 3 and 4).
Below is the brief description of medical images. The Heart is a medical image obtained from biology data repository [44]. It is in "jpeg" format. The 'Kidneys' is a colored MRI scan of a coronal section through a human abdomen, showing the front view of healthy kidneys and liver [45]. It is in 'jpeg' format. The Baboon and Lena are benchmark test images that are found frequently in the literature [46]. These are all in uncompressed "tif" format.
All the algorithms for standard K-Means (KM), standard Fuzzy C-Means (FCM), KMFCM and KMandFCM are executed on each image data with varying number of clusters (k = 10, 11,12,13,14,15). For all algorithms, same cluster seeds are used. Same termination condition Eq. (2) is used for all the experiments. The details of CPU time, clustering fitness and SSE of each algorithm for the all images are given in the following sub-sections (Tables 2-13). The results are also projected in their respective graphs (Figures 5-16).  Table 2. CPU time of each clustering technique (Heart image). Segmenting Images Using Hybridization of K-Means and Fuzzy C-Means  Table 3.
Clustering fitness of each clustering technique (Heart image).

11
Segmenting Images Using Hybridization of K-Means and Fuzzy C-Means    Table 6.
Clustering fitness of each clustering technique (Kidneys image).

13
Segmenting Images Using Hybridization of K-Means and Fuzzy C-Means    Table 9.
Clustering fitness of each clustering method (Baboon image).     Table 11. CPU time of each clustering method (Lena image).   Table 12.
Clustering fitness of each clustering method (Lena image).

19
Segmenting Images Using Hybridization of K-Means and Fuzzy C-Means  6.5 Original images used for present experimentation Figure 13. Heart image. Figure 14.

Comparison of segmentation results on Baboon image
As an example of the present experiments for image segmentation, segmentation results for Baboon image for 10 clusters are presented here. These results are generated by the above proposed hybrid clustering algorithms along with the standard K-Means and standard FCM algorithms.  For segmentation, here, each algorithm is executed using Baboon image data assuming that the number of clusters is 10, i.e., k = 10. Each segment is represented by each cluster. Separate color code is assigned to each cluster. The color codes are red, yellow, green, blue, orange, black, white, gray, cyan and magenta. The projections of all segmentation results generated by the algorithms are shown in Figure 17. The original Baboon image also shown in the figure.
In all the experiments, it is observed that hybrid clustering algorithm KMandFCM is showing better performance in terms of CPU, clustering fitness and SSE than the other algorithms.

Conclusion
The present chapter notably includes the study of hybridization of popular clustering algorithms, K-Means and FCM, and identifies the best hybridization strategy. All experiments are carried out for segmenting four images, which include two medical images also. For all the algorithms CPU time, clustering fitness and sum of squared error (SSE) are taken into consideration while carrying out their performance evaluation. In all the experiments that are conducted, the proposed