Image Clustering based on Artificial Intelligence Techniques

Clustering has been widely used in data analysis and pattern recognition and classification. The objective of cluster analysis is the classification of objects according to similarities among them, and organizing of data into groups. There are several algorithms for clustering large data sets or streaming data sets, Their aims are to organize a collection of data items into clusters. These such items are more similar to each other within a cluster, and different than they are in the other clusters. We have take the advantage of classification abilities of Artificial Intelligence Techniques (AITs) to classify images data set into a number of clusters. The Gath-Geva (GG) fuzzy clustering algorithm, Artificial Bee Colony algorithm(ABC), Radial Basis Function Network(RBF), and then combined Gath-Geva algorithm with (RBF) algorithm to produce Fuzzy RBF (FRBF) method were applied using images data set to classify this data set into a number of clusters (classes). Each cluster will contain data set with most similarity in the same cluster and most dissimilarity with the different clusters. We compute the classification rate, and false rate on this data set. Finally we make comparisons between results obtained after applying these algorithms on this images data set. The FRBF is better than the other three methods that applied in this research such as G-G, ABC, RBF, because the FRBF was obtained higher classification rate in testing state equal (96.8571) and low false alarm equal(3.1429).


Introduction
We are living in a world full of data. Every day, people encounter a large amount of information and store or represent it as data, for further analysis and management. One of the vital means in dealing with these large data is to classify or group them into a set of categories or clusters [1]. Clustering is the process of grouping a data set in such a way that the similarity between data within a cluster is maximized while the similarity between data of different clusters is minimized [2]. Clustering or classification systems are either supervised or unsupervised [1], unsupervised clustering takes an unlabelled set of data and partition it into groups of examples, without additional knowledge. Supervised clustering , on the other hand, assumes that the class structure is already known. It takes a set of examples with class labels [3].
Fuzzy clustering is a branch in clustering analysis and it is widely used in the pattern recognition field, such as Gath-Geva algorithm that takes the size and density of clusters for classification [4].
Artificial Bee Colony(ABC) algorithm, an optimization strategy based on the intelligent behavior of honey bee swarm, is one of the swarm intelligence system, it is a population-based algorithm that can provide the best possible solution for different mathematical problems by using inspiration techniques from nature.
Artificial Neural Networks(ANNs) are the most novel and powerful artificial tool suitable for solving combinatorial problems such as predication and classification. ANNs are being used extensively for solving universal problems intelligently like continuous, discrete, and clustering [5].
The rest of the research is organized as follows: Section 2 previous work, Section 3 Gath-Geve Fuzzy Clustering Algorithm, section 4 describes artificial Bee Colony (ABC) algorithm, section 5 discuses Radial Basis Function (RBF), section 6 presents the proposed method which combined Gath-Geva fuzzy clustering with RBF to produce a proposed method called Fuzzy Radial Basis Function (FRBF), section 7 performance measures, section 8 describes the experimental and results obtained, section 9 conclusion .

Previous Work
In particular several classification or clustering algorithms and artificial intelligence techniques were used for images clustering or classification. In 2012 Thai, Hai, and Thuy [6] used the combination between feedforward artificial neural networks(FANN) with support vector machine (SVM) to classify Roman numeral images, the average classification rate was 86%. Akgundogdu et al. [7], used support vector machine and genetic algorithm (GA) for bone sample images, with 12 training and 6 testing data, the success rate for SVM was equal 83.33%. Banerjee, et al. [8], used artificial bee colony to classify band satellite images for different features, the accuracy rate for barren was equal 93.65%. Arias et al. [9] used multi layer perceptron MLP, and Self Organized Maps SOM neural network to classify satellite images, error for MLP is equal 29.65%, and error for SOM equal 18.00%. Rajendran, Madheswaran [10], used Hybird Association Rule with Decision Tree Classification (HARC) to classify brain images the accuracy rate equal 95%. Zhang and Qiu [11] used unsupervised neuro-fuzzy system and some methods to classify Hyperspectral imaging, the overall accuracy for SOM is 73.1%, fuzzy c-means FCM is equal 80.9%, DFLVQ is equal 83.5%, GFSOM is equal 88.6%, ISODATA is equal 53.4%. Luping and Zhang [12], used SVM to classify fingerprint, the accuracy rate is equal 89.3%. Raja, Shanmugam [13], used wavelet features based on backpropagation network to classify images such as, "opencountry" category, and "War Tank" category, the classification rate is equal 82.5%. Arpita and Mahua [14] proposed a computer assisted treatment planning system implementing genetic algorithm based on Neuro-fuzzy approach. The boundary based features of the tumour lesions appearing in breast have been extracted for classification. The shape features represented by fourier descriptors, introduce a large number of feature vectors with classification rate at 87%.

Gath-Geva Fuzzy Clustering Algorithm
Gath-Geva (GG) fuzzy clustering algorithm takes the size and density of clusters for classification [15]. The objective function based on the minimization of the sum of weighted squared distances between the data points and cluster centers is described in the following [16]: = is the set of centers of the clusters, c is the number of clusters, N is the number of the data, m is the fuzzy coefficient, ik D is the difference between k X and the cluster center, where equation (5) calculates ik D , ik  is the membership degree between the i-th cluster and k-th data, which satisfies conditions: The minimum of ( ) is calculated as follows: The norm of distance between i-th cluster and k-th data is : …………………… (6) where the k X denote the input, mi F is the fuzzy covariance matrix of the i-th cluster, ik  is the fuzzy partitioning matrix, m is the weighting exponent controls the 'fuzziness' of the resulting cluster and i P is aprior probability of selecting the i-th cluster. The distance in Eq. (5) is used in the calculation of i P , the probability of selecting the i-th cluster given the k-th data point, is given by: The step of the GG algorithm are listed as follows [15][16] [17]: Step1: determining the number of clusters; m c and value( let 2 = m ), given the converge error, 0   . Step2: generate the matrix U randomly, U must satisfy the condition of equation (2).
, else go to step 3.

Artificial Bee Colony Algorithm
Artificial Bee Colony(ABC) algorithm is a swarm intelligence method which simulates intelligence forging behavior of honey bees [18]. It was first proposed by Karaboga and then further developed by Karaboga, Basturk and Akay et al. [19]. In the model of ABC algorithm, there are three groups of bees; employed bees, onlooker bees and scout bees in the colony of artificial bees. Firstly, half of the colony consists of the employed bees and the second half consists the onlookers. Employed bees go to the food sources, and then they share the nectar and the position information of the food sources with the onlooker bees which are waiting on the dance area determine to choose a food source. The employed bee whose food source has been abandoned by the bees becomes a scout bee that carries out random search in the simulating model. The goal of bees in the ABC model is to find the best solution, the position of a food source represents a possible solution to the optimization problem and the nectar amount of a food source corresponds to the quality (fitness) of the associated solution. In the ABC algorithm, the number of employed bees is equal to the number of food sources which is also equal to the number of onlooker bees. There is only one employed bee for each food source whose first position is randomly generated. At each iteration of the algorithm, each employed bee determines a new neighboring food source of its currently associated food source by Equation (8), and computes the nectar amount of this new food source: Where k is a randomly selected food source different from j i, is a randomly selected dimension.  is a random number between [-1,1]. If the nectar amount of this new food source is higher than that of its currently associated food source, then this employed bee moves to this new food source, otherwise it continues with the old one. After all employed bees complete the search process, they share the information about their food sources with onlooker bees. An onlooker bee evaluates the nectar information taken from all employed bees and chooses a food source with a probability related to its nectar amount by Equation (9).  (10) where i fitness is the fitness value of the solution i which is proportional to the nectar amount of the food source in the position i and SN is the number of food sources which is equal to the number of employed bees. Once all onlookers have selected their food sources, each of them determines a new neighboring food source of its selected food source and computes it's nectar amount. Providing that this amount is higher than that of the previous one, and then the bee memorizes the new position and forgets the old one. The employed bee becomes a scout bee when the food source which is exhausted by the employed and onlooker bees is assigned as abandoned. In other words, if any solution cannot be improved further through a predetermined number of cycles which is called limit parameter, the food source is assigned as an abandoned source and employed bee of that source becomes a scout bee. In that position, scout generates randomly a new solution by Equation (11). Step6: Depending on the probability i P , onlookers choose food sources, search the neighborhood to generate candidate solutions, and calculate their fitness.
Step7: Use the greedy criteria to update the food sources.
Step8: Memorize the best food source and nectar amount achieved.
Step9: Check whether there are some abandoned solutions or not. If true, replace them with some new randomly-generated solutions by Eq(11). Step10: Repeat steps (3)-(9), until the maximum number of iterations is researched or stop conditions are satisfied. As mentioned above, the fitness function is a key component of ABC algorithm, which evaluates the foraging quality of the colony, i.e. the accuracy of possible solutions. Besides, some control parameters, such as the number of employ bees or onlooker bees, the limit times for abandonment, the maximum number of iterations or stop conditions, need to be assigned. They would have a direct influence on the speed and stability of convergence [21].

Radial Basis Function (RBF).
An RBF network is a three-layer feed forward neural network which consists of an input layer, a hidden layer and an output layer [22]. The hidden layer transforms the input vector from the input layer to a higher dimension, by applying a nonlinear transformation from the input space to the hidden layer vector space. Then, from the hidden layer, there is a linear transformation to the output layer. This Artificial Neural Network (ANN) would be suitable for application to a pattern classification. The dimension of the hidden layer is defined by the number of hidden neurons [23] [24].
The structure of a traditional RBF network is shown in figure 1.
The number of neurons in input layer depends on number of input attributes of training samples. The number of neurons in the output layer depends on number of target class labels. Assume that x is an n-dimensional input vector, there are m neurons in output layer and k neurons in hidden layer. It can be formulated as : where i={1,2,…,m}, j={1,2,…,k},  (x) is the output of the jth hidden node, Cj is the center vector of Gaussian kernel function which has the same number of dimension with x, and  is the width of Gaussian kernel function at the jth hidden node. Besides, yi is the output of the ith output neurons and Wij is the connect weight from the jth hidden node to the ith output node.
And then apply the sigmoid function for each neuron in the output layer by this equation: ..……………...…… (14) where L is the number of hidden nodes. Update the weight between output and hidden layers by using equations (15) - (16).
Where  is the learning rate, it is a small number takes value between (0,1) [25][26].

Fuzzy Radial Basis Function Network
Radial basis functions neural network has its origins in techniques for performing exact interpolation of a set of data point. We can generalized to design a Fuzzy Radial Basis Function by adding Gath-Geva Fuzzy clustering algorithm to weight equation (16) to produce a novel method called Fuzzy Radial Basis Function (FRBF), and the algorithm of new method FRBF as follows: Step1: A vector pair (x,y) of the training set, is selected randomly.
Step2: compute the Gaussian Basis Function by the equation (12).
Step3: update the weight of each neuron, the weight updation is performed in accordance to the following The scaling function i R depends on the fuzzy generator m which is a real number greater than 1.
Stepe4: Calculate the output of output layer by equation (14).

Performance Measures
To evaluate the accuracy of a method that was used in this research, we use some indicators: Classification Rate (CR), is obtained by dividing the number of correctly classified items in each category by the number of training set used for that category, False Rate(FR); FR is the ratio between the number of false positive and the total number of testing dataset. The value of (CR), is expected to be as large as possible, while the value of FR is expected to be as small as possible [27][ [28] [29].

Experimental And Results
MATLAB language was used to implemented for all algorithms and equations in this research. To train Gath-Geva(G-G)fuzzy clustering algorithm, Artificial Bee Colony(ABC), and Radial Basis Function(RBF), Fuzzy Radial Basis Function (FRBF), a larger number of training samples are needed. Training and testing dataset obtained from http//www.flowers.vg/flowers [30]. Dataset consist of (239) images with different category of flowers images. We used (116) images for training state, and (123) images for testing state. Each image is normalized using equation (20): Where, x is the numerical value, min is the lower bound of the attribute values, and max is the upper bound of the attribute values [31] [32].
A number of experiments were performed to evaluate the system as well as to investigate the effect of G-G, ABC parameters and RBF, FRBF network parameters. Training and testing dataset consist of 10 categories of images. Each category consists of number of images samples as shown in Table (1). we applied (G-G) , ABC, RBF, and FRBF on flowers images dataset. In the training state we applied these algorithms on training data set that contain (116) flowers images to classify this dataset into (10) classes or clusters. In training state the results of classification obtained is 100% for all these algorithms. In the testing state we used testing dataset that contain (123) flowers images, and applied these algorithm on this dataset. Table(2) and table (3) show the results of the testing by G-G, ABC algorithms with the classification rate for each category of the flowers and over all classification rate for it. ABC was classified a greater number than the G-G algorithm. The ABC is better than G-G method, because ABC give higher classification rate equal (95.0237) than G-G which give classification rate equal( 93.3451), and also ABC give lower false rate equal ( 4.9763) than G-G method which give false rate equal (6.6549). The input data in training state is 116 images and in testing state is 123 images. And the resolution of each image is (114*86). Different parameter setting for ABC as follows: population size=200; and Maximum Cycle Number (MCN) = 10. In G-G method, the row number of membership function U is (10 rows) according to the number of output classes, and the column number of U is (9804) according to the number of pixels in each image in dataset, and the fuzzy coefficient value (m) equal 2. The output of these algorithms are 10 classes each class contain different number of samples.
Table (4) and table (5) show the results of the testing by RBF, FRBF algorithms with the classification rate for each category of the flowers. FRBF was classified a greater number than the RBF algorithm. The FRBF is better than RBF method, because FRBF give higher classification rate equal (96.8571) than RBF which give classification rate equal(89.6189), and also FRBF give lower false rate equal (3.1429) than RBF method which give false rate equal(10.3811). RBF and FRBF used the input dataset(flower images dataset). And the resolution of each image in training and testing state is (114*86). The number of the input nodes in RBF and FRBF are (114 nodes) in input layer, and 10 nodes in hidden layer, the output is 10 classes which are equal to number of nodes in output layer.
Table (6) shows the comparison between G-G, ABC, RBF, FRBF for 10 classes with over all classification rate that obtained for G-G is equal(93.3451). and for ABC is equal to (95.0237). the classification rate that was obtained for RBF is equal (89.6189) and for FRBF is the higher classification rate which is equal to (96.8571).

9-Conclusion
In this research, we presented some of artificial intelligence techniques such as G-G clustering algorithm, ABC algorithm, RBF neural network, and then we combined G-G method to RBF neural network to produce the proposed method called Fuzzy RBF (FRBF) neural network. In training state all these algorithms obtained high classification rate equal 100%. But in testing state, these algorithms obtained different classification and false rate, The proposed method FRBF has achieved high classification rate equal (96.8571), and low false rate equal (3.1429) on testing large number of flower images.