An Efficient Content-Based Image Retrieval System Using kNN and Fuzzy
Mathematical Algorithm

The implementation of content-based image retrieval (CBIR) mainly depends on two key technologies: image feature extraction and image feature matching. In this paper, we extract the color features based on Global Color Histogram (GCH) and texture features based on Gray Level Co-occurrence Matrix (GLCM). In order to obtain the effective and representative features of the image, we adopt the fuzzy mathematical algorithm in the process of color feature extraction and texture feature extraction respectively. And we combine the fuzzy color feature vector with the fuzzy texture feature vector to form the comprehensive fuzzy feature vector of the image according to a certain way. Image feature matching mainly depends on the similarity between two image feature vectors. In this paper, we propose a novel similarity measure method based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF). Finding out the k nearest neighborhood images of the query image from the image data set according to an appropriate similarity measure method. Using the k similarity values between the query image and its k neighborhood images to constitute the new k-dimensional fuzzy feature vector corresponding to the query image. And using the k similarity values between the retrieved image and the k neighborhood images of the query image to constitute the new k-dimensional fuzzy feature vector corresponding to the retrieved image. Calculating the similarity between the two kdimensional fuzzy feature vector according to a certain fuzzy similarity algorithm to measure the similarity between the query image and the retrieved image. Extensive experiments are carried out on three data sets: WANG data set, Corel-5k data set and Corel-10k data set. The experimental results show that the outperforming retrieval performance of our proposed CBIR system with the other CBIR systems.


Introduction
With the rapid development of the multimedia technology and network, image data is growing at an alarming rate every day [1][2][3][4][5]. How to effectively and quickly retrieve the images of interest to user from huge image database is a very important and challenging research topic [6][7][8][9]. The widely used image retrieval methods are text-based image retrieval (TBIR) and content-based image retrieval (CBIR) [10][11][12]. TBIR is to describe the content of an image by using text annotation, which is easy to implement and its retrieval performance is relatively high due to the manual intervention in the process of image annotation [13]. However, TBIR has some drawbacks justly because of the manual intervention. Firstly, TBIR is only suitable for small scale image database owing to the image annotation will cost a lot of manpower and financial resources. Secondly, different annotators may have different cognitive level, different subjective judgment and different use of key words for the same image, which will lead to different annotation results for the same image. Thirdly, it is difficult for users to describe the content of the query image with precise and short keywords, which will directly affect the retrieval performance of TBIR.
CBIR is used to retrieve the desired images from the huge image database based on the desired image content. In the CBIR system, the features of the query image and every retrieved image in the huge image database are extracted and described by lower dimensional feature vectors. The similarity between the feature vector of the query image and the feature vector of every retrieved image are measured and the coordinated retrieve results are fed back to the user as a yield. CBIR is based on the low-level features of the image such as color features, texture features and shape features, which makes CBIR overcome many problems associated with TBIR of retrieving images by image annotation [1,14]. However, CBIR has some defects and shortcomings to some extent because of the 'semantic gap' between low-level features and high-level semantics of image content [15][16][17][18]. So how to describe the content of image objectively and effectively and how to measure the similarity between two images all play the critical roles in the CBIR system. Although many techniques have been proposed to improve the retrieval performance of the CBIR system based on the low-level features, these techniques fail when describing high-level semantic concepts. In order to improve the retrieval performance of CBIR, we adopt fuzzy mathematical algorithm in the process of image feature extraction, and we propose a novel similarity measure method based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF).
The remainder of this paper is summarized as follows: Related work is introduced in Section 2. Image feature extraction is performed in Section 3. Our proposed novel similarity measure method is discussed in Section 4. The experimental results are shown in Section 5. Finally, conclusions are presented in Section 6.

Related Work
A common scheme to more definitely represent an image is that extracting different kinds of low-level features and integrating them together according to a certain way. Color features, texture features and shape features are the most common visual features for an image, and they are widely used in the CBIR system [19][20][21][22][23][24][25][26]. In this paper, we mainly extract the color features and texture features of the image.
Color features are global features, which can describe the surface properties of the scene corresponding to the image or image area. The general color features are based on the features of pixels, and all the pixels of image or image region have their own contribution. Color features have less dependence on the size, direction and visual angle of the image compared with other visual features, so color features have higher robustness [10,11,27]. Various color feature descriptors have been proposed such as the color histogram, the color coherence vector, the dominant color descriptor, the color correlogram, the vector quantization and color moments, the color co-occurrence matrix (CCM) and so on [11]. The color histogram is the most commonly used descriptor to express the color features of the image because of its characteristics of invariant to the orientation and scale of image, and the extraction of color histogram is simple and convenient [28]. In this paper, we adopt the method of Global Color Histogram based on HSV space. However the color features can not capture the local features of the object in the image. So the retrieval performance of the CBIR system only using color features is not ideal.
Texture features describe the change of image gray level, which depict the repeated local patterns and their arrangement rules in images. Texture features are often used in image classification and scene recognition. Various texture feature descriptors have also been proposed including the Gray Level Co-occurrence Matrix (GLCM), the Gabor filtering, the Tamura texture feature, the local binary patterns (LBP), the Markov random field model, the wavelet coefficients and so on [10,11,[29][30][31][32]. GLCM is recognized as an effective texture feature extraction method, which is simple and easy to implement. GLCM belongs to statistical methods, which has the characteristics of small amount of calculation. In this paper, we adopt the method of GLCM. However texture features can only reflect the characteristics of the object surface, and can not fully reflect the essential attributes of the object, so only using the texture features can not get the high-level content of the image.
Generally, color features can be combined with texture features to improve the discrimination power of the image features.
Image feature matching mainly depends on the similarity between two image feature vectors [33][34][35]. Various similarity measure methods have been proposed in recent years. The most widely used similarity measure methods usually including the similarity measure method based on Manhattan distance (MD) [36], the similarity measure method based on Euclidean distance (ED) [36][37][38], the similarity measure method based on cross correlation distance (CCD) and the similarity measure method based on maximum-minimum distance (MMD) [11]. In order to further improve the retrieval performance of the CBIR system, we propose a novel similarity measure method based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF) in this paper.

Image Feature Extraction
Image feature extraction is the first and key work for the CBIR system. In this paper, we extract the color features and texture features of all the images. In order to improve the effectiveness and representativeness of the color features and the texture features, we adopt the fuzzy mathematical algorithm in the process of color feature extraction and text feature extraction respectively.

Color Feature Extraction
In the process of the color feature extraction, we adopt the method of Global Color Histogram (GCH) based on HSV space. Firstly, convert the image from RGB space into HSV space. Secondly, quantify the HSV color space into histogram. In order to reduce the dimension of the color feature vector to improve the retrieval speed of the CBIR system, we take the following measures: H is quantified into 16 bins, S is quantified into 4 bins, V is quantified into 4 bins. Then we get the 256-dimensional color feature vector C corresponding to the image, which is described as follows: In order to further improve the effectiveness and representativeness of the color feature vector, we adopt the fuzzy mathematical algorithm to blur the vector C, then we get the fuzzy color feature vectorC corresponding to the image, andC is described as follows: The relationship betweenc i (i = 1, 2, …, 256) and c i (i = 1, 2, …, 256) is described as follows: where the values of mean(C) and max(C) correspond to the average value and the maximum value of all the elements in the vector C respectively.

Texture Feature Extraction
In the process of texture feature extraction, we adopt the method of Gray Level Co-occurrence Matrix (GLCM). The texture feature vector T of the image consists of 15 characteristics of the GLCM corresponding to the image: the small gradient advantage, the large gradient advantage, the inhomogeneity of gray distribution, the inhomogeneity of gradient distribution, the energy, the mean gray level, the gradient average, the gray mean square deviation, the gradient mean square deviation, the correlation, the gray level entropy, the gradient level entropy, the hybrid entropy, the inertia and the inverse difference moment. The texture feature vector T is described as follows: In order to further improve the effectiveness and representativeness of the texture feature vector, we also adopt the fuzzy mathematical algorithm to blur the vector T, and we get the fuzzy texture feature vectorT corresponding to the image, which is described as follows: The relationship betweent i (i = 1, 2, ……, 15) and t i (i = 1, 2, ……, 15) is described as follows: where the values of mean(T) and max(T) correspond to the average value and the maximum value of all the elements in the vector T respectively.
Finally, we combine the fuzzy color feature vectorC with the fuzzy texture feature vectorT to form the 271-dimensional comprehensive fuzzy feature vectorZ. The relationship betweenZ andC,T is described as follows: 4 Image Feature Matching Image feature matching is a crucial work for the CBIR system, which mainly depends on the similarity between two image feature vectors. In this section, we describe our proposed similarity measure method based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF) and four traditional similarity measure methods: the similarity measure method based on Manhattan distance (MD), the similarity measure method based on Euclidean distance (ED), the similarity measure method based on cross correlation distance (CCD), the similarity measure method based on maximum-minimum distance (MMD).
Assuming that the vectorZ q corresponds to the query image, andZ r m (m = 1, 2, ……, N) corresponds to the m-th retrieved image in the following paper. Where the parameter N stands for the number of retrieved images in the image data set. The vectorsZ q andZ r m (m = 1, 2, ……, N) are described as follows respectively:

Traditional Similarity Models
The similarity model based on Manhattan distance (MD) can be formulated as follows: The similarity model based on Euclidean distance (ED) can be formulated as follows: The similarity model based on cross correlation distance (CCD) can be formulated as follows: The similarity model based on maximum-minimum distance (MMD) can be formulated as follows:

Our Proposed Similarity Model
Our proposed similarity model based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF) is described as follows.
Firstly, finding out the k nearest neighborhood images of the query image from the image data set according to an appropriate similarity measure method. In Section 5, a series of experiments are carried out to prove that the similarity measure method based on maximum-minimum distance (MMD) is superior to the other traditional methods in terms of the average recall and average precision. The experimental results are shown as Tabs. 1-6.
Calculating the similarity between the query image and all the retrieved images by the similarity model based on maximum-minimum distance (MMD), and using all the similarity to constitute the N-dimensional vector S, which is described as follows: Secondly, finding the k nearest neighborhood images of the query image from the image data set according to the similarity between the query image and all the retrieved images. And the parameter k is described as follows: where the value of max(S) stands for the maximum value of all the elements in the vector S.
AssumingQ kNN ðjÞ (j = 1, 2, …, k) stands for the comprehensive fuzzy feature vector corresponding to the j-th neighborhood image of the query image, which is described as follows: Thirdly, using the k similarity between the query image and its k neighborhood images to constitute the new k-dimensional fuzzy feature vectorF q corresponding to the query image, which is described as follows:  where the j-th elementf qj (j = 1, 2, …, k) in the vectorF q stands for the similarity betweenZ q andQ kNN ðjÞ (j = 1, 2, …, k), which is described as follows: Fourthly, calculating the similarity between every retrieved image and the k neighborhood images of the query image according to the similarity model on maximum-minimum distance (MMD), and using the k similarity to constitute the new k-dimensional fuzzy feature vector corresponding to every retrieved image respectively. AssumingF r ðmÞ (m = 1, 2, …, N) is the k-dimensional fuzzy feature vector corresponding to the m-th retrieved image, which is described as follows: Fifthly, calculating the similarity between the k-dimensional fuzzy feature vectorF q corresponding to the query image and the k-dimensional fuzzy feature vector corresponding to every retrieved image according to the following fuzzy similarity algorithm respectively, and the retrieved images are fed back to the user in descending order of the fuzzy similarity. The fuzzy similarity algorithm is described as follows: sim fuzzy ðF q ;F r ðmÞ Þ ¼ 0:5 Â ½_ðF q^Fr ðmÞ Þ þ 0:5 Â ½1 À^ðF q _F r ðmÞ Þ where _ðF q^Fr ðmÞ Þ indicates the maximum value of all the elements in the vector ðF q^Fr ðmÞ Þ, and ðF q _F r ðmÞ Þ indicates the minimum value of all the elements in the vector ðF q _F r ðmÞ Þ.

Experiments and Results
In order to test the outperforming retrieval performance of our proposed CBIR system with the other CBIR systems, a series of experiments are carried out on three image data sets namely the WANG data set, the Corel-5k data set and the Corel-10k data set, which are widely used for the performance evaluation of the CBIR system. The measurement and evaluation of retrieval performance is a crucial problem for the CBIR system. Many different methods have been proposed and used by researchers. In this paper, we have used two most common performance evaluation indexes namely recall and precision. Which are defined as follows respectively: In this paper, we set N = 100, M = 50. The parameter NTOP 100 indicates the number of the relevant images in the top 100 retrieved images. The parameter NTOP 50 indicates the number of the relevant images in the top 50 retrieved images.

First Experimental
WANG data set consists of 1000 color images in 10 different semantic categories, and 100 images for each semantic category. WANG data set is divided into two data sets: one for training and another for testing. The testing data set consists of 100 images (10 images from each semantic category), and the training data set consists of the remaining 900 images in WANG data set. Fig. 1 Figs. 2-11 we can see that the retrieval performance of CBIR based on comprehensive fuzzy features is superior to CBIR based on unfuzzy color features alone in terms of recall and precision for most query images.
To validate the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) is superior to CBIR based on the other four traditional similarity measure methods, a series of experiments are carried out on WANG data set. The comparative experimental results are shown as Tabs. 1 and 2.    Figure 7: The comparison of precision for CBIR based on CCD and different features on WANG data set

Second Experimental
Corel-5k data set consists of 5000 color images in 50 different semantic categories, and 100 images for each semantic category. Corel-5k data set is divided into two data sets: one for training and another for testing. The testing data set consists of 1000 images (20 images from each semantic category). The training data set consists of the remaining 4000 images. Fig. 12 gives a sample of Corel-5k data set To validate the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) is superior to CBIR based on the other four traditional similarity measure methods, a series of experiments are carried out on Corel-5k data set. The comparative experimental results are shown as Tabs. 3 and 4. From Tabs. 3 and 4 we can see that the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features is superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features in terms of recall and precision for most query images. And the average recall and average precision of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features are superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features for all the query images.

Third Experimental
Corel-10k data set consists of 10000 color images in 100 different semantic categories, each of which has 100 images. Corel-10k data set is divided into two data sets: one for training and another for testing. The testing data set consists of 2000 images (20 images from each semantic category). The training data set consists of the remaining 8000 images. Fig. 13 gives a sample of Corel-10k data set images in To validate the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) is superior to CBIR based on the other four traditional similarity measure methods, a series of experiments are carried out on Corel-10k data set. The comparative experimental results are shown as Tabs. 5 and 6.
From Tabs. 5 and 6 we can see that the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features is superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features in terms of recall and precision for most query images. And the average recall and average precision of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features are superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features for all the query images.
When we input the query images shown as Fig. 14

Conclusions
In this paper, we have extracted the color features based on Global Color Histogram (GCH) and the texture features based on Gray Level Co-occurrence Matrix (GLCM) of the image respectively. In order to obtain the effective and representative features of the image, we have adopted the fuzzy mathematical algorithm in the process of color feature extraction and text feature extraction respectively. And we combine the fuzzy color feature vector of the image with the fuzzy texture feature vector of the image to form the 271dimensional comprehensive fuzzy feature vector of the image. A series of experiments are carried out on WANG data set. The experimental results show that the retrieval performance of CBIR based on comprehensive fuzzy features is superior to CBIR based on unfuzzy color features alone in terms of recall and precision.
Image feature matching mainly depends on the similarity between two image feature vectors. In this paper, we have proposed a novel similarity measure method based on k-Nearest Neighbors (kNN) and fuzzy mathematical algorithm (SBkNNF). A series of experiments are carried out on three data sets: WANG data set, Corel-5k data set and Corel-10k data set. The experimental results show that the retrieval performance of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features is superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features in terms of recall and precision for most query images. And the average recall and average precision of CBIR based on our proposed similarity measure method (SBkNNF) and comprehensive fuzzy features are superior to CBIR based on the other four traditional similarity measure methods and comprehensive fuzzy features for all the query images.
Acknowledgement: In this paper, we have extracted the color features and the texture features of the image. In order to obtain the effective and representative features of the image, we have adopted the fuzzy mathematical algorithm in the process of color feature extraction and text feature extraction respectively. However the retrieval performance of CBIR based on comprehensive fuzzy features is not superior to CBIR based on unfuzzy color features alone in terms of recall and precision for all the query images. So the effectiveness and representativeness of the image features need to be further improved. And our proposed CBIR system is mainly suitable for color image retrieval. In the future, we can develop a content-based image retrieval system that combines texture, shape, and semantic features with color features to represent the image. And image segmentation may be used to extract the significant regions or objects from the image that only the segmented regions or objects are used for similarity matching, which will give good results.
Funding Statement: This research was supported by the National Natural Science Foundation of China (Grant Number: 61702310) and the National Natural Science Foundation of China (Grant Number: 61401260). The authors would like to thank the anonymous reviewers for their constructive comments.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.