Standardization and Its Effects on K -Means Clustering Algorithm

: Data clustering is an important data exploration technique with many applications in data mining. K - means is one of the most well known methods of data mining that partitions a dataset into groups of patterns, many methods have been proposed to improve the performance of the K -means algorithm. Standardization is the central preprocessing step in data mining, to standardize values of features or attributes from different dynamic range into a specific range. In this paper, we have analyzed the performances of the three standardization methods on conventional K -means algorithm. By comparing the results on infectious diseases datasets, it was found that the result obtained by the z-score standardization method is more effective and efficient than min-max and decimal scaling standardization methods.


INTRODUCTION
One of the mosteasiest and generally utilized technique meant for creating groupings by optimizing qualifying criterion function, defined eitherglobally (totaldesign) or locally (on the subset from thedesigns), is the K-means technique (Vaishali and Rupa, 2011). K-means clustering is one of the older predictive n observations in d dimensional space (an integer d) is given and the problem is to determine a set of c points to minimize the mean squared distance from each data point to its nearest center with which each observation belongs. No exact polynomial-time algorithms are known for this problem. The problem can be set up as an integer programming problem but because solving integer programs with a large number of variables is time consuming, clusters are often computed using a fast, heuristic method that generally produces good (but not necessarily optimal) solutions (Jain et al., 1999). The K-means algorithm is one such method where clustering requires less effort. In the beginning, number of cluster c is determined and the centre of these clusters is assumed. Any random objects as the initial centroids can be taken or the first k objects in sequence can also serve as the initial centroids. However, if there are some features, with a large size or great variability, these kind of features will strongly affect the clustering result. In this case, data standardization would be an important preprocessing task to scale or control the variability of the datasets.
The K-means algorithm will do the three steps below until convergence Iterate until stable (= no object move group): • Determine the centroid coordinate • Determine the distance of each object to the centroids • Group the object based on minimum distance The aim of clustering would be to figure out commonalities and designs from the large data sets by splitting the data into groups. Since it is assumed that the data sets are unlabeled, clustering is frequently regarded as the most valuable unsupervised learning problem (Cios et al., 2007).
A primary application of geometrical measures (distances) to features having large ranges will implicitly assign greater efforts in the metrics compared to the application with features having smaller ranges. Furthermore, the features need to be dimensionless since the numerical values of the ranges of dimensional features rely upon the units of measurements and, hence, a selection of the units of measurements may significantly alter the outcomes of clustering. Therefore, one should not employ distance measures like the Euclidean distance without having normalization of the data sets (Aksoy and Haralick, 2001;Larose, 2005).
Preprocessing Luai et al. (2006) is actually essential before using any data exploration algorithms to enhance the results' performance. Normalization of the dataset is among the preprocessing processes in data exploration, in which the attribute data are scaled tofall in a small specifiedrange. Normalization before clustering is specifically needed for distance metric, like the Euclidian distance that are sensitive to variations within the magnitude or scales from the attributes. In actual applications, due to the variations in selection of the attribute's value, one attribute might overpower another one. Normalization prevents outweighing features having a large number over features with smaller numbers. The aim would be to equalize the dimensions or magnitude and also the variability of those features.
Data preprocessing techniques (Vaishali and Rupa, 2011) are applied to a raw data to make the data clean, noise free and consistent. Data Normalization standardize the raw data by converting them into specific range using a linear transformation which can generate good quality clusters and improve the accuracy of clustering algorithms.
There is no universally defined rule for normalizing the datasets and thus the choice of a particular normalization rule is largely left to the discretion of the user (Karthikeyani and Thangavel, 2009). Thus the data normalization methods includes Zscore, Min-Max and Decimal scaling. In the Z-score the values for an attribute X are standardized based on the mean and standard deviation of X, this method is useful when the actual minimum and maximum of attribute X are unknown. Decimal scaling standardized by moving the decimal point of values of attribute X, the number of decimal points moved depends on the maximum absolute value of X. Min-Max transforms the data set between 0.0 and 1.0 by subtracting the minimum value from each value divided by the range of values for each individual value.

MATERIALS AND METHODS
Let, Y = {X 1 , X 2 , …, X n } denote the d-dimensional raw data set. Then the data matrix is an n×d matrix given by: Z-score: The Z-score is a form of standardization used for transforming normal variants to standard score form. Given a set of raw data Y, the Z-score standardization formula is defined as: where, x j and σ j are the sample mean and standard deviation of the jth attribute, respectively. The transformed variable will have a mean of 0 and a variance of 1. The location and scale information of the original variable has been lost (Jain and Dubes, 1988). One important restriction of the Z-score standardization is that it must be applied in global standardization and not in within-cluster standardization (Milligan and Cooper, 1988).

Min-max:
Min-Max normalization is the process of taking data measured in its engineering units and transforming it to a value between 0.0 and 1.0. Where by the lowest (min) value is set to 0.0 and the highest (max) value is set to 1.0. This provides an easy way to compare values that are measured using different scales or different units of measure. The normalized value is defined as: where, µ i is the mean of points in S i :

RESULTS AND DISCUSSION
In this section, details of the overall results have been discussed. A complete program using MATLAB has been developed to find the optimal solution. Few experiments have been conducted on three standardization procedures and compare their performances on K-means clustering algorithm with infectious diseases dataset having 15 data objects and 8 attributes as shown in Table 1. The eight datasets, Malaria dataset, Typhoid fever dataset, Cholera dataset, Measles dataset, Chickenpox dataset, Tuberculosis dataset, Tetanus dataset and Leprosy dataset for X1 to X8 respectively are used to test the performances of the three standardization methods on K-means clustering technique. The sum of squares error representing  distances between data points and their cluster centers and the points attached to a cluster were used to measure the clustering quality among the three different standardization methods, the smaller the value of the sum of squares error the higher the accuracy, the better the result. Figure 1 presents the result of the conventional Kmeans algorithm using the original dataset having 15 data objects and 8 attributes as shown in Table 1. Some points attached to cluster one and one point attached to cluster two are out of the cluster formation with the error sum of squares equal 141.00.
Z-score analysis: Figure 2 presents the result of the Kmeans algorithm using the rescale dataset with Z-score standardization method, having 15 data objects and 8 attributes as shown in Table 2. All the points attached to cluster one and cluster two are within the cluster formation with the error sum of squares equal 49.42 Decimal scaling analysis: Figure 3 presents the result of the K-means algorithm using the rescale dataset with the decimal scaling method of data standardization, having 15 data objects and 8 attributes as shown in Table 3. Some points attached to cluster one and one point attached to cluster two are out of the cluster formation with the error sum of squares equal 0.14 and converted to 140.00. Figure 4 presents the result of the K-means algorithm using the rescale dataset with Min-Max data standardization method, having 15 data objects and 8 attributes as shown in Table 4. Some points attached to cluster one and one point attached to cluster two are out of the cluster formation with the error sum of squares equal 10.07 Table 5 shows the number of points that are out of cluster formations for both cluster 1 and cluster 2. The total error sum of squares for conventional K-means, Kmeans with z-score, K-means with decimal scaling and K-means with min-max datasets.

CONCLUSION
A novel method of K-means clustering using standardization method is proposed to produce optimum quality clusters. Comprehensive experiments on infectious diseases datasets have been conducted to study the impact of standardization and to compare the effect of three different standardization procedures in conventional K-means clustering algorithm. It can be concluded that standardization before clustering algorithm leads to obtain a better quality, efficient and accurate cluster result. It is also important to select a specific standardization procedure, according to the nature of the datasets for the analysis. In this analysis we proposed Z-score as the most powerful method that will give more accurate and efficient result among the three methods in K-means clustering algorithm.