Classification Techniques in Machine Learning: Applications and Issues

: Classification is a data mining (machine learning) technique used to predict group membership for data instances. There are several classification techniques that can be used for classification purpose. In this paper, we present the basic classification techniques. Later we discuss some major types of classification method including Bayesian networks, decision tree induction, k-nearest neighbor classifier and Support Vector Machines (SVM) with their strengths, weaknesses, potential applications and issues with their available solution. The goal of this study is to provide a comprehensive review of different classification techniques in machine learning. This work will be helpful for both academia and new comers in the field of machine learning to further strengthen the basis of classification methods.


INTRODUCTION
Machine Learning (ML) is a vast interdisciplinary field which builds upon concepts from computer science, statistics, cognitive science, engineering, optimization theory and many other disciplines of mathematics and science [1].There are numerous applications for machine learning but data mining is most significant among all [2].Machine learning can mainly classified into two broad categories include supervised machine learning and unsupervised machine learning.
Unsupervised machine learning used to draw conclusions from datasets consisting of input data without labeled responses [3] or we can say in unsupervised learning desired output is not given.Supervised machine learning techniques attempt to find out the relationship between input attributes (independent variables) and a target attribute (dependent variable) [4].Supervised techniques can further classified into two main categories; classification and regression.In regression output variable takes continuous values while in classification output variable takes class labels [5].
Classification is a data mining (machine learning) approach that used to forecast group membership for data instances [6].Although there are variety of available techniques for machine learning but classification is most widely used technique [7].Classification is an admired task in machine learning especially in future plan and knowledge discovery.
*Address correspondence to this author at the Department of Computer Science, Allama Iqbal Open University, Islamabad, Pakistan; Tel: +923217680092; E-mail: aizedamin@yahoo.comClassification is categorized as one of the supreme studied problems by researchers of the machine learning and data mining fields [8].A general model supervised learning (classification techniques) is shown in Figure 1.
Although classification is well known technique in machine learning but it suffers with issues like handling missing data.Missing values in data set can cause problem during both training and classification phases.Some of potential reasons of missing data are presented in [9] includes; Non entry of record due to misconception, data recognized irrelevant at the time of entry, data removal because of deviation with other documented data and equipment malfunction.
Missing data problem can overcome by approaches [10] like; Data miners can overlook the omitting data, swap whole omitting values with an individual global constant, swap an omitting value with its feature mean for the given class, manually observe samples with omitting values and insert a feasible or probable value.In this work we will focus only on some selected classification methods.This paper organized as following; in section 2 methodology of review is presented.Section 3 is divided into four subsections in which selected classification techniques has been discussed.In section 3.1 Logic based technique (decision tree) has been discussed.In section 3.2, statistical learning techniques (Bayesian networks) are discussed.K-Nearest neighbor classifiers are presented in section 3.3.Support Vector Machines has been discussed in section 3.4.

METHODOLOGY
A literature search was performed for the articles by using databases include IEEE xplore, google scholar, science direct and some related web pages that are written in English.The keywords used for literature search include; Machine learning, data mining, classification, classification review, classification applications and classification algorithms.These keywords were used alone and in combination for the initial collection of research material.Only those articles that contain relevant data about classification techniques applications, challenges and solutions were included in this review.It is difficult to provide exhaustive review of all supervised machine learning classification methods in a single article, Therefore we focused only on commonly used classification techniques include Decision Tree (ID3 and C4.5), Bayesian Network, K-Nearest Neighbor and Support Vector Machines.Applications of different classification techniques are presented in Table 1 and issues of classification techniques with their available solutions are presented in Table 2.

CLASSIFICATION TECHNIQUES
Major classification techniques has been discussed in this section with their basic working, advantages and disadvantages.

Decision Tree Induction
Decision tree algorithms are most commonly used algorithms in classification [11].Decision tree provides an easily understandable modeling technique and it also simplifies the classification process [12].The decision tree is transparent mechanism it facilitate users to follow a tree structure easily in order to see how the decision is made [13].In this section basic philosophy of decision tree methods has been discussed with their strengths, limitations and applications.
The core objective of decision tree is to produce a model that calculates the value of a required variable based on numerous input variables [6].Usually all decision tree algorithms are constructed in two phases (i) tree growth; in which training set based on local optimal criteria is splitting recursively until most of the record belonging to the partition having same class label [14] (ii) tree pruning; in which size of tree is reduced making it easier to understand [15].In this section we will focus on ID3 and C4.5 decision tree algorithm.
ID3 (Iterative Dichotomiser 3) decision tree algorithm was introduced in 1986 [16,17].It is one of the widely used algorithms in the area of data mining and machine learning due to its effectiveness and simplicity [16].The ID3 algorithm is based on information gain.Some of the strengths and weaknesses of ID3 decision tree are presented in [18], strengths includes; easy to understand and in final decision whole training example is considered while weaknesses includes; no back tracking search, unable to handle missing values and no global optimization.it minimize its drawbacks caused by ID3.In pruning phase C4.5 tries to eliminate the un-comfort branches by swapping them with leaf nodes by going back through the tree once it has been generated [19].The strengths of C4.5 are dealing training data with missing feature values, deals both discrete and continuous features and providing facility of both pre and post pruning [18,20].The weaknesses includes; not suitable for small data set [18] and high processing time as compare to other decision trees.

Bayesian Networks
A Bayesian Network (BN) refers graphical model for probability associations betwixt a set of variables [21].BN structure S consist directed acyclic graph (DAG) and the nodes in S are in one-to-one communication with the X features.The arcs exemplify unexpected impacts betwixt the nodes while the scarcity of possible arcs in S encodes conditional liberties [2].Normally Bayesian Network learning tasks can be isolated into two subtasks; (a) network DAG structure learning, (b) parameters determination.
One of the problems with Bayesian networks classifier is that it usually requires continuous attributes to be discretized.The process of conversion of continuous attribute into discrete attribute introduced classification issues [22,23].These issues may include noise, missing information and consciousness to the change of the attributes towards class variables [24].The other method of Bayesian network classifier in which continuous attribute does not converted into discrete attribute, needs valuation of the attribute's conditional density [23].
To overcome the problem of conditional density estimation of attributes, in [24] Gaussian kernel function with stable constraints for evaluation of attributes density was used.Then Experiment was [55] [56] [57] [58] performed on data set given at UCI machine learning repository indicate that continuous attributes provides better classification accuracy as compare to other techniques by using Gaussian kernel function in Bayesian Network classifiers.
Some of the advantages of Bayesian network are presented in [25] includes (i) smoothness properties; minor changes in Bayesian network model do not influence the working of the system (ii) Flexible applicability; identical Bayesian Network model can be used for resolving both regression and classification issues (iii) handling missing data; Bayesian network has capability to filled out missing data by assimilating over all opportunities of the missing values.

K-Nearest Neighbor
In K-nearest neighbor (KNN) technique, nearest neighbor is measured with respect to value of k, that define how many nearest neighbors need to be examine to describe class of a sample data point [26].Nearest neighbor technique is divided into two categories i.e, structure based KNN and structure less KNN.The structure based technique deals with the basic structure of the data where the structure has less mechanism which associated with training data samples [27].In structure less technique entire data is categorized into sample data point and training data, distance is calculated between sample points and all training points and the point with smallest distance is known as nearest neighbor [28].
One of the main advantage of KNN technique is that it is effective for large training data and robust to noisy training data [29].Scaling KNN queries over enormous high dimensional multimedia datasets is a stimulating issue for KNN classifiers.To overcome this issue an high performance multimedia KNN query processing system [30] was introduced, in this system the fast distance based pruning methods are coupled with suggested Distance-Pre computation based R-tree (DPR-Tree) index structure.Input/output cost is reduced by this exclusive coupling but it increase the computational work of KNN search.
Two important obstacles with nearest neighbor based classifiers are highlighted in [59] that includes; space requirement and its classification time.Different methods have been introduced to overcome space requirement issue.K-Nearest Neighbor Mean Classifier (k-NNMC) was introduced in [59].K-NNMC independently search k nearest neighbors for every training pattern class and calculate mean for all given k-neighbors.It is presented experimentally by using numerous standard data-sets that the classification accuracy of suggested classifier is better as compare to other classifiers like weighted k-nearest neighbor classifier (Wk-NNC) [60] and it has ability to combine efficiently with any space reduction and indexing methods.
The advantages of KNN include simplicity, transparency, Robust to noisy training data, easy to understand and implement and disadvantages includes computation complexity, memory limitation, poor runtime performance for large training set and irrelevant attributes can cause problems [28,61].

Support Vector Machines
Vapnik proposed statistical learning theory based machine learning method which is known as Support vector machine (SVM) [74].SVM has considered as one of the highest prominent and convenient technique for solving problems related to classification of data [75] and learning and prediction [76].Support vectors are the data points that lie closest to the decision surface [77].It executes the classification of data vectors by a hyper plane in immense dimensional space [78].Maximal margin classifier is the simplest or basic form of SVM that helps to determine the most simple classification problem of linear separable training data with binary classification [27].The maximal margin classifier used to find the hyper plane with maximal margin in real world complications [79].
The main advantage of SVM is its capability to deal with wide variety of classification problems includes high dimensional and not linearly separable problems.One of the major drawback of SVM that it requires number of key parameters to set correctly to attain excellent classification results [80].

CONCLUSION
In this paper various popular classification techniques of machine learning has been discussed with their basic working mechanism, strengths and weaknesses.The potential applications and issues with their available solutions have also been highlighted.Classification methods are typically strong in modeling interactions.The discussed classification techniques can be implemented on different type of data set i.e. health, financial etc.It is difficult to find out which technique is superior to other because each technique has its own merits, demerits and implementation issues.The selection of classification technique depends on user problem domain.However, lot of work has been done in classification domain but it still requires formal attention of research community to overcome classification issues that have been arising due to dealing with new classification problems like problems in classification of Big Data.