Proposing feature engineering method based on deep learning and K-NNs for ECG beat classification and arrhythmia detection

Abstract

Arrhythmia is slow, fast or irregular heartbeat. Manual ECG assessment and disease classification is an error-prone task because of vast differences in ECG morphology and difficulty in accurate identifying ECG components. Moreover, proposing a computer-aided diagnosis system for heartbeat classification can be useful when access to medical care centers is difficult or impossible. Therefore, the main aim of this study is classifying ECG beats for arrhythmia detection (four beat classes are considered). Previous studies have proposed different methods based on traditional machine learning and/or deep learning. In this paper, a novel feature engineering method is proposed based on deep learning and K-NNs. The features extracted by our proposed method are classified with different classifiers such as decision trees, SVMs with different kernels and random forests. Our proposed method has reasonably good performance for beat classification and achieves the average Accuracy of 99.77%, AUC of 99.99%, Precision of 99.75% and Recall of 99.30% using fivefold Cross Validation strategy. The main advantage of the proposed method is its low computational time compared to training deep learning models from scratch and its high accuracy compared to the traditional machine learning models. The strength and suitability of the proposed method for feature extraction is shown by the high balance between sensitivity and specificity.

Appendix A: more details about the classifiers used in this study

In this section, we review the classifiers that are used in this paper including decision trees, support vector machines (SVM), and random forests (RF).

Decision trees

In 1970s and 1980s, decision tree algorithm known as ID3 has been proposed by Quinlan [32]. Further, other extensions of decision tree algorithm such as CART, C4.5 (later C5.0) and J48 have been developed.

Decision trees are built in top-down and divide and conquer manner. The training set is recursively partitioned to make different branches in the decision tree.

A decision tree is represented by a set of nodes which are leaves (terminals) or internal nodes (parents). Each internal node has splits based on a feature to improve the impurity measures (the best split is chosen which reduces the impurity most). The most common and popular impurity measures are Gini Index, Information Gain (Entropy) or misclassification error as formulated in Eq. (A1–A3) [31]:

$$ Gini\,Index\left( D \right) = 1 - \mathop \sum \limits_{j = 1}^{C} p_{j|D}^{2} $$

(A1)

$$ Information\,Gain\left( D \right) = - \left( {\mathop \sum \limits_{j = 1}^{C} p_{j|D} log_{2}^{{p_{j|D} }} } \right) $$

(A2)

$$ Misclassification \,Error\left( D \right) = 1 - max_{1 \le j \le C} \left( {p_{j|D} } \right) $$

(A3)

where D is a node of the decision tree, C is the number of classes and p_j|D is the probability that an arbitrary data record in D belongs to class C_j. The lower values of impurity measures denote better splits.

Each leaf node in the tree is labeled by the majority class label of its data. The classification starts from the root node. In each internal node, the valid branch of the tree which the data record satisfies its corresponding condition is followed. This process continues till the data record reaches to a leaf node and its class label is assigned to the data record.

Pruning decision trees may be done to avoid from overfitting [31].

Decision trees are easily interpretable because of their tree-like structures. The association rules can be extracted easily by traversing the decision tree branches. The number of the association rules extracted from decision trees equals to the number of root-to-leaf paths in the decision tree.

The decision tree obtained by a random sample of our training data based on FS-CNN is displayed in Fig. 13. For simplicity, the classes considered for this sample decision tree are “not N” (abnormal beat) and “N” (normal beat) classes.

Fig. 13

A decision tree inducted by our training dataset based on FS-CNN

Decision tree illustrated in Fig. 13 consists of 15 internal nodes and 16 leaf nodes. The depth of the decision tree is 5 levels. The most important feature appearing in the root node is X9. The second and the third important features are X11 and X34, respectively.

As shown by Fig. 13, 8327 training data records of each class exist at the root node. Entropy of the root node is 1 indicating high impurity. The left branch from the root node is split based on X11 with entropy of 0.508 (much better than the root node) with 944 and 7681 data records of classes “N” and “not N”, respectively.

The first leaf node from the left side of the figure includes 176 samples from which 136 samples (40 samples) belong to “N” (“not N”) class. Therefore, the class label of the leaf node is “N” and every testing data records reach to this node is assigned “N” class label.

The color of the nodes is associated to the class label of the majority data records. The majority data records of blue (orange) nodes belong to “N” (“not N”) class. The color saturation of each node indicates its impurity. Higher saturated nodes have lower impurity measures.

A sample association rule can be extracted by following the root node to the first left leaf node is:

“ If X9 ≤ 0.559 AND X11 ≤ 0.403 AND X16 ≤ 0.431 AND X34 ≤ 0.456 then class Label = N”.

Random forests (RF)

RF is an ensemble classifier of T independent decision trees. Each decision tree is trained by a bootstrap sample of RF training set. For selecting the split feature in the internal nodes of decision trees, the best split among a random subset of features is selected based on the impurity measures. Depth of decision trees should be less than or equal to a user-defined maximal depth to avoid overfitting.

Therefore, RF user-defined hyper-parameters are the number of the trees and the tree maximal depth.

In this paper, RF with 100 decision trees with maximum depth of 3 is applied to our data.

Support vector machines (SVM)

SVM is a classifier trying to find the optimal hyperplane to separate data records of different classes. Original version of SVM is appropriate for classifying linearly separable data. Nonlinear kernels in SVM make nonlinear data linearly separable. Therefore, SVM with nonlinear kernels can classify nonlinear data, too.

The equation of the optimal hyperplane for separating different classes in linear SVM is formulated as Eq. (A4):

$$ \mathop \sum \limits_{i = 1}^{m} w_{i} \cdot FP_{i} + \mathop \sum \limits_{i = 1}^{m} b_{i} = 0 $$

(A4)

where m is the number of the features, w_i is the weight of the i^th feature FP_i in the hyperplane equation and b_i is the i^th element of the bias vector.

Each data record is classified by linear SVM according to Eq. (A5):

$$ y = \left\{ {\begin{array}{*{20}c} { + 1\,if \mathop \sum \limits_{i = 1}^{m} w_{i} \cdot FP_{i} + \mathop \sum \limits_{i = 1}^{m} b_{i} \ge 0} \\ { - 1\,if \mathop \sum \limits_{i = 1}^{m} w_{i} \cdot FP_{i} + \mathop \sum \limits_{i = 1}^{m} b_{i} < 0} \\ \end{array} } \right. $$

(A5)

In this paper, linear (default version of SVM), polynomial, Sigmoid and Radial Basis Function (RBF) kernels are used for SVM with their best tuning of the parameters leading to the best accuracy.