DETECTION OF COVID-19 CHEST X-RAY USING SUPPORT VECTOR MACHINE AND CONVOLUTIONAL NEURAL NETWORK

This study aims to detect whether patients examined are healthy, Coronavirus positive, or just have pneumonia based on chest X-ray data using Convolutional Neural Network method as feature extraction and Support Vector Machine as a classification method or called Convolutional Support Vector Machine. Experiments carried out were comparing the kernel used, feature selection methods, architecture in feature extraction, and separated classes. Our instrument reached the accuracy of 97.33% in the separation of 3 classes (normal, pneumonia, COVID19) and 100% in the separation of 2 classes, that is (normal, COVID19) and (pneumonia, COVID19), respectively. Based on 2 NOVITASARI, HENDRADI, CARAKA ET AL these results, it can be concluded that the feature selection method can improve gained accuracy ±98%.


3
DETECTION OF COVID-19 CHEST X-RAY USING SVM AND CNN classification. Classification is done based on the patient's chest X-ray images. This paper is organized as follows: Section 2 discusses works related to COVID19, Section 2 presents the proposed methodology for COVID19 detection using the Convolutional Support Vector Machine (CSVM) method, Section 3 describes the experiments carried out and presents the results, and finally, Section 4 is about conclusions and future work of this study.

PRELIMINARIES
To obtain important information required regarding COVID19 handling , many studies are conducted on COVID19 for early detection in patients with this disease [5], [13], [14] In line with this, [1] is conducted using deep learning, that is inception v3, resnet50, and inception-resnetv2.
The best result is obtained on resnet50 with the highest accuracy of 98%, but the study has a limitation of classifying only 2 classes, that is normal and COVID19 based on chest X-ray. Another study conducted by [15] uses deep learning method to classify 3 classes, that is COVID-19, influenza-A viral pneumonia, and normal using CT scan or chest X-ray. The results obtained are quite good, with an accuracy of 89.3%. Because of its improved capabilities, this method is suitable for dealing with complex problems or problems that use large-scale data. As a result, the training process on the deep learning method will require a long time [16]. Another method that can be used aside from deep learning method, which functions to classify image data is the Support Vector Machine (SVM) method [17]. According to previous studies, [18] it yields 100% accuracy for the classification of ADHD (Attention Deficit Hyperactivity Disorder). This shows that SVM is beneficial to be used as a classification of 2 classes and requires a faster time. .In another study conducted by [19] using deep learning method that functions to extract features and support vector machine method that functions as a classification method, obtains the best results also on resnet50 with an accuracy of 95.38% but also can only classify two classes based on chest X-ray. The use of both methods are done to obtain a shorter time and better results.

MAIN RESULTS
Main results. The data in this study are chest X-ray images of normal patients, patients infected by Coronavirus, and patients with pneumonia. The data originate from patients who have been tested positive from several countries in the world. Table 1 below shows a breakdown of the data sources used. The data were accessed on April 3, 2020, from each website. The data available on the web will be updated regularly by the data provider, so that if the data are accessed on different dates, it will likely to get different amounts of data. Examples of data obtained can be seen in Figure 1. On chest X-ray images of patients with common pneumonia and patients infected by COVID19, they both have abnormalities in the lung parenchyma. However, based on [20], in the chest X-Ray image, the COVID19 patient shows a degree of turbidity in the lungs which is very visible, whereas pneumonia patient has only white patches, but it does not have excess turbidity.
There are several types of experiments conducted, that is learning models of GoogleNet, Resnet18, Resnet50, Resnet101 with several feature selections, each using PCA and Relief and class separated. Of the several models that are trained, each of them has a level of accuracy. Figure 2 represents the flow of the experiment.

Preprocessing
The initial size of the image owned is very diverse, so it is resized to fit the input size of the architecture used, which is 224 × 224. The size selection is based on [21], where the size of the input image in each architecture for the introduced model is 224 × 224. After resizing, the data is divided into training data and testing data. There are 102 data for each class of pneumonia, normal, and COVID19. 5 DETECTION OF COVID-19 CHEST X-RAY USING SVM AND CNN

3.2.Feature Extraction
Learning in machine learning, after going through the pre-processing stage, is generally continued with feature extraction stage [22], [23]. The method used in this study is convolutional neural network as feature extraction. Convolutional neural network (CNN) is one part of deep learning which is the innovation from the multi-layer perception method and inspired by human artificial neural network [24]. Wiesel and Hubel conducted visual cortex research on the senses of vision of cats [25]. The layer on a convolutional neural network has a 3-dimensional arrangement of length and width, which is the size of the layer and height, which is the depth based on the number of layers. According to the type of layer, [26] represents CNN that can be divided into 2, that is feature extraction layer and fully connected layer. Feature extraction layer, which is located after the input layer in the architectural start, is composed of several layers, each layer is connected to the previous layer. In the feature extraction layer, there are two types of layers, namely convolution layer and pooling layer. Convolutional layers are classified into two ( Figure 3), namely convolution layer one dimension used in vector-shaped data such as signal, time series, etc. [27], and convolution layer two dimensions used in two-dimensional data, for example, imagery and others. Pooling layer or subsampling is a reduction in the size of the matrix. There are two types of pooling layers, that is max pooling and average pooling [28].   Fully Connected Layer consists of several layers, and each layer consists of nodes fully connected to the previous layer. The fully connected layer uses a multi-layer perceptron that functions to process data so that the desired results are obtained.

GoogleNet
GoogleNet is one type of architecture of CNN method created by [31]. GoogleNet has inception modules which carry out various convolutions and unify filters for the next layer [32].
The main characteristic of this model architecture is the excellent utilisation of computing resources in the network. GoogleNet architecture can be seen in Figure 4. The core of GoogleNet architecture is that layers in neural networks are extended to the output of various correlation distributions based on the idea that the neural network output of each layer has optimal efficiency if various distributions are done [33].

ResNet
Residual Neural Network (ResNet) is one type of architecture of CNN method created by [21].
ResNet architecture is quite revolutionary as this architecture became state of the art at that time, that is in classification, object detection, and semantic segmentation. The difference between ResNet and other methods is that there are residual blocks as shown in  The results of each filter of ResNet architecture then go through average pooling before proceeding to the fully connected layer network by using the softmax activation function to 8 NOVITASARI, HENDRADI, CARAKA ET AL determine the classification results [34].
Softmax is an activation function that is commonly used to calculate probabilities and carry out multi-class classifications, where softmax values are between 0 to 1 and have a number of 1 if all elements are added using Equation (1) [35]. This function is used at the end of the layer of the fully connected layer used to produce the probability value of an object's function against the existing class. As CNN method only functions as feature extraction, the process carried out only stops at the last layer before entering the fully connected layer as shown in Figure 6. The description of the ResNet's layer arrangement can be seen in Table 2, and that of the google net's can be seen in Figure 4 and each architecture can be seen in Figure 4.

Principal Component Analysis
Principal Component Analysis (PCA) is a method of feature selection that serves to reduce the number of features used without reducing the characteristics of these features [36]. The PCA steps are as follows: 1. Calculate the covariance matrix using Equation (2). However, X and Y are data, and ̅ , ̅ represents the average.

Relief Algorithm
Relief is a feature selection algorithm using weights to measure the effect of these features.
The higher the weight, the more important the feature is on the data [37]. This algorithm is inspired by instance-based learning [38]. For example, S is training data which has n samples and threshold τ that have values between0 ≤ ≤ 1. This algorithm will detect the effect of the set of features on the target statistically. If it is assumed that each feature is numerical or nominal, then the difference in the value of the feature between the two samples, namely X and Y is defined by the difference function as follows: If and nominal so, If and numeric so, is a normalization unit to make the value of be a range of 0 − 1.

SVM Classification
Support Vector Machine (SVM) is a method used to classify by finding the best hyperplane value and the results obtained from the optimal classification [17], [39]. Vapnik, Guyon, and Boser first presented this method in 1992 in a workshop called the Annual Workshop on Computational Learning Theory [40]. The primary way of working SVM is linear classification and developments are made to be able to work on non-linear problems [41], [42]. Developments made include adding a kernel trick concept to the method that will be used to find the best hyperplane which can separate the distance (margin) between classes from the data maximally [43]. If the distance between the hyperplane with the closest data from each class is the furthest distance that can be obtained [44] [45]. Then the hyperplane can be said to be optimal [46], [47]. Vapnik succeeded in proving that the application of SVM in the real world in the classification problem by separating training data into two classes can work well [48], [49]. The kernels used in SVM are as linear, Radial Basis Function (RBF), sigmoid and polynomial [50]. pooling layers as well as 9 inception blocks as seen in Figure 4. Results of each layer of googlenet architecture can be seen in Figure 8. Next, resnet18, resnet50, and resnet 101 architectures consist of convolution layer, pooling layer, and several residual layers which have different numbers as shown in Table 2. Resnet18 has 8 residual blocks, resnet50 has16 residual blocks, dan resnet101 has 33 residual blocks. As resnet50 and resnet101 have quite many blocks, only results of resnet18 are shown in Figure 7.The first classification model built can be classified into three classes, that is normal, pneumonia, and Covid19. Based on the experiments carried out, the comparison of results from each model is shown in Table 3.The best accuracy results for the kernel and feature selection methods used are linear kernels and using PCA method. The average accuracy obtained is 91.74%. While the best architecture as feature extraction is resnet50 with an average accuracy of 77%. Nevertheless, overall the best accuracy is 97.33% with the architecture of resnet50 using the polynomial kernel and resnet101 using the sigmoid kernel. Both use the same feature selection method, relief. A confusion matrix of the two can be seen in Figure 9.  The second model is a system that can classify images into two classes, that is normal and COVID19. In line with this, we have compared the results from each of the second models shown in Table 4.The best accuracy results for the kernel and feature selection methods used are linear kernels and using PCA method. The average accuracy obtained is 98.04%. However, our best architecture as feature extraction is resnet50 with an average accuracy of 84.25%. Overall, Table  4 representing our second model is quite good at classifying images into two classes as it manages to obtain quite a lot of accuracy by 100%, particularly using feature selection by PCA method and linear kernel in the SVM process.  In the last experiment, we build a system that can classify images into two classes, that is pneumonia and COVID19. However, the best accuracy results for the kernel and feature selection methods used are linear kernels and using PCA method. Table 5 represents our third model and the average accuracy obtained is 98.51%. The best architecture as feature extraction is resnet50 with an average accuracy of 87%. Overall, the third model is quite good at classifying images into two classes as it manages to obtain quite a lot of accuracy by 100%, particularly using feature selection by PCA method and linear kernel in the SVM process

CONCLUSION
Based on several experiments conducted, it can be concluded that the combination of kernel using linear kernel of SVM method and PCA method as a feature selection obtained good results on the three models built. Resnet50 architecture was the best architecture on the three models built.
This is in line with the study conducted [19] which also used deep learning methods as feature