Solving classification problems of visual evoked potentials for the brain-computer interfaces

Development of the electroencephalogram-based neurocomputer interfaces requires application of the efficient algorithms for signal analysis. One of the methods of neurocomputer interface development is based on using single visual evoked potentials for characteristics control. However, it is a difficult task, requiring a combination of various methods of signal processing such as Blind Source Separation method, machine learning method and other modern mathematical and computational tools. In this paper, we drew a comparison between various classifiers for the visual evoked potentials recognition problem. The electroencephalogram records analyzed in this paper were published in the public domain.


Introduction
Neurotechnologies, such as neurocomputer interfaces (NCI), are an important branch of modern science. One of the main concepts for the development of neurocomputer interfaces (NCI) using electroencephalogram (EEG) is a visual evoked potentials (VEP) recognition that occurs as a human brain response to the presence of external visual stimuli [1]. A number of ongoing studies are dedicated to the VEP classification algorithms improvement [2][3][4][5][6].
For researchers, it is becoming a common practice to share in the public domain the data received by using expensive lab equipment or in complex experiment conditions. This might be the next step in science development. In this paper we used the data published in the public domain by a group of scientists L. Vareka, P. Bruha, R. Moucek [7].

Objective of research
The objective of the research is to compare various classifiers for the target and non-target single visual evoked cortical potential recognition.

Materials and methods
In this research, the data published in the public domain by a group of scientists was used [8].
The database contains electroencephalograms of 19 test subjects that were presented with visual stimuli and tested under the three-stimulus paradigm -a type of the odd-ball paradigm [9]. EEG were recorded by electroencephalograph (the device make is not specified) with 19 channels and presented in BrainVision format. Flashes of three (red, green and yellow) LED indicators were used as visual stimuli. Stimuli were presented in a random order with a predefined frequency distribution: 83% of nontarget stimulus presentations (red LED flash), 13.5% of target stimulus presentations (green LED), 3.5% IOP Conf. Series: Materials Science and Engineering 862 (2020) 052051 IOP Publishing doi:10.1088/1757-899X/862/5/052051 2 of distracting stimulus presentations (yellow LED). The stimulus duration was 500 ms, the interstimulus interval (ISI) was 1000 ms. The experiment was conducted in three phases with a short break, each test subject was presented with 30 target, 184 non-target and 8 distracting stimuli.
The detailed description of the experiment is provided in work of researchers [7]. In our study datasets with numbers 76, 85, 86, 87, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 104, 105, 106 were used. The classification features were selected according to the method suggested in the paper [2]. The EEG-signal from seven leads (P3, P4, O1, O2, T5, T6, Pz) was used. The features vector was formed as follows: three time-windows were defined in the EEG-signal of each lead from each epoch with a length of 1500 ms from the stimulus presentation. Time-windows interval from the beginning to the end equaled to 80-250, 250-600, 600-800 ms from the beginning of the epoch. This selection method is supported by general characteristics of VEPs and averaged epoch analysis for all test subjects on the lead Pz. The amplitude values for each time-window were averaged by each lead.
The averaged values were classification features vector components. Consequently, the vector consisted of 21 values (3 windows * 7 channels). The features vector and further classification were created in the statistical data processing R software version 3.3.1 (https://www.r-project.org/).
For comparison, the most efficient for the binary classification problem classifiers were selected based on the research data provided in the work [10]. The classifiers belonged to different groupsneural networks, decision trees, Bayes method, nearest neighbors, support vectors method. All the presented algorithms are described in detail and have been used in various applications [11][12][13]. The modifications of algorithms selected for our research purposes were the following: averaged NNet -Neural Networks Using Model Averaging, an ensemble method for averaging the operation of the same neural network.
ELM -Extreme Learning Machines, a simple neural network with one hidden level. Weights connecting inputs to the hidden level are assigned randomly and are not updated.
GBM -Gradient Boosting Models, an ensemble machine learning method. The method represents one of the decision trees implementations.
k-NNk-Nearest Neighbors algorithm, classification based on the distance for k nearest neighbors. Can be applied with different metrics.
LDA -Fisher'sLinear Discriminant Analysis. This well-known method of classification is based on the application of Bayes theorem.
Naive Bayes -implementation of the classification method based on the application of Bayes theorem. Implies the independence of features in the analyzed dataset.
Random Forest -an ensemble machine learning method. Well-known machine learning algorithm. SVM -Support Vector Machines. In this paper, the classifier was applied with two kernels: linear and Gaussian. They are denoted by SVM linear kernel and SVM rbf kernel respectively.
In the purpose of classifiers training the package caret version 6.0 (https://cran.rproject.org/web/packages/caret/index.html) was used, which can set parameters automatically for a wide range of algorithms. During the training all classifiers were tested by 5-fold cross validation method established in the work [11].
Each classifier had been trained 30 times (cycles). In each training cycle, 380 target and non-target epochs were randomly selected from the entire array of epochs (~ 12,680 epochs). After the training was over, the accuracy of classifier recognition was tested on each of the 19 datasets. In order to assess the performance of the classifiers, the accuracy indicators (for reliability of attributing an epoch to a target or non-target category) and the Cohen's kappa (for consistency between true-positive, truenegative, false-positive, and false-negative recognition cases) were selected.

Results and discussion
The research results are presented in figures 1, 2 and tables 1, 2.   The analysis of the obtained results shows that all indicators are very close to a normal distribution. Therefore, the sample can be described by both average value and standard deviation. Table 1 presents averaged accuracy and Cohen's kappa classifications. Table 2 presents detailed data about classification accuracy for each dataset.
LDA and its regularized modifications swLDA and shrinkage-LDA are used in the work [2]. For further development of the presented method, a new averaged NNet classifier is offered, having higher recognition accuracy than LDA and potential for further accuracy increase.
Averaged NNet has higher accuracy and Cohen's kappa than LDA classifier. Whereas, averaged NNet shows a slight accuracy increase in comparison to LDA (on average +1,51%), it shall be noted that averaged NNet has more training options. Consequently, it provides an opportunity to increase the classification accuracy through the effective parameters selection. Due to some of the specifics of its implementation, LDA has almost zero variable parameters, so the adjustability is not possible.
SVM linear kernel classifier also performed well during the tests. Its accuracy is slightly lower than in averaged NNet (on average -1,56%) and LDA (on average -0,05%). The application of SVM group classifiers for the purpose of VEP classification is very common and fully studied, the comparison between LDA and SVM classifiers is presented in the work [14].
Our classifiers accuracy study contributes to and extends the well-known studies on classifiers comparison. For example, some authors [15] compare the accuracy of Fisher LDA, swLDA, PCM (Pearson's Correlation Method), SVM linear, and SVM Gaussian classifiers. The comparison between Bayesian LDA, Fisher LDA, swLDA, SVM linear, SVM Gaussian, multilayer feed-forward neural network, feature extraction method classifier is represented in the study [16].

Conclusion
Obtaining high classification accuracy was not one of the objectives considered in this paper. The study just compared classification accuracy between several classifiers on the same input data. For further development of the EEG pattern recognition method based on the linear LDA classifiers, presented in the works [2,3,[17][18][19], we suggested using averaged NNet classifier based on the simple neural network, because it performed well on solving binary classification problems for target and non-target VEP. The accuracy of averaged NNet classifier can be increased, if the EEG signal pre-processing, spatial filtration and various methods of reducing the dimension of classifying features, as well as setup of the classifier's parameters, are applied.  Table 2. Classification accuracy of recognition target and non-target single VEPs by nine different classifiers. The accuracy (in percent) is presented for each of the 19 datasets. The best accuracy rate in a row (i.e. for each dataset) is marked with bold font. The best accuracy rate in a column (i.e. for each classifier) is marked with underlined italic font. Note: result 77,64% is the best for the dataset 85 and for the classifier SVM linear kernel.

Dataset
No.