Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue

EEG data was collected from 11 individuals at the University of Essex, England. Before the experiment, the subjects gave their informed consent using a form approved by the Ethics Committee of University of Essex. The consent form contains information about the EEG recording procedure and the purpose of the EEG experiment.

The subjects completed two pre-test self-report measures: visual analogue scale–fatigue (VAS-F) [22] and chalder fatigue scale (CFS) [23]. Thereafter, the subjects performed four different MI tasks: a. left hand movement (Class 1) b. right hand movement (Class 2) c. both feet movement (Class 3), and d. tongue movement (Class 4) for one complete session. A session consists of eight runs. Each run lasts 12 min and consists of 80 trials. Each trial begins with a fixation cross that appears on the computer screen along with a short acoustic warning tone. It is followed by a cue that appears either left, right, down or up of the fixation cross indicating the subjects for the imagination of left hand, right hand, both feet, or tongue movement respectively. The subjects accomplished the desired task until the cue and the fixation cross disappeared from the screen at t  =  6 s. There was a break for 3 s between trials. No extra break was there between runs. The experimental paradigm is illustrated in figure 1.

Zoom In Zoom Out Reset image size

The fatigued state at the end of each run was rated by using a 'fatigue scale'—a subjective scale with a value from 1 to 5 that extends between two extremes (1  =  'Least fatigued' and 5  =  'Most fatigued'). Subjects chose a number along the scale to express the fatigue they were experiencing. This was taken as the fatigue score of that particular run. The subjects completed the post-test self-report using a. VAS-F and b. CFS. Five out of the 11 subjects quit in the middle of the experiment because of high fatigue level.

EEG data was recorded using Biosemi active two system. Sixty four EEG channels were used to record the data following the 10–20 international montage system. The sampling frequency was 256 Hz. EAWICA was used to remove the artefacts [24]. Thereafter, EEG data were low-pass filtered with 40 Hz as the cut off frequency and then subtracted by common average reference. The EEG data from instant t  =  0 to t  =  6 were then analysed.

Monitoring of the growth of mental fatigue through EEG has been carried out on the six subjects who completed all the eight runs of the experiment. Spectral power and spectral entropy were used as features and were computed in four frequency bands, $\delta$ (0.1–3.5 Hz), $\theta$ (4–7.5 Hz), $\alpha$ (8–12 Hz) and $\beta$ (13–30 Hz) from five different areas of the scalp: frontal (F1, F3, F5, F7,Fz, F2, F4, F6, AFz, AF3, AF4 and FPz), parietal (P1, P3, P5, P7, Pz, P2, P4, P6, POz, PO3, PO4 and CPz), temporal (FT8, T8, TP8, FT7, T7, and TP7), central (C1, C3, C5, Cz, C2, C4,C6) and occipital (O1, O2, Oz). The features that exhibit a significant increase during the last run as compared to that during the first run were identified as optimum features and selected for monitoring the growth of mental fatigue. The kernel partial least square (KPLS) algorithm [18] was used for monitoring the growth of fatigue which is a non-linear regression method based on the projection of input (explanatory) variables to the latent vectors (components) [18]. The KPLS algorithm took as input the optimal feature vector and the two class labels, low fatigue (−1) and high fatigue (+1) and predicts KPLS scores as output. The KPLS scores are the projection of the observations onto the KPLS regression coefficients. The KPLS score of each trial can be interpreted as 'fatigue score' of that trial. The fatigue score has been referred as $\mathfrak{f}$ in this study. Since, −1 and  +1 are the two classes, −1 for low fatigue state and  +1 for high fatigue state, KPLS scores lies in the range of [−1 1]. Negative scores represents low fatigue state and positive scores as high fatigue state. The mean of the KPLS scores for each run was then computed which was interpreted as fatigue score of that particular run. The KPLS model was validated with respect to the subjective scores obtained through fatigue scale.

On the basis of the fatigue score obtained through KPLS, each run was classified as either low or high fatigue level using the K-means algorithm. MI EEG feature separability was estimated in terms of DBI, DI, and FS during each level of fatigue. A detailed formulation of fatigue analysis can be found in Talukdar et al [16].

CSP is the most widely used method for feature extraction in MI BCI. It uses a spatial filter to discriminate the spatial patterns for two classes. It takes as input the amplitudes of N channel EEG recorded at T consecutive time points and produces as output an ordered list of distinctive spatial patterns. A few discriminative patterns are substantial for differentiating two classes, and are referred to as CSPs. A detailed formulations of CSP can be found in [14].

CSP computation is based on the covariance matrix of each class. This study proposes a new unsupervised adaptive method named as ADCSP to update CSP during high fatigue level. It updates the covariance matrices of each class by adding regularisation parameters without estimating the target data's class labels. ADCSP is obtained through equations (1) and (2) as follows,

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq1} C_{1}=((1-\gamma)*R\_a)+\frac{\gamma}{N}*trace(R\_a)*I \nonumber \end{align} \tag{ 1 }$$

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq11} C_{2}=((1-\gamma)*R\_b)+\frac{\gamma}{N}*trace(R\_b)*I \nonumber \end{align} \tag{ 2 }$$

where C₁ and C₂ are the updated covariance matrices of class 1 and class 2 respectively, $\beta$ and $\gamma$ are two regularization parameters, I is an identity matrix of $N \times N$ and R$\_a$, R$\_b$ are defined as follows:

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle R\_a=\frac{((1-\beta)*C_{1})+((1-K_{1})*pVect2*\beta)}{N\_a + 1;}; \nonumber \end{align} \tag{ 3 }$$

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle R\_b=\frac{((1-\beta)*C_{2})+((1-K_{2})*pVect2*\beta)}{N\_b + 1;}; \nonumber \end{align} \tag{ 4 }$$

where C₁ and C₂ are the current covariance matrices of class 1 and class 2 respectively, pVect2 is the covariance matrix of testing data with unknown class labels, K₁ and K₂ are the Kullback–Leibler divergence (KL) between the testing and training data of class 1 and class 2 respectively. $N\_a$ and $N\_b$ are the number of training trials of class 1 and class 2 respectively.

KL measures how the probability distribution of target data diverges from the probability distribution of testing data and is computed as follows:

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq123} K_{y}({p_{new}},{p_{train}}) = 0.5 (trace(C_{y}^{-1}C_{y,new}) - log[\frac{det(C_{y,new})}{det(C_{y})}] -M) \nonumber \end{align} \tag{ 5 }$$

where, p _new and p _train are the probability distributions of new target EEG trial and training data respectively, det represents determinant of matrix, C_y ,new and C_y are the covariance matrices of new EEG data and training data respectively, M is the dimension of covariance matrix and y $\in$ $\{1,2\}$.

Since, K_y(p_new,p_train) $\neq$ K_y(p_train,p_new), K_y can be symmetrized as follows [7]

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq3} K_{y}=0.5( K_{y}(\,p_{new},p_{train})+ K_{y}(\,p_{train},p_{new})). \nonumber \end{align} \tag{ 6 }$$

The idea of using $\beta$ and $\gamma$ is derived from the work of Lu et al [25]. Large $\beta$ shrinks the current covariance matrix towards the testing covariance matrix while large $\gamma$ shrinks the current covariance matrix towards the identity matrix to counter the bias due to small training set [25]. The values for $\beta$ was searched in the range of [0,1] and [1,10] while the values for $\gamma$ was searched in the range of [0,1] for all subjects. Two different ranges were used for $\beta$ because in case the high fatigue testing data is quite different from the training data, larger $\beta$ value works. The value of $\beta$ and $\gamma$ within the aforesaid ranges that give the best validation performance (in terms of DBI, DI and FS) is chosen for analysis. However, unlike Lu et al [25], in addition to the two regularisation parameters $\beta$ and $\gamma$, we also used KL between training and testing data to update the spatial filter. KL has been used to update the covariance matrices because based on the KL we can estimate the difference between the training and testing data and update the covariance matrix of each class accordingly.

2.3.1. Implementation of ADCSP

ADCSP can be implemented for both offline or in near real-time analysis. The adaptation in near real-time is done on trial basis while offline adaptation is done on all the runs that come under high fatigue level. In figure 2 we show when the adaptation is activated. The EEG data were divided into training and evaluation data. The training data (T) comprises of first two runs while the evaluation data (E) comprises of runs 3–8 for the offline analysis and runs 3–7 runs for near real-time analysis. For offline analysis, evaluation data were clustered into two levels based on the fatigue score using the K-means clustering: low fatigue (L) and high fatigue (H). No adaptation of the spatial filter was carried out during low fatigue level, but adaptation was performed during high fatigue level. While for near real-time analysis, the trial with positive fatigue score is classified as high fatigue trial and negative fatigue score as low fatigue trials. Adaptation is activated when a high fatigue trial is found.

Zoom In Zoom Out Reset image size

The detailed implementation of ADCSP for offline analysis (named as ADCSP-I) is shown in algorithm 1 and implementation of ADCSP in near real-time (named as ADCSP-II) is shown in algorithm 2.

• (i)  
  ADCSP-I: ADCSP-I relies on fatigue score ($\mathfrak{f}$) of a particular run based on which the fatigue level of the user during that run can be estimated. $\mathfrak{f}$ is estimated as illustrated in section 2.2. Hence, ADCSP-I starts by performing fatigue analysis on the evaluation data to compute $\mathfrak{f}$ of each run. ADCSP-I then categorises the runs of the evaluation data into two levels of fatigue: low fatigue and high fatigue based on the fatigue score using the K-means clustering algorithm. The adaptation is done on the runs that come under high fatigue level. The detailed algorithm is shown in algorithm 1.

Algorithm 1. ADCSP-I—ADCSP for offline analysis.

The fatigue score $\mathfrak{f}$ is calculated by the following function namely Fatigue_Analysis.

• Fatigue_Analysis (i, E): For any run i of evaluation data E, this module performs the fatigue analysis of a subject during the ith run and compute the fatigue score $\mathfrak{f}$. Based on $\mathfrak{f}$, the fatigue level during ith run is estimated using the K-means clustering algorithm.

• (ii)  
  ADCSP-II: ADCSP-II adapts the CSPs in near real-time on trial basis. To explain how this algorithm works, some other parameters, such as the length of the adaptation window and the initial time when the adaptation should start are used and explained as follows:

  • (a)  
    Fatigue score ($\mathfrak{f}$): This gives the score of fatigue level of the user during a particular trial. The fatigue score is estimated through EEG as illustrated in section 2.2. The KPLS score of each trial was computed and interpreted as the fatigue score of that particular trial.
  • (b)  
    Length of the adaptation window: The length of the adaptation window is the time period when the adaptation starts till it reaches the stopping criteria as defined in equation (7)

    $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{ad2} |f2(k)-f2(k-1)|<\phi, \nonumber \end{align} \tag{ 7 }$$

    where, $\phi$ is a pre-determined positive constant, k is the kth iteration and

    $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{ad3} f2(k)=\sum_{i=1}^{N}|\frac{(W_{i}^{(k)})^TS_{L}^{(k)}(W_{i}^{(k)})}{(W_{i}^{(k)})^TS_{R}^{(k)}(W_{i}^{(k)})}| \nonumber \end{align} \tag{ 8 }$$

    where W$_{i}^{(k)}$ is the ith column of the spatial filter, S$_{L}^{(k)}$ and S$_{R}^{(k)}$ are computed as follows

    $$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq41} S_{L}^{(k)} = {C}_{1}-{C}_{2}, S_{R}^{(k)} = {C}_{1} + {C}_{2} \nonumber \end{align} \tag{ 9 }$$

    with C₁ and C₂ being the covariance matrices of Class 1 and Class 2 respectively.Equation (8) computes the sum of Rayleigh coefficients. Since the advantage of CSP feature extraction has been demonstrated in the paradigm of maximization of Rayleigh coefficients, the improvement in the sum of Rayleigh coefficients leads to better class separability of the features [26]. Hence, this study uses this concept as the stopping criterion of the adaptive scheme.
  • (c)  
    Initial time (init): Is the initial time when the adaptation starts which is updated based on the fatigue score estimated through EEG. When fatigue level is high, we hypothesize that adapting the trial with high fatigue level would yield better performance and hence the adaptation starts. The updating of init is described in algorithm 2.

Algorithm 2. ADCSP-II - ADCSP in near real-time.

Both ADCSP-I and ADCSP-II rely on fatigue score as well as values of $\beta$ and $\gamma$. ADCSP-II depends on four major modules namely Fatigue_Analysis, Adapt_CSP, Check, and After_adapt. The evaluation data E is divided into different windows of length 'p' so as to perform the monitoring of the fatigue level efficiently. Hence, ADCSP-II also relies on the values of 'p ' and 'n_t'. n_t is a variable used to denote the number of contiguous high fatigue trials. Different values of p  ($p=50,100,200,300, 400$) and n_t (n_t $\in$ [1 10] is an integer) have been tested and the best one (p   =  200, n_t  =  10) were used for the study. The range of p  and n_t is same for all the subjects.

• Fatigue_Analysis (i, E): Unlike Fatigue_Analysis of ADCSP-I, this module computes the fatigue score of each trial. For any trial i of evaluation data E, the module performs the fatigue analysis of a subject during the ith trial and compute the fatigue score $\mathfrak{f}$. Based on $\mathfrak{f}$, the level of fatigue during ith trial is estimated using K-means algorithm.
• Adapt_CSP (i, E, W_o, t): This module updates the existing CSP projection matrix (W_o) to compute new CSP projection matrix (W_new) considering the ith trial of the evaluation data E.
• Check (W_new, W_o, E, t, i): The stopping criterion of the adaptive process is checked in this module.
• After_adapt(i,E, t, W_new): This module is activated when adaptation stops. It checks for 'n_t' contiguous high fatigue trials. If 'n_t' contiguous high fatigue trials are found, then the module starts adapting the subsequent trials, otherwise no adaptation is performed. This process continues for a specific window length 'p'.

Separability of extracted features can be computed directly by using certain metrics like DBI, FS, etc or indirectly in terms of classification accuracy [27].

In object classification, one needs to predict the class of an unseen object by identifying the class-distribution pattern. This is only possible when the diction ' similar objects tend to cluster together' is true [28]. Due to the high dimensionality of the dataset, it cannot be immediately visualized to identify the class-distribution [28]. Hence, using a method that can give information on the separability of features without using multiple sets of computationally expensive classifiers would be advantageous [28]. And there comes the role of separability indices. Since classification performance depends on the separability of the classes, it can be deduced that the higher separability of features leads to the increase in classification performance [27].

This study uses three such separability indices, DBI, DI and FS to evaluate the separability of features. Lower value of DBI and higher value of DI and FS indicates higher MI EEG feature separability. Unlike using classifiers, no pre-training is required when using separability indices. They are independent of the number of groupings and the grouping algorithm used [29] and hence are simple, feasible and time saving [30]. Friedman statistical test has been carried out to examine statistical significance. The significant difference between the separability of MI EEG features extracted with C-CSP and ADCSP and that with ACSP and ADCSP is shown by $\checkmark$.

The growth of fatigue was tracked on the six subjects who completed the whole experiment using the KPLS algorithm that consists of two key steps: i. KPLS model selection and ii. KPLS model prediction. KPLS model is subject-specific. During the KPLS model selection, the optimal number of KPLS components (KPLS latent vectors) for each subject that provides the maximum classification accuracy with Linear Discriminant Analysis (LDA) was identified. For estimating the optimal number of components, the first and the last runs were used as training as well as testing data, with the first run as active state and the last run as fatigue state. The KPLS components were evaluated in the range of 1 to 10. During KPLS model prediction, the KPLS scores of each trial of all the runs were predicted. In our work [16], the analysis of EEG spectral power and spectral entropy from different frequency bands in different areas of the scalp showed that spectral power increases during the last run as compared to that of the first run. $\delta$, $\theta$ and $\alpha$ power from frontal region, $\alpha$ power from parietal lobe, $\delta$, $\theta$ and $\alpha$ power from temporal lobe and $\theta$ power from occipital lobe has been considered as optimal features since they show significant increase during the last run as compared to that of the first run; while $\beta$ power from all the lobes shows insignificant change between the first and last runs. The KPLS score of each trial can be interpreted as the fatigue score of that trial. The mean of the KPLS scores of each run was then computed and interpreted as the fatigue score of that particular run. For each of the six subjects, the fatigue scores were plotted in a graph as shown in figure 3.

Zoom In Zoom Out Reset image size

The KPLS model was validated with respect to the subjective scores obtained through fatigue scale. Table 1 shows the correlation between the KPLS model and the subjective scores. The first column shows the Subject id, 2nd and 3rd column shows the correlation between KPLS model and the subjective scores taking runs 1–8 and 2–7 into account respectively. This is because, runs 1 and 8 were used for training the KPLS model and hence training may find some combination of input features to maximise the separability between runs 1 and 8. Hence, correlation was also found for runs 2–7.

Table 1. Correlation between the trends of KPLS scores and subjective scores.

	Runs 1–8		Runs 2–7
ID	Corr. Coeff	p-value	Corr. Coeff	p-value
S1	0.9825	$1.32 \times 10^{-5}$ $\checkmark$	0.92	0.025 $\checkmark$
S2	0.891065	0.001 418 $\checkmark$	0.66	0.21$\times$
S3	0.917	0.0004 $\checkmark$	0.88	0.048 $\checkmark$
S5	0.960346	$0.29\times 10^{-5}$ $\checkmark$	0.95	0.005 $\checkmark$
S6	0.89	0.0032 $\checkmark$	0.82	0.065$\times$
S11	0	1 $\times$	0	1$\times$

The result shows that five subjects showed a strong correlation (>0.5) between the KPLS scores and the subjective scores. In case of Subject 11 no correlation was found. Hence the rest of the analysis was carried out on the five subjects. Talukdar et al [16] has detailed formulation for fatigue analysis.

Based on these computed fatigue scores, each run of the evaluation data was then categorized as either low or high fatigue state by using the K-means algorithm [16]. Table 2 shows the runs categorized as low and high fatigue level based on the computed fatigue score. The first column shows the subject id while the second and the third columns show the runs categorized as low and high fatigue level respectively.

3.2.1. Feature extraction and selection

3.2.2. ADCSP-I: ADCSP for offline analysis

• (i)  
  Training and testing sessions: To carry out the offline analysis, the first two runs were used to compute the optimal spatio-temporal filter. The last six runs were used as evaluation data. Evaluation data were categorized as low and high fatigue level by K-means algorithm as described in section 2.2. Table 2 shows the runs of the experiment that have been categorized as low or high fatigue level. The CSPs during high fatigue level were then adapted using algorithm 1. ACSP for offline analysis is implemented in the same way as ADCSP-I.
• (ii)  
  Results: ADCSP-I has been tried on the runs that have been categorized as high fatigue level.

Tables 3–5 show the separability of MI EEG features extracted with ADCSP-I in terms of DBI, FS and DI respectively. In these tables, the top half show the results for FB₁ while the bottom half show the results for FB₂. The first column shows the Subject id, 2nd to 13th columns present the DBI/FS/DI values of C-CSP, ACSP and ADCSP-I for the four segments ($\omega_{1}$,$\delta_{1}$;$\omega_{1}$,$\delta_{2}$; $\omega_{2}$,$\delta_{1}$; $\omega_{2}$,$\delta_{2}$) respectively. The results of the Friedman statistical test are shown in table 6 which demonstrate that ADCSP-I significantly outperformed C-CSP and ACSP in about 80% of the tests. Figure 4 shows the class distributions of each task for Subject 1. The other subjects have similar distribution pattern. The center of each ellipse represents the class mean under that condition while the size of the ellipse represents the 95% confidence interval for the class, oriented along the eigenvectors of the covariance matrix. It is seen that all the four classes are more separable in case of ADCSP while in case of C-CSP, the classess are inseparable.

Table 3. DBI score from offline analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	15	13	12	17	19	14	13	14	12	20	20	15
S2	15	12	12	15	13	13	15	15	11	15	15	11
S3	26	21	19	27	23	18	23	21	15	25	21	18
S5	15	15	12	13	11	11	15	15	12	14	13	13
S6	17	19	16	17	19	16	17	20	14	17	20	14
Avg	17.6	16	14.2	17.8	17	14.4	16.6	17	12.8	18.2	17.8	14.2

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	16	16	13	17	18	14	15	15	12	21	16	15
S2	15	12	12	13	12	11	18	13	11	18	13	11
S3	28	21	20	25	20	18	32	25	18	28	23	17
S5	12	14	10	15	15	12	12	10	10	14	13	13
S6	18	19	15	20	22	14	18	20	14	18	20	14
Avg	17.8	16.4	14.2	18	17.4	13.8	19	16.6	13	19.8	17	14

Table 4. Fisher score from offline analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	0.04	0.074	0.21	0.05	0.05	0.17	0.007	0.097	0.14	0.02	0.097	0.19
S2	0.58	0.29	1.01	0.41	0.87	1.03	0.3	0.83	1.01	0.3	0.83	1.01
S3	0.004	0.01	0.027	0.003	0.03	0.067	0.009	0.04	0.068	0.004	0.03	0.067
S5	0.27	0.14	0.35	0.23	0.24	0.38	0.27	0.12	0.35	0.21	0.12	0.26
S6	0.06	0.049	0.09	0.07	0.06	0.12	0.06	0.07	0.16	0.06	0.07	0.16
Avg	0.19	0.26	0.47	0.15	0.25	0.35	0.13	0.23	0.35	0.12	0.23	0.34

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	0.07	0.11	0.24	0.04	0.052	0.35	0.02	0.09	0.13	0.01	0.04	0.32
S2	0.49	0.93	0.83	0.79	0.87	1.54	0.3	0.82	0.99	0.3	0.81	0.99
S3	0.005	0.01	0.03	0.001	0.02	0.04	0.003	0.03	0.07	0.007	0.01	0.02
S5	0.22	0.32	1.32	0.16	0.24	0.41	0.22	0.32	1.32	0.15	0.25	0.25
S6	0.06	0.05	0.1	0.06	0.05	0.09	0.06	0.1	0.13	0.06	0.1	0.13
Avg	0.17	0.28	0.51	0.21	0.25	0.49	0.13	0.27	0.53	0.11	0.24	0.34

Table 5. DI score from offline analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	0.07	0.1	0.09	0.08	0.07	0.11	0.1	0.08	0.12	0.07	0.06	0.09
S2	0.13	0.14	0.16	0.11	0.12	0.14	0.1	0.11	0.17	0.1	0.11	0.17
S3	0.08	0.08	0.09	0.07	0.08	0.09	0.07	0.09	0.11	0.07	0.08	0.09
S5	0.1	0.13	0.14	0.11	0.15	0.16	0.1	0.12	0.14	0.1	0.13	0.14
S6	0.1	0.097	0.12	0.1	0.094	0.11	0.1	0.014	0.13	0.1	0.01	0.13
Avg	0.09	0.11	0.12	0.09	0.1	0.12	0.09	0.09	0.13	0.09	0.08	0.12

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I	CSP	ACSP	ADCSP-I
S1	0.07	0.08	0.09	0.1	0.07	0.11	0.07	0.08	0.12	0.07	0.08	0.09
S2	0.11	0.14	0.14	0.07	0.14	0.17	0.08	0.12	0.16	0.08	0.13	0.16
S3	0.06	0.07	0.08	0.09	0.09	0.1	0.05	0.07	0.09	0.05	0.08	0.09
S5	0.16	0.12	0.17	0.1	0.12	0.14	0.16	0.12	0.17	0.1	0.13	0.14
S6	0.09	0.09	0.12	0.08	0.07	0.13	0.09	0.09	0.13	0.09	0.09	0.13
Avg	0.09	0.1	0.12	0.09	0.1	0.13	0.09	0.1	0.14	0.08	0.1	0.13

Table 6. Statistical significance test of ADCSP-I versus C-CSP and ACSP for offline analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$		$\omega_{1}$,$\delta_{2}$		$\omega_{2}$,$\delta_{1}$		$\omega_{2}$,$\delta_{2}$
	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I
DBI	0.045 ($\checkmark$)	0.05 ($\times$)	0.045 ($\checkmark$)	0.13 ($\times$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.06 ($\times$)
Fisher Score	0.03 ($\checkmark$)	0.06 ($\times$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.03 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)
DI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.03 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.07 ($\times$)

	FB1
	$\omega_{1}$,$\delta_{1}$		$\omega_{1}$,$\delta_{2}$		$\omega_{2}$,$\delta_{1}$		$\omega_{2}$,$\delta_{2}$
	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I	CSP versus ADCSP-I	ACSP versus ADCSP-I
DBI	0.045 ($\checkmark$)	0.06 ($\times$)	0.045 ($\checkmark$)	0.13 ($\times$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.03 ($\checkmark$)	0.06 ($\times$)
Fisher Score	0.06 ($\times$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.03 ($\checkmark$)	0.045 ($\checkmark$)
DI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.06 ($\times$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.03 ($\checkmark$)

Zoom In Zoom Out Reset image size

3.2.3. ADCSP-II: ADCSP in near real-time

• (i)  
  Training and testing sessions: To evaluate the performance of ADCSP-II, the first two runs were used for training to compute the optimal spatio-temporal filter. The first and the last run were used for training the KPLS model for fatigue analysis while the 3rd to 7th runs were used for evaluation. CSP was adapted on trial basis. ACSP for near real-time analysis is implemented in the same way as ADCSP-II.
• (ii)  
  Results: Tables 7–9 show the separability of MI EEG features extracted with ADCSP-II in terms of DBI, FS, and DI respectively. In these tables, the top half show the results of FB₁ while the results of FB₂ are shown in the bottom half. The first column shows the Subject id, the 2nd to 13th columns present the DBI/FS/DI values of C-CSP, ACSP and ADCSP-II for the four segments (($\omega_{1}$,$\delta_{1}$;$\omega_{1}$,$\delta_{2}$; $\omega_{2}$,$\delta_{1}$; $\omega_{2}$,$\delta_{2}$) respectively. The results of the Friedman statistical test are shown in table 10, which demonstrate that ADCSP-II significantly outperformed C-CSP and ACSP in about 98% of the tests.

Table 7. DBI score from near real-time analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	12	20	11	13	21	12	9	25	11	17	23	16
S2	19	31	18	18	25	16	21	20	18	21	41	18
S3	33	27	25	37	22	26	34	26	26	30	26	24
S5	17	30	16	17	21	16	17	30	16	16	27	15
S6	25	30	20	20	29	19	26	29	17	26	27	17
Avg	21.2	27.6	18	21	23.6	17.8	21.4	26	17.6	22	28.8	18

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	16	25	12	12	27	11	16	25	12	21	30	16
S2	19	31	17	20	30	18	19	25	18	19	25	18
S3	35	28	27	37	24	25	28	27	25	35	30	24
S5	19	27	16	19	28	16	19	29	16	17	27	16
S6	24	30	17	24	26	21	22	24	16	22	23	16
Avg	22.6	28.2	17.8	22.4	27	18.2	20	26	17.4	22.8	27	18

Table 8. Fisher score from near real-time analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	0.06	0.024	0.25	0.14	0.002	0.2	0.12	0.003	0.34	0.06	0.01	0.07
S2	0.14	0.01	0.22	0.17	0.007	0.34	0.14	0.005	0.3	0.14	0.05	0.3
S3	0.005	0.003	0.18	0.004	0.008	0.009	0.005	0.01	0.01	0.005	0.003	0.01
S5	0.15	0.007	0.28	0.15	0.006	0.28	0.15	0.002	0.28	0.25	0.04	0.33
S6	0.02	0.001	0.08	0.03	0.005	0.05	0.02	0.003	0.16	0.02	0.004	0.16
Avg	0.07	0.01	0.17	0.09	0.006	0.18	0.09	0.004	0.22	0.09	0.021	0.17

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	0.12	0.03	0.15	0.12	0.02	0.26	0.1	0.01	0.3	0.07	0.002	0.11
S2	0.14	0.08	0.24	0.15	0.09	0.34	0.20	0.09	0.21	0.2	0.07	0.21
S3	0.005	0.005	0.02	0.004	0.006	0.02	0.009	0.003	0.018	0.005	0.006	0.011
S5	0.03	0.006	0.51	0.03	0.002	0.51	0.03	0.006	0.51	0.24	0.007	0.25
S6	0.06	0.002	0.1	0.06	0.06	0.23	0.03	0.01	0.08	0.03	0.01	0.08
Avg	0.07	0.025	0.21	0.09	0.04	0.27	0.08	0.02	0.3	0.11	0.02	0.13

Table 9. DI score from near real-time analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	0.11	0.06	0.13	0.11	0.04	0.13	0.19	0.05	0.15	0.09	0.05	0.11
S2	0.1	0.03	0.11	0.09	0.08	0.12	0.08	0.03	0.11	0.08	0.03	0.11
S3	0.05	0.07	0.07	0.04	0.08	0.06	0.05	0.07	0.06	0.06	0.06	0.07
S5	0.11	0.06	0.13	0.11	0.06	0.13	0.11	0.07	0.13	0.11	0.06	0.12
S6	0.08	0.05	0.09	0.09	0.05	0.1	0.06	0.002	0.09	0.06	0.06	0.09
Avg	0.09	0.05	0.11	0.09	0.06	0.11	0.09	0.04	0.11	0.08	0.07	0.10

	FB2
	$\omega_{1}$,$\delta_{1}$			$\omega_{1}$,$\delta_{2}$			$\omega_{2}$,$\delta_{1}$			$\omega_{2}$,$\delta_{2}$
Subjects	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II	CSP	ACSP	ADCSP-II
S1	0.08	0.05	0.14	0.13	0.05	0.14	0.08	0.05	0.14	0.07	0.03	0.11
S2	0.1	0.01	0.09	0.11	0.03	0.10	0.09	0.03	0.12	0.09	0.03	0.12
S3	0.04	0.06	0.07	0.04	0.08	0.06	0.05	0.07	0.06	0.05	0.07	0.06
S5	0.10	0.06	0.11	0.1	0.06	0.11	0.1	0.06	0.11	0.11	0.06	0.12
S6	0.07	0.04	0.1	0.007	0.006	0.09	0.08	0.07	0.11	0.08	0.01	0.11
Avg	0.09	0.04	0.1	0.07	0.06	0.1	0.08	0.06	0.1	0.08	0.04	0.11

Table 10. Statistical significance test of ADCSP-II versus C-CSP and ACSP for near real-time analysis.

	FB1
	$\omega_{1}$,$\delta_{1}$		$\omega_{1}$,$\delta_{2}$		$\omega_{2}$,$\delta_{1}$		$\omega_{2}$,$\delta_{2}$
	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II
DBI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.06 ($\times$)
Fisher Score	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)
DI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)

	FB1
	$\omega_{1}$,$\delta_{1}$		$\omega_{1}$,$\delta_{2}$		$\omega_{2}$,$\delta_{1}$		$\omega_{2}$,$\delta_{2}$
	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II	CSP versus ADCSP-II	ACSP versus ADCSP-II
DBI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)
Fisher Score	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)
DI	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)	0.045 ($\checkmark$)

Tables 3–5 show that the average separability of MI EEG features extracted with ADCSP-I across all the five subjects is higher in terms of DBI, FS, and DI than that with C-CSP. Similarly, tables 7–9 portray that the average separability of MI EEG features extracted with ADCSP-II across all the five subjects is higher in terms of DBI, FS, and DI than that with C-CSP. The Friedman statistical test as shown in tables 6 and 10 reveals that the separability of MI EEG features extracted with ADCSP-I and ADCSP-II increases significantly in terms of DBI, FS, and DI when compared to that with C-CSP. The improved separability of MI EEG features with ADCSP over C-CSP demonstrates clear benefits of adapting CSP during high fatigue level. Therefore, the adaptation based on cognitive state provides a means to improve the separability of MI EEG features.

Tables 3–5 show that the average separability of MI EEG features extracted with ADCSP-I across all the five subjects is higher in terms of DBI, FS and DI than that with ACSP in almost all the cases except in the case of S2 with $\omega_{1}$,$\delta_{1}$, FB₁ and $\omega_{1}$,$\delta_{1}$, FB₂ where ACSP shows better performance in terms of FS, while in the case of S2 with $\omega_{1}$,$\delta_{1}$, FB₁, $\omega_{1}$,$\delta_{2}$, FB₁ and $\omega_{1}$,$\delta_{1}$, FB₂, S5 with $\omega_{1}$,$\delta_{2}$, FB₁, $\omega_{2}$,$\delta_{2}$, FB₁, $\omega_{1}$,$\delta_{2}$, FB₂ and $\omega_{2}$,$\delta_{2}$, FB₂, ACSP shows equal performance to that with ADCSP-I in terms of DBI. The Friedman statistical test as shown in table 6 shows the significantly improved separability of MI EEG features with ADCSP-I as compared to that with ACSP except for the aforesaid cases.

However, in the case of near real-time analysis, as shown in tables 7–9, ADCSP-II outperformed ACSP in all the cases except in the case of S3 with $\omega_{2}$,$\delta_{2}$, FB₁. Friedman statistical test as shown in table 10 shows significant improvement of the separability of MI EEG features extracted with ADCSP-II as compared to that with ACSP except in the case of $\omega_{2}$, $\delta_{2}$, FB₁. The performance of ACSP is even worse than that of C-CSP in the near real-time analysis.

The poor performance of ACSP as compared to that of ADCSP-I and ADCSP-II may be because of two reasons: First, ADCSP uses the regularisation parameter $\gamma$ to update the covariance matrices of each class. The use of $\gamma$ counters the bias due to the small training set. Second, ADCSP uses $\beta$ to counter the dissimilarity between the training and testing data during high fatigue level which is not used by ACSP.

It can be concluded that the separability of MI EEG features can be significantly improved through ADCSP. However, there are some limitations of the proposed method. First, the method needs to estimate an optimal value for $\beta$ and $\gamma$ to update the covariance matrices causing some computational drawbacks. The study does not include any automatic way to estimate optimal $\beta$ and $\gamma$ for a particular problem. Future work would include an automatic method of estimating $\beta$ and $\gamma$ using gradient descent algorithm.

Second, the effect of mental fatigue on MI EEG feature separability is subject-specific and is highly individualized. The term 'one size fits all' may not be applicable in implementing an adaptive approach by tracking such user's cognitive state. Therefore, parameters such as the size of the training set, range of $\beta$ and $\gamma$, the range of 'p ' and 'n_t' need to be set individually, causing some computational drawbacks.

Third, the 'init' in case of ADCSP-II is based on high fatigue level instead of the fatigue score. It may happen that a slight increase in fatigue score will activate adaptation, but in reality, the subject may not be experiencing much fatigue and adaptation is actually not required. Hence accurate and robust estimation of the point where fatigue enters high fatigue level and the adaptation process is activated is substantial.

Fourth, although, the proposed methodology works well on the five samples, applying this model to a larger number of subjects of all ages, gender etc would reinforce the study.

Fifth, this study used class separability metrics like DBI, DI, and FS for evaluating the separability of MI EEG features. In practice, the adaptive methods need various evaluation metrics specially because of the issue of unreliable labels. Information based methods, classification accuracy, true-false difference can also be used for such evaluation. However, these were not explored as the thrust of the study was to enhance class separability. Future work would include validation of all the aforesaid approaches using different evaluation metrics.