Steady-State Motion Visual Evoked Potential (SSMVEP) Based on Equal Luminance Colored Enhancement

Steady-state visual evoked potential (SSVEP) is one of the typical stimulation paradigms of brain-computer interface (BCI). It has become a research approach to improve the performance of human-computer interaction, because of its advantages including multiple objectives, less recording electrodes for electroencephalogram (EEG) signals, and strong anti-interference capacity. Traditional SSVEP using light flicker stimulation may cause visual fatigue with a consequent reduction of recognition accuracy. To avoid the negative impacts on the brain response caused by prolonged strong visual stimulation for SSVEP, steady-state motion visual evoked potential (SSMVEP) stimulation method was used in this study by an equal-luminance colored ring-shaped checkerboard paradigm. The movement patterns of the checkerboard included contraction and expansion, which produced less discomfort to subjects. Feature recognition algorithms based on power spectrum density (PSD) peak was used to identify the peak frequency on PSD in response to visual stimuli. Results demonstrated that the equal-luminance red-green stimulating paradigm within the low frequency spectrum (lower than 15 Hz) produced higher power of SSMVEP and recognition accuracy than black-white stimulating paradigm. PSD-based SSMVEP recognition accuracy was 88.15±6.56%. There was no statistical difference between canonical correlation analysis (CCA) (86.57±5.37%) and PSD on recognition accuracy. This study demonstrated that equal-luminance colored ring-shaped checkerboard visual stimulation evoked SSMVEP with better SNR on low frequency spectrum of power density and improved the interactive performance of BCI.


Introduction
Commonly used brain-computer interface (BCI) technologies include motor imagery (MI) [1], P300 event-related potential (ERP) [2], transient visual evoked potential (tVEP) [3], and steady-state visual evoked potential (SSVEP) [4]. Compared with other methods, the SSVEP a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 requires less electrodes for electroencephalography (EEG) recording and no need for training, and can obtain higher recognition accuracy. However, it usually provides stimulation through light flicking or graphic flipping, which is likely to cause visual fatigue and discomfort with a consequent decrease of recognition accuracy. In recent years, brain-computer interface paradigms based on motion perception have been proposed [5] to avoid the negative influence of prolonged strong stimulation on brain neurons. Motion visual evoked potentials (mVEPs) can be divided into transient ones and steady-state ones [6]. In 2009, Gao et al. adopted transient N2 potential based on the perception of motion [7], [8]. This paradigm has an obvious advantage in VEP-based BCI study in which constant luminance and non-flashing techniques were used. However, its shortcoming is that the transient paradigm requires multiple stimulation targets to move along a single direction resulting in motion after-effect (MAF) [9]. Xie et al. designed a BCI paradigm with Newton's rings based on steady-state motion visual evoked potential [10] that increased recognition accuracy to a favorable level [11]. However, the luminance in the central area of the Newton's ring can not be kept constant during the process of motion resulting in a lower signal-to-noise ratio (SNR) of spectral peaks.
SSVEP mainly uses light intensity sensing pathways of human vision. Human visual system (HVS) is elaborate and sophisticated. It is composed of retina [12], lateral geniculate nucleus (LGN), and visual cortex. Visual cortex is the high-level central neural system, which includes striate cortex V1 and extra striate cortex (e.g. V2, V3, V4 and V5 (MT)). As shown in Fig 1, primary visual cortex V1 has two pathways to output information: dorsal pathway and ventral pathway [13].
The dorsal pathway, also known as M-pathway, is associated with motion recognition and spatial processing, which detects motion speed and direction. The ventral pathway, also known as P-pathway, is a color-sense-associated and object recognition pathway, which detects luminance and color [14], [15]. When two different colors with the same luminance are applied for visual stimulation, the sensitivity of the eyes against flicker will be reduced to the lowest [16], [17], [18].
Our hypothesis is that equal-luminance colored checkerboard visual stimulation activates more neuronal response in visual center and M-and P-pathways in the brain with a higher SNR of SSMVEP signals. A colored ring-shaped checkerboard composed of color, shape, luminance, and motion was designed in this study according to neural mechanisms of HVS. The purpose of this study was to determine if the equal-luminance colored ring-shaped checkerboard stimulation paradigm evokes distinct SSMVEPs with a better SNR, and improves the interactive performance of BCI.

Ethics statement
Subjects were studied after giving informed written consent in accordance with a protocol approved by the institutional review board of Xi'an Jiaotong University.

Design of equal-luminance colored checkerboard paradigm
The ring-shaped checkerboard was divided into small grids with the same size and number, with two different colors arranged alternatively. The luminance at the central part of stimulation units was set to be the background luminance value to ensure constant luminance during contraction-expansion of the checkerboard. A white spot with radius of 1 pixel was set at the center to keep the subjects focus on it during the experiment. Based on the color space theory, three pairs of antagonism colors-black and white, red and green, and blue and yellow-are suitable for color stimulation to ensure accurate transmission of colors [19]. Black and white mainly represent eye-to-luminance reaction. Blue and yellow have relatively poor robustness, red and green can present images under equal luminance. Thus, red and green were selected as the colored stimulation paradigm in this study, and the paradigm pattern is as shown in Fig 2. The formula for generating the stimulation pattern of ring-shaped checkerboard is: where sign(x) is a sign function: r(x,y) and ang(x,y) are the radius and angle of the pattern pixel point (x,y) on the display screen; D is the width of the checkerboard, representing spatial resolution, and it was set to be 10 pixels, with the checkerboard divided into 8 rings; ϕ(t) is the phase value function; L is the motion amplitude of the checkerboard, and it was set to be 10 pixels; M is the number of checks which are cut from a single ring, and it was set to be 12 pixels; I 0 is the background luminance, and it was set to be 120 pixels; R inner and R outer are inner and outer diameters of the checkerboard, and they were set to be 3 pixels and 80 pixels. Sine was adopted to show the contraction-expansion of the checkerboard, where: f c is the motion frequency, i.e. the reciprocal of the required time for one time of contractionexpansion of the checkerboard. When the phase ϕ(t) is turned from 0 to π, the checkerboard contracts, and when the phase ϕ(t) is turned from π to 0, the checkerboard expands (The specific forms of movement can refer to the S1 Video). Motion direction changes twice in one cycle. The frequency of motion direction changes is defined to be motion inversion frequency f, and it is 2 times of motion frequency f c . Since SSMVEP mainly comes from brain activities which triggered by direction changes, and the energy mainly focused on motion inversion frequency. Therefore, the inversion frequency is adopted as the fundamental frequency of visual stimulation in this study.
In this study, visual stimulation was presented to subjects through the computer screen, and screen refresh rate was f r . To generate frame images, the t in Formula (3) must be discredited according to the screen refresh rate, i.e. t(n) = n / f r , where n = 1,2,3. . .. . . is the frame number. Formula (3) can be adapted to: Phase value function ϕ(n) is a discrete time sequence. In order to ensure ϕ(n) is a periodic sequence, the value of f r / f c must be an integer. To command F C = f r / f c to be frames required for one cycle of expansion-contraction motion. Formula (4) is rewritten as: Thus, the calculation formula of motion inversion frequency is: When appropriate F C is chosen in actual application, a correct motion inversion frequency f can be calculated based on Formula (6), and the phase value function ϕ(n) can be calculated based on Formula (5).

Experiment data processing
Signals collected from each subject were analyzed off-line. If stimulation lasts t seconds per trial, n trials in total, the sampling frequency of electroencephalogram (EEG) equipment was F s . EEG segment for each stimulation was extracted based on the starting and ending points in each trial, so a n×t Ã F s dimensional matrix was obtained. Then stimulation signals in all trials were superposed and averaged to obtain a new data segment X = {x(1), x(2),. . .x(i)}, i = 1,2,3,. . .,t Ã F s . Then the new data segment was processed by 4-40 Hz band-pass filtering to remove low-frequency drift and high-frequency interference. Then the data segment was analyzed using Welch power spectrum density [20] mentioned below.

Welch Power spectrum density
In the BCI field, SNR is a key index to evaluate the efficiency of stimulation paradigms. A good stimulation paradigm should evoke more powerful SSMVEP, which indicated in the power spectrum is that the peak value is higher at corresponding stimulation frequency.
In this study, the Welch power spectral density was used to estimate random signals by dividing a data with a length of N into M segments, and the length of each segment is l. Its window averaged period formula is: where p 0 refers to the power of window w(n): Canonical correlation analysis Canonical correlation analysis (CCA) is widely applied in SSMVEP target recognition. In this study, EEG of all sampling channels were selected as a set of variable to calculate canonical correlation coefficient with the generated reference signals. The maximum target of correlation coefficient is considered to be the focused target [10]. The SSMVEPs collected from positions of designated electrodes were marked as X = (x 1, x 2,. . ., x n ), where n is the number of electrode channels. The reference signals were constructed at the stimulation frequency f i : Where k is the number of harmonics, which is dependent on how many frequency harmonics existed in SSMVEP. The f s is sampling rate, and m is sample points. By calculating: Correlation coefficient ρ i can be obtained, and i corresponding to the maximum is the focused target.

Feature recognition algorithm based on power spectrum density peak
If the equal-luminance red-green checkerboard can increase higher SNR, a significant peak will appear on the power spectrum density at the stimulation frequency. In consideration of these circumstances, a characteristics recognition method based on power spectrum density peak was used in this study. Several stimulation paradigms with different stimulation frequency were displayed on the screen, and we assumed that the stimulation frequency of all paradigms have a minimum of f min and a maximum of f max . The stimulation duration per trial was set as t seconds and the EEG sampling frequency as F s . Subjects stared on one of the stimulation paradigms, and the data sequence of every sampling channel were {X f (1), X f (2),. . ., X f (n)}. First, singular value decomposition (SVD) was performed for each channel data to eliminate noise. Then Welch power spectrum density was calculated for every channel data to find out the maximum peak {p 1 (w), p 2 (w),. . ., P n (w)} within the [f min , f max ]. Based on p 1 (w), the weight coefficient of each channel was calculated, i.e. a 1 = 1, a 2 = p 2 (w)/p 1 (w),. . ., a n = p n (w)/p 1 (w). Thus the weight coefficient of each channel was obtained {a 1 , a 2 ,. . ., a n }.
The data of the n channels was multiplied by weight coefficient, then superposed and averaged to get a new data vector: The data size is 1×t Ã F s . This method that makes the multi-electrode signals merge into a single channel signal called spatial filtering. In the EEG signal processing, integration of multi-electrode EEG information is effective [21], [22]. After such processing, noises were suppressed, which made stimulation frequency peak outstanding. The Welch power spectrum density was then calculated. The idx of the data point was identified corresponding to the maximum peak within the [f min -Δf, f max +Δf], and located on the corresponding x-coordinate, which was identical to the stimulation frequency f. Where Δf is frequency difference, that was set as 0.2 Hz. The difference between the obtained stimulation frequency f and that of all paradigms was calculated to figure out the absolute value of the difference: i refers to the i stimulation paradigm. When Δf i is the minimum, the corresponding i is the focused target.

BCI experiment platform
In this study, a g.USBamp (g.tec, Austrian) was used to collect and process EEG. This equipment can collect signals of 16 channels at the same time, with sampling frequency of 1,200 Hz. EEG amplifier g.USBamp and active electrode system g.GAMMAbox were combined to form a BCI experiment platform, as shown in Fig 3. Before the experiment, the reference electrode A1 was placed at the left ear of the subjects, and the ground electrode Fpz was placed at forehead. EEG was also collected from the following 15 channels: O1, Oz, O2, PO7, PO3, POz, PO4, Cz, P1, Pz, P2, CP3, CPz, CP4 and Fz, with electrodes places as shown in

Experiment of brain response enhancement under equal-luminance colored stimulation
A black-white checkerboard was used for luminance calibration, which doesn't contain color information. Total luminance value of black-white and equal-luminance red-green checkerboard were the same, which was set as 76 cd/m 2 . We used 5 males and 4 females (20-25 years old) as subjects, who have normal color and visual senses. The experimental process is shown in Fig 5(A) and 5(B). Each subject was requested to finish four tasks. Task 1, black-white checkerboard stimulation: Three stimulation paradigms were displayed on the screen. The frequencies were 11 Hz, 16 Hz and 18 Hz respectively. Subjects were requested to stare on one of the paradigms for 4 s per trial, 15 trials in total, with an interval of 2 s, and then stared on another paradigm, until all were done. Task 2, equal-luminance red-green checkerboard stimulation: Experimental contents were same as Task 1. Task 3, black-white checkerboard stimulation: One stimulation paradigm was displayed on the screen, and the frequency was 8 Hz.
Stimulation lasts 2 s per trial, 15 trials in total, with an interval of 2 s. Task 4, equal-luminance red-green checkerboard stimulation: Experimental contents were same as Task 3. The experiments were designed to compare differences of brain response evoked by equal-luminance red-green and black-white checkerboard to determine the feasibility of improving BCI interactive performance using equal-luminance colored stimulation.

Comparisons of experimental stimulation for recognition accuracy between black-white and equal-luminance red-green checkerboard
Nine people in experiment (a) were selected as subjects. The experimental process is shown in Fig 6. Task 1, black-white checkerboard stimulation: Three stimulation paradigms were displayed on the screen, and the stimulation frequencies were 8 Hz, 9 Hz and 10 Hz respectively. Subjects stared on any of them every trial. The stimulation duration per trial was divided into eight levels, from 1 s to 8 s, and every subject stared on 20 trials in each level of experiments, with an interval of 2 s. Task 2, equal-luminance red-green checkerboard stimulation: Experimental contents were same as Task 1. CCA was used to identify fixation target of each trial.

Characteristic recognition experiment based on power spectrum density peak
Equal-luminance red-green checkerboard was used as stimulus paradigm. Nine people in experiment (a) were selected as subjects. Three stimulation paradigms were displayed on the screen, and the stimulation frequencies were 10.5 Hz, 11.5 Hz, and 12.5 Hz respectively. The experimental process is shown in Fig 7. The subjects stared on any of the paradigms for 4 s per trial, 15 trials in total, with an interval of 2 s. Recognition accuracy result of each trial by both CCA and PSD was analyzed. In order to get the statistical analysis of the accuracy of each subject, the above experiments were carried out 5 runs.

Results
Experimental Results of brain response enhancement under equalluminance colored stimulation Brain response enhancement under colored stimulation. The peak value of the power spectrum density of each sampling channel at a designated stimulation frequency was calculated, superposed, averaged for mean values and standard deviations (SD).  shows that at 11 Hz, the equal-luminance colored checkerboard can evoke more powerful SSMVEP and higher SNR. All subjects had favorable stimulation effects by the equal-luminance red-green checkerboard. Fig 8(B) and 8(C) show that at the middle frequency spectrum (15 Hz-25 Hz), i.e. 16 Hz and 18 Hz, the equal-luminance red-green checkerboard had no significant enhancement effect when compared with the black-white checkerboard. As shown in Fig 8(D), the stimulation frequency was 8 Hz. When the stimulation duration decreased to 2 s per trial, the red-green checkerboard still had enhancement effects.  that brain response at the low frequency spectrum (lower than 15 Hz) was the most intensive, which was consistent with the results of literature [6].
Designated frequency of 11 Hz in low frequency spectrum and 16 Hz in middle frequency spectrum were selected to conduct the significance test. One-way analysis of variance

Experimental Results of recognition accuracy between black-white and equal-luminance red-green checkerboard
Recognition accuracy of the black-white and equal-luminance red-green checkerboard. CCA was used to calculate the recognition accuracy of black-white and equal-luminance red-green checkerboard for each subject. The accuracy of each subject with different stimulation duration was calculated to average the recognition accuracy of 9 subjects according to stimulation duration. Fig 10 is the mean variance histogram of the recognition accuracy, the equal-luminance red-green checkerboard always had higher recognition accuracy than  Steady-State Motion Visual Evoked Potential that of the black-white checkerboard with different stimulation duration. When the stimulus duration was within 4 s, red-green checkerboard recognition accuracy was higher than that of the black-white checkerboard. When the stimulus duration exceeded 4 s, the difference of recognition accuracy was not greater. When the stimulation duration exceeded 6 s, subjects reported visual fatigue and the recognition accuracy decreased.
We used Chi-Square test to conduct significance test and the significant level was set as 0.05. As shown in Table 1, the recognition accuracy of equal-luminance red-green checkerboard significantly improved except when the stimulus duration were 5 s and 6 s. But the Hz. The equal-luminance colored stimulus has a significant effect on the brain response than black-white. (b) Average power spectrum density variance map of subjects at 16 Hz. There is no significant difference on the brain response between black-white and red-green stimulus at middle frequency spectrum (16 Hz). Steady-State Motion Visual Evoked Potential accuracy of equal-luminance red-green checkerboard was higher than black-white checkerboard when the stimulus duration were 5 s and 6 s. When the stimulus duration was 7 s or 8 s, the recognition accuracy of equal-luminance red-green checkerboard was still higher than black-white paradigm.
Experimental Results of characteristic recognition based on power spectrum density peak Recognition of power spectrum density characteristics. Fig 11 shows the power spectrum density of subjects processed by the proposed method for the first stimulation paradigm in 10.5 Hz. Most subjects had significant peaks at 10.5 Hz except subject 9, indicating that SSMVEP firing frequency was concordant to visual stimulation frequency.
As shown in Fig 12, the average off-line recognition accuracy of subjects was about 88.15± 6.56%, indicating that PSD-method based on the peak value of power spectral density was effective. The mean and standard deviation (SD) of the accuracy values between CCA and PSD for each subject are listed in Table 2. We can see that average recognition accuracy are very close between CCA and PSD. However, CCA accuracy results showed that individual viability had an impact on CCA. For example, the recognition accuracy of subject S2 and S8 were only 64.44% and 70.67%, while the recognition accuracy of subject S1 and S3 were 98.67% and 94.67%. Individual viability had less impact on PSD. Therefore, the PSD has better adaptability than CCA.

Discussion
The SSVEP-based BCI technology requires less recording electrodes for electroencephalogram (EEG) signals and no need for training. However, it usually provides stimulation through light flicking or graphic flipping, which is likely to cause visual fatigue and decrease of recognition accuracy. In order to avoid the negative impacts on the brain response caused by prolonged strong stimulation of SSVEP, this study proposed a SSMVEP stimulation method using equalluminance colored ring-shaped checkerboard paradigm based on HVS.

Effects of colored stimulation paradigm
To investigate the efficiency of color-enhanced visual stimulation, red-green and black-white stimulation paradigms were tested in our research. By comparing the power spectrum density of subjects between equal-luminance red-green and black-white checkerboard at 8 Hz, 11 Hz, 16 Hz, and 18 Hz, we found that equal-luminance colored stimulation evoked better SSMVEP signals with a higher SNR at the low frequency spectrum (lower than 15 Hz). The colored

Effects of stimulation frequency
Because the brain response at the low frequency spectrum is the most intensive [6], and the SNR is very low at middle frequency spectrum, therefore, the effect of colored stimulation is difficult to reflect at the middle frequency spectrum as shown in this study. Hence many stimulation frequencies have been set at the low frequency spectrum for better SSMVEP signals. In our study, equal-luminance colored stimulation was selected to determine if the method can effectively enhance brain response and increase SNR to improve BCI interactive performance. We selected 11 Hz in low frequency spectrum and 16 Hz in middle frequency spectrum to compare effects of stimulation frequency on the efficiency of stimulation paradigms. The results demonstrated that equal-luminance colored stimulation at low frequency spectrum (11 Hz) enhanced the brain response better than 16 Hz stimulation paradigm, there was no significant difference between black-white and equal-luminance red-green stimulus at middle frequency spectrum (16 Hz).

Effects of stimulation duration
When the stimulus duration was less than 4 s, recognition accuracy of red-green checkerboard was higher than that of the black-white checkerboard. When the stimulus duration was 5 s or 6  s, there was not a statistical difference on recognition accuracy between these two stimulation paradigms. This is due to the longer stimulation duration reduced the SNR of SSMVEP response, leading to no statistical differences between two stimulation paradigms. Chi-Square test results showed that when the stimulus duration was 7 s or 8 s, the recognition accuracy of equal-luminance red-green checkerboard was higher than that of black-white checkerboard. Black-white checkerboard easily caused visual fatigue with a decrease of SNR, while equalluminance red-green checkerboard did not cause rapid visual fatigue. Achievement of greater speed and recognition accuracy depends on improvements in signal processing, translation algorithms, and user training [23]. In our study, equal-luminance red-green checkerboard shortened the stimulation period yet produced a higher recognition accuracy suggesting that equal-luminance red-green checkerboard BCI interactive performance better than black-white checkerboard.
When the stimulation duration was longer than 6 s, subjects felt visual fatigue with a decrease of the recognition accuracy. These data suggested that the stimulation duration should be neither too short, which would fail to evoke SSMVEP, nor too long, which would cause visual fatigue.

Signal processing and translation algorithms
In this study, a feature recognition method based on the power spectral density peak was used for signal processing. Through a spatial filtering, multi-electrode lead signals were merged into a single channel signal. Results demonstrated that the equal-luminance red-green checkerboard paradigm did not elicit obvious harmonic or sub-harmonic components. This indicates that equal-luminance red-green checkerboard paradigm may not cause cross-frequency interactions with ongoing oscillatory activity. In addition, the experiment results showed that the average recognition accuracy is about 88.15±6.56%, suggesting that the equal-luminance redgreen checkerboard stimulation is a feasible and effective paradigm for SSMVEP. By comparing the recognition accuracy between CCA and PSD, we found that individual differences have less impact on PSD, suggesting that PSD has better adaptability.

Conclusion
This study demonstrated a novel BCI paradigm based on steady-state motion visual evoked potential (SSMVEP). The new paradigm integrated equal-luminance ring-shaped checkerboard into visual stimulation paradigms. Results demonstrated that equal-luminance redgreen checkerboard paradigm had the lower fatigue characteristics, less visual discomfort, and significantly stronger responses than black-white checkerboard paradigm. The signal processing and translation algorithms for feature recognition are feasible and effective. The equalluminance red-green checkerboard paradigm evoked SSMVEP with a better SNR than blackwhite checkerboard at low frequency spectrum (below 15 Hz). The recognition accuracy was 88.15±6.56%.