An Experimental Framework for Assessing Emotions of Stroke Patients using Electroencephalogram (EEG)

This research aims to assess the emotional experiences of stroke patients using Electroencephalogram (EEG) signals. Since emotion and health are interrelated, thus it is important to analyse the emotional states of stroke patients for neurofeedback treatment. Moreover, the conventional methods for emotional assessment in stroke patients are based on observational approaches where the results can be fraud easily. The observational-based approaches are conducted by filling up the international standard questionnaires or face to face interview for symptom recognition from psychological reactions of patients and do not involve experimental study. This paper introduces an experimental framework for assessing emotions of the stroke patient. The experimental protocol is designed to induce six emotional states of the stroke patient in the form of video-audio clips. In the experiments, EEG data are collected from 3 groups of subjects, namely the stroke patients with left brain damage (LBD), the stroke patients with right brain damage (RBD), and the normal control (NC). The EEG signals exhibit nonlinear properties, hence the non-linear methods such as the Higher Order Spectra (HOS) could give more information on EEG in the signal’s analysis. Furthermore, the EEG classification works with a large amount of complex data, a simple mathematical concept is almost impossible to classify the EEG signal. From the investigation, the proposed experimental framework able to induce the emotions of stroke patient and could be acquired through EEG.


Introduction
The previous studies on the investigation of the stroke patients' emotional states were based on the observational-based approaches, which were conducted by filling up the international standard questionnaires and face to face interview. The purpose of the investigation is as an early diagnose of post-stroke depression (PSD). Thus, the EEG signal processing methods are required to develop an observer-independent emotional assessment system for stroke patients. The emotion processing in the brain is formed by a complex network of interconnected neurons. EEG is a non-linear, non-stationary, and non-Gaussian signal. EEG signals are generated by nonlinear deterministic processes, also refers as deterministic chaos theory with nonlinear coupling interactions between neuronal populations. The nonlinear dynamic properties should not be ignored in analysing the EEG signals. In contrast with linear  2 analysis, non-linear analysis methods will give more meaningful information about the emotional state changes of stroke patients. The emotion recognition using EEG signals is the variation in the amount of nonlinearity in EEG among the different emotion. Hence, this research aims to analyse the non-linear behaviour of EEG signals on different emotions of stroke patients by applying non-linear feature extraction methods. According to literature, 'right hemisphere hypothesis' proposes that the right brain is playing the major role in emotional processing regardless of valence. This statement suggests that there will be a difference in emotion perception between LBD and RBD patients.
The previous studies on the investigation of the stroke patients' emotional states were based on the observational-based approaches, which were conducted by filling up the international standard questionnaires and face to face interview. The purpose of the investigation is as an early diagnose of post-stroke depression (PSD). Thus, the EEG signal processing methods are required to develop an observer-independent emotional assessment system for stroke patients. The emotion processing in the brain is formed by a complex network of interconnected neurons. EEG is a non-linear, non-stationary, and non-Gaussian signal. EEG signals are generated by nonlinear deterministic processes, also refers as deterministic chaos theory with nonlinear coupling interactions between neuronal populations. The nonlinear dynamic properties should not be ignored in analysing the EEG signals. In contrast with linear analysis, non-linear analysis methods will give more meaningful information about the emotional state changes of stroke patients. The emotion recognition using EEG signals is the variation in the amount of nonlinearity in EEG among the different emotion. Hence, this research aims to analyse the non-linear behaviour of EEG signals on different emotions of stroke patients by applying non-linear feature extraction methods.
The outline of this research paper consists of 4 sections: Section 1 introduces the research works. Section 2 describes the previous works related to the proposed research. Section 3 discusses the proposed research methodologies. Section 4 discusses the finding of the proposed experimental framework. The conclusions of this research are discussed in section 5.

Related researches
According to the World Health Organization (WHO), stroke was ranked the second in the top ten causes of deaths worldwide, the crude death rate was 85, which means out of 85 deaths caused by stroke per 100, 000 population in the year 2015 [1]. In Malaysia, the stroke, also one of the cerebrovascular diseases, was ranked the third in the report "Statistics on Causes of Death, Malaysia, 2017" which was published by the Department of Statistics Malaysia [2]. The percentage of the medically certified deaths in 201 was 52.8% out of total deaths, whereas the cerebrovascular diseases were 6.9% of it. The damage of the brain cells caused the loss of the brain functions, consequently, patients with brain damage will lose some of their body functions including emotion impairment. For instance, the stroke patients often suffered from emotional and behavioural changes after stroke, there is about one-third of the stroke sufferers affected [3], [4]. These changes are also known as Post-Stroke Depression (PSD) which will increase the difficulty of rehabilitation processes. Therefore, stroke patients need to undergo emotion assessment during their rehabilitation. It is necessary to seek information on mental health problems and to identify the most appropriate method for treatment. In addition, the emotional state assessment can be used as a clinical tool for early detection of PSD symptoms [3].
Emotion is the psychophysiological experience in human that influenced by stimuli. It is an instant response of a human to the stimuli they perceive. The human brain control body activities, the brain is divided into four lobes, known as frontal (F), parietal (P), temporal (T) and occipital (O), these lobes located at the cortical region of the brain. The frontal lobe is responsible for cognitive functions such as emotions, thoughts and voluntary actions, the parietal lobe responsible for sensations such as taste and touch, temporal lobe responsible for memories, and interpreting the sensory perception, whereas occipital lobe responsible for visual information. However, there is no particular brain part that can be known as the emotion canter, emotion processing involves many neural structures of the brain, also known as the emotion network of the brain [5], [6]. The different brain regions have been involved for emotional control in the emotion network [7], [8]. However, most of the studies in the literature reported the emotional impairment in stroke patients using statistical analysis and there is no autonomous emotional state assessment system is developed, where machines have the ability to monitor the emotional states of the human user and to provide an output accordingly. In addition, the earlier works mainly focused on observational based approaches (filling up the international standard questionnaires [3] or symptom recognition from psychological reactions [9] and no experimental study is reported. Also, researchers have worked mostly on facial and lexical based approaches in stroke patients for emotional state assessment [10]. EEG signal is the most powerful bio signal and used in several clinical and real-time systems design applications. Mostly, this signal is used for diagnosis of abnormalities or disorders in the brain region such as traumatic brain injury [11], stroke [12]- [14], Parkinson's disease [15], [16], and Alzheimer's disease [17]. A number of non-linear features have been implemented in the past, either involved in the EEG signal or non-EEG signal studies. In the literature, Higher-Order Spectra (HOS) feature was the most commonly used non-linear feature and reported to be an effective method for analysing EEG signals [18], [19]. The higher order statistical analysis only involved the statistical features with order three and above. The importance to use higher order statistics instead of second or lower order measures is their ability to reveal the amplitude information as well as the phase information, whereas the second order measures only observe the linear and Gaussian information of signals. However, most of the biomedical signals, in particular, the EEG signals are non-Gaussian, consequently, the higher order measures gain its advantage with the ability to extract information deviate from Gaussian and preserve the phase information of the signals [20], [21]. The third order HOS, bi spectrum, is the easiest HOS to be worked out [22]. Bi spectrum is proven in its ability to detect the presence of Quadratic Phase Coupling (QPC), a phenomenon of non-linearity interaction in EEG signals [23] - [25].
EEG classification involved a bunch of feature vectors with complex relationships, a simple calculation may not work well with the EEG data and needs many repetitions for decision making. Therefore, machine learning methods have been applied for EEG classification [14], [15], [18], [26]. Machine learning is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention, it is a method that automates analytical model building. Machine learning has been used as the decision-making algorithm for the unknown EEG signal based on a set of training data, the past studies have been implemented machine learning on emotional classification using EEG signals in stroke patients, Parkinson's disease patients and also in other EEG analysis [14], [15], [18], [26]. Based on the literature reviewed, the emotion assessment system for stroke patients and a normal person can be developed by machine learning algorithm and the brain-behaviour of emotion processing can be analysed from the EEG signals.

Research methodologies
The flow chart of the overall research is shown in Fig. 1. The research activity is divided into signal acquisition, signal processing and signal analysis. The manuscript discusses a framework in setting up the signal acquisition processes. A signal processing and signal analysis are discussed in general.
A signal acquisition involves the data collection protocol design, a pilot study and EEG data collection. In the protocol design, the data collection protocol will be designed to induce six emotional states of the subject in the form of video clips. The protocol will be designed using the pictures and video clips from the International Affective Picture System (IAPS) and International Affective Digitized Sound System (IADS) [14]. The six stimulated emotional states will be the anger (A), disgust (D), fear (FE), happy (H), sad (S), and surprise (SU). In the pilot study, a questionnaire will be designed to examine the emotions felt by a group of participants of 10 people to validate the feasibility of the data acquisition protocol. In a data collection, the EEG data will be collected from three groups of subjects, namely the stroke patients with left brain damage (LBD), stroke patients with right brain damage (RBD), and normal control (NC). Each group is estimated to have 15 subjects. The EEG data will be collected from the subjects using a 14-channel Epoc Emotiv EEG headset with a sampling frequency of 128 Hz, and the electrodes will be placed according to the international 10-20 systems. The Signal processing involves the filtering for artefacts removal in raw EEG signal and the feature extraction. The proposed filtering technique is the Butterworth 6 th order bandpass filter to filter the signals from delta to gamma frequency bands of EEG signals. The MATLAB software will be used to process the signal. As the feature, the third order HOS, known as bi spectrum, will be used to extract the non-linear characteristics of the emotional EEG.
A signal analysis involves identifying pattern of the data for emotion classification and brain mapping. The machine learning-based classifier will be used to classify the emotional states of each group. The MATLAB software will be used to classify the emotional states. The results of the classifiers will be used for brain mapping to study the brain-behaviour for each emotional state.

Designing an experimental protocol
Database has been developed based on the audio-visual stimuli since it induces a strong emotion on the  [14]. Different types of emotional video clips have been collected from different resources such as the International Affective Picture System (IAPS) database, International Affective Digitized Sounds (IADS) database (and video clips collected from various resources on the internet [14]. The elicitation of negative emotions was mainly achieved by using affective pictures from IAPS and IADS databases. The content of the video clips for each emotion is described in Table 1. Before the experiment is began, all the subjects are requested to fill up Mini-Mental State Exam (MMSE), Beck's Depression Inventory (BDI) and Edinburg Handedness Inventory (EHI). Subjects are required to fill in their education background, employment, age and duration of stroke. Then, subjects are given a set of basic instructions explaining about the experiment protocol design, purpose of this experiment, and audio-visual stimuli flow. Experiment is started with recording relax mode (eyes open) as baseline for 3 min. After that, the emotion stimulation is started off with one clip which also acts as an exercise or sample emotion. Right after the sample, a set of 6 clips/trials of the same emotion is played continuously through a computerized system. After each set, subject has to state the emotional condition through a set of questionnaires. The same process repeats for all six emotions. The length of each clips has it ranges from a minimum of 46 s to a maximum of 1 min on different emotions. The total duration for this experiment is 90 min to 120 min depending on the patient's response. Figure 2 shows the configuration of experimental protocol of the proposed works. IAPS and IADS database which shows Special human abilities. Anger (A) Child abuse, police brutal, and bully videos from the internet sources.

Relax
Images and soothing music.

The criteria of the stroke patient
After conducting discussion with the therapist from Medyna Rehablilitation Center, the criteria of the stroke patient are set as shown in Table 2. In a data collection process, the EEG data are collected from three groups of subjects, namely the stroke patients with left brain damage (LBD), stroke patients with right brain damage (RBD), and normal control (NC). Each group contributes 15 subjects.