Fuzzy systems to process ECG and EEG signals for quantification of the mental workload

https://doi.org/10.1016/S0020-0255(02)00155-XGet rights and content

Abstract

In this paper, we analyze the ECG and EEG signals by using a set of fuzzy systems developed for signal processing and try to see if one can quantify the mental workload based on the trend of EEG signal changes and the variation of pulse width from ECG signal. The signal data were taken from nuclear power plant operators while they perform the turbine operations. First, we apply a fuzzy system to extract the heart beat intervals from the ECG and a smoothing algorithm for the variation of the resulting heart beat intervals to obtain a trend variable. Next, we apply the same smoothing algorithm to the α-band frequencies and to the θ-band frequencies to get their trend curves. Finally, the three trend variables; the variation of the heart beat intervals, the α-band power, and the θ-band power are combined by a pair of fuzzy systems to estimate the mental workload during nuclear power plant operations. The results of applying our algorithm to three different data sets are included, along with a comparison between these results and the results obtained by applying a linear combination of the three variables. Compared results show that the `mental workload' computed by fuzzy systems with nonlinear rules reflects the changes more clearly than the ones computed by linear functions.

Introduction

It is generally known that electrocardiogram (ECG) and electroencephalogram (EEG) signals reflect overall arousal or alertness of the brain even though mental workload cannot be quantified accurately using these physiological signal measurements. Rugg and Dickens [1] assigned two psycho-metric tasks to 12 subjects, studied the EEG activity by a statistical method, and found that the α power decreases and θ power increases as the mental workload increases. Lang et al. [2] observed the same pattern while the subjects were asked to transform letters into Morse code. Mecklinger et al. [3] also showed that the θ power increases during memory search work. With regard to heart beat rate, i.e. pulse width, Backs and Ryan [4] showed that its variability decreases with the imposition of a perceptual motor-task.

Instead of the statistical method used in the above papers, we employed a smoothing algorithm for the power of α-band (8–13 Hz) and of θ-band (5–7 Hz) frequencies of EEG curve and the variation of pulse width obtained from ECG curve, to generate their corresponding trend curves. We then fuzzified the three smoothed variables or trend variables and combine them by a set of fuzzy rules to obtain an estimate of the mental workload. The ECG and EEG signals were taken from experienced nuclear power plant operators while they perform turbine operations on a full scope simulator. In each experiment, the operator task was to diagnose and mitigate an abnormal or an emergency situation. Simulated situations were a feed water pump trip, main steam isolation valve fails to open, and a steam generator tube rupture. The data points were taken at 0.01 s intervals for 10–30 min periods and hence there are more than 60,000 points for each experiment. Note that there will be more than 3000 oscillations in case of the 5 Hz frequency. To handle this bulky amount of data, we used a Gaussian smoothing algorithm obtained by a repetitious application of a fuzzy smoothing algorithm.

Section snippets

Extraction of pulse widths from ECG signals

In this section, we describe a fuzzy system developed to locate the end points of pulse width intervals from an ECG curve. The end points of pulse width intervals can be determined by locating the successive parts of the curve consisting of four points at tn, tn+1, tn+2, tn+3 shown in Fig. 1. These parts can be characterized by the inequalitiesx+ϵ<0,z+ϵ<0,y−2ϵ>0,x+y−ϵ>0,where x=tn+1tn, y=tn+2tn+1, z=tn+3tn+2 with a proper constant ϵ>0.

Thus, if there is a positive constant ϵ so that (1) is

A Gaussian smoothing method to generate trend curves

In this section, we describe the smoothing algorithm applied to generate trend curves for the α-power and the θ-power of EEG curves, and the pulse width variation curve from ECG curves. We prove that our algorithm is a Gaussian smoothing algorithm and is better for generating trend curves than the Gaussian smoothing algorithm using the binomial coefficients or the uniform coefficients as its kernel. Let f(x) be a continuous function. Then the Gaussian smoothing [7] of f(x) is defined as the

Application of the smoothing algorithm to ECG and EEG signals

Three sets of ECG and EEG signal data were taken from turbine operators while they perform nuclear power plant operations on a simulator under abnormal situations. The experiments were performed on a nuclear power plant simulator with a set of physiological measurement equipment at Korea Atomic Energy Research Institute [9]. The first example data were taken when the turbine operator was to handle a feed water pump trip. For the second example, the operator was to handle the case of opening a

Mental workload as a function of α-power, θ-power, and the variation of pulse width

Let ρ be the absolute value of the derivative of pulse width, α be the smoothed value of the α power, and θ be the smoothed value of the θ power. In the following, we try to define a function representing the mental workload as a function of the three variables ρ, α, and θ.

First, we fuzzify α by using four triangular fuzzy sets whose centers of support are at α0hα, α0, α0 +hα, α0+2hα where α0 is the average of the smoothed α values for the first few minutes of the experiments when the mental

Conclusion

Based on a fuzzy smoothing algorithm, we developed a method for deriving trend curves for the α-power, θ-power, and the variation of pulse width. Using the values taken from the trend curves, we defined the mental workload as a function of the three variables. From the examples, we found that all three variables; α-power, θ-power, and the variation of pulse width increase during the period when the mental workload is expected to increase. We also found that the results of using nonlinear fuzzy

References (9)

  • M.D. Rugg et al.

    Dissociation of alpha and theta activity as a function of verbal and visuospatial tasks

    Electroencephalogr. Clin. Neurobiol.

    (1982)
  • W. Lang

    Event-related EEG-spectra in a concept formation task

    Hum. Neurobiol.

    (1988)
  • A. Mecklinger

    Event related potentials and EEG components in a semantic memory search task

    Psychophysiology

    (1992)
  • R.W. Backs et al.

    Psychophysiological measures of workload during continuous manual performance

    Hum. Factors

    (1994)
There are more references available in the full text version of this article.

Cited by (30)

  • Driver's mental workload classification using physiological, traffic flow and environmental factors

    2023, Transportation Research Part F: Traffic Psychology and Behaviour
  • Detecting and measuring construction workers' vigilance through hybrid kinematic-EEG signals

    2019, Automation in Construction
    Citation Excerpt :

    José et al. investigated the subjects' risk identification and risk perception through electrocardiogram (ECG) signals that sampled in multiple frequencies [28]. To assess workers' mental and physical workload with these sensing technologies, researchers employed various quantitative methods, such as fuzzy systems [29], artificial neural network [30], and multi-measure frameworks [31,32]. Recently, researchers have begun to introduce wearable EEG devices for the monitoring and analysis on construction workers' risk recognition and concentration assessment [14,33].

  • A novel cepstral-based technique for automatic cognitive load estimation

    2018, Biomedical Signal Processing and Control
    Citation Excerpt :

    To date, various techniques for cognitive load estimation have been proposed [4,22,44–46]. Some of these studies utilized multimodal signal analysis [44–46]. However, the redundant modality could affect the computational complexity and estimation performance [44].

  • Optimizing the number of electrodes and spatial filters for Brain-Computer Interfaces by means of an evolutionary multi-objective approach

    2015, Expert Systems with Applications
    Citation Excerpt :

    For instance, a BCI that is able to classify thoughts into hand-right and left-hand imaginary movement can be used to move a screen cursor right and left. Machine Learning techniques are typically used for computing the classifier from user data (Dornhege, Krauledat, Muller, & Blankertz, 2007; Moon et al., 2002; Sun, 2010). The learning of the classifier and the adjustment of the filters is carried out off-line.

  • Applying evolution strategies to preprocessing EEG signals for brain-computer interfaces

    2012, Information Sciences
    Citation Excerpt :

    From a more applied point of view, [26] discusses recent machine learning classification and pre-processing techniques for an actual BCI system. Finally, some advanced techniques such as Fuzzy systems [29,23], Gaussian Processes [35], Temporal Models [17], Dynamical Bayesian Networks [34], or Hidden Markov Models [31] are also becoming techniques of interest for BCI’s and EEG analysis. In the context of classification of EEG signals, an important issue for the success of the classifier is to determine the relevant input information.

View all citing articles on Scopus
View full text