Deep learning for automated epileptiform discharge detection from scalp EEG: A systematic review

Electroencephalography (EEG) is used for different clinical indications. These include assessing the presence of interictal epileptiform discharges (IEDs), the characteristic biomarkers of epilepsy. IEDs manifest in the form of spikes or sharp waves, often associated with slow frequency waveforms disrupting the normal background [1]. The durations are 20 ms up to 10 s [2]. IED detection is a time-consuming and challenging task as it requires the visual review of EEG signals. Automated IED detection dates back to the 1970s when Gotman and Gloor [3] published the first work to analyze different properties or features of interictal epileptic activities such as duration, sharpness, and amplitude, and a threshold was applied to classify epileptic and background activities. Since then, more complex features [4, 5] and classification approaches have been developed, especially with the development of machine learning [6, 7]. Machine learning (ML) methods have been widely applied to automate epilepsy diagnosis and seizure detection using EEG with great performance to some extent [8–11]. As a result, such automated methods have been employed in clinical practice [5, 12, 13]. Automated IED detection (figure 1) involves classifying sequential temporal windows or segments of the EEG signal into IEDs or normal activity (without IEDs). Predefined features, characterizing distinct aspects of the EEG, derived from these windows of data are passed to a classifier such as a support vector machine [14], K-nearest neighbor classifier [15], decision tree [16], or artificial neural network [17] to discriminate IEDs from artifacts and background signals. These features can be grouped into three domains: time, frequency, and wavelet [5]. A limitation of many of these approaches is that they were only tested on small datasets (⩽50 patients) and the features applied were often hand-picked. This is likely to result in poor generalizability when it comes to seeing if the methods would work well when applied in other clinical centers on new data. Improving the generalizability of automated IED methods is an important area of research to increase the clinical acceptance of these methods. An example of software with approval from the United States Food and Drug Administration (FDA) for automated IED detection is Persyst [18], developed by Persyst Corporation. Persyst has been shown to have comparable performance to skilled senior EEG technologists [19] and also shown to be non-inferior to epilepsy-trained clinicians in a study on EEGs of electrical status epilepticus patients recorded during sleep [20]. This method has been found to be approaching the level of human IED detection performance [21]. Further work is needed to achieve or outperform human-level performance while at the same time maintaining generalizability. Along these lines, promising work suggests that human-level performance can be outperformed using deep learning (DL) [22, 23]. As such, this review specifically focuses on the work that has been done applying DL to automated IED detection.

Zoom In Zoom Out Reset image size

Training on larger datasets will give better generalizability but require a more extensive set of features [24, 25]. Feature selection tasks thereby will be cost-intensive. DL methods solve this by automatically learning latent features from raw high-dimensional data by optimizing a stack of multiple layers of mathematical functions with thousands to millions of parameters. This complexity comes with a tradeoff of high computational expense. DL has been around for many years with the breakthrough application of convolutional neural networks (CNNs) to hand-written number recognition in 1989 [26]. While a CNN's structure mimics the function of the receptive fields in the human visual system, another method, the long short-term memory network, inspired by the human brain's short-term and long-term memory mechanisms, was introduced in 1997 [27]. However, due to the lack of hardware-efficient implementations and computational power limitations at the time, DL was overlooked by researchers [25]. In the ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012), a submission successfully applied a GPU efficient implementation of CNNs (AlexNet) [28] to classify millions of images. Since then, many DL frameworks have been developed (i.e. Tensorflow in 2015 [29], Pytorch in 2016 [30], etc) and widely adopted thanks to their ease of use and efficient execution on both CPU and GPU. As computational resources are becoming cheaper and more accessible, DL methods have emerged as powerful computational methods, approaching human performance in various tasks (i.e. speech processing, image classification, and text analysis) [31–33].

A recent systematic review of the broad applications of DL to EEG analysis [34] reported a rapid increase in the number of papers with promising results across different clinical domains from 2012, especially seizure detection and prediction [35, 36]. These studies analyzed raw EEG signals directly with minimal denoising steps (figure 1). In light of this success, researchers have also experimented with DL on automated IED detection [23, 37, 38]. A recent narrative review of machine learning for IED detection showed that DL methods on raw EEG signals outperformed traditional ML approaches [4]. Given the typical duration of IEDs, short data analysis windows used in DL are either 0.5 s, 1 s, or 2 s [23, 37, 39]. The most common choice of DL architecture among existing works is CNN.

If a study did not explicitly mention the type of epilepsy, we assumed it included both focal and generalized epilepsies (9 studies [23, 42, 48, 50–52, 54–56]) (figure 4). There were two studies focusing only on idiopathic generalized epilepsy [38, 39]. Detecting IEDs from patients with focal epilepsy was the main goal of three studies [43, 44, 54], with four of them focusing on EEGs of patients with BECTS [43–46].

Zoom In Zoom Out Reset image size

All included studies utilized six different EEG recording settings: Routine; Prolonged; Long term video-EEG (LTM-EEG); Ambulatory; ICU; EEG-fMRI (figure 4). Routine EEG recording typically takes 20–30 min. Other EEG types have longer durations (up to 3 d or longer) and a higher chance of capturing IEDs than routine EEG. LTM-EEG combines continuous in-patient EEG and video monitoring over several days to identify and characterize interictal and ictal abnormalities for diagnostic, prognostic, and surgical localization. There was one study [37] with EEG recordings inside an fMRI scanner; however, we still included this as the training of the DL model only involved scalp EEG signals. While the most popular data type is routine EEG (N = 9), eight studies [23, 45, 46, 49, 51, 53, 54, 57] combined at least two different EEG recording types.

The median of total EEGs among studies is 166 (IQR: 110–518). The median number of EEGs with IEDs is 156 (IQR: 26–344). There were 12 [23, 38, 47, 48, 50, 52, 54–59] studies including normal EEG recordings (without IEDs) (median: 106; IQR: 67–496). Of all studies, there were only two studies with more than 1000 recordings in total [23, 52]. Moreover, we looked at the ratio of EEGs with and without IEDs to assess the balance of datasets. While most studies have balanced sets of EEGs, one study had a highly imbalanced ratio of EEGs with IEDs to EEGs without IEDs, 1:10 [23].

Deep learning models for IED detection are trained on window segments from whole EEG recordings. These windows are classified as with or without IEDs. Therefore, the number of annotated IEDs is crucial to the training process. We only included numbers in the studies and ignored datasets whose statistics were not mentioned. The median of annotated IEDs is 11 631 (IQR: 2663–16 402). There were 11 studies containing more than 10 000 annotated IEDs [23, 44–47, 49, 51–53, 55, 60]. The highest number of IEDs is 19 057 [52].

Training DL models on data from a single center may limit the generalizability of models when applied in alternative settings/hospitals, given differences in equipment and protocols. Only six studies [38, 47–49, 52, 58] used multi-center data (median: 3) with the highest number of centers of 6 [52].

Deep learning requires large and well-annotated datasets, often referred to as big data, for high performance [25, 61]. However, there is no indication of how large the dataset size should be. To obtain an overall picture of big data in IED detection, we summarized the largest public and private datasets with at least 10 000 annotated IEDs in table 2. We recommend that readers refer to table S4 for details of datasets in all included studies. The Temple University Hospital (TUH) EEG corpus [51], released in 2016, is the largest known public EEG corpus consisting of approximately 23 000 EEG recordings and 31 000 annotated IEDs. Temple University Events (TUEV), a subset of this corpus, contains labels for the different kinds of IEDs and artifacts (19 057 annotated IEDs) and was used in only one study [51]. Apart from this, Temple University Epilepsy (TUEP) is another subset used for the whole EEG recording classification and patient classification [52]. This dataset has 100 subjects with epilepsy and 100 subjects without epilepsy. TUEV and TUEP were divided into training, and evaluation sets such that there was no data from the same patients in both sets.

Among private datasets in studies, the largest EEG dataset reported was from the Massachusetts General Hospital (MGH) [23], consisting of 9571 EEG recordings. The private dataset with the highest number of annotated IEDs was acquired from the same hospital, with 18 164 annotations [52, 55].

EEG is usually contaminated with artifacts. These could be ocular, muscular, electrode, and powerline noises and occasionally resemble an IED on visual review. Removing artifacts before training any predictive model might help achieve better accuracy. All studies applied artifact removal methods in their preprocessing step. The most common methods are band-pass filtering and applying montages. The choices of high-pass frequencies are 0.5 Hz (N = 10) and 1 Hz (N = 5). Muscle artifacts often occur in high frequency and could be removed with low-pass frequency filtering. Low-pass filtering at 70 Hz was used in one paper [39]. Another solution was to reduce the frequency ranges to the sub-bands alpha, delta, beta, and theta. For this, choices of 30 Hz (N = 2), 32 Hz (N = 1), and 35 Hz (N = 3) were used. Power line noise is at 50 Hz in Europe and 60 Hz in the United States of America. Notch filtering can remove this [48] at these frequencies (N = 3). Low pass filterings of 50 Hz (N = 2) and 49 Hz (N = 1) were also employed to remove the power line noise. Apart from bandpass filtering, the PureEEG algorithm [63] was used in one study [58] to remove artifacts based on a stochastic, spatio-temporal model using a linear minimum mean square error estimator.

There were five choices of montage in studies: temporal central parasagittal (TCP), common average (CA), longitudinal bipolar (LB), source derivation (SD), and laplacian. There were 12 studies [38, 48, 50–59]employing at least one montage in the experiment. CA and LB were combined in one study [23]. On the other hand, LB and laplacian were studied separately and showed comparable results [54]. TCP [64] was the main montage in the TUH corpus [62] and is a combination of longitudinal and transverse montages focusing on focal regions of the scalp. Nhu et al [59] studied longitudinal and transverse montages and showed that the combination of these two had the best performance.

Processing the whole EEG recording at once is resource-intensive. A solution to this is to segment the recording into smaller windows. These can overlap or not overlap. Segmenting the data into windows also assists the time-resolved detection of events in the data. Based on the typical duration of IEDs, the choices for window size were 0.1 s (N = 1), 0.5 s (N = 4), 1 s (N = 6), and 2 s (N = 9).

Channel selection focuses the analysis on a subset of the recorded channels and reduces the computational resources required for analysis. EEG experts select the subset based on the recording settings, historical data, and experience [65]. The 10–20 system was the most popular recording setting (N = 22). The 25 electrode array was used in one study [58]. We did not observe any significant channel selection methods apart from excluding two ear electrodes in 10 studies [23, 38, 42, 47, 48, 52, 54–56, 59].

Normalization of input data might reduce the training time and improve the performance of the classification system [66]. There were two methods: z-score normalization (N = 6) [38, 39, 45, 46, 53, 59] and maxima normalization (N = 1) [49]. z-score normalization reduces the mean of the inputs to 0 by subtracting the mean from the inputs and dividing it by the standard deviation. Maxima normalization rescales the input to between 0 and 1 by subtracting the minimum value and dividing it by the maximum value. These were applied to the windows before the training of the DL model.

Data augmentation is a popular technique in DL, especially in image classification, to tackle overfitting [67] by introducing noises and transforming the data to increase data size. There were six studies [23, 37, 38, 49, 50, 58] using data augmentation on multi-channel data. Among these, two studies [37, 38] reordered channels by the Pearson correlation with a randomly chosen reference channel so that the classification system would learn to be invariant to the order of input channels. Another method is to create jittered signals by randomly translating windows with IEDs by 0.1 s at the beginning and end of a window [23]. Unsupervised Data Augmentation (UDA) [68] was also used in one study to randomly perturb amplitude levels and the order of channels [58]. Synthetic Minority Oversampling Technique (SMOTE) [69] was also employed to synthesize the IED samples by taking the average of the closest samples and was shown to improve the performance in two studies [44, 57].

There were six DL architectures in the included studies, convolutional neural network (CNN), long short-term memory (LSTM), combinations of CNN and LSTM or bidirectional LSTM, combinations of CNN and Gated recurrent unit (GRU), graph convolutional network (GCN), and hybrid. All of them were supervised learning as they involved training on labels annotated by experts. The frequencies of these variants across studies are depicted in figure 5, which includes overlap as several studies explored multiple architectures.

Zoom In Zoom Out Reset image size

CNNs have shown great performance in computer vision and were the most popular method, appearing in 14 studies. A CNN is designed to process grid-like data by optimizing filters or kernels with the convolution operation. In other words, the network applies matrix multiplications on partial data iteratively, as illustrated in figure 6. In IED detection, the EEG signal is considered as a multivariate or univariate time series (EEG channels as features) and passed to a one dimensional (1D) CNN (N = 10) or two dimensional (2D) CNN (N = 9). In the traditional 2D CNN in image classification, the inputs are 3D matrices (width × height × features), and the kernels slide along the width and height dimensions. Temporal or 1D CNN uses causal convolutions and dilations [70], sliding along the time dimension and extracting temporal features from time series. These two types of CNN achieved comparable performance [47]. In terms of 2D CNN, seven studies [23, 37, 39, 47, 49, 54, 55] used the convolution strides with the first dimension of 1, which means they only learned features channel by channel. The maximum probability output across all channels is used as the output of the epoch or window when the inputs are single-channel. This approach was applied to outputs from 1D CNN in two papers [48, 52]. The proposed 2D CNN used a kernel size of 3 × 3 [50, 56, 58]. Jing et al employed 2D CNN which started with a temporal or 1D convolutional layer and then 2D convolutional layers with increasing filter sizes size, from 8 to 128, as the network became deeper [23].

Zoom In Zoom Out Reset image size

Existing 2D CNN architectures in computer vision were also employed, Fast Region-based Convolutional Network (Fast R-CNN) [58] and VGG [50, 56]. The 1D residual network from time series classification (ResNet-TSC) was tested in one study [38]. ResNet-TSC [71] stacks multiple convolutional layers with residual connections to ensure the small residuals do not vanish. Fast R-CNN [72] learns local and global features from different regions of image-like inputs. VGG [73] was invented by the Visual Geometry Group at the University of Oxford, consisting of 16–19 convolutional layers.

LSTM was explored in five studies [43, 44, 46, 53, 54]. LSTM mimics the behavior of the long-short term memory in the human brain by choosing which information of previous time steps to carry to the next steps with the help of a forget gating mechanism. The GRU cell is similar to the LSTM cell but with fewer parameters as the output unit is obsolete. LSTM and GRU were applied to the whole windows in which the voltage values of channels at each time step were used as features (figure 7). Bi LSTM was used in one study [44] to extract features of a sequence in both forward and backward directions. Information to pass through CNN with LSTM or GRU was also combined by stacking the convolutional layer and LSTM cell [54]GRU cell [49, 57]. However, GRU and LSTM suffer from high computational costs and exploding gradient problems [74]. This drawback explains the dominance of CNN in automated IED detection, especially with rectified linear unit (ReLU) activation. ReLU allows for faster convergence by setting inputs to the layers to 0 if they are negative, introducing sparsities, and avoiding the gradient issues of the activation functions in LSTM.

Zoom In Zoom Out Reset image size

In the GCN work [59], a montage was viewed as an undirected graph in which each electrode was a node and the linkage between a pair of electrodes was an edge. A stack of Chebyshev convolutional layers [75] was then applied to estimate the Fourier transform of the graph signals. This particular GCN architecture also applied an embedding layer consisting of a 1D convolutional layer to extract temporal features from each electrode. The global features of the graph were computed as the sum of features from all electrodes.

A hybrid method, combining stochastic processes with DL, was tested with the Temple University EEG Events dataset [51]. This method first extracted a sequence of features: discrete Fourier transform, energy, discrete cosine transform, mel-frequency cepstral coefficients [76], and derivatives. These features were then further processed via three steps: a Hidden Markov model for sequential decoding of EEG events, a stacked denoising autoencoder for temporal and spatial analysis, and finally, a statistical language modeling for final classification.

Deep learning networks consist of stacks of layers. The number of layers and hidden units is often referred to as depth and width, respectively. Carefully balancing these would lead to an increase in performance [77]. We counted all layers, including pooling, dropout, batch normalization, and final classification layers. The number of layers in included studies ranged from 4 to 31 layers (median: 9; IQR: 5–21). Only one study mentioned the number of hidden units of LSTM layers (200 units) [44]. In terms of convolutional layers, the number of filters, starting from 16, 32, or 64, increased linearly with the depth.

Pooling, normalization, and dropout layers are popular in CNNs. Pooling layers reduce the size of inputs. There were two types of pooling layers seen in six studies [23, 37, 38, 55, 56, 58], max-pooling and strided convolution. While max-pooling keeps maximum values of the feature dimensions within non-overlapping moving kernels, strided convolution learns which features to keep by weighting them. When these were applied, the dimensions were reduced by half, and the number of convolution filters was subsequently doubled. Dropout layers [78] help increase the generalizability of the models by randomly setting hidden units (neurons within the layers that are not input or output layers of the network) to zero during the training process and were applied before the last fully connected layer in four studies [37, 38, 48, 52]. There were two studies [52, 56] applying dropout after each LSTM layer. Spatial dropout [79] was used in one study [59], which randomly zeroed features of some timesteps. Training DL models on the entire dataset is computationally expensive. As such, iterating through mini-batches of samples is a strategy for faster training speed. To overcome the statistical variance in batches, four studies [23, 38, 57, 60] used batch normalization to learn how to re-center and re-scale the distribution of inputs to internal layers within every batch [80]. Layer normalization was used along with a graph convolutional network in one study [59] to normalize the features of each electrode.

It is common in included studies for windows without IEDs to significantly outnumber windows with IEDs. The ratio could be up to 1:1000 [47]. The most popular strategy to address this is oversampling minor classes (i.e. the IED windows) or balancing the mini-batch by sampling the same number of samples from different classes in every batch. There was one study implementing focal loss [38] that applied a modulating term to the original cross-entropy loss, preventing the vast number of negative (i.e. interictal) samples from forcing the classifier always to predict the negative class. Learning the projection of the inputs on another coordinate plane has been widely examined in the DL community. Following this, one study [37] utilized the triplet loss [32] to maximize the distance between the projections of IED and normal samples.

In IED detection, the cross-entropy loss measures the difference between two probability distributions of predicted and true labels (in this case, a window is labeled as either IED or normal). The value of the loss function increases when the two distributions diverge from each other. Minimizing the loss is equivalent to reducing the error between the predicted and true labels. It is achieved by optimizing the neural network weights with gradient methods via backpropagation on the training data. The choice of optimizer would affect the training time and performance. Only nine studies mentioned the selection of the optimizer. The traditional Stochastic Gradient Descent (SDG) [81] was implemented in two studies [39, 42]. The most popular choice (N = 7) was the adaptive momentum optimizer (Adam) [82]. Adam uses adaptive learning rates to deal with sparse gradients and non-stationary objectives, allowing faster convergence.

IED detection is the task of classifying a window of EEG signal into either having IED or without IED. There were a variety of reported metrics among studies (figure 8). The most common metric was AUC, which is the area under the curve of the true positive rate (TPR) and the false-positive rate (FPR) at different probability thresholds of the probability outputs, where TPR = TP/(TP + FN), FPR = FP/(FP + TN). True positives (TP) are the correctly classified windows containing IEDs, and the false positives (FP) are the windows without IEDs incorrectly classified as containing IEDs. True negatives (TN) and false negatives (FN) are the correctly and incorrectly classified windows without IEDs, respectively. The median AUC score was 0.94, and the highest AUC was 0.99 [47] (IQR: 0.94–0.96). Figure 9 compared mean AUC scores by DL architecture. Sensitivity was another popular metric measuring how many annotated IEDs were detected (median: 0.91; IQR: 0.89–0.93). By contrast, specificity measures the performance in classifying windows without IEDs or the true positives (median 0.93; IQR: 0.89–0.99). Balanced accuracy (BAC) is the average of the two metrics (median 0.84; IQR: 0.77–0.93). We also plotted the boxplots of the values of the most common metrics in figure 10. From the boxplots, we identified four outliers, Nhu et al—GCN [59], Golhommadi—Hybrid [51], Tjepkema-Cloostermans et al—1D CNN [54], and Tjepkema-Cloostermans et al—1D CNN-LSTM [54]. Nhu et al—GCN had the lowest F1 and sensitivity scores. Golhommadi—Hybrid had the lowest specificity and BAC scores. The AUC scores of Tjepkema-Cloostermans et al—1D CNN and Tjepkema-Cloostermans et al—1D CNN-LSTM were the lowest. In addition, the average AUC score across all studies with multiple clinical centers was 0.93 [52].

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

False positives per minute (FP/minute) were measured in eight studies [37, 39, 44, 47–49, 57, 59] with the lowest value of 0.23 [39]. The measure that characterizes the percent of true detections in the events an algorithm thinks are IEDs is termed precision. F1 score conveys the balance of sensitivity and precision and was used in three studies [44, 47, 49]. Another metric measuring the relationship between these is AUPRC which computes the area under the precision-recall curve [47, 48].

Cross-validation (CV) is an evaluation method dividing the dataset into different subsets for training and testing purposes. CV across multiple subsets would give an overview of the generalizability of models. K-fold cross-validation was employed in six studies [23, 38, 43, 47, 48, 52] which split the data into different k groups, and each of them was used for testing iteratively. The number of folds was 3, 5, and 10. Leave-One-Patient-Out testing is another approach to divide data into different groups without each patient and is useful when the number of patients is relatively small. When the number of centers is large enough, Leave-One-Institution-Out, subgroups without data from a center, might be used to test the generalizability of the models across different clinical settings [52]. Golmohammadi [51] included the most clinical centers (6) and reported a mean Leave-One-Institution-Out AUC of 0.83 and a mean Leave-One-Subject-Out AUC of 0.82. This used the same design of CNN as in Prasanth et al [47]. These AUC scores are lower than the median; however, other studies did not perform any of the two cross-validation strategies.

Whole EEG classification involves categorizing an EEG recording into normal or epileptic and usually leverages the automated detection of IEDs. Similarly, patient classification distinguishes patients with epilepsy from those without epilepsy. There were two approaches to extracting features for an EEG recording. A common method is to consider an EEG recording epileptic if it comprises detected spikes. The other approach is to compute detected spikes per minute as a feature [52] and choose a threshold to maximize the performance. Features across all EEG recordings of a patient are then used for patient classification [52]. Only five studies [23, 38, 47, 52, 59] evaluated the performance of the whole EEG classification, and two studies mentioned patient classification with the best AUC scores of 0.85 [23] and 0.90 [52], respectively.