Feasibility of video/audio monitoring in the analysis of motion and treatment effects on night-time seizures – Interventional study

The aim of the study: This pilot study assessed the ability of a video/audio-based seizure monitoring system to evaluate (I) baseline frequency and severity of nocturnal seizures with motor features in patients with drug-resistant epilepsy (DRE) and (II) the individual effect of brivaracetam (BRV) treatment on number, duration and movement intensity of these seizure types. Algorithmic feature analysis was developed for assessment of qualitative changes in movement intensity measurements within seizure types before and after BRV intervention. Materials and methods: Night-time motor seizures of recruited patients were recorded in two separate four-week monitoring periods. The first period defined a prescreening phase (n = 13 patients) to establish a baseline, and the second period defined the intervention phase (n = 9 patients), with BRV initiated during the second week of the second monitoring period. All recorded nights were analyzed by an expert video reviewer, and all un- equivocal seizures were classified by an epileptologist. Seizure frequencies using both seizure diaries and video monitoring were compared. The effect of BRV on both seizure duration and movement intensity was assessed by numerical comparison of visual features calculated from motion characteristics of the video, as well as spectral features from the recorded audio. The statistical significance of changes in seizure duration and intensity before and after the intervention were investigated by Wilcoxon rank-sum test and visual inspection of Kernel density estimation. Results: 8 patients marked seizures in their seizure diaries during the prescreening phase. During the three-week follow-up, three patients achieved > 50% seizure decrease, four patients did not respond to treatment, and two patients experienced worsening of seizures. Five patients were able to document 40 – 70% of their seizures compared to the video/audio monitoring system. According to the signal feature analysis the intervention decreased movement intensity with clear clinical significance in three patients, whereas statistically significant differences in features appeared in 8 out of 9 patients. Conclusions: The novel video/audio monitoring system improved the evaluation of treatment effect compared to the seizure diaries and succeeded in providing a comparative intra-patient assessment of the movement intensity and duration of the recorded seizures.


Introduction
A single seizure may occur in 8-10% of the population during a person's lifetime, with 2-3% of individuals developing epilepsy (Gavvala and Schuele, 2016). Approximately one-third of patients have drug resistant epilepsy (DRE) defined as continuation of seizures despite using two or more anti-seizure medications (ASMs) with adequate doses either sequentially or in combination (Kwan et al., 2010). Treatment-resistant epilepsy causes significant mortality and morbidity (Laxer et al., 2014), and the risk of premature death due to epilepsy is 11-fold in comparison to the age-matched general population or siblings unaffected by epilepsy (Fazel et al., 2013). Annually, sudden unexplained death in epilepsy (SUDEP) occurs in 1 of 1000 epilepsy patients and in 6 out of 1000 in drug resistant epilepsy patients (Massey et al., 2014).
Outpatient assessment of the type and frequency of seizures is generally based on patient and caregiver reports (seizure diaries) which are used to improve recall of seizure occurrence. However, systematic diary follow-up requires prioritization and a demanding orderly approach, making them prone to inaccuracies. Seizures occurring during sleep or with impaired awareness may go unnoticed, especially in people living alone (Blachut et al., 2017;Geertsema et al., 2018). It is estimated that about half of seizures during wakefulness and up to 90% of nocturnal seizures go unnoticed (Elger and Hoppe, 2018;Hoppe et al., 2007). Complicated or prolonged seizures, or even SUDEP, may occur unexpectedly in situations where a good treatment response has already been assumed (Walczak et al., 2001). Inaccurate documentation of seizure type and frequency makes it challenging to monitor therapeutic outcomes of ASM therapy -both in clinical practice and within drug trials (Elger and Hoppe, 2018;Dalrymple and Appleby, 2000). Improved documentation of seizures could help clinicians to choose the most appropriate treatment based on seizure type and provide more accurate treatment effect data for drug trials.
Several different devices have been developed to detect movement during seizures and these can often be connected to alarm systems (Poppel et al., 2013). Video-based automated analysis of seizure-specific movements can be also used for follow-up of changes in night-time seizure frequency (Geertsema et al., 2018). However, detection of automated computer-assisted methods has been mostly limited to convulsive seizures in previously used devices (Beniczky and Jeppesen, 2019).
The Nelli® seizure monitoring system is an audio/video-based semiautomatic (hybrid) seizure monitoring platform that uses computer vision and machine learning to identify kinematic data (motion, oscillation, and audio) commonly associated with seizures with a positive motor component and human experts to visually assess these epochs (Peciola et al., 2018;Ojanen et al., 2021). In a recent validation study, the Nelli® hybrid system was used in a blinded setting without any prior information on the patients or their seizure types against video-EEG monitoring at a well-established epilepsy center identifying all tonic-clonic and clonic seizures and 82% of focal motor seizures. However, there was low accuracy in identifying seizure types with more discrete or subtle motor phenomena (Peltola et al., 2022). Nelli® has been recommended for clinical use in Finland by a government-appointed committee (the National Coordinating Group for Drug-resistant Epilepsy).
Brivaracetam (BRV) is a selective, high-affinity vesicle protein 2a ligand, which received FDA approval for use as monotherapy and adjunctive therapy for patients with focal epilepsy in 2016. In a phase 3 study, adjunctive BRV (100 and 200 mg/day) significantly reduced frequency of focal seizures compared with a placebo (Klein et al., 2015). BRV is commonly used in Finland as an add-on ASM in patients with DRE.
The aim of the present pilot study was to assess ability of data captured by the video monitoring system to establish (I) the baseline frequency of nocturnal seizures and the sensitivity of seizure diaries during the prescreening phase in patients with DRE scheduled for change of seizure therapy, and (II) the individual effect of ASM BRV treatment on seizure duration and movement intensity. Seizure counts based on subject registrations were compared with conventional seizure diaries in order to evaluate the inaccuracies associated with seizure diaries in the assessment of treatment effect in drug intervention. Additionally algorithmic feature analysis was developed for assessment of qualitative changes in intensity measurements within seizure types before and after BRV intervention. The present study provides proof-ofconcept of how computer-assisted video/audio-based detection may aid in documenting individual responses to treatment interventions in patients with DRE.

Study design
This was an open-label comparison of a computer-vision-assisted seizure monitoring tool and the clinical standard (patient seizure diaries) to observe changes in seizure burden during the initiation of a brivaracetam (BRV). The study consisted of 2 phases. In phase 1 (the prescreening phase), patients underwent a 4-week home monitoring simultaneously as they documented all night-time seizures in their seizure diaries while remaining on stable ASM. No change in medication was done during this phase. Phase 2 (the intervention phase) comprised a 1-week baseline period and a 3-week observational period that began once BRV was administered. Both monitoring tools were used throughout the study. Seizure frequency and semiology captured with each method were compared following each phase.
The study protocol and informed consent forms were reviewed and approved by the Ethics Committee of Tampere University Hospital. Signed informed consent was obtained from each participant. The classification of seizure and epilepsy type and prior knowledge on individual seizure characteristics was available prior to study entry was from VEM recordings obtained as part of routine care.

Patient population
Thirteen patients with focal DRE were enrolled in the study and participated through phase 1 (Table 1). Two patients did not proceed to the second phase due to infrequent seizure events or unobservable motor components; an additional two patients chose not to initiate BRV treatment. Thus, nine subjects completed both phases of the study. Seizure types were classified by an expert epileptologist according to ILAE 2017 classification (Fisher et al., 2017) along with the ILAE codes (Beniczky et al., 2017) in parentheses. Semiology was defined according to semiological classification by (Lüders et al., 1998) for each seizure type using additional descriptors for the observable types of movements manifesting during a seizure. All patients were treated with two or more ASMs, and some of the patients (3, 5, 7, and 8) were concomitantly treated with vagal nerve stimulation (VNS) therapy. Patients' clinical information, including age, sex, age at diagnosis, seizure types, seizure semiology, and ASM(s), are presented in Table 1.

Video monitoring
Video monitoring was conducted by NEL (Neuro Event Labs, Tampere, Finland) using its Nelli video monitoring product. The system includes a camera and microphone, to be installed at the bedside so that the patient is in sight of camera throughout the night. Nelli records epochs of potential seizure activity which is subsequently reviewed by epilepsy technicians and supporting physicians to develop an interactive summary. Information about seizure semiology from VEM reports obtained before the study initiation were used for evaluation of behavioral features of seizures. Behavioral events that were not unequivocally identified as seizures were excluded from assessment. It is important to note that, according to the epilepsy research community, seizures with unequivocal semiology are sufficient to act as a reference standard in this type of phase 2 study and thus video-EEG is not needed as a reference standard (Beniczky and Ryvlin, 2018).

Accuracy of seizure diaries
In phase 1, we compared each patient's seizure count and seizure diary entries during the 4-week monitoring period. We defined the daily average of seizures, diary entries, sensitivity, and positive predictive value of seizure diaries.
In phase 2, we calculated the percentage change of diary entries between the baseline and the third follow-up week for each patient. By comparing the difference between baseline and the third follow-up week based on the seizure average of the monitoring and seizure diary entries, we could evaluate the accuracy of seizure diary on therapy outcome assessment.

Effect of intervention
The seizure average per night was calculated for each week based on both video monitoring and seizure diary entries. Changes in overall seizure frequency between follow-up weeks and the baseline week were calculated based on the seizure average per night to avoid bias from additional recorded nights during the monitoring period and to ensure the results of all patients were comparable with each other.

Movement intensity and duration of seizures
To assess the effect of BRV on the movement intensity and duration of seizures before and after the intervention, we investigated 12 visual and 15 audio-based features. The visual features were derived from off-theshelf optical flow and background subtraction methods in OpenCV. For audio features, power spectral density information was extracted across different frequency bands.
The features in each seizure type and each patient were extracted, and those with significant differences were investigated before and after the intervention using visual inspection and the Wilcoxon rank-sum test. The Wilcoxon rank-sum test was used to assess the difference between the distributions of observations obtained between two separate groups on a dependent variable. The features were normalized based on the length of the features to avoid being biased by the seizure duration. The features are described in Supplementary material 1. The duration of seizures before and after the intervention was also investigated as a separate feature using the Wilcoxon rank-sum test.

Accuracy of seizure diaries
Eleven out of 13 patients during phase 1 had night-time seizures recognizable by the video monitoring system, and eight patients were able to register seizures in their seizure diaries. The sensitivity of seizure diaries varied between 8% and 84%. Overall, four patients (patients 2, 3, 7, and 8) marked seizures which were not observable in the video reference (and therefore considered false positives in the diary); one of these patients (8) marked more seizure diary entries than confirmed by video altogether. This resulted in a positive predictive value of 50-95% for seizure diaries. The daily average seizure count varied between 0 and 10.3, while the daily average of diary entries was between 0 and 3.1. According to seizure diaries, the average number of seizure-free nights for 28 days ranged from 16.6 to 23.2, but the average number of seizurefree nights measured by the video monitoring system varied from 0 to 8.7. Thus, seizure diaries underestimated the daily average of seizures and overestimated the seizure-free night count in 7 of 8 patients.
In phase 2, five patients (1, 3, 5, 7, and 8) recognized and marked seizures in their seizure diaries: two patients (3 and 7) with < 40%, one patient (1) with 60%, and two patients (5 and 8) with 70% of the seizures detected in the video registration marked in the seizure diary  (1600) Rufinamide (2400) Perampanel (8) during the baseline week. Only two patients (5 and 8) marked > 50% of their seizures during the third week of follow-up. According to the video monitoring, patients 1 and 8 reached 28% and 36% seizure reduction, but their seizure diaries showed a 37% decrease and 2% increase, respectively. Patients 5 and 7 experienced 56% and 143% seizure increase, but their seizure diaries showed a 15% and 200% increase, respectively. According to both video monitoring and the seizure diary, one patient (3) did not experience a change in seizure frequency. 4 patients (2, 4, 6, and 9) who did not mark seizures in seizure diaries had a daily seizure average between 0.4 and 6.4 in the third week of follow-up, and three of them reached > 50% seizure reduction. Results from phase 1 and 2 have been summarized in Table 2.

Effect of intervention
Nocturnal seizure count, seizures per day average, and change to baseline were calculated for every week for all patients, and diary entries, entries per day average, and their change to baseline. Based on the change of seizure frequency, patients were classified as follows: patients who experienced > 50% seizure reduction are responders to the medical treatment, patients with < 50% seizure reduction or < 50% seizure increase did not respond to treatment, and patients with more than 50% seizure increase experienced worsening of seizure frequency.
After three weeks of follow-up, three patients (4, 6, and 9) were responders, four patients (1, 2, 3 and 8) did not respond to treatment, and two patients (5 and 7) experienced worsening of seizure frequency. Seizure counts, seizures per day averages and changes to baseline, diary entries, averages of diary entries, and change to baseline are summarized in Table 2. In addition, the change in total seizure count in all patients has been displayed in Fig. 1.

Movement intensity and duration of seizures
To present changes in seizure movement intensity and duration, two graphs are shown for each patient. One represents the feature values of each seizure over the recording periods, and the other is kernel density estimation which visualizes the distributions of feature values before (red) and after (blue) the intervention. All the features with P-value < 0.05 are listed in Table 3. However, only those significantly different features before and after the intervention using both P-value < 0.05 and visual inspection were presented in the graphs. Fig. 2.
Some differences before and after the intervention are visible in all subjects. The analysis indicates that movement intensity decreased after intervention in all subjects, especially in patients 5, 7, and 9, even though the seizure frequency of patients 5 and 7 increased. The number of selected features with a p-value < 0.05 (Table 3) verifies the significance of the changes in patients 5 and 7. However, the results were affected by the changes of external factors in patients 2 and 3. In patient 2, the change in audio features of myoclonic seizures was caused by snoring instead of the change in seizure manifestation. In patient 3, the camera angle and the monitoring setup changed in the second registration, which affected the intensity analysis. The feature values and kernel density estimation graphs from intensity analysis have been gathered in Fig. 3.
Besides the extracted features, the duration of seizures before and after the intervention was studied. According to the Wilcoxon rank-sum test, duration of seizures between before and after intervention in patient 8 (motor seizures), patient 2 (myoclonic seizures), patient 7 (motor and tonic-clonic seizures), and patient 5 (hyperkinetic seizures) is significantly different. However, KDE graphs do not confirm this. Thus, it seems that the intervention does not significantly affect the duration of different seizure types in the nine studied patients. The feature values and kernel density estimation graphs from duration analysis have been gathered in Supplementary material 2.

Discussion
The results from the present study reiterate the inherent problems associated with traditional seizure diaries for assessing both the need for intervention as well as for evaluation of the treatment effect in a group of DRE patients with nocturnal seizures. Furthermore, they demonstrate the feasibility and value of a novel video/audio-based hybrid seizure Table 2 Results from phase 1: seizures, diary entries, daily average (marked in parenthesis), true positives of seizure diaries (marked in square brackets), sensitivity, PPV, seizure-free nights, and overall registered nights. Results from phase 2: seizure counts for baseline and follow-up weeks, diary entries and their daily average during the baseline and third week of follow-up. Daily averages of seizures and diary entries during each week marked in parentheses, and their changes compared to baseline were marked in square brackets. The seizure daily average and its change in follow-up weeks were painted blue to emphasize their comparability. detection system using human annotation for the confirmation of the algorithmically triggered and classified event. Finally, the intensity analysis presents a novel method to quantify intensity of movements due to treatment. However, further validation would be required before using such analysis in clinical contexts.
Reliable detection of seizures is important to improve patient outcomes and to assess treatment effects on various seizure types both in patient care and in drug development. There is no way to foresee the date or time of the next seizure occurrence in patients with seizures uncontrolled by ASMs. This lack of control may cause constant fear of seizures and is a major handicap for patients even when direct harm is not often caused by a single seizure (Laxer et al., 2014). In patients with developmental disability, uncontrolled seizures or fear of unobserved seizures further reduce the possibility for independent living arrangements (Devinsky et al., 2015).
During the prescreening phase, eight patients (1, 2, 3, 5, 7, 8, 10, and 11) documented only from 8% to 84% of seizures in their seizure diaries. During the intervention phase, five patients (1, 3, 5, 7, and 8) documented only from < 40-70% of seizures using seizure diary, which caused underestimation of seizure counts and inaccuracies in the evaluation of treatment effect. Some patients were not able to register any of their seizures in their seizure diaries. Seizure counts according to video monitoring were higher than diary entries in ten patients (1, 2, 3, 4, 5, 6, 7, 9, 10, and 11) during the screening phase, and in eight patients (1, 2,  3, 4, 5, 6, 7, and 9) during the intervention phase, which caused both underestimation of seizure frequency and overestimation of seizure free nights. Three patients (2, 3, 8) also marked non-epileptic events as seizures (false positives) in their seizure diaries during prescreening phase, which caused overestimation of seizure frequency in one patient (8). Thus, our study gives further credence to previous findings reporting that seizure diaries are prone especially to underestimation but also to overestimation of the seizure counts (Stokes et al., 2011;Goldstein et al., 2021).
Seizure diaries indicated a significant change in seizure frequency in only one patient (seizure increase in patient 7) and thus BRV treatment would have been considered a failure in all patients. However, 3 patients were responders according to the video monitoring. In addition, video recording provided information about treatment outcome for those four patients who were unable to document any seizures to their diaries. Two patients (12 and 13) did not experience any seizures during prescreening phase despite the suspicion of active epilepsy both according to their seizure diaries and video registration confirming the reliability of seizure diaries in these patients. Conversely, Nelli® hybrid system detected seizures that were completely missed by seizure diaries in three patients (4, 6 and 9) during prescreening phase, allowing them proceed to the intervention phase. This study indicates that the video monitoring system significantly improved the accuracy of treatment outcome assessment, as it provided additional evidence that the diaries alone could not produce.
There are various methods to quantify the patterns of movement in visual and audio data and utilizing such feature extraction methods highly depends on the application domain. However, the statistical significance before and after the intervention is not reliable enough alone in feature analysis. Therefore, all the changes in features were verified by a medical expert to assure that a selected feature measures the property of interest and were not affected by the environmental noise. For the purpose of clinical validation of the feature analysis a larger patient population is needed to study the efficiency of the proposed features. There are also other limitations of our study. The change of monitoring settings can significantly affect the results based on signal analysis. Seizure detection requires the patient to stay in sight of the camera, a blanket may impede the detection of movements and the device must be turned on. In the Nelli® hybrid system validation study, the performance was good for classifying seizures with clear motor components, but the current challenge for video detection systems was the recognition of seizures with more subtle motor features (Peltola et al., 2022). In the present study, most seizures recorded did indeed have unequivocal motor components. There's a possibility that seizures recorded by patients but not registered by Nelli® hybrid system might represent subtle seizures. However, even though video-EEG could improve identification of subtle motor events, video-EEG is not needed as a reference standard in this phase 2 study according to recent guidelines (Beniczky and Ryvlin, 2018). Furthermore, video-EEG confirmation is not feasible to use for evaluation of treatment effects due to the long duration of the registration required. On the other hand, the assessment of duration and intensity of seizures using video only may be imprecise without confirmation by EEG. In addition, some of the patients were not able to record the nights consecutively. This caused the 3-week follow-up period to vary from 28 to 40 nights causing dissimilarity and hampering comparison of the recording results between patients.

Conclusions
The video/audio-based seizure-monitoring system enabled recognition of a significant effect on seizure frequency and intensity after initiating adjunctive brivaracetam treatment in several patients with drug resistant epilepsy. The significance was based on the ability to accurately detect seizure numbers and types in individual patients with difficult-to-observe predominantly nocturnal seizures. Results between seizure diaries and Nelli® hybrid system recordings varied in our patient population. Seizure diaries often underestimated seizure numbers compared to video recordings. Therefore, the ability of seizure diary usage to detect change in numbers of specific seizure types after therapy modification was inferior to video monitoring tool. The change in nighttime motor seizure count based on diary entries could be larger or smaller than the actual change (as suggested by the video monitoring). Finally, assessment of BRV treatment efficacy was improved by using the video/audio recording system. Further research with larger patient groups is still needed to improve the reliability of feature analysis.

Conflicts of interest
AK, and MZ are employees of Neuro Event Labs, the company that provided the equipment and technology used in the study. PO has Fig. 2. Change in different seizure types during the baseline and 1, 2, and 3 weeks after the initiation of BRV in patients 1-9. FHS = focal hyperkinetic seizure, FMS = Focal myoclonic seizure, FIAMS = focal impaired awareness motor seizure, FBTCS = focal to bilateral tonic-clonic seizure, FTS = focal tonic seizure, FCS = focal clonic seizure. Fig. 3. part 1. Feature values (left) and Kernel density estimation (right) from intensity analysis for patient 5. Features 9, 11 and 12 are visual features calculated by using background subtraction method.part 2. Feature values (left) and Kernel density estimation (right) from intensity analysis for patients 7 and 9. Feature 1 is a visual feature calculated by using optical flow. provided medical consultation for Neuro Event Labs. JP is a shareholder of Neuro Event Labs and has participated in clinical trials for Eisai, UCB, and Bial; received research grants from Eisai, Medtronic, UCB, and Liva-Nova; received speaker honoraria from LivaNova, Eisai, Medtronic, Orion Pharma, and UCB; received support for travel to congresses from LivaNova, Eisai, Medtronic, and UCB; and participated in advisory boards for Arvelle, Novartis, LivaNova, Eisai, Medtronic, UCB, and Pfizer. Other authors claim no conflicts of interest.