Abstract
van Westrhenen A, Lazeron RHC, van Dijk JP, Leijten FSS, Thijs RD; Dutch TeleEpilepsy Consortium. Epilepsia. 2023;64(8):2137-2152. doi:10.1111/epi.17654 There is a pressing need for reliable automated seizure detection in epilepsy care. Performance evidence on ambulatory non-electroencephalography-based seizure detection devices is low, and evidence on their effect on caregiver’s stress, sleep, and quality of life (QoL) is still lacking. We aimed to determine the performance of NightWatch, a wearable nocturnal seizure detection device, in children with epilepsy in the family home setting and to assess its impact on caregiver burden. We conducted a phase 4, multicenter, prospective, video-controlled, in-home NightWatch implementation study (NCT03909984). We included children aged 4-16 years, with ≥1 weekly nocturnal major motor seizure, living at home. We compared a 2-month baseline period with a 2-month NightWatch intervention. The primary outcome was the detection performance of NightWatch for major motor seizures (focal to bilateral or generalized tonic-clonic [TC] seizures, focal to bilateral or generalized tonic seizures lasting >30 s, hyperkinetic seizures, and a remainder category of focal to bilateral or generalized clonic seizures and “TC-like” seizures). Secondary outcomes included caregivers’ stress (Caregiver Strain Index [CSI]), sleep (Pittsburgh Quality of Sleep Index), and QoL (EuroQol five-dimension five-level scale). We included 53 children (55% male, mean age = 9.7 ± 3.6 years, 68% learning disability) and analyzed 2310 nights (28 173 h), including 552 major motor seizures. Nineteen participants did not experience any episode of interest during the trial. The median detection sensitivity per participant was 100% (range = 46%-100%), and the median individual false alarm rate was .04 per hour (range = 0-.53). Caregiver’s stress decreased significantly (mean total CSI score = 8.0 vs. 7.1, p = .032), whereas caregiver’s sleep and QoL did not change significantly during the trial. The NightWatch system demonstrated high sensitivity for detecting nocturnal major motor seizures in children in a family home setting and reduced caregiver stress.Objective:
Methods:
Results:
Significance:
Commentary
I have been recommending seizure detection device/s (SDD) in my pediatric epilepsy practice for many years with mixed feedback. Undetected seizures and false alarm rates are main reasons to give up its use.
How Do You Choose the Best SDD for Your Patient?
While treating epilepsy with anti-seizure medication (ASM), I can rate and compare performance of said ASM based on published rates of efficacy, adverse effects, cost, and so on. The option of choosing from different SDDs while being able to compare and contrast them on level grounds does not yet exist.
How Should We Thus Counsel Parents/Caregivers When Prescribing an SDD in 2023? Will the SDD Detect All Seizures? Will It Prevent SUDEP? Will It Help in Reducing Caregiver Stress, Improve Caregiver Sleep? Will It Be Cost Effective?
After the first International Congress on “Mobile Heath Devices and Seizure Detection in Epilepsy” was held in Copenhagen in 2017; a paper detailing standards of testing and clinical validation of SDDs was published. 1 These standards are noted in Table 1. Similar to drug trials; a phase 3 SDD trial is designed to detect efficacy and safety while a phase 4 trial qualitatively evaluates SDD implementation in the patient’s home. Based on a systematic literature search up to 2019; clinical practice guidelines in the use of SDD were published in 2021. 2 This paper highlighted areas of further SDD research needed to evaluate meaningful clinical outcomes like overall morbidity, seizure quantification, and quality of life. Additionally, a call was made to conduct more phase 4 studies to critically assess performance of the SDD with respect to false alarm rates, costs, and assessment of patient perspectives.
Table for Epilepsy Currents Adapted From Reference 1 With Permission.a
Abbreviations: +, means compulsory; +/− means optional; Na, means not applicable; −, means excluded.
a Standardization of Clinical Testing and Validation of Seizure Detection Devices: Study Phases.
Note that as the robustness of the studies goes up from phase 0 to phase 4: numbers of patients go up. the recordings move from conventional methods to using a dedicated SDD in continuous, multicenter, blinded fashion. Analysis is done using a predefined algorithm with cut off values initially in an unblinded and asynchronous (after the recording) to real-time and blinded fashion. The reference standard for corroborating events moves up from caregiver or patient to video or video plus EEG recordings.
Phase 4 studies are “in home” studies and therefore the “multicenter” standard does not apply.
Most non-EEG SDDs have algorithms based on multimodal signals from accelerometry, photoplethysmography, electrodermal activity, or heart rate in some combination. It is also suggested that there should be some way to compare data quality from all these signals using a multimodal signal quality index 3 to facilitate comparison among different SDD studies.
Recently, a first of its kind, phase 4 SDD study in pediatric patients was published in Epilepsia by Anouk Westrhenen et al on behalf of the Dutch Epilepsy Consortium. 4 This study called the PROMISE trial (Promoting Implementation of Seizure Detection Devices in Epilepsy) satisfies all the above recommendations in SDD research.
The Dutch group first conducted a phase 4 study of an SDD called NightWatch in adults. NightWatch combines 3 dimensional accelerometry and photoplethysmography to detect seizures, while concomitant video recording and seizure diaries provide confirmation. This pivotal out of hospital study was conducted in adults living in a supervised residential facility. 5 After detecting an acceptable sensitivity (86%) and false alarm rate (0.2/night), a validation study of NightWatch in children was conducted using the same adult algorithm; but found unacceptable false alarm rates (0.2/h) due to excessive motion. 6 The adult algorithm was therefore adapted for pediatric use. NightWatch was programmed to alarm if excessive motion was detected in the horizontal plane (eg, motion while sleeping but not when sitting up in bed moving arms).
The PROMISE trial was conducted to validate this adaptation while also researching outcomes of caregiver strain, cost, and caregiver sleep. Patients (ages 4-16 years) with at least 1 major motor seizure per week were recruited from 4 tertiary epilepsy centers. Patients were monitored in their home for 4 months (2 months baseline and 2 months of wearing NightWatch to sleep). Parents kept seizure diaries of events throughout. The primary outcomes included sensitivity, F1 performance, positive predictive value, and false alarm rates. Secondary outcomes included quality of the data signal as well as impact of NightWatch on caregivers’ sleep, stress, and quality of life using validated questionnaires.
Data from 51 patients, 552 seizures, and 2310 nights were assessed: assuring robustness of results. Data from 2 of the 53 total recruited patients was excluded in the performance analysis due to loss of video or recordings of insufficient quality. Although overall sensitivity of NightWatch was 89% (492 out of 552 seizures correctly identified); median sensitivity per participant for major motor seizure identification was 100% for tonic–clonic seizures (range 71%-100%). NightWatch detected other major motor seizures with high sensitivity per participant: tonic seizures > 30 seconds:100% (range 0%-100%), hyperkinetic seizures:75% (range 0%-100%), and other motor seizures:100% (range 0%-100%). Median false negative alarm rate per participant per hour was 0 (NightWatch missed a total of 60 seizures). A total of 1642 false positive alarms were identified but median false (positive) alarm rate per subject per hour was 0.04. In the post hoc analysis; children with developmental delay were found to have greater false alarm rates. As far as quality of data was concerned; approximately 10% (241/2551) of the recorded nights were excluded from analysis due to a combination of insufficient video data (n = 159); computer storage issues (n = 51), inadequate heart rate signal (n = 27); lost connection with the base station (n = 2) or children being out of bed (n = 2). Adverse effects were mild skin irritation (n = 8).
Authors also published secondary outcomes through validated questionnaires. However less than 50% of parents/caregivers completed the online questionnaires regarding caregiver stress, sleep, and quality of life. During the intervention period there was a statistically significant reduction of care giver stress, but sleep and quality of life scores did not change. When offered the chance to buy the NightWatch at the end of the trial at half the price; only 32% agreed.
To further assess perceived value, authors conducted a separate in-depth interview of 23 parents of 19 children from the PROMISE trial to gauge their experience using this SDD. 7 Authors concluded that perceived value of an SDD is determined by a very complex interplay of factors. The amount of assurance from an SDD is balanced against the ability of parents to tolerate the extra burden of care. This extra burden comes from the added cost of the device along with the intrusion of false seizure alarms or technical alarms.
In conclusion, we have come farther since the systematic analysis by Beniczky and Jeppesen, 8 which stated that SDD accuracy for detection of seizures other than generalized tonic–clonic seizures is poor, many SDDs have high false alarm rates and unvalidated devices perform poorly. Westrhenen et al have now validated NightWatch in pediatric patients and it performs reasonably well in detection of non-tonic-clonic seizures too. This study shows the systematic steps needed to truly validate an SDD, making necessary adaptations for best outcomes.
We do not yet have a device that can predict or prevent SUDEP. More studies will continue to bring us closer to our goal of implementing a reimbursable SDD as standard of care to guide medical decisions.
