Abstract
Diagnostic Accuracy of the Salzburg EEG Criteria for Non-Convulsive Status Epilepticus: A Retrospective Study
Markus Leitinger, Eugen Trinka, Elena Gardella, Alexandra Rohracher, Gudrun Kalss, Erisela Qerama, Julia Höfler, Alexander Hess, Georg Zimmermann, Giorgi Kuchukhidze, Judith Dobesberger, Patrick B. Langthaler, Sándor Beniczky. Lancet Neurol 2016;15:1054–1062.
BACKGROUND: Several EEG criteria have been proposed for diagnosis of non-convulsive status epilepticus (NCSE), but none have been clinically validated. We aimed to assess the diagnostic accuracy of the EEG criteria proposed by a panel of experts at the fourth London–Innsbruck Colloquium on Status Epilepticus in Salzburg, 2013 (henceforth called the Salzburg criteria). METHODS: We did a retrospective, diagnostic accuracy study using EEG recordings from patients admitted for neurological symptoms or signs to three centres in two countries (Danish Epilepsy Centre, Dianalund, Denmark; Aarhus University Hospital, Aarhus, Denmark; and Paracelsus Medical University, Salzburg, Austria). Participants were included from the Danish centres if they were aged 4 months or older, and from the Austrian centre if aged 18 years or older. Participants were sorted into two groups: consecutive patients under clinical suspicion of having NCSE (the clinical validation group) or consecutive patients with abnormal EEG findings but no clinical suspicion of NCSE (the control group). Two raters blinded to all other patient data retrospectively analysed the EEG recordings and, using the Salzburg criteria, categorised patients as in NCSE or not in NCSE. By comparing with a reference standard inferred from all clinical and para-clinical data, therapeutic response, and the final outcome, we calculated sensitivity, specificity, overall diagnostic accuracy, positive and negative predictive values, and inter-rater agreement for the Salzburg criteria. The reference standard was inferred by two raters who were blinded to the scorings of the Salzburg criteria. FINDINGS: We retrospectively reviewed EEG data from 220 patients. EEGs in the clinical validation group were recorded in 120 patients between Jan 1, and Feb 28, 2014 (Austria), and Aug 1, 2014, and Jan 31, 2015 (Denmark). EEGs in the control group were recorded in 100 patients between Jan 13 and Jan 22, 2014 (Austria) and Jan 12 and Jan 26, 2015 (Denmark). According to the reference standard, 43 (36%) of the 120 patients in the validation group had NCSE. In the validation cohort sensitivity was 97·7% (95% CI 87·9–99·6) and specificity was 89·6% (80·8–94·6); overall accuracy was 92·5% (88·3–97·5). Positive predictive value was 84·0% (95% CI 74·1–91·5) and negative predictive value was 98·6% (94·4–100). Three people in the control group (n=100) fulfilled the Salzburg criteria and were therefore false positives (specificity 97·0%, 95% CI 91·5–99·0; sensitivity not calculable). Inter-rater agreement was high for both the Salzburg criteria (k=0·87) and for the reference standard (k=0·95). Therapeutic changes occurred significantly more often in the group of patients fulfilling Salzburg criteria (42 [84%] of 50 patients) than in those who did not (11 [16%] of 70; p<0·0001). INTERPRETATION: The Salzburg criteria for diagnosis of NCSE have high diagnostic accuracy and excellent inter-rater agreement, making them suitable for implementation in clinical practice.
Commentary
The difficulty in defining status epilepticus is evidenced by the number of definitional refinements it has undergone (1, 2). Defining nonconvulsive status epilepticus (NCSE) has been equally difficult, even though it has been recognized at a relatively high frequency in critically ill patients. The current study by Leitinger et al. validates the most modern criteria for NCSE, the Salzburg criteria.
Several recent but historically important landmark studies precede and interweave with these criteria. Young et al. (3) introduced the most significant electrographic description of nonconvulsive seizures, now often referred to as the Young criteria. A working criteria for NCSE was introduced by Kaplan (4) encompassing a modified version of the Young Criteria. Among other revisions, it delineated a guideline for patients with known epileptic encephalopathies to which the Young criteria may not necessarily apply. In 2013, a panel of experts developed a consensus definition of NCSE, now known as the Salzburg criteria, drawing heavily from both the Young and Kaplan's works (3, 4). Its organization is in part extracted from the Standardized Computer-Based Organized Reporting of EEG (SCORE) framework, a set of terms and features that have been incorporated into a computerized database software, now in widespread use in parts of Europe (5). The graphical elements of the EEG in the Salzburg criteria are described using the ACNS (American Clinical Neurophysiology Society) standard terminology (6). This terminology precisely describes rhythmic and periodic patterns commonly seen in EEGs of critically ill patients, and has been validated through multisite consortium (7). Subsequent minor modifications to the Salzburg criteria (8) provide clearer instructions on applying it and demonstrate greater specificity by stricter use of the ACNS terminology. Lastly, the International League Against Epilepsy (ILAE) introduced a classification system for status epilepticus that is organized around four principal axes: 1) semiology, 2) etiology, 3) electrographic correlates, and 4) age. The Salzburg Criteria have been incorporated more or less in its current form, particularly into Axis 3.
It is within this context that the authors of the current study rigorously performed a retrospective evaluation of 220 consecutive patients to validate the Salzburg criteria. The authors partitioned the dataset into two: patients with ≥ 10 minutes of decreased level of consciousness suspected of having NCSE (clinical validation group) and patients with no clinical suspicion of NCSE but abnormal EEG (control group). The Salzburg criteria were applied to each patient. All patients were then also evaluated by a “gold” reference standard, which used all clinical, paraclinical, laboratory data, neuroimaging data, response to drugs, follow-up, and final outcome as well as EEG information to make the determination of presence of NCSE. The results of the Salzburg classification were not used in determining the reference standard. It can be reasonably assumed that this reference standard contains the best possible labeled dataset and is truly a gold standard. The Salzburg criteria were then compared with the reference standard.
Several major results were achieved. In the clinical validation group, the Salzburg criteria demonstrated very high sensitivity (97.7%) and negative predictive value (98.6%), high accuracy (92.5), and reasonably good specificity (89.6%) and positive predictive value (84.0). There were more false positives than one would like to have seen (8 of 120 patients) in the clinical validation group, but most were due to EEGs with fluctuations without evolutions, which are typically difficult to interpret. There was very high interrater agreement in both groups for both the Salzburg criteria administration and reference standard determination. In a particularly astute analysis, the authors investigated various epoch lengths (10, 30, and 60 seconds) and empirically found that a 10-second epoch provided the best trade-off for high sensitivity and specificity. This study thus provides welcome validation for the Salzburg criteria which, in one form or another, had been used by clinicians for many years.
This study is not without problems. One of the major issues is that the determination of NCSE by the reference “gold” standard was in part (probably overwhelmingly large part) determined by the EEG. As the Salzburg criteria took inspiration from the relatively ubiquitous Young Criteria (3) as well as the modifications introduced by Kaplan (4), it would have been reasonable to assume that the physicians examining the EEG in the reference standard likely were well versed and heavily influenced by these criteria as well, resulting in circular definition and potentially artificially high inter-rater agreement. This problem seems unavoidable to some degree. The authors acknowledge that defining NCSE due to improvement after administration of AEDs can be misleading and prone to overinterpretation of its effectiveness (8). Documenting clear temporal clinical improvement is frequently quite difficult. AEDs may cause EEG pseudoimprovement of patients with metabolic derangements without any meaningful clinical improvement (or even with clinical deterioration). These patients should not always be considered as NCSE. Furthermore, such a definition limits the usefulness of these criteria for clinical trials. This study was also performed retrospectively, and by investigators who were from the same institution from where the studies originated. There would have been greater external validity had the raters been truly independent. There is concern as to whether the performance of these criteria would apply to a representative consecutive patient population, particularly in the United States where the use of continuous EEG monitoring may be substantially higher than in Europe, but the overall rate of seizure detection by the reference standard (19.5%) is similar to past studies with larger volumes. Additional studies that replicate these findings using truly independent reviewers and additional studies examining distinct pediatric/neonatal populations are needed. Furthermore, we call upon the authors to make the entire dataset public, both for further critical analysis as well as for educational purposes. This should now be technically feasible with relative ease.
One important and potentially confusing point should be noted. This study does not imply that the existence of a 10-second EEG pattern that fulfills the Salzburg Criteria should be considered as NCSE, as nonconvulsive seizures without status epilepticus will fulfill the criteria as well. Other factors, as defined in the ILAE statement, should be considered in determining NCSE. As such, the question arises as to exactly what seizure duration or burden electrographically defines NCSE. The ILAE's operational time dimension for seizures likely to be prolonged leading to continuous activity (t1) is 10 minutes for “focal SE with impaired consciousness” and > 60 minutes for time dimension causing long term consequences (t2). The Neurocritical Care Society guideline in 2012 gives a limited definition as ≥ 5 minutes of clinical and/or electrographic seizures (9). Greater than 50% of an hour long epoch was used by the recently completed Treatment of Recurrent Electrographic Nonconvulsive Seizures (TRENdS) study. As such, this issue appears to remain unresolved.
Finally, the recognition that rhythmic and periodic discharges on continuous EEG often reside in an ictal-interictal continuum should be noted, and an attempt to define hard boundaries upon a process that defies such boundaries may necessarily be artificial. Furthermore, there is increasing concern that periodic patterns that would not necessarily be considered ictal may indeed represent hyperdynamic, deleterious states (10). As such, any classification system may see more modifications as more research emerges.
Despite the difficulties that exist within this study as well as existentially in this field, clinicians are still faced with the task of interpreting the EEGs and making a determination of NCSE. This study demonstrates that by applying validated terminology and thoughtful structural framework, a reasonable and reproducible set of criteria is possible. It remains to be seen whether the Salzburg criteria will stand the test of time in defining NCSE, but it is now supported with critical validation.