Abstract
A retrospective study was performed of adults admitted to the intensive care unit in order to determine the utility of the Acute Physiology and Chronic Health Evaluation II (APACHE II) score in predicting in-hospital mortality in intensive care unit patients with non-cardiac status epilepticus. The cohort consisted of 104 subjects, 50 (48.1%) male, 39 (37.5%) aged ≥65 years, with a mean APACHE II score of 17.88. Four models of the APACHE II system were assessed: numerical score, adjusted score mortality, category, and category mortality. All models demonstrated poor calibration and discrimination, even after adjustment for significantly different covariates. There were independent associations between mortality and acute or toxic-metabolic seizure etiologies, myoclonic seizures, and postoperative status. After multivariate adjustment, only the association with toxic-metabolic seizure etiologies remained. The APACHE II score is a poor predictor of mortality in intensive care unit patients with status epilepticus. Further investigation is warranted to develop better measures of acute physiological disease severity in status epilepticus and its impact on mortality.
Keywords
Introduction
Many scoring systems have been developed to predict in-hospital mortality based on illness severity, of which the Acute Physiology and Chronic Health Evaluation II (APACHE II) score is most commonly used in the ICU setting.1,2 Although subsequent versions of the APACHE scoring system have demonstrated excellent discrimination and calibration, their complexity has limited practical use. 3 The APACHE II score has been validated in multiple populations, including those from medical and surgical ICUs, nations outside of the United States, the elderly, and among patients suffering from specific disease states.4–7 However, limited information is available from patients with primary neurologic disorders, especially in the case of those with status epilepticus (SE), which itself confers high morbidity and reduced survival. 8 Many studies have assessed risk factors for mortality and morbidity in SE, often accounting for illness severity with the APACHE II score.9–11 However, the APACHE II score has never been exclusively tested in the SE population, raising the question of whether it represents a valid indicator of illness severity and mortality. The goal of this study was to assess the utility of the APACHE II score in predicting in-hospital mortality in adult patients with non-cardiac related SE.
Methods
Study design
A single site retrospective study at an urban academic medical center in the United States was performed. Medical charts were reviewed between 1 January 2005 and 31 October 2012. Subjects were identified via review of EEG logs which documented the recording, and summarized findings, of all hospital studies. Studies in which the summary or indication documented concern for SE, periodic, or rhythmic findings were individually reviewed. In addition, all emergently ordered EEGs were reviewed to determine whether epileptiform activity >5 min was present.
The diagnosis of SE was derived from the 2012 Neurocritical Care Society Guidelines for the Evaluation and Management of Status Epilepticus. SE was defined as five or more minutes of: (1) continuous clinical and/or electrographic seizure activity or (2) recurrent seizure activity without recovery (i.e. return to baseline) between seizures. 12
Subjects were included if they were admitted to the medical or surgical ICU, diagnosed with and managed for SE, and greater than or equal to 18 years of age. Subjects who suffered SE related to cardiac arrest were excluded. Study data were collected and managed using REDCap (Research Electronic Data Capture).
Collected clinical variables included age, gender, APACHE II score at the time of ICU admission, postoperative status, history of epilepsy, seizure type (nonconvulsive; myoclonic), duration of SE, and etiology of SE. Etiology of SE was categorized as due to an acute or chronic process; specific etiologies were also identified (antiseizure medication noncompliance, infection, stroke, toxic-metabolic, mass lesion, autoimmune, or other/unknown).
In-hospital mortality was recorded. APACHE II scores and APACHE II mortality risk estimates adjusted for principal diagnosis leading to ICU admission (nonoperative subjects: seizure disorder, postoperative subjects: neurologic) were calculated as per the original description by Knaus et al. on the day of ICU admission. APACHE II scores were also stratified into categories of increasing severity. For example, subjects with APACHE II scores of 0–4 were grouped together as the lowest severity category, followed, in order of increasing severity, by subjects with APACHE II scores of 5–9, 10–14, 15–19, 20–24, 25–29, 30–34, and ≥35). Mortality risk derived from these categories was calculated based on commonly used reference values, 13 not adjusted for etiology.
Ethical standards
This study was approved by the Institutional Review Board (IRB) at Drexel University College of Medicine and Hahnemann University Hospital, and was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. A waiver of consent was granted from the IRB at the site institutions due to the retrospective nature of the study.
Statistical analysis
Statistical analysis was performed with SPSS Statistics 23.0 (IBM). The APACHE II score was investigated via four different models for predicting in-hospital survival: (1) numerical APACHE II score, (2) APACHE II mortality, risk adjusted for principal diagnosis leading to ICU admission (nonoperative subjects: seizure disorder, postoperative subjects: neurologic), (3) APACHE II category, and (4) APACHE II category mortality risk (not adjusted for principal diagnosis leading to ICU admission).
Four APACHE II models were generated and analyzed in order to capture the different manner in which the APACHE II has been interpreted. Raw numerical scores, i.e. the APACHE II score, have been used as a proxy for disease severity, with increasing values indicative of greater disease severity. The risk of mortality based on the numerical APACHE II score can be calculated as per Knaus’s original paper, with the formula accounting for the primary ICU admission diagnosis. This presumably results in more precise risk stratification, as it is specific to the patient’s disease state. Categorical APACHE II scores are frequently referenced in APACHE II calculator applications/programs, with the score category associated with a corresponding mortality risk. With the ability to rapidly calculate the APACHE II score, these applications may be the most frequently used method by which the APACHE II score is applied in the clinical setting, making the category and mortality risk corresponding to that category perhaps the most relevant to daily practice.
To identify potential covariates which would affect the model, Pearson’s chi-square, Mann-Whitney U, and two sample t-tests were used as appropriate. Binary stepwise logistic regression analysis adjusted for significantly different baseline covariates in each APACHE II model. Model calibration was assessed with the chi-square statistic, and a model was considered to demonstrate good calibration if p <0.05.
Discrimination of the APACHE II score was assessed using the area under the receiver operating curve characteristic (AU-ROC). A model was considered fair if both sensitivity and specificity were greater than or equal to 0.70. Cutoff points of fair sensitivity and specificity for each APACHE II model were also determined. A p value <0.05 was considered significant.
Results
Demographics
Baseline demographic characteristics.
APACHE: Acute Physiology and Chronic Health Evaluation; ASM: antiseizure medication; NCSE: nonconvulsive status epilepticus; SE: status epilepticus.
APACHE II model calibration
Calibration of APACHE II Models.
APACHE: Acute Physiology and Chronic Health Evaluation, B: regression coefficient
Comparison of demographic characteristics in survivors versus decedents.
APACHE: Acute Physiology and Chronic Health Evaluation; ASM: antiseizure medication; NCSE: nonconvulsive status epilepticus; SE: status epilepticus.
There was no association between APACHE II outcome and mortality after multivariate adjustment. Among the covariates analyzed, only the presence of a toxic-metabolic seizure etiology remained independently associated with reduced survival (Table 3).
APACHE II model discrimination
APACHE: Acute Physiology and Chronic Health Evaluation

Receiver operating curves for mortality discrimination of APACHE II models.
APACHE II score: The overall model was not predictive of in-hospital survival. APACHE II scores ≤15.5 demonstrated sensitivities ≥73.1%, corresponding with poor specificities ≤39.7%. APACHE II scores ≥21.5 demonstrated specificities ≥73.1%, which corresponded to poor sensitivities ≤38.5%.
APACHE II mortality risk adjusted for principal diagnosis leading to ICU admission (nonoperative subjects: seizure disorder, postoperative subjects: neurologic): The overall model was not predictive of in-hospital survival. Subjects with a mortality risk ≤1.5% demonstrated sensitivities ≥73.1%, corresponding with poor specificities ≤39.7%. Adjusted APACHE II mortality risk ≥8.79% demonstrated specificities ≥71.8%, which corresponded to poor sensitivities ≤50%.
APACHE II category. The overall model was not predictive of in-hospital survival. Subjects in APACHE II categories with scores ≤19 demonstrated sensitivities ≥80.8%, corresponding with poor specificities ≤33.3%. Subjects in the APACHE II category with scores ≥20–24 demonstrated good specificity ≥91%, which corresponded to poor sensitivities ≤19.2%.
APACHE II category mortality risk: The overall model was not predictive of in-hospital survival. Subjects with APACHE II category mortality risk ≤19.5 demonstrated sensitivities ≥76.9%, corresponding with poor specificities ≤35.9%. Subjects with APACHE II category mortality risk ≥47.5% demonstrated good specificity ≥94.9%, which corresponded to poor sensitivities ≤3.8%.
Discussion
In this cohort of adult ICU patients with non-cardiac-related SE, the APACHE II score did not demonstrate good calibration or discrimination for the prediction of in-hospital mortality. Independent of the APACHE II model utilized, subjects with an acute or toxic-metabolic seizure etiology, myoclonic seizures, and postoperative status were less likely to survive. This contests the notion that the APACHE II score is a valid estimate of mortality in SE and suggests against its use in studies of SE outcome.
Evidence regarding the utility of the APACHE II score in stratifying mortality in SE patients is mixed. In a retrospective study by Claassen et al., 9 the APACHE II score was associated with mortality in univariate, but not multivariate, analysis. Mayer’s group compared patients with refractory (RSE) versus non-refractory SE; there was no significant difference in APACHE II scores between groups nor any association with mortality. 10 In another study examining patients with RSE, subjects were divided into high- and low-risk groups based on APACHE II score and age, and no differences in mortality or functional outcome were observed. 11 Alternately, Prasad et al. 14 found that the APACHE II score was an independent predictor of mortality in patients with RSE. In one of the few studies that directly evaluated the APACHE II score in neurologically ill ICU patients, APACHE II demonstrated poor discrimination for patients classified as suffering from “other neurologic disease” (49 of 653 subjects), although it performed well for those with cerebral infarction, intracerebral hemorrhage and neurologic infection. 3 Unfortunately, the number of patients diagnosed with seizures or SE who may have been classified under “other neurologic disease” is not specified.
There are several explanations for these mixed results. In the original study from which APACHE II was derived, few neurologic patients, and even fewer with seizures, were represented. The original APACHE II score utilized data from the medical, surgical, and cardiac ICUs of 13 hospitals. Of the 5030 patients included in score calculation, 707 suffered from a primary neurologic disorder. Within this subgroup, 177 patients were classified as nonoperative, of whom 51 were primarily admitted to the ICU for a seizure disorder; and of 530 postoperative patients, 448 underwent a craniotomy, while 82 were classified as “neurologic” for the major organ system which resulted in ICU admission. Thus, only 7.21% of the original APACHE II cohort definitively demonstrated seizures, and of these, the number with SE is unclear. 1
Although the mortality prediction model by Knaus et al. includes a formula accounting for the primary cause of ICU admission, this adjusted APACHE II mortality risk continued to demonstrate poor calibration and discrimination in this cohort. As the diagnosis of SE does not exist in the APACHE II scoring system, the most similar related diagnoses—that is, “seizure” for the nonoperative group, and “neurologic” for the postoperative group—were substituted in calculating the adjusted score for this study.
In addition to the low representation of patients with seizures, unknown amount of patients with SE, and absence of an SE category for mortality adjustment based on primary cause of ICU admission, the APACHE II score excludes other risk factors known to be associated with mortality in SE. While SE determinants such as age and Glasgow Coma Scale score are included in the APACHE II calculation, it does not account for seizure type (i.e. nonconvulsive, myoclonic, refractory) and acute seizure etiology, each of which are associated with elevated mortality, nor for the presence of a history of seizures, which confers increased odds of survival.15,16
Several well-established risk factors for mortality in SE were assessed.17–19 In this cohort, decedents demonstrated a greater frequency of seizures due to an acute etiology, while a history of seizures was less common, which is in accordance with the majority of the literature.15,16 Age ≥65 years was not a risk factor, although this may reflect ceiling effects from the older overall age of the cohort (mean: 59 years). The duration of SE was also not associated with mortality, likely due to the high number of subjects with prolonged SE (median: 20.50 h). Refractory SE was not separately analyzed, as retrospective review limited the ability to identify all antiseizure medications used, and whether subjects received adequate dosages. However, 83% of the cohort received a benzodiazepine as their first drug. Nonconvulsive seizures were equally represented in survivors and decedents, which could be related to the long duration of SE in this cohort, as NCSE is difficult to clinically recognize, and may delay treatment, which itself has been linked to poor outcome.20,21
In this cohort, novel characteristics which were associated with reduced survival, or alternatively, increased mortality, included postoperative status, myoclonic SE, and toxic-metabolic seizure etiology.
Increased risk of mortality in postoperative patients as classified by the APACHE II system has not been a consistent finding in the literature. Wong’s group found that medical patients demonstrated higher mortality rates compared to surgical patients, specifically for APACHE II scores ranging from 0 to 19. 22 In a study by Rowan et al., medical patients also demonstrated higher mortality than surgical patients, although those who underwent emergency, as opposed to elective, surgery, had greater mortality. 4 Conversely, another group reported that in-hospital mortality was greater for those who did not undergo emergency surgery. 23 These mixed results warrant further investigation, which may help to elucidate the significance of increased mortality risk in postoperative SE patients.
Myoclonic seizures were also associated with mortality. Although myoclonic SE has been studied in patients after cardiac arrest or with primary generalized epilepsy (PGE), its significance is not well-established outside of those populations. In this cohort, of the eight subjects with myoclonic SE, there were five decedents, with SE etiologies including drug overdose, sepsis, hepatic and/or renal failure, and heparin-induced thrombocytopenia (HIT). Of the three survivors with myoclonic SE, two had a history of epilepsy (progressive myoclonic epilepsy secondary to Lafora disease, focal epilepsy), and one was diagnosed with Creutzfeld-Jacob disease (CJD). Increased mortality is a consistent finding across cardiac arrest-related myoclonic SE, but not in adults with PGE, the latter outcome of which concurs with this cohort.24–26 Of note, although the subject with CJD technically survived until the end of hospitalization, the patient was discharged to hospice. This suggests that acutely acquired myoclonic SE, or factors related to it, confers higher mortality risk, even in patients without cardiac arrest.
Another novel finding was that decedents were more likely to suffer from SE due to a toxic-metabolic etiology. Specific toxic-metabolic etiologies of the 14 decedents included drug overdose, sepsis, heparin-induced thrombocytopenia, hepatic and/or renal failure, and graft versus host disease complicated by thrombotic thrombocytopenic purpura. Of the eight survivors with SE precipitated by toxic-metabolic causes, three had a history of epilepsy (two focal epilepsy, one progressive myoclonic epilepsy due to Lafora disease), one had a chronic symptomatic seizure etiology (multiple sclerosis), three were exposed transiently to precipitating substances (alcohol, lidocaine, tacrolimus), and one was due to transient metabolic dysfunction following one missed session of hemodialysis. While toxic-metabolic dysfunction is grouped under acute seizure etiologies, a known predictor of mortality in SE, its independent association has not been separately reported.9–11 Although the APACHE II score assesses physiologic variables, presumably representing toxic-metabolic dysfunction, the lack of association between the APACHE II score and mortality in SE patients suggests that this characteristic is not sufficiently reflected in the APACHE II score for patients with SE. Given the diversity of toxic-metabolic conditions, however, it may be difficult to further elucidate the risks that these etiologies confer. Also of note, those subjects with transient toxic-metabolic dysfunction were more likely to survive, indicating that it may be the degree of reversibility, and not the presence of toxic-metabolic dysfunction alone, which confers higher mortality risk.
Alternative scoring systems for the prediction of SE mortality have been proposed, in which several of the better established risk factors for SE are included, such as the Status Epilepticus Severity Score (STESS) and Epidemiology-based Mortality Score in Status Epilepticus (EMSE).27,28 However, while EMSE calculation involves a comorbidity score (adapted from the Charlson’s Comorbidity Index), neither STESS nor EMSE account for the impact that a patient’s acute physiologic derangements may have on mortality.
The APACHE II score has the advantage of being able to quantify acute physiologic disease severity using objective data which is commonly collected. While this ostensibly influences patient outcomes, 29 the manner and extent in which acute physiologic disturbances contribute to mortality remains uncertain. As the APACHE II scoring system was not devised for the SE population, it is not an accurate representation of acute physiologic disease severity in these patients. Major limitations include the absence of accounting for SE etiology, which plays a vital role in outcome, as well as additional semiologic and historical features of seizures, emphasizing that SE should not be regarded as a uniform disease entity. Further studies investigating accurate measures of acute physiologic disease severity in SE are warranted.
This study has several strengths. The cohort consisted of patients from medical and surgical ICUs, the same setting in which the APACHE II score was derived. The mortality rate of SE in this cohort (25%) is commensurate, although slightly higher than the upper limit of 7.6–22%, which has been reported in the short term. 30 This may be explained by the larger proportion of subjects with prolonged seizure duration, which itself has been associated with elevated mortality (12), and perhaps related to this, the proportion of subjects with NCSE (44.2%), which can be difficult to recognize clinically. Additional pertinent variables which may have affected outcome and have been established in the literature were also adjusted for, including history of epilepsy, seizure semiology (nonconvulsive; myoclonic), duration of SE, and etiology of SE (acute versus chronic).
This study also has several limitations, including the retrospective nature of the analysis. Variation in ICU care may have influenced outcome. Although study subjects were recruited from medical and surgical ICUs, as in the original APACHE II paper, the differences in these patient populations may limit comparability. As a single center study, the results from this cohort may not be generalizable to the population. The number of subjects with SE may have been underestimated or misidentified by retrospective review because the diagnosis of SE changed during the study period. Subgroup analysis of additional potentially relevant risk factors was limited by the size of the cohort. Power also limited the ability to adjust for multiple potential confounders, and resulted in selectively chosen variable collection and analysis. However, no clear consensus exists regarding which characteristics are the most pertinent for mortality prediction in SE. Thus, only well-established risk factors, as culled from the literature, were selected and adjusted for.14,15 Data regarding seizure duration were only available for 95% of subjects (99/104), but otherwise analyzed data were complete. Antiseizure medication treatment choices and dosages were not uniform, and final cause of death (i.e. withdrawal of care versus due to disease) was not adjusted for, which may also affect results. In addition, the APACHE II score was only calculated at the time of ICU admission and may not reflect medical complications which could have arisen later in the course of a patient’s hospitalization.
Conclusions
The APACHE II score does not predict mortality in ICU patients with SE, even after controlling for pertinent risk factors. Independently, patients with an acute seizure etiology, toxic-metabolic seizure etiology, myoclonic seizures or who were postoperative demonstrated elevated risk of in-hospital mortality. Further investigation is warranted to develop better measures of acute physiological disease severity in SE and to determine the impact that this, in conjunction with other pertinent characteristics, may have on mortality.
Footnotes
Acknowledgements
The author would like to thank Dr. Niraj Arora for his assistance with data collection. This work was performed at the Drexel University College of Medicine, Philadelphia, PA, USA.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and publication of this article.
