Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Abstract

Introduction:

Out of hospital cardiac arrest (OHCA) is a common problem. Rates of survival are low and a proportion of survivors are left with an unfavourable neurological outcome. Four models have been developed to predict risk of unfavourable outcome at the time of critical care admission – the Cardiac Arrest Hospital Prognosis (CAHP), MIRACLE₂, Out of Hospital Cardiac Arrest (OHCA), and Targeted Temperature Management (TTM) models. This evaluation evaluates the performance of these four models in a United Kingdom population and provides comparison to performance of the Acute Physiology and Chronic Health Evaluation II (APACHE-II) score.

Methods:

A retrospective evaluation of the performance of the models was conducted over a 43-month period in 414 adult, non-pregnant patients presenting consecutively following non-traumatic OHCA to the five units in our regional critical care network. Scores were generated for each model for where patients had complete data (CAHP = 347, MIRACLE₂ = 375, OHCA = 356, TTM = 385). Cerebral Performance Category (CPC) outcome was calculated for each patient at last documented follow up and an unfavourable outcome defined as CPC ⩾ 3. Performance for discrimination of unfavourable outcome was tested by generating receiver operating characteristic (ROC) curves for each model and comparing the area under the curve (AUC).

Results:

Best performance for discrimination of unfavourable outcome was demonstrated by the high risk group of the CAHP score with an AUC of 0.87 [95% CI 0.83–0.91], specificity of 97.1% [95% CI 93.8–100] and positive predictive value (PPV) of 96.3% [95% CI 92.2–100]. The high risk group of the MIRACLE₂ model, which is significantly easier to calculate, had an AUC of 0.81 [95% CI 0.76–0.86], specificity of 92.3% [95% CI 87.2–97.4] and PPV of 95.2% [95% CI 91.9–98.4].

Conclusion:

The CAHP, MIRACLE₂, OHCA and TTM scores all perform comparably in a UK population to the original development and validation cohorts. All four scores outperform APACHE-II in a population of patients resuscitated from OHCA. CAHP and TTM perform best but are more complex to calculate than MIRACLE₂, which displays inferior performance.

Keywords

Cardiac arrest risk score neuro-prognostication mechanical circulatory support percutaneous coronary intervention

Introduction

Sudden out of hospital cardiac arrest (OHCA) remains a common problem. Resuscitation attempts are made on approximately 30,000 patients a year in the United Kingdom (UK).¹ Rate of survival to hospital discharge varies worldwide from 3% to 20%^2,3 with up to 8% survivors are reported to have unfavourable neurological outcome.⁴

Where resuscitation is successful, but the patient remains unconscious, current European Resuscitation Council (ERC)/European Society of Intensive Care Medicine (ESICM) guidelines on post resuscitation care⁵ recommend delaying formal neuro-prognostication until at least 72 h following ROSC (if a targeted temperature management (TTM)⁶ protocol that targets <37°C is used then this should not happen before 12 h after completion of re-warming) and recommend a multi-modal approach using a combination from clinical assessment, radiological imaging of the brain, multi-channel electroencephalography (EEG) recording, somatosensory evoked potential (SSEP) recording and measurement of neurone specific enolase (NSE). Complications such as aspiration pneumonia,⁷ health care associated infection,⁸ or slow to resolve multi-organ dysfunction⁹ are common in these patients and as a result prognostication is frequently delayed beyond 72 h.

In the resuscitated patient with ongoing haemodynamic instability, interventions such as percutaneous coronary intervention for non ST-elevation ischaemia may be considered and, in some centres, mechanical circulatory support may be available. These interventions require a large amount of resource and it would be desirable to have a tool which allowed targeting the availability of these interventions at those most likely to benefit. There are no established clinical, biochemical, radiological, or neuro-physiological markers that predict neurological outcome with sufficient reliability, such that they could be used in isolation immediately or in the early period following successful resuscitation.

Several statistical models have been developed to predict neurological outcome at the time of critical care admission. These include the Cardiac Arrest Hospital Prognosis (CAHP),¹⁰ MIRACLE₂,¹¹ Out of Hospital Cardiac Arrest (OHCA),¹² and Targeted Temperature Management (TTM)¹³ models (Table 1). This evaluation aims to validate the performance of these models in a UK population and compare their performance to the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, derivates of which are currently used to calculate standardised mortality ratio (SMR) for these patients.

Table 1.

Components for calculation of the four cardiac arrest specific models and the thresholds defining the low, intermediate and high risk of poor outcome groups.

CAHP		MIRACLE₂		OHCA		TTM
Characteristic	Points	Characteristic	Points	Characteristic	Points	Characteristic	Points
Age	1.1 x (years − 10)	Unwitnessed arrest	1	Non-shockable rhythm	−13	Age	<40 = −1 40–44 = 0 45–49 = 1 50–54 = 2 55–59 = 3 60–64 = 4 65–69 = 5 70–74 = 6 75–59 = 7 80-84 = 8 ⩾85 = 9
Setting	Public = 0 Home = 24	Initial rhythm non-shockable	1	Collapse – BLS^a	6 x ln(mins)	Setting	Public = 0 Home = 2
Initial rhythm	Shockable = 0 Non-shockable = 27	Unreactive pupils	1	BLS – ROSC^b	9 x ln(mins)	Initial rhythm non-shockable	No = 0 Yes = 4
Duration: collapse – BLS	2.8 x minutes	Age	0–60 = 0 60–80 = 1 >80	Serum creatinine	−1434/ (µmol/L)	Collapse - BLS	0–4 = 0 5–9 = 1 10–14 = 2 ⩾15
Duration: BLS - ROSC	0.8 x minutes	Changing rhythm (any 2 of VF/PEA/Asystole)	1	Lactate	10 x ln(mmol/L)	BLS - ROSC	0–5 = 0 6–15 = 1 16–30 = 2 31–60 = 3 >60
Initial pH	585 – (77 x pH)	pH < 7.2	1			Adrenaline	No = 0Yes = 2
Total pre-ROSC adrenaline dose (mg)	0 mg = 01–2 mg = 27⩾3 mg = 43	Adrenaline given	1			No corneal/ pupillary reflexes	No = 0Yes = 3
						GCS Motor Score	>11 = 2
						pH	⩾7.35 = −17.2–7.34 = 07.05–7.19 = 16.9–7.04 = 2<6.9
						PaCO2	⩾4.5<4.5
Total	Sum of above	Total	Sum of above	Total	Sum of above	Total	Sum of above
Low risk	<150		0–2		⩽2		⩽10
Intermediate risk	150–200		3–4		2.1–32.4		11–16
High risk	>200		⩾5		⩾32.5		⩾16

CAHP: cardiac arrest hospital prognosis; OHCA: out of hospital cardiac arrest; TTM: targeted temperature management; BLS: basic life support; ROSC: return of spontaneous circulation; VF: ventricular fibrillation; PEA: pulseless electrical activity; GCS: Glasgow coma scale.

Lowest possible value 0.5.

Methods

We conducted a retrospective evaluation of patients referred for critical care admission from the emergency department or cardiac catheterisation laboratory following OHCA to the five critical care units within our local critical care network over a 43-month period from January 2018 to July 2021. Searches of the Scottish Intensive Care Audit Group (SICSAG) WardWatcher local clinical coding dataset in each hospital identified 414 non-pregnant adult patients who presented consecutively after resuscitation from non-traumatic OHCA during this period. The evaluation was discussed with the local regional ethics committee who deemed ethical approval not to be required for this service evaluation of routinely available information. Caldicott approval and prospective authorisation from the local quality improvement committee was received and data governance policies adhered to.

Clinical and laboratory data were obtained from each patient’s paper notes and electronic patient records to allow measurement of baseline demographics & calculation of CAHP, MIRACLE₂, OHCA and TTM scores. APACHE-II scores were available from the WardWatcher dataset. Neurological outcome was determined from the last entries by medical staff, nursing staff, occupational therapists, and physical therapists in the patient record at either the time of discharge from hospital, or from local hospital based neurological rehabilitation units in the cases where patients were transferred for ongoing treatment. Neurological outcome was defined by the Cerebral Performance Category (CPC) tool in the original papers, but modified Rankin scale (mRS) appears to have become the preferred tool in more recently published work involving the assessment of neurological recovery after OHCA. Both CPC and mRS neurological outcomes (Table 2) were calculated for each patient. CPC and mRS outcomes were assessed for all patients by a single author who was blinded to the scores generated by the risk prediction models. An unfavourable outcome was defined as CPC ⩾ 3 or mRS ⩾ 3. A score for each of the risk prediction models (Table 1) was calculated for each patient. High risk, intermediate risk, and low risk groups for unfavourable outcome were defined using the score cut offs from the original papers (Table 1).

Table 2.

Description of neurological outcome scales of cerebral performance category (CPC) and modified Rankin scale. CPC ⩾ 3 were defined as ‘unfavourable’ outcome.

CPC score	Modified Rankin Scale
1 Good cerebral performance: conscious, alert, able to work, might have mild neurologic or psychologic deficit2 Moderate cerebral disability: conscious, sufficient cerebral function for independent activities of daily life. Able to work in a sheltered environment.3 Severe cerebral disability: conscious, dependent on others for daily support because of their impaired brain function. Ranges from ambulatory state to severe dementia or paralysis.4 Coma or vegetative state: any degree of coma without the presence of all brain death criteria. Unawareness, even if appears awake (vegetative state) without interaction with environment; may have spontaneous eye opening and sleep/awake cycles. Cerebral unresponsiveness.5 Death/Brain death: apnoea, areflexia, EEG silence, etc.	0 No symptoms1 No significant disability, despite symptoms; able to perform all usual duties and activities.2 Slight disability; unable to perform all previous activities but able to look after own affairs without assistance.3 Moderate disability; requires some help, but able to walk without assistance.4 Moderately severe disability; unable to walk without assistance and unable to attend to own bodily needs without assistance.5 Severe disability; bedridden, incontinent and requires nursing care and attention.6 Death

CPC score

Modified Rankin Scale

1 Good cerebral performance: conscious, alert, able to work, might have mild neurologic or psychologic deficit2 Moderate cerebral disability: conscious, sufficient cerebral function for independent activities of daily life. Able to work in a sheltered environment.3 Severe cerebral disability: conscious, dependent on others for daily support because of their impaired brain function. Ranges from ambulatory state to severe dementia or paralysis.4 Coma or vegetative state: any degree of coma without the presence of all brain death criteria. Unawareness, even if appears awake (vegetative state) without interaction with environment; may have spontaneous eye opening and sleep/awake cycles. Cerebral unresponsiveness.5 Death/Brain death: apnoea, areflexia, EEG silence, etc.

0 No symptoms1 No significant disability, despite symptoms; able to perform all usual duties and activities.2 Slight disability; unable to perform all previous activities but able to look after own affairs without assistance.3 Moderate disability; requires some help, but able to walk without assistance.4 Moderately severe disability; unable to walk without assistance and unable to attend to own bodily needs without assistance.5 Severe disability; bedridden, incontinent and requires nursing care and attention.6 Death

Scores from the risk prediction models were not available to treating clinicians at any point during a patient’s care and clinicians followed standard practice according to the ESICM guidelines for neurological prognostication following OHCA.⁵ TTM and coronary angiography were implemented in accordance with established guidelines. Mechanical circulatory support became available in one ICU during the evaluation period, with two patients receiving this therapy.

Statistical analyses were performed using SPSS^® (IBM, United States) and Stata^® (StataCorp, United States). Not all patients had complete data for each model and to determine the effect of this missing data, difference between the groups with complete and missing data for each model was assessed using Mann-Whitney or Pearson Chi Squared tests (Supplemental Table 1).

Analysis of the scores performance in our cohort was conducted by assessing relationship between score and outcome, testing calibration, generating receiver operating characteristic (ROC) curves with an area under the curve (AUC) value to assess ability to discriminate for unfavourable outcome, and calculation of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), percentage correctly classified and false positive rate (FPR). We intended to test calibration by applying the coefficients and intercepts from the original development cohorts for each model to data from our cohort. However, this is not published for the CAHP, MIRACLE₂ and TTM models and the authors did not respond to requests for this information. The coefficients and intercepts therefore had to be calculated using our own cohort before calibration could be tested.

Results

Baseline characteristics of the cohort, described by proportions, medians and inter-quartile ranges for variables used in the analysis of the models and APACHE-II data (Supplemental Table 2) are in keeping with previously published cohorts of cardiac arrest data. Of the 414 patients included in the evaluation, 235 (56.7%) died in ICU, 118 patients (28.5%) recovered to a favourable neurological outcome and 61 (14.7%) left ICU alive but with a CPC score ⩾3 (an unfavourable outcome). Of those who left ICU alive but with a CPC score ⩾3, 32 patients (7.7% of the cohort) were discharged from hospital alive but with poor neurological recovery. Of the 235 patients who died in ICU, 82 (19.8% of the cohort) developed progressive multi-organ failure and did not survive to attempt formal neuro-prognostication.

Patients admitted following resuscitation from OHCA occupied a significant proportion of our ICU capacity with a median ICU length of stay for those with an unfavourable outcome of 3.1 [IQR 1.6–5.8] days and for those with a favourable outcome 5.3 [3.0–8.7] days. In both groups, length of ICU stay exceeds the recommended 72 h delay prior to attempting neuro-prognostication,⁵ despite 34.9% of those with unfavourable outcomes dying of progressive multi-organ dysfunction prior to 72 h. This length of stay could represent delays to neuro-prognostication caused by critical care complications such as ventilator acquired pneumonia. However, we did not specifically collect complication data and other reasons may include physiological preparation for brainstem testing, organ donation, delayed discharges or rehabilitation assessment prior to ICU discharge.

Of the 414 patients within the cohort, 347 patients (83.8%) had complete data for the CAHP model, 375 patients (90.6%) had complete data for the MIRACLE₂ model, 356 patients (86.0%) had complete data for the OHCA model, 385 patients (93.0%) had complete data for the TTM model and 380 (91.8%) had recorded APACHE-II scores. Three hundred three patients (73.2%) had complete data for all five models. Tests of difference between ‘complete data’ and ‘missing data’ groups for each model are presented in Supplemental Table 1. Significant difference for shockable versus non-shockable initial rhythm was present between the complete and missing data groups for the OHCA, CAHP and TTM models. There was no significant difference for the MIRACLE₂ model. The authors felt this was unlikely to significantly affect the overall analysis therefore the data were analysed without imputation and only cases where data was complete included for each model.

When each model was applied to our cohort a wide distribution of scores was produced. The relationship of the scores (or predicted risk of poor outcome) to observed outcomes is presented as the percentage of unfavourable CPC outcomes observed for each score value (Figure 1). The relationship for mRS outcomes is presented in Supplemental Figure 1.

Figure 1.

Relationship of observed scores against observed outcomes (plotted as the percentage of outcomes for each possible score across the model where the observed Cerebral Performance Category was ⩾3) with 95% confidence intervals for each risk prediction model.

Calibration plots for prediction of risk of CPC outcome ⩾3 for each model are presented in Figure 2. Best calibration was observed for the CAHP and MIRACLE₂ models. Calibration plots for mRS outcome are presented in Supplemental Figure 2.

Figure 2.

Calibration plots for risk of Cerebral Performance Category outcome ⩾ 3 for (a) Cardiac Arrest Hospital Prognosis (CAHP) score, (b) MIRACLE-2 score, (c) Out of Hospital Cardiac Arrest (OHCA) score, (d) Targeted Temperature Management (TTM) score and (e) Applied Physiology and Chronic Health Evaluation II (APACHE-II) score.

ROC curves and AUC values for discrimination of CPC outcome ⩾3 are presented in Figure 3. The CAHP and TTM models performed best with an AUC of 0.87 [95% CI 0.83–0.91]. MIRACLE₂ had an AUC of 0.81 [95% CI 0.76–0.86], OHCA an AUROC of 0.75 [95% CI 0.69–0.80] and APACHE-II an AUC of 0.74 [95% CI 0.68–0.80].

Figure 3.

Receiver operating characteristic (ROC) curves demonstrating discrimination performance for Cerebral Performance Category outcome ⩾ 3 and resulting area under the curve (AUC) values for the five risk prediction models (n = 308, patients with complete data for all models).

Performance of the scores in our cohort, and performance of the scores in the originally published development and validation cohorts is presented in Table 2. ROC curves and AUC values when discriminating for mRS ⩾ 3 are presented in Supplemental Figure 3.

The sensitivity, specificity, PPV, NPV, percentage correctly classified and FPR for the CAHP, MIRACLE₂, OHCA & TTM models for both the high-risk group and the high and intermediate risk combined group are presented in Table 3.

Table 3.

Performance of the four cardiac arrest specific models, presented as area under the receiver operating characteristic curve values when discriminating for an outcome of Cerebral Performance Category ⩾ 3. Performance is presented in our cohort, the original development cohort, and the external validation cohorts presented by the original models authors.

	Our cohort	Development cohort	Original external validation cohorts
CAHP	0.87 [95% CI 0.83–0.91]	0.93 [95% CI 0.91–0.95]	0.85 [95% CI 0.82–0.91]
MIRACLE₂	0.81 [95% CI 0.76–0.86]	0.90*	0.84*	0.91*
OHCA	0.75 [95% CI 0.69–0.8]	0.82 [95% CI 0.7–0.95]	0.88 [95% CI 0.82–0.94]
TTM	0.87 [95% CI 0.83–0.91]	0.84 [95% CI 0.84–0.85]	n/a

CAHP: cardiac arrest hospital prognosis model; OHCA: out of hospital cardiac arrest model; TTM: targeted temperature management model; 95% CI: 95% confidence interval.

95% confidence interval data not available.

Table 4.

Discrimination of unfavourable neurological outcome (Cerebral Performance Category ⩾ 3). For reader clarity: sensitivity represents the number of those who had an unfavourable outcome and had a score in that group, and PPV represents those with a score in the high risk group who had a unfavourable outcome.

Model	Discrimination of unfavourable neurological outcome
	High risk group alone				High and intermediate risk combined group
	CAHP	MIRACLE₂	OHCA	TTM	CAHP	MIRACLE₂	OHCA	TTM
Sensitivity (95% CI)	31.8% (26.0–37.7)	58.3% (52.4–64.2)	53.8% (47.6–60%)	27.5% (22.2–32.8)	75.5% (70.1–80.9)	86.0% (81.8–90.1)	97.6% (95.7−99.5)	73.6% (68.4–78.9)
Specificity (95% CI)	97.1% (93.8–100)	92.3% (87.2–97.4)	80.4% (72.8–87.9)	97.3% (94.3–100)	79.4% (71.6–87.3)	59.6% (50.2–69.0)	8.4% (3.2–13.7)	77.7% (70.0–85.4)
PPV (95% CI)	96.3% (92.2–100)	95.2% (91.9–98.4)	86.5% (81.1–91.8)	96.2% (91.9–100)	89.8% (85.7–93.9)	84.7% (80.5–89.0)	71.3% (66.5–76.1)	88.9% (84.8–93.0)
NPV (95% CI)	37.2% (31.4–43.0)	45.9% (39.2–52.7)	42.8% (35.9–49.6)	35.5% (30.2–40.9%)	57.4% (49.3–65.6%)	62.0% (52.5–71.5%)	60.0% (35.2–84.8)	54.7% (47.0–62.5)
Correctly classified (95% CI)	51.0% (45.7–56.3)	67.7% (63.0–72.5)	61.8% (56.8–66.8)	47.8% (42.8–52.8)	76.7% (72.2–81.1)	78.7% (74.5–82.8)	70.8% (66.1–75.5)	74.8% (70.5–79.1)
FPR	2.9%	7.7%	19.6%	2.7%	20.6%	40.4%	91.6%	22.3%

CAHP: Cardiac Arrest Hospital Prognosis; OHCA: Out of hospital Cardiac arrest; TTM: Targeted Temperature Management; 95% CI: 95% confidence interval; PPV: positive predictive value; NPV: negative predictive value; FPR: false positive rate.

The highest sensitivity for identifying an unfavourable outcome was 97.6% [95% CI 95.7–99.5] where the high and intermediate risk groups from the OHCA model were combined. The MIRACLE₂ score performed less well at 86.0% [95% CI 81.8–90.1], with the CAHP and TTM scores being even less sensitive. The specificity of the models to exclude a favourable neurological outcome was best where the high-risk group was used in the TTM (97.3% [95% CI 94.3–100]) and CAHP (97.1% [95% CI 93.8–100]) models. The PPV for prediction of poor outcome was best when the high-risk group was used in the CAHP (96.3% [95% CI 92.2–100]), TTM (96.2% [95% CI 91.9–100) and MIRACLE₂ (95.2% [95% CI 91.9–98.4]) models.

Discussion

Patients receiving post-cardiac arrest care remain a challenge to the critical care community. Previously published data reports significant haemodynamic dysfunction and/or cardiogenic shock occurring in >50% of those admitted to an ICU for supportive therapy post-OHCA.^14,15 In many cases this will be irreversible multi-organ failure from global ischaemia, but in a proportion of patients this will be cardiogenic from potentially reversible ongoing myocardial ischaemia or myocardial stunning,¹⁶ which should spontaneously improve with time. In both situations a period of more advanced haemodynamic support may be required to provide continued organ perfusion despite the failing circulation whilst myocardial recovery occurs.

Although in the absence of ST segment elevation percutaneous coronary intervention (PCI) in this population is no longer routinely recommended, where there is haemodynamic instability it should still be considered.^17,18 There is observational data that early angiography benefits those who go on to survive with a good neurological outcome¹⁹ and that this benefit may be higher in younger patients.²⁰ However, neurological outcome will not be known at an early time point following admission where angiography may be beneficial. There is also observational evidence, particularly for those who present with shockable rhythm, that mechanical circulatory support (MCS) via devices such as veno-arterial extracorporeal membrane oxygenation (VA-ECMO) and peripherally inserted ventricular assist devices (pVAD) may benefit patients, either as immediate therapy e-CPR (extra-corporeal cardiopulmonary resuscitation) prior to ROSC or as a bridge to recovery in the setting of myocardial stunning with haemodynamic instability following ROSC.^21
–23

The availability of these interventions is increasing but remains a limited resource which is necessarily centralised through the creation of PCI networks in NHS Scotland and Heart Attack Centres (HAC) in NHS England. Even within the highly specialised centres, the limited availability, resource consumption and risk of complication associated with these interventions is significant and there is unfortunately a requirement to direct resource towards those patients who are most likely to benefit.

The CAHP, MIRACLE₂, OHCA and TTM risk prediction models could all be helpful to assist in the triage of patients who are haemodynamically unstable following OHCA to centres which can employ these interventions. They all perform comparably in our cohort to the original development and validation cohorts and this evaluation supports validity of their use in the UK.

To maximise potential benefit to this population we would wish a risk prediction model to screen in as many patients who could benefit as possible, but to also avoid unnecessary exposure to complication and consumption of a scare resource, by identifying and excluding those at highest risk of a poor outcome. Sensitivity is therefore less important as a highly sensitive score would also inevitably screen out a high number of patients who would have a favourable outcome. More important would be that the score has an adequate specificity and PPV (Table 3) such that it does not mis-identify as unfavourable a large number of patients who would have ultimately displayed neurological recovery. The high-risk group for CAHP and TTM scores display the highest specificity and PPV for unfavourable outcome and should be the tools of choice. However, the MIRACLE₂ high risk group also displays high specificity and PPV and MIRACLE₂ is easier to calculate, with parameters which would be available immediately upon arrival in an Emergency Department, such that it could rapidly be used at the bedside of a deteriorating or arrested patient and could be used as part of an eligibility algorithm for e-CPR/ECLS. The specificity and PPV for unfavourable outcome both fall to unacceptable levels when the high and intermediate risk groups are combined.

The distribution of CPC outcomes across MIRACLE₂ risk groups are displayed graphically in Figure 4. Poor outcomes were distributed across the low, intermediate and high-risk groups, but almost all good outcomes occurred in the low and intermediate risk groups. Therefore, incorporation of MIRACLE₂ into a triage system should not deny advanced intervention to many patients who would have gone on to a favourable outcome and would assist in directing limited resources to those who are most likely to benefit. It is, however, extremely important to recognise that the FPR for all scores is such that they could not be used alone for neuro-prognostication outside of this setting.

Figure 4.

Distribution of observed cerebral performance category outcomes grouped by predicted risk of poor outcome, when calculated using the Miracle-2 model.

The scores may also have another role in benchmarking of outcomes. When attempting to improve outcomes for post-OHCA patients in critical care it is vital that we can accurately assess our observed versus expected performance. APACHE-II, which predicts survival across all causes of admissions to critical care, is necessarily not disease specific. Derivatives of this model are used by the Scottish Intensive Care Society Audit Group (SICSAG) and Intensive Care National Audit and Research Centre (ICNARC) to calculate standardised mortality ratio (SMR) for individual ICUs. APACHE-II performed significantly less well in our cohort than the cardiac arrest specific scores with an AUC of 0.74. This is in keeping with previous estimates for when APACHE-II is applied to patients resuscitated from OHCA of an AUC between 0.58 and 0.71.²⁴ This is unsurprising as many patients will achieve rapid resolution of physiological derangement, such that their APACHE-II score is low and predicted survival high but fail to display neurological recovery and die following withdrawal of support.

APACHE-II has limitations as a tool for benchmarking outcomes following OHCA. It may fail to identify if survival rates for this population fall below that which would be expected for a particular series of cases, potentially preventing timely improvement of unit processes and treatment pathways. It should also be recognised that units which admit a high proportion of patients post-OHCA may have an adverse impact on their SMR.

When selecting a benchmarking tool, ease of calculation is of less concern and the highest performing tool would be the logical choice. CAHP and TTM appear to perform equally when assessing sensitivity, specificity, PPV, NPV and AUC value. CAHP correctly classified marginally more patients, has been externally validated, and is slightly easier to calculate, so we would consider this to be the most appropriate score to use for benchmarking of critical care outcomes following cardiac arrest.

It is important to recognise that this evaluation has several limitations. It is retrospective, longer term follow up was not available and there were missing data (although this did not appear significant). It was not possible to obtain the coefficients and intercepts for testing calibration. Calculation of the scores relied upon accurate pre-hospital documentation of parameters such as time to initiation of basic life support and time to ROSC, but this pragmatic approach would be reflective of use of the score in clinical practice. Additionally, in person neurological assessment by the authors of this paper was not possible and we relied upon accurate documentation in the patient records of neuro-psychiatric and physical recovery by the wider multi-disciplinary team to determine a CPC or mRS outcome.

However, our cohort is the only published cohort which has simultaneously evaluated all the major OHCA specific risk models in a UK population. It is also the largest cohort to provide validation of any of the scores in a UK population. The cohort appears representative of patients successfully resuscitated from OHCA in the UK and is comparable to other published data, where patients who are successfully resuscitated are more likely to be male, of older age, and found most commonly in a shockable rhythm with a cardiac cause.^6,25
–29

Finally, we have used CPC outcomes in our analyses for consistency as these were used by the original papers. However, mRS has become the preferred tool for assessment of neurological outcome following OHCA and we replicated our analysis with similar performance of the scores (see Supplemental Files).

Conclusion

The CAHP, MIRACLE₂, OHCA and TTM risk prediction models all perform comparably in a UK population to the original development and validation cohorts. The risk prediction models may assist in triaging patients to appropriate specialist centres and in the allocation of resource intensive interventions to those who will benefit most. CAHP and TTM perform best and should be the tools of choice where time allows and the parameters are available. MIRACLE₂, which is easier to calculate but displays inferior performance, may be most feasible for use at the bedside of a patient who is arrested or in haemodynamic extremis.

Supplemental Material

sj-docx-1-inc-10.1177_17511437231214146 – Supplemental material for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Supplemental material, sj-docx-1-inc-10.1177_17511437231214146 for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort by John A Livesey, Nazir Lone, Emily Black, Richard Broome, Alastair Syme, Sean Keating, Laura Elliott, Cara McCahill, Gavin Simpson, Helen Grant, Fiona Auld, Sweyn Garrioch, Alasdair Hay and Thomas H Craven in Journal of the Intensive Care Society

Supplemental Material

sj-docx-2-inc-10.1177_17511437231214146 – Supplemental material for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Supplemental material, sj-docx-2-inc-10.1177_17511437231214146 for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort by John A Livesey, Nazir Lone, Emily Black, Richard Broome, Alastair Syme, Sean Keating, Laura Elliott, Cara McCahill, Gavin Simpson, Helen Grant, Fiona Auld, Sweyn Garrioch, Alasdair Hay and Thomas H Craven in Journal of the Intensive Care Society

Supplemental Material

sj-docx-3-inc-10.1177_17511437231214146 – Supplemental material for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Supplemental material, sj-docx-3-inc-10.1177_17511437231214146 for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort by John A Livesey, Nazir Lone, Emily Black, Richard Broome, Alastair Syme, Sean Keating, Laura Elliott, Cara McCahill, Gavin Simpson, Helen Grant, Fiona Auld, Sweyn Garrioch, Alasdair Hay and Thomas H Craven in Journal of the Intensive Care Society

Supplemental Material

sj-docx-4-inc-10.1177_17511437231214146 – Supplemental material for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Supplemental material, sj-docx-4-inc-10.1177_17511437231214146 for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort by John A Livesey, Nazir Lone, Emily Black, Richard Broome, Alastair Syme, Sean Keating, Laura Elliott, Cara McCahill, Gavin Simpson, Helen Grant, Fiona Auld, Sweyn Garrioch, Alasdair Hay and Thomas H Craven in Journal of the Intensive Care Society

Supplemental Material

sj-docx-5-inc-10.1177_17511437231214146 – Supplemental material for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort

Supplemental material, sj-docx-5-inc-10.1177_17511437231214146 for Neurological outcome following out of hospital cardiac arrest: Evaluation of performance of existing risk prediction models in a UK cohort by John A Livesey, Nazir Lone, Emily Black, Richard Broome, Alastair Syme, Sean Keating, Laura Elliott, Cara McCahill, Gavin Simpson, Helen Grant, Fiona Auld, Sweyn Garrioch, Alasdair Hay and Thomas H Craven in Journal of the Intensive Care Society

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

John A Livesey

Nazir Lone

Emily Black

Sean Keating

Thomas H Craven

Supplemental material

Supplemental material for this article is available online.

References

Perkins

Brace-McDonnell

; Group OP. The UK Out of Hospital Cardiac Arrest Outcome (OHCAO) project. BMJ Open 2015; 5: e008736.

Coute

Nathanson

Panchal

, et al.; for the CARES Surveillance Group. Disability-adjusted life years following adult out-of-hospital cardiac arrest in the United States. Circ Cardiovasc Qual Outcomes 2019; 12: e004677.

Kiguchi

Okubo

Nishiyama

, et al. Out-of-hospital cardiac arrest across the world: First report from the International Liaison Committee on Resuscitation (ILCOR). Resuscitation 2020; 152: 39–49.

Benjamin

Muntner

Alonso

, et al. Heart disease and stroke statistics-2019 update: a report from the American Heart Association. Circulation 2019; 139: e56–e528.

Nolan

Cariou

Post-resuscitation care: ERC–ESICM guidelines 2015. Intensive Care Med 2015; 41: 2204–2206.

Nielsen

Wetterslev

Cronberg

, et al. Targeted temperature management at 33°C versus 36°C after cardiac arrest. N Engl J Med 2013; 369: 2197–2206.

Kakavas

Mongardon

Cariou

, et al. Early-onset pneumonia after out-of-hospital cardiac arrest. J Infect 2015; 70: 553–562.

François

Cariou

Clere-Jehl

, et al.; CRICS-TRIGGERSEP Network and the ANTHARTIC Study Group. Prevention of early ventilator-associated pneumonia after cardiac arrest. N Engl J Med 2019; 381: 1831–1842.

Roberts

Kilgannon

Chansky

, et al. Multiple organ dysfunction after return of spontaneous circulation in postcardiac arrest syndrome. Crit Care Med 2013; 41: 1492–1501.

10.

Maupain

Bougouin

Lamhaut

, et al. The CAHP (Cardiac Arrest Hospital Prognosis) score: a tool for risk stratification after out-of-hospital cardiac arrest. Eur Heart J 2016; 37: 3222–3228.

11.

Pareek

Kordis

Beckley-Hoelscher

, et al. A practical risk score for early prediction of neurological outcome after out-of-hospital cardiac arrest: MIRACLE2. Eur Heart J 2020; 41: 4508–4517.

12.

Adrie

Cariou

Mourvillier

, et al. Predicting survival with good neurological recovery at hospital admission after successful resuscitation of out-of-hospital cardiac arrest: the OHCA score. Eur Heart J 2006; 27: 2840–2845.

13.

Martinell

Nielsen

Herlitz

, et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Crit Care 2017; 21: 96.

14.

Lemiale

Dumas

Mongardon

, et al. Intensive care unit mortality after cardiac arrest: the relative contribution of shock and brain injury in a large cohort. Intensive Care Med 2013; 39: 1972–1980.

15.

Dumas

Bougouin

Geri

, et al. Emergency percutaneous coronary intervention in post-cardiac arrest patients without ST-segment elevation pattern: Insights from the PROCAT II Registry. JACC Cardiovasc Interv 2016; 9: 1011–1018.

16.

Jozwiak

Bougouin

Geri

, et al. Post-resuscitation shock: recent advances in pathophysiology and treatment. Ann Intensive Care 2020; 10: 170.

17.

Freund

van Royen

Kern

, et al.; TOMAHAWK, PEARL, and COACT investigators. Early coronary angiography in patients after out-of-hospital cardiac arrest without ST-segment elevation: meta-analysis of randomized controlled trials. Catheter Cardiovasc Interv 2022; 100: 330–337.

18.

Desch

Freund

Akin

, et al. Angiography after out-of-hospital cardiac arrest without ST-segment elevation. N Engl J Med 2021; 385: 2544–2553.

19.

Bougouin

Dumas

Karam

, et al.; Sudden Death Expertise Center. Should we perform an immediate coronary angiogram in all patients after cardiac arrest?: Insights from a large French registry. JACC Cardiovasc Interv 2018; 11: 249–256.

20.

Waldmann

Karam

Rischard

, et al.; On Behalf Paris-SDEC investigators. Low rates of immediate coronary angiography among young adults resuscitated from sudden cardiac arrest. Resuscitation 2020; 147: 34–42.

21.

Belohlavek

Rob

Smalcova

, et al. Effect of intra-arrest transport and extracorporeal cardiopulmonary resuscitation on functional neurologic outcome in refractory out-of-hospital cardiac arrest: a randomized clinical trial. JAMA. 2022; 327(8): 737–747.

22.

Yannopoulos

Bartos

Raveendran

, et al. Advanced reperfusion strategies for patients with out-of-hospital cardiac arrest and refractory ventricular fibrillation (ARREST): a phase 2, single centre, open-label, randomised controlled trial. Lancet 2020; 396: 1807–1816.

23.

Basir

Kapur

Patel

, et al.; on behalf of the National Cardiogenic Shock Initiative Investigators. Improved outcomes associated with the use of shock protocols: updates from the National Cardiogenic Shock Initiative. Catheter Cardiovasc Interv 2019; 93: 1173–1183.

24.

Donnino

Salciccioli

Dejam

, et al. APACHE II scoring to predict outcome in post-cardiac arrest. Resuscitation 2013; 84: 651–656.

25.

Hawkes

Booth

, et al. Epidemiology and outcomes from out-of-hospital cardiac arrests in England. Resuscitation 2017; 110: 133–140.

26.

Government DoHaSC-S. Scottish Out-of-Hospital Cardiac Arrest data linkage project: 2018-2019 results 2020, https://www.gov.scot/publications/scottish-out-hospital-cardiac-arrest-data-linkage-project-2018-19-results/pages/1/ (accessed 2 October 2023).

27.

Dankiewicz

Cronberg

Lilja

, et al. Hypothermia versus normothermia after out-of-hospital cardiac arrest. N Engl J Med 2021; 384: 2283–2294.

28.

Roger

Lloyd-Jones

, et al. Executive summary: heart disease and stroke statistics–2012 update: a report from the American Heart Association. Circulation 2012; 125: 188–197.

29.

Neumar

Nolan

Adrie

, et al. Post-cardiac arrest syndrome: epidemiology, pathophysiology, treatment, and prognostication. A consensus statement from the International Liaison Committee on Resuscitation (American Heart Association, Australian and New Zealand Council on Resuscitation, European Resuscitation Council, Heart and Stroke Foundation of Canada, InterAmerican Heart Foundation, Resuscitation Council of Asia, and the Resuscitation Council of Southern Africa); the American Heart Association Emergency Cardiovascular Care Committee; the Council on Cardiovascular Surgery and Anesthesia; the Council on Cardiopulmonary, Perioperative, and Critical Care; the Council on Clinical Cardiology; and the Stroke Council. Circulation 2008; 118: 2452–2483.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB

0.10 MB

0.09 MB

0.02 MB