Abstract
Introduction:
Accurate prognostic prediction of out-of-hospital cardiac arrest is challenging but important for the emergency team and patient’s family members. A number of prognostic prediction models specifically designed for out-of-hospital cardiac arrest are developed and validated worldwide.
Objective:
This narrative review provides an overview of the prognostic prediction models out-of-hospital cardiac arrest patients for use in the emergency department.
Discussion:
Out-of-hospital cardiac arrest prognostic prediction models are potentially useful in clinical, administrative and research settings. Development and validation of such models require prehospital and hospital predictor and outcome variables which are best in the standardised Utstein Style. Logistic regression analysis is traditionally employed for model development but machine learning is emerging as the new tool. Examples of such models available for use in the emergency department include ROSC After Cardiac Arrest, CaRdiac Arrest Survival Score, Utstein-Based Return of Spontaneous Circulation, Out-of-Hospital Cardiac Arrest, Cardiac Arrest Hospital Prognosis and Cardiac Arrest Survival Score. The usefulness of these models awaits future studies.
Introduction
Out-of-hospital cardiac arrest (OHCA) is a major time-critical medical emergency challenge. In Hong Kong, over 5000 OHCA cases with prehospital resuscitation are managed by the emergency department every year. 1
Accurate prognostic prediction is fundamental to optimal medical care, but clinical outcomes of OHCA are notoriously difficult to predict even by experienced emergency physicians. Physicians often have to rely on intuition to predict the prognosis of patients when a patient with cardiac arrest and ongoing resuscitation arrives at the emergency department. There is no single variable that is definitive for either immediate or late outcome of OHCA. 2
An informative approach would be using evidence-based prognostic prediction models which simultaneously consider a number of variables to give an estimate of a patient’s absolute risk of an event with meaningful predictability. Such models are developed using sophisticated statistical methods to identify a set of predictor variables and estimate the risk of a particular outcome in a selected group of patients. Example of prognostic prediction models commonly used in emergency and critical care medicine include Acute Physiology And Chronic Health Evaluation II (APACHE II) which estimates mortality based on age and sets of physiological parameters measured during the first 24 h of intensive care unit admission 3 and Trauma and Injury Severity Score (TRISS) which predicts survival of trauma patient. 4
Prognostic prediction models for OHCA can provide an early and objective prognostic estimation for patients. This narrative review aims to give an overview of the prognostic prediction models for OHCA for use in the emergency department. Its potential implementation in Hong Kong will be explored in the final section.
Method
The first author searched PubMed for related articles using the following search strategy: “out-of-hospital cardiac arrest” AND prediction AND (score or model or rule). The search was conducted from database inception to 5 March 2020. The literature search was restricted to studies published in English. PubMed yielded 372 results. The author trained for critical appraisal of literature reviewed the articles and decided which studies to include to address the review question, with a focus on emergency medicine relevant articles.
Values of predicting OHCA outcomes with modelling
There are several reasons why a model predicting the outcomes of OHCA is useful. First and foremost, the outcome prediction model is not the same as termination of resuscitation (TOR) rule. TOR rules are designed to apply in clinical settings, which if all criteria of such rule are matched in an OHCA, withholding further resuscitation attempts, may be considered by the rescue team, though ethical issues around TOR rules remain controversial. 5 However, an outcome prediction model is capable of giving an absolute probability of outcome. A low survival outcome predicted by the prediction model does not lead to a decision of withholding resuscitation, as resuscitation attempts should be performed despite a minimal probability of survival.
Although these outcome prediction models cannot directly answer the question of whether resuscitation should be continued or not, they can give a realistic expectation about the likelihood of achieving a reasonable outcome during resuscitation. This piece of information is important for rescue teams, and even more so for family members. The awareness and understanding of what predictive factors play a significant role in a favourable outcome is relevant for acceptance of poor outcomes by relatives and partners. 6
There are other potential roles of OHCA outcome prediction models in medical research and administrative medicine. The probabilities of outcomes generated by these models for a particular group of patients can be treated as a predicted rate of outcome and as a benchmark score. Research can use this benchmark score to evaluate different post-resuscitation management strategies, or after implementation of new resuscitation protocols. Territorial outcomes after OHCA often differ because of different patient characteristics. By comparing the benchmark and the observed rate of outcome, OHCA outcome prediction models can be used as an objective measurement for quality assessment and comparative audit in risk-adjusted analysis.7,8
Developing an OHCA outcome prediction model
There are several points to consider before deriving an outcome prediction model for OHCA. Cardiac arrest is describing essentially a state of cessation of cardiac activity. It can occur at any age from newborn to elderlies. It has a highly variable underlying internal (e.g. cardiac, respiratory, stroke) and external (e.g. trauma, drowning, asphyxia, burn, intoxication) causes, and delineation of the definite cause can be difficult during early resuscitation. 9 If a model excludes patients with particular underlying causes which cannot be rapidly differentiated during early phase of management, the model will lose its practicability.
Paediatric and neonatal OHCA are different from adult cases, as they are more likely due to respiratory causes, and they have a better survival and neurological outcome in general when compared with adult cases. 10 To reflect on these differences, the International Liaison Committee on Resuscitation recommends a different resuscitation approach to paediatric cardiac arrest, and many national resuscitation guideline organisations follow. 11 Most of the resuscitation research separates adult and paediatric cases of cardiac arrest. A prediction model which includes paediatric patients may affect its use in clinical audit and research.
Developers should also consider at which time point from recognition of cardiac arrest to ward admission can the prognostic prediction model be implemented as it determines what predictor variables should be included for the model’s derivation.
Dataset
Most prognostic prediction models for OHCA are built on centre-based or territory-wide resuscitation registries as they contain a large set of convenient OHCA data for derivation. Recommended guidelines with clear data definition for uniform OHCA data reporting have been developed as the Utstein Style. 12 Utstein Style data reporting contains key prognostic variables associated with survival or death following cardiac arrest. A prognostic prediction model based on Utstein Style variables is preferable for in-line adaptation.
The accuracy and comprehensiveness of the resuscitation registry used for analysis are important for developing a good model. Early prediction models with time-to-treatment variables perform poorly due to inaccurate recall or recording during highly stressful events. 13 Automatic machine recording and data entry can prevent human errors of time variables. A comprehensive registry can provide multiple possible predictor variables for evaluation and analysis during model development. Missing data and lost cases are common issues in many registries; their number should be limited to ensure the registries’ integrity and reliability.
Predictor variables
Potential prognostic predictor variables of OHCA can be classified into patient factors and process factors. Patient factors include demographic variables and characteristics of cardiac arrest. Process factors include interventions delivered by paramedics and rescue teams, time to interventions and initial investigation findings.14–16 These predictor variables could be based on review of available publications and guidelines, reports from resuscitation registries and quality analysis. As these variables have different degrees of positive and negative correlations with survival outcomes, a comprehensive prediction model of OHCA prognosis should take into account of multiple independent factors (Table 1). 7
Predictor variables of OHCA prognostic prediction models.
RACA: ROSC after cardiac arrest; CRASS: CaRdiac Arrest Survival Score; CAHP: cardiac arrest hospital prognosis; CASS: Cardiac Arrest Survival Score; CPR: cardiopulmonary resuscitation.
Outcome variables
Utstein guideline 2015 describes four levels of core outcomes of OHCA measurable for surveillance: any return of spontaneous circulation (ROSC), survived event (ROSC sustained until transfer of care at the receiving hospital), survival to discharge (STD) or 30-day survival, and survival with favourable neurological outcome. 17 Neurological outcome may be reported using the Cerebral Performance Category (CPC) or modified Rankin Scale (mRS). Utstein guideline defines favourable neurological outcome as a CPC 1 or 2, or mRS 0 to 3.
These outcome indexes vary greatly across different localities due to variances in resuscitation practices and protocols. 18 Meta-analysis on global OHCA outcome studies showed significant difference in the core outcome rates across continents; for example, Oceania have the highest STD rate (16.2%), followed by Europe (11.7%), North America (7.7%), and Asia (4.5%). 19 The variations in outcome indexes across different locations are the reason why external validation with analysis in calibration and model adjustment is necessary before employing a model developed in other localities.
Statistical approach
Logistic regression analysis is the traditional statistical method to derive a clinical prediction system. Univariate analysis could be used to identify independent significant prognostic factors. Multivariate analysis would combine the independent factors weighted by coefficients to provide an algorithm for the probability of outcomes. New approaches such as machine learning are emerging for use and will be discussed in later sections.
Validation, impact analysis and implementation
Two properties of prognostic prediction model are typically analysed to validate its performance: discrimination and calibration. Discrimination refers to how well the model differentiates those at higher risk of having an event from those at lower risk. As all Utstein core survival outcomes of OHCA are binary variables, discrimination of prediction model can be characterised using the receiver operating characteristic (ROC) curve. Calibration or goodness to fit refers to the accuracy of absolute risk estimates. Hosmer–Lemeshow test is commonly employed to determine whether chance can explain the difference between the predicted and the observed event rate.
Impact analysis can be performed before widespread implementation to measure the usefulness of the prediction tool. A prediction model has to be constantly updated in accordance with changes in resuscitation protocol, new use of resuscitation devices, and advancement in intensive care.
Examples of validated OHCA prognostic prediction models with predictors available at the emergency department
Several prognostic prediction models designed for OHCA specifically using data available in emergency departments have been developed worldwide over the past two decades. They can be categorised into those which are designed for all resuscitated OHCA cases with or without ROSC and for all resuscitated OHCA with ROSC only.
For all resuscitated OHCA with or without ROSC
ROSC after cardiac arrest
The ROSC after cardiac arrest (RACA) 7 score was developed to predict ROSC for all resuscitated OHCA patients. Eight predictive independent categorical variables were selected in a multivariate analysis approach considering available literature: gender, age (<80 years), cardiac aetiology, non-witnessed arrest, location at home and workplace, ventricular fibrillation as the initial rhythm, and no bystander cardiopulmonary resuscitation (CPR). Conditions associated with a negative or positive impact on ROSC were given regression coefficients, respectively, and included in the RACA score equation which generates the probability of ROSC of an individual OHCA patient.
The RACA score was based on 5471 OHCA patients suffering from OHCA in 1998–2008 within the German Resuscitation Registry. Internal validation showed a good discrimination value (AUC = 0.731). The score has been externally validated in other European regions.20–22 An Asian study has found that the original RACA score has a good discrimination performance in Asian patients, but a poor calibration with the constant regression coefficient given (Table 2). 23
Comparison of the examples of OHCA prognostic predictive models on outcome variables and discrimination performance.
RACA: ROSC after cardiac arrest; CRASS: CaRdiac Arrest Survival Score; CAHP: cardiac arrest hospital prognosis; CASS: Cardiac Arrest Survival Score; UB-ROSC: Utstein-based return of spontaneous circulation; OHCA: out-of-hospital cardiac arrest; AUC: area under the curve.
CaRdiac Arrest Survival Score
CaRdiac Arrest Survival Score (CRASS) 8 was developed to predict survival with good neurological outcome after OHCA who had ROSC or ongoing CPR at hospital admission. It was derived from the German Resuscitation Registry, the same registry used to derive the RACA score, with all OHCA patients in 2010–2016 who had ROSC or ongoing CPR at hospital admission. It uses binary logistic regression analysis to identify 12 groups of categorical independent variables which carry positive or negative impact on the primary outcome. Definition of good neurological outcome is in-line with that of the Utstein guideline. The CRASS equation is similar to that of the RACA equation, generating a probability of the primary outcome.
Internal validation found that CRASS has a good discrimination performance (AUC = 0.88), yet it has not been externally validated. Some aspects of CRASS may affect its generalisability, such as the use of prehospital amiodarone as one of the predictor variables, and the high proportion of patients with good neurological function at discharge (23.3%) in its dataset. Interestingly, CRASS identifies use of mechanical CPR devices as a negative predictive factor of survival outcome.
UB-ROSC
The UB-ROSC 6 score aims to predict the probability of survival to hospital admission of any OHCA victim by using only Utstein Style–based variables. Age, aetiology, arrest location, witnessed arrest, bystander CPR, response time and presenting rhythm are all identified as the independent predictive variables in its multivariable logistic model. Each categorical variable carries a positive or negative component score for summing up to the UB-ROSC score of each OHCA victim. The UB-ROSC score is translated to a probability of survival to admission using a nomogram.
The score was developed from all OHCA cases from 2015 to 2017 in two provinces in Italy and Switzerland. Internal validation showed a discrimination value of 0.82, and external validation with a cohort from two other provinces in Italy and Switzerland showed a discrimination value of 0.77.
For OHCA with ROSC/sustained ROSC only
OHCA
The OHCA 13 score was developed to predict poor neurological function in patients suffering from OHCA with sustained ROSC. Derivation data were collected from a French centre in 1999–2003. Logistic regression analysis was used to identify the five significant clinical and biochemical parameters to develop into an equation to generate the probability of primary outcome. One study showed that OHCA score is comparable with APACHE III on outcome prediction. 24
There are several major limitations of the OHCA score. First, the development cohort is small with only 130 patients aged from 47 to 69 years. Second, the analysis does not include most demographic and prehospital parameters that are known to be associated with outcome. Third, arterial lactate is neither routinely taken nor immediately available, limiting the scope of the score’s use.
Cardiac Arrest Hospital Prognosis
The Cardiac Arrest Hospital Prognosis (CAHP) 25 score was developed to predict poor neurological outcomes (CPC 3–5) after ROSC in OHCA. Independent prognostic factors were identified using logistic regression analysis and thresholds defined to stratify low- (39% of unfavourable outcome), moderate- (81% of unfavourable outcome), and high- (100% of unfavourable outcome) risk groups.
The CAHP score was developed in Paris after analysing 859 OHCA patients admitted into ICU in 2011–2012. The major limitation of the CAHP score is that it excluded a lot of cases, particularly ongoing CPR and arrests caused by asphyxia. Therefore, it is not necessarily applicable to the OHCA population in general. The CAHP score has been externally validated in Taiwan using a single-centre cohort with good discriminatory performance (AUC = 0.93). 26
Cardiac Arrest Survival Score
Cardiac Arrest Survival Score (CASS) 27 was created as an in-hospital mortality prediction model for patients experiencing OHCA who achieved sustained ROSC, relying only on information available at first medical contact. It was developed with the resuscitation database in Houston, United States, deriving from a cohort of 2685 OHCA patients in 2007–2015. Five binary variables are included in its final model: age greater than 75 years, unwitnessed arrest, arrest at home, absence of bystander CPR, and non-shockable initial rhythm. The CASS score does not include any parameters related to prehospital resuscitation and intervention.
Machine learning: new tool for prognosis prediction
In recent years, machine learning has shown high performance in several medical domains including radiological image analysis, medical diagnosis and clinical outcome prediction.28–30 Machine learning thrives from large dataset; therefore, the establishment of new territory-wide OHCA registries set an important stage for these powerful algorithms. 31 Examples of supervised learning methods suitable for OHCA outcome prediction include decision trees, k-nearest neighbours, Naive Bayes classifier, artificial neural networks, Support Vector Machine and random forests. 32 Among these tools, artificial neural networks, or deep learning, are the most studied in modelling of OHCA outcomes.
A deep-learning-based OHCA prognostic prediction model has been built in Chicago to predict the neurological outcome of patients with witnessed cardiac arrest events. 33 Another deep-learning-derived model was developed in South Korea used to predict good neurological recovery and STD for patients who experienced ROSC after OHCA. 34 The trade-off of using deep-learning-based algorithms is that they are known as black box and the approach by which it decides on the outcome is not interpretable.
Using OHCA prognostic prediction model in Hong Kong
Prognostic prediction models of OHCA are relatively new clinical prediction tools that await impact analysis. There are several factors to consider before implementing an externally derived OHCA prognostic prediction model to Hong Kong. Hong Kong is among the top for countries and regions with the highest life expectancy; 16% of the population was aged 65 years or above. Elderlies living in residential care homes make up a relatively high proportion of all OHCA cases – about 40%. 35 Bystander CPR rate is 28.8% and bystander defibrillation rate is less than 1.3%; both are much lower than the localities where the models are developed. 1 The median response time by emergency medical services to cardiac arrest call is as short as 9 min. Paramedics do not apply prehospital TOR rules except in a few circumstances, such as injuries incompatible with life or obvious post-mortem changes. 36
These major differences in patient demographics and prehospital resuscitation protocols make it necessary to either validate and calibrate any models developed in other localities before applying it to Hong Kong, or develop a new outcome prediction model for OHCA locally. Either way, a good quality collection of prehospital and hospital OHCA data is the cornerstone of implementing a successful predictive model. 37 The developed model can be used as a benchmark tool for surveillance, audit, and evaluation of the effectiveness of survival enhancement measures. 31 The impact of such tool should be evaluated in future studies.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
Availability of data and materials
No primary data are collected for this clinical review.
Informed consent
No informed consent is obtained as no patient data are included in this clinical review.
Ethical approval
This clinical review does not require ethical approval as it is not a human or animal research.
Human rights
This case report does not require human right permissions
