Abstract
Introduction
The Stockholm Stroke Triage System (SSTS) is a prehospital triage system for detection of patients eligible for endovascular thrombectomy (EVT). Assessment of hemiparesis combined with ambulance-hospital teleconsultation is used to route patients directly to the thrombectomy centre. Some patients are not identified and require secondary transport for EVT (undertriage) while others taken to the thrombectomy centre do not undergo EVT (overtriage). The aims of this study were to characterize mistriaged patients, model for and evaluate alternative triage algorithms.
Patients and methods
Patients with suspected stroke transported by priority 1 ground ambulance between October 2017 and October 2018 (n = 2905) were included. Three triage algorithms were modelled using prehospital data. Decision curve analysis was performed to calculate net benefit (correctly routing patients for EVT without increasing mistriage) of alternative models vs SSTS.
Results
Undertriage for EVT occurred in n = 35/2582 (1.4%) and overtriage in n = 239/323 (74.0%). Compared to correct thrombectomy triages, undertriaged patients were younger and had lower median NIHSS (10 vs 18), despite 62.9% with an M1 occlusion. In overtriaged patients, 77.0% had a stroke diagnosis (29.7% haemorrhagic). Hemiparesis and FAST items face and speech were included in all models. Decision curve analysis showed highest net benefit for SSTS for EVT, but lower for large artery occlusion (LAO) stroke.
Discussion
Undertriaged patients had lower NIHSS, likely due to better compensated proximal occlusions. SSTS was superior to other models for identifying EVT candidates, but lacked information allowing comparison to other prehospital scales.
Conclusion
Using prehospital data, alternative models did not outperform the SSTS in finding EVT candidates.
Introduction
Endovascular thrombectomy (EVT) is an effective treatment of acute ischaemic stroke caused by large artery occlusion (LAO), where the use of intravenous thrombolysis (IVT) alone may be insufficient for recanalization. 1 Observational studies have indicated better outcomes in patients transported directly to a thrombectomy capable stroke centre (TSC), compared to those undergoing secondary transfer. 2–5
Prehospital algorithms have been developed for identification of patients with either LAO stroke or EVT eligibility. 3,6–8 While the end goal is the same, optimal prehospital routing, both triage endpoints have their respective benefits and drawbacks. Not all patients with LAO stroke are eligible for EVT due to contraindications, while an initially well-functioning screening tool for EVT eligibility may fall short as guidelines and indications are updated with new evidence.
The Stockholm Stroke Triage System (SSTS) is a prehospital triage algorithm designed for detection of LAO stroke patients eligible for EVT, implemented in 2017. 9,10 Assessment of hemiparesis is performed by an ambulance nurse, followed by ambulance to hospital teleconsultation for stroke mimic screening and EVT eligibility. Accepted patients bypass the nearest primary stroke centre (PSC) for direct transport to the TSC. At the planning stage of the SSTS, validation of prehospital triage scales was still ongoing, and hemiparesis was chosen due to its association with LAO stroke as well as outcomes. 11–14 Additionally, the simplicity of the test was important for implementation and training by prehospital services. The system has a sensitivity of 0.71 and specificity of 0.91 for EVT initiation, with corresponding values for LAO stroke at 0.42 and 0.93. 9 In comparison to the period before SSTS implementation, when patients were transported to the nearest stroke centre regardless of symptoms, onset to puncture time for EVT was lowered by 69 minutes, while the proportion of patients with 90 day modified Rankin Scale (mRS) score of 0–1 increased from 23.7% to 34.2%. 9,15
The aims of this study were (1) to characterize incorrectly triaged patients based on clinical parameters available in the prehospital setting, as well as occlusion data and final diagnosis and (2) to establish whether alternative triage algorithms, with higher predictive performance for EVT and LAO stroke could be designed with available prehospital data.
Patients and methods
This is a planned sub-analysis of a prospective cohort study that included patients transported by priority 1 ambulance for suspected stroke in the Stockholm Region during the first year after SSTS implementation, October 2017 to October 2018. Patients with in-hospital stroke, those transported by helicopter or private means or taken to hospital with a non-stroke suspicion were excluded. The SSTS algorithm consists of three steps as performed by an ambulance nurse: (1) suspicion of acute ischaemic stroke, using the Swedish modified FAST test (allowing for arm or leg weakness),
16
or other clinical reason, (2) scoring of hemiparesis using the NIHSS subitems 5 and 6 (arm and leg), with ≥ 2 points in both ipsilateral extremities (A2L2-test), prompting (3) teleconsultation with a stroke consultant at the TSC. The teleconsultation includes a stroke mimic screen, symptom assessment and identification of EVT contraindications (onset > 24 h, pre-stroke life expectancy < 3 months and mRS of 4–5). Patients with hemiparesis (positive A2L2 test) are routed directly to the TSC or the nearest stroke centre after teleconsultation. A negative A2L2 test prompts pre-notification of the nearest stroke centre.
9,17
In the Stockholm Region, acute stroke care is provided by one TSC and six PSCs for 2.4 million inhabitants across 6519 km2 (please see Figure 1, online supplement, for map). Figure legend. Flow chart of included patients according to triage status for EVT treatment. TSC: Thrombectomy capable stroke centre, PSC: Primary stroke centre.
Definitions Comprehensive Stroke Centre (CSC) used previously has been changed to Thrombectomy Capable Stroke Centre (TSC). This was done to facilitate comparisons with other regions, as not all centres performing thrombectomy are comprehensive, conforming to Joint Commission terminology.
18
LAO stroke was defined as acute ischaemic stroke with CT angiography evidence of occlusion or subocclusion in arteries accessible by stent retriever EVT in routine practice at our thrombectomy centre: intracranial internal carotid artery, middle cerebral artery (M1 or M2 segments), anterior cerebral artery (A1 or A2 segments), posterior cerebral artery (P1 segment), basilar artery or intracranial vertebral artery. A decision to abstain from CTA could be made due to contraindications to intravenous contrast or EVT. Patients with acute ischaemic stroke and unknown occlusion status were classified as non-LAO ischaemic stroke in the analyses. Two triage endpoints were employed: (1) Arterial puncture with the intention of thrombectomy treatment. (2) LAO stroke diagnosis. Triage category definitions: (1) Overtriage (EVT): patient routed directly to TSC but not treated with EVT. (2) Overtriage (LAO stroke): patient routed directly to TSC but not diagnosed with LAO stroke. (3) Undertriage (EVT): patient routed to PSC undergoing secondary transport to TSC for EVT. (4) Undertriage (LAO stroke): patient routed to PSC and diagnosed with LAO stroke.
Data collection
Data was collected on patient demographics (age, sex), modified FAST-test subitems (facial weakness, arm/leg weakness and speech problems), known onset time, wake up stroke, onset-to-hospital time, as well as NIHSS subitems (evaluated at the first hospital) and arterial occlusion data. Onset to ambulance arrival was approximated by subtracting the median time (27 minutes) from ambulance arrival on scene to hospital arrival in the Stockholm Region. 17 Final diagnosis was categorized as LAO stroke, non-LAO ischaemic stroke, intracerebral haemorrhage and stroke mimic.
Statistical analysis
A univariate comparison was performed between four groups of patients: overtriaged, undertriaged, correctly triaged to TSC and correctly triaged to PSC, for EVT treatment and LAO stroke. Modelling was performed using logistic regression, with three strategies for variable inclusion; (1) clinically relevant variables as decided by the researchers, (2) stepwise regression using the Akaike Information Criterion (AIC) starting with the A2L2-test and iteratively adding variables that lowered the AIC criterion for the model and (3) a model including all variables available in the prehospital setting. Forward stepwise regression using the Akaike Information Criterion (AIC) started with only the A2L2-test as a variable, and then iteratively, one at a time, added the variable that improved AIC the most. Modified FAST-test subitems arm and leg were not considered due to high correlation with the A2L2-test. Only patients with complete prehospital data were included in the modelling. Sensitivity, specificity and area under the curve (AUC) were calculated for all models, as well as the A2L2-test alone and the full SSTS algorithm (A2L2-test + teleconsultation). A sensitivity analysis was performed, excluding patients with unknown occlusion status in the modelling for LAO stroke predictive performance. Decision curve analysis (DCA) was chosen as the main method for model comparison. DCA, introduced by Vickers et al, 19 is used to evaluate the net benefit of prediction tools and diagnostic tests across a range of patient/clinician preferences for accepting risk of undertreatment (undertriage) and overtreatment (overtriage). For the SSTS, benefit is the number of patients treated with EVT transported directly to the TSC (true positive rate). Net benefit is calculated by subtracting this with the false positive rate (overtriage), multiplied by the high risk threshold. The high risk threshold is a weight applied to the false positive rate and can be expressed as the answer to the question: “How many overtriaged patients are acceptable in order to correctly triage one patient in need of EVT?”. In the graphical output of the DCA, net benefit is plotted on the Y-axis, against a range of risk thresholds on the X-axis. Models are compared to strategies of taking all, or none of the patients to the TSC. 20 Statistical analyses were performed in Stata 15.1 (College Station, TX: StataCorp LLC) and R 4.1.0 (https://www.R-project.org/). Results are reported according to STARD. 21
Ethics and informed consent
Ethical approval was obtained from the Stockholm Regional Research Ethics Committee (approval nr 2017/374). The need for active consent was waived. All patients received written information regarding data collection from electronic health records and had the possibility to decline participation and have their personal data expunged from the study database.
Results
Of 2909 eligible patients, four declined participation, leaving 2905 in the study cohort. Among 2582 patients triaged to PSC, secondary transport for EVT occurred in 1.4% (n = 35) while 7.2% (n = 185) were diagnosed with LAO stroke (undertriage). Conversely, among 323 patients triaged to TSC, 74% (n = 239) were not treated with EVT and 59% (n = 192) were not diagnosed with LAO stroke (overtriage). Please see Figure 1 for a flowchart of triage status for EVT and the Supplementary material for LAO stroke.
Univariate comparison of characteristics between incorrectly and correctly triaged patients for EVT
TSC: thrombectomy capable stroke centre, PSC: primary stroke centre, A2L2: Test of hemiparesis with ≥2 points in the NIHSS items for ipsilateral arm and leg, LAO stroke: acute ischaemic stroke caused by large artery occlusion, ICH: intracerebral haemorrhage. *: n=236. **: n=2163
Occlusion sites for patients with acute ischaemic stroke
TSC: Thrombectomy capable stroke centre, PSC: primary stroke centre, IVT: intravenous thrombolysis, MCA: middle cerebral artery/M1: first segment, M2: second segment, ICA-T: internal carotid artery terminus occlusion, PCA: posterior cerebral artery, VA: vertebral artery, BA: basilar artery, ACA: anterior cerebral artery, CTA: CT angiography
For LAO stroke, undertriaged patients were older and more often A2L2 positive (24.5%) compared to those correctly triaged to PSC (6.7%). Among overtriaged patients (LAO stroke), ICH was most common (37.0%), with a lower proportion of stroke mimics compared to those correctly routed to PSC (28.6% vs 51.0%). Differences in other characteristics followed the EVT triage categories: patients correctly triaged to TSC had the highest median NIHSS and undertriaged patients had the highest proportions with unknown exact onset time. Please see the Supplementary material for the comparison of the LAO stroke triage groups.
Prehospital data was complete for n = 2753 patients (94.8%), who were included in the modelling and decision curve analyses. Due to its high predictive performance, the A2L2 test was included in all models for both EVT treatment and LAO stroke. Three models for detection of patients undergoing EVT were computed. For model 1, the variables chosen were FAST items facial weakness and speech problems, based on univariate comparison and clinical relevance. Model 2, starting with the A2L2 test and iteratively adding variables that lowered the AIC, included facial weakness, speech problems (FAST subitems) and age. Model 3 included all available prehospital variables: A2L2 status, age, sex, wake-up stroke, onset-to-ambulance time (or time from last known well), known exact onset time, facial weakness and speech problems.
Decision curve analysis (DCA) for EVT treatment (Figure 2), showed a higher net benefit for SSTS at risk thresholds 0.05 to 0.25. At 0.05, this corresponds to 19 patients overtriaged to TSC for one treated with EVT, while 0.25 means three overtriaged patients for one treated. The corresponding DCA for LAO stroke shows inferiority of the SSTS compared to the A2L2 test alone, as well as models 1–3 (please see Supplementary material). Figure legend: Decision curve analysis for EVT treatment comparing different triage strategies. All: All patients taken to TSC regardless of symptoms. None: No patients taken to the TSC. Risk threshold: Acceptable overtriage rate (e.g. 0.25 = 3 overtriaged patients for 1 treated with EVT).
Precision metrics for EVT triage models at high specificity cutoff (0.90)
Table legend. Model 1: facial weakness and speech problems. Model 2: facial weakness, speech problems and age. Model 3: age, sex, wake-up stroke, onset-to-ambulance time, known onset, facial weakness and speech problems. All models included the A2L2-test. AUC: Area under the curve.

Figure legend: ROC curves for EVT treatment. Model 1: facial weakness and speech problems. Model 2: facial weakness, speech problems and age. Model 3: age, sex, wake-up stroke, onset-to-ambulance time, known onset, facial weakness and speech problems. All models included the A2L2-test.
Discussion
In this sub-analysis of an observational study of a prehospital triage algorithm for detection of patients with LAO stroke eligible for endovascular thrombectomy, the performance of the triage system could not be improved using other variables available in the prehospital setting with statistical modelling using decision curve analysis.
The smallest subgroup in the study was the n=35 patients undergoing secondary transport to TSC for thrombectomy. These patients were younger and presented with lower NIHSS (median 10), compared both to those correctly triaged to TSC as well as n=244 treated with EVT described in a previous study on SSTS (median NIHSS 18 and 17). 15 The >70% EVT undertriaged cases with a proximal anterior circulation LAO may have been well compensated with good collaterals producing milder symptoms, while the remaining 30% had an M2 occlusion or posterior circulation stroke, explaining that only four EVT undertriaged patients had a positive A2L2 test. Importantly, detailed health record review showed that three of these four patients were taken to the nearest PSC in violation of SSTS protocol, while the fourth was initially suspected to have a stroke mimic. Overtriaged patients were more similar to those correctly triaged to TSC than to PSC. Among the overtriaged, the large proportion of patients with intracerebral haemorrhage (29.7%) likely contributed to symptom severity and would have been impossible to distinguish from an acute ischaemic stroke prehospitally without the use of a mobile stroke unit. 22
Decision curve analysis demonstrated that the SSTS had the highest net benefit for EVT treatment when compared to three alternative models using information available in the prehospital stage, despite comparable sensitivity, specificity and a lower AUC. This highlights the difficulty of comparing different triage tools using AUC, as this measure may be harder to interpret for models with one binary predictor. 23 Meanwhile, comparisons using PPV and NPV are highly influenced by prevalence. An illustrative example of an ideal screening tool with very high specificity and sensitivity would at best have a PPV of around 51% at an estimated LAO stroke proportion of 20%. 24 Additionally, equal weighting of sensitivity and specificity may be impractical in a screening tool for EVT, as this treatment was only performed in 119/2905 (4.1%) in our cohort of unselected patients with prehospital stroke suspicion, similar to the 5.0% reported in the study by Nguyen, et al. 25 In the context of prehospital stroke suspicion, a focus on specificity may be a priority especially in healthcare systems with strained TSC bed capacity and limited ambulance resources. 25–28 This was the rationale when designing the SSTS, where access to the electronic health records allows TSC stroke consultants to identify contraindications for EVT, avoiding futile PSC bypass even with a high probability of LAO stroke. The screening for EVT contraindications explains the lower net benefit for LAO stroke demonstrated by DCA, in comparison to all three models and the A2L2 test alone. The SSTS sensitivity (0.41) and specificity (0.93) for LAO stroke were comparable to other prehospital scales. Two prospective studies were recently performed in the Netherlands with head-to-head comparisons of several scales with sensitivities at 0.50–0.67 and specificities at 0.82–0.93. 25,26 A prospective observational study from Madrid, comparing taking all stroke patients to TSC versus triage (PSC or TSC) using the M-DIRECT score for EVT treatment showed a sensitivity of 0.79 and specificity of 0.82, compared to 0.71 and 0.91 for SSTS. 8 M-DIRECT had a PPV of 53% for EVT, compared to 26% in the SSTS. 8 However, the proportion of patients with LAO stroke in M-DIRECT was higher (32% vs 11%), and the proportion of stroke mimics unusually low for a prehospital cohort (22%), both compared to our cohort (44%) and previous studies, likely influencing the comparison. 8,26–28
There are several limitations to our study. The specific geographical and organizational circumstances limit generalizability. However, the proportion of acute ischaemic stroke, LAO stroke, ICH and stroke mimics were very similar to a recent study comparing prehospital LAO scales in a Dutch cohort. 25 Total NIHSS or individual items could not be included as a variable in the modelling, as this information was unavailable in 388 patients, most of them diagnosed with stroke mimics. A full NIHSS examination is not performed in the ambulance and would not have been done on arrival to the emergency department in cases immediately deemed to have an obvious stroke mimic. Therefore, the subset of patients with available in-hospital NIHSS scores would not reflect the true cohort of those with suspected acute stroke in the prehospital setting. Furthermore, some prehospital data was unavailable in 153 patients (5.2%), most commonly FAST subitems. No change in the collection of prehospital information apart from the assessment of hemiparesis was performed, precluding us from including other variables of interest.
Conclusion
Patients requiring secondary transport for thrombectomy, who had been mistriaged by the Stockholm Stroke Triage System to a primary stroke centre, presented with milder symptoms compared to the majority of EVT cases, despite large artery occlusions. Statistical modelling using FAST subitems and patient characteristics available in the ambulance yielded no alternative model with better predictive performance than the SSTS.
Supplemental Material
sj-pdf-1-eso-10.1177_23969873221077845 – Supplemental Material for Analysis and modelling of mistriage in the Stockholm stroke triage system
Supplemental Material, sj-pdf-1-eso-10.1177_23969873221077845 for Analysis and modelling of mistriage in the Stockholm stroke triage system by Boris Keselman, Annika Berglund, Niaz Ahmed, David Grannas, Mia von Euler, Staffan Holmin, Ann-Charlotte Laska, Jan M. Mathé, Christina Sjöstrand, Einar E. Eriksson and Michael V. Mazya in European Stroke Journal
Footnotes
Acknowledgement
The authors would like to kindly acknowledge Prof. em. Nils Wahlgren, MD, for his vision and persistence during the early years of planning of the Stockholm Stroke Triage Project.
Contributorship
All authors participated in the planning of the study. BK, AB, JM and MM were involved in data collection. BK, DG and MM wrote the statistical analysis plan, which was edited and approved by all authors. BK and DG performed data analysis. BK wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: AB and CS were supported by funding the Stockholm County Council (innovation fund). NA was supported by the Stockholm County Council and the Swedish Heart-lung foundation. MM was supported by the Stockholm County Council (clinical postdoctoral appointment). SH was supported by The Söderberg Foundations and MedTechLabs. Other authors received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Informed consent
The need for active consent was waived. All patients received written information regarding data collection from electronic health records and had the possibility to decline participation and have their personal data expunged from the study database.
Ethical approval
Ethical approval was obtained from the Stockholm Regional Research Ethics Committee (approval nr 2017/374).
Guarantor
BK
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
