Abstract
Introduction: Early detection of large vessel occlusion (LVO) is essential to facilitate fast endovascular treatment. CT angiography (CTA) is used to detect LVO in suspected stroke patients. We aimed to assess the accuracy of CTA evaluations in daily clinical practice in a large cohort of suspected stroke patients. Patients and methods: We used data from the PRESTO study, a multicenter prospective observational cohort study that included suspected stroke patients between August 2018 and September 2019. Baseline CTAs were re-evaluated by an imaging core laboratory and compared to the local assessment. LVO was defined as an occlusion of the intracranial internal carotid artery, M1 segment, or basilar artery. Medium vessel occlusion (MeVO) was defined as an A1, A2, or M2 occlusion. We calculated the accuracy, sensitivity, and specificity to detect LVO and LVO+MeVO, using the core laboratory evaluation as reference standard. Results: We included 656 patients. The core laboratory detected 89 LVOs and 74 MeVOs in 155 patients. Local observers missed 6 LVOs (7%) and 28 MeVOs (38%), of which 23 M2 occlusions. Accuracy of LVO detection was 99% (95% CI: 98–100%), sensitivity 93% (95% CI: 86–97%), and specificity 100% (95% CI: 99–100%). Accuracy of LVO+MeVO detection was 95% (95% CI: 93–96%), sensitivity 79% (95% CI: 72–85%), and specificity 99% (95% CI: 98–100%). Discussion and Conclusion: CTA evaluations in daily clinical practice are highly accurate and LVOs are adequately recognized. The detection of MeVOs seems more challenging. The evolving EVT possibilities emphasize the need to improve CTA evaluations in the acute setting.
Introduction
Endovascular thrombectomy (EVT) has become the standard of care for patients with an intracranial large vessel occlusion (LVO) in the anterior circulation up to 6 hours after onset and in selected patients even up to 24 hours after onset.1–3 Because the effect of EVT is time-dependent, early detection of LVO is essential for good clinical outcome. 4 In most stroke centers, CT angiography (CTA) is used to detect LVOs during the acute work-up of suspected stroke patients.5,6 Interrater agreement of LVO detection on CTA has been reported ranging from 0.48 to 0.97.7–10 However, this has only been assessed in a limited number of small studies (sample size range 15–100) that were focused on proximal LVOs and performed in retrospective, controlled, and experimental settings.7–10 Most suspected stroke patients are referred to stroke centers without EVT capabilities and half of the patients present outside office hours. 11 Consequently, presence and location of LVO will often not be assessed by neuroradiologists or interventionalists, but by radiologists, neurologists, or residents with less experience in vascular neuroradiology. So far, the accuracy of CTA evaluations in the acute setting is unknown. We aimed to assess the accuracy of CTA evaluations in daily clinical practice in a large representative cohort of suspected stroke patients in different hospitals.
Methods
Study design
We used data from the prehospital triage of patients with suspected stroke (PRESTO) study, a multicenter prospective observational cohort study that included suspected stroke patients in the ambulance between August 13, 2018, and September 2, 2019.12,13 Inclusion criteria were new neurological deficit defined as at least one point on the Face-Arm-Speech-Time test, age 18 years or older, and blood glucose of at least 2.5 mmol/L. For the current analysis, we included patients with baseline CTA. Patients with intracranial hemorrhage on baseline NCCT were excluded. The Institutional Review Board of the Erasmus MC University Medical Center Rotterdam has reviewed the study protocol and confirmed that the Dutch Medical Research Involving Human Subjects Act was not applicable. Because this study met the exceptions of informed consent regulations, the need for informed consent was waived. Detailed information regarding the study is described elsewhere. 13
Local procedures
The eight participating hospitals had an emergency department and stroke unit and were equipped for rapid diagnosis of ischemic stroke. Three hospitals had a radiology residency training program, of which two centers were intervention centers capable of EVT, including one academic hospital. During the PRESTO study, non-contrast CT (NCCT), CTA, CT perfusion (CTP), magnetic resonance imaging, or digital subtraction angiography could be performed as part of the regular work-up, based on the assessment of the treating physician. CTAs were performed according to local clinical acquisition protocols (Supplementary Table 1). Directly after imaging acquisition, the radiology resident or radiologist on call reviewed the NCCT and CTA. CTA evaluations by radiology residents were always supervised by a radiologist. In clinical practice, especially during the night shift, this supervision was at a later moment. For this analysis, the final supervised imaging report was used as local evaluation. Dutch national guidelines recommend CTA if a diagnosis of ischemic stroke is clinically suspected in patients without EVT contra-indications. Regional protocols advised EVT in all patients with an occlusion of the intracranial part of the internal carotid artery (ICA), the middle cerebral artery segment (M1 or M2), or the anterior cerebral artery segment (A1 or A2), if EVT is possible within 6 hours after last-seen-well, irrespective of stroke severity. During the course of the study, patients with basilar artery occlusions were evaluated for inclusion in the BASilar artery International Cooperation Study. 14
Central imaging storage and evaluation
All neuroimaging was de-identified and stored in an imaging database (XNAT; Neuroinformatics Research Group, St Louis, MO). All baseline CTAs were reviewed by an imaging core laboratory using RadiAnt DICOM Viewer software (Medixant, Poland). Baseline NCCT was provided with the CTA. CTP (if performed) was not provided to the imaging core laboratory. Core laboratory observers were blinded to the final diagnosis or severity of the symptoms but were informed about the following clinical symptoms at baseline: presence and side of the hemiparesis, presence of aphasia, or non-localizing symptoms in case of absence of hemiparesis and aphasia.
LVO was defined as an occlusion of the intracranial ICA: infraclinoid, supraclinoid, or terminal part of the ICA (ICA-T), the M1 segment, or the basilar artery. 15 MeVO was defined as an occlusion of the A1, A2, or M2 segment. We differentiated between proximal and distal M1 occlusions based on the proximal versus the distal half of the M1 segment. The M2 segments were defined as the post-bifurcation branches of the M1 segment. The M2-M3 transition was defined as the location where the arteries rotate to the operculum and return in a horizontal position.
The imaging core laboratory consisted of seven experienced observers (four neuroradiologists and three interventional neuroradiologists). Each scan was evaluated by one of six core laboratory observers (JB, ASP, JJH, ACGMvE, GJL, and PJvD). In case of discordant CTA evaluations between the local observer and the imaging core laboratory, the seventh core laboratory member (AvdL) re-evaluated imaging, blinded to the prior assessments. If the two core laboratory evaluations did not match, disagreements were resolved in consensus by the two co-chairs of the imaging core laboratory (ACGMvE and AvdL).
Statistical analysis
We reported continuous variables as mean and standard deviation or median and interquartile range. We reported categorical variables as numbers and percentages. To calculate the sensitivity, specificity, and accuracy for the detection of LVO and MeVO, we created contingency tables with the local CTA evaluation as index test, and the final core laboratory evaluation as reference standard. For the contingency tables, we approached every LVO and MeVO separately in patients with multiple occlusions. Other occlusion locations were reported but were not used in the analysis. To assess the effect of simultaneous CTP acquisition or presentation in an academic hospital, we calculated the test characteristics for the subgroup of patients with CTP (that was only available to the local observers) and the subgroup of patients that presented in the academic hospital. Test characteristics for these subgroups were compared to the test characteristics of the remaining cohort using Fisher’s exact test.
Additionally, we compared two subgroups of patients with a core laboratory confirmed LVO or MeVO: the patients with an LVO or MeVO detected by the local observer and the patients with an LVO or MeVO missed by the local observer. For this comparison, patients with multiple occlusions were categorized in the subgroup of patients with a locally detected occlusion if at least one LVO or MeVO was locally detected. Between-group comparisons were made with independent-samples t-test, Mann–Whitney U test, chi square test, or Fisher’s exact test, as appropriate. To estimate the clinical impact of occlusions missed by the local team, we explored the potential EVT indication of these patients, based on our regional treatment protocols. We assumed all patients with a clinical diagnosis of ischemic stroke and LVO or MeVO were technically treatable. If, in addition, treatment was possible within 6 hours after last-seen-well and the occlusion was symptomatic (NIHSS>0), we considered the patient eligible for EVT.
We assessed and reported completeness of the data. No data imputation was used for the analyses. All analyses were performed using R software (version 3.6.1) and RStudio (version 1.0.153).
Results
Of the 1334 patients enrolled in the PRESTO study, we included 656 patients for this analysis (Figure 1). Patients had a median age of 73 years (IQR: 62–81), 290/656 (44%) were women, and the median NIHSS score was 4 (IQR: 2–9) (Table 1). Patients presented at the emergency department at a median of 94 mins (IQR: 54–200) since last-seen-well. Baseline NCCT was performed 9 mins (median) after arrival at the emergency department (IQR: 6–13), followed by CTA after another 6 minutes (median, IQR: 3–10). Flowchart of the patient inclusion. Patient characteristics. Data are median (IQR) or n (%), unless otherwise indicated. SBP = systolic blood pressure. SD = standard deviation. NIHSS = National Institutes of Health Stroke Scale. NCCT = non-contrast computed tomography. CTA = computed tomography angiography. TIA = transient ischemic attack. Number of missings: SBP: 4 (0.6%), NIHSS: 3 (0.5%), onset-to-door: 15 (2%), door-to-NCCT: 16 (2%), CT-to-CTA: 20 (3%).
Core laboratory evaluations of 596 CTAs (91%) were consistent with the local evaluations (Figure 2). Sixty CTAs (9%) had to be reviewed by a second core laboratory member. The second core laboratory member agreed with the first core laboratory assessment in 18 CTAs. Disagreement concerning 42 CTAs (70%) was resolved in consensus. Most discordances between the two core laboratory evaluations concerned the presence of an occlusion (30/42, 71%), in particular regarding the M2 segment (21/30, 70%). Flowchart of the core laboratory assessments. BA = basilar artery. ICA = internal carotid artery.
Performance of local CTA evaluations
Occlusion locations.
LVO = large vessel occlusion, defined as an occlusion in the intracranial internal carotid artery, M1 segment or basilar artery. MeVO = medium vessel occlusion, defined as an occlusion of the A1, A2, or M2 segment. ICA = internal carotid artery. ICA-T = internal carotid artery terminus.
of 164 patients had two occlusions, total occlusion count is 182.
The accuracy for the detection of LVO was 99% (95% confidence interval (CI): 98–100%), with a sensitivity of 93 (95% CI: 86–97%) and a specificity of 100 (95% CI: 99–100%) (Supplementary Table 3). The accuracy for the detection of LVO+MeVO was 95% (95% CI: 93–96%), with a sensitivity of 79% (95% CI: 72–85%) and a specificity of 99 (95% CI: 98–100%) (Supplementary Table 4). Sensitivity of MeVO detection was 62% (95% CI: 51–73%). Results for the detection of LVO or LVO+MeVO were not significantly different for patients that underwent CTP or for patients that presented in the academic hospital (Supplementary Table 6–12).
Patients with LVO/MeVO missed by the local observer
Clinical and Imaging aspects of patients with intracranial occlusions confirmed by the core laboratory.
Data are median (IQR) or
Number of missings: NIHSS: 2 (1%), CTA outside office hours: 3 (2%), CTA during night-time: 5 (3%), onset-to-door: 2 (1%), door-to-NCCT: 3 (2%), NCCT-to-CTA: 5 (3%), quality of the CTA: 6 (4%), acquisition phase: 1 (0.6%).
Patients with multiple LVOs/MeVOs were categorized in the subgroup of patients with a locally detected LVO/MeVO if at least one LVO/MeVO was locally detected.
a Two patients were diagnosed with an occlusion in the contralateral (asymptomatic) hemisphere.
b Four patients were diagnosed with an occlusion in a different vascular territory than the clinical symptoms suggested (cerebellar versus hemispheric) or the contralateral hemisphere.
Discussion
This study is the first to assess the accuracy of CTA evaluations of suspected stroke patients in daily clinical practice. CTA evaluations to detect LVO are accurate. However, a considerable proportion of MeVOs were not detected in the acute setting. Missed MeVOs were mostly located in the M2 segment.
The imaging core laboratory was not only more experienced but also benefited from the setting without time pressure. Besides this, the need for a consensus meeting because of discrepancies between core laboratory evaluations confirmed the difficulty of occlusion detection in the M2 segment. Patients with LVO or MeVO missed by the local observer had milder neurologic deficit, which may be explained by the proportion of more distal occlusion locations, but might also be because good collateral circulation made the occlusions hard to detect. Patients with a missed LVO or MeVO more often presented outside the 6-hour time window. Our core laboratory was instructed carefully before the evaluations, but local observers had a different objective, namely, rapidly assess EVT eligibility. In patients that presented beyond 6 hours after last-seen-well, local observers might have been focusing less on MeVO presence. This is because beyond 6 hours after last-seen-well, only LVOs and proximal M2 occlusions could be included in the MR CLEAN LATE Trial. In some patients, clinical symptoms could not be fully attributed to the found occlusion location, so these occlusions might have been pre-existent or asymptomatic. However, it remains important to recognize all intracranial occlusions, in case of acute neurological deterioration of the patient, to gain insight in stroke etiology and the international tendency to treat more distal occlusions.16, 17
One previous single-center study in which a panel of 2 neuroradiologist re-assessed CTAs of ischemic stroke patients (
This study has some limitations. We used our final core laboratory evaluation as reference standard, but since some (mostly distal) occlusions are not easily detected, it is possible there might have been more MeVOs that were even missed by the core laboratory. We have no knowledge regarding the exact training (e.g., radiological focus area) and working experience of the local observers. It is likely that the level of experience of the observers is an important factor in the accuracy of CTA evaluations. 18 Also, we have no information about the time that was needed to detect the intracranial occlusions. However, it is clear that locally missed occlusions were not always easily detectable, even by an experienced core laboratory. Another limitation is that in a substantial proportion of the patients included in the PRESTO study, CTA was not performed. In clinical practice, the indication of a CTA in suspected stroke patients is determined by the treating physician. The Dutch national stroke guidelines recommend that all patients with a diagnosis of acute ischemic stroke undergo CTA. This is most often omitted because the likelihood of ischemic stroke or presence of an intracranial occlusion is considered to be very low. This implies that our cohort reflects the suspected stroke population that would be subjected to CTA.
In contrast to local practices in our region, EVT of MeVOs is not standardly recommended by all guidelines, especially in patients with low NIHSS scores. 15 Currently, the ENDOLOW trial is being conducted to investigate whether EVT improves clinical outcome of patients with occlusions up to the M2 segment and low NIHSS (0–5). Since the prevalence of ischemic stroke in the population is expected to rise and EVT possibilities are developing, it is crucial to improve the detection of MeVOs. Besides, even if patients with an LVO or MeVO would not be treated, occlusion detection may influence stroke management, for example, regarding blood pressure management. The first step toward improvement is to be aware that in patients with minor deficit, intracranial occlusions can still be present and these occlusions might not always be obvious, especially in the M2 segment. It is also important to thoroughly assess the complete intracranial vasculature, both the anterior as the posterior circulation, and not restrict to the clinically suspected area. Additional training in LVO/MeVO detection might also be helpful. Another possible solution to prevent missing occlusions could be the acquisition of multiphase CTA or additional CTP imaging. There is some evidence that multiphase CTA improves the diagnostic accuracy for occlusion detection. 10 Visible perfusion defects on CTP imaging might guide the observer to the occluded vessel and ease the CTA evaluation.19,20 We found no differences in LVO/MeVO detection by local observers in patients with CTP and patients that presented in the academic hospital. However, because these were small subgroup analyses and most CTPs were performed in the academic hospital which was also the only hospital with multiphase CTA, our study is not suitable to assess the added value of either CTP or multiphase CTA. Another development is the use of automated LVO detection software is currently being explored and might also be used to aid the radiologist in rapid detection of LVO. 21
Conclusion
CTA evaluations in daily clinical practice are highly accurate, and in general, LVOs are adequately recognized. The detection of MeVOs seems more challenging. With the evolving endovascular treatment possibilities, this emphasizes the need to improve CTA assessments in the acute setting.
Supplemental Material
sj-pdf-1-eso-10.1177_23969873211058576 – Supplemental Material for Accuracy of CTA evaluations in daily clinical practice for large and medium vessel occlusion detection in suspected stroke patients
Supplemental Material, sj-pdf-1-eso-10.1177_23969873211058576 for Accuracy of CTA evaluations in daily clinical practice for large and medium vessel occlusion detection in suspected stroke patients by Martijne H. C. Duvekot, Adriaan C. G. M. van Es, Esmee Venema, Lennard Wolff, Anouk D. Rozeman, Walid Moudrous, Frédérique H. Vermeij, Hester F. Lingsma, Jeannette Bakker, Aarnout S. Plaisier, Jan-Hein J. Hensen, Geert J. Lycklama à Nijeholt, Pieter Jan van Doormaal, Diederik W.J. Dippel, Henk Kerkhoff, Bob Roozenbeek, Aad van der Lugt and on behalf of the PRESTO investigators in European Stroke Journal
Footnotes
Executive Committee
Diederik W.J. Dippel (1); Bob Roozenbeek (1,2); Henk Kerkhoff (3); Hester F. Lingsma (4); Aad van der Lugt (2); Adriaan C.G.M van Es (5); Anouk D. Rozeman (3); Walid Moudrous (6); Frédérique H. Vermeij (7).
Study coordinators
Esmee Venema (1,4); Martijne H.C. Duvekot (1,3)
Steering Committee
Diederik W.J. Dippel (1); Bob Roozenbeek (1,2); Hester F. Lingsma (4); Aad van der Lugt (2); Adriaan C.G.M. van Es (5); Henk Kerkhoff (3); Anouk D. Rozeman (3); Walid Moudrous (6); Frédérique H. Vermeij (7); Kees C.L. Alblas (7); Laus J.M.M. Mulder (8); Annemarie D. Wijnhoud (9); Lisette Maasland (10); Roeland P.J. van Eijkelenburg (11); Marileen Biekart (12); M.L. Willeboer (13); Bianca Buijck (14)
Imaging Committee
Bob Roozenbeek (1,2); Henk Kerkhoff (3); Aad van der Lugt (2); Adriaan C.G.M. van Es (5)
Imaging Core Laboratory
Aad van der Lugt (2); Adriaan C.G.M. van Es (5); Pieter Jan van Doormaal (2); Jeannette Bakker (15); Jan-Hein Hensen (16), Aarnout Plaisier (17); Geert Lycklama á Nijeholt (18)
Local principal investigators
Diederik W.J. Dippel (1); Bob Roozenbeek (1,2); Aad van der Lugt (2); Amber Hoek (19); Henk Kerkhoff (3); Anouk D. Rozeman (3); Jeannette Bakker (15); Erick Oskam (20); Walid Moudrous (6); Jan-Hein Hensen (16); Frédérique H. Vermeij (7); Mandy M.A. van der Zon (21); Egon D. Zwets (22); Kees C.L. Alblas (7); Laus J.M.M. Mulder (8); Jan Willem Kuiper (23); Annemarie D. Wijnhoud (9); Bruno J.M. van Moll (9); Aarnout Plaisier (17); Lisette Maasland (10); Mirjam Woudenberg (24); Roeland P.J. van Eijkelenburg (11); Arnoud M. de Leeuw (25)
Local trial coordinators
Anja Noordam-Reijm (12); Timo Bevelander (13)
PhD / Medical students
Vicky Chalos (1,2,3); Eveline J.A. Wiegers (4); Lennard Wolff (2); Dennis C. van Kalkeren (1); Jochem van den Biggelaar (1)
List of affiliations
Department of Neurology (1), Radiology & Nuclear Medicine (2), Public Health (4), Emergency Medicine (19), Erasmus MC University Medical Centre, Rotterdam; Department of Neurology (3), Radiology (15), Emergency Medicine (20), Albert Schweitzer hospital, Dordrecht; Department of Radiology (5), Leiden University Medical Center, Leiden; Department of Neurology (6), Radiology (16), Maasstad hospital, Rotterdam; Department of Neurology (7), Radiology (21), Emergency Medicine (21), Franciscus Gasthuis & Vlietland, Rotterdam; Department of Neurology (8), Radiology (23), Ikazia hospital, Rotterdam; Department of Neurology (9), Radiology (17); IJsselland hospital, Rotterdam; Department of Neurology (10), Radiology (24), van Weel Bethesda hospital, Dirksland; Department of Neurology (11), Radiology (25), Rivas Zorggroep Beatrix hospital, Gorinchem; Ambulance service Rotterdam-Rijnmond (12), Barendrecht; Ambulance service Zuid-Holland Zuid (13), Papendrecht; Rotterdam Stroke Service (14), Rotterdam; Department of Radiology (18), Haaglanden Medical Center, The Hague.
Acknowledgments
We would like to thank all PRESTO Registry Investigators for their assistance in this research.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Diederik Dippel and Aad van der Lugt report funding from the Dutch Heart Foundation, Brain Foundation Netherlands, The Netherlands Organisation for Health Research and Development, Health Holland Top Sector Life Sciences & Health, and unrestricted grants from Penumbra Inc., Stryker, Stryker European Operations BV, Medtronic, Thrombolytic Science, LLC and Cerenovus for research, all paid to institution. Pieter Jan van Doormaal reports funding from Stryker, paid to institution and an unrestricted fee from Bayer. All other authors declare no conflict of interest.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the BeterKeten collaboration and Theia Foundation (Zilveren Kruis).
Ethical approval
The Institutional Review Board of the Erasmus MC University Medical Center Rotterdam has reviewed the study protocol and confirmed that the Dutch Medical Research Involving Human Subjects Act was not applicable. Therefore, ethical approval was not required.
Informed consent
Because this study met the exceptions of informed consent regulations, the need for informed consent was waived.
Guarantor
AvdL.
Contributorship
MHCD contributed to study design, data acquisition, statistical analysis, interpretation of data, and wrote the report. ACGMvE, EV, LW, ADR, WM, FHV, HFL, DWJD, HK, BR, and AvdL contributed to study design and data acquisition. JB, ASP, JJH, GJLàN, PJvD, ACGMvE, and AvdL formed the Imaging Core Laboratory committee and assessed neuroimaging. All authors critically reviewed the report and approved the final draft.
Data availability statement
In compliance with the General Data Protection Regulation, source data are not available for other researchers. Information about analytic methods, study materials and scripts of the statistical analyses are available from the corresponding author on reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
