Sage Journals: Discover world-class research

Abstract

Introduction

The Oxfordshire Community Stroke Project (OCSP) proposed a clinical classification for Stroke patients. This classification has proved helpful to predict the risk of neurological complications. However, the OCSP was initially based on findings on the neurological assesment, which can pose difficulties for classifying patients. We aimed to describe the development and the validation step of a computer-based algorithm based on the OCSP classification.

Materials and methods

A flow-chart was created which was reviewed by five board-certified vascular neurologists from which a computer-based algorithm (COMPACT) was developed. Neurology residents from 12 centers were invited to participate in a randomized trial to assess the effect of using COMPACT. They answered a 20-item questionnaire for classifying the vignettes according to the OCSP classification. Each correct answer has been attributed to 1-point for calculating the final score.

Results

Six-two participants agreed to participate and answered the questionnaire. Thirty-two were randomly allocated to use our algorithm, and thirty were allocated to adopt a list of symptoms alone. The group who adopted our algorithm had a median score of correct answers of 16.5[14.5, 17]/20 versus 15[13, 16]/20 points, p = 0.014. The use of our algorithm was associated with the overall rate of correct scores (p = 0.03).

Discussion

Our algorithm seemed a useful tool for any postgraduate year Neurology resident. A computer-based algorithm may save time and improve the accuracy to classify these patients.

Conclusion

An easy-to-use computer-based algorithm improved the accuracy of the OCSP classification, with the possible benefit of further improvement of the prediction of neurological complications and prognostication.

Keywords

Stroke Oxfordshire Community Stroke Project classification algorithm scale

Introduction

The Oxfordshire Community Stroke Project (OCSP) is a clinical classification based on neurological deficits.¹ OCSP was the first large epidemiological study to use CT scanning to distinguish infarct from hemorrhage. It intended to refine the previous grouping of strokes in such studies as simply “anterior” or “posterior” circulation and, among the first to recognize the likely importance of lacunar stroke.¹

The original hypothesis was that certain groups of patients, as defined by their clinical characteristics at the time of maximum deficit, were likely to have different prognoses for survival, recovery, and recurrence. When applied correctly in any healthcare system worldwide, it offers a valid form of comparison between epidemiological studies.² Also, it was hypothesized that the groups were likely to have different frequencies of large artery, small artery, or cardiogenic mechanisms.^3,4 Many of the investigations which we now consider “routine” did not exist at that time and the validation of this part of the classification occurred gradually over the next 20 years.⁵ The intention was always that, when considering the etiology of a stroke, the OCSP classification would complement rather than replace investigation-based etiological classifications such as The Trial of Org 10,172 in Acute Stroke Treatment classification (TOAST),⁶ particularly when all the necessary investigations either were not available or could not be performed. The OCSP has also been helpful to identify patients with a high risk of a hemorrhagic transformation (HT) after thrombolysis, thus improving the currently available scores to predict such risk.^7,8

Despite the obvious advantages, a classification based on neurological deficits relies on inter-rater reliability of certain symptoms, with some dependency on the seniority and special interest of the examiner.^9,10 A large clinical trial adopted a structured flowchart to define the neurological severity of participants,¹¹ but no computer-based algorithm to classify patients according to OCSP has yet been published.

We aimed to describe the development and validation of an easy-to-use computer-based algorithm (COMPACT) to help neurology residents, neurologists, and general practitioners to allocate correctly the OCSP clinical syndromes.

Methods

A PubMed search using the key-words “Oxfordshire Community Stroke Project” and “classification”, yielded 116 articles. We screened them according to the presence of a list (either original¹ or any modified list) of neurological deficits adopted to classify the clinical syndrome, from which we selected 22 papers for the final analysis. Using a list with the neurological deficits described and their equivalent in the OCSP classification, we created a flow-chart describing the algorithm (Figure 1), and then 5 board-certified vascular neurologists (JBCA, GSS, JPM, FLO, JB) reviewed it. Our flow-chart was created using classification and regression tree (CART).¹² The algorithm is derived from the CART, with repeated partitioning of the original data into smaller groups (nodes) on a yes or no basis, which may mimic a clinical pathway reasoning.¹² We adopted a decision tree in our algorithm because the OCSP classification was originally described as a list of findings on the neuro exam, having similarity to a clinical pathway, but it may be not clearly understood by clinicians.

Figure 1.

The algorithm based on the Oxfordshire Community Stroke Project Stroke Classification. LACI: lacunar infarct; PACI: partial anterior circulation infarct; TACI: total anterior circulation infarct; POCI: posterior circulation infarction; higher cerebr. func.: higher cerebral functions; cerebellar dysfunc: cerebellar dysfunction (without ipsilateral long-tract deficit; i.e. ataxic hemiparesis); restricted: deficit restricted to only a part of hemibody (hand, foot, arm, leg); pure: motor and/or sensitivity deficit without other findings on the neuro exam.

Randomization

A total of 62 neurology residents from 12 programs in Brazil took part in the study, out of the 166 individually invited by e-mail to participate. These 62 participants were randomly allocated into two groups (with or without COMPACT) according to their postgraduate year. We adopted random number tables for this randomization process. Participants were required to self-report their level of English comprehension, and a “fair” level was set as a minimum requirement. We asked the invited trainees about their background knowledge and experience of OCSP. Regarding the wide use of OCSP classification in Brazil and based on the answers from the trainees, we did not adopt these variables in the randomization process. In Brazil, all neurology programs follow the same core curriculum – attenuating part of the underlying bias in the randomization process.

Outcomes

We defined the overall final score as the primary outcome. Each correct answer was attributed 1 point for calculating the final score. As a secondary outcome, we evaluated the agreement rate by syndromes.

Instructions to read the flowchart

Identify the most severe and/or well-described neurologic deficits in the clinical case

Start from the top to bottom-red flagged deficits (Hemianopia/Isolated cranial nerve palsy/Gaze palsy OR brain stem syndrome → Higher Cerebral Function → Motor → Sensory). These signs or symptoms are portrayed by darker boxes. If you did not find all the neurological deficits, keep going down beginning from another dark box.

Define an aspect of the deficits (bilateral, pure, restricted, ataxic…), an associated deficits, or a clinical manifestation.

Then, follow the arrows.

Validation

Three authors who are certified vascular neurologists (JBCA, GSS, and FLO) created a 20-item questionnaire with vignettes. The authors answered the cases independently. Conflicting answers were solved in consensus. The vignettes represented the following proportion of the OCSP syndromes: Lacunar 4/20, Partial Anterior circulation infarction 4/20, Total Anterior circulation infarction 5/20, and Posterior Circulation infarction 7/20. Out of the sixty-two trainees participating in the study, thirty-two were randomly assigned to use the COMPACT digital tool (Figure 1) and were instructed to answer the vignettes based on their background knowledge of the OCSP classification. The group selected to use the COMPACT received access through a link. All answers were collected anonymously. None of the sixty-two participants was excluded from the final analysis – all respondents completed integrally the questions. We used Google Analytics™ to measure how long each user took to complete the task using the COMPACT tool.

Statistical analysis

The Kolmogorov-Smirnov test was used to test for normality and numeric data were represented by medians and interquartile ranges. The Mann-Whitney test was used to compare the non-normal data. Categorical data were analyzed with the Chi-square test or ANCOVA. We use Linear logistic regression to analyze the relationship between the overall score of correct answers and dependent variables. Two-tailed tests were adopted and a p-value < 0.05 was considered statistically significant. Data were analyzed using SPSS Statistics for Windows version 25.0. (IBM SPSS Statistics for Windows, Version 25.0, Armonk, NY).

Ethical statement

The responding residents or Stroke fellows agreed to take part in the validation step. They were informed about the objectives of this work and a confidentiality statement was displayed on the first page of the questionnaire. When answering the questions, all participants were informed about ethics statements and agreed with their anonymized participation in this work.

Results

Using MATLAB® 9.4 we then created a logic algorithm with 14 ways starting from 6 main neurological deficits, from which the classifiers were used to define the OCSP classification. After testing, a web-based and cost-free application was developed.

Algorithm

Figure 1. Example of application

Case study

Male, 78 years old. The neurological exam shows hemiparesis (left side), gaze deviation, visual neglect (left), and complete hemianopia (left).

Adopting our algorithm (Figure 1), we begin from the dark box Heminopia, and then following the lines to get to the Motor and Higher Cerebral Function boxes, reaching out the box TACI.

Computer-based application

The cost-free application is available in English at www.compact-stroke.com, and it is compatible with both desktop and smartphone operational systems (Figure 2(a) to (d)).

Figure 2.

In-website screenshots.

The description of deficits (practical examples and clinical tips of how to identify a specific neurological marker) are provided alongside the question, or as a sign along with the symptom (Figure 2(b) and (c)), as it is presented with the tables in the OCSP classification.^13,14

Validation step

We invited 166 neurology residents from 12 programs, out of which 62 accepted to answer the survey with 20 vignettes. Seventy-four percent (n = 45/62) of respondents reported previous knowledge about the OCSP classification. There was no difference regarding the OCSP previous experience between participants who adopted or not our computer-based algorithm (χ²=0.56). The overall score was 16 [13, 17]/20 (75.7%) without difference between each postgraduate year of residency (p = 0.64). In general, the residents (n = 62) had a better performance to classify the vignettes of LACI (87% of correct answers), and a poor performance classifying TACI cases (52%), p < 0.001. Correct answers were obtained in 82% of POCI cases and 72% in PACI cases. Part of these characteristics of respondents and results are summarized in Table 1.

Table 1.

Characteristics of the respondents who were randomly allocated to use COMPACT or not and results.

Characteristics	All participants (n = 62)	Participants with COMPACT (n = 32)	Participants without COMPACT (n = 30)	p value
Postgraduate year
PGY-2, % (n)	30.6 (19)	25 (8)	36.7(11)	0.41
PGY-3, % (n)	40 (25)	40.6 (13)	40(12)	0.61
PGY-4 or Stroke fellows, % (n)	30 (18)	34.4 (11)	23.4(7)	0.40
Frequency of use of OCSP classification
Never (%)	9.7 (6)	6.2 (2)	13.3 (4)	0.41
Rarely (%)	12.9 (8)	12.5 (4)	13.3 (4)	0.99
Sometimes (%)	17.7 (11)	12.5 (4)	23.3 (7)	0.32
Almost daily (%)	59.7 (37)	68.8 (22)	50 (15)	0.19
Correct answers
All cases, points [IQR]	16 [13, 17]	16.5 [14.5, 17]	15 [13, 16]	0.01
In all cases (%)	75.7	79	72.3	0.01
LACI cases (%)	87	88	85.8	0.56
PACI cases (%)	72	77	66.7	0.03
TACI cases (%)	52	57.8	45.8	0.03
POCI cases (%)	82	84.7	80	0.20

COMPACT: Computerized Oxfordshire Community Stroke Project; PGY: postgraduate year; OCSP: Oxfordshire Community Stroke Project; LACI: lacunar infarct; PACI: partial anterior circulation infarct; TACI: total anterior circulation infarct; POCI: posterior circulation infarction; IQR: interquartile range.

Thirty-two participants answered the vignettes using COMPACT In this group, the median score of correct answers was 16.5 [14.5, 17] versus 15 [13, 16]/20 points in the group without the COMPACT (n = 30/62), p = 0.014, Table 1. Respondents who used COMPACT had a higher score for classifying PACI (77 versus 66.7%, p = 0.03) and TACI (57.8 versus 45.8%, p = 0.03) cases.

From a linear regression model, the use of COMPACT was associated with the final score (20 points) of correct answers [stand. coefficient 0.26, 95% CI (0.132–2.49), p = 0.03]. Adjusting the analysis by participants who stated a higher frequency of use of the OCSP classification (almost daily or sometimes use versus rare or never use), the use of COMPACT was still associated with the overall score of correct answers [stand. coefficient 0.24, 95% CI (0.04–2.25), p = 0.04]. In other adjusted analyses by the postgraduate year, the use of COMPACT was also associated with the overall score of correct answers [stand. coefficient 0.25, 95% CI (0.02–2.38), p = 0.04].

During the validation step, the mean time for utilizing the COMPACT was six minutes per user, according to Google Analytics™ metrics.

Discussion

Almost 30 years after the first publication, we have proposed a computer-based algorithm of a well-studied stroke clinical classification scale. The OCSP clinical classification¹ has been used extensively, with moderate accuracy¹⁴ and inter-observer agreement even between neurologists¹⁵ since it was first described. Part of these disagreements occurred based on discordances of neurological signs.¹⁵ However, some authors have stated that the OCSP classification may be a useful instrument for clinical and community-based epidemiological studies.¹⁵

The original publication¹ required the user to have a detailed knowledge of which of the clinical patterns should be allocated to a particular syndrome. We showed that our algorithm effectively assists in the application of identified neurological markers into a validated flowchart, providing the correct OCSP classification, without the clinician necessarily having a detailed knowledge of the classification.

Digital applications like ours using classification and regression tree (CART)¹² have been adopted by numerous medical fields for their model interpretability, scalability to large datasets, and connection to rule-based decision-making. Many published diagrams have been derived directly from diagnostic algorithms or clinical flow-charts and can help specialists or general practitioners in the decision-making process of acute medical conditions. In addition to helping the decision-making process, diagnosing may be also facilitated using these tools.^13,14,16 The CART builds a model by recursively partitioning the instance space, and labeling each partition with either a predicted category,¹⁷ allowing connecting decision tree learning and gradient boosting resources – which may boost algorithms. In comparison to algorithms derived from regression analysis, decision trees do not require clinicians to remember the individual score attributed to the variables.¹⁸

Published interobserver reliability of OCSP has found a good accuracy for hemiparesis and dysphasia but poor agreement for sensory loss.¹¹ In our algorithm, the user can begin to interpret it from the sensory deficits; moreover, we also linked these sensory deficits with each other deficit (portrayed here by the darker boxes) – which may stimulate the user to pay more attention to this group of neurological manifestations.

The OCSP classification consists of four clinical syndromes. Overall, two of those are related to anterior circulation infarcts, one with posterior circulation and the other one with both vascular territories. Total Anterior Circulation Infarcts (TACI) account for about 20% of all Strokes,³ and they are considered as the most severe clinical syndrome according to OCSP.¹ In our proposed algorithm, this group consists of four main neurological deficits portrayed by darker boxes. From the top to the bottom, starting from either Hemianopia or Higher cerebral functions, TACI has been identified by a combination of those main deficits. We aimed to make the identification of TACI easier than other syndromes – emphasizing the clinical relevance of this syndrome. However, only 57.8% of trainees who adopted COMPACT answered those cases correctly. This finding may be associated with the fact of three or more main neurological deficits (represented by darker boxes in our algorithm) – for many trainees, the word “total” means all of these deficits, leading to misdiagnosis.

Lacunar infarcts (LACI) are associated with a good outcome and low rates of recurrence,¹⁹ despite being the major cause of progressive motor deficits.²⁰ In our algorithm, LACS is described as a pure or combined motor and/or sensory syndrome, including ataxia and dysarthria-clumsy hand. Restricted pure either sensory or motor symptoms were allocated to the Partial Anterior Circulation Infarcts (PACI) box in our algorithm. The Posterior circulation syndrome describes deficits associated with infarctions in the vertebrobasilar arterial system. We highlighted in our algorithm neurological deficits related to posterior brain injuries, like brain stem syndromes, gaze palsy, isolated cranial nerve palsy, and cerebellar dysfunction. These conditions have been extensively reported in the literature such as posterior circulation symptoms.^21–24

In our validation step COMPACT seemed a useful tool for any postgraduate year Neurology resident. Moreover, the COMPACT proved to be more useful for classifying PACI and TACI cases. Distinguishing these stroke syndromes may help to predict the extent of an ischemic lesion and therefore plan therapies like mechanical thrombectomy in advance. Partial anterior syndromes, in general, are associated with smaller lesions than total anterior syndromes.¹ Moreover, according to some authors,¹⁴ the agreement rate between the OCSP classification and neuroimaging is about 80%. Therefore, COMPACT might be helpful when using the concept of clinical mismatch in a decision-making process for reperfusion therapies in extended time windows for instance. Our proposed system is a reliable, fast, and accurate web-based tool for routine neurological care, and it might as well be useful for multi-center clinical trials.

Our work has some limitations. First, the COMPACT is only based on the OCSP classification, therefore syndromes from arterial variants, subacute or progressive infarctions, and multiple lesions might not be properly classified. Second, we only assessed the validity of our algorithm in a small number of participants, and we did not include neurologists or general practitioners in our sample. Third, we did not compare the time spent to perform each classification between the groups, however, the mean time using COMPACT was only around six minutes. Fourth, our algorithm is only available in English – thus, our results may not be generalizable based on language proficiency. Fifth, we invited 166 trainees, but a small number of trainees accepted to participate – reducing the generalizability of our results. Sixth, due to small number of participants, we can not assess the influence of the prior use of the OCSP in each allocated groups. Finally, our data were not enough to determine sensitivity and especificity of our algorithm.

Conclusion

Our free and easy-to-use algorithm proved to help classify Stroke subtypes according to the well-studied OCSP classification. A better understanding of the clinical vascular syndrome may be helpful to assess the neurological severity in addition to scales like the National Institutes of Health Stroke Scale. Further studies with larger samples are required to validate the algorithm, which is available on-line.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Dr Andrade's visiting scholarship at Columbia University, USA, is sponsored by the CAPES Foundation, Ministry of Education, Brazil.

Informed consent

The participants were informed about the objectives of this work and a confidentiality statement was displayed on the first page of the questionnaire. When answering the questions, the participants were informed about ethics statements and agreed with their anonymized participation in this work.

Ethical approval

This project has been approved in our local IRB. The responding residents or Stroke fellows agreed to take part in the validation step.

Guarantor

Joao Brainer Clares de Andrade.

Contributorship

All the authors have read and approved the submitted manuscript. The manuscript has not been submitted elsewhere nor published elsewhere in whole or in part. JBC Andrade – planned the study, collected data and wrote the final version. JP Mohr – reviewed the report and reviewed the final version. FB Timbó – planned the study, developed the digital application and reviewed the final version. CR Nepomuceno – collected data and reviewed the paper. JVS Moreira – planned the study, developed the digital application and reviewed the final version. ICG Timbó – planned the study, developed the digital application and reviewed the final version. FO Lima – planned the study, evaluated the digital application and reviewed the final version. GS Silva – supervised the study, reviewed the report and wrote the final version. J Bamford – supervised the study, reviewed the report and reviewed the final version.

Acknowledgements

Full access to all of the data can be requested at any time. All the authors allow the right to publish any and all data from this research.

ORCID iDs

Joao Brainer Clares de Andrade

John Bamford

References

Bamford

Sandercock

Dennis

, et al. Classification and natural history of clinically identifiable subtypes of cerebral infarction. Lancet 1991; 337: 1521–1526.

O’Donnell

, Chin

, Rangarajan

, et al. Global and regional effects of potentially modifiable risk factors associated with acute stroke in 32 countries (INTERSTROKE): a case control study. Lancet 2016; 388: 761–765.

Mead

Lewis

Wardlaw

, et al. How well does the Oxfordshire Community Stroke Project classification predict the site and size of infarct of brain imaging. ? Journal of Neurology, Neurosurgery & Psychiatry 2001; 70: 558–562.

Smith

Emsley

Libetta

, et al. The Oxfordshire Community Stroke Project classification in the early hours of ischemic stroke and relation to infarct site and size on cranial computed tomography. J Stroke Cerebrovasc Dis 2001; 10: 205–209.

Mead

Shingler

Farrell

, et al. Carotid disease in acute stroke. Age Ageing 1998; 27: 677–682.

Adams

Bendixen

Kappelle

, et al. Classification of subtype of acute ischemic stroke. Definitions for use in a multicenter clinical trial. TOAST. Trial of org 10172 in acute stroke treatment. Stroke 1993; 24: 35–41.

Yang

Wang

Zhao

, et al. The Oxfordshire Community Stroke Project classification system predicts clinical outcomes following intravenous thrombolysis: a prospective cohort study. TCRM 2016; 12: 1049–1056.

Sung

S-F

Chen

SC-C

Lin

H-J

, et al. Oxfordshire community stroke project classification improves prediction of post-thrombolysis symptomatic intracerebral hemorrhage. BMC Neurol 2014; 14: 39.

Lodder

Bamford

Kappelle

, et al. What causes false clinical prediction of small deep infarcts? Stroke 1994; 25: 86–91.

10.

Shinar

Gross

Mohr

, et al. Interobserver variability in the assessment of neurologic history and examination in the stroke data bank. Arch Neurol 1985; 42: 557–565.

11.

Lindley

Warlow

Wardlaw

, et al. Interobserver reliability of a clinical classification of acute cerebral infarction. Stroke 1993; 24: 1801–1804.

12.

Gordon

Breiman

Friedman

, et al. Classification and regression trees. Biometrics 1984; 40: 874.

13.

Al-Buhairi

Phillips

Llewellyn

, et al. Prediction of infarct topography using the Oxfordshire Community Stroke Project classification of stroke subtypes. J Stroke Cerebrovasc Dis 1998; 7: 339–343.

14.

Aerden

Luijckx

G-J

Ricci

, et al. Validation of the Oxfordshire Community Stroke Project syndrome diagnosis derived from a standard symptom list in acute stroke. J Neurol Sci 2004; 220: 55–58.

15.

Dewey

Macdonell

Donnan

, et al. Inter-rater reliability of stroke sub-type classification by neurologists and nurses within a community-based stroke incidence study. J Clin Neurosci 2001; 8: 14–17.

16.

Cabitza

Banfi

Machine learning in laboratory medicine: waiting for the flood?

Clin Chem Lab Med 2018; 56: 516–524.

17.

Caruana

Niculescu-Mizil

An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on machine learning – ICML 06, New York, NY, USA, June 2006, pp.161–168. New York: Association for Computing Machinery.

18.

Phan

Chen

Singhal

, et al. Exploratory use of decision tree analysis in classification of outcome in hypoxic–ischemic brain injury. Front Neurol 2018; 9: 126.

19.

Norrving

Long-term prognosis after lacunar infarction. Lancet Neurol 2003; 2: 238–245.

20.

Steinke

Ley

SC.

Lacunar stroke is the major cause of progressive motor deficits. Stroke 2002; 33: 1510–1516.

21.

Brodsky

Donahue

Vaphiades

, et al. Skew deviation revisited. Surv Ophthalmol 2006; 51: 105–128.

22.

Smith

David

Klintworth

Skew deviation. Neurology 1964; 14: 96–105.

23.

Rosemergy

Mossman

Brainstem lesions presenting with nausea and vomiting. N Z Med J 2007; 120U: 2532.

24.

Lekic

Ani

Posterior circulation stroke: animal models and mechanism of disease. J Biomed Biotechnol 2012; 2012: 587590.

Oxfordshire Community Stroke Project Classification: A proposed automated algorithm

Abstract

Introduction

Materials and methods

Results

Discussion

Conclusion

Keywords

Introduction

Methods

Randomization

Outcomes

Instructions to read the flowchart

Validation

Statistical analysis

Ethical statement

Results

Algorithm

Case study

Computer-based application

Validation step

Discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

Informed consent

Ethical approval

Guarantor

Contributorship

Acknowledgements

ORCID iDs

References