Sage Journals: Discover world-class research

Abstract

Background

Early prediction of poststroke motor recovery is challenging in clinical settings. The Prediction recovery potential (PREP2) algorithm is the most accurate approach for prediction of Upper Limb function available to date but lacks external validation.

Objectives

(i) To externally validate the PREP2 algorithm in a prospective cohort, (ii) to study the characteristics of patients misclassified by the algorithm, and (iii) to compare the performance according to the presence of cognitive syndromes (aphasia, neglect, cognitive disorders).

Methods

We enrolled 143 patients with stroke and upper extremity weakness persistent at Day 3. Evaluation to predict the recovery status according to the PREP2 algorithm included age, SAFE and NIHSS scores at Day 3 and transcranial magnetic stimulation to determine the presence of the motor-evoked potential before day seven. Actual recovery (excellent, good, limited, or poor) was defined based on the Action Research Arm test score at 3 months. Accuracy was computed by comparing the predictions of the PREP2 and the actual category of the patient. Additionally, to investigate misclassifications and the impact of cognitive syndromes, we recorded SAFE and NIHSS scores at Day 7, the Montreal Cognitive Assessment (MoCA) score, the presence of aphasia and neglect and Magnetic Resonance Imaging was used to evaluate the corticospinal tract lesion load.

Results

The PREP2 algorithm showed a very good predictive value with 78% accuracy [95% CI: 71.2%-86.1%], especially for the extreme categories of recovery (EXCELLENT 87.5% [95% CI: 78.9%-96.2%] and POOR 94.9% [95% CI: 87.9%-100%]), and only 46.5% [95% CI: 19.05%-73.25%] for the GOOD category and even worse than chance for the LIMITED category 0%. Pessimistic predictions (false-negative cases) had a drastic improvement in the SAFE score acutely compared to that of well-predicted patients with unfavorable recovery (P < 001). The predictive value of PREP2 decreased significantly when patients had cognitive disorders (MoCA score <24) versus not (69.4% [95% CI: 52.8%-86.1%] vs 93.1% [95% CI: 83.9%-100%], P = .01).

Conclusion

Our study provides an external validation of the PREP2 algorithm in a prospective population and underlines the importance of taking into account cognitive syndromes in motor recovery prediction.

Keywords

stroke motor recovery cognitive syndromes prognosis

Introduction

Upper limb function is commonly impaired after stroke. Early and accurate prediction of poststroke motor recovery potential is a substantial goal since it could guide important decisions, such as therapeutic goals and discharge destination, in the initial days after stroke, It could also help inform patients and their families though they could make necessary arrangements for daily life activities after stroke.¹ The majority of the predictive models based on clinical scores showed moderate accuracy and the highest performance when predicting the recovery of patients with mild to moderate impairment.^2,3 Moreover, they often use a dichotomized outcome such as “good” or “poor” outcome/recovery. This binary prediction is neither informative enough nor accurate enough for clinicians or patients to anticipate how much the upper limb will be functional in daily tasks.⁴

To date, the Prediction Recovery Potential (PREP2) algorithm, developed by Stinear et al⁵ in New Zealand, is the most accurate and user-friendly algorithm (75% accuracy) for predicting upper limb motor recovery in the acute phase of stroke. Based on measures recorded within the first week, such as clinical scores (SAFE [sum of Medical Research Council scores for Shoulder Abduction and Finger Extension] and National Institute of Health Stroke Scale (NIHSS]), age and a measure of motor-evoked potential (MEP) using Transcranial Magnetic Stimulation (TMS), the PREP2 algorithm aims to predict upper limb motor outcomes measured by the Action Research Arm Test (ARAT) at 3 months in one of four categories: excellent, good, limited, or poor. Importantly, internal validation of the PREP2 was performed 2 years later, and the predictions remained accurate for 80% of patients at 2 years poststroke.⁶

Few studies have investigated the external validity of this algorithm in other populations to determine its generalizability.^7,8 Lundquist et al⁸ investigated the accuracy of the PREP2 at the subacute stage (13 days poststroke) in 91 patients, arguing that the main obstacle for implementing the PREP2 in clinical practice was the early time points (from Day 3 to Day 7) of the initial assessment with the SAFE score and TMS. However, the accuracy of the PREP2 decreased from 75% in the acute stage⁵ to 60% in the subacute stage of stroke. A recent study by Barth et al⁷ investigated the feasibility of the PREP2 algorithm using clinical variables only in the U.S. health care system. The overall accuracy was 61%, with a low sensitivity, especially for intermediate categories of good (18%) and limited (0%). The characteristics of misclassified patients were not studied.

A potential limitation for the generalisability of the PREP2 model is that the presence of cognitive syndromes of the included patients was not decribed, whereas these could influence motor outcomes and therefore PREP2 predictions. No information on the proportion of patients with cognitive syndromes (neglect, aphasia, or cognitive disorders) was reported in Stinear et al.^5,9 Though, cognitive impairment can indeed be a limiting factor in motor rehabilitation in terms of fatigability, understanding of exercise instructions, attentional resources and motivation.¹⁰ To date, how the algorithm works on the full spectrum of stroke survivors recruited directly in a stroke unit is still unknown, especially since the impact of cognitive disorders on the predictive performance of the PREP2 has not yet been studied.

The aim of this study was to externally validate the PREP2 when applied in a less selected population than in the construction cohort of the PREP2.⁵ The secondary aims of this study were (i) to study the main characteristics of patients misclassified by the algorithm and (ii) to evaluate the prognostic accuracy of the algorithm according to the presence of cognitive syndromes (aphasia, neglect, and cognitive disorders).

Materials and Methods

Patients with motor deficits in the upper limb underwent the assessment needed for the PREP2-prediction: age, SAFE score and NIHSS score at Day 3, and TMS between Days 3 and 7, while they were hospitalized in the stroke unit. A follow-up was planned at 3 months with an ARAT score. This assessment was performed since 2021 when we decided to implement all variables of the PREP2 algorithm in our daily routine evaluation.⁵ Inclusion in the study did not affect patients’ standard care, especially their choice of discharge location (home or rehabilitation centers) or their type of therapy. These choices were made by the clinical team (stroke specialists), who was blinded to the PREP2 prediction. Additionally, for this study and to determine the characteristics of misclassified patients and the impact of cognitive syndromes on the PREP2 performances, we recorded supplementary informations described below.

Patients

All patients aged 18 years or older suffering from acute ischemic or hemorrhagic stroke with a persistent upper limb deficit were considered prospectively for assessment in the Pitié-Salpêtrière Hospital stroke unit, Paris (from 04/2021 to 01/2023). The SAFE score was calculated on Day 3, and patients with a score <10 were included.¹¹ We excluded patients with a life expectancy of less than 6 months based on their medical history, those with life-threatening conditions due to stroke at the time of assessment and patients with upper limb functional limitation previous to stroke, due to any pathology. Oral informed consent was obtained from each participant or closest relative, and the study was approved by the French ethics committee (CER-Sorbonne Université, MR004, ID CER-2020-89). The General Data Protection Regulation (GDPR) was followed in accordance with European legislation. The data should be shared only upon reasonable request to the corresponding author and if they do not violate the GDPR and local ethics committee rules.

Clinical Evaluation

Patients were assessed at the acute phase, on Day 3 (as described in the PREP2 algorithm) and on Day 7 by the SAFE score. A measure of stroke severity was obtained with the NIHSS at Days 3 and 7 (total score and subsections of language disorders [subitems 1c+9] and neglect [subitem 11]). The Montreal Cognitive Assessment (MoCA) score was also collected between Days 3 and 7 in patients without aphasia. The follow-up assessment was performed at 3 months with the ARAT score. Recovery was defined based on the ARAT score at 3 months in four categories as previously published: excellent (ARAT 49-57 points), good (ARAT 34-48 points), limited (ARAT 13-33 points) and poor (ARAT 0-12 points).⁵

TMS Evaluation

TMS was administered between Day 3 and Day 7. Patients were seated in an armchair or bedridden when they could not. Surface electromyography (EMG) was recorded from the first dorsal interosseous (FDI) muscle of both hands using Ag/AgCl surface electrodes (Kendall, Chicopee, MA). EMG signals were amplified (×1000), bandpass filtered with a range of 0.02 to 2 kHz (Digitimer D360, Digitimer Ltd, Hertfordshire, UK), digitalized at 5 kHz via a Power 1401 data acquisition interface (Cambridge Electronics Design, Cambridge, UK) and stored on a computer for offline analysis with Signal software (version 5.02a, Cambridge Electronics Design, Cambridge, UK). Motor evoked potentials (MEPs) were elicited using a MAGSTIM 200 stimulator (Magstim, Dyfed, UK). A figure-of-eight coil with an internal wing diameter of 70 mm was used to apply a monophasic current. The coil was first held or placed tangentially to the scalp over the motor cortex. The primary motor cortex location was defined as 4 cm laterally of the midline (nasion to inion) and 2 cm upfront to the point where the midline crosses the tragus line. Then, the optimal coil position (“hotspot”) was identified as the position where TMS consistently produced the largest MEPs in the contralateral FDI muscle at a suprathreshold intensity. Three pulses were delivered at 100% maximal stimulator output (MSO) in each hemisphere. Patients were classified as MEP+ if MEPs are observed with a consistent latency in response to at least three stimuli, with FDI latencies ≈ 20 to 30 millisecond.

We additionally measured the resting motor threshold (rMT), which was defined as the lowest stimulation intensity (as a percentage of the maximum stimulation output, or MSO) necessary to evoke MEPs in the resting FDI muscle with a peak-to-peak amplitude greater than ≥50 mV in 5 out of 10 consecutive trials.^12,13 The maximum measurable amplitude of the MEP were determined peak-to-peak on the three trials at 100% MSO.

Imaging Data Processing (Supplementary Methods for MRI Acquisition, Preprocessing and the CorticoSpinal Tract Template)

Lesion masks were manually drawn on the diffusion-weighted imaging sequence acquired at the acute stage using Mricron software (https://www.nitrc.org) by a stroke specialist blinded to the clinical status. We calculated a CorticoSpinal Tract (CST) lesion load as the cross-sectional lesion load (CST-cLL), that is, the maximal percentage of a CST template overlapped by the lesion in a plane perpendicular to the axis of direction of the tract.¹⁴

Statistics

The normality of the data was tested with the Shapiro–Wilk test. As the data were not normally distributed, descriptive statistics are presented as medians and interquartile ranges (IQRs). Whenever possible, missing data for the follow-up assessment (3-month ARAT score) were estimated by one physiotherapist together with a rehabilitation physician (SM+LD) blinded to the PREP2 prediction at the time of the imputation. The ARAT score was estimated for 14 patients by reviewing their rehabilitation reports and medical records at the 3-months. This review included extracting available motor scores such as available functional scales and the Medical Research Council (MRC) score for the upper limb. Additionally, we called whenever possible the physio/occupational therapists of the patients and systematically the patients to inquire about their functional performances in their daily life and then we completed a score sheet accordingly. Twelve ARAT scores could not be estimated due to a lack of reports or because the patients were unavailable or lost to follow-up.

First, to externally validate the PREP2 algorithm, the overall accuracy as well as the sensitivity, specificity, and positive and negative predictive values were quantified by comparing the predicted and actual ARAT categories and calculating the proportion of patients whose prediction was correct or not. In addition, accuracy was calculated for each of the four recovery categories.

In the second step, to better characterize the patients misclassified by the PREP2 algorithm, discordant (misclassified) cases were studied. After confirming the nonnormality of the variable distribution using the Shapiro‒Wilk test, the characteristics of the true-positive, false-positive, true-negative, and false-negative cases were compared. Kruskall-Wallis tests were performed for quantitative variables; post hoc Dunn analysis was used when significant, and Fisher’s exact tests were used for categorical variables. Bonferroni adjustment was used due to the multiplicity of tests.

Finally, to investigate whether cognitive status impacted the accuracy of the PREP2 algorithm, subgroups of patients were defined according to (i) the presence or absence of unilateral spatial neglect (USN +/−) based on the neglect subitem of the NIHSS on Day 3; (ii) the presence or absence of language deficits (aphasia +/−) based on the language subitems of the NIHSS on Day 3; and (iii) the presence or absence of cognitive disorders depending on the MoCA score (<24 and ≥24).¹⁵ Accuracy, sensitivity and specificity were calculated with the same methods used for the whole population and compared between groups by chi² tests.

All the statistical analyses were performed with R studio (3.0) and MedCalc software (19.8). A P-value ≤.05 was considered to indicate statistical significance.

Results

Patients

One hundred forty-three patients were included between April 2021 and January 2023. One hundred seventeen patients were analyzed in the final study because of death (n = 9), loss to follow-up (n = 12), stroke recurrence (n = 1), or missing data preventing the use of the PREP2 algorithm, such as the Day 3 SAFE score (n = 3) or TMS recording (n = 1). The flow chart of the study is presented in Figure 1. The baseline characteristics are presented in Table 1. The median SAFE score on Day 3 was 5 (IQR: 0-8) points, with a median NIHSS score on Day 3 of 8 (IQR: 4-12) points, indicating broad recruitment with a wide range of severity profiles. Regarding the cognitive characteristics of the patients, 27% (n = 31) presented unilateral spatial neglect, and 31% (n = 35) had language deficits, The median MoCA score was 23 (IQR: 18-26) points (missing data, 21%).

Figure 1.

Flow chart.

Table 1.

Characteristics of the Patients Values are Median and IQR for Continuous Variables.

	All patients, n = 117	Excellent, n = 56	Good, n = 13	Limited, n = 9	Poor, n = 39	P-value
Demographic data
Age (y)	68 59-79	70 60-77	68 58-83	71 60-79	65 54-77	.88
Sex (%, n, male)	56% (65)	55% (31)	54% (7)	67% (6)	54% (21)	.91
Stroke type I/H (%, n)	81/19% 95/22	91/9% 51/5	69/31% 9/4	67/33% 6/3	74/26% 29/10	.06
Side of lesion Left (%, n)	44% (52)	41% (23)	62% (8)	67% (6)	38% (15)	.24
Motor scores
SAFE D3	5 0-8	8 6-8	7 3-7	2 0-4	0 0-0	<.0001
SAFE D7	6 0-9	9 7-10	7 6-8	2 1-5	0 0-0	<.0001
SAFE score change D3-D7	1 0-1	1 0-2	1 1-3	0 0-1	0 0-0	<.0001
NIHSS D3	8 4-12	5 2-8	7 5-10	12 10-18	14 10-19	<.0001
NIHSS D7	7 3-11	4 2-6	6 5-8	11 8-16	13 10-17	<.0001
ARAT M3	46 1-57	57 55-57	42 39-45	19 13-22	0 0-0	<.0001
Cognitive data
Neglect	27% (31)	20% (11)	23% (3)	33% (3)	36% (14)	.33
Aphasia	30% (35)	21% (12)	38% (5)	56% (5)	33% (13)	.14
MoCA	23 18-26	23 19-26	20 15-26	22 18-22	23 19-26	.45
Electrophysiological and MRI data
Lesion volume (cm³)	9.8 2.9-40.7	5.5 1.4-16.7	15.8 4.1-35.0	29.1 17.6-70.2	15.1 5.2-51.1	.02
CST_cLL (%)	71% 33-100	44% 21-71	30% 16-90	95% 50-100	100% 100-100	<.0001
MEP status (%, n, positive)	55.0% (64)	85.7% (48)	76.9% (10)	55.6% (5)	2.6% (1)	<.0001
Ipsilesional MEPmax (mV)	0.23 0.00-2.71	1.37 0.38-4.43	0.71 0.11- 3.26	0.12 0.00-0.46	0 0.00-0.00	<.0001
Controlesional MEPmax (mV)	3.95 2.18-6.23	3.70 2.43-6.00	5.43 2.70-7.43	3.41 2.09-4.36	4.14 1.81-6.31	.63
Ipsilesional rMT (% MSO)	68% 49-110	56% 43-66	59% 42-91	60% 37-110	110% 110-110	<.0001
Controlesional rMT (% MSO)	46% 40-53	45% 40-51	45% 40-48	45% 37-49	49% 42-57	.26

Abbreviations: I, ischemic; H, hemorrhagic; D3, Day 3; D7, Day 7; M3: 3 months; CST_cLL, cross sectional CST lesion load; MEP, motor evoked potential; rMT, resting motor threshold; MSO, maximal stimulator output.

All patients had an overlap between the stroke lesion and the CST, The lesion locations were as follows: subcortical (38%, n = 44), 20% (n = 24) cortical and 42% (n = 49) cortico-subcortical stroke, The lesion probability map is presented in Figure 2.

Figure 2.

Lesion probability map for left-sided lesions (n = 52) and right-sided lesions (n = 65) overlaid on T1 template images, The color map reflects the percentage of lesioned voxels from 10% to 50%, Images are presented by neurological convention: Left is Left.

Fifty-three patients (45%) had a nonevokable MEP even at 100% MSO (MEP-negative patients). The median amplitude (IQR) for all patients was 0.23 (0.00-2.56) mV, and that for MEP-positive patients was 1.90 (0.59-4.46) mV (n = 64).

At 3 months, the median ARAT score was 46 points for all patients, 56 patients (47.9%) had an EXCELLENT recovery, 13 patients (11.1%) had a GOOD recovery, 9 patients (7.7%) had a LIMITED recovery, and 39 patients (33.3%) had a POOR recovery. Comparisons between recovery categories (clinical and electrophysiological), can be found in the supplementary file.

External Validation of the PREP 2 Algorithm

In our cohort, the external validation of the PREP2 algorithm was conclusive and demonstrated very good accuracy 78.6% [95% CI: 71.2-86.1 %], (n = 92), close to the accuracy of the original PREP2 algorithm⁵ (75% accuracy according to Stinear et al⁵). The algorithm showed high sensitivity (83.0%) and specificity (72.5%). The positive and negative predictive values were 79.9% and 77.5%, respectively. Twenty-five (21.4%) misclassified patients were identified with 11 (44%) pessimistic predictions (i.e, the actual functional recovery was better than the predicted recovery) and 14 (56%) optimistic predictions (ie, the actual functional recovery was worse than the predicted recovery).

The majority of misclassifications were only displaced from one category to the adjacent category (n = 19/25, 76%). The other patients (n = 6, 24%) were inaccurately classified with a displacement of two or more categories: 3 patients were predicted to have POOR, but their actual functional recovery ended up in better categories (GOOD [n = 1] and EXCELLENT [n = 2]). The remaining three patients were predicted to have EXCELLENT but were in the LIMITED category (n = 1) or were predicted to have GOOD and ended up in the POOR category (n = 2).

Patients with EXCELLENT or POOR recovery were well predicted by the PREP2, with 87.5 % [95% CI: 78.8%-96.2%] and 94.9% [95% CI: 87.9%-100%], respectively, of correct predictions. Only 46.5% [95% CI: 19.1%-73.2%] of patients in the GOOD category were correctly predicted, and there was no correct prediction for the LIMITED category (0%).

Characteristics of Misclassified Patients

The second objective of this study was to determine the characteristics of patients who were misclassified by the PREP2 algorithm (n = 25).

Patients were classified as (i) true-positive (TP, n = 55, 47%) patients, who were correctly predicted by the PREP2 algorithm with favorable functional recovery. True-negative patients (TN, n = 37, 32%) were patients correctly predicted to have unfavorable functional recovery; (ii) false-positive patients (FP, n = 14, 12%) were patients who were not correctly predicted but had unfavorable functional recovery; and, finally, false-negative patients (FN, n = 11, 9%) were those who were not correctly predicted but had favorable recovery (Supplemental Table 1).

When comparing false-positive patients (patients who had an unfavorable recovery but were predicted to have a favorable recovery) to true-positive patients, false-positive patients were initially more severe clinically (according to the SAFE and NIHSS) explaining their difference in 3-months outcome (Figure 3). False-positive patients had similar structural damage CST-cLL (P = .86), There was a trend for a lower corticospinal excitability (smaller proportion of MEP-positive patients [P = .09] and a smaller amplitude of the MEP [0.60 mV vs 1.37 mV, P = .13]).

Figure 3.

Patients’ clinical scores according to their prediction status. (A) SAFE score on Day 3, (B) SAFE score on Day 7, and (C) NIHSS score on Day 3.

When comparing false-negative patients (patients with favorable functional recovery despite unfavorable prediction) to true-negative patients, their initial severity was lower (according to the SAFE and NIHSS). Interestingly, false-negative patients had a drastic change in the SAFE scores between Days 3 and 7 (+3 points), while true-negative patients did not (0 points, P < .001). False-negative patients had lower structural damage CST-cLL than true-negative patients (P < .001) and a higher corticospinal excitability [higher proportion of MEP-positive patients and a greater amplitude of the MEP (P < .001 for both comparisons)].

The Accuracy of the PREP2 Algorithm According to the Presence of Cognitive Syndromes

To investigate whether the cognitive status of patients could influence the predictive performance of the PREP2 algorithm, we analyzed patients with and without unilateral spatial neglect (USN+/USN−), with and without language deficit (Aphasia+/Aphasia−), and with and without cognitive disorders depending on their MoCA score (<24 and ≥24 points). SAFE and NIHSS scores according to patients’ cognitive status are presented in Figure 4. Further characteristics are shown in Supplemental Table 2 and Supplemental Results.

Figure 4.

Patients’ clinical scores according to their cognitive status: SAFE scores in the acute phase (Day 3 and Day 7) and NIHSS (Day 3) depending on (A) the presence or absence of unilateral spatial neglect, (B) the presence or absence of aphasia and (C) on the MoCA score Panel D corresponds to the ARAT score at 3-months according to the cognitive status.

The overall accuracy of PREP2 was high and similar in patients with versus without aphasia (74.2% [95% CI: 59.8%-88.8%] vs 80.4% [95% CI: 71.9%-89.1%], P = .45) and with or without USN (80.6% [95% CI: 66.7%-94.6%] vs 78.0% [95% CI: 69.1%-86.7%], P = .74). Interestingly, the accuracy of the PREP2 algorithm in patients with a MoCA score <24 was 69.4% [95% CI: 52.8%-86.1%] lower than that in patients with a MoCA score ≥24 (93.1% [95% CI: 83.9%-100%], P = .01). Accuracy in the four outcome groups according to the presence of aphasia, USN and MoCA score are shown in the Supplemental Table 3.

Discussion

In our cohort, the external validation of the PREP2 algorithm was conclusive and demonstrated very good accuracy (78.6% [95% CI: 71.2%-86.1%], [n = 92]), close to the accuracy of the original PREP2 algorithm⁵ (75% accuracy according to Stinear et al⁵). The algorithm was highly accurate at predicting extreme categories but not intermediate categories. The prediction errors were balanced between too optimistic or pessimistic predictions. The predictive value of the PREP2 score was not influenced by the presence of unilateral spatial neglect or aphasia but decreased significantly, when patients had cognitive disorders assessed by the MoCA score.

Our findings provide an external validation of the PREP2 with new insights for a better understanding of the mispredicted patient and the impact of cognitive syndromes on the performance of this decision tree.

External Validation of the PREP2 Algorithm

To our knowledge, this is the first study reproducing the same experimental design as the original PREP2. Despite having a severe population with cognitive syndromes, the algorithm remained very accurate. Our sample is representative of the stroke population since patients were enrolled prospectively and directly in the stroke unit, which facilitates the generalizability of the approach.¹⁶ The implementation of a predictive model is challenging in clinical practice,¹⁷ and the results imply that the predictive model should be tested in a representative sample of stroke patients hospitalized in a stroke unit or in a rehabilitation department and, whenever possible, in a different health care system. The accuracy of these predictive models relies strongly on the profile of patients included in those studies.¹⁸

In the literature, only one study has investigated the accuracy of the PREP2 algorithm, but the study involved a different setting (at the subacute stage). The accuracy decreased to 60% when it was applied to their population. As in our cohort, a low sensitivity for intermediate categories was found, with 35% and 33% accuracy for good and limited recovery, respectively.⁸ Indeed, no patient was correctly identified in our cohort in the LIMITED category. We had nine patients at 3 months with an ARAT score corresponding to the LIMITED category but none of them was correctly classified by the PREP2 algorithm. Three were predicted POOR, one EXCELLENT and five GOOD. All but one had a SAFE score at Day 3 below 5. Five of them were MEP+ (and predicted GOOD) and three of them were MEP negative but with a NIHSS score >7 (and predicted POOR). For the patients who were predicted in a better category (GOOD or EXCELLENT, n = 6) an explanation could be the presence of cognitive syndromes or shoulder pain which is a factor that hampers an efficient rehabilitation but appears later. For the patients who were predicted POOR, they had indeed a NIHSS above 7 but their motor recovery was actually better.

Considering the use of the PREP2 algorithm in the clinical setting, the accuracy for the EXCELLENT and POOR category is very high, making clinicians probably confident in making individual predictions, and guide rehabilitation decisions for individual patients. However, the results on the intermediate categories did not allow this kind of prediction.

A strength of our study was the investigation of the characteristics of patients misclassified by the PREP2 algorithm. When investigating the optimistic predictions (ie, the false positive patients, with unfavorable recovery while they were predicted with a better outcome), we found that these patients were more severe at baseline. Interestingly, their severity was associated with a lower corticospinal excitability despite similar CST structural damage. The initial severity and low corticospinal excitability could explain the “unexpected” evolution of these patients. The low corticospinal excitability could indicate a low recruitment of CST fibers, which may be explained by excessive inhibitory activity within the primary motor cortex. This hypothesis could account for the discrepancy between the small amplitude of the MEP and the damage to CST structural integrity.^19,20 When investigating the pessimistic predictions (ie, false-negative patients with a favorable recovery while we expected them not to), we found that these patients were less severe at Day 3 but also had a dramatic change in their SAFE score between Days 3 and 7. Introducing repeated measures of SAFE scores during the first 2 weeks could have improved the predictions and must be further explored.^21,22 Due to the small sample size of mispredicted patients (n = 25), further investigations should be conducted with a larger sample.

The Accuracy of the PREP2 Algorithm According to Cognitive Status

Patients with stroke often suffer from cognitive syndromes that must be considered since these symptoms could explain the lack of adherence to rehabilitation and, subsequently, the expected recovery.²³ The originality of our study lies in investigating the impact of cognitive disorders on predicting motor recovery after stroke using the PREP2 algorithm. We studied the characteristics of patients according to the presence or absence of three types of cognitive syndromes (neglect, aphasia, and cognitive disorders). First, the performance of the PREP2 in patients with a MoCA score <24 was lower than that in patients with a better MoCA score.

As the MoCA score is sensitive for detecting cognitive disorders, patients with disorders in information processing, difficulty concentrating, or impaired ability to plan are not well predicted and did not have the expected recovery. These cognitive components (information processing, action planification, attention, etc.) are crucial for motor control, especially when performing complex movements.²⁴ Cognitive abilities, such as attention, may also play a key role in individuals recovering from stroke, as suggested by VanGilder et al²⁵ and D’imperio et al.²⁶ Indeed, they not only are prognostic factors^15,27 but also influence patients’ ability to learn or regain motor skills during therapy.^10,28

Second, the presence of aphasia or unilateral spatial neglect did not affect the performance of the PREP2 algorithm. The best explanation is that this information (aphasia or neglect) is already present in the NIHSS score, which is one of the nodes of the decision tree, Patients with aphasia or neglect had higher NIHSS scores in our cohort, as expected.²⁹ This may also be explained by the fact that the presence of aphasia and neglect was directly based on the subitems of the NIHSS and not by more specific scores. It would be interesting to investigate whether these results would be similar when using specific scales, such as the Rapid Unilateral Neglect Screening (RUNS)³⁰ and the Language Screening Test (LAST scale).³¹

Limitations

Despite the large sample size (n = 117), the intermediate categories were not representative enough to be studied further, especially to understand the failure of the PREP2 algorithm in these patients. Another potential limitation could be the use of a binary endpoint for assessing the presence of a cognitive syndromes (aphasia and neglect), which does not take into consideration the severity of the disorder. Regarding the MoCA score, the proportion of missing data (21%) might be an issue but also highlights the challenges related to patients’ critical clinical state and fatigability in the acute phase of stroke when conducting a cognitive evaluation. Finally, we did not record the Extensor Carpis Radialis using TMS as it is recommended in the PREP2 algorithm, our classification of MEP-positive and negative patients relied only on the FDI muscle.

Conclusion

This study provides an external validation of the PREP2 algorithm with very high confidence in extreme categories in a prospective cohort and in a different health care system. This finding emphasizes the feasibility of the PREP2 algorithm in stroke units and highlights the importance of acquiring and developing expertise in TMS and MEP recordings among stroke neurologists and physiotherapists.

The study also provides additional insights into either the difficulties in predicting intermediate motor recovery categories or misclassifications. Furthermore, the impact of cognitive syndromes on motor recovery, especially cognitive disorders, need to be taken in account and explored more thoroughly with larger samples and more appropriate testing.

Supplemental Material

sj-docx-1-nnr-10.1177_15459683241270056 – Supplemental material for Prediction of Upper Limb Motor Recovery by the PREP2 Algorithm in a Nonselected Population: External Validation and Influence of Cognitive Syndromes

Supplemental material, sj-docx-1-nnr-10.1177_15459683241270056 for Prediction of Upper Limb Motor Recovery by the PREP2 Algorithm in a Nonselected Population: External Validation and Influence of Cognitive Syndromes by Sarah Millot, Lina Daghsen, Thomas Checkouri, Aymeric Wittwer, Romain Valabregue, Damien Galanaud, Jean Charles Lamy and Charlotte Rosso in Neurorehabilitation and Neural Repair

Footnotes

Supplementary material for this article is available on the Neurorehabilitation & Neural Repair website along with the online version of this article.

Author Contributions

Sarah Millot: Conceptualization; Formal analysis; Investigation; Methodology; Writing – original draft; Writing – review & editing.

Lina Daghsen: Conceptualization; Formal analysis; Investigation; Methodology; Writing – original draft; Writing – review & editing.

Thomas Checkouri: Formal analysis; Investigation.

Aymeric Wittwer: Formal analysis; Investigation.

Romain Valabregue: Formal analysis; Investigation.

Damien Galanaud: Formal analysis; Investigation.

Jean Charles Lamy: Formal analysis; Investigation.

Charlotte Rosso: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Supervision; Writing – original draft; Writing – review & editing.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research leading to these results has received funding from “Investissements d’avenir” ANR-10-IAIHU-06. Société Française de Medecine Physique et de Réadaptation (SOFMER) has granted the salary of Sarah Millot.

ORCID iD

Charlotte Rosso

References

Ackerley

Smith

Jordan

Stinear

CM.

Biomarkers of motor outcomes after stroke. Phys Med Rehabil Clin N Am. 2024;35:259-276.

Wynants

Van Smeden

McLernon

Timmerman

Steyerberg

Van Calster

Three myths about risk thresholds for prediction models. BMC Med. 2019;17:192.

Stinear

Smith

Byblow

WD.

Prediction tools for stroke rehabilitation. Stroke. 2019;50:3314-3322.

Langhorne

Bernhardt

Kwakkel

Stroke rehabilitation. Lancet. 2011;377:1693-1702.

Stinear

Byblow

Ackerley

Smith

Borges

Barber

PA.

PREP2: A biomarker-based algorithm for predicting upper limb function after stroke. Ann Clin Transl Neurol. 2017;4:811-820.

Smith

Ackerley

Barber

Byblow

Stinear

CM.

PREP2 algorithm predictions are correct at 2 years poststroke for most patients. Neurorehabil Neural Repair. 2019;33:635-642.

Barth

Waddell

Bland

Lang

CE.

Accuracy of an algorithm in predicting upper limb functional capacity in a United States population. Arch Phys Med Rehabil. 2022;103:44-51.

Lundquist

Nielsen

Arguissain

Brunner

IC.

Accuracy of the upper limb prediction algorithm PREP2 applied 2 weeks poststroke: a prospective longitudinal study. Neurorehabil Neural Repair. 2021;35:68-78.

Stinear

Barber

Petoe

Anwar

Byblow

WD.

The PREP algorithm predicts potential for upper limb recovery after stroke. Brain. 2012;135:2527-2535.

10.

Mullick

Subramanian

Levin

MF.

Emerging evidence of the association between cognitive deficits and arm motor recovery after stroke: a meta-analysis. Restor Neurol Neurosci. 2015;33:389-403.

11.

Stinear

Byblow

Ackerley

Smith

Borges

Barber

PA.

Proportional motor recovery after stroke: implications for trial design. Stroke. 2017;48:795-798.

12.

Groppa

Oliviero

Eisen

, et al. A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee. Clin Neurophysiol. 2012;123:858-882.

13.

Simis

Di Lazzaro

Kirton

, et al. Neurophysiological measurements of affected and unaffected motor cortex from a cross-sectional, multi-center individual stroke patient data analysis study. Neurophysiol Clin. 2016;46:53-61.

14.

Moulton

Magno

Valabregue

, et al. Acute Diffusivity biomarkers for prediction of motor and language outcome in mild-to-severe stroke patients. Stroke. 2019;50:2050-2056.

15.

Abzhandadze

Rafsten

Lundgren Nilsson

Palstam

Sunnerhagen

KS.

Very Early MoCA can predict functional dependence at 3 months after stroke: a longitudinal, cohort study. Front Neurol. 2019;10:1051.

16.

Steyerberg

Harrell

FE.

Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol. 2016;69:245.

17.

Connell

Smith

M-C

Byblow

Stinear

CM.

Implementing biomarkers to predict motor recovery after stroke. NeuroRehabilitation. 2018;43:41-50.

18.

Rosso

Lamy

JC.

Prediction of motor recovery after stroke: being pragmatic or innovative?

Curr Opin Neurol. 2020;33:482-487.

19.

Murphy

Corbett

Plasticity during stroke recovery: from synapse to behaviour. Nat Rev Neurosci. 2009;10:861-872.

20.

Buetefisch

Revill

Haut

, et al. Abnormally reduced primary motor cortex output is related to impaired hand function in chronic stroke. J Neurophysiol. 2018;120:1680-1694.

21.

DeGraba

Hallenbeck

Pettigrew

Dutka

Kelly

BJ.

Progression in acute stroke: value of the initial NIH stroke scale score on patient stratification in future trials. Stroke. 1999;30:1208-1212.

22.

Nijland

RHM

Van Wegen

EEH

Harmeling-Van Der Wel

Kwakkel

Presence of finger extension and shoulder abduction within 72 hours after stroke predicts functional recovery: early prediction of functional outcome after stroke: the EPOS cohort study. Stroke. 2010;41:745-750.

23.

Dutzi

Schwenk

Kirchner

Jooss

Bauer

Hauer

Influence of cognitive impairment on rehabilitation received and its mediating effect on functional recovery. J Alzheimers Dis. 2021;84:745-756.

24.

Salvalaggio

Turolla

Andò

, et al. Prediction of rehabilitation induced motor recovery after stroke using a multi-dimensional and multi-modal approach. Front Aging Neurosci. 2023;15:1205063.

25.

Lingo VanGilder

Hooyman

Peterson

Schaefer

SY.

Post-stroke cognitive impairments and responsiveness to motor rehabilitation: a review. Curr Phys Med Rehabil Rep. 2020;8:461-468.

26.

D’imperio

Romeo

Maistrello

, et al. Research article sensorimotor, attentional, and neuroanatomical predictors of upper limb motor deficits and rehabilitation outcome after stroke. Neural Plast. 2021;2021:8845685.

27.

Zhang

Bai

Chen

Cai

Predictive factors of upper limb motor recovery for stroke survivors admitted to a rehabilitation program. Eur J Phys Rehabil Med. 2020;56:706-712.

28.

Cumming

Marshall

Lazar

RM.

Stroke, cognitive deficits, and rehabilitation: still an incomplete picture. Int J Stroke. 2013;8:38-45.

29.

Grönberg

Henriksson

Lindgren

Accuracy of NIH Stroke Scale for diagnosing aphasia. Acta Neurol Scand. 2021;143:375-382.

30.

Bieth

Beauvais

Romeu

, et al. Validation of a new test for early assessment of unilateral neglect in acute stroke: the Rapid Unilateral Neglect Screening (RUNS) test. Ann Phys Rehabil Med. 2021;64:101399.

31.

Flamand-Roze

Falissard

Roze

, et al. Validation of a new language screening tool for patients with acute stroke: the language screening test (LAST). Stroke. 2011;42:1224-1229.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB