Sage Journals: Discover world-class research

Abstract

Background

The Fugl-Meyer Assessment-Upper Extremity (FMA-UE) is a widely used outcome measure for quantifying motor impairment in stroke recovery. Meaningful change (responsiveness) in the acute to subacute phase of stroke recovery has not been determined.

Objective

Determine responsiveness and sensitivity to change of the FMA-UE from 1-week to 6-weeks (subacute) after stroke in individuals with moderate to severe arm impairment who received standard clinical care.

Methods

A total of 51 participants with resulting moderate and severe UE hemiparesis after stroke had FMA-UE assessment at baseline (within 2 weeks of stroke) and 6-weeks later. Sensitivity to change was assessed using Glass’s delta, standardized response means (SRM), standard error of measure (SEM), and minimal detectable change (MDC). Responsiveness was assessed with the minimal clinically important difference (MCID), estimated using receiver operating characteristic curve analysis with patient-reported global rating of change scales (GROC) and a provider-reported modified Rankin Scale (mRS) as anchors.

Results

The MCID estimates were 13, 12, and 9 anchored to the GROC Arm Weakness, GROC Recovery, and mRS. Glass’s delta and the SRM revealed large effect sizes, indicating high sensitivity to change, (∆ = 1.24, 95% CI [0.64, 1.82], SRM = 1.10). Results for the SEM and MDC were 2.46 and 6.82, respectively.

Conclusion

The estimated MCID for the FMA-UE for individuals with moderate to severe motor impairment from 1 to 6-weeks after stroke is 13. These estimates will provide clinical context for FMA-UE change scores by helping to identify the change in upper-extremity motor impairment that is both beyond measurement error and clinically meaningful.

Keywords

minimal clinically important difference stroke rehabilitation upper extremity ROC curve

Introduction

Accurate assessment of upper extremity (UE) stroke motor impairments is vital for treatment planning and the evaluation of clinical and research treatment efficacy. One of the most widely used UE outcome measures in stroke motor recovery^1,2 is the Fugl-Meyer Assessment-Upper Extremity (FMA-UE).³ The FMA-UE is an impairment-level measure that quantifies clinical observations of the stages of motor recovery after stroke,^4,5 and is considered the gold standard^6-8 for evaluating UE motor recovery. The FMA-UE has high reliability,^9-11 strong validity,¹¹ and is sensitive to motor gains resulting from many different interventions.^12-15 Statistically significant changes in FMA-UE scores are particularly important in determining the effectiveness of research interventions.^12-15

It is known that reliance on statistical significance alone without the context of meaningful improvement fails to reveal important insights into the clinical utility and perceived benefits of treatments. Accordingly, sensitivity to change and responsiveness estimates can provide context to change scores. Sensitivity to change refers to the ability of a measure to detect change that exceeds what can be attributed to chance or error. Sensitivity can be assessed by sample variation (using effect size calculations [Glass’s delta] and the standardized response mean) and measurement uncertainty (using the standard error of measurement [SEM] and minimal detectable change (MDC)).^16,17 However, sensitivity alone does not reveal whether the statistical change is considered relevant or clinically meaningful. Responsiveness is the ability of an assessment to quantify clinically meaningful change and can be estimated using the minimal clinically important difference (MCID). The anchor-based MCID is the threshold estimate of the smallest difference in change scores that participants, providers, or care partners perceive as beneficial.¹⁸ Changes greater than or equal to this threshold value are considered to be clinically important. Furthermore, if an MCID estimate exceeds the MDC, there is confidence that the amount of change achieved is beyond measurement error. Neither MCID (responsiveness) nor MDC (sensitivity) for the FMA-UE has been defined during the acute to subacute stages of recovery after stroke.

This study aimed to determine the responsiveness and sensitivity to change of the FMA-UE from the acute (i.e., 1 week) to subacute (i.e., 6 weeks) phase after ischemic stroke in patients receiving standard clinical care. The FMA-UE MCID has been previously defined in the chronic stroke population.¹⁹ However, limited studies have examined the MCID within the acute to subacute recovery phase^20,21 when the majority of patients transition from acute hospitals to inpatient rehabilitation, the primary clinical focus is intense rehabilitation, and the greatest changes in natural motor recovery occur.^22-25 We constrained our analysis to individuals with moderate and severe UE motor impairment^26,27 to avoid outcome measure ceiling effects and because MCID estimates are known to vary by baseline severity.^28-30 Indeed, evidence has indicated that individuals with moderate to severe impairments recover according to a distinct trajectory compared to those with mild impairment.^31-33

Methods

Study Design and Cohort

Patients were recruited as part of the Stroke Motor reHabilitation and Recovery sTudy (SMaHRT), an ongoing, prospective, natural history, single-site, longitudinal cohort study of UE motor recovery after acute stroke (www.clinicaltrials.gov, NCT03485040). From September 2017 to March 2020, eligible patients who met the following inclusion criteria were recruited to the study from the inpatient stroke service at Massachusetts General Hospital in Boston, MA: (1) age between 18 and 90 year, (2) UE motor weakness after ischemic stroke as defined by a score of ≥1 on the National Institutes of Health Stroke Scale (NIHSS) Arm Motor Drift Questions 5A or 5B, (3) ability to follow simple commands in English, and (4) stable medical status as determined by the primary care team. Exclusion criteria included: (1) prior history of developmental, neurologic, or major psychiatric disorder resulting in functional disability, (2) prior history of visual or auditory disorders limiting ability to participate in testing, or (3) evidence of an uncontrolled seizure disorder. Only individuals classified as having moderate to severe impairment, defined here as a baseline FMA-UE score of ≤ 47,³⁴ and evaluated within 14 days of acute stroke were considered. All participants provided written informed consent before participation in the study. This study was approved by the Mass General Brigham Institutional Review Board (Protocol 2017P000868). This research was conducted in accordance with the use of human participants in research as outlined in the Declaration of Helsinki.

Measures

The following measures were administered by trained occupational therapists or research staff at baseline and again at 6-week follow-up.

Fugl-Meyer Upper Extremity Assessment

The FMA-UE is an impairment index explicitly and specifically designed to assess motor recovery of the UE after stroke. The UE section of the FMA consists of 33 items and is scored on a 3-point ordinal scale with 0 meaning cannot perform, 1 meaning can partially perform, and 2 meaning can perform fully. Scores are summed with a maximum potential score of 66 points.³ The FMA-UE has demonstrated excellent test-retest and inter- and intra-rater reliability, and evidence for its content validity in acute and subacute populations have been widely reported.^9-11,35

Global Rating of Change Scale

At follow-up, participants were asked to rate the amount of change they perceived in their arm weakness and overall recovery using the following question, “With regard to recovery from your stroke, if on your worst day while in the hospital, you were a “10” (with 10 being the worst and 0 being no symptoms at all), what would you rate your level of arm weakness today (Global Rating of Change Scale [GROC], Arm Weakness) and what would you rate your level of overall recovery (GROC Overall Recovery) today?” Individuals were presented with a 10-point Likert scale accompanied by a visual aid for response selection (Supplemental Figure 1). To determine the MCID, we examined the GROC Arm Weakness as an anchor in relation to the FMA-UE as our primary outcome. The GROC Overall recovery was assessed as a secondary anchor to determine whether improvements in UE impairment, as measured using the FMA-UE, are associated with changes in overall recovery.

Modified Rankin Scale

A simplified Modified Rankin Scale (mRS) questionnaire³⁶ was administered and assessed as a secondary anchor. The mRS is a provider-reported assessment of global disability for individuals post-stroke. Clinicians use the mRS to assess functional independence in reference to pre-morbid status. Results were recorded on an ordinal scale from 0 meaning no symptoms to 5 representing severe disability.^37,38

Statistical Analysis

Baseline data on the FMA-UE were analyzed using descriptive statistics for demographic variables, outcome variables, and to characterize baseline severity.

Sensitivity to Change

We employed 2 methods of assessing sensitivity to change: (1) sample variation and (2) measurement uncertainty, the latter of which is due to inherent measurement errors of the instrument. Sample variation was assessed using Glass’s delta (∆) and the standardized response mean. Measurement uncertainty was assessed using the SEM. The SEM estimate can, in turn, be used to calculate the MDC, which is a statistical estimate of the smallest amount of change that can be detected by a measure that exceeds what can be attributed to error or chance (considered “real”).^16,17 The methods that were used to calculate sensitivity to change estimates for the FMA-UE are:

Glass’s ∆ was calculated as $(M_{2} - M_{1}) / {S D}_{b}$ where $M_{1}$ is the mean score at baseline, $M_{2}$ is the mean score at 6-weeks, and ${S D}_{b}$ is the standard deviation (SD) at baseline.³⁹ A value of 0.2, 0.5, and 0.8 represents a small, moderate, and large effect, respectively.⁴⁰

The standardized response mean was calculated as $(M_{2} - M_{1}) / {S D}_{Δ}$ where $M_{1}$ is the mean score at baseline, $M_{2}$ is the mean score at 6 weeks, and ${S D}_{Δ}$ is the SD of $(M_{2} - M_{1}) .$ ⁴¹

The standard error of measurement (SEM) was calculated as ${S D}_{b} \sqrt{1 - r_{x x}}$ where ${S D}_{b}$ is the SD at baseline and $r_{x x}$ is the test-retest reliability coefficient. Data for the test-retest reliability coefficient was obtained from previously published data for the FMA-UE.¹¹ The SEM reflects the precision of individual scores within our sample.

The minimal detectable change (MDC) with 95% confidence intervals (MDC₉₅) was calculated as $1.96 (\sqrt{2}) (S E M)$ . Change scores exceeding this value are considered to be beyond the threshold for error or chance.

Responsiveness

Receiver operating characteristic curve (ROC) analysis, a sensitivity and specificity anchor-based approach derived from diagnostic test methodology,⁴² was used to estimate the MCIDs. For MCID estimations, it is recommended that the patient- and provider-reported anchors (i.e., the GROCs and mRS) are at least moderately positively correlated $(\geq 0.3)$ with the change score of the outcome measure (i.e., FMA-UE) to validate the use of the anchors.^43,44 Correlations between all anchors and the FMA-UE were determined using Spearman’s Rank correlation coefficients. In the ROC analysis, the change in the FMA-UE (calculated by subtracting baseline scores from 6-week scores) was compared with the responses or change in responses in the external anchors. The GROC Arm Weakness served as the primary external anchor while the GROC Overall Recovery and mRS served as secondary external anchors. For the GROC anchors, responses at follow-up were used to dichotomize participants into 2 groups: (1) Improved (considered to have experienced meaningful change): participants who scored a scored 7 or below out of 10 on the GROCs (with 10 meaning no change and 0 meaning no more symptoms) and (2) Not Improved (considered to not have experienced meaningful change): participants who scored above 7 out of 10. Previous literature reporting MCIDs in similar populations support a GROC cut off equivalent to the smallest meaningful change.^45,46 A team of rehabilitation experts including neurologists, OTs and PTs convened, reviewed the GROC figure and prompt, and to reached the consensus opinion that a cut off score of 7, equivalent to a 3-point change on the patient-reported GROC from baseline to follow-up is most reflective of meaningful change in this population. For the mRS anchor, participants were similarly dichotomized into 2 participant groups (1) Improved (considered to have experienced clinically meaningful change): participants who obtained a positive change score of at least 1-point and (2) Not Improved (considered to not have experienced meaningful change: participants who did not obtain a positive change score of at least 1-point). This is consistent with previous MCID analyses using the mRS as an anchor in this population.²¹

Separate ROC curves were constructed to calculate the FMA-UE’s sensitivity and specificity in predicting participant GROC scores or mRS improvement. With this approach, the participant-reported responses on the GROCs or mRS served as the anchors and the FMA-UE served as the “diagnostic test” from which the sensitivity and specificity were calculated to distinguish improved from not improved. The sensitivity is the proportion of patients who reported a meaningful change on the anchor and whose FMA-UE scores are above the threshold MCID value. Specificity is the proportion of patients who did not report a meaningful change on the anchor and whose FMA-UE scores fell below the threshold MCID value. These curves were plotted by using 1- specificity estimates (false positive rate) on the x-axis and sensitivity estimates (true positive rate) on the y-axis for various change scores on the FMA-UE. The optimal MCID then represents the FMA-UE score that produced the greatest combined sensitivity and specificity, as determined by locating the point closest to the top left-hand corner of the ROC curve, or by calculating J = sensitivity + specificity − 1.^16,47 Each ROC curve also generated an area under the curve. The area under the curve represents the probability that scores will accurately discriminate between improved and not improved. An area under the curve of 0.50 suggests that the FMA-UE’s predictive accuracy is no better than chance at discriminating between those that Improved from Not Improved, as determined by their GROC or mRS score. An area under the curve of 1.0 signals perfect accuracy in distinguishing between participants, at least 0.70 is considered adequate, and at least 0.80 is considered excellent.^48,49 Additionally, sensitivity and specificity estimates, positive likelihood ratios, and negative likelihood ratios with confidence intervals were calculated for each MCID.

All statistical analyses were conducted using R Statistical Software (v1.4.1717).⁵⁰ ROC curves were generated using the ROCit R package (v2.1.1).⁵¹

Results

Study Cohort

A total of 73 patients with moderate to severe UE motor hemiparesis consented to participate in the study. Twenty-two patients were excluded from analysis for the following reasons: (4) deceased, (11) did not return for follow-up assessment, and (7) withdrew. These patients did not differ on age (P = .25), gender (P = .12), or baseline motor severity as defined by the and UE FMA-UE (P = .08) from those included in the analysis.

Fifty-one participants (31 females, 59.6%, age 62.1 ± 14.8 years [mean ± SD]) who completed both baseline (FMA-UE and mRS) and follow-up assessments (FMA-UE, mRS, GROC Arm Impairment, and GROC Overall Recovery) were included in the final report. Participants were assessed within 4.7 ± 2.3 (range 1-14) days post-stroke at baseline and at 47.6 ± 7.0 (range 35-81) days at follow-up. Basic demographic information and characteristics at baseline and follow-up (including the FMA-UE, mRS, and NIHSS) are summarized in Table 1. The distribution of FMA-UE scores for change, baseline, and follow-up assessments can be seen in Supplemental Figure 2. Thirty-nine participants were characterized as having severe impairments (baseline FMA-UE score 0-23) and 12 as having moderate impairment (baseline FMA-UE score 23-47). All participants received standard clinical care between assessment time points. Therapy hours, obtained by directed participant interviews, are reported in Table 1. Of the 51 participants included in the analysis, 41 were discharged to an inpatient rehabilitation facility, 2 to a long-term care facility, 3 to a skilled nursing facility, and 5 were discharged to home.

Table 1.

Baseline Demographic and Clinical Characteristics (N = 51).

Age (years)
Mean ± SD	62.5 ± 14.7
Median age (range)	66 [54.5-72.5]
Sex
Female	31 (60.8)
Baseline FMA-UE
Mean ± SD	12.1 + 13.2
Median [Interquartile]	6 [3-18.5]
Follow-up FMA-UE
Mean ± SD	28.4 ± 22.3
Median [Interquartile]	23 [6.5-50.5]
Baseline mRS
Mean ± SD	4.10 ± 0.50
Median [Interquartile]	4 [4-4]
Follow-up mRS
Mean ± SD	3.43 ± 0.81
Median [Interquartile]	3 [3-4]
Baseline NIHSS
Mean ± SD	10.75 ± 4.6
Median [Interquartile]	10 [7-13.3]
Follow-up NIHSS
Mean ± SD	6.9 ± 4.3
Median [Interquartile]	6 [4-8.5]
Therapy hours^a
Total occupational	21.64 ± 9.35 (4-45)
Inpatient	20.60 ± 9.89 (0-45)
Outpatient	6.61 ± 10.98 (0-35)
Home	4.61 ± 4.70 (0-18)
Total physical	21.96 ± 9.81 (3-45)
Inpatient	20.91 ± 9.94 (0-45)
Outpatient	5.75 ± 11.07 (0-35)
Home	4.11 ± 3.62 (0-12)

Demographic and clinical characteristics are summarized as median [interquartile range], N (n%), or mean ± standard deviation.

Abbreviations: FMA-UE, Fugl-Meyer Assessment of Upper Extremity; mRS, Modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale.

Therapy hours are patient-reported and consist of services across the inpatient, outpatient, and home settings summed from discharge to follow-up visit. Of the 51 participants included in the analysis, 41 were discharged to an inpatient rehabilitation facility, 2 to a long-term care facility, 3 to a skilled nursing facility, and 5 were discharged to home. Therapy hours reported as mean ± standard deviation (range).

Sensitivity to Change

Glass’s ∆ and the standardized response mean were used to assess the sensitivity of the FMA-UE to measure change in UE motor impairment within our sample. For the measures of sample variability, there were large effect sizes for both Glass’s ∆ and the standardized response mean, indicating high sensitivity to change (∆ = 1.24, 95% CI [0.64, 1.82], standardized response mean = 1.10). Results for measurement uncertainty indicated an SEM of 2.46 and an MDC of 6.82. In other words, a 7-point change on the FMA-UE is the minimum amount necessary to exceed the threshold for measurement error or chance. In our sample, 66.7% of patients met or exceeded the MDC₉₅.

Responsiveness

Responsiveness, or an instrument’s ability to measure clinically meaningful change, was assessed for the FMA-UE using a sensitivity and specificity anchor-based approach. Spearman’s Rank correlations coefficients between each anchor and the FMA-UE change score were above 0.3 (GROC Arm weakness: r_s = .60, GROC Overall Recovery: r_s = .40, and mRS: r_s = .36). ROC curves were generated to estimate the MCID of the FMA-UE using the GROC Arm Weakness, GROC Overall Recovery, and mRS as anchors (Figure 1). The MCID, estimated by identifying the point on the ROC curve closest to the upper left-hand corner (J = sensitivity + specificity − 1) was determined to be a 13-, 12-, or 9-point change on the FMA-UE when anchored to the GROC Arm Weakness, GROC Overall Recovery, and mRS, respectively. For all anchors, the area under the curve values were above 0.5, indicating the FMA-UE was better than chance at predicting meaningful change (Figure 1). The GROC Arm Weakness yielded the highest area under the curve values (0.83 95% CI [0.70, 0.91]), followed by the GROC Overall Recovery (0.82 95% CI [0.68, 0.90]), and the mRS (0.70 95% CI [0.52, 0.84]). All values demonstrated adequate to excellent discriminative ability.^48,49 Notably, the estimated MCIDs are higher than the MDC₉₅ value (ie, minimum value threshold to exceed chance or measurement error). Sensitivity and specificity values as well as positive and negative likelihood ratios are reported in Table 2.

Figure 1.

Receiver operating characteristic (ROC) curve for all anchors for different changes in Fugl-Meyer Assessment upper extremity (FMA-UE) Scale. ROC curves were constructed to calculate the FMA-UE’s sensitivity (true positive rate) and 1-specificity (false positive rate) in predicting participant-reported meaningful improvement, as defined by the GROC Arm Weakness, GROC Overall Recovery, and mRS anchors. The MCID is identified by the point on the ROC curve closest to the upper left-hand corner of the curve (denoted by arrows). This corresponds to the optimal cut-off score for identifying patients who improved from those who did not. The MCID was estimated to be 13, 12, and 9- point change on the FMA-UE anchored to the GROC Arm Weakness, GROC Overall Recovery, and mRS, respectively. The AUC for each anchor is above the 50% AUC Chance Line (gray diagonal line), indicating that the FMA-UE is better than chance at predicting meaningful change, as defined by each of the 3 anchors. The GROC Arm Weakness yielded the highest AUC values (0.83 95% CI [0.70, 0.91]), followed by the GROC Overall Recovery (0.82 95% CI [0.68, 0.90]), and the mRS (0.70 95% CI [0.52, 0.84]), all demonstrating adequate to excellent discriminative ability.

Table 2.

Results of ROC Analysis.

Anchor	Improved group (N, %)	MCID	% of scale	AUC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	+LR (95% CI)	−LR (95% CI)
GROC arm weakness	28 (54.9)	13	19.7	0.83 [0.70, 0.91]	0.82 [0.67, 0.96]	0.83 [0.68, 0.98]	4.93 [1.98, 12.26]	0.21 [0.09, 0.48]
GROC overall recovery	39 (76.5)	12	18.2	0.82 [0.68, 0.90]	0.68 [0.53, 0.82]	0.92 [0.76, 1.07]	8.1 [1.23, 53.55]	0.35 [0.22, 0.57]
mRS	30 (58.8)	9	13.6	0.70 [0.52, 0.84]	0.80 [0.66, 0.94]	0.64 [0.44, 0.84]	2.19 [1.23, 3.93]	0.31 [0.14, 0.69]

Improved Group are patients who reported a score of 7 or above on the GROC Arm Weakness and GROC Overall Recovery or at least a 1-point change on the mRS at follow-up.

Abbreviations: ROC, receiver operating characteristic curve; MCID, minimal clinically important difference; GROC, global rating of change; mRS, Modified Rankin Scale; AUC, Area under the curve; +LR, positive likelihood ratio; −LR, negative likelihood ratio.

Discussion

This study provides estimates of sensitivity to change and responsiveness of the FMA-UE in individuals with moderate and severe motor impairment receiving standard clinical care from the acute to subacute stages of stroke recovery. We report large effect sizes for the sensitivity to change values for both Glass’s ∆ and the standardized response mean, an SEM of 2.46 and an MDC₉₅ of 6.82. Anchored to the GROC Arm Weakness, we recommend a 13-pt change as the MCID for the FMA-UE to reflect acute to subacute arm motor recovery in moderate and severe stroke. Given that the calculated MCID values exceeded the MDC₉₅ value, our estimations of meaningful change are beyond measurement error.

Robust MCID estimates are critical for both rehabilitation clinical practice and research on novel interventions. Identifying the magnitude of measurement error associated with FMA-UE scores can improve the interpretation of change scores which can establish benchmarks to track patient progress. In order to increase our confidence in our responsiveness values, we used multiple anchors to estimate the MCID. In addition to our primary MCID estimate anchored on GROC arm weakness, we calculated FMA-UE MCIDs anchored to the mRS and the GROC Overall Recovery to provide an enhanced understanding of meaningful change from a provider perspective and to assess whether improvements in UE impairment are associated with changes in overall recovery. These values were 9 and 12 anchored to the mRS and the GROC Overall Recovery, respectively. For the provider-reported mRS, a 9-point change on the FMA-UE corresponds with meaningful change, which is lower than our estimations using the patient-reported GROC anchor. This discrepancy could be attributed to the known differences in the perception of meaningful recovery by patients and their providers. Additionally, the mRS scale’s focus on disability may not entirely reflect the full range of factors that are valuable from the patient perspective,⁵² especially as it relates to arm impairment.⁵³ MCID estimates derived from the GROC Overall Recovery corresponds to a 12-point change on the FMA-UE. Changes in impairment associated with perceptions of overall recovery appear to mirror perceptions of UE motor recovery. This similar estimate enforces the known associations between arm and hand motor deficits to disability after stroke.⁵⁴

Estimations of MCID values are context-specific and may be influenced by the stage of recovery post-stroke, secondary to a patient’s changing perceptions of their impairment and expectations for recovery. Thus, the interpretation of MCID estimates should be specific to the recovery stage. In chronic (mean time post-stroke 59.37 ± 63.22 months) stroke, a FMA-UE MCID of 5.25 points is often cited for patients with mild to moderate hemiparesis.¹⁹ Differences between chronic and acute to subacute values reported here are likely influenced by patient perception in the amount of improvement required to achieve a meaningful change. Differences may be due both to higher expectations for recovery in the acute and early subacute stage, and because the greatest gains in natural motor recovery occur within the early weeks post-stroke, resulting in substantial remediation in arm impairment.^22-25 This is in contrast to the chronic and later subacute stages of stroke where recovery is more stable, gains are less extensive, and smaller, incremental changes (i.e., smaller MCID values) are more likely noticeable and potentially meaningful due to lower expectations for recovery. In this study, the average baseline testing time was 4.7 ± 2.3 days since stroke and the average follow-up testing time was 47.57 ± 7.0 days since stroke. We focused on 6 weeks as the most substantial recovery of upper-extremity impairment, as measured by the FMA-UE, is commonly thought to occur over this time period.³³ For maximum clinical utility, future studies should investigate the influence of specific post-stroke time windows on sensitivity to change and responsiveness estimates. We note that there are currently notable pragmatic limitations (e.g., therapy efficiency standards and push for rehabilitation consultation for discharge planning purposes vs treatment) in administering the FMA-UE measure in acute care.

This analysis informs our understanding of what constitutes meaningful change in arm motor recovery under standard clinical care conditions during the acute to subacute transition. Other MCID estimates have ranged from 9 to 12 when calculated during the later subacute stages of stroke.^20,21 However, to our knowledge, this is the first investigation with an adequate sample size and an inclusion range aligning with the acute to sub-acute transition.^20,21 Furthermore, this is the first study to use a natural history dataset in patients undergoing standard care, and not as a secondary analysis in an interventional clinical trial. Participants in interventional clinical trials often do not mirror the full range of people with stroke. They may fail to have the same characteristics as patients seen in the clinic (e.g., level of impairment, co-morbidities, family support, and socioeconomic status) as therapeutic clinical trials often have more restrictive inclusion and exclusion criteria. Furthermore, MCIDs derived from intervention studies could be subject to inflated perceptions of improvement due to individual expectations for recovery that may ultimately impact MCID estimations. This is particularly noteworthy as there has been growing practice and importance placed on the use of responsiveness in clinical trials, with several studies indicating percent responder rate (i.e., those who achieve MCID) as a primary outcome measure.^15,55 Therefore, the use of an MCID derived from a natural history study (i.e., standard care conditions) may better reflect patient perceptions of recovery. These estimations can help researchers and clinicians to interpret observed changes in the FMA-UE to determine if clinically meaningful progress was experienced and to gain insight into efficacy of interventions.

Our study has limitations. Quantification of clinical and research outcomes will always be limited by the quality of the assessments used. However, in lieu of the development of improved measures to assess arm and hand impairments, we are presently confined to using the imperfect, yet ubiquitous measures reported here. As a field, we recognize the benefit of client-centered care, where the client is an active participant in the rehabilitative goal-setting process.

Therefore, using patient-reported measures is still the best indicator of meaningful change as it directly reflects the patient’s perspective. While the participant was asked to quantify their perceived amount of recovery on the GROC, a limitation of this study was that the participant was not asked for their qualitative perceptions relating to changes in the GROC, which future studies should consider. To help mitigate this limitation, we convened a consensus group of rehabilitation experts to discuss what score on the GROC would best approximate the smallest meaningful change in arm motor recovery to use as a cut off score in dichotomizing participants between improved and not improved. As for our other outcome measures, the limitations of the FMA-UE and the mRS have been well documented in the literature.^56,57 But these assessments, especially the former, are among the most widely used in stroke rehabilitation.^1,2 Indeed, the Neurology Section of the American Physical Therapy Association’s Stroke Taskforce and the Stroke Recovery and Rehabilitation Roundtable highly recommends the FMA-UE for use with patients with acute and subacute stroke acuity and has recommended the measure for use in stroke research.^7,8

The SEM reported here should be interpreted with caution. In this study, the SEM is calculated as a single value with the assumption that it is uniform across the entire range of FMA-UE scores studied. However, there is likely to be variation in the SEM in the middle of the scale as compared to the scores in the tails. This assumption of a uniform SEM to evaluate changes implies that our confidence levels of true change estimates may vary across the FMA-UE score continuum. Using an approximation approach,⁵⁸ we estimate that the 99% confidence interval obtained at the scale location with the lowest standard error corresponds to 61% confidence interval at the scale location with the highest standard error. The assumption of a uniform SEM is a known limitation of the SEM as estimated using classical test theory methods and future studies may consider using item response theory methods to address these limitations.

Lastly, although we constrained our analysis to the moderate-severe subgroup to account for the potential influence of baseline severity, we did not have adequate power to further stratify patients into “moderate” and “severe” subgroups to estimate separate MCIDs for these groups. Given the unique profile of patients recovering from stroke and the context-dependent nature of the MCID, future work should explore these analyses in other subgroups to further our understanding of recovery of motor impairment after stroke.

Conclusion

A 13-pt change on the FMA-UE during the acute to subacute phase of stroke recovery (acute to 6 weeks post-stroke) reflects meaningful change in arm motor impairment recovery and a change of at least 6.82-points is required to be valid beyond measurement error. These estimates will help set benchmarks for research trials and guide clinical decision-making by identifying the meaningful change in motor impairment for acute stroke patients that is both clinically meaningful and beyond measurement error.

Supplemental Material

sj-docx-1-nnr-10.1177_15459683231186985 – Supplemental material for Sensitivity to Change and Responsiveness of the Upper Extremity Fugl-Meyer Assessment in Individuals With Moderate to Severe Acute Stroke

Supplemental material, sj-docx-1-nnr-10.1177_15459683231186985 for Sensitivity to Change and Responsiveness of the Upper Extremity Fugl-Meyer Assessment in Individuals With Moderate to Severe Acute Stroke by Baothy P. Huynh, Julie A. DiCarlo, Isha Vora, Jessica Ranford, Perman Gochyyev, David J. Lin and Teresa J. Kimberley in Neurorehabilitation and Neural Repair

Supplemental Material

sj-docx-2-nnr-10.1177_15459683231186985 – Supplemental material for Sensitivity to Change and Responsiveness of the Upper Extremity Fugl-Meyer Assessment in Individuals With Moderate to Severe Acute Stroke

Supplemental material, sj-docx-2-nnr-10.1177_15459683231186985 for Sensitivity to Change and Responsiveness of the Upper Extremity Fugl-Meyer Assessment in Individuals With Moderate to Severe Acute Stroke by Baothy P. Huynh, Julie A. DiCarlo, Isha Vora, Jessica Ranford, Perman Gochyyev, David J. Lin and Teresa J. Kimberley in Neurorehabilitation and Neural Repair

Footnotes

Acknowledgements

The authors thank the SMaHRT study participants, occupational therapists, and study staff for volunteering their time and contributing to the data.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This manuscript was supported by the American Society of Neurorehabilitation Clinical Research Seed Grant-NIH K23 (David Lin, PI).

ORCID iDs

Baothy P. Huynh

Teresa J. Kimberley

Supplementary material for this article is available on the Neurorehabilitation & Neural Repair website along with the online version of this article.

References

van Wijck

FMJ

Pandyan

Johnson

Barnes

. Assessing motor deficits in neurological rehabilitation: patterns of instrument usage. Neurorehabil Neural Repair. 2001;15(1):23-30. doi:10.1177/154596830101500104

Santisteban

Térémetz

Bleton

Baron

Maier

Lindberg

. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS One. 2016;11(5):e0154792. doi:10.1371/journal.pone.0154792

Fugl-Meyer

Jääskö

Leyman

Olsson

Steglind

. The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand J Rehabil Med. 1975;7(1):13-31.

Twitchell

. The restoration of motor function following hemiplegia in man. Brain. 1951;74(4):443-480. doi:10.1093/brain/74.4.443

Brunnstrom

. Motor testing procedures in hemiplegia: based on sequential recovery stages. Phys Ther. 1966;46(4):357-375. doi:10.1093/ptj/46.4.357

Baker

Cano

Playford

. Outcome measurement in stroke. Stroke. 2011;42(6):1787-1794. doi:10.1161/STROKEAHA.110.608505

Sullivan

Crowner

Kluding

, et al. Outcome measures for individuals with stroke: process and recommendations from the American physical therapy association neurology section task force. Phys Ther. 2013;93(10):1383-1396. doi:10.2522/ptj.20120492

Kwakkel

Lannin

Borschmann

, et al. Standardized measurement of sensorimotor recovery in stroke trials: consensus-based core recommendations from the stroke recovery and rehabilitation roundtable. Int J Stroke. 2017;12(5):451-461. doi:10.1177/1747493017711813

Sullivan

Tilson

Cen

, et al. Fugl-Meyer assessment of sensorimotor function after stroke. Stroke. 2011;42(2):427-432. doi:10.1161/STROKEAHA.110.592766

10.

See

Dodakian

Chou

, et al. A standardized approach to the Fugl-Meyer assessment and its implications for clinical trials. Neurorehabil Neural Repair. 2013;27(8):732-741. doi:10.1177/1545968313491000

11.

Platz

Pinkowski

van Wijck

Kim

di Bella

Johnson

. Reliability and validity of arm function assessment with standardized guidelines for the Fugl-Meyer Test, action research arm test and box and block test: a multicentre study. Clin Rehabil. 2005;19(4):404-411. doi:10.1191/0269215505cr832oa

12.

Takahashi

Der-Yeghiaian

Motiwala

Cramer

. Robot-based hand motor therapy after stroke. Brain. 2008;131(2):425-437. doi:10.1093/brain/awm311

13.

Lindenberg

Renga

Zhu

Nair

Schlaug

. Bihemispheric brain stimulation facilitates motor recovery in chronic stroke patients(Podcast)(e–Pub ahead of print)(LOE Classification). Neurology. 2010;75(24):2176-2184. doi:10.1212/WNL.0b013e318202013a

14.

Page

Szaflarski

Eliassen

Pan

Cramer

. Cortical plasticity following motor skill learning during mental practice in stroke. Neurorehabil Neural Repair. 2009;23(4):382-388. doi:10.1177/1545968308326427

15.

Dawson

Liu

Francisco

, et al. Vagus nerve stimulation paired with rehabilitation for upper limb motor function after ischaemic stroke (VNS-REHAB): a randomised, blinded, pivotal, device trial. Lancet. 2021;397(10284):1545-1553. doi:10.1016/S0140-6736(21)00475-X

16.

Streiner

Norman

Cairney

. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford University Press; 2014. Accessed April 19, 2022. https://oxfordmedicine.com/view/10.1093/med/9780199685219.001.0001/med-9780199685219

17.

Liang

. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care. 2000;38(9):II84-II90.

18.

Jaeschke

Singer

Guyatt

. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-415. doi:10.1016/0197-2456(89)90005-6

19.

Page

Fulk

Boyne

. Clinically important differences for the upper-extremity Fugl-Meyer scale in people with minimal to moderate impairment due to chronic stroke. Phys Ther. 2012;92(6):791-798. doi:10.2522/ptj.20110009

20.

Hiragami

Inoue

Harada

. Minimal clinically important difference for the Fugl-Meyer assessment of the upper extremity in convalescent stroke patients with moderate to severe hemiparesis. J Phys Ther Sci. 2019;31(11):917-921. doi:10.1589/jpts.31.917

21.

Arya

Verma

Garg

R k

. Estimating the minimal clinically important difference of an upper extremity recovery measure in subacute stroke patients. Top Stroke Rehabil. 2011;18:599-610.

22.

Kwakkel

Kollen

van der Grond

Prevo

AJH

. Probability of regaining dexterity in the flaccid upper limb: impact of severity of paresis and time since onset in acute stroke. Stroke. 2003;34(9):2181-2186. doi:10.1161/01.STR.0000087172.16305.CD

23.

Nakayama

Stig Jørgensen

Otto Raaschou

Skyhøj Olsen

. Recovery of upper extremity function in stroke patients: the Copenhagen stroke study. Arch Phys Med Rehabil. 1994;75(4):394-398. doi:10.1016/0003-9993(94)90161-9

24.

Cortes

Goldsmith

Harran

, et al. A short and distinct time window for recovery of arm motor control early after stroke revealed with a global measure of trajectory kinematics. Neurorehabil Neural Repair. 2017;31(6):552-560. doi:10.1177/1545968317697034

25.

Borschmann

Hayward

. Recovery of upper limb function is greatest early after stroke but does continue to improve during the chronic phase: a two-year, observational study. Physiotherapy. 2020;107:216-223. doi:10.1016/j.physio.2019.10.001

26.

Mirbagheri

Rymer

. Time-course of changes in arm impairment after stroke: variables predicting motor recovery over 12 months. Arch Phys Med Rehabil. 2008;89(8):1507-1513. doi:10.1016/j.apmr.2008.02.017

27.

Rabadi

. Comparison of the action research arm test and the Fugl-Meyer assessment as measures of upper-extremity motor weakness after stroke. Arch Phys Med Rehabil. 2006;87(7):962-966. doi:10.1016/j.apmr.2006.02.036

28.

Wang

Hart

Stratford

Mioduski

. Baseline dependency of minimal clinically important improvement. Phys Ther. 2011;91(5):675-688. doi:10.2522/ptj.20100229

29.

Hays

Woolley

. The concept of clinically meaningful difference in health-related quality-of-life research. How meaningful is it? PharmacoEconomics. 2000;18(5):419-423. doi:10.2165/00019053-200018050-00001

30.

Beaton

Boers

Wells

. Many faces of the minimal clinically important difference (MCID): a literature review and directions for future research. Curr Opin Rheumatol. 2002;14(2):109-114.

31.

Bonkhoff

Hope

Bzdok

, et al. Recovery after stroke: the severely impaired are a distinct group. J Neurol Neurosurg Psychiatry. 2022;93(4):369-378. doi:10.1136/jnnp-2021-327211

32.

Kline

Lin

Cloutier

, et al. Arm motor recovery after ischemic stroke: a focus on clinically distinct trajectory groups. J Neurol Phys Ther. 2021;45(2):70-78. doi:10.1097/NPT.0000000000000350

33.

Duncan

Min Lai

Keighley

. Defining post-stroke recovery: implications for design and interpretation of drug trials. Neuropharmacology. 2000;39(5):835-841. doi:10.1016/S0028-3908(00)00003-4

34.

Woodbury

Velozo

Richards

Duncan

. Rasch analysis staging methodology to classify upper extremity movement impairment after stroke. Arch Phys Med Rehabil. 2013;94(8):1527-1533. doi:10.1016/j.apmr.2013.03.007

35.

Woodbury

Velozo

Richards

Duncan

Studenski

Lai

. Longitudinal stability of the Fugl-Meyer assessment of the upper extremity. Arch Phys Med Rehabil. 2008;89(8):1563-1569. doi:10.1016/j.apmr.2007.12.041

36.

Bruno

Shah

Lin

, et al. Improving modified Rankin Scale assessment with a simplified questionnaire. Stroke. 2010;41(5):1048-1050. doi:10.1161/STROKEAHA.109.571562

37.

Rankin

. Cerebral Vascular accidents in patients over the age of 60: II. Prognosis. Scott Med J. 1957;2(5):200-215. doi:10.1177/003693305700200504

38.

Van Swieten

Koudstaal

Visser

Schouten

Van

. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1998;19(5):604-607. doi:10.1161/01.STR.19.5.604

39.

Glass

. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3-8. doi:10.3102/0013189X005010003

40.

Cohen

. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. L. Erlbaum Associates; 1988.

41.

McHorney

Tarlov

. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 1995;4(4):293-307. doi:10.1007/BF01593882

42.

Deyo

Centor

. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986;39(11):897-906. doi:10.1016/0021-9681(86)90038-x

43.

Revicki

Hays

Cella

Sloan

. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61(2):102-109. doi:10.1016/j.jclinepi.2007.03.012

44.

Terwee

Peipert

Chapman

, et al. Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Qual Life Res. 2021;30(10):2729-2754. doi:10.1007/s11136-021-02925-y

45.

Lang

Edwards

Birkenmeier

Dromerick

. Estimating minimal clinically important differences of upper-extremity measures early after stroke. Arch Phys Med Rehabil. 2008;89(9):1693-1700. doi:10.1016/j.apmr.2008.02.022

46.

Beninato

Gill-Body

Salles

Stark

Black-Schaffer

Stein

. Determination of the minimal clinically important difference in the FIM instrument in patients with stroke. Arch Phys Med Rehabil. 2006;87(1):32-39. doi:10.1016/j.apmr.2005.08.130

47.

Stratford

Binkley

Riddle

. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76(10):1109-1123. doi:10.1093/ptj/76.10.1109

48.

Terwee

Bot

SDM

de Boer

, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42. doi:10.1016/j.jclinepi.2006.03.012

49.

Hosmer

Lemeshow

Sturdivant

. Applied Logistic Regression. 3rd ed. Wiley; 2013. Accessed September 15, 2022. https://www.wiley.com/en-us/Applied+Logistic+Regression%2C+3rd+Edition-p-9780470582473

50.

R Core Team. R: a language and environment for statistical computing. 2022. Accessed September 15, 2022. https://www.R-project.org/

51.

Khan

MRA

Brandenburger

. ROCit: performance assessment of binary classifier with visualization. 2020. Accessed September 15 2022. https://CRAN.R-project.org/package=ROCit

52.

Sanchez-Gavilan

Montiel

Baladas

, et al. Added value of patient-reported outcome measures (PROMs) after an acute stroke and early predictors of 90 days PROMs. J Patient-Rep Outcomes. 2022;6(1):66. doi:10.1186/s41687-022-00472-9

53.

Erler

DiCarlo

, et al. Association of modified Rankin Scale with recovery phenotypes in patients with upper extremity weakness after stroke. Neurology. 2022;98(18):e1877-e1885. doi:10.1212/WNL.0000000000200154

54.

Poltawski

Allison

Briscoe

, et al. Assessing the impact of upper limb disability following stroke: a qualitative enquiry using internet-based personal accounts of stroke survivors. Disabil Rehabil. 2016;38(10):945-951. doi:10.3109/09638288.2015.1068383

55.

Winstein

Wolf

Dromerick

, et al. Effect of a task-oriented rehabilitation program on upper extremity recovery following motor stroke: the ICARE randomized clinical trial. JAMA. 2016;315(6):571-581. doi:10.1001/jama.2016.0276

56.

Gladstone

Danells

Black

. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabil Neural Repair. 2002;16(3):232-240. doi:10.1177/154596802401105171

57.

New

Buchbinder

. Critical Appraisal and review of the Rankin Scale and its derivatives. Neuroepidemiology. 2006;26(1):4-15. doi:10.1159/000089536

58.

Wilson

. Constructing Measures: An Item Response Modeling Approach. 1st ed. Routledge; 2004. Accessed September 20 2022. https://www.routledge.com/Constructing-Measures-An-Item-Response-Modeling-Approach/Wilson/p/book/9780805847857

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB

0.18 MB