Responsiveness of the Neuromuscular Recovery Scale During Outpatient Activity-Dependent Rehabilitation for Spinal Cord Injury

Abstract

Background. The Neuromuscular Recovery Scale (NRS) was developed by researchers and clinicians to functionally classify people with spinal cord injury (SCI) by measuring functionally relevant motor tasks without compensation. Previous studies established strong interrater and test-retest reliability and validity of the scale. Objective. To determine responsiveness of the NRS, a version including newly added upper-extremity items, in an outpatient rehabilitation setting. Methods. Assessments using the NRS and 6 other instruments were conducted at enrollment and discharge from a locomotor training program for 72 outpatients with SCI classified as American Spinal Injury Association Impairment Scale grades A to D (International Standards for Neurological Classification of Spinal Cord Injury). Mixed-model t statistics for instruments were calculated and adjusted for confounding factors (eg, sample size, demographic variables) for all patients and subgroups stratified by injury level and/or severity. The resulting adjusted response means (ARMs) and 95% confidence intervals (CIs) were used to determine responsiveness, and significant differences between instruments were identified with pairwise comparisons. Results. The NRS was significantly responsive for SCI outpatients (ARM = 1.05; CI = 0.75-1.35). Changes in motor function were detected across heterogeneous groups. Regardless of injury level or severity, the responsiveness of the NRS was equal to, and often significantly exceeded, the responsiveness of other instruments. Conclusions. The NRS is a responsive measure that detects change in motor function during outpatient neurorehabilitation for SCI. There is potential utility for its application in randomized controlled trials and as a measure of clinical recovery across diverse SCI populations.

Introduction

Spinal cord injury (SCI) results in a wide range of functional limitations and abilities that change over time. Measuring these abilities and their rate and extent of change during the course of rehabilitation, however, remains a major barrier for clinical practice and SCI clinical trials. ¹ Recent clinical trials for SCI used 7 different measures (Functional Independence Measure for Locomotion, overground walking speed, 6-Minute Walk Test [6MWT], Berg Balance Scale [BBS], Walking Index for Spinal Cord Injury [WISCI], Lower-Extremity Sensory and Motor Scores, Ashworth Scale, and SF-54) to quantify performance improvement, yet few of these measures demonstrate robust psychometric properties for SCI.^2,3 Furthermore, 2 expert panels convened to determine the most appropriate measurement tool for SCI but endorsed different assessments.^4,5 Recently, the National Institute of Neurological Disorders and Stroke developed SCI Common Data Elements in which the 10-Meter Walk Test (10MWT), 6MWT, BBS, and Spinal Cord Independence Measure (SCIM) III were recommended for assessing gait, balance, and overall function, respectively. ⁶ Although consensus is important, it is difficult to attain, given the heterogeneity of motor function after SCI. Together, these issues demonstrate the need to specifically design instruments sensitive to the spectrum of SCI-induced deficits and responsive to improvements in motor function.

Common SCI measurement tools rate task performance without distinguishing compensatory versus recovered movement patterns. ⁷ Behavioral compensation is the use of atypical motor patterns, behaviors, body segments, and/or technology to make up for neurological deficits post-injury and accomplish functional tasks.^7-9 In contrast, recovery is the restoration of the neuromuscular system to regain function using normative movement patterns.^7,9 Given the wide diaspora of functional abilities post-SCI and the emerging paradigm shift away from compensation toward emphasizing recovery, a new assessment for recovery after SCI recently was developed and standardized for people with complete and incomplete SCI. ¹⁰ The Neuromuscular Recovery Scale (NRS) measures and classifies the functional capabilities of individuals with SCI relative to normative performance.

The initial version of the NRS focused on the trunk and lower extremities and demonstrated strong interrater and test-retest reliability, as well as good validity, without floor or ceiling effects for chronic SCI of all severities.^11-13 Although the NRS appears to identify functional abilities and detect differences in performance not captured by other outcome measures such as the 6MWT or BBS, some considerations remain. ¹⁰ First, the responsiveness of the NRS to recovery during subacute and chronic periods post-SCI remains unknown. Second, the initial version of the NRS did not assess the upper extremities. Recently, an expanded version of the scale (see the online appendix) was created to assess upper-extremity function and add items of higher skill for the trunk and lower extremities.

The clinical or research value of functional assessment tools requires not only strong reliability and validity but also responsiveness, which is defined as the ability to detect changes across time.^14,15 To date, most studies have only assessed the responsiveness of different measurement tools for SCI for the first 6 months after injury,^16,17 despite the fact that spontaneous improvement or improvement related to rehabilitation can occur beyond this time frame.^10,18-20 Furthermore, many of the responsiveness studies focused on tools that make measurements based on level of impairment, do not distinguish between behaviors that are compensatory versus recovered, and/or are limited in their applicability to the diverse SCI population.^21-23

There is no consensus on how best to measure responsiveness.^24-27 A variety of statistical approaches have been used, which generally determine whether the amount of clinical change adequately surpasses the inherent variability of the tool in a target population. These signal-to-noise comparisons have included paired t tests, standardized effect sizes, standardized response means, and others.^14,26,28 The standardized response mean, which has been used previously to evaluate instrument responsiveness for other neurological conditions,^26,28 has the benefit of being independent of sample size. In addition, the response mean can be adjusted for important factors that may affect clinical change but are predominantly controlled by insurance reimbursement, such as duration, dose, and frequency of treatments.

The purpose of this article was to examine the responsiveness of the expanded NRS, a recovery-based scale, using adjusted response means (ARMs) in people with complete and incomplete SCI receiving outpatient locomotor training. ²⁹ We hypothesized that the NRS would be largely responsive in detecting changes in motor function from enrollment to discharge for SCI outpatients undergoing intense, activity-dependent therapy. Furthermore, we hypothesized that the responsiveness of the NRS would exceed the responsiveness of other instruments traditionally used in SCI.

Methods

The University of Florida and each participating outpatient clinical center in the Christopher and Dana Reeve Foundation NeuroRecovery Network (NRN; Frazier Rehabilitation Institute, Louisville, KY; Kessler Institute for Rehabilitation, West Orange, NJ; Magee Rehabilitation Hospital, Philadelphia, PA; The Ohio State University Medical Center, Columbus, OH; Shepherd Center, Atlanta, GA; and The Institute for Rehabilitation and Research Memorial Hermann, Houston, TX) received approval from local ethics review boards and the Office of Research Protection at the Department of Defense.

Participants

Between May 2011 and January 2013, we examined 72 individuals with SCI categorized as American Spinal Injury Association Impairment Scale (AIS) grade A to D (International Standards for Neurological Classification of Spinal Cord Injury [ISNCSCI]) ³⁰ who completed at least 20 sessions of a standardized locomotor training program across 6 outpatient NRN sites. Specific enrollment criteria for the NRN locomotor training program have been described elsewhere, ³¹ but in brief, requirements include presence of a nonprogressive spinal cord lesion above T11, completion of inpatient rehabilitation, medical clearance by a NRN physician, >18 years of age, and gradual weaning off all antispasticity and neuropathic pain medications. Demographic characteristics of all participants are provided in Table 1.

Table 1.

Demographic, Clinical, and Treatment Characteristics of the 72 Patients Enrolled. ^a

Sex
Male	57 (79%)
Female	15 (21%)
Age (years)	36 (15)
Time since SCI (years)	0.7 [0.1, 14.7]
Subacute (<6 months)	27 (37.5%)
Chronic (>6 months)	45 (62.5%)
Mechanism of injury
MVC	27 (38%)
Sport	17 (24%)
Fall	13 (18%)
Violence	6 (8%)
Medical-Surgical/Nontrauma	6 (8%)
Other	3 (4%)
Neurological level
Cervical	44 (61%)
Thoracic	28 (39%)
AIS
A	17 (24%)
B	10 (14%)
C	20 (28%)
D	25 (35%)
Upper-Extremity Motor Score	34 (14)
Lower-Extremity Motor Score	18 (19)
Sessions	60 [20, 260]
Evaluations	4 [2, 14]
Time enrolled (days)	122 [30, 434]
Cumulative treatment intensity	1.6 [0.3, 7.8]

Abbreviations: AIS, American Spinal Injury Association Impairment Scale; MVC, motor vehicle collision; SCI, spinal cord injury.

a

Values are count (percentage) for nominal variables, mean (SD) for symmetric continuous variables, and median [minimum, maximum] for skewed continuous variables.

Evaluations

Physical therapists conducted standardized patient assessments at enrollment into the NRN program and post-intervention at discharge. The standardization procedure includes training for each outcome measure; an outcome measure manual with details about implementation, scripts, and scoring; as well as monthly video reviews of cases across centers. The NRS and 6 common standardized objective measurement tools used for patients with SCI were utilized for all participants: 6MWT, 10MWT, BBS, Modified Functional Reach (MFR), and Upper- and Lower-Extremity Motor Scores (UEMS and LEMS) derived from the ISNCSCI. The NRS is shown in the appendix. The scaling for each item has been refined to improve separation between subphases within each item and improve alignment of scores between items. The NRS comprises 14 items, each scored by comparing current performance with normative performance of the task. For each item and subphase, the physical therapist looks for deviations from typical performance in the movement pattern, such as toe dragging during walking or posterior pelvic tilt when sitting at the edge of the mat. Major deviations result in lower scores. The assigned subphase reflects the highest performance using a normal, preinjury pattern. For video examples, see http://www.neuromuscularrecoveryscale.com. The total summary score is calculated as the sum of all item scores and is a numerical value (out of 161 total points) representing the amount of recovery. All testing was performed and scored according to standardized procedures.^11-13

Intervention

Locomotor training is an activity-dependent intervention that promotes functional recovery with intense practice in the treadmill, overground (typical clinical), and home/community environments. Optimal task-specific sensory input is provided to the central nervous system with the goal of facilitating neurological recovery below the injury level through activity-dependent neuroplasticity.^29,32 All NRN sites followed a standardized locomotor training protocol where each session consisted of 20 minutes of step retraining on the treadmill, a minimum of 55 minutes of total weight-bearing time on the treadmill, and 15 to 30 minutes of overground, task-specific practice outside the treadmill environment with translation to home program activities and community reintegration. ³¹ The items from the NRS with the lowest scores help guide overground task-specific practice and community reintegration. A standardized discharge algorithm, primarily based on plateau of patient progress, was used to determine discharge from the program.

Data Analysis

Descriptive statistics were calculated for all measurement tools as well as for demographic, clinical, and treatment characteristics. An exploratory factor analysis was conducted on the enrollment assessments of the NRS to examine the factor structure of the NRS. Responsiveness of the NRS and other instruments was determined by calculating ARMs: t-statistics from linear contrasts applied to mixed-effects models fit to each measurement tool, divided by the square root of the sample size. ²⁶ Marginal measures of responsiveness, including the paired t-statistic and other variations on standardized average changes in outcomes,^33,34 were not considered. These measures do not account for sources of variability in the measurement tools that are unrelated to patient response, such as patient functional status at enrollment, treatment intensity and duration, and injury characteristics. The mixed-effects model for each measurement tool included time as the primary fixed factor, dichotomized as enrollment/discharge. The model additionally included ISNCSCI AIS grade (A/B or C/D), neurological injury level (cervical/thoracic), and chronicity (subacute/chronic, with 6 months as the threshold) as fixed factors. Age and cumulative treatment intensity were included as fixed continuous covariates, and a fixed-effect interaction term was included between time, ISNCSCI AIS grade, and neurological injury level. Model intercepts were defined as random effects per patient. The inclusion of AIS grade, neurological level, chronicity, age, and cumulative treatment intensity in the mixed-effects model removes variability in the measurement tools that can be attributed to these characteristics and provides a clearer estimate of average patient response. For each measurement tool, the t-statistic for the “time” fixed factor in the model constitutes an adjusted comparison of patient scores at enrollment and discharge. Dividing this t-statistic by the square root of the sample size defines the responsiveness measure (ARM) and scales the t-statistic to have interpretation similar to effect sizes and other similar measures of responsiveness, such as the standardized response mean. Linear contrasts also were applied to calculate ARMs for subgroups of patients with different AIS grades and neurological injury levels. Residuals from the mixed-effects model for each measurement tool were tested for symmetry and normality to verify the appropriateness of using ARM to quantify responsiveness. We calculated nonparametric bootstrap 95% confidence intervals (CIs) for the ARMs and pairwise differences in responsiveness between the different instruments over 1000 Monte Carlo loops. A measure was significantly responsive if the 95% CI for the ARM excluded 0. Two measures of responsiveness were significantly different if the 95% CI for the difference in their ARM excluded 0. Cohen’s classification of effect sizes,^24,28,35-37 to which ARMs are related, was adopted to distinguish different levels of responsiveness: small (0.2-0.5), moderate (0.5-0.8), or large (>0.8). ³⁸ The smallest real difference (SRD) was calculated for each outcome measure as $\mathrm{SRD}=\mathrm{SD}\times \sqrt{\left(1-r\right)}\times 1.96$ , where SD is the observed standard deviation of the enrollment-to-discharge change and r the intraclass correlation coefficient for each outcome measure derived from the literature. ³⁹ Analyses were conducted in the open source R software environment. ⁴⁰

Results

Demographic, Clinical, and Treatment Characteristics

A total of 72 patients underwent a minimum of 20 treatment sessions and were evaluated at enrollment and discharge using 7 instruments. Demographic characteristics of all participants are provided in Table 1. The sample was largely male, with an average age of 35 years (SD = 15). Most injuries were traumatic, resulting from motor vehicle collisions, sporting incidents, or falls, with representation from all 4 AIS classifications. Chronic patients (>6 months) and those with cervical injuries were in the majority. LEMS and UEMS covered, or nearly covered, the range of possible scores (0-50 for lower, 2-50 for upper). There was substantial variability in treatment duration because the number of locomotor training sessions administered ranged from 20 to 260, and the time of enrollment ranged from 30 to 434 days. Treatment frequency ranged from 3 to 5 sessions per week based on the functional severity of the individual. Cumulative treatment intensity, defined as the sum over all evaluations of the ratio of treatment sessions to days of enrollment between the initial and final evaluations, ranged from 0.3 to 7.8. Although there were between 2 and 14 interim assessments for each patient, only initial and final assessments from enrollment and discharge were considered in this analysis of responsiveness (Table 1).

Exploratory Factor Analysis

The exploratory factor analysis of the enrollment assessments of the NRS revealed 2 primary interpretable factors, accounting for 69% of the variation in the NRS (Table 2). The first factor accounted for 46% of the variation in the NRS and consisted primarily of the trunk and lower-extremity items. The second factor accounted for 23% of NRS variation and consisted primarily of the upper-extremity items. Although the factor model indicated that a minimum of 4 factors was sufficient to describe these data (χ²₄₁ = 36.9; P = .65), the third and fourth factors were not interpretable and accounted for limited NRS variation. We interpreted the first and second factors to represent lower- and upper-extremity function, respectively, and thus included lower-extremity (NRS LE) and upper-extremity (NRS UE) scores—calculated as a simple sum of the trunk and lower-extremity items (NRS LE) and a simple sum of the 3 upper-extremity items (NRS UE)—in the analysis of responsiveness (Table 2).

Table 2.

Loading Coefficients From Factor Analysis of the Neuromuscular Recovery Scale (NRS). ^a

Item	Factor 1 (46%)	Factor 2 (23%)	Factor 3 (15%)	Factor 4 (3%)
Stand retraining	0.53	0.18	0.16	0.31
Stand adaptability	0.88	0.14	0.29	0.19
Step retraining	0.83	0.16	0.30	0.12
Step adaptability	0.89	0.21	0.22	−0.05
Sit	0.70	0.51	0.37	0.25
Sit up	0.51	0.17	0.68	0.15
Reverse sit up	0.54	0.15	0.82	−0.03
Trunk extension in sitting	0.65	0.40	0.46	0.33
Sit to stand	0.87	0.18	0.35	0.13
Standing	0.90	0.21	0.30	0.03
Walking	0.85	0.27	0.29	−0.11
Door open	0.21	0.89	0.21	0.05
Overhead	0.14	0.88	0.04	0.09
Reach and grasp	0.20	0.93	0.08	−0.02

a

Percentage values are the percentage variabilities in the NRS accounted for by each factor.

Changes in Motor Performance in Outpatients With SCI Undergoing Rehabilitation

Positive changes in motor performance were detected at discharge in outpatients that enrolled in the locomotor training program (Table 3). Summary statistics show that average NRS Total Summary Scores improved. This improvement occurred across different AIS grades and neurological injury levels. There were no floor or ceiling effects, and score ranges were separated between the different patient subgroup stratifications at both enrollment and discharge. In contrast, BBS and MFR scores improved only slightly for patients with injuries graded as AIS A or B, and the 6MWT and 10MWT could not be conducted in patients with complete injuries because none of these patients was ambulatory. Furthermore, improvements in UEMS and LEMS were incremental. Floor and ceiling effects also were detected in some patient subgroups for the LEMS and UEMS, respectively. The magnitude of improvement observed in the NRS LE score and NRS UE score was roughly proportional to the relative contribution of each subscale to the NRS total score (Table 3).

Table 3.

Summary Statistics, Overall and by AIS Grade and Neurological Level, for Assessments at Enrollment and Discharge for the Different Instruments. ^a

	NRS (161)		NRS LE (125)		NRS UE (36)		6MWT (m/s)		10MWT (m/s)		BBS (56)		MFR (cm)		UEMS (50)		LEMS (50)
Group	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge	Enrollment	Discharge
All patients (n = 72)	66 ± 32	81 ± 35	52 ± 26	66 ± 30	13 ± 9	15 ± 9	0.14 ± 0.30	0.31 ± 0.53	0.18 ± 0.38	0.40 ± 0.64	14 ± 17	19 ± 21	11 ± 8	14 ± 8	34 ± 14	36 ± 13	18 ± 19	19 ± 19
By AIS
A/B (n = 27)	43 ± 11	52 ± 12	32 ± 7	40 ± 8	10 ± 6	12 ± 7	0	0	0	0	3 ± 2	4 ± 2	4 ± 3	6 ± 3	31 ± 17	32 ± 17	0 ± 1	0 ± 0
C/D (n = 45)	80 ± 32	98 ± 33	64 ± 25	81 ± 28	15 ± 10	17 ± 10	0.23 ± 0.35	0.50 ± 0.59	0.29 ± 0.44	0.63 ± 0.71	19 ± 19	28 ± 22	14 ± 7	18 ± 7	36 ± 11	38 ± 10	29 ± 16	31 ± 16
By Level
Cervical (n = 44)	60 ± 28	76 ± 34	50 ± 24	64 ± 30	10 ± 6	12 ± 7	0.11 ± 0.26	0.30 ± 0.54	0.16 ± 0.34	0.38 ± 0.61	13 ± 17	19 ± 21	10 ± 7	14 ± 8	26 ± 11	28 ± 11	18 ± 18	19 ± 19
Thoracic (n = 28)	74 ± 36	89 ± 37	56 ± 28	69 ± 31	18 ± 11	20 ± 10	0.18 ± 0.35	0.34 ± 0.53	0.22 ± 0.44	0.43 ± 0.71	15 ± 18	20 ± 21	11 ± 9	14 ± 9	47 ± 6	48 ± 5	19 ± 20	19 ± 20
By AIS × Level
A/B, cervical (n = 16)	39 ± 9	48 ± 10	31 ± 6	39 ± 7	8 ± 5	9 ± 4	0	0	0	0	2 ± 2	3 ± 2	4 ± 3	7 ± 2	20 ± 11	20 ± 10	0 ± 1	0 ± 0
A/B, thoracic (n = 11)	48 ± 13	58 ± 14	35 ± 9	43 ± 8	13 ± 6	15 ± 9	0	0	0	0	4 ± 3	5 ± 2	5 ± 3	6 ± 3	50 ± 0	50 ± 0	0 ± 0	0 ± 0
C/D, cervical (n = 28)	73 ± 28	92 ± 33	61 ± 23	78 ± 29	12 ± 6	14 ± 7	0.18 ± 0.31	0.48 ± 0.61	0.25 ± 0.39	0.59 ± 0.67	18 ± 19	27 ± 22	14 ± 6	17 ± 7	30 ± 9	33 ± 9	28 ± 15	30 ± 15
C/D, thoracic (n = 17)	91 ± 36	108 ± 33	70 ± 28	85 ± 28	22 ± 12	23 ± 11	0.30 ± 0.41	0.53 ± 0.58	0.35 ± 0.52	0.68 ± 0.79	22 ± 20	29 ± 22	15 ± 9	20 ± 7	45 ± 7	46 ± 6	31 ± 17	31 ± 18

Abbreviations: 6MWT, 6-Minute Walk Test; 10MWT, 10-m Walk Test; AIS, American Spinal Injury Association Impairment Scale; BBS, Berg Balance Scale; MFR, Modified Functional Reach; LEMS, Lower-Extremity Motor Score; NRS, revised Neuromuscular Recovery Scale; NRS-LE, simple sum of the NRS trunk and lower-extremity items; NRS-UE, simple sum of the NRS upper-extremity items; UEMS, Upper-Extremity Motor Score.

a

Scores are presented as mean ± standard deviation. Maximum possible scores/scoring unit are in parentheses.

Responsiveness of the NRS for Outpatient Rehabilitation in SCI

The NRS was significantly responsive for all outpatients with SCI undergoing locomotor training (ARM = 1.05 [0.75, 1.35]; Figure 1). The scale also was significantly responsive for the different SCI subgroups, stratified by AIS grade or neurological injury level. Though responsiveness was similar among individuals with cervical (ARM = 1.14 [0.78, 1.53], Figure 1) and thoracic (ARM = 1.02 [0.61, 1.46]) injuries, it was higher for individuals classified as AIS C or D (ARM = 1.53 [1.13, 1.96]) compared with individuals classified as AIS A or B (ARM = 0.64 [0.22, 1.05]).

Figure 1.

Responsiveness statistics for the different measurement tools across all patients, and patients stratified by AIS grade and neurological level of impairment. Values are adjusted response means (diamonds) with 95% confidence intervals (CIs; error bars). The NRS can be classified as moderately to largely responsive for all groups based on Cohen’s classification for effect sizes: small (0.2-0.5), moderate (0.5-0.8), or large (>0.8). ³⁸ Tiers of the measurement tools within each group, based on pairwise testing via bootstrap CIs are reported on the left side. Tools in different tiers are statistically significantly different. Statistically significant differences are nontransitive, so that tools can occupy multiple tiers.

The NRS LE score exhibited slightly greater ARMs among all patients and for most patient subgroups considered here (Figure 1). Differences in responsiveness between the NRS score and NRS LE score were not statistically significant. The responsiveness of the NRS UE score was substantially lower than that of the NRS score and NRS LE score, exhibiting ARMs that were comparable with the BBS, MFR, and UEMS.

Consideration of the NRS with 6 other measurement tools revealed that overall, for all patients, the NRS was the most responsive instrument to locomotor training (ARM = 1.05, large responsiveness; Figure 1). Pairwise comparisons of ARMs showed that the NRS was significantly more responsive than the other 6 instruments. The 6MWT, 10MWT, BBS, MFR, and UEMS ranged from moderate to low responsiveness among all patients (ARM = 0.40, 0.45, 0.53, 0.54, 0.21, respectively). The ARM for the ISNCSCI LEMS was low (ARM = 0.15) but not statistically different from the ARM for the 6MWT, 10MWT, and UEMS. The NRS was also the most responsive instrument in subgroups of patients categorized by AIS grade or neurological level of injury. Responsiveness for the NRS among AIS C and D patients was large (ARM = 1.53 [1.13, 1.96]) and significantly greater than the responsiveness for all other measurement tools. The 6MWT, 10MWT, BBS, and MFR occupied a second tier of responsiveness, in that they were significantly less responsive than the NRS, equivalently responsive to each other, and significantly more responsive than the UEMS and LEMS, which occupied the lowest tier of responsiveness. The NRS was moderately responsive (ARM = 0.64 [0.22, 1.05]) among AIS A and B patients. It also was the most responsive instrument in this group but was not significantly more responsive than the MFR (ARM = 0.33 [−0.05, 0.73]). The 95% CIs for the MFR, BBS, UEMS, and LEMS each contained the value 0, showing that these measures may be unresponsive to locomotor training for individuals with AIS A and B injuries.

The NRS was largely responsive in groups of patients with cervical and thoracic injuries. Among patients with cervical injuries, the NRS was most responsive (ARM = 1.14 [0.78, 1.53]) and significantly more responsive than all other instruments (Figure 1). The remaining 6 measurement tools were equivalently responsive to locomotor training in patients with cervical SCI, per the results of pairwise testing. The NRS also was most responsive among patients with thoracic injuries (ARM = 1.02 [0.61, 1.46]) but was not significantly more responsive than the 10MWT (0.45), BBS (0.52), or the MFR (0.51). Sample sizes were prohibitively small to compare the outcome measures in the AIS × Neurological Level subgroups by bootstrapping CIs. We did note that the NRS exhibited the largest responsiveness statistics in each of these subgroups of patients.

Smallest Real Difference

The observed changes in the NRS and NRS LE were 4.1 and 4.9 times greater than their respective SRD, ratios that far exceeded the same ratios calculated for the other measures (Table 4). In all, 86% of patients exhibited “real change” in the NRS and NRS LE, defined as a change exceeding the SRD of a given measure. The percentage of patients exhibiting real change exceeded 50% for only the MFR (69%) and BBS (54%) among the other measures. The NRS UE exhibited a lower mean change-to-SRD ratio (1.5) and lower percentage real change (46%) than the full NRS and NRS LE (Table 4).

Table 4.

Smallest Real Difference (SRD) for the Different Measurement Tools.

		Smallest Real Difference
Measure	Mean Change	Estimate	Change/Estimate Ratio	Percentage Real Change
NRS	16.4	4.0	4.1	86%
NRS LE	14.4	3.0	4.9	86%
NRS UE	2.0	1.3	1.5	46%
6MWT	0.156	0.086	1.8	29%
10MWT	0.215	0.105	2.1	32%
BBS	6.2	2.5	2.5	54%
MFR	3.3	1.4	2.3	69%
UEMS	1.4	1.3	1.1	20%
LEMS	1.1	1.3	0.9	14%

Abbreviations: 6MWT, 6-Minute Walk Test; 10MWT, 10-m Walk Test; BBS, Berg Balance Scale; LEMS, Lower-Extremity Motor Score; MFR, Modified Functional Reach; NRS, revised Neuromuscular Recovery Scale; NRS-LE, simple sum of the NRS trunk and lower-extremity items; NRS-UE, simple sum of the NRS upper-extremity items; UEMS, Upper-Extremity Motor Score.

Model Evaluation

The hypothesis of residual normality, an assumption made by the linear mixed-effect model from which ARMs were calculated, was rejected for all measures (Shapiro-Wilk test, P < .02) except the NRS (P = .64). Among the 6 measures for which the normality assumption failed, tests of skewness showed that residuals were not skewed (D’Agostino test, P > .15) but did exhibit excess kurtosis (Anscombe-Glynn test, P < .02). The nonnormality of the model residuals for these 6 measures was thus attributable to their symmetric, yet heavy-tailed, distribution.

Discussion

The NRS is an expanded version of the original NRS scale^10-13 in which upper-extremity items and more skillful items for the trunk and lower extremities were added. The addition of the upper-extremity items was based, in part, on our previous report, which showed significant changes in upper-extremity strength following locomotor training. ¹⁸ Emphasis on recovery of normative function across different motor tasks for the lower and upper extremities and trunk makes the scale unique. The NRS was sensitive across a spectrum of impairment levels, and unlike other scales, it does not require a minimum level of functional capacity. ¹⁰ In the current study, we extended the psychometric properties of the NRS^11-13 by assessing the responsiveness of the expanded scale. The major findings demonstrate that the NRS was more responsive than other outcome measures tested concurrently in an outpatient rehabilitation setting. Greater responsiveness occurred overall as well as across different AIS grades and neurological injury levels.

The NRS was most responsive for patients classified as AIS C or D (1.53) compared with patients classified as AIS A or B (0.64). It is possible that injury severity could have influenced the therapeutic potential of the intervention and/or spontaneous recovery. These results may, in part, also be explained by the impact of the trunk assessment on the scale. It is interesting to note that a large increase was detected in MFR scores for the group with motor-complete cervical SCI, but improvement in the NRS UE scores for this group was absent. One potential explanation of this is that good trunk alignment and posture are required for several items on the NRS, specifically sitting and the 3 upper-extremity items, to advance to higher levels. This is also true for other items, including standing and walking. Poor trunk posture undermines the kinematics of the upper extremity and stabilization of the pelvis when upright. Thus, individuals with AIS A or B injuries likely have more difficulty recovering good trunk posture and may plateau at lower phase levels in these items. For example, recovery of hand and finger function will not be detected with the NRS until good trunk control and scapular movements have returned. Recovery of the trunk is embedded in 32% (51 of 161 items) of the NRS, which ultimately limits the number of points individuals with impaired trunk control may achieve and could negatively affect the responsiveness of the tool. Alternatively, the NRS may be less sensitive in detecting the type of changes that occur in more severe injuries (ie, AIS A/B), thereby accounting for the lower responsiveness detected in this group. Comparisons of the NRS to more in-depth measures of trunk control after SCI ⁴¹ are needed to validate and potentially refine the scale.

In testing motor function and mobility, lower responsiveness occurs as time from SCI increases and when injury severity is high (ie, AIS A and B).^16,21 Based on the model we used to estimate responsiveness, we cannot comment on the responsiveness of the NRS as a function of time since SCI or duration of rehabilitation. However, similar to published reports assessing other instruments,^16,21 we found that the responsiveness of the NRS generally was highest for individuals with injuries classified as AIS C and D. Low levels of responsiveness have been reported for the SCIM II, 10MWT, and WISCI II for AIS A and B groups between 6 and 12 months after SCI, ²¹ and we report lower levels of responsiveness for AIS A and B groups for the NRS. This trend likely reflects that individuals with motor-complete injuries demonstrate floor effects on impairment-based scales (eg, LEMS or UEMS). Additionally, scales that require a minimum level of function (eg, 6MWT or 10MWT) are rarely used in AIS A and B classifications, which reduce responsiveness. For example, in this study, the AIS C and D patients who underwent locomotor training showed improvements in gait speed during the 6MWT and 10MWT, but the AIS A and B patients were unable to complete these assessments. Finally, levels of improvement and recovery may be influenced by the more limited functional status in more severe SCI, producing lower responsiveness.

The responsiveness of the NRS was lower for patients with impairment classifications of AIS A and B, but the NRS remained more responsive in this group than the other measures we tested concurrently in this study. Other traditional objective outcome measures (6MWT, BBS, MFR) are typically used to measure an impairment or functional limitation (eg, balance, fall risk). The NRS differs from those outcome measures because it relates to recovery of the nervous system in terms of postural control and movement for people while completing a functional task. The scale responds well because it measures small increments of change over time with different phase levels. For example, patients with AIS A or B classified impairments who may be unable to walk or perform items of the BBS may show more change on the NRS because it allows patients to progress and recover from the trunk down, working with their neuromuscular resources that have been stimulated through locomotor training.

Methodological factors of our analyses warrant consideration. Although the sample size of the overall analyses was large, some subgroup analyses contained slightly fewer patients. It is, therefore, possible that these reduced sample sizes may have influenced the accuracy of our estimated responsiveness statistics. In this article, we have not addressed what has been referred to as external responsiveness, in which observed improvements in a candidate instrument like the NRS are compared with improvements in other related measures. We believe that the NRS is substantially different at the measurement construct level from the other outcome measures we considered. As such, an analysis of external responsiveness against these other measures would be difficult to interpret—that is, a strong association between the NRS and these other measures may not be evidence of external responsiveness, and a weak association between the NRS and these other measures may not be evidence of the absence of external responsiveness. Because of the linear mixed-effects model used in this study and the assumptions it makes, it is also possible that the ARMs for the 6 traditional outcome measures may have been underestimated. The mixed-effects model makes the assumption that residuals are normally distributed, but tests of normality failed for all measurement tools except the NRS. Though residuals from these models were symmetric, they were heavy tailed, suggesting that tests of the fixed effects may have been conservative and, thus, the ARMs conservatively estimated. ⁴² However, the impact of this potential underestimation on our comparative conclusions—that the NRS was more responsive than the 6 traditional outcomes—likely was limited because of the large magnitudes of difference between the responsiveness of the NRS and the other instruments.

Factors influencing the greater relative responsiveness of the NRS remain speculative. However, the underlying measurement constructs comprising the NRS and other instruments may provide one explanation. It is possible these constructs are distinct or overlap minimally. This explanation seems reasonable considering that the NRS is focused on function, in contrast to impairment-based measures such as the LEMS and UEMS. ¹⁸ Preliminary factor analyses of the NRS suggest that the instrument comprises at least 2 constructs representing items focused on the lower extremities and trunk versus the upper extremities. Because of the low percentage variability accounted, it is difficult to comment on the third factor identified, necessitating future investigation. However, it is interesting to note that items relying on the trunk were the items contributing most to the third factor. Changes in the NRS were not accompanied by changes in motor scores, and we previously reported that other functional outcome measures such as the 10MWT and 6MWT were not associated with motor scores. ¹⁸ This adds further support that recovery of functional movement may not necessarily be accompanied by gains in motor strength. Additionally, the NRS can be used regardless of functional ability, unlike instruments such as the 6MWT, 10MWT, BBS, or MFR. Finally, the shared emphasis placed on recovery of normal function for the NRS and the study’s rehabilitation intervention (locomotor training) must also be taken into consideration. The generalizability of the responsiveness of the NRS to other rehabilitation interventions is unknown. However, the responsiveness of the NRS exhibited in this study might have been even higher had we used an intervention for both the upper and lower extremities. Nevertheless, it is interesting that high responsiveness occurred with locomotor-based intervention. Musselman and Yang ²³ found that the 6MWT, 10MWT, and Spinal Cord Injury Functional Ambulation Profile demonstrated moderate responsiveness to other forms of walking training, including endurance (treadmill) training and precision (overground) training. ²³ Based on the relative responsiveness of the NRS in comparison to the 6MWT and 10MWT, it therefore seems reasonable that the NRS will also be responsive to other forms of rehabilitation. In the future, however, it will be critical to assess whether the large level of responsiveness for the NRS is maintained across other interventions, varied intensities of rehabilitation (eg, European programs, where length of rehabilitation is substantially longer), and different clinical settings (eg, inpatient rehabilitation).

Conclusions

The NRS provides an additional perspective with which to assess outcomes associated with motor function following SCI. The added value of the NRS is that it assesses different levels of the neuraxis through upper- and lower-extremity and trunk items, is sensitive across a wide spectrum of SCI deficits, and focuses on recovery versus impairment or compensation using a standardized algorithm. It is important to note that the responsiveness for the NRS was high compared with that for the other instruments. This, combined with the fact that this vast data set was collected from a large sample receiving a standardized intervention (locomotor training) in a relatively short timeframe, also suggests that translational use of the NRS for clinical trials may be feasible. Furthermore, it is possible that NRS scores and the functional changes that they can capture may offer some utility in complementing other assessment tools and guiding treatment interventions. To further determine the efficacy of the NRS for application in these contexts, the instrument will continue to be refined and psychometric properties further investigated as the scale evolves.