Abstract
Background. A quadratic formula of the Spinal Cord Injury Ability Realization Measurement Index (SCI-ARMI) has previously been published. This formula was based on a model of Spinal Cord Independence Measure (SCIM95), the 95th percentile of the SCIM III values, which correspond with the American Spinal Injury Association Motor Scores (AMS) of SCI patients. Objective. To further develop the original formula. Setting. Spinal cord injury centers from 6 countries and the Statistical Laboratory, Tel-Aviv University, Israel. Methods. SCIM95 of 661 SCI patients was modeled, using a quantile regression with or without adjustment for age and gender, to calculate SCI-ARMI values. SCI-ARMI gain during rehabilitation and its correlations were examined. Results. A new quadratic SCIM95 model was created. This resembled the previously published model, which yielded similar SCIM95 values in all the countries, after adjustment for age and gender. Without this adjustment, however, only 86% of the non-Israeli SCIM III observations were lower than those SCIM95 values (P < .0001). Adding the variables age and gender to the new model affected the SCIM95 value significantly (P < .04). Adding country information did not add a significant effect (P > .1). SCI-ARMI gain was positive (38.8 ± 22 points, P < .0001) and correlated weakly with admission age and AMS. Conclusions. The original quadratic SCI-ARMI formula is valid for an international population after adjustment for age and gender. The new formula considers more factors that affect functional ability following SCI.
Introduction
The development and advantages of the Spinal Cord Injury Ability Realization Measurement Index (SCI-ARMI) as a measure of the assessment of disability weighted for the neurological deficit in patients with SCI have been described in previous publications.1,2 The scale not only quantitatively evaluates the independent role of rehabilitation in improving function beyond that of neurological recovery alone but also assesses the potential for and the success of rehabilitation (ability realization).
Ability realization is defined as the ratio of the observed functional ability and the maximal possible functional ability gain for any level and severity of SCI. SCI-ARMI is the ability realization expressed as a percentage. The difference between 100 (the SCI-ARMI score that represents 100% ability realization) and the actual SCI-ARMI score before rehabilitation reflects the rehabilitation potential when the neurological deficit is stable. The change in ability realization during rehabilitation is considered to primarily reflect the effect of rehabilitation.
The first SCI-ARMI version was based on the regression of the linear relationship found between the American Spinal Injury Association Motor Scores (AISmo, or AMS), an internationally recognized measure of neurological deficit after a SCI, and the highest-observed values found when using the second version of the Spinal Cord Independence Measure (SCIM II) in patients with discrete AMS ranges (SCIMho); SCIMho represented the maximal possible functional achievement in each range of neurological deficit.3,4 Each individual’s observed SCIM II score (SCIMob) was divided by the linear regression (which represented SCIMho) and multiplied by 100 to calculate the SCI-ARMI formula. However, the linear relationship on which the SCI-ARMI formula was based was found not to hold over the entire AMS range. Furthermore, the method that was used to estimate the SCI-ARMI relied on the highest-observed value (which, itself, depended on the size of the group of patients within each AMS range) and was particularly sensitive to the small number of patients with SCIM and AMS scores in the lower range. These problems reduced the relevance, accuracy, and stability of the formula.
The next iteration of SCI-ARMI was based on the third SCIM version (SCIM III), rather than SCIM II, together with an alternative statistical approach that did not rely on linearity and that was not as sensitive to the number of SCI patients available for each AMS. To produce a conservative estimate for the highest possible SCIM III, for each patient’s AMS value, we used the 95th percentile of SCIM III values 3 at discharge from rehabilitation (SCIM95) for patients with a given AMS at discharge. We obtained a quadratic formula, which was found to be the best estimation of SCIM95. Dividing the individual’s observed SCIM III score by the calculated SCIM95 formula produced an improved measure of ability realization. A recent study 2 on 226 SCI patients supported the accuracy and validity of this new, modified SCI-ARMI formula.3,4 This study, however, included patients from only 1 center, in 1 country (Israel), and we were concerned that specific characteristics of care and rehabilitation in SCI centers in different countries could affect ability realization and differences in length of stay in rehabilitation, and/or in the mix of medical and emotional problems experienced, and/or in environmental conditions or facilities available within SCI rehabilitation facilities. Furthermore, the formula reflected the effect of motor deficit on ability realization but not the effects of such other factors as patient’s age, gender, neurological level, length of stay in rehabilitation, and country.
It was therefore hypothesized that, regardless of differences in the care and rehabilitation characteristics between centers, the SCI-ARMI formula would be valid for various countries, provided it is controlled for any major factors affecting it. To examine this hypothesis, the present study examined the validity of the current formula in a larger patient group from 6 different countries and the effects of several of these potential factors to further develop the formula.
Materials and Methods
Observed admission and discharge SCIM III and AMS scores were obtained from the records of traumatic and nontraumatic SCI in patients (Table 1), treated in 6 different countries. Inclusion criteria were as follows:
a SCI (AIS grades A, B, C, or D) and
age >18 years.
Exclusion criteria were as follows:
patients with concomitant impairments such as brain injury or significant mental impairment;
any condition, other than those caused by SCI, that might influence everyday functional ability;
more than 2 weeks elapsing between SCIM or AIS examination and the corresponding dates of admission or discharge; and
more than 1 week elapsing between the SCIM and AIS examinations either at admission or at discharge.
Patient Data Set: Neurology and Cause of Injury/Impairment.
Initial analysis of the complete data set showed that a total of 189 out of 12 559 values in the 19 SCIM III tasks at admission and 36 out of 12 559 values in the 19 SCIM III tasks at discharge were missing values (1.5% and 0.2%, respectively). These were completed using the imputation method of k nearest neighbors with k = 2. The method calculates the Euclidian distance between 2 participants, and whenever a participant had a missing value, the 2 “nearest” participants’ values (ie, most similar, based on their SCIM III values) were averaged in order to replace the missing values. The missing values of the SCIM III tasks were scattered throughout the data file, and it was assumed that they were missing completely at random.
In addition, 253 patients (out of 330) from the United States were excluded from the study because of not having their AMS data collected at discharge. This data loss was also assumed to be missing at random. The sensitivity of the main SCIM95 analysis was revisited to check the appropriateness of this assumption by imputing these missing values with admission AMS, using a median regression model (with some bias correction adjustment), and rerunning the main SCIM95 analysis with the data set after imputation. When refitting the formula of SCIM95 and comparing the 2 predictions over the entire AMS range (for a representative 50-year-old woman), the 2 predicted SCIM95 lines were very similar to one another, with an average difference of 0.22 SCIM points, differing by −1.16 points at most. The mean age was compared between countries using a 1-way ANOVA followed by Tukey’s pairwise comparisons. The proportion of gender was compared between countries using a χ2 test followed by Holm’s adjustment for the multiplicity of the pairwise comparisons.
An estimate for the highest possible SCIM III, given the patient’s AMS value, was obtained using the 95th percentile of SCIM III values at discharge from rehabilitation (SCIM95) for patients with any given AMS at discharge, as was also undertaken in the authors’previous recent study. 2 The reason for choosing discharge values was that at discharge from rehabilitation, the ability realization score was the highest presumed by the authors to be attainable, and the SCIM95 equation specifies the highest SCIM III score that 95% of patients with similar AMS can achieve.
Although the authors’ experience is that patients usually achieve the highest SCIM score for given neurological states at this point in time, we acknowledge that function may be further enhanced after discharge in some individuals; a further study would therefore be necessary in the future to assess the need to correct SCIM95 accordingly.
The statistical software environment R (version 3.0.1) 5 was utilized to estimate SCIM95 using a range of formulas, implementing the quantile regression method for linear and quadratic formulas. 6 The assessment of the effects of AMS, gender, age, and country (as main effects and their interactions) was performed by serially comparing nested models using tests of linear hypotheses based on regression rank scores. 7
The relationship of the 95th percentile of the SCIM III values for patients with given AMS at discharge from rehabilitation (SCIM95) and the corresponding AMS value was modeled in linear and quadratic formulas. These formulas estimated the effect of AMS alone on SCIM95 or the combined effect of AMS and the additional potentially explanatory factors: age, gender, and country of origin.
The SCIM95 formula for estimating the effect of AMS alone, based on all countries, was graphically compared with the corresponding formula calculated for patients from Israel in the authors’ previous study. 2 A binomial test was also used to check the validity of the previously published SCIM95 formula. The binomial test checked whether 95% of the observed SCIM values in the other 5 countries were comparable with or below the value predicted by the original SCIM95 model. The new SCIM95 formula was used to calculate SCI-ARMI values, defined as the ratio of the observed SCIM score to the respective SCIM95 value.
The relationship of AMS and SCIM III values at discharge was assessed using Spearman’s correlation test. The 95% confidence interval for Spearman correlation was obtained using the percentile bootstrap method, as was the P value, which coincides with the P value obtained by the usual normal approximation for the permutation distribution.
The significance of SCI-ARMI gain during rehabilitation was examined using Student’s t test. The correlation of SCI-ARMI gain with age, AMS, and SCI-ARMI at admission was examined using Pearson’s correlation measure.
Results
A total of 661 patients met the inclusion criteria and were included in the analysis; 254 patients were excluded. The majority of exclusions (n = 253) occurred because of missing AMS data at discharge from the US data set. In addition, 1 patient with extremely disparate overall SCIM and AMS scores (AMS of 34 and SCIM-III of 85) was also excluded. In 60 of the US patients which met the inclusion criteria, 225 missing values of SCIM tasks had to be imputed. Out of 661 participants, 22 participants with an admission SCIM item containing a higher value than is possible and 51 participants with a discharge SCIM item containing a higher value than is possible were identified. These were amended to reflect the true maximum value possible.
The number of patients included from each country, their age, and their gender distribution are detailed in Table 2. The mean age at admission was 47.6 years (standard deviation [SD] = 18.2). The mean age significantly differed between countries (P < .0001). The age in Portugal and Israel was significantly higher than in Italy and in the United Kingdom (P < .05). Men comprised 72% of the patients, and the gender composition significantly differed between countries (P < .03).
Basic Patient Data.
Abbreviations: AMS, American Spinal Injury Association Motor Scores; SCIM, Spinal Cord Independence Measure.
Mean AMS value at discharge was 69 (median = 77; SD = 24.4); mean SCIM III value at discharge was 64.1 (median = 68; SD = 21.7). AMS and SCIM III values at discharge were moderately correlated (r = 0.71; P < .0001). Both AMS and SCIM III scores, however, have different distributions in different countries (P < .0001). After fitting the linear and quadratic formulas for total SCIM95 as a function of AMS alone, the quadratic formula was found to be better than the linear (P = .012) and is given by
Comparing the graphic representation of this formula with that of the formula developed in the authors’ previous study (Figure 1) yielded similar predictions for the 95% quantile. The major difference between the previous and current formula is in the prediction given for lower values of AMS. However, when using the model from the authors’ previous research on the current data from the 5 new countries, 86.6% (0.95 CI = 83.1%-89.7%) of the observations from within the 5 countries were below the prediction of the quadratic model based on data from Israel’s patients (P < .0001). It was, therefore, postulated that this underestimation may well be the result of the differences in age and gender composition between the countries. To explore this possibility, the quadratic model based on the data from all 6 countries was extended by sequentially adding age and gender and inspecting their effect on the predicted SCIM95 values. Age at admission (in years) had an additional significant effect (P = .0014) on SCIM95, and with no interaction between AMS and age (P = .632). The age-adjusted SCIM95 formula (Figure 2) is
Comparison of current and previous Total SCIM95 values for each AMS at discharge (AMSd). The considered effect on SCIM95 was of AMS alone. SCIM95 values are close in both studies, but only 86% (95% CI = 83%, 89.7%) of the observations in the current study are below the previous SCIM95 line.
Age effect on SCIM95: SCIM95 is higher in younger patients. The ages we chose were the 5%, 50%, and 95% quantiles of the participants’ ages in our data set and were chosen to allow clean visualization of age effect.
Adding gender to this formula (inserting 1 for female and 0 for male) showed a further statistically significant effect on SCIM95 (P = .037) and no interaction with AMS and age (P = .1652). The obtained age and gender-controlled formula (Figure 3) is
Age and gender effect on SCIM95: SCIM95 is in higher in males. The ages we chose were the 5%, 50%, and 95% quantiles of the participants’ ages in our data set and were chosen to allow clean visualization of age effect.
Adding country information to the last formula showed no further statistically significant effect on SCIM95 (P = .1436). Similarly, neurological level (below C8 or C8 and above) and length of stay in rehabilitation did not affect SCIM95 significantly (P > .07 and P > .3, respectively).
Based on this formula, a new age- and gender-adjusted SCI-ARMI formula was computed:
Here, SCIMob is the observed SCIM III score. SCI-ARMI values at admission and at discharge were calculated for each patient, and the difference between them (i.e.: SCI-ARMI gain) was positive in almost every case, reflecting the improvement in ability realization during rehabilitation (Figure 4). The average gain in SCI-ARMI was 38.8 points (SD = 22; P < .0001).
Scatter plot of SCI-ARMI scores of all patients before and after rehabilitation: scores are almost always higher after rehabilitation.
SCI-ARMI gain was moderately correlated with admission SCI-ARMI (r = −0.64; P < .0001). Whereas it was found to be weakly correlated with age (r = −0.23; P < .0001), admission AMS (r = −0.14; P < .0001), and AMS gain (r = 0.13, P < .0006). SCIM95 (after adjusting for AMS, age and gender) was not significantly different between patients with traumatic and nontraumatic injuries (P > .8).
Discussion
SCI-ARMI is a measure that enables the assessment of disability to be weighted by the identified level of neurological deficit in people with SCI, irrespective of completeness or incompleteness of injury. It enables investigators to evaluate quantitatively not only the specific impact of rehabilitation beyond that of neurological recovery alone but also an individual’s potential to benefit from rehabilitation. 1 Ensuring its validity and reliability both within functionally different SCI populations and across countries with potentially disparate rehabilitation and care processes is imperative. In fact, these 2 psychometric properties are considered mandatory to ensure the generalizability of the measure and, therefore, justifying its use in different settings.8,9 On the basis of the above analyses, it was concluded that the previously published SCI-ARMI formula, calculated with data of 1 country’s population alone, is valid for an international population after adjustment for age and gender.
Although the identified effects of age and gender on the relationship between functional and neurological outcomes may appear intuitively reasonable, we believe that the analysis was necessary to quantify the effect and to demonstrate that merely adjusting for these factors is sufficient for the same formula to apply to all individuals in various countries. It is perhaps of particular note that this adjustment to the formula eliminates the need to consider possible differences between centers in AMS distribution, neurological level of the injury, and the length of stay in rehabilitation.
The finding that age significantly affects SCIM95 (representing the highest-observed SCIM III values, or the functional potential, for given AMS) may indicate that the functional potential for a given neurological deficit is lower at older age; this may be considered reasonable because functional abilities generally decline as people’s age increases. The concept of reserve (peak) capacity (also termed vitality) 10 suggests that in normal ageing, this capacity peaks at around 30 years of age and gradually reduces year-on-year until death, with disease processes considered to accelerate this process of decline. In support of this concept, there is an increasing amount of literature now being published on the effects of age on the neurological and functional outcomes of SCI patients.11-15 Another possible explanation is that older patients receive less rehabilitation than younger patients. However, Hsieh et al 16 in a recent comparison between old and young concluded that older patients receive the same amount of rehabilitation hours as younger ones, although with less intensity (hours per week) and with different pathways (less physical therapy and more occupational therapy). Despite these findings, the issue of the amount of rehabilitation needs to be addressed in further prospective studies. A third possibility is that older patients may have more difficulties in translating neurological recovery in functional improvement, as already demonstrated by Jakob et al. 17 However, as the present formula aims at evaluating exclusively the effect of rehabilitation, this finding probably does not apply to our data.
When the international data sample was analyzed to consider the impact of gender, it showed that the functional potential for a given neurological deficit is slightly, but significantly (P = .037), lower in women than in men. Figures 2 and 3 demonstrate that although the various quadratic SCIM95 formulas (with and without age and gender as affecting factors) include different coefficients, the practical shapes of their models are similar. Furthermore, although the overall difference in functional potential between men and women might again be considered intuitively reasonable, in practical terms, overall functional potential and outcomes will likely remain person specific, rather than gender specific, within rehabilitation programs.
Literature data on gender-related differences of SCI outcomes are surprisingly scarce and inconsistent. Whereas some authors found that male and female SCI patients had comparable functional outcomes,13,18,19 others demonstrated better outcomes for female patients or for male patients20,21 depending on the level and completeness of the SCI. It should be noted, however, that the present study is the only one comparing gender effect on function while controlling for the neurological deficit, and there are no data from other studies for direct comparison with the current findings of gender effect.
The key aim of this investigation was to analyze the relationship between AMS, SCIM III, age, and gender in a relatively large international SCI patient group. This has allowed for the development of a more accurate SCI-ARMI formula for assessing ability realization after SCI. The calculation of SCI-ARMI using this formula requires patient data that are readily available in most SCI units (AMS and SCIM scores, age, and gender). The nonsignificant country effect on SCIM95 supports the validity of the computed SCI-ARMI quadratic formula in all the countries that participated in the study and possibly in other countries.
The SCI-ARMI gain is considered by the authors to be the most important application of the SCI-ARMI process because it identifies the effectiveness of the rehabilitation process per se. This process together with the change in the neurological deficit between admission to rehabilitation and discharge, age, and gender have been identified in this study as responsible for most of the change in functional ability during specialized SCI rehabilitation and, as such, is considered an effective measure of the success of rehabilitation. Indeed, it is the only measure currently available that has the ability to directly assess the overall success of the rehabilitation process itself, regardless of the completeness of SCI.
For those with complete SCI, in whom the likelihood of significant neurological recovery is unfortunately low, the difference between admission and discharge SCIM III scores is likely to reflect solely the effectiveness of rehabilitation; but for those with incomplete SCI, who experience natural neurological recovery, the differences in disability scores cannot represent the net effect of treatment. 22 However, the weak correlation between the SCI-ARMI gain, calculated by the currently developed formula, and AMS and AMS gain, supports the utility of the formula, demonstrating the minimal effect of the neurological deficit and its change on the measure in patients with incomplete injuries. The negative correlation with admission SCI-ARMI again makes intuitive and clinical sense: the higher the initial SCI-ARMI, the greater is the ceiling effect on the overall functional gain made during rehabilitation.
The present study has examined the effects of age, gender, country, neurological level, length of stay in rehabilitation, and traumatic or nontraumatic etiology, and these seem sufficient to consider SCI-ARMI to be reliable and accurate for the majority of SCI patients. The authors believe that the model as well as the questionnaire and assessments on which it is based provide a valid and reliable evaluation of goal attainment following SCI, and that it has the flexibility to encompass new clinical variables as their significance becomes clear.
Further information concerning SCIM-III and SCI-ARMI can be obtained by visiting the SCIM Web site http://www.scimscore.com. This Web site has been updated to include the modified SCI-ARMI formula detailed in this investigation to allow individual researchers and clinicians the opportunity to calculate patient-specific SCI-ARMI values by inserting patient AMS and SCIM scores, age, and gender into the appropriate fields in the site.
Limitations
We set the final evaluation at the moment of discharge; however, we are aware that further neurological and functional recovery may occur and thus require additional study at later time points after injury
Furthermore, additional factors such as sensory deficit, spasticity, chronic pain, mental health, and environmental and possible socioeconomic and organization factors may have the potential to influence ability realization in certain patients, and these will also require consideration in future SCI-ARMI models.23-26
Footnotes
Acknowledgements
The authors would like to thank J. R. Chowdhury, C. M. Goodwin-Wilson, J. Lains, L. Laurenza, P. Margalho, J. Medina, M. Menarini, J. Nirmal, C. Pilati, and M. Watkins.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was supported by a Legacy Foundation grant. US data collected during the 2008-2010 multicenter study led by K. Anderson. Collaborators on the study were M. E. Acuff, B. G. Arp, D. Backus, S. Chun, K. Fisher, J. E. Fjerstad, D. E. Graves, K. Greenwald, S. L. Groah, S. J. Harkema, J. A. Horton III, M.-N. Huang, M. Jennings, K. S. Kelley, S. M. Kessler, S. Kirshblum, S. Koltenuk, M. Linke, I. Ljungberg, J. Nagy, L. Nicolini, M. J. Roach, S. Salles, W. M. Scelza, M. S. Read, R. K. Reeves, M. D. Scott, K. E. Tansey, J. L. Theis, C. Z. Tolfo, M. Whitney, C. D. Williams, C. M. Winter, and J. M. Zanca.