Abstract
Objective:
This study describes the development and psychometric evaluation of a novel patient-reported single-item mobility measure.
Design:
Prospective cohort study.
Setting:
Four Veteran’s Administration Medical Centers.
Subjects:
Individuals undergoing their first major unilateral lower extremity amputation; 198 met inclusion criteria; of these, 113 (57%) enrolled.
Interventions:
None.
Main measures:
The Amputee Single Item Mobility Measure, a single item measure with scores ranging from 0 to 6, was developed by an expert panel, and concurrently administered with the Locomotor Capabilities Index-5 (LCI-5) and other outcome measures at six weeks, four months, and 12 months post-amputation. Criterion and construct validity, responsiveness, and floor/ceiling effects were evaluated. Responsiveness was assessed using the standardized response mean.
Results:
The overall mean 12-month Amputee Single Item Mobility Measure score was 3.39 ±1.4. Scores for transmetatarsal, transtibial, and transfemoral amputees were 4.2 (±1.3), 3.2 (±1.5), and 2.9 (±1.1), respectively. Amputee Single Item Mobility Measure scores demonstrated “large” and statistically significant correlations with the LCI-5 scores at six weeks (r = 0.72), four months (r = 0.81), and 12 months (r = 0.86). At four months and 12 months, the correlation between Amputee Single Item Mobility Measure scores and hours of prosthetic use were r = 0.69 and r = 0.66, respectively, and between Amputee Single Item Mobility Measure scores and Trinity Amputation and Prosthesis Experience Scales functional restriction scores were r = 0.45 and r = 0.67, respectively. Amputee Single Item Mobility Measure scores increased significantly from six weeks to 12 months post-amputation. Minimal floor/ceiling effects were demonstrated.
Conclusions:
In the unilateral dysvascular amputee, the Amputee Single Item Mobility Measure has strong criterion and construct validity, excellent responsiveness, and does not exhibit floor/ceiling effects.
Introduction
Effective tools that quantify mobility are essential to evaluate therapeutic interventions and translate research findings into clinical practice. 1 This is an area of particular relevance among lower extremity amputees, where mobility is the single most important contributor to quality of life. 2 This article will describe the development and psychometric evaluation of a novel single item mobility measure.
The Amputee Single Item Mobility Measure (AMPSIMM) was developed because of a number of perceived limitations in existing measures. Many measures require clinician input and time, measure mobility only in amputees who use a prosthetic device, or report mobility as a numeric score rather than in clinically meaningful terms.
For example, the Harold Wood Stanmore scale, 3 the Rivermead Mobility Index, 4 the Volpicelli Mobility Grade, 5 the Special Interest Group in Amputee Medicine, 6 and the Amputee Mobility Predictor 7 are useful measures, but all require some clinician input and therefore place a burden on healthcare resources. Well-validated multiple item patient-reported outcome measures, such as the Prosthesis Evaluation Questionnaire,8–10 the Houghton Scale, 11 and the Locomotor Capabilities Index-59,12–14 quantify mobility in a range of domains, but are limited in that they are designed to quantify mobility only in amputees who use prosthetic limbs. The utility of these measures is limited in amputees prior to prosthetic fitting, and in the dysvascular amputee, who will often function through a combination of wheeled and ambulatory mobility.2,15–17 In one study, 68% of the population utilized a prosthesis, while 83% of the population used a wheelchair at least 50% of the time. 2 An instrument that measures amputee function across the continuum of rehabilitation, including the time period prior to prosthetic fitting, or if there is a loss of prosthetic mobility, must be able to quantify mobility utilizing prostheses, ambulatory aids, and wheelchair use. 18
The majority of amputee mobility measures are also limited in that they result in a score that is the sum of scores on individual subitems.8–10,12–14 Although useful for quantifying mobility and mobility change, the resulting score does not often translate easily into clinically meaningful terms that have important relevance for the amputee (e.g. limited to walking inside the home vs. a community ambulator capable of walking long distances).
A team of rehabilitation professionals specializing in amputee rehabilitation, epidemiology, and measurement developed the AMPSIMM. To determine clinically relevant domains for dysvascular amputee mobility, a systematic literature search of existing measures was performed and measures were summarized and reviewed. This was augmented with utilizing the International Classification of Functioning Disability and Health (ICF) classification of mobility, including the subcategories of walking and mobility as described in Deathe et al. 18 This included walking, moving around, moving around in different locations, and moving around using equipment. This information was synthesized and reviewed by investigators with clinical and measurement development expertise through an iterative process to arrive at the final item and response categories. A copy of the AMPSIMM can be found in Figure 1. More detail of the AMPSIMM development process is in the appendix, available online.
The Amputee Single Item Mobility Measure (AMPSIMM).
The goal of this research was to evaluate the psychometric characteristics of the AMPSIMM including its criterion and construct validity, responsiveness, and evaluation of potential floor or ceiling effects.
Methods
Study design
This study was part of a larger multisite prospective cohort study of individuals undergoing major unilateral lower extremity amputation (transmetatarsal, transtibial, transfemoral) secondary to peripheral arterial disease or diabetes at four Veteran’s Administration Medical Centers. Participants were assessed via in-person or telephone interview at baseline (i.e. within seven days of the definitive amputation procedure), six weeks, four months, and 12 months postsurgically. Local institutional review boards approved study procedures. Informed consent was obtained from all individual participants included in the study. All assessments were performed by a trained study coordinator designated for each site that was responsible for recruitment, interviews, completion of case report forms, and routine monitoring of enrolled patients.
Participants
Potential subjects were screened in person or in the medical record before being approached for consent between August 2010 and April 2013. Subjects were considered eligible if they were age 18 years or older, were awaiting (or underwent in the last seven days) a first major lower extremity amputation, and the primary cause of amputation was complications of diabetes or peripheral arterial disease. Subjects were excluded if they had inadequate cognitive or language function to consent or participate, defined by more than four errors on the Short Portable Mental Status Questionnaire, 19 or were non-ambulatory before the amputation for reasons unrelated to peripheral arterial disease or diabetes. Of 415 individuals screened, 198 (48%) met study criteria; 85 (43% of eligible) refused, missed the recruitment window, or died before they could be enrolled; and 113 (57%) participated in the study (Figure 2). Among the 217 who were ineligible, the most common reasons were prior amputation (23% revision surgeries, 22% prior contralateral amputation) or bilateral amputation (23%).
Strobe diagram depicting total numbers excluded, not enrolled, enrolled, and final 12-month follow-up.
Baseline assessment measures
Baseline measures included socio-demographics, smoking status, and common comorbid medical conditions. The primary etiology for amputation was categorized as diabetes or peripheral artery disease, and the anatomic level of amputation was categorized as transmetatarsal, transtibial, or transfemoral (Table 1).
Baseline socio-demographic and general health data.
Data analysis
See Table 1 for descriptive statistics of presurgical variables. While the AMPSIMM can be conceptualized as an ordinal scale, and used in this way to characterize individual amputees for the purposes of establishing validity and for evaluating the effects of interventions on a population of amputees, the AMPSIMM was tested as both a categorical and a continuous variable for statistical purposes. To justify this, we assessed the assumption of normality of the AMPSIMM using the Shapiro–Wilk test. The AMPSIMM was found to display a normal distribution at six weeks, four months, and 12 months (p = 0.10, 0.57, and 0.98, respectively). Non-parametric statistics were employed when evaluating correlations of the AMPSIMM and other measures. The Chi-square test for trend was used when evaluating the ordered AMPSIMM categories by amputation level. Stata 9.1 was used for the statistical analyses described (StataCorp, College Station, TX).
Criterion validity
The Locomotor Capabilities Index-5 was chosen as the reference standard for the measurement of concurrent and predictive criterion validity because it assessed the degree to which ambulation aids were used and covered a relevant range of mobility tasks. In addition to having a relevant range of content, this measure has well-established internal consistency, test–retest reliability, and validity (content, discriminant, and criterion).14,20–23 The Locomotor Capabilities Index-5 was administered at six weeks, four months, and 12 months after amputation.
To evaluate the concurrent criterion validity of the AMPSIMM, the Spearman’s rank correlation coefficient was used to determine the correlation between the AMPSIMM score and the Locomotor Capabilities Index-5 score at six-week, 4-month, and 12-month follow-ups. Correlations of 0.1 were considered “small,” 0.3 as “medium,” and 0.5 as “large.” 24 To evaluate the predictive criterion validity of the AMPSIMM, we evaluated the association of the six-week and four-month AMPSIMM scores with the 12-month Locomotor Capabilities Index-5 score using the Spearman’s rank correlation coefficient. By ensuring that the AMPSIMM scores preceded the reference standard assessment chronologically, this was considered an assessment of predictive validity. 25
Construct validity
Construct validity represents a quantitative form of assessing validity by selecting other measures that evaluate the same or similar constructs and hypothesizing a priori the strength of the correlation. Hours of prosthetic use was measured among individuals who had been fitted with a prosthesis by asking “On average, how many hours per day are you walking with your prosthesis.” Functional restriction was assessed using the functional restriction subscale scores of the Trinity Amputation and Prosthesis Experience Scales (TAPES). 26 The TAPES include nine sub-scales, measuring psychosocial outcomes, activity restriction, prosthetic satisfaction, pain, and general health. The activity restriction subscale is further divided into an athletic activity restriction, functional restriction, and social restriction – the higher the score, the higher the restriction with scores ranging from 0 to 8. The functional restriction subscale was selected a priori because the items were most relevant to a dysvascular amputee population and represented the conceptual inverse of the function domain that the AMPSIMM measures.
Satisfaction with mobility was assessed at four months and 12 months after amputation with a single item measure developed in a prior study. 20 Subjects responded to the question: “How satisfied are you with your current walking ability?” using a 10-point Likert scale, where 0 represented “not at all satisfied” and 10 “extremely satisfied.” 20 This scale was also dichotomized establishing a subject as “satisfied” with a score of 6 to 10 and “not satisfied” with a score of 0 to 5. 20
To evaluate one form of convergent construct validity, the association between the four-month and 12-month AMPSIMM scores and four-month and 12-month hours of daily prosthetic use, TAPES functional restriction score, and satisfaction with mobility scores were evaluated using Spearman’s rank correlation coefficients. It was hypothesized that the strongest correlation would be with hours of prosthetic use, followed by activity restriction (using the TAPES), and then satisfaction with mobility. Non-parametric tests for trend were performed with cross tabulations of the AMPSIMM score and those “satisfied” or “not satisfied” with mobility to assess whether there was a trend in the ordering of the AMPSIMM scores by those “satisfied” vs. “not satisfied.”
To evaluate “known group” validity, mean AMPSIMM scores were compared by anatomic amputation level, hypothesizing that transmetatarsal amputees would have higher mean scores, followed by transtibial amputees, and transfemoral amputees. Cross tabulations were performed using the AMPSIMM response options by amputation level; the same non-parametric test for trend was used to assess whether there was a trend in the ordering of the AMPSIMM score by amputation level.
Responsiveness
As the mobility of a new amputee typically improves during the first year post-amputation, 27 it was hypothesized that the AMPSIMM scores would also improve over time. Several outcomes studies have used different methods to estimate magnitude of change over time in terms of an effect size. Some report that there is no definitive evidence that any method offers specific advantages.28,29 Therefore, to evaluate responsiveness, the change score between the six-week and 12-month assessments was calculated and divided by the standard deviation of the AMPSIMM’s change score to derive the standardized response mean.30,31 The change score standard deviation was imputed by using a formula recommended by the Cochrane collaboration. 32 Using Cohen’s effect size criteria (not to be confused with the previous criteria for correlations), 24 0.2 to 0.49 was considered a “small” effect, 0.5 to 0.79 a “moderate” effect, and 0.8 to infinity, a “large” effect.
Floor and ceiling effects
To assess the floor and ceiling effects of the AMPSIMM score, the percentage of subjects who achieved the minimum and maximum score was computed. Percentages greater than 15% were considered as demonstrating a floor or ceiling effect.33,34
Results
Baseline characteristics
The majority of the 113 subjects enrolled in the study had transtibial amputations (52%) followed by transfemoral (25%) and transmetatarsal level (23%) amputations (Table 1). Differences between baseline variables, comparing all subjects to those that completed each follow-up, were small and not statistically or clinically significant.
Despite quantitative evidence for normality at all time points, raw AMPSIMM scores were more heavily distributed in the lower region at six weeks, with no subjects achieving a score of five or six (Table 2). This was expected, since most patients are very early in their rehabilitation process and therefore higher scores are not expected. The distribution became more evenly distributed at subsequent follow-up – especially at 12 months.
Frequency of AMPSIMM scores by follow-up.
Criterion validity (concurrent)
The mean AMPSIMM and Locomotor Capabilities Index-5 scores at six weeks, four months, and 12 months are presented in Table 3. The AMPSIMM demonstrated “large” correlations with the Locomotor Capabilities Index-5 scores at all follow-up times. The strength of the correlation increased with each subsequent follow-up and the relationship appeared linear by visual inspection.
Criterion validity (concurrent): Mean AMPSIMM and Locomotor Capabilities Index-5 scores and their correlation at six weeks, four months, and 12 months.
Spearman’s rank correlation coefficient = 0.1 (small); 0.3 (medium), 0.5 (large).
Criterion validity (predictive)
The correlation between the six-week AMPSIMM score and the 12-month Locomotor Capabilities Index-5 score was computed. This relationship was considered less than “small” and not statistically significant (r = 0.07; p = 0.56). The correlation between the four-month AMPSIMM score and the 12-month Locomotor Capabilities Index-5 score was considered “medium” and statistically significant (r = 0.40; p = 0.004). This suggests assessing the AMPSIMM at four months has some predictive qualities, but not at six weeks.
Construct validity: Hours of prosthetic use
Among those who had been fitted with a prosthesis (n = 26 and 47, at four months and 12 months, respectively), the mean hours of prosthetic use are presented in Table 4. The correlation between the AMPSIMM score and hours of prosthetic use at these time points were considered “large” correlations.
Construct validity: Correlation of AMPSIMM scores with prosthetic use (hours), TAPES functional restriction subscale scores, and satisfaction with mobility scores at four months and 12 months.
Spearman’s rank correlation coefficient = 0.1 (small); 0.3 (medium), 0.5 (large).
Construct validity: TAPES
The mean TAPES functional restriction scale scores at four months and 12 months are presented in Table 4. The correlations between the AMPSIMM score and the TAPES functional restriction score at these time points were “medium” and “large,” respectively.
Construct validity: Satisfaction with mobility
The mean satisfaction with mobility scores at four months and 12 months are presented in Table 4. The correlations were considered “large” at both time points. Further, those “satisfied” with their mobility were significantly more likely to have a higher AMPSIMM score (test for trend p < 0.001) (Table 5).
12-month AMPSIMM scores comparing those who were and were not satisfied with their mobility. a
Non-parametric test for trend p < 0.001.
Construct validity: Known group
The mean 12-month AMPSIMM scores differed among amputation levels as hypothesized. AMPSIMM scores were highest for transmetatarsal amputees and lowest for transfemoral amputees (means for transmetatarsal, transtibial, and transfemoral amputees were 4.2, 3.2, and 2.9, respectively).
Responsiveness
When measuring the change in score from six weeks to 12 months after amputation, the AMPSIMM score improved significantly (mean change 2.4) with a standardized response mean of 1.0, representing a “large” effect (Table 6).
Responsiveness of the single item mobility measure score.
Imputed using formula recommended by the Cochrane collaboration (see supplementary materials for further information).
Floor/ceiling effects
With respect to floor and ceiling effects, two (2.4%) subjects achieved a minimum score and five (6.1%) subjects achieved a maximum score on the AMPSIMM 12 months after amputation, indicating neither a floor or ceiling effect in the dysvascular amputee population (Table 2).
Discussion
Psychometric evaluation of the AMPSIMM supports the utility of this measure to quantify mobility in the dysvascular amputee population. AMPSIMM has moderate to strong criterion and construct validity, as well as excellent responsiveness and no indication of floor/ceiling effects. Although it was not designed to replace existing measures, the AMPSIMM is unique in terms of its brevity, ease of administration, utility in quantifying mobility across the rehabilitation continuum in the dysvascular amputee population, and ability to define mobility in clinically relevant terms.
The AMPSIMM incorporates both ambulatory and non-ambulatory mobility, mobility in different environments, and mobility utilizing mobility aids. Therefore, it has relevance in the dysvascular population where ambulatory mobility with a prosthetic limb may or may not be achieved or when ambulatory mobility may be lost because of additional amputation or progression in multisystem disease.15–17 The AMPSIMM demonstrated that it is responsive to change when the patient improves in mobility function with or without a prosthesis. It can therefore be used to quantify mobility from time of surgery throughout the continuum of rehabilitation, and as such, offers an objective way of quantifying the impact of various rehabilitation interventions. This differs from the majority of amputee mobility measures, which focus specifically on mobility with a prosthetic limb.8–14
Further, AMPSIMM is scored so that each numeric score is associated with a specific level of mobility in the home and/or the community, whether that level of mobility is achieved by using a wheelchair or through ambulation, and whether or not mobility aids are required. Thus, it enables clear communication of functional mobility to patient and provider. With the increased emphasis on personalized medicine and patient participation in decision making, 35 it is important to have outcome measures that can be used in predictive models that enable clear communication of the difference in outcome associated with key clinical interventions. For example, if a prediction model informs a patient and provider that an intervention would result in a score change from 10 to 14 on a numerical scale, it would be difficult for the patient to weigh the benefits and costs of modifying the health factor. In contrast, if the intervention resulted in a change from using a wheelchair for community mobility to being ambulatory in the community, it would be conceptually easier for the risks of the intervention in relation to the effect of that intervention on outcome to be weighed by the patient. With additional research, AMPSIMM, through its structure and conceptual framework, may fulfill this important goal.
One of the fundamental obstacles to the widespread utilization of amputee mobility outcome measures to assess ongoing function is the clinical burden imposed by the measure3–7 and the lack of ability to interpret the data in real time. 36 AMPSIMM does not require clinician participation and consists of a single question. Its simplicity of structure and direct linkage to daily function will allow ease of interpretation in real time.
The present data demonstrated the preliminary validity of AMPSIMM in a dysvascular amputee population. Content validity was established by ensuring that individuals with relevant clinical and methodology expertise participated in generating the content using a structured and iterative process. 37 Concurrent and predictive validity was established by high correlations with existing measures. The construct validity of AMPSIMM was also well supported. As expected, individuals with more distal levels of amputation reported higher levels of mobility on the AMPSIMM. Similarly, higher AMPSIMM scores were associated with greater hours of prosthetic use, higher levels of mobility satisfaction, and lower levels of functional restriction. There have been a number of studies that have evaluated the psychometric properties of existing mobility outcome measures.18,38–40 Despite differences in the populations studied and the measures used in the validation process, the psychometric properties of the AMPSIMM appear to be similar to or better than existing measures.
Study limitations
The AMPSIMM is inherently an ordinal variable that was evaluated on both an ordinal and interval scale since it was designed to be scored for either individuals or groups. This may be considered a minor limitation; however, the AMPSIMM demonstrated normality at each follow-up point and is highly correlated with other continuous measures. The AMPSIMM can even predict future mobility when assessed in the rehabilitation process; however, this should not be assumed in the early rehabilitation period (i.e. six weeks after amputation).
The test–retest reliability of the AMPSIMM was not assessed. Additional research is required to determine the stability of the measure within different amputee subpopulations and at different time periods in the rehabilitation continuum. Further, the most appropriate wash-out period and an assessment of the test–retest reliability of the AMPSIMM should be considered in this evaluation.
Another potential study limitation is the fact that this measure was developed in individuals undergoing their first major lower extremity amputation. Further, the recruitment rate of those eligible was 57%. The other eligible subjects refused owing to personal issues or logistical burden, passed away before consent, or missed the tight recruitment window we had for this prospective study. Such limitations could suggest this population is healthier and/or more willing to participate in research. Research expanding the use of this measure with dysvascular amputees undergoing revision surgeries and/or contralateral amputations would further its development.
As a single item measure, the AMPSIMM has inherent limits similar to other single question measures and should not be construed as a replacement for other measures of amputee mobility. Single item measures may be less precise in capturing mobility; therefore, when precision is a priority, other more comprehensive patient-reported outcome measures should be used in conjunction with this measure. Similarly, AMPSIMM may be less sensitive for detecting minor changes than measures such as the 2-minute walk test.
Finally, the AMPSIMM was not designed to provide detailed information that would inform mobility outcomes of specific interventions. For example, it does not describe certain aspects of community mobility, such as ability to access public transportation or whether or not an individual is able to increase their walking distance, or reduce ambulatory aids from a walker to crutches. These outcomes may be important for specific programmatic evaluations, but AMPSIMM would not have the sensitivity to detect these small changes. As with all outcome measures, it is important to consider what the measure is designed to assess and in what clinical context.
Clinical messages
A novel, self-report, single item amputee mobility measure (AMPSIMM) has been developed for use in the dysvascular amputee population. Proposed benefits of the AMPSIMM are its ability to quantify mobility with a single question, throughout the continuum of amputee rehabilitation including individuals who use a combination of mobility aids, prosthetic limbs, and wheeled mobility, and it use of clinically descriptive conceptual terms.
Preliminary psychometric evaluation of the AMPSIMM indicates that it has excellent utility because of its brevity and it has strong criteria and construct validity, and it is responsive to change, without significant floor and ceiling effects in the dysvascular population.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based upon work supported by the US Department of Veterans Affairs, Office of Research and Development, Rehabilitation Research and Development (Merit Review A41241 Joseph Czerniecki, PI and Career Development Award B4927W Aaron Turner, PI).