Abstract
Background:
Measures of activity performance for adults with upper limb amputation are needed. The purposes of this study were to evaluate the measurement properties of a new measure of activity performance for adults with upper limb amputation, which we call the Brief Activity Measure for Upper Limb Amputees (BAM-ULA) and to compare BAM-ULA scores for users of different types of prostheses.
Methods:
In all, 35 persons with upper limb amputation were administered the BAM-ULA, twice within 1 week. Internal consistency and test–retest reliability were evaluated. Construct validity was examined by comparing scores by amputation level. Concurrent validity was evaluated by examining correlations with other measures. Exploratory analyses using linear regression compared sub-group scores for users of myoelectric and body-powered devices, and for users of single-degree-of-freedom and multi-articulating devices, controlling for amputation level.
Results:
The scale alpha was 0.83; intraclass correlation coefficient was 0.91. Item scores differed by amputation level and device type. Persons with transradial amputation completed more items than those with amputations at the transhumeral or shoulder level (p < 0.05). Users of myoelectric devices completed more items than users of body-powered devices (p < 0.05). The BAM-ULA strongly correlated with the Activities Measure for Upper Limb Amputees; R = 0.86) and three modified Jebsen–Taylor Hand Function Tests (R = 0.62–0.68) and moderately correlated with one Jebsen–Taylor Hand Function Test (R = 0.46) as well as with the Wong–Baker FACES Pain Rating Scale. No differences in scores were found by degree of freedom of the terminal device after controlling for amputation level.
Conclusion:
Analyses support reliability, construct validity, and concurrent validity of the Brief Activity Measure for Upper Limb Amputees. This new briefer measure is easier to score than the Activities Measure for Upper Limb Amputees.
Clinical relevance
This article reported on the development and evaluation of a brief 10-item measure of activity performance for persons with upper limb impairment, the BAM-ULA. The BAM-ULA takes 10 min to administer and has a simple scoring method, which may facilitate its adoption by clinicians in the field.
Background
Rehabilitation providers and researchers need to utilize standardized, validated outcome measures to assess patient capabilities, track progress, and compare outcomes of therapeutic approaches and technologies. The choice of appropriate metrics is complex and should be guided by evidence that a measure is reliable and valid for the population in which it will be used. This requirement creates challenges for selection of outcome measures, particularly for patients with conditions that are relatively rare, such as upper limb loss. The content of the VA/DoD evidence-based clinical practice guidelines for the rehabilitation of persons with upper limb amputation is based largely on expert opinion, 1 with few outcome studies to draw upon that compare the effectiveness of treatment approaches. A recent systematic review comparing myoelectric and body-powered upper limb prostheses also pointed to the dearth of research using consistent, standardized metrics. 2 There is clearly a need for more research to guide practice and prescription of prostheses. In particular, the need for metrics to assess activity performance of adults with upper limb amputation has been well documented.3-5
In response to that need, Resnik et al. 6 developed the Activities Measure for Adults with Upper Limb Amputation (AM-ULA). The AM-ULA is an 18-item performance-based measure of activity appropriate for upper limb prosthesis users. The AM-ULA takes about 30–35 min to complete and employs a scoring rubric which requires a clinician to make judgments about task completion, skillfulness of prosthesis use, movement quality, independence, and overall time to perform that activity. Given that clinical providers may be challenged to implement a lengthy test such as the AM-ULA within the course of a typical rehabilitation visit, and that the scoring requires clinical judgment, we believed that a briefer measure of activity performance with a simpler scoring system would be welcomed by the field. We believed that the Brief Activity Measure for Upper Limb Amputees (BAM-ULA) scoring system, based on activity completion only, would not require clinical judgment about magnitude of body compensation or awkwardness, and thus could potentially be administered by staff who may not be trained as physical or occupational therapists. Therefore, the purposes of this study were to evaluate the measurement properties of a new measure of activity performance for adults with upper limb amputation, which we call the BAM-ULA and to compare BAM-ULA scores for users of different types of prostheses.
Methods
Study overview
The data used in this study were collected as part of the VA Home Study of an Advanced Upper Limb Prosthesis (Home Study), a multisite study involving three data collection sites. Subjects who were at least 18 years old, with an upper limb amputation at the transradial (TR), transhumeral (TH), shoulder disarticulation, or scapulothoracic (SH) level, were eligible to enroll. Persons with amputation level or skin conditions that prohibited prosthesis wear and those with serious health conditions that the study staff believed might interfere with participation were excluded. This research was approved by the Institutional Review Boards of the Providence VA Medical Center, the James Haley VA Medical Center, the VA New York Harbor Healthcare System (NY HHS), and the Center for the Intrepid at Brooke Army Medical Center.
Data collection
At the study screening visits, an occupational therapist administered a set of self-report and performance-based measures (described below). The BAM-ULA was only administered to persons who were completing their testing using a prosthesis. Most persons who regularly used a prosthesis were tested with their prosthesis; however, there were several subjects who were unable to do so because their prosthesis was broken or they failed to bring it to the testing session.
The BAM-ULA is a 10-item observational measure of activity performance, developed by the first author. The 10 items are as follows: tuck a shirt in pants, lift a 20-lb bag, open a water bottle, remove a wallet from back pocket, replace the wallet in back pocket, take a gallon of water from the refrigerator and place on the counter (lift gallon jug), pour water from a gallon jug, brush or comb hair, use a fork, and open a door with knob. These items were selected from a larger pool of items after preliminary evaluations of scale structure and score distributions were conducted. Exploratory analyses of an initial 11-item set revealed a clear ceiling effect, with 43% of subjects scoring within the estimated minimal detectable change (MDC) of the highest score. Therefore, four of the easiest items were removed, and three more difficult items administered as part of a larger activity item set were included.
Standardized instructions were developed to ensure that subjects were guided through the specific subtasks of each activity (Appendix 1). Each item was rated 0 for unable to complete or 1 for did complete. The scores for each item were summed to obtain the overall task completion score. Summary scores were calculated only when all 10 items were rated. Higher task completion scores indicated better performance.
Other study measures collected were the modified Jebsen–Taylor Hand Function Test (JTHFT),7,8 the Activities Measure for Upper Limb Amputees (AM-ULA), 6 the Upper Extremity Functional Scale (UEFS), the Disabilities of the Arm, Shoulder and Hand Score (QuickDASH), 9 and the Wong–Baker FACES Pain Rating Scale. 10 The modified JTHFT is a measure of dexterity and simple functional activities consisting of seven sub-tests, each of which is scored separately. The modified version has a 2-min cap of allowable time per sub-test. Scores indicate the number of items completed per second. 8 The AM-ULA is a measure of activity performance which considers a variety of factors in its scoring rubric. Every item is scored from 0 to 4 (unable to excellent). The UEFS is a self-reported measure of activity limitation developed as part of the Orthotics and Prosthetics User Survey. 11 The tasks are rated on a 1–5 point scale (very easy to cannot perform activity). The QuickDASH consists of 11 self-reported items that address difficulty performing functional activities and severity of impairments. 12 A higher QuickDASH score indicates greater disability. Pain was rated using the FACES scale as developed by Wong and Baker. 10
Analyses
The characteristics of the sample and scores of baseline measures were tallied and examined. Internal consistency of the BAM-ULA was evaluated using Cronbach’s alphas and item-test and item-rest correlations. Test–retest reliability of total scores was estimated using Shrout and Fleiss intraclass correlation coefficient (ICC) (3,1), a two-way mixed model of consistency for single measurements where raters are considered fixed effects and subjects are treated as random effects. 11 Test–retest reliability of individual task completion was estimated using Kappa (k) statistics. Histograms were utilized to visualize score distributions and to identify potential floor and/or ceiling effects.
To evaluate known group validity, scores were compared by amputation level (TR, TH, or SH) using chi-square analyses. Our main hypothesis was BAM-ULA scores would be lower (worse) for persons with more proximal amputation (shoulder level) as compared to more distal amputation (TR/TH level). To examine concurrent validity, we examined the relationship between the BAM-ULA and other measures we expected would be related, using Pearson product-moment correlations.
We conducted several exploratory analyses to determine whether there were differences between subjects utilizing different types of prostheses and terminal devices. Scores were compared between users of body power devices and users of myoelectric/hybrid devices. Due to potential confounding between amputation level and type of prosthesis used, the analysis by type of prosthesis was repeated controlling for amputation level in a linear regression model. We classified the degree of freedom (DOF) of the prosthetic terminal device used during testing as (1) single DOF (e.g. Greiffer, motion control hand, electric terminal device (ETD), work hook), (2) multi-articulating (e.g. i-limb, bebionic hand, Michelangelo hand, touch bionic hand), or (3) no DOF (cosmetic prosthesis). Because there was one body-powered device user who had a cosmetic hand, we compared scores by terminal device DOFs only for users of single- and multiple-DOF devices and also controlled for amputation level in a linear regression model of the BAM-ULA on DOFs. For analyses with small sample sizes, the non-parametric Wilcoxon signed-rank tests (or Kruskal–Wallis tests) were performed in addition to t-tests (or analyses of variance (ANOVAs)), and Fisher’s exact tests were calculated instead of chi-square tests.
Results
There were 35 subjects, mean age 45 years (standard deviation (SD) 15 years), 89% male, 86% white, 57% TR, 37% TH, and 6% SH who completed the first baseline testing session (Table 1). They were predominantly prosthesis users, with 80% describing themselves as full-time prosthesis users and 63% used a myoelectric device. A total of 31 subjects returned to complete the second baseline testing session. Data from both testing sessions are shown in Table 2.
Characteristics of subjects.
SD: standard deviation; JTHF: Jebsen–Taylor Hand Function; AM-ULA: Activities Measure for Adults with Upper Limb Amputation; UEFS: Upper Extremity Functional Scale; QuickDASH: Disabilities of the Arm, Shoulder and Hand Score.
BAM-ULA scores by visit.
SD: standard deviation.
Reliability analyses
Item-total, item-rest, and scale alphas are shown in Table 3. Item-test correlations ranged from 0.32 to 0.80 and item-rest correlations ranged from 0.18 to 0.72. Cronbach’s alpha for the overall performance rating was 0.83. Kappa statistics showed statistically significant agreement (k = 0.38−1.00) for all items (Table 4). The total BAM-ULA score had an alpha of 0.83 and an ICC of 0.91 (95% confidence interval (CI): 0.82−0.95), with a corresponding MDC90 of 1.9. A total of 23% of the sample was within the MDC90 of the ceiling and no subjects were within the MDC90 of the floor. The distribution of the final BAM-ULA scores is shown in Figure 1.
Internal consistency for 10-item composite scale.
Kappa values for BAM-ULA items.
Distribution of total BAM-ULA scores.
Validity analyses
The results of Fisher’s exact and Kruskal–Wallis tests by amputation level are shown in Table 5. BAM-ULA item scores were significantly different by amputation level (p < 0.05) for the following five tasks: carry a 20-lb bag to shelf, remove wallet from back pocket, replace wallet in back pocket, lift a gallon jug, and open door with knob. The total BAM-ULA scores also differed significantly by amputation level (p = 0.03) with TR amputees completing 7.5 items on average, compared to 4.8 and 4.5 items completed in the TH and SH groups, respectively.
Comparison of BAM-ULA scores by level of amputation.
SH: scapulothoracic; TH: transhumeral; TR: transradial; SD: standard deviation; KW: Kruskal–Wallis.
Correlation analyses found moderate and statistically significant correlations between the BAM-ULA and one of the JTHF sub-tests (R = 0.46), and strong correlations with three others (R = 0.62−0.68) using Cohen’s criteria (Table 6). 13 We also observed strong correlations between the BAM-ULA and the AM-ULA (R = 0.86). We found a moderate correlation (R = −0.45) between the BAM-ULA and self-reported pain ratings.
Pearson product-moment correlations between BAM-ULA and other outcome measures.
JTHF: Jebsen–Taylor Hand Function; AM-ULA: Activities Measure for Adults with Upper Limb Amputation; UEFS: Upper Extremity Functional Scale; QuickDASH: Disabilities of the Arm, Shoulder and Hand Score.
Exploratory analyses
We observed statistically significant differences between body-powered and myoelectric device users on four items (tuck in shirt, move 20-lb bag to shelf, remove wallet, and open door with knob; p < 0.05); in each instance, those using body-powered devices completed the task less often than those using myoelectric/hybrid devices (results not shown). Linear regression revealed that an average increase of 2.5 (p = 0.017) completed BAM-ULA tasks for myoelectric/hybrid device users when controlling for amputation level (Table 7). The only significant difference observed between users of single-DOF and multi-articulating terminal devices in crude analyses was in completion of the “tuck shirt” task (p = 0.04) with single-DOF device users having lower scores. There was no significant linear association between the BAM-ULA and DOFs after controlling for amputation level.
Linear regression of total completed BAM-ULA tasks (N = 35).
TR: transradial; TH: transhumeral; SH: scapulothoracic.
Discussion
This article reported on the development and evaluation of a brief 10-item measure of activity performance for persons with upper limb impairment, the BAM-ULA. The measure was intended to be briefer than the AM-ULA to administer and simpler to score. As compared to the AM-ULA which takes about 35 min to administer, we estimate that the BAM-ULA will take about 11 min to complete. Our analyses confirmed that the measure had very good internal consistency and test–retest reliability. We estimated that the MDC90 of the BAM-ULA was 1.9 points, meaning that in a repeat test administration, there would be 90% confidence that true change had occurred if a score differed from the prior score by 1.9 points or more.
We did observe a ceiling effect for the total score, with approximately 23% of subjects scoring within 1.9 points of the ceiling of the measure. This finding was driven by the prevalence of easier items which had very high completion rates, as well as the inclusion of high proportion of subjects with TR amputation (who on average completed 7.5/10 items). This finding suggests that the BAM-ULA may benefit from the addition of one or more items that are more difficult to complete. This is an area where further research is needed.
We found, as expected, that there were differences in BAM-ULA scores by level of amputation, for the total number of items completed as well as for five of the individual test items. Subjects with TR-level amputation had better performance as compared to subjects with TH or shoulder-level amputation. Our findings of strong correlations between five other performance measures and the BAM-ULA supported concurrent validity.
Although the overall sample size for this analysis provided adequate statistical power for the internal consistency, agreement, and test–retest reliability analyses, we recognize that the sample size was small for analyses of correlations and for comparisons of specific sub-groups. Nevertheless, we found differences between subjects by amputation level and by type of device use (myoelectric vs body powered) after controlling for amputation level.
Our study included a convenience sample of persons with upper limb amputation who were recruited for a larger study. We did not attempt to control for years of prosthesis use or amount of training with the device in these analyses and it is possible that there may have been differences in these factors by group. However, we have no reason to believe that these types of differences would have been systematic and would have biased the study results. Nevertheless, there may be limits to the generalizability of findings to a wider population of persons with upper limb amputation. Another limitation of the study is that we did not examine inter-rater reliability of the BAM-ULA scoring. However, given the simple, dichotomous scoring method, we would expect very strong agreement of scores. Future study is needed to assess inter-rater reliability.
Our finding of no difference in BAM-ULA scores between users of single-DOF and multi-articulating terminal devices should be interpreted cautiously and considered preliminary, given the very small sample sizes and lack of power to detect differences. Furthermore, subjects who used these differing types of components may not have been equivalent in terms of prosthetic training or experience. Despite these limitations, we believe that this research contributes to the literature on prosthetic device types. Further research, using stronger research designs with large samples, is needed to confirm these preliminary findings.
Conclusion
This study reported on the development and preliminary testing of the BAM-ULA, a brief, simple-to-score measure of activity performance for persons with upper limb amputation who utilize a prosthesis. The analyses support the internal consistency, test–retest reliability, construct, and concurrent validity of the new measure. The BAM-ULA was used to compare activity performance of users of body-powered and myoelectric devices. We found that myoelectric device users were able to perform on average 2.5 more of the BAM-ULA items as compared to body-powered device users, after controlling for amputation level. The BAM-ULA is briefer to administer as compared to the AM-ULA and has a simpler scoring method, which may make it more likely to be adopted by clinicians in the field.
Footnotes
Appendix 1
Acknowledgements
The information in this manuscript does not necessary reflect the position or policy of the government; no official endorsement should be inferred. The view(s) expressed herein are those of the author(s) and do not reflect the official policy or position of the US Government.
Author contribution
L.R. secured the funding, led the data collection efforts, directed the analyses, and took the lead on manuscript writing. F.A. was the project coordinator for the study; he coordinated data collection efforts and participated in data analyses and manuscript writing. M.B. was the biostatistician who cleaned the data, conducted the analyses, and assisted in interpretation and manuscript writing.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by VA RR&D A9226-R.