Abstract
This study assesses the reliability and predictive validity of the Lower Limb Extremity Amputee Measurement Scale (LLAMS), which is an assessment tool designed to predict the length of stay (LOS) of patients with lower limb amputations in a rehabilitation program. In order to evaluate inter-rater reliability a prospective evaluation was completed by five independent evaluators (n = 10). Predictive validity was evaluated retrospectively by comparing the LLAMS predicted LOS to actual LOS (n = 147). The ability of the amputee team members to administer the LLAMS to patients was very high (ICC [2,1] = 0.98, CI 95% = 0.96 – 0.99, F[9, 36] = 78.71, p < 0.05). In addition, a moderate positive correlation was found between the LLAMS predicted LOS and the actual LOS (Pearson Correlation Coefficient, r = 0.465, p < 0.01), and the LLAMS was able to identify those patients who required short versus long rehabilitation stays. The incorporation of the LLAMS into the physiatrist's initial assessment of patients in the amputee clinic has enhanced the ability to manage better the LOS and the time patients wait to enter the rehabilitation program.
Introduction
The causes of lower limb amputations (LLAs) are varied. Ephraim and colleagues (2003) report that there are “multiple pathways” to the loss of a limb, including diabetes mellitus (DM), peripheral vascular disease (PVD), trauma, and malignancy. The reported incidence of LLA varies depending on the etiology, gender, race and country of origin. For example, in the United States (US) the incidence of LLA is estimated to be 33 – 114 per 10,000 persons with DM (Ephraim et al. 2003). The incidence of people with PVD who had LLAs in the US ranged from 1.1 – 6.8 per 10,000 (1993 – 1997)(Palumbo and Melton 1995). An estimated 133,000 LLAs are performed annually in the US (Dillingam et al. 2002) with the number of LLAs in the geriatric patients growing from 28,000 – 58,000 by 2024 (Fletcher et al. 2002).
Patients with an LLA who enter the rehabilitation setting often have varying co-morbidities that can affect the length and success of the rehabilitation process. These may include medical conditions, physical impairments, cognitive impairments and motivational barriers (Heinemann et al. 1994; Leung et al. 1996; Muecke et al. 1992). Given the current state of healthcare funding constraints, it is imperative that resources are allocated optimally in order to maximize patient care. It is important that rehabilitation services are used efficiently to maximize patient abilities while demonstrating financial responsibility and sustainability. The ability to predict length of stay (LOS) and the extent of care patients may be a valuable tool in resource management and ensuring financial sustainability.
Currently there is no specific tool available to predict LOS of patients with LLAs in the rehabilitation setting. A few tools that were developed to assess patient's functional abilities have been investigated as predictors of LOS and were found to have limited predictive abilities. The Functional Independence Measure (FIM) has been used to predict the success of the rehabilitation (Heinemann et al. 1994; Leung et al. 1996; Muecke et al. 1992) as well as LOS in the adult population (Heinemann et al. 1994). Muecke et al. (1992) found that the FIM was able to predict the outcome of rehabilitation for patients with high admission scores, but was unable to predict outcome for patients with low admission scores. Muecke et al. (1992) also reported that LOS was largely unrelated to the net difference in FIM scores (Muecke et al. 1992). Johnston et al. (2003) used FIM scores to investigate the characteristics of effective and efficient rehabilitation programs. They have studied data of 37,692 patients from participating rehabilitation hospitals and units and found that admission and discharge FIM scores were correlated. The authors have found that LOS was predicted by a number of factors, especially by percentage of managed cases (r = −0.20)(Johnston et al. 2003). However, they did not discuss the FIM as a specific LOS predictor or study its specific relation to patients with LLAs.
Stineman and Williams (1990) developed a mathematical model for predicting LOS in the rehabilitation setting. They have calculated modified Barthel Index scores retrospectively and found that an improved score correlated to a reduced LOS. However, only 16 patients who underwent LLA were included in their study. In addition, the model described was designed to predict LOS for patients that were admitted up to the year 1991(Stineman and Williams 1990) and thus cannot be applied currently.
Traballesi et al. (1998) examined the prognostic factors in the rehabilitation of patients with PVD who underwent trans-femoral amputation. They have reported that the Barthel Index was negatively correlated with LOS (−0.24); however, they did not evaluate the Barthel Index's potential role as a predictor of LOS.
As reviewed above, few measures are available to predict the functional outcome of patients with LLA (Franchignoni et al. 2004; Larner et al. 2003; Leung et al. 1996; Masedo et al. 2005; Munin et al. 2001; Panesar et al. 2001), however no other measurement tools are specifically designed to predict the rehabilitation LOS with these patients.
The objectives of this study were: (i) To develop a measurement tool that can be used to predict the rehabilitation LOS in adult patients who underwent LLA (Lower Limb Amputee Measurement Scale – LLAMS); (ii) Evaluate the inter-rater reliability of the LLAMS; (iii) Investigate the predictive validity of the LLAMS; and, (iv) Investigate the ability of the LLAMS to dichotomize patients into those who require a long versus short rehabilitation stay.
Methods
Tool development
A review of patients admitted to the Amputee Rehabilitation Center at the Chedoke Campus, Hamilton Health Sciences, was completed looking for key indicators that may affect LOS. The multidisciplinary Amputee team reviewed the common indicators and generated 31 questions that were divided into six subsections. The LLAMS (Appendix A) subcategories include medical (six questions), cognitive (seven questions), social (three questions), physical (eight questions), activities of daily living (four questions), and other (three questions). All questions are answered yes (score one) or no (score zero). A low score on the LLAMS indicates fewer co-morbidities (short LOS), while a high score indicates many co-morbidities (long LOS). The total LLAMS score was used to predict LOS.
Study protocol for inter-rater reliability
Either a physiatrist or physical therapist (PT) assessed each patient, using the LLAMS. Five members of the amputee team, including prosthetist, registered nurse, PT, occupational therapist, and a physiatrist, observed the assessment and completed the LLAMS independently. Each healthcare professional read the three-page instructions sheet (Appendix B) for the administration of the LLAMS prior to the assessment. No other training was provided.
Study protocol for predictive validity (LOS)
A retrospective chart review of all patients who were admitted to the Amputee Rehabilitation Program at Chedoke Campus, HHS, between the years 2000 – 2003 was performed and the data of 157 patients was available for inclusion in the predictive validity analysis. Research ethic board approval was received for this study.
Data analysis
Descriptive analysis
The mean, standard deviation and median were calculated for both the subjects' age and LLAMS scores. The frequencies of gender and amputation types were recorded and percentage presentation calculated (Norman and Streiner 2000).
Internal consistency
Coefficient alpha was used to evaluate the internal consistency of the LLAMS (Streiner and Norman 2003) and 95% confidence intervals (CI) were also calculated (Shrout and Fleiss 1979).
Inter-rater reliability
Model B Intraclass Correlation (Muller and Buttner 1994)(ICC)(Type 2,1)(Shrout and Fleiss 1979) and their related 95% CI were calculated to estimate inter-rater reliability.
Predictive validity (LOS)
Pearson's correlation was used to estimate the relationship between the LLAMS score and the actual LOS (Portney and Watkins 2000). A “2 × 2 table” was used to evaluate the ability of the LLAMS to identify the patients who require short (six weeks) vs long (seven weeks) rehabilitation stay.
Power
Donner and Eliasziw (1987) published guidelines for establishing sample size requirements for reliability studies. Power values for the inter-rater reliability were obtained based on the actual sample size used, the number of measurements and the calculated ICC. The power calculated based on Donner and Eliasziw (1987) guidelines was compared to the results provided by an online power calculator (Bond 2002). The online power calculator was used for the predictive validity analysis.
Results
Inter-rater reliability
Ten patients who consented to participate in the study were chosen from the accessible population at the Amputee Rehabilitation Program. Patients' characteristics are presented in Table I.
Inter-rater reliability patient characteristics.
Intraclass Correlation Coefficients (ICC) were used to establish inter-rater reliability using SPSS© software (Version 11.0), for the total LLAMS score (ICC (2,1) = 0.9873, CI 95% = 0.9683 – 0.9964, F(9, 36) = 78.7113, p < 0.05), and for each subsection (Table II). Power was calculated to be > 0.94 manually as described by Donner and Eliasziw (1987) and was identical to the power calculated by the online calculator.
Reliability analysis on LLAMS subsections.
∗df = (9,36), p < 0.05; ICC, Intraclass Correlation Coefficient; CI, Confidence Interval; †LLAMS, Lower limb extremity amputation measurement scale; ‡ADLs, Activities of daily living.
Predictive validity
In this manuscript LOS days include weekends although no active treatment is provided to the patients. The rationale for including weekends is that the patient still occupies a hospital bed, or if on a day pass home, the bed can not be filled by other patients who may be waiting for a rehabilitation bed. The LOS of 10 patients (6.37%) was uncharacteristically longer than estimated due to their medical condition, patient availability, or home accessibility (Table III). In order to have the data reflect the actual patient population seen, the data of the above 10 outliers were excluded from further analysis.
Excluded patients.
∗Mean; †Standard deviation.
Based on the data of the remaining 147 patients (Table IV), LOS (in days)(m = 47.98, SD = 12.48) and the LLAMS (m = 10.41, SD = 4.13) were moderately positively correlated (r = 0.465, p < 0.01), with the LLAMS accounting for 21.6% of the variability in the predicted LOS. Power was calculated to be > 0.99.
Patient characteristics: predictive validity analysis (n = 147).
In addition to the moderate ability of the LLAMS to predict LOS in days, the LLAMS was used to dichotomize patients into two groups. One group of patients who required six weeks of rehabilitation (LLAMS score of nine and under), and those who required seven weeks of rehabilitation (LLAMS 10 and over). A “2 × 2 table”(Portney and Watkins 2000) was used to evaluate the ability of the LLAMS to identify the patients who require six weeks rehabilitation versus seven weeks rehabilitation. At the current cut-off of nine, the LLAMS identified correctly the patients who needed six weeks of rehabilitation 68% of the time and 80% for those who needed seven weeks of rehabilitation.
Discussion
In the current study, the LLAMS was found to be a reliable tool that can be administered by any health care professional on the amputee team. This was supported by the high ICC scores that were presented above. Furthermore, minimal training is required for administration of the LLAMS and it required only 5 – 10 min to perform. In addition, most of the LLAMS questions are often components of standard physical and medical assessments minimizing duplication of assessment questions.
The moderate correlation that existed between the LLAMS and the actual LOS, in days, indicated that an increase in LLAMS would predict an increase in LOS. This is consistent with previous studies that stated that an increase in co-morbidities would result in an increase of LOS (Heinemann et al. 1994; Leung et al. 1996; Muecke et al. 1992).
The ability of the LLAMS to dichotomize the patients into two groups, as described above, was further improved when the cut-off point was changed from a score of 9 to a cut-off score of 10. With the new cut-off, the LLAMS was able to identify correctly those patients who required six weeks of rehabilitation 82% of the time and 73% for those who required seven weeks of rehabilitation. The importance of this division is discussed later in this paper.
Interruptions in the rehabilitation program due to medical conditions are common (Bonner et al. 1996; Roth et al. 2001; Siegler et al. 1994; Wright et al. 1996) with an incidence ranging from 6.6 – 33.4%. Meikle et al. (2002) reported similar percentages in people with amputations. In the current study a medium incidence rate of interruptions was found (19.31%).
Clinical significance
There are two major benefits for administrating the LLAMS both in the amputee clinic and during admission to the rehabilitation program. The LLAMS score that a patient scores during the clinic visit can assist in educating the patients of the estimated time that they will be in the rehabilitation program, potentially reducing anxiety a patient may be experiencing related to further hospital stays. The LLAMS score can also be used for the management of the amputee rehabilitation waiting list. The waitlist manager can estimate how long a patient will be waiting for a rehabilitation bed based on the LLAMS scores of patients currently on the waiting list and those who are enrolled in the program. Another benefit is the ability to ensure a balanced caseload for both in-patient and out-patient amputee rehabilitation programs. For example, admitting six patients with a LLAMS of 15 or more will require an increase in resources (e.g., nursing staff) compared to admitting three patients with a LLAMS over 12 and three patients with a LLAMS less than nine. This balance may assist in maintaining a consistent caseload for the care providers, potentially reducing work related stress and staff injuries.
Comparing an admission LLAMS to an earlier clinic LLAMS can also help and identify any changes that may have occurred in the patient that may require new considerations and modification of the treatment plan.
In the amputee rehabilitation program at HHS the LLAMS is used to identify which patients will require a longer (seven-week) vs a shorter (six-week) program. Patients who are admitted to the seven-week clinical pathway receive an extra week of “physical build-up” which concentrates on reducing contractures and improving physical strength. This additional week is consistent with previous literature that stated that reducing contractures would improve the success of prosthetic training (Munin et al. 2001).
Limitations
During the period of the study the LOS of the majority of patients in the amputee rehabilitation program (both in-patients and day hospital patients) at HHS was either six or seven weeks (depending on their admission functional level). It is acknowledged that this LOS may be different in this institution compared to other facilities (Munin et al. 2001). It is therefore recommended that other facilities assess whether the LLAMS predict LOS in their settings. Furthermore, the LOS at our facility may seem long to some administrators, however, Munin et al. (2001) reported that successful ambulators had a longer rehabilitation stay compared to those who did not use their prosthesis. To the authors' knowledge, no randomized controlled studies are currently available comparing functional outcomes, quality of life, recurrent admissions or long-term effects on the medical status of patients in short vs long rehabilitation stays. Theoretically maximizing patients' abilities in the rehabilitation setting may have better outcomes and result in long term use of the prosthesis compared to patients who did not maximize their prosthetic use and abilities.
Another limitation of the current study is that the authors did not correlate the LLAMS with other outcome measures such as the FIM and the Barthel index. Furthermore, admission LLAMS had a low correlation with discharge 2-min walk test results (Pearson correlation r = 0.307, p > 0.01), and a low correlation with the gait aid used at discharge (Spearman's rho r = −0.333, p > 0.01) indicating that it was unable to predict functional outcome.
Conclusion
The LLAMS is a useful assessment tool that can be administered to adult patients who had lower limb amputation, by any member of the rehabilitation team, in order to predict their LOS in the rehabilitation setting. The LLAMS can also dichotomize patients into two groups, those who require a shorter rehabilitation program versus those who require a longer rehabilitation stay.
Footnotes
Acknowledgements
The authors would like to thank the Hamilton Health Sciences, Dr Hubert DeBruin, Dr Christopher Lee, Dr Linda Miller, Ms Karla Washington (MSc, BA), Ms Karen Litman (Prosthetist), Ms Pam Barr (PT), Ms Ann Toner (RN), Ms Barb Wood (PT/OT) Ms Kim Hardie, Ms Deborah Tsui (PT), and all the patients involved in this study.