Abstract
Background:
Three-dimensional laser scanning has been used for patient measurement for cranial helmets and spinal braces. Ankle–foot orthoses are commonly prescribed for children with orthopaedic conditions. This trial sought to compare ankle–foot orthoses produced by laser scanning or traditional plaster casting.
Objectives:
Assessment of the effectiveness and efficiency of using laser scanning to produce ankle–foot orthoses.
Study design:
Randomised controlled trial with blinding of orthotists and patients to the construction technique used.
Methods:
A randomised double-blind trial comparing fabrication of ankle–foot orthoses from casts or laser scans.
Results:
The time spent in the rectification and moulding of scanned ankle–foot orthoses was around 50% less than for cast ankle–foot orthoses. A non-significant increase of 9 days was seen in the time to delivery to the patient for laser scanning with computer-aided design and computer-aided manufacturing. There was a higher incidence of problems with the scan-based ankle–foot orthoses at delivery of the device, but no difference in how long the ankle–foot orthoses lasted. Costs associated with laser scanning were not significantly different from traditional methods of ankle–foot orthosis manufacture.
Conclusion:
Compared with conventional casting techniques, laser scan–based ankle–foot orthosis manufacture did not significantly improve either the quality of the final product or the time to delivery.
Clinical relevance
Ankle–foot orthoses (AFOs) are a common requirement for chronic neurological conditions during childhood. Improved efficiency of provision of AFOs would benefit children and families by reducing the delay in provision of devices and would benefit the health service by making best use of valuable orthotist time.
Keywords
Background
Orthoses are devices that are applied externally in order to modify structural and functional attributes of the neuromuscular and musculoskeletal systems, for example, in cerebral palsy. The prevalence of cerebral palsy in the United Kingdom has been estimated as 2 per 1000 live births for birth years 1986–1996. 1
Ankle–foot orthoses (AFOs) have considerable support in the literature in the management of cerebral palsy. 2 Children who are managed in AFOs often require intervention and care from multiple clinicians and agencies, so efficient orthotic provision is important in reducing the logistical demands on their families.
The traditional method of producing an AFO involves taking a negative limb cast from the patient and then filling the negative cast with plaster to make a positive cast. The orthosis is then formed by moulding polypropylene sheets over the positive cast. This method can be replaced by laser scanning with computer-aided design and computer-aided manufacturing (LSCAD/CAM), where a positive of the limb is milled in foam directly from the digital image. A recent systematic review suggests that foot orthotics designed from 3D scans are at least comparable with those made through traditional methods, although none of the studies reviewed were randomised controlled trials and further research is required to confirm this. 3 One study suggested that plaster casting may be preferable to laser scanning when it is important to capture the forefoot-to-rearfoot relationship. 4
We were able to identify four studies comparing plaster casting with LSCAD/CAM technology in the fabrication of spinal orthoses for scoliosis.5–8 Radiological improvement of the scoliotic curves was assessed in three of these studies, and LSCAD/CAM was reported to be as effective as the traditional method. The LSCAD/CAM technique was also found to increase productivity in both of the studies which measured this outcome. None of these studies measured health economic outcomes.
In the National Health Service (NHS), high capital costs and a lack of orthotists with time to learn new technology are likely reasons why LSCAD/CAM technology has not been introduced. If it proved successful, LSCAD/CAM could enable a more efficient process for the provision of a large number of bespoke orthotic devices.
Two hypotheses are explored in this article. First, that there is a difference in the length of time taken to deliver well-fitting AFOs produced by means of LSCAD/CAM compared to AFOs produced by traditional plaster casting. Second, that there is a difference in patient benefit in terms of functioning and satisfaction with AFOs produced by means of LSCAD/CAM compared to AFOs produced by traditional plaster casting. In addition, the cost-effectiveness of AFOs produced by means of LSCAD/CAM versus traditional plaster casting was estimated using a societal perspective.
Methods
Design of study
A double-blind randomised (1:1) controlled trial design in which patients had their AFO(s) fabricated either using LSCAD/CAM technology or by a traditional plaster method.
A positive opinion for the study was received from the North Staffordshire Local Research Ethics Committee on 15 August 2007 (ref: 07/Q2604/85).
Inclusion and exclusion criteria
All patients up to 18 years of age referred for rigid and hinged AFOs at the Trust were candidates for inclusion. Young adults up to the age of 21 years were also included if they were still receiving services for children (e.g. because they were in education). Exclusion of the patient occurred if informed consent was not obtained; the patient was considered by the orthotist to be unsplintable; a Saltiel style device had been specified or if communication was not possible with the subject or their parent/carer.
Setting
The study took place in the Orthotics Research and Locomotion Assessment Unit (ORLAU) and Orthotics Departments at the Robert Jones and Agnes Hunt Orthopaedic Hospital NHS Foundation Trust between February 2008 and August 2010. Eligible subjects were contacted by the study nurse who provided an explanation of the study. Those subjects who assented to participate had demographic data and details about diagnosis and condition collected.
Interventions
All subjects had both a casting and a scan of each limb requiring an AFO, in the same session, and the order in which the two methods were performed was determined according to a computer-generated randomisation list. The allocation sequence was concealed from both the researcher and the participant in sequentially numbered, sealed envelopes. The study orthotists were coached on how to minimise bias in their interaction with the subject (e.g. participants were to receive the same level of encouragement for each method). After experiencing both methods, subjects were asked to state whether they would prefer to have casting or scanning next time. The time taken and consumables for each method and session were measured and recorded by the attending clinician. The subjects or their parents/carers completed a questionnaire about travel and associated costs. The subject was given an appointment to return 2 weeks later to receive their AFO(s). Both the LSCAD/CAM digital image (scan) and the cast were rectified by an orthotic technician, and length of time for each method was recorded. Our usual Standard Operating Procedure was applied to both the cast and scan where 3 mm is applied to malleoli/metatarsals and navicular when requested on the specification sheet. A 1% increase in all dimensions was added to the scans to allow for ease of doffing and donning as well as allowing for growth.
A second randomisation was then performed to determine whether the AFO was manufactured from the cast or from the scan. Allocation to a manufacturing method was achieved through a central telephone randomisation system provided by the University of York Trials Centre and was stratified by age (less than 6 years and 6–21 years), Gillette Functional Walking score (1–5 and 6–10) 9 and required number of AFOs (unilateral or bilateral).
Laser scans were obtained with a FastSCAN (Polhemus, Colchester, VT, USA) and rectified using Rodin4D software (Rodin4D, Pessac, France). Image files were passed to an orthotic milling company (Relief Orthotics, Airdrie, Scotland) which milled a positive of the limb in foam that was returned to the Trust for manufacturing of the polypropylene device. Where cast manufacture was employed, a plaster male form was cast, rectified and then draped in the usual fashion. The length of time for the moulding of the polypropylene AFO was recorded. There were no differences in the manufacturing process between the two groups once the draping of the polypropylene sheet and curing had been performed.
Sample size calculations
The sample size was based on data which suggest the total time taken in the moulding and rectification process is reduced by approximately 60% when using LSCAD/CAM compared to casting. 6 The proposed sample size of 150 patients would enable the detection of a proportional reduction in time taken in the moulding and rectification of a cast of 30% with 90% power using a two-sided test at p = 0.05.
Outcome measures
The primary outcome measure was the length of time taken (in minutes) in the moulding and rectification process.
Secondary outcome measures included the following: (a) length of time (in minutes) spent with subjects to cast and scan limbs; (b) length of time (in minutes) to fit the AFO(s); (c) number of days taken from initial scanning/casting of a subject to the completion of an AFO (or AFOs) that met fitting specifications (any additional time taken for rescanning or remaking was included) and (d) the length of life of the AFO in days.
Patient-focused outcome measures included the ‘Satisfaction with Device’ and ‘Satisfaction with Service’ questionnaires, which were developed in the United States as part of the Orthotics and Prosthetics Users’ Survey (OPUS). 10 Both questionnaires use a 4-point Likert-type scale ranging from Strongly Agree (highest level of satisfaction) to Strongly Disagree (lowest level of satisfaction). Satisfaction with Device was measured with 9 items (e.g. skin abrasions/irritations, whether the orthosis was pain-free to wear, whether it looked good and whether it was easy to put on), and 10 items measured Satisfaction with Service (e.g. amount of time waiting for services, whether the orthotist was responsive to concerns and whether staff coordinated services with doctors and therapists). The questions were administered by post at 3-, 6- and 12-month follow-up. Non-responders were followed up by telephone if possible. In most cases, parents/carers acted as proxies by answering the questions on behalf of their child.
Statistical analysis
All planned analyses were by intention-to-treat. Differences in the primary and secondary outcome measures were analysed using the independent t-test for continuous data and the logrank tests for time to event data. Data from the patient-focused satisfaction questions, using the Likert-type scale, were analysed individually using the Mann–Whitney U test, proportions using chi-squared tests and time-to-event data using the logrank test. All tests were carried out using a 5% level of significance, and no adjustment was made for multiple testing and missing values were not imputed.
Economic analysis
In addition to the above, an economic analysis of LSCAD/CAM was completed compared with traditional casting. The work undertook a cost-effectiveness analysis using the time to functional AFO as the primary outcome and sought to assess costs using a societal perspective. An individual resource collection tool was developed through pilot work with parent-representatives and designed to fully capture all the elements. In addition to intervention costs and health care resource use, this included the cost of patient travel to the hospital and productivity losses associated with time off work due to treatment and care. The time horizon was 12 months; as such, costs and outcomes were not discounted.
Resource use was collected by various means, and in most cases, national unit costs were assigned. Resource use relating to the scanning and casting sessions and the manufacture of a device were recorded by the attending orthotist. This included data on the time spent carrying out specific tasks, staff attending, the use of consumables and the duration of group sessions. Similarly, these data were collected for fittings and adjustments.
Follow-up data were collected from patients at 3, 6, 9 and 12 months. These included medication, time off work, service use and associated travel costs. Published unit costs from NHS Staff Earnings Estimates, 11 Personal Social Services Research Unit 12 and British National Formulary 13 were applied as appropriate. All costings were made at 2011 prices.
An incremental cost-effectiveness ratio (ICER) – the cost per day saved of LSCAD/CAM compared with traditional casting – was estimated, and bootstrapping was used to estimate confidence intervals (CIs) for the ICER. Results are based on a complete case analysis.
Results
Subjects
During the study, the provision of orthotic services at the Trust was reorganised following a ‘sustainable services review’, and the study orthotists left the Trust. At this point, 92 (69% of total) subjects had been recruited into the study. New study orthotists were less experienced in laser scanning and had less time to devote to the study, so the ability to recruit patients into the trial became more difficult. Despite an extension to the study duration, widening of the inclusion criteria from 18 to 21 years of age (provided the subject was in education) and a move to bring more AFO moulding ‘in house’ the sample size target of 150 subjects was not met. A total of 151 individuals were approached and 134 subjects were recruited into the study (two participants were inadvertently randomised twice; in both cases, the first allocation was used). The flow of subjects and numbers randomised into each group, including those that received the planned intervention, are presented in Figure 1. The majority of subjects were affected by cerebral palsy with no statistically significant differences between groups (Table 1). In total, 96 (72%) patients required bilateral AFOs, while assistive devices were used by 42% of the patients.
Consolidated Standards of Reporting Trials (CONSORT) flow chart.
Recruitment allocation by diagnostic category.
CP: cerebral palsy.
Ages ranged from 1.5 to 21.3 years with a mean age of 10.7 years (standard deviation (SD) = 4.9 years) and 9.8 years (SD = 4.1 years) for males and females, respectively. Between the two allocations, 59 girls and 75 boys were evenly distributed.
Protocol deviations
A variety of factors including inability to scan a child’s limb, technical problems with the scanning equipment and errors in allocating the manufacturing method correctly in the factory contributed to a total of 15 crossovers from one arm of the study to the other (Table 2 and Figure 1, CONSORT diagram). The likelihood of crossover occurring was uninfluenced by the need for a bilateral device, sex, previous serial casting, botulinum toxin treatment or calf surgery. All analyses have been performed on an intention-to-treat basis.
Crossovers from allocated method to other method.
CP: cerebral palsy; CTEV: congenital talipes equinovarus.
Time-saving and quality
Rectification and moulding of AFO (primary outcome measure)
Data were available for 83 of the 109 AFOs manufactured using the scanning method and 110 of the 121 AFOs manufactured using the casting method. The mean rectification and moulding time for cast AFOs was 55.1 (SD = 26.0) min and for LSCAD/CAM AFOs was 26.9 (SD = 12.2) min. This represented a reduction in time by 28.2 min or 51% in the LSCAD/CAM group (p < 0.001, two-sample t-test), 95% CI of difference = 22.6 to 33.8.
2(a). Time taken to scan and cast limbs
All subjects were both cast and scanned and data were available for all 134 subjects. There was no significant difference in the time taken to cast or scan the limbs (p = 0.056, paired t-test) with the mean times being 12.5 (SD = 4.9) min and 11.1 (SD = 9.5) min for cast and scan, respectively, and a mean difference of 1.33 (SD = 10.07), 95% CI of paired difference = −0.4 to 3.1.
Evaluation of average casting time throughout the study revealed no training effect or trend in casting time. However, a striking difference was seen over time in the time taken to scan as a result of unplanned staff turnover (Figure 2). If we only include scan times for the two orthotists who had received more training and were already familiar with the technique prior to the start of recruitment, a statistically significant difference (p < 0.001, two-sample t-test) was then seen with average times 13.1 (SD = 4.4) min for casting and 8.9 (SD = 2.9) min for scans.
Trends in first visit scan time during study.
2(b). Supply and fit of AFO(s)
At the second visit, the AFO(s) were provided to the patient. A significant proportion (75%) of patients guessed their actual method of fabrication correctly (p < 0.001, chi-squared test). There were no significant differences in delays in supply between the two methods (p = 0.711, logrank test). A significantly higher proportion of scan-based AFOs failed to meet the specification stipulated by the scanning orthotist (see Table 3) (p = 0.004, chi-squared test). Remaking was required in 17% of scan-based AFOs and 3% of cast-based AFOs (p < 0.001, chi-squared test). While none of the cast patients required recasting, four scanned patients required a rescan, one scanned patient was subsequently cast and eight scanned patients required an adaptation of the original scan. Although there was an increased incidence of problems with the scan-based AFOs, the time taken to fit the device was not significantly different between the groups.
Incidence of problems at delivery of first device.
2(c). Time to deliver device
Data were available for 68 of the subjects allocated to the cast group and 62 subjects allocated to the scanned group. No significant effects were seen (p = 0.12, logrank test) of the allocated group on the time taken from casting/scanning to delivery of a well-fitting AFO/AFOs. The mean (SD) number of days was 46.9 (45.3) and 37.9 (31.9) for scan and cast, respectively, with 95% CI for the difference (scan − cast) of −4.5 to 22.5. The time from receipt of the order in the manufacturing unit to the date the order was completed was close in the two groups (13 ± 6 days in the cast group, 15 ± 8.2 days in the scanned group).
2(d). Length of life of AFO
The comparison of length of AFO life was carried out using Kaplan–Meier survival plots (Figure 3) and a logrank test. This is due to the fact that times to failure are only recorded up to 1 year and hence those surviving beyond 1 year are censored. Of the 70 subjects, 29 were allocated to casting, and 29 of the 64 subjects allocated to scanning returned to clinic for a new AFO within the first year, and around half of these were as a result of growth (see Table 4). No significant difference was seen in the length of life of AFOs manufactured using the casting or scanning method (p = 0.57, logrank test).
Kaplan–Meier plot of time to failure (days) of the device for the two groups.
Reasons for replacement of first AFO.
AFO: ankle–foot orthosis.
Patient preference/satisfaction
After experiencing both casting and scanning (in random order), 70% of the patients said they preferred being scanned to having the limbs cast in plaster. A Mann–Whitney U test to evaluate differences in responses to 4-point Likert-type scale (Strongly Agree–Strongly Disagree) found no significant differences between the allocated groups when applied individually to the 9 ‘Satisfaction with Device’ questions and the 10 ‘Satisfaction with Service’ questions at each time point (3-, 6- and 12-month follow-up). ‘Agree’ was the predominant category selected by respondents for all questions, thus indicating a reasonable level of satisfaction with the AFOs and services.
Economic results
The economic results for plaster versus scan show no advantage of scan over plaster in terms of cost-effectiveness. The average total direct cost of providing an AFO (excluding capital costs) was £586 in the scanning group and £416 in the casting group. This includes consumables, overheads and staff costs for assessments (scanning = £98, casting = £86); manufacture (scanning = £364, casting = £206) and fitting and adjustments (scanning = £120, casting = £117).
The cost of the scanner was taken as £15,000. We assumed that the scanner would completely depreciate in value over 8 years, and that it would be used for the production of around 775 AFOs per year, with 96% of the scanner’s use attributable to AFOs. A discount rate of 3.5% was applied. This gives an equivalent annual capital cost per AFO produced of just £2.69. The standard cost of manufacture for a single AFO was £168.66 for scan-based devices and £90.90 for cast-based devices, though some individuals required multiple AFOs and re-manufacturing.
The average total societal cost for individuals in the scanning group was £2859, and for those in the casting group was £2824. The observed incremental cost of scanning for individuals with complete cost and outcome data was £70. For individuals with complete data, the incremental effect of scanning was an increase in the time to delivery of 10 days. Traditional casting was found to strongly dominate LSCAD/CAM. The observed ICER is therefore negative, with a 95% bias-corrected and accelerated bootstrapped CI of −£128 to £1.
Discussion
Synopsis of key findings
The study does not support the introduction of laser scanning and LSCAD/CAM manufacture for AFO provision in children and young people on the basis of time-savings or improved quality of device. Despite a reduction of around 50% in the time spent in the rectification and moulding processes, this benefit was not transferred to the patient where an average increase in the time to delivery of the devices of 9 days was seen for LSCAD/CAM. This delay was probably due to the proportion of devices failing to meet the required specification being considerably higher after manufacture by LSCAD/CAM than after plaster casting, and thus requiring a remake and lengthening the time to delivery. However, there is potential for improvement in the design of AFOs manufactured by LSCAD/CAM, and this study has highlighted the importance of training for orthotists in this technique.
Pros and cons of LSCAD/CAM compared to plaster casting
Anecdotally, several patients reported that they believed their AFO was produced by a scan because it fitted so well. Conversely, excessively intimate fits, particularly around the malleoli, make donning difficult, and spasticity or calf tightness complicates the process. An intimate fit also presents problems with the longevity of the device in the presence of growth. During the study, a policy was adopted of enlarging the scan by 1% in all dimensions; this was found to give a closeness of fit comparable to a conventionally fabricated device to give room for growth. There is clear potential for improvements in the design of AFOs manufactured by LSCAD/CAM.
The inability of the orthotist to correctly position the patient during the scanning process makes the correction of postural deformity totally dependent on the rectification process, whereas in the process of casting, the foot can be held in a corrected position rendering the rectification process less complex and critical.
Remaking is a costly process. Given that 17% of the scanned patients required a rescan, re-cast or adaptation of the original scan, versus 3% of the plastered patients, plastering is again the favourable option. Where a remake is required, the hospital bears the staff, capital and consumable costs of this, while the families incur increased travel costs and productivity losses. In addition, there is a large difference in the manufacturing costs between AFOs made by scanning and by casting, with scanning being over £70 more expensive.
Being able to store the scan for enlargement in response to growth does offer a potential advantage in terms of a reduction in return visits for families if a previous scan can be stretched and an AFO fabricated. However, given the increased frequency of rescans and defects seen in this study, the cost of such a strategy would be currently unacceptable and the time-saving for families would be reduced.
Reduction in cost due to time-savings in rectification and moulding may be negated by an increase in costs due to problems at delivery of device (e.g. rescanning). Technical difficulties with the hardware and software led to occasional failures of scanning, particularly early on in the study when the technology was relatively new. While the trained orthotists were able to scan in a consistently short period of time, the locums and new staff members frequently had prolonged setup times and were uncomfortable with the process. A study by Telfer et al. 14 demonstrated that only the more experienced CAD operator was able to achieve excellent inter- and intra-operator reliability. However, in this study, the imperfections in the resulting AFOs were seen equally in scans performed by ‘trained’ orthotists as well as those not previously trained in the use of the technology. The consistent times for casting across all orthotists and throughout the study demonstrated the ubiquitous nature of the manual skill of casting instilled in trainee orthotists in the United Kingdom.
Limitations of the study
The study had a number of limitations resulting from factors associated with the pragmatic nature of the trial and unexpected difficulties encountered during the study. While the personnel disruption encountered during the study might seem to detract from the information derived, this was a pragmatic trial of a technology intended to evaluate a ‘real-life’ situation for immediate use in patients.
Since the study started in 2008, technology has progressed and more advanced scanners and fabrication methods are available. 15 The study failed to recruit the target number of 150 participants. However, the eventual sample size was sufficient to achieve 80% power.
A high number of participants successfully guessed the manufacturing method applied to their AFOs (75%). The majority of participants in this study were used to having cast AFOs; thus, many could detect the subtle differences in intimacy of fit of AFOs produced by LSCAD/CAM.
The digital nature of the scanning process allows the initial assessment of the patient to occur remote to the manufacturing facility with the time and expense of transport of fragile casts being eliminated by electronic transmission of the digital mesh. This cost is not applicable at our centre due to the availability of an on-site facility, but would be more relevant where peripatetic orthotic services are in place.
Conclusion
This study demonstrated that compared with conventional casting techniques, laser scan–based AFO manufacture neither significantly improved the quality of the final product nor delivered a useful saving in time. Time-savings of around 30 min were achieved during the rectification and moulding processes in the LSCAD/CAM group, and although this may have implications for staff costs, this would be of no real benefit to the patient. Significant improvements to the quality of devices manufactured using the LSCAD/CAM method would be required before any overall benefit would be seen.
Footnotes
Acknowledgements
Jayne Edwards coordinated subject recruitment and undertook the data collection and collation. Lance Jones and Andy Dewsbury managed the orthotic workshop and implemented the randomisation provided by the York Trials Unit. We thank Professor A Heinemann for giving us permission to use the OPUS instrument. The orthotists taking part in the trial included Gary Gordon, Jose Perez, Phillip Brereton, Jane Dewsbury, Nicole Galea, Stephanie Conover and Cathy Madine ORLAU staff, Karen Edwards and Caroline Stewart, offered ORLAU facilities to support the study.
Author contribution
All authors contributed equally in the preparation of this manuscript.
Declaration of conflicting interests
The authors declare that there is no conflict of interest.
Funding
This work was supported by the National Institute for Health Research (reference no. PB-PG-0706-10626).