Abstract
There is increasing recognition of the credibility and utility of patient reported outcome measures, both in research and as routine quality indicators. This paper reviews the development of a questionnaire for children with foot or ankle problems and integrates the findings from three previously published studies in a cohesive way for the orthotic community. The Oxford Ankle Foot Questionnaire for Children was designed to evaluate the effectiveness of interventions. The development process was conducted in three phases. First the items were devised through focus groups with children affected by foot and ankle problems, and their parents. Second, test versions of child and parent questionnaires were evaluated to enable scales to be developed and tested for validity and reliability. Finally, findings from a prospective study assessing how scores changed over time and/or with treatment supported the longitudinal validity and responsiveness of the scales. The questionnaire offers an inexpensive and expedient means to evaluate the effectiveness of orthoses and other interventions used to treat children's foot or ankle problems. The Oxford Ankle Foot Questionnaire for Children has broad utility both in routine clinical settings, or applied research to evaluate treatment programmes and interventions used in paediatric orthopaedics, trauma and rheumatology.
Introduction
Children's lives can be affected by a variety of foot and ankle problems. These include benign postural deformities, structural problems due to or secondary to congenital conditions, acquired or inflammatory conditions, and trauma. Foot and ankle problems are common in paediatric orthopaedics and rheumatology. Treatment aims to prevent, correct or accommodate deformity, manage related symptoms, and improve physical function, and may include one or more modalities including, advice, orthoses, casting, medication, physiotherapy, or surgery. Typical clinical assessment methods include measurement of range of motion, muscle strength, and the use of X-rays, clinical rating scales and gait analysis. However these assessments do not capture the patients' perspectives and may not accurately reflect how children function in their usual environments, at home and at school.
Orthoses are widely prescribed and provided for children's foot and ankle problems but limited evidence exists of the effectiveness of the intervention. 1 This puts children at risk of undergoing ineffective treatments and potentially wasting health service and family resources. There is a prevailing consensus that better evidence is generally required for orthotic interventions. However, in the past, one of the barriers to clinical research and audit in orthotics has been the lack of appropriate outcome measures that are easily implemented.
Patient reported outcome measures (PROMs) have become established as credible and useful instruments to evaluate the effectiveness of interventions both in research and as routine quality indicators. PROMs have been shown to be particularly useful and influential in orthopaedics for research, benchmarking and clinical audit. 2 In the USA, the Food & Drug Administration (FDA) recognize the importance of PROMs and have issued draft guidance on the use and reporting of such measures in clinical trials. 3 In the UK, the National Institute of Clinical Excellence (NICE) also considers the results of PROMs as part of its technology appraisals; 4 and the UK Government's Department of Health is currently pursuing a programme of implementing routine use of PROMs in the National Health Service for selected elective procedures, 5 and considering using PROMs with children. 6
Patient reported outcome measures can be used to assess a variety of constructs with different populations to answer specific questions. There are generic instruments and site (part of the body) or condition-specific instruments. The constructs assessed may be health status and/or subjective wellbeing; there are single dimension-specific and multi-dimensional instruments. Instruments that have been used to assess outcomes in orthotics include ‘satisfaction with the processes of care’ such as SERVQUAL, 7 and an orthotic or prosthetic treatment-specific instrument, the Orthotic & Prosthetic User Survey. 8 One limitation of outcome measures specific to particular modes of intervention (e.g., orthotic-specific) is that it does not enable different interventions for the same condition to be compared (e.g., orthoses, surgery, etc.).
Criteria for assessing the utility of PROMs for a particular application are the appropriateness, reliability, validity, responsiveness, precision, interpretability, acceptability, and feasibility. 9 Instruments must be shown to be valid for the population with whom they are to be used, and children pose a special case. Qualitative assessment of the dimensions assessed by PROMs for children and adults suggest that they address similar constructs, namely physiological, psychological, and social well-being and functional ability. 10 Nevertheless, the social context of children's lives differs from that of adults because of their dependence on their family, the relative importance of friends, experiences of school and play, and their aspirations for the future. 11 Hence it is broadly accepted that adult questionnaires are not appropriate for children because of deficiencies in their content and face validity. 12,13
While several PROMs have been developed for adults with foot and ankle problems, 14 they have not been shown to be appropriate, valid and reliable for use with children. To date, only two such instruments are available relating to children; both are condition-specific either to Juvenile Arthritis Foot Disability Index, 15 and clubfoot, 16 and the latter was devised only for parents to complete. The Oxford Ankle Foot Questionnaire for Children was developed as a site-specific (ankle/foot) instrument to provide an inexpensive and expedient method for assessing health status and evaluating outcomes from the child's perspective. The purpose of this paper is to integrate knowledge on this outcome measure, which has previously been published in three different journals, and present it in a cohesive way for the orthotic community. The development of the questionnaire was conducted in three phases, and approved by the Oxfordshire Research Ethics Committee (06/Q1604/38).
Questionnaire development
Phase 1
The first stage in developing a patient-reported instrument is to gain an understanding of the patients' experience of their condition through qualitative research. Focus groups were convened with children affected by foot and ankle problems, and their parents. 17 This was a qualitative study involving children with a variety of foot and ankle problems in three age groups: 5–7, 8–11 and 12–15 years; parents took part in separate concurrent focus groups. Learning from the experiences of colleagues, 18,19 the children's focus groups were child-centred and involved creative activities. The activities included agreeing or disagreeing with statements about children with foot and ankle problems and storytelling by exploring a typical ‘day in the life’ of a child with a foot or ankle problem. The parent groups followed a broadly similar structure. The focus groups were audio-recorded, transcribed and analyzed to identify themes participants reported as important.
Consistent themes identified by all groups were: Physical symptoms, activity limitations, reduced participation in certain contexts, and self-consciousness due to appearance and other people's attitudes. There were few differences in the issues raised by children and parents, or between each age-group, although the life-situations children encounter become more complex as they get older. This work identified a list of issues that the affected children themselves felt to be important, and the results formed the basis of the questionnaire. 17
Phase 2
In the second stage of development, test versions of the child and parent questionnaires were evaluated to enable scales to be developed. Questions (items) were constructed using the 25 issues raised by children and parents in the focus groups; three additional items were suggested by clinicians concerning the appearance of the foot, the way in which the child walked, and swelling, giving a total of 28 items. Item responses were rated on a five-point scale indicating how frequently the issue affected the child, from never (score = 0) to always (score = 4). A survey of 158 children and their parents was carried out to determine the scaling, reliability and validity of the instrument. 20 We used exploratory factor analysis and Rasch analysis using both the children's and parent's datasets to devise and refine three domain scales: Physical (six items), Emotional (four items), and School & Play (four items). As in classical approaches to measurement, domain scores were calculated as the total of the scale item scores and were transformed to a percentage scale (0-100), where a higher score represented better functioning. Although an item asking about ‘being able to wear the shoes you want’ did not fit with the scales, it was salient to many children and to most girls. The item had strong face validity and was included as a categorical descriptive variable but not allied to any domain scale. The remaining items were either redundant or did not improve the scaling and were dropped.
The three domain scales were shown to be internally consistent. The scales' scores were also found to be stable at retest within two weeks, and to vary little whether reported by child or parent. 20 Face and content validity were also found to be satisfactory, and construct validity was demonstrated by appropriate convergence and divergence with the domain scores from the Kidscreen questionnaire (a generic quality of life measure for children). 21
Phase 3
In the third phase, a prospective study was conducted with 80 children attending trauma and elective orthopaedic clinics to assess responsiveness and longitudinal validity of the domain scales. 22 We hypothesized that children attending trauma clinics (i.e., those with acute health problems) would have worse health and improve more rapidly over the course of the study than children who attended elective (routinely scheduled) outpatient clinics.
Tests of longitudinal validity assess how scale scores change over time and with treatment, and examine whether these changes reflect what would be expected. Children and parents completed questionnaires at an initial outpatient appointment (baseline). They were then asked to complete and return a second set of questionnaires by mail within two weeks (retest), and mailed a third set of questionnaires to complete again after two months (follow-up). The follow-up questionnaires included an additional global rating of change ‘transition’ item asking respondents to indicate whether they thought, overall, the foot or ankle problem was about the same, better or worse, on a seven-point scale, using increments of ‘slightly’, ‘quite a bit’ and ‘a lot’ better or worse. The global rating of change provided a patient-based anchor for comparing the magnitude of changes in domain scores. Child and parent questionnaires were completed and returned at retest by 34/80 cases (44%), and at follow-up by 53/80 (66%).
Methods for evaluating responsiveness are either distribution-based (effect size, minimal detectable change, index of half a standard deviation) or anchor-based (minimal important difference). However, it is recognized that estimates of minimal change tend to vary across studies owing to differences in population, interventions and context. For this reason Revicki et al. recommend that several indices of minimal change are calculated, and for them to triangulate towards a range of values, in which confidence increases with replication. 23
In our study, apoorers we hypothesized, trauma patients had scores than elective patients at baseline, and showed a greater magnitude of improvement at follow-up. For trauma patients, mean changes in percentage scores were large and all effect sizes were large (>0.8). For elective patients, the mean improvement in scores and effect sizes were more moderate (0.5). The Minimal Detectable Change (MDC90), which is an indication of measurement error, ranged from 6–8 on the percentage scores. The Minimal Important Difference could only be calculated for elective patients (nine child and 13 parent ratings), these ranged from 7–17; none of the responses relating to the trauma patients indicated a slight change. Half the standard deviation of the baseline domain scores, which is often found to closely equate to Minimum Important Differences, 24 ranged from 11–18 on the percentage scale.
The findings of this phase of development supported the responsiveness and longitudinal validity of the domain scales. Changes in domain scores of, or exceeding, the MDC90 (6–8 on the percentage scale) are likely to be beyond measurement error. Further work is required to refine the estimate of smallest change that can be considered important. Nevertheless, the minimally important changes in scores reported generally exceeded potential measurement error and can therefore be considered reliable.
Potential applications
Our work to date has been primarily concerned with the psychometric performance of the instrument (Table 1). Having established that the Oxford Ankle Foot Questionnaire meets prevailing standards, the questionnaire can now be used for clinical and research purposes, to evaluate the effectiveness of interventions used to treat foot or ankle problems in children between 5 and 16 years old. The application of this instrument will allow the evaluation of interventions from a different aspect than has been done in the past, i.e., from the perspective of what is important to the child. The instrument is appropriate for a range of diagnoses, exemplified by the diversity of conditions of the 250 children who took part in development 17,20,22 (Table II). However, it is not suitable for children who are unable to walk, or who have a significant proximal lower limb component to their disability.
Summary of evidence to support psychometric properties of the Oxford Ankle Foot Questionnaire.
ICC, Intraclass Correlation Coefficient.
The Oxford Ankle Foot Questionnaire makes it possible to evaluate the effectiveness of many applications for lower limb orthoses with children relatively expediently and inexpensively. For example, whilst it is already possible to assess the change in gait pattern of a child when they are using an AFO using gait analysis, this questionnaire will be able to provide information on how the child feels it makes a difference not only from a functional perspective, but also emotionally and socially.
It will also be possible to assess the use of orthoses in conditions that cause pain and loss of function; either diffuse pain, for example hind-foot bony coalitions, or locally painful conditions such as plantar fasciitis or sesamoiditis. Currently this evaluation is often performed by subjective reports of pain relief and functional improvement, but rarely by including all of the criteria that have been found to be important to children that this questionnaire allows. Potentially, different types of orthoses could be compared using this questionnaire to see which children find more beneficial and acceptable. Orthoses could be compared with other interventions such as casting to assess which children rate most highly when recovering from ankle and/or foot trauma, or orthoses compared with medication for managing inflammatory conditions.
Part of our ongoing work will be to gather information about the ranges of scores, and change in scores specific to different ankle and foot conditions. Further work can be done to determine whether particular domains are more important in some of these conditions, which will allow treatment to be tailored to address these domains. For instance, if it is evident that in hemiplegia it is the school and play aspects, rather than the physical aspects of using an orthosis that are poor, it may be possible to change the orthosis, or the way that the orthosis is used for that group of children.
Examples of conditions for which the Oxford Ankle Foot Questionnaire has been used.
Fractures include distal tibia and/or fibula, calcaneum, metatarsal
Other trauma includes laceration, joint dislocation.
The Oxford Ankle Foot Questionnaire will be particularly helpful in situations where the clinical indications to intervene are ‘soft’, rather than clearly defined. An example is a child who is high-functioning, where the use of orthoses is debated between the prevention of deformity versus limiting function through immobilizing joints. There are other common conditions where the use of orthoses is a subject of debate amongst clinicians. An example is the use of foot orthoses in flexible flat feet. Although the orthoses may not be changing the shape of the feet when the orthosis is not used, they may be helping some of these children function better or improve other aspects of their life affected by having flat feet. Alternatively, using orthoses may make them more self-conscious and affect whether they can wear the type of shoes they prefer.
The Oxford Ankle Foot Questionnaire has broad utility both in routine clinical settings or applied research comparing different treatment programmes used in paediatric orthopaedics, trauma and rheumatology.
Acknowledgements
This development of the Oxford Ankle Foot Questionnaire was funded by the UK Medical Research Council. The child and parent versions of the questionnaires can be viewed at http://www.publichealth.ox.ac.uk/units/hsru/Ankle/Ankle. Licences to use the Oxford Ankle Foot Questionnaires can be obtained by contacting Isis Innovation www.isis-innovation.com/licensing/healthoutcomes.