Validation of the activPAL activity monitor in children with hemiplegic gait patterns resultant from cerebral palsy

Abstract

Background:

Current investigation of treatment outcomes by clinicians is currently hampered by a lack of clinically viable tools. The use of activity monitors specifically validated for a population could help resolve this situation.

Objectives:

The purpose of the study was to validate an activity monitor for monitoring children with cerebral palsy. The study was designed to validate the duration of time spent sitting, standing and walking and the number of steps taken when being measured by the activPAL activity monitor.

Study design:

A validation study was undertaken.

Methods:

Observations of participants were carried out while completing a specifically designed activity course using video footage, which were then compared to the output from the activity monitor.

Results:

The activity monitor was found to be valid for the time spent standing and walking, the number of steps taken and the number of transitions.

Conclusions:

The results demonstrated that the monitor may prove useful to clinicians as a measurement outcome device for children with hemiplegic gait patterns resultant from cerebral palsy. However, the sensitivity of the device is variable, and further investigations are necessary to confirm it would also be able to detect minor changes after interventions.

Clinical relevance

The monitor may provide clinicians with a simplistic tool that is easily utilised, to enable audit exercises of current and future treatments.

Background

Cerebral palsy (CP) is the most common cause of physical disability in children, with a reported incidence of 2.0–2.5 per 1000 live births. ¹ The motor cortex area of the brain is affected both during and prior to birth and also up to 2 years after birth. ² This results in impairments to movement and postural control. ³ Although the initial brain injury remains static, ² these impairments, including muscle weakness, contractures and balance problems, are progressive in nature. ⁴

When compared to subjects with normal development, children and adolescents with CP have been shown to have lower activity levels,^5,6 which has a significant effect on their well-being. However, the effects of exercise on musculoskeletal impairment and pain remain unknown in this patient group. ⁷ Studies^8,9 have also demonstrated that those children with CP fare worse than their age-gender-matched control groups with respect to quality of life (QOL). However, current understanding of this impact on QOL is limited. ¹⁰

One aspect which may have an impact on the QOL of children with CP is their inability to undertake the same level of activity as their peers. In order to quantify this, activity monitors have therefore been used to record parameters such as the mode, duration, frequency, intensity and the domain when recording an appropriate activity in order to ascertain an overall activity level for patient groups. ¹¹ When presented with a gait pattern such as one of those seen in hemiplegia resulting from CP, the decision on which activity to record is a complex one.

When monitoring activity, an increase in the number of steps or in the step length may be considered as an improvement in the subject’s gait parameters. If the number of steps alone was taken as an indicator of improved performance, a clinician may falsely conclude that an alteration in activity had occurred. ¹² Monitors that are only capable of monitoring the number of steps may therefore be deemed to be unsuitable for monitoring activity in this population. Due to the possible misinterpretation of isolated activity data, it is therefore wise to measure multiple facets of activity to reduce the possibility of false conclusions being drawn. A monitor capable of measuring the number of transitions (e.g. from sitting to standing to walking), by detecting the type of posture adopted and duration of activities performed, as well as step counts, would reduce the possibility of false conclusions being drawn from the data.

When the ultimate purpose of the monitor is to collect data from children within the community, a number of other factors are important. The monitor must be light, small and robust, and its placement upon the body must ensure it can be worn discretely. It must also be able to monitor and store data over an appropriate length of time. The downloading of data must be efficient, and the software must be capable of representing the data in an appropriate manner in order to be quickly and easily interpreted by clinicians.

Relatively few studies have been published that describe the validation of activity monitors for the CP population.^6,12–14 The validation protocols of these studies have varied, including comparisons against the System for Observing Fitness Instruction (SOFIT) ¹³ and observation of 1 h of play. ⁶ Two studies^12,14 employed structured activity laps where observations occurred. In a study by Kanoun ¹⁵ within the healthy population, walking velocities were also calculated. It was determined that velocity had an effect upon the accuracy of the monitor.

Comparison of data obtained from a monitor against observation is considered to be the appropriate criterion measure. ¹⁶ There are few validation studies that have employed observation as the criterion measure for the CP population, including Pirpiris and Graham, ⁶ who employed observation of 1 h of play, and Kuo et al. ¹² and Mackey et al., ¹⁴ who employed structured activity laps where observation occurred. Pirpiris and Graham ⁶ found the Uptimer (National Aging Research Institute of Melbourne) to be both valid and reliable for the amount of time that children spent on their feet each day. The study did not validate the activities of walking and standing as distinct activities, and the monitor was unable to categorise the time spent sitting or lying down. The study by Kuo et al. ¹² investigated the accuracy of the AMP331 and the Minimod monitors to measure distance walked and the number of steps. Although the authors demonstrated accuracy of the data collected, both monitors failed to detect a number of walking events. Mackey et al., ¹⁴ when validating the Intelligent Device for Energy Expenditure and Activity (IDEEA) monitor, found the monitor to have greater difficulty in detecting walking activities than static activities, and in some instances failed to detect a specific activity. It was also stated ‘that many of the younger participants refused to wear the device for an extended period of time because of the wires’.

The evidence available in the literature therefore suggests that activity monitors lack the ability to accurately monitor all activities relevant to the paediatric CP population.

There are also compliance issues. The aim of this study was therefore to investigate whether a monitor, which was identified as having the potential to accurately measure critical activity events by children with CP, could be validated for experimental use. The hypothesis was that validation of an activity monitor may be confirmed by comparing manual observations of video footage to data acquired from an activity monitor using a specifically designed indoor activity course which comprised critical activities such as walking, standing and sitting with transitions between them.

The monitor chosen was the activPAL activity monitor (PAL Technologies Ltd, Glasgow, Scotland). It was chosen as it had a proven validity and inter-device reliability in a healthy population ¹⁷ and had the potential capability to monitor and measure children’s activities. The monitor consisted of a uniaxial accelerometer attached to the user’s thigh, which produced a real-time signal that was related to thigh inclination and limb movement. The monitor was small (35 mm × 53 mm × 7 mm) and lightweight (20 g), and capable of recording the time spent sitting, standing and walking (classed as the activity duration), as well as counting the number of transfers between activities or postures (classed as the activity frequency).

During walking, the pattern of the signal acquired by the activPAL monitor allows the number of steps taken to be determined, and cadence may also be calculated with respect to time. It records processed data at a frequency of 10 Hz and has a memory capacity of 4 MB, making it feasible to record over a period of 7 days.

In a previous validation study of the activPAL monitor within the CP population, Tang ¹⁸ found a lack of agreement for step counts and activity classification. The data were collected within a gait laboratory while undertaking gait assessments, and observation was the criterion measure employed. Large percentage errors were demonstrated in the time spent sitting and standing where a short-duration activity state existed within the confines of the gait laboratory. The study recommended future protocol studies should take place within a set physical activity sequence to reduce potential measurement error within the observer timing recordings taken. Further recommendations were made that more than one observer should identify the timings to remove bias. The above protocol recommendations were therefore incorporated within this study in an attempt to minimise associated errors.

Methods

Ethical approval for this study was granted by University of Salford Ethics Committee and the National Research Ethics Committee. In all, 10 children with CP were recruited from local schools for children with physical disabilities within the Greater Manchester region, and each parent signed an informed consent form before data collection began.

Volunteer participants were included in the study if they were diagnosed with CP and were aged between 4 and 18 years and walked with a type 1, 2 or 3 hemiplegic gait pattern and gross motor function classification (Table 1) as defined by Winters et al. ¹⁹ It was ensured that they were capable of following simple instructions and were able to walk independently with or without an ankle–foot orthosis (AFO). Participants were excluded from the study if they could not walk independently or had poor cognition.

Table 1.

Characteristics of the participants.

Participant classification table
Participant	Gait pattern 19	GMFC 20
1	1	1
2	1	1
3	2a	2
4	1	1
5	1	1
6	2b	2
7	2a	2
8	2a	2
9	2b	2
10	3	3

GMFC: Gross Motor Function Classification.

Each participant walked with their previously prescribed AFO during the activity trials and had an activPAL monitor placed on the affected limb at mid-thigh level. The participant was then asked to walk around a short, specifically designed indoor course (Figure 1). The course involved three periods of walking, interspersed with two set time periods of sitting and one of standing. During the activity session, an observer followed the participant with a video camera, which also captured the times during and between activities. The observation activity categories measured were sitting, sit to stand, walking, stopping and sitting and static stance.

Figure 1.

The activity course.

Data analysis

The sensitivity of the monitor was measured by analysing the degree to which each observation activity category was detected correctly by the monitor.

A direct comparison of readings taken by the activity monitor within the structured activity were analysed against observations from the video recording made by two independent observers. A guidance sheet was provided to each of the observers to provide clarity upon where postural changes occurred in respect of timings recorded, and a practice session was held to ensure accurate foot step recognition. The authors did not have access to the algorithm of the device. It was not the objective of the authors to match the guidance of the observers to the monitor’s algorithm as this would ensure a false accuracy.

The validity of the activPAL monitor was examined using the Bland–Altman level of agreement ²¹ and by calculating the percentage agreement and the sensitivity. The walking velocities were also calculated to determine whether velocity had an effect upon the accuracy of the monitor, as demonstrated in the literature. ¹⁵

Results

Discussion

The range of upper level of agreement (ULOA) and the lower level of agreement (LLOA) scores (Table 2) testify that a high level of agreement existed for the walk timings and the number of steps. Moderate agreement existed for the stand timings and low agreement existed for the sit timings.

Table 2.

The Bland–Altman analysis for 10 participants.

	Sit	Stand	Walk	Step
Mean difference	−6.8	5.9	−2.2	−3.2
ULOA	18.5	19.1	7.8	4.5
LLOA	−32.1	−7.3	−12.3	−10.9

ULOA: upper level of agreement; LLOA: lower level of agreement.

The Bland–Altman graphs reveal a small spread of the limit of agreement for the walk and step measurements, but when standing the limit of agreement was increased. This indicated a low random error when compared to the sit measures. Greater systematic variation existed for the sit and stand activities (Figure 2). The mean difference of the total times recorded for sitting, standing and walking was 2.36 s (standard deviation (SD) = 2.70 s). The graphs also revealed that no proportional error existed and the differences between the two measures were not related to the magnitude of the measurement. In addition, no systematic errors were detected.

Figure 2.

Bland–Altman plots: (a) sitting; (b) walking; (c) standing; (d) number of steps measured by direct observation and activPAL.

The mean percentage agreement (Table 3) was 86.5%. Participants numbered 2, 1 and 6 had the largest recorded errors. The monitor failed to measure two episodes of sitting with participant 2 and one episode of sitting with participant 1 and participant 6, significantly affecting the overall agreement for these two participants. The sensitivity (Table 4) was also similarly affected.

Table 3.

The mean percentage agreement (the number of transitions between activities).

Agreement: hit/miss
	P1	P2	P3	P4	P5	P6	P7	P8	P9	P10	Mean
Total seconds	174.00	204.00	176.00	172.00	171.00	229.00	198.00	239.00	334.00	461.00
Hit	131.00	124.00	170.00	162.00	160.00	192.00	183.00	220.00	283.00	420.00
Miss	43.00	80.00	6.00	10.00	11.00	37.00	15.00	19.00	51.00	41.00
Agreement	75.29	60.78	96.59	94.19	93.57	83.84	92.42	92.05	84.73	91.11	86.46

The number of transitions recorded was accurate for eight of the participants. For two of the participants, inaccuracies occurred on one transition, due to the misclassification of sitting as standing.

Table 4.

The sensitivity.

Sensitivity for each participant’s activity
	P1	P2	P3	P4	P5	P6	P7	P8	P9	P10	Mean
Walk	89.89	97.14	96.43	89.41	90.12	95.45	86.14	89.17	75.26	87.69	89.67
Stand	100.00	72.73	100.00	96.77	93.10	100.00	100.00	93.33	100.00	94.59	95.05
Sit	46.77	0.00	100.00	100.00	95.16	52.31	98.46	95.95	96.47	96.64	78.18
Average	78.89	56.62	98.81	95.39	92.79	82.59	94.87	92.82	90.58	92.97	87.63

The fact that a ‘sit’ was misclassified as a stand resulted in greater intervals of standing time being recorded. This resulted in a mean difference of 5.9% for the total time spent standing (Table 2). This was a larger misclassification than that found by Tang ¹⁸ who recorded a mean difference of 1.4% for total time spent standing in a CP population. During analysis of the video footage, it was noted that misclassification had occurred when the participant had not sat fully on the seat of the chair. This could account for the large differences in results between the two studies. It could be argued that if the chairs had been of a smaller size the children would have been less inclined to perch, resulting in increased monitor accuracy in this respect.

The mean sensitivity for sitting, standing and walking was 78.2%, 95.1% and 89.7%, respectively. These results for standing and walking are comparable with previous results, although the sitting results were 15.6% lower than previous results found by Tang. ¹⁸ This was due to the smaller participants perching on the edge of chairs, as explained earlier, which affected the inclination of the monitor. It is assumed that this led to the algorithm misclassifying the activity. The step count accuracy compares very favourably when compared to findings by Kuo et al. ¹² using monitors in the CP population, exhibiting far less random error of measurement and also lower systematic variation.

Previous findings in a healthy population have stated that accuracy in relation to velocity can decrease at slower speeds. Kanoun ¹⁵ reported that the activPAL monitor underestimated step count by 3.5% at a velocity of 0.45 m s⁻¹. There was a large spread of velocities (Figure 3) for participants, ranging from 0.36 to 1.12 m s⁻¹. Figure 3 demonstrates that a decline in accuracy occurred as the velocity decreased, with a range of accuracy between 76% and 100%. However, this cannot be stated as a general trend for the participants recruited, as the monitor output for participant 6 exhibited poor accuracy at a velocity of over 0.7 m s⁻¹; however, the velocity for participants 9 and 10 was more accurate at far lower speeds. When the gait for participant 6 was observed, it was noted that a short step length occurred and internal rotation and overlay to the contralateral side were present. She also compensated with circumduction of the affected limb in swing phase with little knee flexion occurring. These attributes may have contributed to the increased inaccuracy. This indicated that inaccuracy of the step count is multifactorial, and the results of this study suggest that the complexity of the individuals’ gait pattern can affect the accuracy to a greater degree than a decrease in velocity.

Figure 3.

Step count accuracy in relation to velocity (P = participant number).

The number of transitions recorded was accurate for eight of the participants. The inaccuracies that occurred with two participants were connected to the misclassification of sitting as standing. As stated previously, this was due to the smaller participants perching on the edge of chairs. It was noted that the participant with a more complex gait pattern (type 3) had an increased number of transitions within the trial due to unplanned standing periods within walking events. These standing periods were short in interval. The monitor detected these shorter intervals in the majority of occasions and as such did not affect the transition accuracy adversely.

It should be noted that the absolute agreement between the raters (0.9911), examined with intraclass correlation (Table 5) was very good (95% confidence interval = 0.9814–0.9958).

Table 5.

The intraclass correlation coefficient for the observers.

	Intraclass correlation a	95% confidence interval
Single measures b	0.9911	0.9814–0.9958
Average measures c	0.9955	0.9906–0.9979

a

The degree of absolute agreement among measurements.

b

Estimates the reliability of single ratings.

c

Estimates the reliability of averages of k ratings.

When overall percentage agreement was measured, it was found to be 9.4% less accurate than that within a healthy population. The overall percentage agreement of a second-by-second analysis between the observer and the monitor was 86.5% (Table 3). Overall agreement for healthy subjects has been reported as 95.9%. ¹⁷ This is an expected result due to the increased complexity of the gait pattern being monitored and does not automatically in itself make the monitor unsuitable for clinical application.

The aspect of measuring short time periods was not specifically built into the methodology. However, the authors were confident that the monitor could recognise short time periods. Three of the less-able participants stood for short time periods of varying self-selected duration. This occurred within the walking sections where the participant paused when distracted. These events of duration 3–9 s were detected by the monitor.

Specific indications for using the activPAL may include monitoring and comparing the effects of specific clinical interventions on specific gait parameters and activity following interventions such as the following:

• Use of botulinum toxin with CP

• AFO interventions

• Surgical intervention

Further studies are now required to investigate the increased variable sensitivity detected within the more complex gait patterns of type 2b and type 3 as defined by Winters et al. ¹⁹ Further testing will also be required to measure the sensitivity of the monitor in a community setting. For future studies carried out in the community and studies considering more complex gait patterns, the addition of a monitor to the sound leg could inform the validation process further.

Study limitations

The size of the population for the study may limit generalisation of the results. It should also be noted that the monitor cannot specifically classify facets of activity such as skipping, jumping or crawling.

Conclusion

The aim of the study was to validate an activity monitor for monitoring children with CP. The monitor was proven capable of measuring the number of steps and the time spent walking for the less complex hemiplegic gait patterns. It should be noted that where internal rotation and minimal overlay to the contralateral side exist, decreases in accuracy may occur. It failed to accurately measure the times spent during sitting activities. Further studies are therefore necessary to eliminate this error, potentially including algorithm alterations.