Abstract
In recent years there has been growing recognition of the need to provide effective treatments for children with early onset conduct disorders and problems [1–3]. These concerns have been underwritten by longitudinal evidence that has shown that early onset conduct problems are associated with a wide range of adverse outcomes including crime, mental health problems, substance use, poor physical health and related problems [4–12]. There is no other commonly occurring childhood condition that is associated with this wide range of adverse outcomes. For these reasons there has been advocacy for increased social investments in the prevention and management of childhood conduct problems [3, 13]. This advocacy has been assisted by a growing literature on the effectiveness of parent management training based around the original social learning theory developed by the Oregon Social Learning Centre [14–16]. There are now several validated programmes on the market for implementing this approach. These programmes include the Incredible Years Parent Training programmes [17, 18], Triple P [19, 20] and programmes developed by the Oregon Social Learning Centre [14–16].
While there is now compelling evidence for the efficacy of parent management training programmes in reducing rates of childhood conduct problems, installing these programmes into a new social context such as New Zealand raises a number of issues [13]. In particular, before such programmes can be accepted as part of established practice in a new context there is a requirement to show (i) that these programmes can be delivered effectively within that context; (ii) that programme efficacy in the new context is established; and (iii) that the cultural appropriateness of the programme is assessed.
Against this background, the present research note summarizes the findings of a preliminary examination of the effectiveness and cultural acceptability of the Incredible Years Basic Parent Programme (IYBPP) [18, 21] delivered in New Zealand. The aims of the analysis were threefold: (i) to examine the efficacy of IYBPP using pre-test/post-test comparisons; (ii) to examine parent satisfaction with the programme; and (iii) to examine the extent to which similar outcomes were evident for Maori and non-Maori parents. The evaluation is based on agency records rather than on a systematic research design, and therefore should be seen as preliminary to a more rigorous evaluation of the IYBPP.
Method
Participants
Participants in the study were 214 parents attending 29 IYBPP courses organized by Ministry of Education, Special Education throughout New Zealand. These parents had been referred to the programme by a variety of sources including Special Education Case Managers, preschool health nurses, Resource Teachers Learning and Behaviour (RTLB), Early Childhood centres, and primary schools. The children involved ranged from 2½ to 8 years of age.
Treatment
All parents were provided with a minimum of 12 and up to 14 sessions of IYBPP in a group setting. There were 29 groups that ranged in size from six to 15 participants. Post-test assessments were completed only by participants who attended at least nine sessions. The mean number of sessions attended by those who completed post-test assessments was 10.23. Each group was delivered by two course facilitators who had completed IYBPP training. In all cases IYBPP was delivered according to the specifications in the IYBPP manual. Course facilitators were relatively inexperienced in the delivery of the programme, with most having delivered fewer than three courses.
Measurements
Data were gathered by course facilitators from parents using self-completed questionnaires. The following measures were collected: (i) number of sessions attended; (ii) self-reported ethnicity (Maori/non-Maori); (iii) pre-test and post-test scores on the Eyberg Child Behaviour Inventory (ECBI) Problem and Intensity Scales [22]; and (iv) pre-test and post-test scores on the Social Competence Scale [23]. In addition to this, at the conclusion of the group sessions, parents were asked to complete a 24-question Parent Satisfaction Questionnaire (PSQ) that assessed overall views of the programme (10 questions), usefulness of teaching methods (five questions) and usefulness of parental techniques (nine questions). Between 212 and 214 parents provided pre- and post-test data, with 177 parents providing PSQs.
Results
Pre-test–post-test comparisons
Table 1 compares pre- and post-test means on the ECBI Intensity Scale, Problem Scale and Social Competence scale for Maori parents (n = 41), non-Maori parents (n = 138–139) and parents whose ethnicity was not recorded (n = 33–34). For all groups, the size of the effect for the pre-test–post-test comparison was assessed using Cohen's d [24]. The following conclusions may be drawn from Table 1.
For all three comparisons there were significant improvements in test scores at post-treatment assessment (p < 0.001). The overall values of Cohen's d ranged from 0.56 to 0.70, suggesting that effect sizes were in the range of moderate to large [24]. In all comparisons, results for Maori, non-Maori and not-known groups were similar. A test of interaction using nested regression models showed that the effect sizes for the three groups were not significantly different.
Pre–post-test differences by ethnicity
ECBI, Eyberg Child Behaviour Inventory.
Parent satisfaction
Table 2 shows the association between ethnicity and responses to the PSQ. Table 2 lists (i) the percentages of negative, neutral and positive comments made about the overall programme; the usefulness of the teaching format and the usefulness of parenting techniques; and (ii) a global rating of feeling about the programme. The following conclusions may be drawn.
In general, assessments of the programme were positive, with >80% of ratings being positive. Between 11.4% and 17.5% of responses were neutral and <2% of comments were negative. Overall, nearly 90% of the participants were positive about the programme and 11.4% were neutral. Reports of programme satisfaction were similar for Maori, non-Maori and those of unknown ethnicity. Comparisons between groups using one-way analysis of variance or χ2 showed no significance between group differences.
Parental ratings of overall program satisfaction and program utility by ethnicity
Discussion
These preliminary findings of the IYBPP applied to a group of New Zealand parents are clearly promising. On the basis of pre-test–post-test comparisons, the programme was associated with effect sizes in the moderate–large range. These effect sizes seem to be similar to those reported in other evaluations of IYBPP using pre-test–post-test comparisons in the context of a randomized trial [18, 21]. Levels of parental satisfaction with the programme were high and few parents commented on the programme negatively. Importantly, the programme outcomes did not vary with ethnicity and Maori and non-Maori parents expressed similar levels of satisfaction. These findings suggest that IYBPP in the form it was delivered in the present study was culturally appropriate and equally effective for Maori and non-Maori people.
Although the findings are clearly promising they are also subject to a number of important limitations. First, the results are based on agency records collected on those who completed IYBPP courses. Data were not available on those who declined to attend or on those who dropped out of the programme. Lack of information on these groups may mean that the results of the present evaluation give an overly optimistic view of the efficacy and acceptability of the programme.
Second, the estimates of effect size using a pre-test/post-test design is likely to be subject to retest effects and give overly optimistic estimates of programme effects. For these reasons it is important that the present findings are viewed as being preliminary to a more comprehensive evaluation of the efficacy and acceptability of IYBPP in a New Zealand context. The most effective way of assessing efficacy and acceptability would be through a wait-list randomized design similar to that used in the evaluation of IYBPP in Wales [25, 26] and Norway [27]. Prior to such a trial it would also be useful to conduct pilot research to examine the fidelity of programme delivery.
Footnotes
Acknowledgements
We would like to thank the New Zealand Ministry of Education for giving us permission to publish these data.