Abstract
Intervening in the cycle of intergenerational dysfunction with effective interventions is a challenge for social, educational and health agencies. A substantial literature indicates the importance of environmental influences on children's social, behaviour and emotional development. Children experiencing socioeconomic disadvantage, unstable family composition, harsh parenting styles and the presence of criminality are at greater risk of disruptive behavioural disorders (DBD) [1], although genetic factors are increasingly thought to be contributory [1] and may moderate the effects of maltreatment [2].
Intervention in high-risk families has become the focus of intensive research [1], [3], [4], but influencing outcomes has proved more difficult than originally hoped. The strongest treatment evidence for early onset DBDs is family focused and combines child- and parent-oriented treatments [1]. Debate continues, however, over whether child-focused interventions alone are sufficient and whether gains in parental and family functioning may be achieved indirectly through secondary processes [5], [6].
Poorer treatment response is associated with older age [1], greater severity of behavioural problems [1], [4], social adversity [1], [4], [7] and comorbidity, particularly with attention-deficit–hyperactivity disorder (ADHD) [4]. The Multimodal Treatment study of children with ADHD (MTA) included psychosocial interventions, but found that intensive behavioural interventions with both children and parents did not yield significantly greater benefits than medication alone for the core symptoms of ADHD. The results suggested only modest advantages for comorbid disruptive behavioural problems, such as oppositionality and aggression [8]. The Hechtman et al. ADHD studies concluded ‘parent training did not improve self-rated parental behaviour’, irrespective of the presence of oppositional defiant disorder (ODD) [9].
An alternative intervention strategy has focused on at-risk families with very young infants, who are likely to develop DBDs, with initial promising results from home visiting programmes [3]. Difficulties generalizing home visiting interventions nationally, however, have led to caution [3], [10–12]. For example, identifying any gains on parent and family indices, even after 2–3 years’ intervention [11], [13], has been difficult despite small to modest benefits across a range of child measures [12], [14].
Another approach for managing disruptive children from high-risk families has developed in New Zealand (NZ) through health camps. The Children's Health Camp movement began in 1920, aimed at physically rehabilitating children, but its focus later radically changed to provide intervention for children with high and complex needs, early intervention with children exposed to multiple risks, and for those at risk of poor life outcomes, such as Maori, Pacific, rurally isolated and refugee children.
Glenelg Health Camp (GHC), Christchurch, is one of seven regionally based camps providing residential places for primary school children. Previous GHC reviews, for example Tennant [15], had noted that gains from child-focused interventions were lost on return home, prompting the introduction of a new, broader service delivery model involving the child's parents, school and with ‘before and after’ outreach components.
The present study evaluated the new service model's efficacy, using improvement in child behaviour as the primary outcome. Secondary goals were evaluating the contribution of parenting programme participation and the practical utility of the assessment measures. The specific hypotheses were that (i) children would make significant behavioural and emotional gains after intervention; (ii) children of parents participating in the parenting programme would demonstrate greater gains compared to children whose parents did not; (iii) positive attitudinal change would occur in parents participating in the parenting programme; and (iv) positive parental attitudinal change would be associated with enhanced child outcomes.
Method
Health camp intervention
The referred child was assigned a field social worker who made a home visit, obtained consent and organized data completion. The community school was visited to meet the child and their teacher. After a familiarization visit to the health camp the child was placed in a group (usually of seven) and a 4–6 weeks’ residential stay scheduled. The Camp's associated ‘special school’ keeps the same term dates as community schools, enabling children to attend for a full school day (9:00am–3:00pm), 5 days a week. Outside these hours children participated in a range of group adventure and recreational activities where many ‘here and now’ situations developed, either between the children or residential care staff, typically over participating or adhering to limits. Cooperative behaviour was encouraged and rewarded by receiving individual and group points. Resolving conflictual behaviour was fostered through problem solving and learning new ways to cope with conflict. Camp and school staff utilized cognitive behavioural approaches, social skills training [16] and communication skills training. The residential staff (trained social workers, teachers, nurses and para-professionals) also met regularly with their assigned child. Follow up ranged from one home phone call to the maximum contact of five home phone calls, three home visits and two school visits. Discharge reports were sent to parents, school and referrer.
The parenting programme involved 15 h of interactive group sessions, divided into five 3 h modules. Content included these themes/topics: parent and child needs, coping with change, understanding child development and normal age-related expectations, understanding behaviour disorders and learning difficulties, why children misbehave, effective consequencing, communication and listening skills, social skills and anger management strategies.
Study design
The present study was a prospective service evaluation, assessing short- to medium-term emotional and behavioural outcomes in the children, attitudinal change in parents and the relationship between them. The study was approved by the Canterbury Ethics Committee
Subjects
All children aged 5–13 years referred to GHC between July 2001 and August 2004 were eligible (n = 771). Of these, 614 were accepted into the residential programme, but only those with complete pre- and post-measure assessments (n = 177) qualified for analysis. However, 20 parents did not give consent, leaving 157. Of these 100 participated in the parenting programme and 57 did not. Participation in the parenting programme was defined as completing a minimum of three of the five modules.
The representativeness of the 157 participants, both children and parents, was compared with the 614 non-participants. No differences were found for gender or ethnicity (self-defined European, Maori, Pacific Island or Asian descent), but a significant difference (p < 0.05) was found for age (mean = 8.7 years, SD = 1.8 vs mean = 9.3 years, SD = 2.0), indicating that the study cohort was slightly younger. Additional comparisons were made between children and parents based on participation in the parenting programme. The two groups of children were well matched for age, gender and ethnicity, as were their parents (overwhelmingly mothers) for age and ethnicity (Table 1). There were no significant differences in family structure, although both groups had a high proportion of one-parent families (parenting programme participation first) 46 (46%) vs 25 (44%).
Study cohort characteristics
Measures
The extended Strengths and Difficulties Questionnaire (SDQ) [17] was the child behaviour primary outcome measure. Parent and teacher versions contain 25 items, grouped to generate five clinical subscale scores (hyperactivity, emotional symptoms, conduct problems, peer problems and prosocial behaviour). Subscale scores (0–10) are summed to generate a Total Difficulty score (0–40, excluding the prosocial subscale). Impact scores, derived from five parent distress and social incapacity items and three similar teacher items, give maximum scores of 10 and 6, respectively. Goodman suggests that ‘an impact score of 2 or more…corresponds conceptually to a score of 60 or less on the Children's Global Assessment Scale’ [18], which is regarded as a good indicator of caseness [17]. The SDQ predictive algorithm (available from the SDQ website: www.iop.kcl.ac.uk/Iop/Departments/ChildPsy/sdq/a1.stm) variably combines parent and teacher scale scores with impact scores to generate ‘unlikely, possible and probable disorder’ categories (hyperactivity, conduct, emotional and any psychiatric disorder). The algorithm excludes autism and psychotic disorders. The SDQ has primarily been used as a screening measure [19], although follow-up versions are available.
The Adult Adolescent Parenting Inventory–Second version (AAPI-2) [20] has two (before and after) 40-item parent self-report questionnaires, using a 5 point Likert scale (strongly agree to strongly disagree). Items are grouped into five constructs: A, Inappropriate Parental Expectations; B, Parental Lack of Empathic Awareness of Children's Needs; C, Strong Belief in the Use and Value of Corporal Punishment; D, Parent–Child Role Reversal; and E, Oppressing Children's Power and Independence. It has been standardized across a range of American groups, allowing each subscale's raw score to be converted into normally distributed sten scores (1–10). Nearly 16% of the population respectively rank as low scorers (1–3, less nurturing parenting attitudes) or high scorers (8–10, adaptive parenting attitudes). Internal consistency has been demonstrated using Cronbach alpha (range 0.70–0.86) with a test–re-test correlation of 0.76. Lower scores are associated with at-risk and abusive parenting [20]. It has been used as a measure of change in young (often adolescent), disadvantaged and at-risk parents [20]. The AAPI-2 was selected as the primary indicator of change in parenting attitudes.
Statistical analysis
The representativeness of the study sample, compared to all other health camp referrals, was evaluated using χ2 and independent samples t-tests.
The SDQ and AAPI-2 were the primary outcome measures for the children and parents, respectively. Within-group pre–post-intervention SDQ and AAPI-2 score changes were statistically tested using paired t-tests. SDQ categories were dichotomized into normal + borderline versus abnormal and compared between pre- and post-intervention assessments using McNemar's χ2 test. The changes in SDQ and AAPI-2 scores were compared between those who attended the parenting programme and those who did not, using repeated measures ANOVA. p ≤ 0.05 was used to indicate statistical significance.
Effect sizes were calculated by using pre- and post-intervention standard deviations for continuous data or pre- and post-intervention percentages for categorical data. All statistical analyses were undertaken using SPSS version 13.0 (SPSS, Chicago, IL, USA).
Results
The study cohort's mean age was 8.7 years (range 5–12 years) and nearly two-thirds were boys (62%). Ethnicity was predominantly New Zealand European (83%). The proportion of Maori children (12%; Table 1) was comparable with the Canterbury population aged 5–14 years (13%) [21].
Mean family contact was 10.9 months (SD = 4.3). The ‘contracting’ lead-in period was 4.4 months (SD = 2.5); mean residential stay was 31 days (SD = 7.8); post-residential follow up mean duration was 5.4 months (SD = 3.5), while outcome measures were completed a mean of 2.5 months (SD = 1.7) into this period.
SDQ Symptom Ratings
At post-intervention assessment both parent and teacher SDQ mean scores uniformly showed significant reductions in Total and subscale scores, with the exception of the prosocial subscale, which remained unchanged. Symptom scale effect sizes range from 0.17 to 0.45, with most scores indicating a moderate effect (Table 2) [22]. Total difficulties effect sizes for parents and teachers were, respectively, 0.45 and 0.35, followed by emotional problems (0.37 and 0.30), hyperactivity (0.30 and 0.38), then conduct problems (0.32 and 0.20). The least intervention effect was on peer difficulties (0.24 and 0.17). There was no effect on prosocial (positive) behaviour (0.08 and 0.04).
SDQ parent and teacher outcome scores
SDQ, Strengths and Difficulties Questionnaire.
†t-test; ‡χ2 test.
The proportion of children with impact scores ≥2 was halved at post-intervention assessment (parents 72–33.8%, teachers 65.6–28%; Table 2), This significant finding was reflected in a large effect size for parent rating (0.85) and teacher rating (0.79) [22].
Categorical results
A similar pattern of effect sizes was seen when SDQ scores were analysed categorically (0.16–0.42). Only abnormal scores above the cut-offs for each subscale were analysed (Table 3).
Parent and teacher SDQ category changes
SDQ, Strengths and Difficulties Questionnaire.
When the parent and teacher categorical and impact data were combined to generate diagnoses (Table 4) a uniform pattern of decrease in the proportion of ‘probable’ diagnoses was found after intervention. A more variable pattern was seen in the ‘possible’ diagnosis categories, where ‘conduct’ and ‘any diagnosis’ proportions increased. The number of children receiving an ‘unlikely’ diagnosis categorization uniformly increased after intervention. Small cell sizes did not allow sensible statistical comparisons, thus only the effect size for the pre- and post-intervention Total Group ‘any diagnosis’ cells was calculated and, at 0.78, approached a large effect [22].
Diagnosis before and after intervention×parent programme participation and Total Group
SDQ, Strengths and Difficulties Questionnaire.
Sub-analyses (data not shown) clearly indicated that a small number of children became worse over the course of the intervention, with 15 (9.6%) developing a possible ‘any diagnosis’ categorization. Similarly, sub-analyses of the SDQ categories showed that some children's functioning deteriorated, which is reflected in the Total Difficulties category for both parent and teacher ratings. For example, using parent ratings, four of 38 initially ‘normal’ categorized children moved to either borderline (n = 1) or abnormal (n = 3) groupings (respective teacher-rated changes were normal to borderline, n = 5, and to abnormal, n = 1). Similarly, 11 of 21 parentally and three of 26 teacher-rated children moved from borderline into the ‘abnormal’ category, giving adverse change totals by parents of 15 (9.6%) and by teachers of nine (5.7%).
Group results
SDQ changes were compared between the group of children whose parents participated in the parenting programme (100) and those with non-participating parents (57). No significance differences were found between Total Difficulties or any SDQ subscales, irrespective of whether ratings were completed by parents or teachers (Table 5), with one exception: teacher-rated improvement in the conduct subscale was greater in the parenting ‘no’ versus ‘yes’ group (p = 0.040). t-tests, however, indicated that the parenting ‘yes’ children initially had significantly fewer teacher-rated conduct problems (p < 0.05), and significantly less parent-rated hyperactivity (p < 0.05). There were no other between-group differences (Table 5).
Mean SDQ parent and teacher scores by parenting programme
SDQ, Strengths and Difficulties Questionnaire.
†Comparison between parenting programme groups of before–after changes.
∗Group difference: Yes vs No: significant at p < 0.05.
The lack of parenting programme effect was examined further by post-hoc evaluation comparing GHC programme delivery to the parenting ‘no’ versus ‘yes’ groups. A range of significant differences emerged. Mean camp duration was similar (‘no’ 30 days vs ‘yes’ 31 days), but mean duration of service contact was longer for the ‘no’ group (11.8 months, SD = 4.2 vs 10.3 months, SD = 4.3, p = 0.035), as were the mean times from referral to first measurements (5.6 months, SD = 2.3 vs 4.5 months, SD = 1.6, p = 0.006), the interval between measures (5.4 months, SD = 2.4 vs 4.7, SD = 1.7, p = 0.01) and the interval from camp end to obtaining second measures (2.9 months, SD = 2.0 vs 2.2 months, SD = 1.4, p = 0.007). There were no interval differences between first measurement and camp commencement (1.4 months, SD = 1.1 for both groups, p = 0.454) or duration of follow up (5.4 months, SD = 3.4 vs 5.3 months, SD = 3.5, p = 0.505). Overall, these results suggest that more variability occurred in the parenting ‘no’ group and that this was particularly related to the longer duration of the engagement period, which took, on average, 1.1 months longer.
Parental attitude results
There were several significant attitudinal changes within the whole parent group (Table 6). The respective pre- and post-intervention changes were: Inappropriate Parental Expectations (construct A), 5.75–5.10 (p < 0.001), Strong Belief in Corporal Punishment (construct C) 4.65–4.92 (p = 0.048) and Parent–Child Role Reversal (construct D) 4.75–5.25 (p = 0.002). Change in Oppressing Child's Independence (construct E) approached significance (4.91–4.45, p = 0.062). The changes in constructs C and D represent favourable changes in attitudes, while those in construct A indicate unfavourable change. Between-group comparisons showed no significant contribution of the parenting programme to any of these changes, with one exception. Parent programme participation positively influenced Parent–Child Role Reversal (Construct D, p = 0.006, Table 6). Overall, these results suggest that attitudinal change was variable and that participating in the parenting programme made a minimal contribution to changing parental attitudes.
AAPI-2 means
AAPI-2, Adult Adolescent Parenting Inventory–Second version.
†Comparison total group before vs after; ‡between parenting programme groups before vs after.
∗p < 0.05; ∗∗p < 0.001.
Discussion
This study provides moderate support for the effectiveness of the GHC intervention on a range of common childhood emotional and behavioural problems. The greatest impact was expected for conduct problems, but comparable changes were found across all SDQ domains and these occurred whether rated by parents or teachers and scored continuously or categorically. The effect size for improvement in the ‘any diagnosis’ category was relatively large (0.78; Table 4).
The children referred to GHC had clinically significant problems and appear similar to international outpatient samples. Recent research in Germany and Australia also using the SDQ as a measure of severity generated SDQ scores and diagnostic percentages comparable to the GHC children [23], [24].
Although normative data are not yet available for New Zealand children, a recent Australian study supported both the SDQ subscale reliability and the original five-factor structure validated in UK children [25]. Small revisions of Australian teacher SDQ cut-off norms were suggested [26] but, even with these revisions, our recalculations showed no changes in the pattern of results or significances (data not shown).
Few previous studies have used the SDQ as an outcome measure. One small study (n = 27), using the SDQ to measure the effect of a mental health outreach service on homeless children and families, found that the Total SDQ score improved significantly [27]. A longer term follow-up study (n = 40, mean age = 9.5 years) using the SDQ as a repeat measure of change, found all of the highly significant post-intervention SDQ gains had been lost at 3 years [28]. Another larger study (n = 141) used changes in SDQ scores to measure the effect of videotaped Parent Management Training (PMT) on parent-rated child behaviour (children aged 3–8 years) and demonstrated moderate effect size reductions in Total and conduct SDQ scores [29]. Taken together, these studies suggest that the SDQ shows sensitivity to change. The present study appears to be the largest outcome study to date and suggests sufficient sensitivity to change to support using the SDQ as an outcome measure with primary school children. Additionally, finding that a small number of SDQ scores worsened (9.6%) over the course of the intervention argues against the changes simply reflecting a general tendency for psychometric measure scores to improve with repetition.
Although nearly half (48%) of the sample received a ‘possible’ or ‘probable’ hyperactivity diagnosis (Table 4), the pattern of improvement was notably similar across the emotional, conduct and hyperactivity SDQ subscales (Tables 2 and 3). The improvement in hyperactivity symptoms was unexpected and led to extensive retrospective review of the files to ascertain whether additional pharmacological intervention accounted for the improvement. A total of 22 of the 29 children previously diagnosed with ADHD were found to be on stimulant medications. No child started on stimulant medication during the study period, one child stopped medication and another's dexamphetamine dosage was increased. Sub-analyses were independently conducted comparing to the remainder of the sample, first, those children on medication and, second, those diagnosed with ADHD, with respect to the SDQ data generated ‘hyperactivity disorder’. No contributing effect of medication on outcome was found (data not shown).
Contrary to expectations, parenting programme participation did not appear to augment gains in the children's behaviour (Table 5). This was not explained by demographic differences, because the children and parents were closely matched (Table 1). Nor did a few significant differences in problem severity at intake seem a likely explanation, because participating parents rated their children less hyperactive and the teachers rated them as having fewer conduct problems (both p < 0.05). Indeed, participating parents uniformly, but non-significantly, tended to rate their children as having less severe problems (Table 5). Taken together these differences would be expected to lead to more positive results for the children of participating parents, because children with less severe problems are thought to be more treatment responsive [4]. Last, parents’ willingness to commit to the parenting programme might indicate less negative feeling about their children simply because they had fewer problems. All of these potential biases would be expected to lead to more favourable results for the children of parenting programme participants. This was not found.
Surprisingly, neither did parenting programme participation lead to change in parenting attitudes, with the exception of Parent–Child Role Reversal (p = 0.006; Table 6). In contrast, for the whole sample, construct A (which reflects inappropriate parental expectations) worsened significantly (p < 0.001), despite the children's improvements. Thus, hypotheses 2, 3 and 4 were not supported: children of participating parents did not make greater gains than those whose parents did not (Table 5); parenting programme participation did not promote more favourable parental attitudes (Table 6) and, consequently, there could be no correlation between the improvement in the children's difficulties and parent participation (hypothesis 4).
Several explanations for the lack of parenting effect were considered. It is possible that parents lacked opportunities to apply the advice given, but post-intervention assessments were completed a mean of 2.9 months following the camp stay, allowing ample time for advice to be implemented. Non-participating parents might have absorbed key messages regarding parenting just through the general contact occurring throughout the GHC experience. The present data did not permit evaluation of this possibility. Last, it is well recognized that attitudinal change may not translate into behavioural change, but the present findings (Table 5) suggest that actual parenting behaviour did not change either or, at least, not enough to significantly influence their child's behaviour.
The lack of additional benefit from combined child and parent interventions contrasts with the Webster-Stratton and Hammond findings, where combined child and parent intervention was superior to either alone at 1 year follow up [30]. The major multi-modal ADHD study designs [8], [9] did not permit separating out the relative contributions of child versus parent focused interventions.
The study findings of change in the children, but not the parents, is consistent with at-risk studies [13], [31] including, most recently, the Healthy Families Alaska Study [11]. These were home visiting programmes, however, which was only a small component of the GHC intervention. It could be argued that the intensity of the GHC Parenting Programme was insufficient, but the Alaskan intervention found ‘no impact on outcomes for families receiving a ‘high dose’ of home visiting’. Nor does age of the child seem important [12], because the MacMillan et al. study involving older children (mean age 5.1 years) similarly found no parent gains [31].
An interesting question remains over how to reconcile the apparent discrepancy between the typically encouraging PMT results [4], [5], [7], [29], [30] and the generally less encouraging effects on parents from home visiting programmes [10], [12], [31]. Caldera et al. identified a potential explanation lying in PMT interventions specifically fostering parental observation, while promoting adaptive parenting practices through ‘practice, feedback and shaping’ [4], [5], [12]. In contrast, the Healthy Families America ‘model and implementation system is relatively weak in addressing the ineffective or negative behaviours observed by home visitors’ [12]. Another important difference is the breadth of approach usually adopted in home visiting programmes compared to PMT, which may be the sole intervention [29]. Nevertheless, explaining the lack of PMT effect found in the present study remains difficult. The simplest explanation lies in the relative ‘dose’ of the child intervention being far greater than the parenting one, but addressing this question would require further research.
A number of limitations qualify the findings. As a service evaluation, randomization was precluded. No confirmation of previous clinical diagnoses was made because this was not required for referral purposes. While raters were not blinded, the teachers rated all children at GHC regardless of their study inclusion status, as part of the health camp protocol. The variation in dose of the GHC programme delivered reflected the flexibility needed when intervening with children and families in a real-world setting and was unable to be controlled for. Additionally, including observational ratings would have increased confidence in the findings, but even introducing the two study outcome measures became a significant service undertaking. Last, despite our best efforts to track medication changes, we cannot exclude unrecorded alterations occurring during follow up, which may have influenced outcomes.
Conclusion
The GHC intervention appeared moderately beneficial for a range of common emotional and behavioural problems seen in primary school-age children over the short to medium term. Contrary to expectations, parent participation in PMT conferred no additional advantage for their children and had minimal impact on parenting attitudes. The results suggest that the SDQ has potential utility as an outcome measure for this age group, although this utility is limited to commonly occurring, broadly defined problems.
Footnotes
Acknowledgements
The authors thank and acknowledge John Horwood for his valuable assistance and advice in preparing this paper. The study would not have been possible without the contribution of participating parents, staff of Glenelg Health Camp and the teachers of Glenelg School – their efforts are warmly and gratefully appreciated.