Abstract
Keywords
The negative long-term outcomes for toddlers and preschool-aged children displaying early behavioural problems are known to be far reaching. Left untreated, early onset conduct problems typically persist, placing the child at risk of developing treatment-resistant disruptive behaviour disorders, social and emotional difficulties in middle childhood and adolescence, and psychopathology in adulthood [1], [2]. Hence, early intervention for toddlers displaying early behavioural problems is of outmost importance.
Parent–child interaction therapy (PCIT) [3] is a short-term, evidence-based parent training intervention used widely in the treatment of behaviourally disordered preschool-aged children. Outcome studies have shown PCIT to be associated with lasting improvements in child and sibling behaviours at home and school [4–10], as well as improvements in the interactional styles, stress levels, confidence, and psychological functioning of parents [4], [11], [12].
One of the most pressing issues currently facing evidence-based clinical practice is whether programmes developed in research clinics can be successfully transported to community settings, where treatments are often adapted to meet the individual needs of patients and where the characteristics of patients are typically more varied [13]. To date, all PCIT outcome studies have been conducted in university research clinics, which has meant that understanding of the effectiveness of PCIT applied in the real world is limited [14].
The aim of the present study was to evaluate the effectiveness of PCIT delivered to families in an Australian community-based early childhood clinic. The Karitane Toddler Clinic, located in South Western Sydney, Australia, was established in January 2005 and forms part of the wider suite of Karitane services for families with children aged 0–5 years. In the present study it was hypothesized that at the end of treatment at the Clinic: (i) children would demonstrate significant reductions in dysfunctional behaviours; (ii) mothers would show significant improvements on measures of stress and mood; and (iii) mothers would evaluate the programme positively.
Method
Parent–child interaction therapy
PCIT is a treatment approach designed for families with young children aged 2–7 years experiencing a broad range of behavioural, emotional and developmental problems. PCIT integrates elements of attachment theory, traditional child psychotherapy and behavioural techniques. It emphasizes (i) the importance of a positive parent–child relationship in the healthy behavioural and emotional development of a child; (ii) the role of play as the medium through which children develop the problem-solving skills needed for successful resolution of developmental problems; and (iii) the effectiveness of direct parental coaching of parent–child interactions as a way of facilitating the parent's ability to teach pro-social skills to their child, thereby increasing positive behaviours and decreasing inappropriate and maladaptive behaviours [3].
The PCIT programme consists of two phases, child-directed interaction (CDI) and parent-directed interaction (PDI). Both phases begin with a didactic parental teaching session followed by weekly sessions conducted around a dyadic play situation between parent and child, with the therapist as a coach. Based on attachment theory [7], the CDI aims to bring about behaviour change by enhancing the quality of the parent–child relationship. In this phase the child leads the play while the parent describes, reflects and praises appropriate child behaviours, and at the same time ignores inappropriate behaviours, does not criticize, ask questions or interfere. Once an appropriate CDI skill level has been attained, the PDI, which is based largely on social learning theory [7], is commenced. In the PDI the parent continues the same behaviours and also gives clear, direct and developmentally appropriate verbal commands to the child while consistently applying specific consequences for the child's compliance (e.g. labelled praise) and non-compliance (e.g. time-out).
Participants
Eighty-four families were referred to the Karitane Toddler Clinic between April 2005 and October 2006. Referrals came from the Karitane Residential Unit (a 5 day residential programme for families with children aged 0–4 years experiencing early parenting difficulties; 49%), community-based early childhood nurses (19%), psychologists (10%), paediatricians (6%), community health services (4%), the Karitane Volunteer Home Visiting service (3%), general practitioners (2%), the Karitane Careline (24 h telephone parenting counselling service; 1%), play groups (1%), speech pathologists (1%), and other services (4%). Most children were described by referral agents as displaying multiple presenting problems, including demanding behaviours (76.3%), aggression (58.0%), non-compliance (65.0%), sibling rivalry (36.3%) and social withdrawal (7.5%).
Of the 84 referred families, 32 did not begin treatment (23 did not take up the referral and nine attended the initial assessment session but did not begin treatment). Of these 32 families, some identified practical issues such as difficulties obtaining childcare (9.4%, n = 3) or transport (6.3%, n = 2) as the reason for deciding not to begin treatment. Five parents (15.6%) felt that things had improved since the referral, and 14 (43.8%) felt that the programme did not suit them or their child's needs. Seven families (21.9%) were referred to alternative clinical services in the community because they were thought to be inappropriate for treatment at the Clinic (e.g. because of severe maternal psychopathology); one family was not able to be contacted.
Fifty-two families attended the Clinic for at least one treatment session in addition to the initial assessment session. The mean number of treatment sessions (not including the initial assessment session) attended by these 52 families was 11.2 (SD = 3.9, range = 2–19). Families who discontinued after attending at least one treatment session were classed as having dropped out of the programme. Of the 52 families who commenced treatment, six (11.5%) dropped out before treatment was completed. Treatment completers attended a mean of 12.2 sessions (SD = 3.0, range = 5–19) and treatment dropouts completed a mean of 3.7 sessions (SD = 1.4, range = 2–6). Of the 52 families who received treatment, 14 (15.4%) participated in the CDI phase only and 38 (73.1%) participated in both the CDI and the PDI. In 21.2% of cases (n = 11), siblings were incorporated into the treatment programme. In addition to the treatment sessions reported here, two families received a booster session in the 2 months following the final session.
Of the 46 families who completed treatment, two families concluded treatment abruptly and so did not complete post-treatment evaluation measures. One family was excluded from the final sample because pre- and post-treatment measures were completed by different parents. This left a final study sample (i.e. families who completed the PCIT programme and post-treatment measures) of 43 families. The mean age of the 43 children was 33.8 months (SD = 9.0, range = 19–52 months). More than two-thirds (67.4%) were boys and 32.6% were girls. In all cases it was the mother who participated in the programme. Most (82%) were in a married or de facto relationship, 14% were divorced or separated and 6% were single. The mean maternal age was 32.6 years (SD = 4.7, range 19–41 years). Thirty-one percent were high school educated, 41% were university educated, and 28% had a TAFE qualification/ trade. More than one-third (34%) endorsed coming from a cultural background other than Australian.
Procedure
All families who attended the Karitane Toddler Clinic between April 2005 and October 2006 received PCIT treatment delivered by one of the Clinic's three nurses. The treatment protocol followed prescriptions given by Hembree-Kigin and McNeil [15]. In line with recommendations from these authors [15], treatment was tailored to meet the individual needs and abilities of families. This meant that treatment duration was performance based rather than being strictly time limited, with families being discharged from the programme when the clinician felt that the parent had successfully mastered CDI skills and when there was a clear improvement in the child's behaviour. Some families were discharged from treatment after completing the CDI due to a decision (made by the clinician) that progression to the PDI was unnecessary. Home visits, sessions in which siblings and extended family members were included, booster sessions, follow-up phone calls, and referrals to appropriate additional services (e.g. referrals for an allied health assessment where maternal depression or anxiety were evident) were provided where indicated.
Pre-treatment measures were obtained for all of the families who attended the Clinic. Prior to the first treatment session, parents completed the Child Behaviour Checklist (CBCL) [16], the Eyberg Child Behaviour Inventory (ECBI) [17], the Parenting Stress Index–Short Form (PSI-SF) [18] and the Depression, Anxiety and Stress Scales (DASS) [19]. The same measures were completed at a post-treatment assessment session (within 1 week of the final treatment session), at which time the mother also completed the Therapy Attitude Inventory (TAI) [20]. Ethics approval was obtained from the Sydney South West (Western Zone) Human Research Ethics Committee.
Measures
Child Behaviour Check List for ages 1½–5
The CBCL is a 99-item parent/teacher report scale measuring behavioural, emotional and social functioning in children aged 1½–5 years [16]. The scale consists of 99 child behavioural problems, which respondents are asked to rate as 0, not true of the child; 1, somewhat or sometimes true; or 2, very true or often true. Individual item scores are summed to yield a Total Problems score, Internalizing and Externalizing subscale scores, and seven Syndrome subscale scores (Emotionally reactive, Anxious/Depressed, Somatic complaints, Withdrawn, Sleep problems, Attention problems, and Aggressive Behaviour). The CBCL has been shown to be a valid instrument [16]. The CBCL Total Problems scale has been shown to have a test–retest reliability coefficient of 0.90. Across all syndrome scales a mean correlation coefficient of 0.85 has been reported [16].
Eyberg Child Behaviour Inventory
The ECBI is a 36-item parent report measure of conduct problem behaviours in children aged 2–16 years of age [17]. The ECBI consists of an Intensity score that measures the frequency of disruptive child behaviours, and a Problem score, which measures the number of behaviours the parent views as problematic. Scores >132 on the Intensity scale and ≥15 on the Problem scale are considered to be clinically significant [17]. The ECBI has been shown to be a reliable instrument, with internal consistency coefficients of 0.95 for the Intensity scale and 0.93 for the Problem scale [21]. Test–retest coefficients across 12 weeks (rs = 0.85 and 0.80) and 10 months (r = 0.75) for the intensity and problem scales, respectively, have been reported [22]. The ECBI has also been shown to possess satisfactory construct validity [23] and to be sensitive to change during treatment [11].
Parenting Stress Index–Short Form
The PSI-SF is a 36-item self-report measure of stress in the parent–child system [18]. The PSI-SF is a direct derivative of the full-length Parenting Stress Index and consists of three subscales: Parental Distress, Parent–Child Dysfunctional Interaction, and Difficult Child. Higher subscale and Total Stress scores indicate greater dysfunction. Total Stress scores >90 are considered indicative of clinically significant levels of parenting stress. The PSI-SF Total Stress scale has been shown to have a test–retest reliability coefficient (over a 6 month interval) of 0.84 and an internal consistency coefficient alpha of 0.91 [18].
Depression, Anxiety and Stress Scales
The DASS is a 42-item self report measure of psychological symptoms [19]. The DASS consists of three subscales: Depression, Anxiety, and Stress, with higher scores indicating greater psychological distress. The Depression scale includes items that measure symptoms typically associated with dysphoric mood (e.g. sadness or worthlessness); the Anxiety scale includes items that measure symptoms of physical arousal, panic attacks and fear; and the Stress scale includes items measuring symptoms such as tension, irritability and a tendency to overreact to stressful events [19]. The factor structure and validity of the DASS have been demonstrated and severity labels have been reported [19], [24], [25]. Cronbach alphas of 0.96, 0.89 and 0.93 have been reported for the Depression, Anxiety and Stress scales, respectively. Reported test–retest correlation coefficients for the subscales have ranged from 0.71 to 0.81 [26].
Therapy Attitude Inventory
The TAI is a self-report scale designed to measure parental satisfaction with the process and outcome of parent training and parent–child treatments [20], [27]. The TAI consists of 10 questions relating to issues such as confidence in discipline skills, quality of the parent–child interaction, the child's behaviour, and overall family adjustment. Each item is answered on a scale of 1–5, where 1 indicates dissatisfaction with treatment or a worsening of problems and 5 indicates maximum satisfaction with treatment or improvement of problems. The 10 item scores are summed to yield a total score between 10 and 50. The internal consistency (Cronbach's α = 0.88) and discriminative validity of the TAI have been demonstrated [4].
Statistical analysis
Data were analysed using SPSS for Windows (version 15; SPSS, Chicago, IL, USA). Repeated measures t-tests were used to compare scores on measures obtained before and after treatment. McNemar's χ2 tests and binomial distribution comparisons were used to compare changes in the percentages of mothers reporting clinically significant levels of distress. Bonferroni corrections were used to account for multiple comparisons, and statistical significance was set at p < 0.05. Clinical significance was evaluated using a reliable change index (RCI) [28], published ECBI clinical cut-off scores [17] and improvement categories similar to those described by Matthey [29].
Results
The clinical severity of this group is evident in the fact that they had a mean pretreatment ECBI Intensity scale score of 145.8 (SD = 29.9; i.e. 1.4 SD above the normative mean) and a mean pretreatment CBCL Externalizing scale score of 27.6 (SD = 7.7; i.e. 1.9 SD above the normative mean). Mean scores on all measures at pre- and post-treatment assessment are shown in Table 1.
Paired t-tests showed significant differences between pre- and post-treatment scores on the ECBI Intensity scale (t(36) = 6.3, p = 0.000), ECBI Problem scale (t(33) = 5.1, p = 0.000), CBCL Externalizing scale (t(38) = 7.4, p = 0.000), and the CBCL Internalizing scale (t(37) = 4.9, p = 0.000). Statistically significant improvements were also seen on maternal self-report measures: PSI-SF, t(39) = 3.1, p = 0.003; DASS Depression, t(35) = 2.7, p = 0.012; DASS Anxiety, t(35) = 3.3, p = 0.002; and DASS Stress, t(35) = 3.7, p = 0.001. There were medium effect size differences between pre- and post-treatment scores on the CBCL and ECBI scales (CBCL Externalizing, d = 0.6; CBCL Internalizing, d = 0.4; ECBI Intensity, d = 0.6; ECBI Problem, d = 0.4). Effect size differences between pre- and post-treatment scores on the PSI-SF (d = 0.2) and DASS (Depression, d = 0.2; Anxiety, d = 0.3; Stress, d = 0.3) indicated smaller improvements on these measures.
Table 1 shows the percentages of families scoring within the clinical ranges on all measures before and after treatment. Statistically significant changes in proportions scoring in the clinical ranges at post-treatment assessment were found on the PSI-SF Total stress scale (p = 0.001), CBCL Internalizing (p = 0.021), CBCL Externalizing (p = 0.000), the ECBI Intensity scale (p = 0.001), and ECBI Problem scale (p = 0.003). Comparisons of the proportions of mothers scoring within the clinical ranges on the DASS (severe or extremely severe) [19] were non-significant (Depression, p = 0.625; Anxiety, p = 0.063; and DASS Stress, p = 0.250).
Parent report measures (n = 43)
CBCL, Child Behaviour Checklist; DASS, Depression, Anxiety and Stress Scales; EBCI, Eyberg Child Behaviour Inventory; PSI-SF, Parenting Stress Index–Short Form.
∗p < 0.05, binomial distribution.
RCI calculations [28], [30] were used to evaluate the clinical significance of differences between pre- and post-treatment ECBI scores. Using this method, the pre–post-treatment score differences needed for 95% confidence that a real change had occurred were calculated, resulting in scores of 3.33 points for the ECBI Intensity scale and 0.83 points for the ECBI Problem scale. Using pre–post-treatment score differences, established norms for clinical caseness [17] and a classificatory approach similar to that described by previous researchers [30], individuals were classified into one of five ‘improvement’ categories. Results are shown in Table 2.
Improvement classifications
EBCI, Eyberg Child Behaviour Inventory.
†Sample size variation due to missing data.
The mean total TAI score was 44.05 (SD = 5.0, range = 33–50). This score indicates a high level of parental satisfaction with the programme.
Discussion
The present study evaluated the effectiveness of a PCIT programme tailored for delivery in a community clinic setting in Australia. This research was motivated by an identified need for translational research to examine the effectiveness of PCIT in real-world clinics [14], to extend evidence about the effectiveness of PCIT conducted in university clinic settings [4–8], [11], [12], [31]. The present results confirmed previous findings about the effectiveness of PCIT, identifying clinically and statistically significant improvements in child behaviours and maternal well-being, and high levels of patient satisfaction with treatment.
Results showed that children who received this intervention displayed significant behavioural improvements, evident in the statistically significant differences between pre- and post-treatment CBCL and ECBI mean scores and in the fact that so many were classified as ‘recovered’ or ‘improved’ (84% ECBI Intensity, 83% ECBI Problem). Nevertheless, there were a significant proportion of children (16% ECBI Intensity, 17% ECBI Problem) who did not show significant improvement or whose behaviour had deteriorated at the end of the programme. In light of the fact that early child behavioural problems are known to develop and be maintained by an interaction between child characteristics (e.g. temperament, personality factors), parental factors (e.g. maternal psychopathology, drug abuse, relationship difficulties) and environmental factors (e.g. stressful life events, family disruption) [1], [14], [32], [33], treatment response was likely to be influenced by a combination of these factors. Indeed, recent research has identified multiple child psychological/behavioural problems [34], personality variables of the child (e.g. rational–unemotional personality traits) [35], [36] and parental variables such as maternal psychological functioning, as predictors of treatment response [37]. In the current study, high rates of parenting stress (70%) and moderately high rates of maternal depression (9%), anxiety (19%), and stress (16%) were identified, thus highlighting the importance of providing interventions that address maternal issues as well as child behaviour problems. Allied health assessment and treatments (including individual or couple counselling for parents) have recently been introduced at the Karitane Toddler Clinic as an adjunct to parent–child interventions. As maintained by Herschell et al., there is still a need for identification of specific predictors of PCIT treatment response [14]. Further research should systematically examine the role of maternal psychological dysfunction and other social adversity factors such as unemployment and substance abuse in the prediction of treatment success in this clinical population.
Attrition from early childhood psychotherapy intervention programmes is an issue of concern for many services and clinicians, due to the frequency with which it occurs (47% in general child psychotherapy programmes [38]; 33% in PCIT programmes [37]) and the demonstrated poorer outcomes for families who not complete treatment [39]. Socioeconomic disadvantage and family adversity have been identified as some of the strongest predictors of attrition from programmes for children with disruptive behaviours, but child characteristics have also been shown to play an important predictive role [40–45]. In the current study, families were considered to have dropped out of the programme when they discontinued after attending one treatment session (in addition to the initial assessment). Compared to previous studies [37], [38], the rate of attrition in the present study (11.5%) was low. This may have been related to the age group of the children, because families with younger children (3–7 years of age) have been shown to be more likely to complete treatment compared to families with older children (aged 7–13 years) [46]. But in light of previous research identifying the importance of parental perceptions about barriers to treatment and the quality of the therapeutic relationship in predicting treatment dropout [45], the low dropout rate in the current sample is also likely to be attributable to the community-based clinic setting in which the treatment was delivered, the flexible approach that was taken in the delivery of the intervention and the strong therapeutic relationship established between clinicians and patients.
In addition to the 11.5% who dropped out after having commenced treatment, a sizable proportion (38%) of the families who were referred to the clinic either declined the referral or did not attend for treatment after the initial assessment session (‘non-starters’). Kendall and Sugarman found that the only significant difference between treatment dropouts and non-starters in a sample of families referred for treatment of childhood anxiety disorders was that non-starters were younger in age [47]. Therefore, factors known to be associated with attrition from child behaviour treatment programmes such as socioeconomic status, life stressors, child behavioural problem severity and parental attitudes towards treatment [40–45] are also likely to be relevant for the non-starters in the present study. Without detailed psychosocial information about these families, however, it is difficult to comment meaningfully about the reasons that they did not commence treatment. As found in previous studies [39], practical constraints and perceived barriers to treatment were of relevance for the non-starters in the present study, with 15.7% of the non-starters identifying practical issues such as transport or childcare as the reason for not commencing treatment. In addition, more than half (59.4%) of the families who did not commence treatment said that their child's behaviour problems had improved since the referral or that they did not feel that the programme suited their needs. Further investigation of these issues could lead to programme improvements such as better education for referral agents about referring appropriate families, or better systems to encourage and enable the most vulnerable families to engage in the service.
It is important to mention the limitations of the present study. The absence of a control group meant that improvements due to maturational or other factors could not be ruled out. But, considering the severity of the behavioural problems displayed by the children in this sample (1.4 SD above the normative ECBI Intensity scale mean and 1.9 SD above the normative CBCL Externalizing scale mean) and the demonstrated stability of child behaviours during the preschool years [4], [7], it seems unlikely that improvements would have come about spontaneously. Also, measures were administered only at the start and end of treatment; data about the longer term effectiveness of the programme would have been useful in evaluating the success of the intervention. Additional shortcomings of the current study included its reliance on parental reports of child behaviours and child behavioural outcomes. The study could have been improved with the inclusion of a diagnostic interview for child behaviour/conduct problems, and with observational ratings of the quality of parent–child interactions (e.g. Dyadic Parent–Child Interaction Coding System [48]) and child attachment styles (e.g. Ainsworth et al. Strange Situations procedure [49]). Additional improvements could have included independent ratings of child behaviours (e.g. from pre-school teachers and fathers); a more detailed assessment of parental psychopathology (including a structured clinical interview for psychiatric diagnoses); assessment of parental history of parenting and discipline practices; evaluation of the impact of the intervention on other family members such as fathers, grandparents and siblings; and independent ratings of PCIT treatment integrity. Unfortunately, practical restraints associated with the pilot nature of the present study precluded the addressing of these methodological limitations, and future research will address these shortcomings.
In summary, the present study demonstrated that tailoring PCIT for application in a real-world community clinic setting (Karitane Toddler Clinic in Sydney, Australia) was associated with a range of statistically and clinically significant improvements for families with young children displaying early onset disruptive behaviour. The present study highlights a number of issues relevant to community-based child and family clinical intervention services and to evaluating clinical practice in real-world settings, and is an important first step in the evaluation of PCIT effectiveness outside the university-based clinic setting.
Footnotes
Acknowledgements
We would like to thank Sheila Eyberg and David Hawes for providing feedback on early versions of this manuscript.