Abstract
There is a lack of evidence about the efficacy of behavioural or cognitive-behavioural training interventions for foster carers. The programmes are intended to assist foster carers in the management of the difficult behaviour of looked-after children and to provide competent care.
This Campbell/Cochrane systematic review, supported by Nordic Campbell Center, finds no evidence of the efficacy of the programmes. On the contrary, there is a need to further investigate whether such programmes may be harmful to looked-after children
Abstract
Background
The provision of training for foster carers is now seen as an important factor contributing to the successful outcome of foster care placements. Since the late 1960s, foster carer training programs have proliferated, and few of the many published and unpublished training curricula have been systematically evaluated. The advent of cognitive-behavioural therapy (CBT) and the research evidence demonstrating its effectiveness as a psychotherapeutic treatment of choice for a range of emotional and behavioural problems, has prompted the development of CBT-based training programmes. CBT approaches to foster care training derive from a ‘skill-based’ training format that also seeks to identify and correct problematic thinking patterns that are associated with dysfunctional behaviour by changing and/or challenging maladaptive thoughts and beliefs.
Objectives
To assess the effectiveness of cognitive-behavioural training interventions in improving a) looked-after children's behavioural/relationship problems, b) foster carers’ psychological well-being and functioning, c) foster family functioning, d) foster agency outcomes.
Search strategy
We searched databases including: CENTRAL (Cochrane Library Issue 3, 2006), MEDLINE (January 1966 to September 2006), EMBASE (January 1980 to September 2006), CINAHL (January 1982 to September 2006), PsyclNFO (January 1872 to September 2006), ASSIA (January 1987 to September 2006), LILACS (up to September 2006), ERIC (January 1965 to September 2006), Sociological Abstracts (January 1963 to September 2006), and the National Research Register 2006 (Issue 3). We contacted experts in the field concerning current research.
Selection criteria
Random or quasi randomised studies comparing behavioural or cognitive-behavioural-based training intervention (in a group or one-to-one settings or both) versus a no-treatment or wait-list control, for foster parents/carers.
Data collection & analysis
Two authors independently assessed trial quality and extracted data. We contacted study authors for additional information.
Main results
Six trials involving 463 foster carers were included. Behavioural and cognitive-behavioural training interventions evaluated to date appear to have very little effect on outcomes relating to looked-after children, assessed in relation to psychological functioning, extent of behavioural problems and interpersonal functioning. Results relating to foster carer(s) outcomes also show no evidence of effectiveness in measures of behavioural management skills, attitudes and psychological functioning. Analysis pertaining to fostering agency outcomes did not show any significant results. However, caution is needed in interpreting these findings as their confidence intervals are wide.
Reviewers’ conclusions
There is currently little evidence about the efficacy of behavioural or cognitive-behavioural training intervention for foster carers. The need for further research in this area is highlighted.
Internal sources of support
University of Bristol, UK
External sources of support
Centre for Evidence-Based Social Services, UK
Nordic Campbell Center, DENMARK
Contribution of reviewers
William Turner (WT) and Geraldine Macdonald (GM) both contributed to the writing of the protocol and the development of a search strategy. Searches were run by Jo Abbott and Eileen Brunt, trials search coordinators of the Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG). WT, Jane Dennis (JD) and GM independently reviewed titles and abstracts of potential trials and selected trials to include within the review. WT extracted data (double-checked by JD).
WT, JD and GM analysed results and wrote up the final text of the review and share responsibility for the current update of this review.
Acknowledgements
We would like to acknowledge the contribution made by the Centre for Evidence-Based Social Services (Exeter, UK) and the Nordic Campbell Centre (Copenhagen, Denmark) for their financial support for the completion of this review. We would like also to thank Professor Will Shadish (USA), Dr Jeff Valentine (USA) and Dr Julian Higgins (UK) for their helpful comments at many stages of the review process.
We would like to thank Professor Richard Barth, Dr Patti Chamberlain, Professor Judy Dunn, Dr Mair Edwards, Dr Louise Guerney, Dr Oriana Linares, Professor Kathy Lowe, Dr Helen Minnis and Dr Andy Pithouse for their help in clarifying details of trials with which they are associated. We would like to thank Jo Abbott and Eileen Brunt, trial search coordinator and maternity leave cover for trial search coordinator for the CDPLPG, for their help in helping devise and run the search strategy for this review, and Julie Millener (Research Secretary of the CDPLPG) for her unstinting support.
Potential conflict of interest
Two of the review authors (WT and GM) have undertaken a randomised controlled trial in the area of CBT for foster carers in the southwest of England (UK) (Macdonald 2004; Macdonald 2005).
What's new
Minor errors concerning significance of results have been corrected in the following areas for re-publication at Issue 1, 2008:
A1.
(c) Strengths and difficulties scale (both teacher and child reports)
(d) Reactive attachment disorder scale
Synopsis
Behavioural and cognitive behavioural training interventions for assisting foster carers in the management of difficult behaviour - no strong evidence yet
Providing training for foster carers is thought to enhance caring attitudes and skills, help foster carers deal more effectively with foster children's behaviour, and decrease foster carer attrition. Although training programs have proliferated, there has been minimal evaluative research to determine whether they are effective.
This review attempted to determine the effectiveness of cognitive-behavioural training interventions. Only six trials involving 463 foster carers were included. Results suggest little evidence of effect on looked-after children, foster carers and fostering agency outcomes.
Background
The concept of child welfare encompasses all of the policies, programmes, and practices centred on the well-being of children. The child welfare system, however, refers to a discrete network of services created to protect and serve children and their families when the families are either not able or not willing to provide an acceptable level of care (McGowan 1991). If children are unable to remain with their families because of court-ordered removal due to maltreatment or through voluntary arrangements with the family, the child welfare system assumes responsibility for caring for the children. State legislation and legal precedent establish the legal framework for placement procedures and for the periodic review of the child's out-of-home placement status. Traditional foster care, kinship care, residential group home care, and therapeutic foster care are the primary types of substitute care (Zukoski 1999). Traditional foster care could be further distinguished into respite care, short-term (either for emergencies, assessment or preparation for long-term), medium or long-term, and specialist foster care (such as care provided by those who ‘specialise’ in looking after young people with disabilities, on remand, or who have been identified as potential abusers) (Kelly 2000).
Foster care has been defined as care “provided in the carer's home, on a temporary or permanent basis, through the mediation of a recognised authority, by specific carers, who may be relatives or not” (Colton 1997). Foster carers are individuals who agree to provide care in a family environment for children who are in the custody of the child welfare system. Foster care is sometimes designed as a temporary measure, lasting until children can be reunited with their families or some other permanent placement can be arranged. Sometimes it can be planned as a long-term, alternative form of care for children who cannot be cared for by their natural parents. In some circumstances children can find themselves in unplanned foster care, for example following a crisis within the family or investigation of child abuse.
Foster carers are responsible for providing the daily care for children placed with them, and carry a significant responsibility for the children's experiences while in public care. The profile of children in foster care has changed significantly over time, with the majority now requiring much more than a stable home environment and adequate supervision. Foster parents are often asked to care for children with complex medical and behavioural problems. This has been increasingly the case as child care policy in many countries has moved away from placing children with special needs in residential schools, seeking instead to place them within families. A substantial number of children now in foster care have some form of disability and often come from home environments that have contributed to the development of behavioural and emotional difficulties (Parker 1992). They are likely to have histories of parent-child conflict, aggressive or self-destructive behaviour or both, disruptive community behaviour, school difficulties and emotional disturbance (Berridge 1997). Blatt and Simms (Blatt 1997) reported that foster children “have three to seven times more chronic medical conditions, birth defects, emotional disorders, and academic failure than children from similar socio-economic backgrounds”; later studies also emphasise the specific mental health problems of children placed in the foster care system (Richardson 2000; Meltzer 2004). Foster carers, therefore, often face exceptional challenges in caring for children placed with them. The difficulties inherent in working with children with challenging behaviour have turned the task of fostering into an extremely demanding job.
Providing foster care to children with special medical or emotional needs or both requires not only extra time, but also greater patience, skill and endurance in dealing with the child's demands and their effects on the foster carers’ personal and social life (Cliffe 1991). Those carers fostering adolescents may need to be particularly skilled and well-supported (e.g. Bebbington 1989; Sinclair 1995; Stern 1989; Triseliotis 1995). Utting (Utting 1997) has suggested that many foster parents are unprepared to meet the demands of behaviourally and emotionally disordered children. This situation can result in placement breakdowns, which further strain the resources of foster care. The effects of placement disruption pervade all aspects of foster care, foster children, foster carers and indeed family placement workers. Disruptions to placements and frequent changes of carers can undermine children's capacity for developing meaningful attachments, disrupt friendships, and contribute to discontinuities in education and health (Macdonald 2004). Unplanned terminations of placement have the potential to drain the field of experienced carers, a much needed but limited resource, and compound the problems of finding and training an adequate supply of foster carers to provide children with stable care. Foster carers look to child welfare agency staff for guidance, support and reassurance but the latter are generally hard-pressed and often crisis-driven, which hinders their ability to respond to the on-going needs of foster carers not experiencing crises. In the UK, the Audit Commission (1994) identified the absence of support and training as the feature most likely to affect recruitment, retention and quality of foster carers (Audit Comm. 1994).
Foster parent training
The term ‘foster carer training’ typically refers to an educational or training process or both designed to provide foster carers with information and skills designed to help them fulfil their responsibilities. Foster carer training differs from training programs designed for parents whose children are in foster care or residential care. The latter generally concentrate on helping parents to address those aspects of their parenting which contributed to the decision to place their child(ren) in foster care. This may be knowledge (of child development or management strategies), particular skills, or personal problems which may interfere with their ability adequately to parent their children. Pasztor and Evans (Pasztor 1992) noted that while parenting training programs can be traced to the early 1800s, foster parent training is a relatively recent development, as indeed is foster care in its present form. Pasztor and Wynne (Pasztor 1995) note that because parents did not typically receive training to parent their own children, the need to train foster carers was not recognised until relatively recently.
The provision of training for foster parents is now seen as an important factor contributing to the successful outcome of foster care placements. Foster carer training is believed to be associated with enhancing caring attitudes and skills, reducing behaviour problems in foster children, improving relationships between foster carers and child welfare agencies, and decreasing foster carer attrition (Boyd 1978; Hampson 1983; Lee 1991; Runyan 1981; Simon 1982; Sinclair 2004). Likewise, a lack of training is associated with failed placements (Audit Comm. 1994; Runyan 1981). Since the late 1960s, foster carer training programs have proliferated, and numerous unpublished curricula have been developed by both state and private child welfare agencies (cf. Zukoski 1999). They use numerous formats and a broad range of training methods (for example, Berry 1988). Foster carers generally are trained in groups, with training broadly aimed at meeting the needs of foster carers looking after children of all ages.
It is possible to identify two broad categories of foster care training. One is skill-based training, providing information about the typical developmental needs of children together with child management techniques. The other type focuses on providing foster parents with information and support to assist them in understanding their roles and responsibilities and supporting their efforts as they encounter the variety of issues associated with being a foster carer (Hampson 1985). Lee and Holland (Lee 1991a) found that “the content of many fostering training efforts often includes attention to three broad areas: (a) understanding child development and preparing for anticipated difficulties between child and parents, (b) orientation of foster carers to the agency and community services available to them, (c) support for the functioning of the foster family in order to increase placement stability”.
Behavioural and cognitive-behavioural approaches
Behavioural and cognitive-behavioural approaches fall into the ‘skill-based’ category, combining the use of techniques from cognitive therapy (based on cognitive theories) and behavioural therapy (derived from learning theory). Whilst behavioural approaches emphasised the environmental antecedents and consequences of behaviour in their development and maintenance, cognitive-behavioural (CBT) approaches emphasise that the role of cognition as a major determinant of behaviour and mood. CBT approaches combine behavioural (e.g., reinforcement and response cost) and cognitive techniques (such as challenging negative automatic thoughts) to identify and correct problematic thinking patterns that are associated with dysfunctional behaviour. The term ‘cognitive-behavioural’ is now often used to describe interventions using behavioural approaches as well as those incorporating more explicitly the use of cognitive techniques, and in the body of this review the term CBT will be used to refer to both.
Recent research evidence suggests that CBT approaches are effective interventions for a range of behavioural problems (cf. DOH 2001; Scott 2001). As behaviour problems are a major contributory factor to placement breakdown (cf. Cliffe 1991; Kazdin 1997; Nissim 1994) the use of CBT approaches in foster carer training has potential to help carers better manage a range of challenging behaviour that children in their care may present, thereby optimising the chances of placement stability and its consequences.
To date, no systematic review of the effectiveness of any foster carer training approach has been undertaken. This review begins to address this gap in our knowledge by examining the efficacy of cognitive-behavioural training programmes in this context.
Objectives
To assess the effectiveness of behavioural and cognitive-behavioural training interventions in improving a) placement stability, b) foster carers’ psychological well-being and functioning, and c) looked-after children's behavioural and relationship problems.
Criteria for considering studies for this review
Types of studies
Studies were eligible for the review if the allocation of study participants to experimental or control groups was by random allocation or quasi-random allocation, i.e., by day of the week, alternate numbers, case number or alphabetical order. Only studies comparing a CBT-based intervention versus a no-treatment or wait-list control were included.
Types of participants
Foster parents/carers (either from single- and/or two-parent families) looking after children and adolescents up to and including 18 years of age.
Types of interventions
Interventions (whether in a group or one-to-one) which were described by the authors as behavioural or cognitive-behavioural or which described the use of interventions derived from one or more of the following or both CBT and one of the following: operant learning: for example, positive and negative reinforcement, differential reinforcement, extinction, response cost (WebsterStratton 1997), ‘time out’; classical (respondent or Pavlovian) learning e.g. graded in vivo exposure, enuresis alarms; social learning theory: e.g. modelling, behaviour rehearsal; cognitive theories of learning e.g. understanding the influence of past events on current behaviour, challenging attributions and faulty information processing, reality testing, (Clarke 1997)
Types of outcome measures
A. Looked-after child outcomes
Psychological functioning (including psychiatric symptoms), e.g. depression, PTSD, anxiety; Behaviour problems (at foster home and/or at school, i.e. attendance, achievement, failure); Interpersonal functioning of the looked-after child (e.g. relationships with peers or other members of foster home or both).
B. Foster carer(s) outcomes
Measures of skills (e.g., behaviour management skills), knowledge, attitudes and behaviour change; Psychological functioning.
C. Foster family functioning
Foster family functioning (e.g., communication patterns and interpersonal relationships); Foster parent(s) - looked-after child relations.
D. Fostering agency outcomes
Placement stability (e.g., number of requests for removal, number of unrequested removals) or completion of allocated stay, or both.
Data sources included:
1) Rating scales. Taking into consideration that instruments vary in quality and validity, the minimum standard for the inclusion of the data from outcome instruments was that the psychometric properties for the instrument had been documented. Instruments could include either self report, or reports completed by an independent rater, teacher or other foster carer.
2) Assessment and agency Records, e.g. records of placement.
Search strategy for identification of studies
We searched the following databases
CENTRAL (The Cochrane Library Issue 3, 2006)
MEDLINE (January 1966 to September 2006)
EMBASE (January 1980 to September 2006)
CINAHL (January 1982 to September 2006)
PsycINFO (January 1887 - September 2006)
Sociological Abstracts (January 1963 to September 2006)
ERIC (January 1965 to September 2006)
Assia (January 1987 to September 2006)
National Research Register 2006 (Issue 3)
LILACS (Up to September 2006)
The following search strategy was used to search CENTRAL:
foster-home-care*:ME
(foster-care*)
(foster near care*)
(foster near parent*)
(foster near mother*)
(foster near father*)
((substitute near (care or carer*))
(((((#1 or #2) or #3) or #4) or #5) or #6) or #7)
behavior-therapy*:ME
(cognitiv* near (therap* or train*))
((behavior* or behaviour*) near (therap* or train*)
(parent* near train*)
(family near therap*)
((((#9 or #10) or #11) or #12) or #13) child*:ME
((((((((((child* or adolescen*) or boy*) or girl*) or teen*) or schoolchild*) or preschool*) or pre-school*) or infant*) or baby) or babies)
(#15 or #16)
((#8 and #14 and #17)
Search terms were modified to meet the requirements of individual databases as regards to differences in fields (see Table 04, Table 05, Table 06, Table 07, Table 08, Table 09, Table 10, Table 11 and Table 12 for details of other search strategies used). There were no language restrictions. Other sources of information included the bibliographies of systematic and non-systematic reviews and reference lists of articles identified through the search strategy. We also contacted authors and known experts to identify any additional or unpublished data.
Methods of the review
1. Selection of trials
All review authors (WT, JD and GM) screened titles and abstracts from the searches. WT and JD retrieved relevant articles, and all three authors assessed each article against the inclusion criteria, independently. No disagreement occurred which was not resolved by communication with the authors of a particular study (apart from the case of Brown 1980, which remains in the category ‘Studies awaiting assessment’ due to difficulties in locating the author). No recourse to a fourth adjudicator was necessary.
2. Quality assessment
Three review authors (WT, JD and GM) independently assessed the methodological quality of the included studies against agreed criteria (see below) for which a pro-forma was developed. No disagreement occurred which was not resolved by communication with the authors of a particular study. We contacted authors and others involved in included studies were contacted (Barth 2004; Barth 2006; Chamberlain 2006; Dunn 2004; Edwards 2002; Lowe 2004; Lowe 2006; Pithouse 2004) to supply missing information from five included studies (Barth 1994; Edwards 2002; Minnis 2001; Pithouse 2002). Blinding of providers and participants was identified at protocol stage as unlikely to be applicable within this review and was not used as a criterion to assess internal validity of included trials, however blinding of outcome assessors was assessed (see point 2iii, below).
i) Concealment of allocation
The two review authors independently assigned each included study to quality categories according to their degree of concealment of allocation to intervention groups, as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2005). These were as follows:
(A) indicates adequate concealment of allocation (e.g. by sealed envelopes or telephone randomisation)
(B) indicates uncertainty about whether the allocation was adequately concealed (possibly where the method of allocation concealment is unknown)
(C) indicates that the allocation was not adequately concealed (e.g. open random number lists or quasi-randomisation methods)
ii) Selection bias
In studies where allocation was not adequately concealed, the degree to which evidence of baseline imbalance and attempts to control for imbalance were reported were sought and summarised.
In addition, included studies were assessed with regard to the following criteria.
iii) Outcome assessment
MET: assessor unaware of the assigned treatment when collecting outcome measures.
UNCLEAR: blinding of assessor not reported and could not be verified by contacting investigators.
NOT MET: assessor aware of the assigned treatment when collecting outcome measures.
iv) Losses to follow up
MET: losses to follow up equally distributed between comparison groups.
UNCLEAR: losses to follow up not reported.
NOT MET: losses to follow up in excess of 20% OR unevenly distributed between comparison groups.
v) Intention-to-treat
MET: intention-to-treat analysis performed or possible with data provided.
UNCLEAR: intention-to-treat not reported, and cannot be verified by contacting the investigators.
NOT MET: intention-to-treat analyses not done and not possible with the data provided.
Other methodological issues were documented in the data extraction form.
3. Data management
The review authors independently extracted data, using a pro-forma. Data extracted included population, age, the control group, baseline characteristics, intervention characteristics and duration, compliance and outcome measures. These are summarised in the table ‘Characteristics of included studies.’ Citations and data were entered and organised in RevMan 4.2. We contacted authors of studies with missing data were contacted and some additional data were obtained as a result (in some cases; no attempts were made to impute missing data).
4. Missing data
If relevant missing data could not be obtained missing data and dropouts were assessed for each included study. The review reports the number of participants who are included in the final analysis as a proportion of all participants in each study. Reasons for missing data are provided in the narrative summary.
5. Measures of treatment effect
Data synthesis was conducted with RevMan 4.2., the latest version of the Cochrane's Collaboration meta-analysis software. Dichotomous outcomes were not reported in any of the included studies. Should additional studies be available at future updates, for dichotomous outcomes we will calculate the pooled odds ratio, because the odds ratio has statistical advantages relating to its sampling distribution and its suitability for modelling. Continuous data were analysed if means and standard deviations were available or if effect sizes could be calculated by other means (see ‘Results’ for further details). If continuous outcomes were measured with similar, but not identical, instruments across studies standardised mean differences (SMD) were compared across studies. Confidence intervals of 95% were used for individual study data and pooled estimates. We expected and found evidence of heterogeneity, hence we report results of random effects models, as planned in our protocol. Effects were examined at specific follow-up periods: short-term (up to three months) and medium to long term (six to nine months).
6. Assessment of heterogeneity
Consistency of results was assessed visually and by examining I2 (Higgins 2002), a quantity which describes the approximate proportion of variation in point estimates that is due to heterogeneity rather than sampling error. We supplemented this with the chi-squared test of heterogeneity, and by comparing results of fixed and random-effects models.
7. Sensitivity analyses
Should sufficient data be available for updates of this review, primary analyses will be based on available data from all included studies relevant to the comparison and outcome of interest. In order to assess the robustness of conclusions in relation to quality of data and approaches to analysis, sensitivity analyses will be performed. These will include: study design. Studies with adequate allocation concealment will be synthesised separately and a comparison of the overall estimates will be undertaken; intention-to-treat. The authors will report whether or not studies analysed data on an intention-to-treat basis. If this is not the case studies with fewer than 80% participants at follow up will be examined separately and results compared with overall effects for all included studies.
For dichotomous outcomes, such as ‘did placement break down?’, the authors will assume that those who were lost to follow up (i) had proportionately the same outcomes as those who completed in the control group, (ii) experienced the successful outcome, (iii) all experienced the unsuccessful outcome.
8. Subgroup analyses
Should sufficient data become available, the impact of certain study characteristics will be investigated by means of subgroup analyses. Depending on the profile of included studies the following subgroup analyses will be undertaken: the differential impact of group- versus individual-based versus combined training interventions in outcome; the differential impact of studies in which children are asymptomatic at the commencement of the intervention versus those in which children are symptomatic i.e. with extant psychological or behavioural problems or both, whether formally or informally diagnosed; differences in the type, intensity or length/period of the intervention.
Such analyses need to be treated with caution and this will be considered in the textual discussion.
9. Assessment of bias
If sufficient studies are identified in future, funnel plots will be drawn to investigate any relationship between effect size and study precision (closely related to sample size). Such a relationship could be due to publication or related biases or due to systematic differences between small and large studies. If a relationship is identified, the experimental diversity of the studies will be further examined as a possible explanation. (See also Egger 1997).
Description of studies
Studies included in this review
Searches for the initial version of this review were undertaken in April 2004 and 197 records of potential trials were identified. Following assessment of titles and abstracts, 32 full text copies were obtained. After assessment, 25 studies were excluded from this review. Seven papers were assessed as meeting the eligibility criteria.
Five studies were included in the original version of this review (first published in 2005) and all were based on published data (Barth 1994; Chamberlain 1992; Macdonald 2004; Minnis 2001; Pithouse 2002). Two trials still awaited assessment despite attempts to clarify their eligibility -Brown 1980 (an unpublished American PhD dissertation) and Edwards 2002 (a British study for which results had yet to appear).
For the 2006 update of this review searches were rerun across all databases in September 2006. Thirteen results were located, of these two were duplicates; a further two had been identified in previous searches, six were clearly irrelevant, leaving three for assessment by all authors. Two RCTs (Linares 2006; Pacifici 2005) were later excluded, as was one paper (Patterson 2005) which proved to be a review article. In the meantime, data for a study previously classed as ‘awaiting assessment’ appeared, although still in unpublished form, and is now included in this review (Edwards 2002). Thus, the current version of this review leaves six studies in the current version of this review. Details and reasons for exclusion can be found in the table ‘Characteristics of excluded studies’.
Setting
Two studies included in this review were conducted on the west coast of the USA (California (Barth 1994); Oregon (Chamberlain 1992)). The remaining four were conducted in different regions of the UK: south Wales (Pithouse 2002), north Wales (Edwards 2002), the south west of England (Macdonald 2004), Scotland (Minnis 2001). No set of researchers conducted more than one study. The studies span the years 1988 to 2002. All studies included in this review are published journal articles except Edwards 2002, which is based on a unpublished research report.
Baseline data
All studies provided some demographic data for the foster carers with varying degrees of detail, covering gender, family characteristics, number of biological children, occupation, income, educational level, number of previous placements, number and length of current placements, levels and years of fostering experience, and exposure to previous training opportunities (see Table 01). Foster carers in the studies usually were from two-parent households. The age of foster carers in each study ranged from early 30s to late 40s. Where demographic and other related information on the fostering context are provided, all investigators make a point of highlighting the equivalence of foster carers assigned to conditions on the reported characteristics.
All studies also provided information on the profiles of looked-after children/young persons with varying degrees of detail on age, gender, ethnic origin, diagnosis, health, general ability, natural family characteristics, current and previous placements, behaviour problems/psychopathology, and school adjustment (see Table 02). The age of the participants being looked-after ranged from 3 to 17 years of age within the included studies. Placement histories varied widely amongst individual studies. Length of current placement varied widely amongst individual children in all studies ranging from one month to 10 years. One source of possible clinical heterogeneity occurs in Barth's 1994 study wherein the focus was on children who were in foster care following sexual abuse within their own biological families; this study also may contain some heterogeneity as to the foster carers themselves (a proportion of which were recorded as being related to the children they looked after in a ‘kinship care’ arrangement (Barth 1994)). Most children/young persons were reported to exhibit difficult/challenging behaviour based on foster carers’ self reports or social work staff assessments or both.
All of the studies dealt with the effectiveness of training foster carers on a) children's emotional, psychological or behavioural functioning or both, and b) behaviour management methods. Training in all studies was delivered in groups led by experienced and qualified group leaders. Training arrangements varied widely among studies ranged from two hours per week over nine months (Chamberlain 1992) to six hours daily over three consecutive days (Minnis 2001). All of the studies were single two-group experiments except one evaluation of provision of enhanced services and stipends for foster parents, which included comparisons between that intervention, increased payment only, and a third group of ‘foster care only’ (Chamberlain 1992). (For this study, we chose to include only the first group (which involved a CBT component) and the no-treatment control in our analysis, as this approach best conformed to the original protocol). All studies employed a pre-post research design and all reported post-intervention data. All studies included data from self-report measures and two (Chamberlain 1992; Pithouse 2002) included data rated by researchers employed by the study. Follow-up periods were diverse and included measurements at one, three, six and nine months.
In addition to study details provided in the Table of ‘Characteristics of included studies’ and Table 01 and Table 02, we summarise each briefly below.
PSYCHOEDUCATIONAL GROUPS WITH FOSTER PARENTS OF SEXUALLY ABUSED CHILDREN (Barth 1994).
The authors acknowledged the lack of evaluations relating to training programmes designed to enhance the abilities of foster carers and set out to evaluate the effectiveness of training provision to foster carers looking after sexually abused children (Barth 1994). The sample, consisting of 27 foster carers, were allocated to a training group (N = 15) and a control group (N = 12). Method of allocation was not reported in the paper but identified as alternate allocation following personal communication with the first author (Barth 2004). Training was delivered in groups and utilised a psychoeducational approach whose origins, according to the authors, can be traced to the experience of the Child Protection Team at the Children's Hospital in Oakland, California (Barth 1994). The training was delivered over ten sessions; the content of each session is described in detail although no details are provided in relation to session's duration. Evaluation and assessment were based on self-reports from foster carers before the start of the training and approximately two months after the end of training. Outcome measures included psychopathology (measured by the Child Behavior Checklist (CBCL) (Achenbach 1983); the Child Sexuality Inventory (CSI) (Friedrich 1986); and foster carer satisfaction (quantified using a measure developed for the study (Barth 1994).
ENHANCED SERVICES AND STIPENDS FOR FOSTER PARENTS: EFFECTS ON RETENTION RATES AND OUTCOMES FOR CHILDREN (Chamberlain 1992).
The study involved the experimental evaluation of some of the reforms that the investigators themselves had hypothesised as being associated with measurable improvements in foster care. Effects of extra compensation and support/training were evaluated for both foster parents and children in their care. Seventy-two foster carers were randomly assigned to one of three groups: (1) enhanced support and training (ES & T) plus an increased payment of $70/month (N = 31); (2) increased payment of $70/month only (IPO) (N = 14); (3) control group involving foster care as usual (FCU) (N = 27). The investigators took care to point out that whilst the foster carers in their sample did not appear to differ at baseline, the number of ‘child problem behaviours’ between groups differed considerably at baseline, with the children in the ES & T group reported as exhibiting on average 7.5 ‘problem behaviours’ per day as compared to 5.71 in the IPO group and 3.71 in the FCU group (no standard deviations were provided in the published paper, nor were they available from the investigator following extensive correspondence) (Chamberlain 2006)). The training intervention is described in detail and involved weekly two-hour group foster carer meetings aimed at increasing skills in child behaviour management and thrice-weekly telephone contact by the group leader/facilitator. Foster carers in all three groups were first assessed within three weeks of the child's placement, and then three months, six months, and nine months later. Assessments and evaluation were based on self reports from foster carers and on input from staff at the three Oregon State Children's Services Division. Outcomes included number and frequency of child ‘problem behaviours’ as measured by the Parent Daily Report (PDR) (Chamberlain 1987); foster carer and caseworker satisfaction ratings (as measured by a seven-point scale developed for the trial); staff members’ impressions of foster carer effectiveness (discipline, social skills and ‘personal strength’); effects on foster carer dropout rates were also evaluated (Chamberlain 1992).
HELPING FOSTER CARERS TO MANAGE CHALLENGING BEHAVIOUR: EVALUATION OF A COGNITIVE-BEHAVIOURAL PROGRAMME FOR FOSTER CARERS (Macdonald 2004).
This study was designed to test whether training foster carers in cognitive-behavioural methods designed to help them manage challenging behaviour would have benefits for looked-after children and foster carers. A total of 164 foster carers were initially recruited from six local authorities. The final sample comprised of 117 foster carers comprised the final sample, with 67 allocated to the intervention group and 50 to the control. All foster carers had reported having problems managing children's behaviour within the home, but no formal screening took place. Due to the geographical distribution of foster carers it was not possible to make random allocation from the whole, and randomisation to training versus control conditions happened within smaller geographical locations. The CBT-based intervention training programme was based on Webster-Stratton's model (behaviour development and social learning theory, ‘ABC’ analysis) (WebsterStratton 1994; WebsterStratton 1998). Whilst primarily concerned to test whether training would enable carers to manage difficult behaviour, researchers were also interested in whether it would enhance carers’ confidence in their capacity to care for challenging children and young people, and, ultimately, whether improved skills and/or confidence would enhance placement stability. The programme was delivered initially as five weekly, three-hour sessions plus one follow-up session for two groups, then changed to four weekly, five-hour sessions plus one follow-up session. Training groups were run separately for foster carers looking after children under or over 10 years of age. Foster carers in both groups were assessed before the start of the training, at the end of the training, and six months later. Data were collected through interviews and self report measures. Outcomes included child psychopathology (Child Behaviour Checklist (CBCL) (Achenbach 1983) number of unplanned breakdown of placements; ‘time at risk’ (referring to the length of time, in months, that an opportunity for unplanned ending(s) existed for each placed child/young person); skills in the management of behaviour problems (based on a list of 21 behavioural strategies); frequency and/or severity of behavioural problems (a composite measure based on a list of 25 behaviours); knowledge of behavioural principles as applied to children (KBPAC), (O’Dell 1979); confidence in carers’ ability to manage difficult behaviour (measured qualitatively, based on questioning about the impact of the training); foster carer satisfaction (as measured by the Foster Carer Satisfaction Questionnaire) developed for the study.
MENTAL HEALTH AND FOSTER CARER TRAINING (Minnis 2001).
This study was designed to evaluate whether a training programme for foster carers focusing on communication skills and attachment would have a beneficial effect on the emotional and behavioural functioning of foster children. Out of a total sample of 268 foster families recruited from 17 Scottish areas, 121 families were assigned using random permuted blocks (block size 12) to either intervention (N = 57 families, 76 children in total) and control (N = 64 families, 106 children). The training was developed based on a modified version of an internationally used training manual (Richman 1993). The theoretical assumptions underlying the rationale for the training programme's development and results from its piloting have been reported separately in another published journal article (Minnis 1999). The intervention was a three-day programme on communication and attachment delivered by an experienced social worker/trainer. Training sessions ran for six hours per day, the first two days running consecutively with a follow-up day one week later. Foster carers, children and their teachers were assessed before the start of the training, immediately after, and nine months later (not applicable to all measures employed in the study). All data were collected through self report measures. Outcome measures for children included self-esteem as measured by a modified version of the Rosenberg Self-Esteem Scale (Rosenberg 1965); child psychopathology as measured by the Strengths and Difficulties Questionnaire (SDQ) (Goodman 1998) and the Reactive Attachment Disorder Scale (RAD) (Minnis 1999). Foster carer measures included financial ones, as measured by the Costs of Foster Care Questionnaire (Netten 1997) and satisfaction, as measured by a six-item questionnaire designed for the study (Minnis 2001).
TRAINING FOSTER CARERS IN CHALLENGING BEHAVIOUR: A CASE STUDY IN DISAPPOINTMENT? (Pithouse 2002)
This study examined the impact of training foster carers in techniques to manage challenging behaviour and is described by the authors as ‘semi-experimental’. 114 children specifically recruited to meet seven criteria for challenging behaviour provided the operational definition and the basis for selection by local authority social work staff. This number dropped to 103 after efforts were made to obtain complete baseline datasets. Children were divided into treatment (N = 54) and control (N = 49). Information about the method assignment to the training intervention and to the comparison groups was obtained after personal communication with two of the authors (Pithouse 2002: Pithouse 2004: Lowe 2004). The rationale and the theoretical underpinnings of the intervention are described in detail: in summary, the training programme sought to provide carers with skill development, clear plans for coping with emergencies and a proactive strategy aimed at smoothing the ‘fit’ between the person and the environment (Pithouse 2002). Training was delivered by a clinical psychologist from the study team to groups of approximately 15 carers over a period of three days. Post-intervention measures include the follow-up period which was set three to four weeks after the end of the training. Outcome measures included a version of the Index of Community Integration (Raynes 1989), modified to incorporate elements relevant to young people; a version of a section of the Disability Assessment Schedule (Felce 1994; Holmes 1982), modified in an attempt to enable recording of frequency and severity of problem behaviours; carers’ reactions to challenging behaviours as assessed by the Emotional Responses to Challenging Behaviour Scale (Hastings 1994); carers’ beliefs about the causes of challenging behaviour (as measured by the Challenging Behaviour Attributions Scale (Hastings 1997); emotional and physical well-being of carers was measured by the Malaise Inventory Scale (Rutter 1970); carers’ stress was measured by the Spielberger Self-Evaluation Questionnaire (Spielberger 1983). Finally, a short questionnaire was designed by the authors to assess whether carers had an analytic understanding of behaviour (Pithouse 2002).
EVALUATION OF THE APPLICATION OF THE “INCREDIBLE YEARS” PROGRAMME WITH FOSTER CARERS OF LOOKED AFTER CHILDREN IN GWYNEDD (Edwards 2002)
This study was designed to evaluate whether the “Incredible Years” training programme for natural parents (or step or adoptive/long-term parental figures or all of the aforementioned) could be implemented successfully with foster carers. The rationale of the study was based on the positive evaluations of the programme both the US (for example, WebsterStratton 1988) and in the UK (for example, Scott 2001a) with regards to improvements in parental attitudes and parent-child interactions, reductions in parents’ use of violent discipline style and child conduct problems. Twenty-nine foster carers were randomly assigned to one of two groups: the first intervention group (N = 13) and the wait-list control group (N = 16, second intervention group). The wait-list control group (second intervention group) provided baseline comparison data for the first intervention group. This group received its training a few months after the end of the first round of training and data from both groups was then combined to increase statistical power and the reliability of findings. The training intervention is described in detail and involved 12 weekly group training sessions each lasting 2 to 2.5 hours aimed at increasing child management skills, as well as foster carer -child communication patterns and foster carers’ problem-solving skills. The training also utilised 10 videotape programmes which were used throughout training by the trainers to stimulate group discussions about the usefulness/effectiveness of the techniques/methods demonstrated in the audio-visual material. Assessment and evaluation were based on self completed measures from participating foster carers and on placement information accessed through the Social Services database. Outcome measures included a) foster carers’ functioning measures (measured by the Parenting scale (PS) (Acker 1993)and the General Health Questionnaire (GHQ) (Goldberg 1972); b) child behaviour measures (measured by the Strengths and Difficulties Questionnaire (SDQ) (Goodman 1997) and the Eyberg Child Behaviour Inventory (ECBI) (Eyberg 1980) and c) foster carer satisfaction with the training received (by employing Weekly Evaluation Forms and the Parent's Satisfaction Questionnaire (PSQ) (WebsterStratton 1984) (the latter was completed at the end of each training cycle).
Methodological quality of included studies
Further information was acquired from almost all authors as papers varied greatly in their reporting of data. See also Table 03.
i) Concealment of allocation
This review includes six studies, of which four were randomised controlled trials (Chamberlain 1992; Edwards 2002, Macdonald 2004; Minnis 2001) and two were quasi-randomised controlled trials (Barth 1994; Pithouse 2002). Macdonald 2004 reported use of a computer-generated numbers table and Edwards 2002 used alternate ‘out of a hat’ technique; both studies were coded as ‘A’ for allocation concealment. Minnis 2001 used random permuted blocks (block size 12) (Pocock 1983) and reported stringent efforts to conceal assignment list and identifying information (Minnis 2001). Chamberlain 1992 was not explicit about method of randomisation or concealment of allocation in the published paper, but reported in personal communication that a ‘coin-flip’ method was used and thus rates an ‘A’ (adequate) (Chamberlain 2005). Of the quasi-randomised controlled trials, one study stratified participants by age prior to assignment of participants to conditions (Pithouse 2002; Pithouse 2004). As the investigator in charge of allocation used a list with names and birth dates and was acquainted with some of the participants, it was decided this trial was subject to some potential bias and it received a ‘C’. Barth 1994 did not report method of randomisation in the published paper but personal communication revealed that alternate allocation had been used and the allocation given to this paper is therefore ‘C’ (Barth 2004).
ii) Selection bias
We planned in our protocol to report (concerning studies where allocation is not adequately concealed) the degree to which there was evidence of baseline imbalance and attempts to control for imbalance. We found that, regardless of method of assignment, where demographic and other related information on the fostering context were provided, all investigators made a point of highlighting the equivalence of foster carers assigned to conditions on the reported characteristics (Barth 1994; Chamberlain 1992; Edwards 2002; Macdonald 2004; Minnis 2001; Pithouse 2002; Table 01). The same cannot be said of differences amongst children in the five studies included in this systematic review, which varied considerably. For more information, see Table 02.
iii) Outcome assessment
As predicted in the protocol, none of the included studies attempted blinding of participants or providers of the intervention (which was felt to be, as Minnis put it “unfeasible and unethical” (Minnis 2001). Only one of the studies reported blinding of outcome assessors (Minnis 2001) and so for this category, Barth 1994, Chamberlain 1992, Edwards 2002, Macdonald 2004 and Pithouse 2002 received a rating of ‘criterion not met.’
iv) Losses to follow up
Ratings for each trial are as follows:
Barth 1994. The loss to follow up in the treatment group from pre- to post-treatment was 15 to 13, representing a 13% loss; for the control, the loss was 12 pre to 10 post, representing a 17% loss. Therefore, this trial merits a ‘MET’ for this criterion.
Chamberlain 1992. The dropout rate was not evenly distributed across the three groups involved in the trial (9.6 % for ES & T group, 14.3 % for the IPO group, and 25.9 % for the control group). We rate this trial as ‘NOT MET’ for this criterion, because losses to follow up exceed 20% and are unequal enough to bias the results.
Edwards 2002. The loss to follow up in the first intervention group from pre-to post training was 13 to 9, representing a 31% loss; for the second intervention group (which acted as the wait-list control for the first intervention group), the loss was 16 pre to 11 post, representing a 31% loss. Therefore, this trial is given a rating of ‘NOT MET’ for this criterion.
Minnis 2001 began with a sample of 160 families; after randomisation the sample was reduced to 121 families (23 families withdrew from the intervention group and 16 families withdrew from the control group) with 182 children; 57 families were assigned to treatment (76 children) and 64 families (106 children) to control. While the variance in the number of children between groups is quite large, this is due to the fact that the unit of allocation during randomisation was the family. Data for completion are presented for children (loss to follow up 18% in the treatment arm and 17% in the control. Minnis 2001 is thus given a rating of ‘MET’ for this criterion.
Macdonald 2004 began with 67 foster carers allocated to CBT training, dropping to 55 at time 1 and 49 at time 3 (losses to follow-up of 18% and 27%); in the wait list control condition, the initial group of 50 fell to 45 at time 1 and 40 at time 3 (losses to follow-up of 10% and 20%). This trial is given a rating of ‘NOT MET’ for this criterion.
Pithouse 2002 began with 53 carers in each assigned group (treatment and control) and reports nothing concerning rate of dropout. This trial is thus rated as ‘UNCLEAR’ for the criterion ‘loss to follow up’.
v) Intention-to-treat
None of the studies included in this review reported intention-to-treat analysis; therefore, all studies were given a rating of ‘NOT MET’ on this criterion of quality.
Results
The results of this review are organised into two sections:
Section A: Individual results of the studies in the four general categories of outcome measures (e.g. looked-after children, foster carers, foster family, and fostering agency) and their respective sub-categories.
Section B: Meta-analyses of the data for the three outcomes for which there was a sufficient number of trials this, namely -looked-after child psychological functioning (assessed immediately after training and six to nine months after training) and foster carers’ behaviour management skills and knowledge.
Section A
The following section provides the results for each of the four general outcomes and their respective sub-categories as specified in the methods of the review. If it has been possible to calculate an effect size and 95% confidence intervals, these are reported in the form of standardised mean differences (SMDs). If effect sizes have been calculated and reported, a minus sign indicates that the result favours the training group (unless stated otherwise).
An effect size of 0.15 or less has been treated as having no clinically meaningful effect, and an effect size ranging between .015 and 0.40 has been treated as a clinically meaningful but small effect (Thalheimer 2002). An effect size greater than 0.4 up to 0.75 has been treated as a clinically meaningful effect of moderate size, and an effect size greater than 0.75 has been treated as a large, clinically meaningful effect size.
The findings from studies in which it was not possible to obtain the necessary data with which to calculate an effect size have also been summarised. We are aware that effect sizes can be computed in a number of ways. Chiefly, we made strenuous efforts to obtain means and SDs when these were not reported in papers. We also paid attention to data given in other forms, but found these were insufficient to calculate further effect sizes. Briefly: Barth 1994. In the published paper, only significance levels were reported. The investigator communicated that no other data were now available (Barth 2006); Chamberlain 1992. Means were provided for some measures, but SDs could not be retrieved by investigator despite strenuous efforts (Chamberlain 1992); some F tests were reported in the published paper, but incomplete (no degrees of freedom were supplied). Edwards 2002. Means and SDs were provided in the report, which were used in meta-analysis; Macdonald 2005. Means and SDs were provided in the initial report and published paper, which were used in meta-analysis; Minnis 2001. Means and SDs were provided in the published papers and in an unpublished doctoral thesis, which were used in meta-analysis; Pithouse 2002. In the published papers (three), investigators described statistical analyses using ‘non-parametric tests’ but only to the level of whether results were ‘statistically significant or not’. Investigators have been unable to respond to requests for further information (Lowe 2006); we concluded no further effect sizes could be calculated.
A. Looked-after child outcomes
Trials included in this review used a variety of measures to capture information on changes in child psychopathology, self-esteem, attachment, inappropriate sexual behaviour, and relationships with peers.
A1. Psychological functioning (including psychiatric symptoms)
All included studies evaluated the effectiveness of CBT-based foster carer training with regards to a range of measures of looked after children's psychological functioning. There was insufficient data in three of these studies to calculate effect sizes (standardised mean differences) and confidence intervals (CI), and the results of these three studies have not been included in the graph displaying the results (Barth 1994; Chamberlain 1992; Pithouse 2002). Of the remaining three studies which provided sufficient data to calculate effect sizes, a total of 11 assessment of outcome were produced on a range of measures relating to looked-after children's psychopathology, self-esteem, and attachment disorder.
(a) Child Behaviour Checklist (CBCL)
Two studies (Barth 1994 and Macdonald 2004) used the CBCL (Achenbach 1983) to assess the extent of child psychopathology before and after training.
Macdonald 2004 examined levels of looked-after children's psychopathology as reported by the foster carers six months after training. The results were compatible with no evidence of effectiveness regarding CBCL-derived internalising (SMD -0.04; 95% CI -0.62 to 0.54), externalising (SMD -0.05; 95% CI -0.64 to 0.53), and total (SMD -0.02; 95% CI -0.60 to 0.57)] behavioural profile scores (Achenbach 1983).
Barth 1994 also assessed the effectiveness of foster carer training in improving looked-after children's psychopathology, but did not provide sufficient data to calculate an effect size. In Barth's study, scores from the treatment group on four CBCL behaviour dimensions (schizoid-obsessive, aggressive, cruel, and immature-hyperactive) were in the clinical range (that is, T equal to or greater than 70) prior to training and scores on 17 behaviour dimensions after training. For the control group, scores on two CBCL behaviour dimensions were in the clinical range (e.g., anxious-obesity, immature-hyperactive) prior to training, and on seven CBCL behaviour dimensions after training (e.g., summary, external, delinquent, aggressive, cruel, depressed-withdrawal, immature-hyperactive). Differences between pre-test and post-test scores were significant for only two scores (Barth 1994): For the training group, CBCL ratings pertaining to Social Behaviour significantly improved from pre- to post-test (31.0 to 44.2 respectively). For the control group, CBCL ratings pertaining to Aggression significantly increased from pre- to post-test (63.7 to 78.3 respectively). Results from statistical analyses comparing training versus control group at post-intervention are not reported but as the authors state: “the treatment group was more disturbed before and after training, but both groups appeared to have worsened in equal measure from the pre-test to follow-up” (Barth 1994a.
(b) Rosenberg Self-esteem scale
One study (Minnis 2001) considered self-esteem as an outcome and used the modified Rosenberg Self-esteem scale (MRS) (Rosenberg 1965) to measure change. The results suggested a positive but not statistically significant effect on foster children's self-esteem (SMD -0.17; CI -0.58 to 0.23).
(c) Strengths and Difficulties Scale (SDQ)
Two studies evaluated the effectiveness of foster carer training in improving looked-after children's psychopathology using the Strengths and Difficulties Scale (SDQ) (Goodman 1998). In the Minnis 2001 study scores on the SDQ were obtained separately by children, foster carers and teachers. The results for foster carers suggested a negative, i.e., favouring the children in the control group, but not statistically significant effect as regards levels of psychopathology (SMD 0.25; 95% CI -0.08 to 0.57). The results for teachers also suggested a negative and statistically significant effect (SMD 0.80; 95% CI 0.47 to 1.14)]. Additionally, a negative, statistically significant effect (SMD 0. 40; 95% CI 0.07 to 0.73) was obtained for the results for the foster children.
Edwards 2002 study reports separate scores on the SDQ for the ProSocial and Total subscales. The results suggested positive but not statistically significant effects on levels of psychopathology for looked-after children for both subscales, ProSocial subscale (SMD 0.58; 95% CI -0.32 to 1.49), Total subscale (SMD -0.41; 95% CI -1.30 to 0.48).
(d) Reactive Attachment Disorder Scale
Additionally, Minnis 2001 employed of the Reactive Attachment Disorder Scale (RAD) (Minnis 1999) to assess extent of children's attachment disorders. Results relating to scores at post-intervention favoured the children in the control group (SMD 0.46; 95% CI 0.09 to 0.84). This result is significant. Analysis of RAD scores at nine months interval suggested a similar negative, i.e. favouring the control group, but not statistically significant effect on children's attachment (SMD 0.31; 95% CI -0.02 to 0.65).
A2. Behaviour problems (at foster home and/or at school, i.e. attendance, achievement, failure)
(a) Child Sexuality Inventory (CSI)
One study, Barth 1994, assessed the effectiveness of foster carer training in improving children's inappropriate sexual behaviour, but did not provide sufficient data to calculate an effect size. Statistical analysis of data pertaining to factor scores (i.e. boundary permeability, sexual aggression, self stimulation, and sexual inhibition) on the Child Sexuality Inventory (CSI) (Friedrich 1986) did not reveal any significant change or improvement or both between pre- and post-test for both training and control groups. No significant differences were found between pre- and post-intervention on mean child sexual abuse scores (37 dimensions) for the control group. For the training group, a significant increase in the dimension titled ‘looks at people when nude’ was recorded between pre- and post-test (.60 to 1.23 respectively) and a significant decrease was observed in the dimension titled ‘shy about undressing’ (.93 to .46 respectively).
(b) Frequency of ‘problem behaviours’ (PDR and modified scale designed by Pithouse)
Three studies examined the effectiveness of foster carer training on reduction in the reported frequency of ‘problem behaviours’. Two of these studies did not provide sufficient data to calculate an effect size (Chamberlain 1992; Pithouse 2002) and the Macdonald 2004 study reports non-normally distributed data for this outcome and can not be used.
Both Chamberlain 1992 and Pithouse 2002 sought to measure frequency of ‘problem behaviours’, Chamberlain used a 36-item scale (the Parent Daily Report) of ‘problem behaviours’ (e.g., arguing, destructiveness, short attention span, temper tantrums, inappropriate sexual behaviour, etc) (Chamberlain 1987). Pithouse 2002, divided behaviours into ‘severe’ and non-severe, based on a measure designed for the study derived from a modified version of two other scales (Felce 1994; Holmes 1982).
In Chamberlain's study (Chamberlain 1992), the authors report a repeated measures analysis on the frequency of problem behaviours which indicates a ‘significant’ decrease in problem behaviours for the Enhanced Support and Training (ES & T) condition relative to the other two conditions (Increased Payment Only [IPO] and Foster Care as Usual [FCU]); however, the fact that the groups differed significantly at baseline raises questions about the validity of their result. The investigators took care to point out that whilst the foster carers in their sample did not appear to differ at baseline, the number of ‘child problem behaviours’ between groups differed considerably, with the children in the ES & T group reported as exhibiting on average 7.5 ‘problem behaviours’ per day as compared to 5.71 in the IPO group and 3.71 in the FCU group. The number of problems reported post-test the ES & T group fell to 3.85 per day; and those in the IPO group dropped to 3.94; whilst the control group of FCU actually worsened (increasing numbers of problem behaviours to 4.56 per day). T he investigators note that the ‘norm’ for non-referred children would be an average of 5 ‘daily problems’ (Chamberlain 1987, Chamberlain 1992). Mean scores of the Parent Daily Report (PDR) (Chamberlain 1987) indicated a marked drop in the number of child problem behaviours over a three-month period (with ‘problem behaviours) halved in the ES & T group, substantially decreased in the IPO group, and slightly worse in the FCU (foster care only) group. Again, these findings should be treated with some caution, as the samples were small and the three groups were markedly non-equivalent at baseline (Chamberlain 1992).
In the Pithouse 2002 study, analysis of data pertaining to the number and frequency of presenting behaviours showed for both measures a slight decrease from pre- to post-intervention for both intervention and control groups and were not significant. Data pertaining to the ‘severity’ (undefined) of presenting behaviours did not show any significant differences between the intervention and comparison groups either at baseline or post-intervention periods.
Macdonald 2004 used a composite measure relating to carers’ perception of frequency and severity of difficult behaviour(s). Non-parametric analysis did not reveal any statistically significant differences between training versus control group at any time point, although the number of reported problems was significantly reduced over time for both groups.
(c) Eyberg Child Behaviour Inventory
The inventory was employed in the study by Edwards 2002 in an attempt to examine frequency of commonly child behaviour problems and the extent to which such behaviours were perceived as problematic. The study reports separate scores for the Intensity and Problem subscales. The results suggested positive but not statistically significant effects on both subscales, Problem subscale (SMD -0.10; 95% CI -0.98 to 0.78), Intensity subscale (SMD -0.40; 95% CI -1.30 to 0.49).
A3. Interpersonal functioning of the looked-after child (e.g. relationships with peers and/or other members of foster home)
Only one study (Pithouse 2002) examined this outcome but did not provide sufficient data to calculate an effect size. The study reported analysis of data relating to children's use of community facilities, both for the number of facilities used and the number of facilities from which children were excluded (e.g. school, job, sports, shops, library, discos, etc). The study reports a statistically significant difference at baseline between the intervention group (9.6 facilities used per child) and control (8 facilities used per child). At post-intervention this had altered only slightly (an average of 9.4 in the intervention group and 8.3 in the control). No statistical differences were reported for the analysis of data relating to the number of community facilities from which children were excluded.
B. Foster carer(s) outcomes
Trials included in this review used a variety of measures to capture information on changes in foster carers’ skills, knowledge of CBT principles, attitudes and behaviour with regard to the children in their care.
B1. Measures of skills (e.g. behaviour management skills), knowledge, attitudes and behaviour change
Three of the included studies evaluated the effectiveness of CBT-based training on foster carers’ behaviour management skills and knowledge (Edwards 2002; Macdonald 2004; Pithouse 2002). Pithouse 2002 did not provide sufficient data to calculate effect sizes (standardised mean differences) and confidence intervals (CI), and the results of this study have not been included in the graph displaying the results. Of the two studies (Edwards 2002; Macdonald 2004) which provided sufficient data to calculate effect sizes, two assessments of outcome were produced on knowledge of behavioural principles and child management techniques.
(a) Challenging behaviour attribution scale
One study (Pithouse 2002) assessed the effectiveness of CBT training for foster carers in establishing attribution models of a child's difficult behaviour, but did not provide sufficient data to calculate an effect size. The Challenging Behaviour Attribution Scale (Hastings 1997) contains 33 items, classed under five subscales to represent five causal models of challenging behaviour. These subscales include ‘learned behaviour’, ‘biomedical factors’ ‘emotional factors’, ‘stimulation’, and ‘physical environment’. Results presented separately for the five subscales show significant differences between groups for the Emotional subscale at both pre- and post-intervention, and the Biomedical subscale at post-intervention. According to the authors, these results indicate that carers across both groups attributed challenging behaviour mostly to learned positive and learned negative behaviours and to emotional causes. No significant changes in total scores were found (Pithouse 2002).
(b) Knowledge of Behavioural Principles as Applied to Children (KBPAC)
One study assessed the effectiveness of training in improving foster carers’ knowledge in behavioural principles (Macdonald 2004). The ‘Knowledge of Behavioural Principles as Applied to Children’ (KBPAC) is a 50-item forced-choice test designed to measure knowledge/understanding of ‘good’ behavioural parenting skills (O’Dell 1979) with higher scores indicating greater understanding of behavioural principles with children. The results show a marked improvement in knowledge for the participants in the training group as indicated by the KBPAC scale (SMD 0.75; 95% CI 0.31 to 1.19).
(c) The Parenting Scale (PS)
Edwards 2002 used the Parenting Scale (Acker 1993) to assess self-reported methods of child management techniques commonly used by the carer. Respondents indicate on a 7-point Likert scale the response that best describes their typical disciplining style during the last two months. Scores toward the lower end of the 1 to 7 scale are considered to reflect more effective child management strategies. Results suggested a positive but not statistically significant effect on levels of child management strategies between training vs. control participants (SMD -0.27; 95% CI -1.15, 0.62).
(d) Use of behavioural strategies
In addition to exploring knowledge of behavioural techniques, Macdonald 2004 sought to measure their application. Results of this outcome are summarised narratively as data do not comply to normal distribution assumptions. Investigators surveyed foster carers pre- and post-intervention and analysed the range of behavioural techniques/strategies that foster carers employed in dealing with difficult behaviour. At post-intervention there was evidence that training had a significant effect (Fisher's Exact Test) on whether or not carers made use of tokens (0% in control condition versus 10% in the training condition), ‘grounding’ (confining to the house) (16% in the control condition versus 4% in the training condition) and ABC analysis (a CBT technique designed to encourage ‘observation, description and analysis’ of behaviour) (9% in the control condition versus 42% in the training condition). At six-month follow up, the difference in use of ABC by carers in the training condition remained statistically significant (Fisher's Exact Test) compared to those in the control condition. In a finding the authors reported as ‘unexpected’, the use of ‘response cost’ (the removal of privileges contingent on unwanted or inappropriate behaviours) as a behavioural strategy was more taken up by those who did not receive CBT training (i.e., only 18% in the training condition as compared to 37% in the control group) (Macdonald 2004).
B2. Psychological functioning
Psychological functioning of foster carers (thought to be important to foster carer retention) was measured by gathering data on stress and anxiety associated with their foster care duties. Four studies examined the effectiveness of CBT-based foster carer training on levels of carers’ psychological functioning. There was insufficient data in three of these studies to calculate effect sizes (standardised mean differences) and confidence intervals (CI), and the results of these three studies have not been included in the graph displaying the results (Chamberlain 1992; Macdonald 2004; Pithouse 2002). Edwards 2002 study provided sufficient data to calculate effect sizes for one outcome assessment relating to foster carers psychological functioning
(a) Evaluation of programme by foster carers
In the Chamberlain 1992 study foster carers in the ‘Enhanced support and training’ (ES & T) group gave positive feedback about the weekly group meetings, reported that the group had helped them deal effectively with their foster child's problems, and that they would recommend the group to other foster carers. It is reported that most foster carers rated the groups at 6.5 on a seven-point scale (i.e., between ‘very informative’ and ‘extremely informative’). Additionally, caseworkers’ impressions of the ability of this group of foster carers to manage children's behaviour problems were favourable (Chamberlain 1992). In the Macdonald 2004 study qualitative data collected suggested that attendance on the course gave participants more confidence to deal with difficult situations and behaviour. The authors claim that increased confidence was probably the most significant finding from the study. Satisfaction ratings relating to the programme in general and other related aspects of the training (e.g. teaching format, materials, and trainers) were high overall amongst those in the training condition (16 ‘very satisfied’, 25 ‘satisfied’, 4 ‘slightly satisfied’ and 2 ‘slightly dissatisfied’).
(b) Emotional Responses to Challenging Behaviour Scale
One study (Pithouse 2002) used the Emotional Responses to Challenging Behaviour Scale (Hastings 1994) but provided insufficient data to calculate an effect size. The study reported scores showing a statistically significant decrease in score for both groups at post- intervention (Pithouse 2002). While this is a positive finding, Pithouse 2002 does not report results between trained and control-group carers at post-intervention.
(c) Malaise Inventory
Additionally, Pithouse 2002 employed the Malaise Inventory (Rutter 1970) (currently used primarily to measure stress in those caring for dependents) but narrative description of findings does not permit computation of an effect size. The authors report that there were no significant differences between the two groups at either pre- or post-intervention assessment.
(d) Spielberger Self-Evaluation questionnaire
Further, in Pithouse 2002 scores on the two subscales (State and Trait) of the Spielberger Self-Evaluation questionnaire were analysed separately (Spielberger 1983). With respect to the State subscale scores no significant differences were found between comparison and intervention group and no significant changes were evident over time. With respect to scores of the Trait subscale, significant decreases in score were reported over time for both groups. Participants’ satisfaction with the training overall was very positive at post-intervention and at follow up.
(e) General health questionnaire (GHQ)
Edwards 2002 used the General Health Questionnaire (Goldberg 1972) to assess foster carers’ mental health. The instrument is widely used in community settings to assess fluctuations in normal psychological functioning and is sensitive to anxiety and depressive symptoms. The results suggested a positive but not statistically significant effect on general psychological health (SMD -0.40; 95% CI -1.30 to 0.49)].
C. Foster family functioning
C1. Foster family functioning (e.g. communication patterns and interpersonal relationships)
No data/measures on foster family functioning were reported by any of the studies.
C2. Foster parents(s) - looked-after child relations
No data/measures on foster carers-looked-after child relation were reported by any of the studies.
D. Fostering agency outcomes
Four studies examined the effectiveness of CBT-based foster carer training on fostering agency outcomes (Chamberlain 1992; Edwards 2002; Macdonald 2004; Minnis 2001). There was insufficient data in three of these studies to calculate effect sizes (standardised mean differences) and confidence intervals (CI), and the results of these three studies have not been included in the graph displaying the results (Chamberlain 1992; Edwards 2002; Minnis 2001). No data relating to foster agency outcomes were reported in the studies by Barth 1994 and Pithouse 2002.
Placement stability (e.g. number of requests for removal, number or unrequested removals) and/or completion of allocated stay
In the Macdonald 2004 study, data pertaining to the number of unplanned placement breakdowns were examined for group conditions and time periods. Effect size has not been computed due to skewed data. The authors report that differences in the number of unplanned placement terminations both at post-training and follow-up periods between training and control groups were not found to be significant (Mann Whitney U statistic). Additionally, the study also examined ‘months at risk’ data for carers in the training and control conditions; results did not show evidence of effectiveness regarding this outcome post-training (SMD 0.30; 95% CI -0.12 to 0.73)] or after a six-month period (SMD 0.33; 95% CI -0.10 to 0.75).
In the Chamberlain 1992 study, 12 out of the 72 participating foster families discontinued providing care. The dropout rates for the three groups were 9.6% for the ES & T, 14.3% for the IPO, and 25.9% for the control group. The authors note that even the control group had a substantially lower dropout rate than foster families across the state dropout rate (i.e. 40% in Oregon, USA). Data relating to the stability of the child in foster care indicated that out of the 72 study children 18 were returned home during the study (the reasons for the move were improvements in the family of origin's situation rather than factors associated with the child's adjustment in care). Analyses of the data pertaining to the number of successful days in care were concentrated on the remaining 54 children who were classified as either successful (remained in the foster home) or unsuccessful (ran away, were moved to another home, or were placed in residential care). Children in the ES& T group had significantly more successful days in care than children in the other two conditions (F = 3.45, P < 0.04). Caution must be applied to this finding due to the heterogeneity in the three groups described at length above.
In the Minnis 2001 study, the cost of care in terms of service use (as measured by the Cost of Foster Care Questionnaire (see Minnis 2001 and Netten 1997) increased in the intervention group, but the increase was not statistically significant in comparison to the control group.
Edwards 2002 provides a narrative description of significant incidents of placement breakdowns (one incident), complaints against foster carers (one incident) and de-registration (eight incidents) during the period of the study.
Section B: Meta-analysis
Three studies provided sufficient data with which to conduct a meta-analysis for three outcomes -looked-after child psychological functioning (assessed immediately after training and at six to nine months after training); and foster carers’ skills. These analyses combine results from a range of instruments, all measuring the same outcome. Also of note is that the confidence intervervals for the three outcomes are wide, indicating a great deal of uncertainty about the size of the effects.
A1. Looked-after child psychological functioning (assessed immediately after training)
Two studies (Minnis 2001, Edwards 2002) evaluated the effectiveness of training in improving child psychopathology assessed soon after the end of training using two standardised instruments -Reactive Attachment Disorder Scale (RAD) (Minnis 2001) and Strengths & Difficulties Questionnaire - (SDQ-Total subscale scores) (Edwards 2002). The two studies provided data from a total of 134 participants (61 training group and 73 control group). The combined data suggest no clinically meaningful effects, but the confidence interval was wide and included negative effects that, if real, indicate large and clinically meaningful harms and positive effects that, if real, indicate large and clinically meaningful benefits (SMD 0.13; 95% CI -0.71 to 0.96).
Looked-after child psychological functioning (assessed six to nine months after training)
Minnis 2001, Macdonald 2004 assessed the effectiveness of foster carer training in improving child psychopathology 6 to 9 months following training using the Child Behavior Checklist (CBCL-Total scores, Achenbach 1983) (Macdonald 2004) and the Reactive Attachment Disorder Scale (RAD, Minnis 1999) (Minnis 2001). The two studies provide data from 188 participants (85 training group and 103 control group). The combined data suggest a clinically meaningful but small effect, but this result was not statistically significant (SMD 0.23; 95% CI -0.06 to 0.52).
B1. Foster carer skills
Two studies (Edwards 2002; Macdonald 2004) examined the effectiveness of training in improving foster carers’ behaviour management skills and knowledge using the PS (Acker 1993) and the KBPAC (O’Dell 1979) respectively. The two studies provide data from a total of 106 participants (57 training group and 49 control group). The combined data suggest that a clinically meaningful but small effect, but the confidence interval was wide and included negative effects that, if real, indicate large negative and clinically meaningful harms and positive effects that, if real, indicate large and clinically meaningful benefits (SMD 0.32; 95% CI -0.67 to 1.31).
Discussion
The results reported within the six randomised trials, conducted over a 12-year period provide no evidence that training foster carers in cognitive-behavioural methods has a significant impact on psychological functioning of looked after children, their behavioural profile or their interpersonal functioning.
All studies start with the assumption, either explicit or implicit, that children's behaviour would improve as a result of enhancing their carers’ knowledge about effective behaviour management and mastery of the associated skills in their use. This premise is based on the success of parent training programmes with birth parents but does not seem to be confirmed in this context.
A number of competing explanations could be offered for this finding. Studies were, for one thing, underpowered to detect small (but possibly important) effect sizes. At present, this means that we really do not have much information about the true effects of the intervention, due to the fact that in many cases, the confidence intervals in the outcomes measured are so wide that the interval encompasses negative effects that, if real, would be evidence of clinically meaningful harms and also positive effects that, if real, would be evidence of clinically meaningful benefits. Furthermore, particular attention should be given to the baseline characteristics of the looked-after children participated in these studies. Table 02 confirms that the children in these studies have come from disadvantaged backgrounds and have suffered serious abuse or neglect, or both sometimes over many years. Such children have often experienced a wide range of emotional, psychological and behavioural traumas, resulting in problems which manifest themselves in a range of situations over which foster carers have little direct influence, and some of which may continue to act as painful stimuli e.g. ongoing contact with birth parents or other relatives. Three of the studies (e.g., Barth 1994; Macdonald 2004; Minnis 2001) used in this review make particular reference to the measurable psychopathology of these young people (on average very close to the clinical range). It is probable that in order for foster care intervention to be successful, training needs to be supported by other services and interventions designed directly to address children's traumas and to help manage their behaviour outside of the foster home. The effectiveness of other foster care arrangements, e.g., treatment foster care, needs to be explored in this context, and a systematic review of these interventions is in progress.
Researchers have offered a number of recommendations for potential improvements both to the nature of CBT training given and study design for its evaluation. In terms of study design, the employment of more sensitive outcome measures and outcome assessments over longer time periods are recommended (Minnis 2001; Pithouse 2002). Studies are often small and increases in the number of participants (Barth 1994, Macdonald 2004) have been recommended alongside attention to grouping of participants according of severity/type of challenging behaviours (Pithouse 2002). CBT programmes themselves may need to be longer and more intensive, offering carers better opportunities to develop and consolidate their skills both within and between training sessions. They may also require more resources for more specialised programme content (Barth 1994; Macdonald 2004). All these factors may need to be considered in future studies given the overall strong support for cognitive-behavioural approaches in other areas (DOH 2001; Scott 2001).
Reviewers’ conclusions
Implications for practice
It is difficult to offer practice guidelines and recommendations based on results from these six studies. Training interventions evaluated to date appear to have very little effect on outcomes relating to looked-after children, assessed in relation to psychological functioning, extent of behavioural problems and interpersonal functioning. Results relating to foster carer(s) outcomes also show no evidence of effectiveness in measures of behavioural management skills, attitudes and psychological functioning.
Implications for research
This review has highlighted the need for further research in this area incorporating the key factors identified above. As regards study design, more sensitive outcome measures and outcome assessments over longer time periods are recommended; studies should be adequately powered to detect effects; attention should be paid to grouping participants according of severity/type of challenging behaviours. CBT programmes themselves may need to be longer and more intensive, offering carers better opportunities to develop and consolidate their skills both within and between training sessions and /or require more resources for more specialised programme content.
In accordance with Cochrane guidelines, we plan to update this review within 24 months to incorporate any new studies or respond to cogent criticisms or both.
Characteristics of included studies
Characteristics of excluded studies
Footnotes
Table of comparisons
01 Foster carer training vs. Control Group (Child psychological functioning)
01 Child Behaviour Checklist (CBCL- Externalising Scores)
02 Child Behaviour Checklist (CBCL- Internalising Scores)
03 Child Behaviour Checklist (CBCL-Total Scores)
04 Self-esteem (children)
05 Strengths & Difficulties Questionnaire (SDQ- completed by foster carers)
06 Strengths & Difficulties Questionnaire (SDQ-completed by teachers)
07 Strengths & Difficulties Questionnaire (SDQ - completed by foster children)
08 Strengths & Difficulties Questionnaire (ProSocial Scores)
09 Strengths & Difficulties Questionnaire (Total Scores)
10 Reactive Attachment Disorder Scale (RAD-post training)
11 Reactive Attachment Disorder (RAD- 9 months post-training)
02 Foster carer training vs. Control Group (Child behaviour)
01 Eyberg Child Behavior Inventory (ECBI-Problem)
02 Eyberg Child Behaviour Inventory (ECBI-Intensity)
03 Foster carer training vs. Control Group (Foster carer skills)
01 Knowledge of Behavioral Principles (KBPAC)
02 Use of behavioural strategies
03 Parenting Scale (PS)
04 Foster carer training vs. Control Group (Foster carers’ psychological functioning)
01 General Health Questionnaire (GHQ)
05 Foster carer training vs. Control Group (Foster family functioning)
06 Foster carer Training vs. Control Group (Fostering agency outcomes)
01 Placement Stability (assessed post-training)
02 Placement stability (assessed 6 months after training)
07 Meta-analysis - looked-after child psychological functioning
01 Training participants assessment (post training)
02 Training participants assessment (6-9 month post training)
09 Meta-analysis - foster carers skills
01 Various measures of skills & knowledge