Sage Journals: Discover world-class research

Abstract

Background

Parent–Child Interaction Therapy (PCIT) is a parent training intervention for childhood conduct problems, distinctive in its use of live clinician coaching of the parent–child dyad via a one-way mirror and discrete earpiece. However, despite a compelling evidence base, uptake of evidence-based parent training programmes such as PCIT by clinicians in routine care settings remains poor. This systematic review aimed to identify and synthesise implementation interventions that have sought to increase clinician adoption of PCIT in usual care settings.

Methods

We searched MEDLINE (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (EBSCO), Science Citation Index and Social Sciences Citation Index, and Web of Science Core Collection from inception to October 2020. Articles were included if they tested (by way of randomised controlled trials, controlled clinical trials, interrupted time series and controlled before and after trials) implementation interventions across any and all of the patient, clinician, clinic, system or policy domains. Two independent reviewers screened and selected studies, assessed risk of bias and extracted data – summarising implementation intervention components according to items from the Template for Intervention Description and Replication (TIDieR) checklist ( Hoffmann et al., 2014).

Results

Of the 769 articles identified once duplicates were removed, 13 papers relating to three studies met the inclusion criteria – all were quantitative or mixed-methods examinations of the effectiveness of different PCIT clinician training or training-related consultation methods. A narrative description of interventions was provided, as quantitative synthesis was not possible.

Conclusions

Research attention has to date been focussed on the establishment of an evidence-base for PCIT's effectiveness, with relatively little attention to the dissemination, implementation and sustainment of this treatment. Those studies that do exist have focused on training methods and training-related expert consultation. Research attention could usefully turn to both adoption and sustainment of this effective treatment in usual care settings.

Plain language summary

In this review, we aimed to summarise what is already known about how to implement PCIT in community settings after clinicians have received training in the approach. While research relating to the implementation of other parent training programmes is interesting and informative, implementation efforts are most effective when tailored to a specific programme in a specific context. As such, it was important to review published studies relating to PCIT specifically. We identified three relevant studies, one of which is yet to publish its main implementation findings. The three studies have focused on how best to train clinicians in PCIT, including how best to provide post-training support from expert trainers. We concluded that a fruitful line for future research would be to focus on the post-training period, particularly how best to support clinicians to adopt and sustain PCIT in their practice.

Systematic review registration

The study was prospectively registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 01/10/2020 (CRD42020207118).

Keywords

Parent–Child Interaction Therapy parent training behavioural parent training parent management training implementation adoption determinants systematic review

Background

A degree of defiance, non-compliance and aggression is common in young children, particularly at certain stages of development. However, for some children, these problems persist and begin to detrimentally impact child and family wellbeing. Such conduct problems represent one of the most common reasons for children to be referred to mental health services (Coghill, 2013). The social and economic consequences of untreated childhood conduct problems are significant, and it has been suggested that ‘… there is no other commonly occurring childhood condition that has such far-reaching and pervasive consequences for later health, development and social adjustment’ (Fergusson et al., 2011, p. 60). Longitudinal studies indicate that left untreated, children with conduct problems may develop into adults who require intensive support from services across criminal justice, social welfare and health domains (Rivenbark et al., 2017). As such, ensuring the effectiveness of community-based treatment is essential (Coghill, 2013; Furlong et al., 2013).

Parent training interventions based on social learning theory and operant learning principles (also known as parent management training or behavioural parent training; hereafter referred to as ‘parent training’) are well established as an effective treatment for childhood conduct problems, being amongst the most comprehensively reviewed treatments for childhood psychiatric conditions worldwide (Leijten et al., 2019). These interventions typically include both a parent–child relationship enhancement component and behaviour management skills training for parents (Leijten et al., 2018). They have been shown to be effective in reducing child conduct problems (Furlong et al., 2013; Ward et al., 2020). Through improving parental mental health and psychosocial wellbeing, parent training interventions influence the nature and quality of the parent–child relationship, in turn contributing to enhanced child wellbeing (Barlow et al., 2014; Furlong et al., 2013). Parent training programmes may also reduce the incidence of child physical abuse by providing alternative, non-contact methods of discipline, reducing harsh or negative parenting practices and increasing positive parenting skills (Furlong et al., 2013; Knerr et al., 2013).

Parent–Child Interaction Therapy (PCIT; Eyberg and Funderburk (2011)) is a well-established example of a parent training intervention (Lieneman et al., 2019; Thomas et al., 2017; Ward et al., 2016). PCIT is somewhat distinctive from many parent training approaches in its use of live coaching of parents with their child by trained therapists, typically by way of a one-way mirror and coaching headset. Originally developed by Sheila Eyberg in the USA in the 1980s, PCIT has been disseminated internationally to countries including Germany, Hong Kong, Australia, New Zealand, the Netherlands and Japan. Accreditation in PCIT requires Master's level licensed clinicians to attend a five-day intensive training, followed by fortnightly supervision until the successful completion of two PCIT cases (PCIT International; www.pcit.org).

However, PCIT faces a similar challenge to other evidence-based parent training interventions, in that, despite demonstrated effectiveness, and considerable international dissemination efforts, uptake in usual care settings remains poor (Forgatch et al., 2013; Lyon et al., 2020; Michelson et al., 2013; Williams & Beidas, 2019; Woodfield et al., 2020). Instead, children and families may be offered clinician-preferred interventions, and/or lower-intensity delivery of evidence-based protocols, both of which are less likely to represent an efficacious treatment (Williams & Beidas, 2019). To address this, research into how to integrate parent training effectively and sustainably into clinical care settings is critical, particularly given suggestions that implementation research within mental health services has lagged behind other disciplines (Powell et al., 2014).

Implementation difficulties may occur at a patient, clinician, clinic, system or policy level or indeed, often across multiple levels (Bauer et al., 2015). Yet despite this complexity, human behaviour remains at the core of successful implementation (Presseau et al., 2019). A key requirement of successful implementation is ‘for someone (usually more than one person or group), somewhere (from organisational leadership through to those providing direct patient care) doing something (usually more than one thing) differently … taking up new evidence requires healthcare providers and other health system stakeholders to change their behaviour’ (Presseau et al., 2019, p. 2, emphasis original).

Implementation strategies are single-component or, more typically, multifaceted co-ordinated attempts to address identified determinants of practice, with the ultimate aim of bringing about behaviour change (Michie et al., 2011). Many implementation strategies working in harmony are often required to implement a treatment successfully (Rudd et al., 2020). Of note, the terms implementation strategy and implementation intervention have a variety of meanings in different contexts but are used interchangeably in this paper.

Within the field of implementation science, there are calls for better specification and operationalisation of implementation strategies, and the field is also being challenged to better understand the intermediary in this relationship, that is, the mechanism by which a particular strategy may act on an implementation outcome (Lewis et al., 2020; Lewis et al., 2018; Powell et al., 2019). Hypothesised mechanisms of action are often not specified, let alone tested (Powell et al., 2019). Those working in the field are being encouraged to attempt to understand ‘the processes and mechanisms by which (implementation) strategies exert their effects rather than simply establishing whether or not they were effective’ (Powell et al., 2019), as connecting implementation strategies to hypothesised mechanisms of action can enhance understanding of why, and for whom, such strategies are effective (Lewis et al., 2020; Powell et al., 2019). By way of example, in Table 1, we have listed the Cochrane EPOC group's taxonomy of implementation strategies targeted at clinicians and have extended this with examples from the literature of the mechanisms thought to bring about behaviour change.

Table 1.

Examples and definitions of implementation strategies, and hypothesised mechanisms for action.

Example of behaviour change strategy targeting clinician behaviour (Cochrane Collaboration Effective Practice & Organisation of Care (EPOC) Group, 2015)	Definition (adapted from (Cochrane Collaboration Effective Practice & Organisation of Care (EPOC) Group, 2015)	Hypothesised or possible mechanism(s)
Audit and feedback	A summary of clinical performance over a period of time, may include recommendations for clinical action, provided via written, verbal or electronic means.	Alters clinician awareness of their current practice and clinical consequences of this; alters clinician's perceived social norms; impacts self-efficacy; draws clinician attention to a particular group of tasks and activities (Ivers et al., 2012).
Clinical incident reporting	A system for reporting critical incidents.	Organisational and individual learning from the accumulated data, used to reduce actual harm and potential harm (Stavropoulou et al., 2015).
Monitoring of the performance of the delivery of healthcare	Monitoring of service provision by healthcare organisations or individuals, e.g. comparing with an external standard.	Improves knowledge about performance, processes and results (Berwick et al., 2003).
Communities of practice	Groups or collectives of professionals with a common interest who interact regularly to deepen their knowledge or understanding in a particular area.	Knowledge generation and information sharing, collective problem-solving, reducing professional isolation, development of practice aids (Ranmuthugala et al., 2011).
Continuous quality improvement	An iterative process to evaluate, refine and improve healthcare that may include analysis of a system or process, use of data analysis to evaluate changes, and/or a structured problem-solving approach.	Shared valuing of clinical data for practice improvement, collective change efficacy and/or the development of a population health orientation (Schierhout et al., 2013).
Educational Games Materials Meetings Outreach visits or academic detailing Inter-professional education	Continuing education for clinicians, that may involve seminars, workshops, online learning, personal outreach visits, feedback on performance, educational materials to support clinical care, or interactive learning with other healthcare clinicians.	Improved awareness of best practice and/or new developments (O'Brien et al., 2007). Educational games may allow for ‘safe’ experimentation with problem-solving and decision-making in a controlled learning environment, stimulating use of analytical skills and reflective practice (Akl et al., 2013).
Clinical Practice Guidelines	Systematically and rigorously developed (often printed) resources to aid healthcare clinician decision-making around optimal healthcare for specific clinical circumstances.	Passive dissemination of scientific information to improve practice via increased awareness, knowledge, skills and changes in attitudes (Giguère et al., 2012).
Local consensus processes and/or local opinion leaders	Promotion of good clinical practice through the use of identified opinion leaders in the local context, or a consensus agreement on a local process.	Opinion leaders (cf. ‘champions’, who are selected by management) are respected by peers, and exert social influence to promote improved practice (Flodgren et al., 2019).
Managerial supervision	Routine supervision by healthcare staff in a management role.	Mechanism likely to depend on context – for example in low- and middle-income countries, may serve to reduce professional isolation and provide support to clinicians, or line manage logistics and finances around care provision (Bosch-Capblanch et al., 2011).
Patient-mediated interventions, including routine patient-reported outcome measures	Interventions which use information provided by/to patients and/or interactions with patients to change clinician performance.	Patient feedback may broaden information available to the clinician and prompt improvement. Informed patients may be more actively involved in their care, alongside increased self-efficacy and self-care (refer Table 1 in Fønhus et al., 2018 for a full description of possible mechanisms and positive and adverse effects).
Public release of performance data	Written or electronic performance data, released to inform the public about healthcare clinicians.	Informed patients may select better performing clinicians, perhaps providing an indirect stimulus or incentive for quality improvement (Metcalfe et al., 2018).
Reminders	Interventions (manual or computerised) which involve prompting clinicians to perform a particular behaviour, for example, during a consultation.	Serve as prompts for clinicians to recall known information, and/or present information in a more comprehensible format (Pantoja et al., 2019).
Tailored interventions	Interventions to change practice which are based on an assessment of barriers to change.	The implementation strategy is deliberately chosen to address specific pre-identified barriers or obstacles in a particular context or setting (Baker et al., 2015).
Financial incentives (Not an EPOC example, but often included – for example, Dombrowski et al. (2016))	Interventions which involve providing a monetary reward to clinicians for a particular time period; service, episode or visit; for providing care for a particular population group; a change in activity or quality of care – with the aim of increasing the quality and efficiency of care (Flodgren et al., 2011).	Extrinsic source of motivation, conditional on performing a particular behaviour or set of behaviours, though may inadvertently reduce intrinsic motivation (Flodgren et al., 2011).

Existing reviews

Despite a recent upswing in interest in implementation research in child psychology and psychiatry, there are very few existing reviews of implementation strategies to increase therapist use of parent training approaches in usual care settings. Forgatch et al. (2013)'s review related solely to the implementation of the Parent Management Training-Oregon Model (PMTO) programme and illustrated the process of implementing PMTO in three locations: Norway, Michigan and Kansas. The review was narrative rather than systematic and aimed to highlight important influences on successful implementation, along with challenges. The authors wrote of the importance of ‘combin[ing] the puzzle pieces, captur[ing] the key components, and build[ing] a complete picture’ (p. 692) – a suggestion to move the parent training implementation field forward through collaboration and integration. As applied to the review context, this suggests a need to rigorously and systematically synthesise individual implementation studies. However, this may need to occur through synthesising the implementation literature relating to a specific programme, as implementation strategies are ideally tailored to specific treatments and contexts (Baker et al., 2015). A rigorous and systematic review of the literature relating to an individual parent training intervention sets the scene for the field-wide collaboration that Forgatch et al. (2013) promote.

Also, a new look at the evidence is timely. Baumann et al.'s (2015) review of four evidence-based manualised programmes: Parent–Child Interaction Therapy, The Incredible Years, Parent-Management Training-Oregon Model (PMTO) and the Positive Parenting Programme (Triple P) included articles published up to January 2013. Their review assessed the extent to which implementation strategies and implementation outcomes have been evaluated when integrating these parent training interventions into usual care. At that time, the authors suggested that parent training research had been disproportionately weighted towards effectiveness studies but was ‘poised’ to undertake more research into implementation strategies (Baumann et al., 2015). Given that implementation, science is a new and rapidly evolving field, and the importance of tailoring interventions to specific treatments, an up-to-date review of implementation strategies specific to PCIT is particularly important.

The aim of this systematic review was to synthesise the existing literature on implementation strategies designed to increase clinician use of PCIT in the treatment of childhood conduct problems. Specifically, we aimed to:

Describe the nature of implementation strategies used and their constituent components according to items from the Template for Intervention Description and Replication (TIDieR) checklist (Hoffmann et al., 2014).

Examine the effectiveness of existing implementation strategies to support the use of PCIT in the treatment of childhood conduct problems.

Methods

Protocol and registration

The protocol for this systematic review was prospectively registered on PROSPERO (01/10/2020; CRD42020207118) and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Page et al., 2021) reporting guidelines.

Eligibility criteria

In keeping with the Cochrane Effective Practice and Organisation of Care Group recommendations (Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) Group, 2017b) we included randomised controlled trials, controlled clinical trials, interrupted time series and controlled before and after trials. We included English language studies published in a peer-reviewed journal, peer-reviewed conference proceedings, and research dissertations. Opinion papers, case series, case reports/single case studies, and cross-sectional studies were excluded.

We were interested in ‘real world’ delivery of PCIT, and as such, we included implementation studies that related to clinicians delivering PCIT to children and their parent(s) or caregiver(s) in usual care settings, that is, university laboratories were excluded. No restriction was placed on the modality of PCIT delivery (i.e. whether it was delivered in a group, online or individually), or the nature of the client group (PCIT with foster or adoptive parents, and other caregivers were included; children could have co-existing difficulties alongside conduct problems). Studies of any intervention described as PCIT, regardless of the age of the child, were included.

Trials could involve health professionals from any professional discipline. Implementation interventions across any and all of the patient, clinician, clinic, system or policy domains were included.

Outcomes

Proctor et al. (2011) suggested a taxonomy of implementation outcomes including adoption, feasibility, appropriateness, fidelity, penetration, acceptability, sustainability, and costs of the implementation intervention. Our primary outcome of interest was clinician adoption of PCIT 6 months after the implementation intervention. The time period of 6 months was chosen, as we wanted to see a sustained response from any intervention, while maintaining a pragmatic lens, as we are aware that many trials do not measure outcomes beyond this point. Where studies had multiple time points that were 6 months or beyond, we were interested in the time point that was closest to 6 months. We understood adoption to refer to the ‘intention, initial decision or action to try or employ an innovation or evidence-based practice’ (Proctor et al., 2011, p. 69). Adoption is variously known as uptake, utilisation, initial implementation and intention to try (Proctor et al., 2011). Adoption data is drawn from a variety of sources, which may include administrative data (e.g. a mental health service database record), observation (e.g. of practitioner behaviour), survey or semi-structured interviews with stakeholders (Proctor et al., 2011). Attempts to ascertain the validity and reliability of psychometric measures of adoption have been undertaken (e.g. Lewis et al., 2015), though as yet, there is not one inventory or instrument that is widely used or understood to represent best practice in the assessment of adoption. Indeed, practitioner-report surveys of adoption of evidence-based practices are typically study-specific (Beidas & Kendall, 2014). Surveys are also typically dichotomous (i.e. the practitioner did or did not adopt the particular practice). It has been suggested that this approach is overly simplistic, and that the quality of adoption is as important as the quantity of adoption (Beidas & Kendall, 2014). While acknowledging these concerns, our primary outcome will be a participant (at any level – practitioner, clinic or system) self-reported dichotomous adoption of PCIT. Other measures of adoption will also be reported.

Secondary outcomes included acceptability, appropriateness and feasibility, as these are thought to be particularly influential on clinician adoption or uptake (Lewis et al., 2015). Acceptability to service users is an important consideration but for the purposes of this review, we are particularly interested in acceptability to clinicians. We understood acceptability to refer to the palatability of PCIT to clinicians, or the extent to which it is satisfactory or acceptable (Proctor et al., 2011). Acceptability is often assessed by semi-structured interviews or questionnaires, and, importantly, is considered to be dynamic, with the potential to change according to a clinician's experience (Proctor et al., 2011). Appropriateness, also often measured by rating scales or interviews, is conceptually similar, though related more to clinicians’ perception that PCIT is relevant or a ‘good fit’ for their context (Proctor et al., 2011). Finally, feasibility may be defined as the extent to which PCIT can be fruitfully utilised within a particular agency or service (Proctor et al., 2011).

We were interested in identifying and recording any implementation theories, models or frameworks described by study authors, and any mechanisms by which the implementation intervention was hypothesised to act. Material costs associated with implementation interventions were also included in our review, along with any reported adverse or undesirable effects of implementation strategies.

Information sources and searching methods

We searched Cochrane Central Register of Controlled Trials (CENTRAL; latest issue), MEDLINE Ovid, PsycInfo Ovid, Embase Ovid, CINAHL (Cumulative Index to Nursing and Allied Health Literature) EBSCO, Science Citation Index and Social Sciences Citation Index, and Web of Science Core Collection from the date of database inception to 10th October 2020. We also screened ClinicalTrials.gov, the World Health Organisation's International Clinical Trials Registry Platform (ICTRP) and ISRCTN, and the Australian and New Zealand Clinical Trials Registry. A specialist librarian was consulted in the process of determining specific search strings for each database, and these are available from the first author. They typically involved the phrase ‘parent child interact* therap*.mp’ or similar (i.e. adapted for the specific database context). We made the decision to utilise a broader search, and manually search results for relevant studies, as the number of results was manageable. We manually reviewed the reference lists of included studies and relevant reviews and carried out cited reference searches for all included studies in ISI web of knowledge. We also searched the Cochrane Database of Systematic Reviews and Google Scholar for related systematic reviews to identify additional studies and wrote to experts in the field to ask for relevant studies.

Selection of studies and data extraction

All titles and abstracts identified by electronic searching were downloaded to Endnote X9 and duplicates removed. Two review team members (MW and HK (see Acknowledgements)) each screened all retrieved titles and abstracts independently of each other. Where studies appeared to meet inclusion criteria, or where there was any uncertainty, full-text study reports were obtained. Consensus was achieved through discussion between MW and HK, along with SM and SH. Full-text articles were reviewed by MW, SM and SH.

Data collection and analysis

Data extraction and quality assessment

Extraction criteria were developed by the research team based on the key aims of the research. We used a modified version of the Cochrane EPOC Group data collection form (Cochrane Collaboration’s Effective Practice and Organisation of Care (EPOC) Group, 2017a), adapted to reflect the characteristics of our study and outcomes of interest. For example, we extracted data relating to whether studies described any identified barriers to be addressed by the intervention(s), and whether any theories, models or frameworks were cited as a basis for implementation intervention development. We piloted this form on one study, prior to formal use. Two review authors (MW and SM) then, independently of each other, extracted study characteristics and outcomes according to those specified in the protocol, from the included studies and entered the data into Microsoft Excel. Disagreements were resolved by discussion and consensus.

Assessment of risk of bias in individual included studies

Two review authors (MW and SH) independently assessed risk of bias for each study, using the criteria outlined in Section 8.5 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2020), which covers random sequence generation, allocation concealment, blinding of participants and personnel and outcome assessment, incomplete outcome data (e.g. dropouts and withdrawals) and selective outcome reporting. For non-randomised studies, we used the ROBINS-I (‘Risk Of Bias In Non-randomised Studies – of Interventions’) (Sterne et al., 2016), which is a tool for assessing risk of bias relating to studies that did not use randomisation to allocate to groups. Disagreements were resolved by discussion and consensus.

Data synthesis

It had been intended that if there were sufficient data, and included studies were sufficiently comparable, we would have undertaken a meta-analysis using the random effects model with 95% confidence intervals. Details of the planned parameters are available in the protocol or from the first author. Unfortunately, meta-analysis was not possible, and as outlined in the protocol, we have consequently provided a narrative synthesis of outcomes.

Results

Article selection

The search was run on 10 October 2020. The article selection and screening process is outlined in the PRISMA flow diagram (Figure 1). The search yielded 2150 results and after duplicates were removed, 769 abstracts were manually and independently reviewed by MW and HK. Full texts of 17 papers were retrieved, and reviewed by MW, SH and SM. A total of 13 papers relating to three studies were included (Table 2).

Figure 1.

PRISMA flow diagram.

Table 2.

Summary of included studies.

	Protocol	Primary publication(s)	Secondary publications
1		Herschell (2003) Evaluation of techniques for disseminating PCIT (PhD dissertation)	Herschell, A. D., McNeil, C. B., Urquiza, A. J., McGrath, J. M., Zebell, N. M., Timmer, S. G., & Porter, A. (2009). Evaluation of a treatment manual and workshops for disseminating, parent–child interaction therapy. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 63-81.
2	Funderburk et al. (2011) Trial protocol (NCT01294488) Using technology to augment the implementation and effectiveness of PCIT	Funderburk, B., Chaffin, M., Bard, E., Shanley, J., Bard, D., & Berliner, L. (2015). Comparing Client Outcomes for Two Evidence-Based Treatment Consultation Strategies. Journal of Clinical Child & Adolescent Psychology, 44(5), 730-741.	Nelson, M. M., Shanley, J. R., Funderburk, B. W., & Bard, E. (2012). Therapists’ attitudes toward evidence-based practices and implementation of parent–child interaction therapy. Child Maltreatment, 17(1), 47–55.
3	State wide trial to compare three training models for implementing an evidence-based treatment (NCT02543359; Herschell; ClinicalTrials.gov) Herschell, A. D., Kolko, D. J., Scudder, A. T., Taber-Thomas, S., Schaffner, K. F., Hiegel, S. A., … Mrozowski, S. (2015). Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment. Implementation Science, 10(1), 1–16.		Scudder, A. T., Taber-Thomas, S. M., Schaffner, K., Pemberton, J. R., Hunter, L., & Herschell, A. D. (2017). A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives. Health research policy and systems, 15(1), 1–13. Jackson, C. B., Herschell, A. D., Schaffner, K. F., Turiano, N. A., & McNeil, C. B. (2017). Training community-based clinicians in parent–child interaction therapy: The interaction between expert consultation and caseload. Professional Psychology: Research and Practice, 48(6), 481. Jackson, C. B., Macphee, F. L., Hunter L. J., Herschell, A. D., & Carter, M. J. (2017). Lessons learned: Enrolling family participants in a statewide implementation trial of an evidence-based treatment. Progress in Community Health Partnerships: Research, Education, and Action 11(3), 233–241; Brabson, L. A., Herschell, A. D., Kolko, D. J., & Mrozowski, S. J. (2019). Associations among job role, training type, and staff turnover in a large-scale implementation initiative. The Journal of Behavioral Health Services & Research, 46(3), 399–414. Jackson, C. B. (2020). Making Implementation Last: Understanding the Sustainability of an Evidence-based Treatment. Dissertation. Brabson, L. A., Herschell, A. D., Snider, M. D., Jackson, C. B., Schaffner, K. F., Scudder, A. T., … Mrozowski, S. J. (2020). Understanding the Effectiveness of the Cascading Model to Implement Parent-Child Interaction Therapy. The Journal of Behavioral Health Services & Research, 1–19.

Article characteristics

Type and quality of articles

Articles were published between 2003 and 2020 and described studies that ran between 2000 and 2017. Two of the three included studies were led by the same author (Herschell et al., 2015; Herschell et al., 2009). Herschell et al. (2009) described a parallel-group (two-arm), controlled and non-randomised trial. The authors described randomisation as being pragmatically challenging, with descriptions of random assignment to groups not being possible ‘due to contractual agreements and community relationships’ (Herschell et al., 2009, p. 65). Funderburk et al. (2015) described a stepped wedge design, with random assignment by the clinic to a position in the implementation sequence. Herschell et al. (2015) described a parallel-group (three-arm) randomised controlled trial.

The control condition in all included studies involved an active comparator of some kind, typically training-as-usual. Even where distance education/online training acted as the control (Herschell et al., 2015), participants in this condition received phone consultations with a trainer, consistent with the other conditions.

The overall risk of bias for Herschell et al. (2015) was not fully assessed, as the primary outcomes had not been reported at the time of writing. However, based on the protocol, there appears to have been adequate sequence generation, that is, risk of bias in this domain is likely to be low (SAS was used ‘to write a routine for the randomization’, p. 5); allocation concealment was unclear; and blinding of participants was not possible, so risk of bias in this domain may be high. As this is a cluster randomised controlled trial with randomisation at the county level, analysis ought to be at the cluster level (rather than individual) and measures at this level are not described in the protocol. Risk of bias was assessed as high for Funderburk et al. (2015), partly as a result of unclear reporting. As Herschell et al. (2009) was not randomised, the ROBINS-I tool was employed (Sterne et al., 2016) and risk of bias was assessed as moderate. Refer to Supplemental Material for supporting material for these conclusions.

Types of settings and participants

All studies were carried out in the USA: specifically, in licensed psychiatric outpatient clinics in Pennsylvania (Herschell et al., 2015), Washington and Oklahoma-based state-run community mental health centres, tribal social services, community-based or church-affiliated not-for-profit services, and for-profit providers (Funderburk et al., 2015) and California-based agencies specialising in the treatment of children with histories of maltreatment (Herschell et al., 2009).

In order to participate in the research, agencies where participating clinicians were based were required to be in a position to fund site preparation costs (Herschell et al., 2015); have existing funding for PCIT implementation, a commitment to providing all resources required for a PCIT programme including an adequate referral stream (Funderburk et al., 2015) or have expressed willingness to provide PCIT through a requirement to apply to participate (Herschell et al., 2009).

Participating clinicians’ qualifications were typically consistent with PCIT International's requirement for PCIT trainees, namely licensed mental health clinicians with a Master's degree or higher.

Types of implementation interventions

Implementation interventions all concentrated on training and training-related consultation methods. A detailed account of the components of the three studies’ implementation interventions is provided in the Supplemental Material, described according to the Template for Intervention Description and Replication (TIDieR) checklist (Hoffmann et al., 2014).

In summary, Herschell et al. (2009) had two conditions, didactic and experiential. On the first day of the PCIT training workshop, all participants received the same training experience: information was presented didactically, through video and live modelling, with no opportunity to practice skills. On the second day, both groups received the same content, but the experiential group ‘participated in role-plays, individually practiced coding videotapes, and received frequent, individualized feedback on their performance’ and the didactic group ‘reviewed client session videotapes, discussed PCIT skills, and coded videotapes as a group’ – with no practice or feedback.

Funderburk et al. (2015) involved two post-training consultation conditions, telephone consultation and live video consultation. Telephone consultation included an average of six therapists per call and was led remotely by a PCIT expert. Calls involved collecting data from therapists on client outcomes, then discussing individual cases ‘with an emphasis on reinforcing proper PCIT technique, offering examples for case-specific tailoring, tracking progress, and setting the agenda’ (p. 734). In the live video consultation condition, the PCIT consultant joined client sessions in real time via a remote link. This allowed for direct observation, listening to therapist coaching, communication back and forth with the therapist, and (very rarely) the PCIT consultant taking over coaching of a client in order to model skills to the therapist.

Herschell et al. (2015) involved three training conditions: a cascading model (also known as ‘train the trainer’), a learning collaborative, and distance education. In the cascading model condition, participants undertook a 40-h initial face-to-face training, an advanced 16-h training 6 months later, and fortnightly contact with a trainer for 12 months (this represents a typical or standard PCIT training). In the learning collaborative condition, which involved four representatives from each clinic (a ‘core team’ of an administrator, supervisor and two clinicians), participants undertook a three-phase process: (1) A pre-work phase including readings, material review, conference calls (2) learning sessions – three 2-day meetings over a nine-month period and (3) Action periods. In the distance education condition, participants completed an online training, which took approximately 10 h to complete and received phone consultations with a trainer, consistent with other conditions.

Descriptions of mechanisms of action of implementation strategies

Implementation theories, models or frameworks that formed the foundation of the implementation intervention were relatively poorly reported across the three studies. Hypothesised mechanisms of action of the implementation strategies incorporated in the intervention were also often not stated. Funderburk et al. (2015) identified specific barriers to be addressed, namely post-training drift/poor fidelity; and poor transferability of traditional training models from university to community settings, but did not specify an implementation model, or hypothesized mechanism of action. Herschell et al. (2015) referenced the Training Transfer Conceptual Model (Ford & Weissbein, 1997) and the draft model of implementation research (Proctor et al., 2009). The authors did not make reference to specific hypothesised mechanisms of action.

Implementation outcomes

Herschell et al. (2009) assessed clinician skill acquisition, knowledge gain, satisfaction with training and training integrity. They found that the training conditions they tested (i.e. didactic and experiential) were equally effective in increasing clinician knowledge and skills (as assessed at the end of the final day of training), and in terms of clinician satisfaction. Simply reading a treatment manual was found to be useful, but insufficient to result in sustained increases in clinician knowledge and skills (Herschell et al., 2009). The authors also explored clinician characteristics ‘potentially associated with successful [evidence-based treatment] adoption’ at the end of the second day of training (the final assessment point) but cautioned that their modest sample size undermined their findings in this area.

The Funderburk (2011) study protocol suggested that they intended to examine ‘practitioner fidelity and competency in implementing PCIT [and] practitioner acceptance and satisfaction with PCIT’ alongside client/family outcomes. Funderburk et al. (2015) primarily reported client outcomes rather than implementation outcomes and found that child behaviour problems fell ‘from well above to below clinical cut off values’ (p. 730), with greater improvements in the group where the therapist was supported by increased rates of the live video consultation. The dose of telephone consultation appeared to have had no effect on child outcomes. Of interest to our review, the authors discussed the material cost of the two interventions, noting that live video consults didn't incur the same opportunity cost as phone calls as therapists were seeing the client at the time, whereas phone consults required taking time away from clients. They calculated the hypothetical per-case cost differential at $200USD (p. 738).

Herschell et al. (2015)'s primary outcome paper was under review at the time of writing (A. Herschell, personal communication) and therefore unavailable. The results from this study will be particularly informative, given that both implementation outcomes (including outcomes of particular interest to our review – adoption, feasibility, acceptability and cost) and client outcomes are being assessed up to 12 months post-training.

Discussion

Overall, the studies identified in this systematic review of PCIT implementation interventions were modest in number (three). Strengths included the ‘real world’ context within which studies were based, and the detailed examination of characteristics associated with successful training and post-training consultation methods. Implementation outcomes have not yet been published from one large study (Herschell et al., 2015), were not directly reported in another (Funderburk et al., 2015) or were limited to those assessed at the end of the final day of training and not upon the clinicians’ return to their treatment delivery context (Herschell et al., 2009).

In recent years, there has been increasing recognition of the importance of describing mechanisms of action of implementation strategies. However, mechanisms tended not to feature prominently in the design of these studies, which in some cases were carried out some time ago. Robust training of clinicians in a particular treatment approach is often a necessary pre-condition for successful implementation in a clinical context and warrants research attention (Frank et al., 2020). Training serves an important role – it may increase clinician knowledge and satisfaction with an approach (Valenstein-Mah et al., 2020). However, it remains unclear whether and how training results in enhanced delivery of treatments by clinicians, and better outcomes for clients (Valenstein-Mah et al., 2020). Future research into training ought to seek to understand mechanisms by which training does (or does not) result in clinician behaviour change as it relates to adoption and sustainment, along with how other contextual factors (such as organisational context) inter-relate with training experiences to result in adoption (Frank et al., 2020). As an example of a description of a hypothesised mechanism, training and training-related consultation are thought to impact implementation behaviour by way of expert modelling/rehearsal of skills, problem-solving, improving engagement and motivation, therapist accountability, adaptation or tailoring to improve fit, planning for sustainability of the treatment (Motamedi et al., 2021). Herschell et al. (2009) justifiably note that several limitations in the design and execution of their study were a result of its ‘real world’ nature, ‘which is both a strength and weakness of the current study … (there is a) need to balance rigor with relevance in dissemination and implementation studies, which leads to necessary compromises at all levels’ (p. 77). Indeed, while researchers may be aware of the importance of testing hypothesised mechanisms of action, factors outside of researchers’ control, for example, contractual agreements and/or community relationships, can influence the design and execution of an implementation study (Herschell et al., 2009).

There was an emphasis in all included studies on training as an implementation strategy. Funderburk et al. (2015) importantly observed that post-training drift (i.e. poor fidelity to the treatment protocol) and poor transferability of traditional training models from university to community settings are challenges facing the field. But across the three studies, there does not appear to have been a systematic assessment of barriers to implementation of PCIT in the particular context (ideally in partnership with stakeholders, including clinicians), followed by a deliberate selection of implementation strategies to address the specific barriers in the setting. Considering clinician perspectives is particularly relevant to the delivery of evidence-based interventions (Motamedi et al., 2021). In a recent mixed-methods study, the majority of implementation strategies perceived by child therapists as essential for sustained delivery of interventions (connection to a community of trained therapists, ongoing consultation and supervision, availability of internal supervisors trained in the intervention) were indeed significantly associated with the use of evidence-based practices, while only one (access to intervention-related resources or materials) was not (Motamedi et al., 2021). It is apparent that comprehensive consultation is required with those charged with PCIT implementation (i.e. clinicians) to better understand barriers, and shape and prioritise implementation interventions to address these.

Also, while the included studies ostensibly explored training and training-related consultation as implementation strategies, their intervention often had other – potentially very influential – components. For example, inclusion criteria for participating agencies typically required that the agency had PCIT equipment in place and a commitment to a suitable referral stream (e.g. Funderburk et al., 2015), which meant some of the more common barriers which may be faced by PCIT clinicians downstream, such as having an appropriate client flow and installing the equipment needed, were pre-emptively addressed (Barnett et al., 2021; Woodfield et al., 2020). It may have been useful to consider this as an implementation strategy in and of itself. Though it is likely that the authors intentionally addressed logistical or practical barriers in order to examine the impact of training conditions alone and minimise confounding factors.

Within the field of implementation science, there is a need for increased consistency in relation to the terms used and their definitions, as overlapping and variable use of terms is common and problematic (Lewis et al., 2020). Mechanisms may be referred to as mediators, and determinants or barriers as outcomes – for example, poor fidelity was highlighted as a barrier to be addressed by the implementation intervention in studies included in this review and is also an implementation outcome (Proctor et al., 2011). We share the call by Lewis et al. (2020) for increased clarity in distinctions between determinants, strategies and outcomes. Finally, different implementation outcomes are relevant at different stages of an implementation process, with factors influencing the sustainment of an intervention likely to be different from those influencing its initial adoption (Shoesmith et al., 2021). While adoption is, by definition, necessary for sustainment, few would define a successful implementation as one which simply results in the initial adoption of an intervention without any sustained use. A decline in the use of evidence-based practices such as PCIT following implementation is often described, but not well understood (Birken et al., 2020). Yet apart from Herschell et al. (2015)'s study, the outcomes of which are yet to be published, there are no data on implementation outcomes after the PCIT training period. As such, while these are valiant and important studies, more work is required to better understand downstream (i.e. post-training) influences and processes.

Limitations

We limited our search to publications in English only and did not search the grey literature. It is possible that relevant publications may have been missed with these search parameters. The studies included in this review were relatively homogeneous, in that they were all conducted in the USA, and did not include specific consideration of the needs of minority groups such as Indigenous clinicians. The review resulted in a modest, and heterogeneous pool of included studies. This is a key limitation to any conclusions that can be drawn relating to PCIT implementation interventions. Also of note, one researcher (Dr Amy Herschell) was involved in the design and execution of two of the three included studies, which introduces the possibility that effects may be associated with her considerable expertise, rather than the implementation interventions themselves. This may also limit the generalisability of the findings of this review.

The process of conducting a systematic review requires one primary outcome to be identified, for the purposes of data analysis. We elected to use adoption as a primary outcome, and this decision may warrant further exploration. As stated earlier, it has been suggested that the quality of adoption is as important as the quantity of adoption (Beidas & Kendall, 2014). Also, implementation outcomes are known to be inter-related, with some outcomes even thought to be predictive of others. For example, acceptability, feasibility and appropriateness of the intervention being implemented may predict adoption by clinicians (Proctor et al., 2011).

Conclusion

In summary, while the methodology of studies is increasingly robust, and there has been an increase in implementation-related PCIT research in the past decade, it is clear that the focus remains on training and training-related consultation as implementation strategies, with increases in clinician knowledge and/or skills, and acceptability or satisfaction of the training itself as implementation outcomes. The stage is set for consideration of how best to intervene to sustain clinician use of PCIT in community settings post-training.

PCIT has a compelling evidence-base, and the potential to effectively treat childhood conduct problems – a relatively commonly occurring and potentially debilitating condition. However sustainably embedding evidence-based interventions such as PCIT in usual care environments requires thoughtful and deliberate effort, and rarely occurs naturally (Shelton et al., 2020). Existing PCIT implementation initiatives have focussed on training and training-related consultation strategies and the stage is set for implementation interventions that attempt to improve sustainment and even attempt ‘re-implementation’ where implementation has failed or fallen short in real-world settings. This process ought to involve pre-identified barriers identified in partnership with stakeholders, and set out to test specific mechanisms of action, to not only understand what works in implementation interventions, but why and how.

Supplemental Material

sj-docx-1-irp-10.1177_26334895221082330 - Supplemental material for Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions

Supplemental material, sj-docx-1-irp-10.1177_26334895221082330 for Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions by Melanie J Woodfield, Sally Merry and Sarah E Hetrick in Implementation Research and Practice

Supplemental Material

sj-docx-2-irp-10.1177_26334895221082330 - Supplemental material for Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions

Supplemental material, sj-docx-2-irp-10.1177_26334895221082330 for Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions by Melanie J Woodfield, Sally Merry and Sarah E Hetrick in Implementation Research and Practice

Footnotes

Acknowledgements

Anne Wilson, Research Services Adviser from the University of Auckland's Libraries and Learning Services, advised on the search strategy and methodology. Harshali Kumar, Research Assistant from the Department of Psychological Medicine at the University of Auckland, independently screened all titles and abstracts identified by electronic searching.

Contributions

MW designed the study, wrote the protocol, executed the search, reviewed articles for inclusion, drew conclusions and wrote the final paper. SM contributed to the design of the study, edited the protocol, reviewed included articles, participated in summarising outcomes, drawing conclusions and drafting and revising the final paper. SH contributed to the design of the study, edited the protocol, reviewed included articles, participated in summarising outcomes, drawing conclusions and drafting and revising the final paper.

Declaration of conflicting interests

MW is a within-agency trainer for PCIT, though receives no additional remuneration for this role.

Funding

The research is supported by a Clinical Research Training Fellowship for Dr Melanie Woodfield from the Health Research Council (HRC) of New Zealand. Professor Sally Merry holds the Cure Kids Duke Family Chair in Child and Adolescent Mental Health. Associate Professor Sarah Hetrick holds an Auckland Medical Research Foundation (AMRF) Douglas Goodfellow Repatriation Fellowship and is a Cure Kids Research Fellow. The HRC, AMRF and Cure Kids were not involved in protocol development, conducting the review, analysing and/or interpreting data or dissemination of findings.

ORCID iDs

Melanie J Woodfield

Sally Merry

Supplemental material

Supplemental material for this article is available online.

References

Akl

E. A.

Sackett

K. M.

Erdley

W. S.

Mustafa

R. A.

Fiander

Gabriel

Schünemann

. (2013). Educational games for health professionals. Cochrane Database of Systematic Reviews, (3). https://doi.org/10.1002/14651858.CD006411.pub3

Baker

Camosso–Stefinovic

Gillies

Shaw

E. J.

Cheater

Flottorp

Robertson

Wensing

Fiander

Eccles

M. P.

(2015). Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews, (4). https://doi.org/10.1002/14651858.CD005470.pub3

Barlow

Smailagic

Huband

Roloff

Bennett

. (2014). Group-based parent training programmes for improving parental psychosocial health. Cochrane Database of Systematic Reviews, (5). https://doi.org/10.1002/14651858.CD002020.pub4

Barnett

Brookman-Frazee

S. H.

Lind

Lui

Timmer

Boys

Urquiza

Innes-Gomberg

Quick-Abdullah

Lau

A. S

. (2021). Train-to-sustain: Predictors of sustainment in a large-scale implementation of parent–child interaction therapy. Evidence-based Practice in Child and Adolescent Mental Health, 6(2), 262–276. https://doi.org/10.1080/23794925.2020.1855613

Bauer

M. S.

Damschroder

Hagedorn

Smith

Kilbourne

A. M

. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3, 32. https://doi.org/10.1186/s40359-015-0089-9

Baumann

A. A.

Powell

B. J.

Kohl

P. L.

Tabak

R. G.

Penalba

Proctor

E. K.

Domenech-Rodriguez

M. M.

Cabassa

L. J

. (2015). Cultural adaptation and implementation of evidence-based parent-training: A systematic review and critique of guiding evidence [article]. Children and Youth Services Review, 53, 113–120. https://doi.org/10.1016/j.childyouth.2015.03.025

Beidas

R. S.

Kendall

P. C.

(2014). Dissemination and implementation of evidence-based practices in child and adolescent mental health. Oxford University Press.

Berwick

D. M.

James

Coye

M. J

. (2003). Connections between quality measurement and improvement. Medical Care, 41(1), I30–I38. https://www.jstor.org/stable/3767726

Birken

S. A.

Haines

E. R.

Hwang

Chambers

D. A.

Bunger

A. C.

Nilsen

. (2020). Advancing understanding and identifying strategies for sustaining evidence-based practices: A review of reviews. Implementation Science, 15(1), 1–13. https://doi.org/10.1186/s13012-020-01040-9

10.

Bosch–Capblanch

Liaqat

Garner

(2011). Managerial supervision to improve primary health care in low- and middle-income countries. Cochrane Database of Systematic Reviews (9). https://doi.org/10.1002/14651858.CD006413.pub2

11.

Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) Group. (2015). EPOC taxonomy .

12.

Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) Group. (2017a). Data collection form .

13.

Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) Group. (2017b). EPOC Resources for review authors .

14.

Coghill

. (2013). Editorial: Do clinical services need to take conduct disorder more seriously? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(9), 921–923. https://doi.org/10.1111/jcpp.12135

15.

Dombrowski

S. U.

Campbell

Frost

Pollock

McLellan

MacGillivray

Gavine

Maxwell

O’Carroll

Cheyne

Presseau

Williams

. (2016). Interventions for sustained healthcare professional behaviour change: A protocol for an overview of reviews. Systematic Reviews, 5(1), 173. https://doi.org/10.1186/s13643-016-0355-9

16.

Eyberg

S. M.

Funderburk

(2011). Parent-child interaction therapy protocol. Florida: PCIT International, Inc.

17.

Fergusson, D., Boden, J., & Hayne, H. (2011). Childhood conduct problems. Improving the Transition, 59.

18.

Flodgren

Eccles

M. P.

Shepperd

Scott

Parmelli

Beyer

F. R

. (2011). An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database of Systematic Reviews, (7). https://doi.org/10.1002/14651858.CD009255

19.

Flodgren

O'Brien

M. A.

Parmelli

Grimshaw

J. M

. (2019). Local opinion leaders: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (6). https://doi.org/10.1002/14651858.CD000125.pub5

20.

Fønhus

M. S.

Dalsbø

T. K.

Johansen

Fretheim

Skirbekk

Flottorp

S. A

. (2018). Patient-mediated interventions to improve professional practice. Cochrane Database of Systematic Reviews, (9). https://doi.org/10.1002/14651858.CD012472.pub2

21.

Ford

J. K.

Weissbein

D. A

. (1997). Transfer of training: An updated review and analysis. Performance improvement quarterly, 10(2), 22–41. https://doi.org/10.1111/j.1937-8327.1997.tb00047.x

22.

Forgatch

M. S.

Patterson

G. R.

Gewirtz

A. H

. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science: a Journal of the Association for Psychological Science, 8(6), 682–694. https://doi.org/10.1177/1745691613503478

23.

Frank

H. E.

Becker-Haimes

E. M.

Kendall

P. C.

(2020). Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice, n/a(n/a), e12330. https://doi.org/10.1111/cpsp.12330

24.

Funderburk

. (2011). NCT01294488: Using technology to augment the implementation and effectiveness of PCIT. http://clinicaltrials.gov/show/NCT01294488

25.

Funderburk

Chaffin

Bard

Shanley

Bard

Berliner

. (2015). Comparing client outcomes for two evidence-based treatment consultation strategies. Journal of Clinical Child and Adolescent Psychology, 44(5), 730–741. https://doi.org/10.1080/15374416.2014.910790

26.

Furlong

McGilloway

Bywater

Hutchings

Smith

S. M.

Donnelly

. (2013). Cochrane review: Behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years (review). Evidence-Based Child Health, 8(2), 318–692. https://doi.org/10.1002/ebch.1905

27.

Giguère

Légaré

Grimshaw

Turcotte

Fiander

Grudniewicz

Makosso-Kallyth

Wolf

F. M.

Farmer

A. P.

Gagnon

M. P.

(2012). Printed educational materials: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (10). https://doi.org/10.1002/14651858.CD004398.pub3

28.

Herschell

A. D.

Kolko

D. J.

Scudder

A. T.

Taber-Thomas

Schaffner

K. F.

Hiegel

S. A.

Iyengar

Chaffin

Mrozowski

(2015). Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment. Implementation Science, 10(1), Article 133. https://doi.org/10.1186/s13012-015-0324-z

29.

Herschell

A. D.

McNeil

C. B.

Urquiza

A. J.

McGrath

J. M.

Zebell

N. M.

Timmer

S. G.

Porter

. (2009). Evaluation of a treatment manual and workshops for disseminating, parent-child interaction therapy [article]. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 63–81. https://doi.org/10.1007/s10488-008-0194-7

30.

Higgins

J. P. T.

Savović

Page

M. J.

Elbers

R. G.

Sterne

J. A.

(2020). Assessing risk of bias in a randomized trial. In Higgins

J. P. T.

Thomas

Chandler

Cumpston

Page

M. J.

Welch

V. A.

(Eds.), Cochrane handbook for systematic reviews of interventions (pp. 205–228). https://doi.org/10.1002/9781119536604.ch8 .

31.

Hoffmann

T. C.

Glasziou

P. P.

Boutron

Milne

Perera

Moher

Altman

D. G.

Barbour

Macdonald

Johnston

(2014). Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ, 348, g1687. https://doi.org/10.1136/bmj.g1687

32.

Ivers

Jamtvedt

Flottorp

Young

J. M.

Odgaard-Jensen

French

S. D.

O'Brien

M. A.

Johansen

Grimshaw

Oxman

A. D.

(2012). Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (6). https://doi.org/10.1002/14651858.CD000259.pub3

33.

Knerr

Gardner

Cluver

. (2013). Improving positive parenting skills and reducing harsh and abusive parenting in low- and middle-income countries: A systematic review. Prevention Science, 14(4), 352–363. https://doi.org/10.1007/s11121-012-0314-1

34.

Leijten

Gardner

Melendez-Torres

G. J.

van Aar

Hutchings

Schulz

Knerr

Overbeek

. (2019). Meta-analyses: Key parenting program components for disruptive child behavior. Journal of the American Academy of Child and Adolescent Psychiatry, 58(2), 180–190. https://doi.org/https://doi.org/10.1016/j.jaac.2018.07.900

35.

Leijten

Melendez-Torres

G. J.

Gardner

van Aar

Schulz

Overbeek

. (2018). Are relationship enhancement and behavior management “The golden couple” for disruptive child behavior? Two meta-analyses. Child Development, 89(6), 1970–1982. https://doi.org/10.1111/cdev.13051

36.

Lewis

C. C.

Boyd

M. R.

Walsh-Bailey

Lyon

A. R.

Beidas

Mittman

Aarons

G. A.

Weiner

B. J.

Chambers

D. A.

(2020/04/16). A systematic review of empirical studies examining mechanisms of implementation in health. Implementation Science, 15(1), 21. https://doi.org/10.1186/s13012-020-00983-3

37.

Lewis

C. C.

Fischer

Weiner

B. J.

Stanick

Kim

Martinez

R. G.

(2015). Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10(1), 155. https://doi.org/10.1186/s13012-015-0342-x

38.

Lewis

C. C.

Klasnja

Powell

B. J.

Lyon

A. R.

Tuzzio

Jones

Walsh-Bailey

Weiner

(2018). From classification to causality: Advancing understanding of mechanisms of change in implementation science [perspective]. Frontiers in Public Health, 6(136). https://doi.org/10.3389/fpubh.2018.00136

39.

Lieneman

C. C.

Quetsch

L. B.

Theodorou

L. L.

Newton

K. A.

McNeil

C. B

. (2019). Reconceptualizing attrition in parent-child interaction therapy: “Dropouts” demonstrate impressive improvements. Psychology Research and Behavior Management, 12, 543–555. https://doi.org/10.2147/PRBM.S207370

40.

Lyon

A. R.

Dopp

A. R.

Brewer

S. K.

Kientz

J. A.

Munson

S. A.

(2020). Designing the future of children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research. https://doi.org/10.1007/s10488-020-01038-x

41.

Metcalfe

Rios Diaz

A. J.

Olufajo

O. A.

Massa

M. S.

Ketelaar

Flottorp

S. A.

Perry

D. C

. (2018). Impact of public release of performance data on the behaviour of healthcare consumers and providers. Cochrane Database of Systematic Reviews, (9). https://doi.org/10.1002/14651858.CD004538.pub3

42.

Michelson

Davenport

Dretzke

Barlow

Day

. (2013). Do evidence-based interventions work when tested in the “real world?” A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clinical Child and Family Psychology Review, 16(1), 18–34. https://doi.org/10.1007/s10567-013-0128-0

43.

Michie

van Stralen

M. M.

West

. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science, 6(1), 42. https://doi.org/10.1186/1748-5908-6-42

44.

Motamedi

Lau

A. S.

Lind

Lui

J. H.

Rodriguez

Smith

Brookman-Frazee

. (2021). What educational strategies and mechanisms facilitate EBP use? A mixed methods examination of therapist perceptions within a system-driven implementation of multiple EBPs. Implementation Research and Practice, 2, 2633489520982903. https://doi.org/10.1177/2633489520982903

45.

O'Brien

M. A.

Rogers

Jamtvedt

Oxman

A. D.

Odgaard-Jensen

Kristoffersen

D. T.

Forsetlund

Bainbridge

Freemantle

Davis

Haynes

R. B.

Harvey

. (2007). Educational outreach visits: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (4). https://doi.org/10.1002/14651858.CD000409.pub2

46.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372. https://doi.org/https://doi.org/10.1016/j.jclinepi.2021.02.003

47.

Pantoja

Grimshaw

J. M.

Colomer

Castañon

Leniz Martelli

. (2019). Manually-generated reminders delivered on paper: Effects on professional practice and patient outcomes. Cochrane Database of Systematic Reviews, (12). https://doi.org/10.1002/14651858.CD001174.pub4

48.

Powell

B. J.

Fernandez

M. E.

Williams

N. J.

Aarons

G. A.

Beidas

R. S.

Lewis

C. C.

McHugh

S. M.

Weiner

B. J.

(2019). Enhancing the impact of implementation strategies in healthcare: A research agenda [perspective]. Frontiers in Public Health, 7(3). https://doi.org/10.3389/fpubh.2019.00003

49.

Powell

B. J.

Proctor

E. K.

Glass

J. E

. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24(2), 192–212. https://doi.org/10.1177/1049731513505778

50.

Presseau

McCleary

Lorencatto

Patey

A. M.

Grimshaw

J. M.

Francis

J. J

. (2019). Action, actor, context, target, time (AACTT): A framework for specifying behaviour. Implementation Science, 14(1), 102. https://doi.org/10.1186/s13012-019-0951-x

51.

Proctor

E. K.

Landsverk

Aarons

Chambers

Glisson

Mittman

. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34. https://doi.org/10.1007/s10488-008-0197-4

52.

Proctor

E. K.

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

53.

Ranmuthugala

Plumb

J. J.

Cunningham

F. C.

Georgiou

Westbrook

J. I.

Braithwaite

(2011). How and why are communities of practice established in the healthcare sector? A systematic review of the literature. BMC Health Services Research, 11(1), 273. https://doi.org/10.1186/1472-6963-11-273

54.

Rivenbark

J. G.

Odgers

C. L.

Caspi

Harrington

Hogan

Houts

R. M.

Poulton

Moffitt

T. E

. (2017). The high societal costs of childhood conduct problems: Evidence from administrative records up to age 38 in a longitudinal birth cohort. Journal of Child Psychology and Psychiatry and Allied Disciplines, 59(6), 703–710. https://doi.org/10.1111/jcpp.12850

55.

Rudd

B. N.

Davis

Beidas

R. S.

(2020). Integrating implementation science in clinical research to maximize public health impact: A call for the reporting and alignment of implementation strategy use with implementation outcomes in clinical research. Implementation Science, 15(1), 103. https://doi.org/10.1186/s13012-020-01060-5

56.

Schierhout

Hains

Kennedy

Cox

Kwedza

O’Donoghue

Fittock

Brands

Lonergan

Dowden

Bailie

. (2013). Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: Developing a realist theory of change. Implementation Science, 8(1), 119. https://doi.org/10.1186/1748-5908-8-119

57.

Shelton

R. C.

Lee

Brotzman

L. E.

Wolfenden

Nathan

Wainberg

M. L

. (2020). What is dissemination and implementation science? An introduction and opportunities to advance behavioral medicine and public health globally. International Journal of Behavioral Medicine, 27(1), 3–20. https://doi.org/10.1007/s12529-020-09848-x

58.

Shoesmith

Hall

Wolfenden

Shelton

R. C.

Powell

B. J.

Brown

McCrabb

Sutherland

Yoong

Lane

. (2021). Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: A systematic review. Implementation Science, 16(1), 1–20. https://doi.org/10.1186/s13012-021-01134-y

59.

Stavropoulou

Doherty

Tosey

. (2015). How effective are incident-reporting systems for improving patient safety? A systematic literature review. The Milbank Quarterly, 93(4), 826–866. https://doi.org/10.1111/1468-0009.12166

60.

Sterne

J. A.

Hernán

M. A.

Reeves

B. C.

Savović

Berkman

N. D.

Viswanathan

Henry

Altman

D. G.

Ansari

M. T.

Boutron

(2016). ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355. https://doi.org/10.1136/bmj.i4919

61.

Thomas

Abell

Webb

H. J.

Avdagic

Zimmer-Gembeck

M. J.

(2017). Parent-child interaction therapy: A meta-analysis [article]. Pediatrics, 140(3), Article e20170352. https://doi.org/10.1542/peds.2017-0352

62.

Valenstein-Mah

Greer

McKenzie

Hansen

Strom

T. Q.

Wiltsey Stirman

Wilt

T. J.

Kehle-Forbes

S. M

. (2020/05/27). Effectiveness of training methods for delivery of evidence-based psychotherapies: A systematic review. Implementation Science, 15(1), 40. https://doi.org/10.1186/s13012-020-00998-w

63.

Ward

C. L.

Wessels

I. M.

Lachman

J. M.

Hutchings

Cluver

L. D.

Kassanjee

Nhapi

Little

Gardner

. (2020). Parenting for lifelong health for young children: A randomized controlled trial of a parenting program in South Africa to prevent harsh parenting and child conduct problems. Journal of Child Psychology and Psychiatry, 61(4), 503–512. https://doi.org/10.1111/jcpp.13129

64.

Ward

M. A.

Theule

Cheung

. (2016). Parent–child interaction therapy for child disruptive behaviour disorders: A meta-analysis [article]. Child and Youth Care Forum, 45(5), 675–690. https://doi.org/10.1007/s10566-016-9350-5

65.

Williams

N. J.

Beidas

R. S

. (2019). Annual research review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, 60(4), 430–450. https://doi.org/10.1111/jcpp.12960

66.

Woodfield

M. J.

Cargo

Barnett

Lambie

. (2020). Understanding New Zealand therapist experiences of parent-child interaction therapy (PCIT) training and implementation, and how these compare internationally. Children and Youth Services Review, 119, 105681. https://doi.org/10.1016/j.childyouth.2020.105681

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB