Abstract
Intervention scientists have proposed a focus on empirically supported principles of change (ESPCs) in psychotherapies. We explored this proposition as applied to youth psychotherapies, focusing on five candidate ESPCs—calming, increasing motivation, changing unhelpful thoughts, solving problems, and practicing positive opposites. We synthesized 348 treatment–control comparisons from 263 randomized controlled trials (RCTs) spanning six decades, testing treatments for anxiety, depression, attention-deficit/hyperactivity disorder, and conduct problems. We found that ESPCs could be reliably identified and distinguished by independent coders and that psychotherapies most often included fewer than three ESPCs. However, across the entire study pool and the anxiety subsample, when we controlled for dose, treatments with all five ESPCs showed effects about twice as large as treatments with fewer ESPCs. The findings suggest that ESPCs are reliably identifiable, that they are associated with variations in treatment effect size, and that treatments containing more ESPCs may produce greater therapeutic benefit.
Since the 1960s, researchers have conducted hundreds of randomized controlled trials (RCTs) of psychotherapies for the most common mental health concerns among youths (anxiety, depression, conduct, attention-deficit/hyperactivity disorder [ADHD]; Weisz & Kazdin, 2017). In these RCTs, many psychotherapies have shown beneficial effects (mean Hedges’s g = 0.46; Weisz, Kuppens, et al., 2017), but meta-analyses (Jones et al., 2019; Weisz et al., 2019; Weisz, Kuppens, et al., 2017) and epidemiological studies (Achenbach et al., 2003; Merikangas et al., 2010) suggest that there may be value in assessing how youth psychotherapies have historically been designed and considering whether modifications might be useful in boosting their benefits and implementability.
This theme is consistent with growing calls to revisit and redesign dominant conceptualizations of psychopathology (e.g., National Institute of Mental Health [NIMH] Research Domain Criteria [RDoC]; Hierarchical Taxonomy of Psychopathology [HiTOP]; Conway et al., 2019). These calls have largely been driven by efforts to address challenges identified with the dominant psychiatric classification system—the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association [APA], 2013). For example, the DSM has been criticized for organizing psychological disorders according to specific theoretical orientations and as discrete, standardized disorders, subsequently failing to fully capture the dimensional nature of psychopathology (e.g., Conway et al., 2019; Lilienfeld & Treadway, 2016; Sauer-Zavala et al., 2017; Zimmerman et al., 2015). Thus, although the DSM has definite strengths, such as creating a common language of psychiatric disorders for providers, experts have argued that there may be value to rethinking this system and, ultimately, the psychotherapy development and delivery it has guided and continues to inform.
Indeed, operating within the DSM framework, most youth psychotherapies have been designed to target one specific disorder (e.g., obsessive-compulsive disorder), problem domain (e.g., conduct), or homogenous cluster (e.g., several anxiety disorders) through a relatively standardized sequence of sessions (e.g., Coping Cat—a particularly successful cognitive-behavioral therapy for youth anxiety; Kendall & Hedtke, 2006). RCTs testing these kinds of treatments have been prominent in reviews documenting the success in youths of evidence-based psychotherapies (EBPs) for anxiety (e.g., Chronis-Tuscano et al., 2018; Higa-McMillan et al., 2016; Reynolds et al., 2012), depression (e.g., Curry, 2014; Oud et al., 2019; Weersing, Brent, et al., 2017), conduct (e.g., Kaminski & Claussen, 2017; Lochman et al., 2019; McCart & Sheidow, 2016), ADHD (e.g., DuPaul et al., 2012; Evans et al., 2018), and posttraumatic stress (e.g., Dorsey et al., 2017; Silverman et al., 2008). These psychotherapies have made and will certainly continue to make enormous contributions to the field.
At the same time, experts have recommended shifting from this DSM-based approach to one that is more transdiagnostic and transtheoretical, with the goal of better reflecting the dynamic, personalized, and dimensional nature of mental health (Conway et al., 2019; Lilienfeld & Treadway, 2016; Sauer-Zavala et al., 2017; Zimmerman et al., 2015). For example, most clinically referred youths exhibit multiple co-occurring mental health concerns (Merikangas et al., 2010) that fluctuate across episodes of clinical care (e.g., Powell et al., 2013; Thomassin et al., 2019; Weisz et al., 2015), which highlights the need for more flexible and customizable approaches (e.g., transdiagnostic and modular psychotherapies; e.g., Chorpita & Weisz, 2009; Chu et al., 2016; Ehrenreich-May et al., 2017; Galla et al., 2021; García-Escalera et al., 2020; Kennedy et al., 2019; Lyon et al., 2011; Murray et al., 2018; Weersing, Brent, et al., 2017; Weisz et al., 2012; Wood et al., 2009, 2015). One framework that has been proposed by several prominent leaders in the field involves focusing on empirically supported principles of change (ESPCs; see, e.g., Castonguay & Beutler, 2006; Davison, 2019; Goldfried, 1980, 2009; Hofmann & Hayes, 2019; Oddli et al., 2016; Rosen & Davison, 2003; Tolin et al., 2015).
Empirically Supported Principles of Change
An historical overview
The ESPC framework is in line with historical and recent recommendations that psychologists focus attention on the processes of change that may be activated during effective psychotherapy (e.g., Bandura, 1969; Davison, 1994) and was popularized by Goldfried (1980). In his seminal piece, Goldfried defined ESPCs as a “middle ground” between more abstract theoretical orientations (e.g., cognitive-behavioral therapy, psychodynamic therapy) and more concrete, specific treatment procedures (e.g., cognitive restructuring, exposure, transference assessment). According to Goldfried (1980, 2019), ESPCs—which are common across most forms of psychotherapy, regardless of theoretical orientation—may include, for example, promoting client expectation that therapy can help and strengthening client motivation for change (e.g., as found in motivational interviewing techniques; Rollnick et al., 2010), as well as establishing optimal therapeutic alliance (e.g., through personal attributes, such as flexibility and warmth, and therapeutic techniques, such as reflection and accurate interpretation of the client’s presenting concerns; Ackerman & Hilsenroth, 2003).
This recommendation to focus on ESPCs is in line with that of other experts in the field, such as Rosen and Davison (2003), who argued that this focus has the potential to advance the field by “redirecting the attention . . . where it should be—on mechanisms of change” (p. 309). Overall, these proposals have contributed to valuable work in the field (e.g., Castonguay et al., 2019), from the development of task forces designed to identify ESPCs (Castonguay & Beutler, 2006) to important discussions about the factors of effective psychotherapies that are common across, versus specific to, different treatment modalities (e.g., Crits-Christoph, 1997; Gaines et al., 2021; McAleavey & Castonguay, 2015; Wampold et al., 1997). Leveraging the strong foundation laid by these scientific leaders, experts have increasingly argued that the field is well-positioned to and would greatly benefit from applying the ESPC framework through scientific research and clinical practice (e.g., Gaines et al., 2021; Goldfried, 2019).
Where are they now?
Despite the existence of the ESPC concept in the literature for several decades, the empirical examination of its application has been limited to date. There may be several reasons for what appears to be a rather slow pace. First, the ESPC concept is a rather abstract one. This abstraction may need to be translated into a more specific definition and to be populated with more concrete content that is amenable to empirical assessment in order to provide intervention scientists with an opportunity to evaluate its potential merit. Second, given the abstract nature of ESPCs, it is not clear whether they can be reliably identified by independent evaluators. Third, lacking a clear definition and concrete examples that can be reliably identified, it has been difficult to document the extent to which, and the manner in which, ESPCs may have been included in youth psychotherapies studied to date. Of note, experts, such as Goldfried (2019), have argued that targeted tests of ESPCs should focus on empirically grounded “research findings associated with the process of change or general principles” as opposed to “placing an emphasis on developing new approaches to treatment” (p. 492). To this end, one potentially valuable step in the empirical assessment of the ESPC framework may be examining the effects associated with various ESPCs within the context of existing psychotherapies.
We aimed to contribute to this effort, as described in detail below, by providing a working definition of ESPCs, generating concrete examples of candidate ESPCs, assessing whether ESPCs can be reliably identified, providing data on the extent and manner of ESPC inclusion in the youth psychotherapy evidence base to date, and offering preliminary data on the magnitude of treatment effects associated with the presence of different ESPCs within extant treatments. In this article, we define ESPCs as constructs that encompass and characterize groups of conceptually related treatment procedures for which RCTs have shown significant evidence of benefit. Given the emphasis on empirically supported principles of therapeutic change among experts, we emphasized ESPCs within EBPs in this study.
Potential usefulness in youth psychotherapy
A focus on ESPCs found within EBPs could contribute usefully to research and practice in youth psychotherapy for several reasons. First, this model is positioned to leverage the strengths of EBPs, as ESPCs are broad principles (e.g., calming) that subsume clusters of EBP-based elements (e.g., diaphragmatic breathing, progressive muscle relaxation). Second, this approach has the potential to be transdiagnostic, with each ESPC being appropriate for a range of mental health concerns. Third, an ESPC approach might be personalizable, with principles selected and sequenced to fit the needs of each youth. Fourth, this approach might be more readily accessed and mastered by clinicians than an array of different standardized EBPs. That is, a relatively modest number of broad ESPCs that subsume many specific treatment procedures might be assimilated rather readily, and identifying the ESPCs that a therapy is intended to activate could provide an organizing framework for clinician decision-making—illuminating which procedures should be selected and why. Fifth, ESPCs could serve a similar function for clients, helping them understand the rationale of their treatment, thereby enhancing uptake, mastery, and generalization of skills. Finally, although the search for mechanisms of change in EBPs has been highly incentivized in recent years (e.g., NIMH RDoC and the experimental therapeutics model, with guidance to identify treatment “targets” in funding proposals), there is some question about how productive the search has been to date. The ESPC approach, which emphasizes therapeutic change processes, may contribute to this search by specifying candidate ESPCs and providing a framework for conceptualizing and investigating them.
Identifying candidate ESPCs
An important first step toward scientifically evaluating the ESPC framework is identifying testable candidate ESPCs. A valuable and growing body of work has been dedicated to pursuing this goal, such as that produced by the APA Task Force created for this purpose (Castonguay & Beutler, 2006), theoretical recommendations from experts (e.g., Goldfried, 2019), and reviews of the psychotherapy literature (e.g., McAleavey & Castonguay, 2015; McAleavey et al., 2019). The ESPCs identified within this body of work have included, for example, promoting client motivation to engage with therapy, facilitating client awareness of the factors associated with their difficulties for which they are seeking treatment, and building optimal therapeutic alliance, as well as establishing the facilitative conditions Rogers’ (1957) proposed as necessary and sufficient for therapeutic change—genuineness, accurate empathy, and unconditional positive regard (Castonguay & Beutler, 2006; Goldfried, 2019; McAleavey & Castonguay, 2015; McAleavey et al., 2019). As demonstrated by these efforts, there are many possible ESPCs, with recent work suggesting as many as 38 candidate principles of therapeutic change that can be categorized within five broader groups (e.g., client prognostic principles, therapy relationship principles; McAleavey et al., 2019).
Building on this important work, a potentially useful step toward empirically assessing ESPCs might be concretizing them by clustering treatment procedures of demonstrated effectiveness (e.g., praise, rewards, contingency management) into broader ESPC groups (e.g., increasing the client’s motivation to engage with therapy, which aligns with previously identified ESPCs). In the current research, we pursued this step within the context of youth mental health disorders that account for most treatment referrals (i.e., anxiety, depression, ADHD, and conduct; Weisz & Kazdin, 2017). As a starting point, we focused on five candidate ESPCs that had been identified through an iterative process that included three primary steps. First, we searched published RCTs to identify an initial pool of potential ESPCs that appeared to be represented in the treatments that had been tested in those RCTs and reviews of those RCTs (e.g., Comer et al., 2019; Dorsey et al., 2017; Evans et al., 2018; Freeman et al., 2014; Higa-McMillan et al., 2016; Kaminski & Claussen, 2017; Kendall & Hedtke, 2006; Lonigan et al., 1998; McCart & Sheidow, 2016; Silverman & Hinshaw, 2008; Southam-Gerow & Prinstein, 2014; Weersing, Jeffreys, et al., 2017; Weisz, 2004). Second, we reduced the pool to those that met a specific operational definition of “empirically supported” (see below), and finally, a team of seven child and adolescent psychotherapy practitioners and five prominent clinical psychologists who are well known as treatment developers reviewed this pool (see Weisz & Bearman, 2020; Weisz, Bearman, et al., 2017). The operational definition of empirically supported that we employed was that there had to be at least one published RCT showing beneficial effects of a treatment that included only one of the potential ESPCs in the pool. We identified 26 published youth RCTs in which this was the case (see Weisz & Bearman, 2020, for a full list). Within these 26 studies, five ESPCs emerged that satisfied our operational definition. The descriptive language characterizing these was modified for clarity and accuracy via review by the clinician and treatment developer team.
As described in Table 1, the resulting list of five ESPCs included (a) calming (reducing tension and emotional arousal; e.g., progressive muscle relaxation, quick calming techniques, using calming imagery), (b) increasing motivation (using environmental contingencies, such as differential attending, praise, and tangible rewards, to make desirable behavior more rewarding than undesirable behavior), (c) changing unhelpful cognitions (identifying and changing biased, distorted, or otherwise unhelpful cognitions; e.g., overestimates of threat, self-blaming thoughts, hostile attributions), (d) solving problems (building skill in sequential problem solving; e.g., identify the problem, set a goal, think of solutions, weigh the pros and cons, choose a solution), and (e) practicing the positive opposite (engaging in activities that directly counter unhelpful behavior; e.g., graduated exposure to address avoidance, behavioral activation to address depression, practicing effective responses to interpersonal conflict).
Empirically Supported Principles of Change (ESPCs)
The Current Study
In the present study, we investigated these ESPCs as they appeared in a large database of youth psychotherapy RCTs spanning the past 60 years. We addressed several preregistered questions (PROSPERO No. CRD42020177503) about the feasibility of an ESPC approach and its potential to contribute to research and practice. First, we explored whether it is possible to reliably code therapy procedures according to the ESPCs that subsume them (Question 1). This was a critical first step toward understanding the feasibility of identifying ESPCs and, subsequently, developing treatments based on them. Then, we explored frequency of use—the extent to which treatments over the past 60 years have included one through five ESPCs (Question 2)—and the frequency of each one of the ESPCs (Question 3) across the full study collection and within each of the four problem domains. We synthesized this information to provide a foundational understanding of how ESPCs have been used within youth psychotherapies to date, as well as to contribute to growing efforts to understand which therapeutic approaches might be associated with outcomes for which disorders and problems (e.g., Ng & Weisz, 2016). Next, we examined the association between each ESPC and posttreatment effect size of the tested treatments that included them across the full study collection and within each of the four problem domains (Question 4). 1 Finally, we examined the association between the number of ESPCs included in treatments and posttreatment effect size across the full study collection and within each of the four problem domains (Question 5) to shed light on whether treatment efficacy varied by the number of ESPCs included and whether this differed by target problem. Number of ESPCs, we reasoned, might be construed as one aspect of the conceptual density of treatments. That is, therapies might be seen as tiered systems with multiple specific treatment procedures organized within superordinate ESPC categories. Following this reasoning, treatments may vary in both procedural density (the number of treatment procedures included) and conceptual density (the number of principles of change the treatment is designed to engage). As a complement to seminal work focusing on procedural density (e.g., Chorpita & Daleiden, 2009), we focused, in Question 5, on conceptual density as a predictor of treatment outcome. The question relates to ongoing debates about the efficacy of more streamlined treatments that pose fewer cognitive and memory demands versus treatments that are richer conceptually but also more complex and cognitively demanding (see, e.g., Schleider & Weisz, 2017; Weisz et al., 2019; Zieve et al., 2019).
Method
Data sources
Our literature search included peer-reviewed RCTs testing psychological therapies for youth psychopathology encompassing four broad domains that account for most mental health referrals (Weisz, 2004; Weisz & Kazdin, 2010)—depression (e.g., Mufson et al., 1999), anxiety (e.g., Kendall et al., 1997; including obsessive-compulsive disorder and posttraumatic stress disorder), conduct problems (e.g., Scott et al., 2010), and ADHD-related problems (e.g., The MTA Cooperative Group, 1999). We searched PsycINFO and PubMed from January 1960 through December 2020. For PsycINFO, we used 21 key terms related to psychological therapy (e.g., “psychother-,” “counseling”) that had been used in previous youth therapy meta-analyses, crossed with outcome assessment topic and age-group constraints. PubMed’s indexing system (Medical Subject Headings [MeSH]) searches publishers who may use different keywords for the same concepts; we used “mental disorders,” with the following search limits: clinical trial, child, published in English, and human subjects. In addition, we searched related reviews and meta-analyses, followed reference trails in the reports we identified, and obtained additional studies identified by youth therapy researchers whom we contacted.
Inclusion criteria
Criteria for study inclusion were as follows: (a) participants selected and treated for anxiety, depression, ADHD, or conduct; (b) random assignment of youth to treatment and control conditions (waitlist, placebo, usual care), in which at least one of the conditions was psychological therapy (we excluded study conditions involving pharmacotherapy alone or in combination with psychotherapy); (c) mean participant age of 3.5 to 18.4 years; (d) outcome measures administered to both conditions; and (e) published in English. We defined psychopathology as either meeting criteria for a DSM (APA, 2013) diagnosis or showing elevated symptoms on measures of psychopathology for several reasons: (a) Both definitions of psychopathology are common in the youth psychotherapy literature (Weisz, 2004; Weisz & Kazdin, 2010), (b) youths with elevated symptoms experience serious impairment (Angold et al., 1999; Costello et al., 1999; Silverman & Hinshaw, 2008; Weisz, 2004), (c) such youths are often referred for mental health services (Jensen & Weisz, 2002; Weisz et al., 2013), and (d) diagnostic categories and their definitions within the DSM system have varied markedly across the five decades encompassed in this study.
Study selection
Using the pool of studies that met inclusion criteria, we then narrowed our sample for the proposed analyses. Studies were initially screened as part of an ongoing meta-analysis database-building process (Weisz et al., 2019). After removing duplicate articles in PsycInfo and PubMed, a team of trained research assistants reviewed each article’s title and abstract for inclusion and retrieved 4,896 full-text studies for further double screening using standardized procedures. In total, 4,633 of these studies were excluded. For the current analyses, we included only studies that (a) tested a treatment that was coded (using methods reported below) as including one or more of the five ESPCs, 2 (b) provided sufficient data to calculate effect sizes, and (c) provided sufficient data on the total number of treatment hours to permit calculation of treatment dose as a covariate in our models. See Figure S1 in the Supplemental Material available online for a detailed flowchart of study retrieval, review, and inclusion. Ultimately, 348 treatment–control comparisons in 263 RCT articles met the criteria, with 3,155 effect sizes. The RCTs involved treatment targeting anxiety (n = 152), conduct (n = 115), ADHD (n = 44), and depression (n = 37). Across included studies, the mean participant age was 10.50 years (SD = 3.80), the mean percentage of males was 61.02% (SD = 25.35), and 64.57% of the study samples were White. References for all included studies that met these criteria can be found in the References of Included Studies section in the Supplemental Material.
Data extraction, coding, and processing
All included studies were coded for the (a) target problem domain, (b) treatment dose (total maximum number of planned treatment hours), (c) type of ESPCs featured in the tested treatment, and (d) number of ESPCs featured in the tested treatment. Target problem domain (mean κ = .89) and treatment dose (mean κ = .98) were coded as part of a broader meta-analysis database project by a team of seven clinical psychology postdoctoral fellows and graduate students. These codes were then compared with a gold standard provided by the most experienced coder, an RCT researcher with a doctorate in clinical psychology and 15 years of post-PhD experience with youth psychotherapy trials.
Coded ESPC data were specific to this study and were captured by one master’s-level doctoral student (O. M. Fitzpatrick) in clinical psychology and one doctoral-level clinical psychology researcher and clinician (E. Cho). The coders created and refined a codebook (see the Codebook for ESPCs section in the Supplemental Material) with the senior author (J. R. Weisz), an expert in youth psychotherapy, through discussions of the literature and iterative applications of the codebook to a series of small batches of studies (10% of studies). Once the codebook was finalized, the coders independently coded 80% of included studies, resolving discrepancies through discussions during weekly meetings and consultation with the senior author. Following the achievement of interrater reliability (reported below in Results), the second author then coded the remaining 10% of studies independently.
Effect-size calculation
All effect-size calculations were conducted using the metafor package (Viechtbauer, 2010) in R. Hedges’s g, which corrects for small sample bias for each effect size within each study, was calculated for each comparison between a psychotherapy and a control condition at posttreatment.
Robust variance estimation models
Analyses were conducted using robust variance estimation (RVE), which accounts for dependence among effect sizes from the same sample by correcting study standard errors. Given that RVE adjusts only study standard errors, it relies on fewer distributional assumptions and requires less power than similar approaches, such as multilevel meta-analysis. The robumeta (Fisher et al., 2018) package in R was used to implement all models, which included weighted, random-effects to account for between-study variance. In line with established recommendations (Tipton, 2015), we specified a small sample-size correction in models with fewer than 40 treatment versus control comparisons. As preregistered, treatment dose—operationalized as the maximum number of planned treatment hours—was included as a covariate in all models to account for its significant correlation with the number of included ESPCs (r = .13, p < .001).
To answer Question 4, we implemented separate intercept-only RVE models (a) to obtain a pooled posttreatment effect size for each type of ESPC across the entire sample and within each subsample and (b) to determine whether these pooled posttreatment effect sizes significantly differed from an effect size of zero. To answer Question 5, we implemented RVE models testing whether number of ESPCs moderated posttreatment effect sizes. We operationalized number of ESPCs in two ways: (a) continuously, to test whether the number of ESPCs was linearly related to posttreatment effect size, and (b) categorically, to test the average difference in posttreatment effect size between treatments with different numbers of ESPCs. All analyses were conducted within the entire sample and each problem domain subsample.
Results
Question 1: Is it possible to reliably code treatment procedures according to ESPCs?
Intercoder agreement (κs) for the type and number of ESPCs featured in included treatments was .88 and .90, respectively.
Question 2: What are the frequencies of each number of ESPC category?
Across all the 348 treatment–control comparisons, most tested treatments featured one ESPC (n = 106) or two ESPCs (n = 102), followed by three (n = 79), four (n = 42), and five (n = 19) ESPCs. This breakdown within each problem domain is shown in Figure 1.
Number of empirically supported principles of change (ESPCs) within each problem domain. The figure shows the frequency of treatments with each number of ESPC category (from one to five ESPCs) within each problem domain category. Taller blocks represent a greater proportion of treatments within a given ESPC category. Wider blocks represent a greater number of total treatments in a given problem domain. For example, among the included treatments for anxiety (n = 152), 36 included one ESPC, 53 included two, 28 included three, 23 included four, and 12 included five. ADHD = attention-deficit/hyperactivity disorder.
Question 3: What are the frequencies of each type of ESPC category?
First, across all treatment–control comparisons, we examined the frequency of each ESPC when used as a standalone treatment. As shown in Figure 2, increasing motivation (n = 48) was most commonly tested as a standalone treatment, followed by practicing the positive opposite (n = 37), changing unhelpful cognitions (n = 7), calming (n = 7), and solving problems (n = 6). Second, we examined the frequency of each ESPC when used with one or more additional ESPCs. Practicing the positive opposite was most frequently included in tested treatments that included more than one ESPC, followed by changing unhelpful cognitions, increasing motivation, solving problems, and calming.
Relationship between number of empirically supported principles of change (ESPCs) and each ESPC. The figure shows the frequency of treatments with each number of ESPC category (from one to five ESPCs) within each ESPC category (calming, increasing motivation, etc.). Taller blocks represent a greater proportion of treatments within a given number of ESPC category. Wider blocks represent a greater number of total studies in a given ESPC category. For example, among the included treatments with relaxation, seven included one ESPC, 18 included two, 27 included three, 29 included four, and 19 included five. Note that these totals do not equal the sample N because most treatments included a combination of ESPCs.
We then examined the frequency of each ESPC within each problem domain. For ADHD, increasing motivation and engaging in the positive opposite emerged as the ESPCs most often included in tested treatments alone and with other ESPCs. For conduct, increasing motivation and practicing the positive opposite were most frequently included in tested treatments alone, and practicing the positive opposite and solving problems were most often tested with other ESPCs. For anxiety, practicing the positive opposite and changing unhelpful cognitions emerged as the ESPCs most frequently included in tested treatments alone and with other ESPCs. Finally, for depression, only three treatments tested an ESPC alone, and each featured a different one; however, changing unhelpful cognitions and practicing the positive opposite were most often tested with other ESPCs.
Question 4: Is treatment efficacy related to the type of ESPCs included?
We then explored the pooled posttreatment effect size for each ESPC type (see Fig. 3), as well as whether these effect sizes significantly differed from an effect size of zero. As shown in Table 2, across the entire sample, the anxiety subsample, and the conduct subsample, calming (0.42–0.58), increasing motivation (0.43–0.68), changing unhelpful cognitions (0.50–0.69), solving problems (0.41–0.90), and practicing the positive opposite (0.52–0.56) generated significant posttreatment effect sizes, relative to an effect size of zero. Within the anxiety subsample, practicing the positive opposite also yielded a significant pooled posttreatment effect size of .54 when used as a standalone intervention. Within the ADHD subsample, significant pooled posttreatment effect sizes emerged for treatments with increasing motivation (0.56), solving problems (0.32), and practicing the positive opposite (0.39) and, within the depression subsample, significant pooled posttreatment effect sizes emerged for treatments with changing unhelpful cognitions (0.24) and practicing the positive opposite (0.28).
Empirically supported principle of change (ESPC) type and pooled posttreatment effect size across the entire sample. Asterisks indicate effect sizes that could not be calculated because of insufficient degrees of freedom (dfs < 4). Values in parentheses represent the sample size of the treatment–control comparison. Overall effect sizes represent pooled Hedges’s g of each ESPC used as a standalone treatment and with other ESPCs.
Pooled Posttreatment Effect Size of Treatments With Each Empirically Supported Principle of Change (ESPC)
Note: Pooled effect sizes are Hedges’s gs. p values and confidence intervals (CIs) represent whether the pooled effect size of a given ESPC is significantly greater than an effect size of zero. No values are reported when a particular subsample did not have sufficient degrees of freedom (dfs < 4) to yield reliable results.
Question 5: Is treatment efficacy related to the number of ESPCs included?
First, we tested the number of ESPCs per psychotherapy as a continuous moderator of posttreatment effect size. Number of ESPCs did not significantly moderate treatment efficacy in the entire sample (b = 0.42, df = 345, SE = 0.03, 95% confidence interval [CI] = [0.36, 0.48], p = .119), depression subsample (b = 0.22, df = 9.52, SE = 0.05, 95% CI = [0.09, 0.34], p = .608), ADHD subsample (b = 0.69, df = 41, SE = 0.07, 95% CI = [0.55, 0.83], p = .060), or conduct subsample (b = 0.38, df = 112, SE = 0.06, 95% CI = [0.26, 0.51], p = .376). However, it did significantly moderate treatment efficacy in the anxiety subsample (b = 0.38, df = 149, SE = 0.04, 95% CI = [0.30, 0.46], p = .024); specifically, therapies with more ESPCs yielded higher pooled posttreatment effect sizes.
We then tested the number of ESPCs as a categorical moderator of posttreatment effect size. The results from these models using the entire sample and each subsample are shown in Table 3. Treatments with five ESPCs were specified as the reference category in these models, except for the ADHD subsample because none of these treatments included five ESPCs; instead, treatments with four ESPCs were specified as the reference category for this subsample. In the entire sample and anxiety subsample, treatments with five ESPCs yielded significantly larger posttreatment effect sizes relative to those with one, two, three, and four ESPCs. No significant effects were found in the conduct or ADHD subsamples, and these models could not be implemented with the depression subsample because of insufficient degrees of freedom for reliable estimates (dfs < 4; Fisher et al., 2018).
Number of Empirically Supported Principles of Change (ESPCs) as a Categorical Moderator
Note: These models could not be implemented within the depression subsample because of insufficient degrees of freedom. Treatment dose (total number of planned treatment hours, as reported by the authors) was included as a covariate in all models. We tested each number of ESPC category as the reference to which all others were compared, and all of these models yielded the same results; for ease of interpretation, we selected the category of five ESPCs as the reference to which all others were compared, except for attention-deficit/hyperactivity disorder (ADHD), for which four ESPCs was used because there were no five-ESPC treatments for ADHD. CI = confidence interval.
Discussion
Over the past two decades, ideas advanced by experts in psychotherapy have suggested the potential value of a focus on ESPCs (Castonguay & Beutler, 2006; Davison, 2019; Goldfried, 2009; Hofmann & Hayes, 2019; Oddli et al., 2016; Rosen & Davison, 2003; Tolin et al., 2015). Here, we sought to provide empirical data bearing on the feasibility and utility of this notion within the context of youth psychotherapy by conducting a systematic review and meta-analysis of 263 youth psychotherapy RCTs (with 348 treatment–control comparisons) spanning 60 years. Focusing on five potential ESPCs, we addressed five questions relevant to evaluating the feasibility and utility of an ESPC approach.
A basic feasibility requirement is that ESPCs, once defined, must be reliably identifiable by independent reviewers (Question 1); we found high intercoder reliability (κs ranging from .88 to .90) in the identification of specific ESPCs and in tallying the number of ESPCs present within the various treatments. We next assessed the density of ESPCs used across all the treatments (Question 2), and we found a decreasing pattern, with one ESPC per treatment most common and five ESPCs least common, suggesting a preference among intervention developers for leaner treatments. Our analysis of the frequency of the different ESPCs within the various treatments (Question 3) showed that increasing motivation to engage in desired behavior (e.g., using attention, praise, or reward) was the most frequently used ESPC in those treatments that included only one ESPC and that practicing the positive opposite of unhelpful behavior (e.g., using exposure or behavioral activation) was the ESPC used most frequently in treatments that included multiple ESPCs; the specific ESPCs that were most and least often used varied across the four problem domains in ways that appear to be clinically meaningful.
Our calculation of pooled posttreatment effect sizes for treatments with each ESPC (Question 4) showed informative variations across the different ESPCs and problem domains. In tests of whether effects differed by the number of ESPCs included in treatments (Question 5), we found evidence that treatments with five ESPCs yielded effect sizes that were roughly two times greater than those of treatments with fewer ESPCs in the entire sample and the anxiety subsample. Recent evidence has supported the idea that simple single-session treatments can produce significant benefit (e.g., Schleider & Weisz, 2017), and other evidence has suggested that greater complexity and memory demands may be associated with reduced treatment benefit (Zieve et al., 2019), but the findings presented here suggest the testable possibility that, for some target problems, treatments that are conceptually rich in the number of ESPCs included may produce more substantial benefit than conceptually simpler treatments. This overall pattern warrants a deeper analysis at the level of individual problems targeted in treatment, as we will discuss later.
Our initial investigation served, in part, as a proof-of-concept study, demonstrating that it may be possible and potentially useful to characterize youth psychotherapies according to the ESPCs from which they are constructed. The high reliability that independent reviewers achieved in identifying ESPCs suggests that the information provided in documentation of treatment protocols may be sufficient for locating and distinguishing among the principles. This suggests, in turn, the possibility that an ESPC framework may be accessible and potentially useful for clinicians who deliver treatments to which that framework is applicable.
The ESPCs that were the focus of the current study are those identified in our multistep synthesis of scientific literature spanning six decades of youth psychotherapy RCTs. It is quite possible that other ESPCs might be identified via other methods applied to other bodies of evidence. Indeed, even the coding of our own study pool identified other groupings of specific treatment approaches (e.g., components of executive function training, bio/neurofeedback) that may warrant attention in the future. Determining whether these and other groupings of treatment procedures constitute coherent psychotherapeutic principles that are empirically supported will be an important part of the developmental agenda for an ESPC framework. The application of our process to RCTs of youth psychotherapies did yield resulting ESPCs that are transdiagnostic and transtheoretical, but they naturally reflect the nature of the evidence base in youth psychotherapy, which involves more behavioral and cognitive treatments than other approaches. Thus, there may be value in examining the relevance of ESPCs, including those identified in the current study, across treatments that are grounded in a more diverse array of theoretical orientations (e.g., experts in insight-oriented therapies may or may not similarly identify changing cognitions as an ESPC). More broadly, it may be fruitful for scholars to investigate additional candidate ESPCs that have been identified by experts in the field—such as those related to client characteristics and prognostic factors, treatment/provider factors, therapy relationship factors, and therapist intervention factors (McAleavey et al., 2019; Castonguay & Beutler, 2006)—within the context of youth psychotherapy. We are hopeful that the current article will serve as a springboard for future work dedicated to these efforts, which we suspect may uncover additional ESPCs.
We found that a majority of included treatments featured one to three ESPCs, with a considerable drop in frequency of treatments with four and five ESPCs. This pattern suggests that the youth psychotherapies included in this meta-analysis were relatively streamlined in terms of ESPCs. That said, the number of ESPCs included in treatments varied across problem domains. Whereas treatments targeting ADHD and conduct most frequently included one ESPC (increasing motivation), those targeting anxiety most frequently included two ESPCs (engaging in the positive opposite and changing unhelpful cognitions), and those targeting depression most frequently included three ESPCs (changing unhelpful cognitions, engaging in the positive opposite, and solving problems). One potential explanation for this pattern is that, relative to ADHD, conduct, and anxiety, depression has been historically more resistant to available treatments (e.g., Eckshtain et al., 2020; March et al., 2007; Weisz, Bearman, et al., 2017; Weisz et al., 2019; Weisz, Kuppens, et al., 2017). It is possible that therapies for youth depression have become relatively complex as researchers and practitioners have sought to produce benefit by adding more and more ingredients. In contrast, problems that respond more readily to treatments with fewer ESPCs (e.g., anxiety, often treated with exposure, and conduct problems, often treated with behavioral parent training) have faced less pressure to add more ESPCs. Indeed, increasing motivation (the principle underlying most behavioral parent training procedures) and engaging in the positive opposite (the principle underlying exposure treatment) emerged as the ESPCs most commonly tested as standalone treatments—a finding that may be driven partly by the success of behavioral parent training with youth conduct problems (Kaminski & Claussen, 2017) and of exposure therapy with youth anxiety (Higa-McMillan et al., 2016).
The varying representation and varied effect sizes of the different ESPCs, within the broader sample and each target problem subsample, make it interesting to consider the possibility of ESPC hierarchies that are specific to the different target problems. One possibility is that each problem domain may have its own “spotlight” ESPCs—those that are central and perhaps key to producing benefit—and “supporting” ESPCs—those that provide additional benefits when used in combination with spotlight ESPCs, perhaps depending on which co-occurring problems and comorbidities require attention. For example, within the conduct subsample, a large majority of treatments included increasing motivation, which is the most thoroughly studied ESPC for this domain (see, e.g., Leijten et al., 2019). At the same time, our findings suggest that there may be measurable added benefit to including repairing thoughts (e.g., reframing hostile attributions) and/or practicing the positive opposite (e.g., resisting the urge to respond to anger) in treatments for conduct, most often in conjunction with increasing motivation. Indeed, treatments with these ESPCs yielded statistically significant effect sizes within the medium to large range. However, repairing thoughts and practicing the positive opposite were rarely, if ever, tested alone for conduct; rather, they were most frequently tested within the context of a treatment featuring increasing motivation. Similarly, treatments for anxiety that included practicing the positive opposite, increasing motivation, calming, and/or changing unhelpful cognitions yielded medium to large posttreatment effect sizes, with increasing motivation producing a potentially meaningfully greater effect size. Importantly, increasing motivation was very rarely tested alone for anxiety. Instead, it was most frequently examined within the context of a treatment involving practicing the positive opposite (most often in the form of exposure therapy), which is the most effective and well-studied ESPC for this domain (e.g., Davis & Ollendick, 2005; Higa-McMillan et al., 2016; In-Albon & Schneider, 2007; Peris et al., 2017). Thus, practicing the opposite may be the spotlight ESPC for anxiety (i.e., the central ESPC that drives therapeutic effects), and there may be added benefits in some cases from the supporting ESPCs of increasing motivation (e.g., rewards for bravery; i.e., an ESPC that provides additional benefits when used with practicing the opposite). This possibility may be one explanation for why anxiety treatments with more than one ESPC were observed to yield relatively greater effects than those with one ESPC. For example, evidence suggests that exposure therapy for youth anxiety may be most effective when used in conjunction with other ESPCs, such as increasing motivation (e.g., Tiwari et al., 2013).
Interestingly, this pattern was different for ADHD and depression, further highlighting the unique roles that spotlight and supporting ESPCs may play for different problem domains. Specifically, among treatments for ADHD, those with increasing motivation yielded the strongest posttreatment effect sizes. Although increasing motivation was most often tested in conjunction with at least one other ESPC, it appears that this ESPC might be the spotlight for ADHD, which is consistent with evidence on effective psychosocial interventions for youth ADHD (e.g., Evans et al., 2018). Supporting ESPCs (e.g., solving problems, practicing the positive opposite) may offer gains when used with the spotlight ESPC of increasing motivation but do not appear to be driving therapeutic effects, as evidenced by their relatively small pooled posttreatment effect sizes. Finally, for depression, treatments that included calming and increasing motivation yielded the greatest posttreatment effect sizes. As noted above, treatments for depression commonly included multiple ESPCs. Consequently, it is difficult to disentangle additional possible spotlight ESPCs for this domain. Overall, these results highlight the promising scientific and clinical value of efforts to dismantle depression treatments and identify the potential spotlight ESPC (or ESPCs) for this problem domain (e.g., van den Heuvel et al., 2019).
Overall, psychotherapies with five ESPCs yielded significantly greater posttreatment effect sizes, relative to those with fewer ESPCs. This pattern was replicated in the anxiety subsample when this relationship was tested both linearly and categorically. Treatments with more ESPCs also appeared to yield nonsignificantly greater posttreatment effects within the conduct subsample, but the opposite was true for the ADHD subsample, as including more than two ESPCs in ADHD treatments did not appear to confer added benefit. Given the limited variability in the number of ESPCs included in treatments for depression, these models could not be implemented within that subsample. Thus, it appears that the effect-size advantage associated with treatments that included all five ESPCs was primarily accounted for by treatments for anxiety and conduct, which represented the largest portion of the total sample.
Taken together, these findings raise interesting questions and opportunities for future research. For example, one possibility is that there may be clinically meaningful differences in which specific ESPCs and/or the number of ESPCs needed to optimize treatment benefit within different problem domains. An alternate interpretation may be that each of the identified ESPCs offers similar benefits across problem domains, suggesting that these ESPCs may truly be transdiagnostic and transtheoretical. This issue may link the science of ESPCs in interesting ways to ongoing discussions between advocates of common factors versus specific factors in psychotherapy (see, e.g., Wampold, 2015; Wampold & Imel, 2015). Another important future empirical question will be how effectively each ESPC actually addresses and produces clinical outcomes via the goal it targets. For example, does calming lead to symptom relief via reduction of muscle tension or via some other process (see Bruijniks et al., 2019; Conrad & Roth, 2007)? Finally, empirical work dedicated to discerning differences in treatment efficacy among psychotherapies that feature different ESPCs versus those that do not could advance our understanding of the precise role that ESPCs may play in generating therapeutic benefits. We hope the current article inspires research into these and related questions.
Limitations
Our meta-analytic review has certain limitations that warrant attention. One of these is the differential representation of studies across the different problem domains, with anxiety and conduct-problem RCTs being much better represented than ADHD and depression RCTs. Although this is the nature of the study database that exists, it nonetheless reduces confidence in findings related to ADHD and depression. Second, as noted previously, although our analysis focused on five ESPCs that emerged from an iterative identification process described in the introduction and that appear particularly robust, our guess is that this is not an exhaustive list. It seems likely that additional ESPCs will be identified in the future, and that may well enrich the process we have sought to initiate in this article. The multifaceted process this identification process may require can be illustrated by considering the three most commonly noted additional candidates we found when coding the studies, as described previously: executive functioning training, general supportive counseling, and bio/neurofeedback. In addition to assessing evidence of treatment benefit for these three when tested alone, review and conceptual analysis may be required to determine whether changing executive processes or electroencephalogram patterns constitute psychotherapy and whether supportive counseling reflects a concrete change process. Finally, as noted previously, there is likely variability within and across ESPCs in terms of the number of specific treatment procedures used in each treatment (i.e., procedural density). We addressed this variability by controlling for treatment dose in our analyses, but our findings may nonetheless reflect an undetected influence of these variations. Our hope is that the current study’s focus on the conceptual density of youth psychotherapies (number of ESPCs included) will help to stimulate future work focusing on the procedural density of youth psychotherapies. For example, future studies could test the association between the procedural density and conceptual density of youth psychotherapies, as well as the potential association of each kind of density with treatment outcomes.
Conclusions
Our findings suggest that it is possible to identify ESPCs within youth psychotherapies with sufficient clarity and integrity that independent coders can detect them in studies and with a high degree of reliability. Additional findings have helped us begin to document the frequency with which the various ESPCs are included within treatments for four broad problem domains that account for most youth mental health referrals and to document the effect sizes of treatments in relation to the ESPCs they include. Differences in findings across the different target problem domains suggest the possibility of a tiered network of principles, with spotlight and supportive ESPCs for each problem area and roles for the supportive ESPCs that may vary depending on co-occurring problems and comorbidities. The broad array of findings presented here suggests that an important mission for future research, and potentially for treatment design, may be discerning which ESPCs (when used alone and in combination with other ESPCs) for which treated problems offer the best prospects for optimizing treatment benefit.
Supplemental Material
sj-docx-1-cpx-10.1177_21677026221120230 – Supplemental material for Empirically Supported Principles of Change in Youth Psychotherapy: Exploring Codability, Frequency of Use, and Meta-Analytic Findings
Supplemental material, sj-docx-1-cpx-10.1177_21677026221120230 for Empirically Supported Principles of Change in Youth Psychotherapy: Exploring Codability, Frequency of Use, and Meta-Analytic Findings by Olivia M. Fitzpatrick, Evelyn Cho, Katherine E. Venturo-Conerly, Ana M. Ugueto, Mei Yi Ng and John R. Weisz in Clinical Psychological Science
Supplemental Material
sj-pdf-2-cpx-10.1177_21677026221120230 – Supplemental material for Empirically Supported Principles of Change in Youth Psychotherapy: Exploring Codability, Frequency of Use, and Meta-Analytic Findings
Supplemental material, sj-pdf-2-cpx-10.1177_21677026221120230 for Empirically Supported Principles of Change in Youth Psychotherapy: Exploring Codability, Frequency of Use, and Meta-Analytic Findings by Olivia M. Fitzpatrick, Evelyn Cho, Katherine E. Venturo-Conerly, Ana M. Ugueto, Mei Yi Ng and John R. Weisz in Clinical Psychological Science
Footnotes
Transparency
Action Editor: Pim Cuijpers
Editor: Jennifer L. Tackett
Author Contributions
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
