Sage Journals: Discover world-class research

Abstract

Background

Most evidence-based treatments (EBTs) for posttraumatic stress disorder (PTSD) and anxiety disorders include exposure; however, in community settings, the implementation of exposure lags behind other EBT components. Clinician-level determinants have been consistently implicated as barriers to exposure implementation, but few organizational determinants have been studied. The current study examines an organization-level determinant, implementation climate, and clinician-level determinants, clinician demographic and background factors, as predictors of attitudes toward exposure and changes in attitudes following training.

Method

Clinicians (n = 197) completed a 3-day training with 6 months of twice-monthly consultation. Clinicians were trained in cognitive behavioral therapy (CBT) for anxiety, depression, behavior problems, and trauma-focused CBT (TF-CBT). Demographic and background information, implementation climate, and attitudes toward exposure were assessed in a pre-training survey; attitudes were reassessed at post-consultation. Implementation climate was measured at the aggregated/group-level and clinician-level.

Results

Attitudes toward exposure significantly improved from pre-training to post-consultation (t(193) = 9.9, p < .001; d = 0.71). Clinician-level implementation climate scores did not predict more positive attitudes at pre-training (p > .05) but did predict more positive attitudes at post-consultation (ß = −2.46; p < .05) and greater changes in those attitudes (ß = 2.28; p < .05). Group-level implementation climate scores did not predict attitudes at pre-training, post-consultation, or changes in attitudes (all ps > .05). Higher frequency of self-reported CBT use was associated with more positive attitudes at pre-training (ß = −0.81; p < .05), but no other clinician demographic or background determinants were associated with attitudes at post-consultation (all p > .05) or with changes in attitudes (all p > .05).

Conclusions

Clinician perceptions of implementation climate predicted greater improvement of attitudes toward exposure following EBT training and consultation. Findings suggest that organizational determinants outside of training impact changes in clinicians’ attitudes. Training in four EBTs, only two of which include exposure as a component, resulted in positive changes in clinicians’ attitudes toward exposure, which suggests non-specialty trainings can be effective at changing attitudes, which may enable scale-up.

Keywords

community mental health exposure evidence-based treatment anxiety posttraumatic stress disorder organizational climate

About 30% of youth in the United States have an anxiety disorder or posttraumatic stress disorder (PTSD; Merikangas et al., 2010). Exposure therapy (“exposure”), which is a treatment component of cognitive behavioral therapy (CBT) that involves approaching feared stimuli, is a key ingredient of evidence-based treatments (EBTs) of anxiety and PTSD. Exposure is included in 100% of PTSD treatment protocols (Borntrager et al., 2013) and nearly 90% of anxiety treatment protocols (Higa-McMillan et al., 2016). There is strong evidence for the efficacy of exposure in treating anxiety disorders and PTSD (Dorsey et al., 2017; Higa-McMillan et al., 2016). In treatments for anxiety disorders, the introduction of exposure tasks is associated with acceleration in treatment progress (Peris et al., 2015), larger treatment effects (Whiteside et al., 2020), greater improvements in anxiety severity and functional impairment (Peris et al., 2017), higher odds of treatment completion (Bilek et al., 2021), and greater anxiety reductions (Bilek et al., 2021). In treatments for PTSD, exposure has been shown to be more effective than psychopharmacological treatments (Morina et al., 2016) and to lower anxiety and abuse-related fear (Deblinger et al., 2011). However, exposure is rarely implemented in community mental health (CMH) settings (Becker-Haimes et al., 2017; Borntrager et al., 2013; Higa-McMillan et al., 2017; Whiteside et al., 2016; Whiteside et al., 2020; Wolitzky-Taylor et al., 2015). In CMH settings, exposure was only used in 15% of youth anxiety cases (Higa-McMillan et al., 2017) and less than 25% of youth PTSD cases (Borntrager et al., 2013).

A primary barrier to the implementation of exposure therapy in CMH settings is a lack of clinician training (Reid et al., 2017a, 2017b; Wolitzky-Taylor et al., 2018). Most clinicians receive limited or no training in exposure and report difficulty finding training opportunities (Reid et al., 2017a, 2017b; Reid et al., 2018; Wolitzky-Taylor et al., 2019). Furthermore, a one-time training may not lead to sufficient increases in attitudes, knowledge, and competency if it is not followed by ongoing consultation (Foa et al., 2020; Frank et al., 2020). A recent meta-analysis found that trainings were effective in changing knowledge, attitudes, self-efficacy, intentions to use, and use of exposure (Trivasse et al., 2020). However, studies included in this meta-analysis trained clinicians in only one EBT containing exposure and many trainings had an explicit focus on changing exposure attitudes. While these training were effective, they are limited in scope and costly, which may limit the ability to scale-up trainings to meet the large demand.

Implementation science frameworks emphasize the need to consider contextual determinants that might contribute to infrequent implementation of EBTs even following extensive training (Damschroder et al., 2009). Studies have shown that attitudes toward EBTs influence uptake and use (Aarons, 2004; Addis & Krasnow, 2000; Becker-Haimes et al., 2019; Beidas et al., 2015). Clinician-level determinants of exposure implementation have been well documented, with negative attitudes consistently emerging as barriers to high-quality delivery (Deacon et al., 2013a, 2013b). Many clinicians believe that exposures will lead to client attrition (Olatunji et al., 2009), not generalize to the real world (Feeny et al., 2003), damage emotionally fragile clients (Rosqvist, 2005), and worsen client's symptoms (Deacon et al., 2013a, 2013b). Negative attitudes toward exposure have profound and pervasive impacts on the use of exposure. Specifically, clinicians’ negative attitudes toward exposure are associated with less frequent use (de Jong et al., 2020; Reid et al., 2018; Whiteside et al., 2016), lower proficiency (Harned et al., 2013), and suboptimal delivery (Deacon et al., 2013a, 2013b; Deacon et al., 2013a, 2013b; Farrell et al., 2016). Therefore, it is important to understand what influences clinicians’ attitudes toward exposure.

Potential influences on clinician attitudes may include organization- and clinician-level factors, such as organizational climate and clinician demographics. Theoretical models of training emphasize the importance of understanding how pre-training clinician-level determinants influence the training process (McLeod et al., 2018). One study found that higher self-reported CBT use was associated with more positive attitudes toward exposure (Whiteside et al., 2016). Other studies found that female gender (Deacon et al., 2013a, 2013b), higher caseload (Becker-Haimes et al., 2017), and older age (Deacon et al., 2013a, 2013b) predicted more negative attitudes toward exposure. While most studies have been in non-CMH settings (Deacon et al., 2013a, 2013b; Whiteside et al., 2016), the current sample consists of CMH clinicians who are similar to other CMH samples in that they are largely White, female, Master's-level clinicians (Amaya-Jackson et al., 2018; Bartlett et al., 2015; Jensen-Doss et al., 2020), which suggests that results from the current study may be generalizable to other CMH contexts. Further research is necessary to understand how clinician-level determinants influence changes in attitudes during training and consultation.

Implementation science frameworks also emphasize the importance of organization-level determinants of EBT implementation (Damschroder et al., 2009), which may be especially important for difficult-to-implement EBT components like exposure. Implementation climate, the extent to which the use of an innovation is expected, supported, or rewarded within an organization (Klein & Sorra, 1996), has emerged as a potential determinant of EBT use. There is evidence that implementation climate predicts use of an EBT (Williams et al., 2018), implementation effectiveness of an innovation (Jacobs et al., 2015; Sawang & Unsworth, 2011), and attitudes toward an EBT (Powell et al., 2017). Clinicians’ individual implementation climate perspectives can be considered a determinant, or they can be aggregated to the group-level (Jacobs et al., 2015). Theory conceptualizes implementation climate as an organization-level construct (Jacobs et al., 2014; Weiner et al., 2011). However, it is also theorized that individual perceptions of implementation climate may be more appropriate for an innovation that is a smaller piece of an individual's practice (Jacobs et al., 2014; Weiner et al., 2011), such as exposure, which is only one component of EBTs for anxiety and PTSD. Some previous studies have aggregated implementation climate within organizations (Becker-Haimes et al., 2017; Ehrhart et al., 2016; Jacobs et al., 2014; Williams et al., 2018), and some have used individual perceptions (Dong et al., 2008; Jacobs et al., 2014; Kratz et al., 2019; Osei-Bryson et al., 2008). Weiner et al. (2011) offer analytic recommendations to determine the most appropriate level of analysis by assessing the extent to which clinician-level responses show sufficient agreement to justify aggregation. The current study follows the analytic recommendations of Weiner et al. (2011) for the main analysis. However, to explore when it is appropriate to use each level of analysis for implementation climate and to offer transparency in the research process, the current study will present results of implementation climate analyzed at both the individual clinician-level and the aggregated group-level as an exploratory aim. This distinction has an impact on the future conceptualization and analysis of implementation climate.

Among the few studies that have examined organizational determinants of exposure implementation, most are primarily descriptive in nature with a focus on generating potential predictors of exposure use (e.g., Becker-Haimes et al., 2020; Wolitzky-Taylor et al., 2018; Wolitzky-Taylor et al., 2019). The only study that has quantitatively examined organizational determinants found that group-level implementation climate was marginally associated with exposure use (Becker-Haimes et al., 2017). Despite the need to consider both individual- and organization-level determinants (Becker-Haimes et al., 2019), to our knowledge, no studies have quantitively examined implementation climate as a determinant of exposure attitudes. An organization with a more positive implementation climate communicates to clinicians that they expect, support, and reward the use of exposure, which may mean clinicians have more knowledge and practice using exposure, which may lead to more positive attitudes before training. In an organization with a more negative implementation climate, clinicians may be unfamiliar with and hesitant about using exposure, which may lead to more negative attitudes before training. Implementation climate may also predict changes in attitudes throughout a training initiative. For example, an organization with a more positive implementation climate may serve as a supportive environment as clinicians engage in training, receive consultation, and practice delivering exposure, which may lead to greater changes in attitudes.

Given the need to identify factors that influence clinicians’ attitudes toward exposure, we conducted a study with CMH clinicians who participated in a state-funded EBT learning collaborative (LC), which trains clinicians in four EBTs and provides 6 months of consultation. To our knowledge, this is the first study examining changes in attitudes toward exposure following a multiple-EBT training, which is more scalable than a specialty training focused on a single EBT. The current study is one of few to examine changes in attitudes toward exposure following training with consultation, which is a key ingredient for effective training (Foa et al., 2020; Frank et al., 2020). Furthermore, the current study considers both organization- and clinician-level determinants of exposure, which has been called for (Becker-Haimes et al., 2019). This study examined three pre-specified hypotheses involving key determinants of exposure at the organization-level (implementation climate) and at the clinician-level (clinicians’ attitudes and beliefs toward exposure, referred to as “attitudes”). Hypothesis 1: Given previous evidence suggesting the influence of implementation climate on clinician attitudes toward EBTs (e.g., Powell et al., 2017), we expected that higher pre-training implementation climate scores would be associated with more positive attitudes toward exposure at pre-training and greater changes in attitudes after training and consultation. Hypothesis 2: We also sought to replicate and extend previous findings (e.g., Becker-Haimes et al., 2017; Deacon et al., 2013a, 2013b; Whiteside et al., 2016) by examining clinician-level demographic and background determinants of attitudes toward exposure. We expected to replicate previous findings that being older, identifying as female, having a larger caseload, and lower self-reported CBT use would be associated with more negative attitudes toward exposure. Hypothesis 3: We follow analytic recommendations (Jacobs et al., 2014) to determine if implementation climate should be aggregated for main analyses. However, as an exploratory aim, we present the results of both aggregated and non-aggregated implementation climate perceptions to contribute to the understanding of implementation climate measurement and analysis.

Method

Procedure

We follow the preferred reporting guidelines for observational studies (STROBE) to enhance transparency (von Elm et al., 2014). Data were collected from a Washington state-funded LC, called the CBT+ Initiative (see Dorsey et al., 2016) from September 2018 to October 2019. Three-day trainings were held in five locations. Clinicians were trained in CBT for anxiety, depression, behavioral problems, and trauma-focused CBT (TF-CBT; Cohen et al., 2006). Participating organizations must have a supervisor who already completed, or was currently completing, the CBT+ training. Before the training, participants (n = 236) completed an electronic survey, which included demographic and background information, the Implementation Climate Scale (ICS), the Therapist Beliefs about Exposure Scale, and measures used for program evaluation. Informed consent was obtained from participants included in the study.

In line with best practices for training (Foa et al., 2020; Frank et al., 2020; Valenstein-Mah et al., 2020), clinicians applied CBT+ techniques and received expert consultation via telephone or video-based group calls twice-monthly for 6 months (12 calls total). Each group contained 10–15 clinicians, who were led by a CBT+ faculty member or a local stakeholder (see Triplett et al., 2020). Consultants and clinicians used an online system that allowed clinicians to enter de-identified case information (e.g., standardized assessment results, CBT+ model delivered, self-reported fidelity) and allowed consultants to track clinician attendance and progress with cases.

Clinicians completed the second electronic survey after the 6 months of consultation (n = 202, 85.6% completed both surveys). The post-consultation survey included the Therapist Beliefs about Exposure Scale and measures for program evaluation. Clinicians were emailed reminders once-a-week for 3 weeks, then were considered lost to follow-up. Values were missing from eight respondents, who did not appear to systematically differ after visual inspection of covariates. The final sample includes 194 clinicians (96.0% of the post-consultation survey response). Certificates of completion were given to clinicians who completed the pre-training and post-consultation surveys, were present for nine of the 12 group calls, presented one case during consultation, and submitted information demonstrating standardized measure use and CBT+ application with clients (judged by consultants using the data entered into the online system). University of Washington Institutional Review Board determined study activities exempt from review.

Measures

Demographics and background information

Clinicians self-reported demographics and background information including gender identity, race and ethnicity, theoretical orientation, years of experience delivering therapy, number of active cases, and frequency of CBT use. Frequency of CBT use was measured with an item at pre-training asking, “How frequently have you used cognitive behavioral therapies with your psychotherapy clients?” This item was rated on a 7-point rating scale from 0 = “Have not provided psychotherapy in the past 5 years,” 1 = “never [0% of clients],” 2 = “rarely (1–5% of clients),” 3 = “occasionally (6–25% of clients),” 4 = “sometimes (26–50% of clients),” 5 = “often (51–75% of clients),” and 6 = “Almost always (76% of clients or more).” These items were recoded to approximate an interval variable with equal spacing between responses (i.e., 1 = 0%–25% of clients [responses 1, 2, 3], 2 = 26%–50% of clients [response 4], 3 = 51%–75% of clients [response 5], and 4 = 76%–100% of clients [response 6]). The clinicians who selected response 0 were excluded (n = 7).

Attitudes toward exposure

The Therapist Beliefs about Exposure Scale (TBES) is a 21-item scale assessing participants’ negative attitudes and beliefs toward exposure therapy (“attitudes” in the current study; Deacon et al., 2013a, 2013b). For example, the scale assessed clinicians’ perceptions about the potential for vicarious traumatization, iatrogenic harm, confidentiality breaches, and damage to the clinician–client relationship. Participants indicated agreement with statements on a 5-point Likert scale, 0 = “strongly disagree” and 4 = “strongly agree.” The total score included all items summed (range = 0–84). Higher scores indicate stronger negative attitudes toward exposure therapy. The initial validation sample found the TBES to have high internal consistency (αs = .90–.96) and 6-month test–retest reliability (r = .89; Deacon et al., 2013a, 2013b). Reliability in our sample was good (a = .85).

Implementation climate

Five of six subscales (15 items) of the ICS (Ehrhart et al., 2014) assessed how much clinicians perceive that their organization expects, supports, and rewards the use of “evidence-based practices”. The full ICS has 18 items (Ehrhart et al., 2014), including six subscales. The rewards for evidence-based practice subscale was excluded because it had weaker correlations with other ICS subscales when validated in a child welfare organization context (Ehrhart et al., 2016), which may indicate publicly funded settings are rarely able to use financial incentives. All ICS items were rated on a 5-point Likert scale, 0 = “not at all” and 4 = “very great extent.” For each subscale, a mean score was computed. For the overall score, a mean of the subscale means was computed. Scores range from 0 to 4 with higher scores indicating higher levels of organizational support for EBTs. Overall scale reliability in our sample was excellent (a = .91), and subscale reliability ranged from a = .84 to a = .91.

Analytic plan

We calculated descriptive statistics to characterize our sample, including demographics and attitudes toward exposure at pre-training and post-consultation. We used paired t-tests to examine change in attitudes toward exposure from pre-training to post-consultation and present Cohen's d effect sizes.

We used linear mixed-effects regression models to determine the associations between implementation climate, clinician demographic and background determinants (i.e., age, gender, caseload, theoretical orientation, and self-reported frequency of CBT use), and attitudes toward exposure. Final linear mixed-effects models included random intercepts for organization to account for nesting of clinicians within organizations. Fixed effects were added to account for all clinician- and organization-level determinants. Clinician-level demographic and background determinants included age, identification as female (vs. all other gender identities), clinician-level implementation climate perceptions, theoretical orientation, active caseload, and self-reported frequency of CBT use. Organization-level determinants included group-level and clinician-level implementation climate. We followed an iterative model building approach for each outcome variable (pre-training attitudes, post-consultation attitudes, and change in attitudes). For each of the three models, first, only random effects for organizations were included to provide an estimate of organization and residual variance as well as intraclass correlation coefficients (ICCs) for each model. Next, clinician- and group-level determinants were run individually as fixed effects to estimate the variance in attitudes toward exposure that was attributable to clinician-level determinants. Nonsignificant effects (p > .05) were removed for parsimony. Our final models included significant determinants and a random effect for group. All analyses were conducted using the lme4 package in R.

We examined both clinician-level perceptions of implementation climate and group-level perceptions of implementation climate as determinants of clinicians’ attitude toward exposure in separate models. For clinician-level implementation climate, we utilized the Implementation Climate Scale (ICS) mean score for that clinician. For group-level implementation climate, we averaged the clinician-level ratings within organizations with three or more respondents. Because it is not yet known which level of implementation climate measurement is most appropriate for assessing the impact on clinicians’ attitudes toward exposure, we followed Weiner et al.’s (2011) recommendations and examined the ICC(1,1), ICC(1,k), r_wg(J), and eta-squared of implementation climate ratings before analyzing the impact of either level on attitudes toward exposure (for more information on ICC, see Shrout & Fleiss, 1979). Our ICCs indicated that most of the variance in implementation climate was at the individual level: (ICC(1,1) = 0.085 and ICC(1,k) = 0.20). Some studies recommend a cut-off of 0.80 to determine if data are homogenous enough to justify aggregation (Lance, 2006), which would indicate an aggregated implementation climate score may not accurately represent our clinicians’ “shared” perceptions. The average r_wg(J) values for all groups was 0.59, and r_wg(J) values ranged from −0.35 to 0.87, which indicates there was substantial heterogeneity in within-group agreement between groups. Eta-squared was 0.42, suggesting 42% of variance in implementation climate scores is attributable to group; however, this estimate may be biased and overestimate variance (Bliese, 1998). These analyses led us to conclude that our data are best represented through clinician-level implementation climate perceptions, however, previous literature has often included aggregated implementation climate scores for organizations with more than three respondents. Thus, we present the results of implementation climate at both the clinician- and group-level and consider both to be measuring an organization-level construct based on theory underlying the measure (Jacobs et al., 2014; Weiner et al., 2011).

Results

Demographics and descriptives

Participants (N = 194) were predominantly female (79.9%, n = 155) and White (65.5%, n = 127). Most had a Master's degree (95.9%, n = 186). Nearly half identified cognitive-behavioral as their primary theoretical orientation (40.2%; n = 78), with the next most common being integrative or eclectic (23.2%; n = 45). On average, participants were 35.4 (SD = 10.1) years old with 3.5 (SD = 4.3) years of experience providing psychotherapy. The average active caseload was 27.6 clients (SD = 17.6). Clinicians represented 72 organizations. The average clinician-level implementation climate rating was 2.7 (SD = 0.6; range of scale: 0–4), and average group-level implementation climate for organizations with three or more participants (n = 29, 40.3% of organizations) was also 2.7 (SD = 0.3). The clinician- and group-level implementation climate scores indicate that participants perceived their organizations to expect, support, and reward the use of “evidence-based practices” at a slight-to-moderate extent. The average TBES score at pre-training was 38.3 (SD = 6.6; range of scale: 0–84). Organization-level random effects accounted for 2.5% of the total variance in TBES at pre-training. The average TBES score at post-consultation was 33.0 (SD = 8.5). Organization-level random effects accounted for 11.2% of the total variance at post-consultation. In our sample, TBES scores significantly decreased from pre-training to post-consultation, t(193) = 9.9, p < .001; d = 0.71 (Figure 1). Full demographic and descriptive statistics are included in Table 1. Correlations between all individual- and organization-level variables are shown in Table 2.

Figure 1.
Clinician attitudes toward exposure at pre-training and post-consultation. Higher scores indicate stronger negative attitudes toward exposure therapy.

Table 1.
Demographic information.

Variable n (194)* Percentage

Race/Ethnicity

Asian/Asian American 11 5.67%

Native American/Alaska Native 2 1.03%

Black/African American 5 2.58%

Native Hawaiian/Other Pacific Islander 1 0.52%

White 127 65.46%

Multiracial 16 8.25%

Hispanic/Latin(a/o)/Latinx 27 13.92%

Other 5 2.58%

Gender

Female 155 79.90%

Male 31 15.98%

Transgender/Nonbinary 5 2.58%

Self-described 1 0.52%

Prefer not to say 2 1.03%

Academic degree

Bachelor's level 4 2.06%

Master's level 186 95.88%

PhD/PsyD 1 0.52%

Other 3 1.55%

Theoretical oientation

Integrative/Eclectic 45 23.20%

Cognitive Behavioral 78 40.21%

Humanistic/Essential 22 11.34%

Psychodynamic/Analytic 5 2.58%

Interpersonal 12 6.19%

Systems 18 9.28%

Other 14 7.22%

M SD

Age 35.36 10.13

Years in field 3.48 4.26

Caseload size 27.60 17.58

Individual clinician-level implementation climate (n = 194) 2.73 0.59

Aggregated group-level implementation climate (n = 128) 2.70 0.34

Table 2.
Pearson correlation coefficients between individual- and organization-level variables.

Variable 1 2 3 4 5 6 7 8 9 10

1. Age —

2. Gender −0.04 —

3. Caseload 0.03 −0.02 —

4. Theoretical orientation −0.02 0.04 0.12 —

5. Frequency of CBT use 0.16* 0.15* 0.25* 0.33* —

6. Pre-training attitudes −0.08 0.05 −0.01 −0.04 −0.13 —

7. Post-consultation attitudes −0.1 0.01 0.04 −0.02 −0.08 0.54* —

8. Change in attitudes −0.04 −0.03 0.06 0.01 0.03 −0.28* 0.66*** —

9. Individual-level ICS −0.12 0.04 0.09 0.05 0.11 −0.01 −0.14 −0.15* —

10. Organization-level ICS −0.15 0.22* 0.01 0.1 0.07 0.12 0.08 −0.01 0.55*** —

p < .05. p < .01. p < .001.

Note. CBT = cognitive behavioral scale; ICS = Implementation Climate Scale.

Implementation climate and exposure

Single-factor model associations between implementation climate and attitudes toward exposure are presented in Table 3. At pre-training, neither clinician- nor group-level implementation climate scores were associated with TBES scores (both p > .05). At post-consultation, higher clinician-level implementation climate scores were associated with lower TBES scores (i.e., fewer negative attitudes; ß = −2.21; p < .05), but group-level implementation climate scores were not associated with TBES scores (p > 0.05). Following a similar pattern, higher clinician-level implementation climate scores were also associated with greater changes in attitudes toward exposure from pre-training to post-consultation (ß = −2.07; p < .05; Figure 2), but group-level implementation climate scores were not (p > .05).

Figure 2.
Changes in clinician attitudes toward exposure by clinician-level implementation climate scores. Higher attitudes scores indicate stronger negative attitudes toward exposure therapy.

Table 3.
Single factor models examining associations between implementation climate and clinician attitudes toward exposure.

Determinant Pre-training attitudes Post-consultation attitudes Change in attitudes

ß SE p 95% CI ß SE p 95% CI ß SE p 95% CI

Clinician-level implementation climate(n = 194) −0.12 0.80 0.89 [−1.68, 1.46] −2.21* 1.01 0.03 [−4.20, −0.21] −2.07* 0.89 0.02 [−3.81, −0.32]

Group-level implementation climate(n = 128) 2.22 1.71 0.21 [−1.12, 5.54] 1.79 2.56 0.49 [−3.24, 6.76] −0.66 2.33 0.78 [−5.26, 3.88]

p <0.05.

Clinician-level determinants and exposure

Single-factor model associations between clinician demographic and background determinants (i.e. age, caseload, gender, theoretical orientation, and self-reported frequency of CBT use) and attitudes are presented in Table 4. At pre-training, higher rates of self-reported CBT use were associated with lower TBES scores (i.e., fewer negative attitudes; ß = −1.04; p < .05). At pre-training, no other clinician demographic or background determinants were associated with TBES (all p > .05). At post-consultation, no clinician demographic or background determinants were associated with TBES (all p > .05). No clinician demographic or background determinants were associated with changes in TBES scores (all p > .05).

Table 4.
Single factor models examining associations between clinician determinants and clinician attitudes toward exposure.

Determinant Pre-training attitudes Post-consultation attitudes Change in attitudes

ß SE p 95% CI ß SE p 95% CI ß SE p 95% CI

Age(n = 194) −0.05 0.05 0.25 [−0.15, 0.04] −0.08 0.06 0.19 [−0.20, 0.04] −0.03 0.05 0.60 [−0.13, 0.07]

Gender(n = 194) 0.86 1.18 0.47 [−1.41, 3.21] 0.15 1.52 0.92 [−2.84, 3.14] −0.43 1.34 0.75 [−3.06, 2.18]

Caseload(n = 193) 0.00 0.03 0.93 [−0.06, 0.05] 0.02 0.04 0.50 [−0.05, 0.09] 0.02 0.03 0.43 [−0.04, 0.09]

Theoretical orientation(n = 194) −0.58 0.96 0.55 [−2.47, 1.31] −0.46 1.23 0.71 [−2.87, 1.95] 0.11 1.08 0.92 [−2.00, 2.23]

Frequency of CBT use(n = 186) −1.04 0.49 0.03 [−2.02, −0.05] −0.90 0.64 0.16 [−2.16, 0.37] 0.18 0.56 0.75 [−0.91, 1.27]

p <0.05.

Note. CBT = cognitive behavioral therapy.

Discussion

Efforts to support clinicians delivering EBTs in CMH settings have highlighted the need to address determinants across multiple levels, especially for difficult-to-implement and infrequently implemented treatment components, such as exposure. Results from this study indicate that organizational determinants within a clinician's agency, and outside of the training process, may influence how much clinicians’ attitudes toward exposure change throughout training and consultation. Higher clinician-level pre-training implementation climate scores were not associated with more positive attitudes toward exposure at pre-training but were associated with more positive attitudes at post-consultation. Furthermore, higher clinician-level pre-training implementation climate scores were associated with greater changes* in attitudes toward exposure from pre-training to post-consultation. Implementation climate was found to be a potentially helpful factor for improving clinicians’ attitudes toward exposure; however, in our sample, implementation climate only influenced attitudes in the presence of training and consultation. As such, implementation climate appears to have acted as a helpful, but not sufficient, factor. Given that negative attitudes toward exposure have consistently been associated with less frequent (de Jong et al., 2020; Reid et al., 2018; Whiteside et al., 2016) and less competent use of exposure (Deacon et al., 2013a, 2013b; Deacon et al., 2013a, 2013b; Farrell et al., 2016; Harned et al., 2013), it is important to identify factors that may influence these attitudes over time and in different organizational climates.

The current study provides evidence for an improvement in attitudes toward exposure following a non-specialty training, which offers a less resource-intensive alternative to meet the large demand for exposure training among CMH clinicians (Wolitzky-Taylor et al., 2019). While other studies have had success in changing attitudes toward exposure, prior studies focused on training clinicians in only one EBT containing exposure (Trivasse et al., 2020). Prior to the current study, the only work investigating the effects of a multiple EBT training on intentions to use exposure found that clinicians had the lowest intentions to deliver exposure compared to other CBT components (Wolk et al., 2019). Even in modular treatments designed to be responsive to comorbidities and flexible in implementation (Chorpita et al., 2005; Weisz et al., 2012), exposure was the least used therapeutic component at a 5-year follow-up, which suggests some difficulty or discomfort with the implementation of exposure (Thomassin et al., 2019). In contrast to prior studies, the current study did not explicitly focus on addressing attitudes toward exposure and included training for four EBTs, of which only two include exposure as a treatment component.

We predicted that higher pre-training implementation climate would lead to more positive attitudes toward exposure at pre-training. However, we found that higher pre-training implementation climate did not predict more positive attitudes at pre-training but did predict more positive attitudes at post-consultation and greater changes in attitudes throughout training and consultation. While beyond the scope of the current study, we theorize that there is some interplay between implementation climate and skill development. For example, this pattern of results could reflect that a higher implementation climate does not necessarily shape attitudes before training, but rather it functions (1) to increase clinicians’ motivation to learn and practice EBT skills following the training and during the consultation period and (2) as a supportive environment while clinicians practice new skills in the context of workplace-based supervision and consultation. Both processes could lead to more positive attitudes toward exposure at post-training for clinicians who perceive more positive implementation climates in their organization. Future work should seek to investigate the interplay between skill development and implementation climate to understand whether implementation climate functions as a mechanism underlying clinician behavior change.

While item-level analyses were not conducted in the current study, we can speculate about how some of the TBES items are likely to interact at the organization level. For instance, if participants are in an organization where using exposure and/or other EBPs is the norm, they are perhaps less likely to worry about damage to the clinician-client relationship from exposure. If they have peers and/or supervisors who also use exposure, they would be able to consult them regarding the ethics of exposure (Becker-Haimes et al., 2020). Future work should investigate changes in specific attitudes toward exposure and the association with implementation climate.

The current study examined clinician demographic and background factors predicting attitudes toward exposure. At pre-training, more positive attitudes toward exposure were predicted only by higher ratings of self-reported CBT use. Additionally, no clinician-level determinants were associated with attitudes at post-consultation or with changes in attitudes. While another study also found that higher self-reported CBT use was associated with more positive attitudes toward exposure (Whiteside et al., 2016), other studies found that gender (Deacon et al., 2013a, 2013b), caseload (Becker-Haimes et al., 2017), and age (Deacon et al., 2013a, 2013b) predicted attitudes toward exposure as well, but those findings were not replicated. Most clinicians in Deacon et al. (2013a) (2013b) worked in private practice or hospital settings, and there were more doctoral-level and male clinicians than in our sample. Compared to Becker-Haimes et al. (2017), the current study had less experienced and less racially diverse clinicians. In general, our sample may be more representative of the average CMH clinician population, including largely Master's-level, White, female clinicians (see Table 1; Amaya-Jackson et al., 2018; Bartlett et al., 2015; Jensen-Doss et al., 2020), but our sample may have less years of experience than other studies (Bartlett et al., 2015; Becker-Haimes et al., 2017; Jensen-Doss et al., 2020). Although not compared statistically, our sample had more negative attitudes toward exposure on average than previous samples in non-CMH settings (Deacon et al., 2013a, 2013b; Farrell et al., 2016; Kaye, 2018; Reid et al., 2017a, 2017b; Whiteside et al., 2016). Although the current sample of CMH clinicians had more negative attitudes toward exposure initially, a training that focused on four EBTs with no explicit emphasis on addressing exposure attitudes was still effective at significantly improving attitudes toward exposure (Cohen's d = 0.71).

To contribute understanding when it is appropriate to examine individual versus aggregated perceptions of implementation climate, we report both the group-level and individual-level perceptions. The variation in group-level implementation climate scores indicates clinicians did not highly agree on their perceptions, which means that the aggregated implementation climate ratings could represent few clinicians’ actual perceptions. Theoretically, it makes sense that clinician-level perceptions were most valid because, in this study, exposure was a small piece of an individual's practice (one component in only two of the four EBTs taught; Jacobs et al., 2014). Group differences within organizations may have emerged due to the number of respondents within an organization. Because our data is collected as part of a state-funded training which aimed to include as many organizations as possible, organizations could only send a maximum of three clinicians and one supervisor to the training. As such, only 40% of the organizations in our study had enough respondents to be aggregated (66% of the clinician sample), which could have influenced our ability to fully compare clinician versus group perceptions. Additionally, emerging literature shows clinical supervisors play a role in shaping clinicians’ perceptions about their organization (Birken et al., 2018; Meza et al., 2021). Thus, clinicians having different supervisors may have contributed to the variability of climate perceptions within each organization. More work must be done to distinguish what constructs are being measured with individual versus group implementation climate perceptions to ensure high construct validity and appropriate future use. This distinction has an impact on the conceptualization and approach to analyzing implementation climate across disciplines.

The current study has several strengths. To our knowledge, this is the first study to investigate changes in attitudes toward exposure following a multiple-EBT training, which is more scalable than a specialty training. We are also among the first to examine changes in attitudes toward exposure following training with consultation, which is a key ingredient for effective training (Frank et al., 2020). We use both organization- and clinician-level determinants of exposure, which has been called for in the literature (Becker-Haimes et al., 2019). Second, this study included a large and representative sample of CMH clinicians from many different organizations in Washington participating in a state-funded training. This sample is similar to other CMH training studies (Amaya-Jackson et al., 2018; Bartlett et al., 2015; Jensen-Doss et al., 2020), which suggests that results may be generalizable to other CMH contexts. Third, the current study focuses on clinician perceptions of implementation climate following analysis of ICCs as recommended (Jacobs et al., 2014), but we also report the results based on group-level implementation climate perceptions. This allows us to contribute to discussion around appropriate future measurement and analysis of implementation climate. Lastly, a strength of this study is the assessment of attitudes toward a specific EBT component, exposure, which has strong evidence for effectiveness (Higa-McMillan et al., 2019).

In addition to its strengths, this study also has several limitations. First, we did not examine cross-level interactions between organization and individual characteristics, thus, we cannot rule out the complex interplay between determinants across levels (Becker-Haimes et al., 2019). Although other studies demonstrate that attitudes impact delivery for exposure (e.g., de Jong et al., 2020), we had no measure of exposure use after the consultation period ends, so we cannot be confident that improved attitudes led to behavior change. Measurement was limited by the fact that data were collected as part of an evaluation of a state-funded training. Since this study relies on clinician self-report, the data are subject to self-report biases (Borntrager et al., 2015; Donaldson & Grant-Vallone, 2002; Eccles et al., 2006). Additionally, our sample included many CBT-leaning clinicians, which may differ from other CMH contexts. Due to the constraints of being a state-funded training initiative aiming to include as many organizations as possible, clinicians represented 72 organizations in our study, but only 29 organizations (40.3%) had three or more respondents necessary to aggregate implementation climate to the group-level, which may have affected ICC analysis indicating that clinician-level analyses were most appropriate. Next, implementation climate was only measured at pre-training, so we cannot rule out the possibility that climate shifted throughout the training, and that that shift led to attitude changes. However, it is somewhat unlikely that the implementation climate shifted significantly over the 6-month study period, especially given that only four people from each organization (maximum) could attend the training. Lastly, the implementation climate measure assessed climate for evidence-based practices broadly while the attitudinal measure assessed attitudes toward exposure specifically. Future research may want to examine the relation between clinician perceptions of implementation climate for specific EBT components like exposure and attitudes toward those specific components.

Building on the results of this study, we recommend four future directions for further investigation. First, future work could examine the individual items on the TBES scale, which each assess a specific exposure attitude. For example, the changes in specific attitudes and the relationship between specific attitudes and implementation climate could offer insights for targeted research and intervention. Second, few studies have examined if and how changes in attitudes toward exposure translates to clinical practice (Trivasse et al., 2020). Future work should seek to measure exposure use during and after training. Third, long-term follow-up studies assessing clinicians’ practice following training and consultation are necessary to fully understand the effectiveness of training efforts by evaluating if practices are sustained (Chu et al., 2015; Thomassin et al., 2019). Fourth, future work should investigate how to improve organizational climate since it may be important in shaping clinicians’ attitudes (Powell et al., 2017), behaviors (Williams et al., 2018), and effectiveness of training and consultation. Organizational interventions have shown success by reducing clinician turnover (Glisson et al., 2006), improving EBT adoption and use (Williams et al., 2017), and improving client outcomes (Glisson et al., 2010). Interventions aimed at enhancing implementation climate specifically are currently being evaluated (e.g., Aarons et al., 2015; Aarons et al., 2017).

Conclusions

Training in four EBTs, only two of which include exposure as a component, resulted in positive changes in clinicians’ attitudes toward exposure. These results suggest that non-specialty trainings may provide a less resource-intensive alternative to address clinician's attitudes about exposure than specialized trainings. Further, CMH clinicians who endorsed a higher implementation climate within their organization had more positive attitudes toward exposure following EBT training and consultation. Clinicians who perceived higher implementation climates also showed greater improvements in attitudes after EBT training and consultation. These findings suggest that organizational determinants within a clinician's agency, and outside of the training and consultation process, may influence the changes in attitudes that happen throughout training, treatment implementation, and consultation.

Variable	n (194)	Percentage
Race/Ethnicity
Asian/Asian American	11	5.67%
Native American/Alaska Native	2	1.03%
Black/African American	5	2.58%
Native Hawaiian/Other Pacific Islander	1	0.52%
White	127	65.46%
Multiracial	16	8.25%
Hispanic/Latin(a/o)/Latinx	27	13.92%
Other	5	2.58%
Gender
Female	155	79.90%
Male	31	15.98%
Transgender/Nonbinary	5	2.58%
Self-described	1	0.52%
Prefer not to say	2	1.03%
Academic degree
Bachelor's level	4	2.06%
Master's level	186	95.88%
PhD/PsyD	1	0.52%
Other	3	1.55%
Theoretical oientation
Integrative/Eclectic	45	23.20%
Cognitive Behavioral	78	40.21%
Humanistic/Essential	22	11.34%
Psychodynamic/Analytic	5	2.58%
Interpersonal	12	6.19%
Systems	18	9.28%
Other	14	7.22%
	M	SD
Age	35.36	10.13
Years in field	3.48	4.26
Caseload size	27.60	17.58
Individual clinician-level implementation climate (n = 194)	2.73	0.59
Aggregated group-level implementation climate (n = 128)	2.70	0.34

Variable	1	2	3	4	5	6	7	8	9	10
1. Age	—
2. Gender	−0.04	—
3. Caseload	0.03	−0.02	—
4. Theoretical orientation	−0.02	0.04	0.12	—
5. Frequency of CBT use	0.16*	0.15*	0.25***	0.33***	—
6. Pre-training attitudes	−0.08	0.05	−0.01	−0.04	−0.13	—
7. Post-consultation attitudes	−0.1	0.01	0.04	−0.02	−0.08	0.54***	—
8. Change in attitudes	−0.04	−0.03	0.06	0.01	0.03	−0.28***	0.66***	—
9. Individual-level ICS	−0.12	0.04	0.09	0.05	0.11	−0.01	−0.14	−0.15*	—
10. Organization-level ICS	−0.15	0.22*	0.01	0.1	0.07	0.12	0.08	−0.01	0.55***	—

Determinant	Pre-training attitudes	Post-consultation attitudes	Change in attitudes
Clinician-level implementation climate(n = 194)	−0.12	0.80	0.89	[−1.68, 1.46]	−2.21*	1.01	0.03	[−4.20, −0.21]	−2.07*	0.89	0.02	[−3.81, −0.32]
Group-level implementation climate(n = 128)	2.22	1.71	0.21	[−1.12, 5.54]	1.79	2.56	0.49	[−3.24, 6.76]	−0.66	2.33	0.78	[−5.26, 3.88]

Determinant	Pre-training attitudes	Post-consultation attitudes	Change in attitudes
Age(n = 194)	−0.05	0.05	0.25	[−0.15, 0.04]	−0.08	0.06	0.19	[−0.20, 0.04]	−0.03	0.05	0.60	[−0.13, 0.07]
Gender(n = 194)	0.86	1.18	0.47	[−1.41, 3.21]	0.15	1.52	0.92	[−2.84, 3.14]	−0.43	1.34	0.75	[−3.06, 2.18]
Caseload(n = 193)	0.00	0.03	0.93	[−0.06, 0.05]	0.02	0.04	0.50	[−0.05, 0.09]	0.02	0.03	0.43	[−0.04, 0.09]
Theoretical orientation(n = 194)	−0.58	0.96	0.55	[−2.47, 1.31]	−0.46	1.23	0.71	[−2.87, 1.95]	0.11	1.08	0.92	[−2.00, 2.23]
Frequency of CBT use(n = 186)	−1.04*	0.49	0.03	[−2.02, −0.05]	−0.90	0.64	0.16	[−2.16, 0.37]	0.18	0.56	0.75	[−0.91, 1.27]

Footnotes

Acknowledgments

This publication was made possible in part by funding from the Washington State Department of Social and Health Services, Division of Behavioral Health and Recovery, and the trainers and community partners for the CBT+ Initiative, Lucy Berliner, Dr. Nathaniel Jungbluth, Amanda Russell, Nikki Sharp, Sean Wright, Dan Fox, Naomi Leong, Melissa Gorsuch-Clark, Amber Ard, Dr. Georganna Sedlar, Angelica De Anda, Jamilyn Ellis, Derrick Conley, Rachel Barrett, Cassandra Ellsworth, Louisa Hall, Laura Merchant, Dr. Minu Ranna-Stewart, Jennifer Evinger, Crystal Hynek, and Claudia Kirkland. Furthermore, as a co-founder and former co-director of the CBT+ Initiative, Lucy Berliner has expertly served as a liaison between government agencies, community stakeholders, and researchers around the country to bring high-quality, accessible mental health treatment for children and families across Washington State, which made this and many other studies possible.

Declaration of conflicting interests

Dr. Dorsey was paid by the Washington State Department of Social and Health Services, Division of Behavioral Health and Recovery to serve as a trainer, consultant, and evaluator for the activities outlined in this study.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Grace S. Woodard

Noah S. Triplett

Shannon Dorsey

Plain language summary

Exposure is highly effective for treating trauma symptoms and anxiety-based disorders, but it is not commonly used in community mental health settings. Clinicians who endorsed higher expectations, support, and rewards for using exposure in their agency had more positive attitudes toward exposure after training and consultation. Additionally, clinicians who endorsed that exposure is expected, supported, and rewarded in their agency showed a greater improvement in attitudes throughout the training process. Organizational culture can affect clinicians’ attitude changes in the training process, and therefore should become a focus of training efforts.

References

Aarons

G. A.

(2004). Mental health provider attitudes toward adoption of evidence-based practice: The evidence-based practice attitude scale (EBPAS). Mental Health Services Research, 6(2), 61–74. https://doi.org/10.1023/B:MHSR.0000024351.12294.65

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

Hurlburt

M. S.

(2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 1–12. https://doi.org/10.1186/s13012-014-0192-y

Aarons

G. A.

Ehrhart

M. G.

Moullin

J. C.

Torres

E. M.

Green

A. E.

(2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implementation Science, 12(1), 29. https://doi.org/10.1186/s13012-017-0562-3

Addis

M. E.

Krasnow

A. D.

(2000). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68(2), 331–339. https://doi.org/10.1037//0022-006X.68.2.331

Amaya-Jackson

Hagele

Sideris

Potter

Briggs

E. C.

Keen

Murphy

R. A.

Dorsey

Patchett

Ake

G. S.

Socolar

(2018). Pilot to policy: Statewide dissemination and implementation of evidence-based treatment for traumatized youth. BMC Health Services Research, 18(1), 1–14. https://doi.org/10.1186/s12913-018-3395-0

Bartlett

J. D.

Barto

Griffin

J. L.

Fraser

J. G.

Hodgdon

Bodian

(2015). Trauma-informed care in the massachusetts child trauma project. Child Maltreatment, 21(2), 101–112. https://doi.org/10.1177/1077559515615700

Becker-Haimes

E. M.

Byeon

Y. V.

Frank

H. E.

Williams

N. J.

Kratz

H. E.

Beidas

R. S.

(2020). Identifying the organizational innovation-specific capacity needed for exposure therapy. Depression and Anxiety, April 2019, 37(10), 1–10. https://doi.org/10.1002/da.23035

Becker-Haimes

E. M.

Okamura

K. H.

Wolk

C. B.

Rubin

Evans

A. C.

Beidas

R. S.

(2017). Predictors of clinician use of exposure therapy in community mental health settings. Journal of Anxiety Disorders, 49, 88–94. https://doi.org/10.1016/j.janxdis.2017.04.002

Becker-Haimes

E. M.

Williams

N. J.

Okamura

K. H.

Beidas

R. S.

(2019). Interactions between clinician and organizational characteristics to predict cognitive-behavioral and psychodynamic therapy use. Administration and Policy in Mental Health and Mental Health Services Research, 46(6), 701–712. https://doi.org/10.1007/s10488-019-00959-6

10.

Beidas

R. S.

Marcus

Aarons

G. A.

Hoagwood

K. E.

Schoenwald

Evans

A. C.

Hurford

M. O.

Hadley

Barg

F. K.

Walsh

L. M.

Adams

D. R.

Mandell

D. S.

(2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382. https://doi.org/10.1001/jamapediatrics.2014.3736

11.

Bilek

Tomlinson

R. C.

Whiteman

A. S.

Johnson

T. D.

Benedict

Luan Phan

Monk

C. S.

Fitzgerald

K. D.

(2021). Exposure-focused CBT outperforms relaxation-based control in an RCT of treatment for child and adolescent anxiety. Journal of Clinical Child & Adolescent Psychology, 1–9. https://doi.org/10.1080/15374416.2021.1901230

12.

Birken

Clary

Tabriz

A. A.

Turner

Meza

Zizzi

Larson

Walker

Charns

(2018). Middle managers’ role in implementing evidence-based practices in healthcare: A systematic review. Implementation Science, 13(1), 1–14. https://doi.org/10.1186/s13012-018-0843-5

13.

Bliese

(1998). Group size and measures of group-level properties: An examination of eta-squared and ICC values. Journal of Management, 24(2), 157–172. https://doi.org/10.1016/s0149-2063(99)80058-0

14.

Borntrager

Chorpita

B. F.

Higa-McMillan

C. K.

Daleiden

E. L.

Starace

(2013). Usual care for trauma-exposed youth: Are clinician-reported therapy techniques evidence-based? Children and Youth Services Review, 35(1), 133–141. https://doi.org/10.1016/j.childyouth.2012.09.018

15.

Borntrager

C. F.

Chorpita

B. F.

Orimoto

Love

Mueller

C. W.

(2015). Validity of clinician’s self-reported practice elements on the monthly treatment and progress summary. The Journal of Behavioral Health Services & Research, 42(3), 367–382. https://doi.org/10.1007/s11414-013-9363-x

16.

Chorpita

B. F.

Daleiden

E. L.

Weisz

J. R.

(2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7(1), 5–20. https://doi.org/10.1007/s11020-005-1962-6

17.

Chu

B. C.

Crocco

S. T.

Arnold

C. C.

Brown

Southam-Gerow

M. A.

Weisz

J. R.

(2015). Sustained implementation of cognitive-behavioral therapy for youth anxiety and depression: Long-term effects of structured training and consultation on therapist practice in the field. Professional Psychology: Research and Practice, 46(1), 70–79. https://doi.org/10.1037/a0038000

18.

Cohen

J. A.

Mannarino

A. P.

Deblinger

(2006). Treating trauma and traumatic grief in children and adolescents. Guilford Press.

19.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50

20.

Deacon

B. J.

Farrell

N. R.

Kemp

J. J.

Dixon

L. J.

J. T.

Zhang

A. R.

McGrath

P. B.

(2013a). Assessing therapist reservations about exposure therapy for anxiety disorders: The therapist beliefs about exposure scale. Journal of Anxiety Disorders, 27(8), 772–780. https://doi.org/10.1016/j.janxdis.2013.04.006

21.

Deacon

B. J.

Lickel

J. J.

Farrell

N. R.

Kemp

J. J.

Hipol

L. J.

(2013b). Therapist perceptions and delivery of interoceptive exposure for panic disorder. Journal of Anxiety Disorders, 27(2), 259–264. https://doi.org/10.1016/j.janxdis.2013.02.004

22.

Deblinger

Mannarino

A. P.

Cohen

J. A.

Runyon

M. K.

Steer

R. A.

(2011). Trauma-focused cognitive behavioral therapy for children: Impact of the trauma narrative and treatment length. Depression and Anxiety, 28(1), 67–75. https://doi.org/10.1002/da.20744

23.

de Jong

Lommen

M. J. J.

van Hout

W. J. P. J.

de Jong

P. J.

Nauta

M. H.

(2020). Therapists’ characteristics associated with the (non-)use of exposure in the treatment of anxiety disorders in youth: A survey among Dutch-speaking mental health practitioners. Journal of Anxiety Disorders, 73(May), 102230. https://doi.org/10.1016/j.janxdis.2020.102230

24.

Donaldson

S. I.

Grant-Vallone

E. J.

(2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245–260. https://doi.org/10.1023/A:1019637632584

25.

Dong

Neufeld

D. J.

Higgins

(2008). Testing klein and sorra’s innovation implementation model: An empirical examination. Journal of Engineering and Technology Management, 25(4), 237–255. https://doi.org/10.1016/j.jengtecman.2008.10.006

26.

Dorsey

Berliner

Lyon

A. R.

Pullmann

M. D.

Murray

L. K.

(2016). A statewide common elements initiative for children's mental health. The Journal of Behavioral Health Services & Research, 43(2), 246–261. https://doi.org/10.1007/s11414-014-9430-y

27.

Dorsey

McLaughlin

K. A.

Kerns

S. E. U.

Harrison

J. P.

Lambert

H. K.

Briggs

E. C.

Revillion Cox

Amaya-Jackson

(2017). Evidence base update for psychosocial treatments for children and adolescents exposed to traumatic events. Journal of Clinical Child and Adolescent Psychology, 46(3), 303–330. https://doi.org/10.1080/15374416.2016.1220309

28.

Eccles

M. P.

Hrisos

Francis

Kaner

E. F.

Dickinson

H. O.

Beyer

Johnston

(2006). Do self-reported intentions predict clinicians’ behaviour: A systematic review. Implementation Science, 1, 1–10. https://doi.org/10.1186/1748-5908-1-28

29.

Ehrhart

M. G.

Aarons

G. A.

Farahnak

L. R.

(2014). Assessing the organizational context for EBP implementation: The development and validity testing of the implementation climate scale (ICS). Implementation Science, 9(1), 157. https://doi.org/10.1186/s13012-014-0157-1

30.

Ehrhart

M. G.

Torres

E. M.

Wright

L. A.

Martinez

S. Y.

Aarons

G. A.

(2016). Validating the implementation climate scale (ICS) in child welfare organizations. Child Abuse and Neglect, 53, 17–26. https://doi.org/10.1016/j.chiabu.2015.10.017

31.

Farrell

N. R.

Kemp

J. J.

Blakey

S. M.

Meyer

J. M.

Deacon

B. J.

(2016). Targeting clinician concerns about exposure therapy: A pilot study comparing standard vs. Enhanced training. Behaviour Research and Therapy, 85, 53–59. https://doi.org/10.1016/j.brat.2016.08.011

32.

Feeny

N. C.

Hembree

E. A.

Zoellner

L. A.

(2003). Myths regarding exposure therapy for PTSD. Cognitive and Behavioral Practice, 10(1), 85–90. https://doi.org/10.1016/S1077-7229(03)80011-1

33.

Foa

E. B.

McLean

C. P.

Brown

L. A.

Zang

Rosenfield

Zandberg

L. J.

Ealey

Hanson

B. S.

Hunter

L. R.

Lillard

I. J.

Patterson

T. J.

Rosado

Scott

Weber

Wise

J. E.

Zamora

C. D.

Mintz

Young-Mccaughan

Peterson

A. L

. (2020). The effects of a prolonged exposure workshop with and without consultation on provider and patient outcomes: A randomized implementation trial. Implementation Science, 15(1), 1–14. https://doi.org/10.1186/s13012-020-01014-x

34.

Frank

H. E.

Becker-Haimes

E. M.

Kendall

P. C.

(2020). Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice, December 2019, 27(3), 1–30. https://doi.org/10.1111/cpsp.12330

35.

Glisson

Dukes

Green

(2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse & Neglect, 30(8), 855–880. https://doi.org/10.1016/j.chiabu.2005.12.010

36.

Glisson

Schoenwald

S. K.

Hemmelgarn

Green

Dukes

Armstrong

K. S.

Chapman

J. E.

(2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 535–550. https://doi.org/10.1037/a0019160

37.

Harned

M. S.

Dimeff

L. A.

Woodcock

E. A.

Contreras

(2013). Predicting adoption of exposure therapy in a randomized controlled dissemination trial. Journal of Anxiety Disorders, 27(8), 754–762. https://doi.org/10.1016/j.janxdis.2013.02.006

38.

Higa-McMillan

C. K.

Ebesutani

Stanick

C. F.

(2019). What therapy practices do providers value in youth behavioral health? A measure development study. Journal of Behavioral Health Services and Research, 46(4), 607–624. https://doi.org/10.1007/s11414-019-09651-x

39.

Higa-McMillan

C. K.

Francis

S. E.

Rith-Najarian

Chorpita

B. F.

(2016). Evidence base update: 50 years of research on treatment for child and adolescent anxiety. Journal of Clinical Child & Adolescent Psychology, 45(2), 91–113. https://doi.org/10.1080/15374416.2015.1046177

40.

Higa-McMillan

C. K.

Kotte

Jackson

Daleiden

E. L.

(2017). Overlapping and non-overlapping practices in usual and evidence-based care for youth anxiety. Journal of Behavioral Health Services and Research, 44(4), 684–694. https://doi.org/10.1007/s11414-016-9502-2

41.

Jacobs

S. R.

Weiner

B. J.

Bunger

A. C.

(2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9(1),1–14. https://doi.org/10.1186/1748-5908-9-46

42.

Jacobs

S. R.

Weiner

B. J.

Reeve

B. B.

Hofmann

D. A.

Christian

Weinberger

(2015). Determining the predictors of innovation implementation in healthcare: A quantitative analysis of implementation effectiveness. BMC Health Services Research, 15(1), 6. https://doi.org/10.1186/s12913-014-0657-3

43.

Jensen-Doss

Smith

A. M.

Walsh

L. M.

Mora Ringle

Casline

Patel

Shaw

A. M.

Maxwell

Hanson

Webster

(2020). Preaching to the choir? Predictors of engagement in a community-based learning collaborative. Administration and Policy in Mental Health and Mental Health Services Research, 47(2), 279–290. https://doi.org/10.1007/s10488-019-00985-4

44.

Kaye

J. L.

(2018). Targeting therapists’ experiential avoidance during exposure delivery: An experimental investigation to improve the quality of exposure-based interventions [Unpublished doctoral dissertation]. Drexel University.

45.

Klein

K. J.

Sorra

J. S.

(1996). The challenge of innovation implementation. Academy of Management Review, 21(4), 1055–1080. https://doi.org/10.5465/AMR.1996.9704071863

46.

Kratz

H. E.

Stahmer

Xie

Marcus

S. C.

Pellecchia

Locke

Beidas

Mandell

D. S.

(2019). The effect of implementation climate on program fidelity and student outcomes in autism support classrooms. Journal of Consulting & Clinical Psychology, 87(3), 270–281. https://doi.org/10.1037/ccp0000368

47.

Lance

C. E.

(2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9(2), 202–220. https://doi.org/10.1177/1094428105284919

48.

McLeod

B. D.

Cox

J. R.

Jensen-Doss

Herschell

Ehrenreich-May

Wood

J. J.

(2018). Proposing a mechanistic model of clinician training and consultation. Clinical Psychology: Science and Practice, 25(3), e12260. https://doi.org/10.1111/cpsp.12260

49.

Merikangas

K. R.

Burstein

Swanson

S. A.

Avenevoli

Cui

Benjet

Georgiades

Swendsen

(2010). Lifetime prevalence of mental disorders in US adolescents: Results from the national comorbidity survey replication–adolescent supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980–989. https://doi.org/10.1016/j.jaac.2010.05.017

50.

Meza

R. D.

Triplett

N. S.

Woodard

G. S.

Martin

Khairuzzaman

A. N.

Jamora

Dorsey

(2021). The relationship between first-level leadership and inner-context and implementation outcomes in behavioral health: A scoping review. Implementation Science, 16(69), 1–21. https://doi.org/10.1186/s13012-021-01104-4

51.

Morina

Koerssen

Pollet

T. V.

(2016). Interventions for children and adolescents with posttraumatic stress disorder: A meta-analysis of comparative outcome studies. Clinical Psychology Review, 47, 41–54. https://doi.org/10.1016/j.cpr.2016.05.006

52.

Olatunji

B. O.

Deacon

B. J.

Abramowitz

J. S.

(2009). The cruelest cure? Ethical issues in the implementation of exposure-based treatments. Cognitive and Behavioral Practice, 16(2), 172–180. https://doi.org/10.1016/j.cbpra.2008.07.003

53.

Osei-Bryson

K. M.

Dong

Ngwenyama

(2008). Exploring managerial factors affecting ERP implementation: An investigation of the klein-sorra model using regression splines. Information Systems Journal, 18(5), 499–527. https://doi.org/10.1111/j.1365-2575.2008.00309.x

54.

Peris

T. S.

Caporino

N. E.

O’Rourke

Kendall

P. C.

Walkup

J. T.

Albano

A. M.

Bergman

R. L.

McCracken

J. T.

Birmaher

Ginsburg

G. S.

Sakolsky

Piacentini

Compton

S. N.

(2017). Therapist-reported features of exposure tasks that predict differential treatment outcomes for youth with anxiety. Journal of the American Academy of Child and Adolescent Psychiatry, 56(12), 1043–1052. https://doi.org/10.1016/j.jaac.2017.10.001

55.

Peris

T. S.

Compton

S. N.

Kendall

P. C.

Birmaher

Sherrill

March

Gosch

Ginsburg

Rynn

McCracken

J. T.

Keeton

C. P.

Sakolsky

Suveg

Aschenbrand

Almirall

Iyengar

Walkup

J. T.

Albano

A. M.

Piacentini

. (2015). Trajectories of change in youth anxiety during cognitive-behavior therapy. Journal of Consulting and Clinical Psychology, 83(2), 239–252. https://doi.org/10.1037/a0038402

56.

Powell

B. J.

Mandell

D. S.

Hadley

T. R.

Rubin

R. M.

Evans

A. C.

Hurford

M. O.

Beidas

R. S.

(2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings ? A cross- sectional observational study. Implementation Science, 12(64), 1–13. https://doi.org/10.1186/s13012-017-0593-9

57.

Reid

A. M.

Bolshakova

M. I.

Guzick

A. G.

Fernandez

A. G.

Striley

C. W.

Geffken

G. R.

McNamara

J. P.

(2017a). Common barriers to the dissemination of exposure therapy for youth with anxiety disorders. Community Mental Health Journal, 53(4), 432–437. https://doi.org/10.1007/s10597-017-0108-9

58.

Reid

A. M.

Guzick

A. G.

Balkhi

A. M.

McBride

Geffken

G. R.

McNamara

J. P. H.

(2017b). The progressive cascading model improves exposure delivery in trainee therapists learning exposure therapy for obsessive-compulsive disorder. Training and Education in Professional Psychology, 11(4), 260–265. https://doi.org/10.1037/tep0000159

59.

Reid

A. M.

Guzick

A. G.

Fernandez

A. G.

Deacon

McNamara

J. P. H.

Geffken

G. R.

McCarty

Striley

C. W.

(2018). Exposure therapy for youth with anxiety: Utilization rates and predictors of implementation in a sample of practicing clinicians from across the United States. Journal of Anxiety Disorders, 58(March), 8–17. https://doi.org/10.1016/j.janxdis.2018.06.002

60.

Rosqvist

(2005). Exposure treatments for anxiety disorders: A practitioner’s guide to concepts, methods, and evidence-based practice. Routledge.

61.

Sawang

Unsworth

K. L.

(2011). A model of organizational innovation implementation effectiveness in small to medium firms. International Journal of Innovation Management, 15(5), 989–1011. https://doi.org/10.1142/S1363919611003398

62.

Shrout

P. E.

Fleiss

J. L.

(1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420

63.

Thomassin

Marchette

L. K.

Weisz

J. R.

(2019). Practitioners’ use and evaluation of transdiagnostic youth psychotherapy years after training and consultation have ended. Administration and Policy in Mental Health and Mental Health Services Research, 46(6), 821–832. https://doi.org/10.1007/s10488-019-00962-x

64.

Triplett

N.S.

Sedlar

Berliner

Jungbluth

Boyd

Dorsey

(2020). Evaluating a train-the-trainer approach for increasing EBP training capacity in community mental health. The Journal of Behavioral Health Services & Research, 47(2), 189–200. https://doi.org/10.1007/s11414- 019-09676-2

65.

Trivasse

Webb

T. L.

Waller

(2020). A meta-analysis of the effects of training clinicians in exposure therapy on knowledge, attitudes, intentions, and behavior. Clinical Psychology Review, 80(July), 101887. https://doi.org/10.1016/j.cpr.2020.101887

66.

Valenstein-Mah

Wilt

Greer

McKenzie

Strom

Hansen

Stirman

S. W.

Kehle-Forbes

(2020). The effectiveness of training methods for the delivery of evidence-based psychotherapies: A systematic review. Implementation Science, 15(40), 1–17. https://doi.org/10.1186/s13012-020-00998-w

67.

von Elm

Altman

D. G.

Egger

Pocock

S. J.

Gøtzsche

P. C.

Vandenbroucke

J. P.

(2014). The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. International Journal of Surgery, 12(12), 1495–1499. https://doi.org/10.1016/j.ijsu.2014.07.013

68.

Weiner

B. J.

Belden

C. M.

Bergmire

D. M.

Johnston

(2011). The meaning and measurement of implementation climate. Implementation Science, 6(1), 78. https://doi.org/10.1186/1748-5908-6-78

69.

Weisz

J. R.

Chorpita

B. F.

Palinkas

L. A.

Schoenwald

S. K.

Miranda

Bearman

S. K.

Daleiden

E. L.

Ugueto

A. M.

Martin

Gray

Alleyne

Langer

D. A.

Southam-Gerow

M. A.

Gibbons

R. D.

the Research Network on Youth Mental Health (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147

70.

Whiteside

S. P. H.

Deacon

B. J.

Benito

Stewart

(2016). Factors associated with practitioners’ use of exposure therapy for childhood anxiety disorders. Journal of Anxiety Disorders, 40, 29–36. https://doi.org/10.1016/j.janxdis.2016.04.001

71.

Whiteside

S. P. H.

Sim

L. A.

Morrow

A. S.

Farah

W. H.

Hilliker

D. R.

Murad

M. H.

Wang

(2020). A meta-analysis to guide the enhancement of CBT for childhood anxiety: Exposure over anxiety management. Clinical Child and Family Psychology Review, 23(1), 102–121. https://doi.org/10.1007/s10567-019-00303-2

72.

Williams

N. J.

Ehrhart

M. G.

Aarons

G. A.

Marcus

S. C.

Beidas

R. S.

(2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-year observational study. Implementation Science, 13(85), 1–13. https://doi.org/10.1186/s13012-018-0781-2

73.

Williams

N. J.

Glisson

Hemmelgarn

Green

(2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 269–283. https://doi.org/10.1007/s10488-016-0742-5

74.

Wolitzky-Taylor

Chung

Bearman

S. K.

Arch

Grossman

Fenwick

Lengnick-Hall

Miranda

(2019). Stakeholder perceptions of the barriers to receiving and delivering exposure-based cognitive behavioral therapy for anxiety disorders in adult community mental health settings. Community Mental Health Journal, 55(1), 83–99. https://doi.org/10.1007/s10597-018-0250-z

75.

Wolitzky-Taylor

Fenwick

Lengnick-Hall

Grossman

Bearman

S. K.

Arch

Miranda

Chung

(2018). A preliminary exploration of the barriers to delivering (and receiving) exposure-based cognitive behavioral therapy for anxiety sisorders in adult community mental health settings. Community Mental Health Journal, 54(7), 899–911. https://doi.org/10.1007/s10597-018-0252-x

76.

Wolitzky-Taylor

Zimmermann

Arch

J. J.

De Guzman

Lagomasino

(2015). Has evidence-based psychosocial treatment for anxiety disorders permeated usual care in community mental health settings? Behaviour Research and Therapy, 72, 9–17. https://doi.org/10.1016/j.brat.2015.06.010

77.

Wolk

C. B.

Becker-Haimes

E. M.

Fishman

Affrunti

N. W.

Mandell

D. S.

Creed

T. A.

(2019). Variability in clinician intentions to implement specific cognitive-behavioral therapy components. BMC Psychiatry, 19(1), 1–7. https://doi.org/10.1186/s12888-019-2394-y

Determinant	Pre-training attitudes				Post-consultation attitudes				Change in attitudes
	ß	SE	p	95% CI	ß	SE	p	95% CI	ß	SE	p	95% CI
Clinician-level implementation climate(n = 194)	−0.12	0.80	0.89	[−1.68, 1.46]	−2.21*	1.01	0.03	[−4.20, −0.21]	−2.07*	0.89	0.02	[−3.81, −0.32]
Group-level implementation climate(n = 128)	2.22	1.71	0.21	[−1.12, 5.54]	1.79	2.56	0.49	[−3.24, 6.76]	−0.66	2.33	0.78	[−5.26, 3.88]

The impact of implementation climate on community mental health clinicians’ attitudes toward exposure: An evaluation of the effects of training and consultation

Abstract

Background

Method

Results

Conclusions

Keywords

Method

Procedure

Measures

Demographics and background information

Attitudes toward exposure

Implementation climate

Analytic plan

Results

Demographics and descriptives

Implementation climate and exposure

Clinician-level determinants and exposure

Discussion

Conclusions

Footnotes

Acknowledgments

Declaration of conflicting interests

Funding

ORCID iDs

Plain language summary

References