Abstract
Background
Inner context organizational factors proximally shape therapist experiences with evidence-based practice (EBP) implementation and may influence therapist self-efficacy, which has been linked to sustained use of EBPs in community mental health settings. Research has primarily focused on constructs such as implementation leadership and climate. However, the effects of such factors may depend upon other inner context dimensions, such as psychological safety. Psychologically safe environments are conducive to taking risks, speaking up about problems, and requesting feedback and may promote therapist self-efficacy during implementation. This study examines whether organizational sustainment leadership and sustainment climate relate to therapist EBP self-efficacy only under conditions of psychological safety.
Methods
Data were collected from 410 clinicians in 85 programs during the sustainment phase of a system-driven implementation of multiple EBPs in children's mental health services. Therapists reported on their organization's sustainment leadership, sustainment climate, psychological safety, and their own self-efficacy in delivering specific EBPs. Multilevel regression analyses were conducted to account for nested data structure.
Results
Among program-level variables, sustainment leadership and psychological safety both significantly predicted therapist self-efficacy. However, there were no significant interactions between program-level sustainment climate and psychological safety. Exploratory post-hoc analyses revealed a significant interaction between program-level sustainment leadership and therapist-level perceptions of psychological safety such that that the conditional effect of psychological safety on EBP self-efficacy was significant at high levels of sustainment leadership, but not at low or average levels.
Conclusion
We noted independent links between sustainment leadership, organizational psychological safety and therapists feelings of confidence and mastery with EBPs. Therapists’ individual perceptions of psychological safety were linked to self-efficacy only in programs with high sustainment leadership. Thus, sustainment leadership and psychological safety may both represent implementation intervention targets, but it may not be critical to assess for perceptions of psychological safety before deploying organizational leadership strategies.
Keywords
Over the past 50 years, numerous evidence-based practices (EBPs) have been developed and identified as effective treatments for youth with psychiatric disorders (Williams & Beidas, 2019). However, access to EBPs is highly variable in community mental health settings (Bruns et al., 2015; Wang et al., 2010). In response, public behavioral health systems have pursued scale-up efforts to promote the implementation of multiple EBPs. These systems provide training and consultation for community therapists, as well as incentives or requirements such as collection of outcome measures, enhanced reimbursement for EBP use, and public recognition of EBP providers to support EBP learning, adoption and use (Regan et al., 2017; Stewart et al., 2018). As these efforts have increased, so has the pressure and expectation for community mental health therapists to learn these EBPs and for organizations to support and provide resources for them. Guided by the Exploration, Preparation, Implementaiton, and Sustainment (EPIS) framework, (Aarons et al., 2011) in the past decade, implementation research has examined inner context elements, such as therapist and organizational factors that facilitate the adoption and implementation of EBPs, in different outer context settings in behavioral health systems across the United States (Beidas et al., 2015, 2019; Hoagwood et al., 2014; Lau & Brookman-Frazee, 2016; Nakamura et al., 2014). This research has mostly focused on use of EBPs with fidelity after the implementation phase as an indicator of successful implementation. However, as many systems have already significantly invested in EBP training and implementation strategies, it becomes important to explore inner context factors and structures that predict sustained EBP use. A readily assessed therapist factor that has been shown to relate to sustained delivery of EBPs (Lau et al., 2020) is therapist self-efficacy with EBPs.
Self-efficacy – the belief that one is capable, knowledgeable, and skilled enough to complete a task (de Vries et al., 1988) – has robust links to future behavior change in many workplace sectors, such as health care (Holden, 1991), social work (Collins-Camargo & Royse, 2010; Montcalm, 1999), and technology (Hsu et al., 2009). In mental health service settings, therapist EBP self-efficacy is hypothesized to be an important factor in EBP delivery across many phases of implementation, including preparation, implementation, and sustainment (Aarons et al., 2012; McLeod et al., 2018). Among community therapists within a system-driven implementation effort of multiple EBPs, therapist-reported self-efficacy with a given EBP was associated with decreased likelihood of discontinuation of that EBP (Lau et al., 2020) and increased observed extensiveness of EBP strategy delivery in session (Lind et al., 2021), indicating that self-efficacy may be an important target for EBP sustainment efforts. Self-efficacy is also hypothesized to be an important therapist factor in improving clinical outcomes. In a randomized controlled trial of cognitive processing therapy, therapist self-efficacy ratings were related to patient clinical improvement. In addition, among therapists with low baseline self-efficacy, receipt of consultation resulted in better client outcomes (Pace et al., 2020). These findings suggest that organizations may be able to improve care and EBP implementation outcomes by investing in strategies that promote therapist self-efficacy.
To date, research on inner context organizational factors related to EBP implementation has largely focused on improving the EBP implementation specific components, such as EBP implementation leadership and climate (Aarons et al., 2009; Williams et al., 2020). Strong implementation leadership is characterized by leaders who are knowledgeable about, supportive of, and perseverant through challenges associated with EBP implementation (Aarons et al., 2014). Positive EBP implementation climate is denoted by employees’ shared perception of the importance of EBP implementation within the organization (Ehrhart, Aarons, et al., 2014). Both implementation climate and leadership have been posited to improve EBP implementation through supporting therapist acquisition of EBP knowledge and skills through educational implementation strategies, removing barriers, and enhancing motivation to EBP use (Aarons & Sommerfeld, 2012; Jacobs et al., 2014). Previous studies have also found that higher levels of EBP implementation climate and leadership are associated with more positive therapist EBP attitudes, increased EBP knowledge, and increased use of EBP techniques (Powell et al., 2017; Williams et al., 2020). These findings suggest that organizational leaders’ persistent and proactive promotion of EBPs can improve provider implementation outcomes in community settings. It should also be noted that implementation climate is theorized to mediate the relationship between implementation leadership and employee behavior (Ehrhart, Schneider, et al., 2014). However, less is known about how these factors interact to influence EBP self-efficacy. Organizational commitment to training and supportive leadership have been posited to influence individual self-efficacy regarding EBPs, (Cannon-Bowers et al., 1995; McLeod et al., 2018) but links to this outcome have yet not been empirically tested. Furthermore, little data is available concerning how these organizational factors operate in the sustainment phase of implemenation.
There is also a need to examine inner context organizational characteristics (beyond those specific to EBP implementation) that may also determine the outcomes of implementation leadership and climate. For example, one study by found a significant interaction between implementation climate and molar climate, such that organizations with positive molar climates and high levels of implementation climate predicted the highest EBP use for their clinicians (Williams et al., 2018). Given considerable organizational variation in workplace climate, it may be important to identify inner context organizational characteristics that have implications for the impact of implementation strategies and interventions. One organizational factor that may be important to consider is psychological safety, i.e., the degree to which employees perceive that their workplace is conducive to taking interpersonal risks, such as speaking up or asking for help (Edmondson, 1999). Psychologically safe environments, where people feel accepted, supported, and trusted, promote sharing of information (Collins & Smith, 2006), experimenting with new skills (Baer & Frese, 2003), and upward communication to management (Edmondson & Lei, 2014) - all processes that underlie workplace effectiveness and organizational learning. While psychological safety can be studied at multiple levels (e.g., individual-, group-, organizational-), it is most commonly studied at the group- and organizational-levels (Edmondson & Lei, 2014) Group- and organizational-level psychological safety has been found to promote employee learning and behavior change in corporate settings and complex work environments such as health care and education sectors (Edmondson, 2003; Edmondson et al., 2016; Nembhard & Edmondson, 2006; Singer et al., 2009). Yet, organizational-level psychological safety has yet to be examined in relation to EBP implementation or sustainment outcomes in behavioral health settings (Williams & Glisson, 2020).
Organizational interventions aimed at improving implementation leadership and climate have also implicated organizational-level psychological safety as an essential mechanism to ready organizational contexts for successful EBP implementation (Aarons et al., 2017; Glisson et al., 2010, 2016). Fostering psychologically safe environments may be especially important for frontline therapists who are responsible for delivering multiple EBPs in resource-limited community mental health settings serving high-need populations. As part of EBP implementation initiatives, therapists have been asked to perform new behaviors (e.g., learning and using new techniques, asking for and receiving supervision and performance feedback), which can be viewed as risky because they could open up individuals to negative evaluation. Even if organizations provide clear messages about the priority of EBPs through its leadership and climate, therapists may hestitate to take risks that promote learning if they perceive that the organization is not psychologically safe. In contexts where client well-being is at stake and complex multicomponent behavioral interventions are being introduced, it is critical for therapists to feel safe to grow from experience with the innovation, learn from mistakes, and benefit from fidelity monitoring (Edmondson, 2003; Edmondson et al., 2016). Psychologically safe environments are conducive to improving staff learning and performance by setting the conditions to try out practice innovations without negative consequences (Edmondson & Lei, 2014). EBP self-efficacy is theorized to change as therapists gain experience applying skills and knowledge and benefitting from training and consultation (McLeod et al., 2018). Environments in which therapists are willing to pose questions, discuss errors, and learn from their mistakes set the stage for therapists to be comfortable engaging in active learning components of training and activities such as fidelity monitoring and performance feedback (Dorsey et al., 2013; Lucid et al., 2018) ultimately impacting their sense of self-efficacy. Additionally, a psychologically safe work environment may also encourage therapists to share information and problem-solve challenges in implementation at the local level, further boosting agency and self-efficacy.
Thus, a critical unexamined question is whether the effects of implementation leadership and climate on EBP self-efficacy may depend on psychological safety. In particular, a concerted emphasis on EBP implemenation at the organizational-level may have differential effects, depending upon whether therapists perceive the environment as safe for learning practice innovations. Therefore, the current study examines the interplay of sustainment leadership, sustainment climate, and organizational-level psychological safety in predicting therapists’ self-efficacy with EBP delivery.
Within the context of a system-driven implementation of multiple EBPs for children in the Los Angeles County Department of Mental Health (LACDMH; Regan et al., 2017), the current study examined how organizational sustainment leadership and EBP sustainment climate are associated with therapist perceptions of EBP self-efficacy and whether these relationships may be moderated by organizational-level psychological safety (See Figure 1). Moving forward, this is described as “program-level” psychological safety in the present study to accurately describe the organizational structure of this context in which therapists are aligned to programs within specific agencies. Specifically, we hypothesized that more positive sustainment leadership and higher levels of sustainment climate would predict higher therapist self-efficacy for EBPs under conditions of program-level psychological safety for learning. Under positive conditions for learning, an organizational climate focused on the importance of sustaining EBPs and sustainment leadership may provide the conditions under which therapists can develop self-efficacy. However, a high-stakes focus on EBPs through implementation-specific workflows and leadership messaging may inhibit therapist EBP self-efficacy when they perceive the organizational climate is not safe for learning.
Conceptual model of the effect of inner context organizational constructs on EBP self-efficacy.
Methods
Setting
Data for the current study were collected via an online survey fielded in 2019 as part of the Knowledge Exchange on Evidence-Based Practice Sustainment (4KEEPS) study (Lau & Brookman-Frazee, 2016). The 4KEEPS study is an observational study that examined therapist and organizational factors associated with sustainment of multiple EBPs in the context of a system-driven implementation in Los Angeles County. LACDMH is the largest county mental health department in the United States, serving more than 250,000 residents annually (Los Angeles County Department of Mental Health, n.d.). In 2010, the LACDMH launched the Prevention and Early Intervention (PEI) program which facilitated EBP implementation by furnishing EBP training and consultation and offering agencies reimbursement for the delivery of EBPs. The 4KEEPS survey administered in 2019 focused on nine EBPs that were among the most commonly delivered within PEI: Incredible Years Parenting Program (IY), Interpersonal Therapy (IPT), Parent Child Interaction Therapy (PCIT), Child-Parent Psychotherapy (CPP), Managing and Adapting Practice (MAP), Seeking Safety, Triple P Positive Parenting Program (Triple P), Trauma Focused Cognitive Behavior Therapy (TF-CBT), and Cognitive-Behavioral Intervention for Trauma in Schools (CBITS).
Participants
The participants for the current study included 410 therapist respondents from 85 programs that were contracted by LACDMH to deliver EBPs through PEI in children's mental health services. Eligible participants included any therapist who billed for psychotherapy services to at least one of the nine commonly delivered EBPs to youth. Most therapists were female (91.22%), had a master's degree (91.71%), were not licensed (66.10%), and had CBT or behavioral theoretical orientation (63.90%). The mean age and SD were 34.47 and 7.67, respectively. This sample was racially and ethnically diverse, with 53.66% identifying as Hispanic, 20.00% as white, and 26.34% as other ethnic minorities (e.g., Asian, Black, etc.). Table 1 presents the therapist demographics of the sample.
Descriptive statistics (N = 410).
Procedure
Data collection for this survey occurred in May through August 2019. Participants were recruited by gathering therapist and program leader contact information from leaders of organizations that had participated in a prior therapist survey from the 4KEEPS project fielded in 2016 (Barnett et al., 2017). Of 82 agencies eligible for youth PEI service contracts, 37 (45.1%) provided contact information for therapists employed in 87 distinct programs. All program leaders and therapists were then invited by email to participate in the online survey. Email invitations were sent directly to 1447 individuals. The survey was completed by 448 therapists, 113 program leaders, and 33 individuals who identified as both therapist and program leader, leading to a response rate of 41.1%. This is within the range of previous surveys of previous community mental health therapists (Hawley et al., 2009). We are unable to produce specific response rates for program leaders and therapists because we do not know the which of the 1447 individuals were therapists or leaders. Participants received a $40 electronic gift card for completing the survey. All procedures for this study were reviewed and approved by the Institutional Review Boards at LACDMH, University of California, Los Angeles, and University of California, San Diego. The study was granted a waiver of signed consent and a written information sheet was provided to each participant on the first page of the online survey. For the current study, 38 therapists were excluded from the analyses because they did not provide complete data for the dependent variable, resulting in N = 410 as described above. Program leader data were not included in the present study due to the focus on front-line therapist perceptions of organizational constructs, which is the convention in the literature (Weiner et al., 2011).
Measures
Psychological safety scale
Psychological safety was measured using an adapted 4-item version of the Psychological Safety Scale (Edmondson, 1999). Therapists reported to what extent each of the four statements applied to their organizations on a 5-point Likert scale from “Strongly disagree” (1) to “Strongly agree” (5). Example statements include “It is safe to take a risk in this program,” and “Therapists who work here are able to bring up problems and tough issues.” “Program” referred to the therapist's organization and work setting. The adapted scale differs from the original scale in two ways: 1) text in the items were modified from “team” and “someone” to reflect the current context (e.g., “program” and “therapists”), 2) we removed 3 items from the original scale to focus on items that were relevant to the EBP implementation context. The content of the 4 included items assessed climate that promoted safety in learning a practice innovation (e.g., “It is difficult to ask other staff in this program for help” and “Therapists who work here are able to bring up problems and tough issues) rather than a more general atmosphere of interpersonal trust and acceptance (e.g., “People on this team sometimes reject others for being different” and “Working with members of this team, my unique skills and talents are valued and utilized”). In the present study, internal consistency for the scale was acceptable (α = .72) and there was significant program-level variance (ICC = .18).
Sustainment leadership scale
Sustainment Leadership was measured using an adapted 9-item version of the psychometrically validated Implementation Leadership Scale (ILS) (Aarons et al., 2014). This scale assesses the degree to which leaders are knowledgeable (“Leaders in my program know what they are talking about when it comes to EBPs”), supportive (“Leaders in my program support staff efforts to learn more about EBPs”) and perseverant (“Leaders in my program persevere through the ups and downs of sustainment of EBPs”) in implementing EBPs. The adapted version differs from the original ILS in three ways: 1) the proactive subscale was removed due to concerns about overburdening participants and lack of relevance for the current study's system-driven sustainment context, 2) “leaders in my program” was included in each item rather than the name of each supervisor, and 3) the items were reworded to reflect EBP sustainment rather than implementation (“Leaders in my program recognizes and appreciates employee efforts toward successful
EBP sustainment climate
Perceptions of organizational climate on EBPs were measured using a 3-item subscale of the Sustainment Climate Scale, which was adapted from the Implementation Climate Scale (ICS) (Ehrhart, Aarons, et al., 2014). The subscale assesses the degree to which organization members prioritize EBP use and sustainment (“One of this program's main goals is to use evidence-based practices effectively”) and only one word was adapted from the subscale item of the ICS to capture sustainment rather than implementation (“People in this program think that the
Therapist report of EBP self-efficacy
Therapist self-efficacy in delivering EBPs was measured using 4 items that included two from a previously published EBP-specific self-efficacy measure (Lau et al., 2020). The prior two-time measure assessing confidence in delivering each of the EBPs the therapist is currently using with clients (“I am confident in my ability to implement this EBP,” “I am well prepared to deliver this EBP even with challenging clients”). Following a qualitative analysis of in-depth interviews with community therapists, two items were added to reflect additional concepts related to self-efficacy – feeling knowledgeable about (“I am knowledgeadble about this EBP”) and understanding the EBP model (“I have a good understanding of all of the components of this EBP”). Therapists rated themselves on a 5-point Likert scale from “Not at all” (1) to a “Very great extent” (5). Therapists completed this scale for each EBP they were currently using with clients at their agency. On average, therapists reported using 1.95 EBPs with clients (SD = 1.10). The four items were averaged to create a composite score. Internal consistency for this scale was excellent (α = .94). Also, we calculated ICCs at level 2 (.32) and level 3 (.08) for self-efficacy, indicating that between-therapist, and between-program differences do account for variability in reported self-efficacy.
Statistical analyses
This study intended to examine 1) if sustainment leadership and EBP sustainment climate were associated with therapists’ self-efficacy for EBPs and 2) whether psychological safety moderated the relationships in the first aim. Multilevel models were conducted to account for the nested data structure, with EBPs (level 1; n = 932) nested within therapists (level 2; n = 410) nested within programs (level 3; n = 85). For program-level variables, therapist reported perceptions were aggregated to the program-level by taking the mean score of all therapists in a particular program.To justify aggregation, we calculated interrater agreement (i.e., rwg(j),) for each program-level variable (LeBreton & Senter, 2008). The mean rwg(j)s for program-level psychological safety (rwg(j) = .73), sustainment leadership (rwg(j) = .74), and sustainment climate (rwg(j) = .76) all demonstrated strong agreement (.71 < rwg(j) <.91) according to LeBreton & Senter's revised standards for interpreting interrater agreement estimates (2008). In terms of the models, first, we ran a multilevel regression with random intercepts using the “mixed” command and maximum likelihood estimator in Stata with two level 3 interactions with program level psychological safety (i.e., psychological safety x sustainment leadership, psychological safety x EBP sustainment climate) as predictors and EBP self-efficacy (at level 1) as the outcome. Based on previous research on therapist-level factors associated with similar therapist-reported EBP implementation outcomes (Lau et al., 2020), we included the following therapist-level covariates in our analyses: race/ethnicity, licensure status, education, primary discipline, and number of learned EBPs. We also included a categorical variable that identified whether an EBP had structured content as an EBP-level covariate to control for EBP content differences (Barnett et al., 2017). All variables were centered at the grand mean prior to calculating interaction terms.
A second, exploratory model examining therapist-level psychological safety was conducted after testing our initial hypothesis. This variable was group-mean centered prior to calculating interaction terms (Enders & Tofighi, 2007). We added two cross-level interactions to the previous model (i.e., therapist-level psychological safety and program-level sustainment leadership and therapist-level psychological safety and program-level EBP sustainment climate). For this regression analysis with random intercepts and random slope (for therapist-level psychological safety), we used the same “mixed” command and maximum likelihood estimator in Stata. Missing data were addressed using listwise deletion, which resulted in 1.1% or 6.2% of data missing depending on the model. All analyses were completed using Stata/IC version 16 and SPSS version 26.
Results
Table 2 presents the results of multilevel regression analyses. Model 1 presents the proposed model, but without interaction effects. In this model, only program-level sustainment leadership was found to be associated with therapists’ EBP self-efficacy (b = 0.35, p = 0.001). In model 2, which is the proposed model including interaction effects, no significant interactions were found. However, the main effects of program-level sustainment leadership and program-level psychological safety on therapists’ EBP self-efficacy were significant such that higher sustainment leadership (b = 0.36, p < 0.001) and psychologicsal safety (b = 0.21, p = 0.049) predicted higher therapist EBP self-efficacy. Effect sizes were calculated for these main effects (Selya et al., 2012) and both variables had the same small-to-medium effect, f2 = 0.077 (Lorah, 2018). However, program-level EBP sustainment climate was not associated with therapist-reported EBP self-efficacy.
Regression results predicting therapist EBP self-efficacy for models with direct effects only (model 1), program-level psychological safety (model 2) and therapist-level psychological safety (model 3).
*p < .05, **p < .01, ***p < .001.
In model 3, our exploratory model including therapist-level psychological safety, the interaction between program-level sustainment leadership and therapist-level psychological safety was significant (b = 0.39, p = 0.021), but no other interactions with sustainment leadership or climate were significant. The interaction also had a small-to-medium effect (f2 = 0.076). The main effect of program-level EBP sustainment climate remained significant as well (b = 0.35, p = 0.001), but program-level psychological safety was no longer significant. Similar to the previous model, the main effect of program-level EBP sustainment climate was not significant.
To further probe the significant cross-level interaction, we conducted post hoc simple slope analyses. This analysis revealed that the conditional effect of psychological safety on EBP self-efficacy was significant at high levels (one SD above the mean) of sustainment leadership (dy/dx = 0.31., p = 0.013), but not at low (one SD below the mean) or average levels (at the mean of programs). (See Figure 2).
Simple slopes of therapist-level psychological safety predicting EBP self-efficacy for low (−1 SD), medium (mean), and high (+ 1 SD) levels of sustainment leadership.
In terms of model covariates, in both models 2 and 3 higher number of EBPs being used by therapists (p = 0.030) and professional licensure were positively associated with self-efficacy (p = 0.016). Also, therapists who identified as Hispanic (p < 0.001) or as a member of another ethnic minority group (p = 0.012) reported higher EBP self-efficacy than therapists who identified as White. Additionally, therapists reported higher EBP-efficacy for structured EBPs compared to those that are unstructured (b = 0.16, p = 0.003). There were no differences in reported self-efficacy based on therapists’ primary discipline (e.g., social work, counseling, psychology, or marriage and family therapy).
Discussion
As public mental health systems make costly investments in EBP implementation efforts, there is utility in examining conditions that relate to on-going EBP sustainment to inform targeted improvements. In this paper, we were interested in examining how organizational sustainment leadership and climate may interact with psychological safety to create conditions conducive to high levels of EBP self-efficacy, an important predictor of sustained EBP use. The current study examined how psychological safety may moderate the effects of organizational sustainment leadership and sustainment climate on therapists’ EBP self-efficacy in the context of a system-driven implementation of multiple EBPs in children's mental health.
In our initial analyses, program-level sustainment leadership and psychological safety showed associations with therapist self-efficacy, but no significant interactions emerged. Although our original hypothesis was unsupported, we conducted an additional exploratory analysis examining individual (or therapist)-level psychological safety as the factor that may more proximally relate to outcomes of interest. In community mental health clinics, therapists mostly work with clients individually rather than in teams and perceptions of psychological safety may vary considerably across staff members. In our exploratory analysis, we found a significant interaction between program-level sustainment leadership and therapist-level psychological safety. Given the nature of this multi-level moderation analysis, therapist-level psychological safety cannot be interpreted as moderating the higher-level relationship between program-level sustainment leaderhip and EBP self-efficacy (Preacher et al., 2016). Instead, we may infer only that the positive link between therapist's sense of psychological safety and EBP self-efficacy only held in programs characterized by high sustainment leadership. Thus, although we did not obtain support for our a priori hypothesis that organizational psychological safety was a precondition for positive effects of sustaintment leadership and climate, we did find evidence that perceived psychological safety promoted self-efficacy when leadership was strong in championing EBP implementation.
Overall, we noted independent links between organizational sustainment leadership and organizational psychological safety and therapists feelings of confidence and mastery self-efficacy with EBPs. Thus, sustainment leadership and psychological safety both represent potentially important and distinct targets of organizational interventions but it may not be critical to assess for workplace psychological safety before deploying organizational leadership strategies. Again, therapists’ individual perceptions of psychological safety were linked to self-efficacy only in programs with high sustainment leadership.
Our findings suggests that organizations with program leaders who are knowledgeable about, able to support the tasks associated with EBP delivery, and tenacious about EBP sustainment may be able to support therapists feelings of mastery when overall levels of psychological safety are less than optimal. We also found that sustainment leadership impacts self-efficacy above and beyond perceptions of organizational EBP sustainment climate. This may be because strong leaders are attuned to barriers to implementation and sustainment in their programs and are able to work with employees to remove these barriers. These findings point to the centrality of supporting community mental agencies to cultivate sustainment focused leadership, which may in turn increase self-efficacy in front-line therapists.
Our results differ from previous studies that describes implementation climate as potential mediator of the relationship between implementation leadership and EBP use or behavior. Although the two variables were moderately strongly correlated (Table 3), there was no evidence that the relationship between sustainment leadership and the outcome was diminished by sustainment climate. Rather, with both variables in the model, only sustainment leadership explained variance in therapist self-efficacy. A main difference in this study was our focus on therapist EBP self-efficacy as an outcome, rather than EBP use. Although self-efficacy is related to and predicts actual EBP use, these are distinct constructs that may have different relationships with organizational constructs. In the Theory of Planned Behavior, self-efficacy is theorized to be associated with intentions and behaviors, and is a precursor to both (Ajzen, 2002). Even if a therapist holds positive attitudes toward an EBP and understands that EBP use is expected, if they do not believe they are capable and prepared, adoption, implementation and sustainment may not ensue. However, given the study limitations described below, additional resarch is needed to replicate these findings prior to taking action on implementation strategy design.
Correlation matrix with therapist- and program-level study variables.
Our findings provide some support that it may be important to attend to individual differences in perceptions of psychological safety in community mental health, in addition to organizational psychological safety. Outside of training and supervision, community therapists largely work individually to learn EBPs and provide care to their clients. The independent nature of this role is also reflected in the growing number of independent contractors, rather than salaried employees, that are hired by community mental health organizations (Beidas et al., 2016). In addition, previous work has observed that the relevant relationships that build and maintain perceptions of psychological safety may vary across workplace contexts (Edmondson & Lei, 2014). One study comparing psychological safety in medical and education settings found that psychological safety perceptions varied by small groups in hospital settings (i.e., intensive care units within medical centers) while perceptions differed by entire schools in high school settings (Edmondson et al., 2016). It is possible that psychological safety is shaped by specific salient interpersonal working relationships (e.g., direct supervisor-therapist dyads) instead of group or organization-wide dynamics in community mental health settings. Thus, individual perceptions of psychological safety may be most relevant to provider self-efficacy. When participating in group supervision and consultation, therapists’ own perceptions of interpersonal safety may determine whether they elect to engage in active EBP learning behaviors (e.g., asking questions, seeking feedback, demonstrating EBP skills at risk of making a mistake), which may impact their self-efficacy. Future research is needed to examine how individual- and organizational-level psychological safety may impact proximal learning behaviors among therapists in implementation contexts.
To develop effective strategies for improving sustainment leadership and perceived psychological safety, both leaders and providers should be engaged to assess the health and functioning of key working relationships (e.g., supervisor and therapist dyads, mid-level management and organizational leadership interactions) and groups (e.g., supervision groups, EBP training cohorts). Comprensive and systematic organizational assessments that examine the social context of organizations (such as the profiles developed for organizations in Williams et al., 2019) may be useful identifying appropriate targets and goals. Once the appropriate targets have been determined, future qualitative studies can assess the feasibility and acceptability of interventions that integrate individual- and organization-directed strategies aimed at developing supportive and knowledgeable leaders who address appropriate EBP sustainment barriers and fostering psychological safety. For example, interventions in burnout prevention in healthcare workers have been shown to be most effective when strategies are aimed at both levels (e.g., peer support groups and consultation and organizational strategies to increase therapist involvement in organizational policy decision-making) (Dreison et al., 2018; Panagioti et al., 2017). Stakeholder input in the development and implementation of these strategies will be essential to tailor them to the specific needs and context of each organization.
There are several limitations to this study that necessitate tempered conclusions and indicate important directions for future research. First, we are unable to draw conclusions about the directionality of associations because constructs were assessed in a cross-sectional analysis. We are unable to rule out alternative interpretations, for example, staff therapists’ self-efficacy with EBPs may have compelled more organizational leadership support for EBPs. It is also plausible that an unmeasured individual difference factor (e.g., neuroticism) contributes to therapists’ low confidence with EBPs, negative perceptions about their organization, and greater apprehensiveess about making mistakes and thus lower pereceived psychological safety. Future prospective studies are needed to clarify the temporal relations between psychological safety, sustainment leadership, and EBP self-efficacy. Future intervention research manipulating these organizational constructs may provide further insight into the magnitude of effects. Second, the study relied on self-report surveys to assess all constructs. Although reliance on self-report is common, this contributed to shared method variance and subjective reports of EBP-focused leadership, sustainment climate, and perceptions of psychological safety, and self-efficacy. Future studies may benefit from collection of multi-informant multi-method data, including direct observation of EBP leadership behavior and therapist implementation behavior. Furthermore, the present study used modified and shortened versions of scales that have not been validated within multi-method, multi-informant studies. Replication of findings from this study using complete, validated scales within prospective designs is needed to guide future implementation and sustainment efforts. Third, although our survey response rate of 41.1% is acceptable for surveys with community mental health therapists, (Hawley et al., 2009) it leaves room for concerns about response bias and generalizability. Finally, the present study examined provider self-efficacy with EBP delivery as an outcome, and this is not equivalent to EBP use. While self-efficacy is a theorized predictor to behavior change, future studies should prospectively examine whether and how organizational sustainment leadership, climate, and psychological safety impact sustained EBP use and fidelity among community therapists.
Despite these limitations, this study contributes new data to our emerging understanding of how therapist and organizational factors may individually and jointly impact EBP implementation in community mental health settings. In particular, organizational perceptions of sustainment leadership and psychological safety were identified as predictors of EBP self-efficacy, and there was some evidence that individual perceptions of psychological safety were associated with EBP self-efficacy only when there was sufficiently high EBP sustainment leadership support. After further study and replication, implementation scientists and organizational leaders may consider how to prioritize implementation strategies related to leadership and climate to optimize the impact of costly system-driven implementation efforts.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute of Mental Health (NIMH), (grant number R01 MH100134) to the second and last author.
