Abstract
Background
School leaders play an integral role in the use of implementation strategies, which in turn support special education teachers in the implementation of evidence-based practices (EBPs). In this convergent mixed methods study, we explored school leaders’ perceptions of the facilitators and barriers to EBP implementation, particularly for students receiving special education, as well as the importance and feasibility of 15 implementation strategies.
Method
School leaders (N = 22, principals, assistant principals, school psychologists, etc.) participated in a semistructured interview that included three parts—qualitative questions, quantitative ratings of strategies’ importance and feasibility, and discussion of the top three implementation strategies. Data strands were analyzed independently and then integrated to generate meta-inferences.
Results
The qualitative data identified facilitators such as access to resources about the intervention (e.g., professional development) and collaboration and teamwork, while barriers centered on lack of school supports, culture/climate, and organizational factors (e.g., lack of communication). The quantitative data indicated that the implementation strategy provide ongoing consultation/coaching was rated as important and feasible. Monitor the progress of the implementation effort was rated as important but less feasible, while conduct educational meetings and change the environment were rated as feasible, but less important. Build partnerships to support implementation was rated as less important and feasible. There was convergence and divergence in mixed methods findings.
Conclusion
This study underscores the critical need to increase school leaders’ knowledge and skills related to implementation science to better leverage implementation strategies that address the confluence of relevant implementation determinants.
Plain Language Summary
School leaders, such as principals, assistant principals, and school psychologists, are responsible for supporting special education teachers in doing interventions (e.g., through teacher training). In this study, we interviewed 22 school leaders to understand what factors support or hinder teachers in doing interventions, especially for students receiving special education. We also provided school leaders with a list of 15 strategies that may be used to support teachers, to determine which strategies school leaders think are the most important and feasible in schools. Our results indicated that it is important to train school leaders on how to move research into practice settings, such as schools. It is equally important for researchers and policymakers to understand the priorities of school leaders.
Introduction
Schools are the primary service setting for children with mental and behavioral health needs, including those who receive special education (Duong et al., 2021). Students receiving special education are provided with an Individualized Education Program (IEP) that delineates their needs, goals, and services (Reich, 2010). School-based providers are encouraged to address IEP goals using evidence-based practices (EBPs; Individuals with Disabilities Education Act [IDEA], 2004); however, the uptake and scale-up of EBPs are onerous in schools (Lyon et al., 2020; Lyon & Bruns, 2019). School leaders are often equated with school administrators (e.g., principals; Melgarejo et al., 2020; Stadnick et al., 2019). However, common team-based leadership models recognize administrative and practice-level leaders (e.g., school psychologists, behavioral coaches; Bush & Glover, 2014). Each type of leader plays a key role in supporting the implementation of EBPs by using implementation strategies (e.g., professional development [PD]). However, there is limited research on school leaders’ perceptions of the barriers and facilitators that influence the implementation of EBPs and which implementation strategies they find the most important and feasible.
Barriers and Facilitators to School-Based Implementation of EBPs
Despite national laws mandating EBPs in educational settings (IDEA, 2004), EBP adoption, implementation, and sustainability in schools and special education settings is limited (Burns & Ysseldyke, 2009; Lyon et al., 2020). Determinant frameworks, such as the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009, 2022a), delineate multilevel contextual factors that influence implementation outcomes. Barriers and facilitators of EBP in special education are noted at the individual (e.g., knowledge and attitudes), inner setting (e.g., competing priorities and school resources), and innovation (e.g., complexity of EBPs) levels (B. G. Cook & Odom, 2013; Edmunds et al., 2022; Forman et al., 2013). Studies on implementation determinants in special education have often focused on autism services and have primarily included teachers (Lawson et al., 2022), direct service providers (Suhrheinrich et al., 2021), or a combination of personnel (e.g., administrators, teachers, and other staff; Locke et al., 2019b). There are limited studies, to our knowledge, that examine the perspective of school leaders—both administrative- and practice-level—on implementation barriers and facilitators.
Implementation Strategies in Schools
Implementation strategies were developed to address barriers and bolster facilitators to expedite the translation of EBPs into routine service delivery. The Expert Recommendations for Implementing Change (ERIC) project compiled and defined 73 implementation strategies, distributed across nine categories (e.g., engage consumers, develop stakeholder interrelationships, etc.; Powell et al., 2015; Waltz et al., 2015); however, this compilation was limited to health care. Through an iterative expert review process, the School Implementation Strategies, Translating ERIC Resources (SISTER) project delineated 75 school-based implementation strategies, with definitions, across the same nine categories. The ERIC and SISTER compilations were developed to increase consistency and terminology when describing implementation strategies in research and practice (C. R. Cook et al., 2019; Powell et al., 2015).
The literature on implementation strategies in schools has largely focused on individual-level strategies, such as PD including training, consultation, and coaching (Lyon & Bruns, 2019). Yet, research suggests that high-quality implementation is supported when a variety of tailored strategies are used at multiple levels, including the inner and outer settings (Lyon et al., 2019; Powell et al., 2017). In research, Fernandez and colleagues (2019) suggest implementation mapping as a systematic process for developing implementation strategies based on needs assessment data, as well as the identification of determinants and mechanisms of change.
In practice, it is unclear how implementation strategies are chosen. Lyon and colleagues (2019) attempted to understand this process by asking 200 school-based consultants (e.g., school psychologists and behavior specialists) to rate the importance and feasibility of the 75 SISTER strategies. The three most important implementation strategies were: (1) conduct ongoing training, (2) make training dynamic, and (3) provide ongoing consultation/coaching. The three most feasible strategies were: (1) make training dynamic, (2) distribute educational materials, and (3) remind school personnel. Although the participants in the Lyon et al. (2019) study had a variety of roles, administrative leaders were not included. Their study also targeted school-based implementation in general education, therefore, these findings may not generalize to special education settings.
Contribution of the Present Study
School leaders, including administrators and practice leaders, often have the executive decision-making power to utilize implementation strategies, which in turn promote the effective use of EBPs by teachers (Pauling et al., 2021). However, there is limited research focusing on their perceptions of the barriers and facilitators to implementation and which implementation strategies they find most important and feasible for special education settings. This knowledge would make an important contribution to the literature by providing insight into school leaders’ decision-making, including how funds may be allocated and which initiatives are prioritized. This study explored the perspective of school leaders with three research questions: (1) What factors facilitate the implementation of EBPs for students receiving special education? (2) What barriers impede the implementation of EBPs for students receiving special education? (3) What implementation strategies were identified as most important and feasible?
Method
Participants
Participants were 22 school leaders from nine schools in one Mid-Atlantic state. They primarily identified as female (81.8%) and as either White (86.4%) or African American/Black (13.6%). Leaders’ roles included administrators (59.1%) or related service providers (40.9%) with some experience or exposure supporting students with autism in either public (54.5%) or nonpublic (45.5%) schools. The nonpublic schools (part of one university-affiliated alternative school system) served public school students with disabilities (e.g., autistic, intellectual disability, and emotional disability) whose educational needs could not be met in public schools. School leaders were prompted to discuss implementation by special education teachers serving students with autism. Table 1 depicts leaders’ demographics.
Participant Demographics (N = 22, Nschools = 9)
Note. IEP = Individualized Education Program.
Procedure
Research procedures were approved by the university's and school district's Institutional Review Boards. Participants were recruited from a larger study aimed at modifying a school-based implementation package to enhance the use of EBPs by parents and teachers of autistic children. The project did not involve EBP delivery nor implementation strategy testing, thus participants’ exposure to EBPs was not affected because of their affiliation with the project. As part of the larger study, the senior author had contact information for all school leaders. She emailed them to explain the research opportunity which included surveys and an optional qualitative interview. For those that expressed interest (N = 34), the senior author met with each school leader to provide further information about herself (e.g., qualifications, experience, and relationship with teachers) and the study, including risks and benefits, and answer any questions. One school leader did not consent to participate. Three leaders (from public schools) consented but did not return their surveys. The subsample who expressed interest in the qualitative interviews is the focus of this study (N = 22). Sampling saturation was achieved when all possible leaders whose school sites were participating in the larger study were approached for recruitment.
Interviews were conducted by the senior author at participating schools and audio recorded. Data were collected between September 2019 and February 2020. The interview (analyzed between May 2022 and April 2023) was broken into three parts: (1) open-ended questions, (2) rating implementation strategies’ importance and feasibility, and (3) discussion of top three implementation strategies. Each part was completed consecutively during the 45–60-min interview.
Measures
Interview Part 1: Open-Ended Qualitative Questions
A semistructured interview protocol was developed to gather in-depth information about implementation in special education. We analyzed school leaders’ responses to two prompts about (a) the culture and climate at their school (i.e., “Tell me about the culture and climate at this school”) and (b) how special education teachers were supported in using EBPs (i.e., “How does your school support special education teachers in doing evidence-based practices?”).
Interview Part 2: Quantitative Ratings
School leaders rated the importance and feasibility of 15 implementation strategies identified from the SISTER compilation (C. R. Cook et al., 2019). The 15 implementation strategies were selected through a multistep process (see Supplementary File 1): (1) The senior author and colleagues with expertise in schools, implementation science, and special education reviewed the 75 SISTER strategies and eliminated strategies that were already embedded in the implementation package being modified in the larger project, resulting in a narrowed list of 58 strategies. (2) The remaining 58 implementation strategies were examined for conceptual relevance and applicability for serving special education students. An additional 16 strategies were removed from consideration. (3) The importance and feasibility of the 42 remaining implementation strategies were considered using average ratings provided by Lyon et al. (2019). The 42 strategies were first ordered by importance. Then, we examined the feasibility of strategies with importance ratings above the median value. Strategies for which the average feasibility rating was below 3 (range = 1–5) were eliminated from consideration. (4) This final list included 15 implementation strategies spanning five categories: training and educating stakeholders, using evaluative and iterative strategies, developing stakeholder relationships, supporting educators, and changing infrastructure.
Replicating processes used in the ERIC (Waltz et al., 2015) and SISTER (Lyon et al., 2019) studies, participants were given separate surveys to rate the importance and feasibility of the 15 selected strategies. Each survey provided: (1) a definition of importance (i.e., how critical the implementation strategy is to successful implementation efforts) or feasibility (i.e., the extent to which a strategy is practical and can be successfully used to support implementation) from Lyon et al. (2019), and (2) a definition of each implementation strategy from C. R. Cook et al. (2019). Participants rated the degree to which each strategy was important (1 = relatively unimportant to 5 = extremely important) and feasible (1 = not at all feasible to 5 = extremely feasible) for implementation efforts at their school (Lyon et al., 2019; Waltz et al., 2015).
Interview Part 3: Qualitative Responses to Top Three Implementation Strategies
Next, the interviewer and participant visually examined the participant's ratings to identify the top three implementation strategies to discuss. Strategies with the highest ratings (i.e., “5” for both importance and feasibility) were identified. If only three strategies were identified, then those were discussed. If more than three strategies were identified with equally high ratings (e.g., there was a tie between two strategies’ ratings), participants were asked to pick which of those they wanted to discuss. If fewer than three strategies were identified with the highest ratings, participants were also asked to pick an additional strategy or strategies to discuss.
Data Analyses
Qualitative Analyses
Audio recordings of interviews were deidentified, transcribed verbatim, and imported into MAXQDA. Interviewer field notes supported clarity and interpretation of transcripts. Data were analyzed using constant comparison coding methods and grounded theory. Multiple steps were taken to enhance trustworthiness. Specifically, the coding team included three independent coders (an master’s-level research coordinator and a bachelor’s-level research assistant led by a doctoral student with extensive experience with qualitative analysis) who were not involved in study development or data collection. The codebook was developed iteratively, such that additional codes were added and codes were refined throughout the process. Coders first reviewed a subset of interviews (n = 4) to familiarize themselves with the data. A combination of inductive and deductive coding practices was used. Some codes were identified a priori (e.g., “importance of implementation strategies”), while other codes were developed inductively, based on ideas that emerged during the coding process (e.g., “access to resources”). All interviews were coded independently. Consensus meetings were held to discuss discrepancies, finalize codes, and update the codebook to reflect coder agreement; an audit trail was used to track the rationale for codebook changes. Interrater reliability following initial coding was 74%. Upon completion of the coding process, the coding team developed overarching categories and themes to organize the final codebook by topic area. Our reporting of qualitative results was guided by the Standards for Reporting Qualitative Research checklist (O’Brien et al., 2014).
Quantitative Analyses
Descriptive analyses (mean, standard deviation, and range) provided information regarding school leaders’ perceptions of the importance and feasibility of implementation strategies. The frequency with which participants identified each strategy as one of their top three during the qualitative interview was also generated.
Mixed Methods Analyses
In mixed methods, quantitative and qualitative data are collected and analyzed, and then integrated (Creswell, 2022). We used a convergent mixed method design such that the quantitative (QUAN) and qualitative (QUAL) data were collected concurrently and given equal weight (QUAL + QUAN; Palinkas et al., 2011). Integration involved comparing the results from the qualitative and quantitative data by merging, which provided a more complete understanding than what would have been provided by the qualitative or quantitative results alone (Creswell, 2022).
First, the quantitative and qualitative data were analyzed independently, and inferences were drawn from each of these results. Additionally, the qualitative data were quantified to present the frequency of each code (i.e., the number of times the code was assigned across all transcripts; Tables 2 and 4); this provided information regarding the saliency of each code (Landrum & Garza, 2015). Second, both data strands were merged to integrate findings. A joint display was developed that linked qualitative themes to related quantitative constructs to facilitate integration (Guetterman et al., 2015). Third, we generated meta-inferences by merging the data.
Themes and Subthemes of Factors Facilitating and Hindering Implementation With Illustrative Quotes
Results
What Factors Facilitate EBP Implementation in Special Education?
Theme 1: Professional Development and Training
This theme captured PD opportunities and training resources that supported implementation. Specific codes included resources/access to resources, PD opportunities, and teacher training. Table 2 presents illustrative quotes and code frequencies.
Resources/Access to Resources
School leaders identified a variety of resources/access to resources including receiving support from web-based training modules (e.g., about autism interventions) and video models. Others described specific types of resources such as a book study, during which teachers worked in small groups and focused on several topics (e.g., behavior management). Another participant described an “avatar lab” which allowed teachers to practice conversations with families. Finally, participants highlighted the importance of understanding how to distinguish accurate versus inaccurate information.
Professional Development Opportunities
School leaders discussed the importance of PD activities including “professional learning” or providing teachers with the necessary knowledge to teach a variety of topics. Participants also mentioned engaging in PD outside of the school to gain information about specific interventions and have greater exposure to different EBPs. They described attending weekly PD meetings as well as collaborative planning meetings, which included training and support for professional practices. One participant mentioned engaging with lead teachers; lead teachers were provided with a stipend and opportunities to gain leadership experience by providing PD support to newer teachers/staff.
Teacher Training
PD support also included teacher trainings that covered a range of topics, including “on-the-job” training with Board Certified Behavior Analysts, developing goals/objectives, addressing student behavior with parents, building classroom structure, behavioral consultation and support, live practice and support, access to conferences on specific topics (e.g., identifying EBPs and specific childhood disorders), and training on the use of specific interventions (e.g., social stories). Additionally, teachers received mentorship from “mentor teachers” and/or were given a half day to complete training from experts outside of the school. Leaders believed that teacher training was a very important component of providing EBP support.
Theme 2: Collaboration and Working in Teams
This theme captured facilitating factors related to collaboration and working in teams across staff and leadership at the schools.
School Supports
School supports were described as “robust” and included several built-in social support systems such as “team-led environments” and supportive teams, support from the district, use of walk-throughs and training, meetings to review EBPs, receiving mentorship, quarterly feedback from staff, and open communication across teachers and the team. Additionally, monthly community meetings among the teachers allowed school staff to focus on implementation each month.
Meetings
School leaders mentioned the use of weekly team meetings with teachers and paraprofessionals, special team meetings for focused discussions on a student or a student's programing, and meetings with administrators. Staff meetings covered a range of topics, including programing/interventions, developing goals/objectives, discussions around training/support needed to provide relevant programing, working with parents, identifying priorities, weekly planning, and preparing for behavioral concerns. Weekly meetings were instrumental in providing new teachers with opportunities to learn about EBPs and how to “apply these interventions…or procedures.” Other meetings included monthly meetings for different staff, including all-teacher meetings, meetings for assistant teachers, and meetings for program aids. Supervision meetings were held with leadership and curriculum/instructional/assessment specialists to address teaching practices, use of assessment data, IEPs, and classroom interventions.
Support From Nonteaching Staff
Nonteaching staff reportedly contributed greatly to the implementation of EBPs. School leaders mentioned teachers working in collaboration with (1) curriculum experts who helped set up programming and addressed academic issues, (2) behavioral teams who supported behavior interventions, (3) speech-language pathologists, and (4) committees that helped evaluate best practices and program-wide implementation. The important role of IEP chairs and autism coordinators was also described. Curriculum, instruction, and assessment teams supported teachers when developing IEPs and conducting assessments, often meeting weekly to discuss academic programming. Communication and collaboration between nonteaching staff and classroom teachers were noted as primary facilitators.
Culture/Climate
School leaders described the culture/climate of their organization as “unique,” “accepting,” “welcoming,” “positive,” and “safe.” There was great emphasis on the need for all stakeholders to collaborate, including embracing/celebrating individual differences while trying to be as adaptive and functional as possible. Some participants discussed the need for an “open door policy” to build better parent–teacher relationships. Leaders indicated that conveying appropriate expectations and encouraging continued education and PD among staff members immensely contributed to a successful climate/culture. Lastly, culture/climate was linked to the school's size and the “population served”; one participant explained “I think for our specific school that the culture climate is much different than in other schools just given the student population that we serve. I think that is very difficult to maintain a positive culture all the time.”
Family/Student-Level Factors
Participants discussed the need to collaborate with parents/caregivers when engaging in service delivery. Leaders attempted to connect with parents at the start of the academic year to ensure that parents felt comfortable about where their children were spending their day. Given that students come from different cultural backgrounds and have different learning styles, participants highlighted the need to collaborate with parents to meet unique student needs and ensure consistency in activities between home and school. Recommended steps for building positive relationships with families included: (1) introducing yourself to parents/caregivers during IEP meetings, (2) showing parents around to make them feel comfortable in an unfamiliar environment, and (3) conducting parent training to help address concerns at home.
Teacher/Staffing-Level Factors
Participants who viewed their school's culture/climate as positive, also described their staff as “caring,” “tenacious,” “friendly,” and “problem-solvers.” School leaders highlighted the need to positively acknowledge their staff's efforts, rather than focusing on the negatives, to empower teachers and encourage them to attempt new practices. Participants emphasized the need to have staff with a collaborative mindset to work with parents and general educators, and to cater to individual student needs. Leaders discussed the importance of staff actively communicating with each other and working as a team, and for schools to provide PD and training to inculcate staff into organizational culture and keep them updated.
What Barriers Impede EBP Implementation in Special Education?
Theme 3: Barriers to Implementation
This theme captured several factors, including lack of school support, culture/climate, organization-level factors, and teachers/staff-level factors.
Lack of School Supports
Some school leaders noted that there was a lack of support for the implementation of EBPs. While participants did not indicate who they perceived a lack of support from, common issues mentioned included schools being too focused on test scores, curriculum, and a lack of communication throughout the school leading to confusion among teachers.
Culture/Climate
One participant mentioned the difficulty in maintaining a consistently good culture/climate when there is a great deal of staff turnover. Staff turnover was mentioned often as being a consequence of the nature of special education. Another participant mentioned the culture/climate as being a “work in progress” and something they were “always striving to do better.”
Organization-Level Factors
A subcode of culture/climate factors included several organizational challenges, including a lack of consistent and transparent communication across the school, inadequate financial support, and limited resources. Participants highlighted the need for frequent and smooth communication between school-based teams to ensure that good ideas did not get overlooked and could be successfully implemented. One participant noted that “administrators can do a lot of things to maintain a positive work culture and collaborative environment.” One leader noted that being a Positive Behavioral Interventions and Supports school helped them shift the focus to school-wide positive approaches, which in turn helped their culture.
Teachers/Staffing-Level Factors
One major barrier to the implementation of EBPs was staff burnout. This is often due to staff turnover, especially of classroom staff (i.e., lead teachers and paraprofessionals). Leaders noted that teachers often felt stressed/overwhelmed due to understaffing and being overworked. Managing difficult behaviors was noted as a reason that teachers and staff felt burnt out.
What Are Important and Feasible Implementation Strategies?
Descriptive Statistics for Implementation Strategy Importance and Feasibility
Note. Implementation strategies are organized by category (italicized text), as identified in the SISTER project (C. R. Cook et al., 2019). The possible response range for strategy importance was 1 = relatively unimportant to 5 = extremely important and for strategy feasibility was 1 = not at all feasible to 5 = extremely feasible. Interview ranking describes the frequency with which school leaders identified each strategy as one of the top three strategies during the qualitative interview. SISTER = School Implementation Strategies, Translating ERIC Resources.
Average importance ratings were highest for provide ongoing consultation/coaching (M = 4.8), conduct ongoing training (M = 4.7), and monitor the progress of the implementation effort (M = 4.6). The least variability was observed in the ratings of provide ongoing consultation and coaching (SD = 0.4). Build partnerships to support implementation, conduct educational meetings, and change/alter environment were rated as relatively less important (M = 3.8, 4.0, 4.0, respectively), with slightly more variability (SD = 0.9) in participants’ ratings.
Conduct educational meetings, change/alter environment, conduct ongoing training, and provide ongoing consultation/coaching were rated as relatively more feasible (M = 3.9, 3.8, 3.8, 3.8, respectively). Ratings were the most variable (SD = 1.1) for conduct educational meetings. Build partnerships to support implementation, facilitate relay of intervention fidelity and student data to school personnel, and monitor the progress of the implementation effort (M = 3.2, 3.2, 3.3, respectively) were rated as relatively less feasible.
Overall, participant ratings indicated that provide ongoing consultation/coaching was both important and feasible. Monitor the progress of the implementation effort, in contrast, was rated as one of the more important strategies, but was also considered less feasible. Similarly, conduct educational meetings and change/alter the environment were rated, on average, to be relatively less important but more feasible. Build partnerships to support implementation was rated as less important and feasible.
Discussion
School leaders, including administrative and practice leaders, play an integral role in the use of implementation strategies, which in turn support special education teachers in the implementation of EBPs. This convergent QUAL + QUAN mixed methods study is a first attempt to investigate school leaders’ perceptions of the facilitators and barriers to EBP implementation, particularly for students receiving special education, as well as the importance and feasibility of 15 implementation strategies.
Facilitators and Barriers Identified Through Qualitative Data
Facilitators primarily included access to resources related to interventions/EBPs (e.g., training and PD), as well as collaboration and teamwork (e.g., consistent communication between teachers, nonteaching staff, and family members; team meetings). Our findings about special education are consistent with the CFIR determinant framework (Damschroder et al., 2022a) that highlights the relevance of inner-setting factors. Specifically, facilitators related to Professional Development and Training (Theme 1) are consistent with multiple CFIR inner-setting domains. School leaders discussed the importance of available resources (money, training, and education) for EBP implementation, as well as access to knowledge and information (e.g., about the intervention/EBP and its contextual fit). Similarly, the second theme, Collaboration and Working in Teams, includes determinants in the domains of relational connections and communications, which refer to formal or informal relationships, networks, and teams as well as information-sharing practices within organizations.
School leaders’ perceptions of barriers also included inner-setting determinants. For example, participants reported that implementation of EBPs was not prioritized, consistent with the relative priority domain (i.e., individuals responsible for implementation may not view a novel innovation as a priority) as well as with implementation climate, which was classified as a critical antecedent for implementation in the revised CFIR and CFIR outcomes addendum (Damschroder et al., 2022a, 2022b). Lastly, structural characteristics were noted (e.g., high staff turnover and burnout) as factors hindering implementation.
Important and Feasible Implementation Strategies Identified Through Quantitative Data
Our participants’ ratings of the importance and feasibility of 15 selected implementation strategies were relatively restricted in range, paralleling prior research (Waltz et al., 2015). On average, leaders rated provide ongoing consultation/coaching, conduct ongoing training, and monitor the progress of the implementation effort as relatively most important, and provide ongoing consultation/coaching, conduct ongoing training, conduct educational meetings, and change/alter environment as relatively most feasible. Our findings are consistent with prior research investigating implementation strategy importance and feasibility in general education settings (Lyon et al., 2019) and with broader trends in school-based implementation research (Lyon & Bruns, 2019). Individual, education-focused implementation strategies, such as direct training (Fallon et al., 2017), consultation (Noell et al., 2005), and coaching (Pas et al., 2022), are commonly investigated in school-based implementation research and used during intervention trials (Moore et al., 2021). In research with school-based implementation consultants, education-focused strategies were also rated to be the most important and feasible (Lyon et al., 2019). However, school leaders’ mean importance and feasibility ratings were, on average, slightly higher in this study.
One strategy—build partnerships to support implementation—was rated to be relatively less important and feasible. This finding suggests that although researchers continuously seek to establish partnerships with schools, school leaders may view partnerships as less pertinent than other strategies. Misalignment in the priorities and timelines of researchers with those of community partners can impede partnership effectiveness. As such, researchers are encouraged to move toward embedded research approaches that prioritize partners’ needs (Beidas et al., 2022). School leaders considered two additional strategies—facilitate relay of intervention fidelity and student data to school personnel and monitor the progress of the implementation effort—to be less feasible. Despite the priority placed on data-based decision-making processes in schools (Lyon & Bruns, 2019), this finding indicates a need to improve school data systems and build capacity for school leaders and educators to effectively use data to guide implementation.
Integration of Qualitative and Quantitative Findings through Mixed Methods
After completing the importance and feasibility surveys, leaders identified three strategies that they considered most important and/or feasible and discussed how these were implemented. All 15 strategies were described by at least one participant. The top strategies participants discussed varied somewhat from the quantitative ratings. Whereas provide ongoing consultation/coaching and conduct ongoing training continued to be described as one of the top three strategies by several leaders (n = 8 and 6, respectively), model and simulate change was one of the top strategies that participants discussed (n = 7). Additionally, change/alter environment was discussed by eight leaders despite a relatively lower importance rating.
Factors Influencing Implementation
Consistent with qualitative findings about the importance of PD, training, and related resources, school leaders quantitatively rated and discussed strategies supporting EBP implementation through training/education to be some of the most important and feasible. Yet, leaders’ awareness of inner-setting factors that impede EBP implementation in special education was not prominently reflected in the strategies they rated as important for supporting EBP implementation. Research suggests that the organizational context is important for implementation (Forman & Selman, 2011; Locke et al., 2019a), and indicates that implementation strategies are most effective when tailored to relevant implementation determinants (Baker et al., 2015). Together, prior literature and our findings indicate that there is a need to consider a broader range of implementation strategies, including organizational-level strategies (e.g., bolstering school-wide communication).
Importance and Feasibility of Implementation Strategies
Joint Display of Most Discussed Implementation Strategies by Importance and Feasibility
QUAL = Qualitative; QUAN = Quantitative.
Limitations and Future Directions
There are important limitations to note. First, participants in this study only rated 15 implementation strategies across five categories, potentially leading to the oversight of other relevant strategies. Future studies that examine school leaders’ perceptions of a comprehensive range of strategies using qualitative and quantitative methods can build upon our work. Second, this study's small sample of school leaders were predominantly female, White, and recruited from elementary schools in one Mid-Atlantic state. These leaders were recruited from a larger project focused on modifying an implementation strategy package to support EBPs with autistic students in special education. Thus, results may have limited transferability to other school settings or to racially, ethnically, and linguistically diverse leader and student populations, and may reflect perceptions of leaders who value EBPs. Additional research with representative samples of school leaders is needed.
Third, in contrast to prior research with school principals, our sample included school leaders with varied roles (administrators and service providers) in either public or nonpublic school settings and who likely have varied experiences with implementing EBPs. This sample heterogeneity may have contributed to the variability and restricted ranges observed in leaders’ strategy ratings. Our findings should be interpreted with these characteristics in mind. Individuals’ perceptions of the implementation context may differ depending on their role (Locke et al., 2019b). Thus, future research with larger samples should investigate whether leaders’ perceptions of the implementation context and strategies vary by their role, school setting, or prior experiences, as well as whether leaders’ perceptions differ from perceptions of educators delivering classroom interventions. Finally, although the time frame for the study did not enable participants to review the accuracy of their transcripts and findings, members of an advisory board for the overarching project were able to provide feedback on the findings.
Implications
Despite their recognition of the importance of inner-setting factors for EBP implementation in special education, few of the school leaders in this study prioritized implementation strategies that may be capable of addressing these implementation determinants. Inner-setting factors and related strategies have received insufficient attention in school contexts (Lyon & Bruns, 2019), suggesting that school leaders may be less familiar with strategies addressing these determinants. Future researchers may collect data on leaders’ experiences with various strategies to investigate whether prior exposure affects how strategies are evaluated and prioritized. To fill potential knowledge or experience gaps, implementation science concepts can be integrated into training and certification programs for school leaders and in-service training opportunities (Forman et al., 2013; Pauling et al., 2021). Building school leaders’ knowledge and skills for identifying and addressing implementation determinants at multiple levels as part of these trainings is important.
While implementation scientists recommend that feasibility and importance ratings may guide the selection of implementation strategies (Lyon et al., 2019; Waltz et al., 2015), our mixed method findings suggest that school leaders’ ratings of importance and feasibility may not directly align with their beliefs about the most important strategies for EBP implementation in special education. Prior work suggests that a range of factors influence the selection of implementation strategies to address target determinants (Waltz et al., 2019). Future research that investigates school leaders’ decision-making processes surrounding implementation strategy selection (beyond feasibility and importance) and how implementation strategies address targeted determinants is needed (Powell et al., 2019). Investigations that leverage explanatory sequential mixed method designs (e.g., Bustos et al., 2021) may illuminate how importance and feasibility considerations, as well as leaders’ role and prior experience with EBPs and implementation strategies, contribute to school leaders’ implementation planning and strategy selection. In practice, integrating a systematic process for identifying determinants and tailoring strategies (e.g., implementation mapping) may benefit school-based implementation of EBPs in special education (Fernández et al., 2019). Importantly, the observed discrepancies in this study also signal potential incompatibility between the complex concepts, frameworks, theories, and terminology applied in implementation science and those used by school-based implementation partners (Beidas et al., 2022). Both research- and practice-based evidence, generated through stakeholder-driven partnerships, will be needed to inform the use of tailored implementation strategies and, ultimately, to support high-quality implementation of EBPs in schools.
Supplemental Material
sj-docx-1-irp-10.1177_26334895231220279 - Supplemental material for The perspective of school leaders on the implementation of evidence-based practices: A mixed methods study
Supplemental material, sj-docx-1-irp-10.1177_26334895231220279 for The perspective of school leaders on the implementation of evidence-based practices: A mixed methods study by Stephanie A. Moore, Aksheya Sridhar, Isabella Taormina, Manasi Rajadhyaksha and Gazi Azad in Implementation Research and Practice
Footnotes
Author Contributions
Stephanie A. Moore, PhD (she/her), is an assistant professor of school psychology whose research is focused on school mental health systems and implementation science. She has conducted primarily quantitative and survey-based research and has experience with direct service provision in schools. She coconceptualized this study's aims, led the quantitative analyses, wrote the quantitative portions of the method and results, wrote the discussion, and reviewed all sections of the manuscript. Aksheya Sridhar (she/her) is a doctoral candidate in clinical science, and focuses her research at the intersection of autism services and implementation science. She has conducted primarily qualitative and mixed methods research. She led the qualitative analysis, wrote the qualitative portions of the method and results, and wrote the discussion for this study. Isabella Taormina (she/her) is a research coordinator within the division of child and adolescent psychiatry and works primarily on school-based implementation research. She conducted qualitative coding, wrote the qualitative portion of the results, and wrote the introduction for this study. Manasi Rajadhyaksha (she/her) is a doctoral student in school psychology, with a research focus on school-based mental health and implementation science. She has conducted primarily quantitative and mixed methods research. She conducted qualitative coding and wrote the introduction for this study. Gazi Azad is an assistant profession in the division of child and adolescent psychiatry. She is the PI on the grant that funded this study. She conceptualized the aims, conducted the interviews, and facilitated with all components of the manuscript (i.e., helped write the introduction, methods, and discussion, as well as supervised quantitative and qualitative results).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported in part by a Faculty-Community Award from the Urban Health Institute at Johns Hopkins University (74210). Preparation of this article was supported in part by grants from the National Institute of Mental Health (5K23 MH1119331). Open access publishing of this article was supported in part by the University of California libraries.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
