Exploring Organizational Differences in Perceptions of Implementation Climate and Leadership in Schools: A Mixed Methods Study of Autism EBP Implementation

Abstract

Public programs have been charged with implementation of evidence-based practices (EBPs) to improve outcomes for children with autism. However, research indicates that scale-up of EBPs poses challenges. This study identifies perceived variables linked to effective statewide scale-up of EBPs in special education by exploring implementation climate and leadership across special education organization types (e.g., schools, districts, and regional consortiums). A simultaneous QUAL + QUAN mixed methods design was employed with the primary function of convergence and triangulation. Data were drawn from focus groups with 30 special educators and a statewide survey completed by 656 school personnel in one U.S. state. In general, perceptions of implementation climate and leadership are weak in special education organizations, with strengths at regional levels focused on special education and increased challenges at the school and district levels. Implications for practice and future research are identified.

Keywords

autism implementation climate implementation leadership evidence-based practices

Get full access to this article

View all access options for this article.

References

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

(2014). The Implementation Leadership Scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), Article 45. https://doi.org/10.1186/1748-5908-9-45

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

Hurlburt

M. S.

(2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 1–12. https://doi.org/10.1186/s13012-014-0192-y

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

Sklar

Horowitz

(2017). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health and Mental Health Services Research, 44(4), 480–491. https://doi.org/10.1007/s10488-015-0672-7

Aarons

G. A.

Horowitz

J. D.

Dlugosz

L. R.

Ehrhart

M. G.

(2012). The role of organizational process in dissemination and implementation research. In Brownson

R. C.

Colditz

G. A.

Proctor

E. K.

(Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 128–153). Oxford University Press.

Aarons

G. A.

Hurlburt

Horwitz

S. M.

(2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7

Brookman-Frazee

Stahmer

A. C.

(2018). Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science, 13(1). https://doi.org/10.1186/s13012-018-0757-2

California Autism Professional Training and Information Network. (2017). Welcome. http://www.captain.ca.gov/

Ehrhart

M. G.

Aarons

G. A.

Farahnak

L. R.

(2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 157–168. https://doi.org/10.1186/s13012-014-0157-1

Fixsen

Blase

Metz

Van Dyke

(2013). Statewide implementation of evidence based programs. Exceptional Children, 79(2), 213–230. https://doi.org/10.1177/001440291307900206

10.

Gibbs

(1997). Focus groups. Social Research Update, 19(8), 1–8. https://sru.soc.surrey.ac.uk/SRU19.html

11.

Green

A. E.

Albanese

B. J.

Cafri

Aarons

G. A.

(2014). Leadership, organizational climate, and working alliance in a children’s mental health service system. Community Mental Health Journal, 50(7), 771–777. https://doi.org/10.1007/s10597-013-9668-5

12.

Hamilton

A. B.

Finley

E. P.

(2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, Article 112516. https://doi.org/10.1016/j.psychres.2019.112516

13.

Harvey

Fitzgerald

Fielden

McBride

Waterman

Bamford

Kislov

Boaden

(2011). The NIHR collaboration for leadership in applied health research and care (CLAHRC) for Greater Manchester: Combining empirical, theoretical, and experiential evidence to design and evaluate a large-scale implementation strategy. Implementation Science, 6(1), Article 96. https://doi.org/10.1186/1748-5908-6-96

14.

Hess

K. L.

Morrier

M. J.

Heflin

L. J.

Ivey

M. L.

(2008). Autism treatment survey: Services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders, 38(5), 961–971. https://doi.org/10.1007/s10803-007-0470-5

15.

Kratz

H. E.

Stahmer

Xie

Marcus

S. C.

Pellecchia

Locke

Beidas

Mandell

D. S.

(2019). The effect of implementation climate on program fidelity and student outcomes in autism support classrooms. Journal of Consulting and Clinical Psychology, 87(3), 270–281. https://doi.org/10.1037/ccp0000368

16.

Lasky

Karge

B. D.

(2006). Meeting the needs of students with disabilities: Experience and confidence of principals. NASSP Bulletin, 90(1), 19–36. https://doi.org/10.1177%2F0192636505283950

17.

Locke

Lawson

G. M.

Beidas

R. S.

Aarons

G. A.

Xie

Lyon

A. R.

Stahmer

Seidman

Fredrick

Spaulding

Dorsey

Mandell

(2019). Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: A cross-sectional observational study. Implementation Science, 14(1), Article 29. https://doi.org/10.1186/s13012-019-0877-3

18.

Lyon

A. R.

Cook

C. R.

Brown

E. C.

Locke

Davis

Ehrhart

Aarons

G. A.

(2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13(1), 1–14. https://doi.org/10.1186/s13012-017-0705-6

19.

Maenner

M. J.

Shaw

K. A.

Baio

(2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69(4), 1. https://doi.org/10.15585/mmwr.ss6904a1

20.

Melgarejo

Lind

Stadnick

N. A.

Helm

J. L.

Locke

(2020). Strengthening capacity for implementation of evidence-based practices for autism in schools: The roles of implementation climate, school leadership, and fidelity. American Psychologist, 75(8), 1105–1115. https://doi.org/10.1037/amp0000649

21.

Merton

R. K.

Kendall

P. L.

(1946). The focused interview. American Journal of Sociology, 51(6), 541–557. https://doi.org/10.2307/2088873

22.

Novins

D. K.

Green

A. E.

Legha

R. K.

Aarons

G. A.

(2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child & Adolescent Psychiatry, 52(10), 1009–1025. https://doi.org/10.1016/j.jaac.2013.07.012

23.

Odom

S. L.

Sam

A. M.

Tomaszewski

(2022). Factors associated with implementation of a school-based comprehensive program for students with autism. Autism, 26(3), 703–715. https://doi.org/10.1177/13623613211070340

24.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

(2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z

25.

Pellecchia

Connell

J. E.

Beidas

R. S.

Xie

Marcus

S. C.

Mandell

D. S.

(2015). Dismantling the active ingredients of an intervention for children with autism. Journal of Autism and Developmental Disorders, 45(9), 2917–2927. https://doi.org/10.1007/s10803-015-2455-0

26.

QSR International. (2012). NVivo 10 [Computer software]. https://www.qsrinternational.com/nvivo/home

27.

Robins

C. S.

Ware

N. C.

dosReis

Willging

C. E.

Chung

J. Y.

Lewis-Fernandez

(2008). Dialogues on mixed methods and mental health services research: Anticipating challenges, building solutions. Psychiatric Services, 59, 727–731. https://doi.org/10.1176%2Fappi.ps.59.7.727

28.

Sam

A. M.

Cox

A. W.

Savage

M. N.

Waters

Odom

S. L.

(2020). Disseminating information on evidence-based practices for children and youth with autism spectrum disorder: AFIRM. Journal of Autism and Developmental Disorders, 50(6), 1931–1940. https://doi.org/10.1007/s10803-019-03945-x

29.

Sanetti

L. M. H.

Collier-Meek

M. A.

(2019). Increasing implementation science literacy to address the research-to-practice gap in school psychology. Journal of School Psychology, 76, 33–47. https://doi.org/10.1016/j.jsp.2019.07.008

30.

Snyder

T. D.

De Brey

Dillow

S. A.

(2019). Digest of education statistics 2017 (NCES 2018-070). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

31.

Spillane

(2006). Distributed leadership. Jossey-Bass.

32.

Stahmer

A. C.

Brookman-Frazee

Lau

Roesch

Caplan

(2022, September 9). Testing a multi-level implementation strategy for two evidence-based autism interventions: Preliminary results [Paper Presented in L. Brookman-frazee (Chair) Symposium: Utilizing the TEAMS implementation-effectiveness trial to assess the joint impact of multilevel implementation strategies and implementation context on autism EBI implementation outcomes in schools and child mental health services]. Society for Implementation Research Collaboration Conference, San Diego, CA, United States.

33.

Stahmer

A. C.

Suhrheinrich

Schetter

P. L.

Hassrick

E. M.

(2018). Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): A study protocol. Implementation Science, 13, Article 3. https://doi.org/10.1186/s13012-017-0698-1

34.

Steinbrenner

J. R.

Hume

Odom

S. L.

Morin

K. L.

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yücesoy-Özkan

Savage

M. N.

(2020). Evidence-based practices for children, youth, and young adults with autism. The University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Institute, National Clearinghouse on Autism Evidence and Practice Review Team.

35.

Suhrheinrich

Rieth

S. R.

Dickson

K. S.

Stahmer

A. C.

(2020a). Exploring associations between inner-context factors and implementation outcomes. Exceptional Children, 86(2), 155–173. https://doi.org/10.1177%2F0014402919881354

36.

Suhrheinrich

Schetter

England

Melgarejo

Nahmias

A. S.

Dean

Yasuda

(2020b). Statewide interagency collaboration to support evidence-based practice scale up: The California Autism Professional Training and Information Network (CAPTAIN). Evidence-Based Practice in Child and Adolescent Mental Health, 5, 468–482. https://doi.org/10.1080/23794925.2020.1796545

37.

Suhrheinrich

Stahmer

A. C.

Reed

Schreibman

Reisinger

Mandell

(2013). Implementation challenges in translating pivotal response training into community settings. Journal of Autism and Developmental Disorders, 43(12), 2970–2976. https://doi.org/10.1007/s10803-013-1826-7

38.

Thayer

A. J.

Brown

E. C.

Cook

C. R.

Ehrhart

Locke

Davis

Picozzi

Aarons

Lyon

A. R.

(2022). Construct validity of the school-implementation climate scale (SICS). Implementation Research and Practice, 3, 1–14. https://doi.org/10.1177/26334895221116065

39.

U.S. Department of Education. (2016, January 8). RDA: Results driven accountability. https://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

40.

Weiner

B. J.

Belden

C. M.

Bergmire

D. M.

Johnston

(2011). The meaning and measurement of implementation climate. Implementation Science, 6(1), Article 78. https://doi.org/10.1186/1748-5908-6-78

41.

Williams

N. J.

Hugh

M. L.

Cooney

D. J.

Worley

J. A.

Locke

(2022). Testing a theory of implementation leadership and climate across autism evidence-based interventions of varying complexity. Behavior Therapy, 53, 900–912. https://doi.org/10.1016/j.beth.2022.03.001

42.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Brock

M. E.

Plavnick

J. B.

Fleury

V. P.

Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z