Practice-driven research for statewide scale up: Implementation outcomes of the California Autism Professional Training and Information Network

Abstract

Scaling up the use of evidence-based practice (EBP) for autism across service sectors and regions has presented a considerable challenge indicating a clear need for continued development. The California Autism Professional Training and Information Network (CAPTAIN) integrates implementation drivers into specific procedures and methodology as an implementation strategy to support statewide scale up. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, and knowledge, fidelity and use of autism EBPs, and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. CAPTAIN-trained providers reported more openness to EBP. Significantly more CAPTAIN-trained direct service providers reported collecting fidelity of implementation data (χ²(2, N = 1515) = 10.95, p = 0.004), collecting student data (χ²(2, N = 1509) = 14.19, p = 0.001), and reported using their primary EBP with “most or all students” (χ²(2, N = 1514) = 11.41, p = 0.003) than providers not trained by CAPTAIN. In summary, these preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.

Lay abstract

Supporting use of evidence-based practice in public service programs for autistic individuals is critical. The California Autism Professional Training and Information Network (CAPTAIN) brings together best practices from intervention and implementation research to support scale up of autism services. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, knowledge, fidelity, and use of autism EBPs and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. These preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.

Keywords

education services implementation science professional development scale up

Get full access to this article

View all access options for this article.

References

Aarons

G. A.

(2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74. https://doi.org/10.1023/b:mhsr.0000024351.12294.65

Aarons

G. A.

Hurlburt

Horwitz

S. M.

(2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7

Beidas

R. S.

Edmunds

Ditty

Watkins

Walsh

Marcus

Kendall

(2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 788–799. https://doi.org/10.1007/s10488-013-0529-x

Brock

M. E.

Dynia

J. M.

Dueker

S. A.

Barczak

M. A.

(2020). Teacher-reported priorities and practices for students with autism: Characterizing the research-to-practice gap. Focus on Autism & Other Developmental Disabilities, 35, 67–78. https://doi.org/10.1177/1088357619881217

Brookman-Frazee

Baker-Ericzén

Chan

Dickson

Rieth

Haine-Schlagel

Stadnick

Stahmer

Suhrheinrich

(in press). Applying dissemination and implementation science to facilitate community implementation of evidence-based interventions. Handbook of autism and pervasive developmental disorder. Springer.

Brookman-Frazee

Baker-Ericzén

Stahmer

Mandell

Haine

R. A.

Hough

R. L.

(2009). Involvement of youths with autism spectrum disorders or intellectual disabilities in multiple public service systems. Journal of Mental Health Research in Intellectual Disabilities, 2(3), 201–219. https://doi.org/10.1080/19315860902741542

California Department of Education. (n.d.). California’s longitudinal pupil achievement data system. https://www.calpads.org

Cook

C. R.

Davis

Brown

E. C.

Locke

Ehrhart

M. G.

Aarons

G. A.

Larson

Lyon

A. R.

(2018). Confirmatory factor analysis of the Evidence-Based Practice Attitudes Scale with school-based behavioral health consultants. Implementation Science, 13(1), Article 116. https://doi.org/10.1186/s13012-018-0804-z

Dynia

J. M.

Walton

Brock

M. E.

Tiede

(2020). Early childhood special education teachers’ use of evidence-based practices with children with autism spectrum disorder. Research in Autism Spectrum Disorders, 77, Article 101606. https://doi.org/10.1016/j.rasd.2020.101606

10.

Eccles

M.P.

Mittman

B.S.

(2006) Welcome to Implementation Science . Implementation Sci 1, 1. https://doi.org/10.1186/1748-5908-1-1

11.

Ehrhart

M. G.

Aarons

G. A.

Farahnak

L. R.

(2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science: IS, 10, Article 65. https://doi.org/10.1186/s13012-015-0255-8

12.

Every Student Succeeds Act. (2015). Public Law 114—95—GovInfo. https://www.govinfo.gov/app/details/PLAW-114publ95

13.

Fixsen

D. L.

Blasé

K. A.

Metz

Dyke

M. V.

(2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230. https://doi.org/10.1177/001440291307900206

14.

Fixsen

D. L.

Blasé

K. A.

Timbers

G. D.

Wolf

M. M.

(2007). In search of program implementation: 792 replications of the Teaching-Family Model. The Behavior Analyst Today, 8(1), 96–110. https://doi.org/10.1037/h0100104

15.

Green

A. E.

Albanese

B. J.

Shapiro

N. M.

Aarons

G. A.

(2014). The roles of individual and organizational factors in burnout among community-based mental health service providers. Psychological Services, 11(1), 41–49. https://doi.org/10.1037/a0035299

16.

Harned

M. S.

Dimeff

L. A.

Woodcock

E. A.

Contreras

(2013). Predicting adoption of exposure therapy in a randomized controlled dissemination trial. Journal of Anxiety Disorders, 27(8), 754–762.

17.

Hess

Morrier

Heflin

Ivey

(2008). Autism treatment survey: Services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders, 38, 961–971. https://doi.org/10.1007/s10803-007-0470-5

18.

Individuals With Disabilities Education Act. (2004). Public Law 101-476, GovInfo. https://www.govinfo.gov/app/details/CREC-2004-11-20/CREC-2004-11-20-pt1-PgS11669

19.

Jobin

Stahmer

Dickson

K. S.

Nahmias

Chlebowski

Brookman-Frazee

(2018, November). Characterization of inner context implementation factors related to the implementation of evidence-based interventions for children with ASD: Comparing implementation climate and leadership in education and mental health settings [Conference session]. In K. S. Dickson (Chair), Examination of Implementation Leadership and Climate on Implementation in Schools and Community Mental Health Services. Symposium conducted at the 52nd Annual Convention of the Association for Behavioral and Cognitive Therapies, Washington, DC, United States.

20.

McGhee Hassrick

Freidman

Schetter

Melgarejo

Nahmias

Suhrheinrich

Stahmer

(2020, January). The impact of social networks on scaling up evidence-based autism practices in schools [Poster presentation]. Annual Principal Investigators Meeting for the Institute for Education Sciences, Washington, DC, United States.

21.

McNeill

(2019). Social validity and teachers’ use of evidence-based practices for autism. Journal of Autism and Developmental Disorders, 49, 4585–4594. https://doi.org/10.1007/s10803-019-04190-y

22.

Morrier

M. J.

Hess

K. L.

Heflin

L. J.

(2010). Teacher training for implementation of teaching strategies for students with autism spectrum disorders. Teacher Education and Special Education, 34, 119–132. https://doi.org/10.1177/0888406410376660

23.

National Professional Development Center on Autism Spectrum Disorder. (2011). https://autismpdc.fpg.unc.edu/national-professional-development-center-autism-spectrum-disorder

24.

Nelson

J. R.

Oliver

R. M.

Hebert

M. A.

Bohaty

(2015). Use of self-monitoring to maintain program fidelity of multi-tiered interventions. Remedial and Special Education, 36(1), 14–19. https://doi.org/10.1177/0741932514544970

25.

Novins

D. K.

Green

A. E.

Legha

R. K.

Aarons

G. A.

(2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52(10), 1009–1025.e1018. https://doi.org/10.1016/j.jaac.2013.07.012

26.

Odom

S. L.

Cox

Sideris

J. H.

Hume

K. A.

Hedges

Kucharczyk

Shaw

Boyd

B. A.

Reszka

S. S.

Neitzel

(2018). Assessing quality of program environments for children and youth with autism: Autism Program Environment Rating Scale (APERS). Journal of Autism and Developmental Disorders, 48, 913–924. https://doi.org/10.1007/s10803-017-3379-7

27.

Odom

S. L.

Cox

A. W.

Brock

M. E.

(2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79(2), 233–251. http://libproxy.sdsu.edu/login?url=https://www-proquest-com.libproxy.sdsu.edu/scholarly-journals/implementation-science-professional-development/docview/1270781690/se-2?accountid=13758

28.

Powell

B. J.

Waltz

T. J.

Chinman

M. J.

Damschroder

L. J.

Smith

J. L.

Matthieu

M. M.

Kirchner

J. E.

(2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 1–14.

29.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Hensley

(2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.

30.

Reding

M. E. J.

Chorpita

B. F.

Lau

A. S.

Innes-Gomberg

(2013). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter, too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1

31.

Ruble

McGrew

J. H.

Toland

M. D.

(2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42(9), 1974–1983. https://doi.org/10.1007/s10803-012-1446-7

32.

Snyder

T. D.

De Brey

Dillow

S. A.

(2019). Digest of education statistics 2018 (NCES 2020-009) (54th ed.). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. https://nces.ed.gov/pubs2020/2020009.pdf

33.

Stahmer

A. C.

Ingersoll

(2004). Inclusive programming for toddlers with autism spectrum disorders. Journal of Positive Behavior Interventions, 6(2), 67–82. https://doi.org/10.1177/10983007040060020201

34.

Steinbrenner

J. R.

Hume

Odom

S. L.

Morin

K. L.

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yücesoy-Özkan

Savage

M. N.

(2020). Evidence-based practices for children, youth, and young adults with Autism. National Clearinghouse on Autism Evidence and Practice Review Team, Frank Porter Graham Child Development Institute, The University of North Carolina at Chapel Hill.

35.

Suhrheinrich

(2011). Training teachers to use pivotal response training with children with autism: Coaching as a critical component. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 34(4), 339–349. https://doi.org/10.1177/0888406411406553

36.

Suhrheinrich

Rieth

S. R.

Dickson

K. S.

Stahmer

A. C.

(2020). Exploring associations between inner-context factors and implementation outcomes. Exceptional Children, 86(2), 155–173. https://doi.org/10.1177/0014402919881354

37.

Suhrheinrich

Schetter

England

Melgarejo

Nahmias

A. S.

Dean

Yasuda

(2020). Statewide interagency collaboration to support evidence-based practice scale up: The California Autism Professional Training and Information Network (CAPTAIN). Evidence-Based Practice in Child and Adolescent Mental Health, 5(4), 468–482. https://doi.org/10.1080/23794925.2020.1796545

38.

Suhrheinrich

Stahmer

Reed

Schreibman

Reisinger

E. M.

Mandell

D. S.

(2013). Implementation challenges in translating Pivotal Response Training into community settings. Journal of Autism and Developmental Disorders, 43, 2970–2976. https://doi.org/10.1007/s10803-013-1826-7

39.

Suhrheinrich

Stahmer

A. C.

Schreibman

(2007). A preliminary assessment of teachers’ implementation of Pivotal Response Training. The Journal of Speech and Language Pathology—Applied Behavior Analysis, 2(1), 1–13. https://doi.org/10.1037/h0100202

40.

Tabachnick

Fidell

(2018). Using multivariate statistics (7th ed.). Pearson.

41.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

Brock

M. E.

Plavnick

J. B.

Fleury

V. P.

Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z