Fidelity of a Teacher-Implemented Intervention for Preschoolers With Autism Spectrum Disorder: No,Some,and Unexpected Effects

Abstract

Advancing Social-Communication and Play (ASAP) is a classroom-based intervention targeting the social-communication and play development of preschool-age children with autism spectrum disorder. We used a multimethod approach to measuring intervention fidelity, including adherence, exposure, treatment differentiation, and quality of treatment delivery. Overall, teachers in the ASAP classrooms increased their adherence to the ASAP intervention compared with business-as-usual teachers. Teacher characteristics did not affect the ability to implement ASAP with fidelity. There were no group differences in coded video instructional quality scores. Differences in child engagement between the ASAP and business-as-usual groups were not mediated by fidelity adherence scores, contrary to our hypothesis. Finally, end-of-year summary of treatment delivery ratings did not align with adherence scores. Continued research regarding fidelity of school-based interventions should focus on distinguishing active ingredients from general instructional practices and learning from teachers about the child-level factors they consider when implementing an intervention.

Keywords

fidelity ASAP school interventions autism spectrum disorder

Get full access to this article

View all access options for this article.

References

Aarons

G. A.

Cafri

Lugo

Sawitzky

(2012). Expanding the domains of attitudes towards evidence-based practice: The Evidence Based Practice Attitude Scale–50. Administration and Policy in Mental Health and Mental Health Services Research, 39, 331–340.

Abry

Rimm-Kaufman

S. E.

Larsen

R. A.

Brewer

A. J.

(2013). The influence of fidelity of implementation on teacher–student interaction quality in the context of a randomized controlled trial of the Responsive Classroom approach. Journal of School Psychology, 51, 437–453.

Achenbach

T. M.

(1997). Guide for caregiver-teacher report form for ages 2–5. Burlington: Department of Psychiatry, The University of Vermont.

Aguiar

McWilliam

R. A.

(2013). Consistency of toddler engagement across two settings. Early Childhood Research Quarterly, 28, 102–110.

Berry

Axford

Blower

Taylor

R. S.

Edwards

R. T.

Tobin

. . . Bywater

(2016). The effectiveness and micro-costing analysis of a universal, school-based, social–emotional learning programme in the U.K.: A cluster-randomised controlled trial. School Mental Health, 8, 238–256.

Boyd

B. A.

Watson

L. R.

Reszka

S. S.

Sideris

Alessandri

Baranek

G. T.

. . . Belardi

(2018). Efficacy of the ASAP intervention for preschoolers with ASD: A cluster randomized controlled trial. Journal of Autism and Developmental Disorders, 48, 3144–3162. doi:10.1007/s10803-018-3584-z

Burchinal

Nelson

Poe

(2006). Growth curve analysis: An introduction to various methods for analyzing longitudinal data. In McCartney

Burchinal

Bub

(Eds.), Monographs of the Society for Research in Child Development: Vol. 71. Best practices in quantitative methods for developmentalists (pp. 65–87). Hoboken, NJ: Wiley-Blackwell.

Century

Rudnick

Freeman

(2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31, 199–218.

Classroom Practice Inventory. (2007). Comparison of two comprehensive treatment models for preschool-aged children with autism (Project staff of TEACCH/LEAP project). Chapel Hill, NC: Frank Porter Graham Child Development Institute.

10.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

11.

Dane

A. V.

Schneider

B. H.

(1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.

12.

Dykstra

Boyd

B. A.

Watson

L. R.

Crais

E. R.

Baranek

G. T.

(2012). The impact of the advancing social-communication and play intervention on preschoolers with autism spectrum disorders. Autism, 16, 33–49.

13.

Estes

Munson

Rogers

S. J.

Greenson

Winter

Dawson

(2015). Long-term outcomes of early intervention in 6-year-old children with autism spectrum disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 54, 580–587.

14.

Gresham

F. M.

MacMillan

D. L.

Beebe-Frankenberger

M. E.

Bocian

K. M.

(2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research & Practice, 15, 198–205.

15.

Howlin

Moss

Savage

Rutter

(2013). Social outcomes in mid- to later adulthood among individuals diagnosed with autism and average nonverbal IQ as children. Journal of the American Academy of Child & Adolescent Psychiatry, 52, 572–581.

16.

Hume

Boyd

McBee

Coman

Gutierrez

Shaw

. . . Odom

(2011). Assessing implementation of comprehensive treatment models for young children with ASD: Reliability and validity of two measures. Research in Autism Spectrum Disorders, 5, 1430–1440.

17.

Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004).

18.

Institute of Education Sciences. (2018). RFA 84.324A. Retrieved from https://ies.ed.gov/funding/pdf/2019_84324A.pdf

19.

Kasari

(2002). Assessing change in early intervention programs for children with autism. Journal of Autism and Developmental Disorders, 32, 447–461.

20.

Kasari

Smith

(2013). Interventions in schools for children with autism spectrum disorder: Methods and recommendations. Autism, 17, 254–267.

21.

Kishida

Kemp

(2009). The engagement and interaction of children with autism spectrum disorder in segregated and inclusive early childhood center-based settings. Topics in Early Childhood Special Education, 29, 105–118.

22.

Lakens

. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article ID 863.

23.

Lord

Rutter

DiLavore

P. C.

Risi

. (1999). Autism Diagnostic Observation Schedule-WPS (ADOS-WPS), Los Angeles, CA: Western Psychological Services.

24.

Mandell

D. S.

Stahmer

A. C.

Shin

Xie

Reisinger

Marcus

S. C.

(2013). The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. Autism, 17, 281–295.

25.

Maslach

Jackson

S. E.

Schwab

(1986). Maslach Burnout Inventory Manual: Educators survey. Menlo Park, CA: Mind Garden.

26.

Mowbray

C. T.

Holter

M. C.

Teague

G. B.

Bybee

D. B.

(2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.

27.

Mullen

E. M.

(1995). Mullen scales of early learning. Circle Pines, MN: American Guidance Service.

28.

Odom

S. L.

Boyd

B. A.

Hall

L. J.

Hume

(2010). Evaluation of comprehensive treatment models for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 40, 425–436.

29.

O’Donnell

C. L.

(2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33–84.

30.

Pellecchia

Connell

J. E.

Beidas

R. S.

Xie

Marcus

S. C.

Kelleher

Mandell

D. S.

(2015). Dismantling the active ingredients of an intervention for children with autism. Journal of Autism and Developmental Disorders, 45, 2917–2927.

31.

Power

T. J.

Blom-Hoffman

Clarke

A. T.

Riley-Tillman

T. C.

Kelleher

Manz

P. H.

(2005). Reconceptualizing intervention integrity: A partnership-based framework for linking research with practice. Psychology in the Schools, 42, 495–507.

32.

Professional Development in Autism Center. (2008). PDA program assessment. Seattle: PDA Center at University of Washington.

33.

Sawilowsky

(2009). New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 8, 467–474.

34.

Schultes

M. T.

Jöstl

Finsterwald

Schober

Spiel

(2015). Measuring intervention fidelity from different perspectives with multiple methods: The Reflect program as an example. Studies in Educational Evaluation, 47, 102–112.

35.

Smith

S. W.

Daunic

A. P.

Taylor

G. G.

(2007). Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice. Education & Treatment of Children, 30, 121–134.

36.

Sparapani

Morgan

Reinhardt

V. P.

Schatschneider

Wetherby

A. M.

(2016). Evaluation of classroom active engagement in elementary students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 46, 782–796.

37.

Strain

P. S.

Bovey

(2008). LEAP preschool. In Handleman

Harris

(Eds.), Preschool education programs for children with autism (pp. 249–280). Austin, TX: Pro-Ed.

38.

Sutherland

K. S.

Conroy

M. A.

McLeod

B. D.

Algina

(2018). Teacher competence of delivery of BEST in CLASS as a mediator of treatment effects. School Mental Health, 10, 214–225.

39.

Sutherland

K. S.

McLeod

B. D.

Conroy

M. A.

Cox

J. R.

(2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders: Conceptual issues and recommendations. Journal of Early Intervention, 35, 129–149.

40.

Trochim

W. M.

Donnelly

J. P.

(2001). Research methods knowledge base. Cincinnati, OH: Atomic Dog.

41.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

. . . Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966.