Sampling Engagement in Shared Reading Activities With Children With Autism Spectrum Disorder: A Generalizability Study

Abstract

Researchers seeking to make valid conclusions about engagement for young children with autism spectrum disorder (ASD) must first determine the reliability of estimates obtained across the conditions sampled. Working from that premise, we conducted a secondary data analysis of shared book readings between caregivers and their children with ASD, examining the contribution of measurement error on estimates of four states of child engagement. Caregivers read two different book types, each three times, with their children. With book type and occasion as measurement facets, we conducted a generalizability study and a series of decision studies. With the interaction of Persons × Book Type × Occasion contributing the most measurement error for four engagement variables, we only found stable estimates for unengaged behaviors. For the active engagement, visual engagement, and disruptive variables, 4, 5, and more than 10 book types, respectively, were necessary to obtain stable estimates across two occasions.

Keywords

engagement autism spectrum disorders measurement shared book reading

Get full access to this article

View all access options for this article.

References

Adamson

L. B.

Bakeman

Deckner

D. F.

(2004). The development of symbol-infused active engagement. Child Development, 75(4), 1171–1187. https://doi.org/10.1111/j.1467-8624.2004.00732.x

Akhtar

Gernsbacher

M. A.

(2007). Joint attention and vocabulary development: A critical look. Language and Linguistics Compass, 1(3), 195–207. https://doi.org/10.1111/j.1749-818X.2007.00014.x

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders. https://doi.org/10.1176/appi.books.9780890425596

Atilgan

(2013). Sample size estimation of G and Phi coefficients in generalizability theory. Eurasian Journal of Educational Research, 51, 215–227.

Azur

M. J.

Stuart

E. A.

Frangakis

Leaf

P. J.

(2011). Multiple imputation by chained equations: What is it and how does it work? International Journal of Methods in Psychiatric Research, 20(1), 40–49. https://doi.org/10.1002/mpr.329

Berk

R. A.

(1979). Generalizability of behavioral observations: A clarification of interobserver agreement and interobserver reliability. American Journal of Mental Deficiency, 83(5), 460–472.

Bottema-Beutel

Kim

S. Y.

Crowley

Augustine

Kecili-Kaysili

Feldman

Woynaroski

(2019). The stability of active engagement states in infant siblings of children with and without ASD: Implications for measurement practices. Autism Research, 12(3), 495–504. https://doi.org/10.1002/aur.2068

Bottema-Beutel

Lloyd

Carter

E. W.

Asmus

J. M.

(2014). Generalizability and decision studies to inform observational and experimental research in classroom settings. American Journal on Intellectual and Developmental Disabilities, 119(6), 589–605. https://doi.org/10.1352/1944-7558-119.6.589

Bottema-Beutel

Malloy

Lloyd

B. P.

Louick

Joffe-Nelson

Watson

L. R.

Yoder

P. J.

(2017). Sequential associations between caregiver talk and child play in autism spectrum disorder and typical development. Child Development, 89(3), 157–166. https://doi.org/10.1111/cdev.12848

10.

Brennan

R. L.

(2000). (Mis)conception about generalizability theory. Educational Measurement: Issues and Practice, 19(1), 5–10. https://doi.org/10.1111/j.1745-3992.2000.tb00017.x

11.

Brennan

R. L.

(2001). Generalizability theory. Springer.

12.

Brennan

R. L.

(2003). Coefficients and indices in generalizability theory (Issue 1). Center for Advanced Studies in Measurement and Assessment.

13.

Briesch

A. M.

Swaminathan

Welsh

Chafouleas

S. M.

(2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52(1), 13–35. https://doi.org/10.1016/j.jsp.2013.11.008

14.

Brown

H. M.

Oram-Cardy

Johnson

(2013). A meta-analysis of the reading comprehension skills of individuals on the autism spectrum. Journal of Autism and Developmental Disorders, 43(4), 932–955. https://doi.org/10.1007/s10803-012-1638-1

15.

Bruckner

C. T.

Yoder

P. J.

McWilliam

R. A.

(2006). Generalizability and decision studies: An example using conversational language samples. Journal of Early Intervention, 28(2), 139–153. https://doi.org/10.1177/105381510602800205

16.

Burstein

N. D.

(1986). The effects of classroom organization on mainstreamed preschool children. Exceptional Children, 52(5), 425–434. https://doi.org/10.1177/001440298605200504

17.

Cardinet

Johnson

Pini

(2010). Applying generalizability theory using EduG. Routledge.

18.

Carpenter

Tomasello

(2000). Joint attention, cultural learning, and language acquisition: Implications for children with autism. In Wetherby

A. M.

Prizant

B. M.

(Eds.), Autism spectrum disorders: A transactional developmental perspective (Vol. 9, pp. 31–54). Paul H. Brookes Publishing.

19.

Cone

J. D.

(1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8(3), 411–426. https://doi.org/10.1016/S0005-7894(77)80077-4

20.

Cronbach

L. J.

Rajaratnam

Gleser

G. C.

(1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163. https://doi.org/10.1111/j.2044-8317.1963.tb00206.x

21.

Dickinson

D. K.

Porche

M. V.

(2011). Relation between language experiences in preschool classrooms and children’s kindergarten and fourth-grade language and reading abilities. Child Development, 82(3), 870–886. https://doi.org/10.1111/j.1467-8624.2011.01576.x

22.

Fleury

V. P.

Ford

A. L.

(2021). Shared reading extratextual talk with children with autism who have spontaneous speech. The Journal of Special Education, 55(1), 23–33. https://doi.org/10.1177/0022466920926165

23.

Fleury

V. P.

Hugh

M. L.

(2018). Exploring engagement in shared reading activities between children with autism spectrum disorder and their caregivers. Journal of Autism and Developmental Disorders, 48(10), 3596–3607. https://doi.org/10.1007/s10803-018-3632-8

24.

Fleury

V. P.

Lease

E. M.

(2018). Early indication of reading difficulty? A descriptive analysis of emergent literacy skills in children with autism spectrum disorder. Topics in Early Childhood Special Education, 38(2), 82–93. https://doi.org/10.1177/0271121417751626

25.

Fleury

V. P.

Schwartz

I. S.

(2017). A modified dialogic reading intervention for preschool children with autism spectrum disorder. Topics in Early Childhood Special Education, 37(1), 16–28. https://doi.org/10.1177/0271121416637597

26.

Fredricks

J. A.

Blumenfeld

P. C.

Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

27.

Godin

Freeman

Rigby

(2019). Interventions to promote the playful engagement in social interaction of preschool-aged children with autism spectrum disorder (ASD): A scoping study. Early Child Development and Care, 189(10), 1666–1681. https://doi.org/10.1080/03004430.2017.1404999

28.

Greenwood

C. R.

(1991). Longitudinal analysis of time, engagement, and achievement in at-risk versus non-risk students. Exceptional Children, 57(6), 521–535. https://doi.org/10.1177/001440299105700606

29.

Hollo

Staubitz

J. L.

Chow

J. C.

(2020). Applying generalizability theory to optimize analysis of spontaneous teacher talk in elementary classrooms. Journal of Speech, Language, and Hearing Research, 63(6), 1947–1957. https://doi.org/10.1044/2020_JSLHR-19-00118

30.

Hume

Steinbrenner

J. R.

Odom

S. L.

Morin

K. L.

Nowell

S. W.

Tomaszewski

Szendrey

McIntyre

N. S.

Yücesoy-Özkan

Savage

M. N.

(2021). Evidence-based practices for children, youth, and young adults with autism: Third generation review. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-020-04844-2

31.

Institute of Education Sciences. (2009). Dialogic reading (What Works Clearinghouse Intervention Report).

32.

Kane

(1982). A sampling model for validity. Applied Psychological Measurement, 6(2), 125–160. https://doi.org/10.1177/014662168200600201

33.

Kane

(2002). Inferences about variance components and reliability-generalizability coefficients in the absence of random sampling. Journal of Educational Measurement, 39(2), 165–181. https://www.jstor.org/stable/1435254

34.

Kemp

Kishida

Carter

Sweller

(2013). The effect of activity type on the engagement and interaction of young children with disabilities in inclusive childcare settings. Early Childhood Research Quarterly, 28(1), 134–143. https://doi.org/10.1016/j.ecresq.2012.03.003

35.

Koegel

L. K.

Koegel

R. L.

Frea

Green-Hopkins

(2003). Priming as a method of coordinating educational services for students with autism. Language, Speech, and Hearing Services in Schools, 34(3), 228–235. https://doi.org/10.1044/0161-1461(2003/019)

36.

Krippendorff

(2018). Content analysis: An introduction to its methodology. SAGE.

37.

Landis

J. R.

Koch

G. G.

(1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://www.jstor.org/stable/2529310

38.

Shavelson

R. J.

Yin

Wiley

(2015). Generalizability theory. In. The Encyclopedia of clinical psychology (pp. 1322–1341). John Wiley. https://doi.org/10.1002/9781118625392.wbecp352

39.

Livingston

. (2017). Types of children’s literature. In Evans

Joubert

Meier

(Eds.), Introducing children’s literature: A guide to South African classroom (pp. 112–128). Van Schaik.

40.

Loeb

Dynarski

McFarland

Morris

Reardon

Reber

(2017). Descriptive analysis in education: A guide for researchers. U.S. Department of Education, Institute of Education Sciences & National Center for Education Evaluation and Regional Assistance. https://doi.org/10.1094/PDIS.2003.87.5.550

41.

Lonigan

C. J.

Wagner

R. K.

Torgeson

(2007). Test of preschool early literacy. Super Duper Publications.

42.

Lord

Novick

M. R.

(1968). Statistical theories of mental test scores. Addison-Wesley.

43.

Mantzicopoulos

French

B. F.

Patrick

Watson

J. S.

Ahn

(2018). The stability of kindergarten teachers’ effectiveness: A generalizability study comparing the framework for teaching and the classroom assessment scoring system. Educational Assessment, 23(1), 24–46. https://doi.org/10.1080/10627197.2017.1408407

44.

McIntyre

N. S.

Solari

E. J.

Gonzales

J. E.

Solomon

Lerro

L. E.

Novotny

Oswald

T. M.

Mundy

P. C.

(2017). The scope and nature of reading comprehension impairments in school-aged children with higher-functioning autism spectrum disorder. Journal of Autism and Developmental Disorders, 47(9), 2838–2860. https://doi.org/10.1007/s10803-017-3209-y

45.

McWilliam

R. A.

Ware

W. B.

(1994). The reliability of observations of young children’s engagement: An application of generalizability theory. Journal of Early Intervention, 18(1), 34–47. https://doi.org/10.1177/105381519401800104

46.

Nation

Clarke

Wright

Williams

(2006). Patterns of reading ability in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 36(7), 911–919. https://doi.org/10.1007/s10803-006-0130-1

47.

National Early Literacy Panel. (2008). Developing early literacy: Report of the national early literacy panel.

48.

National Research Council. (2001). Educating children with autism. National Academy Press. http://www.nap.edu/catalog/10017.html

49.

Newborg

(2005). Battelle Developmental Inventory (2nd ed., Examiner’s Manual). Riverside.

50.

Petscher

Cabell

S. Q.

Catts

H. W.

Compton

D. L.

Foorman

B. R.

Hart

S. A.

Lonigan

C. J.

Phillips

B. M.

Schatschneider

Steacy

L. M.

Terry

N. P.

Wagner

R. K.

(2020). How the science of reading informs 21st-Century education. Reading Research Quarterly, 55(Suppl. 1), S267–S282. https://doi.org/10.1002/rrq.352

51.

R Core Team. (2015). RStudio: Integrated development for R. R Studio. http://www.rstudio.com/

52.

Reutzel

D. R.

Fawson

(2002). Your classroom library: New ways to give it more teaching power. Scholastic.

53.

Richter

Courage

M. L.

(2017). Comparing electronic and paper storybooks for preschoolers: Attention, engagement, and recall. Journal of Applied Developmental Psychology, 48, 92–102. https://doi.org/10.1016/j.appdev.2017.01.002

54.

Shavelson

R. J.

Webb

N. M.

(2006). Generalizability theory. In Green

J. L.

Camili

Elmore

P. B.

(Eds.), Handbook of complementary methods in education research (3rd ed., pp. 309–322). American Educational Research Association.

55.

Sigman

Dijamco

Gratier

Rozga

(2004). Early detection of core deficits in autism. Mental Retardation and Developmental Disabilities Research Reviews, 10(4), 221–233. https://doi.org/10.1002/mrdd.20046

56.

Suen

H. K.

Ary

(1989). Reliability: The generalizability approach. In. Analyzing quantitative behavioral observation data (pp. 131–156). Lawrence Erlbaum.

57.

Swiss Society for Research in Education Working Group. (2012). EduG user guide (6.1). https://www.irdp.ch/data/secure/1968/document/EduGUserGuide.pdf

58.

Tapp

Wehby

Ellis

(1995). A multiple option observation system for experimental studies: MOOSES. Behavior Research Methods, Instruments, & Computers, 27(1), 25–31. https://doi.org/10.3758/BF03203616

59.

Tipton

L. A.

Blacher

J. B.

Eisenhower

A. S.

(2017). Young children with ASD: Parent strategies for interaction during adapted book reading activity. Remedial and Special Education, 38(3), 171–180. https://doi.org/10.1177/0741932516677831

60.

Wasik

B. A.

Bond

M. A.

(2001). Beyond the pages of a book: Interactive book reading and language development in preschool classrooms. Journal of Educational Psychology, 93(2), 243–250. https://doi.org/10.1037/0022-0663.93.2.243

61.

Westerveld

M. F.

Paynter

Wicks

(2020). Shared book reading behaviors of parents and their verbal preschoolers on the autism spectrum. Journal of Autism and Developmental Disorders, 50(8), 3005–3017. https://doi.org/10.1007/s10803-020-04406-6

62.

Whitehurst

Lonigan

(1998). Child development and emergent literacy. Child Development, 69(3), 848–872. http://www.jstor.org/stable/1132208

63.

Wicks

Paynter

Westerveld

M. F.

(2020). Looking or talking: Visual attention and active engagement during shared book reading of preschool children on the autism spectrum. Autism, 24(6), 1384–1399. https://doi.org/101177/1362361319900594

64.

Wong

Kasari

(2012). Play and joint attention of children with autism in the preschool special education classroom. Journal of Autism and Developmental Disorders, 42(10), 2152–2161. https://doi.org/10.1007/s10803-012-1467-2

65.

Yoder

Lloyd

B. P.

Symons

(2018). Observational measurement of behavior. Paul H. Brookes Publishing.

66.

Young

T. A.

Moss

Cornwell

(2007). The classroom library: A place for nonfiction, nonfiction in its place. Reading Horizons, 48(1), 1–18.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB