Given oral language’s role in writing proficiency and to address measurement issues in oral and written language, we trialed complementary scoring metrics in language sample analysis (LSA) with the sentence-level Picture Word Writing Curriculum-Based Measure (CBM-W). Using the Picture Word CBM-W samples of 123 students with writing difficulties, we investigated (1a) alternate form reliability, (1b) criterion-related validity with existing Picture Word CBM-W metrics, (2) criterion-related validity with a standardized written expression measure, and (3) sensitivity to growth from fall to spring for LSA and Picture Word CBM-W scoring mechanisms. Pearson product-moment correlations, Spearman’s correlations, and Bonferroni-corrected paired-samples t-tests revealed two promising LSA metrics with evidence of technical quality and sensitivity to growth as a complementary scoring mechanism for Picture Word CBM-W: mean length of T-Unit in morphemes (MLTU-M) using the mean of two forms in the fall, and number of different words (NDW) using the mean of two forms in fall and spring. Results support the role of oral language in sentence-level writing proficiency, as well as MLTU-M and NDW as complementary scoring mechanisms to provide more specific estimates of oral language skills in grammar/morphosyntax and semantics/lexical diversity not possible with current Picture Word CBM-W scoring mechanisms.
AdlofS.HoganT. (2019). If we don’t look, we won’t see: Measuring language development to inform literacy instruction. Policy Insights from the Behavioral and Brain Sciences, 6(2), 210–217. https://doi.org/10.1177/2372732219839075
2.
AllenA. A.JungP.-G.PochA. L.BrandesD.ShinJ.LembkeE. S.McMasterK. M. (2020). Technical adequacy of curriculum-based measures in writing in grades 1-3. Reading & Writing Quarterly: Overcoming Learning Difficulties, 36(6), 563–587. https://doi.org/10.1080/10573569.2019.1689211
3.
AllenA. A.PochA. L.LembkeE. S. (2018). An exploration of alternative scoring methods using curriculum-based measurement in early writing. Learning Disability Quarterly, 41(2), 85–99. https://www.jstor.org/stable/26742893
4.
BaoX.KomesidouR.HoganT. P. (2024). A review of screeners to identify risk of developmental language disorder. American Journal of Speech-Language Pathology, 33(3), 1548–1571. https://doi.org/10.1044/2023_AJSLP-23-00286
5.
BeersS.NagyW. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?Reading & Writing, 22(2), 185–200. https://doi.org/10.1007/s11145-007-910
6.
BerningerV. W.AmtmannD. (2003). Preventing written expression disabilities through early and continuing assessment and intervention for handwriting and/or spelling problems Research into practice. In SwansonH. L.HarrisK. R.GrahamS. (Eds.), Handbook of learning disabilities (pp. 345–363). Guilford Press.
7.
DenmanD.SpeyerR.MunroN.PearceW. M.ChenY.-W.CordierR. (2017). Psychometric properties of language assessments for children aged 4–12 years: A systematic review. Frontiers in Psychology, 8, 1–28. https://doi.org/10.3389/fpsyg.2017.01515
8.
DenoS. L.MarstonD.MirkinP. (1982). Valid measurement procedures for continuous evaluation of written expression. Exceptional Children, 48(4), 368–371. https://doi.org/10.1177/001440298204800417
9.
DenoS. L.MirkinP.MarstonD. (1980). Relationships among simple measures of written expression and performance on standardized achievement tests (Vol. IRLD-RR-22). Institute for Research on Learning Disabilities, University of Minnesota.
10.
DockrellJ.ConnellyV. (2009). The impact of oral language skills on the production of written text. Teaching and Learning Writing, 11(6), 45–62. https://doi.org/10.1348/000709909X421919
11.
DockrellJ.ConnellyV.WalterK.CrittenS. (2015). Assessing children’s writing products: The role of curriculum-based measures. British Educational Research Journal, 41(4), 575–595. https://doi.org/10.1002/berj.3162
12.
DockrellJ.LindsayG.ConnellyV. (2009). The impact of specific language impairment on adolescents’ written text. Exceptional Children, 75(4), 427–446. https://doi.org/10.1177/001440290907500403
13.
DockrellJ.LindsayG.ConnellyV.MackieC. (2007). Constraints in the production of written text in children with specific language impairments. Exceptional Children, 73(2), 147–164. https://doi.org/10.1177/001440290707300202
14.
DockrellJ.RickettsJ.CharmanT.LindsayG. (2014). Exploring writing products in students with language impairments and autism spectrum disorders. Learning and Instruction, 32, 81–90. https://dx.doi.org/10.1016/j.learninstruc.2014.01.008
15.
EbertK. D. (2020). Language sample analysis with bilingual children: Translating research to practice. Topics in Language Disorders, 40(2), 182–201. https://doi.org/10.1097/TLD.0000000000000209
16.
FumeroK.WoodC. (2021). Verb errors in 5th-grade English learners’ written responses: Relation to writing quality. Languages, 6(2), 71–86. https://doi.org/10.3390/languages6020071
17.
GansleK. A.NoellG. H.VanDerHeydenA. M.NaquinG. M.SliderN. J. (2002). Moving beyond total words written: The reliability, criterion validity, and time cost of alternate measures for curriculum-based measurement in writing. School Psychology Review, 31(4), 477–497. https://doi.org/10.1080/02796015.2002.12086169
18.
GrahamS.CollinsA.Rigby-WillsH. (2016). Writing characteristics of students with learning disabilities and typically achieving peers. Exceptional Children, 83(2), 199–218. https://doi.org/10.1177/0014402916664070
19.
GrahamS.HebertM.FishmanE.RayA.RouseA. (2020). Do children classified with specific language impairment have a learning disability in writing? A meta-analysis. Journal of Learning Disabilities, 53(4), 292–310. https://doi.org/10.1177/0022219420917338
20.
Hall-MillsS.ApelK. (2015). Linguistic feature development across grades and genre in elementary writing. Language, Speech & Hearing Services in Schools, 46(3), 242–255. https://doi.org/10.1044/2015_LSHSS-14-0043
21.
HammillD. D.LarsenS. C. (2009). Test of Written Language (TOWL-4) (4th ed.). Pro-Ed.
22.
HughesD.McGillivaryL.SchmidekM. (1997). Guide to narrative language: Procedures for assessment. Pro-Ed.
23.
HuntK. W. (1970). Syntactic maturity in schoolchildren and adults. Monographs of the Society for Research in Child Development, 35(1), 1–67. https://doi.org/10.2307/1165818
JusticeL.BowlesR.KaderavekJ.UkrainetzT.EisenbergS.GillamR. (2006). The index of narrative microstructure: A clinical tool for analyzing school-age children’s narrative performances. American Journal of Speech-language Pathology, 15(2), 177–191. https://doi.org/10.1044/1058-0360(2006/017)
KaufmanA. S.KaufmanN. L. (2014). Kaufman Test of Educational Achievement, Third Edition (KTEA-3) [Technical manual]. Pearson.
28.
KimY.Al OtaibaS.FolsomJ.GreulichL.PuranikC. (2014). Evaluating the dimensionality of first-grade written composition. Journal of Speech, Language, and Hearing Research, 57(1), 199. https://doi.org/10.1044/1092-4388(2013/12-0152)
29.
KimY.Al OtaibaS.WanzekJ. (2015). Kindergarten predictors of third grade writing. Learning and Individual Differences, 37, 27–37. https://dx.doi.org/10.1016/j.lindif.2014.11.009
30.
KimY.GrahamS. (2022). Expanding the direct and indirect effects model of writing (DIEW): Reading–writing relations, and dynamic relations as a function of measurement/dimensions of written composition. Journal of Educational Psychology, 114(2), 215–238. https://doi.org/10.1037/edu0000564
31.
KimY.ParkS. (2019). Unpacking pathways using the direct and indirect effects model of writing (DIEW) and the contributions of higher order cognitive skills to writing. Reading & Writing, 32(5), 1319. https://doi.org/10.1007/s11145-018-9913-y
32.
KimY.PuranikC.OtaibaS. (2015). Developmental trajectories of writing skills in first grade. The Elementary School Journal, 115(4), 593–613. https://doi.org/10.1086/681971
33.
KimY.SchatschneiderC. (2017). Expanding the developmental models of writing: A direct and indirect effects model of developmental writing (DIEW). Journal of Educational Psychology, 109(1), 35–50. https://dx.doi.org/10.1037/edu0000129
34.
KoutsoftasA.GrayS. (2012). Comparison of narrative and expository writing in students with and without language- learning disabilities. Language, Speech & Hearing Services in Schools, 43(4), 395–409. https://doi.org/10.1044/0161-1461(2012/11-0018)
35.
LeeY.De La PazS. (2021). Writing scientific explanations: Effects of a cognitive apprenticeship for students with LD and English learners. Exceptional Children, 87(4), 458–475. https://doi.org/10.1177/0014402921999310
McMasterK. L.LembkeE. S.ShanahanE.ChoiS.AnJ.SchatschneiderC.Deusenberg-MarshallM.BirinciS.McCollomE.GarmanC.MooreK. (2024). Supporting teachers’ data-based individualization of early writing instruction: An efficacy trial. Journal of Learning Disabilities, 58(4), 287–303. https://doi.org/10.1177/00222194241300324
40.
McMasterK. L.LembkeE. S.ShinJ.PochA.SmithR. A.JungP.AllenA.WagnerK. (2020). Supporting teachers’ use of data-based instruction to improve students’ early writing skills. Journal of Educational Psychology, 112, 1–21. https://doi.org/10.1037/edu0000358
41.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). New York American Councilon Education Macmillan.
42.
MillerJ.IglesiasA. (2023). Systematic Analysis of Language Transcripts (SALT) (Version 21) [Computer Software]. SALT Software, LLC.
National Center for Education Statistics [NCES]. (2022). Students with disabilities. U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/programs/coe/indicator/cgg
45.
PavelkoS.OwensR.IrelandM.Hahs-VaughnD. (2016). Use of language sample analysis by school-based SLPs: Results of a nationwide survey. Language, Speech & Hearing Services in Schools, 47(3), 246–258. https://doi.org/10.1044/2016_LSHSS-15-0044
46.
PetersenD. B.StoddardA. (2018). Psychometric requirements of oral and written language progress monitoring assessments. Perspectives of the ASHA Special Interest Groups, 3(Part 4), 180–197. https://doi.org/10.1044/persp3.SIG1.180
47.
PuranikC.DuncanM.LiH.YingG. (2020). Exploring the dimensionality of kindergarten written composition. Reading & Writing, 33(10), 2481–2510. https://doi.org/10.1007/s11145-020-10053-1
48.
RatnerN. B.HanY.YangJ. S. (2024). Should we stop using lexical diversity measures in children’s language sample analysis?American Journal of Speech-Language Pathology, 33(4), 1986–2001. https://doi.org/10.1044/2024_AJSLP-23-00457
49.
RenoE. A.McMasterK. L. (2024). Measuring linguistic growth in sentence-level writing curriculum-based measures: Exploring complementary scoring methods. Language, Speech, and Hearing Services in Schools, 55(2), 529–544. https://doi.org/10.1044/2023_LSHSS-23-00056
50.
SarmientoC. M.TruckenmillerA. J.ChoE.HwangH. (2024). Academic language use in middle school informational writing. British Journal of Educational Psychology, 95(2), 384–404. https://doi.org/10.1111/bjep.12724
51.
SchrankF. A.MatherN.McGrewK. S. (2014a). Woodcock-Johnson IV Tests of Achievement. Riverside.
52.
ScottC. (2020). Language sample analysis of writing in children and adolescents: Assessment and intervention contributions. Topics in Language Disorders, 40(2), 202–220. https://doi.org/10.1097/TLD.0000000000000213
53.
SillimanE. R.BerningerV. W. (2011). Cross-disciplinary dialogue about the nature of oral and written language problems in the context of developmental, academic, and phenotypic profiles. Topics in Language Disorders, 31(1), 6–23. https://doi.org/10.1097/TLD.0b013e31820a0b5b
54.
TroiaG.ShenM.BrandonD. (2019). Multidimensional levels of language writing measures in grades four to six. Written Communication, 36(2), 231–266. https://doi.org/10.1177/0741088318819473
WoodC.SchatschneiderC. (2021). Examining writing measures and achievement for students of varied language abilities and linguistic backgrounds using structural equation modeling. Reading Writing Quarterly, 37(1), 65–81. https://doi.org/10.1080/10573569.2020.1716284
57.
WoodC. L.SchatschneiderC.HartS. (2020). Average one year change in lexical measures of written narratives for school age students. Reading & Writing Quarterly, 36(3), 260–277. https://doi.org/10.1080/10573569.2019.1635544
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
