Modeling Growth in Reading With Curriculum-Based Measurement Versus Computer-Adaptive Tests

Abstract

Curriculum-based measurement of reading (CBM-R) is a common assessment educators use to monitor student growth in broad reading skills and evaluate the effectiveness of instructional programs. Computer-adaptive tests (CATs), such as Star Reading, have been cited as a viable option to formatively assess reading growth. We used Bayesian multivariate-multilevel modeling to compare growth measured via concurrently collected CBM-R and CAT data for 3,192 students across Grades 1 (n = 298), 2 (n = 1,149), 3 (n = 1,062), 4 (n = 462), and 5 (n = 221). After standardizing outcomes, the average rate of weekly improvement on each measure was highly similar across grade levels. Between-student variability for growth on each assessment was highly similar across grade levels. The magnitude of residual variance, or error, differed markedly between assessments. In Grades 1 to 3, Star Reading yielded less precise estimates of growth relative to CBM-R, and the opposite results were observed in Grade 5.

Keywords

progress monitoring response to intervention computer-adaptive tests curriculum-based measurement

Get full access to this article

View all access options for this article.

References

Al Otaiba

. (2005). Response to intervention in early literacy instruction. Practical issues for early childhood personnel preparation. Journal of Early Childhood Teacher Education, 25, 201–209. https://doi.org/10.1080/1090102050250303

Baker

S. K.

Smolkowski

Katz

Fien

Seeley

J. R.

Kame’enui

E. J.

Beck

C. T.

(2008). Reading fluency as a predictor of reading proficiency in low-performing, high-poverty schools. School Psychology Review, 37, 18–37. https://doi.org/10.1080/02796015.2008.12087905

Boorse

Van Norman

E. R.

(2021). Modeling within-year growth on the mathematics measure of academic progress. Psychology in the Schools, 58, 2255–2268. https://doi.org/10.1002/pits.22590

Bulut

Cormier

D. C.

(2018). Validity evidence for progress monitoring with Star Reading: Slope estimates, administration frequency, and number of data points. Frontiers in Education, 3(68), 1–12. https://doi.org/10.3389/feduc.2018.00068

Bürkner

(2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01

Carlson

R. D.

(1994). Computer adaptive testing: A shift in the evaluation paradigm. Journal of Educational Technology Systems, 22(3), 213–224. https://doi.org/10.2190/QP36-WV9L-ATT3-FYWK

Children’s Educational Services. (1987). Test of Oral Reading Fluency.

Christ

T. J.

(2006). Short-term estimates of growth using curriculum-based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35(1), 128–133. https://doi.org/10.1080/02796015.2006.12088006

Christ

T. J.

Ardoin

S. P.

(2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development. Journal of School Psychology, 47(1), 55–75. https://doi.org/10.1016/j.jsp.2008.09.004

10.

Christ

T. J.

White

M. J.

Ardoin

S. P.

Eckert

T. L.

(2013a). Curriculum based measurement of reading: Consistency and validity across best, fastest, and question reading conditions. School Psychology Review, 42(4), 415–436. https://doi.org/10.1080/02796015.2013.12087463

11.

Christ

T. J.

Zopluoglu

Monaghen

B. D.

Van Norman

E. R.

(2013b). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51(1), 19–57. https://doi.org/10.1016/j.jsp.2012.11.001

12.

Cohen

(1983). Applied multiple regression/correlational analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

13.

Cummings

K. D.

Park

Bauer Schaper

H. A.

(2013). Form effects on DIBELS Next oral reading fluency progress-monitoring passages. Assessment for Effective Intervention, 38(2), 91–104. https://doi.org/10.1177/1534508412447010

14.

Deno

S. L.

(1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52(3), 219–232. https://doi.org/10.1177/001440298505200303

15.

Deno

S. L.

(2003). Curriculum-based measures: Development and perspectives. Assessment for Effective Intervention, 28(3–4), 3–12. https://doi.org/10.1177/073724770302800302

16.

Denton

C. A.

Barth

A. E.

Fletcher

J. M.

Wexler

Vaughn

Cirino

P. T.

Romain

Francis

D. J.

(2011). The relations among oral and silent reading fluency and comprehension in middle school: Implications for identification and instruction of students with reading difficulties. Scientific Studies of Reading, 15(2), 109–135. https://doi.org/10.1080/10888431003623546

17.

Francis

D. J.

Santi

K. L.

Barr

Fletcher

J. M.

Varisco

Foorman

B. R.

(2008). Form effects on the estimation of students’ oral reading fluency using DIBELS. Journal of School Psychology, 46(3), 315–342. https://doi.org/10.1016/j.jsp.2007.06.003

18.

Fuchs

L. S.

(2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33(2), 188–192. https://doi.org/10.1080/02796015.2004.12086241

19.

Fuchs

L. S.

Deno

S. L.

(1991). Effects of curriculum within curriculum-based measurement. Exceptional Children, 58(3), 232–243. https://doi.org/10.1177/001440299105800306

20.

Fuchs

L. S.

Fuchs

Hamlett

C. L.

Walz

Germann

(1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22(1), 27–48. https://doi.org/10.1080/02796015.1993.12085636

21.

Fuchs

L. S.

Fuchs

Hosp

M. K.

Jenkins

J. R.

(2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. In The role of fluency in reading competence, assessment, and instruction (pp. 239–256). Routledge. https://doi.org/10.4324/9781410608246-3

22.

Good

R. H.

Kaminski

R. A.

Shinn

Bratten

Shinn

Laimon

Smith

Flindt

(2004). Technical Adequacy of DIBELS: Results of the Early Childhood Research Institute on measuring growth and development (Technical Report, No. 7). University of Oregon.

23.

Hambleton

R. K.

Jones

R. W.

(1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38–47. https://doi.org/10.1111/j.1745-3992.1993.tb00543.x

24.

Harlacher

J. E.

Sakelaris

T. L.

Kattleman

N. M.

(2014). Multi-tiered system of support. In Practitioner’s guide to curriculum-based evaluation in reading (pp. 23–45). Springer. https://doi.org/10.1007/978-1-4614-9360-0

25.

Hoffman

M. D.

Gelman

(2014). The no-u-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1593–1623.

26.

Individuals with Disabilities Education Improvement Act of 2004, 20 U.S.C. §1400 et seq. (2004).

27.

Kovaleski

J. F.

VanDerHeyden

A. M.

Runge

T. J.

Zirkel

P. A.

Shapiro

E. S.

(2013). The RTI approach to evaluating learning disabilities. Guilford Publications.

28.

Lai

S. A.

George Benjamin

Schwanenflugel

P. J.

Kuhn

M. R.

(2014). The longitudinal relationship between reading fluency and reading comprehension skills in second-grade children. Reading & Writing Quarterly, 30, 116–138.

29.

Lenth

R. V.

(2011). Some practical guidelines for effect sample size determination. The American Statistician, 55(3), 187–193. https://doi.org/10.1198/000313001317098149

30.

Long

J. D.

(2011). Longitudinal data analysis for the behavioral sciences using R. Routledge.

31.

Mahdavi

J. N.

Tensfeldt

(2013). Untangling reading comprehension strategy instruction: Assisting struggling readers in the primary grades. Preventing School Failure: Alternative Education for Children and Youth, 57, 77–92. https://doi.org/10.1080/1045988X.2012.668576

32.

Meijer

R. R.

Nering

M. L.

(1999). Computerized adaptive testing: Overview and introduction. Applied Psychological Measurement, 23(3), 187–194. https://doi.org/10.1177/01466219922031310

33.

Nese

J. F.

Biancarosa

Cummings

Kennedy

Alonzo

Tindal

(2013). In search of average growth: Describing within-year oral reading fluency growth across grades 1–8. Journal of School Psychology, 51(5), 625–642. https://doi.org/10.1016/j.jsp.2013.05.006

34.

Pinheiro

Bates

& R Core Team. (2022). nlme: Linear and nonlinear mixed effects models (R package version 3.1-160). https://CRAN.R-project.org/package=nlme

35.

Plewis

(2005). Modelling behaviour with multivariate multilevel growth curves. Methodology, 1(2), 71–80. https://doi.org/10.1027/1614-1881.1.2.71

36.

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

37.

Renaissance Learning. (2016). The next generation of response to intervention [White paper]. https://doc.renlearn.com/KMNet/R60382.pdf

38.

Renaissance Learning. (2021). Star CBM Reading Technical Manual.

39.

Renaissance Learning. (2022). Star Assessments™ for Reading Technical Manual. https://help.renaissance.com/US/PDF/SR/SRRPTechnicalManual.pdf

40.

Segall

D. O.

Moreno

K. E.

Hetter

R. D.

(1997). Item pool development and evaluation. In Sands

W. A.

Waters

B. K.

McBride

J. R.

(Eds.), Computer adaptive testing: From inquiry to operation (pp. 117–130). American Psychological Association. https://doi.org/10.1037/10244-012

41.

Shapiro

E. S.

Gibbs

D. P.

(2014). Comparison of progress monitoring with computer adaptive tests and curriculum-based measures. Center for Promoting Research to Practice, Lehigh University. http://coe.lehigh.edu/cprp/research/current

42.

Stan Development Team. (2022). RStan: The R interface to Stan (R package version 2.21.7). https://mc-stan.org/

43.

Thornblad

S. C.

Christ

T. J.

(2014). Curriculum-based measurement of reading: Is 6 weeks of daily progress monitoring enough? School Psychology Review, 43(1), 19–29. https://doi.org/10.1080/02796015.2014.12087451

44.

Tindal

(2013). Curriculum-based measurement: A brief history of nearly everything from the 1970s to present. ISRN Education. https://doi.org/10.1155/2013/958530

45.

Van Norman

E. R.

Forcht

E. R

. (2022). An evaluation of the validity of growth on two computer adaptive tests to predict performance on end-of-year achievement tests using quantile regression. Assessment for Effective Intervention, 0(0), 1–10. https://doi.org/10.1177/15345084221100421

46.

Van Norman

E. R.

Ysseldyke

J. E

. (2020). The impact of data collection frequency and trend estimation method on the consistency of growth estimates from two computer-adaptive tests. School Psychology Review, 49(1), 20–30. https://doi.org/10.1080/2372966x.2020.1716634

47.

Wang

Shin

C. D.

(2010). Comparability of computerized adaptive and paper-pencil tests. Test, Measurement and Research Service Bulletin, 13, 1–7.

48.

Weiss

D. J.

(2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37(2), 70–84. https://doi.org/10.1080/07481756.2004.11909751

49.

Weiss

D. J.

Kingsbury

G. G.

(1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361–375. https://doi.org/10.1111/j.1745-3984.1984.tb01040.x