Restricted accessResearch articleFirst published online 2023-3
An Evaluation of the Validity of Growth on Two Computer Adaptive Tests to Predict Performance on End-of-Year Achievement Tests using Quantile Regression
This study explored the validity of growth on two computer adaptive tests, Star Reading and Star Math, in explaining performance on an end-of-year achievement test for a sample of students in Grades 3 through 6. Results from quantile regression analyses indicate that growth on Star Reading explained a statistically significant amount of variance in performance on end-of-year tests after controlling for baseline performance in all grades. In Grades 3 through 5, the relationship between growth on Star Reading and the end-of-year test was stronger among students who scored higher on the end-of-year test. In math, Star Math explained a statistically significant amount of variance in end-of-year scores after statistically controlling for baseline performance in all grades. The strength of the relationship did not differ among students who scored lower or higher on the end-of-year test across grades.
ArdoinS. P.ChristT. J.MorenaL. S.CormierD. C.KlingbeilD. A. (2013). A systematic review and summarization of the recommendations and research surrounding curriculum-based measurement of oral reading fluency (cbm-r) decision rules. Journal of School Psychology, 51, 1–18. https://doi.org/10.1016/j.jsp.2012.09.004
2.
BatesD.MächlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48. https://doi.10.18637/jss.v067.01
3.
BergstromB. A. (1992, April). Ability measure equivalence of computer adaptive and pencil and paper tests: A research synthesis [Paper presentation]. Annual Meeting of the American Education Research Association, San Francisco, CA, United States.
CarlsonR. D. (1994). Computer adaptive testing: A shift in the evaluation paradigm. Journal of Educational Technology Systems, 22, 213–224. https://doi.org/10.2190/QP36-WV9L-ATT3-FYWK
6.
CattsH. W.PetscherY.SchatschneiderC.Sittner BridgesM.MendozaK. (2009). Floor effects associated with universal screening and their impact on the early identification of reading disabilities. Journal of Learning Disabilities, 42, 163–176. https://doi.org/10.1177/0022219408326219
7.
ChoE.CapinP.RobertsG.VaughnS. (2018). Examining predictive validity of oral reading fluency slope in upper elementary grades using quantile regression. Journal of Learning Disabilities, 51, 565–577. https://doi.org/10.1177/0022219417719887
8.
ChoE.ComptonD. L.JosolC. K. (2020). Dynamic assessment as a screening tool for early identification of reading disabilities: A latent change score approach. Reading and Writing, 33, 719–739. https://doi.org/10.1007/s11145-019-09984-1
9.
ChristT. J.ZopluogluC.MonaghenB.Van NormanE. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19–57. https://doi.org/10.1016/j.jsp.2012.11.001
10.
ClemensN. H.LeeK.HenriM.SimmonsL. E.KwokO.Al OtaibaS. (2020). Growth on sublexical fluency progress monitoring measures in early kindergarten and relations to word reading acquisition. Journal of School Psychology, 79, 43–62. https://doi.org/10.1016/j.jsp.2020.01.003
11.
CronbachL. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–127. https://doi.org/10.1037/h0076829
DenoS. L. (1990). Individual differences and individual difference: The essential difference of special education. Journal of Special Education, 24, 160–173. https://doi.org/10.1177/002246699002400205
DenoS. L.MirkinP. K. (1977). Data-based program modification: A manual. Council for Exceptional Children.
16.
FoegenA.JibanC.DenoS. (2007). Progress monitoring measures in mathematics A review of the literature. The Journal of Special Education, 41(2), 121–139. https://doi.org/10.1177/00224669070410020101
17.
FuchsD.FuchsL. S. (2006). Introduction to response to intervention: What, why, and how valid is it?Reading Research Quarterly, 41(1), 93–99. https://doi.org/10.1598/RRQ.41.1.4
FuchsL. S.FuchsD.ZumetaR. O. (2008). A curricular-sampling approach to progress monitoring: Mathematics concepts and applications. Assessment for Effective Intervention, 33, 225–233. https://doi.org/10.1177/1534508407313484
20.
GerstenR.BeckmannS.ClarkeB.FoegenA.MarshL.StarJ. R.WitzelB. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009-4060). What Works Clearinghouse. http://eric.ed.gov/?id=ED504995
21.
Individuals with Disabilities Education Improvement Act of 2004. 20 U. S. C. §1400 et seq. (2004).
22.
JenkinsJ.GraffJ. J.MigliorettiD. L. (2009). Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151–163. https://doi.org/10.1177/001440290907500202
23.
KaneM. (2006). Validation. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 17–64). American Council on Education; Praeger.
24.
KlingbeilD. A.McComasJ. J.BurnsM. K.HelmanL. (2015). Comparison of predictive validity and diagnostic accuracy of screening measures of reading skills. Psychology in the Schools, 52, 500–514. https://doi.org/10.1002/pits.21839
LongJ. D. (2011). Longitudinal data analysis for the behavioral sciences using R. Routledge.
28.
NelsonP. M.Van NormanE. R.KlingbeilD. A.ParkerD. C. (2017). Progress monitoring with computer adaptive assessments: The impact of data collection schedule on growth estimates. Psychology in the Schools, 54, 463–471. https://doi.org/10.1002/pits.22015
29.
PetscherY.LoganJ. A. R. (2014). Quantile regression in the study of developmental sciences. Child Development, 85, 861–881. https://doi.org/10.1111/cdev.12190
30.
PetscherY.LoganJ. A. R.ZhouC. (2013). Extending conditional means modeling: An introduction to quantile regression. In PetscherY.SchatschneiderC.ComptonD. (Eds.), Applied quantitative analysis in education and social sciences (pp. 3–33). Routledge.
31.
R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
32.
Renaissance Learning. (2019). Star reading: Technical manual.
33.
Renaissance Learning. (2020). Star math: Technical manual.
34.
ShapiroE. S.MinyiS. D.QiongF. (2015). Comparing computer adaptive and curriculum-based measures of math in progress monitoring. School Psychology Quarterly, 30, 470–487. https://doi.org/10.1037/spq0000116
TigheE. L.SchatschneiderC. (2016). A quantile regression approach to understanding the relations among morphological awareness, vocabulary, and reading comprehension in adult basic education students. Journal of Learning Disabilities, 49, 424-436. https://doi.org/10.1177/0022219414556771
40.
Van NormanE. R.NelsonP. M.ParkerD. C. (2017). Technical adequacy of growth estimates from a computer adaptive test: Implications progress monitoring. School Psychology Quarterly, 32, 379–391. https://doi.org/10.1037/spq0000175
41.
Van NormanE. R.YsseldykeJ. E. (2020). The impact of data collection frequency and trend estimation method on the consistency of growth estimates from two computer-adaptive tests. School Psychology Review, 49, 20–30. https://doi.org/10.1080/2372966X.2020.1716634
42.
WeissD. J.KingsburyG. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361–375. https://doi.org/10.1111/j.1745-3984.1984.tb01040
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
