Sage Journals: Discover world-class research

Abstract

The purpose of this study was to compare the classification accuracy of data produced by the previous year’s end-of-year New York state assessment, a computer-adaptive diagnostic assessment (i-Ready), and the gating combination of both assessments to predict the rate of students passing the following year’s end-of-year state assessment in English Language Arts (ELA) and math. Overall, the previous year’s end-of-year state assessment yielded stronger classification accuracy over the use of the fall i-Ready diagnostic assessment. The combination of i-Ready with the previous year’s state test demonstrated nominally greater strength in determining student risk level. Therefore, the previous year’s end-of-year assessment may be economically advantageous over administering i-Ready as an additional screening assessment for the strict purpose of determining risk within a multi-tiered system of supports. Implications of these findings and recommendations for further research are discussed.

Keywords

screening MTSS test validity classification accuracy

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, & National Council on Measure-ment in Education. (2014). Standards for educational and psychological testing. American Educational Research As-sociation. https://www.testingstandards.net/open-access-files.html

Clemens

N. H.

Hagan-Burke

Luo

Cerda

Blakely

Frosch

Gamez-Patience

Jones

(2015). The predictive validity of a computer-adaptive assessment of kindergarten and first-grade reading skills. School Psychology Review, 44(1), 76–97. https://doi.org/10.17105/SPR44-1.76-97

Compton

D. L.

Fuchs

L. S.

Bryant

J. D.

(2006). Selecting at-risk readers in first grade for early intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98(2), 394–309. https://doi.org/10.1037/0022-0663.98.2.394

Curriculum Associates. (2015). i-Ready diagnostic and instruction: User guide. https://www.franklinboe.org/cms/lib/NJ01000817/Centricity/Domain/54/Teacher%20User%20Guide.pdf

Curriculum Associates. (2017). The science behind i-Ready’s adaptive diagnostic. http://www.sarasota.k12.fl.us/agenda2/January%206,%202015%20Board%20Meeting%20on%20Tuesday,%20January%2006,%202015/40AC09CC-59D8-4801-9EA8-4F305FE4C4D5.pdf

Curriculum Associates. (2020). i-Ready diagnostic: Linking study with New York State Testing Program (NYSTP). https://www.poughkeepsieschools.org/cms/lib/NY01813902/Centricity/Domain/234/iready%20impact%20ny.pdf

Curriculum Associates. (2023). I-Ready diagnostic Grades K-12 scale score placement tables (2023–2024 school year). https://www.rcboe.org/cms/lib/GA01903614/Centricity/Domain/20337/iReady%20Scale%20Score%20Placement%20Results.pdf

Davey

(2011). Educational Testing Service (ETS): Practical considerations in computer-based testing. www.ets.org/Media/Research/pdf/CBT-2011.pdf

Farley-Ripple

E. N.

Jennings

A. S.

Buttram

(2019). Toward a framework for classifying teachers’ use of assessment data. AERA Open, 5(4), 1–18. https://doi.org/10.1177/2332858419883571

10.

Hall

G. J.

Doyle

(2025). Learning from learning loss: Bayesian updating in academic universal screening during learning disruptions. Journal of School Psychology, 109, 101426. https://doi.org/10.1016/j.jsp.2024.101426

11.

Jenkins

J. R.

Hudson

R. F.

Johnson

E. S.

(2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36(4), 582–600. https://doi.org/10.1080/02796015.2007.12087919

12.

Jiang

Metz

C. E.

Nishikawa

R. M.

(1996). A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology, 201(3), 745–750. https://doi.org/10.1148/radiology.201.3.8939225

13.

Johnson

E. S.

Jenkins

J. R.

Petscher

Catts

H. W.

(2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24(4), 174–185. https://doi.org/10.1111/j.1540-5826.2009.00291.x

14.

Kane

M. T.

(1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535. https://doi.org/10.1037/0033-2909.112.3.527

15.

Kilgus

S. P.

Methe

S. A.

Maggin

D. M.

Tomasula

J. L.

(2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377–405. https://doi.org/10.1016/j.jsp.2014.06.002

16.

Kilgus

S. P.

Taylor

C. N.

Van Wie

M. P.

von der Embse

N. P.

Sims

W. A.

(2018). Diagnostic accuracy of a universal screening multiple gating procedure: A replication study. School Psychology Quarterly, 33(4), 582–589. https://doi.org/10.1037/spq0000246

17.

Klingbeil

D. A.

Van Norman

E. R.

Nelson

P. M.

Birr

(2018). Evaluating screening procedures across changes to the statewide achievement test. Assessment for Effective Intervention, 44(1), 17–31. https://doi.org/10.1177/1534508417747390

18.

Kuhn

(2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1–26. https://doi.org/10.18637/jss.v028.i05

19.

Linan-Thompson

Ortiz

Cavazos

(2021). An examination of MTSS assessment and decision making practices for English learners. School Psychology Review, 51(4), 484–497. https://doi.org/10.1080/2372966X.2021.2001690

20.

Martin

A. J.

Lazendic

(2018). Computer-adaptive testing: Implications for students’ achievement, motivation, engagement, and subjective test experience. Journal of Educational Psy-chology, 110(1), 27–45. https://doi.org/10.1037/edu0000205

21.

Meehl

P. E.

Rosen

(1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–215. https://doi.org/10.1037/h0048070

22.

Messick

(1995). Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10/1037/0003-066X.50.9.741

23.

National Center on Intensive Intervention. (2024). Academic screening tools chart. American Institutes for Research. https://charts.intensiveintervention.org/ascreening

24.

New York State Education Department. (2018/2019). New York State grades 3-8 English Language Arts and mathematics tests. New York State Education Department. https://www.nysed.gov/state-assessment/grades-3-8-tests

25.

Ochs

Keller-Margulis

M. A.

Santi

K. L.

Jones

J. H.

(2020). Long-term validity and diagnostic accuracy of a reading computer-adaptive test. Assessment for Effecrtive Intervention, 45(3), 210–225. https://doi.org/10.1177/1534508418796271

26.

Petscher

Kim

Foorman

B. R.

(2011). The importance of predictive power in early screening assessments: Implications for placement in the response to intervention framework. Assessment for Effective Intervention, 36(3), 158–166. https://doi.org/10.1177/1534508410396698

27.

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/

28.

Robin

Turck

Hainard

Tiberti

Lisacek

Sanchez

Muller

(2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12(77), 1–8. https://doi.org/10.1186/1471-2105-12-77

29.

Shapiro

E. S.

Gebhardt

S. N.

(2012). Comparing computer-adaptive and curriculum-based measurement methods of assessment. School Psychology Review, 41(3), 295–305. https://doi.org/10.1080/02796015.2012.12087510

30.

Sutter

C. C.

Campbell

L. O.

Lambie

G. W.

(2020). Predicting second-grade students’ yearly standardized reading achievement using a computer-adaptive assessment. Computers in the Schools, 37(1), 40–54. https://doi.org/10.1080/07380569.2020.1720611

31.

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). (2022). Reading assessment.

32.

VanDerHeyden

A. M.

Witt

J. C.

Naquin

(2003). Development and validation of a process for screening referrals to special education. School Psychology Review, 32(2), 204–227. https://doi.org/10.1080/02796015.2003.12086194

33.

Van Norman

E. R.

Nelson

P. M.

Klingbeil

D. A

. (2017). Single measure and gated screening approaches for identifying students at-risk for academic problems: Implications for sensitivity and specificity. School Psychology Quarterly, 32(3), 405–413. https://doi.org/10.1037/spq0000177

34.

Van Norman

E. R.

Nelson

P. M.

Parker

D. C

. (2017). Technical adequacy of growth estimates from a computer adaptive test: Implications for progress monitoring. School Psychology Quarterly, 32(3), 379–391. https://doi.org/10.1037/spq0000175

35.

Walker

H. M.

Small

J. W.

Severson

H. H.

Seeley

J. R.

Feil

E. G.

(2014). Multiple-gating approaches in universal screening within school and community settings. In Kettler

R. J.

Glover

T. A.

Albers

C. A.

Feeney-Kettler

K. A.

(Eds.), Universal screening in educational settings: Evidence-based decision making for schools (pp. 47–75). American Psychological Association. https://doi.org/10.1037/14316-00

36.

Weiss

D. J.

Sahin

(2024). Computerized adaptive testing: From concept to implementation. The Guilford Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

Classification Accuracy of i-Ready and Prior Year State Exams on Year-End Outcomes

Abstract

Keywords

Get full access to this article

References

Supplementary Material