Early identification of students needing additional support is a foundational component of Multi-Tiered Systems of Support (MTSS). Due to the resource-intensive nature of implementing MTSS, it is critical that universal screening procedures are maximally accurate and efficient. The purpose of this study was to compare the classification accuracy of aimswebPlus reading scores to the Benchmark Assessment System scores. We used data from a mid-size city in Texas to retrospectively compare the classification accuracy between fall aimswebPlus reading composites to the Benchmark Assessment System scores when predicting student performance on the statewide reading test. When classification decisions were made based on the vendor-recommended cut-scores, both measures were insufficiently sensitive for screening in MTSS. Following aimswebPlus’ recommended method for establishing local-cut scores improved the sensitivity of decisions, but the specificity values were well below minimally acceptable levels. Limitations, directions for future research, and implications for practice are discussed.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing.https://www.testingstandards.net/open-access-files.html
2.
BaluR.ZhuP.DoolittleF.SchillerE.JenkinsJ.GerstenR. (2015). Evaluation of response to intervention practices for elementary school reading(NCEE 2016-4000). https://files.eric.ed.gov/fulltext/ED560820.pdf
3.
BaroneJ.KhairallahP.GabrielR. (2020). Running records revisited: A tool for efficiency and focus. The Reading Teacher, 73(4), 525–530. https://doi.org/10.1002/trtr.1861
4.
BensonN. F.FloydR. G.KranzlerJ. H.EckertT. L.FeferS. A.MorganG. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 national survey. Journal of School Psychology, 72, 29–48. https://doi.org/10.1016/j.jsp.2018.12.004
5.
D’AgostinoJ. V.RodgersE.WinklerC.JohnsonT.BerenbonR. (2021). The generalizability of running record accuracy and self-correction scores. Reading Psychology, 42(2), 111–130. https://doi.org/10.1080/02702711.2021.1880177
FienH.ChardD. J.BakerS. K. (2021). Can the evidence revolution and multi-tiered systems of support improve education equity and reading achievement?Reading Research Quarterly, 56(S1), S105–S118. https://doi.org/10.1002/rrq.391
FuchsD.FuchsL. S. (2017). Critique of the national evaluation of response to intervention frameworks. Exceptional Children, 83(3), 255–268. https://doi.org/10.1177/0014402917693580
11.
FuchsL. S.VaughnS. (2012). Responsiveness-to-intervention: A decade later. Journal of Learning Disabilities, 45(3), 195–203. https://doi.org/10.1177/0022219412442150
12.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45(1), 117–135. https://doi.org/10.1016/j.jsp.2006.05.005
GraneyS. B.MartinezR. S.MissallK. N.AricakO. T. (2010). Universal screening of reading in late elementary school: R-cbm vs. cbm maze. Remedial and Special Education, 31(5), 368–377. https://doi.org/10.1177/0741932509338371
15.
HonakerJ.KingG.BlackwellM. (2011). Amelia II: A program for missing data. Journal of Statistical Software, 45(7), 1–47. https://www.jstatsoft.org/v45/i07/
16.
JenkinsJ. R.HudsonR. F.JohnsonE. S. (2007). Screening for at-risk readers in a response to intervention framework. School Psychology Review, 36(4), 582–600. https://doi.org/10.1080/02796015.2007.12087919
KentS. C.WanzekJ.YunJ. (2019). Screening in upper elementary grades: Identifying fourth-grade students at-risk for failing the state reading assessment. Assessment for Effective Intervention, 44(3), 160–172. https://doi.org/10.1177/1534508418758371
19.
KilgusS. P.MetheS. A.MagginD. M.TomasulaJ. L. (2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377– 405. https://doi.org/10.1016/j.jsp.2014.06.002
20.
KlingbeilD. A.MauriceS. A.Van NormanE. R.NelsonP. M.BirrC.HanrahanA. R.SchrammA. L.CopekR. A.CarseS. A.KoppelR. A.LopezA. L. (2019). Improving mathematics screening in middle school. School Psychology Review48(4): 383–398. https://doi.org/10.17105/SPR-2018-0084.V48-4
21.
KlingbeilD. A.McComasJ. J.BurnsM. K.HelmanL. (2015). Comparison of predictive validity and diagnostic accuracy of screening measures of reading skills. Psychology in the Schools, 52(5), 500–514. https://doi.org/10.1002/pits.21839
22.
KlingbeilD. A.NelsonP. M.Van NormanE. R.BirrC. (2017). Comparison of universal screening methods for students in upper elementary grades. Remedial and Special Education, 38(5), 308–320. https://doi.org/10.1177/0741932517697446
23.
LeeflangM. G.RutjesA. W. S.ReitsmaJ. B.HooftL.BossuytP. M. M. (2013). Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ, 185(11), E537–E544. https://doi.org/10.1503/cmaj.121286
NCS Pearson, Inc. (2018b). aimswebPlus Tiers guide. Author.
28.
NilssonN. L. (2008). A critical analysis of eight informal reading inventories. The Reading Teacher61(7): 526–536. https://doi.org/10.1598/RT.61.7.2
29.
ParkerD. C.ZaslofskyA. F.BurnsM. K.KaniveR.HodgsonJ.ScholinS. E.KlingbeilD. A. (2015). A brief report of the diagnostic accuracy of oral reading fluency and reading inventory levels for reading failure risk among second-and third-grade students. Reading & Writing Quarterly, 31(1), 56–67. https://doi.org/10.1080/10573569.2013.857970
30.
R Core Team. (2020). R: A language and environment for statistical computing.R Foundation for Statistical Computing. https://www.R-project.org/
RubinD. B. (1987). Multiple imputation for nonresponse in surveys. John Wiley & Sons, Inc.
33.
ShinJ.McMasterK. (2019). Relations between CBM (oral reading and maze) and reading comprehension on state achievement tests: A meta-analysis. Journal of School Psychology, 73, 131–149. https://doi.org/10.1016/j.jsp.2019.03.005
34.
SilvaM. R.Collier-MeekM. A.CoddingR. S.KleinertW. L.FeinbergA. (2020). Data collection and analysis in response-to-intervention: A survey of school psychologists. Contemporary School Psychology, 1–18. https://doi.org/10.1007/s40688-020-00280-2
TierneyN.CookD.McBainM.FayC. (2021). naniar: Data structures, summaries, and visualizations for missing data(version 0.6.1). https://CRAN.R-project.org/package=naniar
40.
University of Oregon. (2020). 8th edition of the dynamic indicators of basic early literacy skills (DIBELS): Technical Manual. https://dibels.uoregon.edu
41.
VanDerHeydenA. M. (2013). Universal screening may not be for everyone: Using a threshold model as a smarter way to determine risk. School Psychology Review, 42(4), 402–414.
42.
VanDerHeydenA. M.BurnsM. K. (2018). Improving decision making in school psychology: Making a difference in the lives of students, not just a prediction about their lives. School Psychology Review, 47(4), 385–395. https://doi.org/10.17105/SPR-2018-0042.V47-4
43.
VanDerHeydenA. M.BurnsM. K.BonifayW. (2018). Is more screening better? The relationship between frequent screening, accurate decisions, and reading proficiency. School Psychology Review, 47(1), 62–82. https://doi.org/10.17105/SPR-2017-0017.V47-1
YeoS. (2010). Predicting performance on state achievement tests using curriculum-based measurement in reading: A multilevel meta-analysis. Remedial and Special Education, 31(6), 412–422. https://doi.org/10.1177/0741932508327463
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
