Restricted accessResearch articleFirst published online 2016-9
Predictive Utility and Classification Accuracy of Oral Reading Fluency and the Measures of Academic Progress for the Wisconsin Knowledge and Concepts Exam
This study examined the predictive validity and classification accuracy of two commonly used universal screening measures relative to a statewide achievement test. Results indicated that second-grade performance on oral reading fluency and the Measures of Academic Progress (MAP), together with special education status, explained 68% of the variance in third-grade standardized test performance (N = 399). Six methods of dichotomous risk classification were also explored to predict whether students would pass or fail the state test. A random sample of half of the participants was used to establish locally appropriate risk cut-scores, holding sensitivity constant at .95; the cut-scores were then cross-validated on the remaining participants. Cross-validation reflected sensitivity of above .90 for all methods, with MAP Language Usage and two multi-screener methods demonstrating the strongest overall technical adequacy for risk classification. The limitations and implications of these findings are discussed.
AkobengA. K. (2007). Understanding diagnostic tests 2: Likelihood ratios, pre- and post-test probabilities and their use in clinical practice. Acta Paediatrica, 96, 487–491. doi:10.1111/j.1651-2227.2006.00179.x
2.
AndrenK. J. (2010). An analysis of the concurrent and predictive validity of curriculum-based measures (CBM), the Measures of Academic Progress (MAP), and the New England Common Assessment Program (NECAP) for reading (Unpublished doctoral dissertation). University of Southern Maine, Portland.
3.
ArdoinS. P.ChristT. J. (2008). Evaluating curriculum-based measurement slope estimates using data from triannual universal screenings. School Psychology Review, 37, 109–125.
4.
BallC. R.ChristT. J. (2012). Supporting valid decision making: Uses and misuses of assessment data within the context of RTI. Psychology in the Schools, 49, 231–244. doi:10.1002/pits.21592
5.
BallC. R.FinchW. H.GettingerM., & K-3 Reading and Behavior Intervention Project. (2014). Classroom-level effects on the reading and behavior of at-risk kindergartners. Preventing School Failure: Alternative Education for Children and Youth, 58, 80–89.
6.
BishopA. G.LeagueM. B. (2006). Identifying a multivariate screening model to predict reading difficulties at the onset of kindergarten: A longitudinal analysis. Learning Disability Quarterly, 29, 235–252.
7.
ClemensN. H.ShapiroE. S.ThoemmesF. (2011). Improving the efficacy of first grade reading screening: An investigation of word identification fluency with other early literacy indicators. School Psychology Quarterly, 26, 231–244. doi:10.1037/a0025173
8.
ComptonD. L.FuchsD.FuchsL. S.BryantJ. D. (2006). Selecting at-risk readers in first grade for intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98, 394–409. doi:10.1037/0022-0663.98.2.394
9.
FuchsL. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188–192.
10.
FuchsL. S.DenoS. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–500.
11.
GloverT. A.AlbersC. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135. doi:10.1016/j.jsp.2006.05.005
12.
GoffredaC. T.DiPernaJ. C. (2010). An empirical review of psychometric evidence for the Dynamic Indicators of Basic Early Literacy Skills. School Psychology Review, 39, 463–483.
13.
GoodR. H.KaminskiR. A. (Eds.). (2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Available from http://dibels.uoregon.edu
14.
GoodR. H.KaminskiR. A.SmithS.BrattenJ. (2001). Technical adequacy of second grade DIBELS Oral Reading Fluency passages (Technical Report No. 8). Eugene: University of Oregon.
15.
HintzeJ. M.SilberglittB. (2005). A longitudinal analysis of the diagnostic accuracy and predictive validity of R-CBM and high-stakes testing. School Psychology Review, 34, 372–386.
16.
JenkinsJ. R.HudsonR. F.JohnsonE. S. (2007). Screening for at-risk readers in a response-to-intervention framework. School Psychology Review, 36, 582–600.
17.
JohnsonE. S.JenkinsJ. R.PetscherY. (2010). Improving the accuracy of a direct route screening process. Assessment for Effective Intervention, 35, 131–142. doi:10.1177/1534508409348375
18.
JohnsonE. S.JenkinsJ. R.PetscherY.CattsH. W. (2009). How can we improve the accuracy of screening instruments?Learning Disabilities Research & Practice, 24, 174–185. doi:10.1111/j.1540-5826.2009.00291.x
19.
KilgusS. P.MetheS. A.MagginD. M.TomasulaJ. L. (2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377–405. doi:10.1016/j.jsp.2014.06.002
20.
Kingsbury Center at Northwest Evaluation Association. (2010). Wisconsin linking study: A study of the alignment of the NWEA RIT scale with the Wisconsin Knowledge and Concepts Examination. Lake Oswego, OR: Author.
21.
MacDonaldH. H.SullivanA. L.WatkinsM. W. (2013). Multivariate screening model for later word reading achievement: Predictive utility of prereading skills and cognitive ability. Journal of Applied School Psychology, 29, 52–71. doi:10.1080/15377903.2013.751476
22.
MaziarzA. L. (2010). The exploration of demographics and computer adaptive testing in predicting performance on state-mandated reading assessments (Unpublished doctoral dissertation). Indiana University of Pennsylvania, Indiana.
23.
MellardD. F.McKnightM.WoodsK. (2009). Response to intervention screening and progress monitoring practices in 41 local schools. Learning Disabilities Research & Practice, 24, 186–195.
24.
MerinoK.BeckmanT. O. (2010). Using reading curriculum-based measurements as predictors for the Measures of Academic Progress (MAP) standardized test in Nebraska. International Journal of Psychology: A Biopsychosocial Approach, 6, 85–98.
25.
MoyerE. L.GalindoJ. L.DoddB. G. (2012). Balancing flexible constraints and measurement precision in computerized adaptive testing. Educational and Psychological Measurement, 72, 629–648. doi:10.1177/0013164411431838
26.
National Association of School Psychologists. (2009). School psychologists’ involvement in assessment (Position statement). Bethesda, MD: Author.
Northwest Evaluation Association. (2009). Technical manual for measures of academic progress and measures of academic progress for primary grades. Lake Oswego, OR: Author.
PetscherY.KimY.FoormanB. R. (2011). The importance of predictive power in early screening assessments: Implications for placement in the response to intervention framework. Assessment for Effective Intervention, 36, 158–166. doi:10.1177/1534508410396698
31.
ReschlyA. L.BuschT. W.BettsJ.DenoS. L.LongJ. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427–469. doi:10.1016/j.jsp.2009.07.001
32.
VanDerHeydenA. M. (2011). Technical adequacy of RtI decisions. Exceptional Children, 77, 335–350.
33.
VanDerHeydenA. M. (2013). Universal screening may not be for everyone: Using a threshold model as a smarter way to determine risk. School Psychology Review, 42, 402–414.
34.
WangS.McCallM.JiaoH.HarrisG. (2013). Construct validity and measurement invariance of computerized adaptive testing: Application to Measures of Academic Progress (MAP) using confirmatory factor analysis. Journal of Educational and Developmental Psychology, 3, 88–100. doi:10.5539/jedp.v3n1p88
35.
WaymanM. M.WallaceT.WileyH. I.TichaR.EspinC. A. (2007). Literature synthesis on curriculum-based measurement in reading. The Journal of Special Education, 41, 85–120. doi:10.1177/00224669070410020401
36.
Wisconsin Department of Public Instruction. (2005). Wisconsin student assessment system criterion-referenced test framework for the WKCE statewide assessment. Madison: Author.
37.
ZellmerM. B.FrontierA.PheiferD. (2006). What are NCLB’s instructional costs?Educational Leadership, 64, 43–46.
