This study examined the diagnostic accuracy associated with decision making as is typically conducted with curriculum-based measurement (CBM) approaches to progress monitoring. Using previously published estimates of the standard errors of estimate associated with CBM, 20,000 progress-monitoring data sets were simulated to model student reading growth of two-word increase per week across 15 consecutive weeks. Results indicated that an unacceptably high proportion of cases were falsely identified as nonresponsive to intervention when a common 4-point decision rule was applied, under the context of typical levels of probe reliability. As reliability and stringency of the decision-making rule increased, such errors decreased. Findings are particularly relevant to those who use a multi-tiered response-to-intervention model for evaluating formative changes associated with instructional intervention and evaluating responsiveness to intervention across multiple tiers of intervention.
ArdoinS. P.ChristT. J. (2009). Curriculum-based measurement of oral reading: Standard errors associated with progress monitoring outcomes from DIBELS, AIMSweb, and an experimental passage set. School Psychology Review, 38, 266-283.
2.
ArdoinS. P.ChristT. J.MorenaL. S.CormierD. C.KlingbeilD. A. (2013). A systematic review and summarization of the recommendations and research surrounding Curriculum-Based Measurement of oral reading fluency (CBM-R) decision rules. Journal of School Psychology, 51, 1-18.
3.
Brown-ChidseyR.SteegeM. W. (2010). Response to intervention: Principles and strategies for effective practice (2nd ed.). New York, NY: The Guilford Press.
4.
ChristT. J.SilberglittB. (2007). Estimates of the standard error of measurement for curriculum-based measures of oral reading fluency. School Psychology Review, 36, 130-146.
5.
ChristT. J.ZoplugluC.LongJ. D.MonaghenB. D. (2012). Curriculum-based measurement of oral reading: Quality of progress monitoring outcomes. Exceptional Children, 78, 356-373.
6.
ChristT. J.ZoplugluC.MonaghenB. D.Van NormanE. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19-57.
7.
ConeJ. D. (1989). Is there utility for treatment utility?American Psychologist, 44, 1241-1242.
8.
DenoS. L.FuchsL. S. (1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus on Exceptional Children, 19, 1-15.
9.
DenoS. L.FuchsL. S.MarstonD.ShinJ. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507-524.
10.
DeProsperoA.CohenS. (1979). Inconsistent visual analysis of intrasubject data. Journal of Applied Behavior Analysis, 12, 573-579.
11.
FuchsD.MockD.MorganP. L.YoungC. L. (2003). Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research and Practice, 18, 157-171.
12.
FuchsL. S.DenoS. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488-500.
13.
FuchsL. S.DenoS. L.MirkinP. (1984). The effects of frequent curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449-460.
14.
FuchsL. S.FuchsD. (1986a). Curriculum-based assessment of progress toward long- and short-term goals. The Journal of Special Education, 20, 69-82.
15.
FuchsL. S.FuchsD. (1986b). Effects of systematic formative evaluation on student achievement: A meta-analysis. Exceptional Children, 53, 199-208.
16.
FuchsL. S.FuchsD. (1998). Treatment validity: A unifying concept for reconceptualizing identification of learning disabilities. Learning Disabilities Research and Practice, 14, 204-219.
17.
FuchsL. S.FuchsD.HamlettC. L. (1990). Curriculum-based measurement: A standardized long-term goal approach to monitoring student progress. Academic Therapy, 25, 615-632.
18.
FuchsL. S.FuchsD.HamlettC. L.WalzL.GermannG. (1993). Formative evaluation of academic progress: How much growth can we expect?School Psychology Review, 22, 27-48.
19.
HarbstK. B.OttenbacherK. J.HarrisS. R. (1991). Interrater reliability of therapists’ judgements of graphed data. Physical Therapy, 71, 107-115.
20.
HayesS. C.NelsonR. O.JarrettR. (1987). The treatment utility of assessment: A functional approach to evaluating assessment quality. American Psychologist, 42, 963-974.
21.
HintzeJ. M. (2009). Conceptual and empirical issues related to developing a response-to-intervention framework. Journal of Evidence-Based Practices for Schools, 9, 128-147.
22.
HintzeJ. M.ChristT. J. (2004). An examination of variability as a function of passage variance in CBM progress monitoring. School Psychology Review, 33, 204-217.
23.
HintzeJ. M.DalyE. J.ShapiroE. S. (1998). An investigation of the effects of passage difficulty level on outcomes of oral reading fluency progress monitoring. School Psychology Review, 27, 433-445.
24.
HintzeJ. M.OwenS. V.ShapiroE. S.DalyE. J. (2000). Generalizability of oral reading fluency measures: Application of G theory to curriculum-based measurement. School Psychology Quarterly, 15, 52-68.
25.
HospM. K.HospJ. L.HowellK. W. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: The Guilford Press.
26.
MercerS. H.SterlingH. E. (2012). The impact of baseline trend control on visual analysis of single-case data. Journal of School Psychology, 50, 403-419.
27.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13-103). New York, NY: Macmillan.
28.
MessickS. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performance as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
29.
National Association of State Directors of Special Education. (2005). Response to intervention: Policy considerations and implementation. Alexandria, VA: Author.
30.
NeseJ. F. T.BiancarosaG.CummingsK.KennedyP.AlonzoJ.TindalG. (2013). In search of average growth: Describing within-year oral reading fluency growth across grades 1-8. Journal of School Psychology, 51, 625-642.
31.
OttenbacherK. J. (1990). Visual inspection of single-subject data: An empirical analysis. Mental Retardation, 28, 283-290.
32.
ParkH.MarascuiloL. A.Gaylord-RossR. (1990). Visual inspection and statistical analysis of single-case designs. Journal of Experimental Education, 58, 311-320.
33.
ReschlyA. L.BuschT. W.BettsJ.DenoS. L.LongJ. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427-469.
34.
ReschlyD. J.TillyW. D.GrimesJ. (Eds.). (1999). Special education in transition: Functional assessment and noncategorical programming. Longmont, CO: Sopris West.
35.
SechrestL. (1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23, 153-158.
36.
ShapiroE. S. (2004). Academic skills problems: Direct assessment and intervention (3rd ed.). New York, NY: The Guilford Press.
37.
ShinnM. R. (Ed.). (1989). Curriculum-based measurement: Assessing special children. New York, NY: The Guilford Press.
38.
ShinnM. R.BamontoS. (1998). Advanced applications of curriculum-based measurement: “Big ideas” and avoiding confusion. In ShinnM. R. (Ed.), Advanced application of curriculum-based measurement (pp. 1-31). New York, NY: The Guilford Press.
39.
ThornbladS. C.ChristT. J. (2014). Curriculum-based measurement of reading: Is 6 weeks of daily progress monitoring enough?School Psychology Review, 43, 19-29.
40.
Van NormanE. R.ChristT. J. (2016a). Curriculum-based measurement of reading: Accuracy of recommendations from three-point decision rules. School Psychology Review, 45, 296-309.
41.
Van NormanE. R.ChristT. J. (2016b). How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules. Journal of School Psychology, 58, 41-55.
