The accuracy of decision rules for progress monitoring data is influenced by multiple factors. This study examined the accuracy of decision rule recommendations with over 4,500 second–and third–grade students receiving a tier II reading intervention program. The sensitivity and specificity of three decision rule recommendations for predicting year–end spring benchmark targets was evaluated over different data collection durations under good and poor dataset conditions. Across grade level and dataset quality, the sensitivity of decisions made using trend lines and the median of recent data points tended to improve to acceptable levels after 18 weeks. Specificity tended to decrease around that same time but was less pronounced with the median decision rule method. Limitations and directions for future research are discussed.
Get full access to this article
View all access options for this article.
References
1.
AIMSweb (2012). AIMSweb roi growth norms. Pearson Inc. Available upon request from Publisher.
2.
ArdoinS. P., ChristT. J., MorenaL. S., CormierD. C., & KlingbeilD. A. (2013). A systematic review and summarization of the recommendations and research surrounding curriculum–based measurement of oral reading fluency (CBM–R) decision rules. Journal of School Psychology, 51(1), 1–18.
3.
BetebennerD. W. (2008). Norm and criterion reference student growth. The National Center for the Improvement of Educational Assessment. Retrieved April 7, 2017 from http://www.nceia.org.
4.
ChristT. J. (2006). Short–term estimates of growth using curriculum–based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35, 128–133.
5.
ChristT. J., ArañasY. A., KemberJ. M., KissA. J., McCarthy–TrentmanA., MonaghenB. D., et al. (2014). Formative assessment system for teachers technical manual: EarlyReading, CBMReading, aReading, aMath, and early–Math. Minneapolis, MN: Formative Assessment System for Teachers.
6.
ChristT. J., ZopluogluC., MonaghenB. D., & Van NormanE. R. (2013). Curriculum–based measurement of oral reading: Multi–study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51(1), 19–57.
7.
CummingsK. D., StoolmillerM. L., BakerS. K., FienH., & Kame'enuiE. J. (2015). Using school–level student achievement to engage in formative evaluation: comparative school–level rates of oral reading fluency growth conditioned by initial skill for second grade students. Reading and Writing, 28(1), 105–130.
8.
DenoS. L. (1985). Curriculum–based measurement: The emerging alternative. Exceptional Children, 52, 219–232.
9.
DenoS. L., FuchsL. S., MarstonD., & ShinJ. (2001). Using curriculum–based measurements to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.
10.
FuchsD., & FuchsL. S. (2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41, 93–99.
11.
FuchsL. S., & FuchsD. (1986). Effects of systematic formative evaluation: A meta–analysis. Exceptional Children, 53, 199–208.
12.
FuchsL. S., & FuchsD. (2011). Using CBM for progress monitoring in reading. Washington, D.C.: Student Progress Monitoring, U.S. Department of Education. Retrieved 19 April 2017 from http://files.eric.ed.gov/fulltext/ED519252.pdf.
13.
FuchsL. S., FuchsD., HamlettC. L., WalzL., & GermannG. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27–27.
14.
GerstenR., ComptonD., ConnorC. M., DiminoJ., SantoroL., Linan–ThompsonS., et al. (2008). Assisting students struggling with reading: Response to Intervention and multi–tier intervention for reading in the primary grades. A practice guide. (NCEE 2009–4045). Washington, D.C.: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practiceguides.
15.
GoodR. H., Powell–SmithK. A., & DeweyE. (2015). Making reliable and stable progress decisions: Slope or pathways of progress. In Poster presented at the Annual Pacific Coast Research Conference, Coronado, CA.
16.
HintzeJ. M., & SilberglittB. (2005). A longitudinal examination of the diagnostic accuracy and predictive validity of R–CBM and high–stakes testing. School Psychology Review, 34, 372–386.
17.
JenkinsJ., & TerjesonK. J. (2011). Monitoring reading growth: Goal setting, measurement frequency, and methods of evaluation. Learning Disabilities Research & Practice, 26(1), 28–35.
18.
JimersonS. R., BurnsM. K., & VanDerHeydenA. M. (Eds.) (2016). The handbook of response to intervention: Science and practice of multi–tiered systems of support (2nd ed.). New York: Springer Science. https://doi.org/10.1007/978-1-4899-7568-3
NewellK. W., & ChristT. J. (2017). Novice interpretations of progress monitoring graphs: Extreme values and graphical aids. Assessment for Effective Intervention, 42, 224–236.
21.
RaudenbushS. W., & BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
22.
ReschlyA. L., BuschT. W., BettsJ., DenoS. L., & LongJ. D. (2009). Curriculum–based measurement oral reading as an indicator of reading achievement: A meta–analysis of the correlational evidence. Journal of School Psychology, 47, 427–469.
23.
ShinnM. R., GoodR. H., & SteinS. (1989). Summarizing trend in student achievement: A comparison of methods. School Psychology Review, 18, 356–370.
24.
SilberglittB., & HintzeJ. M. (2007). How much growth can we expect? A conditional znalysis of R–CBM growth rates by level of performance. Exceptional Children, 74, 71–84.
25.
SteckerP. M., FuchsL. S., & FuchsD. (2005). Using curriculum–based measurement to improve student achievement: Review of research. Psychology in the Schools, 42, 795–819.
26.
TusingM. E., & BreikjernN. A. (2017). Using curriculum–based measurement for program evaluation: Expanding roles for school psychologists. Journal of Applied School Psychology, 33, 43–66.
27.
U.S. Department of Education (2004). Individuals with Disabilities Improvement Act of 2004, Pub. L. 108–466. Federal Register, 70, 35802–35803
28.
Van NormanE. R., & ChristT. J. (2016). Curriculum–based measurement of reading: Accuracy of recommendations from three–point decision rules. School Psychology Review, 45, 296–309.
29.
Van NormanE. R., ChristT. J., & NewellK. W. (2017). Curriculum–based measurement of reading progress monitoring: The importance of growth magnitude and goal setting in decision making. School Psychology Review, 46(3), 320–328.
30.
Van NormanE. R., ChristT. J., & ZopluogluC. (2013). The effects of baseline estimation on the reliability, validity, and precision of CBM–R growth estimates. School Psychology Quarterly, 28, 239–255.
31.
Van NormanE. R., & ParkerD. C. (2017). A comparison of common and novel curriculum–based measurement of reading decision rules to predict spring performance for students receiving supplemental interventions. Assessment for Effective Intervention. Advance online publication.
32.
VanDerHeydenA. M., WittJ. C., & BarnettD. W. (2005). The emergence and possible futures of response to intervention. Journal of Psychoeducational Assessment, 23, 339–361.
33.
VujnovicR. K., FabianoG. A., MorrisK. L., NormanK., HallmarkC., & HartleyC. (2014). Examining school psychologists’ and teachers’ application of approaches within a response to intervention framework. Exceptionality, 22, 129–140.
YeoS., KimD. I., Branum–MartinL., WaymanM. M., & EspinC. A. (2012). Assessing the reliability of curriculum–based measurement: An application of latent growth modeling. Journal of School Psychology, 50, 275–292.
