Curriculum-Based Measurement of Reading (CBM-R) is frequently used to monitor instructional effects and evaluate response to instruction. Educators often view the data graphically on a time-series graph that might include a variety of statistical and visual aids, which are intended to facilitate the interpretation. This study evaluated the effects of brief training, graphical aids, and presence of extreme/outlier values on accuracy. Novice visual analysts (N = 173) were randomly assigned to a training condition: control, trend line calculation, or extreme value identification. All participants interpreted graphical displays of progress monitoring data with and without trend lines and extreme values. Novice visual analysts’ performance on a knowledge test was uniformly high. Results indicate that the overall probability of an accurate response was .80, which improved to a maximum of .94 within the condition with trend lines and no extreme values. Specifically, results indicate that even in the presence of extreme values, trend lines facilitate novice visual analysts’ accuracy.
ArdoinS. P.ChristT. J. (2009). Curriculum-based measurement of oral reading: Standard errors associated with progress monitoring outcomes from DIBELS, AIMSweb, and an experimental passage set. School Psychology Review, 38, 266–283.
2.
ArdoinS. P.ChristT. J.MorenaL. S.CormierD. C.KlingbeilD. A. (2013). A systematic review and summarization of the recommendations and research surrounding Curriculum-based measurement of oral reading fluency (CBM-R) decision rules. Journal of School Psychology, 51, 1–18. doi:10.1016/j.jsp.2012.09.004
BegenyJ. C.MartensB. K. (2006). Assessing pre-service teachers’ training in empirically-validated behavioral instruction practices. School Psychology Quarterly, 21, 262–285.
5.
ChristT. J. (2006). Short-term estimates of growth using curriculum-based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35, 128–133.
6.
ChristT. J. (2015). Formative Assessment System for Teachers. Minneapolis, MN: FastBridge Learning.
7.
ChristT. J.WhiteM. J.ArdoinS. P.EckertT. L. (2013). Curriculum-based measurement of reading: Consistency and validity across best, fastest, and question reading conditions. School Psychology Review, 42, 415–436.
8.
ChristT. J.ZopluogluC.MonaghenB. D.Van NormanE. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19–57. doi:10.1016/j.jsp.2012.11.001
9.
CummingsK. D.BiancarosaG.SchaperA.ReedD. K. (2014). Examiner error in curriculum-based measurement of oral reading. Journal of School Psychology, 52, 361–375.
10.
DenoS. L. (1985). Curriculum-based measurement: The emerging alternative. The Council for Exceptional Children, 52, 219–232.
11.
DenoS. L. (1986). Direct and frequent curriculum-based measurement: An alternative for educational decision making. Special Services in the Schools, 2, 5–27.
12.
DenoS. L. (1990). Individual differences and individual difference: The essential difference of special education. The Journal of Special Education, 24, 160–173. doi:10.1177/002246699002400205
13.
DenoS. L.FuchsL. S.MarstonD.ShinJ. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.
14.
DeprosperoA.CohenS. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 12, 573–579.
15.
DerrT. F.ShapiroE. S. (1989). A behavioral evaluation of curriculum-based assessment of reading. Journal of Psychoeducational Assessment, 7, 148–160.
16.
FrancisD. J.SantiK. L.BarrC.FletcherJ. M.VariscoA.FoormanB. R. (2008). Form effects on the estimation of students’ oral reading fluency using DIBELS. Journal of School Psychology, 46, 315–342.
17.
FuchsD.FuchsL. S. (2006). Introduction to response to intervention: What, why, and how valid is it?Reading Research Quarterly, 41, 93–99.
18.
FuchsL. S. (1989). Evaluating solutions: Monitoring progress and revising intervention plans. In ShinnM. R. (Ed.), Curriculum-based measurement: Assessing special children (pp. 153–181). New York, NY: Guilford.
19.
FuchsL. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188–192.
20.
FuchsL. S.FuchsD. (1986). Effects of formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.
21.
FuchsL. S.FuchsD. (1993). Formative evaluation of academic progress: How much growth can we expect?School Psychology Review, 22, 1–30.
22.
GastD. L. (2010). Single subject research methodology in behavioral sciences. New York, NY: Routledge.
23.
HixsonM. D.ChristT. J.BruniT. (2013). Best practices in the analysis of progress-monitoring data and decision making. In HarrisonP. L.ThomasA. (Eds.), Best practices in school psychology (Vol. 44, pp. 1–12). Bethesda, MD: NASP.
24.
KahngS.ChungK.-M.GutshallK.PittsS. C.KaoJ.GirolamiK. (2010). Consistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 43, 35–45. doi:10.1901/jaba.2010.43-35
25.
KazdinA. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
26.
MarstonD. (1988). Measuring progress on IEPs: A comparison of graphing approaches. Exceptional Children, 55, 38–44.
27.
MercerS. H.LyonsA. F.JohnstonL. E.MillhoffC. L. (2015). Robust regression for slope estimation in curriculum-based measurement progress monitoring. Assessment for Effective Intervention, 40, 176–183. doi:10.1177/1534508414555705
28.
OttenbacherK. (1986). Reliability and accuracy of visually analyzing graphed data from single-subject designs. The American Journal of Occupational Therapy, 40, 464–469.
29.
R Core Team. (2014). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org/
30.
ShapiroE. S. (2011). Academic skills problems (4th ed.). New York, NY: Guilford.
31.
ShinnM. R.GoodR. H.SteinS. (1989). Summarizing trend in student achievement: A comparison of methods. School Psychology Review, 18, 356–370.
32.
SkibaR. J.DenoS. L.MarstonD.CaseyA. (1989). Influence of trend estimation and subject familiarity on practitioners’ judgments of intervention effectiveness. The Journal of Special Education, 22, 433–446.
33.
SkibaR. J.DenoS. L.MarstonD.WessonC. (1986). Characteristics of time-series data collected through CBM-R.pdf. Diagnostique, 12, 3–15.
34.
SteckerP. M.FuchsL. S.FuchsD. (2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42, 795–819.
35.
ThornbladS. C.ChristT. J. (2014). Curriculum-based measurement of reading: Is six weeks of daily progress monitoring enough?School Psychology Review, 43, 19–29.
36.
Van NormanE. R.ChristT. J. (2016). How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules. Journal of School Psychology, 58, 41–55.
37.
Van NormanE. R.NelsonP. M. (2013, February). Visual analysis of time series data: Obstacles and solutions. Paper presented at the National Association of School Psychologists Annual Convention, Seattle, WA.
38.
Van NormanE. R.NelsonP. M.ShinJ.-E.ChristT. J. (2013). An evaluation of the effects of graphic aids in improving decision accuracy in a continuous treatment design. Journal of Behavioral Education, 22, 283–301. doi:10.1007/s10864-013-9176-2
