Within a response to intervention model, educators increasingly use progress monitoring (PM) to support medium- to high-stakes decisions for individual students. For PM to serve these more demanding decisions requires more careful consideration of measurement error. That error should be calculated within a fixed linear regression model rather than a classical test theory model, which has been more common. Seven practical skills are described for improving the use of PM data for medium- to high-stakes decisions: (a) estimating a static performance level in PM, (b) fitting a level of confidence to an educational decision, (c) expressing an estimated score (Yhat) with its measurement error, (d) judging reliable improvement from one time to a later time, (e) properly using slope versus trendedness, (f) expressing “rate of improvement” (slope) with error, and (g) controlling autocorrelation. An example data set and PM graphs are used to illustrate each.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
2.
ArdoinS. P.ChristT. J. (2009). Curriculum based measurement of oral reading: Estimates of standard error when monitoring progress using alternate passage sets. School Psychology Review, 38, 266–283.
3.
ArmitageP.BerryG.MatthewsJ. N. (2002). Statistical methods in medical research (4th ed.). Oxford, UK: Blackwell.
4.
BoltS. (2005). Reflections on practice within the heartland problem-solving model: The perceived value of direct assessment of student needs. California School Psychologist, 10, 65–80.
5.
BoxO. E. P.JenkinsG. M. (1976). Time-series analysis: Forecasting and control (2nd ed.). San Francisco, CA: Holden-Day.
6.
BrownJ. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall.
7.
Brown-ChidseyR.SteegeM. W. (2005). Response to intervention: Principles and strategies for effective practice. New York, NY: Guilford.
8.
BrowneR. H. (1979). On visual assessment of the significance of a mean difference. Biometrics, 35, 657–665.
9.
BuskP. L.MarascuiloL. A. (1988). Autocorrelation in single-subject research: A counter-argument to the myth of no autocorrelation. Behavioral Assessment, 10, 229–242.
10.
ChristT. J.Coolong-ChaffinM. (2007). Interpretations of curriculum-based measurement outcomes: Standard error and confidence intervals. School Psychology Forum, 1, 75-86.
11.
CohenJ.CohenP. (1983). Applied multiple regression/ correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
12.
CohenJ.CohenP.WestS. G.AikenL. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
13.
CottrellA.LucchettiR. (2009). Gnu regression, econometrics and time-series library [Computer software]. Retrieved from http://gretl.sourceforge.net/
14.
CrockerL.AlginaJ. (1986). Introduction to classical and modern test theory. Orlando, FL: Harcourt Brace.
15.
DenoS. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.
16.
DenoS. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37, 184–192.
17.
DenoS. L.DenoD.MarstonD.MarstonD. (1987). Test of reading fluency: Measures for screening and progress monitoring. Minneapolis, MN: Children’s Educational Services.
18.
DenoS. L.FuchsL. S.MarstonD.ShinJ. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.
19.
DjulbegovicB.HozoI.SchwartzA.McMastersK. (1999). Acceptable regret in medical decision making. Medical Hypotheses, 53, 253–259.
20.
DraperN. R.SmithH. (1981). Applied regression analysis (2nd ed.). New York, NY: John Wiley.
21.
DraperN. R.SmithH. (1998). Applied regression analysis (3rd ed.). New York, NY: John Wiley.
22.
FrancisD. J.ShaywitzS. E.StuebingK. K.ShaywitzB. A.FletcherJ. M.LyonsG. R. (Ed.). (1994). The measurement of change: Assessing behavior over time and within a developmental context. In LyonsG. R. (Ed.), Frames of reference for the assessment of learning disabilities: New views on measurement issues (pp. 29–58). Baltimore, MD: Brookes.
FuchsL. S.FuchsD.HamlettC. L.WaltzL.GermannG. (1993). Formative evaluation of academic progress: How much growth can we expect?School Psychology Review, 22, 27–48.
25.
GlassG. V.WillsonV. L.GottmanJ. M. (1975). Design and analysis of time-series experiments. Boulder, CO: Associated University Press.
26.
GoldsteinH.HealyM. J. R. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society A, 158(1), 175–177.
27.
GoodR. H.SimmonsD. C.Kame’enuiE. J. (2001). The importance and decision making utility of a continuum of fluency based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257–288.
28.
GoodmanS. N. (1999). Toward evidence-based medical statistics. 1: The p value fallacy. Annals of Internal Medicine, 130, 995–1004.
29.
GraybillF. A.IyerH. K. (1998). Regression analysis: Concepts and applications. Belmont, CA: Duxbury Press.
30.
GreshamF. M. (2002). Responsiveness to intervention: An alternative approach to the identification of learning disabilities. In BradleyR.DonaldsonL.HallahanD. (Eds.), Identification of learning disabilities: Research to practice (pp. 467–519). Mahwah, NJ: Lawrence Erlbaum.
31.
Hagan-BurkeS.JeffersonG. L. (2002). Using data to promote academic benefit for included students with mild disabilities. Preventing School Failure, 46, 112–118.
32.
HintzeJ. (2007). Number cruncher statistical systems [Computer software]. Kaysville, UT: NCSS.
33.
HintzeJ. M.ChristT. J. (2004). An examination of variability as a function of passage variance in CBM progress monitoring. School Psychology Review, 33, 204–217.
34.
HoldnackJ. A.WeissL. G. (2006). IDEA 2004: Anticipated implications for clinical practice—Integrating assessment and intervention. Psychology in the Schools, 43, 871–892.
35.
HospM. K.HospJ. L.HowellK. W. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: Guilford.
36.
IoannidisJ. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. doi:10.1371/journal.pmed.0020124
37.
KavaleK. A.FornessS. R. (2000). History, rhetoric, and reality: Analysis of the inclusion debate. Remedial and Special Education, 21, 279–296.
38.
KozakA.KozakR. A.StaudhammerC. L.WattsS. B. (2008). Introductory probability and statistics: Applications for forestry and natural resources. New York, NY: CABI.
39.
KutnerM. H.NachtsheimC. J.NeterJ. (2004). Applied linear regression models (4th ed.). New York, NY: McGraw-Hill.
40.
MacMillanD. L.SemmelM. I.GerberM. M. (1996). The social context: Then and now. In KauffmanJ. M.HallahanD. P. (Eds.), The illusion of full inclusion: A comprehensive critique of a current special education bandwagon (pp. 19–38). Austin, TX: Pro-Ed.
41.
ManolovR.SolanasA. (2008). Comparing N = 1 effect size indices in presence of autocorrelation. Behavior Modification, 32, 860–875.
42.
MatyasT. A.GreenwoodK. M. (1996). Serial dependency in single-case time series. In FranklinR. D.AllisonD. B.GormanB. S. (Eds.), Design and analysis of single-case research (pp. 215–243). Mahwah, NJ: Lawrence Erlbaum.
43.
McGlincheyM. T.HixsonM. D. (2004). Using curriculum-based measurement to predict performance on state assessments in reading. School Psychology Review, 33, 193–203.
44.
NeterJ.KutnerM. H.NachtsheimC. J.WassermanW. (1996). Applied linear statistical models (4th ed.). New York, NY: McGraw-Hill.
45.
NeterJ.WassermanW.KutnerM. H. (1983). Applied linear regression models. Homewood, IL: Irwin.
46.
ParkerR. I. (2006). Increased reliability for single case research results: Is the bootstrap the answer?Behavior Therapy, 37, 326–338.
47.
ParkerR. I.CryerJ.ByrnsG. (2006). Controlling trend in single case research. School Psychology Quarterly, 21, 418–440.
48.
PaytonM. E.GreenstoneM. H.SchenkerN. (2003). Overlapping confidence intervals or standard error intervals: What do they mean in terms of statistical significance?Journal of Insect Science, 3(34), 1–6.
49.
PaytonM. E.MillerA. E.RaunW. R. (2000). Testing statistical hypotheses using standard error bars and confidence intervals. Communications in Soil Science and Plant Analysis, 31, 547–552.
50.
PedhazurE. J. (1982). Multiple regression in behavioral research: Explanation and prediction. Fort Worth, TX: Holt, Rinehart & Winston.
51.
PoncyB. C.SkinnerC. H.AxtellP. K. (2005). An investigation of the reliability and standard error of measurement of words read correctly per minute using curriculum based measurement. Journal of Psychoeducational Assessment, 23, 226–238.
52.
RestoriA. F.GreshamF. M.CookC. R. (2008). Old habits die hard: Past and current issues pertaining to response-to-intervention. California School Psychologist, 13, 67–79.
53.
SalkindN. J. (Ed.). (2007). The encyclopedia of measurement and statistics (Vol. 1). Thousand Oaks, CA: Sage.
54.
SchenkerN.GentlemanJ. F. (2001). On judging the significance of differences by examining overlap between confidence intervals. American Statistician, 55, 182–186.
55.
SchwarzC. J. (2007). Sampling, regression, experimental design and analysis for environmental scientists, biologists and resource managers. Burnaby, Canada: Simon Fraser University. Retrieved from http://www.stat.sfu.ca/~cschwarz/Stat-650/Notes/PDF/Chapter17.pdf
56.
SharpleyC. F.AlavosiusM. P. (1988). Autocorrelation in behavioral data: An alternative perspective. Behavioral Assessment, 10, 243–251.
57.
SheskinD. J. (2007). Handbook of parametric and nonparametric statistical procedures. Boca Raton, FL: Chapman & Hall.
58.
StageS. A.JacobsenM. D. (2001). Predicting student success on a state-mandated performance-based assessment using oral reading fluency. School Psychology Review, 30, 407–419.
59.
SuenH. K.AryD. (1987). Autocorrelation in applied behavior analysis: Myth or reality?Behavioral Assessment, 9, 125–130.
