ABSTRACT: More and more, schools are considering the use of progress monitoring data for high-stakes decisions such as special education eligibility, program changes to more restrictive environments, and major changes in educational goals. Those high-stakes types of data-based decisions will need methodological defensibility. Current practice for summarizing progress monitoring data is to use a hand-fit trend lines (for practitioner use) or linear regression (for research). This study critically examines both approaches and compares them to a new nonparametric slope called the Theil–Sen. A field test with 372 published data series compared hand-fit, linear regression, and Theil–Sen slopes against evaluative criteria of power and precision, meeting data assumptions, and agreement with visual judgments. Results indicate promise for Theil–Sen slope in defensible high-stakes decision making.
Get full access to this article
View all access options for this article.
References
1.
ArdoinS. P., & ChristT. J. (2009). Curriculum based measurement of oral reading: Estimates of standard error when monitoring progress using alternate passage sets.School Psychology Review38, 266–283.
2.
BaerD. M., WolfM. M., & RisleyT. R. (1968). Some current dimensions of applied behavior analysis.Journal of Applied Behavior Analysis1, 91–97.
3.
BerkeleyS., BenderW. N., PeasterG. L., & SaundersL. (2009). Implementation of Response to Intervention: A snapshot of progress.Journal of Learning Disabilities42, 85–95.
4.
BijouS. W., & BaerD. M. (1961). Child development: A systematic and empirical theory.New York: Appleton-Century-Crofts.
5.
BolgerH. (1965). The case study method. In WolmanB. B. (Ed.), Handbook of clinical psychology.New York: McGraw-Hill.
6.
BoltS. (2005). Reflections on practice within the heartland problem-solving model: The perceived value of direct assessment of student needs.The California School Psychologist10, 65–80.
7.
Brown-ChidseyR., & SteegeM. W. (2005). Response to intervention: Principles and strategies for effective practice.New York: Guilford.
8.
ChristT. J. (2006). Short term estimates of growth using curriculum-based measurement of oral reading fluency: Estimates of standard error of the slope to construct confidence intervals.School Psychology Review35(1), 128–133.
9.
ChristT. J., & ArdoinS. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development.Journal of School Psychology47, 55–75.
10.
ChristT. J., & Coolong-ChaffinM. (2007). Interpretations of curriculum-based measurement outcomes: Standard error and confidence intervals.School Psychology Forum1, 75–86.
11.
DenoS., FuchsL. S., MarstonD. B., & ShinJ. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities.School Psychology Review30, 507–524.
12.
EstesW. K. (1956). The problem of inference from curves based on group data.Psychological Bulletin53, 134–140.
13.
FantuzzoJ. W., PoliteK., & GraysonN. (1990). An evaluation of reciprocal peer tutoring across elementary school settings.Journal of School Psychology28, 309–323.
14.
FrancisD.J., SantiK. L., BarrC., FletcherJ. M., VariscoA., & FoormanB. R. (2008). Form effects on the estimation of students’ oral reading fluency using DIBELS.Journal of School Psychology46, 315–342.
15.
FuchsD., DeshlerD., & ReschlyD. (2004). National Research Center on Learning Disabilities: Multi-method approaches to the study of LD identification and classification.Learning Disability Quarterly27, 189–195.
16.
FuchsD., FuchsL. S., McMasterK. N., & AI OtaibaS. (2003). Identifying children at risk for reading failure: Curriculum-based measurement and the dual-discrepancy approach. In SwansonH. L., HarrisK. R., & GrahamS. E. (Eds.), Handbook on learning disabilities (pp. 431–449). New York: Guilford.
17.
GerberM. (2005). Teachers are still the test: Limitations of response to intervention strategies for identifying children with learning disabilities.Journal of Learning Disabilities38(6), 516–524.
18.
GoodR. H., & JeffersonG. (1988). Contemporary perspectives on Curriculum Based measurement Validity. In ShinnM. R. (Ed.), Advanced applications of Curriculum-based Measurement (pp. 61–88). New York: Guilford.
19.
GreshamF. M. (2002). Teaching social skills to high-risk children and youth: Preventive and remedial strategies. In ShinnM. R., WalkerH. M., & StonerG. (Eds.), Interventions for academic and behavior problems II: Preventive and remedial approaches (pp. 403–432). Bethesda, MD: National Association of School Psychologists.
20.
HaleJ. B. (2006). Differential medication response in ADHD subtypes.LAS Special Populations Pediatrics Review, 62–69.
21.
HaleJ. B., NaglieriJ. A., KaufmanA. S., & KavaleK. A. (2004). Specific learning disability classification in the new Individuals with Disabilities Education Act: The danger of good ideas.The School Psychologist58(1), 6–14,
HoldnackJ. A., & WeissL. G. (2006). IDEA 2004: Anticipated implications for clinical practice-Integrating assessment and intervention.Psychology in the Schools43, 871–892.
24.
JenkinsJ. R., ZumetaR., DupreeO., & JohnsonK. (2005). Measuring gains in reading ability with passage reading fluency.Learning Disabilities Research and Practice20(4), 245–253.
25.
JohnstoneI., & VellemanP. F. (1985). Efficient scores, variance decompositions, and Monte Carlo swindles.Journal of the American Statistical Association80, 851–862.
26.
KavaleK. A., & FornessS. R. (2000). What definitions of learning disability say or don't say: A critical analysis.Journal of Learning Disabilities33(3), 239–256.
27.
KavaleK. A., HoldnackJ. A., & MostertM. P. (2006). Responsiveness to intervention and the identification of specific learning disability: A critique and alternative proposal.Learning and Disability Research & Practice29, 113–127.
28.
KernL., & BambaraL. (2002). Class-wide curricular modification to improve the behavior of students with emotional or behavioral disorders.Behavior Disorders27(4), 317–326.
29.
KoenigC. (1972). Charting the future course of behavior.Kansas City, KS: Precision Media.
30.
MarstonD. B. (1989). A Curriculum-based Measurement approach to assessing academic performance: Wate it is and why to do it. In ShinnM. (Ed). Curriculum-based Measurement Assessing Special Children.New York: Guildford.
31.
MoodA. M. (1950). Introduction to the theory of statistics.New York: McGraw-Hill.
32.
MoserB. K., & StevensG. R. (1992). Homogeneity of variance in the two-sample means test.American Statistician4, 19–21.
33.
NaglieriJ. A., & CrockettD. P. (2005). Response to intervention (RTI): Is it a scientifically proven method?Communique34(2), 38–39.
34.
ParkerR. I., CryerJ., & ByrnsG. (2006). Controlling baseline trend in single case research.School Psychology Quarterly21(4), 418–444,
35.
ParkerR., SteinM., & TindalG. (1992). Estimating trend in progress monitoring data: A comparison of simple line-fitting methods.School Psychology Review2, 300–313.
36.
PoncyB. C., SkinnerC. H., & AxtellP. K. (2005). An investigation of the reliability and standard error of measurement of words read correctly per minute using curriculum based measurement.Journal of Psychoeducational Assessment23, 226–238.
37.
RestoriA. F., GreshamF. M., & CookC. R. (2008). Old habits die hard: Past and current issues pertaining to response-to-intervention.The California School Psychologist13, 67–79.
38.
ReynoldsC. R., & ShaywitzS. E. (2009). Response to intervention: Ready or not? Or, from wait-to-fail to watch-them-fail.School Psychology Quarterly24(2), 130–145.
39.
SenP. K. (1968). Estimates of the regression coefficient based on Kendall's tau.American Statistical Association Journal63, 1379–1389.
40.
ShapiroS. S., & WilkM. B. (1965). An analysis of variance test for normality (complete samples).Biometrika52(3–4), 591–611.
41.
SidmanM. (1960). Tactics for scientific research: Evaluating experimental data in psychology.New York: Basic Books.
42.
SiegelS., & CastellanN. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.
43.
SkinnerB. F. (1966). Science and human behavior.New York: McMillian.
44.
SteckerP. M., LembkeE. S., & SáenzL. (2007). Advanced application of CBM in reading: Instructional decision-making strategies. Presentation at 2007 Summer Institute on Student Progress Monitoring, Nashville, TN. Retrieved from http://www.studentprogress.org/weblibrary.asp#data
45.
SwansonH. L., KozleskiE., & SteginkP. (1987). Disabled readers’ processing of prose: Do any processes change because of intervention?Psychology in the Schools24, 378–384.
46.
TheilH. (1950). A rank-invariant method of linear and polynomial regression analysis, III.Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen A, 53, 1397–1412.
47.
TukeyJ. W. (1977). Straightening out plots (using three points). In Exploratory data analysis (pp. 169–181). Menlo Park, CA: Addison-Wesley.
48.
WaldA. (1940). The fitting of straight lines if both variables are subject to error.Annals of Mathematical Statistics11, 284–300. U.S. Office of Special Education Programs. National Center on Student Progress Monitoring http://www.studentprogress.org/weblibrary.asp dowloaded August 14, 2012.
49.
WhiteO. R. (1972). The prediction of human performances in the simple case: An example of four techniques. (Working Paper No. 15), University of Oregon Regional Resource Center for Handicapped Children, Eugene, OR.
50.
WhiteO. R. (1974). The split middle: A quickie method of trend estimation (3rd revision). Unpublished manuscript, University of Washington, Experimental Education Unit, Child Development and Mental Retardation Center, Seattle, WA.
51.
WodrichD. L., & SchmittA. J. (2006). Patterns of learning disabilities.New York: Guilford Press.
