This study outlines a formative assessment sys tem using a consistent set of cut scores on Curriculum-Based Measurement-Reading (CBM-R) probes and investigates four statisti cal methods for establishing cut scores. Cut scores were established using the Minnesota statewide achievement test in reading at grade 3 as the criterion for a successful outcome. Participants were 2,191 students from five dis tricts in rural and outer-ring suburban east cen tral Minnesota, each of whom completed the statewide achievement test in reading and also CBM-R probes between grade 1 and grade 3. This study examined the relationship between CBM-R and the state test and compared the util ity and accuracy of each of four statistical meth ods for establishing cut scores. Receiver Operating Characteristic (ROC) curve analysis provided the most flexibility in establishing desired levels of diagnostic accuracy (i.e., sensi tivity, specificity, positive predictive power, and negative predictive power). The benefits of a set of consistent cut scores for use within a response to intervention (RTI) framework are discussed.
Get full access to this article
View all access options for this article.
References
1.
Amrein, A., & Berliner, D. (2002). High-stakes testing, uncertainty, and student learning. EducationalPolicy Analysis Archives, 10(18). Retrieved May 13, 2003, from http://epaa.asu.edu/epaa/v10n18.
2.
Carnoy, M., & Loeb, S. (2003). Does external accountability affect student outcomes ? A cross-state analysis. Educational Evaluation and Policy Analysis, 24, 305-331.
3.
Crawford, L., Tindal, G., & Stieber, S. (2001). Using oral reading rate to predict student performance on statewide achievement tests. Educational Assessment, 7, 303-323.
4.
Crocker, L.M., & Algina, J. (1986). Introduction to classical and modern test theory . Orlando, FL: Harcourt Brace Jovanovich .
5.
Deno, S.L. (1985). Curriculum-based measurement: The emerging alternative . Exceptional Children, 52, 219-32.
6.
Deno, S.L., Fuchs, L.S., Marston, D., & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30(4), 507-524.
7.
Deno, S.L., & Mirkin, P.K. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.
8.
Fewster, S., & MacMillan, P.D. (2002). School-based evidence for the validity of curriculum-based measurement of reading and writing. Remedial and Special Education, 23(3), 149-156.
9.
Fuchs, D., Fuchs, L.S., & Compton, D.L. (2004). Identifying reading disabilities by responsiveness-to-instruction: Specifying measures and criteria. Learning Disability Quarterly , 27, 216-227.
10.
Fuchs, L.S. (2003). Assessing intervention responsiveness: Conceptual and technical issues. Learning Disabilities Research & Practice, 18(3), 172-186.
Fuchs, L.S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research & Practice, 13, 204-219.
13.
Gibbons, K.A., Howe, K.B., Phaneuf, R., & Silberglitt, B. (2003, February). Creating and implementing a general outcome measurement system from preschool to grade 3: One education district's experience. Presentation at the 11th annual Pacific Coast Research Conference, La Jolla, CA.
14.
Good, R.H., Gruba, J., & Kaminski, R.A. (2002). Best practices in using Dynamic Indicators of Basic Early Literacy Skills in an outcomes-driven model . In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 699-720). Bethesda, MD: National Association of School Psychologists.
15.
Good, R.H., & Kaminski, R.A. (2002). Dynamic Indicators of Basic Early Literacy Skills: Benchmark goals (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Available at http://dibels.uoregon.edu.
16.
Good, R.H., Simmons, D.C., & Kame'enui, E.J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.
17.
Hasbrouck, J.E., & Tindal, G. (1992). Curriculum-based oral reading fluency norms for students in grades 2 through 5. Teaching Exceptional Children , 24, 41-44.
18.
Hintze, J.M., Ryan, A.L., & Stoner, G. (2003). Concurrent validity and diagnostic accuracy of the Dynamic Indicators of Basic Early Literacy Skills and the Comprehensive Test of Phonological Processing. School Psychology Review , 32, 541-556.
19.
Howe, K.B., Scierka, B.J., Gibbons, K.A., & Silberglitt, B. (2003). A school-wide organization system for raising reading achievement using general outcome measures and evidence-based instruction: One education district's experience. Assessment for Effective Intervention, 28, 59-72.
20.
Howe, K.B., & Shinn, M.M. (2002). Standard reading assessment passages (RAPs) for use in General Outcome Measurement: A manual describing development and technical features. Eden Prairie, MN: Edformation .
21.
Howell, D.C. (1997). Statistical methods for psychology. Belmont, CA: Duxbury.
22.
Huberty, C.J. (1994). Applied discriminant analysis. New York: Wiley.
23.
Kame'enui, E.J. (1993). Diverse learners and the tyranny of time: Don't fix blame; Fix the leaky roof. The Reading Teacher, 46, 376-383.
24.
Klecka, W.R. (1980). Discriminant analysis. Beverly Hills, CA: Sage.
25.
Linn, R.L., & Haug, C. (2002). Stability of school-building accountability scores and gains. Educational Evaluation and Policy Analysis, 24, 29-36.
26.
Madelaine, A., & Wheldall, K. (2002). Further progress towards a standardised curriculum-based measure of reading: Calibrating a new passage reading test against the New South Wales Basic Skills Test. Educational Psychology, 22(4), 461-471.
27.
Marston, D. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 18-78). New York: Guilford Press.
28.
McGlinchey, M.T., & Hixson, M.D. (2004). Using curriculum-based measurement to predict performance on state assessments in reading. School Psychology Review, 33(2), 193-203.
29.
Minnesota Department of Children, Families, and Learning. (1998 through 2002). Test specifications: Minnesota Comprehensive Assessment-Reading (Grade 3). St. Paul, MN: Author.
30.
Minnesota Department of Children, Families, and Learning & NCS Pearson. (2001 & 2002). Minnesota Comprehensive Assessments: Technical Manual. St. Paul, MN: Author.
31.
National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel, Teaching children to read, An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction-repost of the subgroups. D. N. Langenberg, Chair, National Reading Panel.
32.
Neter, J., Kutner, M.H., Nachtsheim, C.J., & Wasserman, W. (1996). Applied linear statistical models. Boston: McGraw-Hill.
33.
No Child Left Behind of 2001. Pub. L. No. 107-110.
34.
Office of Educational Accountability. (1999 ). Interpreting the Minnesota Comprehensivc Assessment Levels: A link to national percentile ranks. OEA Briefcase, 1(1). Retrieved on May 19, 2000 from http://edpsy.coled.umn.edu/oea.
35.
Pommerich, M., Hanson, B.A., & Harris, D.J. (2004). Issues in conducting linkages between distinct tests. Applied Psychological Measurement, 28(4), 247-273.
36.
Press, S.J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association , 73, 699-705.
37.
Rothman, K.J. (2002). Epidemiology: An introduction. New York: Oxford University Press.
38.
Shinn, M.R. (1988). Development of curriculum-based local norms for use in special education decision-making. School Psychology Review, 17(1), 61-80.
39.
Shinn, M.R. (1989). Identifying and defining academic problems: CBM screening and eligibility procedures. In M.R. Shinn (Ed.), Curriculum-based measurement: Assessing special children. New York: Guilford .
40.
Shinn, M.R. (2002). Best practices in curriculum-based measurement . In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 671-698). Bethesda, MD: National Association of School Psychologists.
41.
Silberglitt, B., Scierka, B.J., Howe, K.B., & Gibbons, K.A. (2003). Using General Outcome Measures in a reading initiative: Increasing reading achievement while decreasing LD incidence. Communiqué, 31 (8), 30-34.
42.
Speece, D.L., & Case, L. (2001). Classification in context: An alternative to identifying early reading disability. Journal of Educational Psychology , 93, 735-749.
43.
SPSS, Inc. (1999). SPSS (Version 10.0) [Computer software]. Chicago: Author.
44.
Stage, S.A., & Jacobsen, M.D. (2001). Predicting student success on a state-mandated performance-based assessment using oral reading fluency . School Psychology Review, 30, 407-419.
45.
Swets, J.A. (1996). Signal detection theory and ROC analysis in psychology and diagnostics. Mahwah, NJ: LEA .
46.
Swets, J.A., Dawes, R.M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions . Psychological Science in the Public Interest, 1, 1-26.
