This study examined three aspects of technical adequacy with respect to two general outcome measures in mathematics. The first two aspects were the effects of aggregating scores and correcting for random guessing on the reliability and validity of the measures. The third issue was the extent to which the measures were sensitive to changes in student performance. One hundred and five sixth-grade students completed two general outcome measures (basic facts and estimation) weekly for a 10-week period. With respect to aggregation, the results indicated that while fluency with basic facts is measured quite precisely with a single score, aggregation of scores for the estimation task increases both reliability and validity. Correcting the estimation scores for random guessing is recommended to increase criterion validity. Analyses of sensitivity to growth revealed that although both the fact and estimation measures reflect statistically significant changes across the 10-week period, the fact measure is likely to have greater practical significance. The article closes with a discussion of the relative merits of the measures for special education practice.
Get full access to this article
View all access options for this article.
References
1.
Baker, D.P. (1997). Surviving TIMSS: Or, everything you blissfully forgot about international comparisons. Phi Delta Kappan,79, 295-300.
2.
Ben-Shakhar, G., & Sinai, Y. (1991). Gender differences in multiple-choice tests: The role of differential guessing tendencies. Journal of Educational Measurement, 28, 23-35.
3.
Budescu, D., & Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement, 30, 277-291.
4.
Deno, S.L. (1985). Curriculum-based measurement: The emerging alternative . Exceptional Children, 52, 219-232.
5.
Deno, S.L., & Fuchs, L.S. (1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus onExceptional Children, 19(8), 1-16.
6.
Epstein, S. (1979). The stability of behavior: I. On predicting most of the people much of the time. Journal of Personality and Social Psychology, 37, 1097-1126.
7.
Epstein, S. (1980). The stability of behavior. II. Implications for psychological research. American Psychologist, 35, 790-806.
8.
Espin, C.A. (1998, February). Use of general outcome measures in reading and written expression for decision making. Paper presented at the 6th Annual Pacific Coast Research Conference, LaJolla, CA.
9.
Espin, C.A., & Foegen, A. (1996). Validity of GOMs for predicting secondary students' performance on content-area tasks. Exceptional Children, 62, 497-514.
10.
Espin, C.A., & Tindal, G. (1998). Curriculum-based measurement for secondary students . In M. R. Shinn (Ed.), Advanced applications of curriculum-based measurement (pp. 214-253). New York: Guilford.
11.
Foegen, A., & Deno. S.L. (in press). Identifying growth indicators for low-achieving students in middle school mathematics. The Journal of Special Education .
12.
Frary, R.B. (1982). A simulation study of reliability and validity of multiple-choice test scores under six response-scoring modes. Journal of Educational Statistics, 7,333-351.
13.
Frary, R.B., Cross, L.H., & Sewell, E.H. (1985). Partial information and the "correction" for guessing . Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago. (ERIC Document Reproduction Service No. ED 255 575)
Fuchs, L.S., Deno, S.L., & Marston, D. (1983). Improving the reliability of curriculum-based measures of academic skills for psychoeducational decision making. Diagnostique, 8, 135-149.
16.
Fuchs, L.S., & Fuchs, D. (1992). Identifying a measure for monitoring student reading progress. School Psychology Review, 21, 45-58.
17.
Fuchs, L.S., Fuchs, D., & Hamlett, C.L. (1994). Strengthening the connection between assessment and instructional planning with expert systems. Exceptional Children, 61, 138-146.
18.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., & Allinder, R.M. (1991). The contributions of skills analysis to curriculum-based measurement in spelling. Exceptional Children, 57, 443-452.
19.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., & Ferguson, C. (1992). Effects of expert system consultation within curriculum-based measurement using a reading maze task. Exceptional Children , 58, 436-450.
20.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., & Stecker, P.M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Educational Research Journal, 28, 617-641.
21.
Fuchs, L.S., Fuchs, D., Hamlett, C.L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect?School Psychology Review, 22, 27-48.
Kazdin, A.E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.
24.
Kurz, T.B. (1999). A review of scoring algorithms for multiple-choice tests. Paper presented at the annual meeting of the Southwest Education Research Association, San Antonio, TX. (ERIC Document Reproduction No. ED 428 076)
25.
Lloyd, J.W., Fomess, S.R., & Kavale, K.A. (1998). Some methods are more effective than others. Intervention in School and Clinic, 33, 195-200.
26.
Lord, F.M. (1963). Formula scoring and validity. Educational and Psychological Measurement, 23, 663-672.
27.
Lord, F.M. (1975). Formula scoring and number-right scoring. Journal of Educational Measurement, 12, 7-11.
28.
iMarston, D.B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 18-78). New York: Guilford .
29.
Marston, D., Fuchs, L.S., & Deno, S.L. (1986). Measuring pupil progress: A comparison of standardized achievement tests and curriculum-related measures. Diagnostique , 11, 77-90.
30.
Mehrens, W.A., & Lehmann, I.J. (1991). Measurement and evaluation in education and psychology . Fort Worth, TX: Holt, Rinehart and Winston.
31.
Mostert, M.P. (1996). Reporting meta-analyses in learning disabilities . Learning Disabilities Research & Practice, 11, 2-14.
32.
National Council of Teachers of Mathematics. ( 1989). Curriculum and evaluation standards for school mathematics . Reston, VA: Author.
33.
Reys, R.E. (1986). Evaluating computational estimation. Estimation and mental computation, 1986 yearbook of the National Council of Teachers of Mathematics (pp. 225-238). Reston, VA: NCTM .
34.
Schoen, H.L., Blume, G., & Hoover, H.D. (1990). Outcomes and processes on estimation test items in different formats. Journal for Research in Mathematics Education, 21, 61-73.
35.
Shinn, M. R. (Ed.). (1998). Advanced applications of curriculum-based measurement. New York: Guilford.
36.
Sowder, J. (1992). Estimation and number sense. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 371-389). New York: Macmillan.
37.
SPSS [Computer software]. (1995). Chicago : SPSS.
38.
Taylor, R.L. (2000). Assessment of exceptional students: Educational and psychological procedures (5th ed.). Boston: Allyn and Bacon.
39.
Tindal, G., & Parker, R. (1989). Development of written retell as a curriculum-based measure in secondary programs. School Psychology Review, 18, 328-343.
40.
Wiseman, D.C. (1999). Research strategies for education. Belmont, CA: Wadsworth.
41.
Yell, M.L., & Shriner, J.G. (1997). The IDEA amendments of 1997: Implications for special and general education teachers, administrators, and teacher trainers . Focus on Exceptional Children, 30 (1), 1-19.
