Evaluation of special education effectiveness has stirred considerable controversy about the quality of services delivered to mildly handicapped students. However, the reliability of these evaluations has been questioned due to numerous methodological problems. Foremost among the concerns is the choice of metric to represent academic growth. The present study examines the hypothesis that different evaluative interpretations may be a function of the manner in which data are summarized and reported. Using a norm-referenced evaluation design, multiple metrics of reading and spelling achievement were obtained from mildly handicapped students in grades 1 through 6. Four metrics were compared, including raw score (level of correct performance), grade-equivalent score, z-score, and discrepancy index. Different interpretations may be warranted according to the metric employed. Criteria for selecting metrics for evaluating programs are considered with respect to statistical adequacy and the objective of special education programs.
Get full access to this article
View all access options for this article.
References
1.
Bacher, J.H. (1965). The effect of special class placement on the self-concept, social adjustment, and reading growth of slow learners (Doctoral dissertation, New York University, 1964). Dissertation Abstracts, 25,7071.
2.
Baker, K., & Pelavin, S. (undated). Implications for evaluation models from bilingual evaluations . (Unpublished manuscript). Washington, DC: U.S. Department of Education.
3.
Becker, W., & Engelmann, S. (1976). Teaching 3: Evaluation of instruction. Chicago: Science Research Associates.
4.
Bennett, R. (1982). The use of grade and age equivalent scores in educational assessment. Diagnostique, 7(3), 41-52.
5.
Berquist, C.G. (1982). A methodology for validating placement of children in exceptional child programs. Exceptional Children, 49(3), 269-270.
6.
Carlberg, C., & Kavale, K. (1980). The efficacy of special versus regular class placement for exceptional children: A meta-analysis. The Journal of Special Education, 14(3), 295-309.
7.
Cotugno, A.J. (1982). Cognitive control methods in the treatment of special education children. Psychology in the Schools, 19, 517-525.
8.
Deno, S.L. (1985). Curriculum-based measurement: The emerging alternative . Exceptional Children, 52, 219-232.
9.
Deno, S., & Mirkin, P. (1977). Data-based program modification: A manual. Minneapolis: University of Minnesota Leadership Training Institute.
10.
Deno, S., Mirkin, P., & Chiang, B. (1982). Identifying valid measures of reading. Exceptional Children, 49(1), 36-45.
11.
Deno, S., Mirkin, P., Lowry, L., & Kuehnle, K. (1980). Relationships among simple measures of spelling and performance on standardized achievement tests (Research Report No. 21). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities .
12.
Fuchs, L., Deno, S., & Marston, D. (1982). Use of aggregation to improve the reliability of simple direct measures (Research Report No. 90). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
13.
Fuchs, L., Fuchs, D., & Deno, S.L. (1985). Importance of goal ambitiousness and goal mastery to student achievement. Exceptional Children, 52(1), 63-71.
14.
Germann, G., & Tindal, G. (1985). Applications of direct and repeated measurement using curriculum-based assessment. Exceptional Children, 51(2), 110-121.
15.
Glass, G. (1978). Standards and criteria. Journal of Educational Measurement, 15(4), 237-261.
16.
Glass, G. (1980). When educators set standards. In E. L. Baker & E. S. Quellmalz (Eds.), Educational testing and evaluation (pp. 185-193). Beverly Hills, CA: Sage.
Idol-Maestas, L., Lloyd, S., & Ritter, S. (1982). A model for direct data based reading instruction . Champaign, IL: Department of Special Education. (ERIC Document Reproduction Service No. ED 219-738)
19.
Kelley, T.L., Madden, R., Gardner, E.R., & Rudman, H.C. (1964). Stanford achievement test. New York: Harcourt Brace Jovanovich.
20.
Lakin, K.C. (1983). A response to Gene V. Glass. Policy Studies Review, 2, 233-240.
21.
Lew, M., Mesch, D., & Lates, B.J. (1982). The Simmons College generic consulting teacher program. TEASE, 5(2), 11-16.
22.
Marston, D. (1986, July). The effectiveness of special education: A time-series evaluation approach. (Available from: Director of Special Education, Department of Education, State Department, 826 Capitol Square Building, 550 Cedar St., St. Paul, Minnesota .)
23.
Marston, D., & Deno, S. (1982). Implementation of direct and repeated measurement in the school setting (Research Report No. 106). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
24.
Salvia, J., & Ysseldyke, J. (1985). Assessment in special and remedial education (3rd edition). Boston: Houghton Mifflin.
25.
Shinn, M. (1986). Does anyone care what happens after the refer-test-place sequence: The systematic evaluation of special education effectiveness. School Psychology Review, 15, 49-58.
26.
Tindal, G. (1985). Investigating the effectiveness of special education: An analysis of methodology. Journal of Learning Disabilities , 18(2), 101-112.
27.
Tindal, G., Fuchs, L., Fuchs, D., Shinn, M., Deno, S., & Germann, G. (1985). Empirical validation of criterion referenced tests . Journal of Educational Research, 78(4), 203-209.
28.
Tindal, G., Germann, G., & Deno, S. (1983). Descriptive research on the Pine County norms: A compilation of findings (Research Report No. 132). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
29.
Tindal, G., Germann, G., Marston, D., & Deno, S. (1983). The effectiveness of special education: A direct measurement approach (Research Report No. 123). Minneapolis: University of Minnesota Institute for Research on Learning Disabilities.
30.
Tindal, G., Marston, D., & Deno, S. (1983). The reliability of direct and repeated measurement (Research Report No. 109). Minneapolis : University of Minnesota Institute for Research on Learning Disabilities.
31.
White, O., & Haring, N. (1980). Exceptional teaching. Columbus, OH: Charles E. Merrill.
