Effects of Systematic Formative Evaluation: A Meta-Analysis

Abstract

This meta-analysis investigated the effects of formative evaluation procedures on student achievement. The data source was 21 controlled studies, which generated 96 relevant effect sizes, with an average weighted effect size of .70. The magnitude of the effect of formative evaluation was associated with publication type, data-evaluation method, data display, and use of behavior modification. Implications for special education practice are discussed.

Get full access to this article

View all access options for this article.

References

American Psychological Association. (1982). Thesaurus of psychological index terms (3rd ed.). Washington, DC: Author.

Arter

J. A.

, & Jenkins

J. R.

(1977). Examining the benefits and prevalences of modality considerations in special education. Journal of Special Education, 11, 281–298.

Arter

J. A.

, & Jenkins

J. R.

(1979). Differential diagnosis-prescriptive teaching: A critical appraisal. Review of Educational Research, 49, 517–555.

Baldwin

(1976). Curriculum concerns. In Thomas

M. A.

(Ed.), Hey, don't forget about me. Reston, VA: Council for Exceptional Children.

Bangert-Drowns

R. L.

Kulik

J. A.

, & Kulik

C. C.

(1983). Effects of coaching programs on achievement test performance. Review of Educational Research, 53, 571–585.

Bloom

B. S.

(1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educ. Researcher, 13(6), 4–16.

Cohen

(1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.

Corrigan

D. C.

(1978). Public Law 94–142: A matter of human rights; A call for change in schools and colleges of education. In Grosenick

J. K.

& Reynolds

M. C.

(Eds.), Teacher education: Renegotiating roles for mainstreaming. Reston, VA: Council for Exceptional Children.

Council for Exceptional Children Delegate Assembly. (1983). Code of ethics and standards for professional practice. Exceptional Children, 50, 205–209.

10.

Deno

S. L.

, & Mirkin

P. K.

(1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.

11.

Fuchs

, & Fuchs

L. S.

(1986). Test procedure bias: A meta-analysis of examiner familiarity effects. Review of Educational Research, 56, 243–262.

12.

Fuchs

L. S.

Power

M. H.

, & Dailey

A. M.

(1985). Bias in the assessment of handicapped children. American Educational Research Journal, 22, 185–198.

13.

Fuchs

L. S.

Deno

S. L.

, & Mirkin

P. K.

(1983). Data-based program modification: A continuous evaluation system with computer technology to facilitate implementation. Journal of Special Education Technology, 6(2), 50–57.

14.

Fuchs

L. S.

Deno

S. L.

, & Mirkin

P. K.

(1984). The effects of frequent curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449–460.

15.

Glass

McGaw

, & Smith

M. L.

(1981). Metaanalysis in social research. Beverly Hills: Sage.

16.

Greenwald

A. G.

(1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20.

17.

Hammill

D. D.

, & Larsen

(1974). The effectiveness of psycholinguistic training. Exceptional Children, 41, 5–14.

18.

Hammill

D. D.

, & Wiederholt

J. L.

(1973). Review of the Frostig visual perception test and the related training program. In Mann

& Sabatino

D. A.

(Eds.), First review of special education (Vol. 1, pp. 33–48). Philadelphia: JSE Press.

19.

Hasselbring

T. S.

(1985). Toward the development of expert assessment systems. Working paper. (Available from T. Hasselbring, Box 328, Peabody College, Vanderbilt University, Nashville, TN 37203.)

20.

Hasselbring

T. S.

, & Hamlett

C. L.

(1983). AIMSTAR: A data management and decision-making system. Portland, OR: ASIEP Education Co.

21.

Hedges

(1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 359–361.

22.

Hedges

(1982). Fitting categorical models to effect sizes from a series of experiments. Journal of Educational Statistics, 7, 119–137.

23.

Hedges

(1984). Advances in statistical methods for meta-analysis. In Yeaton

W. H.

& Wortman

P. N.

(Eds.), Issues in data synthesis. New Directions for Program Evaluation, 24. San Francisco: Jossey-Bass.

24.

Kavale

(1981). Functions in the Illinois Test of Psycholinguistic Abilties (ITPA): Are they trainable? Exceptional Children, 47, 496–510.

25.

Linn

R. L.

(1983). Testing and instruction: Links and distinctions. Journal of Educational Measurement, 20, 179–191.

26.

Lloyd

J. W.

(1984). How shall we individualize instruction—Or should we? Remedial and Special Education, 5, 7–15.

27.

Lysakowski

R. S.

, & Walberg

H. J.

(1982). Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. American Educational Research Journal, 19, 559–578.

28.

Mann

P. H.

Suiter

P. A.

, & McClung

(1979). Handbook in diagnostic-prescriptive teaching (2nd ed.). Boston: Allyn & Bacon.

29.

Rosenthal

(1978). Combining results of independent studies. Psychological Bulletin, 85, 185–193.

30.

Rosenthal

(1979). The “file drawer problem” and tolerance for null effects. Psychological Bulletin, 86, 638–641.

31.

Rosenthal

, & Rubin

D. B.

(1978). Interpersonal expectancy effects: The first 345 studies. The Behavioral and Brain Sciences, 3, 377–415.

32.

Salvia

, & Ysseldyke

(1981). Assessment in special and remedial education (2nd ed.). Boston: Houghton-Mifflin.

33.

Schenck

S. J.

(1980). The diagnostic-instructional link in individualized education programs. Journal of Special Education, 14, 337–345.

34.

Snow

R. E.

, & Lohman

D. F.

(1984). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 76, 347–376.

35.

Spears

(1973). Kappans ponder typical school procedures. Phi Delta Kappan, 54, 615–618.

36.

Tarver

S. G.

, & Dawson

M. M.

(1978). Modality preference and the teaching of reading: A review. Journal of Learning Disabilities, 11, 5–17.

37.

Thompson

R. H.

White

K. R.

, & Morgan

D. P.

(1982). Teacher-student interaction patterns in classrooms with mainstreamed mildly handicapped students. American Educational Research Journal, 19, 220–236.

38.

Tindal

Fuchs

L. S.

Christenson

Mirkin

P. K.

, & Deno

S. L.

(1981). The relationship between student achievement and teacher assessment of short- or long-term goals (Research Report No. 61). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities. (ERIC Document Reproduction Service No. Ed 218 846).

39.

Turnbull

Strickland

B. B.

, & Brantley

J. C.

(1978). Developing and implementing individualized educational programs. Columbus: Merrill.

40.

Wang

, & Reynolds

M. C.

(1985). Avoiding the “Catch 22” in special education reform. Exceptional Children, 51, 497–502.

41.

Wesson

King

R. P.

, & Deno

S. L.

(1984). Direct and frequent measurement of student performance: If it's good for us, why don't we do it? Learning Disability Quarterly, 7, 45–48.

42.

West

T. L.

, & Bates

(1977). The implications of mainstreaming for teacher education. In Bates

West

T. L.

, & Schmerl

R. B.

(Eds.), Mainstreaming: Problems, potentials, and perspectives. Minneapolis: University of Minnesota.

43.

White

O. R.

(1974). Evaluating educational process (Working paper). Seattle: University of Washington, Child Development and Mental Retardation Center, Experimental Education Unit.

44.

White

O. R.

, & Haring

N. G.

(1980). Exceptional teaching (2nd ed.). Columbus, OH: Merrill.

45.

Ysseldyke

J. E.

(1979). Psychqeducational assessment and decision making. In Ysseldyke

J. E.

& Mirkin

P. K.

(Eds.), Proceedings of the Minnesota Roundtable Conference on Assessment of Learning Disabled Children (Monograph No. 8). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities. (ERIC Document Reproduction Service No. ED 185 765).

46.

Beck

(1976). Report for the office of education dissemination review panel. (Unpublished manuscript available at Precision Teaching Project, 3300 Third St. N.E., Great Falls, MT 59404.)

47.

Beck

(1979). Report for the office of education dissemination review panel. (Unpublished manuscript available at Precision Teaching Project, 3300 Third St. N.E., Great Falls, MT 59404.)

48.

Beck

(1981). Curriculum management through a data base. (Unpublished manuscript available at Precision Teaching Project, 3300 Third St. N.E., Great Falls, MT 59404.)

49.

Beck

(1981). High school basic skills improvement project. (Unpublished manuscript available at Precision Teaching Project, 3300 Third St. N.E., Great Falls, MT 59404.)

50.

Bohannon

R. M.

(1975). Direct and daily measurement procedures in the identification and treatment of reading behaviors in children in special education. Unpublished doctoral dissertation, University of Washington.

51.

Bradfield

R. H.

Brown

Kaplan

Rickert

, & Stannard

(1973). The special children in the regular classroom. Excptnl. Chldrn., 39, 384–391.

52.

Brandstetter

, & Merz

(1978). Charting scores in precision teaching for skill acquisition. Exceptional Children, 45, 42–48.

53.

Bruening

S. E.

(1978). Precision teaching in the high school classroom: A necessary step towards maximizing teaching effectiveness and student performance. American Educational Research Journal, 15, 125–140.

54.

Crutcher

C. E.

, & Hofmeister

A. M.

(1975). Effective use of objectives and monitoring. Teaching Exceptional Children, 7(3), 78–79.

55.

Dubrule

M. N.

(1984). The study of precision teaching as a remedial method. Unpublished doctoral dissertation, Clark University.

56.

Frumess

S. C.

(1983). A comparison of management groups involving the use of the standard behavior chart and setting performance aims. Unpublished doctoral dissertation, University of Houston.

57.

Fuchs

L. S.

Deno

S. L.

, & Mirkin

P. K.

(1984). The effects of frequent curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449–460.

58.

Haring

N. G.

(1971). Investigation of systematic instructional procedures to facilitate academic achievement in mentally retarded disadvantaged children. Final report. (ERIC Document Reproduction Service No. ED 071 246).

59.

Haring

N. G.

, & Krug

D. A.

(1975a). Evaluation of a program of systematic instructional procedures for extremely poor retarded children. American Journal of Mental Deficiency, 79, 627–631.

60.

Haring

N. G.

, & Krug

D. A.

(1975b). Placement in regular programs: Procedures and results. Exceptional Children, 41, 413–417.

61.

King

Deno

S. L.

Mirkin

P. K.

, & Wesson

(1983). The effects of training teachers in the use of formative evaluation in reading: An experimental-control comparison (Research Report No. 111). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities.

62.

Lovitt

T. C.

, & Fantasia

(1983). A precision teaching project with learning disabled children. Journal of Precision Teaching, 3, 85–91.

63.

Mirkin

P. K.

(1978). A comparison of the effects of three formative evaluation strategies and contingent consequences on reading performance. Unpublished doctoral dissertation, University of MN.

64.

Mirkin

P. K.

Deno

S. L.

Tindal

, & Kuehnle

(1980). Formative evaluation: Continued development of data utilization systems (Research Report No. 23). University of Minnesota, Institute for Research on Learning Disabilities. (ERIC Document Reproduction Service No. ED 197 510).

65.

Peniston

E. G.

(1975). An evaluation of the Portage Project: A comparison of a home-visit program for multiple handicapped preschoolers and Head-start program. (ERIC Document Reproduction Service No. ED 112 570).

66.

Sevcik

Skiba

Tindal

King

Wesson

Mirkin

P. K.

, & Deno

S. L.

(1983). Curriculum-based measurement: Effects on instruction, teaching estimates of student progress, and student performance (Research Report No. 124). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities.

67.

Skiba

Wesson

, & Deno

S. L.

(1982). The effects of training teachers in the use of formative evaluation in reading: An experimental-control comparison (Research Report No. 88). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities.

68.

Tindal

Fuchs

L. S.

Christenson

Mirkin

P. K.

, & Deno

S. L.

(1981). The relationship between achievement and teacher assessment of short- or long-term goals (Research Report No. 61). Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities. (ERIC Document Reproduction Service No. ED 218 846).