It has been 20 years since Gene Glass reinvigorated research integration through the formalization of meta-analysis. This article explores the nature of meta-analysis by placing it within the context of research synthesis. The methods of meta-analysis are described briefly and compared and contrasted with other forms of research integration. In addition, the findings from several meta-analyses are provided to show the advantages of quantitative review methods. Finally, criticisms of meta-analyses are analyzed; as are several unanswered questions about the direction of meta-analysis.
Get full access to this article
View all access options for this article.
References
1.
Abrami, P. C.
, Cohen, P. A., & d'Apollonia, S. (1988). Implementation prblems in meta-analysis. Review of Educational Research, 58, 151-179.
2.
Bakan, D.
(1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
3.
Bangert-Drowns, R. L.
(1986). Review of developments in meta-analytic methods. Psychological Bulletin, 99, 388-399.
4.
Bolles, R. C.
(1962). The differences between statistical hypotheses and scientific hypotheses. Psychological Reports, 11, 639-645.
5.
Bryant, F. B.
, & Wortman, P. M. (1984). Methodological issues in the meta-analysis of quasi-experiments. InW. H. Yeaton & P. M. Wortman (Eds.), Issues in data synthesis (pp. 5-24). San Francisco: Jossey-Bass.
6.
Carbo, M.
(1983). Research in reading and learning style: Implications for exceptional children. Exceptional Children, 49, 486-494.
7.
Carlberg, C. G.
, Johnson, D. W., Johnson, R., Maruyama, G., Kavale, K. A., Kulik, C-L. C., Kulik, J. A., Lysakowski, R. S., Pflaum, S. W., & Walberg, H. J. (1984). Meta-analysis in education: A reply to Slavin. Educational Researcher, 3, 16-23.
8.
Carlberg, C. G.
, & Kavale, K. A. (1980). The efficacy of special versus regular class placement for exceptional children: A meta-analysis. Journal of Special Education, 14, 295-309.
9.
Center, B. A.
, Skiba, R. J., & Casey, A. (1985-1986). Amethodology forthequantitative synthesis of intra-subject design research. Journal of Special Education, 19, 387-400.
10.
Cochran, W. G.
(1937). Problems arising in the analysis of a series of similar experiments. Journal of the Royal Statistical Society, 4(Suppl. 4), 102-118.
11.
Cohen, J.
(1977). Statistical power analysis for the behavioral sciences (Rev. ed.). New York: Academic Press.
12.
Cook, T. D.
, & Leviton, L. C. (1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48, 449-472.
13.
Cooper, H.
, & Rosenthal, R. (1980). Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin, 87,442-449.
14.
Cooper, H. M.
(1984). The integrative research review. Beverly Hills, CA: Sage.
15.
Dunn, R. S.
, & Dunn, K. J. (1978). Teaching students through their individual learning style. Reston, VA: Reston.
16.
Eysenck, H. J.
(1978). An exercise in mega-silliness. American Psychologist, 33, 517-517.
17.
Fisher, R. A.
(1932). Statistical methodsfor research workers (4th ed.). London: Oliver & Boyd.
18.
Gentile, J. R.
, Roden, A. H., & Klein, R. D. (1974). An analysis of variance model for the intrasubject replication design. Journal of Applied BehaviorAnalysis, 5, 193-198.
19.
Ghiselli, E. E.
(1949). The validity of commonly employed occupational tests. University of California Publications in Psychology, 5, 253-288.
20.
Glass, G. V.
(1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5,3-8.
21.
Glass, G. V
, & Kliegl, R. M. (1983). An apology for research integration in the study of psychotherapy. Journal of Consulting and Clinical Psychology, 51, 28-41.
22.
Glass, G. V
, McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
23.
Green, B. F.
, & Hall, J. A. (1984). Quantitative methods for literature reviews. Annual Review of Psychology, 35, 37-53.
24.
Hallahan, D. P.
, & Cruickshank, W. M. (1973). Psychoeducational foundations of learning disabilities. Englewood Cliffs, NJ: Prentice-Hall.
25.
Hammill, D. D.
, & Larsen, S. C. (1974a). The effectiveness of psycholinguistic training. Exceptional Children, 41, 5-14.
26.
Hammill, D. D.
, & Larsen, S. C. (1974b). The relationship of selected auditory perceptual skills and reading ability. Journal of Learning Disabilities, 7,429-435.
27.
Hammill, D. D.
, & Larsen, S. C. (1978). The effectiveness of psycholinguistic training: A reaffirmation of position. Exceptional Children, 44, 402-414.
28.
Hedges, L. V.
(1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.
29.
Hedges, L. V.
, & Olkin, I. (1980). Vote counting methods in research synthesis. Psychological Bulletin, 88, 359-369.
30.
Hedges, L. V.
, & Olkin, I. (1982). Analyses, reanalyses, and meta-analysis. Contemporary Education Review, 1, 157-165.
31.
Hedges, L. V.
, & Olkin, I. (1985). Statistical methodsfor meta-analysis. Orlando, FL: Academic Press.
32.
Hempel, C. G.
(1955). Aspects of scientific explanation. New York: Free Press.
33.
Hogbon, L.
(1957). Statistical theory. New York: Norton.
34.
Hunter, J. E.
, Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
35.
Jackson, G. B.
(1980). Methods for integrative reviews. Review of Educational Research, 50, 438-460.
36.
Jones, L. V.
, & Fiske, D. W. (1953). Models for testing the significance of combined results. Psychological Bulletin, 50, 375-382.
37.
Kaplan, A.
(1964). The conduct of inquiry. San Francisco: Chandler.
38.
Kavale, K. A.
(1981a). Functions of the Illinois Test of Psycholinguistic Abilities (ITPA): Are they trainable?Exceptional Children, 47,496-510.
39.
Kavale, K. A.
(1981b). The relationship between auditory perceptual skills and reading ability: A meta-analysis. Journal of Learning Disabilities, 14, 539-546.
40.
Kavale, K. A.
(1982a). The efficacy of stimulant drug treatment for hyperactivity: Ameta-analysis. Journal of Learning Disabilities, 15, 280-289.
41.
Kavale, K. A.
(1982b). Meta-analysis of the relationship between visual perceptual skills and reading achievement. Journal of Learning Disabilities, 15, 42-51.
42.
Kavale, K. A.
(1985). Auditory perceptual skills andreading: Arejoinderto Gersten and Carnine about what Kavale did. Remedial and Special Education, 6, 43-45.
43.
Kavale, K. A.
, & Forness, S. R. (1983). Hyperactivity and diet treatment: Ameta-analysis of the Feingold hypothesis. Journal of Learning Disabilities, 16, 324-330.
44.
Kavale, K. A.
, & Fomess, S. R. (1987). Substance over style: Assessing the efficacy of modality testing and teaching. Exceptional Children, 54, 228-234.
45.
Kavale, K. A.
, & Mattson, P. D. (1983). "One jumped off the balance beam": Meta-analysis of perceptual-motor training. Journal of Learning Disabilities, 16, 165-173.
46.
Kavale, K. A.
, & Nye, C. (1984). The effectiveness of drug treatment for severe behavior disorders: A meta-analysis. Behavioral Disorders, 9, 117-130.
47.
Kazdin, A. E.
(1978). Methodological and interpretive problems of single-case experimental designs. Journal of Consulting and Clinical Psychology, 46, 629-642.
48.
Kraemer, H. C.
, & Andrews, G. (1982). Anonparametric technique for meta-analysis effect size calculation. Psychological Bulletin, 91, 404-412.
49.
Kuhn, T. S.
(1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press.
50.
Larsen, S. C.
, & Hammill, D. D. (1975). The relationship of selected visual perceptual abilities to school learning. Journal of Special Education, 9, 281-291.
51.
Light, R. J.
(1979). Capitalizing on variation: How conflicting research findings can be helpful for policy. Educational Researcher, 8, 7-11.
52.
Light, R. J.
, & Pillemer, D. B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
53.
Light, R. J.
, & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review, 41, 429-471.
54.
Lund, K. A.
, Foster, G. E., & McCall-Perez, F C. (1978). The effectiveness of psycholinguistic training: A reevaluation. Exceptional Children, 44, 310-319.
55.
Lykken, D. T.
(1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159.
56.
McCain, L. J.
, & McCleary, R. (1979). The statistical analysis of the simple interrupted time-series quasi-experiment. In T. D. Cook & D. T. Campbell (Eds.), Quasi-experimentation: Design and analysis issues forfield settings (pp. 233-294). Chicago: Rand McNally.
57.
McGaw, B.
, & Glass, G. V. (1980). Choice of the metric foreffectsize in meta-analysis. American Educational Research Journal, 17, 325-337.
58.
Minskoff, E.
(1975). Research on psycholinguistic training: Critique and guidelines. Exceptional Children, 42, 136-144.
59.
Mitroff, I. I.
, & Featheringham, T. R. (1974). On systematic problem solving and the error of the third kind. Behavioral Science, 19, 383-393.
60.
Mosteller, F. M.
, & Bush, R. R. (1954). Selected quantitative techniques. In G. Lindzey (Ed.), Handbook of social psychology: Vol 1. Theory and method. Cambridge, MA: Addison-Wesley.
61.
Nagel, E.
(1961). The structure of science. New York: Harcourt, Brace, & World.
62.
Newcomer, P.
, Larsen, S. C., & Hammill, D. D. (1975). A response. Exceptional Children, 42, 144-148.
63.
Orwin, R. G.
(1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157-159.
64.
Parsonson, B. S.
, & Baer, D. M. (1978). The analysis and presentation of graphic data. In T. R. Kratchowill (Ed.), Single subject research: Strategiesfor evaluating change (pp. 101-166). New York: Academic Press.
65.
Pearson, E. S.
(1938). The probability integral transformation for testing goodness of fit and combining tests of significance. Biometrika, 30, 134-148.
66.
Pearson, K.
(1933). On a method of determining whether a sample size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika, 25, 379-410.
67.
Pillemer, D. B.
, & Light, R. J. (1980). Benefitting from variation in study outcomes. New Directions for Methodology of Social and Behavioral Science, 5, 1-12.
68.
Rosenthal, R.
(1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641.
69.
Rosenthal, R.
(1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage.
70.
Salzberg, C. L.
, Strain, P. S., & Baer, D. M. (1987). Meta-analysis for single-subject research: When does it clarify, when does it obscure?Remedial and Special Education, 8, 43-48.
71.
Scheffler, I.
(1963). The anatomy of inquiry. New York: Knopf.
72.
Scruggs, T. E.
, Mastropieri, M. A., & Casto, G. (1987a). The quantitative synthesis of singlesubject research: Methodology and validation. Remedial and Special Education, 8, 24-33.
73.
Scruggs, T. E.
, Mastropieri, M. A., & Casto, G. (1987b). Reply to Owen White. Remedial and Special Education, 8, 40-42.
74.
Scruggs, T. E.
, Mastropieri, M. A., & Casto, G. (1987c). Response to Salzberg, Strain, and Baer. Remedial and Special Education, 8, 49-52.
75.
Sindelar, P. T.
, & Wilson, R. J. (1984). The potential effects of meta-analysis on special education practice. Journal of Special Education, 18, 81-92.
76.
Slavin, R. E.
(1984). Meta-analysis ineducation: How has it been used?Educational Researcher, 3,6-15.
77.
Slavin, R. E.
(1986). Best-evidence synthesis: An alternative to meta-analytic and traditional reviews. Educational Researcher, 15, 5-11.
78.
Slavin, R. E.
(1987). Best-evidence synthesis: Why less is more. Educational Researcher, 16, 15-16.
79.
Smith, M. L.
, & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
80.
Smith, M. L.
, Glass, G. V, & Miller, T. I. (1980). The benefits ofpsychotherapy. Baltimore: Johns Hopkins University Press.
81.
Stouffer, S. A.
, Suchman, E. A., DeVinney, L. C., Star, S. A., & Williams, R. M. (1949). The American soldier: Adjustment during army life, Vol. 1. Princeton, NJ: Princeton University Press.
82.
Tawney, J. W.
, & Gast, D. L. (1984). Single-subject research in special education. Columbus, OH: Merrill.
83.
Tippett, L.H.C.
(1931). The methods of statistics. London: Williams & Norgate.
84.
Toulmin, S.
(1961). Foresight and understanding. New York: Harper & Row.
85.
Underwood, B. J.
(1957). Interference and forgetting. Psychological Review, 64, 49-60.
86.
Wachter, K. W.
, & Straf, M. L. (1993). The future of meta-analysis. New York: Russell Sage.
87.
White, O. R.
(1987). Some comments concerning "the quantitative synthesis of single-subject research."Remedial and Special Education, 8, 34-39.
88.
Wolf, F. M.
(1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage.
89.
Wortman, P. M.
(1983). Evaluation research: A methodological perspective. Annual Review of Psychology, 34, 223-260.
90.
Yates, F.
, & Cochran, W. G. (1938). The analysis of groups of experiments. Journal of Agricultural Science, 28, 556-580.
