In spite of an abundance of data, the empirical evidence as yet does not make clear whether meta-analysis will bring about progress in psychological science. Therefore, it is still useful and desirable to engage in rational analysis of the methodology. Such analysis is done in the present essay by posing five questions that go to the logical and conceptual foundation of meta-analysis. The questions are (a) What are the grounds for believing that the review of the literature, even a quantitative one, will bring about scientific discovery? (b) Why is the individual study devalued when the history of successful science seems largely the story of the success of the individual study? (c) What is the rationale for believing that data analysis by itself can markedly improve the fortunes of psychological science? (d) Is there a basis for claims made on behalf of meta-analysis that it is more accurate than either the traditional literature review or the individual study? (e) Is there justification for the claim that de facto meta-analysis has been used effectively in physical science?
Get full access to this article
View all access options for this article.
References
1.
BeggC. B. (1994) Publication bias. In CooperH.HedgesL. V. (Eds.), The handbook of research synthesis. New York: Russell Sage Found. Pp. 399–409.
2.
BerginA. E. (1971) The evaluation of therapeutic outcomes. In BerginA. E.GarfieldS. L. (Eds.), Handbook of psychotherapy and behavior change. New YorkWiley Pp. 217–270.
3.
BerginA. E.GarfieldS. L. (1994) Overview, trends, and future issues. In BerginA. E.GarfieldS. L. (Eds.), Handbook of psychotherapy and behavior change. (4th ed.) New York: Wiley. Pp. 821–830.
4.
CohenJ. (1990) Things I have learned (so far). American Psychologist, 45, 1304–1312.
5.
CohenJ. (1994) The earth is round (p < .05). American Psychologist, 49, 997–1003.
6.
CookA. H. (1990) Gravitation. In The Encyclopaedia Britannica. (15th ed.) Vol. 20. Chicago, IL: Encyclopaedia Britannica. Pp. 169–177.
7.
CooperH. M. (1989) Integrating research: a guide for literature reviews. (2nd ed.) Newbury Park, CA: Sage.
8.
CooperH. M.HedgesL. V. (1994) Potentials and limitations of research synthesis. In CooperH. M.HedgesL. V. (Eds.), The handbook of research synthesis. New York: Russell Sage Found. Pp. 521–529.
9.
CooperH. M.RosenthalR. (1980) Statistical versus traditional procedures for summarizing research findings. Psychological Bulletin, 87, 442–449.
10.
EysenckH. J. (1995) Meta-analysis squared—does it make sense?American Psychologist, 50, 110.
11.
GlassG. V. (1976) Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.
12.
HallJ. A.RosenthalR.Tickle-DegnenMostellerF. (1994) Hypotheses and problems in research synthesis. In CooperH.HedgesL. V. (Eds.), The handbook of research synthesis. New York: Russell Sage Found. Pp. 17–28.
13.
HedgesL. V. (1987) How hard is hard science, how soft is soft science? The empirical cumulativeness of research. American Psychologist, 42, 443–455.
14.
HedgesL. V. (1992) Meta-analysis, Journal of Educational Statistics, 17, 279–286.
HunterJ. E.SchmidtF. L. (1990) Methods of meta-analysis: correcting error and bias in research findings. Newbury Park, CA: Sage.
17.
JohnsonB. T.MullenB.SalasE. (1995) Comparison of three major meta-analytic approaches. Journal of Applied Psychology, 80, 94–106.
18.
JuddC. M.McClellandG. H.CulhaneS. E. (1995) Data analysis: continuing issues in the everyday analysis of psychological data. Annual Review of Psychology, 46, 433–465.
19.
LambertM. J.BerginA. E. (1994) The effectiveness of psychotherapy. In BerginA. E.GarfieldS. L. (Eds.), Handbook of psychotherapy and behavior change. (4th ed.) New York: Wiley. Pp. 143–189.
20.
LipseyM. W.WilsonD. B. (1993) The efficacy of psychological, educational, and behavioral treatment. American Psychologist, 48, 1181–1209.
21.
OakesM. (1986) Statistical inference, a commentary for the social and behavioural sciences. Chichester, UK: Wiley.
22.
OlkinI. (1990) History and goals. In WachterK. W.StrafM. (Eds.), The future of meta-analysis. New York: Sage. Pp. 3–10.
23.
PavlovI. P. (1927) Conditioned reflexes. London: Oxford Univer. Press.
24.
PocockS. J. (1983) Clinical trials: a practical approach. Chichester, UK: Wiley.
25.
RobinsonA. (1990) The principles of genetics and heredity. In The Encyclopaedia Britannica. (15th ed.) Vol. 19. Chicago, IL: Encyclopaedia Britannica. Pp. 700–742.
26.
RosenthalR. (1991) Meta-analytic procedures for social research. (Rev. ed.) Newbury Park, CA: Sage.
27.
RosenthalR. (1994) Science and ethics in conducting, analyzing, and reporting psychological research. Psychological Science, 5, 127–134.
28.
SchmidtF. L. (1992) What do data really mean?: research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173–1181.
29.
SchmidtF. L. (1996) Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods, 1, 115–129.
30.
SmithM. L.GlassG. V.MillerT. I. (1980) The benefits of psychotherapy. Baltimore, MD: Johns Hopkins Univer. Press.
31.
SohnD. (1995) Meta-analysis as a means of scientific discovery. American Psychologist, 50, 108–110.
32.
SohnD. (1996a) Meta-analysis and science. Theory & Psychology, 6, 229–246.
33.
SohnD. (1996b) Publication bias and the evaluation of psychotherapy efficacy in reviews of the research literature. Clinical Psychology Review, 16, 147–156.
34.
ThwaitesJ. G. (1958) Modern medical discoveries. New York: Dutton.
35.
WortmanP. M. (1992) Lessons from the meta-analysis of quasi-experiments. In BryantF. B.EdwardsJ.TindaleR. S.PosavacE. J.HeathL.HendersonE.Suarez-BalcazarY. (Eds.), Methodological issues in applied social psychology. New York: Plenum. Pp. 65–81
