Four tests for the difference between pooled estimators of effect size from separate meta-analyses are discussed, and small sample performance compared via Monte Carlo simulation. In terms of Type I error and power, a z-test based on the difference between pooled estimators appears most accurate, while a confidence interval approach modified to permit testing between two mean effect sizes and Hedges and Olkin’s (1985) chi-square test with one degree of freedom appear roughly equivalent, and each of these is superior to a z-test based on an r-transformation. The modified confidence interval approach has particular promise for graphical examination of presumed moderators in meta-analyses.
Get full access to this article
View all access options for this article.
References
1.
Alliger, GM., Janak, E.A., Streeter, D., Byrne, D. & Turban, D. (1993). Psychological similarity effects in personnel selection decisions and work relations: A meta-analysis. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, San Francisco.
2.
Feingold, A. (1990). Gender differences in effects of physical attractiveness on romantic attraction: A comparison across five research paradigms. Journal of Personality and Social Psychology, 59: 981-993.
3.
Feingold, A. (1992). Good-looking people are not what we think. Psychological Bulletin, 111: 304-341.
4.
Hedges, L.V. (1982). Estimation and testing for differences in effect sire: A comment on Hsu. Psychological Bulletin, 91: 691-693.
5.
Hedges, L.V. & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando: Academic Press.
6.
Hsu, L.M. (1980). Tests of differences in p levels as tests of differences in effect sizes. Psychological Bulletin, 88: 605-708.
7.
Rosenthal, R. & Rubin, D.B. (1979). Comparing significance levels of independent studies. PsychologicalBulletin, 86: 1165-1168.
8.
Wolf, F.M. (1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage.
