Sage Journals: Discover world-class research

Abstract

A comparison of the power and Type I error rates of three tests of homogeneity for independent product-moment correlation coefficients (X;, Q, and LRS) were evaluated using Monte Carlo methods. Conditions simulated include data having varying sample sizes, reliabilities, and range restriction, with central and noncentral conditions, and having normal and nonnormal distributions. The LRS test almost always had a Type I error rate higher than the nominal alpha. Correspondingly, it was always more powerful than the other two tests given the same condition. The X² and Q tests performed comparably regarding their Type I and Type II error rates. However, for all three tests power decreases substantially with the introduction of range restriction or measurement error when fewer than 18 correlations are being tested. Average Type II error rate for "perfect" data was less than .01 for both a = .05 and .01. The presence of range restriction and unreliability increased the average Type II error rate to approximately .56 for a = .05 and to .76 for a = .01 with no effect on Type I error rate. The Q test is recommended based on its performance in this paper and findings presented by Alexander, Scozzaro, and Borodkin (1989). A general computer program was used to generate these results which can be employed to estimate power for specific situations.

Get full access to this article

View all access options for this article.

References

Alexander, R. A. , Scozzaro, M. J. , and Borodkin, L. J. (1989). Statistical and empirical examination of the Chi-Square test for homogeneity of correlations in meta-analysis.Psychological Bulletin, 106, 329-331.

Bangert-Drowns, R. L. (1985, August). Meta-analytic techniques: Five different approaches to meta-analytic research. Paper presented at the annual meeting of the American Psychological Association, Los Angeles, California.

Bangert-Drowns, R. L. (1986). Review of developments in Meta-Analytic methods.Psychological Bulletin, 99, 388-399.

Burke, M. J. (1984). Validity generalization: A review and critique of the correlation model.Personnel Psychology, 37, 93-115.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press

Cornwell, J. M. (1987). An investigation of the statistical quality of meta-analytic estimators used in industrial and organizational psychology: Content analysis, derivation, and Monte Carlo analyses. Unpublished doctoral dissertation, University of Tennessee, Knoxville, TN.

Cornwell, J. M. (1989). Content analysis of Meta-Analytic studies from I/O Psychology. New Orleans, LA: Tulane University, Department of Psychology. (ERIC Document Reproduction Service No. TM 012 844).

Cornwell, J. M. and Ladd, R. T. (in press). Power and accuracy of the Schmidt and Hunter meta-analytic procedures. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT.

Fleishman, A. L. (1978). A method for simulating, non-normal distributions.Psychometrika, 43, 521-532.

10.

Ghiselli, E. E. , Campbell, J. P. , and Zedeck, S. (1981). Measurement theory for the behavioral sciences. New York: W. H. Freeman and Company.

11.

Glass, G. V. , McGaw, B. , and Smith, M. L. (1981). Meta-Analysis in social research. Beverly Hills, CA: Sage Publications.

12.

Hedges, L. V. and Olkin, I. (1985). Statistical methods for meta- analysis. Orlando, FL: Academic Press.

13.

Hunter, J. E. , Schmidt, F. L. , and Jackson, G. B. (1982). Meta- analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage Publications.

14.

Marascuilo, L. A. (1971). Statistical methods for behavioral science research. New York: McGraw-Hill.

15.

Matarazzo, J. D. (1972). Wechsler measurement and appraisal of adult intelligence (5th ed.). Baltimore: Williams and Wilkins Company.

16.

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

17.

Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage Publications.

18.

SAS Institute. (1985). SAS Version 5 [Computer program]. Cary, NC: Author.

19.

Spector, P. E. and Levine, E. L. (1987). Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors.Journal of Applied Psychology, 72, 3-9.

20.

Thomas, H. (1990). A likelihood-based model for validity generalization.Journal of Applied Psychology, 75, 13-20.

21.

Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage Publications.

Monte Carlo Comparisons of Three Tests for Homogeneity of Independent Correlations

Abstract

Get full access to this article

References