Sage Journals: Discover world-class research

Abstract

Many Monte Carlo studies have evaluated the Schmidt and Hunter (S&H) (1977) meta-analytic procedures for combining independent correlation coefficients and their 75% rule for testing the homogeneity of variance. This study is different in using simulated data that are more typical of those data sets included in meta-analyses (Cormwell, 1989) such that sample size, effect size, degree of range restriction, measurement error, and number of correlations were simultaneously manipulated in evaluating the S&H procedures' reliability, Type I and II error, bias, and standard error. Results supported other studies which revealed that estimates of mean rho tend to be accurate even in the presence of small sample sizes, few correlations, and substantial measurement error. The power to detect variance among the rhos, however, is low in most situations; thus, the S&H procedures do not find variance among rhos when it exists (Type II error) and also have a high Type I error rate under all conditions simulated. These results are summarized in terms of the reliability of the S&H procedure in estimating mean rho and the variance of rhos.

Get full access to this article

View all access options for this article.

References

Bobko, P. (1983). An analysis of correlations corrected for attenuation and range restriction. Journal of Applied Psychology, 68, 584-589.

Burke, M. J. (1984). Validity generalization: A review and critique of the correlation model. Personnel Psychology, 37, 93-115.

Cornwell, J. M. (1989). Content analysis of meta-analytic studies from I/O psychology. New Orleans, LA: Tulane University. (ERIC Document Reproduction Service No. TM 012 844).

Green, B. F., Jr., (Chair). (1986, August). Panel discussion: Meta-analysis: The first ten years and beyond. Symposium conducted at the 94th annual convention of the American Psychological Association, Washington, D.C.

Hunter, J. E. , Schmidt, F. L. , and Jackson, G. B. (1982). Meta-Analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage Publications.

Jackson, S. E. (1984, August). Can meta-analysis be used for theory development in organizational psychology? Paper presented at the annual meeting of the American Psychological Association, Toronto, Canada.

James, L. R. , Demaree, R. G. , and Mulaik, S. A. (1986). A note on Validity Generalization procedures. Journal of Applied Psychology, 71, 440-450.

Kemery, E. R. , Mossholder, K. W. , and Roth, L. (1987). The power of the Schmidt and Hunter additive model of validity generalization. Journal of Applied Psychology, 72, 30-37.

Marascuilo, L. A. (1971). Statistical methods for behavioral science research. New York: McGraw-Hill.

10.

Osburn, H. G. , Callender, J. C. , Greener, J. M. , and Ashworth, S. (1983). Statistical power of tests of the situational specificity hypothesis in validity generalization studies: A cautionary note. Journal of Applied Psychology, 68, 115-122.

11.

Sackett, P. R. , Harris, M. M. , and Orr, J. M. (1986). On seeking moderator variables in the Meta-Analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. Journal of Applied Psychology, 71, 302-310.

12.

Sackett, P. R. , Tenopyr, M. L. , Schmitt, N. , and Kehoe, J. (1985). Commentary on forty questions about validity generalization and Meta-Analysis. Personnel Psychology, 38, 697-798.

13.

SAS User's Guide, Version 5 Edition. (1985). Cary, NC: SAS Institute, Inc.

14.

Schmidt, F. L. (1984, August). Meta-analysis: Implications for cumulative knowledge in the behavioral and social sciences. Paper presented at the annual meeting of the American Psychological Association, Toronto, Canada.

15.

Schmidt, F. L. and Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.

16.

Schmidt, F. L. , Pearlman, K. , Hunter, J. E. , and Hirsch, H. R. (1985). Forty questions about validity generalization and Meta-analysis. Personnel Psychology, 38, 697-798.

17.

Spector, P. E. and Levine, E. L. (1987). Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors. Journal of Applied Psychology, 72, 3-9.

Power and Accuracy of the Schmidt and Hunter Meta-Analytic Procedures

Abstract

Get full access to this article

References