As meta-analytic methods are being applied to an ever wider variety of research domains, it is becoming apparent that researchers need more guidance than is currently available to produce replicable and consistent sets of findings. The present paper suggests some general guidelines to govern four critical steps in the meta-analytic process These are (1) selection and coding of studies, (2) correction of coefficients for artifacts, (3) interpretation of results, and (4) resolution of conflicting meta-analyses.
Get full access to this article
View all access options for this article.
References
1.
Bangert-DrownsR. L.Review of developments in meta-analytic method. Psychological Bulletin, 1986, 99, 388–399.
2.
CooperH. M.The integrative research review: A systematic approach. Beverly Hills, CA: Sage, 1984.
3.
EysenckH. J.An exercise in mega-silliness. American Psychologist, 1978, 73, 517.
4.
FisherC. D.GitelsonR.A meta-analysis of the correlates of role conflict and ambiguity. Journal of Applied Psychology, 1983, 68, 320–333.
5.
GlassG. V.McGawB.SmithM. L.Meta-analysis in social research. Beverly Hills, CA: Sage, 1981.
6.
GuzzoR.JacksonS. E.KatzellR. A.Meta-analysis analysis. In CummingsL. L.StawB. M. (Eds.), Research in organizational behavior. Vol. 9. Greenwich, CT: JAI Press, 1986. Pp. 407–422.
7.
HackettR.GuionR. M.A re-evaluation of the absenteeism-job satisfaction relationship. Organizational Behavior and Human Decision Processes, 1985, 35, 340–381.
8.
HirshH. R.NorthropL. C.SchmidtF. L.Validity generalization results for law enforcement occupations. Personnel Psychology, 1986, 39, 389–420.
9.
HirshH. R.SchmidtF. L.HunterJ. E.Estimation of employment test validities by less experienced judges. Personnel Psychology, 1986, 39, 337–344.
10.
HunterJ. E.Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery. Washington, DC: USES, Department of Labor, 1980.
11.
HunterJ. E.HirshH. R.Applications of meta-analysis. In CooperC.RobertsonI. (Eds.), International review of industrial and organizational psychology. London: Wiley, 1986. Pp. 321–357.
12.
HunterJ. E.SchmidtF. L.JacksonG. B.Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage, 1982.
13.
HydeJ. S.LinnM. C.Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 1988, 104, 53–69.
14.
IaffaldanoM.MuchinskyP.Job satisfaction and job performance: A meta-analysis. Psychological Bulletin, 1984, 97, 231–243.
15.
JacksonS. E.SchulerR.A meta-analysis and conceptual critique on role ambiguity and role conflict in work settings. Organizational Behavior and Human Decision Processes, 1985, 38, 16–78.
16.
JamesL. R.DemareeR. G.MulaikS. A.A note on validity generalization procedures. Journal of Applied Psychology, 1986, 71, 440–450.
17.
LightR. J.PillemerD. B.Summing up: The science of reviewing research. Cambridge, MA: Harvard Univer. Press, 1984.
18.
LordR. G.DeVaderC. L.AlligerG. M.A meta-analysis of the relation between personality traits and leadership perceptions: Application of validity generalization procedures. Journal of Applied Psychology, 1986, 71, 402–410.
19.
McDanielM. A.Job experience as a determinant of job performance. Paper presented at the 95th Annual Convention of the American Psychological Association, New York, August, 1987.
20.
McDanielM. A.HirshH. R.SchmidtF. L.RajuN. B.HunterJ. E.Interpreting the results of meta-analytic research: A comment on Schmitt, Gooding, Noe, and Kirsch (1984). Personnel Psychology, 1986, 38, 141–148.
21.
McDanielM. A.WhetzelD. L.SchmidtF. L.HunterJ. E.MaurerS.RussellJ.The validity of employment interviews: A review and meta-analysis. (Unpublished manuscript, 1987).
22.
PearlmanK.SchmidtF. L.HunterJ. E.Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology, 1980, 65, 373–406.
23.
PettyM. M.McGeeG. W.CavenderJ. W.A meta-analysis of the relationship between individual job satisfaction and individual performance. Academy of Management Review, 1984, 8, 712–721.
24.
RasmussenJ. L.LoherB. T.Appropriate critical percentages for the Schmidt and Hunter meta-analysis procedure: Comparative evaluation of Type I error rate and power. Journal of Applied Psychology, 1988, 73, 683–687.
25.
RodgersR. C.HunterJ. E.The impact of management by objectives on organizational productivity. (Unpublished manuscript, 1986).
26.
RosenthalR. R.The “file drawer problem” and tolerance for null results. Psychological Bulletin, 1979, 85, 185–193.
27.
RosenthalR. R.Meta-analytic procedures for social research. Beverly Hills, CA: Sage, 1984.
28.
SackettP. R.HarrisM. M.OrrJ. M.On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. Journal of Applied Psychology, 1986, 71, 302–330.
29.
SchmittN.GoodingR. Z.NoeR. A.KirschM.Meta-analyses of validity studies published between 1964 and 1982 and the investigation of study characteristics. Personnel Psychology, 1984, 37, 407–422.
30.
SchmidtF. L.HunterJ. E.Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 1977, 62, 529–540.
31.
SchmidtF. L.HunterJ. E.CrollP.McKenzieR.Estimation of employment test validities by expert judgment. Journal of Applied Psychology, 1983, 68, 580–601.
32.
SchmidtF. L.HunterJ. E.PearlmanK.HirshH. R.Forty questions about validity generalization and meta-analysis. Personnel Psychology, 38, 697–798.
33.
ShapiroD. A.ShapiroD.Meta-analysis of comparative psychotherapy outcomes studies: A replication and refinement. Psychological Bulletin, 1982, 92, 581–604.
34.
SmithM. L.Sex bias in counseling and psychotherapy. Psychological Bulletin, 1980, 87, 292–307.
35.
SmithM. L.GlassG. V.Meta-analyses of psychotherapy outcome studies. American Psychologist, 1977, 32, 752–760.
