Meta-analysis is commonly used to quantitatively review research findings in the social sciences. This article looks at what happens next, after a meta-analysis is published. The authors examine how meta-analytic findings are cited in subsequent studies and whether the citing authors take full advantage of the information meta-analyses provide. A review of 1,489 citations to meta-analyses in 319 empirical studies published in three journals over two decades indicates that the frequency of citing meta-analyses is accelerating. An analysis of citing practices indicates that authors use data for a variety of purposes in subsequent research studies. However, the citing studies often underreported important aspects of meta-analytic data, and additional opportunities exist to build on the data provided by meta-analytic reviews.
AguinisH.DaltonD. A.BoscoF. A.PierceC. A.DaltonC. M. (2009, August). Meta-analytic choices and judgment calls: Implications for theory and scholarly impact. Paper presented at the annual meeting of the Academy of Management, Chicago, IL.
2.
AguinisH.SturmanM.PierceC. (2008). Comparison of three meta-analytic procedures for estimating moderating effects of categorical variables. Organizational Research Methods, 11, 9–34.
3.
American Psychological Association. (2009). Publication manual of the American Psychological Association6th ed.Washington, DC: Author.
4.
BarrickM. R.MountM. K. (1991). The big five personality dimensions and job performance. Personnel Psychology, 44, 1–26.
5.
BarrickM. R.MountM. K.StraussJ. P. (1993). Conscientiousness and performance of sales representatives: Test of the mediating effects of goal-setting. Journal of Applied Psychology, 78(5), 715–722.
6.
BirdiK.CleggC.PattersonM.RobinsonA.StrideC. B.WallT. D.WoodS. J. (2008). The impact of human resource and operational management practices on company productivity: A longitudinal study. Personnel Psychology, 61, 467–501.
7.
BondC. F.WiitalaW. L.RichardF. D. (2003). Meta-analysis of raw mean differences. Psychological Methods, 8, 406–418.
8.
BonettD. G. (2008). Meta-analytic interval estimation for bivariate correlations. Psychological Methods, 13, 173–181.
9.
ChanD.SchmittN. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143–159.
10.
ChapmanD. S.UggerslevK. L.CarrollS. A.PiasentinK. A.JonesD. A. (2005). Applicant attraction to organizations and job choice: A meta-analytic review of the correlates of recruiting outcomes. Journal of Applied Psychology, 90, 928–944.
11.
ChiaburuD. S.HarrisonD. A. (2008). Do peers make the place? Conceptual synthesis and meta-analysis of coworker effects on perceptions, attitudes, OCBs, and performance. Journal of Applied Psychology, 93(5), 1082–1103.
12.
CohenJ. (1992). A power primer. Psychological Bulletin, 112, 155–159.
13.
CombsJ.LiuY.HallA.KetchenD. (2006). How much do high-performance work practices matter? A meta-analysis of their effects on organizational performance. Personnel Psychology, 59, 501–528.
14.
CortinaJ. M. (2003). Apples and oranges (and pears, oh, my!): The search for moderators in meta-analysis. Organization Research Methods, 6, 415–439.
DetertJ. R.TrevinoL. K.SweitzerV. L. (2008). Moral disengagement in ethical decision making: A study of antecedents and outcomes. Journal of Applied Psychology, 93(2), 374–391.
17.
DulacT.Coyle-ShapiroJ. A. M.HendersonD. J.WayneS. J. (2008). Not all responses to breach are the same: The interconnection of social exchange and psychological contract processes in organizations. Academy of Management Journal, 51(6), 1079–1098.
18.
EllingsonJ. E.GruysM. L.SackettP. R. (1998). Factors related to the satisfaction and performance of temporary employees. Journal of Applied Psychology, 83(6), 913–921.
19.
FieldA. P. (2005). Is the meta-analysis of correlation coefficients accurate when population correlations vary?. Psychological Methods, 10, 444–467.
20.
GillettR. (2003). The metric comparability of meta-analytic effect-size estimators from factorial designs. Psychological Methods, 8, 419–433.
21.
GlassG. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.
22.
GlassG. V. (1977). Integrating findings: The meta-analysis of research. Review of Research in Education, 5, 29.
23.
GrantA. M.DuttonJ. E.RossoB. D. (2008). Giving commitment: Employee support programs and the prosocial sensemaking process. Academy of Management Journal, 51(5), 898–918.
24.
GullyS. M.DevineD. J.WhitneyD. J. (1995). A meta-analysis of cohesion and performance: Effects of levels of analysis and task interdependence. Small Group Research, 26, 497–520.
25.
HarrisonD. A.PriceK. H.BellM. P. (1998). Beyond relational demography: Time and the effects of surface- and deep-level diversity on work group cohesion. Academy of Management Journal, 41(1), 96–107.
26.
HarterJ. K.SchmidtF. L.HayesT. L. (2002). Business-unit-level relationship between employee satisfaction, employee engagement, and business outcomes: A meta-analysis. Journal of Applied Psychology, 87, 268–279.
27.
HedgesL. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93, 388–395.
28.
HedgesL. V.OlkinI. (1985). Statistical methods for meta-analysisOrlando, FL: Academic Press.
29.
HedgesL. V.VeveaJ. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.
30.
HigginsF. P. T.ThompsonS. G. (2002). Quantifying heterogeneity in meta-analysis. Statistics in Medicine, 21, 1539–1558.
31.
HinkinT. R.SchriesheimC. A. (2008). An examination of “nonleadership”: From laissez-faire leadership to leader reward omission and punishment omission. Journal of Applied Psychology, 93(6), 1234–1248.
32.
Huedo-MedinaT. B.Sanchez-MecaJ.Marin-MartinezF.BotellaJ. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index?. Psychological Methods, 11, 193–206.
33.
HunterJ. E.SchmidtF. L. (1990a). Dichotomization of continuous variables—The implications for meta-analysis. Journal of Applied Psychology, 75, 334–349.
34.
HunterJ. E.SchmidtF. L. (1990b). Methods of meta-analysis: Correcting error and bias in research findingsNewbury Park, CA: Sage.
35.
HunterJ. E.SchmidtF. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings2nd ed.Thousand Oaks, CA: Sage.
36.
HunterJ. E.SchmidtF. L.JacksonG. B. (1982). Meta-analysis: Cumulating research findings across studiesBeverly Hills, CA: Sage.
37.
JacksonS. E.SchulerR. S. (1985). A meta-analysis and conceptual critique of research on role ambiguity and role conflict in work settings. Organizational Behavior and Human Decision Processes, 36, 16–78.
38.
JudgeT. A.PiccoloR. F. (2004). Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology, 89, 755–768.
39.
JudgeT. A.ThorstenC. J.BonoJ. E.PattonG. K. (2001). The job satisfaction–job performance relationship: A qualitative and quantitative review. Psychological Bulletin, 127, 376–407.
40.
KisamoreJ. L.BrannickM. T. (2008). An illustration of the consequences of meta-analysis model choice. Organizational Research Methods, 11, 35–53.
41.
LambertL. S.EdwardsJ. R.CableD. M. (2003). Breach and fulfillment of the psychological contract: a comparison of traditional and expanded views. Personnel Psychology, 56, 895–934.
42.
LeH.SchmidtF. L. (2006). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic procedure. Psychological Methods, 11, 416–438.
43.
MalzahnU.BohningD.HollingG. (2000). Nonparametric estimators of heterogeneity variance for standardised difference used in meta-analysis. Biometrika, 87, 619–632.
44.
MeehlP. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195–244.
45.
Mesmer-MagnusJ. R.ViswesvaranC. (2005). Convergence between measures of work-to-family and family-to-work conflict: A meta-analytic examination. Journal of Vocational Behavior, 67, 215–232.
46.
MeyerJ. P.StanleyD. J.HerscovitchL.TopolnytskyL. (2002). Affective, continuance, and normative commitment to the organization: A meta-analysis of antecedents, correlates, and consequences. Journal of Vocational Behavior, 61, 20–52.
47.
MillerP.EisenbergN. (1988). The relation of empathy to aggressive and externalizing/antisocial behavior. Psychological Bulletin, 103, 324–344.
48.
MorrisS. B.DeShonR. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125.
49.
PodsakoffP. M.BommerW. H.PodsakoffN. P.MacKenzieS. B. (2006). Relationships between leader reward and punishment behavior and subordinate attitudes, perceptions, and behaviors: A meta-analytic review of existing and new research. Organizational Behavior and Human Decision Processes, 99, 113–142.
50.
PorterC.HollenbeckJ. R.IlgenD. R.EllisA. P. J.WestB. J.MoonH. (2003). Backing up behaviors in teams: The role of personality and legitimacy of need. Journal of Applied Psychology, 88(3), 391–403.
51.
RhoadesL.EisenbergerR. (2002). Perceived organizational support: A review of the literature. Journal of Applied Psychology, 87, 698–714.
52.
RosenthalR. (1984). Meta-analysis procedures for social researchBeverly Hills, CA: Sage.
53.
RosenthalR. (1991). Meta-analysis procedures for social researchRev ed.Newbury Park, CA: Sage.
54.
RothP. L.BobkoP.McFarlandL. A. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology, 58, 1009–1037.
55.
SagieA.KoslowskiM. (1993). Detecting moderators with meta-analysis: An evaluation and comparison of techniques. Personnel Psychology, 46, 629–640.
56.
Sanchez-MecaJ.Marin-MartinezF. (2008). Confidence intervals for the overall effect size in random effects meta-analysis. Psychological Methods, 13, 31–48.
57.
SchaubroeckJ.ShawJ. D.DuffyM. K.MitraA. (2008). An under-met and over-met expectations model of employee reactions to merit raises. Journal of Applied Psychology, 93(2), 424–434.
58.
SchmidtF. L.HunterJ. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529–540.
59.
SchmidtF. L.OhI.HayesT. L. (2009). Fixed- versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62, 97–128.
60.
SchmidtF. L.RajuN. S. (2007). Updating meta-analytic research findings: Bayesian approaches versus the medical model. Journal of Applied Psychology, 92, 297–308.
61.
SidikK.JonkmanJ. N. (2007). A comparison of heterogeneity variance estimators in combine results of studies. Statistics in Medicine, 26, 1964–1981.
62.
SilverN. C.DunlapW. P. (1987). Averaging correlation coefficients: Should Fisher’s Z transformation be used?. Journal of Applied Psychology, 72, 146–148.
63.
SmitherJ. W.LondonM.ReillyR. R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58, 33–66.
64.
SonnentagS. (1998). Expertise in professional software design: A process study. Journal of Applied Psychology, 83(5), 703–715.
65.
ViechtbauerW. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine, 26, 37–52.
66.
WalkerA. G.SmitherJ. W.WaldmanD. A. (2008). A longitudinal examination of concomitant changes in team leadership and customer satisfaction. Personnel Psychology, 61, 547–577.
67.
WanousJ. P.SullivanS. E.MalinakJ. (1989). The role of judgment calls in meta-analysis. Journal of Applied Psychology, 74, 259–264.
68.
WhitenerE. M. (1990). Confusion of confidence intervals and credibility intervals in meta-analysis. Journal of Applied Psychology, 75, 315–321.
69.
WilliamsM. L.McDanielM. A.NguyenN. T. (2006). A meta-analysis of the antecedents and consequences of pay level satisfaction. Journal of Applied Psychology, 91, 392–413.
70.
WittL. A.FerrisG. R. (2003). Social skill as moderator of the conscientiousness-performance relationship: Convergent results across four studies. Journal of Applied Psychology, 88(5), 809–820.
71.
WrightP. M.GeorgeJ. M.FarnsworthS. R.McMahanG. C. (1993). Productivity and extra-role behavior: The effects of goals and incentives on spontaneous helping. Journal of Applied Psychology, 78(3), 374–381.
72.
ZhaoH.WayneS. J.GlibkowskiB. C.BravoJ. (2007). Impact of psychological contract breach on work-related outcomes: A meta-analysis. Personnel Psychology, 60, 647–680.
