Systematic reviews with meta-analysis represent the gold standard for conducting reliable and transparent reviews of the literature. The purpose of this article is threefold: (a) to address why and when it is worthwhile to conduct a systematic review with meta-analysis, covering advantages of this approach in the context of the statistics reform in the behavioral sciences; (b) to explain how to conduct and publish a systematic review with meta-analysis, describe the main steps, and suggest best practices for each of them; and (c) to discuss the relevance of conducting a systematic review with meta-analysis for the emerging adulthood field, suggesting how this approach can be applied to address research questions about the specificity of this period. In addressing these issues, a fictitious systematic review with meta-analysis aimed at examining gender differences in the view of emerging adulthood as a period of exploration, instability, self-focus, feeling-in-between, and possibilities is presented. Furthermore, individual participant data systematic review with meta-analysis is proposed as an important future direction for conducting reviews within the social sciences.
AdachiP.WilloughbyT. (2014). Interpreting effect sizes when controlling for stability effects in longitudinal autoregressive models: Implications for psychological science. European Journal of Developmental Psychology, 12, 116–128.
2.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
3.
ArnettJ. J. (2000). Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist, 55, 469–480.
4.
ArnettJ. J. (2004). Emerging adulthood: The winding road from the late teens through the twenties. New York, NY: Oxford University Press.
5.
BeckerB. J. (2005). Failsafe N or file-drawer number. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis (pp. 111–125). Chichester, England: John Wiley.
6.
BeggC. B.MazumdarM. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.
7.
BorensteinM.HedgesL. V.HigginsJ. P. T.RothsteinH. R. (2009). Introduction to meta-analysis. Chichester, England: John Wiley.
8.
BurrD.DossH. (2005). A Bayesian semiparametric model for random-effects meta-analysis. Journal of the American Statistical Association, 100, 242–251.
9.
BushmanB. J.WellsG. L. (2001). Narrative impressions of literature: The availability bias and the corrective properties of meta-analytic approaches. Personality and Social Psychology Bulletin, 27, 1123–1130.
Centre for Reviews and Dissemination. (2009). Systematic reviews. York, England: York Publishing Services.
12.
ClarkI. A.MackayC. E.HolmesE. A. (2015). Low emotional response to traumatic footage is associated with an absence of analogue flashbacks: An individual participant data meta-analysis of 16 trauma film paradigm experiments. Cognition and Emotion, 29, 702–713.
13.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
14.
CooperH. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Thousand Oaks, CA: Sage.
15.
CooperH.DeNeveK.CharltonK. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2, 447–452.
16.
CooperH.HedgesL. V.ValentineJ. (Eds.). (2009). The handbook of research synthesis (2nd ed.). New York, NY: Russell Sage.
17.
CooperH.PatallE. A. (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14, 165–176.
18.
CrocettiE. (2015). Rassegne sistematiche, sintesi della ricerca e meta-analisi [Systematic reviews, research syntheses, and meta-analysis]. North Charleston, SC: CreateSpace.
19.
CrocettiE.TagliabueS.SugimuraK.NelsonL.TakahashiA.NiwaT. … JinnoM. (2015). Perceptions of emerging adulthood: A study with Italian and Japanese university students and young workers. Emerging Adulthood, 3, 229–243.
20.
CuijpersP.WeitzE.TwiskJ.KuehnerC.CristeaI.DavidD. … HollonS. D. (2014). Gender as predictor and moderator of outcome in cognitive behavior therapy and pharmacotherapy for adult depression: An “individual patient data” meta-analysis. Depression and Anxiety, 31, 941–951.
21.
CummingG. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta–analysis. New York, NY: Routledge.
22.
CummingG. (2014). The new statistics: Why and how. Psychological Science, 25, 7–29.
23.
DickersinK. (2005). Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta–analysis (pp. 127–144). Chichester, England: John Wiley.
24.
DuvalS. (2005). The trim and fill method. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis (pp. 11–33). Chichester, England: John Wiley.
25.
DuvalS.TweedieR. (2000). A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. Journal of American Statistical Association, 95, 89–98.
26.
EggerM.Davey SmithG.SchneiderM.MinderC. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.
27.
EllisP. D. (2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge, England: Cambridge University Press.
28.
GreenwaldA. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20.
29.
HaleIII, W. W.CrocettiE.RaaijmakersQ.MeeusW. (2011). A meta-analysis of the cross–cultural psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED). Journal of Child Psychology and Psychiatry, 52, 80–90.
30.
HedgesL. V.VeveaJ. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.
31.
HigginsJ. P. T.GreenS. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions, Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Retrieved fromwww.cochrane-handbook.org
32.
HigginsJ.ThompsonS. G.DeeksJ. J.AltmanD. G. (2003). Measuring inconsistency in meta-analysis. British Medical Journal, 327, 557–560.
33.
Huedo-MedinaT. B.Sánchez-MecaJ.Marin-MartinezF.BotellaJ. (2006). Assessing heterogeneity in meta-analysis: Q Statistic or I2 index?Psychological Methods, 11, 193–206.
34.
HuntM. (1997). How science takes stock: The story of meta-analysis. New York, NY: Russell Sage Foundation.
35.
KlineR. B. (2013). Beyond significance testing: Statistics reform in the behavioral sciences (2nd ed.). Washington, DC: American Psychological Association.
36.
KrzyzanowskaM. K.PintilieM.TannockI. F. (2003). Factors associated with failure to publish large randomized trials presented at an oncology meeting. Journal of the American Medical Association, 290, 495–501.
37.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
38.
LiberatiA.AltmanD. G.TetzlaffJ.MulrowC.GøtzscheP. C.IoannidisJ. P. A. … MoherD. (2009). The PRISMA statement for reporting systematic reviews and meta–analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6, e1000100.
39.
LightR. J.PillemerD. B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
MagrinM. E.D’AddarioM.GrecoA.MigliorettiM.SariniM.ScrignaroM. … CrocettiE. (2015). Social support and adherence to treatment in hypertensive patients: A meta-analysis. Annals of Behavioral Medicine, 49, 307–318.
42.
MahoneyM. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy & Research, 1, 161–175.
43.
ManteN.HaenszelW. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.
44.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta–analyses: The PRISMA statement. PLoS Medicine, 6, e1000097.
45.
NegruO. (2012). The time of your life: Emerging adulthood characteristics in a sample of Romanian high-school and university students. Cognition, Brain, Behavior, 16, 357–367.
46.
NorthouseL. L.KatapodiM. C.SongL.ZhangL.MoodD. W. (2010). Interventions with family caregivers of cancer patients, meta-analysis of randomized trials. CA Cancer Journal for Clinicians, 60, 317–339.
47.
PetticrewM.RobertsH. (2006). Systematic reviews in the social sciences. Malden, MA: Blackwell Publishing.
48.
ReifmanA.ArnettJ. J.ColwellM. J. (2007). Emerging adulthood: Theory, assessment, and application. Journal of Youth Development, 2, 1–12.
49.
RileyR. D.LambertP. C.Abo-ZaidG. (2010). Meta-analysis of individual participant data: Rationale, conduct, and reporting. British Medical Journal (Online), 340, 521–525.
50.
RobertsB. W.WaltonK. E.ViechtbauerW. (2006). Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132, 1–25.
51.
RosenthalR. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
52.
RothsteinH. R.HopewellS. (2009). Grey literature. In CooperH.HedgesL. V.ValentineJ. (Eds.), The handbook of research synthesis (2nd ed., pp. 103–125). New York, NY: Russell Sage.
53.
RothsteinH. R.SuttonA. J.BorensteinM. (Eds.). (2005). Publication bias in meta-analysis. Chichester, England: John Wiley.
54.
SimmondsM. C.HigginsJ. P. T.StewartL. A.TierneyJ. F.ClarkeM. J.ThompsonS. G. (2005). Meta-analysis of individual patient data from randomized trials: A review of methods used in practice. Clinical Trials, 2, 209–217.
55.
SterneJ. A.EggerM. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54, 1046–1055.
56.
SterneJ. A.BeckerB. J.EggerM. (2005). The funnel plot. In RothsteinH. R.SuttonA. J.BorensteinM. (Eds.), Publication bias in meta-analysis (pp. 75–98). Chichester, England: John Wiley.
57.
StewartL. A.TierneyJ. F. (2002). To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Evaluation in the Health Professions, 25, 76–97.
58.
StewartL. A.TierneyJ. F.ClarkeM. (2011). Reviews of individual patient data. In HigginsJ. P. T.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions, Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Retrieved fromwww.cochrane-handbook.org
59.
StroupD. F.BerlinJ. A.MortonS. C.OlkinI.WilliamsonG. D.RennieD. … ThackerS. B. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. Journal of the American Medical Association, 283, 2008–2012.
60.
SuttonA. J.AbramsK. R. (2001). Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10, 277–303.
61.
TurnerR. M.JacksonD.WeiY.ThompsonS. G.HigginsJ. P. T. (2015). Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Statistics in Medicine, 34, 984–998.
62.
ValentineJ. C. (2009). Judging the quality of primary research. In CooperH.HedgesL. V.ValentineJ. (Eds.), The handbook of research synthesis (2nd ed., pp. 129–146). New York, NY: Russell Sage.
63.
ViechtbauerW. (2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. Journal of Psychology, 215, 104–121.
