Meta-analysis of panel data is uniquely suited to uncovering phenomena that develop over time, but extant approaches are limited. There is no straightforward means of aggregating findings of primary panel studies that use different time lags and different numbers of waves. We introduce continuous time meta-analysis (CoTiMA) as a parameter-based approach to meta-analysis of cross-lagged panel correlation matrices. CoTiMA enables aggregation of studies using two or more waves even if there are varying time lags within and between studies. CoTiMA thus provides meta-analytic estimates of cross-lagged effects for a given time lag regardless of the frequency with which that time lag is used in primary studies. We describe the continuous time underpinnings of CoTiMA, its advantages over discrete-time, correlation-based meta-analysis of structural equation models (MASEM), and how CoTiMA would be applied to meta-analysis of panel studies. An example is then used to illustrate the approach. We also conducted Monte Carlo simulations demonstrating that bias is larger for time category–based MASEM than for CoTiMA under various conditions. Finally, we discuss data requirements, open questions, and possible future extensions.
Aguirre-UrretaM. I.EllisM. E.SunW. (2012). Performance of a proportion-based approach to meta-analytic moderator estimation: Results from Monte Carlo simulations. Research Synthesis Methods, 3, 11–29. doi:10.1002/jrsm.1083
2.
AndersonC. A.ShibuyaA.IhoriN.SwingE. L.BushmanB. J.SakamotoA.…SaleemM. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review. Psychological Bulletin, 136, 151–173. doi:10.1037/a0018251
3.
*AshforthB. E.SaksA. M. (1996). Socialization tactics: Longitudinal effects on newcomer adjustment. Academy of Management Journal, 39, 149–178. doi:10.2307/256634
AtkinsonL.NiccolsA.PagliaA.CoolbearJ.ParkerK. C. H.PoultonL.…SitareniosG. (2000). A meta-analysis of time between maternal sensitivity and attachment assessments: Implications for internal working models in infancy/toddlerhood. Journal of Social and Personal Relationships, 17, 791–810. doi:10.1177/0265407500176005
6.
*BechtoldS. E.SimsH. P.SzilagyiA. D. (1981). Job scope relationships: A three-wave longitudinal analysis. Journal of Occupational Behaviour, 2, 189–202. doi:10.1002/job.4030020304
7.
BokerS. M.NealeM.MaesH.WildeM.SpiegelM.BrickT.…FoxJ. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.
8.
BollenK. A.BrandJ. E. (2010). A general panel model with random and fixed effects: A structural equations approach. Social Forces, 89, 1–34. doi:10.1353/sof.2010.0072.
9.
*BondF. W.BunceD. (2003). The role of acceptance and job control in mental health, job satisfaction, and work performance. Journal of Applied Psychology, 88, 1057–1067. doi:10.1037/0021-9010.88.6.1057
10.
BorensteinM.HedgesL. V.HigginsJ. P. T.RothsteinH. R. (2009a). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1, 97–111.
11.
BorensteinM.HedgesL. V.HigginsJ. P. T.RothsteinH. R. (2009b). Introduction to meta-analysis. New York, NY: Wiley.
12.
CheungM. W.-L. (2015). Meta-analysis: A structural equation modeling approach. New York, NY: Wiley.
13.
CheungM. W.-L.CheungS.-F. (2016). Random-effects models for meta-analytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods, 7, 140–155. doi:10.1002/jrsm.1166
14.
CohenA. (1993). Organizational commitment and turnover: A meta-analysis. Academy of Management Journal, 36, 1140–1157. doi:10.2307/256650
15.
CohenJ.CohenP.WestS. G.AikenL. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Erlbaum.
16.
ColeD. A.MaxwellS. E. (2003). Testing meditational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112, 558–577. doi:/10.1037/0021-843X.112.4.558
17.
CrookT. R.ToddS. Y.CombsJ. G.WoehrD. J.KetchenD. J.Jr (2011). Does human capital matter? A meta-analysis of the relationship between human capital and firm performance. Journal of Applied Psychology, 96, 443–456. doi:10.1037/a0022147.
18.
DalgardO.SørensenT.SandangerI.NygårdJ. F.SvenssonE.ReasD. L. (2009). Job demands, job control, and mental health in an 11-year follow-up study: Normal and reversed relationships. Work & Stress, 23, 284–296. doi:10.1080/07399332.2014.980889
19.
DemeroutiE.BakkerA. B.BultersA. J. (2004). The loss spiral of work pressure, work-home interference and exhaustion: reciprocal relations in a three-wave study. Journal of Vocational Behavior, 64, 131–149. doi:10.1016/S0001-8791(03)00030-7
20.
DormannC. (2001). Modeling unmeasured third variables in longitudinal studies. Structural Equation Modeling, 8, 575–598. doi:10.1207/S15328007SEM0804_04
21.
DormannC.GriffinM. A. (2015). Optimal time lags in panel studies. Psychological Methods, 20, 489–505. doi:10.1037/met0000041
22.
DormannC.ZapfD. (2001). Job satisfaction—A meta-analysis of stabilities. Journal of Organizational Behavior, 22, 1–22. doi:10.1002/job.98
23.
DriverC.OudJ. H. L.VoelkleM. C. (2017). Continuous time structural equation modeling with R package ctsem. Journal of Statistical Software, 77, 1–35. doi:10.18637/jss.v077.i05
24.
DunlapW. P.CortinaJ. M.VaslowJ. B.BurkeM. J. (1996). Meta-ANALYSIS of experiments with matched groups or repeated measures designs. Psychological Methods, 1, 170–177.
25.
FinkelS. E. (1995). Causal analysis with panel data. London: Sage.
26.
FordM. T.MatthewsR. A.WoolridgeJ. D.MishraV.KakarU. M.StrahanS. R. (2014). How do occupational stressor-strain effects vary with time? A review and meta-analysis of the relevance of time lags in longitudinal studies. Work & Stress, 28, 9–30. doi:10.1080/02678373.2013.877096
27.
GlasmanL. R.AlbarracínD. (2006). Forming attitudes that predict future behavior: A meta-analysis of the attitude-behavior relation. Psychological Bulletin, 132, 778–822. doi:10.1037/0033-2909.132.5.778
28.
GriffethR. W.HomP. W.GaertnerS. (2000). A meta-analysis of antecedents and correlates of employee turnover: Update, moderator tests, and research implications for the next millennium. Journal of Management, 26, 463–488. doi:10.1177/014920630002600305
29.
*GriffinR. W. (1991). Effects of work redesign on employee perceptions, attitudes, and behaviors: A long-term investigation. Academy of Management Journal, 34, 425–435. doi:10.2307/256449
30.
HangesP. J.WangM. (2012). Seeking the holy grail in organizational science: Uncovering causality through research design. In KozlowskiS. W. J. (Ed.), Oxford library of psychology. The Oxford handbook of organizational psychology (Vol. 1, pp. 79–116). New York, NY: Oxford University Press.
HigginsJ. P. T.ThompsonS. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558.
33.
HoldenG.MoncherM. S.SchinkeS. P.BarkerK. M. (1990). Self-efficacy of children and adolescents: A meta-analysis. Psychological Reports, 66, 1044–1046. doi:10.2466/PR0.66.3.1044-1046
34.
HomP. W.Caranikas-WalkerF.PrussiaG. E.GriffethR. W. (1992). A meta-analytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77, 890–909. doi:10.1037/0021-9010.77.6.890
35.
HulinC. L.HenryR. A.NoonS. L. (1990). Adding a dimension: Time as a factor in the generalizability of predictive relationships. Psychological Bulletin, 107, 328–340. doi:10.1037/0033-2909.107.3.328
36.
HunterJ. E.SchmidtF. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
37.
HunterJ. E.SchmidtF. L.LeH. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594–612.
38.
JinJ.RoundsJ. (2012). Stability and change in work values: A meta-analysis of longitudinal studies. Journal of Vocational Behavior, 80, 326–339. doi:10.1016/j.jvb.2011.10.007
39.
JudgeT. A.PiccoloR. F.IliesR. (2004). The forgotten ones? The validity of consideration and initiating structure in leadership research. Journal of Applied Psychology, 89, 36–51. doi:10.1037/0021-9010.89.1.36
40.
KennyD. A. (1979). Correlation and causality. New York, NY: Wiley.
41.
KesslerR. C.GreenbergD. F. (1981). Linear panel analysis. New York, NY: Academic Press.
42.
KetchenD. J.JrCombsJ. G.RussellC. J.ShookC.DeanM. A.RungeJ.…BecksteinB. A. (1997). Organizational configurations and performance: A meta-analysis. Academy of Management Journal, 40, 223–240. doi:10.5465/257028
43.
MarcelissenF. H.WinnubstJ. A.BuunkB.de WolffC. J. (1988). Social support and occupational stress: A causal analysis. Social Science and Medicine, 26, 365–373. doi:10.1016/0277-9536(88)90402-9
44.
MaricuţoiuL. P.SuleaC.IancuA. (2017). Work engagement or burnout: Which comes first? A meta-analysis of longitudinal evidence. Burnout Research, 5, 35–43. doi:10.1016/j.burn.2017.05.001
45.
MathieuJ. E.KukenbergerM. R.D’InnocenzoL.ReillyG. (2015). Modeling reciprocal team cohesion–performance relationships, as impacted by shared leadership and members’ competence. Journal of Applied Psychology, 100, 713–734. doi:10.1037/a0038898
46.
MehtaP. D.NealeM. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259–284. doi:10.1037/1082-989X.10.3.259
47.
MitchellT. E.JamesL. R. (2001). Building better theory: Time and the specification of when things happen. Academy of Management Review, 26, 530–547. doi:10.5465/AMR.2001.5393889
48.
*NathanB. R.MohrmanA. M.JrMillimanJ. (1991). Interpersonal relations as a context for the effects of appraisal interviews on performance and satisfaction: A longitudinal study. Academy of Management Journal, 34, 352–369. doi:10.2307/256446
49.
NielsenK.NielsenM. B.OgbonnayaC.KänsäläM.SaariE.IsakssonK. (2017). Workplace resources to improve both employee well-being and performance: A systematic review and meta-analysis. Work & Stress, 31, 101–120. doi:10.1080/02678373.2017.1304463
50.
NoheC.MeierL. L.SonntagK.MichelA. (2015). The chicken or the egg? A meta-analysis of panel studies of the relationship between work–family conflict and strain. Journal of Applied Psychology, 100, 522–536. doi:10.1037/a0038012
51.
NyeC. D.BradburnJ.OlenickJ.BialkoC.DrasgowF. (2018). How big are my effects? Examining the magnitude of effect sizes in studies of measurement equivalence. Organizational Research Methods. Advance online publication. doi:10.1177/1094428118761122.
52.
R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
53.
RikettaM. (2008). The causal relation between job attitudes and performance: A meta-analysis of panel studies. Journal of Applied Psychology, 93, 472–481. doi:10.1037/0021-9010.93.2.472
54.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. doi:10.18637/jss.v048.i02
55.
RuguliesR.AustB.MadsenI. E. H. (2017). Effort-reward imbalance at work and risk of depressive disorders. A systematic review and meta-analysis of prospective cohort studies. Scandinavian Journal of Work, Environment & Health, 43, 294–306. doi:10.5271/sjweh.3632
56.
SaridakisG.LaiY.CooperC. L. (2017). Exploring the relationship between HRM and firm performance: A meta-analysis of longitudinal studies. Human Resource Management Review, 27, 87–96. doi:10.1016/j.hrmr.2016.09.005
57.
SchmidtF. L.HunterJ. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage.
58.
*SheridanJ. E.SlocumJ. W. (1975). The direction of the causal relationship between job satisfaction and work performance. Organizational Behavior and Human Performance, 14, 159–172. doi:10.1016/0030-5073(75)90021-5
59.
SonnentagS. (2001). Work, recovery activities, and individual well-being: A diary study. Journal of Occupational Health Psychology, 6, 196–210. doi:10.1037/1076-8998.6.3.196
60.
SowisloJ. F.OrthU. (2013). Does low self-esteem predict depression and anxiety? A meta-analysis of longitudinal studies. Psychological Bulletin, 139, 213–240. doi:10.1037/a0028931
61.
SteelR. P.HendrixW. H.BaloghS. P. (1990). Confounding effects of the turnover base rate on relations between time lag and turnover study outcomes: An extension of me-ta-analysis findings and conclusions. Journal of Organizational Behavior, 11, 237–242. doi:10.1002/job.4030110306
62.
SteelR. P.OvalleN. K. (1984). A review and meta-analysis of research on the relationship between behavioral intentions and employee turnover. Journal of Applied Psychology, 69, 673–686. doi:10.1037/0021-9010.69.4.673
63.
*SzilagyiA. D. (1980). Causal inferences between leader reward behavior and subordinate performance, absenteeism, and work satisfaction. Journal of Occupational Psychology, 53, 195–204. doi:10.1111/j.2044-8325.1980.tb00025.x
64.
*TharenouP. (1993). A test of reciprocal causality for absenteeism. Journal of Organizational Behavior, 14, 269–290. doi:10.1002/job.4030140306
65.
ThompsonS. G.HigginsJ. P. T. (2002). How should meta-regression analyses be undertaken and interpreted?Statistics in Medicine, 21, 1559–1573. doi:10.1002/sim.1187
66.
VerkuilB.AtasayiS.MolendijkM. L. (2015). Workplace bullying and mental health: A meta-analysis on cross-sectional and longitudinal data. PloS one, 10, e0135225. doi:10.1371/journal.pone.0135225
67.
VoelkleM. C.OudJ. H. L.DavidovE.SchmidtP. (2012). A SEM approach to continuous time modeling of panel data: Relating authoritarism and anomia. Psychological Methods, 17, 176–192. doi:10.1037/a0027543
68.
*WanousJ. P. (1974). A causal-correlational analysis of the job satisfaction and performance relationship. Journal of Applied Psychology, 59, 139–144. doi:10.1037/h0036532
69.
ZapfD.DormannC.FreseM. (1996). Longitudinal studies in organizational stress research: A review of the literature with reference to methodological issues. Journal of Occupational Health Psychology, 1, 145–169. doi:10.1037/1076-8998.1.2.14
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
