Correlation-Based Meta-Analytic Structural Equation Modeling: Effects of Parameter Covariance on Point and Interval Estimates

Abstract

More and more researchers use meta-analysis to conduct multivariate analysis to summarize previous findings. In the correlation-based meta-analytic structural equation modeling (cMASEM), the average sample correlation matrix is used to estimate the average population model. Using a simple mediation model, we illustrated that random effects covariation in population parameters can theoretically bias the path coefficient estimates and lead to nonnormal random effects distribution of the correlations. We developed an R function for researchers to examine by simulation the impact of random effects in other models. We then reanalyzed two real data sets and conducted a simulation study to examine the magnitude of the impact on realistic situations. Simulation results suggest parameter bias is typically negligible (less than .02), parameter bias and root mean square error do not differ across methods, 95% confident intervals are sometimes more accurate for the two-stage structural equation modeling approach with a diagonal random effects model, and power is sometimes higher for the traditional Viswesvaran-Ones approach. Given the increasing popularity of cMASEM in organizational research, these simulation results form the basis for us to make several recommendations on its application.

Keywords

meta-analysis structural equation modeling meta-analytic structural equation modeling

Get full access to this article

View all access options for this article.

References

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Process, 50, 179–211.

Aroian

L. A.

(1947). The probability function of the product of two normally distributed variables. The Annals of Mathematical Statistics, 18, 265–271.

Becker

B. J.

(1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362.

Becker

B. J.

(1995). Corrections to “using results from replicated studies to estimate linear models.” Journal of Educational Statistics, 20, 100–102.

Becker

B. J.

Schram

C. M.

(1994). Examining explanatory models through research synthesis. In Cooper

Hedges

L. V.

(Eds.), The handbook of research synthesis (pp. 357–381). New York, NY: Russell Sage Foundation.

Beus

J. M.

Dhanani

L. Y.

Mccord

M. A.

(2015). A meta-analysis of personality and workplace safety: Addressing unanswered questions. Journal of Applied Psychology, 100, 481–498.

Cheung

M. W.-L.

(2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46, 29–40.

Cheung

M. W.-L.

(2015a). Meta-analysis: A structural equation modeling approach. Chichester, UK: John Wiley & Sons.

Cheung

M. W.-L.

(2015b). metaSEM: An R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5, 1521.

10.

Cheung

M. W.-L.

Chan

(2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40–64.

11.

Cheung

M. W.-L.

Cheung

S. F.

(2016). Random-effects models for meta-analytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods, 7, 140–155.

12.

Cheung

S. F.

Chan

D. K.-S.

(2000). The role of perceived behavioral control in predicting human behavior: A meta-analytic review of studies on the theory of planned behavior. Unpublished manuscript, Chinese University of Hong Kong.

13.

Colquitt

J. A.

LePine

J. A.

Noe

R. A.

(2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707.

14.

Courtright

S. H.

Thurgood

G. R.

Stewart

G. L.

Pierotti

A. J.

(2015). Structural interdependence in teams: An integrative framework and meta-analysis. Journal of Applied Psychology, 100, 1825–1846.

15.

Craig

C. C.

(1936). On the frequency function of xy. The Annals of Mathematical Statistics, 7, 1–15.

16.

Hedges

L. V.

(2016). Applying meta-analysis to structural equation modeling. Research Synthesis Methods, 7, 209–214.

17.

Hedges

L. V.

Vevea

J. L.

(1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.

18.

Hong

Liao

Jiang

(2013). Missing link in the service profit chain: A meta-analytic review of the antecedents, consequences, and moderators of service climate. Journal of Applied Psychology, 98, 237–267.

19.

Hunter

J. E.

Schmidt

F. L.

(2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage Publications.

20.

Joseph

D. L.

Jin

Newman

D. A.

O’Boyle

E. H.

(2015). Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI. Journal of Applied Psychology, 100, 298–342.

21.

Kalaian

H. A.

Raudenbush

S. W.

(1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227–235.

22.

Knight

A. P.

Eisenkraft

(2014). Positive is usually good, negative is not always bad: The effects of group affect on social integration and task performance. Journal of Applied Psychology, 100, 1214–1227.

23.

Liu

Huang

J. L.

Wang

(2014). Effectiveness of job search interventions: A meta-analytic review. Psychological Bulletin, 140, 1009–1041.

24.

T. W. H.

Feldman

D. C.

(2015). Ethical leadership: Meta-analytic evidence of criterion-related and incremental validity. Journal of Applied Psychology, 100, 948–965.

25.

Ouellette

J. A.

Wood

(1998). Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior. Psychological Bulletin, 124, 54–74.

26.

Premack

S. L.

Hunter

J. E.

(1988). Individual unionization decisions. Psychological Bulleting, 103, 223–234.

27.

R Development Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Downloaded from https://www.R-project.org/.

28.

Raudenbush

S. W.

Becker

B. J.

Kalaian

(1988). Modeling multivariate effect sizes. Psychological Bulletin, 103, 111–120.

29.

Rosseel

(2012). lavaan: An r package for structural equation modeling. Journal of Statistical Software, 48, 1–36.

30.

Schmidt

F. L.

Hunter

J. E.

(2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage.

31.

Shadish

W. R.

(1996). Meta-analysis and the exploration of causal mediating processes: A primer of examples, methods, and issues. Psychological Methods, 1, 47–65.

32.

Sheng

Kong

Cortina

J. M.

Hou

(2016). Analyzing matrices of meta-analytic correlations: Current practices and recommendations. Research Synthesis Methods, 7, 187–208.

33.

Tallis

G. M.

(1961). The moment generating function of the truncated multi-normal distribution. Journal of the Royal Statistical Society, Series B (Methodological), 23, 223–229.

34.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36, 1–48.

35.

Viswesvaran

Ones

D. S.

(1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865–885. doi:10.1111/j.1744-6570.1995.tb01784.x

36.

Viswesvaran

Schmidt

F. L.

Ones

D. S.

(2005). Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology, 90, 108–131.

37.

World Values Study Group. (1994). World Values Survey, 1981-1984 and 1990-1993 [Computer file]. Ann Arbor, MI: Inter-university Consortium for Political and Social Research.

38.

Downes

P. E.

Carter

K. M.

O’Boyle

E. H.

(2016). The problem of effect size heterogeneity in meta-analytic structural equation modeling. Journal of Applied Psychology, 101, 1457–1473. doi:10.1037/apl0000141

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.24 MB