Entities such as individuals, teams, or organizations can vary systematically from one another. Researchers typically model such data using multilevel models, assuming that the random effects are uncorrelated with the regressors. Violating this testable assumption, which is often ignored, creates an endogeneity problem thus preventing causal interpretations. Focusing on two-level models, we explain how researchers can avoid this problem by including cluster means of the Level 1 explanatory variables as controls; we explain this point conceptually and with a large-scale simulation. We further show why the common practice of centering the predictor variables is mostly unnecessary. Moreover, to examine the state of the science, we reviewed 204 randomly drawn articles from macro and micro organizational science and applied psychology journals, finding that only 106 articles—with a slightly higher proportion from macro-oriented fields—properly deal with the random effects assumption. Alarmingly, most models also failed on the usual exogeneity requirement of the regressors, leaving only 25 mostly macro-level articles that potentially reported trustworthy multilevel estimates. We offer a set of practical recommendations for researchers to model multilevel data appropriately.
AikenL. S.WestS. G.MillsapR. E. (2008). Doctoral training in statistics, measurement, and methodology in psychology—Replication and extension of Aiken, West, Sechrest, and Reno’s (1990) survey of PhD programs in North America. American Psychologist, 63(1), 32–50.
AngristJ. D.PischkeJ.-S. (2008). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.
4.
AngristJ. D.PischkeJ.-S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24(2), 3–30.
5.
AngristJ. D.PischkeJ.-S. (2014). Mastering metrics: The path from cause to effect. Princeton, NJ: Princeton University Press.
6.
AntonakisJ. (2011). Predictors of leadership: The usual suspects and the suspect traits. In BrymanA.CollinsonD.GrintK.JacksonB.Uhl-BienM. (Eds.), Sage handbook of leadership (pp. 269–285). Thousand Oaks, CA: Sage.
7.
AntonakisJ.BastardozN.LiuY.SchriesheimC. A. (2014). What makes articles highly cited?The Leadership Quarterly, 25(1), 152–179.
8.
AntonakisJ.BendahanS.JacquartP.LaliveR. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21, 1086–1120.
9.
AntonakisJ.BendahanS.JacquartP.LaliveR. (2014). Causality and endogeneity: Problems and solutions. In DayD. V. (Ed.), The Oxford handbook of leadership and organizations (pp. 93–117). New York, NY: Oxford University Press.
10.
ArellanoM. (1993). On the testing of correlated effects with panel data. Journal of Econometrics, 59(1-2), 87–97.
11.
BallingerG. A. (2004). Using generalized estimating equations for longitudinal data analysis. Organizational Research Methods, 7(2), 127–150.
12.
BascleG. (2008). Controlling for endogeneity with instrumental variables in strategic management research. Strategic Organization, 6(3), 285–327.
13.
BatesD.MächlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.
14.
BellA.JonesK. (2015). Explaining fixed effects: Random effects modeling of time-series cross-sectional and panel data. Political Science Research and Methods, 3(1), 133–153.
15.
BlieseP. D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1(4), 355–373.
16.
BlieseP. D. (2000). Within-group agreement, non-independence, and reliability: Implications for aggregation and analysis. In KozlowskiS. W. J.KleinK. J. (Eds.), Multilevel theory, research, and methods in organizations (pp. 349–381). San Francisco, CA: Jossey-Bass.
17.
BlieseP. D.PloyhartR. E. (2002). Growth modeling using random coefficient models: Model building, testing, and illustrations. Organizational Research Methods, 5(4), 362–387.
18.
BollenK. A. (2012). Instrumental variables in sociology and the social sciences. Annual Review of Sociology, 38(1), 37–72.
19.
CameronA. C.GelbachJ. B.MillerD. L. (2011). Robust inference with multiway clustering. Journal of Business & Economic Statistics, 29(2), 238–249.
20.
CameronA. C.MillerD. L. (2015). A practitioner’s guide to cluster-robust inference. Journal of Human Resources, 50(2), 317–372.
21.
CertoS. T.SemadeniM. (2006). Strategy research and panel data: Evidence and implications. Journal of Management, 32(3), 449–471.
22.
CertoS. T.WithersM. C.SemadeniM. (2017). A tale of two effects: Using longitudinal data to compare within- and between-firm effects. Strategic Management Journal, 38(7), 1536–1556.
23.
ClarkT. S.LinzerD. A. (2015). Should I use fixed or random effects?Political Science Research and Methods, 3(2), 399–408.
24.
CohenJ. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
25.
CroissantY.MilloG. (2008). Panel data econometrics in R: The plm package. Journal of Statistical Software, 27(2), 1–43.
26.
CurranP. J.BauerD. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619.
27.
DalalD. K.ZickarM. J. (2012). Some common myths about centering predictor variables in moderated multiple regression and polynomial regression. Organizational Research Methods, 15(3), 339–362.
28.
DawsonJ. F. (2014). Moderation in management research: What, why, when, and how. Journal of Business and Psychology, 29(1), 1–19.
29.
EndersC. K.TofighiD. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121–138.
30.
GennetianL. A.MagnusonK.MorrisP. A. (2008). From statistical associations to causation: What developmentalists can learn from instrumental variables techniques coupled with experimental data. Developmental Psychology, 44(2), 381–394.
31.
GreeneW. H. (2012). Econometric analysis. Boston, MA: Prentice Hall.
32.
GuoG. (2017). Demystifying variance in performance: A longitudinal multilevel perspective. Strategic Management Journal, 38(6), 1327–1342.
33.
HalabyC. N. (2004). Panel models in sociological research: Theory into practice. Annual Review of Sociology, 30, 507–544.
34.
HausmanJ. A. (1978). Specification tests in econometrics. Econometrica, 46(6), 1251–1271.
35.
HelsonR.JonesC.KwanV. S. Y. (2002). Personality change over 40 years of adulthood: Hierarchical linear modeling analyses of two longitudinal samples. Journal of Personality and Social Psychology, 83(3), 752–766.
36.
HofmannD. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23(6), 723–744.
37.
HofmannD. A.GavinM. B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24(5), 623–641.
38.
HolcombT. R.CombsJ. G.SirmonD. G.SextonJ. (2010). Modeling levels and time in entrepreneurship research: An illustration with growth strategies and post-IPO performance. Organizational Research Methods, 13(2), 348–389.
39.
KreftI. G.De LeeuwJ.AikenL. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30(1), 1–21.
40.
KromreyJ. D.Foster-JohnsonL. (1998). Mean centering in moderated multiple regression: Much ado about nothing. Educational and Psychological Measurement, 58(1), 42–67.
41.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
42.
La PortaR.Lopez-De-SilanesF.ShleiferA. (2008). The economic consequences of legal origins. Journal of Economic Literature, 46(2), 285–332.
43.
LarckerD. F.RusticusT. O. (2010). On the use of instrumental variables in accounting research. Journal of Accounting and Economics, 49(3), 186–205.
44.
LeeV. E. (2000). Using hierarchical linear modeling to study social contexts: The case of school effects. Educational Psychologist, 35(2), 125–141.
45.
LukeD. A. (2004). Multilevel modeling. Thousand Oaks, CA: Sage Publications.
46.
McNeishD.KelleyK. (2018). Fixed effects models versus mixed effects models for clustered data: Reviewing the approaches, disentangling the differences, and making recommendations. Psychological Methods, 24(1), 20–35.
47.
McNeishD.StapletonL. M.SilvermanR. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114–140.
48.
McNeishD.WentzelK. R. (2017). Accommodating small sample sizes in three-level models when the third level is incidental. Multivariate Behavioral Research, 52(2), 200–215.
49.
MundlakY. (1978). Pooling of time-series and cross-section data. Econometrica, 46(1), 69–85.
50.
NeuhausJ. M.KalbfleischJ. D. (1998). Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics, 54(2), 638–645.
51.
PetersenM. A. (2009). Estimating standard errors in finance panel data sets: Comparing approaches. Review of Financial Studies, 22(1), 435–480.
52.
PustejovskyJ. E.TiptonE. (2018). Small-sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models. Journal of Business & Economic Statistics, 36(4), 672–683.
53.
Rabe-HeskethS.SkrondalA. (2008). Multilevel and longitudinal modeling using Stata. College Station, TX: Stata Press.
54.
Rabe-HeskethS.SkrondalA. (2012). Multilevel and Longitudinal Modeling Using Stata (3rd ed). College Station, TX: Stata Press Publication.
55.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
56.
SachsJ. D. (2003, February). Institutions don’t rule—Direct effects of geography on per capita income (Working Paper 9490). Cambridge, MA: National Bureau of Economic Research.
57.
SchafferM. E.StillmanS. (2006). Xtoverid: Stata module to calculate tests of overidentifying restrictions after xtreg, xtivreg, xtivreg2 and xthtaylor. Retrieved fromhttp://ideas.repec.org/c/boc/bocode/s456779.html
58.
SchunckR. (2013). Within and between estimates in random-effects models: Advantages and drawbacks of correlated random effects and hybrid models. Stata Journal, 13(1), 65–76.
59.
StataCorp. (2017). Stata statistical software: Release 15. College Station, TX: Author.
60.
WooldridgeJ. M. (2002). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press.
61.
WooldridgeJ. M. (2013). Introductory econometrics: A modern approach (5th ed.). Mason, OH: South-Western Cengage Learning.
62.
YammarinoF. J.DansereauF. (2011). Multi-level issues in evolutionary theory, organization science, and leadership. The Leadership Quarterly, 22, 1042–1057.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
