Advancing Theory by Assessing Boundary Conditions With Metaregression: A Critical Review and Best-Practice Recommendations

Abstract

Understanding boundary conditions, or situations when relations between variables change depending on values of other variables, is critical for theory advancement and for providing guidance for practice. Metaregression is ideally suited to investigate boundary conditions because it provides information on the presence and strength of such conditions. In spite of its potential, results of our review of 63 metaregression articles published in the Journal of Management, Journal of Applied Psychology, Personnel Psychology, Journal of Management, Academy of Management Journal, and Strategic Management Journal uncovered a surprising lack of transparency, frequently implemented erroneous practices, and a lack of attention to important methodological choices. Results also suggest that many substantive conclusions are ambiguous at best and, unbeknownst to authors and readers, potentially misleading. Drawing from our review of the substantive literature as well as the latest statistical and methodological research, we offer evidence-based best-practice recommendations on how to conduct and report the results of a metaregression study. We offer recommendations on calculating statistical power and heterogeneity, choosing an appropriate model, testing boundary condition hypotheses, adjusting R² for known variance, explaining methodological choices, and reporting and interpreting model coefficients and other results. Also, we conducted two illustrative metaregression studies that incorporate all of our recommendations with accompanying syntax and data. Our recommendations can be used by authors, readers, journal editors, and reviewers wishing to conduct and evaluate metaregression studies, as well as practitioners interested in understanding conditions under which organizational practices are more or less likely to be effective.