r2_mlm: A command for computing R-squared measures for models fit by mixed

Abstract

The article by Rights and Sterba (2023, Multivariate Behavioral Research 58: 340-367) provides a comprehensive framework for computing $R^{2}$ measures for (linear) multilevel models. In this article, we introduce r2_mlm, a postestimation command for mixed that computes $R^{2}$ measures using Rights and Sterba’s framework. We explain how this $R^{2}$ framework works and demonstrate how r2_mlm can be used to compute various $R^{2}$ measures for models fit by mixed. One of the most useful features of r2_mlm is that it will produce warning messages if the user specifies the model in a way that may lead to conflation bias (an easily overlooked issue). Finally, we walk through a simple example and explain how to interpret the various $R^{2}$ measures.

Keywords

st0789 r2_mlm mixed multilevel modeling explained variation conflation bias

Get full access to this article

View all access options for this article.

References

Aguinis

Culpepper

S. A.

. 2015. An expanded decision-making procedure for examining cross-level interaction effects with multilevel modeling. Organizational Research Methods 18: 155–176. 10.1177/1094428114563618.

Allison

P. D

. 2009. Fixed Effects Regression Models. Thousand Oaks, CA: Sage. 10.4135/9781412993869.

Cronbach

L. J.

1976. Research on classrooms and schools: Formulation of questions, design, and analysis. Occasional paper, Stanford Evaluation Consortium. https://files.eric.ed.gov/fulltext/ED135801.pdf .

Guo

Dhaliwal

Rights

J. D.

. 2023. Disaggregating level-specific effects in cross-classified multilevel models. Behavior Research Methods 56: 3023–3057. 10.3758/s13428-023-02238-7.

Jann

2004. estout: Stata module to make regression tables. Statistical Software Components S439301, Department of Economics, Boston College. https://ideas.repec.org/c/boc/bocode/s439301.html .

------------. 2007. Stata tip 44: Get a handle on your sample. Stata Journal 7: 266–267. 10.1177/1536867X0700700210.

Johnson

P. C. D

. 2014. Extension of Nakagawa & Schielezeth’s RG_lmm to random slopes models. Methods in Ecology and Evolution 5: 944–946. 10.1111/2041-210X.12225.

Mundlak

. 1978. On the pooling of time series and cross section data. Econometrica 46: 69–85. 10.2307/1913646.

Nakagawa

Schielzeth

. 2013. A general and simple method for obtaining R² from generalized linear mixed-effects models. Methods in Ecology and Evolution 4: 133–142. 10.1111/j.2041-210x.2012.00261.x.

10.

Preacher

K. J.

Zyphur

M. J.

Zhang

. 2010. A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods 15: 209–233. 10.1037/a0020141.

11.

Raudenbush

S. W.

Bryk

A. S.

. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods. Vol. 1 of Advanced Quantitative Techniques in the Social Sciences, 2nd ed. Thousand Oaks, CA: Sage.

12.

Rights

J. D

. 2023. Aberrant distortion of variance components in multilevel models under conflation of level-specific effects. Psychological Methods 28: 1154–1177. 10.1037/met0000514.

13.

Rights

J. D.

Sterba

S. K.

. 2019. Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological Methods 24: 309–338. 10.1037/met0000184.

14.

------------. 2020. New recommendations on the use of R-squared differences in multilevel model comparisons. Multivariate Behavioral Research 55: 568–599. 10.1080/00273171.2019.1660605.

15.

------------. 2021. Effect size measures for longitudinal growth analyses: Extending a framework of multilevel model R-squareds to accommodate heteroscedasticity, autocorrelation, nonlinearity, and alternative centering strategies. New Directions for Child and Adolescent Development 2021: 65–110. 10.1002/cad.20387.

16.

------------. 2023a. On the common but problematic specification of conflated random slopes in multilevel models. Multivariate Behavioral Research 58: 1106–1133. 10.1080/00273171.2023.2174490.

17.

------------. 2023b. R-squared measures for multilevel models with three or more levels. Multivariate Behavioral Research 58: 340–367. 10.1080/00273171.2021.1985948.

18.

Shaw

Rights

J. D.

Sterba

S. K.

Flake

J. K.

. 2022. r2mlm: An R package calculating R-squared measures for multilevel models. Behavior Research Methods 55: 1942–1964. 10.3758/s13428-022-01841-4.

19.

Snijders

T. A. B.

Bosker

R. J.

. 2012. Multilevel Analysis: An Introduction to Basic and Advance Multilevel Modeling. 2nd ed. Thousand Oaks, CA: Sage.

20.

Vonesh

E. F.

Chinchilli

V. M.

. 1997. Linear and Nonlinear Models for the Analysis of Repeated Measurements. Vol. 154 of Statistics: A Series of Textbooks and Monographs. New York: Marcel Dekker. 10.1201/9781482293272.

21.

. 2003. Measuring explained variation in linear mixed effects models. Statistics in Medicine 22: 3527–3541. 10.1002/sim.1572.

22.

Zhang

Hedeker

. 2022. Defining R-squared measures for mixed-effects location scale models. Statistics in Medicine 41: 4285–4483. 10.1002/sim.9521.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB