Sage Journals: Discover world-class research

Abstract

The presence of clustered data is common in the sociobehavioral sciences. One approach that specifically deals with clustered data but has seen little use in education is the generalized estimating equations (GEEs) approach. We provide a background on GEEs, discuss why it is appropriate for the analysis of clustered data, and provide worked examples using both continuous and binary outcomes. Comparisons are made between GEEs, multilevel models, and ordinary least squares results to highlight similarities and differences between the approaches. Detailed walkthroughs are provided using both R and SPSS Version 26.

Keywords

generalized estimating equations GEEs clustered data population average models

Get full access to this article

View all access options for this article.

References

Agresti

(2007). An introduction to categorical data analysis (2nd ed.). John Wiley.

Allison

P. D.

(2005). Fixed effects regression methods for longitudinal data using SAS. SAS Institute.

Ballinger

G. A.

(2004). Using generalized estimating equations for longitudinal data analysis. Organizational Research Methods, 7(2), 127–150. https://doi.org/10.1177/1094428104263672

Bates

D. M.

(2010). lme4: Mixed-effects modeling with R. Springer.

Bates

D. M.

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bauer

D. J.

Sterba

S. K.

(2011). Fitting multilevel models with ordinal outcomes: Performance of alternative specifications and methods of estimation. Psychological Methods, 16(4), 373–390. https://doi.org/10.1037/a0025813

Bliese

P. D.

Hanges

P. J.

(2004). Being both too liberal and too conservative: The perils of treating grouped data as though they were independent. Organizational Research Methods, 7(4), 400–417.

Cameron

A. C.

Miller

D. L.

(2015). A practitioner’s guide to cluster-robust inference. Journal of Human Resources, 50(2), 317–372. https://doi.org/10.3368/jhr.50.2.317

Carey

(2019). yags: Yet another GEE solver. https://yags.r-forge.r-project.org/

10.

Carey

Lumley

Ripley

(2019). gee: Generalized estimation equation solver. https://CRAN.R-project.org/package=gee

11.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Associates.

12.

Crespi

C. M.

Wong

W. K.

(2011). A new dependence parameter approach to improve the design of cluster randomized trials with binary outcomes. Clinical Trials: Journal of the Society for Clinical Trials, 8(6), 687–698. https://doi.org/10.1177/1740774511423851

13.

Cui

Qian

(2007). Selection of working correlation structure and best model in GEE analyses of longitudinal data. Communications in Statistics—Simulation and Computation, 36(5), 987–996. https://doi.org/10.1080/03610910701539617

14.

Dahmen

Ziegler

(2004). Generalized estimating equations in controlled clinical trials: Hypotheses testing. Biometrical Journal, 46(2), 214–232. https://doi.org/10.1002/bimj.200310018

15.

Dedrick

R. F.

Ferron

J. M.

Hess

M. R.

Hogarty

K. Y.

Kromrey

J. D.

Lang

T. R.

Niles

J. D.

Lee

R. S.

(2009). Multilevel modeling: A review of methodological issues and applications. Review of Educational Research, 79(1), 69–102.

16.

Enders

C. K.

Keller

B. T.

Levy

(2018). A fully conditional specification approach to multilevel imputation of categorical and continuous variables. Psychological Methods, 23(2), 298.

17.

Feng

Diehr

Peterson

McLerran

(2001). Selected statistical issues in group randomized trials. Annual Review of Public Health, 22(1), 167–187. https://doi.org/10.1146/annurev.publhealth.22.1.167

18.

Fieberg

Rieger

R. H.

Zicus

M. C.

Schildcrout

J. S.

(2009). Regression modelling of correlated data in ecology: Subject-specific and population averaged response patterns. Journal of Applied Ecology, 46(5), 1018–1025. https://doi.org/10.1111/j.1365-2664.2009.01692.x

19.

Fitzmaurice

G. M.

Laird

N. M.

Rotnitzky

A. G.

(1993). Regression models for discrete longitudinal responses. Statistical Science, 8(3), 284–299.

20.

Fox

(2016). Applied regression analysis and generalized linear models (3rd ed.). Sage.

21.

Ghisletta

Spini

(2004). An introduction to generalized estimating equations and an application to assess selectivity effects in a longitudinal study on very old individuals. Journal of Educational and Behavioral Statistics, 29(4), 421–437. https://doi.org/10.3102/10769986029004421

22.

Grund

Lüdtke

Robitzsch

(2018). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods, 21(1), 111–149. https://doi.org/10.1177/1094428117703686

23.

Grund

Robitzsch

Luedtke

(2019). mitml: Tools for multiple imputation in multilevel modeling. https://CRAN.R-project.org/package=mitml

24.

Hardin

Hilbe

(2013). Generalized estimating equations (2nd ed.). CRC Press.

25.

Hedeker

du Toit

S. H. C.

Demirtas

Gibbons

R. D.

(2018). A note on marginalization of regression parameters from mixed models of binary outcomes: A note on marginalization. Biometrics, 74(1), 354–361. https://doi.org/10.1111/biom.12707

26.

Hilbe

J. M.

Robinson

A. P.

(2013). Methods of statistical model estimation. CRC Press.

27.

Hin

L.-Y.

Carey

V. J.

Wang

Y.-G.

(2007). Criteria for working–correlation–structure selection in GEE: Assessment via simulation. The American Statistician, 61(4), 360–364. https://doi.org/10.1198/000313007X245122

28.

Hin

L.-Y.

Wang

Y.-G.

(2009). Working-correlation-structure identification in generalized estimating equations. Statistics in Medicine, 28(4), 642–658.

29.

Højsgaard

Halekoh

Yan

(2005). The R package geepack for generalized estimating equations. Journal of Statistical Software, 15(1), 1–11. https://doi.org/10.18637/jss.v015.i02

30.

Horton

N. J.

Lipsitz

S. R.

(1999). Review of software to fit generalized estimating equation regression models. The American Statistician, 53(2), 160. https://doi.org/10.2307/2685737

31.

Hox

J. J.

Moerbeek

van de Schoot

(2018). Multilevel analysis: Techniques and applications (3rd ed.). Routledge.

32.

Huang

(2014). Analyzing group level effects with clustered data using Taylor series linearization. Practical Assessment, Research, and Evaluation, 19(13), 1–9.

33.

Huang

(2016). Alternatives to multilevel modeling for the analysis of clustered data. Journal of Experimental Education, 84, 175–196. https://doi.org/10.1080/00220973.2014.952397

34.

Huang

(2018a). Multilevel modeling and ordinary least squares regression: How comparable are they? Journal of Experimental Education, 86, 265–281. https://doi.org/10.1080/00220973.2016.1277339

35.

Huang

(2018b). Multilevel modeling myths. School Psychology Quarterly, 33(3), 492–499. https://doi.org/10.1037/spq0000272

36.

Huang

(2018c). Using cluster bootstrapping to analyze nested data with a few clusters. Educational and Psychological Measurement, 78, 297–318. https://doi.org/10.1177/0013164416678980

37.

Huang

(in press). Using cluster robust standard errors when analyzing group randomized trials with few clusters. Behavior Research Methods.

38.

Hubbard

A. E.

Ahern

Fleischer

N. L.

Laan

M. V. der

Lippman

S. A.

Jewell

Bruckner

Satariano

W. A.

(2010). To GEE or not to GEE: Population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology, 21(4), 467–474. https://doi.org/10.1097/EDE.0b013e3181caeb90

39.

Ivers

N. M.

Taljaard

Dixon

Bennett

McRae

Taleban

Skea

Brehaut

J. C.

Boruch

R. F.

Eccles

M. P.

Grimshaw

J. M.

Weijer

Zwarenstein

Donner

(2011). Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: Review of random sample of 300 trials, 2000-8. British Medical Journal, 343, d5886. https://doi.org/10.1136/bmj.d5886

40.

Kleinbaum

D. G.

Klein

(2010). Logistic regression. Springer. https://doi.org/10.1007/978-1-4419-1742-3_14

41.

LaHuis

D. M.

Ferguson

M. W.

(2009). The accuracy of significance tests for slope variance components in multilevel random coefficient models. Organizational Research Methods, 12(3), 418–435.

42.

Laird

N. M.

Ware

J. H.

(1982). Random-effects models for longitudinal data. Biometrics, 38(4), 963–974.

43.

Redden

D. T.

(2015). Small sample performance of bias-corrected sandwich estimators for cluster-randomized trials with binary outcomes. Statistics in Medicine, 34(2), 281–296. https://doi.org/10.1002/sim.6344

44.

Liang

K.-Y.

Zeger

S. L.

(1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22.

45.

Long

J. A.

(2020). jtools: Analysis and presentation of social scientific data. https://cran.r-project.org/package=jtools

46.

Luke

D. A.

(2004). Multilevel modeling. Sage.

47.

Maas

Hox

J. J.

(2004). The influence of violations of assumptions on multilevel parameter estimates and their standard errors. Computational Statistics & Data Analysis, 46(3), 427–440.

48.

Maas

Hox

J. J.

(2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92.

49.

Mancl

L. A.

DeRouen

T. A.

(2001). A covariance estimator for GEE with improved small-sample properties. Biometrics, 57(1), 126–134.

50.

McDaniel

L. S.

Henderson

N. C.

Rathouz

P. J.

(2013). Fast pure R implementation of GEE: Application of the matrix package. The R Journal, 5(1), 181–187.

51.

McNeish

D. M.

(2014). Modeling sparsely clustered data: Design-based, model-based, and single-level methods. Psychological Methods, 19, 552–563. https://doi.org/10.1037/met0000024

52.

McNeish

D. M.

(2019). Effect partitioning in cross-sectionally clustered data without multilevel models. Multivariate Behavioral Research, 54(6), 1–20.

53.

McNeish

Stapleton

L. M.

(2016). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51, 495–518. https://doi.org/10.1080/00273171.2016.1167008

54.

McNeish

Stapleton

L. M.

Silverman

R. D.

(2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22, 114–140. http://dx.doi.org/10.1037/met0000078

55.

McPherson

Barbosa-Leiker

McDonell

Howell

Roll

(2013). Longitudinal missing data strategies for substance use clinical trials using generalized estimating equations: An example with a buprenorphine trial. Human Psychopharmacology, 28(5), 506–515. https://doi.org/10.1002/hup.2339

56.

Murnane

R. J.

Willett

J. B.

(2011). Methods matter: Improving causal inference in educational and social science research. Oxford University Press.

57.

Musca

S. C.

Kamiejski

Nugier

Méot

Er-Rafiy

Brauer

(2011). Data with hierarchical structure: Impact of intraclass correlation and sample size on type-I error. Frontiers in Psychology, 2, 1–6. https://doi.org/10.3389/fpsyg.2011.00074

58.

Muth

Bales

K. L.

Hinde

Maninger

Mendoza

S. P.

Ferrer

(2016). Alternative models for small samples in psychological research: Applying linear mixed effects models and generalized estimating equations to repeated measures data. Educational and Psychological Measurement, 76(1), 64–87.

59.

Muthén

(2008). Mplus discussion: Adjusting standard errors for cluster sampling. http://www.statmodel.com/discussion/messages/12/305.html?1532439573

60.

Norton

E. C.

Bieler

G. S.

Ennett

S. T.

Zarkin

G. A.

(1996). Analysis of prevention program effectiveness with clustered data using generalized estimating equations. Journal of Consulting and Clinical Psychology, 64(5), 919.

61.

Pan

(2001). Akaike’s information criterion in generalized estimating equations. Biometrics, 57(1), 120–125.

62.

Pedroza

Truong

V. T. T.

(2017). Estimating relative risks in multicenter studies with a small number of centers—which methods to use? A simulation study. Trials, 18(1), 512. https://doi.org/10.1186/s13063-017-2248-1

63.

Pekár

Brabec

(2018). Generalized estimating equations: A pragmatic and flexible approach to the marginal GLM modelling of correlated data in the behavioural sciences. Ethology, 124(2), 86–93. https://doi.org/10.1111/eth.12713

64.

Pustejovsky

Tipton

(2016). Small-sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models. Journal of Business & Economic Statistics, 1–12. https://doi.org/10.1080/07350015.2016.1247004

65.

Raudenbush

Bryk

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage.

66.

Raudenbush

Bryk

Congdon

(2013). HLM (7.01) [Computer software]. Scientific Software International Inc.

67.

R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

68.

Rizopoulos

(2020). GLMMadaptive: Generalized linear mixed models using adaptive Gaussian quadrature. https://CRAN.R-project.org/package=GLMMadaptive

69.

Rogers

Stoner

(2015). Modification of the sandwich estimator in generalized estimating equations with correlated binary outcomes in rare event and small sample settings. American Journal of Applied Mathematics and Statistics, 3(6), 243–251.

70.

Rotnitzky

Jewell

N. P.

(1990). Hypothesis testing of regression parameters in semiparametric generalized linear models for cluster correlated data. Biometrika, 77(3), 485–497.

71.

Rubin

D. B.

(2004). Multiple imputation for nonresponse in surveys (Vol. 81). John Wiley.

72.

Salazar

Ojeda

Dueñas

Fernández

Failde

(2016). Simple generalized estimating equations (GEEs) and weighted generalized estimating equations (WGEEs) in longitudinal studies with dropouts: Guidelines and implementation in R. Statistics in Medicine, 35(19), 3424–3448. https://doi.org/10.1002/sim.6947

73.

Shults

Sun

Kim

Amsterdam

Hilbe

J. M.

Ten-Have

(2009). A comparison of several approaches for choosing between working correlation structures in generalized estimating equation analysis of longitudinal binary data. Statistics in Medicine, 28(18), 2338–2355. https://doi.org/10.1002/sim.3622

74.

Singer

J. D.

(1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23(4), 323–355. https://doi.org/10.2307/1165280

75.

Snijders

Bosker

(2012). Multilevel analysis (2nd ed.). Sage.

76.

Subramanian

S. V.

O’Malley

A. J.

(2010). Modeling neighborhood effects: The futility of comparing mixed and marginal approaches. Epidemiology, 21(4), 475–478. https://doi.org/10.1097/EDE.0b013e3181d74a71

77.

Tabachnick

B. G.

Fidell

(2019). Using multivariate statistics (7th ed.). Pearson.

78.

Wang

Kong

Zhang

(2016). Covariance estimators for Generalized Estimating Equations (GEE) in longitudinal analysis with small samples. Statistics in Medicine, 35(10), 1706–1721. https://doi.org/10.1002/sim.6817

79.

White

(1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817–838. https://doi.org/10.2307/1912934

80.

Crespi

C. M.

Wong

W. K.

(2012). Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemporary Clinical Trials, 33(5), 869–880.

81.

Xue

Zhang

Wang

(2019). An R package for model fitting, model selection and the simulation for longitudinal data with dropout missingness. Communications in Statistics - Simulation and Computation, 48(9), 2812–2829. https://doi.org/10.1080/03610918.2018.1468457

82.

Yang

Zhang

(2017). The performances of several modified CIC criteria for working intra-cluster correlation structure selection in GEE analysis. Communications in Statistics—Simulation and Computation, 46(8), 6034–6048. https://doi.org/10.1080/03610918.2016.1189565

83.

Yelland

L. N.

Salter

A. B.

Ryan

(2011). Relative risk estimation in cluster randomized trials: A comparison of generalized estimating equation methods. The International Journal of Biostatistics, 7(1), 1–26. https://doi.org/10.2202/1557-4679.1323

84.

Zeger

S. L.

Liang

K.-Y.

(1986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42(1), 121–130.

85.

Zeger

S. L.

Liang

K.-Y.

Albert

P. S.

(1988). Models for longitudinal data: A generalized estimating equation approach. Biometrics, 44(4), 1049–1060.

86.

Zeileis

(2006). Object-oriented computation of sandwich estimators. Journal of Statistical Software, 16(9), 1–16.

87.

Zorn

C. J.

(2001). Generalized estimating equation models for correlated data: A review with applications. American Journal of Political Science, 45(2), 470–490.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.65 MB

0.02 MB

Analyzing Cross-Sectionally Clustered Data Using Generalized Estimating Equations

Abstract

Keywords

Get full access to this article

References

Supplementary Material