Correcting Fixed Effect Standard Errors When a Crossed Random Effect Was Ignored for Balanced and Unbalanced Designs

Abstract

Previous studies have detailed the consequence of ignoring a level of clustering in multilevel models with straightly hierarchical structures and have proposed methods to adjust for the fixed effect standard errors (SEs). However, in behavioral and social science research, there are usually two or more crossed clustering levels, such as when students are cross-classified by schools and neighborhoods, yet it is not uncommon that researchers focus only on one level of clustering. Using the generalized least squares framework, in this study, we derive the bias in the fixed effect SE estimators when one crossed random effect is omitted. We then showed, using data from the Scotland Neighborhood Study, how one can correct for the SEs and obtain corrected statistical inference when a misspecified two-level model was used in a primary study, which is useful when evaluating observational studies or cluster randomized trials that ignored a crossed random effects or when conducting meta-analyses. In addition, our analytic results provide theoretical insights on how one can quantify imbalance with cross-classified data by the strength of association between the two-crossed random effects in a contingency table and how the degree of imbalance relates to the correction factor for the fixed effect SEs.