Dealing With Dependence (Part I): Understanding the Effects of Clustered Data

Abstract

This article provides a conceptual introduction to the issues surrounding the analysis of clustered (nested) data. We define the intraclass correlation coefficient (ICC) and the design effect, and we explain their effect on the standard error. When the ICC is greater than 0, then the design effect is greater than 1. In such a scenario, the standard error produced under the assumption of independence is underestimated. This increases the Type I error rate. We provide a short illustration of the effect of non-independence on the standard error. We show that after accounting for the design effect, our decision about the statistical significance of the test statistic changes. When we fail to account for the clustered nature of the data, we conclude that the difference between the two groups is statistically significant. However, once we adjust the standard error for the design effect, the difference is no longer statistically significant.