Hierarchical Dependence in Meta-Analysis

Abstract

Meta-analysis is a frequent tool among education and behavioral researchers to combine results from multiple experiments to arrive at a clear understanding of some effect of interest. One of the traditional assumptions in a meta-analysis is the independence of the effect sizes from the studies under consideration. This article presents a meta-analytic review of 13 experiments with 18 study reports all involving the effect of native-language (L1) vocabulary aids on second-language (L2) reading comprehension. Some experiments produced multiple study reports, creating a dependence structure among the resulting effect size estimates. The covariance among these effect size estimates is estimated and incorporated into a proposed meta-analysis model that accounts for the dependence at a hierarchical level. The overall effect size estimate (g =.63) indicates that L1 vocabulary aids can be an effective L2 reading comprehension aid in the short term. An interpretation of the hierarchical components is discussed.