Sensitivity Analysis for Fixed Effects in the Hierarchical Model: A Gibbs Sampling Approach

Abstract

Many recent applications of the two-level hierarchical model (HM) have focused on drawing inferences concerning fixed effects—that is, structural parameters in the Level 2 model that capture the way Level 1 parameters (e.g., children’s rates of cognitive growth, within-school regression coefficients) vary as a function of Level 2 characteristics (e.g., children’s home environments and educational experiences; school policies, practices, and compositional characteristics). Under standard assumptions of normality in the HM, point estimates and intervals for fixed effects may be sensitive to outlying Level 2 units (e.g., a child whose rate of cognitive growth is unusually slow or rapid, a school at which students achieve at an unusually high level given their background characteristics, etc.). A Bayesian approach to studying the sensitivity of inferences to possible outliers involves recalculating the marginal posterior distributions of parameters of interest under assumptions of heavy tails, which has the effect of downweighting extreme cases. The goal is to study the extent to which posterior means and intervals change as the degree of heavy-tailedness assumed increases. This strategy is implemented in the HM setting via a new Monte Carlo technique termed Gibbs sampling (Gelfand & Smith, 1990; cf Tanner & Wong, 1987) and is illustrated through reanalyses of the data from a study of vocabulary growth in children (Huttenlocher, Haight, Bryk, Seltzer, & Lyons, 1991). The Gibbs sampling approach presented builds on the work of Dempster, Laird, and Rubin (1980); Little and Rubin (1987); and Lange, Little, and Taylor (1989) concerning the use of the t distribution in robust statistical estimation. Extensions of this approach are discussed in the final section of the article.