Performance of Empirical Bayes Estimators of Level-2 Random Parameters in Multilevel Analysis: A Monte Carlo Study for Longitudinal Designs

Abstract

Multilevel analysis is a useful technique for analyzing longitudinal data. To describe a person’s development across time, the quality of the estimates of the random coefficients, which relate time to individual changes in a relevant dependent variable, is of importance. The present study compares three estimators of the random coefficients: the Bayes estimator (BE), the empirical Bayes estimator (EBE), and the ordinary least squares estimator (OLSE). Using MLwiN, Monte Carlo simulations are carried out to study the performance of the estimators, while systematically varying the size of the sample as well as the number of measurement occasions. First, we examine for normally distributed random coefficients to what extent the EBE performs better than the OLSE and to what extent the EBE preserves the good properties of the BE. Second, we examine the robustness of the EBE which is based on a normal distribution of the random parameters, by comparing its performance to the OLSE for data being generated from two distributions other than the normal distribution: a modified t-distribution and a modified exponential distribution. As performance criteria we examine the Bayes risk as well as a criterion based on the frequentist notion of mean squared error.