A Confidence Interval Approach for Variance Component Estimates in the Context of Generalizability Theory

Abstract

The accuracy of the conclusions drawn from applications of generalizability theory are largely a function of the accuracy of the statistics resulting from a G study analysis. Previous research has shown that errors in variance component estimation due to sampling are often larger than one might suspect. This is particularly true when small samples are used and for complex designs such as multifaceted G studies. For such designs, confidence intervals for variance component estimates could be of great benefit to researchers using G theory. The present study uses Monte Carlo methods to explore the accuracy of a method for establishing confidence intervals for variance component estimates. The results indicate that the technique is only moderately accurate, but if used with caution can guide the researcher as to the accuracy of his/her estimates.