A Confidence Interval Approach for Variance Component Estimates in the Context of Generalizability Theory

Abstract

The accuracy of the conclusions drawn from applications of generalizability theory are largely a function of the accuracy of the statistics resulting from a G study analysis. Previous research has shown that errors in variance component estimation due to sampling are often larger than one might suspect. This is particularly true when small samples are used and for complex designs such as multifaceted G studies. For such designs, confidence intervals for variance component estimates could be of great benefit to researchers using G theory. The present study uses Monte Carlo methods to explore the accuracy of a method for establishing confidence intervals for variance component estimates. The results indicate that the technique is only moderately accurate, but if used with caution can guide the researcher as to the accuracy of his/her estimates.

Get full access to this article

View all access options for this article.

References

Collins, J. D. Jackknifing generalizability. Unpublished doctoral dissertation, University of Colorado, 1970.

Cronbach, L. J. , Gleser, G. , Nanda, H. and Rajaratnam, N. The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley, 1972.

Graybill, F. A. An introduction to linear statistical models: Volume I. New York: McGraw-Hill, 1961.

Hartley, H. O. and Rao, J. N. K. Maximum likelihood estimation for the mixed analysis of variance model. Biometrika, 1967, 54, 93-108.

Henderson, C. P. Estimation of variance and covariance components . Biometrics, 1953, 9, 226-252.

Herbach, L. H. Properties of model IL type analysis of variance tests . Annals of Mathematical Statistics, 1959, 39, 939-959.

Klotz, J. H. , Milton, R. C. and Zachs, S. Mean square efficiency of estimates of variance components. Journal of the American Statistical Association, 1969, 64, 1383-1402.

Leone, F. C. and Nelson, L. L. Sampling distributions of variance components I. Empirical studies of balanced nested designs. Technometrics, 1966, 8, 457-468.

Miller, R. J. Jackknifing variances . Annals of Mathematical Statistics, 1968, 39, 567-582.

10.

Mosteller, R. and Tukey, J. W. Data analysis, including statistics. In G. Lindzey & E. Aronson (Eds.), Handbook of social psychology: Volume II. Reading, MA: Addision-Wesley, 1968.

11.

Satterthwaite, F. E. An approximate distribution of estimates of variance components . Biometrics, 1946, 2, 110-114.

12.

Scheffe, H. The analysis of variance. New York: Wiley, 1959.

13.

Searle, S. R. and Henderson, C. R. Computing procedures for estimating components of variance in the two way classification mixed model. Biometrics, 1961, 17, 607-616.

14.

Smith, P. L. Sampling errors of variance component estimates in small sample multifacet generalizability studies . Journal of Educational Statistics, 1978, 3, 319-346.

15.

Thompson, W. A. The problem of negative estimates of variance components . Annals of Mathematical Statistics, 1962, 33, 273-289.

16.

Wang, Y. Y. A comparison of several variance component estimators . Biometrika, 1967, 54, 301-305.

17.

Welch, B. L. On linear combinations of several variances . Journal of the American Statistical Association, 1956, 51, 132-148.

18.

Yates, F. The analysis of multiple classification with unequal numbers in the different classes . Journal of the American Statistical Association, 1934, 29, 51-66.