The Calculation of Reliability From a Split-Plot Factorial Design

Abstract

This paper treats the question, "How should one estimate the reliability of school (or classroom) means when persons are nested within schools (or classrooms)?" We begin by reviewing the use of variance components in the estimation of reliability from a randomized block (RB) design. Then we extend this rationale to the estimation of reliability (or generalizability) coefficients in a split-plot factorial (SPF) design with persons nested within schools.

Through the use of variance components from the SPF design, we derive estimates of reliability for schools and for persons within schools. Then we compare the reliability for persons within schools from a SPF design with the reliability for persons from a RB design. Finally, we compare the reliability for schools from a SPF design with the reliability for school means from a RB design.

Get full access to this article

View all access options for this article.

References

Brennan, R. L. and Kane, M. T. The generalizability of class means. Paper presented at the Annual Meeting of the American Educational Research Association, Washington, D.C, April, 1975.

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika, 1951, 16, 297-334.

Cronbach, L. J. , Gleser, G. C. , Nanda, H. , and Rajaratnam, N. The dependability of behavioral measurements. New York: Wiley, 1972.

Cronbach, L. J. , Rajaratnam, N. , and Gleser, G. C. Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 1963, 16, 137-163.

Hoyt C. Test reliability estimated by analysis of variance. Psychometrika, 1941, 6, 153-160.

Kirk, R. E. Experimental design: Procedures for the behavioral sciences. Belmont, California: Wadsworth, 1968.

Kuder, G. F. and Richardson, M. W. The theory of the estimation of test reliability . Psychometrika, 1937, 2, 151-160.

Lindquist, E. F. Design and analysis of experiments in psychology and education. Boston: Houghton Mifflin, 1953.

Millman, J. and Glass, G. V. Rules of thumb for writing the anova table. Journal of Educational Measurement, 1967, 4, 41-51.

10.

Stanley, J. C. Reliability. In R. L. Thorndike (Ed.), Educational Measurement (2nd ed.). Washington: American Council on Education, 1971. Pp. 356-442.