The analytical model typically used to perform generalizability analysis assumes that design effects are uncorrelated. Often, the assessment of behavioral data involves designs that employ multiple occa sions or repeated trials (as in many observational and rating studies). In these cases, design effects may be serially correlated. The implications of serially correlated effects on the results of gen eralizability analyses are discussed. Simulated data are provided that demonstrate the biases that serially correlated effects introduce into the results.
Boardman, T.J. (1974). Confidence intervals for variance components: A comparative Monte Carlo study. Biometrics, 30, 251-262.
2.
Box, G.E.P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems II: Effects of inequality of variance and of correlation between errors in the two way classification. Annals of Mathematical Statistics, 29, 885-891.
3.
Brennan, R.L. (1983). Elements of generalizability theory. Iowa City IA: The American College Testing Program .
4.
Brennan, R.L., & Kane, M.T. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 14, 277-289.
5.
Cronbach, L.J., & Gleser, G.C. (1964). The signal/noise ratio in the comparison of reliability coefficients. Educational and Psychological Measurement, 24, 467-480.
6.
Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for tests and profiles. New York: Wiley.
7.
Cronbach, L.J., Rajaratnam, N., & Gleser, G.C. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology , 16, 137-163.
8.
Crump, S.L. (1946). The estimation of variance components in analysis of variance. Biometrics, 2, 7-11.
9.
Eisenhart, C. (1947). The assumptions underlying analysis of variance . Biometrics, 3, 1-21.
10.
Gleser, G.C., Cronbach, L.J., & Rajaratnam, N. (1965). Generalizability of scores influenced by multiple sources of variance. Psychometrika , 30, 395-418.
11.
Henderson, C.R. (1953). Estimation of variance and covariance components . Biometrics, 9, 226-253.
12.
Lord, F.M., & Novick, M. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
13.
Maxwell, A.E. (1968). The effect of correlated errors on estimates of reliability coefficients. Educational and Psychological Measurement , 28, 803-811.
14.
Rowley, G.L. (1989). Assessing error in behavioral data: Problems of sequencing. Journal of Educational Measurement, 26, 273-284.
15.
Searle, S.R. (1970). Large sample variances of maximum likelihood estimators of variance components using unbalanced data. Biometrics, 26, 505-524.
16.
Searle, S.R. (1971). Linear models. New York: Wiley.
17.
Smith, P.L. (1978). Sampling errors of variance components in small sample generalizability studies. Journal of Educational Statistics , 3, 319-346.
18.
Smith, P.L. (1982). A confidence interval approach for variance component estimates in the context of generalizability theory. Educational and Psychological Measurement, 42, 459-466.
Suen, H.K., & Lee, P.S.C. (1987, April). Generalizability assessment of autocorrelated data via Box-Jenkins back forecasting. Paper presented at the annual meeting of the American Educational Research Association, Chicago IL.
