Reliability is a concept central to testing. Whatever measure of reliability is used, its value will always have to be estimated from a sample and will therefore be subject to sampling variation. The magnitude of sampling error involved in measuring reliability from a sample should be indicated by quoting the standard error of a reliability estimate or by giving a confidence interval for its true value. This paper considers several simple ANOVA models which can be used to define and estimate reliability coefficients. It discusses the statistical properties of commonly used measures of reliability and explains how to test hypotheses about their true values.
Get full access to this article
View all access options for this article.
References
1.
Bolus, R.E., Hinofotis, B. and Bailey, K.M.1982: An introduction to generalizability theory in second language research. Language Learning32, 245-57.
2.
Feldt, L.S.1965: The approximate sampling distribution of Kuder-Richardson reliability coefficient 20. Psychometrika30, 357-70.
3.
1969: A test of the hypothesis that Cronbach's alpha or Kuder-Richardson coefficient 20 is the same for two tests. Psychometrika34, 363-73.
4.
1975: Estimation of the reliability of a test divided into two parts of unequal length. Psychometrika40, 557-61.
5.
1980: A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample. Psychometrika45,99-105.
6.
Hakistian, A.R.L. and Whalen, T.E.1976: A k-sample significance test for independent alpha coefficients . Psychometrika41, 219-31.
7.
Hunter, J.E.1968: Probabilistic foundations for coefficients of generalizability . Psychometrika33, 1-18.
8.
Kraemer, H.C.1975: On estimation and hypothesis testing problems for correlation coefficients. Psychometrika40, 473-85.
9.
1979: Tests of homogeneity of independent correlation coefficients . Psychometrika44, 329-35.
10.
1981: Extension of Feldt's approach to testing the homogeneity of coefficients of reliability. Psychometrika46, 41-45.
11.
Kristof, W.1963: The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika28,221-38.
12.
1970: On the sampling theory of reliability estimation. Journal of Mathematical Psychology7, 371-77.
13.
1974: Estimation of reliability and true score variance from a split of a test into three arbitrary parts. Psychometrika39, 491-99.
14.
Lord, F.M.1974: Variance stabilising transformation of the stepped-up reliability coefficient. Journal of Educational Measurement11, 55-57.
15.
Novick, M.R. and Lewis, C.1967: Coefficient alpha and the relaibility of composite measurements . Psychometrika32, 1-13.
16.
Pandey, T.N. and Hubert, L.1975: An empirical comparison of several interval estimation procedures for coefficient alpha. Psychometrika40, 169-81.
17.
Payne, W.H. and Anderson, D.E.1968: Significance levels for the Kuder- Richardson 20: an automated sampling experiment approach. Educational and Psychological Measurement28, 23-39.
18.
Zimmerman, D.W.1972: Test reliability and the Kuder-Richardson formulas: derivation from probabilistic theory. Educational and Psychological Measurement32, 939-54.
