Testing the Equality of Independent Alpha Coefficients Adjusted for Test Length

Abstract

In comparing the merits of different item types or test formats, test developers are often called on to make a rigorous comparison of the internal consistency reliabilities of the scores from two competing tests. Such comparisons should be made for tests of equal length in terms of time demands and speededness. If the experimental instruments used in the field trials were not equal in length, estimates of the reliabilities of scores on equal-length tests can be obtained by applying the Spearman-Brown formula. In this article, the authors develop a statistical test for the hypothesis that [.alpha]₁ ^[.minute] = [.alpha]₂ ^[.minute], where [.alpha]_i ^[.minute] is the extrapolated value of Cronbach’s alpha reliability for Test i. The test is shown to exercise tight control of Type I error.

Get full access to this article

View all access options for this article.

References

Alsawalmeh, Y. M. , & Feldt, L. S. (1994). A modification of Feldt’s test of the equality of two dependent alpha coefficients. Psychometrika, 59(1), 49-57.

Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357-370.

Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363-373.

Feldt, L. S. (1980). A test of the hypothesis that Cronbach’s alpha coefficient is the same for two tests administered to the same sample. Psychometrika, 49, 99-105.

Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (pp. 105-146). New York: Macmillan.

Hakstian, A. R. , & Whalen, T. E. (1976). A k-sample significance test for independent alpha co-efficients. Psychometrika, 41, 219-231.

Kendall, M. G. , & Stuart, A. (1977). The advanced theory of statistics (Vol. 1, 4th ed.). New York: Macmillan.

Kristof, W. (1963). The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika, 28, 221-238.

Novick, M. R. , & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1-13.

10.

Odell, P. L. , & Feiveson, A. N. (1966). A numerical procedure to generate a sample covariance matrix. American Statistical Association Journal, 61, 199-203.

11.

Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309-316.

12.

Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110-114.

13.

Scheffé’ H. (1959). The analysis of variance. New York: John Wiley.

14.

Woodruff, D. J. , & Feldt, L. S. (1986). Tests for equality of several alpha coefficients when their estimates are dependent. Psychometrika, 51, 393-413.