The Effects of Sampling Model on Inference with Coefficient Alpha

Abstract

The distinction between Type I and Type 12 sampling in connection with measurement data is discussed, and a method is presented for simulating data arising from Type 12 sampling. Next, an empirical study is described in which the perfornance of inferential procedures for coefficient alpha was examined when these procedures were applied with data arising from both kinds of sampling. The results indicate that as long as the data are drawn from a population exhibiting strict essential parallelism among the conditions (an assumption underlying the procedures), the methods maintain precise confidence level control under Type 12 sampling. For data departing from essential parallelism, however, these procedures are not robust under Type 12 sampling. In contrast, actual confidence coefficient values were close to nominal levels under Type I sampling. Implications are noted for practice and for future research on inference with coefficient alpha.

Get full access to this article

View all access options for this article.

References

Alsawalmeh, J. M. , & Feldt, L. S. (1992). Test of the hypothesis that the intraclass reliability coefficient is the same for two measurement processes. Applied Psychological Measurement, 16, 195-205.

Barchard, K. A. , & Hakstian, A. R. (1997). The robustness of confidence intervals for coefficient alpha under violation of the assumption of essential parallelism. Multivariate Behavioral Research, 32, 169-191.

Bay, K. S. (1973). The effect of non-normality on the sampling distribution and standard error of reliability coefficient estimates under an analysis of variance model. British Journal of Mathematical and Statistical Psychology, 26, 45-57.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.

Cronbach, L. J. , Rajaratnam, N. , & Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16, 137-163.

Eom, H. J. (1993). The interactive effects of data categorization and noncircularity on the sampling distribution of generalizability coefficients in analysis of variance models: An empirical investigation. Unpublished doctoral dissertation, University of British Columbia, Vancouver.

Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357-370.

Feldt, L. S. (1969). A test of the hypothesis that Cronbach's alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363-373.

Feldt, L. S. (1980). A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample. Psychometrika, 45, 99-105.

10.

Hakstian, A. R. , & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219-231.

11.

Hoyt, C. (1941). Test reliability estimated by analysis of variance. Psychometrika, 6, 153-160.

12.

Kristof, W. (1963). The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika, 28, 221-238.

13.

Kuder, G. F. , & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151-160.

14.

Lord, F. M. (1955). Sampling fluctuations resulting from the sampling of test items. Psychometrika, 20, 1-22.

15.

Novick, M. R. , & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1-13.

16.

Woodruff, D. J. , & Feldt, L. S. (1986). Tests for equality of several alpha coefficients when their sample estimates are dependent. Psychometrika, 51, 393-413.