A paradoxical phenomenon of decreases in reliability as the number of elements averaged over increases is shown to be possible in multifacet reliability procedures (intraclass correlations or generalizability coefficients). Conditions governing this phenomenon are presented along with implications and cautions.
Get full access to this article
View all access options for this article.
References
1.
1. Wallander, J. L. Unpublished doctoral dissertation, Purdue University, 1981.
2.
Bartko, J. J.On various intraclass reliability coefficients . Psychological Bulletin, 1976, 83, 762-765.
3.
Cronbach, L. J. , Gleser, G. C., Nanda, H. and Rajaratnam, N.The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley, 1972.
4.
Gleser, G. C. , Cronbach, L. J., and Rajaratnam, N.Generalizability of scores influenced by multiple sources of variance. Psychometrika, 1965, 40, 395-418.
5.
Haggard, E. A.Intraclass correlation and the analysis of variance. New York: Dryden Press, 1958.
6.
Lumsden, J.Test theory . Annual Review of Psychology, 1976, 27, 251-280.
7.
Mitchell, S. K.Interobserver agreement, reliability, and generalizability of data collected in observational studies . Psychological Bulletin, 1979, 86, 376-390.
8.
Shrout, P. E. and Fleiss, J. L.Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 1979, 86, 420-428.
