This article describes a SAS MACRO for computing various indices of interrater agreement, including a new generalizability coefficient, for categorical data in a single-facet, crossed design.
Get full access to this article
View all access options for this article.
References
1.
Cicchetti, D. V. , & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127-137.
2.
Fleiss, J. L. (1971). Measuring nominal scales agreement among many raters. Psychological Bulletin, 76, 378-382.
3.
Fleiss, J. L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: John Wiley.
4.
Li, M. N. , & Lautenschlager, G. J. (1997). Generalizability theory applied to categorical data. Educational and Psychological Measurement, 57, 813-822.
5.
Light, R. J. , & Margolin, B. H. (1971). An analysis of variance for categorical data. Journal of the American Statistical Association, 66, 534-544.
6.
Rae, G. (1988). The equivalence of multiple rater kappa statistics and intraclass correlation coefficients. Educational and Psychological Measurement, 48, 367-374.
