Some suggestions for measuring marginal symmetry in agreement matrices for categorical data are discussed, together with measures of item-by-item agreement conditional on marginal asymmetry. Connections with intraclass correlations for dichotomous data are noted.
Get full access to this article
View all access options for this article.
References
1.
Bishop, Y. M. M. , Fienberg, S. E. and Holland, P. W. (1975). Discrete multivariate analysis. Cambridge: MIT Press.
2.
Brennan, R. L. and Prediger, D. J. (1981). Coefficient kappa: some uses, misuses and alternatives. Educational and Psychological Measurement, 41, 687-699.
3.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
4.
Fleiss, J. L. (1975). Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 31, 651-659.
5.
Fleiss, J. L. and Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational And Psychological Measurement, 33, 613-619.
6.
Goodman, L. A. and Kruskal, W. H. (1959). Measures of association for cross classifications. II: Further discussion and references. Journal of the American Statistical Association, 54, 123-163.
7.
Maxwell, A. E. and Pilliner, A. E. G. (1968). Deriving coefficients of reliability and agreement for ratings. British Journal of Mathematical and Statistical Psychology, 21, 105-116.
