Large sample chi square tests of the significance of the difference between two correlated kappas, which may be weighted or unweighted, are derived. Two cases are presented: one judge in common between the two kappas and no judge in common. An illustrative calculation is included.
Get full access to this article
View all access options for this article.
References
1.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
2.
Cohen, J. (1968). Weighted Kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.
3.
Cohen, J. (1972). Weighted Chi square: An extension of the Kappa method. Educational and Psychological Measurement, 32, 61-74.
4.
Fleiss, J. L. , Cohen, J., and Everitt, B. S. (1969). Large sample standard errors of Kappa and weighted Kappa. Psychological Bulletin, 72, 323-327.
5.
Rao, C. R. (1952). Advanced Statistical Methods in Biometric Research. New York: Wiley.
6.
Mannuzza, S. , Fyer, A. J., Martin, L. Y., Gallops, M. S., Endicott, J., Gorman, J., Liebowitz, M. R., and Klein, D. F. (1989). Reliability of anxiety assessment: I Diagnostic agreementArchives of General Psychiatry, 46, 1093-1101.
