This program computes specific category agreement levels for both nominally and ordinally scaled data. For ordinally scaled data, an option is available for collapsing the original scale to a smaller number of categories, with the goal of improving the level of inter-rater reliability for the rating scale.
Get full access to this article
View all access options for this article.
References
1.
Cicchetti, D. V.Assessing inter-rater reliability for rating scales: Resolving some basic issues. British Journal of Psychiatry, 1976, 129, 452-456.
2.
Cicchetti, D. V., Aivano, S. L., and Vitale, J.Computer programs for assessing rater agreement and rater bias for qualitative data. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1977, 37, 195-201.
3.
Cicchetti, D. V., Fontana, A. F., and Dowds, B. Noel.Assessing specific category reliabilities for rating scales in behavioral research. Paper presented at the annual meeting of the American Psychological Association, San Francisco, California, August 1977.
4.
Cohen, J.A coefficient of agreement for nominal scales. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1960, 23, 37-46.
5.
Cohen, J.Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 1968, 70, 213-220.
6.
Dice, L. R.Measures of the amount of ecologic association between species. Ecology, 1945 , 26, 297-302.
7.
Fleiss, J. L., Cohen, J. , and Everitt, B. S.Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 1969, 72, 323-327.
8.
Hall, J. N.Inter-rater reliability of ward rating scales. British Journal of Psychiatry, 1974, 125, 248-255.
9.
Helzer, J. E., Clayton, P. J., Pambakian, R., Reich, T., Woodruff, R. A., and Reveley, M. A.Reliability of psychiatric diagnosis. II. The test/retest reliability of diagnostic classification. Archives of General Psychiatry , 1977, 34, 136-141.
10.
Sandifer, M. G., Pettus, C., and Quade, D.A study of psychiatric diagnosis. Journal of Nervous and Mental Disease , 1964, 139, 350-356.
11.
Spitzer, R. L., Cohen, J. , Fleiss, J. L., and Endicott, J.Quantification of agreement in psychiatric diagnosis: A new approach. A rchives of General Psychiatry, 1967, 17, 83-87.
12.
Spitzer, R. L., Endicott, J., Cohen, J., and Fleiss, J. L.Constraints on the validity of computer diagnosis. Archives of General Psychiatry, 1974, 31, 197-203.
13.
Spitzer, R. L. and Fleiss, J. L.A re-analysis of the reliability of psychiatric diagnosis . British Journal of Psychiatry, 1974, 125, 341-347.
