A computer program entitled Program CONGRU is presented which analyzes the overall conjoint agreement among many observers for their classifications on categorical scales. A special feature assesses the significance of observers' congruence in assignments to each nominal category considered.
Get full access to this article
View all access options for this article.
References
1.
Cicchetti, D. V., Aivano, S. L., and Vitale, J.Computer programs for assessing rater agreement and rater bias for qualitative data. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1977, 37, 195-201.
2.
Cicchetti, D. V., Lee, C. , Fontana, A. F., and Dowds, B. N.A computer program for assessing specific category rater agreement for qualitative data. West Haven, Conn.: Veterans Administration Hospital , 1978.
3.
Cohen, J.A coefficient of agreement for nominal scales. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1960, 20, 37-46.
4.
Hartmann, D. P.Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 1977, 10, 103-116.
5.
Fleiss, J. L.Measuring nominal scale agreement among many raters. Psychological Bulletin, 1971, 76, 378-382.
6.
Light, R. J.Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 1971, 76, 365-377.
7.
McDermott, P. A.The grouping and classification of school children: Actuarial and legal principles. In B. D. Sales and M. R. Novick (Eds.), Perspectives in law and psychology: Volume III. Testing and evaluation. New York: Plenum, 1979.
8.
Yelton, A. R., Wildman, B. G., and Erickson, M. T.A probability-based formula for calculating interobserver agreement. Journal of Applied Behavior Analysis, 1977, 10, 127-131.
