The degree of agreement between two raters who rate a number of objects on a certain characteristic can be expressed by means of an association coefficient (e.g., the product-moment correlation). A large number of association coeffi cients have been proposed, many of which belong to the class of Euclidean coefficients (ECs). A discussion of desirable properties of ECs demonstrates how the identity coefficient and its generalizations, which constitute a family of ECs, can be used to assess interrater agreement. This family of ECs contains coefficients for both nominal and non-nominal (ordinal and metric) data. In particular, it is pointed out which infor mation contained in the data is accounted for by the various coefficients and which information is ignored.
Cohen, J. (1960). A coefficient of agreement for nominal scales . Educational and Psychological Measurement, 20, 37-46.
2.
Cohen, J. (1969). r,: A profile similarity coefficient invariant over variable reflection. Psychological Bulletin, 71, 281-284.
3.
Daniels, H.E. (1944). The relation between measures of correlation in the universe of sample permutations. Biometrika, 36, 129-135.
4.
Fagot, R.F., & Mazo, R.M. (1989). Association coefficients of identity and proportionality for metric scales. Psychometrika, 54, 93-104.
5.
Gower, J.C. (1966). Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika, 53, 315-328.
6.
Gower, J.C. (1971). A general coefficient of similarity and some of its properties. Biometrics, 27, 857-871.
7.
Hubert, L. (1977). Nominal scale response agreement as a generalized correlation. British Journal of Mathematical and Statistical Psychology, 30, 98-103.
8.
Janson, S., & Vegelius, J. (1978a). Correlation coefficients for more than one scale type and symmetrization as a method of obtaining them (Research Rep. No. 78-2) . Uppsala, Sweden: University of Uppsala, Department of Statistics.
9.
Janson, S., & Vegelius, J. (1978b). On construction of E correlation coefficients (Research Report 78-11). Uppsala, Sweden: University of Uppsala, Department of Statistics.
10.
Janson, S., & Vegelius, J. (1979). On generalizations of the G index and the phi coefficient to nominal scales. Multivariate Behavioral Research , 14, 255-269.
11.
Janson, S., & Vegelius, J. (1982a). Correlation coefficients for more than one scale type. Multivariate Behavioral Research, 17, 271-284.
12.
Janson, S., & Vegelius, J. (1982b). The J index as a measure of nominal scale response agreement. Applied Psychological Measurement, 6, 111-121.
13.
Popping, R. (1983). Overeenstemmingsmaten voor nominale data (Agreement coefficients for nominal data). Doctoral dissertation. University of Groningen, The Netherlands.
14.
Saporta, G. (1975). Liaison entre plusieurs ensembles des variables et codage de donnés qualitatives (Relation between various sets of variables and the coding of qualitative data). Doctoral dissertation. University of Paris, France.
15.
Sokal, R.R., & Sneath, P.H.A. (1963). Principles of numerical taxonomy. San Francisco: Freeman & Co.
Tschuprow, A.A. (1939). Principles of the mathematical theory of correlation . New York: William Hodge.
18.
Tucker, L.R. (1951). A method for synthesis of factor analysis studies (Report No. 984). Washington DC: Department of the Army, Personnel Research Section.
19.
Vegelius, J. (1976). On generalizations of the G index. Educational and Psychological Measurement, 36, 595-600.
20.
Zegers, F.E. (1986a). A family of chance-corrected association coefficients for metric scales. Psychometrika, 51, 559-569.
21.
Zegers, F.E. (1986b). A general family of association coefficients . Groningen, Netherlands: Boomker .
22.
Zegers, F.E., & ten Berge, J.M.F. (1985). A family of association coefficients for metric scales. Psychometrika, 50, 17-24.
23.
Zegers, F.E., & ten Berge, J.M.F. (1986). Correlation coefficients for more than one scale type: An alternative to the Janson and Vegelius approach . Psychometrika, 51, 549-557.
