Measures of agreement for ordinal scaled data are suggested that make use of the k categories with the highest ranks. The proposed measures are applied to the Self-Directed Search in order to evaluate its agreement with self-assessment (translation ability) or with the work environment (congruence).
Get full access to this article
View all access options for this article.
References
1.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
2.
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.
3.
Conger, A. J. (1980). Integration and generalization of kappas for multiple raters. Psychological Bulletin, 88, 322-328.
4.
Fleiss, J. L. (1975). Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 31, 651-659.
5.
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378-383.
6.
Fleiss, J. L. , Cohen, J., and Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 72, 323-327.
7.
Fleiss, J. L. and Cohen, J. (1973). The equivalent of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613-619.
8.
Holland, J. L. (1973). Making Vocational Choices: A Theory of Careers, Prentice-Hall, NJ.
9.
Holland, J. L. (1979). Professional Manual for the Self-Directed Search, 2nd Ed.Palo Alto, CA: Consulting Psychologists Press.
10.
Iachan, R. (1984). Measures of agreement for Holland's classification system. Journal of Vocational Behavior, 24, 133-141.
11.
Kendall, M. G. , Kendall, S. F. H., and Smith, B. B. (1938). The distribution of Spearman's coefficient of rank correlation in a universe in which all rankings occur an equal number of times. Biometrika, 30, 251-273.
12.
Kendall, M. G. and Smith, B. B. (1939). The problem of m rankings. Annals of Mathematical Statistics, 10, 275-287.
13.
Kraemer, H. C. (1979). Ramifications of a population model for K as a coefficient of reliability. Psychometrika, 44, 461-472.
14.
Kwak, J. C. and Pulvino, C. J. (1982). A mathematical model for comparing Holland's personality and environment codes. Journal of Vocational Behavior, 21, 231-241.
15.
Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
16.
Light, R. J. (1971). Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychological Bulletin, 76, 365-377.
17.
Zener, T. B. and Schnuelle, L. (1976). Effects of the self-directed search on high school students. Journal of Counseling Psychology, 23, 353-359.
