A PC program for comparing two kappa coefficients of agreement obtained from the same sample of observations is described. The program employs a Monte Carlo permutation test to assess the statistical significance of the difference between kappa values.
Get full access to this article
View all access options for this article.
References
1.
AMERICAN PSYCHIATRIC ASSOCIATION. (1987) Diagnostic and statistical manual of mental disorders: DSM-III–R. (3rd ed., Rev.) Washington, DC: Author.
2.
BeckA. X.WardC. H.MendelsonM.MockJ., & ErbaughJ. (1961) An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571.
3.
BlochD. A. (1997) Comparing two diagnostic tests against the same “gold standard” in the same sample. Biometrics, 53, 73–85.
4.
BrysbaertM. (1991) Algorithms for randomness in the behavioral sciences: a tutorial. Behavior Research Methods, Instruments and Computers, 23, 45–60.
5.
ClarkeD. M.SmithG. C., & HerrmanH. E. (1993) A comparative study of screening instruments for mental disorders in general hospital patients. International Journal of Psychiatry in Medicine, 23, 323–337.
6.
CohenJ. (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.
7.
EdgingtonE. S. (1995) Randomization tests. (3rd ed.) New York: Marcel Dekker.
8.
EfronB., & TibshiraniR. J. (1993) An introduction to the bootstrap. New York: Chapman & Hall.
9.
FleissJ. L. (1981) Statistical methods for rates and proportions. (2nd ed.) New York: Wiley.
10.
GoodP. (1994) Permutation tests: a practical guide to resampling methods for testing hypotheses. New York: Springer-Verlag.
11.
GoodP. (1997) Resampling methods. New York: Springer-Verlag.
12.
Isml. (1991) User's manual, STAT/Library: FORTRAN subroutines for statistical analysis. Version 2. [computer program]. Houston, TX: Author.
13.
Lahey Computer Systems, Inc. (1992) Lahey F77L-EM/32 FORTRAN programmer's reference. revision C. [computer program] Incline Village, NV: Author.
14.
McKenzieD. P.MacKinnonA. J.PeladeauN.OnghenaP.BruceP. C.ClarkeD. M.HarriganS., & McGorryP. D. (1996) Comparing correlated kappas by resampling: is one level of agreement significantly different from another?Journal of Psychiatric Research, 30, 483–492.
15.
OnghenaP (1993) A theoretical and empirical comparison of mainframe, microcomputer, and pocket calculator pseudorandom number generators. Behavior Research Methods, Instruments and Computers, 25, 384–395.
16.
OnghenaP., & MayR. B. (1995) Pitfalls in computing and interpreting randomization test p-values: a commentary on Chen and Dunlap. Behavior Research Methods, Instruments and Computers, 27, 408–411.
17.
SimonJ. L. (1969) Basic research methods in social science. (3rd ed.1985, with BursteinP.) New York: Random House.
18.
Simon, J. L., & Resampling Stats, Inc. (1996) Resampling Stats for Windows, user's guide. Version 4.0.1. [computer program] Arlington, VA: Resampling Stats, Inc.
19.
WichmannB. A., & HillJ. D. (1982) Algorithm AS183: an efficient and portable pseudo random number generator. Applied Statistics, 31, 188–190.
20.
WichmannB. A., & HillJ. D. (1984) An efficient and portable random number generator: correction. Applied Statistics, 33, 123.
21.
ZigmondA. S., & SnaithR. P. (1983) The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavia, 67, 361–370.
