Studies in applied behavior analysis have used two expressions of reliability for human observations: percentage agreement (including percent occurrence and percent nonoccurrence agreement) and cor-relational techniques (including the phi coefficient). The formal relationship between these two expressions is presented and the overstatement of reliability by percentage agreement, relative to phi, is discussed. It is suggested that both expressions be reported in order to communicate reliability unambiguously and to facilitate comparison of the reliabilities from different studies.
Get full access to this article
View all access options for this article.
References
1.
Freund, J. E.Mathematical statistics. Englewood Cliffs, N. J.: Prentice-Hall , 1971.
2.
Guilford, J. P.Psychometric methods (2nd ed.). New York: McGraw-Hill, 1954.
3.
Hartmann, D. P.Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 1977, 10, 103-116.
4.
Kelly, M. B.A review of the observational data-collection and reliability procedures in The Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 1977, 10, 97 -101.
5.
Kratochwill, T. R. and Wetzel, R. J.Observer agreement, credibility, and judgement: Some considerations in presenting observer agreement data. Journal of Applied Behavior Analysis, 1977, 10 , 133-139.
6.
Lord, F. M. and Novick, M. R.Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley, 1968.
7.
Yelton, A. R., Wildman, B. G., and Erickson, M. T.A probability-based formula for calculating interobserver agreement . Journal of Applied Behavior Analysis, 1977 , 10, 127 -131 .
