On Bartko's Intraclass Correlations of Interrater Reliability

Abstract

Bartko's justification for rejecting Winer's “anchor point method” of intraclass correlation is reviewed. The issues of rater variance and the average intercorrelation were examined in terms of Winer's method and Bartko's two alternative formulations. There were three major conclusions: (1) Bartko's arguments regarding additive bias were statistically sound, however, they are virtually meaningless in the application of the intraclass correlation in an experimental or practical context; (2) Bartko's intraclass correlation for the one-way model of analysis of variance is appropriate when it is desirable to include the rater variance term; (3) Winer's intraclass correlation for the two-way model of analysis of variance is appropriate when it is desirable to exclude rater variance.