Internal Consistency of Verbal Response Classes

Abstract

The traditional determination of adequacy of measurement in verbal conditioning studies is questioned. Reliance on the single criterion of inter-rater reliability ignores other critical aspects of the typical study, namely, the internal consistency of response classes and problems intrinsic to derived scores. Illustrative data are presented in which inter-rater reliabilities are quite respectable but the typical significance test for experimental effects would be applied to a measure that contained little non-error variance because of low internal consistency. In such a case, experimental effects could not be detected.