A More Nuanced View of Reliability

Abstract

Retest reliability is a better predictor of validity than is internal consistency. One explanation for this is item-specific variance, which distinguishes different nuances of a facet and contributes to retest reliability but not internal consistency. Specific variance at the facet level is temporally stable, consensually validated, and heritable; a consideration of the role of specific variance in personality measures leads to a distinction between traits as the intersection (∩) versus the union (∪) of their constituents. I discuss specific variance at the item level and its implications for scale development and argue that retest reliability outpredicts internal consistency because item-specific variance has been shown to be observable, and is probably heritable and stable. I consider some implications of these ideas for the use of single-item scales, the causal interpretation of traits, and the notion of scalar equivalence. Finally, I note that the sources of random error in scales are still poorly understood.