The Reliability of Difference Scores When Errors Are Correlated

Abstract

The usual formulas for the reliability of differences between two test scores X and Y are based on the assumption that the error scores E_X and E_Y are uncorrelated. In modern developments of test score theory, such as that of Lord and Novick, a true score is defined as the expected value of an individual's observed score. This definition implies that true scores on any test are uncorrelated with error scores on any test, but it does not imply that error scores on distinct tests X and Y are uncorrelated. A zero correlation between the errors can be obtained only by introducing an additional assumption of "experimental independence" that does not follow from the other axioms in the model. This assumption restricts severely the class of random variables to which the usual formulas for reliability of differences will apply. The present paper investigated the reliability of difference scores in more general cases where it is not assumed that error scores on distinct tests are uncorrelated. The formulas derived are relatively simple, and they reduce to the usual ones of the classical model when E_X and E_Y are uncorrelated.