Inferential Conditions in the Statistical Detection of Measurement Bias

Abstract

Measurement bias in an observed variable Y as a measure of an unobserved variable W exists when the relationship of Y to W varies among popula tions of interest. Bias is often studied by examin ing population differences in the relationship of Y to a second observed measure Z that serves as a substitute for W. Whether the results of such studies have implications for measurement bias is addressed by first defining two forms of invariance— one corresponding to the relationship of Y to the unmeasured W, and one corresponding to the rela tionship of Y to the observed Z. General theoreti cal conditions are provided that justify the inference of one form of invariance from the other. The implications of these conditions for bias detection in two broad areas of application are discussed: differential item functioning and predictive bias in employment and educational settings. It is con cluded that the conditions for inference are restric tive, and that bias investigations that rely strictly on observed measures are not, in general, diag nostic of measurement bias or the lack of bias. Some alternative approaches to bias detection are discussed. Index terms: differential item function ing, invariance, item bias, item response theory, measurement bias, predictive bias.