Using Group Differences in True Score Relationships to Evaluate Measurement Bias

Abstract

This paper makes three contributions to our understanding of measurement bias and predictive bias in testing. First, we develop a linear model for assessing measurement bias across two tests and two groups in terms of the estimated true-score relationships between the two tests in the two groups. This new model for measurement bias is structurally similar to the Cleary model for predictive bias, but it relies on the Errors-in-Variables (EIV) regression model, rather than the Ordinary-Least-Squares (OLS) regression model. Second, we examine some differences between measurement bias and predictive bias in three cases in which two groups have different true-score means, and we illustrate how regression toward the mean in OLS regression can lead to questionable conclusions about test bias if the differences between measurement bias and predictive bias are ignored. Third, we reevaluate a body of empirical findings suggesting that the tests employed in college-admissions and employment-testing programs tend to over-predict criterion performance for minorities, and we show that these findings are consistent with the occurrence of substantial measurement bias against the minority group relative to the majority group.