The most commonly used and accepted model of assessing bias in a selection context is that proposed by Cleary in which predictor-criterion regression lines are tested for both slope and intercept equality. With this approach, any difference in intercepts or slopes is considered an indication of bias. We argue that differing regression lines intercepts is indicative of differential prediction but not test bias. We describe several fundamentally different potential causes of differences in groups’ regression line intercepts, many of which are unrelated to test properties. We argue that differential prediction because of such sources should not preclude the use of the test in selection contexts. We propose a new procedure to potentially identify the source of regression line differences and illustrate this framework using a job incumbent sample.
Aguinis, H. ( 1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141-1158.
2.
Aguinis, H., & Smith, M.A. ( 2007). Understanding the impact of test validity and bias on selection errors and adverse impact in human resource selection. Personnel Psychology, 60, 165-199.
3.
American Educational Research Association/American Psychological Association/ National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4.
Austin, J.T., & Crespin, T.R. ( 2006). Problems of criteria in industrial and organizational psychology: progress, pitfalls, and prospects. In W. J. Bennett, C. E. Lance, & D. J. Woehr (Eds.). Performance measurement: Current perspectives and future challenges (pp. 9-48). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
5.
Bartlett, C.J., Bobko, P., Mosier, S.B., & Hannan, R. ( 1978). Testing for fairness with a moderated multiples regression strategy. Personnel Psychology, 31, 233-242.
6.
Bartlett, C.J., & O’Leary, B. (1969). A differential prediction model to moderate the effects of heterogeneous groups in personnel selection and classification. Personnel Psychology, 22, 1-19.
7.
Bennett, W.J., Lance, C.E., & Woehr, D.J. ( 2006). Performance measurement: Current perspectives and future challenges. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
8.
Bingham, W.V. ( 1926). Measures of occupational success. Harvard Business Review, 5, 1-10.
9.
Binning, J.F., & Barrett, G.V. ( 1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478-494.
10.
Camilli, G., & Shepard, L.A. ( 1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
11.
Cleary, T.A. ( 1968). Test bias: Prediction of grades of Negro and White students in integrated colleges. Journal of Educational Measurement , 5, 115-124.
12.
Cohen, J., & Cohen, P. ( 1975). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
13.
Crocker, L.M., & Algina, J. ( 1986). Introduction to classical and modern test theory. New York: Harcourt Brace Jovanovich College Publishers .
14.
Darlington, R.B. ( 1971). Another look at ‘‘cultural fairness’’ . Journal of Educational Measurement, 8, 71-82.
15.
Dunnette, M.D. ( 1963). A note on ‘‘the’’ criterion. Journal of Applied Psychology, 47, 251-254.
16.
Eagly, A.H., Makhijani, M.G., & Klonsky, B.G. ( 1992). Gender and the evaluation of leaders: A meta-analysis . Psychological Bulletin, 111, 3-22.
17.
Erwin, F. ( 2001a). Customer Service and Clerical Potential Index (CSCPI) Technical Manual. Developed by Richardson, Bellows, Henry & Company (RBH) . Minneapolis, MN: PreVisor, Incorporated .
18.
Erwin, F. ( 2001b). Test of Learning Ability (TLA) Technical Manual. Developed by Richardson, Bellows, Henry & Company (RBH). Minneapolis, MN: PreVisor, Incorporated.
19.
Ford, J.K., Kraiger, K., & Schechtman, S.L. (1986). Study of race effects in objective indices and subjective evaluations of performance: A meta-analysis of performance criteria. Psychological Bulletin, 99, 330-337.
Humphreys, L.G. ( 1986). An analysis and evaluation of test and item bias in the prediction context. Journal of Applied Psychology, 71, 327-333.
25.
Hunter, J.E., & Schmidt, F.L. ( 1976). Critical analysis of the statistical and ethical implications of various definitions of test bias. Psychological Bulletin , 83, 1053-1071.
26.
James, L.R. ( 1980). The unmeasured variables problem in path analysis. Journal of Applied Psychology, 65, 415-421.
27.
Johnson, E.C., Meade, A.W., & DuVernet, A.M. (in press). The role of referent indicators in tests of measurement invariance. Structural Equation Modeling .
28.
Kraiger, K., & Ford, J.K. ( 1985). A meta-analysis of ratee race effects in performance ratings. Journal of Applied Psychology, 70, 56-65.
29.
Kraiger, K., & Ford, J.K. ( 1990). The relation of job knowledge, job performance, and supervisory ratings as a function of ratee race. Human Performance, 3, 269-279.
30.
Landy, F.J., Shankster, L.J., & Kohler, S.S. ( 1994). Personnel selection and placement. Annual Review of Psychology, 45, 261-296.
Lautenschlager, G.J., & Mendoza, J.L. ( 1986). A step-down hierarchical multiple regression analysis for examining hypotheses about test bias in prediction. Applied Psychological Measurement, 10, 133-139.
33.
Ledvinka, J. ( 1979). The statistical definition of fairness in the federal selection guidelines and its implications for minority employment. Personnel Psychology, 32, 551-562.
34.
Linn, R.L. ( 1973). Fair test use in selection. Review of Educational Research, 43, 139-161.
35.
Linn, R.L., & Werts, C.E. ( 1971). Considerations for studies of test bias. Journal of Educational Measurement, 8, 1-4.
36.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems (). Hillsdale, NJ: Lawrence Erlbaum Associates.
37.
Lord, F.M., & Novick, M.R. ( 1968). Statistical theories of mental test scores. Oxford, England: Addison-Wesley.
38.
Lyness, K.S., & Heilman, M.E. ( 2006). When fit is fundamental: Performance evaluations and promotions of upper-level female and male managers. Journal of Applied Psychology, 91, 777-785.
39.
Mauro, R. ( 1990). Understanding L.O.V.E. (left out variables error): A method for estimating the effects of omitted variables. Psychological Bulletin, 108, 314-329.
40.
McKay, P.F., & McDaniel, M.A. (2006). A reexamination of black-white mean differences in work performance: More data, more moderators. Journal of Applied Psychology, 91, 538-554.
41.
McNemar, Q. ( 1975). On so-called test bias. American Psychologist , 30, 848-851.
42.
Meade, A.W., & Lautenschlager, G.J. (2004). A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance. Organizational Research Methods, 7, 361-388.
43.
Motowidlo, S.J. ( 2003). Job performance. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.). Handbook of psychology: Vol. 12, Industrial and organizational psychology (pp. 39-53). Hoboken, NJ: John Wiley & Sons.
44.
Murphy, K.R., & Davidshofer, C.O. (2001). Psychological testing: Principles and applications (5th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
45.
Navas-Ara, M.J., & Gomez-Benito, J. (2002). Effects of ability scale purification on the identification of DIF. European Journal of Psychological Assessment, 18, 9-15.
46.
Neisser, U., Boodoo, G., Bouchard, T.J.J., Boykin, A.W., Brody, N., & Ceci, S.J., et al. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101.
47.
Pazy, A., & Oron, I. ( 2001). Sex proportion and performance evaluation among high-ranking military officers. Journal of Organizational Behavior, 22, 689-702.
48.
Pulakos, E.D., White, L.A., Oppler, S.H., & Borman, W.C. ( 1989). Examination of race and sex effects on performance ratings . Journal of Applied Psychology, 74, 770-780.
49.
Rensvold, R.B., & Cheung, G.W. ( 2001). Testing for metric invariance using structural equation models: Solving the standardization problem. In C. A. Schriesheim, & L. L. Neider (Eds.). Research in management (Vol. 1): Equivalence in measurement (pp. 21-50). Greenwich, CT: Information Age.
50.
Roth, P.L., Bevier, C.A., Bobko, P., Switzer, F.S.I., & Tyler, P. ( 2001). Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology , 54, 297-330.
51.
Roth, P.L., Huffcutt, A.I., & Bobko, P. ( 2003). Ethnic group differences in measures of job performance: A new meta-analysis. Journal of Applied Psychology, 88, 694-706.
52.
Rotundo, M., & Sackett, P.R. ( 1999). Effect of rater race on conclusions regarding differential predication in cognitive ability tests. Journal of Applied Psychology, 84, 815-822.
Sackett, P.R., Laczo, R.M., & Lippe, Z.P. ( 2003). Differential prediction and the use of multiple predictors: The omitted variables problem. Journal of Applied Psychology , 88, 1046-1056.
55.
Sackett, P.R., & Wilk, S.L. ( 1994). Within-group norming and other forms of score adjustment in preemployment testing. American Psychologist, 49, 929-954.
56.
Schmidt, F.L., & Hunter, J.E. ( 1981). Employment testing: Old theories and new research findings . American Psychologist, 36, 1128-1137.
57.
Schmidt, F.L., & Hunter, J.E. ( 1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
58.
Schmitt, N., Clause, C.S., & Pulakos, E.D. ( 1996). Subgroup differences associated with different measures of some common job-relevant constructs. In C. L. Cooper, & I. T. Robertson (Eds.). International review of industrial and organizational psychology (Vol. 11, pp. 115-139). New York: Wiley .
59.
Society for Industrial and Organizational Psychology. ( 2003). Principles for the validation and use of personnel selection procedures. Bowling Green, OH: Society for Industrial and Organizational Psychology.
60.
Stauffer, J.M., & Buckley, M.R. ( 2005). The existence and nature of racial bias in supervisory ratings. Journal of Applied Psychology, 90, , 586-591.
61.
Swim, J., Borgida, E., Maruyama, G., & Myers, D.G. ( 1989). Joan McKay versus John McKay: Do gender stereotypes bias evaluations? Psychological Bulletin, 105, 409-429.
62.
Terris, W. ( 1997). The traditional regression model for measuring test bias is incorrect and biased against minorities. Journal of Business and Psychology, 12, 25-37.
63.
Thayer, P.W. ( 1992). Construct validation: Do we understand our criteria? Human Performance, 5, 97-108.
64.
Thorndike, R.L. ( 1971). Concepts of culture-fairness. Journal of Educational Measurement, 8, 63-70.
65.
U.S. Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and Department of Justice. ( 1978). Uniform guidelines on employee selection procedures. Federal Register, 43, 38290-38315.
66.
Vandenberg, R.J., & Lance, C.E. ( 2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
67.
Waldman, D.A., & Avolio, B.J. ( 1991). Race effects in performance evaluations: Controlling for ability, education, and experience. Journal of Applied Psychology, 76, 897-901.
68.
Wang, W.-C., & Yeh, Y.-L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498.
69.
Wendelken, D.J., & Inn, A. ( 1981). Nonperformance influences on performance evaluations: A laboratory phenomenon? Journal of Applied Psychology, 66, 149-158.
