We present a data-driven prescriptive framework for fair decisions, motivated by hiring. An employer evaluates a set of applicants based on their observable attributes. The goal is to hire the best candidates while avoiding bias with regard to a certain protected attribute. Simply ignoring the protected attribute will not eliminate bias due to correlations in the data. We present a provably optimal fair hiring policy that depends on the protected attribute functionally, but not statistically. The policy does not set rigid quotas, and does not withhold information from decision-makers. Both synthetic and real data indicate that the policy can greatly improve equity for underrepresented and historically marginalized groups, often with negligible loss in objective value.
BehaghelLCréponBLe BarbanchonT (2015) Unintended effects of anonymous résumés. American Economic Journal: Applied Economics7(3): 1–27.
7.
BerkRHeidariHJabbariS, et al. (2017) A convex framework for fair regression. Preprint, arXiv:1706.02409.
8.
BertrandMMullainathanS (2004) Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American Economic Review94(4): 991–1013.
9.
BertsimasDFariasVFTrichakisN (2011) The price of fairness. Operations Research59(1): 17–31.
10.
BertsimasDFariasVFTrichakisN (2012) On the efficiency-fairness trade-off. Management Science58(12): 2234–2250.
11.
BertsimasDKallusN (2020) From predictive to prescriptive analytics. Management Science66(3): 1025–1044.
12.
BertsimasDO’HairARelyeaSSilberholzJ (2016) An analytics approach to designing combination chemotherapy regimens for cancer. Management Science62(5): 1511–1531.
13.
BjarnadóttirMVAndersonD (2020) Machine learning in healthcare: fairness, issues, and challenges. In: Druehl C and Elmaghraby W (eds) Pushing the Boundaries: Frontiers in Impactful OR/OM Research. Catonsville, MD: INFORMS, pp. 64–83.
14.
BlackJSvan EschP (2020) AI-enabled recruiting: What is it and how should a manager use it?Business Horizons63(2): 215–226.
15.
BreugemTVan WassenhoveLN (2022) The price of imposing vertical equity through asymmetric outcome constraints. Management Science68(11): 7977–7993.
16.
CaldersTVerwerS (2010) Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery21(2): 277–292.
17.
ChoiT-MWallaceSWWangY (2018) Big data analytics in operations management. Production and Operations Management27(10): 1868–1883.
18.
ChouldechovaA (2017) Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data5(2): 153–163.
19.
ChzhenEDenisCHebiriM, et al. (2020) Fair regression with Wasserstein barycenters. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF and Lin H (eds) Advances in Neural Information Processing Systems, vol. 33. Red Hook, NY: Curran Associates, pp. 7321–7331.
20.
CohenMCElmachtoubANLeiX (2022) Price discrimination with fairness constraints. Management Science68(12): 8536–8552.
21.
CuiRLiJZhangDJ (2020) Reducing discrimination with reviews in the sharing economy: Evidence from field experiments on Airbnb. Management Science66(3): 1071–1094.
22.
CuiTHRajuJSZhangZJ (2007) Fairness and channel coordination. Management Science53(8): 1303–1314.
23.
del BarrioEGordalizaPLoubesJ-M (2020) Review of mathematical frameworks for fairness in machine learning. Preprint, arXiv:2005.13755.
24.
DworkCHardtMPitassiT, et al. (2012) Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, New York: ACM, pp. 214–226.
25.
EfronBTibshiraniRJ (1994) An Introduction to the Bootstrap. Chapman & Hall/CRC.
26.
ElmachtoubANGrigasP (2022) Smart “predict, then optimize”. Management Science68(1): 9–26.
27.
FeldmanMFriedlerSAMoellerJ, et al. (2015) Certifying and removing disparate impact. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York: ACM, pp. 259–268.
28.
FinocchiaroJMaioRMonachouF, et al. (2021) Bridging machine learning and mechanism design towards algorithmic fairness. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, ACM, New York, NY, pp. 489–503.
29.
FoleyMWilliamsonS (2018) Does anonymising job applications reduce gender bias? Understanding managers’ perspectives. Gender in Management33(8): 623–635.
30.
FuRHuangYSinghPV (2020) Artificial intelligence and algorithmic bias: Source, detection, mitigation, and implications. In: Druehl C and Elmaghraby W (eds) Pushing the Boundaries: Frontiers in Impactful OR/OM Research. Catonsville, MD: INFORMS, pp. 39–63.
31.
GengXHuhWTNagarajanM (2015) Fairness among servers when capacity decisions are endogenous. Production and Operations Management24(6): 961–974.
32.
GillenSJungCKearnsMRothA (2018) Online learning with an unknown fairness metric. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N and Garnett R (eds) Advances in Neural Information Processing Systems, vol. 31, pp.2600–2609.
33.
GrantMBoydS (2014) CVX: Matlab software for disciplined convex programming, version 2.1. Available at: http://cvxr.com/cvx.
34.
GuptaSKambleV (2021) Individual fairness in hindsight. Journal of Machine Learning Research22: 144:1–144:35.
35.
HardtMPriceEPriceESrebroN (2016) Equality of opportunity in supervised learning. In: Lee D, Sugiyama M, von Luxburg U, Guyon I and Garnett R (eds) Advances in Neural Information Processing Systems, vol. 29. Red Hook, NY: Curran Associates, pp. 3315–3323.
36.
HarrisCI (2000) Equal treatment and the reproduction of inequality. Fordham Law Review69(5): 1753–1784.
37.
HeidariHKrauseA (2018) Preventing disparate treatment in sequential decision making. In: Proceedings of the 2018 International Joint Conference on Artificial Intelligence. Cambridge, MA:AAAI Press, pp. 2248–2254.
38.
HoppWJLiJWangG (2018) Big data and the precision medicine revolution. Production and Operations Management27(9): 1647–1664.
39.
IancuDATrichakisN (2014) Fairness and efficiency in multiportfolio optimization. Operations Research62(6):1285–1301.
40.
IlventoCJagadeesanMChawlaS (2020) Multi-category fairness in sponsored search auctions. In: Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency. New York:ACM, pp. 348–358.
41.
JabbariSJosephMKearnsMMorgensternJRothR (2017) Fairness in reinforcement learning. In: Proceedings of the 34th International Conference on Machine Learning. Red Hook, NY: Curran Associates, pp. 1617–1626.
42.
JohndrowJELumK (2019) An algorithm for removing sensitive information: Application to race-independent recidivism prediction. The Annals of Applied Statistics13(1):189–220.
43.
JohnsonMPChichirauGR (2020) Diversity, equity, and inclusion in operations research and analytics: A research agenda for scholarship, practice, and service. In: Druehl C and Elmaghraby W (eds) Pushing the Boundaries: Frontiers in Impactful OR/OM Research. Catonsville, MD: INFORMS, pp. 1–38.
44.
JosephMKearnsMMorgensternJH, etal (2016) Fairness in learning: Classic and contextual bandits. In: Advances in Neural Information Processing Systems. vol. 29. Red Hook, NY: Curran Associates, pp. 325–333
45.
KallusNZhouA (2021) Fairness, welfare, and equity in personalized pricing. In: Proceedings of the 2021 ACM conference on Fairness, Accountability, and Transparency. New York: ACM, pp. 296–314.
46.
KearnsMNeelSRothAWuZS (2018) Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In: Proceedings of the 35th International Conference on Machine Learning. Red Hook, NY: Curran Associates, pp. 2564–2572.
47.
KearnsMRothAWuZS (2017) Meritocratic fairness for cross-population selection. In: Proceedings of the 34th International Conference on Machine Learning. Red Hook, NY: Curran Associates, pp. 1828–1836.
48.
KleinbergJMullainathanSRaghavanM (2017) Inherent trade-offs in the fair determination of risk scores. In: Papadimitriou CH (ed) Proceedings of the 8th Innovations in Theoretical Computer Science Conference. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, pp.43:1–43:23.
49.
KosinskiMStillwellDGraepelT (2013) Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences110(15): 5802–5805.
50.
LambrechtATuckerC (2019) Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science65(7): 2966–2981.
51.
LeviRPerakisGUichancoJ (2015) The data-driven newsvendor problem: New bounds and insights. Operations Research63(6): 1294–1306.
52.
LoiMHerlitzAHeidariH (2021) Fair equality of chances for prediction-based decisions. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. New York: ACM, pp. 756–756.
53.
LuceyCRSaguilA (2020) The consequences of structural racism on MCAT scores and medical school admissions: The past is prologue. Academic Medicine95(3): 351–356.
54.
MaYWangTZhengH (2023) On fairness and efficiency in nonprofit operations: Dynamic resource allocations. Production and Operations Management32(6): 1778–1792.
55.
McCoyJHLeeHL (2014) Using fairness models to improve equity in health delivery fleet management. Production and Operations Management23(6): 965–977.
56.
MejiaJParkerC (2021) When transparency fails: Bias and financial incentives in ridesharing platforms. Management Science67(1): 166–184.
57.
MisićVV (2020) Optimization of tree ensembles. Operations Research68(5): 1605–1624.
58.
MisićVVPerakisG (2020) Data analytics in operations management: A review. Manufacturing & Service Operations Management22(1): 158–169.
59.
PetersonERRubie-DaviesCOsborneDSibleyC (2016) Teachers’ explicit expectations and implicit prejudiced attitudes to educational achievement: Relations with student achievement and the ethnic achievement gap. Learning and Instruction42: 123–140.
60.
RaghavanMBarocasSKleinbergJLevyK (2020) Mitigating bias in algorithmic hiring: Evaluating claims and practices. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. New York: ACM, pp. 469–481.
61.
ReaDFroehleCMastersonS, et al. (2021) Unequal but fair: Incorporating distributive justice in operational allocation models. Production and Operations Management30(7): 2304–2320.
62.
SantelicesMVWilsonM (2010) Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review80(1): 106–134.
63.
SpectorARRaileyKM (2019) Reducing reliance on test scores reduces racial bias in neurology residency recruitment. Journal of the National Medical Association111(5): 471–474.
64.
SührTHilgardSLakkarajuH (2021) Does fair ranking improve minority outcomes? Understanding the interplay of human and algorithmic biases in online hiring. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. New York: ACM, pp. 989–999.
65.
ThomsenFK (2015) Stealing bread and sleeping beneath bridges–indirect discrimination as disadvantageous equal treatment. Moral Philosophy and Politics2(2): 299–327.
66.
U.S. Supreme Court (1987) Johnson v. Transportation Agency. 480 US 616.
67.
WightmanLF (1998) LSAC national longitudinal bar passage study. LSAC Research Report Series. Available at: https://eric.ed.gov/?id=ED469370.
68.
ZemelRWuYSwerskyK, et al. (2013) Learning fair representations. In: Proceedings of the 30th International Conference on Machine Learning, Curran Associates, Red Hook, NY, pp. 325–333.
69.
ZinkARoseS (2020) Fair regression for health care spending. Biometrics76(3): 973–982.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
