Scoring systems, as a type of predictive model, have significant advantages in interpretability and transparency and facilitate quick decision-making. As such, scoring systems have been extensively used in a wide variety of industries, such as healthcare and criminal justice. However, the fairness issues in these models have long been criticized, and the use of big data and machine learning (ML) algorithms in the construction of scoring systems heightens this concern. This article proposes a general framework to create fairness-aware, data-driven scoring systems. First, we develop a social welfare function that incorporates both efficiency and group fairness. Then, we transform the social welfare maximization problem into the risk minimization task in ML, and derive a fairness-aware scoring system with the help of mixed-integer programming. Lastly, several theoretical bounds are derived for providing parameter selection suggestions. Our proposed framework provides a suitable solution to address group fairness concerns in developing scoring systems. It enables policymakers to set and customize their desired fairness requirements as well as other application-specific constraints. We test the proposed algorithm with several empirical data sets. Experimental evidence supports the effectiveness of the proposed scoring system in achieving the optimal welfare of stakeholders and in balancing the needs for interpretability, fairness, and efficiency.
AdelettiKBryanCBrown-CrowellMDysonJShahDKTerrellJMaruyamaDRPriceNC (2017) Laboratory values interpretation resource. Technical report, Academy of Acute Care Physical Therapy.
2.
AgarwalADudíkMWuZS (2019) Fair regression: Quantitative definitions and reduction-based algorithms. In: International conference on machine learning, pp.120–129. PMLR.
3.
AndrejevićMSmillieLDFeuerriegelDTurnerWFLahamSMBodeS (2022) How do basic personality traits map onto moral judgments of fairness-related actions?Social Psychological and Personality Science13(3): 710–721.
4.
BanasikJCrookJ (2007) Reject inference, augmentation, and sample selection. European Journal of Operational Research183(3): 1582–1594.
5.
BarocasSHardtMNarayananA (2017) Fairness in machine learning. NIPS Tutorial1: 2017.
6.
BarocasSSelbstAD (2016) Big data’s disparate impact. California Law Review104: 671–732.
7.
BerkRHeidariHJabbariSKearnsMRothA (2021) Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research50(1): 3–44.
8.
BertsimasDFariasVFTrichakisN (2011) The price of fairness. Operations Research59(1): 17–31.
9.
BjarnadottirMAndersonDZiaLRhoadsK (2018) Predicting colorectal cancer mortality: Models to facilitate patient-physician conversations and inform operational decision making. Production and Operations Management27(12): 2162–2183.
10.
BrayneS (2014) Surveillance and system avoidance: Criminal justice contact and institutional attachment. American Sociological Review79(3): 367–391.
11.
CaldersTKamiranFPechenizkiyM (2009) Building classifiers with independency constraints. In: 2009 IEEE international conference on data mining workshops, pp.13–18. IEEE.
12.
CampbellCAD’AmatoCPappJ (2020) Validation of the Ohio Youth Assessment System Dispositional Tool (OYAS-DIS): An examination of race and gender differences. Youth Violence and Juvenile Justice18(2): 196–211.
13.
CappelenAListJSamekATungoddenB (2020) The effect of early-childhood education on social preferences. Journal of Political Economy128(7): 2739–2758.
14.
CappelenAWHoleADSørensenEØTungoddenB (2007) The pluralism of fairness ideals: An experimental approach. American Economic Review97(3): 818–827.
15.
CecconiMEvansLLevyMRhodesA (2018) Sepsis and septic shock. The Lancet392(10141): 75–87.
16.
CelisLEHuangLKeswaniVVishnoiNK (2019) Classification with fairness constraints: A meta-algorithm with provable guarantees. In: Proceedings of the conference on fairness, accountability, and transparency, pp.319–328.
17.
CharnessGRabinM (2002) Understanding social preferences with simple tests. The Quarterly Journal of Economics117(3): 817–869.
18.
ChenVXHookerJN (2023) A guide to formulating fairness in an optimization model. Annals of Operations Research326(1): 581–619.
19.
ChouldechovaA (2017) Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data5(2): 153–163.
20.
CoffmanKBExleyCLNiederleM (2021) The role of beliefs in driving gender discrimination. Management Science67(6): 3551–3569.
21.
Corbett-DaviesSPiersonEFellerAGoelSHuqA (2017) Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp.797–806.
22.
CoxJCSadirajV (2012) Direct tests of individual preferences for efficiency and equity. Economic Inquiry50(4): 920–931.
23.
De-ArteagaMFeuerriegelSSaar-TsechanskyM (2022) Algorithmic fairness in business analytics: Directions for research and practice. Production and Operations Management31(10): 3749–3770.
24.
DoniniMOnetoLBen-DavidSShawe-TaylorJPontilM (2018) Empirical risk minimization under fairness constraints. In: Proceedings of the 32nd international conference on neural information processing systems, pp.2796–2806.
25.
DworkCHardtMPitassiTReingoldOZemelR (2012) Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference, pp.214–226.
26.
EisenhandlerOTzurM (2019) The humanitarian pickup and distribution problem. Operations Research67(1): 10–32.
27.
EvansLRhodesAAlhazzaniWAntonelliMCoopersmithCMFrenchCMachadoFRMcintyreLOstermannMPrescottHC, et al. (2021) Surviving sepsis campaign: International guidelines for management of sepsis and septic shock 2021. Intensive Care Medicine47(11): 1181–1247.
28.
FehrESchmidtKM (1999) A theory of fairness, competition, and cooperation. The Quarterly Journal of Economics114(3): 817–868.
29.
FeldmanMFriedlerSAMoellerJScheideggerCVenkatasubramanianS (2015) Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp.259–268.
30.
Fernández-GutiérrezMVan de WalleS (2019) Equity or efficiency? Explaining public officials’ values. Public Administration Review79(1): 25–34.
31.
FleurbaeyMManiquetF (2008) Utilitarianism versus fairness in welfare economics. In: Justice, Political Liberalism, and Utilitarianism: Themes from Harsanyi and Rawls. Cambridge University Press, pp.263–280.
32.
FoleyDK (1967) Resource Allocation and the Public Sector. vol. 7, New Haven, CT: Yale Economics Essays.
33.
FuRHuangYSinghPV (2020) Artificial intelligence and algorithmic bias: Source, detection, mitigation, and implications. In: Pushing the Boundaries: Frontiers in Impactful OR/OM Research. INFORMS, pp.39–63.
34.
FuRHuangYSinghPV (2021) Crowds, lending, machine, and bias. Information Systems Research32(1): 72–92.
35.
Grgic-HlacaNZafarMBGummadiKPWellerA (2016) The case for process fairness in learning: Feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law. vol. 1, pp.2–11.
36.
HammanJRLoewensteinGWeberRA (2010) Self-interest through delegation: An additional rationale for the principal-agent relationship. American Economic Review100(4): 1826–46.
37.
HardtMPriceESrebroN (2016) Equality of opportunity in supervised learning. In: Proceedings of the 30th international conference on neural information processing systems, pp.3323–3331.
38.
HenrichJBoydRBowlesSCamererCFehrEGintisHMcElreathR (2001) In search of homo economicus: Behavioral experiments in 15 small-scale societies. American Economic Review91(2): 73–78.
39.
HenryKEHagerDNPronovostPJSariaS (2015) A targeted real-time early warning score (TREWScore) for septic shock. Science Translational Medicine7(299): 299ra122.
40.
HossainSMladenovicAShahN (2020) Designing fairly fair classifiers via economic fairness notions. In: Proceedings of the web conference 2020, pp.1559–1569.
41.
HuLChenY (2018) A short-term intervention for long-term fairness in the labor market. In: Proceedings of the 2018 world wide web conference, pp.1389–1398.
42.
HuLChenY (2020) Fair classification and social welfare. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp.535–545.
43.
JohnsonAEPollardTJShenLLi-WeiHLFengMGhassemiMMoodyBSzolovitsPCeliLAMarkRG (2016) MIMIC-III, a freely accessible critical care database. Scientific Data3(1): 1–9.
44.
KalaiESmorodinskyM (1975) Other solutions to Nash’s bargaining problem. Econometrica43(3): 513–518.
45.
KallusNMaoXZhouA (2022) Assessing algorithmic fairness with unobserved protected class using data combination. Management Science68(3): 1959–1981.
46.
KallusNZhouA (2019) The fairness of risk scores beyond classification: Bipartite ranking and the xAUC metric. In: Proceedings of the 33rd international conference on neural information processing systems, vol. 32, 8 December, pp. 3438–3448. Curran Associates Inc.
47.
KamiranFKarimAZhangX (2012) Decision theory for discrimination-aware classification. In: 2012 IEEE 12th international conference on data mining, pp.924–929. IEEE.
KozodoiNJacobJLessmannS (2022) Fairness in credit scoring: Assessment, implementation and profit implications. European Journal of Operational Research297(3): 1083–1094.
50.
KurowskiASzarpakŁFrassM, etal (2016) Glasgow Coma Scale used as a prognostic factor in unconscious patients following cardiac arrest in prehospital situations: Preliminary data. The American Journal of Emergency Medicine6(34): 1178–1179.
51.
LambrechtATuckerC (2019) Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science65(7): 2966–2981.
52.
Le GallJ-RLemeshowSSaulnierF (1993) A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA270(24): 2957–2963.
53.
LiuLTDeanSRolfESimchowitzMHardtM (2018) Delayed impact of fair machine learning. In: Proceedings of the 35th international conference on machine learning, pp.3150–3158. PMLR.
54.
LohausMPerrotMVon LuxburgU (2020) Too relaxed to be fair. In: Proceedings of the 37th international conference on machine learning, pp.6360–6369. PMLR.
55.
McMahanHB (2017) A survey of algorithms and analysis for adaptive online learning. Journal of Machine Learning Research18(1): 3117–3166.
56.
MehrabiNMorstatterFSaxenaNLermanKGalstyanA (2021) A survey on bias and fairness in machine learning. ACM Computing Surveys54(6): 1–35.
57.
MillerGA (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review63(2): 81.
MukherjeeVEvansL (2017) Implementation of the surviving sepsis campaign guidelines. Current Opinion in Critical Care23(5): 412–416.
60.
ObermeyerZPowersBVogeliC, etal (2019) Dissecting racial bias in an algorithm used to manage the health of populations. Science366(6464): 447–453.
61.
RajkomarAHardtMHowellMDCorradoGChinMH (2018) Ensuring fairness in machine learning to advance health equity. Annals of Internal Medicine169(12): 866–872.
RudinCUstunB (2018) Optimized scoring systems: Toward trust in machine learning for healthcare and criminal justice. INFORMS Journal on Applied Analytics48(5): 449–466.
64.
SamoraniMHarrisSLBlountLGLuHSantoroMA (2022) Overbooked and overlooked: Machine learning and racial bias in medical appointment scheduling. Manufacturing & Service Operations Management24(6): 2825–2842.
65.
SouderWE (1972) A scoring methodology for assessing the suitability of management science models. Management Science18(10): B526–B543.
66.
ThanMFlawsDSandersSDoustJGlasziouPKlineJAldousSTroughtonRReidC, et al. (2014) Development and validation of the Emergency Department Assessment of Chest pain Score and 2 h accelerated diagnostic protocol. Emergency Medicine Australasia26(1): 34–44.
67.
UstunBRudinC (2016) Supersparse linear integer models for optimized medical scoring systems. Machine Learning102(3): 349–391.
68.
WangGLiJHoppWJ (2022) An instrumental variable forest approach for detecting heterogeneous treatment effects in observational studies. Management Science68(5): 3399–3418.
69.
WickMPandaSTristanJ-B (2019) Unlocking fairness: A trade-off revisited. In: Proceedings of the 33rd international conference on neural information processing systems, pp.8783–8792.
70.
WilliamsonRMenonA (2019) Fairness risk measures. In: Proceedings of the 36th international conference on machine learning, vol. 97, pp.6786–6797.
WoodworthBGunasekarSOhannessianMISrebroN (2017) Learning non-discriminatory predictors. In: Proceedings of the 2017 conference on learning theory, vol. 65, pp.1920–1953.
73.
WuYHuangSChangX (2021) Understanding the complexity of sepsis mortality prediction via rule discovery and analysis: A pilot study. BMC Medical Informatics and Decision Making21(1): 1–15.
74.
YonaGRothblumG (2018) Probably approximately metric-fair learning. In: Proceedings of the 35th international conference on machine learning, vol. 80, pp.5680–5688.
75.
ZafarMBValeraIGomez-RodriguezMGummadiKP (2019) Fairness constraints: A flexible approach for fair classification. Journal of Machine Learning Research20(75): 1–42.
76.
ZafarMBValeraIGomez RodriguezMGummadiKP (2017a) Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In: Proceedings of the 26th international conference on world wide web, pp.1171–1180.
77.
ZafarMBValeraIRogriguezMGGummadiKP (2017b) Fairness constraints: Mechanisms for fair classification. In: Proceedings of the 20th international conference on artificial intelligence and statistics, vol. 54, pp.962–970.
78.
ZengJUstunBRudinC (2017) Interpretable classification models for recidivism prediction. Journal of the Royal Statistical Society: Series A (Statistics in Society)3(180): 689–722.
79.
ZhangXKhaliliMMTekinCLiuM (2019) Group retention when using machine learning in sequential decision making: The interplay between user dynamics and fairness. In: Proceedings of the 33rd international conference on neural information processing systems, vol. 32, pp.15269–15278.
80.
ZhangYBradlowETSmallDS (2015) Predicting customer value using clumpiness: From RFM to RFMC. Marketing Science34(2): 195–208.
81.
ZhaoHGordonGJ (2022) Inherent tradeoffs in learning fair representations. Journal of Machine Learning Research23(57): 1–26.
82.
ZouJSchiebingerL (2018) AI can be sexist and racist—It’s time to make it fair. Nature559: 324–326.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
