There is growing interest in ensuring equity and guarding against bias in the use of risk scores produced by machine learning and artificial intelligence models. Risk scores are used to select patients who will receive outreach and support. Inappropriate use of risk scores, however, can perpetuate disparities. Commonly advocated solutions to improve equity are nontrivial to implement and may not pass legal scrutiny. In this article, we introduce pragmatic tools that support better use of risk scores for more equitable outreach programs. Our model output charts allow modeling and care management teams to see the equity consequences of different threshold choices and to select the optimal risk thresholds to trigger outreach. For best results, as with any health equity tool, we recommend that these charts be used by a diverse team and shared with relevant stakeholders.
ChenT.GuestrinC. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). https://doi.org/10.1145/2939672.2939785
6.
CowgillB. (2018). Bias and productivity in humans and algorithms: Theory and evidence from resume screening. Columbia Business School, Columbia University.
DworkC.HardtM.PitassiT.ReingoldO.ZemelR. (2012). Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (pp. 214–226). Association for Computing Machinery. https://doi.org/10.1145/2090236.2090255
9.
EaneffS.ObermeyerZ.ButteA. J. (2020). The case for algorithmic stewardship for artificial intelligence and machine learning technologies. Journal of the American Medical Association, 324(14), 1397–1398. https://doi.org/10.1001/jama.2020.9371
FeldmanM.FriedlerS. A.MoellerJ.ScheideggerC.VenkatasubramanianS. (2015). Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 259–268). Association for Computing Machinery. https://doi.org/10.1145/2783258.2783311
12.
FlanaganB. E.HalliseyE. J.AdamsE.LaveryA. (2018). Measuring community vulnerability to natural and anthropogenic hazards: The centers for disease control and prevention’s social vulnerability index. Journal of Environmental Health, 80(10), 34–36.
GervasiS. S.ChenI. Y.Smith-McLallenA.SontagD.ObermeyerZ.VenneraM.ChawlaR. (2022). The potential for bias in machine learning and opportunities for health insurers to address it: Article examines the potential for bias in machine learning and opportunities for health insurers to address it. Health Affairs, 41(2), 212–218. https://doi.org/10.1377/hlthaff.2021.01287
15.
GillN.HallP.MontgomeryK.SchmidtN. (2020). A responsible machine learning workflow with focus on interpretable models, post-hoc explanation, and discrimination testing. Information, 11(3), 137. https://doi.org/10.3390/info11030137
16.
GulshanV.PengL.CoramM.StumpeM. C.WuD.NarayanaswamyA.WebsterD. R. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Journal of the American Medical Association, 316(22), 2402–2410. https://doi.org/10.1001/jama.2016.17216
17.
HardtM.PriceE.SrebroN. (2016). Equality of opportunity in supervised learning. ArXiv:1610.02413 [Cs]. http://arxiv.org/abs/1610.02413
18.
HeidariH.LoiM.GummadiK. P.KrauseA. (2018). A moral framework for understanding of fair ml through economic models of equality of opportunity. Arxiv:1809.03400 [Cs, Econ, Stat]. http://arxiv.org/abs/1809.03400
19.
HildebrandtM.CastilloC.CelisE.RuggieriS.TaylorL.Zanfir-FortunaG. (2020). In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. FAT* ’20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain
KindA. J. H.BuckinghamW. R. (2018). Making neighborhood-disadvantage metrics accessible—The neighborhood atlas. The New England Journal of Medicine, 378(26), 2456–2458. https://doi.org/10.1056/NEJMp1802313
LitjensG.KooiT.BejnordiB. E.SetioA. A. A.CiompiF.GhafoorianM.SánchezC. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https://doi.org/10.1016/j.media.2017.07.005
MehrabiN.MorstatterF.SaxenaN.LermanK.GalstyanA. (2019). A survey on bias and fairness in machine learning. ArXiv:1908.09635 [Cs]. http://arxiv.org/abs/1908.09635
26.
Nondiscrimination in health and health education programs or activities, delegation of authority, 85 Fed. Reg. 37160 June 19, 2020.
27.
ObermeyerZ.EmanuelE. J. (2016). Predicting the future—Big data, machine learning, and clinical medicine. The New England Journal of Medicine, 375(13), 1216–1219. https://doi.org/10.1056/NEJMp1606181
28.
ObermeyerZ.PowersB.VogeliC.MullainathanS. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science (New York, N.Y.), 366(6464), 447–453. https://doi.org/10.1126/science.aax2342
29.
PersadG. (2021). Allocating medicine fairly in an unfair pandemic (SSRN Scholarly Paper No. ID 3699769). Social Science Research Network. https://papers.ssrn.com/abstract=3699769
The Public Health and Welfare, 42 C.F.R. § 456 (1964a).
32.
The Public Health and Welfare, 42 C.F.R. § 2000d (1964b).
33.
QuanH.LiB.CourisC. M.FushimiK.GrahamP.HiderP.SundararajanV. (2011). Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. American Journal of Epidemiology, 173(6), 676–682. https://doi.org/10.1093/aje/kwq433
34.
RajkomarA.DeanJ.KohaneI. (2019). Machine learning in medicine. New England Journal of Medicine, 380(14), 1347–1358. https://doi.org/10.1056/NEJMra1814259
35.
SantelicesM. V.WilsonM. (2010). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106–134. https://doi.org/10.17763/haer.80.1.j94675w001329270
36.
SchmidtH.GostinL. O.WilliamsM. A. (2020). Is it lawful and ethical to prioritize racial minorities for COVID-19 vaccines?Journal of the American Medical Association, 324(20), 2023. https://doi.org/10.1001/jama.2020.20571
37.
SendakM.ElishM. C.GaoM.FutomaJ.RatliffW.NicholsM.O’BrienC. (2020). “The human body is a black box”: Supporting clinical decision-making with deep learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 99–109). ACM. https://doi.org/10.1145/3351095.3372827
38.
SingT.SanderO.BeerenwinkelN.LengauerT. (2005). ROCR: Visualizing classifier performance in R. Bioinformatics, 21(20), 3940–3941.
WallaceP. J.ShahN. D.DennenT.BleicherP. A.BleicherP. D.CrownW. H. (2014). Optum Labs: Building a novel node in the learning health care system. Health Affairs (Project Hope), 33(7), 1187–1194. https://doi.org/10.1377/hlthaff.2014.0038
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
