Recent research has helped to cultivate growing awareness that machine-learning systems fueled by big data can create or exacerbate troubling disparities in society. Much of this research comes from outside of the practicing data science community, leaving its members with little concrete guidance to proactively address these concerns. This article introduces issues of discrimination to the data science community on its own terms. In it, we tour the familiar data-mining process while providing a taxonomy of common practices that have the potential to produce unintended discrimination. We also survey how discrimination is commonly measured, and suggest how familiar development processes can be augmented to mitigate systems' discriminatory potential. We advocate that data scientists should be intentional about modeling and reducing discriminatory outcomes. Without doing so, their efforts will result in perpetuating any systemic discrimination that may exist, but under a misleading veil of data-driven objectivity.
Get full access to this article
View all access options for this article.
References
1.
BarocasS, SelbstAD. Big data's disparate impact. 104 California Law Review 671 (2016). Available at SSRN: https://ssrn.com/abstract=2477899
2.
O'NeilC.Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Crown Publishing Group. 2016.
3.
McNamaraRMJr. Fair credit reporting act: A legislative overview. J Pub L. 1973; 22:6–7.
RubinDB.Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974; 66:68–8.
7.
ZliobaiteI.A survey on measuring indirect discrimination in machine learning. arXiv preprint arXiv:1511.00148. 2015.
8.
FryerRGJr. An empirical analysis of racial differences in police use of force (No. w22399). Cambridge MA: National Bureau of Economic Research. 2016.
9.
RomeiA, RuggieriS. A multidisciplinary survey on discrimination analysis. Knowl Eng Rev. 2014; 29:582–638.
10.
KamiranF, CaldersT, PechenizkiyM. Discrimination aware decision tree learning. In: 2010 IEEE International Conference on Data Mining, IEEE, 2010, pp. 869–874.
11.
CaldersT, VerwerS. Three naive Bayes approaches for discrimination-free classification. Data Min Knowl Discov. 2010; 21:277–292.
12.
ZemelRS, WuY, SwerskyK, et al.Learning fair representations. ICML. 2013; 28:325–333.
13.
ZliobaiteI.On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723. 2015.
14.
KamiranF, ZliobaiteI, CaldersT. Quantifying explainable discrimination and removing illegal discrimination in automated decision making. Knowl Inf Syst. 2013; 35:613–644.
15.
LuongBT, RuggieriS, TuriniF. k-NN as an implementation of situation testing for discrimination discovery and prevention. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2011, pp. 502–510.
16.
CaldersT, KarimA, KamiranF, et al.Controlling attribute effect in linear regression. In: Proceedings of the 13th International Conference on Data Mining, ICDM, 2013, pp. 71–80.
17.
HajianS, BonchiF, CastilloC. Algorithmic bias: From discrimination discovery to fairness-aware data mining. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016.
PedreshiD, RuggieriS, TuriniF. Discrimination-aware data mining. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2008, pp. 560–568.
20.
KamishimaT, AkahoS, AsohH, SakumaJ. Fairness-aware classifier with prejudice remover regularizer. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2012, pp. 35–50.
21.
DalessandroB, HookR, PerlichC, ProvostF. Evaluating and optimizing online advertising: Forget the click, but there are good proxies. Big Data, 2015; 3:90–102.
ElkanC.The foundations of cost-sensitive learning. In: International Joint Conference on Artificial Intelligence (Vol. 17, No. 1). Lawrence Erlbaum Associates Ltd., 2001, pp. 973–978.
SettlesB.Active learning literature survey. Madison, WI: University of Wisconsin. 2010; 52.55–52.66:1–1.
26.
KamiranF, CaldersT. Data preprocessing techniques for classification without discrimination. Knowl Inf Syst. 2012; 33:1–33.
27.
FeldmanM, et al.Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2015.
28.
HandDJ.Classifier technology and the illusion of progress. Stat Sci. 2006; 21:1–14.
29.
VeeramachaneniK.Why you're not getting value from your data science. Harvard Business Review. December7, 2016.
30.
MarinW.The problems with using personality tests for hiring. Harvard Business Review. August27, 2014.
