Sage Journals: Discover world-class research

Abstract

Creating ethical, equitable and racially-just machine learning is currently impossible for six reasons, summarized by the acronym DO-GOOD: (1) data for training is historically racially biased, (2) ownership of data is disproportionately racially tilted, (3) guidelines/ethical algorithms have been largely lacking, especially in the space of racial equity, (4) opacity of algorithm design prevents testing for racial bias, and then improvement of racial equity, (5) over-reliance on math as a false racially-neutral tool, and (6) decision errors by humans who are subject to their own lived experiences and prejudices. For concreteness, each is demonstrated using examples from economics and finance applications, concluding with suggestions about how to address them (and indeed, how we are already tackling them): they require not only thoughtfulness but regulatory oversight to prevent harm.

Keywords

machine learning artificial intelligence structural racism

Get full access to this article

View all access options for this article.

References

Aaronson

Hartley

Mazumder

(2021). The effects of the 1930s HOLC “redlining” maps. American Economic Journal: Economic Policy, 13(4), 355–392. https://doi.org/10.1257/pol.20190414

Abnar

Dehghani

Zuidema

(2020). Transferring inductive biases through knowledge distillation. arXiv preprint arXiv:2006.00555.

Adebayo

J. A.

(2016). FairML: ToolBox for diagnosing bias in predictive modeling. Massachusetts Institute of Technology.

Adomavicius

Yang

(2019). Integrating behavioral, economic, and technical insights to address algorithmic bias: Challenges and opportunities for is research. Economic, and Technical Insights to Address Algorithmic Bias: Challenges and Opportunities for IS Research (September 3, 2019).

Agarwal

Amromin

Ben-David

, et al. (2014). Predatory lending and the subprime crisis. Journal of Financial Economics, 113(1), 29–52. https://doi.org/10.1016/j.jfineco.2014.02.008

Ajunwa

(2021). An auditing imperative for automated hiring. Harvard Journal of Law & Technology, 34(2), 1. https://doi.org/10.2139/ssrn.3437631

Akhtar

(2019). New York is investigating UnitedHealth's use of a medical algorithm that steered black patients away from getting higher-quality care. Business Insider.

Akinwumi

Merrill

Rice

, et al. (2021). An AI fair lending policy agenda for the federal Financial regulators. Economic Studies at Brookings, 2, 1–18.

Almada

(2019). Human intervention in automated decision-making: Toward the construction of contestable systems. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law, 2–11. https://doi.org/10.1145/3322640.3326699

10.

Amini

Soleimany

A. P.

Schwarting

, et al. (2019). Uncovering and mitigating algorithmic bias through learned latent structure. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 289–295. https://doi.org/10.1145/3306618.3314243

11.

Angwin

Larson

Mattu

, et al. (2016). Machine bias. In Ethics of data and analytics (pp. 254–264). Auerbach Publications.

12.

Babuta

Oswald

(2019). Data analytics and algorithmic bias in policing. Briefing paper for The Royal United Services institute for defence and security studies.

13.

Baker

Dellaert

(2017). Regulating robo advice across the financial services industry. Iowa L. Rev, 103(5), 713. https://10.2139/ssrn.2932189

14.

Banker

Khetani

(2019). Algorithm overdependence: How the use of algorithmic recommendation systems can increase risks to consumer well-being. Journal of Public Policy & Marketing, 38(4), 500–515. https://doi.org/10.1177/0743915619858057

15.

Bankrate.com. (2020, January 15). Minorities, Millennials Among Those Who Pay the Most Bank Fees. https://www.bankrate.com/pdfs/pr/20200115-best-banks-survey.pdf

16.

Baptiste

(2013, October 13). Staggering loss of Black wealth due to subprime scandal continues unabated. The American Prospect.

17.

Barocas

Selbst

A. D.

(2016). Big data's disparate impact. California Law Review, 104, 671. https://doi.org/10.2139/ssrn.2477899

18.

Barzilay

A. R.

Ben-David

(2016). Platform inequality: Gender in the gig-economy. Seton Hall Law Review, 47(2), 393. https://doi.org/10.2139/ssrn.2995906

19.

Bellamy

R. K.

Dey

Hind

, et al. (2019). AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5), 4: 1–4: 15. https://doi.org/10.1147/JRD.2019.2942287

20.

Berghel

(2018). Malice domestic: The Cambridge analytica dystopia. Computer, 51(05), 84–89. https://doi.org/10.1109/MC.2018.2381135

21.

Berk

(2017). An impact assessment of machine learning risk forecasts on parole board decisions and recidivism. Journal of Experimental Criminology, 13(2), 193–216. https://doi.org/10.1007/s11292-017-9286-2

22.

Bertrand

Weill

(2021). Do algorithms discriminate against African Americans in lending? Economic Modelling, 104: 105619. https://doi.org/10.1016/j.econmod.2021.105619

23.

Bhutta

Chang

A. C.

Dettling

L. J.

(2020). Disparities in wealth by race and ethnicity in the 2019 Survey of Consumer Finances.

24.

Blanchflower

D. G.

Levine

P. B.

Zimmerman

D. J.

(2003). Discrimination in the small-business credit market. Review of Economics and Statistics, 85(4), 930–943. https://doi.org/10.1162/003465303772815835

25.

Blattner

Nelson

(2021). How costly is noise? Data and disparities in consumer credit. arXiv preprint arXiv:2105.07554.

26.

Blattner

Nelson

Spiess

(2021). Unpacking the black box: Regulating algorithmic decisions. arXiv preprint arXiv:2110.03443.

27.

Bond

R. M.

Fariss

C. J.

Jones

J. J.

, et al. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415), 295–298. https://doi.org/10.1038/nature11421

28.

Broady

McComas

Ouazad

(2021). An analysis of financial institutions in black-majority communities: black borrowers and depositors face considerable challenges in accessing banking services. Brookings Institution.

29.

Broussard

(2018). Artificial unintelligence: How computers misunderstand the world. MIT Press.

30.

Brynjolfsson

Mitchell

(2017). What can machine learning do? Workforce implications. Science, 358(6370), 1530–1534. https://doi.org/10.1126/science.aap8062

31.

Bryson

Winfield

(2017). Standardizing ethical design for artificial intelligence and autonomous systems. Computer, 50(5), 116–119. https://doi.org/10.1109/MC.2017.154

32.

Buolamwini

(2022). Facing the Coded Gaze with Evocative Audits and Algorithmic Audits. Massachusetts Institute of Technology.

33.

Burk

D. L.

(2021). Racial Bias in Algorithmic IP.

34.

Burt

(2017, November 13). Facebook files patent for facial recognition for physical payments. BiometricUpdate.com. https://www.biometricupdate.com/201711/facebook-files-patent-for-facial-recognition-for-physical-payments.

35.

Cambo

S. A.

Gergle

(2022). Model positionality and computational reflexivity: Promoting reflexivity in data science. CHI Conference on Human Factors in Computing Systems, 1–19. https://doi.org/10.1145/3491102.3501998

36.

Canaday

Reback

(2010). Race, literacy, and real estate transactions in the postbellum south. The Journal of Economic History, 70(2), 428–445. https://doi.org/10.1017/S0022050710000343

37.

Chan-Olmsted

S. M.

(2019). A review of artificial intelligence adoptions in the media industry. International Journal on Media Management, 21(3-4), 193–215. https://doi.org/10.1080/14241277.2019.1695619

38.

Chouldechova

(2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047

39.

Clark

W. A.

(1986). Residential segregation in American cities: A review and interpretation. Population Research and Policy Review, 5(2), 95–127. https://doi.org/10.1007/BF00137176

40.

Coates

D. L.

Martin

(2019). An instrument to evaluate the maturity of bias governance capability in artificial intelligence projects. IBM Journal of Research and Development, 63(4/5), 7: 1–7: 15. https://doi.org/10.1147/JRD.2019.2915062

41.

Collins

B. W.

(2007). Tackling unconscious bias in hiring practices: The plight of the Rooney Rule. New York University Law Review, 82(4), 870.

42.

Conn. Fair Hous. (2020). Ctr. v. CoreLogic Rental Prop. Sols.

43.

Cooper

(2022, July 19). The Impact Of AI & How It Is Used In Social Media. StatusBrew. https://statusbrew.com/insights/social-media-ai/

44.

Corbett-Davies

Pierson

Feller

, et al. (2017). Algorithmic decision making and the cost of fairness. Proceedings of the 23rd international conference on knowledge discovery and data mining. 797–806.

45.

Corwin

T. S.

III Johnson

D. K. N.

(2021). Plus a life sentence? Incarceration’s effects on expected lifetime wage growth. Research in Applied Economics, 13(1), 1–4. https://doi.org/10.5296/rae.v13i1.18283

46.

Danks

London

A. J.

(2017). Algorithmic bias in autonomous systems. Ijcai, 4691–4697. https://doi.org/10.24963/ijcai.2017/654

47.

Dash

McMurtrey

Rebman

Kar

U. K.

(2019). Application of artificial intelligence in automation of supply chain management. Journal of Strategic Innovation and Sustainability, 14(3), 43–53. https://doi.org/10.33423/jsis.v14i3.2105

48.

Dastin

(2018). Amazon Scraps secret AI recruiting tool that showed bias against women. In Ethics of data and analytics (pp. 296–299). Auerbach Publications.

49.

Datta

Sen

Zick

(2016). Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. 2016 IEEE symposium on security and privacy (SP). IEEE, 598-617.

50.

Dobbe

Dean

Gilbert

, et al. (2018). A broader view on bias in automated decision-making: Reflecting on epistemology and dynamics. arXiv preprint arXiv:1807.00553.

51.

Domanski

J. R.

(2019). The AI pandorica: Linking ethically-challenged technical outputs to prospective policy approaches. Proceedings of the 20th Annual International Conference on Digital Government Research, 409–416. https://doi.org/10.1145/3325112.3325267

52.

Edelson

(2021, April 8). Target Leans Into Racial Equity With $2 Billion Commitment To Black-Owned Businesses. Forbes.

53.

Ensign

Friedler

S. A.

Neville

, et al. (2018). Runaway feedback loops in predictive policing. Conference on Fairness, Accountability and Transparency. PMLR, 160–171.

54.

Eubanks

(2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin's Press.

55.

Eubanks

(2020). Coded Bias. Netflix documentary.

56.

Evans

Miller

(2019). From catalogs to clicks: The fair lending implications of targeted, internet marketing. Consumer Compliance Outlook, 3(1), 1–9.

57.

Executive Office of the President. (2016). Artificial Intelligence, Automation and the Economy. White House.

58.

Faber

J. W.

(2013). Racial dynamics of subprime mortgage lending at the peak. Housing Policy Debate, 23(2), 328–349. https://doi.org/10.1080/10511482.2013

59.

Fairlie

Robb

Robinson

D. T.

(2022). Black and white: Access to capital among minority-owned start-ups. Management Science, 68(4), 2377–2400. https://doi.org/10.1287/mnsc.2021.3998

60.

Federal Deposit Insurance Corporation, FDIC. (2020). Minority Depository Institutions Program. https://www.fdic.gov/minority-depository-institutions-program.

61.

Federal Register. (2021, March 31). Comment on Financial Institutions’ Use of Artificial Intelligence, including Machine Learning. https://www.federalregister.gov/documents-/2021 /03/31/2021-06607/request-for-information-and-comment-on-financial-institutions-use-of-artificial-intelligence

62.

Federal Trade Commission. (2018, October 16). Press Release: RealPage FTC Settlement Statement.

63.

Firlej

Taeihagh

(2021). Regulating human control over autonomous systems. Regulation & Governance, 15(4), 1071–1091. https://10.1111/rego.12344

64.

Fishback

Rose

Snowden

K. A.

, et al. (2022). New evidence on redlining by federal housing programs in the 1930s. Journal of Urban Economics, 103462. https://doi.org/10.1016/j.jue.2022.02.002

65.

Fligstein

Goldstein

(2011). Catalyst of disaster: Subprime mortgage securitization and the roots of the great recession. UC Berkeley Working Paper Series, 2011-09-02. https://escholarship.org/uc/item/83(2h03n

66.

Flynn

(2020). 13 Cities Where Police Are Banned From Using Facial Recognition Tech. Innovation & Tech Today (13).

67.

Fox

Tariq

Thomas

, et al. (2019). Bank branch closures take greatest toll on majority-black Areas. S&P Global Report.

68.

Freedman

(2017). The information value of online social networks: Lessons from peer-to-peer lending. International Journal of Industrial Organization, 51: 185–222. https://doi.org/10.1016/j.ijindorg.2016.09.002

69.

Fuster

Goldsmith-Pinkham

Ramadorai

, et al. (2022). Predictably unequal? The effects of machine learning on credit markets. The Journal of Finance, 77(1), 5–47. https://doi.org/10.2139/ssrn.3072038

70.

Garcia

(2016). Racist in the machine. World Policy Journal, 33(4), 111–117. https://doi.org/10.1215/07402775-3813015

71.

Garvie

Frankle

(2016). Facial-Recognition Software Might Have a Racial Bias Problem. The Atlantic.

72.

Gendler

T. S.

(2011). On the epistemic costs of implicit bias. Philosophical Studies, 156(1), 33–63. https://doi.org/10.1007/s11098-011-9801-7

73.

Goddard

Roudsari

Wyatt

J. C.

(2012). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127. https://doi.org/10.1136/amiajnl-2011-000089

74.

Granitz

(2022). Texas sues Meta, saying it misused facial recognition data. National Public Radio.

75.

Guidotti

Monreale

Giannotti

, et al. (2019). Factual and counterfactual explanations for black box decision making. IEEE Intelligent Systems, 34(6), 14–23.

76.

Hacker

(2018). Teaching fairness to artificial intelligence: existing and novel strategies against algorithmic discrimination under EU law. Common Market Law Review, 55(4), 1145–1186. https://doi.org/10.54648/cola2018095

77.

Haenschen

(2016). Social pressure on social Media: Using Facebook Status updates to increase voter turnout. Journal of Communication, 66(4), 542–563. https://doi.org/10.1111/jcom.12236

78.

Hagendorff

(2020). The ethics of AI ethics: An evaluation of guidelines. Minds and Machines, 30(1), 99–120. https://doi.org/10.1007/s11023-020-09517-8

79.

Hamilton

I. A.

(2018, October 13). Why it's totally unsurprising that Amazon's recruitment AI was biased against women. Business Insider.

80.

Hardt

(2014, September 26). How big data is unfair. Medium.

81.

Hardt

Price

Srebro

(2016). Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems, 29, 3315–3323. https://doi.org/10.48550/arXiv.1610.02413

82.

Harvard Business Review Staff. (2014). With big data comes big responsibility. Harvard Business Review, 92(11), 100–104.

83.

Heaven

W. D.

(2022). Bias isn't the only problem with credit scores—and No, AI can't help. In Ethics of data and analytics (pp. 300–302). Auerbach Publications.

84.

Henderson

Herring

Horton

H. D.

, et al. (2015). Credit where credit is due?: Race, gender, and discrimination in the credit scores of business startups. The Review of Black Political Economy, 42(4), 459–479. https://doi.org/10.1007/s12114-015-9215-4

85.

Hill

(2012). How target figured out a teen girl was pregnant before her father did. Forbes, Inc, 7, 4–1.

86.

Homonoff

O'Brien

Sussman

(2017). Does Knowing Your FICO Score Change Financial Behavior? Evidence From a Field Experiment With Student Loan Borrowers. NYU Wagner Research Paper. https://doi.org/10.3386/w26048

87.

Horowitz

J. M.

Igielnik

Kochhar

(2020, January 9). Trends in income and wealth inequality. Pew Research Center Report.

88.

(2020). Cambridge analytica’s black box. Big Data & Society, 7(2), 2053951720938091. https://doi.org/10.1177/2053951720938091

89.

Huq

A. Z.

(2018). Racial equity in algorithmic criminal justice. Duke LJ, 68(6), 1043.

90.

Hutchinson

Mitchell

(2019). 50 Years of test (un) fairness: Lessons for machine learning. Proceedings of the Conference on Fairness, Accountability, and Transparency, 49–58. https://doi.org/10.1145/3287560.3287600

91.

Johnson

G. M.

(2021). Algorithmic bias: On the implicit biases of social technology. Synthese, 198(10), 9941–9961. https://doi.org/10.1007/s11229-020-02696-y

92.

Kahn

(2021). HireVue drops facial monitoring amid AI algorithm audit. Fortune. http://fortune.com/2021/01/19/hirevue-drops-facial-monitoring-amid-ai-algorithm-audit

93.

Kahn

(2022). After Uvalde: Could A.I. prevent another school shooting? Forbes.

94.

Kahneman

(2011). Thinking, fast and slow. Macmillan.

95.

Kamiran

Karim

Verwer

, et al. (2012). Classifying socially sensitive data without discrimination: An analysis of a crime suspect dataset. 2012 IEEE 12th International Conference on Data Mining Workshops. IEEE, 370–377.

96.

Kammerer

(2021). Hired by a robot: The legal implications of artificial intelligence video interviews and advocating for greater protection of job applicants. Iowa Law Review, 107(2), 817–850.

97.

Kaoudi

Quiané-Ruiz

J.-A.

(2018). Cross-platform data processing: use cases and challenges. 2018 IEEE 34th International Conference on Data Engineering (ICDE). IEEE, 1723–1726.

98.

Katyal

S. K.

(2019). Private accountability in the age of artificial intelligence. UCLA Law Review, 66(1), 54–148.

99.

Kay

Matuszek

Munson

S. A.

(2015). Unequal representation and gender stereotypes in image search results for occupations. Proceedings of the 33rd Annual acm Conference on Human Factors in Computing Systems, 3819–3828. https://doi.org/10.1145/2702123.2702520

100.

Kilbertus

Rojas Carulla

Parascandolo

, et al. (2017). Avoiding discrimination through causal reasoning. Advances in Neural Information Processing Systems, 30, 656–666. https://doi.org/10.48550/arXiv.1706.02744

101.

Kirkpatrick

(2016). Battling algorithmic bias: How do we ensure algorithms treat us fairly? Communications of the ACM, 59(10), 16–17. https://doi.org/10.1145/2983270

102.

Kleinberg

(2018). Inherent trade-offs in algorithmic fairness. In Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems. 40-40.

103.

Kleinberg

Ludwig

Mullainathan

, et al. (2018). Algorithmic fairness. AEA papers and proceedings. 22–27.

104.

Koene

(2017). Algorithmic bias: Addressing growing concerns [leading edge]. IEEE Technology and Society Magazine, 36(2), 31–32. https://doi.org/10.1109/MTS.2017.2697080

105.

Koene

Dowthwaite

Seth

(2018). IEEE P7003™ standard for algorithmic bias considerations: Work in progress paper. Proceedings of the International Workshop on Software Fairness, 38–41. https://doi.org/10.1145/3194770.3194773

106.

Kordzadeh

Ghasemaghaei

(2022). Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388–409. https://doi.org/10.1080/0960085X.2021.1927212

107.

Kotras

(2020). Mass personalization: Predictive marketing algorithms and the reshaping of consumer knowledge. Big Data & Society, 7(2), 2053951720951581. https://doi.org/10.1177/2053951720951581

108.

Kraft

(2016, March 25). Microsoft shuts down AI chatbot after it turned into a Nazi. CBS News. https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/

109.

Kubrin

C. E.

Stewart

E. A.

(2006). Predicting who reoffends: The neglected role of neighborhood context in recidivism studies. Criminology, 44(1), 165–197. https://doi.org/10.1111/j.1745-9125.2006.00046.x

110.

Lambrecht

Tucker

C. E.

(2020). Apparent algorithmic discrimination and real-time algorithmic learning in digital search advertising. Available at SSRN 3570076.

111.

Laperruque

(2022). The Facial Recognition Act: A Promising Path to Put Guardrails on a Dangerously Unregulated Surveillance Technology. Lawfare.

112.

Leadership Conference of Civil and Human Rights. (2020). Civil Rights Leaders Announce Principles to Protect Civil Rights and Technology. https://civilrights.org/2020/10/21/ civil-rights-leaders-announce-principles-to-protect-civil-rights-and-technology/#

113.

Lee

N. T.

(2018). Detecting racial bias in algorithms and machine learning. Journal of Information, Communication and Ethics in Society, 16(2), 148–162. https://doi.org/10.1108/JICES-06-2018-0056

114.

Lippens

Baert

Ghekiere

, et al. (2022). Is labour market discrimination against ethnic minorities better explained by taste or statistics? A systematic review of the empirical evidence. Journal of Ethnic and Migration Studies, 48(1), 1–34. https://doi.org/10.1080/1369183X.2022.2050191

115.

Ludwig

(2015). Credit scores in America perpetuate racial injustice. Here’s how. The Guardian 13.

116.

Lynch

E. E.

Malcoe

L. H.

Laurent

S. E.

, et al. (2021). The legacy of structural racism: Associations between historic redlining, current mortgage lending, and health. SSM-Population Health, 14, 100793. https://doi.org/10.1016/j.ssmph.2021.100793

117.

Maas

(2022). Machine learning and power relations. AI & SOCIETY, 37(1), 1–8. https://doi.org/10.1007/s00146-022-01400-7

118.

Marinchak

C. L. M.

Forrest

Hoanca

(2018). The impact of artificial intelligence and virtual personal assistants on marketing. In Encyclopedia of information science and technology (Fourth Edition, pp. 5748–5756). IGI Global.

119.

Marr

(2016). 4 Mind-Blowing Ways Facebook Uses Artificial Intelligence.

120.

Martinez

Kirchner

(2021). The Secret Bias Hidden in Mortgage-Approval Algorithms. Center for Public Integrity report. https://publicintegrity.org/inequality-poverty-opportunity/bias-mortgage-approval-algorithms/

121.

Mayer

C. J.

Pence

(2008). Subprime mortgages: what, where, and to whom? National Bureau of Economic Research Working Paper w14083.

122.

Mazurowski

M. A.

Habas

P. A.

Zurada

J. M.

, et al. (2008). Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance. Neural Networks, 21(2-3), 427–436. https://doi.org/10.1016/j.neunet.2007.12.031

123.

McKinney

(2019, August 30). Looking Back at the History of America’s Black Banks, Even as They Strive For Vitality. Black Enterprise.

124.

Menczer

(2021, October 7). Here’s exactly how social media algorithms can manipulate you. Big Think. https://bigthink.com/the-present/social-media-algorithms-manipulate-you/

125.

Moore

Tom

O’Neil

(2024). The Truth Behind Crime Statistics: Avoiding Distortions and Improving Public Safety. Thurgood Marshall Institute: Criminal Justice No. 8.

126.

Mui

Y. Q.

(2012, July 8). For black Americans, financial damage from subprime implosion is likely to last. Washington Post.

127.

National Farmers Union. (2020). Juneteenth and the Broken Promise of 40 Acres and a Mule. https://nfu.org/2020/06/19/juneteenth-and-the-broken-promise-of-40-acres-and-a-mule/

128.

National Park Service. (2022). African American Homesteaders in the Great Plains. https://www.nps.gov/articles/african-american-homesteaders-in-the-great-plains.htm

129.

Noble

S. U.

(2018). Algorithms of oppression. Algorithms of Oppression. New York University Press.

130.

Obermeyer

Powers

Vogeli

, et al. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342

131.

Okun

A. M.

(2015). Equality and efficiency: The big tradeoff. Brookings Institution Press.

132.

O'neil

(2017). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.

133.

Paige

M. A.

Amrein-Beardsley

(2020). Houston, we have a lawsuit”: A cautionary tale for the implementation of value-added models for high-stakes employment decisions. Educational Researcher, 49(5), 350–359. https://doi.org/10.3102/0013189X20923046

134.

Parasuraman

Min

B.-C.

(2019). Consensus control of distributed robots using direction of arrival of wireless signals. In Distributed autonomous robotic systems (pp. 17–34). Springer.

135.

Paul

(2024). When did credit scores start? A brief look at the long history behind credit reporting. CNBC. https://www.cnbc.com/select/when-did-credit-scores-start

136.

Pedersen

(2005). Stanislav Petrov World Hero. Fellowship, 71(7-8), 9.

137.

Petrasic

Saul

Greig

, et al. (2017). Algorithms and bias: What lenders need to know. White and Case, LLP.

138.

Petty

R. D.

Harris

A.-M. G.

Broaddus

, et al. (2002). Regulating target marketing and other race-related advertising practices. Mich. J. Race & L, 8(2002), 335. repository.law.umich.edu/mjrl/vol8/iss2/1

139.

Prince

(2014). How media have shaped our perception of race and crime. The Root, September 4. theroot.com/how-media-have-shaped-our-perception-of-race-and-crime-1790885677

140.

Pullin

(2013). Legal issues in the use of student test scores and value-added models (VAM) to determine educational quality. Education Policy Analysis Archives, 21(6), n6. https://doi.org/10.14507/epaa.v21n6.2013

141.

Rice

Swesnik

(2013). Discriminatory effects of credit scoring on communities of color. Suffolk University Law Rev, 46(4), 935–976.

142.

Ritter

(2013). Predicting recidivism risk: New tool in Philadelphia shows great promise. National Institute of Justice Journal, 271(February), 4–13.

143.

Robinson

Koepke

(2016). Stuck in a Pattern. Early evidence on “predictive policing” and civil rights. https://www.upturn.org/reports/2016/stuck-in-a-pattern/[Zugriffsdatum : 05.07.2019 .

144.

Romei

Ruggieri

(2014). A multidisciplinary survey on discrimination analysis. The Knowledge Engineering Review, 29(5), 582–638. https://doi.org/10.1017/S0269888913000039

145.

Sanober

Alam

Pande

, et al. (2021). An enhanced secure deep learning algorithm for fraud detection in wireless communication. Wireless Communications and Mobile Computing, 2021. https://doi.org/10.1155/2021/6079582

146.

Sapir

(2009). From financial crisis to turning point. How the US subprime crisis turned into a world-wide one and will change the global economy. Internationale Politik und Gesellschaft, 1(2009), 27–44.

147.

Schwartz

(2019). In 2016, microsoft’s racist chatbot revealed the dangers of online conversation. IEEE Spectrum, 11(2019), 2019.

148.

Seyyed-Kalantari

Zhang

McDermott

, et al. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182. https://doi.org/10.1038/s41591-021-01595-0

149.

Sheridan

T. B.

Verplank

W. L.

Brooks

T. L.

(1978). Human/computer control of undersea teleoperators. NASA. Ames Res. Center The 14th Ann. Conf. on Manual Control.

150.

Shirgave

Awati

, et al. (2019). A review on credit card fraud detection using machine learning. International Journal of Scientific & Technology Research, 8(10), 1217–1220.

151.

Shrestha

Y. R.

Yang

(2019). Fairness in algorithmic decision-making: Applications in multi-winner voting, machine learning, and recommender systems. Algorithms, 12(9), 199. https://doi.org/10.3390/a12090199

152.

Silva

Kenney

(2019). Algorithms, platforms, and ethnic bias. Communications of the ACM, 62(11), 37–39. https://doi.org/10.1145/3318157

153.

Simonitis

(2021). Facial recognition technology and the constitution. Notre Dame Journal on Emerging Technologies (JET), 2(2), 357–378.

154.

Skirpan

Gorelick

(2017). The Authority of” Fair” in Machine Learning. arXiv preprint arXiv:1706.09976.

155.

Sood

Speagle

Ehrman-Solberg

(2019). Long shadow of racial discrimination: Evidence from housing covenants of Minneapolis. Available at SSRN 3468520.

156.

Steele

C. M.

Aronson

(1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69(5), 797. https://doi.org/10.1037/0022-3514.69.5.797

157.

Stone

Brooks

Brynjolfsson

, et al. (2016). Artificial intelligence and life in 2030. One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel 52.

158.

Sun

(2018). The dangerous radicalization of crime in US news media. Center for American Progress, August 29.

159.

Sweet

(2022). Anti-discriminatory bank law getting update for modern era. AP News. May 5. https://apnews.com/article/business-race-and-ethnicity-1cba97a43a966228c0faf0ba18434f52.

160.

Taeihagh

Lim

H. S. M.

(2019). Governing autonomous vehicles: Emerging responses for safety, liability, privacy, cybersecurity, and industry risks. Transport Reviews, 39(1), 103–128. https://doi.org/10.1080/01441647.2018.1494640

161.

Tait

W. W.

(2001). Beyond the axioms: The question of objectivity in mathematics. Philosophia Mathematica, 9(1), 21–36. https://doi.org/10.1093/philmat/9.1.21

162.

Thaler

R. H.

Sunstein

C. R.

(2021). Nudge. Yale University Press.

163.

Thorbecke

(2019, November 11). New York probing Apple Card for alleged gender discrimination after viral tweet. ABC News. https://abcnews.go.com/US/york-probing-apple-card-alleged-gender-discrimination-viral/story?id=66910300

164.

Tiku

(2018). Why Netflix features Black actors in promos to Black users. Wired, 20, 20.

165.

Trouiller

Olliaro

Torreele

, et al. (2017). Drug development for neglected diseases: A deficient market and a public-health policy failure. Globalization and Health, 13(1), 267–273. https://doi.org/10.1016/s0140-6736(02)09096-7

166.

Tufekci

(2017). Twitter and tear gas: The power and fragility of networked protest. Yale University Press.

167.

UNESCO. (2019). Preliminary study on the Ethics of Artificial Intelligence .

168.

Verma

Rubin

(2018). Fairness definitions explained. 2018 IEEE/ACM international workshop on software fairness (fairware). IEEE, 1–7.

169.

Vincent

(2016). Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day. The Verge March 24.

170.

Vogell

(2022). Rent Going Up? One Company’s Algorithm Could Be Why.Rent Going Up? One Company’s Algorithm Could Be Why.: ProPublica.

171.

Wake

(2019). The Shocking Truth 50 Years After The 1968 Fair Housing Act: The Black Homeownership Paradox. Forbes May 16.

172.

Walton

G. M.

Murphy

M. C.

Ryan

A. M.

(2015). Stereotype threat in organizations: Implications for equity and performance. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 523–550. https://doi.org/10.1146/annurev-orgpsych-032414-111322

173.

Webb

(2019). The big nine: How the tech titans and their thinking machines could warp humanity. Hachette UK.

174.

West

Baker

A. C.

, et al. (2021). Reversing the gains of the civil rights and women’s movements: How housing strain and market exclusion led to wealth depletion during the Great Recession. Journal of the Society for Social Work and Research, 12(2), 263–282. https://doi.org/10.1086/714544

175.

Williams

B. A.

Brooks

C. F.

Shmargad

(2018). How algorithms discriminate based on data they lack: Challenges, solutions, and policy implications. Journal of Information Policy, 8(1), 78–115. https://doi.org/10.5325/jinfopoli.8.2018.0078

176.

Wong

P.-H.

(2020). Democratizing algorithmic fairness. Philosophy & Technology, 33(2), 225–244. https://doi.org/10.1007/s13347-019-00355-w

177.

Yglesias

(2013). What’s All the Land in America Worth? Slate December 20.

178.

Yinger

(2018). Discrimination in mortgage lending: A literature review. Mortgage lending, racial discrimination, and federal policy. 29–73.

179.

Zliobaite

(2015). A survey on measuring indirect discrimination in machine learning. arXiv Preprint arXiv:1511.00148.

180.

Zunger

(2018, March 22). Computer science faces an ethics crisis. The Cambridge Analytica scandal proves it. Boston Globe.

Gaslighting Ourselves: Racial Challenges of Artificial Intelligence in Economics and Finance Applications

Abstract

Keywords

Get full access to this article

References