Sage Journals: Discover world-class research

Abstract

This article discusses the dangers of the Moral Machine (MM) experiment, alerting against both its uses for normative ends and the whole approach it is built upon to address ethical issues. It explores additional methodological limits of the experiment on top of those already identified by its authors; exhibits the dangers of computational moral systems for modern democracies, such as the “voting-based system” recently developed out of the MM’s data; and provides reasons why ethical decision-making fundamentally excludes computational social choice methods.

Keywords

autonomous vehicle self-driving car AI ethics Moral Machine trolley problem

Get full access to this article

View all access options for this article.

References

Stone

(2010). Motion planning algorithms for autonomous intersection management. In Proceedings of the 1st AAAI Conference on Bridging the Gaps Between Task and Motion Planning (pp. 2–9). AAAI Press.

Awad

Sohan

Richard

Jonathan

Joseph

Azim

Jean-François

Iyad

(2018). The moral machine experiment. Nature, 563, 59–64.

Bauman

C. W.

Peter

M. A.

Daniel

B. M.

Caleb

(2014). Revisiting external validity: Concerns about trolley problems and other sacrificial dilemmas in moral psychology. Social and Personality Psychology Compass, 8, 536–554.

Bauman

Youngblood

(2017). Why waiting for perfect autonomous vehicles may cost lives. RAND Corporation.

Bonnefon

Azim

Iyad

(2015). Autonomous vehicles need experimental ethics: Are we ready for utilitarian cars? ArXiv. https://128.84.21.199/abs/1510.03346v1

Bonnefon

Azim

Iyad

(2016). The social dilemma of autonomous vehicles. Science, 352, 1573–1576.

Bostyn

D. H.

Sybren

Arne

(2018). Of mice, men, and trolleys: Hypothetical judgment versus real-life behavior in trolley-style moral dilemmas. Psychological Science, 29, 1084–1093.

Conitzer

Walter

Jana

B. S.

Yuan

Max

(2017). Moral decision making frameworks for artificial intelligence. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (pp. 4831–4835). AAAI Press.

Dredge

(2015). Elon Musk: Self-driving cars could lead to ban on human drivers. The Guardian. https://www.theguardian.com/technology/2015/mar/18/elon-musk-self-driving-cars-ban-human-drivers

10.

Dwork

Moritz

Toniann

Reingold

Rich

Z. S.

(2012). Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (pp. 214–226). Association for Computing Machinery.

11.

Etienne

(in press). A practical role-based approach to solve moral dilemmas for self-driving cars.

12.

European Commission. (2018). On the road to automated mobility: An EU strategy for mobility of the future (p. 283). COM.

13.

Federal Ministry of Transport and Digital Infrastructure. (2017). Ethics commission on automated and connected driving [Report]. https://www.bmvi.de/SharedDocs/EN/publications/report-ethics-commission.pdf?__blob=publicationFile

14.

Food and Agriculture Organization. (2018). The state of food security and nutrition in the world. United Nations.

15.

Foot

(1967). The problem of abortion and the doctrine of double effect. Oxford Review, 5, 5–15.

16.

Francis

K. B.

Charles

Ian

H. S.

Michaela

Giorgio

Grace

Sylvia

(2017). Virtual morality: Transitioning from moral judgment to moral action? PLoS One, 12, e0171793.

17.

Greene

Francesca

John

Kristen

V. B.

Brian

(2016). Embedding ethical principles in collective decision support systems. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (pp. 4147–4151). Association for Computing Machinery.

18.

Hevelke

Nida-Rümelin

(2015). Responsibility for crashes of autonomous vehicles: An ethical analysis. Science and Engineering Ethics, 21, 619–630.

19.

Kalra

Groves

D. G.

(2017). The enemy of good: Estimating the cost of waiting for nearly perfect automated vehicles. RAND Corporation.

20.

Lin

(2013). The ethics of saving lives with autonomous cars is far murkier than you think. Wired. https://www.wired.com/2013/07/the-surprising-ethics-of-robot-cars/

21.

Lin

(2014). Here’s a terrible idea: Robot cars with adjustable ethics settings. Wired. https://www.wired.com/2014/08/heres-a-terrible-idea-robot-cars-with-adjustable-ethics-settings/

22.

Lin

(2015). The ethical dilemma of self-driving cars. Ted-Ed. https://www.ted.com/talks/patrick_lin_the_ethical_dilemma_of_self_driving_cars?language=en

23.

Loh

(2017). Autonomy and responsibility in hybrid systems: The example of autonomous cars. In Lin

Abney

Jenkins

(Eds.), Robot ethics 2.0 (pp. 35–50). Oxford University Press.

24.

Millar

(2014). Technology as moral proxy: Autonomy and paternalism by design. In Proceedings of the 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering. Association for Computing Machinery.

25.

Mohr

Hans-Werner

Paul

Dominik

Timo

(2016). Automotive revolution-perspective towards 2030. How the convergence of disruptive technology-driven trends could transform auto industry. McKinsey & Company.

26.

Noothigattu

Snehalkumar

Neil

Edmond

Sohan

Iyad

Pradeep

Ariel

P. D.

(2018). A voting-based system for ethical decision making. In Proceedings of the 32th AAAI Conference on Artificial Intelligence. AAAI Press.

27.

Nyholm

Smids

(2016). The ethics of accident-algorithms for self-driving cars: An applied trolley problem? Ethical Theory and Moral Practice, 19, 1275–1289.

28.

Parfit

(1986). Reasons and persons. Oxford University Press.

29.

Rozieres

(2018). Les voitures autonomes doivent-elles vous sacrifier pour sauver un enfant ou un chien? [Should autonomous cars sacrifice you to save a child or a dog?] The Huffington Post. https://www.huffingtonpost.fr/2018/10/24/les-voitures-autonomes-doivent-elles-vous-sacrifier-pour-sauver-un-enfant-ou-un-chien_a_23570383/

30.

Sandberg

Bradshaw-Martin

(2013). What do cars think of trolley problems: Ethics for autonomous cars? In Proceedings of the 2013 International Conference, Beyond AI. Springer.

31.

Shariff

Jean-François

Iyad

(2017). Psychological roadblocks to the adoption of self-driving vehicles. Nature Human Behaviour, 1, 694–696.

32.

Singh

(2015). Critical reasons for crashes investigated in the National motor vehicle crash causation survey [Traffic Safety Facts Crash Stats. Report No. DOT HS 812 115]. National Highway Traffic Safety Administration.

33.

Sparrow

Howard

(2017). When human beings are like drunk robots: Driverless vehicles, ethics, and the future of transport. Transportation Research Part C: Emerging Technologies, 80, 206–215.

34.

Taylor

(2017, October 14). Political identity and the problem of democratic exclusion. ABC Religion and Ethics. http://www.abc.net.au/religion/articles/2016/04/29/4452814.htm

35.

Thomson

(1976). Killing, letting die, and the trolley problem. The Monist, 59, 204–217.

36.

Thomson

J. J.

(1985). The trolley problem. Yale Law Journal, 94, 1395–1415.

37.

Vincent

(2018). Global preferences for who to save in self-driving car crashes revealed. The Verge. https://www.theverge.com/2018/10/24/18013392/self-driving-car-ethics-dilemma-mit-study-moral-machine-results

38.

Wallach

Allen

(2008). Moral machines: Teaching robots right from wrong. Oxford University Press.

39.

Wood

(2011). Humanity as an end in itself. In Parfit

Scheffler

(Eds.), On what matters (Vol. 2, pp. 58–82). Oxford University Press.

40.

World Health Organization. (2018). Global status report on road safety 2018. https://www.who.int/violence_injury_prevention/road_safety_status/2018/en/

When AI Ethics Goes Astray: A Case Study of Autonomous Vehicles

Abstract

Keywords

Get full access to this article

References