Against generalisation: Data-driven decisions need context to be human-compatible

Abstract

During the past two decades, there have been a number of breakthroughs in the fields of data science and artificial intelligence, made possible by advanced machine learning algorithms trained through access to massive volumes of data. However, their adoption and use in real-world applications remains a challenge. This paper posits that a key limitation in making AI applicable has been a failure to modernise the theoretical frameworks needed to evaluate and adopt outcomes. Such a need was anticipated with the arrival of the digital computer in the 1950s but has remained unrealised. This paper reviews how the field of data science emerged and led to rapid breakthroughs in algorithms underpinning research into artificial intelligence. It then discusses the contextual framework now needed to advance the use of AI in real-world decisions that impact human lives and livelihoods.

Keywords

Algorithm behaviour big data cognition context decision

Get full access to this article

View all access options for this article.

References

Alvi

Kharya

(2021) Using DeepSpeed and megatron to train megatron-turing NLG 530B, the world’s largest and most powerful generative language model. Available at: https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/ (accessed 23 October 2021).

Anderson

(2008) The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine. Available at: https://www.wired.com/2008/06/pb-theory/(accessed 25 October 2021).

Bengio

(2009) Learning Deep Architectures for AI. Foundations and Trends in Machine Learning 2: 1–127. DOI: 10.1561/2200000006

Best

(2013) IBM watson: The inside story of how the jeopardy-winning supercomputer was born, and what it wants to do next. Available at: https://www.techrepublic.com/article/ibm-watson-the-inside-story-of-how-the-jeopardy-winning-supercomputer-was-born-and-what-it-wants-to-do-next/ (accessed 15 August 2021).

Box

Draper

(1987) Empirical Model-Building and Response Surfaces. Oxford, UK: John Wiley & Sons.

Codd

(1970) A relational model of data for large shared data banks. Communications of the ACM 13: 377–387. DOI: 10.1145/362384.362685. S2CID 207549016

Daniels

(1952) The Average Man? WCRD technical note. Dayton, OH: Wright-Patterson Air Force Base, 53–7.

Dean

Ghemawat

(2004) MapReduce: Simplified Data Processing on Large Clusters. OSDI'04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA (2004), pp. 137-150

DeepMind (2019) AlphaStar: Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning. Available at: https://deepmind.com/blog/article/AlphaStar-Grandmaster-level-in-StarCraft-II-using-multi-agent-reinforcement-learning (accessed 3031 October 2021).

10.

Ghemawat

Gobioff

Leung

S-T

(2003) The google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles. New York, NY, 1 January 2003. Bolton Landing, pp. 20–43.

11.

Goodman

(1990) Fact, Fiction and Forecast: Fourth Edition Paperback. Harvard University Press.

12.

Gray

(2009) eScience: A transformed scientific method. In: Hey

Tansley

Tolle

(eds) The Fourth Paradigm: Data-Intensive Scientific Discovery (Second printing, version 1.1). Redmond, WA: Microsoft Research.

13.

Hassabis

(2016) AlphaGo: Using machine learning to master the ancient game of go. Available at: https://blog.google/technology/ai/alphago-machine-learning-game-go/ (accessed 24 October 2021).

14.

Hayes

(2002) The Story So Far. Available at: http://www.computerworld.com/databasetopics/data/story/0,10801,70102,00.html (accessed 21 April 2021).

15.

Zhang

Ren

, et al. (2015) Delving deep into rectifiers: Surpassing. Human- level performance on imagenet classification. ArXiv Preprint arXiv:1502.0185.

16.

Hinton

Osindero

Teh

(2006) A fast learning algorithm for deep belief nets. Neural Computation 18(7): 1527–1554. DOI: 10.1162/neco.2006.18.7.1527

17.

Jiang

Hwang

Bhagavatula

, et al (2021) Delphi: Towards machine ethics and norms. ArXiv Preprint arXiv:2110.07574. https://arxiv.org/pdf/2110.07574.pdf.

18.

Jumper

Evans

Pritzel

, et al. (2021) Highly accurate protein structure prediction with AlphaFold. Nature 596: 583–589. DOI: 10.1038/s41586-021-03819-2

19.

Laney

(2001) 3D data management: Controlling data volume, velocity and variety. Available at: https://blogs.gartner.com/doug-laney/files/2012/01/ad949- 3D-Data-Management-Controlling-Data-Volume-Velocity-and-Variety.pdf (accessed 13 January 2016).

20.

Merton

(1967) On sociological theories of the middle range. In: Merton

(ed) On Theoretical Sociology. New York, NY: Free Press, pp. 39–72.

21.

Norvig

(2008) All we want are the facts. Available at: http://www.norvig.com/fact-check.html (accessed 25 October 2021).

22.

K-S

Jung

(2004) GPU implementation of neural networks. Pattern Recognition 37: 1311–1314. DOI: 10.1016/j.patcog.2004.01.013

23.

Page

(2018) The Model Thinker: What You Need to Know to Make Data Work for You. New York, NY: Basic Books.

24.

Patel

(2017) Why doctors aren’t afraid of better, more efficient AI diagnosing cancer. Available at: https://www.thedailybeast.com/why-doctors-arent-afraid-of-better-more-efficient-ai-diagnosing-cancer (accessed 30 June 2021).

25.

Ribeiro

Singh

Guestrin

(2016) “Why should I trust you?”: explaining the predictions of any classifier. ArXiv Preprint arXiv:1602.04938 [cs, stat].

26.

Richardson

(2020) Modelling Socio-Spatial Dynamics from Real-Time Data: Towards a Context-Aware Framework for Modelling Behaviour Change in Urban Space from Mobile dataPhD Thesis. London, UK: University College London.

27.

Russell

(2019) Human Compatible: AI and the Problem of Control. London, UK: Allen Lane.

28.

Seif

(2018) Deep learning vs classical machine learning. Available at: https://towardsdatascience.com/deep-learning-vs-classical-machine-learning-9a42c6d48aa (accessed 31 October 2021).

29.

Tukey

(1962) The future of data analysis. The Annals of Mathematical Statistics 33(1): 1–67.

30.

Vincent

(2021) The AI oracle of delphi uses the problems of reddit to offer dubious moral advice. Available at: https://www.theverge.com/2021/10/20/22734215/ai-ask-delphi-moral-ethical-judgement-demo (accessed 23 October 2021).

31.

White

(2019) Our Top Data and Analytics Predicts for 2019. Available at https://blogs.gartner.com/andrew_white/2019/01/03/our-top-data-and-analytics-predicts-for-2019/ (accessed 23 October 2021).

32.

Williams

Burnap

Sloan

(2016) Crime sensing with big data: The affordances and limitations of using open source communications to estimate crime patterns. British Journal of Cancer 57: 320–340. DOI: 10.1093/bjc/azw031