Sage Journals: Discover world-class research

Abstract

This article uncovers how computers learned to recognize emotion from a neurological difference often conceived of as marked by a difficulty in recognizing it at all: autism. Developers of facial emotion recognition technologies have repeatedly mobilized autism to produce new kinds of knowledge at the interface of machine learning and human feeling, deploying it as a charismatic use-case, as a source for conceptual schemata for how emotion should be made computable, and as a testbed for development. The entanglement of emotion recognition and autism research shows how disability has been transformed into a rhetorical, conceptual, and material resource for the expansion of surveillance capitalism. This history has also underwritten a larger reconceptualization of emotional data, from a neglected variable in human–computer interaction to an exploitable corporate asset, a transformation that has allowed platform users’ emotional lives to be mined for new forms of knowledge, value, and power.

Keywords

Affective computing autism disability and technology emotion recognition human–computer interaction surveillance

Get full access to this article

View all access options for this article.

References

Alper

(2018) Inclusive sensory ethnography: studying new media and neurodiversity in everyday life. New Media & Society 20(10): 3560–3579.

Alper

(forthcoming) Kids across the Spectrums: Growing Up Autistic in the Digital Age. Cambridge, MA: MIT Press.

Ames

(2019) The Charisma Machine: The Life, Death, and Legacy of One Laptop Per Child. Cambridge, MA: MIT Press.

Andrejevic

(2011) The work that affective economics does. Cultural Studies 25(4–5): 604–620.

Balsamo

(1996) Technologies of the Gendered Body: Reading Cyborg Women. Durham, NC: Duke University Press.

Baron-Cohen

(1995) Mindblindness: An Essay on Autism and Theory of Mind. Cambridge, MA: MIT Press.

Baron-Cohen

(1997) How to build a baby that can read minds: cognitive mechanisms in mindreading. In: Baron-Cohen

(ed.) The Maladapted Mind. Hove: Psychology Press, pp. 204–240.

Baron-Cohen

Wheelwright

Lawson

, et al. (2002) The exact mind: empathising and systemising in autism spectrum conditions. In: Goswami

(ed.) Blackwell Handbook of Cognitive Development. Oxford: Blackwell Publishing, pp. 491–508.

BusinessWire (2021) Affectiva awarded six patents for multi-modal human perception AI and advanced in-cabin sensing. Businesswire.com, 23 March. Available at: https://www.businesswire.com/news/home/20210323005132/en/Affectiva-Awarded-Six-Patents-for-Multi-Modal-Human-Perception-AI-and-Advanced-In-Cabin-Sensing (accessed 26 June 2021).

10.

Crawford

(2021) Atlas of AI. New Haven, CT: Yale University Press.

11.

Dawson

(2004) The misbehaviour of behaviourists. In: No Autistics Allowed. Available at: http://www.sentex.net/~nexus23/naa_aba.html (accessed 17 May 2020).

12.

Demo

(2017) Hacking agency: apps, autism, and neurodiversity. Quarterly Journal of Speech 103(3): 277–300.

13.

Dinishak

(2016) The deficit view and its critics. Disability Studies Quarterly 36(4). Available at: https://dsq-sds.org/article/view/5236/4475

14.

Dinishak

Akhtar

(2013) A critical examination of mindblindness as a metaphor for autism. Child Development Perspectives 7(2): 110–114.

15.

Duffy

Dorner

(2011) The pathos of “mindblindness”: autism, science and sadness in “Theory of Mind” narratives. Journal of Literary & Cultural Disability Studies 5(2): 201–215.

16.

El Kaliouby

(2005) Mind-reading machines: automated inference of complex mental states. University of Cambridge Computer Laboratory, Technical Report no. 636. Available at: https://affect.media.mit.edu/projectpages/esp/elkaliouby-Phd.pdf

17.

El Kaliouby

Kross

(2014) Joni Ernst makes ‘em squeal. Affectiva (Blog), 9 December. Available at: https://blog.affectiva.com/joni-ernst-makes-em-squeal (accessed 26 June 2021).

18.

El Kaliouby

Robinson

(2002) Theory of mind in intelligent user interfaces. In: International workshop on the philosophy and design of socially adept technologies at ACM conference on computer-human interaction, Minneapolis, MN, 21 April.

19.

El Kaliouby

Robinson

(2005a) Generalization of a vision-based computational model of mind-reading. In: Tao

Tan

Picard

(eds) Affective Computing and Intelligent Interaction. Berlin: Springer, pp. 582–589.

20.

El Kaliouby

Robinson

(2005b) The emotional hearing aid: an assistive tool for children with Asperger syndrome. Universal Access in the Information Society 4(2): 121–134. https://doi.org/10.1007/s10209-005-0119-0

21.

El Kaliouby

Picard

Baron-Cohen

(2006) Affective computing and autism. Annals of the New York Academy of Sciences 1093(1): 228–248. https://doi.org/10.1196/annals.1382.016

22.

Ellcessor

Kirkpatrick

(eds) (2017) Disability Media Studies. New York: New York University Press.

23.

Ellis

Goggin

(2015) Disability and the Media. New York: Palgrave Macmillan.

24.

Ensmenger

(2012) Is chess the Drosophila of artificial intelligence? A social history of an algorithm. Social Studies of Science 42(1): 5–30.

25.

Fletcher-Watson

Bird

(2020) Autism and empathy: what are the real links? Autism 24(1): 3–6.

26.

Foster

(2016) Soon, these companies will track your emotions. get ready. Inc.com, 21 June. Available at: https://www.inc.com/magazine/201607/tom-foster/lightwave-monitor-customer-emotions.html (accessed 25 July 2020).

27.

Gernsbacher

(2017) Editorial perspective: the use of person-first language in scholarly writing may accentuate stigma. Journal of Child Psychology and Psychiatry, and Allied Disciplines 58(7): 859–861. https://doi.org/10.1111/jcpp.12706

28.

Gilberg

(1992) Autism and autistic-like conditions: subclasses among disorders of empathy. Journal of Child Psychology and Psychiatry 33(5): 813–842.

29.

Goodwin

(2007) Social-emotional technologies for autism spectrum disorders. National Science Foundation Award Abstract: 0705508. Available at: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0705508&HistoricalAwards=false (accessed 5 July 2022).

30.

Hacking

(2010) Autism fiction: a mirror of an Internet decade? Toronto Quarterly 72(2): 632–655.

31.

Hamraie

(2017) Building Access: Universal Design and the Politics of Disability. Minneapolis, MN: University of Minnesota Press.

32.

Harrison

Brownlow

Ireland

, et al. (2022) Empathy measurement in autistic and nonautistic adults: a COSMIN systematic literature review. Assessment 29(2): 332–350.

33.

Higginbotham

(2012) Welcome to Rosalind Picard’s touchy-feely world of empathic tech. Wired, 27 November. Available at: https://www.wired.co.uk/article/emotion-machines (accessed 2 July 2020).

34.

Howlin

Baron-Cohen

Hadwin

(1999) Teaching Children with Autism to Mind-read: A Practical Guide for Teachers and Parents. Chichester: John Wiley & Sons.

35.

Hutchinson

Prabhakaran

Denton

, et al. (2020) Unintended machine learning biases as social barriers for persons with disabilities. ACM SIGACCESS Accessibility and Computing 125: 9.

36.

Jack

(2014) Autism and Gender: From Refrigerator Mothers to Computer Geeks. Urbana, IL: University of Illinois Press.

37.

Jaysane-Darr

(2020) Enabling and disabling emotional diversity: negotiating autism spectrum disorder in therapeutic encounters. Children & Society 34(4): 261–275.

38.

Kenny

Hattersley

Molins

, et al. (2016) Which terms should be used to describe autism? Perspectives from the UK autism community. Autism 20(4): 442–462.

39.

Keyes

(2020) Automating autism: disability, discourse, and artificial intelligence. Journal of Sociotechnical Critique 1(1): 1–31.

40.

Kohler

(1994) Lords of the Fly: Drosophila Genetics and the Experimental Life. Chicago, IL: University of Chicago Press.

41.

Korosec

(2021) Emotion-detection software startup affectiva acquired for $73.5M. Techcrunch, 25 May. Available at: https://techcrunch.com/2021/05/25/emotion-detection-software-startup-affectiva-acquired-for-73-5m/ (accessed 6 June 2022).

42.

Lovaas

(1977) The Autistic Child: Language Development through Behavior Modification. New York: Irvington.

43.

McDonagh

(2013) Autism in an age of empathy: a cautionary critique. In: Davidson

Orsini

(eds) Worlds of Autism: Across the Spectrum of Neurological Difference. Minneapolis, MN: University of Minnesota Press, pp. 31–52.

44.

McStay

(2018) Emotional AI: the Rise of Empathic Media. Los Angeles, CA: SAGE.

45.

Madsen

El Kaliouby

Eckhardt

, et al. (2009) Lessons from participatory design with adolescents on the autism spectrum. In: Proceedings of the 2009 SIGCHI conference on human factors in computing systems, Boston, MA, 4–9 April, pp. 3835–3840. Boston, MA: ACM Press.

46.

Madsen

el Kaliouby

Goodwin

, et al. (2008) Technology for just-in-time in-situ learning of facial affect for persons diagnosed with an autism spectrum disorder. In: Proceedings of the10th international ACM SIGACCESS conference on computers and accessibility, Halifax, NS, Canada, 13–15 October, pp. 19–26. Boston, MA: ACM Press.

47.

Mills

(2010) Deaf jam: from inscription to reproduction to information. Social Text 28(1): 35–58.

48.

Mills

(2011a) Deafening: noise and the engineering of communication in the telephone system. Grey Room 43: 118–143.

49.

Mills

(2011b) Hearing aids and the history of electronics miniaturization. IEEE Annals of the History of Computing 33(2): 24–45. https://doi.org/10.1109/MAHC.2011.43

50.

Mills

(2011c) On disability and cybernetics: Helen Keller, Norbert Wiener, and the hearing glove. Differences 22(2–3): 74–111.

51.

Mills

Sterne

(2017) Afterword II: dismediation—three proposals, six tactics. In: Ellcessor

Kirkpatrick

(eds) Disability Media Studies. New York: New York University Press, pp. 365–378.

52.

Milton

(2012) On the ontological status of autism: the “double empathy problem.” Disability & Society 27(6): 883–887.

53.

Milton

(2014) Autistic expertise: a critical reflection on the production of knowledge in autism studies. Autism 18(7): 794–802.

54.

Milton

Kapp

Bovell

, et al. (2019) Deconstructing diagnosis: multi-disciplinary perspectives on a diagnostic tool. Autonomy, the Critical Journal of Interdisciplinary Autism Studies 1(6). Available at: http://www.larry-arnold.net/Autonomy/index.php/autonomy/article/view/AR26

55.

Murray

(2010) Autism functions/the functions of autism. Disability Studies Quarterly 30(1). Available at: https://dsq-sds.org/article/view/1048/1229

56.

Ochs

Solomon

(2010) Autistic sociality. Ethos 38(1): 69–92.

57.

Organization for Autism Research (2020) 1,000 people surveyed, survey says . . . OAR News. Available at: https://researchautism.org/-people-surveyed-survey-says/.1000

58.

Picard

(1995) Affective computing. Report, MIT Media Laboratory Perceptual Computing Section, no. 321. Available at: https://hd.media.mit.edu/tech-reports/TR-321.pdf

59.

Picard

(2000) Affective Computing. Cambridge, MA: MIT Press.

60.

Picard

(2005) Social-emotional intelligence prosthetic. National Science Foundation Award Abstract #0555411. Available at: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0555411&HistoricalAwards=false (accessed 5 July 2022).

61.

Picard

(2009) Future affective technology for autism and emotion communication. Philosophical Transactions of the Royal Society B: Biological Sciences 364(1535): 3575–3584.

62.

Picard

El Kaliouby

(2007) Social-emotional technologies for autism spectrum disorders. National Science Foundation Award Abstract #0705647. Available at: https://www.media.mit.edu/publications/social-emotional-technologies-for-asd/

63.

Picard

Klein

(2002) Computers that recognize and respond to user emotion: theoretical and practical implications. Interacting with Computers 14(2): 141–169.

64.

Pinchevski

(2005) Displacing incommunicability: autism as an epistemological boundary. Communication and Critical/cultural Studies 2(2): 163–184.

65.

Pinchevski

Peters

(2016) Autism and new media: disability between technology and society. New Media & Society 18(11): 2507–2523.

66.

Pullin

(2009) Design Meets Disability. Cambridge, MA: MIT Press.

67.

Research and Markets (2021) Emotion detection and recognition market by component. Available at: https://www.researchandmarkets.com/reports/5310824/emotion-detection-and-recognition-market-by (accessed 12 May 2021).

68.

Rodas

(2018) Autistic Disturbances: Theorizing Autism Poetics from the DSM to Robinson Crusoe. Ann Arbor, MI: University of Michigan Press.

69.

Rouvroy

(2013) The end(s) of critique. In: Rouvroy

Hildebrandt

DeVries

(eds) Privacy, Due Process, and the Computational Turn. London: Routledge, pp. 143–167.

70.

Schreibman

(2005) The Science and Fiction of Autism. Cambridge, MA: Harvard University Press.

71.

Silverman

(2013) Understanding Autism. Princeton, NJ: Princeton University Press.

72.

Spiel

Frauenberger

Keyes

, et al. (2019) Agency of autistic children in technology research: a critical literature review. ACM Transactions on Computer-Human Interaction 26(6): 38.

73.

Stark

(2018) Algorithmic psychometrics and the scalable subject. Social Studies of Science 48(2): 204–231.

74.

Stark

(2020) The emotive politics of digital mood tracking. New Media & Society 22(11): 2039–2057.

75.

Whittaker

Alper

Bennett

, et al. (2019) Disability, bias, and AI. Report, AI Now Institute. Available at: https://ainowinstitute.org/disabilitybiasai-2019.pdf

76.

Yergeau

(2018) Authoring Autism: On Rhetoric and Neurological Queerness. Durham, NC: Duke University Press.

77.

Zeavin

(2021) Hot and cool mothers. Differences: A Journal of Feminist Cultural Studies 32(3): 53–84.

78.

Zuboff

(2015) Big other: surveillance capitalism and the prospects of an information civilization. Journal of Information Technology 30(1): 75–89.

79.

Zuboff

(2019) The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. New York: PublicAffairs.

Autism and the making of emotion AI: Disability as resource for surveillance capitalism

Abstract

Keywords

Get full access to this article

References