Critics now articulate their worries about the technologies, social practices and mythologies that comprise Artificial Intelligence (AI) in many domains. In this paper, we investigate the intersection of two domains of criticism: identity and scientific knowledge. On one hand, critics of AI in public policy emphasise its potential to discriminate on the basis of identity. On the other hand, critics of AI in scientific realms worry about how it may reorient or disorient research practices and the progression of scientific inquiry. We link the two sets of concerns—around identity and around knowledge—through a series of case studies. In our case studies, about autism and homosexuality, AI figures as part of scientific attempts to find, and fix, forms of identity. Our case studies are instructive: they show that when AI is deployed in scientific research about identity and personality, it can naturalise and reinforce biases. The identity-based and epistemic concerns about AI are not distinct. When AI is seen as a source of truth and scientific knowledge, it may lend public legitimacy to harmful ideas about identity.
AnannyM., and CrawfordK.. 2018. “Seeing Without Knowing: Limitations of the Transparency Ideal and Its Application to Algorithmic Accountability. ” New Media & Society 20 (3): 973–989.
2.
BautistaM. A., Hernández-VelaA., EscaleraS., IgualL., PujolO., MoyaJ., ViolantV., and AngueraM. T.. 2015. “A Gesture Recognition System for Detecting Behavioral Patterns of Adhd. ” IEEE Transactions on Cybernetics 46 (1): 136–147.
3.
BeerD. G.2017. “The Social Power of Algorithms. ” Information, Communication and Society 20: 1–13.
4.
BhagatR. B.2006. “Census and Caste Enumeration: British Legacy and Contemporary Practice in India. ” Genus 62: 119–134.
5.
BivensR., and HaimsonO. L.. 2016. “Baking Gender Into Social Media Design: How Platforms Shape Categories for Users and Advertisers. ” Social Media+ Society 2 (4). doi: 10.1177/2056305116672486.
6.
BowkerG. C.2014. “Big Data, Big Questions— the Theory/Data Thing. ” International Journal of Communication 8: 1795–1799.
7.
BraunL.2014. Breathing Race into the Machine: The Surprising Career of the Spirometer From Plantation to Genetics . Minneapolis: University of Minnesota Press.
8.
BrookeyR. A.2002. Reinventing the Male Homosexual: The Rhetoric and Power of the Gay Gene . Bloomington: Indiana University Press.
9.
BrowneS.2015. Dark Matters: On the Surveillance of Blackness . Durham: Duke University Press.
10.
BucherT.2016. “Neither Black nor Box: Ways of Knowing Algorithms.” In:
Innovative Methods in Media and Communication Research, 81–98. Cham: Springer.
11.
BurrellJ.2016. “How the Machine ‘Thinks’: Understanding Opacity in Machine Learning Algorithms. ” Big Data & Society 3 (1). doi: 10.1177/2053951715622512.
12.
CaludeC. S., and LongoG.. 2017. “The Deluge of Spurious Correlations in Big Data. ” Foundations of Science 22 (3): 595–612.
13.
CalvoR. A., and D'MelloS.. 2010. “Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. ” IEEE Transactions on Affective Computing 1 (1): 18–37.
14.
CampoloA., and CrawfordK.. 2020. “Enchanted Determinism: Power Without Responsibility in Artificial Intelligence. ” Engaging Science, Technology, and Society 6: 1–19.
15.
CarrasquillaJ., and MelkoR. G.. 2017. “Machine Learning Phases of Matter. ” Nature Physics 13 (5): 431.
16.
CetinaK. K.2009. Epistemic Cultures: How the Sciences Make Knowledge . Cambridge, MA: Harvard University Press.
17.
Cheney-LippoldJ.2011. “A New Algorithmic Identity: Soft Biopolitics and the Modulation of Control. ” Theory, Culture & Society 28 (6): 164–181.
18.
CoopmansC., VertesiJ., LynchM. E., and WoolgarS.. 2014. Representation in Scientific Practice Revisited . Cambridge, MA: MIT Press.
19.
De VriesK.2010. “Identity, Profiling Algorithms and a World of Ambient Intelligence. ” Ethics and Information Technology 12 (1): 71–85.
20.
DowningL., MorlandI., and SullivanN.. 2015. Fuckology: Critical Essays on John Money's Diagnostic Concepts . Chicago: University of Chicago Press.
21.
DumitJ.2004. Picturing Personhood: Brain Scans and Biomedical Identity . Princeton: Princeton University Press.
22.
EckerC.2011. “Autism Biomarkers for More Efficacious Diagnosis. ” Biomarkers in Medicine 5 (2): 193–195.
23.
ElishM. C., and BoydD.. 2018. “Situating Methods in the Magic of Big Data and AI. ” Communication Monographs 85 (1): 57–80.
EubanksV.2018. Automating Inequality: How High-tech Tools Profile, Police, and Punish the Poor . New York: St. Martin's Press.
26.
EyalG.2010. The Autism Matrix . Cambridge: Polity.
27.
FeltU., IgelsböckJ., SchikowitzA., and VölkerT.. 2016. “Transdisciplinary Sustainability Research in Practice: Between Imaginaries of Collective Experimentation and Entrenched Academic Value Orders. ” Science, Technology, & Human Values 41 (4): 732–761.
28.
FitzgeraldD.2017. Tracing Autism: Uncertainty, Ambiguity, and the Affective Labor of Neuroscience . Seattle: University of Washington Press.
29.
FuS., HeH., and HouZ. G.. 2014. “Learning Race From Face: A Survey. ” IEEE Transactions on Pattern Analysis and Machine Intelligence 36 (12): 2483–2509.
30.
GalisonP., and DastonL.. 2007. Objectivity.
31.
GeertzC.1973. Local Knowledge: Further Essays in Interpretive Anthropology . New York: Basic books.
32.
GillespieT.2014. “The Relevance of Algorithms. ” Media Technologies: Essays on Communication, Materiality, and Society 167: 167.
33.
Hames-GarciaM. R.2011. Identity Complex: Making the Case for Multiplicity . Minneapolis: University of Minnesota Press.
34.
HashemiJ., SpinaT. V., TepperM., EslerA., MorellasV., PapanikolopoulosN., and SapiroG.. 2012. “A Computer Vision Approach for the Assessment of Autism-Related Behavioral Markers.” In:
2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL). 1–7, IEEE.
35.
HaslamN., and LevyS. R.. 2006. “Essentialist Beliefs About Homosexuality: Structure and Implications for Prejudice. ” Personality and Social Psychology Bulletin 32 (4): 471–485.
36.
HayesJ., McCabeR., FordT., and RussellG.. 2020. “Drawing a Line in the Sand: Affect and Testimony in Autism Assessment Teams in the UK. ” Sociology of Health & Illness 42: 825–843.
37.
HelbingD., FreyB. S., GigerenzerG., HafenE., HagnerM., HofstetterY., Van Den HovenJ., ZicariR. V., and ZwitterA.. 2019. “Will democracy survive big data and artificial intelligence?” In:
Towards Digital Enlightenment, 73–98. Cham: Springer.
38.
HirdM. J.2004. Sex, Gender, and Science . Basingstoke: Palgrave Macmillan.
39.
HollinG.2014. “Constructing a Social Subject: Autism and Human Sociality in the 1980s. ” History of the Human Sciences 27 (4): 98–115.
40.
Jordan-YoungRebecca M.2011. Brain Storm: The Flaws in the Science of Sex Differences . Cambridge, MA
: Harvard University Press.
41.
Kassraian-FardP., MatthisC., BalstersJ. H., MaathuisM. H., and WenderothN.. 2016. “Promises, Pitfalls, and Basic Guidelines for Applying Machine Learning Classifiers to Psychiatric Imaging Data, with Autism As An Example. ” Frontiers in Psychiatry 7: 177.
42.
KeyesO.2018. “The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. ” Proceedings of the ACM on Human-Computer Interaction 2 (CSCW): 1–22.
43.
KeyesO.2019. Counting the Countless: Why Data Science is a Profound Threat for Queer People. Cambridge: Real Life 2.
44.
KitchinR.2017. “Thinking Critically About and Researching Algorithms. ” Information, Communication & Society 20 (1): 14–29.
45.
KrafftP., YoungM., KatellM., HuangK., and BugingoG.. 2019. Policy Versus Practice: Conceptions of Artificial Intelligence.
Available at SSRN 3431304.
46.
LatourB.1987. Science in Action: How to Follow Scientists and Engineers Through Society . New York: Harvard University Press.
47.
LawJ.2004. After Method: Mess in Social Science Research . New York: Routledge.
48.
LazerD., PentlandA., AdamicL., AralS., BarabásiA. L., BrewerD., ChristakisN., et al. 2009. “Computational Social Science. ” Science (New York, N.Y.) 323 (5915): 721–723.
49.
L'heureuxA., GrolingerK., ElyamanyH. F., and CapretzM. A.. 2017. “Machine Learning with Big Data: Challenges and Approaches. ” IEEE Access 5: 7776–7797.
50.
LugonesM.2016. “The coloniality of gender.” In:
The Palgrave handbook of gender and development, 13–33. New York: Springer.
51.
MaclureJ.2019. “The New AI Spring: A Deflationary View. ” AI & Society 35: 1–4.
52.
M'charekA., SchrammK., and SkinnerD.. 2014. “Topologies of Race: Doing Territory, Population and Identity in Europe. ” Science, Technology, & Human Values 39 (4): 468–487.
MorandiniJ. S., BlaszczynskiA., RossM. W., CostaD. S., and Dar-NimrodI.. 2015. “Essentialist Beliefs, Sexual Identity Uncertainty, Internalized Homonegativity and Psychological Wellbeing in Gay Men. ” Journal of Counseling Psychology 62 (3): 413.
55.
NataleS., and BallatoreA.. 2017. “Imagining the Thinking Machine: Technological Myths and the Rise of Artificial Intelligence. ” Convergence 26: 3–18.
56.
NobleS. U.2018. Algorithms of Oppression: How Search Engines Reinforce Racism . New York: NYU Press.
57.
OrtegaF., and ChoudhuryS.. 2011. “‘Wired Up Differently’: Autism, Adolescence and the Politics of Neurological Identities. ” Subjectivity 4 (3): 323–345.
58.
OrtolevaP.2009. “Modern Mythologies, the Media and the Social Presence of Technology. ” Observatorio (OBS*) 3 (1).
59.
PassiS., and SengersP.. 2020. “Making Data Science Systems Work. ” Big Data & Society 7 (2). doi: 10.1177/2053951720939605.
60.
RobertsD.2011. Fatal Invention: How Science, Politics, and Big Business Re-create Race in the Twenty-first Century . Princeton: New Press/ORIM.
61.
RoseN., and Abi-RachedJ. M.. 2013. Neuro: The New Brain Sciences and the Management of the Mind . New York: Princeton University Press.
62.
SamuelsE.2014. Fantasies of Identification: Disability, Gender, Race . Vol.
10, NYU Press.
63.
SatoJ. R., HoexterM. Q., de Magalhães OliveiraP. P.Jr, BrammerM. J., MurphyD., EckerC., and ConsortiumM. A.. 2013. “Inter-regional Cortical Thickness Correlations are Associated with Autistic Symptoms: A Machine-learning Approach. ” Journal of Psychiatric Research 47 (4): 453–459.
SeaverN.2018. “What Should an Anthropology of Algorithms Do?
” Cultural Anthropology 33 (3): 375–385.
66.
Seaver, 2019. “Knowing Algorithms. ”
in DigitalSTS: A Field Guide for Science & Technology Studies, edited by Janet Vertesi and David Ribes. Princeton: Princeton University Press.
67.
SinghJ. S.2015. Multiple Autisms: Spectrums of Advocacy and Genomic Science . Minneapolis: University of Minnesota Press.
68.
StarkL.2018. “Algorithmic Psychometrics and the Scalable Subject. ” Social Studies of Science 48 (2): 204–231.
69.
TerryJ.1999. An American Obsession: Science, Medicine, and Homosexuality in Modern Society . Chicago:
University of Chicago Press.
70.
ThompsonD.2015. “What Lies Beneath: Equality and the Making of Racial Classifications. ” Social Philosophy & Policy 31 (2): 114.
Van der PloegI.2012. “The Body as Data in the Age of Information.” In:
Routledge Handbook of Surveillance Studies, 176–183. Cham: Springer.
73.
VerhoeffB.2013. “Autism in Flux: A History of the Concept From Leo Kanner to DSM-5. ” History of Psychiatry 24 (4): 442–458.
74.
WaidzunasT.2015. The Straight Line: How the Fringe Science of Ex-gay Therapy Reoriented Sexuality . Minneapolis: University of Minnesota Press.
75.
WaidzunasT., and EpsteinS.. 2015. “For Men Arousal is Orientation: Bodily Truthing, Technosexual Scripts, and the Materialization of Sexualities Through the Phallometric Test. ” Social Studies of Science 45 (2): 187–213.
76.
WangY., and KosinskiM.. 2018. “Deep Neural Networks are More Accurate Than Humans At Detecting Sexual Orientation From Facial Images. ” Journal of Personality and Social Psychology 114 (2): 246.
77.
WolfC. T., and PaineD.. 2018. Sensemaking practices in the everyday work of AI/ML software engineering.
interface.
78.
WoodsR.2017. “Pathological Demand Avoidance: My Thoughts on Looping Effects and Commodification of Autism. ” Disability & Society 32 (5): 753–758.
79.
YangK. K., WuZ., BedbrookC. N., and ArnoldF. H.. 2018. “Learned Protein Embeddings for Machine Learning. ” Bioinformatics (Oxford, England) 34 (15): 2642–2648.
80.
ZhouJ., ParkC. Y., TheesfeldC. L., WongA. K., YuanY., ScheckelC., FakJ. J., et al. 2019. “Whole-genome Deep-learning Analysis Identifies Contribution of Noncoding Mutations to Autism Risk. ” Nature Genetics 51 (6): 973–980.
81.
ZimmerF., StockM., ScheibeK., and StockW. G.. 2019. “Fake News in Social Media: Bad Algorithms or Biased Users?
” Journal of Information Science Theory and Practice 7 (2): 40–53.
