Diversity has become increasingly relevant in computer science and technology development, both in terms of inclusive design teams and as a concept used to design “diversity-aware” technologies. The latter part has received less attention and notions of diversity leveraged to design technologies remain understudied. This paper critically examines diversity concepts employed by computer scientists and designers by reviewing 120 articles in two subfields of computer science: personalized recommendations and human–computer interaction. Drawing on Black feminism and critical race theory-inspired literature, I offer a critique of demographic and cultural user representations and underscore the shortcomings of diversity as a strategy to increase user satisfaction and fairness. Particularly concerning is the unreflected use of binary, static, and individual-level diversity concepts in the design of technologies. Such concepts render structural inequalities between and within groups of users invisible and thereby risk reinforcing existing injustices. The paper closes with considerations on whether and how diversity concepts can be leveraged by designers in a social justice–oriented manner.
AbascalJulioGardeazabalXabierPérezJuan EduardoValenciaXabierArbelaitzOlatzMuguerzaJavierYeraAinhoa. 2019. “Personalizing the User Interface for People with Disabilities.” In Proceedings of the 23rd International Workshop on Personalization and Recommendation on the Web and Beyond - ABIS ’19, edited by Augstein, Mirjam, Herder, Eelco, Wörndl, Wolfgang, and Yigitbas, Enes, 29. New York: ACM.
2.
AhmedSara. 2012. On Being Included: Racism and Diversity in Institutional Life. Durham, UK: Duke University Press.
AndersonAndrewDikkalaRupikaLiTianyiVorvoreanuMihaelaBurnettMargaret. 2021. “Understanding Diversity in Human-AI Data: What Cognitive Style Disaggregation Reveals.” https://arxiv.org/pdf/2108.00588.
5.
BanovicNikolaOulasvirtaAnttiKristenssonPer Ola. 2019. “Computational Modeling in Human-Computer Interaction.” In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, edited by Brewster, Stephen, Fitzpatrick, Geraldine, Cox, Anna, Kostakos, Vassilis, 1-7. New York: ACM.
BenjaminRuha. 2019. Race after Technology: Abolitionist Tools for the New Jim Code. Medford, MA: Polity Press.
8.
BerreyEllen. 2015. The Enigma of Diversity: The Language of Race and the Limits of Racial Justice. Chicago, IL: The University of Chicago Press.
9.
Bonilla-SilvaEduardo. 2015. “Why We Cannot Just Get Along? Race Matters in the Color-Blind Racial Moment.” Race Today: A Symposium on Race in America, Providence, RI, February 27. Accessed October 24, 2019. https://www.youtube.com/watch?v=T9ChSyjxjUI.
10.
Bonilla-SilvaEduardo. 2018. Racism without Racists: Color-blind Racism and the Persistence of Racial Inequality in America, 5th edition. Lanham, MD: Rowman & Littlefield.
11.
BurkeRobinSonboliNasimOrdonez-GaugerAldo. 2018. “Balanced Neighborhoods for Multi-Sided Fairness in Recommendation.” Proceedings of the 1st Conference on Fairness, Accountability, and Transparency. Proceedings of Machine Learning Research81: 202–14.
12.
CastellsPabloHurleyNeil J.VargasSaul. 2015. “Novelty and Diversity in Recommender Systems.” In Recommender Systems Handbook, edited by RicciFrancescoRokachLiorShapiraBracha, 881–918. Boston, MA: Springer US.
13.
ChiNicoleLurieEmmaMulliganDeirdre K.. 2021. “Reconfiguring Diversity and Inclusion for AI Ethics.” In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, edited by FourcadeMarionKuipersBenjaminLazarSethMulliganDeirdre, 447–57. New York: ACM.
14.
CollinsPatricia Hill. 2000. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment, 2nd ed. Perspectives on Gender. New York: Routledge.
Costanza-ChockSasha. 2020. Design Justice: Community-led Practices to Build the Worlds We Need. Information Policy. Cambridge, MA: The MIT Press.
17.
CrenshawKimberlé. 1989. “Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics.” University of Chicago Legal Forum (1): 139–67. Accessed June 02, 2020. https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1052&context=uclf.
18.
CrenshawKimberléGotandaNeilPellerGaryThomasKendall. 1995. “Introduction.” In Critical Race Theory: The Key Writings That Formed the Movement, edited by CrenshawKimberléGotandaNeilPellerGaryThomasKendall. New York: New Press; Distributed by W. W. Norton & Co.
19.
D’IgnazioCatherineKleinLauren F.. 2020. Data Feminism. Cambridge, MA: The MIT Press.
20.
EreteSheenaIsraniAartiDillahuntTawanna. 2018. “An Intersectional Approach to Designing in the Margins.” Interactions25 (3): 66–69. https://doi.org/10.1145/3194349.
21.
EskandanianFarzadMobasherBamshad. 2020. “Using Stable Matching to Optimize the Balance between Accuracy and Diversity in Recommendation.” In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, edited by Kuflik, Tsvi, Torre, Ilaria, Burke, Robin, and Gena, Cristina, 71–70. New York: ACM.
22.
FordRichard T.2009. Racial Culture: A Critique. Princeton, NJ: Princeton University Press.
23.
FriedmanBatyaNissenbaumHelen. 1996. “Bias in Computer Systems.” ACM Transactions on Information Systems14 (3): 330–47. https://doi.org/10.1145/230538.230561.
24.
GildemeisterRegine. 2008. “Doing Gender: Soziale Praktiken Der Geschlechterunterscheidung.” In Handbuch Frauen-und Geschlechterforschung, edited by BeckerRuthKortendiekBeate, 137–45. Wiesbaden, Germany: VS Verlag für Sozialwissenschaften.
25.
GoetheOleNiksiratKavous SalehzadehHirskyj-DouglasIlyenaSunHuatongLawEffie L.-C.RenXiangshi. 2019. “From UX to Engagement: Connecting Theory and Practice, Addressing Ethics and Diversity.” In Universal Access in Human-computer Interaction. Theory, Methods and Tools, edited by AntonaMargheritaStephanidisConstantine, 91–99. Cham, Switzerland: Springer International Publishing.
26.
GoffmanErving. 1990. Stigma: Notes on the Management of Spoiled Identity. Penguin Psychology. London, UK: Penguin.
27.
Guy-SheftallBeverly, ed. 1996. Words of Fire: An Anthology of African-American Feminist Thought. 2nd edition. New York: The New Press.
28.
HannaAlexDentonEmilySmartAndrewSmith-LoudJamila. 2020. “Towards a Critical Race Methodology in Algorithmic Fairness.” FAT Conference Proceedings, Barcelona, January2020, 501–12.
29.
HaslangerSally Anne. 2012. Resisting Reality: Social Construction and Social Critique. New York: Oxford University Press.
HimmelsbachJuliaSchwarzStephanieGerdenitschCorneliaWais-ZechmannBeatrixBobethJanTscheligiManfred. 2019. “Do We Care About Diversity in Human Computer Interaction.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems—CHI ‘19, edited by BrewsterStephenFitzpatrickGeraldineCoxAnnaKostakosVassilis, 1–16. New York: ACM Press.
32.
Hirskyj-DouglasIlyenaLuceroAndrés. 2019. “On the Internet, Nobody Knows You’re a Dog…Unless You’re Another Dog.” In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, edited by Brewster, Stephen, Fitzpatrick, Geraldine, Cox, Anna, Kostakos, Vassilis, 1–12. New York: ACM.
33.
HoffmannAnna Lauren. 2020. “Terms of Inclusion: Data, Discourse, Violence.” New Media & Society (September). Accessed February 20, 2022. https://doi.org/10.1177/1461444820958725.
34.
HoltzDavidCarteretteBenChandarPraveenNazariZahraCramerHenrietteAralSinan. 2020. “The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify.” In Proceedings of the 21st ACM Conference on Economics and Computation, edited by BiróPéterHartlineJasonOstrovskyMichaelProcacciaAriel, 75–76. New York: ACM.
35.
HsuYen-ChiaNourbakhshIllah. 2020. “When Human-computer Interaction Meets Community Citizen Science.” Communications of the ACM63 (2): 31–34. https://doi.org/10.1145/3376892.
36.
HullGloria T.ScottPatricia BellSmithBarbara, eds. 2010. All the Women Are White, All the Blacks Are Men, but Some of Us Are Brave: Black Women’s Studies. [13. repr.; Original: 1982]. New York: Feminist Press at the City University of New York.
37.
JackGavin. 2016. “Advancing Postcolonial Approaches in Critical Diversity Studies.” In The Oxford Handbook of Diversity in Organizations, edited by BendlRegineBleijenberghIngeHenttonenElinaMillsAlbert J., 153–74. Oxford, UK: Oxford University Press.
38.
JinYuchengTintarevNavaVerbertKatrien. 2018. “Effects of Individual Traits on Diversity-aware Music Recommender User Interfaces.” In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, edited by MitrovicTanjaZhangJieChenLiChinDavid, 291–99. New York: ACM.
39.
KaminskasMariusBridgeDerek. 2017. “Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-accuracy Objectives in Recommender Systems.” ACM Transactions on Interactive Intelligent Systems7 (1): 1–42. https://doi.org/10.1145/2926720.
40.
KeyesOs. 2019. “Counting the Countless: Why Data Science Is a Profound Threat for Queer People.” Real Life Magazine. https://reallifemag.com/counting-the-countless/. Accessed July 18, 2019.
41.
KouYuboGuiXinningChenYunanNardiBonnie. 2019. “Turn to the Self in Human-Computer Interaction.” In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, edited by Brewster, Stephen, Fitzpatrick, Geraldine, Cox, Anna, Kostakos, Vassilis, 1-7. New York: ACM.
42.
KravčíkMilosSchmidKatharinaIgelChristoph. 2019. “Towards Requirements for Intelligent Mentoring Systems.” In Proceedings of the 23rd International Workshop on Personalization and Recommendation on the Web and Beyond - ABIS '19, edited by Augstein, Mirjam, Herder, Eelco, Wörndl, Wolfgang, and Yigitbas, Enes, 29. New York: ACM.
43.
LakehalAbderrahimAltiAdelLaborieSebastienPhilippeRoose. 2018. “Ontology-based Context-aware Recommendation Approach for Dynamic Situations Enrichment.” In 2018 13th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), 81–6: IEEE.
44.
LazarJonathanBarbosaSimone Diniz Junqueira. 2019. “Introduction to Human-computer Interaction.” In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, edited by Brewster, Stephen, Fitzpatrick, Geraldine, Cox, Anna, Kostakos, Vassilis, 1–7. New York: ACM.
45.
LoecherbachFeliciaWelbersKasperMoellerJudithTrillingDamianAtteveldtWouter van. 2021. “Is This a Click Towards Diversity? Explaining When and Why News Users Make Diverse Choices.” In 13th ACM Web Science Conference 2021, 282–90. New York: ACM.
46.
LyonDavid, ed. 2003. Surveillance as Social Sorting: Privacy, Risk, and Digital Discrimination. London, UK: Routledge.
47.
MonacoJohn V.2020. “Bug or Feature? Covert Impairments to Human Computer Interaction.” In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, edited by BernhauptReginaMuellerFlorianVerweijDavidAndresJoshMcGrenereJoannaCockburnAndyAvellinoIgnacio, et al., 1–15. New York: ACM.
48.
NashJennifer C.2019. Black Feminism Reimagined: After Intersectionality. Next Wave. Durham, UK: Duke University Press.
NobleSafiya Umoja. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press.
51.
NormanKent L.KirakowskiJurek, eds. 2018. The Wiley Handbook of Human Computer Interaction. Chichester, UK: John Wiley & Sons.
52.
NwokoyeChukwuNonso. 2018. “Human Computer Interaction for Development (HCI4D): Implementing Reformation/Design Opportunities for Support and Care Coordination in Nigerian Prisons.” In Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities, edited by Winschiers-TheophilusHeikeZylIzak vanGoagosesNaskaJatDharm SinghBelayElefelious G.OrjiRitaPetersAniciaBouhlelMed S.JereNobert, 1–3. New York: ACM.
53.
PataranutapornPatSuminiValentinaEkblawArielYasharMelodieHäuplik-MeusburgerSandraTestaSusannaObristMariannaDonovielDoritParadisoJosephMaesPattie. 2021. “SpaceCHI: Designing Human-computer Interaction Systems for Space Exploration.” In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, edited by KitamuraYoshifumiQuigleyAaronIsbisterKatherineIgarashiTakeo, 1–6. New York: ACM.
54.
PataranutapornPatVujicAngelaKongDavid S.MaesPattieSraMisha. 2020“Living Bits: Opportunities and Challenges for Integrating Living Microorganisms in Human-computer Interaction.” In Proceedings of the Augmented Humans International Conference, edited by BoZhouChengJingyuanIshimaruPaulStrohmeierShoyaKoschThomas, 1–12. New York: ACM.
55.
RicciFrancescoRokachLiorShapiraBracha, eds. 2015. Recommender Systems Handbook. Boston, MA: Springer US.
RoughgardenJoan. 2013. Evolution’s Rainbow: Diversity, Gender, and Sexuality in Nature and People. Berkeley: University of California Press.
58.
Sanz-CruzadoJavierCastellsPablo. 2018. “Enhancing Structural Diversity in Social Networks by Recommending Weak Ties.” In Proceedings of the 12th ACM Conference on Recommender Systems, edited by PeraSole, 233–41. New York: ACM.
59.
SchelenzLaura. 2021. “Diversity-Aware Recommendations for Social Justice? Exploring User Diversity and Fairness in Recommender Systems.” In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, edited by MasthoffJudithHerderEelcoTintarevNavaTkalčičMarko, 404–10. New York: ACM.
60.
SilvaDiogo Vinícius de SousaDurãoFrederico Araújo. 2020. “Dynamic Clustering Personalization for Recommending Long Tail Items.” In Proceedings of the 2020 Federated Conference on Computer Science and Information Systems, edited by Ganzha, Maria, Maciaszek, Leszek, and Paprzycki, Marcin, 417–25. Annals of Computer Science and Information Systems, New York: IEEE.
61.
SkenderiErjonOlshannikovaEkaterinaOlssonThomasHuhtamäkiJukkaKoivunenSamiYaoPengHuttunenHeikki. 2019. “Investigation of Egocentric Social Structures for Diversity-Enhancing Followee Recommendations.” In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization—UMAP’19 Adjunct, edited by PapadopoulosGeorge A.SamarasGeorgeWeibelzahlStephanJannachDietmarSantosOlga C., 257–61. New York: ACM Press.
62.
SørensenJannick Kirk. 2019. “Public Service Media, Diversity and Algorithmic Recommendation: Tensions between Editorial Principles and Algorithms in European PSM Organizations.” CEUR Workshop Proceedings, 6–11. Accessed June 15, 2021. http://ceur-ws.org/Vol-2554/paper_01.pdf.
63.
SteelDanielFazelpourSinaGilletteKinleyCreweBiancaBurgessMichael. 2018. “Multiple Diversity Concepts and Their Ethical-epistemic Implications.” European Journal for Philosophy of Science8 (3): 761–80. https://doi.org/10.1007/s13194-018-0209-5.
StrohmayerAngelikaBelliniRosannaMeissnerJanisFinniganSamantha MitchellAlabdulqaderEbtisamToombsAustinBalaamMadeline. 2018. “#CHIversity: Implications for Equality, Diversity, and Inclusion Campaigns.” In Asian CHI Symposium 2018: Emerging Research Collections: Seoul, Korea, 19 April 2015, edited by SariEunice, 1–10. New York: ACM.
66.
SubramaniamBanu. 2014. Ghost Stories for Darwin: The Science of Variation and the Politics of Diversity. Urbana: University of Illinois Press.
67.
SunYingChaoXuLiGongfaWanfenXuKongJianyiJiangDuBoTaoChenDisi. 2020. “Intelligent Human Computer Interaction Based on Non Redundant EMG Signal.” Alexandria Engineering Journal59 (3): 1149–57. https://doi.org/10.1016/j.aej.2020.01.015.
WangNingxiaChenLiYangYonghua. 2020. “The Impacts of Item Features and User Characteristics on Users’ Perceived Serendipity of Recommendations.” In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, edited by Kuflik, Tsvi, Torre, Ilaria, Burke, Robin, and Gena, Cristina, 71–70. New York: ACM.
