Abstract
In this article, we draw inspiration from participatory action research (PAR) and the work of Latin American thinkers such as Freire and Fals Borda to interrogate artificial intelligence (AI). We propose a South-North flow by utilising PAR approaches that stem from Latin America, challenging how the North's centrality is taken for granted regarding AI epistemologies, experiences, and understandings. Conducting workshops in London with a diverse group of students, tech workers and activists, we argue that PAR can not only empower marginalised communities in the Global South; we can also learn more from its application in the Global North, in contexts where people deal with different struggles. Our analysis delves into three specific concepts around AI and data (in)justice: autonomy, empathy and dialogue. First, inspired by PAR principles, participants started to problematise what they called an empty interpretation of empathy, establishing parallels with transnational dynamics of data capitalism, which disadvantage marginalised communities in the Global South. Second, PAR offered a critical lens to analyse issues of AI and autonomy in ways that are less individualistic and more collective and politically engaged. Third, PAR's dialogical spirit enabled participants to locate various intersections between AI and dialogue. Critiquing the idea of a superior AI, participants were reminded of the possibilities offered by human intelligence and the combination of thinking, making and feeling or what Fals Borda (2003. Ante la crisis del país: Ideas acción para el cambio, 1st. Ed. Bogotá: Panamericana) calls our sentipensante nature.
This article is a part of special theme on Critical Data Studies in Latin America. To see a full list of all articles in this special theme, please click here: https://journals.sagepub.com/page/bds/collections/critical_data_studies__in_latin_america
Introduction
After a long pandemic break, we were standing in a room as part of a conference panel. Hosted by a European university, the conference focused on the theme of decolonising the internet. Our paper reflected on the possibilities of applying Latin American participatory action research (PAR) to critical data and artificial intelligence (AI) studies. Citing authors like Paulo Freire and Orlando Fals Borda in an early version of this work, and as she faced the mostly European attendees, author 1, Andrea Medrado, could not avoid feeling imposter syndrome: Are we doing justice to the work of these authors? Would it make sense to use PAR, a radical approach that originated in the past, in the 1970s and 1980s, to analyse AI, which is deemed the future?
As Andrea dealt with her internal questions, it was time for the Q&A. One of the questions was about the profile of the participants. We conducted workshops in London with a predominantly middle-class group of university students, tech workers and activists. The question was: “Given that these approaches have been developed to work with groups that are oppressed (to use a Freirean term), would they still work with people who have not experienced as much social marginalisation?” The question was helpful and invited us to reflect on the suitability of using approaches that stem from contexts of marginalisation and employ them in different contexts. To put it simply, which lessons can be learned from applying in the Global North, methodological approaches rooted in the Global South?
The question led us to think about how and why, as critical data scholars, we should embrace epistemological and methodological approaches that originate from the Global South, particularly the Latin American tradition of PAR. This article aims to share this journey and identify the unique contributions Latin American PAR can offer to critical data studies, specifically AI and data justice issues. Our main research question is:
In what ways can the Latin American tradition of PAR (participatory action research) inspire us to inquire about AI and data (in)justice? Which epistemological frameworks are offered by PAR to help us understand critical issues around AI and power asymmetries between the so-called Global North and Global South? Which research priorities emerge by embracing South-to-North epistemological and methodological flows to critical AI and data studies? Which insights can Latin American PAR approaches offer when applied in Global North contexts with participants who are not as affected by issues of marginalisation?
Additional questions are:
As critical data scholars, our task goes beyond asking the best critical questions: it entails seeking the best-suited critical ways to answer them. Thus, as we look for answers about issues of AI and data (in)justice, we argue that they must stem from the realities of those who are precisely the most affected by extractive (digital) capitalism: marginalised groups in the Global South. In this way, we invite critical data scholars to be more open to what we call South-North epistemological and methodological flows. By doing this, we challenge the treatment of Southern contexts (such as the Latin American context), as mere test grounds for applying Global North theoretical and methodological models. Echoing scholars who focus on data, technology and decoloniality (Aguilar, 2020; Aouragh and Chakravartty, 2016) we aim to turn around such (colonial) epistemological dynamics.
Here, we pause for a few conceptual explanations: what do we mean by the Global South? What do we mean by data justice? And what are we calling critical data studies? Let's start with the first question. The term Global South is problematic because it lumps together the other, reinforcing the othering of the others. It would be simplistic to impose similarities between a Latin American and an African country, for example, simply because they have a weaker economy than, say, a Western European country. Thus, by adopting the term we might indirectly contribute to homogenising what constitutes the true South. This is further complicated by the fact that there are Souths in the Norths and vice-versa (such as pockets of poverty in the Global North and pockets of wealth in the Global South).
At the same time, the term Global South can be useful and strategic, helping to connect marginalised realities although they are, of course, very distinct. Referring to more than a geographic location, the South features as a metaphor for oppression, social inequalities, and human suffering. To cite Santos (2016), the Global South is about drawing an abyssal line that separates those deemed to be human (located in the Metropolis zone, the North) from those deemed sub-human (Located in the Colonial zone, the South). Importantly, the South is connective in nature (Medrado and Rega, 2023). The term can be a conversation starter, triggering exchanges about colonial legacies, oppression, and marginalisation. It conveys the meaning of a political solidarity project 1 , particularly amongst former colonies subjected to violence. It also expresses an openness to listen to the other and to each other as others. It communicates the willingness to change things for the better, to right what is wrong, to join forces and fight injustices.
To turn to the second question, what is data justice? In contemporary societies, the vast availability of data – “as a by-product of people's use of technological devices and services” (Taylor, 2017: 1) – has implications for the way people are treated by the state and the private sector. Therefore, the power of data to produce social categorisations and interventions in the world should be connected to an agenda of social justice. Data justice can be defined as “fairness in the way people are made visible, represented and treated as a result of their production of digital data” (Ibid.). Additionally, Taylor (2017) proposes three pillars that should work as the basis for data justice: (in)visibility, (dis)engagement with technology and antidiscrimination (Ibid). The first aspect deals with representation and privacy; the second with attention to sharing the benefits of data and autonomy in technology choices; and the third aspect includes the ability to challenge bias.
Linked to an agenda of social justice, data justice is a crucial concept in critical data studies (CDS). Kitchin and Lauriault (2014) argue that CDS needs to unpack the socio-technical assemblages in which data are produced, circulated and used, including how they shape policy and regulation. Power and politics are key themes of CDS, which is informed by critical social theory, including feminism, political economy, postcolonialism, STS, etc. The study of artificial intelligence (AI) is part of CDS too. We cannot really talk about AI without talking about data. The link between them is illustrated in this basic definition of AI as “computer programming that learns from and adapts to data” (Verdegem, 2021: 5). In the context of AI, data is not only a commodity but also a source for value creation (Sadowski 2019). Narrow AI, the type of AI we talk about in this article, works by analysing datasets, identifying patterns and probabilities, and codifying them into a model (Broussard 2018). This means that problems and challenges surrounding data are present in AI, and vice versa.
As we interrogate AI using PAR, we are inspired by critical perspectives focused on political economy, colonialism and critical race studies. Political economy analyses how capitalist societies produce media and communication as commodities. AI is part of data capitalism (Sadowski 2019), which is characterised by the centrality of commodification and extraction of data, as well as a concentrated industrial landscape (Verdegem, 2022). Only a few companies – GAFAM (Google/Alphabet, Apple, Facebook/Meta, Amazon and Microsoft) in the Western world and BAT (Baidu, Alibaba and Tencent) in China – have simultaneous access to large amounts of data and can invest in a high-performance computational structure needed to run AI models. But data and AI are also part of a project of white supremacy. Looking at how the corporate power of big tech develops and sustains inequalities is the subject of what Benjamin (2019) calls race critical code studies. Studies like this rebuke the claim that technology is neutral. Noble (2018: 4) calls algorithmic oppression the structural racism and sexism that happens online, which is based on “algorithmically driven data failures that are specific to people of colour and women.” Having been developed in the largely white military-industrial-academic complex, data is often captured by institutions predicated on white supremacy, reproducing the shape of “whiteness as an ideology” (Katz, 2020: 8).
Embracing these critical perspectives, we conducted four workshops in London in June and July 2022 (described in the following sections). With its less rigid knowledge production hierarchies, PAR approaches were vital in raising participants’ critical consciousness (or conscientização, in Freirean vocabulary). Additionally, with its explicit political dimensions, paying special attention to issues of power asymmetries in AI and data (in)justice, such consciousness-raising manifested in three main axes – empathy, autonomy and dialogue. First, inspired by PAR principles, participants started to problematise what they called an empty interpretation of empathy, establishing parallels with transnational dynamics of data capitalism, which disadvantage marginalised communities in the Global South. Second, PAR offered a critical lens to analyse issues of AI and autonomy in ways that are less individualistic and more collective and politically engaged. Third, PAR's dialogical spirit enabled participants to locate various intersections between AI and dialogue. Critiquing the idea of a superior AI, participants were reminded of the possibilities offered by human intelligence and the combination of thinking, making and feeling or what Fals Borda (2003) calls our sentipensante nature.
Revisiting the Latin American roots of participatory action research (PAR)
In a context in which digital technology itself is behind societal harms such as discrimination and racism (Benjamin, 2019; Noble, 2018) increasing participation is an attempt to understand the perspectives and needs of historically marginalised communities, contributing to more inclusive AI. However, while increased participation has potential for emancipatory technology design, there is less agreement about what meaningful participation is (Birhane et al., 2022).
As they try to identify good participation practices in a study about AI as a catalyst for social progress, Bondi et al. (2021) call for the need to include as equal partners members of communities that AI systems are likely to impact. The authors present a list of questions that can serve as guiding principles, such as: How can we understand and incorporate viewpoints from many stakeholder groups? How are impacted communities identified and represented? Does this include marginalised groups? How are community concerns addressed? What are the long-term plans for maintaining work (Bondi et al., 2021)?
These questions provide a valuable set of guidelines for socially conscious research in the field of AI and, indeed, any field of study. However, given what we know about the unevenness between how tech giants in the Global North and marginalised communities in the Global South benefit from AI, are these expectations of equal relationships credible? The indigenous thinker Yásnaya Aguilar writes about how the Western myth of perpetual growth disadvantages those who are most vulnerable in the global capitalist system. To cite Aguilar (2020), this growth advances through a digestive system that uses technology as one of its core components to basically chew up marginalised people, turning our planet into an inhospitable place. It is not an exaggeration to claim that this represents a new colonial order.
What is then our task as critical data scholars delving into issues of AI and data (in)justice? Asking critical questions that relate to the power asymmetries between the so-called Global North and South is a good starting point, but it is not enough. We suggest that in addition to asking the right (as in critical) questions, we need to seek the right (also critical) ways to answer them. We argue that the best-suited methodological and epistemological approaches for this endeavour are precisely the ones that stem from the realities of those who are excluded in the global techno-colonialist ecosystem (Madianou, 2019). Here, as enthusiasts state that AI is the future and there is no turning back, we suggest a return to a radical past. We aim to do this by revisiting the Latin American roots of participatory action research (PAR) and particularly the work of authors like Paulo Freire (1972) and Orlando Fals Borda (1987).
From a broader Global South perspective, PAR emerged in different regions of the world as a response to developmentalist top-down approaches. Unlike the North American tradition of action research (AR) (Whyte, 1994), these strands of PAR are connected to the need for new research methodologies to answer the needs of twentieth-century social movements, in particular land reform and the anti-colonial struggle. PAR was developed simultaneously across various countries, such as Tanzania, India, Brazil, Chile, and Colombia (Rahman, 2006; Vio Grossi, 1982). Despite the contextual differences, it evolved through bottom-up processes co-developed with marginalised groups.
In Latin America, PAR was an attempt to combine theory and practice to tackle unequal social realities. One of its essential characteristics was the rejection of asymmetries, which are often part of dependence paradigms and research processes. For PAR, there are no distinctions between subjects and objects of research – all participants are active subjects. When research on participatory AI, for instance, points to the need to treat all involved actors as equal partners, this shows a commitment to what Freire (1972) called a critical pedagogy. The aim is to challenge the banking form of knowledge building in which students (or the researched) are considered blank sheets to be filled in by teachers (or the researchers). By embracing this approach, we can challenge top-down power relations with teachers/researchers being deemed knowledgeable whilst students/researched are considered knowledgeless.
Additionally, authors such as Fals Borda (2003) have questioned other forms of demarcation, namely the distinctions between making, thinking and feeling (or hands, brains and hearts) in knowledge building. The author has developed ideas around sentipensante (thinking-feeling) approaches. Deriving from the combination of the Spanish words sentimiento and pensamiento, sentipensante refers to combining the mind with the heart to guide one's journey on the right path, enduring its many setbacks (Fals Borda, 2003: 9). This is a call for a different approach to be embraced by researchers, educators, activists, intellectuals: an approach that stands in direct opposition to the cold and supposedly neutral attitude of the positivist Eurocentric scientist. In this way, knowledge is not obtained from making, as opposed to thinking, or as opposed to feeling. Rather, all knowledges are sentipensante and can be acquired by engaging in respectful and empathic dialogues.
Aligned with feminist and decolonial perspectives, PAR explicitly recognises that knowledge building does not occur in a vacuum. Rather, knowledge is situated (Haraway, 1988), demanding a practice of positioning that attends to the power relations at play in the processes of knowledge production. For this reason, vivências, or the lived experiences, are crucial for PAR. Vivências can be defined “as full experiences of an event with its all possibilities”. In other words, vivências cannot be observed (in an objective manner); “they can only be lived, felt, and experienced” (Glassman and Erdem, 2014: 212). Whether they are experienced by the participants, by the researchers, or both (as the boundaries between the two are blurred), what matters is that these vivências can be transformative in sentipensante ways (Fals Borda, 2003). This transformative nature is connected to another key characteristic: a commitment to social change.
Additionally, according to Freire (1972), a process of collective participation can only happen if accompanied by a process of consciousness raising (or conscientização). The goal is for the community to oversee this process, knowing how to operationalise it, understand it logically and critically evaluate it (Mendez and Sanabria, 2003: 123). PAR is not an approach that is carried out by an expert. Instead, it requires the formation of teams allowing people to discover their capacities for action. The action-reflection processes must be participatory and developed collectively around notions of solidarity (Mendez and Sanabria, 2003: 124).
Here, we refer to our understanding of the Global South as a political solidarity project. Indeed, authors such as Fals Borda and Freire are insightful because they explicitly position participatory research in political ways. If participatory research is about getting a plurality of voices and echoing them on equal levels, Latin American PAR is about taking sides. It is about taking the side of the colonised, or oppressed, in Freirean terms. If we apply these ideas to the concerns of this article – interrogating AI - participation should start from the acknowledgement that ignoring patterns of systemic oppression and privilege has led to AI systems that are opaque and unfair. In what follows we discuss how these issues were analysed in a series of workshops conducted in London.
Methodological approach
Inspired by Latin American PAR, we organised four workshops during June and July 2022. 2 The participants (see Table 1) consisted of 20 people in four main categories: 11 undergraduate students; 5 postgraduate students; 2 tech workers and 2 human rights activists. The group consisted of 11 females and 9 males (all cisgendered) with ages ranging from 20 to 43. We obtained approval from the university's research ethics committee before carrying out the workshops.
Workshop participants’ profiles.
As discussed earlier, we were inspired by the principles of vivência (lived experience), conscientização (consciousness raising), praxis/action and critical reflection (Freire, 1972; Fals Borda, 1987; Glassman and Erdem, 2014). Gathering inputs from participants in our first meeting, we designed the workshops as follows:
Day 1 – Vivências of AI Day 2 – My Day in AI Day 3 – Discussion of AI Intervention Day 4 – Design of AI Intervention
We shared stories of AI in the everyday life of invisible workers in the Global South. This triggered a conversation about AI, data (in)justice and social inequality, while we also introduced the Latin American PAR tradition to the group. At the end of the day, it was decided that participants would document their own days in AI by producing a piece of text, video or illustration.
The participants returned two weeks later having produced a creative piece to document their daily experiences in and with AI. Building on the vivências of AI, this exercise was aligned with research that focuses on datafication as it lived, felt and experienced at the level of the everyday (Kennedy, 2018: 27). The artefacts were made available for all participants so that they could engage with each other's content before the second workshop. During this session, participants were asked which pieces of content produced by others they empathised with and why.
Participants spent time articulating problems in terms of AI not contributing to social justice, such as AI inputs come from one predominant source (Western, white, male perspective) (powerful companies in the Global North); AI-driven recommendation systems interfering with our autonomy; business models do not support dialogue with empathy; we have little information about the big players/actors in AI and what their agendas are. All these problems, which were connected to critical questions, were shared on a Padlet.
We divided the larger group into two smaller ones. In collective brainstorming, the participants proposed two interventions to tackle the abovementioned problems. The first one related to the media's non-critical reporting and representations of AI (either with hype or dystopian Hollywood narrative tropes). The group's solution would be to propose a reality show where people would spend a week with other people's devices, such as tablets or phones, engaging with the recommendations they get from search engines and social media. Based on these recommendations, the participants would then be asked to imagine what living that other person's life is like. During this exercise, they would also be asked to empathise with this person and use their critical consciousness (conscientização) to identify if (in what ways) the person is subjected to AI-driven discrimination. The second intervention related to issues of media policy. Participants proposed the development of a platform called Have you also heard? The aim was to enable social media platforms to point users to diverse perspectives on the same issue with the rationale of your views are… but have you also heard…? The group believed that this would help reduce a radicalisation of views which might be caused by algorithmic logics of exposing users to the same types of content. Figure 1 provides a visual overview of how the three main PAR concepts – empathy, dialogue and autonomy – emerged from the workshops.
Visual timeline of the workshops.
We have presented the structures of the four workshops to demonstrate how our PAR approach works in practice. However, we do not have the space here to delve into the feasibility of these proposed interventions. By focusing on the workshops as our empirical site, we argue that despite having middle-class participants who were not as affected by issues of marginalisation, PAR led us to conversations that were deeply infused with concerns around AI and data (in)justice.
Another caveat is worth making here. Whilst we were inspired by Latin American PAR, we could not stay as true to its ethos of open agenda as we would have liked. There are significant challenges in terms of applying PAR, as it is understood by radical Latin American scholars (Freire, 1972; Fals Borda, 1987), in the context of the UK Higher Education system. The project was financed by a seed funding round of impact grants within our institution. Even if indirectly, the impact agenda places the researcher in a higher hierarchical position than the recipients of their expertise because impact is about delivering successful case study stories (Wróblewska, 2021).
Finally, as we discussed earlier, the workshop participants comprised mostly middle-class university students from diverse backgrounds in terms of country of origin. In this case, arguably, as opposed to participants from economically vulnerable backgrounds, there were fewer concerns in terms of potentially (economically) exploitative relationships, even if the usual (human, humane) reciprocity principles applied. However, a question that emerged was: from a PAR perspective, would these participants still be able to offer valuable insights into issues of AI and data (in)justice as they are not as affected by issues of inequality and marginalisation? In the next sections, we demonstrate that our participants were clearly grappling with issues of AI, extraction and inequalities even if they were not initially identified as research priorities.
Empathy and AI: sharing feelings or taking the side of the oppressed?
At the end of the first workshop day, we decided to document our vivências of AI with a creative exercise called My Day in AI. Pieces of creative writing, videos and illustrations were produced and uploaded to a shared drive. We then returned for the second workshop, two weeks later. This second meeting started with us reflecting on how it felt to document the role that AI played in our daily lives. We then shared the writings, videos and illustrations again and asked: by looking at these artefacts, can you tell us one you empathise with? Why? In what follows we reproduce a piece of writing (in diary style) that resonated with many participants. My Day in AI “One of the things I need to do in a day is to pay the deposit for my room, but for some reason I can't. I need help and I rely on the live chat on the portal. I try to get answers from a bot, which unfortunately cannot help me. But even if the bot could help me, it doesn’t really care about my financial strains to go to university. Bots don’t really care if we are in debt”. (Excerpt from Participant 11's writing, 19/06/2022).
Empathy was a concept that emerged organically from the workshop activities. Sharing our creative artefacts meant that we could connect across our daily experiences of/with AI in empathic ways. Additionally, participants were also able to identify points of intersection between AI and empathy. This was articulated by Participant 3, a male tech worker: “Today we hear a lot that there are many problems with the world because we have no empathy, so empathy is talked about as a solution to social ills. And we hear a lot of similar things about AI. So, if our life is supposed to become so much better because of AI, are we now to expect that these systems must somehow have a capacity for empathy built within them?”. (Workshop 2 notes, 23/06/2022).
Within the same strand of critical literature, Pedwell (2016: 2) asks: “how might we understand the complex links between empathy and transnational relations of power?”. By transnational, the author refers to the “inter-related and shifting processes of colonialism, slavery, diaspora, migration, development, globalisation, neoliberalism and global media” (Ibid). When placing it in a transnational ecosystem, Pedwell (2016: 13) demonstrates that empathy represents an effective technology for maximising economic competitiveness within circuits of capital. She adds that the mobilisations of empathy depend on gendered, raced, classed and geo-political distinctions. Such distinctions also determine who can capitalise on empathy as a mode of neoliberal capital and who is confined to performing unrecognised emotional labour (Ibid). Simply put, when empathy is understood as transnational politics, its uneven effects are revealed, being linked to geo-political distinctions and exclusions.
Here, we can establish linkages with empty discourses of AI shared by the companies that dominate what is called data capitalism (Sadowski, 2019). The AI industrial landscape is characterised by the centrality of commodification and extraction of data, as well as a concentration of compute capacity and AI talent (Verdegem, 2022). Only a few big tech companies have simultaneous access to large amounts of data and can invest in a high-performance computational structure needed to run AI models. In sum, as a prime mover of capital accumulation (Brevini, 2022), AI is used to increase capital accumulation in the Global North while also worsening inequalities in the Global South. Thus, the explicit geo-political exclusions that Pedwell (2016) critiques in empty understandings of empathy emerge in relation to the transnational dynamics of data capitalism. With this critical framing in mind, we started discussing another creative My Day in AI output, an illustration accompanied by a commentary produced by Participant 2, a female postgraduate student (Figure 2). Just a few individuals create AI software, but billions are using it on daily basis. Consciousness won't be extracted but it could be shared and used as Augmented Intelligence. The symbiosis between humans and machines gets stronger. Who will benefit from it? The few or the billions of us? (Excerpt from Participant 2's commentary on her illustration, 18/06/2022).
Illustration entitled A non-linear process of continual becoming, produced by Participant 2, 18/06/2022.
In complementary ways, empathy acquired deeper levels than simply connecting to the feelings of others; rather it became associated with a willingness to listen to the oppressed (Freire, 1972). Empathy surfaced as being central to achieving a critical consciousness (conscientização), including an understanding of social, political, and economic contradictions. In other words, empathy could be seen as a political praxis that allowed people to identify with oppressed groups, not just for the sake of finding things in common, but rather to transform the realities that oppress them (Manyozo, 2022: 43). In what follows, we analyse how empathy functioned as a socially engaged lens to look at another critical issue, that of AI and autonomy.
From an individualistic autonomy to a collective sense of Autonomia
In the second workshop, participants revisited another piece of writing by Participant 12. This male postgraduate student was doing gigs at different concerts in London to earn extra cash before he returned to Italy. The piece described the following experience with AI algorithms: “A new notification pops up on my iPhone. It's LinkedIn texting: ‘hey, do you know …?’. I am surprised because the person is a former girlfriend. How did the application propose her contact since we have no direct connection on LinkedIn? Later that morning I got a call from my mother. I tell her that on Saturday I will be busy preparing for Harry Styles’ concert. She has no idea who he is and I realise I also don’t know much about him. After lunch, TikTok's algorithms began to offer me a - previously non-existent - amount of Harry Styles videos”. (Participant 12's Notes on My Day in AI, 20/06/2022).
As participants identified with Participant 12's experience of being pushed content, the focus was on how flawed and invasive these systems were. After all, “the recommendation system could not really tell that Participant 12 was not truly interested in Harry Styles” (Participant 18, Workshop 2 notes, 23/06/2022). Another participant said that she felt disturbed by Participant 12's writing because “AI is telling Participant 12 who to reconnect with, how to be better informed for a conversation with his mum….” She then asked: does this mean we have no more autonomy over our interactions, what we know and what we say?” (Participant 2, Workshop 2, 23/06/2022).
Initially, the groups’ interpretations of autonomy seemed to revolve around an individualistic notion, which is common in Western/Global Northern comprehensions of the term. Participants seemed annoyed with what they perceived as technology reducing the power of individual decisions. According to Floridi et al. (2018: 698), in the context of AI, the conceptualisation of autonomy relates to the decisions of people whether to adopt AI and “willingly cede some of their decision-making power to machines”. Autonomy – in AI ethics – thus deals with how to safeguard autonomy for humans and keeping their control over machines/computer systems. This brings us to a fundamental discussion in AI circles, and the so-called control problem, or the need to keep human beings in control over machines and AI systems (Russell, 2019). Similarly, our workshop participants criticised recommendation systems for their insubordination to our (human) true needs and wants. “Are these recommendation systems”, they asked, “having too much control over what we say, do, and feel?” (Workshop 2 notes, 23/06/2022).
Participant 12 further complicated the debate by linking automated recommendations to automated decision-making systems. In response to group members’ critiques of what they perceived as “stupid recommendations that obviously do not reflect what we like and want”, he noted that “at least these are just annoying, but not necessarily harmful” (Workshop 2 notes, 23/06/2022). “Things get more serious”, he added, “when the systems start making decisions for us. I live in a remote village in Italy and when I order things online, these systems have decided that my address does not work. Am I citizen of a lesser value because my family lives in a village?” (Participant 12, Workshop 2 notes, 23/06/2022).
Here, again, PAR contributed to a development of these debates in ways that are consistent with a process of consciousness raising or conscientização (Freire, 1972). Talking about how systems classified some addresses as recognisable and therefore deserving of home deliveries, while other addresses and people were not, participants started to address ethical dilemmas about AI and autonomy. As we asked in relation to empathy, what does autonomy really mean from the perspective of the oppressed, to use a Freirean term?
Participant 5, a male tech activist, raised the issue of data and predictive security policies. He reminded the group that preventing crime was often a justification for discriminatory data-driven decisions, targeting people of colour and working-class citizens (Participant 5, Workshop 2 notes, 23/06/2022). Scholars such as Ricaurte (2022) have written on predictive policies. Such policies, she argues, are problematic because they mean that AI technologies are being used geopolitically as part of a military industrial complex and as a strategic tool for surveillance of the global poor.
Discussing these examples, participants realised that autonomy might have different meanings and implications, depending on whether we are speaking about a privileged Northern or a marginalised Southern context. They also concluded that the implications of loss of autonomy might acquire more drastic dimensions from a Global South perspective. They include serious human rights violations. This was the case with the Rohingya refugees in Bangladesh who had their biometric data shared in ways that could further endanger them (Madianou, 2021). Thus, combining PAR with our reflection in the workshops reveals how AI can make recommendations on who you interact with, what you see, what you talk about and how, but also who gets to be perceived as human, who gets to be free or incarcerated, who gets to live or to die.
Additionally, we can infer that PAR offered a critical lens to analyse issues of AI and autonomy in ways that are less individualistic and more collective and politically engaged, taking the side of the oppressed, as it was the case with the Freirean and decolonial feminist interpretations of empathy (Freire, 1972; Chouliaraki, 2013; Pedwell, 2016). Along these lines, Couldry and Mejias (2019: 155) define autonomy not as a self-rule but rather as a socially grounded integrity without which we cannot recognise ourselves or others as selves. To cite the authors, “autonomy is crucially understood as something that we cannot trade, a minimal integrity of human life – this what we give up when we give up autonomy” (Ibid).
Thus, we are reminded of how important it is to salvage the concept of minimal integrity of human life as autonomy. When a person is branded as a future criminal given her/his childhood via an algorithmic decision system, such integrity is severely disrespected and thus there is no autonomy. Authors such as Arturo Escobar (2018: 172) add that autonomy acquires an additional layer of political density. Using the word in its Spanish form, autonomia becomes about an “altogether different form of rule anchored in people's lives, a struggle for liberation and for a new type of society in harmony with other people and cultures”. Furthermore, the PAR approaches in our workshops led us to an understanding of autonomia as a political horizon guiding political practice based on inter existence and interbeing.
During our discussions about empathy and autonomy in the second workshop, we realised that none of these concepts could be analysed in isolation. Rather, they were all interconnected in ways which were complex and needed to be mapped out. We decided then to generate a visual scheme on a Padlet board, creating columns that identified themes/problems, common issues and questions in relation to AI and data (in)justice. Each participant was asked to add her/his contributions on a shared Padlet (see below) in her/his own time.
We then revisited the Padlet and our discussions about autonomy in the third workshop. We were confronted with a chain of interrelated problems, issues and questions, such as: (1) “prejudices are encoded in AI, one size does not fit all” (column 1 in Figure 3), which then leads to “discrimination, bias, exclusion/inclusion” (column 2 in Figure 3) and to the question “how can we devise technologies that are more responsive to the complexities of society?” (column 3 in Figure 3); (2) “recommendation systems are flawed, they can trigger anxieties and interfere with your autonomy (make you feel like you are being told)” (column 1 in Figure 3), which is connected to the “autonomy/control issue” (column 2 in Figure 3) and to the question “how can users have more autonomy over their social interactions” (column 3 in Figure 3) and (3) “gender bias – AI technologies dominated by white men” (column 1 in Figure 4), linked to the issue of “power asymmetries and big players – hidden agendas, lack of transparency” (column 2 in Figure 4) and to the question “how can we bridge the gaps between activists, policy makers and technologists in a way that this can generate constructive dialogues on technology and AI” (column 2 in Figure 4)? The questions, particularly the last one, were chosen to be tackled by a creative intervention in our fourth and last workshop.
Screenshot of a Padlet created by workshop participants to identify problems, common issues and questions in relation to AI and data (in)justice (first half of the screen).
Screenshot of a Padlet created by workshop participants to identify problems, common issues and questions in relation to AI and data (in)justice (second half of the screen).
Dialogue: challenging hierarchies of (artificial) intelligence
One week after the second workshop, we were back at the university room – all staring at the Padlet projected onto a screen and scratching our heads. Participant 3, a male tech worker, broke the silence: “Yeah, I wonder if this bridging of dialogue is at all possible”. I mean, I work in tech: The business models that we have in tech are not built to support dialogue. There is so much more profit to be made from polarisation than from dialogue” (Participant 3, Workshop 3 notes, 30/06/2022).
Before we turn our attention to the issues that revolve around AI and dialogue, there are some critical points to be made about Freire's conceptualisation of dialogue (Freire and Macedo, 1995) and how they are key to PAR approaches. Dialogues are multilayered, and journeys of dialogue are comprised of various stages. Humanisation represents one of these stages. It is not just about repositioning the oppressed in a better condition. It is about redrawing the relationship oppressed-oppressor and addressing the power asymmetry of this relationship (Suzina and Tufte, 2020). Freire has also made essential points on humility as a pathway in the journey of dialogue. Indeed, humility features as a requirement to recognise that people – any people – are knowledgeable. Humility also suggests that the authentic truth and indeed the authentic intelligence, to use a term that is central to this article on AI and data (in)justice, do not reside with any individual or group, nor can it be forced upon one group by another.
Our participants were inspired by PAR's dialogical principles in a sense that they started to question: if we are all supposed to benefit from AI, collectively as a society, why is it that marginalised communities in the Global South offer so little input when it comes to AI developments? In this context, we (the convenors) mentioned processes of developing AI ethical guidelines. Referring to the literature in this field, we noted how the development of AI ethics is concentrated in North America, the EU, Japan and a handful of other Global North countries (Jobin, Ienca and Vayena, 2019). This underrepresentation of large parts of the world – Africa, Latin America, Southeast Asia, among other areas – means that the Global South is excluded from having a say and contributing to this debate. Beyond this, questions need to be asked about who exactly is involved in the initiatives that develop ethical guidelines and whether they are representative of society. Participant 8, a female tech worker, added to the debate: “it is easy to identify a lack of diversity. All we need to do is to look at the membership of expert panels and task forces and how they are dominated by white male experts” (Participant 8, Workshop 3 notes, 30/06/2022). Indeed, participants had written on our shared Padlet that “AI technologies dominated by white men” and this was identified as one of the fundamental problems of AI.
Like what happened with AI and empathy, the participants started to establish critical parallels between AI and dialogue. First, participants problematised the idea of an all-encompassing, omnipresent and omniscient AI as well as the premise that “AI systems have either matched or exceeded the capacities of human thought” (Katz, 2020: 94–95). As we discussed earlier concerning AGI, this is based on the questionable promise that machines can do, think and even feel things, not only like humans, but also in ways that are superior to human capabilities. Participant 8, a female tech worker from Peru who was familiar with PAR and Freire's work made the accurate observation that “this idea stands in direct opposition to the principles of humility, dialogue”. She also asked: “why is the tech industry so obsessed with ranking intelligence to begin with?” (Participant 8, Workshop 3 notes, 30/06/2022).
Participant 8's question referred to a common trope in AI discourses: a concern with the categorisation of different types of artificial intelligence in terms of technological innovations. The AI literature is helpful here. Huang, Rust and Maksimovic (2019), for instance, distinguish between AI systems that are mechanically intelligent and designed to perform repetitive tasks; systems that are thinking intelligent, and designed to learn and adapt from data autonomously; and AI systems that may become feeling intelligent and designed to interact empathetically with people (2019: 45). Our participants acknowledged these differences, either by showing concerns about machines replacing humans on an emotional level (meaning that machines could then totally outperform humans) or by showing a scepticism towards the feeling intelligent machines. When we discussed Participant 11's writing about bots who did not care, other participants were quick in saying things like: “come on, AI simply cannot have empathy” (Workshop 2 notes, 23/06/2022). Interestingly, this discussion on distinctions of intelligence-led some participants to reflect upon what makes us human. What makes us human, they said, is precisely the fact that thinking, making and doing are not mutually exclusive. In other words, in humans, brains, hands and hearts can co-exist. “Why do we keep worrying so much about super smart machines instead of focusing on the possibilities of human talent? We can feel while we think”, said participant 20, a female undergraduate student. These observations remind us of the importance of Fals Borda's (2003) sentipensante approaches to questioning Western/Global Northern rigid demarcation and binarisms.
Here, the group also offered us what they referred to as a pragmatic perspective. When asked about the problematic implications of the exclusion of Global South marginalised communities, Participant 19, a male undergraduate student, said bluntly that “including everyone is simply too idealistic” (Workshop 3 notes, 30/06/2022). Another observation related to the inevitability of AI. “AI is here to stay”, said Participant 18, another male undergraduate student, “so we better find some good in it” (Workshop 3 notes, 30/06/2022). We, the convenors then asked: “should we just accept the fact that the rise of AI is inevitable, even if it comes at the expense of excluding large amounts of the global population (and even if large exclusions are inherently non conducive to dialogue?). The response was nearly unanimous with participants saying that AI is indeed inevitable, requiring a pragmatic attitude to just make the “best of these technologies so that they can advantage as many people as possible”, as Participant 18 put it (Workshop 3 notes, 30/06/2022).
Here, one could argue that the acceptance of the inevitability of AI as a technology of exclusion is not consistent with PAR perspectives on (1) empathy, as in taking the sides of those who are oppressed (Freire, 1972), (2) autonomy, as in struggling for a new and more just social horizon (Escobar, 2018); and (3) dialogue, as in creating horizontal relationships in which all realities matter (Freire and Macedo, 1995). At the same time, empathic dialogues represent complex and ongoing processes. They do not only involve agreement but also conflict, negotiation. (Pedwell, 2016: 20). In any case, in our workshops, Latin American PAR functioned as a tool to share reflections, ask critical questions and imagine alternative realities.
Concluding remarks
In this article, we focused on AI and some of its problematic implications for data (in)justice, placing it within the emerging field of critical data studies (CDS). By embracing a South-North flow, we aim to inspire alternatives to epistemological and methodological perspectives that stem from the Global North. In the introduction, we asked how PAR, a radical approach developed in the past, could inspire us when addressing AI and data (in)justice in the present and future. We also questioned how an approach focused on empowering the oppressed can be applied to people and communities who have not experienced as much social marginalisation. After conducting and analysing the data from the workshops, we found that one can learn a lot from the application of PAR in the Global North, in contexts where people face different struggles and inequalities.
We have shared a journey in which the Latin American tradition of PAR inspired us in multiple ways to delve into issues of AI and data (in)justice. The first inspiration came from its focus on vivências, making it easier to share the lived experiences of AI. We hope to have demonstrated how PAR's fluid boundaries between subjects and objects, researchers and researched, teachers and students, were conducive to an environment of asking critical questions that are insightful for CDS. Here, PAR's less hierarchical processes of collective knowledge building contributed to making participants more attentive to power dynamics. After four days of workshops of making (creative artefacts), thinking and feeling about AI and data (in)justice in sentipensante ways (Fals Borda, 2003), three main PAR-enabled frameworks emerged to understand critical issues that revolve around AI: empathy, autonomy and dialogue.
Examining these three elements in ways that are politically and socially committed rather than empty emerged as a research priority. Here, empathy became more aligned with its decolonial and feminist interpretations (Chouliaraki, 2013; Pedwell, 2016) as well as with the Freire (1972) notion that empathy should be about taking the side of the oppressed. Reflecting about autonomy, the debates also shifted – from concerns about recommendation systems and individual control over machines to automated decision-making and its implications for who gets to be perceived as human, who gets to be free or incarcerated, who gets to live or to die. In sum, autonomy started to be comprehended less as an individualistic phenomenon and more as a collective fight for fairer social realities (Escobar, 2018). Finally, participants realised that humanising all actors - by placing them all on equal grounds - was a prerequisite for dialogue. It also constituted a form of humanising intelligence, which stood in opposition to the idea of superior artificial intelligence.
These three elements – autonomy, empathy and dialogue – are deeply intertwined, with one contributing to the other. From a Global South angle, for marginalised groups, autonomy is about self-determination, or the ability to have a greater say on one's own life destinies. This possibility is something that is at the centre of the struggles of the oppressed. Thus, if we aim for social change, we must show empathy in a political solidarity sense. Also, by engaging in open and horizontal dialogues, we can humanise and empathise with each other.
Although Latin American PAR offered us valuable lessons, our approach had limitations. Constraints in applying PAR are related to how (most) scholarship is organised in the Global North. Often, this must start from clear objectives to generate maximal impact, which stands in opposition to the open agenda and bottom-up approach of PAR. Although impact generation is aimed at delivering a positive impact on non-academic stakeholders, the rhetoric of co-creation and development is often hollow, as it is challenging to engage marginalised communities in the research process (as they need to prioritise their time and energy). We support Lehuedé's (forthcoming) call for a radical critical reflexivity, to avoid a double helix of extraction: data extraction by the tech industry and knowledge extraction by researchers and their institutions.
Finally, we were inspired by Latin American PAR, situating it within Global South approaches, acknowledging the strength of South-North and South-South dialogues as tools to enhance projects of political solidarity among the oppressed (Medrado and Rega, 2023). We do not claim a kind of Latin American universalism stemming from the South. Instead, our aim is to call attention to the need to treat the Souths as sources of inspiration in humble attempts to learn from and with each other in CDS. One future direction would be to establish dialogues with media outlets and other partners across the Global South and North to analyse AI and data (in)justice. Can this (Latin American inspired) dialogical approach shift our attention from AI myths of superior intelligence and top-down problem-solving to asking more questions about AI, autonomy, empathy and dialogue? Yes, it is true, these issues are complex. However, while we might not always do justice to them, this must not prevent us from being guided by a concern with (data) justice in our shared journeys of knowledge building.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
