Agency in a datafied society: an introduction

Abstract

In media and communication research the datafied society has become a core concept for describing a society in which data-based sense making has become a fundamental principle of the construction of reality. In datafied societies it is often suggested that the possibilities of human agency are evolving, while also recognizing that a new form of machinic agency has become a ubiquitous phenomenon of daily life. The thematic issue Agency in a datafied society takes up such a discussion and aims to contribute to further clarification through theoretical and empirical articles. This introduction aims to achieve two objectives: First, it addresses the intellectual risks posed by empirical and conceptual ambiguity linked to vague interpretations of a datafied society. In tandem, it systematically categorizes the diverse applications of the term agency in the context of investigating a datafied society. Both serve as a foundation for situating the further contributions to this special issue in within the broader discourse of the field.

The intellectual risks of empirical and conceptual ambiguity

It is not solely media corporations that have heralded data as the ‘new oil’ underpinning their business models. In recent years, the term datafication has emerged as a talking point within media and communications research. Alongside the wide public excitement surrounding big data and data analytics (Mayer-Schönberger and Cukier 2013), media and communication scholars have increasingly grappled with the implications of digital media and their serving not only as means of communication but also as generators and processors of data in real time. Various theoretical works have criticized simplistic understandings of digital data (e.g., boyd & Crawford, 2012; Puschmann and Burgess 2014), while empirical studies have looked at the processes of data generation itself (Athique 2018; Schäfer and Van Es 2017). Another significant impetus for this trend has been the dynamic progress of computational communication research (Atteveldt and Peng 2021; Rogers 2019), while others have criticized the ‘surveillance capitalism’ (Zuboff 2019) and ‘data colonialism’ (Couldry and Mejías 2019) that have emerged. Supported by scholars from media and communication studies, the field of critical data studies has evolved (Dalton et al., 2016; Hepp et al., 2022). In the meantime, datafication has gained recognition as a distinct area of media and communication research (Flensburg and Lomborg 2021). Broadly, datafication is defined as ‘the transformation of social action into online quantified data, thus allowing for real-time tracking and predictive analysis’ (Van Dijck 2014: 2, discussing Mayer-Schönberger and Cukier 2013). Therefore, a datafied society is understood as a society in which datafication has become a pervasive and everyday phenomenon (Hintz et al., 2019: 54–60).

Considering the dynamic development of research on datafication and the corresponding notion of datafied societies, it is essential to approach claims about the dominance of any single-word type of society with caution, especially when viewed through the lens of the history of social sciences. Reflecting on past trends, such as the propositions of an ‘information society’ (Beniger, 1986), a ‘network society’ (Castells 1996) or a ‘platform society’ (Van Dijck et al., 2018), it becomes evident that such declarations can quickly become outdated or closely tied to their specific historical contexts within just a few years of their proclamation. There is an obvious rationale for this: The concepts mentioned above all share a fundamental pattern – namely, that they postulate one singular aspect, closely tied to media and their infrastructures, as dominant forces shaping our society. For instance, electronic media proponents argue that ‘information’ should define everything, while proponents of internet-based media argue that the ‘network’ holds the primary defining power. Similarly, proponents of social media contend that it's the ‘platform’ that shapes society. This framework poses risks of both empirical and conceptual ambiguities, a concern that also applies to the notion of datafied societies.

Sonia Livingstone (2019) raises a critical point for audience research within the context of datafication. While acknowledging its undeniable presence in contemporary media communication and its subsequent importance as a research area, she challenges existing approaches. However, she considers the type of research and theorizing that takes place in relation to datafication to be problematic, especially when it comes to audiences and people in everyday life. Digital media, their infrastructures and the underlying corporations that own and operate them are assumed to have an unquestioned power, which corresponds to a new adaptation of the political economy approaches of the 1960s and 1970s. In research on the datafication of audiences, users once again appear to be at the mercy of the media; under the banner of datafication, the focus is more on the (assumed) structuring power of the media itself than on the practices of humans. As Livingstone puts it: ‘Away with the socio-cultural, displaced by individual “behaviors.” Away with context, meaning, interpretation, for it is the hidden patterns beneath awareness that matter. Away with audiences’ motivations, commitments, and concerns – for if data reveal what people “really” do on and through digital media, why talk to them anymore?’ (Livingstone 2019: 176f).

We can see the questions raised by Livingstone as the crystallization point of the discussion in this thematic issue: If we do not want to follow a simplistic discourse that revives the old assumptions of an all-encompassing and one-directional power of media corporations under the framing of datafication, if we do not want to replace the analysis of media practices with the analysis of online digital traces only, then we are inevitably confronted with questions of agency: How do humans act in today’s media environment? Which forms of practice are supported, which are limited? Do we need different understandings of agency than those we have had up to now if we address such questions in relation to today’s digital media and their infrastructures?

This last question in particular is anything but trivial, especially when we realize that the phenomena discussed under the term datafication were also only a snapshot of the ongoing change of our media environment and related processes of (deep) mediatization: As datafication has progressed, the automation of communication has recorded momentum (Andrejevic 2020; Napoli 2014), as the latest forms of communicative AI based have revealed (Guzman and Lewis, 2020; Hepp et al., 2023a; Stenbom et al., 2021). How exactly can we grasp agency when – based on datafication – machines are now ‘able to communicate’ (Esposito 2017: 250)?

The tech industry in particular – whether framed in utopian or dystopian terms – assumes that communicative AI has a great capacity for action of its own. This is exemplified by the mission statement formulated by Sam Altman himself as CEO for OpenAI, in which he puts communicative AI on an equal footing with humans in terms of its potential for action when he defines ‘artificial general intelligence’ as ‘AI systems that are generally smarter than humans’ (Altman 2023). The attribution of such a comprehensive agency can be understood as part of a process of ‘talking into being’ (Bareis and Katzenbach 2022) in which communicative AI is constructed in public discourse as a major, disruptive invention with an assumed agency of its own. As a consequence, any lack of investment in communicative AI would come with the danger of lagging behind as individuals, corporations, or nation-states in the ongoing technological advancement. Such narratives are deeply ingrained within the current ‘AI hype’ (Züger et al., 2023).

We know from sociological research on the economy that these ‘fictional expectations’ (Beckert 2016: 61) are a fundamental aspect of enabling potential future developments in the present (cf., Barbrook 2007; Daub 2020; McCray 2013; Turner 2006). However, there is another crucial point to consider: the intellectual risk that we have already discussed in relation to the datafied society also applies to communicative AI. There is a danger that we may not fully grasp – empirically – the actual challenges that communicative AI poses for human agency due to conceptual ambiguities surrounding agency.

With this thematic issue, we want to counteract the risk of empirical and conceptual ambiguity. All of its contributions focus on investigating the challenges for human agency posed by the latest digital media and their infrastructures. The phenomena covered in the articles are diverse. They deal with self-service checkouts at super markets (Andreas Bischoff & Christian Pentzold), intelligent personal assistants (David Waldecker, Tim Hector, and Dagmar Hoffmann), social media and influencers (Astri Moksnes Barbala), video advertising for personal assistants (Miriam Lind and Sascha Dickel), interaction with Google Maps (Jeffrey Wimmer and Peter Gentzel), the institutionalization of machinic agency after anthropocentrism (Jernei Markelj and Claudio Celis) and subject rights in data protection (Phoebe Moore). The various articulations aim to reflect on what each phenomenon means for rethinking our concepts and theories of agency in general. Overall, we hope that this hermeneutic move from empirical investigation to conceptual reflection can advance our understanding of agency in what we call datafied societies.

Sorting agency: Humans, machines and everything in between

In summary, there is a pressing need to reevaluate issues of agency in light of the ongoing transformations within our media environment. However, this examination runs the risk of being incomplete if it solely relies on simplistic assumptions of technological efficacy. To address these risks effectively, it is essential to lay down foundational considerations about agency as a precursor to further argumentation. ¹

According to social science definitions, ‘agency describes the locus of action, whether in the person, in language, or in some other structure or process’ (Denzin 2014: 75). This broad understanding already underscores the interdisciplinary nature of the discussion, as agency is situated differently depending on the disciplinary and theoretical positions adopted. For instance, linguistic speech act theory has traditionally viewed humans as possessors of agency (Searle 1969). Social theories have developed understandings of supra-individual agency, such as the agency of organizations or collectives (Giddens 1984). Foucauldian research assumes that certain discursive practices reveal agency (Foucault 1970). These examples demonstrate that the social sciences and humanities have very different ways of locating agency and determining which social entities (such as animals, natural phenomena, ghosts, or machines) they consider capable of agency (Lindemann 2016). Does agency in processes of communication solely belong to humans? Or does, for example, communicative AI possess its own agency? If the latter is true, how does this relate to and interact with human agency?

Even if – as the quote from Altman exemplifies – corporations developing communicative AI themselves assume that it possesses independent agency and repeatedly argue that communicative AI might eventually replace humans (or even become a new super-intelligence, see Kurzweil 2005), the social science discussion about the digital media and their infrastructures is much more restrained.

On the one hand, constructivist theories such as social phenomenology, communicative constructivism, and systems theory emphasize the idea that machines (apps, digital assistants, bots, etc.) can be described as an ‘objectification’ of human action and that the agency attributed to them is a projection of human actors or a personification of their expectations (Knoblauch 2020; Lindemann et al., 2016; Muhle 2016). The machine is understood as the ‘objectification’ of human intentionality (Knoblauch 2020: 118; Pfadenhauer 2014: 144). Nevertheless, humans may experience the machine as intentional, that is, as having an agency of its own. Here, agency is understood as an observer-related category. We are then dealing with a human form of ‘projection’ (Knoblauch 2020: 114), just as in certain cultures nature is conceived as having agency. A specific variant of this kind of thinking can be found in systems theory, which assumes, based on its particular approach to communications, that the decisive question is whether a ‘personification’ of the machine takes place in the act of human-machine communication and that agency is attributed as a personalized formation of expectations (Muhle 2018: 155-159, see also Baecker 2011: 23–25). Such a theoretical turn is not so different to some computer science approaches. We can see at this point a parallel to critical work such as that of Weizenbaum (Natale 2021) who wanted to demonstrate with ELIZA that even simple systems can evoke in humans the assumption of their own agency without being capable of ‘understanding’ (Weizenbaum 1967: 474). The same can be said for today’s machine translation which is based on pattern recognition using large amounts of data, therefore developing some form of agency without any ‘understanding’ of meaning (Gunkel 2020: 97–134).

On the other hand, new materialism, actor-network theory, and extended action theory emphasize the idea of the distributed or joint agency of humans and machines (Fink and Weyer 2014; Gunkel 2020; Hanson 2009; Latour 2007; Neff and Nagy 2016; Rammert and Schulz-Schaeffer 2002; Schapals and Porlezza 2020). From this perspective, machines (apps, digital assistants, bots, etc.) do not develop an agency on their own, but in their ‘entanglement’ (Barad 2007; Scott and Orlikowski 2014) with humans, a new form of agency ‘in-between’ emerges that would otherwise not exist (Hepp and Couldry 2023). Relational approaches are seen as a way of overcoming simplistic dichotomies and arriving at more complex understandings of agency (Gunkel 2020: 277–281; Coeckelbergh and Gunkel 2023). One starting point is provided by actor-network theory which focuses on ‘collectives’ (Latour 2007: 247), not only of humans but of assemblages of humans and things which collectively build upon the social and, therefore, possess agency. In parallel, F. Allan Hanson has developed what he calls ‘extended agency theory’, in which he theorizes the ‘joint responsibility’ of human and machine, where ‘moral agency is distributed over both human and technological artifacts’ (Hanson 2009: 94). In a similar way, Werner Rammert has written about the ‘hybrid interdependencies of action’ (Rammert 2007: 79) from the perspective of the sociology of technology (see also Matsuzaki 2011).

However, discussing agency in relation to a datafied society is not just a question of its ‘holder’ (human, thing, both; see Gunkel 2023). Another crucial aspect pertains to the discussion about ‘talking into being’, as previously mentioned. This aspect involves the future orientation of human agency. If we follow the arguments of action and practice theory, one fundamental aspect of human agency is its ‘projective element’ (Emirbayer and Mische 1998: 971). This refers to actors’ potential futures that emerge from their own practices: A conception that involves ‘fantasizing, imaginative anticipation of future behavior’ (Schütz 1971: 22). It is true that one assumption of ‘big data’ is the idea that the future could be predicted through the analysis of large amounts of digital data using probability statistics (Esposito 2022). Irrespective of the fact that this aligns more closely with the ‘ideology of dataism’ (Van Dijck 2014: 198) than with actual possibilities, the ‘projective element’ also carries another significance here: it signifies that human beings imagine the future consequences of their current actions, and this future orientation is an important element of how human agency unfolds in the present.

It is precisely this projective orientation towards possible futures that is a significant element of agency when it comes to emergent digital media and their infrastructures. A well-known saying (often attributed to Alan Kay, one of the XeroxPARC developers of the graphical interface) is that ‘the best way to predict the future is to invent it’. This kind of thinking was also embraced by the network formed around the Whole Earth catalog and disseminated not only to the up-and-coming digital companies in the San Francisco Bay Area, but also worldwide through publications like Wired (Turner 2006). In a similar vein, institutions also wielded a lasting influence on the imaginaries surrounding today’s digital media, such as the MIT Media Lab (Negroponte 1995). These ideas about technological futures were carried into the world by pioneer communities and tech movements (Hepp et al., 2023b), underscoring Silicon Valley’s reputation as the epicenter of shaping the ‘global imaginary’ of a digital future (Marwick 2017: 321).

When examining digital media, it is imperative to take into account the sociotechnical ‘visions’ (Hilgartner 2015: 33) of technologically advanced groups, as well as broader societal ‘imaginaries’ (Jasanoff and Sang-Hyun 2015: 189). Our contributions in this special issue emphasize that agency in relation to digital media and their infrastructures is always located in particular sociotechnical contexts and therefore does not solely depend on individual capabilities. Sociotechnical visions and their accompanying imaginaries are an important link between individual practice and societal realities. Agency – in whatever way related to the datafication and automation of communication – is always defined by its social localization and situational realization. This includes questions of language, emotion, and power relations which are part of agency’s social construction.

Lines of argument in this special issue

In this special issue on ‘Agency in a datafied society’, comprising eight articles, it becomes evident that this topic demands examination from various angles. Following these contributions, four perspectives stand out as particularly significant in the current discourse: perspectives on situations, discourses, interrelations, and institutions.

The importance of situations for the establishment and implementation of agency is shown in the first two articles by Andreas Bishoff and Andreas Pentzold on the one hand, and David Waldecker, Tim Hector, and Dagmar Hoffmann on the other. Waldecker, Hector and Hoffmann refer to how users interact with and control intelligent personal assistants in specific situations. Examples show users actively intervening, such as unplugging the device during installation or taking control over the suggested commands. This situational embeddedness of agency helps users to maintain control over the interaction process with the IPAs. Bischof and Pentzold explore how users overcome challenges in self-checkout situations, relying on support from fellow customers and store assistants. It emphasizes the role of social dynamics within the situation on users’ ability to complete transactions, highlighting their dependence on others for help in the face of technical malfunctions or scanning difficulties. Both articles show that agency is not only based on individual skills and knowledge but is also strongly influenced by the situation and the embedded character of social interactions. In the self-service checkout situation, agency is characterized by dependence on other customers and store staff, while in the case of smart speakers, agency is constructed by situational user intervention, conflict and the manifestation of individual notions of control. Overall, these two articles show how important it is to consider in our discussion around agency in the datafied society that – despite society’s saturation society by media technologies – human agency is produced and carried out in social situations.

However, the production of agency is always also a discursive construction. Accordingly, discourses are the focus of the following two articles in this thematic issue which underline the need to consider the role of language, emotion, and power relations. In her contribution, Astri Moksnes Barbala analyses the engagement of feminist influencers on Instagram as part of the emerging #MeToo discourse. She highlights the emotional and transformative dimensions of agency during the #MeToo campaign and individual voices and experiences are incorporated to underline different perceptions and expressions of agency. Astri reflects on narrative changes through the #MeToo campaign, particularly in media coverage, revealing the discursive construction of agency. The discourse highlights how language, framing, and personal narratives shape possible forms of agency within the #MeToo discourse. Miriam Lind and Sascha Dickel are more interested in newer technologies in the form of intelligent personal assistants (IPAs). They explore the representation of IPAs’ assumed agency by examining advertising campaigns. Their analysis shows limited agency in the portrayal of IPAs as obedient helpers carrying out human commands with limited autonomy. There emerges an asymmetry between humans and the artificial. IPAs therefore exhibit a pre-defined agency, confined to specific roles. The perlocutionary intent of advertising is to evoke positive perceptions, emphasizing close connections, safety, and convenience, while downplaying concerns about privacy and surveillance. The discourse often overlooks the significant influence of corporations such as Amazon and Google as third parties shaping human-machine relationships. Together, the two articles show us how important, but also how complex, the dimension of discourse is when considering agency. In a sense, we can see both articles as cornerstones of a spectrum. Astri is concerned with what discourses about digital platforms mean for the production of human agency. Lind and Dickel explore how discourses on the agency of technical systems can shape our perceptions of them and the assumptions made in the process. In any case, both articles underscore the importance of considering discourses in our examination of agency in datafied societies.

Another important perspective is that of the interrelations between human and machine. These interrelations are the subject of the articles by Ignacio Siles on the one hand and Jeffrey Wimmer and Peter Gentzel on the other. Siles argues for a move away from a deterministic understanding of technological power or approaches that present it as an all-or-nothing condition. He points out that similar techno-deterministic assumptions were also prevalent in communication studies and have been refuted by empirical work, which has led to the development of fruitful theories that have emphasized the agency of media users (e.g., the ‘two-step flow’ of communication or the ‘uses and gratifications’ approach). Siles proposes a relational framework for analyzing the power of algorithms and the question of human agency in this context to explain through which intersections and entanglements agency arises in everyday life. This relational framework considers tensions, mediations, and transversalities. We find a different emphasis on interrelations in Jeffrey Wimmer and Peter Gentzel’s study which examines agency as it emergences in people’s everyday interactions with Google Maps and other mobile map applications. This qualitative study deals with a different form of relationality, namely, that between cartographic representation and one’s own place-related practices. They show how a new form of agency emerges in this relationality, which they describe as an ‘everyday world navigation’ and characterize as an ‘extended agency’ at the interface of human practice and technical system. While one article approaches this concept more from a theoretical standpoint and the other from an empirical perspective, both highlight the significance of addressing interrelations to comprehend the potential forms of agency facilitated by contemporary digital media and their infrastructures. Grasping the forms of agency possible at the intersections of both humans and things is crucial. It’s essential to reject any techno-deterministic tradition and acknowledge that in the complex interplay between humans and technologies, agency emerges from their intricate intersections.

A fourth and final necessary perspective is that on institutions. The production of agency within what is called datafied societies remains closely tied to social institutions, particularly in the form of organizations. This theme is explored in distinct ways in the articles by Jernei Markelj and Claudio Celis, as well as by Phoebe V. Moore. Markelj and Celis discuss the necessity of a shift from an anthropocentric to a post-anthropocentric perspective on value and labor, challenging Marx’s understanding of human labor. Drawing on Deleuze and Guattari’s machinic agency, the authors argue that in the digital platform economy, agency and value result from institutionalized interactions among various human and non-human actors. By contrast, Moore explores wage labor from a Marxist perspective, emphasizing the ‘right to the subject’ and arguing that current European data protection laws are inadequate to protect workers’ agency in algorithmic management under unequal social conditions. Both contributions explore the role of institutions regarding agency. Markelj and Celis focus on the connections forming an institutionalized machinic agency, while Moore examines the institutionalized social dynamics between workers and employers, limiting worker agency through data-based technologies. While such a focus on the institutionalization of work is certainly a specific perspective, both articles demonstrate how important it is to see questions of agency in a datafied society in its broader institutional contexts. This inevitably leads our gaze to questions of power, conflict and exploitation, a discussion that is currently taking place, for example, with regard to ‘surveillance capitalism’ (Zuboff 2019) and ‘data colonialism’ (Couldry and Mejías 2019).

Overall, the articles in this thematic issue demonstrate how important it is to discuss ‘agency in a datafied society’ from a variety of perspectives. The perspectives addressed here, particularly those on situations, discourses, interrelations, and institutions are indeed significant. However, it should not be assumed that these are the only important ones. As demonstrated in the individual articles, we are merely at the outset of the academic debate on agency concerning the swiftly evolving media environment. The next trend emerging after datafication is the further automation of communication through communicative AI. This development brings us back to the fundamental question: What does this mean for questions of agency? Therefore, we do not see this thematic issue as the conclusion, but rather as a further step in an ongoing and highly relevant scientific discussion.