Artificial companions,social bots and work bots: communicative robots as research objects of media and communication studies

Abstract

The aim of this article is to outline ‘communicative robots’ as an increasingly relevant field of media and communication research. Communicative robots are defined as autonomously operating systems designed for the purpose of quasi-communication with human beings to enable further algorithmic-based functionalities – often but not always on the basis of artificial intelligence. Examples of these communicative robots can be seen in the now familiar artificial companions such as Apple’s Siri or Amazon’s Alexa, the social bots present on social media platforms or work bots that automatically generate journalistic content. In all, the article proceeds in three steps. Initially, it takes a closer look at the three examples of artificial companions, social bots and work bots in order to accurately describe the phenomenon and their recent insinuation into everyday life. This will then allow me to grasp the challenges posed by the increasing need to deal with communicative robots in media and communication research. It is from this juncture from where I would like to draw back on the discussion about the automation of communication and clearly outline how communicative robots are more likely than physical artefacts to be experienced at the interface of automated communication and communicative automation.

Introduction

We live in times of deep mediatization. This means that our societies are increasingly saturated by digital media and their infrastructures, which have become constitutive for the social worlds in which we live. Being software-based, these digital media are not only means of communication, they also perform the tasks of continuous data generation and processing. Automation has thus become a key aspect of digital media as algorithms are designed to be ‘functionally automatic, to act when triggered without any regular human intervention or oversight’ (Gillespie, 2014: 170). It is for this reason that a number of researchers use the terms ‘automation’ or even ‘automated media’ (Napoli, 2014: 340) to describe digital media’s latest stage of development.

Automation is a broad phenomenon that can relate to both media use and media production (Loosen and Scholl, 2017; Napoli, 2014). At the level of media use, algorithms enable users to navigate complex, highly networked media using ‘automated orientation aids’ such as recommendation and aggregation systems. In terms of media production, automatically generated user data are fast becoming an important basis from which productive or editorial decisions are made. In addition, individual forms of media content, such as weather forecasts, are created automatically on the basis of information gleaned from databases. Automation also connects media use to media production by automatically processing and visualising data traces of media usage on editorial dashboards, which are then reflected in production processes.

If we take a closer look at automation processes, they refer to a more particular phenomenon, which I would like to call ‘communicative robots’. Current examples of these communicative robots can be seen in the now familiar artificial companions such as Apple’s Siri or Amazon’s Alexa, the social bots that populate Facebook, Twitter and other platforms, or work bots that automatically generate journalistic content. As dissimilar as these examples may appear, they share one fundamental characteristic that clearly illustrates that they represent something far beyond simple automation: They are autonomous systems that serve the needs of human communication. For these systems to function, extensive prerequisites are necessary that local software cannot provide single-handedly, namely, the presence of globalised digital infrastructures on which these systems are based and the information processing that takes place through them.

However, we must be wary of labelling these systems as artificially intelligent agents if we understand artificial intelligence (AI) as a technology based on complex machine learning systems (for the multilayered character of the term, see Russell and Norvig, 2010: 1–5). Twitter bots, for example, are more often than not merely simple scripts designed for the automatic generation of content (Veale and Cook, 2018); in this way, automated modes of media production can be very limited. What actually links these phenomena are several other, more defined components: (a) automation, (b) a communicative purpose, (c) an embeddedness within digital infrastructures and (d) an apparently ‘humanlike’ user interface. These systems are more proximal to the popular representations of the ‘robot-as-servant’ archetype (Jordan, 2016: 56), which leads to prosaic metaphors such as ‘robotic reporters’ (Carlson, 2015: 416) or ‘software robots’ (Jung et al., 2017: 291) when they are addressed by media and communication research. It is my aim in this article to move beyond these kinds of representations so as to more clearly and precisely define the phenomenon of ‘communicative robots’.

In terms of media and communication studies, the preoccupation with communicative robots has so far been characterised by the fact that examples such as ‘bots’ (Gehl and Bakardjieva, 2016a) or ‘robot journalism’ (Clerwall, 2014) have been considered in isolation leading to their exponential rise in relevance. This is all the more remarkable as many robots operate as ‘interactive media’ that are ‘constructed by humans as social counterparts’ (Höflich, 2016: 200; see also Krotz, 2007: 125–160). From a similar point of view, Sherry Turkle (2015) has described the human desire to communicate with machines as the ‘fourth chair’ of conversation – the other three ‘chairs’, inspired by the writing of David Thoreau, represent our needs for, and the effects digital devices have had on solitude, communication with certain others and communication with society.

In everyday language, we typically associate terms such as ‘machine’ or ‘robot’ with discrete artefacts while many scientific definitions consider ‘embodiment’ a crucial characteristic of robots (Gunkel, 2020: 14). But what does embodiment really mean? The embodiment of today’s communicative robots is less about their appearance as discrete, physical artefacts. Rather, we can notice a ‘communicative’ as well as an ‘infrastructural embodiment’: Their communicative embodiment manifests from their representation as ‘human-like’ actors mediated through the software applications with which we communicate (Barile and Sugiyama, 2015: 407–408). Their infrastructural embodiment refers to the embedding of artificial companions, social bots and work bots in material infrastructures on the basis of which they ‘act’. The equation of embodiment with a single and isolated physical artefact thus corresponds more to a replication of popular cultural discourses about robots (Jordan, 2016: 39–68) than it does to an analytical rapprochement with the form of embodiment that we are dealing with: Communicative robots are material and, therefore, embodied, but in a different way to humans.

Based on these reflections, I would like to deal with communicative robots as an increasingly relevant subject for media and communication studies. I will proceed across three stages. Initially, I will take a closer look at the three aforementioned examples – artificial companions, social bots and work bots – in order to accurately describe the phenomenon and their recent insinuation into everyday life (Hepp, 2020). This will then allow me to grasp the challenges posed by the increasing need to deal with communicative robots in media and communication research. It is from this juncture from where I would like to come back to the discussion on the automation of communication and outline how communicative robots are more likely to emerge in the form of software rather than as physical artefacts and how they will be experienced at the interface of automated communication and communicative automation.

From social to communicative robots: an approach to the phenomenon of automated communication

If we cast our minds back over the past few decades, what is noticeable is that household robots have become prevalent in both popular and academic discourse. Fortunati (2018) argues that we are witness to an incipient ‘robotization’ (p. 2675) of private and domestic life. Service robots such as automated vacuum cleaners, lawn mowers and window cleaners are beginning to seep into the popular imagination and into our homes. Part of this more general interest in robotization is also the social science studies guided investigation into ‘social robots’, that is, in autonomous systems that are, as physical artefacts, designed to interact with humans (Böhle and Pfadenhauer, 2014: 3). ¹ One domain in which these social robots are emerging is in the nursing home environment, in which robots designed to resemble small seals are used to care for sufferers of dementia (Pfadenhauer and Dukat, 2015).

To equate the ‘robotization’ of everyday life with isolated artefacts appears problematic, however, if the relevance of communication is neglected (see Bakardjieva, 2015: 247; Fortunati, 2018: 7). On closer examination, and in the context of deep mediatisation, we are confronted less with social robots in the form of artefacts but with communicative robots in the form of software and digital infrastructures. Communicative robots are (partially) automated and (partially) autonomous communication media that serve the purposes of quasi-communication with humans. Their modus operandi is not ‘that the machine is able to think but that it is able to communicate’ (Esposito, 2017: 250). As this type of media is based on an exchange of digital data, the automated processing of these data opens up a wide range of additional functionalities that go beyond the actual act of communicating, functionalities that typically remain ‘invisible’ to their human counterparts.

It makes sense, therefore, to begin an analysis of these systems by looking at communication or, to put it more precisely, quasi-communication – quasi-communication because it is essentially a matter of attributing communication to a machine and not communication in the sense of human symbolic exchange as theorised in symbolic interactionism. Interestingly, this quasi-communication refers to a recursive loop: Human communication has served as a central model for the development of human–machine interfaces, the analysis of which now reveals that machines are perceived and treated by humans in a similar way to fellow human interaction partners. As Andrea Guzman (2018b) aptly puts it, ‘The technologies that are now inspiring communication scholars to pay attention to human-machine communication are the result of decades of research guided by the idea of communication outside the discipline’ (p. 7, see also Guzman and Lewis, 2020: 5).

As part of contemporary everyday life, we can preliminarily identify three types of communicative robots: artificial companions, social bots and work bots.

Artificial companions ² include Amazon’s Alexa, Apple’s Siri, Google’s Assistant and Microsoft’s Cortana. Much rarer, but still present, are artificial companions that take the form of a physical artefact. Anki’s Vector, for example, is a robot that to some degree resembles a miniature bulldozer to whom queries can be directed and is able to take photos on request. One of the Vector’s stand-out features is its ability to seemingly express emotions through a range of animated expressions that appear on its front-facing display essentially signalling to its user its ‘emotional status’ (see https://www.anki.com/en-us/vector (accessed 19 February 2020)). This is also a prominent feature of Sony’s Aibo robot dog which was produced between 1999 and 2006 and subsequently updated and reissued in 2017 (for the history of this robot, see the following fan page: http://www.sony-aibo.com (accessed 19 February 2020)).

The typical structure of artificial companions follows a general scheme: They have a software-based interface that mediates its functions between itself and the user through which he or she can communicate with the system using (spoken) language. This is achieved either through a smartphone app, an application on the computer or through voice commands directed at an ‘intelligent’ speaker or other device. The spoken commands or questions are not then processed by the device itself but are transferred via the internet to data servers. Once a command reaches a server, speech recognition processes configured by AI systems are triggered, the questions and commands are processed and the results are transferred back to the device. The output then takes shape either in the form of a specific activity (execution of a software command) or a spoken response (output of relevant information). Such companions present themselves as ‘conversational vis-à-vis, artificial playmates and interdependent actors’ (Böhle and Bopp, 2014: 164).

These artificial companions are not simply media in the sense that they serve as interaction nodes between people; their functionality is based on datafication, that is, the collection and processing of large amounts of data and they form part of an automated, data-based construction of reality. The Shazam application, which was adopted by Apple in 2017/2018 and can be understood as an artificial companion for identifying songs and discovering music, is one clear example: Shazam identifies songs via the smartphone’s microphone; a snippet of the song is recorded, the resulting audio file is transmitted to the ‘cloud’, it is then compared to an existing database of pre-digitised songs and, if recognised, the result is communicated back to the user through the app (otherwise a message appears stating that the song could not be recognised). Shazam saves these search results in a user database and on this basis creates personal playlists, recommends new music titles, highlights new releases and suggests new songs that the user might be interested in. What is happening here is what Barile and Sugiyama refer to as the ‘automation of taste’ (Barile and Sugiyama, 2015: 413) through an artificial companion. On the one hand, Shazam provides users with extended control of their music consumption. Searching and selecting music becomes an individual act augmented by technical support. On the other hand, the accompanying automated processing and taste classification determines to some degree individual variation and gives the communicative robot a considerable position in the composition of the user’s respective music repertoire.

Besides artificial companions, social bots represent an increasingly common form of communicative robot. In essence, most social bots are automated social media accounts that operate as if ‘real’ people were doing the communicating. More precisely, social bots are ‘software processes that are programmed to appear to be human-generated within the context of social networking sites such as Facebook and Twitter’ (Gehl and Bakardjieva, 2016b: 2). While the idea of bots has a much longer legacy and can be traced back to Weizenbaum’s Eliza (Natale, 2019), the more concrete history of social bots goes back to at least 2008, when scripts imitating Twitter users were first developed. These scripts tweeted text elements that were synthesised from other tweets made by human users. In 2017, research found that between 9% and 15% of active Twitter accounts might actually be bots (see Ferrara et al., 2016; Varol et al., 2017). Instagram is assumed to have similar numbers (see https://www.theinformation.com/articles/instagrams-growing-bot-problem (accessed 19 February 2020)). According to Facebook, in the same year, around 10% of its users were ‘fake’ accounts, which does not mean that each of them is also a social bot (some are human-operated accounts running under a ‘false’ identity; see the company’s stock exchange report: http://d18rn0p25nwr6d.cloudfront.net/CIK-0001326801/06205619-7ced-42ed-b8c8-4621b5a121e9.pdf (accessed 19 February 2020)). These numbers, however, are highly speculative, as there is no real possibility of identifying bots accurately (Lazer et al., 2018). Nevertheless, this phenomenon is becoming increasingly relevant and can be explained by what Sherry Turkle called the ‘robotic moment’. Turkle describes how that ‘even before we make the robots, we remake ourselves as people ready to be their companions’ (Turkle, 2015: 338). In the case of bots, this means that with platforms such as Facebook, Twitter or Instagram, users are represented in a highly structured manner, as are their interactions, hashtags and other elements of their communicative practices, and are defined according to their interaction history. The representation of humans and their communicative practices is pre-structured to such a degree that their replication by machines is comparatively simple. These platforms are therefore, a ‘levelled middle ground’ where humans and robots appear on a ‘significantly equalised footing’ and where social bots ‘have good chances to be successful in presenting themselves as humans’ (Bakardjieva, 2015: 248).

In addition, there exists in various forms, so-called work bots: bots that are used to perform communicative labour. There are several examples of work bots operating in the real world, even in the field of journalism: The LA Times’ Quakebot is an automatic article-generating programme that uses algorithms to convert email alerts from the US Geological Survey (USGS) into news stories (Oremus, 2014). The San Francisco-based Hoodline generates hyperlocal news articles scraped from public databases using work bots (Wang, 2018). Quill is an AI application developed by Narrative Science which transforms data into contextually rich information. In the field of media and communication, these work bots are typically discussed in terms of ‘robot journalism’ (Thurman et al., 2017: 1251) or ‘automated journalism’ (Loosen, 2018: 11). This kind of automated reporting is principally based on ‘algorithmic processes that convert data into narrative news texts with limited to no human intervention beyond the initial programming’ (Carlson, 2015: 417). Or, to put it another way, algorithms arrange data and information from different sources in such a way that the outcome is a text, video or any other kind of media product (Dörr, 2016).

The example of work bots demonstrates that we are most likely less confronted with a simple replacement of humans by communicative robots, rather these work bots operate as something more akin to colleagues: companions in content production that reveal transformations to the journalistic work process. This concerns the role of the journalist who, when using, for example, Narrative Science technologies, is thought of more as a ‘meta-writer’ or ‘meta-journalist’ (Carlson, 2015: 423) who facilitates the production of automated stories. In their application of these technologies journalists are required to pre-programme their stories’ scripts (the line-up of players, the match result, etc.) into an appropriate descriptive language. These scripts push the ‘meta-journalists’ into the use of a certain compositional form as afforded or required by the software. If the data processed by these scripts rely on existing databases the underlying data models have a comprehensive influence over which ‘stories’ can be told and how (Carlson, 2015: 423; Caswell and Dörr, 2018: 492).

Based on the examples outlined so far, it is possible to further specify our definition of communicative robots. Communicative robots are (a) (partially) automated and (partially) autonomous media that (b) serve as quasi-communication interfaces with humans, (c) they are embedded in a comprehensive digital infrastructure and (d) have in most instances an apparently human-like user interface. In a certain sense, they are the next incarnation of what Joseph C.R. Licklider (1960) already imagined as ‘man-computer symbiosis’: Their purpose is not to replace humans but to approach human communication in new ways.

By investigating communicative robots we can see that advanced automation has once again increased the media’s processual dynamics. First of all, this concerns data processing. In the examples discussed so far, the core aspect of automation is the automated processing of data (language, images, video) through the use of databases. The central role played by databases is also the reason why they have become an important subject in the debate on deep mediatisation (Andersen, 2018). What we are dealing with here are new forms of institutionalisation and materialisation based on processes of categorisation. Relational in their structure, these databases are highly flexible, and only loosely connected objects can be ordered in a potentially unlimited number of ways (Burkhardt, 2015: 121–148; Rieder, 2012). Because of this flexibility categorisation is fundamental as even metadata are not strictly ‘raw’ in nature but somehow already organised and, therefore, structured (Gitelman and Jackson, 2013). Categorisation constitutes a powerful semantic and political intervention because ‘what the categories are, what belongs to a category, and who decided how to implement these categories in practice, are all powerful assertions about how things are and supposed to be’ (Gillespie, 2014: 171). Overall, categorisation is an ongoing process with its own dynamics as new data are added based on the assignments made by the designers of those categories. We are thus confronted with processes of an automated, data-based construction of the social world: a type of ‘data structuring’ (Flyverbom and Murray, 2018: 1) which is based on the qualities of specific databases.

It eventually becomes apparent that this automated processing relies on and results in procedures of continuous feedback. One special quality communicative robots possess is that they are designed to communicate data to humans in a targeted way. Targeted communication can take on a variety of forms: It could be the precise answer to a question addressed to an artificial companion, but it is also possible that a bot communicates information on an event – an earthquake, a goal scored during a football match, an exceeded limit value of air pollution – almost instantaneously. Furthermore, it can mean that information about one’s self (tastes, habits, etc.) is collected on an ongoing basis and that suggestions for visiting events, buying new books or accessing additional information are made based on the automated processing of personal data. With communicative robots, we as humans engage with a process of dynamic feedback through machines.

Communicative robots as an object of research: theoretical and methodological challenges

As the research discussed in the last section demonstrates, there currently exists a range of arguments in favour of media and communication studies focusing some of its attention on communicative robots. In a recent paper, Sakari Taipale and Leopoldina Fortunati (2018) described social robots as the ‘next new media’ (p. 206). Their analysis of EU barometer data shows that those people who already make extensive use of information and communications technologies are also those who are most open to the idea of social robots. Furthermore, the interest in social robots lies not so much in the areas assumed by the industry – health and social care, for example – as in those areas closer to personal and public communication. They conclude that ‘perhaps it would make sense to focus more on ICT-like than human-like robots’ (Taipale and Fortunati, 2018: 216). Once again, the phenomenon of social robots is conflated with the idea of communicative robots.

A phenomenon of the everyday becomes an object of research if one approaches it with a certain research question. That said, the phenomenon of communicative robots can be examined along many directions of inquiry. What remains decisive is that a research question is always anchored disciplinarily: It is ‘always placed within the framework of a scientific discipline and formulated in appropriate terms, the suitability of which can nevertheless be questioned in the research process’ (Krotz, 2005: 119; author’s translation). Here, we come to the question of from which perspective media and communication studies might approach the idea of communicative robots.

It is clear that media and communication research will pose different questions than computer science (the field developing these systems) would, and will approach the subject from a different angle to many sociological studies (which are more concerned with social relations). Assuming that media and communication studies is defined by its interest in (technologically based) mediation (Silverstone, 2005), a media and communication approach to communicative robots should focus on questions that deal with processes of communicative mediation: What distinguishes the communicative relationship between humans and robots? How does public and personal communication change in light of the existence of communicative robots? How significant are communicative robots for the communicative construction of the social world and how does this relate to power relations in society? It is questions like these that render the phenomenon of communicative robots an analytical object for media and communication studies and justify the initiation and maintenance of a wide discourse on them.

Nevertheless, this presents various, closely related challenges for media and communication research which can be systematised as specifically theoretical and methodical issues that need confronting. Communicative robots are transversely positioned across the field’s traditional concepts and theories and they require an extension of its methodological instruments.

Theoretical challenges

How far the theoretical challenges of the phenomenon of communicative robots extend becomes clear when we look at media and communication studies’ two basic concepts, namely, ‘communication’ and ‘media’.

The field of human–machine communication (HMC), a newly emerging and increasingly important field in media and communication research, does most of its work based on the concept of communication. ³ If one leaves aside cybernetics (Wiener, 1948) and its application in communication and media research (Merten, 1977), the concept of communication in media and communication studies is based on theories of action, practice and symbolic interaction. Over and above any pre-existing differences – for example, regarding the question of whether communication is always intentional or not (Reichertz, 2009: 81–122) or if communication actually means connection, dialogue, expression, information, persuasion, or symbolic interaction (Waisbord, 2019: 23–39) – each theorisation of communication have a shared understanding that communication is a mutually oriented, symbolic human practice that is oriented towards the production of shared meanings.

This understanding of communication reaches a theoretical and methodological limit when communicative robots enter the fray. Communication with them is typically constructed as a ‘delegation’ of human communication capabilities to technical systems. However, if we look at the examples of communicative robots described above, we are forced to question just how far this direction of argumentation can go. Or, to quote Shanyang Zhao (2006), the ‘emerging movement of social roboticization is causing a fundamental change in the meaning of social interaction and the nature of human communication in society’ (p. 402). If the use of communicative robots continues to expand it will be necessary to adequately grasp how communication with these systems operates – systems that were created based on models of human communication for the purpose of communicating with humans (Esposito, 2017; Guzman, 2018b: 7).

The suggestions for appropriate theoretical approaches vary depending on the theoretical position of those making them (see the chapters in Gehl and Bakardjieva, 2016a; Guzman, 2018a). Beyond their variance, however, they manage to come together in that they emphasise the necessity of overcoming simple metaphors of delegating human practice to technology and include the idea that people, together with machines, can enter into permanent, open-ended communicative relationships. We can understand these conceptual formations as the first steps towards an appropriate descriptive language. Nevertheless, there remains an urgent need for more theoretical work if we are to ever generate useful descriptors for the study of communicative robots: The unique nature of communication between humans and robots forces us to question the specifics of (mediated) communication among humans. Research on communicative robots also contributes to the formation of a general theory of communication – a development that has been faltering for some years.

This leads directly to another fundamental challenge, that of media. As already mentioned, we can understand communicative robots as a certain kind of (communication) medium. If we understand a medium as a mediating instance of communication which is based on a technologically based system of signs that works to form social institutions, comprises organisations and provides certain services (for individuals, groups, organisations, social worlds), then communicative robots are clearly media, but of a special kind. Similar to computer games and the avatars that come with them, we are dealing with media based on ‘virtualised communication’ (Hepp, 2013: 65), that is, an instance of media acting at their core as ‘independent’ communication partners in a predefined space.

It is this question of agency that challenges the formation of concepts of what a medium is and it goes a long way to explain the fact that actor-network theory (ANT) is currently being widely adopted as an approach to research communicative robots (Lutz and Tamò, 2018). One of ANT’s principal considerations is that ‘objects’ in actor networks or assemblages with humans can assume the role of an ‘actant’. It is obvious that such a theoretical approach possesses a certain appeal when dealing with communicative robots. However, in its assumption that society – consisting of these actor networks or assemblages – is ‘flat’, ANT suffers from a range of inadequacies when grasping more comprehensive social forms, especially in regard to power and inequality (Couldry and Hepp, 2017: 61f.). Nevertheless, the burgeoning discussion carried out through an ANT perspective can be seen as an indicator that a meaningful description of communicative robots is not possible without theorising their potential for agency.

Communicative robots are typically ‘media in media’. This means that social bots are based on platforms such as Twitter or Facebook, that Alexa, Echo and Siri require that the information they communicate is already available online, and so on. The point is that an investigation into communicative robots implies a need to research the platforms ‘upon’ which they ‘act’. One important argument in this respect is that digital platforms increasingly assume the role of infrastructures (Plantin and Punathambekar, 2018). ‘Infrastructural platforms’ (Van Dijck et al., 2018: 11) – social media platforms such as Facebook, game platforms such as Steam or software platforms such as the Apple App or Google’s Play Store – form the meta-structure for the supply of other media. This is also the infrastructure through which many communicative robots become available or from which many communicative robots operate.

However, the concept of infrastructure can be considered even more broadly in the case of communicative robots. As has already been argued, communicative robots cannot be adequately understood if they are not considered in light of their embeddedness within advanced digital infrastructures. This does not simply mean that they are based on the infrastructure of the internet or infrastructural platforms, rather, they actually have their own infrastructures. Using Amazon’s Alexa as an example, Kate Crawford and Vladan Joler (2018) have traced the ‘anatomy of an AI system’ (https://anatomyof.ai (accessed 19 February 2020)), which clearly demonstrates that communicative robots are not only the artefacts and applications that form what Lucy Suchman (2007) refers to as the ‘agents at the interface’ but also include the various data centres around the world that enable speech recognition and the responses these systems provide in real time. Communicative robots such as Alexa also demonstrate that such systems are based on (often exploitative) physical labour, the extraction of rare Earth materials and the free labour carried out by users in training the AI systems.

Communicative robots pose further challenges to media and communication research: ‘privacy’, for example, when considered in contrast to the (mediated) ‘public sphere’ – where exactly do we draw the line between public and private when communicative robots occupy private space to such an extent and accumulate highly private data? (Lutz and Tamò, 2018: 146, 152); or the notion of ‘responsibility’ – who bears the responsibility for the ways in which communicative robots actually communicate? (Gunkel, 2018: 222). While these and other complex matters represent a variety of challenges to the field, they can also be grasped as an opportunity to question media and communication research’s fundamental concepts in order to perhaps reach a deeper and more nuanced understanding of what it is exactly that distinguishes human communication and what does not through work applied to the phenomenon of communicative robots.

Methodological challenges

The theoretical challenges outlined above bring with them a variety of methodological ones. There are certainly many aspects of communicative robots that can be described using established qualitative and quantitative methods applied by the social sciences. For example, the use and appropriation of artificial companions can be investigated through observation and interviews or content analyses of the news distributed by online work bots. In this respect, though, it is not a question of completely reinventing the wheel. However, two methodological challenges can be identified that are of particular importance for media and communication research into communicative robots: the challenge of opening up their ‘black boxes’ to reveal what lies inside and the challenge of researching emerging technologies.

In the realm of the everyday, much like many other advanced technologies, communicative robots are commonly presented to us as a kind of black box. From a sociological point of view, the expression ‘black boxing’ means that something is not further problematised in its functioning by people in everyday life (Nassehi, 2019: 205f.). Research that deals with communicative robots that does not want to accept this state of affairs is forced to open this black box. Variations on ‘digital methods’ (Rogers, 2013; Schäfer and Van Es, 2017) and ‘computational social sciences’ (Lazer et al., 2009; Maireder et al., 2015) may provide an effective methodological route to uncovering some aspects of what can be found ‘inside’ it. However, the fact that there is a considerable need for further development of methodological approaches has been demonstrated by the previous discussion on the identification of social bots, the current status of which is that existing automated procedures are not (yet) sufficient (Lazer et al., 2018). In this sense, the determination of what the black box actually is already represents a methodological challenge.

This challenge becomes even more significant if we look at communicative robots in more detail. Due to their software-based character, the methodological discussion about the social-scientific reconstruction of the functionality of algorithms for communicative robots is of particular importance. The potential approaches are manifold and range from a qualitative reconstruction of the power structures inscribed within algorithms (Beer, 2019; Bucher, 2018) to approaches to ‘reverse engineering’ algorithmic functionality (Demeyer et al., 2008). The prying open of the black box is also associated with the development of a methodical view of the complexity of the systems and their infrastructures at hand. As communicative robots are part of comprehensive digital infrastructures, a mapping out of complex technological ‘eco-systems’ (Lutz and Tamò, 2018: 145f., 158) is required if we are to carry out research into them. With the current discussion on platforms and infrastructures in media and communication research, the methodological reflection on how exactly such a mapping might be undertaken is only just beginning.

It goes without saying that this methodological work should be carried out with an eye towards what Van Es et al. (2018) refer to as ‘tool criticism’. Tool criticism reflects on the technological specificity of the (software) tools used for such analyses. Depending on the specific question being asked different methods will be required. Beyond this differentiation, however, it is essential that the various strands of methodological discussion need to come together more closely if one wants to achieve an adequate investigation into communicative robots.

These methodological challenges are amplified further by the fact that we are dealing with emerging technologies. Communicative robots are only now becoming part of our everyday ‘media repertoires’ and ‘media ensembles’ (Hasebrink and Hepp, 2017). One could say that – to take a comparison from communication and media history – we find ourselves in a moment much like the early days of radio, television and the birth of home computing: It is apparent that alternative types of communication media are emerging and that their establishment is part of a comprehensive change in the media environment (Couldry and Hepp, 2017: 34–56). However, we cannot simply observe this burgeoning phenomenon as the completion of the ongoing diffusion of media. We are, in fact, faced with the co-construction of emerging communication media that, in terms of their development and introduction into our lives, is highly fluid (Hepp, 2020: 79–86).

Typically, media and communication research has worked with Delphi studies (see Beck et al., 2000, among others) to investigate emerging developments: Adopting a layered procedure to this approach, experts are questioned on possible developments from which scenarios are generated and the probability of particular occurrences is queried. These studies provide insights into the shapes that the future dissemination and appropriation of media might take. What I see as a methodological challenge here, however, is less the exploration of such scenarios than the concrete investigation of the emergence of these technologies, the inscription of certain societal models and the interactions that take place within them. This will require a new form of ‘laboratory studies’ (Knorr Cetina, 1981; Latour, 1992). These should not only seriously consider the laboratories where communicative robots are developed but should also consider the broader social, economic and cultural contexts in which their emergence takes place. The significance of these broader contexts can be seen, for example, in historical research on the ‘Whole Earth Network’ (Turner, 2006) or current research on media-related ‘pioneer communities’ (Hepp, 2016).

Conclusion: between automated communication and communicative automation

I have set out to outline communicative robots as an increasingly important and relevant area for media and communication research. As mentioned at the beginning of this article, the notion of the communicative robot is a preliminary concept that is to be further formulated in future research in order to define this emerging field. However, it makes sense to construct an analytical bridge so that our attention may be drawn to the fact that addressing only single phenomena such as artificial companions, social bots and work bots, and treating them as discrete analytical objects, leads to a lack of focus on the broader context of automation. Media and communication studies needs to maintain this broader focus because we must assume that public and private communication and the manifold gradations between them are increasingly taking place through means of automated actants. If we want to effectively comprehend communication in an age of progressively intense deep mediatisation we cannot avoid turning to what I have described as communicative robots.

The arguments laid out here have hopefully made clear that the importance I apply to the study of communicative robots is about more than ‘just’ automated communication: Fundamentally, communicative robots serve to automate communication. But there also rests in the background of this discussion what can only be described as communicative automation. Automated communication with communicative robots is integrated into far-reaching and complex functionalities that not only serve to collect data in the pursuit of recruiting communicative robots as agents in the projects of ‘data colonialism’ (Couldry and Mejias, 2019) and ‘surveillance capitalism’ (Zuboff, 2019): the colonialisation of more and more parts of everyday life through data collection procedures, which are subsequently monetised, and, in turn, maintain the project of capitalist value creation. At the same time, there is also the matter of how communicative robots take part in the trajectory towards the general automation of the social world.

This can be seen in current developments of Amazon’s virtual assistant, Alexa, which can now control a partially automated smart oven besides presenting you with your favourite playlists on demand (see https://www.theguardian.com/technology/2019/sep/25/amazon-privacy-echo-devices (accessed 19 February 2020)). Even smart home applications from Alphabet (Google) and Apple are by no means inferior to the idea of automating the domestic environment. Current research on ‘Smart Cities’ (Mosco, 2019) reveals the integration of the systems described in this article as communicative robots linked to the automation of a vast array of urban functions such as public administration, traffic systems and public transport. From the point of view of media and communication studies, the main focus should, therefore, be on a consideration of communicative robots in regard to the transformation of public and personal communication and any associated consequences. For this to take place, however, it will be necessary to keep abreast of the broader contexts of communicative automation.

Communicative robots not only represent an important analytical object for media and communication research because they stand for the automation of communication, they are equally important simply because they enable communicative automation and assign a new sphere of influence to what we normatively refer to as public and personal communication. This all goes well beyond questions of the public sphere without abandoning the principal discussion on mediated communication. It is these dynamics that become increasingly vital in times of deep mediatisation.