Abstract
Chatbots have become a mundane experience for Internet users. Public sector institutions have recently been introducing more advanced chatbots. In this article, we consider two cases of public sector chatbots, one in Estonia and one in Sweden, seeking to challenge the seemingly coherent understanding of artificial intelligence (AI) in the public sector. The aim is to both question the “thingness” of AI and show AI chatbots can be very different things. The material in this article is based on in-depth interviews and observations at public sector institutions that have relatively recently implemented chatbots. We employ the notion of AI frictions as a sensitizing concept to engage with the material and the diverging character of the public sector chatbots in the two countries. In the analysis, we identify AI frictions related to expectations of AI, organizational logics, as well as values connected with the digitalization of the public sector.
Keywords
Introduction
Recently, the Swedish public have engaged in a heated debate about whether and how entry into the health system should be automated with the help of a chatbot (Dagens nyheter 20231122). 1 The discussion revolved around potential risks, increased inequalities between social groups with different needs and capabilities as well as fundamental societal values when it comes to access to the healthcare system. While this specific debate was especially intense, chatbots have become a rather uncontroversial, mundane experience in our online interactions with companies, and this is increasingly the case with public agencies and public sector organizations as well. In this context, we consider public sector chatbots as part of the emergence of the data welfare state (Andreassen et al., 2021; Dencik and Kaun, 2020).
In this article, chatbots are considered a form of datafication infrastructure, as they efficiently turn interactions with public institutions into data points that can be analyzed for purposes beyond the initial interaction. More generally, within the data welfare state, citizens are increasingly advised to interact with data-based infrastructure, as both frontend and backend digitalization is increasing. Chatbots are increasingly framed as in-use AI applications that build on complex cutting-edge technologies. However, there are various types of chatbots, which are more or less complex and often not based on AI technology. In many ways, the framing of (public sector) chatbots as in-use AI applications contributes to what Lucy Suchman (2023) has called the “thingness of AI,” a stable and coherent entity that needs more critical attention. Although there are controversies around AI and in particular chatbots, as illustrated by the Swedish health debate, they do not fundamentally harm the idea of AI as a coherent thing. Rather, they reinforce the idea and expectation of AI as a “thing,” further mystifying and stabilizing the power structures within which AI applications have emerged. At the same time, we still know little about the underlying data-based infrastructure and the broader implications of this tool for mediating state–citizen relations. Although the number of chatbots is increasing, this mundane aspect of the digital welfare state has been largely overlooked empirically. With this article, we aim to destabilize the coherence of AI by examining public sector chatbots that are presented as AI systems and exploring the implications of chatbots in the public sector, which are often introduced without broader discussions and in an uncontroversial manner. Empirically, we explore two examples of public sector chatbots, one in Estonia and one in Sweden. Based on the empirical findings, we consider the broader societal implications of chatbots as a form of public datafication infrastructure. We employ the notion of AI frictions as an analytical lens, based on Anna Tsing’s (2005) conceptualization of friction and Minna Ruckenstein’s (2023) algorithmic friction. Here, AI friction is understood as confrontations that emerge in AI-mediated interactions within the data welfare state.
Chatbots serve as an empirical entry point to study the broader implications of the data welfare state. They are mundane examples of the data welfare state, especially in comparison to scandals related to algorithmic automation, such as the childcare benefit case in the Netherlands, which forced the whole government to resign after causing serious damage to several welfare beneficiaries. 2 Scandals like this are often mobilized to emphasize problematic aspects of the data welfare state and why we as citizens and researchers should care. However, focusing on spectacular and especially gruesome cases invisibilizes the way in which intricate data infrastructure is interwoven with mundane experiences.
Context: the data welfare state
In the following, we situate the exploration of public sector chatbots in the broader context of the data welfare state and suggest that AI frictions are a productive way to engage with infrastructural shifts of datafication in that area. The concept of welfare played a central role in post-war Western societies, but it has lost prominence in recent years and is increasingly contested. Originally, as Raymond Williams (1976) noted, it denoted happiness and prosperity. The idea of organized welfare, with institutionalized support for basic needs, began to take shape in the 20th century, and the term “welfare state” was first coined during the Second World War in 1939. This welfare state is founded on normative principles of universalism, equality, and decommodification, as summarized by Jakobsson, Lindell, and Stiernstedt (2022). These principles serve as the basis for arguments in favor of the welfare state, emphasizing its potential to enhance social cohesion and mitigate risks while upholding human dignity (Jakobsson et al., 2022).
Some of these arguments have also touched on the role of media and technology in either facilitating welfare provision or as an integral precondition for organized and institutionalized welfare, such as public service media (Nikunen and Hokka, 2020). In fact, certain scholars have positioned the media as a central component of welfare, encapsulated in the concept of the “media welfare state” proposed by Syvertsen et al. (2014). In the context of digitalization and datafication, researchers are increasingly delving into the evolving landscape of welfare provision, with Virginia Eubanks (2018) among the first to identify algorithmic systems in the welfare sector and their historical roots of managing poor populations in the USA. According to Lina Dencik (2022), the datafied welfare state operates based on two underlying logics: first, an actuarial logic that individualizes risks associated with social problems, and second, a logic of rentierism, which pertains to the economic model reinforcing the production and circulation of data. Dencik (2022) suggested that these dynamics lead to a restructuring of the social power dynamics that underpin the welfare state. Building on the insights of sociologist Marion Fourcade (2021), we can further add that these changes in the data-driven welfare landscape also entail a reconfiguration of the social contract between citizens and the welfare state. This relationship is increasingly mediated by digital data and technologies, particularly those facilitating algorithmic automation. Chatbots are a central example of this shift in mediating state–citizen relations.
The concept of the data welfare state, characterized by an increased reliance on digital data for decision-making and welfare provision, represents both an evolution and a continuation of earlier trends, including the fundamental principles of certain welfare state models (Esping and Andersen, 1989). Specifically, the social democratic welfare model, built on the idea of universal access, was originally motivated by and structured around the use of scientific knowledge and information. Concepts of social engineering were rooted in statistical data and extensive population registries. The contemporary data welfare state, which is fundamentally based on digitalization, builds on these historical foundations but often emphasizes a shift toward managerial arguments focused on cost efficiency and rapid service delivery rather than the traditional emphasis on universal access to welfare and the promotion of a progressive society. However, the shifts toward the data welfare state come with hurdles and broader societal implications, potentially reinforcing existing inequalities and structural exclusion.
AI frictions
To better understand the implications of infrastructure for datafication in the context of welfare, we analytically tie the notion of data welfare to AI frictions. These frictions challenge the notion of AI thingness, that is, the idea of AI as a coherent product or technology. By examining AI frictions, we shed light on both the diverging understandings of AI’s role in the context of public sector chatbots as well as the frictions in the implementation process. This approach allows us to counter narratives that present AI as a unified seamless technology. To conceptualize AI frictions, we draw on Anna Tsing’s (2005) work on friction and Minna Ruckenstein’s (2023) research on algorithmic friction. We define AI friction as forms of impedance that emerge in AI-mediated interaction within the data welfare state. Crucially, we view these frictions not only as sources of inconvenience but also as having productive potential for positive change.
Tsing wrote her book in the context of a vivid alter-globalization movement with its culmination in the Battle of Seattle in 1999. The movement highlighted the environmental, social and economic inequalities that global corporatization caused. In this context, Tsing explored the connections between the local, national, and global in her multi-sited ethnography of Borneo’s links to the global economy. Instead of following specific objects in global streams of capital, she explores what kind of frictions, diverging interests and perspectives make the universal or global understanding possible in the first place. In her book, she puts forward that the universal of globalization only becomes possible through frictions, for example in the form resistance and feelings of alienation. Part of this process is the production of a coherent presence or what she calls economies of appearance of for example local resources that are supposed to attract global capital.
All this is relevant to understand the process of producing AI in the public sector as coherent project or “thing.” The local frictions of different understandings of what AI means and does in the organization and society at large are an integral part of the process of producing AI as a larger coherent object that the organization aspires to gain from. Minna Ruckenstein invokes the notion of friction in the context of algorithmic everyday encounters. Similarly, frictions in the form of provocations and misunderstandings of what algorithms are and do are necessarily part of producing what we might term algorithmic culture that appears both attractive and repulsive. Through creative acts of engaging and sense-making mundane users contribute to the emergence of algorithmic culture as universal invocation.
Tsing’s understanding of friction relates to the ambiguous character of technological infrastructures. Using the example of roads, she argued as follows:
“Friction is not just about slowing things down. Friction is required to keep global power in motion. It shows us where the rubber meets the road. (. . .) Roads create pathways that make motion easier and more efficient, but in doing so they limit where we go. The ease of travel they facilitate is also a structure of confinement. Friction inflects historical trajectories, enabling, excluding, and particularizing.” (p. 19).
Similarly, data-based technologies are viewed as easing our everyday lives. Especially in the public sector context, arguments of efficiency and effectiveness are often put forward as the main way to legitimize the introduction of digital infrastructure. Chatbots are an example, which supposedly allow for simple access to information and provide an additional pathway into public administration. At the same time, they direct and hence structure the ways in which citizens interact with public sector institutions.
Taking this as a starting point, the interaction between citizens and the state is always also characterized by frictions that emerge in the translation between the two parties. Paul Edwards argued that data frictions emerge with every movement of data across interfaces that involve time, energy, and attention (Edwards et al., 2011). Ruckenstein (2023) suggested that in designing digital tools, friction is considered an inconvenience that needs to be diminished as much as possible. Digital experiences and interactions are supposed to be smooth, seamless, and frictionless. However, interactions with digital machine are rarely frictionless. At the same time, friction is productive, as Tsing stated, “rubbing two sticks together produces heat and light; one stick alone is just a stick. As a metaphorical image, frictions remind us that heterogeneous and unequal encounters can lead to new arrangements of culture and power” (p. 18). From friction, something new and productive can emerge. This productive aspect of AI frictions is of particular interest here, and we ask how AI frictions emerging in connection with the introduction of public sector chatbots lead to new cultural arrangements. Tsing (2005) also emphasized that frictions are not about resistance and questioning hegemony per se. Hegemony, she argued, is produced and reproduced through frictions. Frictions do not automatically lead to the fundamental questioning of power arrangements but rather serve to reinforce and uphold them.
AI frictions serve as a sensitizing concept that guides the initial analysis of our material. We are on the outlook for frictions to destabilize and defamiliarize the seemingly well-known chatbot format. We are looking for the places and moments where frictions emerged but also how different actors involved strategically try to gloss over AI frictions.
Public sector chatbots as a form of datafication infrastructure
It is difficult to estimate the extent of chatbot use. However, a market research report approximated the global chatbot market size at 5132.8 million USD, with growth of more than 23% expected over the next seven years (Grand View Research, 2023). 3 As for the public sector in Sweden and Estonia, the two contexts explored here, many public agencies and, increasingly, municipalities, seeking to enhance interaction with citizens and residents, have introduced or are in the process of implementing chatbots. Furthermore, a 2020 report by AI Watch (Misuraca and van Noordt, 2020) noted that the dominant form of automated and AI-based systems deployed in the public sector are chatbots (compared to less prominent forms like automated profiling of citizens). Accordingly, public sector chatbots should be seen as an integral part of the data welfare state, or more specifically, an infrastructure enabling the datafication of interactions between public sector organizations and citizens.
The deployed chatbots vary in complexity and accessibility, from advanced models answering frequently asked questions to interactive, machine learning-based applications to live chats. According to Følstad and Bjerkreim-Hanssen (2023), the most common public sector chatbots are applications for service triaging to support users in navigating information, for example, by providing structured replies to frequently asked questions. In rare cases, interactive chatbots are even linked to live chat interfaces.
Chatbots have increasingly been implemented to allow citizens to interact with public sector institutions. Together with other data-based applications, they constitute an infrastructure of datafication and have become part of what Susan Leigh Star and Ruhleder (1996) referred to as an ecology of infrastructure, that is, technologies that are turning state–citizen relations into data points. Chatbots are part of the broader field of communicative technologies that have recently been conceptualized as communicative AI. During the last 20 years, numerous communicative technologies, programs, and devices that take the role of communicator by exchanging messages with people or by performing a communicative task on their behalf have been introduced by technology companies (Guzman, 2020: 37) and increasingly in the public sector (e.g. van Noordt and Misaruca, 2019). Communicative AI tools, which include conversational agents, social robots, and automated writing software, vary in how they function as communicators, from interpersonal interlocutors to content producers (Guzman and Lewis, 2020). Sociologists have conceptualized communicative AI in terms of machine habitus, which is socialized within machine learning (Airoldi, 2021), and as artificial communication, where AI is understood as a communicative interlocutor rather than an intelligent entity (Esposito, 2022). Similarly, Natale (2021) focused on the meaning-making around AI by users. AI is intelligent only as long as we believe it is intelligent and we are deceived by it.
In the public sector context, AI is anticipated to overcome four essential public service problems: a problem of control (e.g. ensuring compliance with rules), a problem of cost (e.g. how to meet demand with reduced funds), a problem of convenience (e.g. how to meet growing customer expectations), and a problem of connection (e.g. how to maintain trust and mutual empathy) (Jeffares, 2020: 10). Not all of these problems can be addressed by deploying chatbots. However, according to previous research, chatbots can reduce public sector organizations’ administrative burden and enhance and standardize state–citizen communication (van Noordt and Misuraca, 2019). Digitized self-services like chatbots, which are available 24 hours a day 365 days a year, can provide more convenient services for citizens when interacting with the state. Research on chatbot use in Norwegian welfare services has shown that they can handle peak demand periods, corresponding to the capacity of 220 service agents, with only one-fifth of inquiries transferred for a live chat with a human (Vassilakopoulou et al., 2023). Thus, amid labor shortages, chatbots can save staff time and help street-level bureaucrats allocate their time to more specific tasks that machines cannot handle.
Chatbots, as conversational agents, have the ability to engage in a conversation or interaction with humans through text or voice (Brandtzaeg and Følstad, 2018) through various methods, including natural language processing technology (NLP) (Shawar and Atwell, 2007), rule-based systems, or simple keyword recognition. Chatbots can be categorized in multiple ways using different parameters, such as the knowledge domain, the service provided, the goals, the input processing and response generation methods, the amount of human aid, and the build method (Adamopoulou and Moussiades, 2020: 377). Usually, however, they do not belong to one category exclusively.
Cortes-Cediel et al. (2023: 7) analyzed the literature on e-government chatbots, noting a trend in e-government chatbots shifting from basic keyword matching and rule-based methods to advanced, large-scale implementations using machine learning NLP models. These models often utilize technologies from companies like RASA, DialogFlow from Google, AzureBot from Microsoft, or Watson from IBM. Development platforms can be open source, such as RASA, or closed (Adamopoulou and Moussiades, 2020: 378). Adamopoulou and Moussiades (2020: 378) suggested that although open-source platforms give the designer the ability to intervene in most aspects of development, closed development platforms using proprietary code may be preferred by larger companies because of their state-of-art technologies. However, closed development platforms also lead to “black box” systems, where the knowledge based on the chatbot is not fully known.
The current body of knowledge on chatbots in the public sector is still limited. Previous studies have provided analytical evaluations (van Noordt and Misaruca, 2019) or user insight through methods like questionnaires or interviews (Abbas et al., 2023; Henk and Nilssen, 2021; Makasi et al., 2022; Tisland et al., 2022). Meanwhile, fewer studies have examined how users interact with chatbots, for example, based on the chat logs (e.g. Følstad and Bjerkreim-Hanssen, 2023; Simonsen et al., 2020; Verne et al., 2022). This is not surprising, considering that chatbots are mainly used for providing users answers to frequently asked questions, so authentication or user data collection are not needed. While there is some emerging research on the use and perception of public sector chatbots, the broader societal implications of their introduction as part of the data welfare state remain largely understudied. Before delving into the analysis of AI frictions in the implementation of public sector chatbots, we detail the methods and materials gathered for the case studies.
Approaching public sector chatbots: method and material
To explore the character and implications of public sector chatbots, we utilized a case study approach that comprises the analysis of publicly available documents, public records requests, interviews, and participant observations within different parts of organizations working with chatbot development and maintenance. The specific methods and data produced are detailed in Table 1. In the gathered material, we systematically identified actors involved in the implementation and development process as well as those maintaining the existing data infrastructure of chatbots. We also examined specific discourses around the chatbots that legitimize and motivate their deployment.
Overview of the materials and methods.
The two cases studied here were chosen based on the assumption that they both represent public sector chatbots with similar target groups (i.e. citizens and residents). They also share a similar goal to make public administration more accessible. The methodological approach was ethnographically inspired, aiming to gather as much relevant context information on the chatbots as possible (Karasti and Blomberg, 2018).
However, we also assumed important differences between the two welfare contexts of Estonia and Sweden, including different levels of ambition: The Estonian chatbot is presented as one unified national entry point to all public services in the country, independent of the specific domain. In the Swedish case, we were dealing with a municipal chatbot catering to residents in the specific municipality sharing information about the services provided there. Hence, the two cases of public sector chatbots represent two different ways of imagining automated interaction with citizens. The Estonian Bürokratt is part of a large-scale AI platform project that not only encompasses chatbot interaction but also other, broader applications of AI in the public sector. Both the chatbot and its link to other AI applications indicate an all-round approach that aims to provide adaptable general solutions across different domains of the public sector. In contrast, the Swedish Kringla represents a specialized chatbot adapted to the informational needs of one municipality. Hence, the two technical systems are both similar and different.
Public sector chatbots
Bürokratt—the all-round Estonian chatbot
Bürokratt – a universal chatbot for Estonia –, as a way to change and enhance citizens’ interactions with the state, was first referenced in the report of the Estonia AI taskforce. Bürokratt’s was inspired by the Aurora concept in Finland (AuroraAI Operating Model–Valtiovarainministeriö, n.d.). In the long term, the aim is to develop an interoperable network of public as well as private sector AI solutions (Bürokratt, n.d. a) that would act as “personalized virtual (autonomous) assistant for citizens” and have not just speech-related capabilities but also sign language capabilities to offer seamless interaction with the state. The chatbot, based on machine learning language models, is one of the central components of Bürokratt. Bürokratt is based on RASA, an open-source machine learning framework for automated text and voice-based conversations. Bürokratt has often been branded and introduced to the international public as the “Siri of public digital services” (Grzegorczyk, 2021) or “Siri on steroids” (Paraskevopoulos, 2022). Tracing the chatbot’s history means interacting with a plethora of actors: from municipality representatives to state agencies and national strategists. The chatbot does not have a clear home and is instead developed and implemented across different sites.
The funding, in total 53 million EUR, for Bürokratt comes from the European Commission REPowerEU Plan (Estonia’s recovery and resilience plan) and must meet its targets by 2026. Hence, the development of the Bürokratt virtual assistant should be concluded by the end of 2025. If the chatbot’s introduction is managed at the ministry level, then its development will be managed by the Republic of Estonia Information System Authority (hereafter RIA). The Bürokratt team (8 people) outsources all developments to smaller and bigger companies through procurements. The team is also responsible for information exchange and communication with the participating institutions and chatbot initial training. Altogether, there are 25 development partners and about 40 for whom the paperwork is in progress (Bürokratt, n.d. b). Bürokratt’s team also cooperates with Microsoft (CEE Multi-Country News Center, 2022), and based on the interviews with the project management, has been looking for ways to cooperate with Google and Open AI. The development of Bürokratt has followed open development model (Majandus-ja Kommunikatsiooniministeerium, 2022), meaning that all the code produced as well as the overall process is open to the public and all the development partners. Bürokratt, first piloted in 2021 in the first two institutions, was implemented near the end of 2023 in eight additional institutions, including one municipality.
In the current implementation, there is a separate chatbot for each institution, although they seem to be the same when looking at the interface. To allow the chatbots in different institutions to “communicate” with each other, Bürokratt’s classifier must be finalized and implemented. This development would change citizens’ interaction from one specific public sector organization to statewide interaction, as citizens could ask questions related to whichever public sector organization from whichever chatbot they are interacting with at particular moment to get an answer.
Bürokratt chatbot training requires several organizational steps. 7 Before Bürokratt is trained, the organization needs to analyze the information requests coming through e-mails or other main communication channels and compile the training questions and answers used to train the chatbot models. There should be at least 30 different variants of the same question (e.g. the order of words in the question may differ; they can include slang and have spelling mistakes) asked by a citizen for the model to better identify the question and provide the correct answer. RIA advises organizations to limit the answers to 450 words and avoid formal language and links in answering. This is considered important “to preserve natural conversation”. The initial training is done and supported by the RIA chatbot trainers. After the chatbot is live on the organization’s web page and new topics and questions emerge through communication, the training is mainly done on the training interface by the organization representatives. In some organizations, this is done on a weekly basis, while other interviewed organizations do so less often.
The chatbot, usually already visible on the institution's landing page, does not open for interaction automatically and must be opened by the citizen. However, the chatbot can also only appear on specific subpages, for example, in the way the Viimsi local government used it (see Figure 1). As their chatbot was not ready to answer inquiries other than those related to children’s benefits and services, they decided to only make it visible on the children’s benefits subpage. However, this has also meant that there has not much interaction with citizens, who have difficulty finding it.
Bürokratt on the Viimsi local government subpage related to child benefits and services, Available at: https://www.viimsivald.ee/teenused/haridus-ja-noorsootoo/lasteaiad/koduse-lapse-toetus
The chatbot identifies a specific keyword that matches a topic in its database (same question written in at least 15 different ways) and gives the citizen a prewritten answer. It can encounter difficulty when sentences are too long or include keywords for several different topics. Thus, the current version of Bürokratt is mainly able to answer frequently asked questions about particular organizations and their services and provide weather information about Estonia (through weather API). As all the greetings can be personalized by the organizations themselves, the way Bürokratt represents itself varies. It introduces itself as a “digital state virtual assistant” (see Figure 2), as a “chatbot,” or simply by saying “hi.” (see Figure 2) Organizations have autonomy in choosing how it ends interactions as well based on predefined possibilities (i.e. fallback answers like asking if the citizen prefers to continue the conversation with customer support or leave the conversation). The chatbot window also has a separate section for the information related to the “terms of use” and information about funding. The use of “terms of service” is usually associated with private sector software and could be considered to redefine the way in which citizens communicate.
Chat window (Bürokratt on Estonian Statistics webpage), Available at: https://www.stat.ee/.
The aim is not just to offer an anonymized communication channel but also to give citizens the possibility to receive personalized information about specific services. For this purpose, citizens can authenticate themselves and submit personalized inquiries (i.e. ask if some of their documents are expiring). This aligns with Estonia’s larger aim to develop a personal digital state 8 characterized by daily interaction between the state and citizens seamlessly and efficiently through different digital channels (including a mobile phone app).
Kringla—specialized Swedish municipal chatbot
The second case explored here is the municipal chatbot Kringla, implemented by Södertälje municipality.
The fieldwork around this chatbot has been concentrated on the municipal main building, where most of the administrative units are located. Besides interviews with the group developing and maintaining the chatbot, we also shadowed one of the employees in the call center for one of their scheduled sessions. The municipality prides itself that the building, which was opened in 2008, is one of the first to be built after the accessibility law was implemented and hence follows the principles of accessibility as well as transparency, featuring large open areas and glass elements. The communication department that is responsible for the chatbot operates in a flexible, open office with different kinds of workstations and areas, including cubicles for video meetings, call center areas, and quiet zones. Many of the architectural values are mirrored in the features of the chatbot, which is intended to enhance transparency and accessibility for citizens.
The chatbot is the second of its kind used by this municipality. Previously, Södertälje worked with Kommun-Kim, an application based on the boostAI platform solution that has previously been used in several Norwegian municipalities (Følstad and Bjerkreim-Hanssen, 2023). However, after the five-year contract lapsed the municipality decided to contract another platform following a public procurement process. The service has been provided by Kindly since 2023. This shift was motivated by the greater freedom it gives the municipality to change and add information, or in their words, to train the AI. As one of the interviewees put it, “before we were sending files to the chatbot company that employed AI trainers. Today we are AI trainers ourselves and can add whatever information that is needed” (Interviewees 1 & 2).
As illustrated in Figure 3, the chatbot presents itself as an AI chatbot. In practice, prompting the chatbot returns responses that resemble answers to frequently asked questions listed on the website as well as the internal database that public information officers use during live interactions at the call center or visitor center in the municipal hall. The chatbot does not involve any live interaction with service staff. It has been integrated into the routine communication work of the information department, which includes a reception desk at the town hall, a call center, an email service, and the chatbot. The information department is a service unit that caters to other municipal units and has both communication and administrative responsibilities, serving as “the pathway into the municipality” (Interviewee 2). There is a specific specialist group that works on maintaining and improving the chatbot approximately two hours per week.
Interface of the Södertälje municipality’s chatbot on the landing page, Available at: https://www.sodertalje.se/.
The chatbot interactions are based on matching users’ queries with a database of potential, pre-programmed answers. NLP comes in on the user side, where the prompts are processed and linked to predefined answers. The answers that are returned to users have been formulated and formatted (sometimes including emojis) by municipal service workers. The training of the AI consists of assembling, updating, and maintaining the answer database (by the employees). When we were conducting the interviews, the Kindly application used by Södertälje municipality was not integrated with ChatGPT for the assessment and interpretation of user prompts. This might change, however, since Kindly is collaborating with OpenAI and increasingly integrating ChatGPT capabilities into its products.
In many cases, the chatbot returns links to relevant sections of the municipality’s website as well as related forms, for example, in the case of applying for school placement. However, the chatbot does not provide personalized service, such as through logging in or linking to digital ID solutions. After an interaction concludes, the user is able to download the conversation (Figure 4). These logs are deleted after a set period, but the unit maintains weekly and monthly reports of interactions, including the number of interactions and character/area of questions (Figure 5).
Downloaded conversation with the Södertälje chatbot Kringla, Available at: https://www.sodertalje.se/.
Illustration of the elements of Kindly’s AI chatbot, Available at https://www.kindly.ai/.
AI frictions in public sector chatbots
Expectations of AI
Kringla, the Swedish municipal chatbot, is framed as an AI chatbot, whereas Bürokratt is introduced as a virtual assistant. In both cases, the descriptions raise certain expectations among users in terms of complexity and level of interactivity. The Södertälje municipality relies on an off-the-shelf solution provided by Kindly—a Norwegian company that builds AI and NLP solutions that mainly cater to commercial businesses, including e-commerce, travel, logistics, and finance businesses. It presents its chatbot solution as a conversational AI that moves beyond static chatbots that simply connect keywords with preset answers.
In the Swedish case, the replies to prompts are drawn from a preset database of potential answers and often feature links to the municipal website, including further information and forms. The Swedish chatbot currently does not offer personalized service (e.g. log in and digital identification) or real-time interaction with a service worker. The current version of Bürokratt resembles Kringla in this sense. It is mainly capable of answering frequently asked questions, but it is also connected to the state’s weather API and can thus answer weather-related questions. Differently from Kringla, in most cases Bürokratt allows real-time interaction with a service worker when it is unable to answer the questions by itself. Hence, while the chatbots are presented as complex tools of the latest AI and NLP solutions, the actual user experience does not necessarily surpass the use of an interactive FAQ list. However, both the Swedish municipality and Estonian national public agencies position themselves in the AI ecology and emphasize the technologically enhanced future orientation of the organizations.
Further expectations emerged from the organizations’ own previous experiences with chatbots. One of the interviewed public sector organizations in Estonia, Estonian Statistics, experimented with its own chatbot named Iti before Bürokratt. As the chatbot provider decided to discontinue development, the company did not see any reason to choose a new provider if the state itself had started developing Bürokratt. Compared to other organizations, Estonian Statistics had much broader experiences in the development of chatbots. At the same time, it was not in the lead of the Bürokratt development process, which created specific frictions between units. While previous experiences helped in the implementation process, they also created specific expectations about the possibilities and capabilities, both from the organizations as well as from the citizens’ side. As the interviewed specialist expressed, the company felt distraught by the slow development process and loss of interaction through the chatbot with citizens after implementing Bürokratt.
More generally, the two public sector chatbots have been implemented with a vision and expectation of 24/7 remote access to public services, as underlined by our Swedish informant:
“The reason why we wanted a chatbot was mainly to be open and accessible for our citizens. That is the reason why. It is not to reduce caseworkers at all or to replace our staff in any kind of way. It is about openness and access. We have telephone hours and visiting hours 8–17, Monday to Friday, or rather Monday to Thursday. On Fridays we close already at 15.00. And not everybody can come to the municipal hall during these hours, or questions emerge at another point in time. So, it is mainly access, but also to unburden the staff from general questions that are easily answered.” (Interviewee 2—Södertälje municipality)
This research participant repeatedly emphasized the focus on access rather than replacement of staff to mitigate AI frictions that have been discussed publicly, specifically the reduction in the number of case workers due to the help of automation technology. Instead, she highlighted the additional service that would be gained with the chatbot, explicitly avoiding critical aspects related to reduced resources or enhanced efficiency goals.
In the Estonian case, the availability of the chatbot is currently linked to the office hours of public information officers. In the Swedish case, the chatbot is available 24/7, and the unit manager proudly noted the increased use of the chatbot outside of office hours, which now amounts to about 30% of all chatbot interactions. There are also frictions emerging between promises and the actual use of the implemented chatbots. In both cases, the use remains comparatively low. In the case of the Swedish municipality, there are approximately 200 chatbot interactions registered per month, compared to 1800 emails and 7500 telephone calls during the same period. In Estonia, the use of Bürokratt differs considerably depending on the specific public sector institution. In the interviewed local government agency, there has been minimal interaction through the chatbot so far compared to other public sector institutions, where approximately one-third of the daily requests can be through chatbot interactions.
Although the Bürokratt interfaces are the same, the interaction itself can differ in several aspects. During the implementation, as the interviewees noted, the public sector organization needs to gather all the data and enter it into the system. Hence, all the chatbots are not actually replicas of each other. They may all say “common” things like “hi,” but their style in answering questions and fallback messages (e.g. suggesting talking with case workers) can be different and are highly reflective of the values of the specific organization. As the one public sector representative expressed, in their case, when questions involve obscenities, they are hidden or replaced for the information worker, and specific ironic answers are given. In that sense, the lived experience of users might vary considerably depending on how the organization chooses to pre-set the chatbot responses, and the chatbots’ different roles depend on preset scripts.
“Yes, we are a state institution, and we are not making any kind of cuckoo here and we are still polite. But we can joke a little, just like in a live conversation. That it doesn't have to be a big state institution from top to bottom like that. I come to that level (citizen level) and we can, we can also joke.” (Interviewee 6—Estonia Statistics)
The imagined possibilities of public sector chatbots are largely based on applications that are already common in the private sector. The chatbots are designed to serve as “public sector assistants and servants” and a “public sector Siri,” as one of our Swedish interviewees expressed it, to “give citizens help faster” (interview Södertälje municipality). For their part, citizens adopt the role of users who follow learned scripts of how to interact and communicate with particular systems.
Imaginaries regarding the interoperable work of chatbots are fostering the idea of their generalized entry into public administration. The Bürokratt is imagined as performing the “networking” between different public sector entities on behalf of citizens (i.e. a citizen goes to the local government webpage and asks how to get children’s benefits, and the chatbots start to “talk” with each other, and based on the interaction between different databases, provide the citizens with the right response). From the citizen’s perspective, the chatbot does this work “behind the curtain” as a consultant and then proposes solutions for the client. The difficulty is in the “data flow” between institutions, as all the data currently produced through interactions with chatbots (which could also be used in future chats with digitally authenticated citizens) must be securely kept on the public institution’s own servers or in the Estonian Government Cloud, separately from each other. Thus, while the aim of the chatbot is to provide “seamless” interaction with the state, the responsibility for this is always on specific institutions in connection to the data processing. An interoperable network of chatbots aims to shift the burden from the citizen to the state based on the keywords assigned to the questions. This assumes that the service provision and responsibilities are clearly divided beforehand, however, which can, in real life, be quite complicated, as one of the Estonian interviewees expressed. Furthermore, it assumes citizens’ requests in prompts are clear enough and contain enough domain-specific information for the machine to select the correct recipient of the question. As previous scholarly work has shown, this is not always the case (Simonsen et al., 2020; Verne et al., 2022).
While citizens are increasingly experienced chatbot users, every institutional chatbot provides a unique experience. This is because they are based on slightly different training datasets, even though the underlying software infrastructure is the same. At the same time, there are anticipations and expectations related to previous interactions with other, perhaps more advanced chatbot solutions as well as mediated depictions of chatbots that rarely match interactions in reality. Accordingly, there are increasing attempts to develop strategies for further standardizing chatbot interactions across public sector organizations.
Frictions between organizational logics and between systems
A second aspect that potentially introduces AI frictions in the mediation of state–citizen relations through chatbots is friction between organizational logics as well as between systems. The implementation of chatbots is primarily project based. In the Estonian case, for example, all the developments are outsourced to development partners through both national and international procurements. To keep all the partners up to date, an open development methodology was chosen. In practice, this means that much of the documentation and code is publicly accessible through GitHub. It is financed by a specific fund from the European Union, and it is unclear what happens after this funding period is lapsing. Similarly, in the Swedish case, the development is linked to cycles of public procurement that allow for contracts of up to five years. This cyclical character leads to changes in service provision of the applications as well as in the specific people working on the chatbot development. In the Swedish case, the new chatbot was introduced fairly recently and required new input data and training by the public information officers working with the application. Although the retraining of the new chatbot required a considerable amount of time, our research participants repeatedly emphasized that this does not cause immense frictions. Rather, the opposite is true. It is part of their normal work routine to interact with new types of digital infrastructure, and the new system promises so much more functionality:
“Previously we would send files to the chatbot company who had AI-trainers. Today we could say that we are the AI trainers ourselves. We can add any kind of information that we want, but we stick to the most general level still. So it is mainly general questions, and we have not activated personalized service with the electronic ID, for example. But the chatbot would allow us to be very, very specific” (Interviewee 3—Södertälje municipality).
The previously mentioned example describing the implementation of Bürokratt by Estonian Statistics also highlights the important problem arising from situations where private vendors are chosen for chatbot development. A change in chatbot provider disrupts the process of implementing new communication channels. Channels that were previously successful and widely used by citizens can become less used due to the new interface and interaction capabilities.
The development of chatbots is also related to other, existing systems and infrastructure. In Estonia, all inhouse data produced through interactions with the chatbot are stored on the organization’s servers or the Estonian Government Cloud, which is available for paying public state institutions. In the future, an interoperable chatbot network is imagined, which would be linked to other projects like M-State or life-event services development, intending to change state–citizen interaction and service provision in other ways. In addition, other, not specifically related systems become part of the larger infrastructure if the chatbot’s own systems create friction. Several interviewed client service workers from different Estonian public sector institutions described the need to use Teams chat to manage citizen inquiries, as they had experienced problems with chat sound notifications. Using Teams, they sought to make sure that the client service worker responsible for answering inquiries noticed the chat even if Bürokratt did not make a sound. Thus, glitches in the original systems can lead workers to look for workarounds or adopt practices outside of the chatbot’s own infrastructure to make it work successfully.
In the Swedish case, the chatbot is one “path into the municipality,” but there are others, including a call center, a visitor center, and email. Here, the chatbot contributes to the communicative assemblage citizens and information officers must navigate and maintain. In addition, the introduction of efficient communication channels with citizens has led to a rearrangement and assignment of new organizational tasks to the information center in the Swedish case. While previously being primarily responsible for citizen contacts, the unit is now increasingly taking over administrative service tasks of other units within the municipality, including archiving and record handling. Hence, the specific work assignments for the public information officers have been broadened while also being standardized to improve efficiency and effectiveness in communication efforts.
Accordingly, the development of chatbots is embedded in a broader data and organizational infrastructure, or as Leigh Star and Ruhleder suggested, “the ecology of infrastructure” linking different systems together while not always being compatible. Focusing on emerging AI frictions highlights the embeddedness of specific AI solutions, here chatbots, into a broad network and organizational infrastructure of datafication that do not necessarily share the same logics.
Frictions of values
There are a plethora of actors and professions involved in the chatbot development, including contracted companies for software solutions, digital consultants, case workers, and public information officers (often without specialized training in public administration). All of them have different professional values to the development projects, potentially leading to AI frictions. Among our interviewees, for example, the aim to move forward and innovate was seen as standing in conflict with maintaining the institutions’ administrative routines. This also includes the push toward standardization across domains and citizen–state interactions. Often, a standardized entry point into the public sector and public administration is the larger aim of the chatbot introduction. In the case of Södertälje, the communication department responsible for maintaining the chatbot not only had to take care of the technical training but also convince other parts of the municipality of the advantages of the chatbot. This resulted in a specialized group responsible for chatbot maintenance that also acts as what could be called digital ambassadors within the organization. The head of the unit described this role in the following way:
“We have adjusted our way to work also more generally within the contact center during the last couple of years. We have introduced so-called specialist groups instead. They have a bit more administrative time instead of meeting citizens in the visitor center or responding to calls at the call center. [. . .] And we are working as a service unit for other parts of the municipality. So, we are taking over administrative tasks from them. So one could say that we are assigned tasks by other units to unload some work from them [. . .].” (Interviewee 2—Södertälje municipality)
There are also AI frictions emerging in relation to the imagined users of chatbots. The current imagined user of chatbots is already a digitally literate citizen. In the Estonian case, where the chatbot is still under development, the imagined future capabilities of the chatbot include allowing people with disabilities to use the system. Indeed, sign-language, voice recognition, and machine translation capabilities are explicitly written into planning documents and have been expressed by the Bürokratt’s development team leader.
“The vision is that you should also be able to convey the voice. Well, we also must think about different groups of people, who also have a disability or something. [. . .] We want Bürokratt have voice to voice capability and in the same way we can also speak, then voice is converted to texts, a person can actually communicate with this Bürokratt by voice even in the everyday language of the conversation, then the text will arrive still in the form of a Bürokratt’s text. In the same way, sometime in the distant future, we will also think about having sign language. Maybe create an avatar that communicates in sign language.” (Interviewee 2—Bürokratt management)
These visions of chatbots, when realized, can lead to productive frictions since they may not just improve the inclusivity and accessibility of public services but also lead to more versatile chatbot systems in the public sector.
The Swedish municipality is planning an information campaign to inform residents about the chatbot and increase user numbers. Generally, the chatbot and other projects that are part of the overall digitalization strategy 9 of the municipality suggest that the implementation of AI figures into public relations campaigns and branding attempts. Here, the Södertälje municipality has been particularly ambitious, launching several Internet of Things and open data initiatives, including live updates on lake temperatures in the municipalities and live tracking of bike traffic across one of the bridges. The contribution to the positive framing and branding of the municipality as future oriented and heavily investing in smart technology is not only a positive side effect but potentially also the main outcome of the projects.
In the Estonian case, Bürokratt is well branded outside of Estonia and has already won several prizes for its brand design or vision10, 11 and been selected by the IRCAI as one of the top 100 projects solving problems related to the 17 United Nations Sustainable Development Goals. 12 However, it has not yet really been introduced to the residents of Estonia. As the development is still ongoing and the chatbot does not resemble the visions in the strategy documents, the development team has been hesitant to launch any public campaigns. Another, more current reasoning relates to the possible expectations related to the widespread use of ChatGPT in other life fields—introducing a chatbot in the current stage could lead to disappointment on the part of citizens.
While the chatbot projects are presented along with ambitious future promises of easy and inclusive access to public services, there are often underlying values that drive investments and shifts in resources dedicated to the development of datafication infrastructure (Jakobsson et al., 2022). There are important AI frictions between motivations to introduce AI chatbots, including efficiency and accessibility, and values of the welfare state, such as universality, equality, and decommodification. Our informants play an important role in negotiating and mitigating these AI frictions related to values by either emphasizing the overall benefits for the organization or focusing on the future potential that might be reached one day.
Conclusion: AI frictions in the datafication infrastructure of the welfare state
In this article, we have employed AI frictions as a sensitizing concept to explore the implementation of public sector chatbots in a Swedish municipality and in Estonian public agencies. The identified areas of AI frictions concern frictions in expectations of AI, frictions in organizational logics and between systems, as well as frictions in values. The findings highlight the specific ways in which our research participants, who have worked with the implementation, have emerged as intermediaries with the important task of moderating AI frictions and easing the experience both within the organization and for the citizens interacting with the chatbots. The AI frictions that we have encountered contribute—while constituting the messy reality of AI implementations in the public sector—a pre-condition for the AI as a overarching social and political project to emerge. Like globalization as a process in the late 1990s and early 2000s emerged as a coherent project out of a messy entanglement of the local, national, and transnational encompassing contradictions, frictions, and conflicts. As we have shown, much additional work goes into balancing local frictions within the municipality to involve all employees in the process. Our research findings imply that rather than contradicting the implementation of AI and algorithms as a coherent process, AI frictions are an integral part of the project of producing AI as thing itself and should hence be at the center of investigations in critical algorithm and AI studies.
While the aim of the Estonian chatbot project is to implement an all-encompassing solution for all levels of the administration, the Swedish public sector chatbot development remains fractured, with several independent initiatives on different levels using different off-the-shelf solutions. In Estonia, the specific outlook of the chatbot project is related to the country’s small size and high level of digitalization, which foster a centralized approach to technological innovation in the public sector. In Sweden, the municipalities are procuring software solutions individually and independently. The procurement process reflects the different needs of the municipalities depending on size and budget. Hence, the implementation is characterized by a decentralized approach. These different contexts for chatbot development lead to different ways in which AI frictions play out. In the Estonian case, for example, organizational frictions have emerged between different public sector actors that are developing the chatbot, which is happening across different levels of the backend of the data welfare state. In the Swedish context, the chatbot is an important branding tool for a municipality struggling with its image, and frictions have emerged around procurement processes within the municipality. In both the Estonian and Swedish cases, the mitigation of AI frictions has become an important cornerstone of the implementation along with maintenance work on the chatbots. Public sector chatbots represent one specific form of datafication infrastructure within the welfare state. Such chatbots make state–citizen interactions measurable and open for analysis on a new scale. Although both the Swedish municipality and the national Estonian project are yet to make use of the data generated beyond compiling basic statistics on the kinds of requests and peak interaction times, the potential of systematically being able to monitor citizen requests is evident.
Approaching AI implementation projects like public sector chatbots from the perspective of AI frictions makes it possible to challenge the idea of AI as a coherent entity while highlighting its shaky base. In fact, the analysis of AI frictions reveals ambiguities and contradictions in the development process but also creative and productive practices for negotiating and mitigating frictions by individual employees as well as on an organizational level.
Future research should take seriously how citizens and clients relate to public sector chatbots and explore for example in how far expectations differ from commercial context. A longitudinal study should also explore how the adaptation changes over time including shifts in trust in the municipality as a public entity. In combination with case studies of specific chatbot application across domains, citizens’ perspectives could provide important insights how state-citizen relations shift with new forms of mediation such as public sector chatbots and what the implications for social trust as well as civic participation and engagement are. Comparative studies across domains but also across countries remain rare and would allow for more nuanced analyses of the above discussed aspects.
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Project AUTO-WELF is supported by Austrian Science Fund: [I 6075-G], Austria; Independent Research Fund, Denmark; BMBF, Germany; National Science Centre, Poland (grant no.2021/03/Y/HS5/00263); FORTE, Sweden, under CHANSE ERA-NET Co-fund programme, which has received funding from the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no 101004509.
