Abstract
Generative AI requires us to rethink what constitutes technology, how technology is socially constructed, and how technology is used. Our common understanding of technology is mainly derived from designed technology. Generative AI, in contrast, is a learned technology and, moreover, a technology that mainly relies on unsupervised learning. The paper argues that this has far-reaching consequences with respect to the task-relatedness of generative AI systems, the user interaction with generative AI systems, and the agency of those systems. Designed technological behavior is usually designed with respect to particular tasks. AI systems based on unsupervised learning, in contrast, are not task-related is such a manner. Consequently, generative AI systems cannot be operated in the same way as designed technology. Rather than instructing them, operating them requires what can be described as strategic interaction. Interacting with generative AI systems leads to new actor roles and role relationships. The agency of technological artifacts that are designed and used for particular tasks tends to be the agency of a tool. In contrast, the new agency of generative AI systems lies in their capability to mobilize machine-learned versions of human experiential knowledge and thereby to become in some respects similar to a human interaction partner.
Keywords
Introduction
In this paper, I argue that generative AI is different from designed technology in three main respects: task-relatedness/toolness, user interaction, and agency. Designed technology is usually developed and put to use by establishing the artifacts as predefined means for particular ends and thus as tools. This mode of establishing technological artifacts requires the artifacts’ features to be designed with respect to their task-related roles. Generative AI systems, in contrast, acquire their features by learning. To the extent that they are trained based on unsupervised learning, neither designers nor users initially know which features the systems possess and for which purposes they may turn out to be useful. In their pre-trained state, generative AI models are “task-agnostic” and lack the toolness of designed technology (Section, The task-relatedness of technological design and the task-agnosticism of unsupervised learning).
Technological artifacts are commonly used by operating and instructing them. Both activities presuppose that, to some extent, it is predefined which parts of the sequence of events that brings about the intended result are to be performed by the artifacts and which parts the users or other parties involved have to contribute. Operating a technological artifact means that the users have to perform their part of the task, and thereby enable the artifacts to play their part. Instructing refers to triggering the technology's contributions to the task by giving the related orders. Using generative AI systems, in contrast, requires interacting with them, it involves finding out which text input (prompt) elicits which response from the system. Getting useful responses often requires to interact with generative AI systems, for example, by asking follow-up questions or giving additional information in response to unsatisfactory previous output. Prompt engineering, which is essentially about turning these interactions into strategic interactions, has become a major source for dealing with the fact that generative AI systems cannot be instructed in the same way as designed technology (Section, Interacting with generative AI as the way of using it).
The roles humans and artifacts assume within the sociotechnical constellations (networks, assemblages) they are involved in defines their agency towards each other. In the case of technological artifacts, an important part of this role-related agency is about how the artifacts’ functional features make a difference. With designed technology, the resulting agency of the artifacts is related to their designed features and to how these features become meaningful within the artifacts’ contexts of use. To the extent that the designed features are used as predefined contributions to particular tasks, the resulting agency thus is the agency of an artifact in the role of a tool. The agency of generative AI, in contrast, results from internal representations that contain regularities and patterns these systems have learned from products of human knowledge. Being pre-trained with large amounts and a wide variety of products of human knowledge, generative AI models can be turned by fine-tuning or by prompt engineering into an unforeseen number of different experts. In a way, generative AI systems act based on machine-learned versions of human experiential knowledge. They assume the roles assigned to them by referring to knowledge that is similar to some of the experiential knowledge humans would refer to in the respective roles. This is what constitutes the new agency of generative AI systems (Section, The agency of generative AI).
Since the behavior of learned systems is not designed but only indirectly reflects design choices, it is much more difficult to analyze how the new sociotechnical constellations that include such systems are socially constructed. Studying similarities and differences between the knowledge that is employed to prompt and interpret the responses of generative AI systems on the one hand and the machine-learned knowledge as it is reflected in these responses on the other hand may be a promising way to analyze the new inter-agent relationships between generative AI systems and their human counterparts (Section, Conclusion).
The task-relatedness of technological design and the task-agnosticism of unsupervised learning
There is a fundamental difference between technologies whose features and functions are designed and technologies who acquire their capabilities by unsupervised learning. As I will argue, AI systems based on unsupervised learning are the first technology to substantially break with the task-relatedness and thus with the toolness of technology. To substantiate this argument, I will first discuss the task-relatedness of designed technology (Section, Task-relatedness and toolness as characteristics of designed technology). Second, I shortly describe the main different forms of supervised and unsupervised learning and how generative AI systems are based on unsupervised learning (Section, How generative AI is based on unsupervised deep learning). Finally, I show how generative AI systems differ from designed technology with respect to task-relatedness and toolness (Section, The lack of toolness resulting from unsupervised learning).
Task-relatedness and toolness as characteristics of designed technology
Most of the technological artifacts that surround us possess features and functions that have been designed in order to support their users in particular ways. As Akrich (1992: 207–208) puts it, developers design new technology by envisioning the role they shall play in the future within particular contexts of use. The features and functions of the artifacts, the developers come up with, are their suggestions as to how the artifacts should support their future users in performing particular tasks. Another way to put this is to say that designing technology means to try to establish technological artifacts as predefined means for particular ends and thus as tools (Schulz-Schaeffer, 2023).
The notion of technological artifacts as tools for instrumental action has been criticized early in the history of science and technology studies (STS). One main argument has been that the artifacts’ functions as implemented by their designers do not determine the artifacts’ use because there is interpretative flexibility of how to use them and for which purposes. Thus, only after a particular interpretation has been established as to which problems an artifact is the solution does that artifact turn into a working technology (Pinch and Bijker, 1984). In a similar vein, actor-network theory has argued that the properties of all human and non-human actors of an actor-network are a result of how they mutually “translate” each other by inscribing particular properties into each other and prescribing particular behaviors (Latour, 1988). According to these considerations, which are well supported by empirical findings, technology's role is defined neither by the instrumental relations as embodied in the artifacts’ technological functions nor by any predetermined difference between human actors and technological artifacts. For these (and other) reasons, many STS scholars have abandoned the idea of technology as a tool (cf. e.g. Schatzberg, 2018: 232–235).
However, the picture changes if the unit of analysis is not just the artifacts involved but the sociotechnical constellation (network, assemblage). Establishing a technological artifact as the solution to a particular problem means that a specific role becomes assigned to it within the resulting sociotechnical constellation (Schulz-Schaeffer, 2021: 79–85). This is usually the role of being instrumental in supporting other entities within this constellation to solve specific problems, that is, the role of a tool (Schulz-Schaeffer, 2023). Many of Latour's (1988, 1991, 1992) examples, such as the door-closer, the Berlin Key, or the seat-belt, illustrate this clearly: Their agency lies in their ability to take over a task or a part of a task within a particular sociotechnical constellation, thus assuming the role of a tool. 1
The role of a technological artifact as a tool emerges in relation to particular sequences of activities that serve to achieve particular desired results. A part of such a sequence of activities becomes translated into an algorithm (that is, into a procedural rule that can be mechanically or digitally inscribed in artifacts, cf. Schulz-Schaeffer and Rammert, 2023: 42). This part is delegated to the artifact while the users (and the other entities involved) adopt the complementary roles. It is a fundamental feature of designed technology to provide output that is predefined in such a way and usually predefined with respect to specific tasks. AI based on deep learning deviates from this pattern but only AI based on unsupervised learning breaks with it.
How generative AI is based on unsupervised deep learning
The term deep learning designates a class of machine learning techniques that are based on multi-layered artificial neural networks, complex algorithmic structures consisting of connected layers of artificial neurons that operate on adjustable parameters. The large number of parameters of deep neural networks allows these systems to store large amounts of information. Deep neural networks acquire this information by being trained with training data. There are basically two ways of training, by supervised or unsupervised learning.
The training is called supervised if the learning system is provided with information about what to learn. The two most important forms of supervised learning are learning based on labeled training data and reinforcement learning from human feedback (RLHF). With labeled data, the labels contain the information about what is to be learned. For instance, if a system shall learn to categorize certain objects from images these images would be labeled with the relevant categories (such as cat, dog, horse, etc. if the objective is learning to categorize animals). The training is a trial-and-error learning controlled by the intended learning result as represented by the labeled data. The system learns by adjusting its internal parameters (LeCun et al., 2015: 436). If successfully trained, the system will be able to categorize new data correctly according to the learned categories (Murphy, 2022: 1–14). RLHF serves to improve the output of deep neural networks that have already been trained to some extent. Here, a reward function is used to teach the system the desired output behavior where the reward function is derived from information about what counts as more or less desirable output, information which is provided by humans (Ouyang et al., 2022: 27731, 27733).
In contrast to supervised learning, “unsupervised learning algorithms seek to find structure in data without recourse to labels (as in supervised learning) or reward signals (as in reinforcement learning)” (Watson, 2023: 28). There are different unsupervised learning methods: clustering methods are “grouping data-points into different clusters” (Tyagi et al., 2022: 33) in order to discover similarities in the data; density estimation methods are determining the probability distribution in data; and dimensional reduction methods allow “to reduce the least impact features within a dataset” (Tyagi et al., 2022: 42) and thus to reduce the noise in data. With generative AI, self-supervised learning, another form of unsupervised learning, has become important. Self-supervised learning methods “leverage the data's inherent co-occurrence relationships as the self-supervision” (Liu et al., 2023b: 858). For instance, a system can learn the co-occurrence of words in sentences in a self-supervised manner by masking words in sentences from its training data and by training itself to predict the masked words, using the knowledge about the masked words for evaluating and improving its own performance. With such strategies to “[o]btain ‘labels’ from the data itself” and to “[p]redict part of the data from other parts” (Liu et al., 2023b: 857) self-supervised learning derives the information used for supervising the learning process from structural features of the data itself. Self-supervised learning provides deep learning algorithms with the predictive capabilities which are the basis of generative AI.
Generative AI systems are systems that “can create new data samples based on learned patterns” (Feuerriegel et al., 2024: 112). ChatGPT represents a special type of generative AI called large language models (LLMs). Language models are “statistical models that describe the probability distribution of natural language” which allows them to “compute the probability of a given sentence […] or the probability of generating other contents given a part of the sentence” (Wu et al., 2023: 1024). Their ability to generate new text basically relies on “predicting the next possible word of a sentence” (Wu et al., 2023: 1129; cf. Yenduri et al., 2023: 7).
ChatGPT is a version of the underlying GPT (Generative Pre-trained Transformer) models (currently the version GPT 4.0) that is fine-tuned for conversational interaction. The underlying GPT models are trained using self-supervised learning as the main strategy (Wu et al., 2023: 1124–1126; Yenduri et al., 2023: 2). To adapt the GPT models to conversational interaction—for example to make the responses more polite and helpful and to avert “unintended behaviors such as making up facts, generating biased or toxic text, or simply not following user instructions” (Ouyang et al., 2022: 27730)—ChatGPT is fine-tuned using RLHF. Another objective of fine-tuning the GPT models is to improve their performance on specific tasks such as writing summaries.
Training the GPT or other LLM models using self-supervised learning and additional unsupervised learning methods is called “pre-training” though it actually “constitutes the main training (understood as the process by which one improves or acquires new capacities)” (Barandiaran and Almendros, 2024: 13). Most current generative AI systems and not only those based on LLM models are trained by such a combination of unsupervised pre-training and supervised fine-tuning. Bommasani et al. (2022: 1) call the pre-trained models of these systems “foundation models to underscore their critically central yet incomplete character.”
The lack of toolness resulting from unsupervised learning
Deep learning algorithms are “
Many of the problems of deep learning systems that are at the heart of critical algorithm studies and AI ethics are independent of whether the systems are trained in a supervised or an unsupervised manner. Major problems are the problem of biased learning and resulting discrimination (Barocas and Selbst, 2016; Kitchin, 2017) and the problem of the specific kind of opacity of deep learning algorithms (Burrell, 2016) that makes it very different to understand the algorithms’ internal reasoning and to arrive at responsible AI use (Taylor, 2025). However, supervised deep learning algorithms are still similar to designed technology in one respect: if trained correctly, they acquire task-related capabilities. They are not as far outside of the instrumental remit as unsupervised deep learning algorithms are.
Supervised learning is task-related learning. The tasks to be learned are defined by humans via labeled data or reward functions. Unsupervised learning, in contrast, is task-agnostic learning. The foundation models of generative AI systems “are trained on raw data that is typically extremely diverse and task-agnostic” (Bommasani et al., 2022: 122). Thus, with unsupervised learning, the systems learn whatever regularities and patterns they identify in the training data (Alpaydin, 2016: 111). Whether or not the learned patterns help to accomplish any tasks depends on what the systems have learned. With unsupervised learning this cannot be controlled directly. 2 It is also “very hard to evaluate the quality of the output of an unsupervised learning method, because there is no ground truth to compare to” (Murphy, 2022: 16).
It is this difference regarding task-relatedness that distinguishes unsupervised deep learning from designed technology as well as from deep learning based on labeled training data. Designed technology as we know it is usually task-specific to some extent. The names of many technologies indicate their task-specificity: sewing machine, word processor, enterprise resource planning system, etc. Supervised deep learning still shares this characteristic while unsupervised deep learning is different in this respect.
Not being task-specific in the same way, however, also means that the actual or potential usefulness of unsupervised deep learning systems is not limited in the same way to supporting specific predefined tasks. Unsupervised deep learning opens up the possibility to train the systems on a wide variety of knowledge sources so that they become potentially useful for a wide variety of intelligent tasks. This is what actually happens with generative AI systems like ChatGPT, which has been characterized as “one of the first set of programmes that are tending towards AGI [Artificial General Intelligence, author's note] […] as they have a wide range of seemingly intelligent capabilities that may not exceed expert human levels at individual tasks, but are overwhelming owing to their scale, speed, and scope.” (Dwivedi et al., 2023: 11).
GPT-3 has been trained on 45 terabyte of text data (Wu et al., 2023: 1123), “a diverse dataset of naturally used text obtained from different internet sources such as web pages, books, research articles and social chatter” (Dwivedi et al., 2023: 3). With 175 billion parameters (Brown et al., 2020), GPT-3 is able to store an immense number of patterns inferred from the data. Due to the diversity of the training data from which the GPT models learn, ChatGPT can be used for a wide variety of specific tasks, “such as preparing slides in a specific style, writing marketing campaigns for a specific demographic, online gaming commentary and generating high resolution images” (Dwivedi et al., 2023: 6). These and many other purposes for which ChatGPT is used demonstrate the systems capability to generate “answers to a plethora of questions” (Dwivedi et al., 2023: 3).
LLMs do operate in ways that differ from tools “because the language modeling objective used for many recent large LMs—predicting the next token on a webpage from the internet—is different from the objective ‘follow the user's instructions helpfully and safely’” (Ouyang et al., 2022: 27731). The same applies accordingly to the models underlying text-to-image generators such as DALL-E, Midjourney, or Stable Diffusion. Fine-tuning via supervised learning and reinforcement learning are the designers’ interventions to make the systems more reliable and predictable in providing the contributions the users would want for their respective purposes—in other words: to improve the systems’ toolness. Fine-tuning, however, does not change the characteristics of the knowledge acquired by unsupervised learning or how generative AI generates new content based on such knowledge. These characteristics “are blindly inherited by all adapted models” (Bommasani et al., 2022: 6). Thus, with unsupervised pre-trained generative AI systems such as ChatGPT, “we currently lack a clear understanding of how they work, when they fail, and what they are even capable of” (Bommasani et al., 2022: 1). As Dwivedi et al. (2023) put it, the basic functionality of ChatGPT is “‘ask me anything’ and ‘I may have a good answer’” (Dwivedi et al., 2023: 4). Even if fine-tuning improves the answers’ reliability, this is still quite different from how technology usually works.
Interacting with generative AI as the way of using it
The use of generative AI systems differs from the usual way of using technological artifacts. The accustomed ways of using technology require some knowledge about how the task to be performed is distributed between the artifact and its users. It is knowledge about the artifact's role and the user's role: about which parts of the sequence of events that brings about the intended result will be conducted by the artifact or have to be contributed by the user. With more complex electrical and digital machines, operating them means instructing them. But the mode of procedure is basically the same. Again, the users need to know what tasks are supported by the artifact and which parts of these tasks are to be performed by the artifact. In addition, they need to know the instructions that cause the artifacts to contribute as expected.
From instruction to interaction and co-creation
Describing the basic functionality of ChatGPT as asking the system anything and hoping for a good answer, as cited above, aptly illustrates how using generative AI is different from using designed technology. Prompting LLMs involves crafting questions for which the system has answers and asking them in a way that elicits the answers the user is looking for. Often, this process means changing or refining the questions in reaction to an unsatisfactory previous answer, asking follow-up questions, for instance asking the AI system to elaborate on certain aspects of previous answers, adding further information, and so on. Using LLMs, thus, is more akin to interacting with a conversation partner than to instructing a machine. This process is similar for all current generative AI systems, be it text-to-text generation as with ChatGPT, text-to-image generation as with Midjourney, text-to-music generation as with MusicLM, or the generation of other content from textual input (for a taxonomy of different kinds of generative models see Gozalo-Brizuela and Garrido-Merchan, 2023). With all of these generative AI systems, getting the results the user looks for requires changing, refining, and rephrasing the textual inputs (the prompts) based on the outputs (the text, images, music, or whatever the systems generate) until a result is reached with which the user is satisfied or until the user gives up.
An important aspect of this interaction is that not only does the system react to the user's inputs but users also adapt based on the system's responses. For the user, the system's outputs in response to their initial prompts become an input on which they adapt their prompts. That is, both the user and the system produce inputs for each other and generate outputs in reaction to the other side's inputs. This interaction continues until a user is satisfied with the result or gives up. Feuerriegel et al. (2024) characterize this as a “co-creation pattern” since it represents a “practice of collaborating in different roles to align and offer diverse insights to guide a design process” (Feuerriegel et al., 2024: 116).
The question of how interacting with generative AI systems constitutes a form of co-creation has been discussed most extensively in connection with text-to-image generative AI. There have already been a number of legal disputes over who should be considered the author of works of art created with generative AI. The US Copyright Office has argued that such art is a creation of the AI systems without substantial creative input by the artists. In two decisions on an AI-generated paintings by Stephen Thaler and on a comic book created by Kris Kashtanova via Midjourney, the US Copyright Office denied copyright protection for this reason, because, according to US law, only human creativity is protectable by copyright (Frosio, 2024). The previous considerations regarding the toolness of generative AI systems are consistent with the US Copyright Office's reasoning for its decision to deny Kris Kashtanova authorship for the comic. It argues that unlike with computer-based tools “such as Adobe Photoshop […] users of Midjourney do not have comparable control over the initial image generated, or any final image. Instead, ‘rather than a tool that Ms. Kashtanova controlled and guided to reach her desired image, Midjourney generates images in an unpredictable way’” (Frosio, 2024).
Empirical research on AI-generated art practitioners, however, suggests that there is considerable knowledge and work involved for AI artists to achieve the results they are looking for. According to Buraga's (2022) analysis of posts from an online community of AI art practitioners, the opinion that AI art requires hard work is one of the major topics (Buraga, 2022: 39). She quotes the following comment as an example: “People, in general, have not a deeper idea of how hard it is to do art with AI. I have invested tons of hours in my piece dropped today, mint and burned twice, and I am not completely happy with the results. So this is not at all push a button and obtain a masterpiece” (Buraga, 2022: 43). AI art practitioners emphasize that creating art with text-to-image AI is a collaboration (Buraga, 2022: 39–40). According to Oppenlaender (2024: 3766), it “resembles a conversation with the text-to-image system. A practitioner typically will run a prompt, observe the outcome, and adapt the prompt to improve the outcome.” The work involved and the skills AI artists develop over time and share as a community has been described by Oppenlaender (2022, 2024).
Another aspect of co-creation, which is not the focus of this paper but is nevertheless worth noting, relates to the creators of the content on which generative AI systems are pre-trained. Most of this content (texts, images, audio samples, etc.) is gathered from web pages, and much of it is protected by copyright (Buick, 2025: 183). AI developers argue that they do not need the permission of the rightsholders because “machine learning is a transformative use of the underlying data” (Chesterman, 2025: 26). This transformative use, however, generates content “that may, in fact, compete directly with past and present works produced by the authors and artists whose works trained those models” (Chesterman, 2025: 26). How to deal with this aspect of co-reaction is the subject of several lawsuits. Writers and authors have developed several countermeasures to prevent their creations from being used for machine learning without their consent (Frenkel and Thompson, 2023). Though this paper focuses on the interaction between technology and its users it is important to bear in mind this conflictual aspect of co-creation.
Prompt engineering: Strategic interaction with machines
For adapting pre-trained generative AI models to downstream tasks, an alternative to task-specific fine-tuning has emerged over the past few years. It is called prompt engineering. A foundational research paper on this topic characterizes the rise of prompt engineering as “a second sea change, in which the ‘pretrain, fine-tune’ procedure is replaced by one which we dub ‘
So far, the most elaborate prompt engineering techniques have been developed for text-to-text generative AI systems. More recently, prompting techniques for text-to-image or text-to-video generation have also become the subject of intensive research (Gu et al., 2023). They sometimes are adaptions of techniques developed in text-to-text prompting (Qi et al., 2023) but often represent “best practices learned from experience” (Oppenlaender, 2024: 3766) or rely on experimenting with prompt modifiers (Gu et al., 2023: 11).
Prompt engineering for LLMs such as ChatGPT contains basic strategies similar to those used for successfully instructing humans, such as giving clear and specific instructions, specifying explicit constraints concerning the task and its intended result, or giving examples of how the task should be carried out and what the result should look like (Chen et al., 2023; Ekin, 2023: 3–6). It also contains increasingly advanced prompting techniques. The first of them is called few-shot prompting. It is a more formalized way to provide the AI systems with examples of how to solve the task (Brown et al., 2020). Chain-of-thought prompting is a refinement of few-shot-prompting. It provides the AI systems with examples of how to break down the task into subtasks (Wei et al., 2023). Another prominent prompting technique is role prompting (Chen et al., 2023: 4–5; Kong et al., 2024) or expert prompting (Van Buren, 2023; Xu et al., 2023) where the AI system is asked to take the role of an expert in the task to be performed and to perform the task in this role. Role prompting or expert prompting makes generative AI systems referring in particular to learned patterns that represent knowledge associated with the respective role holders or experts. Accordingly, they generate outputs that are—compared to the same prompt without role prompting—more in line with what the experts of the topic in question would provide. Meta-prompting as suggested by Suzgun and Kalai (2024) combines ideas from chain-of-thought prompting and expert prompting in that it “guides the LM to break down complex tasks into smaller, more manageable subtasks. These subtasks are then handled by distinct ‘expert’ instances of the same LM, each operating under specific, tailored instructions.” (Suzgun and Kalai, 2024: 1). A fascinating aspect of these and other prompting techniques is that they can compete with and often even outperform LLMs that are fine-tuned for the respective tasks (Wei et al., 2023; for an overview of prompt patterns see White et al., 2023).
Some scholars suggest to understand prompting “as a particular form of
The question then arises what kind of interaction engineered prompting represents. The designers of new prompting techniques often refer to strategies of instructing humans and strategies of human problem-solving as their source of inspiration. The inventors of few-shot prompting explain that it “closely matches the way in which some tasks are communicated to humans. For example, when asking humans to generate a dataset on a human worker service (for example Mechanical Turk), it is common to give one demonstration of the task. By contrast it is sometimes difficult to communicate the content or format of a task if no examples are given.” (Brown et al., 2020: 6). The inventors of chain-of thought prompting point out that “chain of thought emulates the thought processes of human reasoners” (Wei et al., 2023: 9). And Jung et al. (Jung et al., 2022: 1) argue: “Explanation-based prompting is intuitively motivated by the reasoning steps humans typically employ to solve a problem.”
The patterns of human interaction on which the prompting techniques are modeled can be described as strategic. Strategic interaction is oriented at influencing the actions of other actors so that they comply with the goals and intentions of the first actor. The patterns of strategic interaction to which prompt engineering refers are aimed at getting other actors to perform tasks correctly by providing them with relevant context and with instructions of how to perform the tasks. These instructions, however, are not just “technical rules of action” (which is Habermas’ criterion to distinguish instrumental action from strategic action, see Habermas, 1985: 285–286). They are rather about providing the other actors with a pattern of how to solve a task and enable them to apply this pattern to new instances of the task.
After being transformed into prompting techniques, these patterns of strategic interaction still work in a similar manner. However, this does not imply that generative AI systems are actually reasoning. Even with explanation-based prompting and with the advent of the most recent reasoning models, the systems’ “ability to perform rigorous logical reasoning remains an open question” (Liu et al., 2025; see also Wei et al., 2023: 9). So, what does it mean to provide generative AI systems with patterns of how to solve a task and to enable them to apply these patterns to new instances of the task? It means to find prompting patterns that fit with the learned patterns of the algorithm such that they activate from “the plurality of ‘knowledges’ present in the underlying training corpora” (Burkhardt and Rieder, 2024: 4) those internal representations that are relevant and useful for solving the task. Just as with instructing a human counterpart, prompting a generative AI has strategically to take into account the “understanding” of the counterpart, that is, what learned knowledge can be assumed and what contextual and instructional information has to be provided.
The fact that LLMs respond so well to common strategies of instructing humans reflects their learned knowledge. A prompt engineering technique called zero-shot chain-of-thought exemplifies this nicely. Instead of giving the system a few examples of how the task should be solved step by step, as in few-shot-chain-of-thought prompting, this technique consists of simply adding “Let's think step by step.” at the end of the prompt. Nevertheless, it performs not too bad compared to the much more elaborate few-shot chain-of-thought prompting (Kojima et al., 2023: 9). Its success indicates the existence of step-by-step reasoning patterns in the learned knowledge of LLMs that are evoked by this prompting technique. This observation can be generalized: Prompt engineering successfully transfers patterns of strategic interaction to interaction with generative AI systems, as the knowledge learned by the systems already contains patterns that are sufficiently similar to these patterns of strategic interaction.
The agency of generative AI
Considering the properties of generative AI, scholars from the field of information systems research have argued “that a new generation of ‘agentic’ IS [Information System, author's note] artifacts requires revisiting the human agency primacy assumption. Agentic IS artifacts are no longer passive tools waiting to be used, are no longer always subordinate to the human agent” (Baird and Maruping, 2021: 314; Feuerriegel et al., 2024: 116–117). The presupposition implicit in that position—that technological artifacts not based on generative AI are passive tools—appears to contradict basic assumptions of actor-network theory. After all, the basic lesson to be learned from actor-network theory is that the identities of human and non-human entities are not given but a result of how these entities define and redefine each other within heterogeneous networks. Thus, both human and non-human actors play active as well as passive roles in these processes (cf. e.g. Latour, 1987: 108–144, 1991). The above argument is nevertheless worthy of being taken seriously because it addresses a different aspect of agency: the agency entities acquire due to their roles within the resulting heterogeneous networks rather than the agency they exert in the processes of building, establishing, and maintaining them.
Two kinds of technological agency
According to actor-network theory and other conceptual and empirical research on material agency (Schulz-Schaeffer and Rammert, 2023), every technology that works is the result of establishing heterogeneous networks and every technological artifact exerts agency when becoming involved in such network-building. The designers of new technology inscribe particular properties into the artifacts. The properties of the artifacts, in turn, prescribe to the users’ particular ways of how to deal with them. The artifacts thus exert agency by influencing the users’ behavior. The users may accept or deny the behavior assigned to them, thus exerting agency as well (Akrich, 1992: 207–209). With technological artifacts that eventually become established for specific purposes in particular contexts of use, this process of mutual definition and redefinition enrolls all entities involved in such a way that they assume roles that match with each other to some extent and that are sufficiently stable for some time (Callon, 1991: 144–152; Latour, 1991: 120–128). The entities thus acquire a particular identity, the identity of their role within the respective network (Schulz-Schaeffer, 2017: 277–280). This is what Latour means when he says: “An actant [a term for referring to human and non-human actors, author's note] is a list of answers to trials—a list which, once stabilized, is hooked to a name of a thing and to a substance” (Latour, 1991: 122).
Consequently, there are two different kinds of agency of technology. There is the agency all human or non-human entities exert by (re-)defining each other in the process of establishing and stabilizing a new heterogeneous network (and in changing or destabilizing it as well). But there is also the agency defined by the roles that become established as temporarily stabilized results of these processes of network-building. 3 It is with respect to this latter agency that generative AI is different from designed technology.
The new expert roles of generative AI systems
Whether task-specific fine-tuning is applied or prompt engineering, assigning task-related roles to generative AI systems is very different from how roles are assigned to designed technology. The general strategy of enrolling artifacts, which has been so overwhelmingly successful with designed technology of any kind, is to inscribe into artifacts parts of predefined sequences of activities that lead to the desired results. If successful, this strategy assigns an artifact the role of a more or less sophisticated tool to be used for specific tasks and humans the corresponding role of a user. This strategy, however, is not easily applicable to learned systems, especially when they have been trained in an unsupervised manner.
For task-specific fine-tuning, the pre-trained AI models are additionally trained using task-specific datasets. The additional training serves to teach them task-related patterns. The aim is that the algorithms adapt their general knowledge to the task but without unlearning it. The task-specific abilities generative AI systems acquire in this way are different from the technical functions of designed technology. They are not technical rules of action embodied in algorithms. Rather, these abilities resemble what with humans is called experiential knowledge. Experiential knowledge is practical knowledge based on experience. It results from accumulated experience humans gain when repeatedly engaged in particular practices. This experience condenses into heuristics, classification hierarchies, and domain-specific patterns related to the respective practices (Kingston, 2012: 4–5). Experiential knowledge is often characterized as a kind of tacit knowledge (Polanyi, 1966) because it is a knowledge people acquire without explicitly noticing it and use without explicit knowledge of its content.
The internal representations generative AI algorithms gain as result of being pre-trained and fine-tuned can also be conceived as a kind of experiential tacit knowledge. Being pre-trained with large amounts of data means that the algorithms are repeatedly exposed to data with similar content and structure thus having repeatedly similar experiences, so to speak. With task-specific fine-tuning this kind of exposure is intensified for task-related data, which in a way represents task-related experiences. The classifications and patterns the algorithms learn from these data are tacit in a similar way as the experiential knowledge of humans: generative AI systems can apply them but not reflect on them.
In the terms of role theory, the difference between contributing to tasks in an algorithmically predefined manner and doing so based on experiential knowledge can be described as the difference between role-taking and role-making. For role theorists like Linton (1936), role-taking means to play a role according to prescribed role behaviors. Role-making, in contrast, refers to the ability to interpret a role with regard to the situation at hand and to exercise it accordingly (Turner, 1962: 21–22). With human actors, role-making abilities usually grow with experience. While the beginners in a particular field of action tend to stick to the rules, the experts know how to deal with the specifics of different situations, and this “know how” is to a large part experiential tacit knowledge (Dreyfus and Dreyfus, 1986: 16–51).
Technological artifacts whose contributions to tasks are prescribed via their algorithmic structure are role-takers in the strictest sense. Operating on the basis of learned patterns that resemble experiential knowledge, in contrast, enables generative AI systems to adapt roles assigned to them to the particular circumstances of different situations. Like human experts, they become to some extent role-makers. Here, too, it is a “know how” learned from many different instances of carrying out particular tasks as represented in the training data that enable them to contribute to these tasks in a situation-specific manner.
The reason for this strength of experiential knowledge is, at the same time, the reason for two of its main weaknesses. First, it is knowledge based on past experiences (in the case of deep learning, on training data that represent past events). The use of experiential knowledge thus presupposes that the present in which it is applied is or shall be similar to the past from which it originates. This (often tacit) presupposition has been criticized as one of the main reasons for biases of deep neural networks—for instance by unintentionally reproducing existing inequalities or prejudices (cf. e.g. Barocas and Selbst, 2016: 681–684). Second, the patterns that constitute the experiential knowledge is correlational knowledge and not causal knowledge. It is knowledge about co-occurrences but does not include explanatory knowledge about causes and effects. 4 For this reason, human experts usually do not rely solely on experiential knowledge but also on approved causal and explanatory knowledge (which is also the kind of knowledge previous AI expert systems are based on, cf. e.g. Hayes-Roth et al., 1983; Lucas and Van Der Gaag, 1991). In this respect there is, thus, still a huge difference between human experts and the expertise provided by generative AI systems.
With task-specific fine-tuning, generative AI systems can be trained to become experts in a wide range of areas “including software development and testing, poetry, essays, business letters, and contracts” (Dwivedi et al., 2023: 3). Similar to human experts, their performance depends on how much relevant experience they gain and on how good they are in deriving relevant patterns from that experience (with both aspects more controllable during fine-tuning than pre-training). Different to human experts, however, their “experiential” knowledge is the only kind of knowledge they can refer to.
Prompt engineering, the other major way of assigning task-related roles to generative AI systems, can also be employed to turn generative AI systems into experts for specific tasks. However, prompt engineering does this differently and also opens up a much larger and more diverse range of possibilities of enrolling generative AI systems for all kinds of tasks. The prompt engineering technique of role prompting or expert prompting described above exemplifies the general rationale behind all prompting techniques: to tell the AI system the role it is supposed to assume with respect to the task it is asked to fulfill. As we have seen above, this can be done in many different ways, including strategies of describing what the results to be generated by the AI system should look like, strategies of specifying how the system should proceed in generating the result, or strategies of describing in which capacity the system should act.
The new agency of generative AI systems
Due to fine-tuning and advanced prompting techniques, generative AI systems can become quite reliable in generating useful results—just as users would expect from their technological tools. However, as described above, they generate these results not like designed technology by activating predefined sequences of actions but by creating them based on processes of matching the input with the learned patterns. This constitutes a new kind of agency of generative AI systems and a new form of inter-agent relationship between them and their human counterparts. The key to understanding the new agency of generative AI systems is in my opinion to consider the interaction with them and their human counterparts as meaningful interaction. The responses of generative AI systems to prompts should be viewed as the result of their meaningful interpretations of these prompts. And it should be taken into account that the human prompters can treat these responses with some success as meaningful.
What “meaningful interpretation” and “treat as meaningful” means, however, is still different for generative AI systems and their human counterparts. For the AI systems, treating and interpreting human input as meaningful basically means to identify fitting patterns from the learned knowledge. It is meaningful in the sense that referring to knowledge as the basis for action can be considered meaningful. For their human counterparts, in contrast, relying on experiential tacit knowledge is only one way of generating or interpreting meaningful actions. The ability to give an account of one's actions relies mainly on a different kind of knowledge: explicit knowledge about the beliefs and intentions that motivate these actions (Giddens, 1984: 6; Schulz-Schaeffer and Rammert, 2023: 45). The explications provided by generative AI systems upon request are not of this kind. These are not “real” explanations but also merely the result of matching inputs to learned patterns. Treating them as explanations and acting accordingly, nevertheless often leads to useful refinements of the original prompt. Why is this? The answer resides in a particular form of common ground between generative AI systems and humans.
Satyanarayan and Jones (2024: 5–7, 15–16) introduce accountability as a means of establishing common ground in collaboration between agents, be they humans or generative AI systems. In deliberative cooperation between humans this certainly is true. But even there, there is a much more basic common ground: common knowledge. According to Schutz and Luckmann (1973), common knowledge consists on the one hand of general knowledge, that is, of “those elements of the social stock of knowledge that are socially established as relevant for ‘everyone’” and are “routinely transmitted to ‘everyone’” (Schutz and Luckmann, 1973: 310). It consists on the other hand of special knowledge “that exhibits a role-specific, and thus uneven, distribution” (Schutz and Luckmann, 1973: 312). Common knowledge is the most basic common ground when it comes to understanding actions. If actions are generated and interpreted based on common knowledge, the common knowledge facilitates mutual understanding and thus the coordination of actions. For the human actors involved, it is not even necessary to be aware of this knowledge. It is enough that it exists as tacit knowledge.
The most basic common ground of the interaction between human actors and generative AI systems is also a kind of common knowledge, based on which the prompts and responses are generated. The patterns generative AI models learn during training are derived from products of human knowledge. To the extent that their human authors have generated these products based on common knowledge, the learned patterns will somehow reflect this common knowledge. Being pre-trained with large amounts and a wide variety of products of human knowledge, generative AI models learn a multitude of patterns. This learned knowledge somehow resembles Schutz and Luckmann's general knowledge. The data sets on which generative AI models are pre-trained also contain role-specific knowledge for a wide range of roles, from which the AI models learn knowledge that somehow resembles what Schutz and Luckmann call special knowledge. Fine-tuning serves to improve such special knowledge; prompt engineering aims at getting the generative AI systems to refer specifically to the internal representations of such special knowledge.
Obviously, the learned knowledge of generative AI systems is not identical with the social stock of knowledge human actors rely on. It is just “somehow” similar to it. To which extent turns out only when human actors refer to it in anticipating and interpreting the AI systems’ responses to their prompts. However, it seems that the learned knowledge often is similar enough to facilitate successful interaction with generative AI systems. A striking example is the success with transferring human patterns of strategic interaction to the interaction with generative AI systems as discussed above.
The new agency of generative AI systems, thus, lies in its ability to generate new content using learned representations that resemble elements of the social stock of knowledge. Due to this resemblance, the generated content is somehow similar to what humans would generate based on the respective elements from the social stock of knowledge. In a way, generative AI systems can be considered as agentified human knowledge. A machine-learned version of patterns of knowledge that inform human thought and action, derived from products of human thought and action, becomes agentic due to the capability of generative AI systems to generate new content on its basis.
Conclusion
This paper focused on main differences between generative AI and designed technology, which are related to the fact that generative AI systems contribute to tasks based on internal representations that are learned in a mainly unsupervised manner and not based on predefined sequences of action. The paper looked at how making use of this kind of content affects the task-relatedness, user interaction, and agency of generative AI systems. It showed how fine-tuning and prompt engineering is employed to deal with the task-agnostic nature of the knowledge generative AI models learn during their pre-training and how interacting with generative AI nevertheless remains fundamentally different from instructing designed technology. The paper suggests to understand the new agency of generative AI as the capability to mobilize machine-learned versions of human experiential knowledge and thus to view generative AI systems as a kind of agentified human knowledge.
These differences between generative AI and designed technology have far-reaching consequences regarding the social construction of generative AI systems and their uses. One of these consequences is that the classic approach of studying the social construction of technology becomes much more difficult to apply. From a social constructivist perspective one would analyze how society and culture is inscribed in the artifacts by the design choices of their developers, how potential users subscribe to, reject, or modify the designers assumptions about them and the new technology's contexts of application, and how interpreting and negotiating the usefulness of the new technology's functions and features eventually results (or fails to result) in a new sociotechnical constellation (cf. e.g. Pinch and Bijker, 1984; Akrich, 1992; Oudshoorn et al., 2004). With deep learning AI, society and culture is inscribed in the artifacts via training data sets. The composition of the training data sets is designed and thus can be analyzed and assessed in a social constructivist manner. This is what the discourses about biases in training data, their effects, and ways of how to deal with them is about (cf. e.g. Barocas and Selbst, 2016; Johnson et al., 2022; Mavrogiorgos et al., 2024). The patterns learned from the training data, however, are not designed and their intended uses are not predefined as with designed technology. They require a different approach that views the social construction of generative AI and its uses as a much more indirect process. Following from the considerations of this paper, the relationships and connections between the social stock of knowledge and the machine-learned versions thereof is crucial for understanding this process. This requires on the one hand to analyze how developers and users apply elements from the social stock of knowledge to make use of the learned patterns of generative AI, for instance as part of prompting. On the other hand, it requires to better understand in which ways the learned patterns represent machine-learned versions of elements of the social stock of knowledge.
Footnotes
Acknowledgments
I thank three reviewers for their valuable comments on earlier versions of this text. I thank Anne Wegner for her thorough language editing, which significantly improved the readability of this manuscript.
Ethical considerations
Ethical approval was not required because this study does not involve human participants, human data or human tissue.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a fellowship at the Käte Hamburger Center for Advanced Study in the Humanities: Cultures of Research (KHK c:o/re) at RWTH Aachen University and funded by the German Federal Ministry of Research, Technology, and Space (BMFTR). I acknowledge support by the Open Access Publication Fund of TU Berlin.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data availability
All data referred to in this manuscript are publicly available.