Abstract
In May 2019, the “Automation and Me” event invited digital artists from around the world to come together in an unused shopping center unit in the heart of Leeds, United Kingdom, to critically respond to and explore the themes of automation, embodiment, and identity. In this article, we draw on Tobias Matzner’s argument that what algorithms are and do “emerges in a complex interplay of social practices, material properties, discourses, mathematical abstractions, code” to think through the event. In so doing, we explore a number of complex interplays as they were felt and lived by all of us who participated in it. We reflect on the experiences during the workshops through the particular framework of automation and feminist new materialism to explore how AI technologies and the complex surveillance of the smart environment replicated, on the one hand, existing and oppressive prejudice and, on the other hand, worked to constantly recalibrate and manipulate the context that humans and nonhumans navigate.
The Automation and Me hack was a weeklong feminist collaboration in 2019 to deliver a three-day workshop as part of the Leeds International Festival UK where we engaged with critical issues relating to technologies and bodies through creative practices. In this paper, we draw on our experiences of Automation and Me in order to think about automation as complex, embedded, and a significant element of the material, sociotechnical, and embodied experience of space. We argue that automation shapes and generates contexts, including architecture and the conditions of possibility for everyday life. At the same time, automation is political and powerful because technologies, algorithms, bodies, and automation play a role in upholding white, masculine logics.
Our event invited twenty-four feminist and activist digital artists from around the world (including Japan, Argentina, the Netherlands, and the United Kingdom) to come together in an unused shopping center unit in the heart of Leeds (see Figure 1). The artists were recruited by the Northern Sound Collective, 1 who issued an open call for the event as well as inviting artists directly to try to ensure diversity, including practices and experiences. For the open call, artists wrote a paragraph addressing how their work fit the theme of automation and me, and from that, we selected twelve artists prioritizing fit to call diversity of practices and perspectives. The agreement forms included consent for the documentary maker and ethnographer to record and interview and for wider academic and public use of quotes and images (the latter with a secondary stage of consent following initial private screening). In keeping with our own ethics processes, verbal consent was also sought during the initial session. In this paper, we also refer to anonymized security personnel, and while the security personnel were aware of the event and our activities, including the presence of a documentary maker and researchers because of health and safety, privacy, and due diligence protocols, which required consent for their presence; they did not have the same agency in terms of consent. It is for these reasons that the only direct quote is from a publicly witnessed and often repeated phrase, and for the rest of the paper, we have chosen to explain conversations rather than quote them. One of the participants created a film, took photos, and interviewed others, sharing her work during and after the event. 2 The material for this paper comes in part from that corpus and in part from the notes and reflections of the ethnographer written during the event.
Looking at Automation and Me from the outside.
The arguments for this paper, then, stem from the Automation and Me event, as well as ongoing critical work around automation, embodiment, and identity. The activities we draw from were part of the Leeds International Festival, which were in turn designed to both showcase the city and engage in creative digital playful work. We chaired the opening discussion, touching on themes of care and ethics, power and technology, digital colonialism, and identity: topics that connected all the artists and their work with those of the festival. Then, we invited the artists to reflect on this discussion, and they began to create responses through build-and-design sprints over the course of the week that culminated in an interactive installation on the closing evening of workshops. 3 We facilitated the sessions, drawing out connective themes but the artists themselves created the projects and direction of work. Working in unit C18/19 (the space donated to us for the activities by Leeds council)—a disused shopping unit in a (then) newly built, smart shopping center (Victoria Gate) adjacent to the historic market in Leeds—also shaped the activities both in terms of what was digitally possible and through the specific interactions that occurred between the longtime occupants of the space (security personnel, shoppers, and shop owners) and us. Over the course of the week, groups made immersive and playful installations, reprogrammed Alexa to be less subservient, labored with embroidery machines to build reconfigured twister mats, and built literal webs through algorithms. The shopping unit was open door, and shoppers would wander in, ask questions, or look through the windows.
Our title is taken from an explanation of automation offered by one of the digital artists (explored in the later sections) but also relates to what we were trying to do during the week as we stayed late and arrived early to the unit, creating different rhythms and patterns of movement through the shopping center. Much of what we talk about in this article comes from an increasing awareness over the course of the week that we were being surveilled and that these forms of surveillance created and were evoked to produce power relations that positioned us in highly unequal relations. We talked about this as a group during the week, sharing experiences, recounting conversations; and we have continued to discuss this in increasingly theoretical terms following the event partly because of the work of scholars such as Matzner (2019), Parisi (2013), and Introna (2016), who have informed our reflections, and partly because we have increasingly understood the event as metaphor through which we can think about contemporary everyday experience per se. The structure for the theoretical work comes from Tobias Matzner’s (2019, 216) argument that what algorithms are and what they do “emerges in a complex interplay of social practices, material properties, discourses, mathematical abstractions, code.” We use Matzner’s argument and explore each of his concepts (social practices, material properties, discourses, mathematical abstractions, and code), putting them together in different ways as a critical thought exercise. First, we reflect on how material properties and mathematical abstraction could be said to generate the lived and material elements of the building. We then look at mathematical abstraction and social practices, exploring the sociotechnical ways we experienced the building and the extent to which we could claim the building generated experiences through the parametric design (where buildings are shaped by algorithms) and relations of the building. Following this, we move on to discuss social practices and discourse: focusing on the ways that automation became discursively evoked along power structures and in so doing, generated discursive and material power. We think about discourse and code through the experience of one of our participants, drawing issues such as expertise and labor into our conceptions of automation and extending this to explore code and the material (coming in a full circle at different scales).
While our movement then is from architecture (macro) to bodies (micro), our conception of automation becomes increasingly open and layered—temporally, spatially, materially, and discursively. Introna (2016, 23) argues that what is valued or understood about algorithms and automation depends on not just where but how you make “the cut”—across scales, perspectives, and experiences. Our central aim is to make several different cuts and in so doing critically open up how we are able to think about automation to incorporate a variety of scales, temporalities, and speeds. We reflect on the experiences during the workshops (see Figure 2) through the particular framework of automation and feminist new materialism to explore how artficial intelligence (AI) technologies and the complex surveillance of the smart environment replicated, on the one hand, existing and oppressive prejudice and, on the other hand, worked to constantly recalibrate and manipulate the context that humans and nonhumans navigate. In what follows we firstly discuss Victoria Gate as a smart, parametrically designed building before discussing what that means for, and how it shaped the ways that we could live within it.
An image from one of the group projects developed through the week.
Material Properties, Mathematical Abstractions
Victoria Gate in the northern English city Leeds was completed in 2016 at a cost of £165 million. 4 It is a smart and automated environment: living, sensory, data-driven system(s) that uses real-time data capture to feed machine learning systems through algorithmic processes to regulate the light and temperature of the space, for example. To draw on Matzner (2019) and Parisi (2013), Victoria Gate is parametrically designed: built according to “external conditions” (Matzner 2019, 136; see also Parisi 2013, 104) which inform the building and to which it responds. The notion of parametric design is a useful concept for us not only because it encapsulates the use of automation in relation to the control and management of buildings (as we explore later)—such as the use of sensor driven, ambient lighting to mimic sunlight that are designed to brighten and dim throughout the day along with the natural rhythms of the sun. Parametric design also refers to the conditions of possibility of the building itself in terms of it always already being an outcome of automation, software, and labor, on the one hand, and the wider economic, ideological, and material contexts for its design and creation, on the other hand. The idea of parametric design—being built according to “external conditions” (ibid.)—reminds us of the complex ways that material artifacts come into existence—through negotiations with wider structures (e.g., technological, infrastructural, sociopolitical). Indeed, as Varnelis (2005, 27) reminds us, “new infrastructures do not so much supersede old ones as ride on top of them.” This is a form of what Varnelis (2005, 1) calls “path dependent” evolution, which “tends to reinforce urban development patterns” (Mattern 2017, 30). For Victoria Gate, the fact that it was designed to “pay homage” 5 to the existing architecture of Leeds, most notably Victoria Quarter in the city center, shapes the material aesthetics of the space, which in turn frames activities within that space, as we shall see.
The design of the building does not just relate to aesthetics of course. The acoustic engineers of the building talk about the need for the building to be a low or zero energy space not only for environmental but also economic reasons. 6 Their acoustic design is a fundamental element of their remit as “engineers of human experiences.” 7 These designs underpin our discussion later of the management of sound within such environments and how important it is in the shaping of lived experience. Taken together then, we can see how the remit and design principles of the building relate closely to and form part of the “external conditions” (Matzner 2019, 136) in which it is located: materially, geographically, and also ideologically.
All of which is to say that we should understand Victoria Gate as bound up in the “conditions of possibility” (Suchman 2007, n.p.) that imagined and produced it (see Figure 3). Such “conditions of possibility” relate not only to the wider external conditions that produce it (such as remit, partners, economic and environmental concerns, consumption), as well as design techniques and software used to imagine and shape the build. Hoare Lea & Waterman Structures used BREEAM 8 modeling to assess the sustainability of Victoria Gate and BIM 9 systems for designing the acoustics including vibration and audiovisual qualities: specific software widely adopted which as Gardner (2019) notes that software shapes how a building can be imagined, planned, and built. The modeling software does not facilitate the “translation” of design into a digital environment: “computation informs and shapes design decision-making” (Gardner 2019, 109-10).
The notion of parametric design encapsulates these practices and ways that technology, code, and ideologies all frame conditions of possibility for architecture. This reminds us in turn that technology is always already embedded within architecture: woven in, meaningful within, and familiar and, in so doing, directs us to think about algorithms as “embedded in social practices” (Matzner 2019, 126) rather than somehow separate from them. This relationality bleeds into the computational systems of design so that separating ideologies and systems, gender, and automation, for example, is impossible. Shopping centers have long been researched as spaces of control, discipline, and surveillance (see, e.g.,, Goss 1993), so it should be no surprise that the politics and ideologies that underpin the desire for control and management of such spaces bleed across, for example, the technologies, software, material, and labor practices that constitute their design (and vice versa). The first cut we would like to make then is to suggest that automation is increasingly enmeshed within architectural processes. Parametric design builds according to (ideologies, principles, materials of) “external conditions” (Matzner 2019, 136; see also Parisi 2013, 104) but also—as we go on to discuss—the building is managed by real-time algorithmic processes (Parisi 2013, 104). The second cut is that contextualizing our environment in this way forces us to acknowledge that automation is neither materially nor temporally contained: it bleeds across spatial, material, and temporal boundaries. In what follows, we further unpack these relations drawing on two experiences from our event. The first of them relates to walking slowly.
Mathematical Abstractions, Social Practices
On day one of the event, the twenty-four women and non-binary people stood up as instructed and walked as slowly as possible around the empty shopping unit. Our instructions were to go slower, go slower still: to shift our weight with full consciousness and awareness from the heel of one foot, along the outside edge of it, pressing down first the little toe and then moving to the big toe. We were told to shift our balance onto that leg, noting pauses, imbalances, keeping our head high, extending our spine, finding space, transferring weight, pressing, lifting: trying for a slow continuous movement but not quite achieving it. Our shopping unit—while fine as a boutique of fancy things—was far too small for more than twenty bodies to walk around in, so we trickled into the shopping center: a kind of non-flashy flash mob of attentive automata. As we slowly swarmed out, a small boy ran away; barista staff paused; coffee-drinkers stared; shoppers sped up and past. Some sped up then, at a safe distance, stopped, snapped. Caught up in the mechanics of movement, concentrating on balance, trying to go slower and slower still, and then suddenly, a flash of high vis. One, then two, then four bored faces of shopping center security guards. One woman: Janet (who later introduced herself); three men: nameless (who never introduced themselves). They descended from their security room, radios cracking and cackling, earpieces in, and on the edge: not aggressive, not stopping us, but implicating us, encircling us, then stopping. Meaningful, representative bodies: hardened by the layers of uniforms and immobility. We stopped, feeling a shift in something. Glanced around, looked at each other and returned to C18/19. Later, when we went to return the keys to the security room at the end of the day, we were told: “Get permission next time girls, if you want to be disruptive: there’s no hiding in here.”
The group performing Pauline Oliveros’s “Extreme Slow Walk” from Deep Listening: A Composer’s Sound Practice (2005).
In order to think about how our actions came to be understood as “disruptive,” we need to begin with an understanding of how decision-making works within Victoria Gate. Parametric buildings increasingly incorporate machine learning processes and “real-time” data capture and input (Parisi 2013, 104) to manage the space. Simply put, every time a door opens or closes, it is registered as a line of text on a central computer. These automated and algorithmic processes further locate buildings as always-already relational and networked but, more than this, foreground “neoliberal forms of governance” (Parisi 2019, 94), which are constituted not by norm, reason, or the law but by “control functions, behavioral operations based on procedures within self-regulating autopoietic agencies” (ibid.). To begin to think about how it was that our actions came to be understood discursively as “disruptive,” we need to explore the building’s sociotechnical and automated processes within which we were located. This in turn gets us a little closer to our third cut: an understanding of algorithms as always-already more than technological automation: as something that is complexly sociotechnical (Suchman 2007). So what are some of the elements that constitute the sociotechnical in this instance? We begin by noting that Victoria Gate is built for and with pattern recognition software that uses real time and big data to monitor and automate responses to movement within and through an environment. For example, sensors embedded within the building observe the environment and note “unusual” behavior. Bounded by its exterior walls, actions outside the “norm” trigger an intervention in space, which may be human (security guards) or machinic (locking doors, sounding alarms, and calling for outside intervention). These processes serve to maintain the building as designed, in this case, to ensure a flow of movement between different shops.
These “norms” of behavior are constituted in turn through pattern recognition software that uses the “rendered” data from everyday experiences (Zuboff 2017, 324)—the number of people moving through the shopping center at any one time, for example—to co-constitute a range of automated processes, calculating, for example, movement trajectories through environments or changes in temperature or lighting to inform decision-making that will produce automated responses and adjustments. Pattern recognition software is built through “routines of organizational culture that are standardized” (Rossiter 2017, 138). The standardization of routine is what enables patterns to be made visible, norms to be established, and pattern recognition software. This, in turn, enables what Hu et al (2004, 334; see also Nguyen et. al 2005) have called “intelligent surveillance” where real-time data co-constitutes the conditions within which decisions are made about the building from a security center nestled within. Those decisions can be human, technological, or both—they may be as mundane as changing the volume of the background music within an environment to tracking bodies visually and spatially across a cityscape. Intelligent surveillance redistributes decision-making factors to automated and human processes (Matzner 2019, 134), while recalibrating those factors into more modular data that can constitute automation. In other words, the line on the central computer that indicated a door opened could initiate a variety of responses that might be human or machinic (that would then be logged); initiate a particular action such as a visit from a security guard; and be nonconstitutive until a certain point (be ignored until it has happened X number of times in a particular timeframe). At the same time, the door opening also forms new constitutive data about “normal” movement (if the door was repetitively opened, it might stop being “seen” as abnormal).
Pattern recognition software can be taught to cross-tabulate (with facial recognition software, for example) and share data using multiple interactive cameras across environments. A person can be tracked across a city in this way, through a process that is both automated and human. Here, anomaly detection of movement trajectories trigger real-time and emplaced processes, as well as feeding into and constituting what is understood as normative behavior over time. Other applications include crowd flux and congestion—calculating flux and then automating management of movement processes to redirect that movement. Biometric data might also be used in place of access cards to areas that may be tabulated with and through biometric data (see Hu et al 2004, 334). In each of these scenarios, the systems are taught to recognize normative movement trajectories through pattern recognition to then highlight nonnormative movement through “anomaly detection and alarming” (Hu et al 2004, 334). All of these human–technological, automated, and semi-automated responses build on pattern recognition which is crucial to establishing a norm against which real-time data are compared. As machine learning processes are increasingly engaged in anomaly detection and alarming, what constitutes the “norm” is constantly recalibrated through real-time data processes.
When we talked to Janet (the security guard) about what had happened that day, she told us that opening a door in Victoria Gate triggers an alert that is both sent to the mobile devices carried by the security guards and to the central control room. This made us think about (and consciously notice) the building in relation to parametric design and automation, and later when we reflected on the event, it was these experiences and themes we kept returning to. The automated response to a sensor (the door opening) produced a human–technological response insofar as the mobile phone nudges a security guard by vibrating. When we walked slowly out of C18/19, the security personnel all received an initial prompt through their mobiles, but as a standalone nudge, which may have been ignored. By contrast, real-time, crowd flux data triggered alerts in the security room because a mass of bodies had suddenly appeared and were not dispersing. In terms of automated processes, real-time data tabulates and alerts on a range of levels and through a range of connected but distinct processes and alerts. Parisi (2013) has argued against Zuboff (2017, 324), that it is not only our experiences that are rendered into data, which suggests a one-way process in which humans produce data for systems. Instead, automation also shapes experience in iterative ways, and we need to also consider the normative movement trajectories that co-constituted the conditions in which we moved slowly: As data are recorded, so they evolve into predictive scenarios aiming not simply at presetting your movement, but rather at generating its future conditions through the generative interaction of parameters with real-time data. (Parisi 2013, 105)
What was interesting about our experience of course was the way that it enabled the outcome of such processes to be made temporarily visible. Our witnessing of a flash mob later in the week (as we discuss below) produced a different outcome and it is interesting to wonder if the two experiences were connected: Later in the week, the (“real”) flash mob took over the space outside unit C18/19: 10 then 20 then thirty bodies appropriating the piano to sing loudly about God. The volume of what turned out to be an electric piano (on closer inspection: Yamaha) decreased as the crowd reached a certain number. No high viz jackets this time. The decrease in volume was (Janet told us later) an entirely automated response to the algorithms flagging changing movement trajectories at scale. The background “nature” noises of birdsong—which we had stopped hearing after a few minutes inside the shopping center—increased. The volume and the sound, calculated in relation to the spatial dimensions of the building and longstanding socio-psychological research into optimum soundscapes for consumption practices, increased, returning our attention to the carefully orchestrated soundscape which we had forgotten. The 30 (or so) singing bodies also heard (or sensed) the increase in volume and responded initially by singing louder. The different human-technology ensembles (the audio systems of the shopping center on the one hand, the flash mob on the other) “created and legitimized” (Matzner 2019, 141) through the constant reconfiguring and recalibrating of decisions (ibid., 135). The flash mob sang without their piano: lifting their arms, waving their bodies, closing their eyes. And then dispersed, quietly to the eerily loud sound of birdsong.
Social Practices, Discourses
We now turn to social practices and discourse to examine how automation is evoked and enacted though power structures to think about the social practices that shaped our experience of working within the space. On the one hand, social and discursive practices form part of Matzner’s (2019, 126) “complex interplays,” but to us they were also highly visible power structures during the event. When we recounted the words of the security guard (who told us: “Get permission next time girls, if you want to be disruptive”) to the group, his words became a sort of ironic mantra for the remainder of the week, partly (we think) because labels of “girls” and “disruptive” were jarring but also because of overt discursive constructions of power into which we were all positioned.
Reflecting on this moment offers a range of considerations around gender, expertise, power, and discursive framings (to name a few). Journeying to the security room, through back corridors, up an elevator, we had to knock on an outer door within which was a kitchenette and leading from this the security room “itself.” This was a journey entirely designed to impress us into a particular relationship not only with the security guards but also with the building “itself.” Emerging from a security room to discursively name us as “disruptive” and as “girls” was as much a performance of (white, masculine) power which was embodied in very particular ways in this moment. Not only could he/they name us and our activities in particular ways, but the unimpeachability of his position, and the conditions which produced this scenario, all worked to underpin it as particularly disciplinary.
The exchange also made us aware of a particular discursive act in which the recourse to surveillance technology or automated processes enabled power in really interesting ways, and there were a number of interactions with the security personnel where a similar move was enacted. We want to briefly unpack this maneuver because of the way it discursively enmeshed technology to generate power relations. In this exchange on the edge of the security room, the male security personnel were not only, as we have said, able to discursively name us (as girls) and what happened (as disruptive), but they also constructed themselves as agents who acted on (and were able to act on) pattern recognition algorithms and surveillance systems. To borrow from Parisi (2019, 105) above, this was a moment where discourse, automation, data, and materiality were all performatively evoked in order to “pre-set” our movement and to generate the “future conditions” of our activities. The relationship into which we were interpellated was also one in which recourse to automation and surveillance was enacted, but the claim of us being disruptive (and “girls”) was also produced in that moment as an outcome of automation because we were framed as disruptive because we were visible to, and understood as nonnormative by, automated systems. In other words, the statement that we were being disruptive was a claim made possible through a recourse to automation. In that moment, automation was evoked as a useful tool to underpin his statement (“disruption” was not claimed as an interpretation but as an unimpeachable, apolitical “fact” generated by technology). His own power was similarly cemented because he was seemingly “just” articulating the outcome of an algorithmic equation that promises gender and ethnicity-free decision-making (see also Thornham 2019, 36).
Security guards often lingered outside the windows, loitered in the doorway, or obtrusively “invaded” C18/19 asking us questions that made it clear we were being watched, and referring to us all as “girls.” A number of participants were followed throughout the building and to the car parking areas, and when we asked Janet why Kameela was followed the previous night by one of her (male) colleagues, she told us that when an alert is triggered (because a door is opened, for example), they have to “act on it, or explain why you haven’t.” As Janet described it, ignoring the prompt requires a later explanation which is set against the digital record of the evenings’ activities. In this instance, we see the processes of automated surveillance of security personnel (not just the general public) at work: the “always-on ubiquitous monitoring” (Andrejevic 2019, 10) and its consequences. Responding to a prompt is understood and conceived as the normative action that requires neither an explanation nor accountability. But ignoring a prompt requires work, accountability, and intention. There are lots of issues to think about here, including how human visibility within automated systems is understood, framed, and valued. But what is also worth noting is how Janet’s explanation (like the man’s above) sought to construct the technology as the (unimpeachable) prompt/demand for action, and the human agent as “only” an outcome of automation. More than this, in their explanation, Kameela was constructed as the accountable—visible, agential—element. Her seemingly mundane movements through the shopping center, security claimed, provided a rational explanation for the fact that she had a hugely unsettling and scary experience. For Kameela, being followed throughout the shopping center was highly politically charged: bound in power politics, enacted as masculine and white power: she was followed because she was a Black woman in Victoria Gate. Janet explained that it was not a political, racial or gendered act because it was “only” part of an automated process that Kameela herself initiated. In explaining the event in this way, Janet sought to (perhaps unconsciously) depoliticize the experience through recourse to technology (which was constructed as apolitical) and construct her colleague as an outcome—rather than constituent element—of decision-making.
Constructing automated decision-making processes as apolitical is a familiar (see, e.g., Gitelman 2013, 2; Kitchin 2014, 19) and incredibly destructive (see Benjamin 2018, D’Ignazio and Klein 2020) maneuver: one that produces and generates the perception of automation as powerful and as “separate, logical, unbiased and transparent,” while simultaneously working to mask the politics and prejudices through which automation is generated and to which automation ascribes (Thornham 2019, 36). It is a two-fold maneuver that enables racism, sexism, and discrimination while claiming such actions are logical and apolitical outcomes of automated systems (see also Benjamin 2018; D’Ignazio and Klein 2020). The recourse to automation as an explanation for action serves to enforce the discursive construction of automation as apolitical, logical and rational; and further, it sidesteps any need or demand for political, lived, human accountability, because it constructs such events as normative and normatively apolitical, when in fact they are the opposite. Seen here, agency is discursively located within the processes of automation, thus also sidestepping any accusations or discussions about power (racism, sexism, classism, etc.). Our fourth cut, then, draws attention to the way automation is discursively evoked to enable certain power relations that are increasingly irrational precisely because of the adherence to the normative logics of automation.
In the final section of this article, we want to think about Matzner’s argument in relation to one of the participants of our event: Sadie-Love, the textile machinist. In thinking about the experience of Sadie-Love, we introduce another scale for thinking about automation and return to and extend questions of automation, disruption, and embodiment. Indeed, in thinking about Sadie-Love, we open up different understandings of automation as social practices, material properties, and code (drawing on Matzner once again). In juxtaposing the automation of Sadie-Love’s lived and embodied practice with the lived and embodied automation of the Shopping Center, we can also ultimately begin to see what is at stake here.
Discourses, Code
During the Automation and Me event, Sadie-Love worked with her Pfaff textile machine to create performative pieces of artwork with a group of other women. Her “task” was to sew embroidered precoded factory-setting images onto squares of material that would eventually form the different tiles of a mat. She sewed with conductive thread so that an electrical current could be moved through each pattern and reach every tile of the mat, generating feedback in the form of basic code (on/off) when touched. The idea was to create a textile mat inspired by the game Twister; the on/off sensors arranged in a grid of three by three, each to a separate pin on a microcontroller to trigger “ons” and “offs” in different combinations. A computer nearby would note which squares were being touched and in what order (Figure 4). Each series of movements as bodies moved and pressed different tiles on the mat were cross-tabulated against a sequence that eventually unlocked a box: the key for which was formed from the sequences of code. You had to press tiles in a particular order to open the box, but the key (the sequence) constantly recalibrated depending on the sequence of data that were inputted (the box never opened, nor could it). The artwork understood automation as the logic of the game and as the programming of a system that sought to identify patterns and also as continually recalibrated real-time data.
An automated, sonic twister mat produced by participants.
Sadie-Love sewed fast in a mundane and familiar rhythm that was entirely embodied and unthought. She described her work as “listening to the machine” paying attention to “judders, jabs, slips.” She explained her artwork as embodying and exploring the history of women, textiles, labor, and embodiment, telling us: “gender is embedded in the materiality of the textiles. It might not be at the forefront of your thinking, but it’s lived in what you do.” Sadie-Love operated at a range of temporalities—in time with; a little in advance of; as a response to—the rhythm of the machine. These speeds and temporalities didn’t just relate to machinic speed, but to the “living present” (White 1987) of Sadie-Love’s practice over time as well as the material elements which she responded to and which were bound up in, and beyond the machine. By comparison with Sadie-Love, the machine sewed slowly: the conductive thread had knots and got tangled. It needed less tension, a slower rhythm.
As well as creating a piece of work that sought to enmesh automation with bodies; and as well as mediating and predicting the Pfaff machine through code and through sensory and embodied interactions with it; Sadie-Love was also part of the automated systems of the shopping center in a number of ways. Positioned in the window of the unit, she was visible on the monitors in the security control room—not as “erratic” or normative movement but instead contained within a shopping unit and repetitively “caught” in the cycling through the shopping center’s fixed camera viewpoints of the shopping center. Sadie-Love may have been contributing to the environmental systems of automation insofar as she generated thermal heat and sound, but being in the window of the shopping unit, seemingly immobile, contained for long periods of time, meant she was minimally legible to the pattern recognition software—what Parisi (2019, 90) has called the “mediatic infrastructure”—by comparison to shoppers moving through the building, for example.
This means that there was little reason for security guards to notice her. She was minimally visible within pattern recognition, movement trajectory, or crowd flux algorithms; she contributed very little to statistical modeling or predictive analytics that constituted these systems. She was not flagged via alerts or prompts: hers did not constitute erratic or nonnormative movement. Yet she was frequently commented on to us by the security personnel particularly when we picked up or returned the keys at either end of the day. Her visual appearance, her performance on the machine, her presence and absence, and her arrival and departure were all noted (Figure 5). There are several issues we want to think about here in relation to surveillance, automation, and power that these conversations with security personnel about Sadie-Love made us reflect on and we want to frame these issues around the distinction Andrejevic (2019, 8) makes between what he calls “symbolic surveillance” (older camera/visual surveillance systems) and “automated surveillance” (real-time data capture and input).
At the sewing machine.
Andrejevic (2019, 8) argues that there has long been a blurring between the monitoring of activity (through data capture) and visual surveillance, yet symbolic surveillance can be defined as an asymmetrical power relation between “watcher and watched.” It is a form of “panoptic power” (drawing on Foucault 1977) which is bound up in visual power, whereby being able to look is elided with knowledge and power. Automated surveillance, by comparison, is the term Andrejevic uses to refer to mediatic infrastructures: automated processes whereby action is “rendered” (Zuboff 2017, 234) into data. His argument is that mediatic infrastructures displace the long-held assumptions that to look equates with power and knowledge. Mediatic infrastructures, he argues, favor constitutive data that can feed predictive analysis: what is being valued has shifted from the visual to data, and from capturing to prediction—from disciplining bodies to conditioning them (as discussed above). In Andrejevic’s terms, automated surveillance are the processes that feed statistical modeling, pattern recognition, data mining, predictive analytics, and self-organizing adaptive systems which comprise and constitute smart parametrically designed and managed buildings.
Seen within this distinction, Sadie-Love is positioned as an object of symbolic rather than automated surveillance within a parametrically designed and smart building that values automated surveillance over the symbolic surveillance (as we have already seen above). So, what is going on here? It seems to us reflecting on these conversations with the security personnel about Sadie-Love that symbolic surveillance was being drawn on in order to evoke and affirm an unequal power relation, in which we (and Sadie-Love) were being watched. The fact that this occurred within a wider context of automated surveillance is a further indication not only of the complex interplay between automation and social practices but of the many layers and complexities of automation per se. These interactions and processes of symbolic surveillance are not outside of automation: they are encompassed and validated within “it.” The exchanges about Sadie-Love also reminds us that automation is far from apolitical—not least because in this instance it enables a discursive claim to a wholly masculine, white position of power, which via these exchanges is exploited and claimed through these exchanges. To say that there was an absence of predictive analysis, code, algorithms, or automation in Sadie-Love’s work would also be a misrepresentation. Our fifth cut then is to argue that we should think about automation in embodied and lived relational ways that are also framed within and speak to wider long-standing power structures in which automation and technology are also configured.
Code, Material Properties
Sadie-Love’s role is also partly that of repair work of maintenance (Jackson 2017, 174). She constantly had to care for the machine: ensure the tension, retie the thread, and restart the needle. We could also conceive her role in relation to technological nudges that demand action, or accountability, that has implications for what is made visible within processes (or not). Each change is forged through the enactment of the repair process is both a correction and a redirection so that the machine—and the tile being stitched—“now works differently, containing some but not all the marks of its labor as well as the breakdown that occasioned it” (Jackson 2017, 179). This is not only because of the mechanic: the machine also “informates” (Zuboff 1988, 9), learning from past processes, bearing the scars of breakdowns, and forging new patterns. The knots and broken threads, the restarts, judders, and slips are sewn into each design.
Through the example of Sadie-Love, we see multiple—very different—ways she is understood and valued within and as automation. Yet it is much easier to think of her work—and her—within different narratives of repair work or even in relation to histories of industrialized labor. Indeed, there are parallels to be drawn with Kylie Jarret’s work on the shift to industrialization, where embodied and sentient knowledge “amounted to deskilling and disenfranchisement” (Jarrett 2017, 35), because sentient knowledge was unquantifiable and intangible during a period of industrialization that valued abstracted rational and datalogical information instead (Zuboff 1988, 9). What is it then that invites us to understand some embodied experiences (such as the security guards) as automation and others as something else?
Fazi (2019) argues that there is a long-standing “impasse” or a “deadlock” (p. 3) that relates to our inability to think about technology and ontology together. Fazi (2019, 3), tracing arguments of digital technology through the work of Deleuze, highlights what she calls a fundamental “ontological discrepancy” between the “continuity of sensation,” on the one hand, and “the discreteness of digital technology,” on the other hand. In thinking about how Sadie-Love describes her work, we could say that the continuity of sensation referred to by Fazi—the sentient and embodied knowledge of the listening to the machine, forged over time and through experience—is devalued and understood as “deskilled” at key moments (or stabilizations) when technologies—be they tangible machines, automation, and data (metrics)—become valued as indicators and representations of productivity.
In this scenario, productivity could relate to the production of an output and the idea of working linearly to create a specific outcome (e.g., the conductive mat). Or thinking more broadly, it could relate to the security staff members’ interest in Sadie-Love as constitutive data within an automated system that values some data over others. What is interesting is that Sadie-Love also articulates these ideologies, telling us that she is “quick, but not that quick”; reframing her project in that instance within the terms of productivity-as-speed—evoking a goal and an outcome, and conceptualizing technology and her ontological knowledge within those terms. In both cases (a tangible creation or constitutive data), productivity values discreteness over continuity, especially if we consider how abstractable frameworks, which are often today conceptualized as automation, AI, or machine learning systems, produce forms of exclusion.
With Jarrett’s example, this took the form of excluding women from “skilled” work, but we could also think of the many forms of exclusion detailed in this article on bodies and space. We could also understand how productivity values discreteness if we think about forms of knowledge where representations (i.e., metrics/data) are taken to stand in for—or supersede (Parisi 2019)—knowledge. This also generates exclusions, which we have detailed in this article. We also need to understand productivity as bound up with specific and historical concepts of temporality and what Lucy Wajcman (2019, 332) calls the valorization of “time optimization”—in keeping with logics of accelerationism (ibid., 3) associated with technology and also in terms of productive labor that can never be understood in sentient terms insofar as it needs to be measured. Seen here, accelerationism is not only an ideology but also a “moral enterprise” (ibid., 4) to which algorithms, machine learning, and automation are seen to both ascribe and deliver. Perhaps here then we can get to the crux of why it is so difficult to think of Sadie-Loves’s work in these terms: because accelerationism excludes slow rhythmic repair work—even though it is enmeshed with these logics—which is derided as the ideal against which accelerationism (and productivity) work.
Yet if we follow Matzner’s (2019) argument that what algorithms are and what they do, and emerge in a “complex interplay” (p. 126), then we need to think at several scales and temporalities and speeds. As he goes on to argue, there are many “equally justified” perspectives on algorithms (ibid.), but perhaps more importantly as Introna (2016, 23) has suggested, this means that what is valued or understood about algorithms and automation depends on not just where but how you make “the cut.” For us, making the “cut” in the ways we have in this article, puts together lived and embodied experiences, power structures, smart architecture, automation, and visual culture to demonstrate just how embedded and embroiled algorithms and automation are in the material, in lived experience, in discourse, and in bodies. That we have come to this conclusion in a myriad of ways, at a range of scales and through different approaches, demonstrates for us just how deeply and problematically entrenched and highly political automation is.
Footnotes
Authors’ Note
Automation and Me was funded by Leeds International Festival 2019. This event was organized and led by Northern Sound Collective.
Acknowledgments
We would like to extend a particular thanks to Lucy Cheeseman and Lizzie Wilson who co-organized and supported facilitation of the event. We also extend our gratitude to those who participated, and so generously shared their expertise, practice, and creativity.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
