Abstract
Sensors have become embedded in all kinds of environments. Their ubiquity has prompted a boom in scholarly engagement centered around the effects of digitally interconnected sensor media and the data traces they ceaselessly produce. This research generally focuses either on theoretical or philosophical macro-perspectives—approaching the topic “from above,” so to speak—or it is ethnographic, “from below,” analyzing the practices of sensory (self-)surveillance, for example in the context of the quantified self-movement. In this article, we propose a situated, praxeological approach that combines both foci: expanding on the epistemically productive concepts of glitches and breakdowns, we follow three strands of what we call “counter-practices”—hiding in plain sight, dis/simulation, and the exploitation of sensor logics—through which we explore the inherent operations of technological sensing and sensemaking. We trace these counter-practices through historic and recent contexts, considering the interrelation of bodies, media and environments, and extrapolating epistemological conclusions that are meaningful for critiquing the codes, logics, and logistics of recent sensor-media societies. Asking how certain practices elude and subvert intended processes of sensor datafication provides a methodological blueprint for media studies and cultural theory that are faced with an algorithmic and technological situation that has rendered itself largely unobservable.
(Research on) Sensor media
It is often said that the digital revolution has already taken place (Beyes et al., 2017). The same holds true for sensors: the sensor revolution has already taken place, and we are now confronted with a situation in which sensors are everywhere, at least in Western or “Global North” societies. We are living in a postdigital era, as termed by Negroponte (1998), but also in a post-sensor era. In his book Sensing Machines, media artist and researcher Salter (2022) notes that in the year 2022, 30 to 50 billion sensors inhabited the earth, compared to only 7.8 billion people. We therefore already are interacting, wittingly or unwittingly, with a wide variety of technological sensors. Indeed, “sensors have become our mostly invisible companions” (Klimburg-Witjes et al., 2021: 23) in our everyday lives. And they are also part of logistics networks and Internet of Things architectures where RFID tags track commodities along global supply chains. This is the reason for media theorist Sebastian Scholz (2021: 135) to claim that, paraphrasing Friedrich Kittler, “sensors determine our situation,” often literally, since they pervasively and continually locate and track our positions and behaviors when we interact with “near-body technologies” (Kaerlein, 2018) such as our smartphones, duplicating our bodily movements as data traces (Hayles, 2009).
Smartphones, for example, already contain a variety of sensors, many of which are based on MEMS technology—an acronym standing for Micro Electro-Mechanical Systems. These include, among others, an image sensor, an accelerometer, a gyroscope, an air pressure sensor, a digital compass, an optical proximity sensor, an ambient light sensor, a humidity sensor, a magnetic field sensor, touch sensors, and a microphone (Parks et al., 2023). This manner of enriching media with sensors is not limited to smartphones, but also applies to watches, refrigerators, digital TVs, and every “smart” object embedded in Internet of Things infrastructures. This constitutes a new media convergence, an amalgamation of previously separate technologies. It follows that digitally interconnected media and sensors can actually no longer be separated from one another. This is meant both theoretically and technically: in the age of plastic-welded and glued-together cases, sensors have become almost inseparable from their host devices.
Sensorization applies not only to objects, but also to environments in which potentially everything of economic, political, logistical, constitutional, geographical, or ecological interest is datafied. This holds true for the measurement of environmental qualities such as storm forces and wind directions (James and Kane, 2008); radiation, noise, rain, and air quality (English et al., 2022); temperature (Starosielski, 2021); energy harvesting and management (Fahmy, 2020); for the datafication of potentially everything which is of agricultural interest (Paul et al., 2022); for the self-tracking of bodily functions and activities (Swan, 2012); and for the mobile sensing of complex urban environments carried out by autonomous cars (Hind, 2022), to give just a few examples. In light of these developments, scholars like Sprenger (2019) conclude that today’s environments are becoming media themselves, while media become environmental, albeit mostly unseen, at the same time. Sensors make environments algorithmic and, ultimately, sensitive (Hörl, 2017). Simply put, sensors as environmental media of communication “create and communicate data about the world and those in it” (Meikle and Bunz, 2018: 11).
Scholars in media studies, cultural theory and adjacent disciplines have been developing different perspectives on sensors. From a techno-political perspective, Salter (2022) investigates the ways in which we are tracked and surveilled by ubiquitous sensor media. This tracking is not limited to the public sphere but also takes place in our private spaces and so-called smart homes, where autonomous sensing machines such as Roombas move freely, and smart speakers are always prepared to intercept our private communication. Such a perspective reveals how new forms of surveillance capitalism are being developed and how they conceal themselves through sensors (Woods, 2018). Arguing that our world has become a computational planet, Gabrys (2016) explores from an ecological perspective how sensors are programming our earth, thus generating new technological environments—“technogeographies,” in Gabrys’ terminology—rather than just datafying already existing spaces. Along similar lines, Sprenger (2019) uses the current multiplicity of sensor environments to examine the possibility of designing artificial environments in a historical perspective. Whereas theoretical perspectivations of our technocultural condition have emphasized the crucial role of sensors as an emblematic sign of today’s data logistics and chrono-politics (Scholz, 2021), they also require asking anew which modes of justice and discrimination are induced by new data technologies. This is why Chun (2021) has investigated the entanglement of sensor data processing with new forms of “data discrimination” and algorithmically driven social polarization. Andrejevic and Burdon (2015) have even gone as far as proclaiming the advent of a “Sensor Society” in which the ubiquity of sensor technology fundamentally changes our understanding of surveillance and interactivity.
What these approaches have in common is that they tend to argue from a macro-perspective, which emphasizes sensor technology’s impact on the planet, specific groups of people or societies at large. They do this for good reasons, and their observations lead to well-founded conclusions. Nevertheless, we propose a more grounded approach that—in a reversal of perspective—takes ubiquitous sensorization as a fait accompli and puts the ways in which people elude, subvert and adapt to being sensed at center stage. In doing so, we expand on praxeological approaches to questions of sensing and datafication that have recently gained popularity.
Studying sensor media through practice
Traditionally, media studies—especially so-called German Media Theory—has revolved around the study of singular technological artefacts (Kittler, 1999). Now that most of our media exist as networked devices and computers have incorporated many of the functionalities formerly spread across a wide variety of individual media, talking about the singular medium-as-object has turned into a futile exercise. Consequently, praxeological approaches decenter the singular artefact, turning their attention toward practices and the operational chains in which technical objects become effective as parts of human practice (Borbach and Thielmann, 2022). This development within media studies (Schüttpelz et al., 2021) has been part of a larger “practice turn,” a re-orientation toward micro-sociological and ethnographic methodologies within the humanities and social sciences (Cetina et al., 2001) that understand observable human actions as their analytical starting point.
More recently, social interaction researcher Lorenza Mondada has shown that questions of “taste” and the negotiation of human perception in social situations can be understood through what she calls a “praxeological approach of sensing practices” (2021: 39). Taking a cue from her research, we will show that a praxeological approach also proves fruitful for analyzing sensing and sensemaking operations in situations that involve both technological objects and human subjects. Up until now, these situations have been mostly discussed from the point of the technological actors, emphasizing the translational processes by which things and actions in the physical world are captured or “datafied.”
There have been various calls to shift scholarly attention toward how the “practical accomplishment of datafication” (Burkhardt et al., 2022: 12) is being carried out both from “above and below” (Dencik, 2019), that is, by those who produce and maintain media technologies and infrastructures that facilitate datafication, and by those who are subjected to them. Focusing on sensor practices, we explicitly situate ourselves in the context of microanalysis that “allows for bringing actors and agency back into the analysis, something that is usually missing in macro-social analysis of cultures or societies” (Struck et al., 2011: 577). Human interaction with technical objects—sensors in our case—often reveals more about those objects than could be experienced through block diagrams, reverse engineering, or pure philosophy. We therefore argue for studying sensors in a praxeological manner. This allows us to reflect the transformation of the “Global North” into sensor data societies on both sociotechnical and everyday levels. Only through such an analysis can we hope to comprehend how media reconfigure “our situation,” as Kittler would say, when, first, we are confronted with a ubiquity of sensors not only in specialized environments, but in our daily lives; second, when datafication exceeds active media usage and continues whether we intentionally engage with sensors or not; and third, when the previously genuinely human practice of decision-making is now similarly carried out by sensor algorithms.
The epistemic value of counter-practices
Approaches that promise to decenter the media in favor of human practices and the ensuing operational chains always run the risk of losing sight of the technological objects that were the original impetus for media studies. This applies even more to our contemporary culture, in which media are increasingly being miniaturized, backgrounded, black-boxed, and driven by algorithmic processes that elude the human gaze. Media and cultural research is thus faced with a methodological problem. Although the advantages of a praxeological approach to technological objects—focusing on observable actions, that is—are, quite literally plain to see, some things necessarily remain unobservable in praxeological studies inspired by conventional sociological approaches. How can we analyze media and processes, which seemingly elude praxeological investigation while, and this is crucial, still utilizing the advantages of praxeology?
With this problem in mind, we propose to examine sensor datafication not through practices that align with the intended functionality of those sensors and their algorithmic processes, but rather, on the contrary, through practices that aim to circumvent or subvert their sensing and sensemaking operations. We call these specific bundles of practices “counter-practices.” Like everyday practices of sensor use, counter-practices and the sensing situations they are part of are easily observable. But to media scholars interested in the technological underpinnings of sensors, engaging with counter-practices offers a distinct epistemological advantage over engaging with other usage practices. This advantage lies in counter-practices’ ability to reveal information about sensor datafication processes. That is, observing counter-practices allows us to make statements that extend beyond the practices themselves, touching upon the logics and technological underpinnings of sensor data processing—a feat rarely accomplished through the analysis of simple usage practices. Getting at the technological object through studying counter-practice not only solves the problem of getting access to otherwise invisible algorithmic processes, but can also be understood as our attempt to reconcile the technology focus of German Media Theory with current practice theory.
Our approach is part of a digital praxeology oriented toward understanding actions in sensor environments that allows us to move from general critiques of big data, surveillant assemblages, or AI to a situated and nuanced understanding of datafication processes. As a methodology, attending to counter-practices answers several challenges commonly faced by researchers of digital technology. In the face of secretive companies guarding their data silos and closed-off APIs (Bruns, 2019), “follow the data”-approaches offer limited benefits when it comes to understanding situated sensing and sensemaking operations. On the other hand, micro-sociological studies of media—while providing excellent descriptions—often fail to prove epistemologically powerful enough for assessing and critiquing “our situation” in general. Studying how counter-practitioners trick sensing technologies—or fail to do so—allows researchers to tap into the technical skill of the actors while attending to their political-economic motivations at the same time. The unique relationship between counter-practices and sensor technology allows us to start with a praxeological account of a specific “sensing situation” and arrive at epistemic conclusions about our “situation” in the more abstract sense, that is, about the way in which sensor technologies reconfigure our economic and social relations.
Our focus on counter-practices can be situated in a longer history of research that is not concerned with the regular functioning of technologies, but with the ways in which they break down and get disrupted. Disruption has long been problematized as a central aspect of technical systems anyway. Claude Shannon’s “Mathematical Theory of Communication,” (1948) which has been popularized by Warren Weaver in Scientific American (1949), already outlined communication as a question of signal transmission that had to withstand “noise.” Shannon’s theory thus made it possible to address disturbance, or noise, as a central dimension of technological systems in general. In this sense, beyond engineering approaches, scholars from Science and Technology Studies (STS), and other disciplines have begun to understand moments of disturbance to be a heuristic principle for understanding technological systems and media as such. This view is encapsulated by Susan Leigh Star’s oft-cited dictum “[i]nfrastructure becomes visible upon breakdown” (1999: 382). Actor network theory (ANT) also has recognized the analytical value of disruptions for a humanities analysis of technologized societies. The technical object, a projector in Bruno Latour’s seminal example, only becomes visible at the moment of disruption: “The crisis reminds us of the projector’s existence,” Latour (1999: 183) stated. The reason for focusing on the technical object and its inherent logics, thus rendering it “visible,” is a moment of non-functioning, of disruption, of crisis.
While STS and ANT approaches draw on their observations of accidentally occurring crisis in technological systems, we shift the focus toward the active usage of sensors aimed at subverting their automated sensing and subsequent sensemaking processes. In STS, the heuristic promise of breakdowns is to unveil mostly embedded infrastructures and the technicity of supposedly invisible technological objects. In contrast, being a counter-practitioner presupposes awareness of the technological objects’ existence. The epistemic surplus provided by the crises induced by counter-practitioners lies not in merely revealing a sensors existence, but in surfacing its working logics in relation to human activity that occurs around and with it. Miyazaki (2019), in his eponymous article, calls for “taking back the algorithms” by acts of “making [them] affordable” (271), that is, by rendering “intentionally concealed, blurred, obfuscated, and protected processes of measurement, counting, control, and surveillance […] visible, understandable, accessible” (270–271). We posit that studying counter-practices is a valuable tool toward this end.
Paul Virilio reminds us that “when you invent the ship, you also invent the shipwreck” (1999: 89). In the same vein, inventing the sensor necessarily means inventing the glitch, the breakdown and the counter-practice. When “every technology carries its own negativity,” as Virilio (1999: 89) argues, we should be able to generate an epistemic surplus by attending to these negativities. Counter-practitioners intentionally negate or confuse processes of sensor datafication, but they can only do so because the systemic architecture of the targeted sensor system leaves open possibilities for specific disruptions. Like glitches, these openings have to be understood not as bugs, but as built-in property of the system, that is, as features (Benjamin, 2019). In their concept of “glitch epistemologies,” media geographers Agnieszka Leszczynski and Sarah Elwood note, that this understanding requires a sort of double-vision, a “‘queer orientation’ to glitches that stays with the counterintuitive tensions of the glitch as simultaneously systemic and interrupting” (2022: 363). Following their example, we advance counter-practices not as an answer to the ontological question of what a sensor is, but to the epistemological question of how we can understand the complex entanglements between sensor, sensed subjects and the processes of sensing and sensemaking themselves.
The notion of the glitch is also instructive when theorizing the figure of the counter-practitioner. Being part of both the situated sensing operation and of the wider socio-technical structure in which the sensing takes part, counter-practitioners can be considered systemic. At the same time, they act disruptively by creatively utilizing the wiggle room found within the system’s architecture. This understanding of counter-practice and counter-practitioners is already laid out in what can be considered one of the founding texts of practice theory. In “The Practice of Everyday Life,” de Certeau (1988) theorizes everyday practice through the dichotomy between tactics and strategy. Everyday practice is tactical, as it is carried out in opposition to—and therefore has to adapt to—powerful economic and political entities that act on them on a strategic level. This distinction between strategy and tactics not only inspired later theoretical work, like the notion of “tactical media” (Raley, 2009) used to describe flexible and often short-lived acts of resistance that turn media back on themselves through hacking, art and activism, but also draws attention to the “warlike” (de Certeau, 1988: xix) dynamic between strategic rule-setting and tactical disruptions. We posit that this dynamic also characterizes counter-practices aimed at subverting the algorithmic sensemaking faculties of sensor media.
In the following subsections, we will showcase a variety of interventions that exploit the rules of a given sensing system in a tactical fashion. Our aim is not to produce a clear-cut and exhaustive typology of (sensor) counter-practices. Instead, we identify three principles found across a variety of them: hiding in plain sight, dis/simulation, and the exploitation of sensor logics. We outline these archetypes and their relationships through a series of case studies, drawing on a wide range of materials such as media art exhibits and their theoretical underpinnings, historical accounts of camouflaging practices and ethnographic work on the day-to-day lives of delivery drivers. The archetypes synthesized from our examples share certain characteristics, overlap and are commonly found together, often within the same operational chain. As we will show, counter-practices adapt to the specific sensorium they target. Consequently, they may look wildly different against the backdrop of recent sensor technology when compared to earlier periods, when they mainly served to confuse human perception. Under these conditions, identifying and tracing key motifs becomes a vital method in describing the historic trajectories of counter-practices without falling back into a historiography of technological artifacts.
Hiding in plain sight
Media artist Adam Harvey hides human faces from facial-recognition systems by designing garish patterns of makeup that he calls “Computer Vision [CV] Dazzle” (see Figure 1). Harvey describes the process of finding suitable looks to evade sensor detection as both playful and activist, as it is about productively exploring and testing the specifics of the sensor software to find vulnerabilities in the algorithms. This makes the counter-practice highly individualized and situational: dazzling is difficult to generalize across different people and algorithms, as it requires an original rather than a replicable look. Nevertheless, Harvey gives some universal “style tips” that help to evade detection in various sensor-media environments: “1. 2. 3. 4.
“CV dazzle Look 5.” Commission for The New York Times Op-Art. 2013. © Adam Harvey 2013. From Adam Harvey (ns).
Faces are nowadays detected by camera sensors and subsequent image-data processing. Once a face has been detected within the image, the next steps in the operation chain usually involve classification and, ultimately, identification. The question of whether a human face can be found in a video image has thus become a question of probabilistic determination: of quantitative image data and its algorithmic processing. If there are faces in certain images of digital cultures is no longer solely a question of human perception. What a face is, and what it is not, has also become the result of an algorithmic operation in digital space that is characterized by certain parameters and generates a probabilistic assertion as to how likely it is that certain image data refers to a face—or not. There are only two ways to avoid this potentially non-consensual sensor detection of one’s face: hiding it completely, for example, behind a mask, or modifying it in a way that throws a wrench in the algorithms. Accordingly, there are special ways of disrupting this automated facial recognition by computer vision and causing targeted misinterpretation by computers. The sensor counter-practice of dazzling therefore consists of modifying one’s own face to deliberately deceive statistical assumptions that are algorithmically inscribed in automatic facial-recognition programs. As Harvey attests, common image recognition algorithms detect patterns that typically correspond to certain regions of human faces and can be intentionally masked by applying his styling tips (Harvey, 2021).
An early form of dazzling by Harvey, which he conceived between 2010 and 2013, was directed against the “Viola-Jones” face-detection algorithm. Viola-Jones operates by detecting characteristic features based on simple black-white contrasts, rendering the algorithm susceptible to racial bias. Obfuscating symmetrical regions like the eyes, switching up typically light and dark regions of the face, and using geometric patterns that not commonly found in faces, CV dazzle made it difficult for Viola-Jones to properly function. The counter-practice is therefore based on non-technological optical forms of facial modification—hair styling and makeup—in order to remove one’s own face and thus one’s own person from sensor recognition, monitoring, and control. Although the Viola-Jones algorithm became technologically obsolete around 2015–2016, it is surprising that a rather simple counter-practice could fool the entire security infrastructure of facial recognition of its time with very simple forms, as Harvey describes: “All of sudden a key technology in the previously infallible post 9/11 security apparatus could be foiled by makeup and hairstyles” (Harvey, ns). Counter-practices might be forms of media art, but as such they are media-pedagogically oriented: one learns something about the algorithmic specificity of the sensors they address simply by looking at the esthetics of face dazzling, for example.
This counter-practice and the face-recognition algorithm Viola-Jones are in a certain way co-constituted. In order to deceive the algorithm, its functionality must be known in order to develop adequate counter-practices. Although the counter-practice establishes an esthetic of its own, this is not the decisive factor for us. What is essential is that dazzling operates at the algorithmic limits of software sensemaking and thus unfolds a political-activist potential that starts at the level of the logic of digital technology itself. Since automated facial recognition using sensors and data processing is only the first step in an algorithmic chain for facial analysis, dazzling promises to remove people’s faces not only from the sensor logic of control, but also from subsequent datafication, such as emotional analysis.
Computer vision dazzling does not address human perception but is specifically dedicated to algorithmic sensemaking. Significantly, these dazzle patterns can be clearly and easily recognized as human faces by human observers, but not by cameras. The logic of dazzling is not to try to make one’s own face invisible, but to irritate algorithmic processes of recognition through a form of obscurity through hyper-visibility. Turning their face into a glitch, the dazzling counter-practitioner becomes both systemic and disrupting—their makeup can act disruptively precisely because it addresses the system on its own terms. Dazzling thus follows a well-documented historical tradition of military subterfuge of the same name. Activist artists, such as Georgina Rowlands and Evie Price of the so-called anti-surveillance makeup movement, make this connection explicit: “The project [computer vision dazzle] was inspired by the dazzle ships used in World War [1 and 2] where warships would be painted with bolds – kind of black and white geometric shapes to obscure them out at sea. So we have the same kind of idea: hiding in plain sight with this kind of old graphic aesthetic.” (Sharma, 2020)
The dazzling of warships in the First World War (see Figure 2), usually associated with the British artist Norman Wilkinson, did not serve to conceal ships (just as Harvey did not propose a concealer for Dazzle make-up), but rather to make the practice of recognition more difficult; namely the targeting of ships with telescopes, which was important to indicate their exact course and speed in order to shoot them. In the context of sensor-media technologies, the phrase “hidden in plain sight” attains a double meaning. While to the eyes of a human observer an object may indeed be “in plain sight,” it can be completely unrecognizable and thus hidden for the machinic gaze.
The French cruiser Gloire, probably during exercises off Africa, 1943/1944. Note how the dazzle pattern obscures the direction in which the bow faces. U.S. Navy photo 80-G-K-1208.
Had the practice of camouflaging ships in dazzle paint been aimed at the sensorium of human observers, recent dazzling practices like anti-surveillance masks, makeup, and haircuts are geared toward deceiving sensors and the image recognition models fueled by their data. This highlights how counter-practices are always tailored to target specific forms of sensing and sensemaking. Once the mode of sensing changes, the counter-practice has to follow, or it becomes obsolete. For example, ship’s dazzle paint became obsolete once the trajectory of enemy ships could be determined via radar, an electromagnetic sensor medium that did not rely on the visual ability of a human observer. In much the same fashion, Adam Harvey’s CV Dazzle only worked against older image recognition methods and fails to hide faces from newer models.
Dis/simulating presence
Dazzle patterns are not designed to outright deny the existence of the objects they are applied to, but to obfuscate certain properties of said objects, rendering adversarial processes of recognition and classification more difficult. In contrast, there are counter-practices used in situations where it is deemed advantageous to feign one’s presence or absence.
In most major cities, platformized delivery services draw on a host of precarious gig workers standing at the ready at any time of the day. For these platforms, optimizing the time to delivery starts with the algorithmic choosing of a suitable worker. In this process, proximity has emerged as a central category, i.e., platforms place orders with the delivery driver closest to the shop or restaurant from which the customer has ordered. This in turn incentivizes delivery workers to be, or at least appear to be, as close to popular businesses as possible in order to be presented with valuable orders. To enact this ordering regime, platforms take our digital near-body technologies up on their promise to never leave our side, to be “always on and always on us” (Turkle, 2008). They tacitly assume that the geospatial position of a personal navigation device and its user are one and the same. But, as anybody who has used a GPS tracking app to search for a misplaced device can attest to, this congruence cannot be taken for granted. This is an often overlooked but crucial aspect of mobile media: even without being attached to the body of the user, these sensors can and regularly do generate forms of independent agency.
In the case of food delivery, “skipping the line” to get the next order could be achieved by physically placing one’s device in the vicinity of the shop or restaurant (Qadri, 2021). But there are more subtle ways to subvert the positional coupling between device and user: Specialized “GPS spoofing” applications act as a go-between the smartphone’s sensors and the platforms themselves, feeding the latter positional data that corresponds to locations favored by the order-sorting algorithm while the actual driver can be in an entirely different location. The difference between the geophysical location of the subject and the machinic perception of the location of the technical object that acts as a stand-in for the user renders GPS spoofing productive as a counter-practice. While these practices introduce false data into the platforms’ calculations to gain a personal advantage, they notably do not subvert the system itself (see Figure 3). They only exacerbate the competition between drivers, who now have to deal with the proprietary software applications needed for spoofing and must acquire additional technical know-how in order to make a living.
Screenshots of apps used by Indonesian delivery drivers to spoof their GPS location to increase the likelihood of receiving orders. The app shows the locations of restaurants and shops so that delivery drivers can choose where to simulate their presence based on proximity to the businesses they want to receive orders from. From Qadri (2021).
In a similar fashion, users of the popular location-based game Pokémon Go, which has players moving to specific real-world places in order to catch virtual “monsters” on their smartphones, spoof their locations to simulate presence at locations that promise encounters with rare and therefore desirable Pokémons. But places that are relevant to the gameplay are not scattered through geophysical space at random. Rather, the developer of the game, Niantic, lets business owners pay to place “sponsored Pokéstops” at a position within the game world that corresponds to the physical location of their businesses in the hopes of turning passing players into customers. Under these conditions, spoofing actively undermines Niantic’s economic interests, resulting in the practice becoming prohibited under threat of being permanently banned from the game.
Following Friedrich Kittler, we call these counter-practices of dis/simulation, since Kittler related dis/simulations to the double logic of affirmation and denial: “In simulations (…), this more than aesthetic process, negation is always already built in (…) [T]o simulate is to affirm what is not, and to dissimulate is to deny what is” (Kittler, 1989: 63–64, our translation). GPS spoofing is always an act of simultaneous affirmation and negation, where presence at a desired location is simulated while presence at one’s real, algorithmically disadvantageous position is dissimulated. Here, Kittler’s apt description of simulation as a “more than aesthetic” process takes on a new meaning. The spoofing of sensor technology is not concerned with human aisthesis, but with “more-than-human” (Whatmore, 2006) perception: it acts on the real by manipulating the digital platform, which can only sense its users’ positions via the stream of location data provided by the users’ own devices.
Feigning the presence or absence of people and objects has a long tradition in military contexts. While fictional accounts, like the use of the wooden horse during the siege of Troy, or Macbeth’s downfall at the hands of an army hidden behind boughs and foliage, are set in earlier times, the systematic development of camouflaging military operations begins in the nineteenth century, under the impression of the ever-increasing accuracy of rifles and cannons, and really comes to the fore with the advent of aerial reconnaissance in the First World War. To elude the watchful eyes of the newly airborne observers, units of so-called camoufleurs were tasked with masking uniforms, infrastructure, and weaponry in a way that rendered them indistinguishable from their natural surroundings. By taking cues from nature, but also from modern art movements such as Cubism—painters like Paul Klee and Franz Marc joined the ranks of the camoufleurs, eager to support the war effort—the camoufleurs’ goal was to call into question the relationship between figure and ground, or to put it in the language of signal processing, to adapt an object’s appearance “to the background noise of its spatial environment” (Bucheli, 2021: 13, our translation). These efforts at feigning absence were not limited to two-dimensional patterning. In an act of mimicry, French artist André Mare famously disguised observation posts as trees—a camouflaging practice that stood the test of time, as the photographic work of Dillon Marsh (ns), who documents modern cell phone towers disguised as trees, attests to (see Figure 4).
Cell phone towers disguised as trees in Cape Town to appear as part of the natural environment, as captured by photo artist Dillon Marsh. Screenshot taken from Marsh (ns).
Camouflage operates under the assumption that environments and the objects and people within them will be seen, usually from above. The practice then boils down to confusing the perception of the observers as best as one can. As Allan Sekula has put it in his seminal work on wartime aerial photography, “[w]ith the development of camouflage, a low-level language game evolved in which the indexical status of the sign was thrown into question, thereby inflating the suspicions of the photo-interpreter” (Sekula, 1975: 28). Another move available to the players of this “game” was the creation of wholly new signs, a practice exhibiting a logic akin to Kittler’s understanding of simulation: To feign the presence of troops, intricate setups of decoys and dummies ranging from singular vehicles to entire armies were employed such as the allied so-called Ghost Army in World War II (Beyer and Sayles, 2015).
Conversely, important industry buildings could be concealed by simulating a more harmless environment on top of it. Following the attack on Pearl Harbor, the Douglas Aircraft Company rooftop in Santa Barbara and the rooftop of Boeing’s B17 bomber factory in Seattle, Washington, were camouflaged by erecting nearly 26 acres of simulated suburbs on top of them, complete with actors and actresses strolling the fake streets, or hanging fake laundry to dry (see Figure 5). As in the case of GPS spoofing, this is affirmation and negation at the same time. If the camouflage proves successful, the presence of the factory is negated while the presence of the non-existent suburb is affirmed within the perception of the (enemy) beholder.
An aerial view of Boeing plant 2 in Seattle, nicknamed “Boeing wonderland” due to its elaborate camouflage. The photograph clearly shows the “street grid” of the sprawling faux suburb that covered the factory. Figure from Warfare History Network (2013).
A spoofing practice popularized in 2019 is the addition of so-called adversarial patterns to captured scenes in order to deceive automatic image recognition systems. One possible goal of this counter-practice is the dissimulation of objects and persons that are actually present in the scene, be it bananas, people, or airplanes. Likewise, adversarial patterns can be used to simulate objects which are not actually present within the sensed environment, like patterns that generate fake license plates (see Figure 6), stop signs, or walls to keep autonomous vehicles from moving. These practices not only foreground the sensor algorithmic principles fueling AI-based machine vision, but also follow the historic trajectory of simulation and dissimulation that includes dummy-objects like fake-tanks and camouflage to hide objects from sight, often in military contexts.
The successful sensor spoofing of automated license plate readers (ALPRs) with a shirt print: license plates on the shirt are identified by the sensor algorithm as if they were actual plates on cars. Figure from Adversarial Fashion (ns).
Exploiting sensor logics
In 2023, citizens of San Francisco took to the streets, placing traffic cones on autonomous cars that were stopped at stop signs and red lights. If such a cone is placed on the hood of such a car, the algorithmic sensor logic of the cameras on or in the cars identifies this cone as part of a roadworks or road closure, for example, causing the car to stop driving and park until the cone is removed. This example foregrounds that the car's perceptual topology follows a pars pro toto or proxy logic by which a situation is extrapolated based on singular recognized elements. The car senses a construction site, and stops accordingly, even if it recognizes only a single pylon. Thus through the placement of a single signifying element, the whole reading of the situation changes.
The counter-practice was a protest against the increase in autonomous vehicles on the city’s streets. Through cone-placing, the activist group “Safe Street Rebel” opposed the industry’s efforts to automate cab operations and establish so-called “robotaxis.” This simple positioning of objects of attention on self-driving cars fundamentally raises the issue of the right to public space and the “right to the street”: producers of self-driving car technologies, despite their sometimes-considerable shortcomings and the chaos they often cause on public roads (Zipper, 2022), were previously granted the right to carry out tests on the streets of San Francisco, without the public having agreed to this technological intervention in their space. Cone placement thus shows how major social criticism, which affects urban space as such, can be articulated via the irritation of sensors. Cone-placing illustrates how elaborate sensor technologies can be successfully disrupted by very simple analog procedures. As a “low-tech” method, it shows that counter-practices are the result of empirical investigations into the logics of sensor algorithmic sensemaking. After the group noticed that self-driving cars from Cruise or Waymo brake and freeze at the sight of traffic cones, it was obvious to use this disruption in an intentional and activist way as a form of protest.
In 2020, citizens of Berlin were treated to a rare sight when a usually congested street remained virtually car-free for an extended period of time thanks to a counter-practice employed by artist Simon Weckert that seems equally banal as that of cone-placing. Weckert obtained a total of 99 smartphones from friends and rental companies, and pulled these in a handcart through selected streets of Berlin (see Figure 7). The smartphones were active, they had working SIM cards, and they all used the Google Maps app. Moving these phones around the city, generating GPS sensor positions, and thereby sharing their GPS position data with Google, Weckert caused Google Maps to display a traffic jam. In other words: the accumulation of sensor data in conjunction with the phones’ movement pattern, which potentially corresponded to that of a traffic jam (a certain number of GPS receivers in a certain street area with a movement speed similar to the speed of cars in a traffic jam), actually created a “traffic jam”—without actual cars, that is: a digital traffic jam, a data jam, that became visible as a red marker on Google Maps, causing the app to show users traffic routes other than the supposedly congested ones.
Impressions of Weckert’s media art project of a synthetic traffic jam production. Screenshot from Weckert (ns).
Weckert was inspired by a personal experience. Years earlier, his smartphone had displayed a traffic jam during a May Day march in Berlin, even though there was not a single car on the road. Apparently, the large gathering of slow-moving people with smartphones whose GPS data were sent to Google Maps resulted in a traffic jam being mapped within the application. For Weckert’s activist media art project, he needed only to adapt what he had seen in order to generate a traffic jam as an individual: this time, he needed only 99 smartphones, without the people. Weckert was thus able to demonstrate the inherent working logic of Google Maps without assuming any knowledge of algorithms, software, or digital data processing on the part of the audience he reached, via, for example, YouTube.
Discussion
Counter-practices come in various shapes and forms, ranging from the use camouflage and dummies in warzones, activist interventions and artistic statements against sensor-based surveillance, to what could, in a nod to de Certeau, be called counter-practices of everyday life: day-to-day practices carried out in a tactical fashion with the goal of gaining an advantage against powerful rule makers, like the sensor spoofing aimed at tricking game companies and delivery platforms. Each of these counter-practices presented here could be rightfully called singular. This is because counter-practices are difficult to stabilize over longer periods of time. Due to the “warlike” dynamic they are part of, they rapidly lose their usefulness once sensing equipment and sensor algorithms catch up to them. The fact that they appear historically ephemeral, however, does not make them any less relevant for a situated analysis of contemporary culture. It is precisely their ephemeral nature that renders an analysis of counter-practices all the more urgent, as they would otherwise be forgotten just as quickly as they emerged—as supposedly marginal micro-phenomena of history, whose documentation (to date) has been primarily taken up by journalists rather than media scholars.
The practices we described share a common orientation toward an “other,” a mode of perception that acts as their target. Following Offert and Bell, we employ the term “perceptual topology” to describe a machine-vision model’s “specific way of representing the visual world” (2021: 1133). While biases introduced into machine-learning models via skewed data sets have been at the center of scholarly and journalistic critique of these models, the specific perceptual biases of technological actors received less attention. Acknowledging the existence of different perceptual topologies raises the question of what perspective on the world these technological actors exhibit. Although different actors live together in a shared physical space, they live in different environments or umwelten shaped by their sensorial “makeup,” as Jakob von Uexküll (1934) notes in his seminal work on the topic. As it turns out, this applies not only to the biological creatures that inspired von Uexküll's original research, but also to humans and sensors.
What our case studies show is that perceptual processes exhibit certain emergent qualities that render them difficult to understand and to predict through exclusively engaging with physiological descriptions, technical instructions or programming code. Rather, certain peculiarities of perception may only surface in the situation, in practical application, that is, in sensing real world phenomena. Descriptions of machine vision processes regularly concede that “[t]heir cameras work like our eyes do, but the space between the image that a camera captures and actionable information about that image is filled with a black box of machine-learning algorithms that are trying to translate patterns of features into something that they’re familiar with” (Ackerman, 2017). Here, sensemaking is marked as the site where a gap opens up between the perceptual subjectivities of humans and machines. Although a face modified with makeup according to Adam Harvey’s recommendations can still be easily identified as a human face by a human observer, some image recognition algorithms are unable to do so. In these cases, there is no face in the environment as perceived by the sensor algorithm; while for human observers, however, there evidently is. The same applies to our other case studies of sensor datafication: From a data-centric perspective, the GPS-spoofing delivery riders are at the optimal spot to receive the next order, and the agglomeration of data points produced by Weckert’s artwork does constitute a traffic jam. The systems involved are simply not equipped to discern between an actual, physical traffic jam and a mere “data traffic jam.” All these forms of counter-practice therefore show that a top-down perspective is insufficient when it comes to sensor datafication. Rather, we have to account for the situatedness of data production to really see what is going on.
The heuristic potential of attending to Weckert’s artistic practices lies not in surfacing the readily understood sensing mechanism—each smartphone receives a GPS signal that enables it to locate itself accurately within the geographic framework provided by Google Maps—but in providing a glimpse into the sensemaking, that is, into the ways in which Google evaluates the ensuing situation. Starting again with the assumption that a smartphone running the vehicle-navigation function is actually located within a moving vehicle, we can employ Weckert’s methodology to discern what the concept of “traffic jam” means to Google: how many smartphones have to be involved, how slow they have to be moving, etc., before the Maps application starts to reroute other traffic participants. The same applies to the other counter-practices we have considered. For example, while Adam Harvey’s dazzle make-up tips certainly have an esthetic dimension, their epistemic relevance lies in the fact that they are revealing beyond this esthetic dimension: They literally show what specific image-sensing algorithms consider to be a face and what exactly irritates this interpretation of an image as “containing a face.”
Counter-practitioners like Simon Weckert, Adam Harvey, and the cone-placing activists share a political message, their stated goal being to explore the relationship between technology, people and urban spaces, as well as the possibility of reappropriating said spaces (Barrett, 2020). They manage to uncover elements of both the data politics and data biases of the systems they target, but without acting at the level of the code itself. This is an important characteristic that connects contemporary counter-practices in sensor environments with earlier forms of counter-practices: Although counter-practices that deceive sensors have effects in the digital space, they are genuine practices in analog environments, such as putting make-up on one’s face, dragging smartphones in a handcart through the streets of a city, or placing cones in front of autonomous cars. The McLuhanian message of counter-practices, as simple as they may appear at first glance, is the same as Weckert’s summary of his media activist project: “small, simple hacks can break down, [sic] complex systems” (Weckert, 2020)—in order to understand, critique and ultimately modify them, one might add. If we are to “take back the algorithms,” as Miyazaki (2019) puts it, counter-practices like Simon Weckert’s are our best bet for gaining insight into the black-boxes of Google Maps and co. and understanding the real-world effects that they have on local environments and communities.
Consequently, the counter-practitioners mentioned above are not only concerned with disrupting sensor logics in specific situations, but also with questioning the societal re-orderings that manifest themselves with and through sensor technologies: surveillance, automation, autonomous decision-making, workplace rationalization, etc. From this perspective, counter-practices emerge as instruments for enacting specific forms of social criticism through the disruption of technology. In doing so, they connect to older, more radical forms of protest based on the same principle, for example, the sabotage of machinery by the Luddites or their twenty-first-century Neo-Luddite counterparts (Mueller, 2021). Acting tactically and taking an active role in shaping how the sensor game plays out, counter-practitioners can formulate differentiated critiques that go beyond the emphatic negation signaled by outright destruction of the technology in question.
Operating productively at the epistemic boundaries of sensor datafication, counter-practices foreground the difference between algorithmic and human sensemaking. In doing so, counter-practices can reveal the blind spots of sensors to us—an effect that is especially striking when the machinic blind spot happens to be glaringly obvious for human observers. Consequently, Lingel (2021) theorizes dazzling as a practice of queer counter conduct that relies on strategies of hypervisibility to elude and challenge the hegemonic gaze, especially in the context of everyday digital surveillance. While “[d]azzle camouflage forces an encounter with and acknowledgment of power dynamics underlying surveillance,” (Lingel, 2021: 1119) it forces us to reckon with the technological basis of said surveillance at the same time. In counter-practices like dazzling we (en)counter the technological other of sensor perception. Under these conditions, a new regime of multiple visibilities comes to the fore: Dazzle face painting, although characterized by its hypervisibility for other humans, removes the human face from recognition by sensors and subsequent data analysis. In this way, the practice of dazzle make-up is uniquely positioned: It acts as a hypervisible “glitch” that queers everyday esthetics and reminds human observers of the technological surveillance that surrounds them. At the same time, it evades the eye of surveillance devices altogether.
While this multiplication of visibilities seems to be a rewarding subject for selective interventions and media art—Adam Harvey’s activist work, for example, has been shown several times at art exhibitions such as re:publica in Berlin in 2016—it seems hardly feasible to keep up with what the ever-changing combinations of sensors and algorithms present within any given sensor environment. Acknowledging these practical hindrances, Harvey nevertheless emphasizes the critical insights gained through his work at the operational limits of sensors—and by extension, the epistemological potential of counter-practices: “When we see the world through computer vision systems, we begin to see the flaws and limitations of a world mediated by algorithms, data, and sensors” (Harvey, 2021).
Conclusion
According to Norbert Wiener (1989 [1950]: 24), the declared task of cybernetics—the “theory of messages”—was the analysis of the “machine on the basis of its actual performance rather than its expected performance.” And, as Wiener (1989 [1950]: 24) remarkably adds, this analysis “involves sensory members” who are able to gauge the performance. Wiener thus emphasizes both the situatedness and the importance of perceiving subjects for the analysis of technical systems. We can take two things away from Wiener’s remarks: On a fundamental level, counter-practices aimed at specific modes of perception can only be tested in confrontation with the respective sensorium, as the example of simulated targeting tests involving dazzle-camouflaged ship models (see Figure 8) show. Through these confrontations, counter-practices act as sites for negotiating what, in any specific situation, is considered as “signal,” and what is discarded as “noise”—a distinction that may rapidly shift, once the potentially open-ended arms race between the development of counter-practices and sensor technologies unfolds across a variety of perceptual topologies. It follows that counter-practices not only allow insight into various forms of machine vision, but also open avenues for questioning the notion of supposedly “normal” modes of sensing and, ultimately, perceptual ability or disability. This understanding is supported by our historical findings: Several reports from the Second World War have suggested that a specific type of vision “deficiency,” dichromacy, proves advantageous in detecting color-camouflaged objects (Morgan et al., 1992), allowing dichromatic observers to filter signal from noise when faced with a counter-practice that would have rendered other observers unable to do so.
A model vessel covered in dazzle camouflage is being put to the test (Kaempffert, 1919).
The second takeaway stems from Wiener’s idea of evaluating machines by their perceived performance. From this perspective, counter-practices emerge as a specific form of testing, a methodology for analyzing technical systems by productively subverting their expected performance. Following Noortje Marres and David Stark, we argue that in “today’s total test environments,” where ubiquitous sensor media facilitate that “testing can be taking place without one’s knowledge at any time” (Marres and Stark, 2020: 437), counter-practices have become an important part of the “testing ecology.” As “playful and situated practices, by which opaque media technologies are appropriated” (Gießmann and Gerlitz, 2023: 12, our translation), counter-practices provide an avenue for critically engaging with technology’s sense and sensemaking capabilities. Both from Wiener’s perspective of putting the actual technology to the test, and from the point of view of a sociology of testing, where the social is itself tested by technology, counter-practices are, and have always been, forms of counter-testing.
As a tool for producing glitches in the socio-technical fabric of society, counter-practices are intrinsically political. They are not merely carried out within, but also produce insights about sensor and data environments, rendering them “affordable” (Miyazaki, 2019) in the process. Since we are confronted with a social situation in which we are constantly the objects being measured, recognized, and located by sensors—even in our everyday lives, in our public and private spaces—it is counter-practices that allow us, at least partially, to regain what is now being discussed as personal “data sovereignty” (Hummel et al., 2021). The Foucauldian question of What is Critique? (1996), which Foucault related to analyzing how we are governed, has therefore to be transformed into questioning the procedures of how, via sensors, we are datafied—and how we can subvert these processes.
Regardless of the specific form of protest against recent sensor surveillance technologies, counter-practices reveal insights into the ways in which sensors make sense of their surroundings, and, symbolically speaking, they also provide insight into the language of algorithmic sensor societies. Counter-practices can be understood as forms of symbolic interaction which communicate or intentionally miscommunicate with sensors. This makes counter-practices so exciting for cultural theory—in order for them to be effective, their practitioners must have first empirically analyzed the language of sensors in order to specifically disrupt them in a subsequent step: Counter-practices require knowledge of the codes with which sensors operate. Finding out what could produce a counter-practice for any particular sensor is simultaneously the empirical analysis of the sensing and sensemaking processes and algorithms of those very sensors. This is fundamentally a symbolic practice because sensors “read” in operational chains of data, processing landscapes, people, faces, expressions, voices, and movements through numbers. It follows that sensor “perception” is not based on the same pictorial logic as human sensory perception. Sensor perception is fundamentally algorithmic and shows—in a re-actualization of an axiom of German Media Theory—that the digital image does not exist (Hagen, 2002; Pias, 2003), especially for the sensor media themselves. Counter-practices evidence this specific non-seeing of sensors. In the case of Weckert’s traffic jam, Google Maps’ algorithm is “blind” to the fact that there are actually no cars on the street. In the same manner, dazzle makeup hides faces that would be easily identifiable for human observers from being detected by camera sensors. This example, in which simple geometric manipulations derail the sensing process, foregrounds that faces actually do not exist for sensors that only process abstract assemblages of color, brightness, and position data. Sensors do not perform facial recognition in the truest sense of the term, but analyze a multiplicity of data patterns that allows judgments as to probable presences. “The truth of the technical world,” in the spirit of Kittler (2013), is different for sensors than it is for people. This relationship to the world of sensors shows that there is not merely one monolithic technical world, but at best a plurality of environments, corresponding to the sensory equipment of human and non-human actors.
In line with Seb Franklin (2015: 107–134), who understands the logic of the digital as fundamentally symbolic rather than pictorial, our investigation supports the claim that while there may be images for humans, there are not for sensors. Sensors dramatize the question of perception anew, or establish, paraphrasing Virilio (1989), a new “sensor logistics of perception” that is less about questions of operative imagery and more about data chains. Perhaps we should therefore abandon the concept of the image in relation to sensor environments. Jussi Parikka hints at this in his book, Operational Images, yet ultimately still insists on the concept of the image: “Treating the world as an image beyond its pictorial qualities is one key instance where operations of epistemic value take place—but there are multiple operations already in place in order to produce such images” (Parikka, 2023: x). If we follow Ian Bogost’s “alien phenomenology” (2012) and take the ways in which non-human actors perceive the world seriously, we would have to habituate ourselves with a way of thinking that does not refer to images but to data patterns.
Technology that recedes into the background while becoming ever-more miniaturized and encapsulated raises the question of what critical engagement with technology will look like in the future. An epistemologically oriented media praxeology, as we have suggested here using the case study of sensor counter-practices, could be a solution to pressing methodological problems. Counter-practices are technologically “deep.” They teach something about sensing devices and sensor media cultures, even though we are “only” looking at practices, not at block diagrams. An investigation of counter-practices provides insight into the working logics—the “code,” symbolically speaking—of sensor technologies, while at the same time epistemologically illuminating our cultural and societal situation.
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Collaborative Research Center “Media of Cooperation” [Deutsche Forschungsgemeinschaft (DFG)—Project number 262513311].
