Bringing experiments down to earth

Abstract

A commentary on Thao Phan’s ‘Testing-in-the-wild: innovation nationalism and the colonial dynamics of new technology testbeds’.

With the 2022 release of ChatGPT, hundreds of millions of us became aware that we were part of a social experiment. We were invited to regard ourselves as early adopting pioneers, but many felt less like scientists and more like research subjects. In smaller ways, public experiments currently taking place with self-driving cars and drones are raising profound questions of governance, not just of the technologies themselves but of the processes that are bringing these technologies into the world.

Phan (2025) sets this type of contemporary experiment against a historical backdrop of colonial exploitation. Her focus is Australia, where the language of science was used to both justify and to condemn the practice of ‘transporting’ people as part of Britain's imperial experimentation. But we might also think about abusive experimentation from the mid-20th Century Tuskegee Syphilis Study to Pfizer's much more recent drug trials in Kano, Nigeria (Wollensack, 2007). Recognition of these outrages has contributed to the experimental rules we now have. When it comes to 21st century innovation, it is not clear which rules might apply. Where experiments, testbeds and living labs are built in the service of an innovation economy, they are often designed to evade accountability, and they can be understood as experiments in rule-formation (cf Wynne, 1988).

Beneath Phan's big story of innovative power sits a smaller one. The story of how a Silicon Valley upstart – Wing – moved into and then out of the town of Logan tells us so much about the new politics of testbed innovation. Wing had initially scoped out the Canberra suburb of Bonython, near Australia's seat of Government, but had found rather too much democracy for their tastes, in the form of local protesters and civil servants who knew their way around the system. The move to Logan, Phan explains, was about finding somewhere with enough financial capital to see if the business could work, but not so much political capital that local opposition would jeopardise the experiment.

Logan was down on its luck and desperate for opportunities. As Phan describes, the town did not have the deepest pockets, but it had enthusiasm and a multicultural diversity that offered experimental possibilities. This was not an old-fashioned colonial experiment with bodies that were seen as cheap, land seen as unoccupied and human rights that were seen as malleable. In Logan, the town's diversity became a feature rather than a bug, but not one that local citizens could use to their advantage. This real-world experiment was judged to be more real than some other ‘smart cities’ (see Sadowski and Pasquale, 2015), and a better scale model for the country and then the world. The paradox is that this unusual place was chosen by Wing to stand in for Anytown, USA, providing experimental findings that could be generalised and exported.

In an excellent film made during her fieldwork, Phan interviewed enthusiastic early adopters and local shopkeepers, who expressed their enthusiasm for the idea of a drone company enabling rapid deliveries from local businesses to local customers. ¹ In a process that has now become familiar with tech platforms, characterised by Doctorow (2025) as ‘enshittification’, the company then turned on its users and targeted economic opportunities with bigger, multinational players. The enthusiasts described their disappointment on seeing the company's priorities change. One shopkeeper realised, with real pathos, ‘we were just like a stepping stone… we were just a guinea pig’. He said that Logan's function for the company was merely ‘to prove to the bean counters back in the US that we can make this work’. The local economy had been picked up, repackaged for the purposes of the experiment and then dropped again. Logan briefly became ‘the drone delivery capital of the world’, but then the circus moved on to another town. The language of experimentation allowed the company to pretend that this was a mere test, but for the customers and local businesses that had bought into the promise and begun to reconfigure their lifestyles and livelihoods around it, the stakes were all too real.

Phan's paper fits into a line of work in science and technology studies that problematises technologies as social experiments, in which the effects are radically uncertain (Krohn and Weingart, 1987; Wynne, 1988; Collins, 1988). Successful experiments, in this view, are about the controlled production of surprises (Gross, 2010). The politics of social experiments are about who gets to define, control and learn from these surprises. More recent work in this vein considers how, as testbeds and ‘living labs’ are increasingly discussed as explicit modes of innovation, society and its institutions are themselves put to the test (Engels et al. 2019; Marres. 2025).

The difference is that innovators are now more willing, with the backing of policymakers and venture capitalists, to admit the experimentality of their operations. This is not the situation observed by Collins (1988), in which public ‘experiments’ were in fact mere performances, with all of the uncertainties kept under tight control. But it is still one in which, as Amoore (2023) observes, failure is never total. Failure is reconfigured as learning, but the learning is privatised. Technologies are released, before they are ready, into the wild and then iterated on the fly. This is a mode of beta-testing innovation rehearsed in Silicon Valley, in the world of software, where a computer crashing may not be as consequential as a car or a drone crashing. As the experiment expands in material and economic ways to include robots, markets, public opinions and political machinery, the privilege required to fail often and fail safely demands a societal safety net, the building of which requires an elaborate form of what actor network theory would call ‘heterogenous engineering’ (Law, 1987).

Phan asks us to examine the politics of failure. Many people in Logan would regard the visit from Wing as a failure. For the company, however, this was merely a data point on their journey to a standardised system that they hoped would eventually scale across any number of Logans and Bonythons. In Logan, they tried something, learned from it and moved on. A Silicon Valley mantra that asks companies to ‘fail fast’ thinks nothing of the collateral damage, and experiments are not set up such that real accountability is built in. We have recently seen in the United States this mode of disruption being wrought upon institutions in the form of a ‘Department of Government Efficiency’. The aim here appears to be not just a radical cutting of administrative capacity but also a sabotage of administrative intelligence by removing research activities and data-collection. By the time the scale of destruction is understood, the disruptors have moved on, away from the normal politics of accountability.

Phan's story reveals some moments of possibility for democratised learning. We read, for example, about the company's discovery that, while it is focussing on getting its technology to deliver packages safely, people are more concerned and understandably irritated by the drones’ mosquito-like whine. This moment of learning offers a fixable problem for the company's engineers. But it suggests a challenge to a mode of innovation that intentionally overlooks issues of scale. Some of Logan's enthusiastic early adopters might not mind the occasional noise. But what happens if and when the technology becomes a new infrastructure and the noise becomes an unavoidable soundscape for citizens, including those who see no benefit? In cities where self-driving cars are thriving, we are seeing similar concerns, which engineers might dismiss as mundane, come to the fore, as robots become an additional traffic problem.

So what are the prospects for improved governance if failure is costless and accountability can never catch up with a constantly changing, ever-experimenting state of innovation? In theory, when society is the laboratory, its democratic representatives should be able to take some degree of control of the experiment, sharing in the learning and claiming some fraction of its benefits. In practice, however, the valuable learning is privatised and the downsides are socialised.

During my own interviews on the politics of self-driving car innovation, a disenchanted civil servant sarcastically told me that a testbed-based innovation policy could, if done badly, give a local community what ‘Christmas Island got from being a testbed for nuclear weapons. You get all the pain, but you don't necessarily get much of the gain’. (see Stilgoe and O’Donovan, 2023) (In the 1950s, Christmas Island had been brutally turned into a laboratory for some of the hydrogen bomb tests Phan refers to in her history of Britain's imperial experimentation).

Exploitative experimentation is, as Phan argues, not new, and it is instructive to see it in historical and colonial context. But when experiments begin, it is often not clear what they are about. Phan's analysis shows us that a simple drone test has far more at stake than an assessment of technical performance or cost-cutting potential. It is a test of a mode of innovation. For policymakers, the object of governance is therefore unclear. Should they be targeting their regulation at the thing itself and its risks, the claims being made by its operators, the company's business model or something else? In these circumstances, a company's most successful strategy is not so much regulatory arbitrage as regulatory misdirection.

The tech industry's testbeds and pilots can sometimes feel like prestidigitation, mere distractions in a process of what is sometimes called ‘blitzscaling’. The aim is to acquire oligopolistic power, and the rents that come with it, before regulators are able to recognise what is going on. The tactics are, as Phan describes, ‘endless versioning’ (Halpern and Günel, 2017, p. 8). But the strategy steering these permalabs tends to be intentionally ambiguous, so it is not clear what the ends of experimentation are, nor how experimenters are aiming to make money should their innovations succeed. So even if permission is sought and granted for a local experiment (which is often not the case), the eventual innovation can still be regarded as permissionless. The companies’ hope is that, by the time regulators come to evaluate the goods and bads, the technology is entrenched and its benefits are clear to its users.

Next to this hyperfast, hyperscaled juggernaut, the contribution of slow, detailed social research might seem negligible, but, as I think Phan's research shows us, the local detail matters precisely because these innovators want to hide it from public view. If the companies achieve the requisite exit velocity, they escape accountability, becoming ‘too big to care’ (Doctorow, 2025), and accountability becomes practically impossible. Uber cannot care about all of the myriad issues its millions of drivers encounter every day, just as Amazon cannot understand or correct the mistakes of its ‘delivery service partners’ and Meta cannot directly moderate the content of Facebook's billions of newsfeeds. These companies claim that automation solves these scale issues; the reality is that they are dealt with by humans who these companies do not normally regard as employees. Against this model of carelessness-by-design taking place up in the air, it is more important than ever to pay attention to what's happening on the ground.