From rule of law to rule of algorithm: Generative Artificial Intelligence's threat to democracy

Abstract

Generative AI is transforming state power, replacing transparent legal processes with opaque algorithmic systems. This commentary argues that this shift is not simply a technical upgrade but a qualitative break from previous forms of digital governance. Where earlier predictive systems classified and ranked citizens, generative AI constructs the informational environment within which citizens act. I argue that this shift operates along two axes: generative AI’s layered opacity dissolves the chains of public answerability that link transparency to accountability, while its capacity for synthetic content generation fragments the shared factual ground that deliberation depends on. Regulatory responses such as the EU AI Act represent important but insufficient counterweights. The central challenge remains ensuring these technologies serve democratic governance rather than reduce citizens to objects of algorithmic management.

This article is a part of special theme on Digital Authoritarianism and Gen AI. To see a full list of all articles in this special theme, please click here: https://journals.sagepub.com/page/bds/collections/digital_authoritarianism_and_genai?

Introduction

Power in democratic societies is changing, and AI is a significant part of why. Two broad types matter here: predictive systems that classify and forecast, and generative systems that create new content. Rouvroy and Berns (2023) define this as “algorithmic governmentality”—governance enacted through aggregated data patterns rather than law or democratic deliberation. This commentary uses that concept to examine what happens when algorithmic governmentality meets established liberal democracies, where the clash between automated processes and longstanding democratic norms is sharpest. ¹ Unlike earlier predictive AI, generative systems like OpenAI's GPT-4 produce novel content that is already shaping citizen behavior and political discourse. In many countries, these systems are becoming core elements of government services—drafting policies, simulating public consultations, and generating citizen communications in ways that directly influence democratic processes (Anand and Wan Mohamed, 2025).

The essential argument here is that generative AI, as it enters public administration, poses an existential challenge to democracy—specifically because algorithmic systems that prioritize efficiency over rights are displacing deliberative policymaking (Fountain, 2023; Kreps and Kriner, 2023). This commentary traces this development in three moves: outlining its evolution from welfare state administration through predictive algorithms to current generative systems; examining how these technologies erode transparency, accountability, deliberation, and pluralism; and considering what legal, technical, and civic responses can preserve democratic control. My animating concern is that the use of generative AI's capacities for content creation, surveillance, and automated decision-making is concentrating power in opaque corporate algorithms that can foreclose opposition and dissent through synthetic manipulation (Kingsmith, 2026; Kingsmith and Zehner, 2025).

From welfare statistics to algorithmic governance

Traditional democratic governance relied on stable rules and formal processes that citizens could understand and challenge. The postwar welfare state favored clear authority chains from elected officials through civil servants to policy implementation. Decisions followed published procedures, affected parties had appeal rights, and elections provided accountability. As Kessler-Harris and Vaudagna (2018) summarize, this postwar consensus specifically linked social welfare to democratic citizenship.

The New Public Management reforms of the 1980s disrupted this consensus. Driven by a logic of austerity, NPM introduced metrics and digitization that recast citizens as efficiency units rather than democratic participants (Veale and Brass, 2019). Decisions, however, remained traceable to human officials—citizens could contest processes through known channels. What distinguished this period was not a break from democratic accountability but the steady infusion of efficiency logic that administrators and consultants actively promoted as modernization (Karlsson, 2024). Early digitalization in the 1990s and 2000s largely automated existing processes rather than transforming governance logic (Dunleavy et al., 2005). The basic architecture of accountability (traceable decisions, appeal rights, known procedures) held.

By the 2010s, that architecture was under more direct pressure. Algorithms evolved from tools that automated tasks to systems that actively shaped public decisions, managed urban life, and concentrated power in ways that resisted scrutiny (Katzenbach and Ulbricht, 2019). Chicago's 2013 “heat list” algorithm used arrest data to generate risk scores determining police deployment (Brayne, 2017). From 2015 to 2019, Australia's Robodebt system automatically issued debt notices based on algorithmic income averaging that reversed the burden of proof onto citizens (Hannah and Botterill, 2025). Beyond simply introducing new metrics for administrators to apply, these systems removed human agents from key decision points.

The Dutch SyRI system characterizes the predictive power of pre-generative AI tools. Operational from 2014 until courts declared it unlawful in 2020, SyRI cross-referenced databases to generate welfare fraud risk scores—but did not generate content or interact directly with citizens (Blauw, 2020). As Rouvroy and Stiegler (2016) elaborate, this form of governance dispenses with causation or understanding, focusing instead on continuous monitoring and correlation. Decisions were now probabilistic rather than rule based. Surveillance had become pre-emptive. But citizens were still, in a formal sense, legal subjects—there was still something to appeal, someone to confront.

Generative AI removes that foothold. When the UK's Department for Work and Pensions began testing GPT-based systems in 2024 to handle benefit inquiries and flag potential fraud, it acquired capabilities that earlier systems did not have (Booth, 2024). These tools can generate personalized responses that subtly discourage benefit applications, produce synthetic consultation documents that appear to represent citizen input, and create targeted messaging that shapes political preferences at scale (Ulnicane, 2025). Herein lies the paradox of generative AI governance: a technology widely used by the public increasingly governs through means that exclude the public from any meaningful contestation.

What marks this qualitative shift is not just what these systems decide, but what they produce. Earlier algorithms detected patterns in citizen behavior; generative systems now construct the informational environment within which citizens act. Microsoft's Azure OpenAI Service, deployed across government agencies in over 60 countries, allows civil servants to use Copilot to draft policy documents—with the AI shaping policy direction through its suggestions before any human deliberation occurs (Brandthav and Elzaki Adam, 2025; Chappell, 2023). The UK Home Office's visa processing tool goes further: it does not just assess applicants but effectively compels them to reshape their documentation to fit algorithmic categories, creating a loop in which the state's classificatory logic becomes the terms on which citizenship itself is negotiated (Murray, 2024). Where SyRI imposed a predictive score, these systems generate a new reality.

Erosion of democratic values

Transparency and accountability

Transparency and accountability are not merely parallel democratic requirements, they are mutually constitutive ones. Transparency enables public scrutiny of government decisions, while accountability establishes clear lines of answerability for the exercise of power. I argue that generative AI undermines this connection. By introducing profound and multi-layered opacity, it dismantles the mechanisms of public scrutiny and creates a vacuum of responsibility where no actor can be held answerable for automated outcomes (Taeihagh, 2025).

Real transparency requires comprehensible decision criteria. The Dutch SyRI case—in which a court ruled the system unlawful precisely because its risk-scoring logic could not be meaningfully examined by those it affected—demonstrated this for a rule-based system. Generative AI represents a more profound shift. It operates through neural architectures whose decision paths are intrinsically opaque at multiple levels: technical (the complexity of the models themselves), linguistic (the mismatch between how models compute and how humans explain), and institutional (the deliberate concealment of system workings) (Burrell, 2016; Fui-Hoon Nah et al., 2023).

Such layered opacity is compounded by a structural governance gap, in which public officials lack the expertise to scrutinize the systems they deploy while private vendors strategically hide behind trade secret protections (Kronblad et al., 2024). The result is the systemic “black box” Pasquale (2016) describes. A disabled citizen denied benefits cannot discern whether the decision was justified, discriminatory, or a statistical error (Piridi and Asundi, 2025)—a problem linked to the erosion of what Citron (2008) calls “technological due process.”

This opacity obscures decisions while also dissolving the chain of responsibility needed to challenge them. When decisions are opaque, assigning responsibility becomes impossible, producing what Danaher (2022) identifies as a “responsibility gap” in governance. Canada's Chinook immigration system illustrates how this plays out: the system showed disproportionate rejection rates for visa applicants from African countries, yet no accountability chain existed to explain or remedy this. The department blamed the tool; developers said it merely aided human decisions (Robitaille, 2023; Stacey, 2022). As Meurrens (2024) documents, this diffusion of responsibility left applicants with no entity to hold answerable.

When opacity and unaccountability merge, the result is a democratic deficit in the precise sense Coeckelbergh (2025) gives the term: power exercised without a responsible actor. Citizens are left in a maze where vendors invoke intellectual property protections, governments claim reliance on certified systems, and individual workers lack the authority to challenge algorithmic outputs. The fused connection between transparency and accountability is severed. I argue that this is the result of institutional design choices that consistently prioritize efficiency and proprietary control over democratic oversight. Without enforceable mechanisms that maintain human responsibility for AI decisions, governance becomes a process where power is exercised but cannot be located.

Deliberation and pluralism

Democratic legitimacy, as theorists from Marti (2017) to Young (2001) have elaborated, depends on a relationship between deliberation and pluralism that is symbiotic rather than sequential. Deliberation provides the process of collective reasoning through which contested issues are resolved; pluralism ensures the full range of societal voices is present in that process of debate. Each condition sustains the other: deliberation without genuine plurality produces rationalized consensus, while plurality without deliberative process produces mere aggregation of preferences. I argue that generative AI disrupts this symbiosis in three core ways.

The first disruption is temporal. Deliberation requires time for reflection, debate, and integration of competing viewpoints. The deployment of AI to accelerate governance, as seen in Estonia's use of AI to draft legislation faster than parliamentary review can accommodate, creates what Kerikmäe and Pärn-Lee (2021) identify as a fundamental asymmetry between algorithmic speed and human deliberators. By the time legislators convene, the system may have already generated new iterations, rendering democratic deliberation perpetually reactive. This is not a manageable inefficiency but a structural shift that transfers authority from a pluralist assembly to a monolithic algorithmic process, eroding the conditions under which genuine compromise becomes possible (Rönnblom et al., 2024).

The second disruption is epistemic. A functioning pluralism depends on the ability to distinguish genuine citizen voices from artificial ones. As Calvo and Saura Garcia (2025) argue, generative AI undermines this distinction by enabling the mass production of unique, context-aware submissions that can drown out authentic public participation. The UK's “Consult” tool for public consultation drew precisely this concern: citizens and observers feared that AI analysis would flatten response diversity and introduce vendor bias into what purported to be a record of public views (Berditchevskaia et al., 2025). Narrow interests can now automate simulated consensus or opposition, making it structurally impossible to gage authentic democratic expression from synthetic noise.

The third disruption is informational. Deliberation requires, at minimum, a shared factual baseline from which disagreement can proceed. When government systems use generative AI for personalized policy explanations, they risk creating inconsistent or biased interpretations of the same law for different citizens—a danger serious enough that the EU classifies such systems as “high-risk” (European Commission, 2023). I argue that this produces what Lazovich (2023) terms “filter bubbles” of administrative communication, where citizens receive fundamentally distinctive explanations of a policy based on their demographic profile. The consequence is not merely confusion but what Rodilosso (2024) describes as radical epistemic fragmentation—a condition where no shared facts exist from which collective reasoning can begin.

Resisting the eclipse of democracy

The integration of generative AI into public administration represents more than a technical upgrade; it signals a profound transformation in the logic of governance itself. This commentary has briefly traced the shift from rule of law to rule of algorithm, arguing that it actively undermines democracy's foundational pillars. Generative AI's opacity—arising from technical complexity, institutional choices, and commercial secrecy—creates accountability vacuums where traditional chains of public answerability dissolve (Kneer and Christen, 2024). Its capacity for synthetic content generation and personalized communication simultaneously corrupts deliberative integrity and fragments the shared factual ground pluralist societies depend on. These are not technical failures, but political ones.

Yet this erosive trajectory is not inevitable. Courts, regulators, and civil society have begun to push back. The EU AI Act establishes crucial red lines by classifying public sector AI as a key risk and mandating transparency and human oversight. The task ahead is not to refuse technological innovation but to subordinate it to democratic values. This requires moving beyond reactive regulation to establish enforceable, proactive lines of human responsibility—demanding explainability, protecting deliberative spaces from synthetic manipulation, and ensuring that public consultation means something. Citizens must remain the authors of their collective future, not the processed subjects of an inscrutable algorithmic logic.