Between risk mitigation and labour rights enforcement: Assessing the transatlantic race to govern AI-driven decision-making through a comparative lens

Abstract

In this article, we provide an overview of efforts to regulate the various phases of the artificial intelligence (AI) life cycle. In doing so, we examine whether—and, if so, to what extent—highly fragmented legal frameworks are able to provide safeguards capable of preventing the dangers that stem from AI- and algorithm-driven organisational practices. We critically analyse related developments at the European Union (EU) level, namely the General Data Protection Regulation, the draft AI Regulation, and the proposal for a Directive on improving working conditions in platform work. We also consider bills and regulations proposed or adopted in the United States and Canada via a transatlantic comparative approach, underlining analogies and variations between EU and North American attitudes towards the risk assessment and management of AI systems. We aim to answer the following questions: Is the widely adopted risk-based approach fit for purpose? Is it consistent with the actual enforcement of fundamental rights at work, such as privacy, human dignity, equality and collective rights? To answer these questions, in section 2 we unpack the various, often ambiguous, facets of the notion(s) of ‘risk’—that is, the common denominator with the EU and North American legal instruments. Here, we determine that a scalable, decentralised framework is not appropriate for ensuring the enforcement of constitutional labour-related rights. In addition to presenting the key provisions of existing schemes in the EU and North America, in section 3 we disentangle the consistencies and tensions between the frameworks that regulate AI and constrain how it must be handled in specific contexts, such as work environments and platform-orchestrated arrangements. Paradoxically, the frenzied race to regulate AI-driven decision-making could exacerbate the current legal uncertainty and pave the way for regulatory arbitrage. Such a scenario would slow technological innovation and egregiously undermine labour rights. Thus, in section 4 we advocate for the adoption of a dedicated legal instrument at the supranational level to govern technologies that manage people in workplaces. Given the high stakes involved, we conclude by stressing the salience of a multi-stakeholder AI governance framework.

1. Introduction

In recent years, workers in all industries have become inexorably acquainted with a set of human resource practices partially ‘outsourced’ to digital devices and software that rely on artificial intelligence (AI) to optimise processes, enhance efficiency, and minimise costs. ¹ Moreover, managerial functions can be performed, or at least supported, by data-driven tools across the entire range of contractual phases, from recruitment to termination, task administration to performance assessment. Numerous accounts have detailed how hiring procedures are now completed online and processed by algorithms, ² while prerogatives such as scheduling shifts, forecasting personnel requirements and allocating assignments can equally now be offloaded to online applications with relative ease and speed. To nurture these new functions, worker monitoring is constantly and ubiquitously performed, thanks to the virtually infinite set of data-capturing and -processing systems currently available, and the large-scale information derived provides management with granular knowledge that informs both day-to-day and real-time decision-making.

Popular awareness of so-called ‘algorithmic bosses’ is increasing, ³ and not without reason. Several traditional guardrails are at risk of being unsettled due to how these systems are conceived and operated. First, they are able to collect and elaborate huge swathes of data almost instantaneously and from multiple sources, due to their seemingly infinite scalability, and they can do so without the observed being cognisant of the tracking. Second, once instructed to pursue a particular objective by their designers and deployers, algorithmic models are unbeatably efficient, ‘mechanistic’, and mission-oriented, ⁴ meaning that they fail to consider any exceptions. Third, as they are being adopted within a far-from-perfect social fabric, they end up calcifying biases, inequalities, and stereotypes, despite being erroneously depicted as a silver bullet with which to overcome shortcomings known to plague analogue reality, such as inefficiencies, disparities, inaccuracies, and arbitrariness. To complicate things further, existing redress mechanisms, which are designed to address a different form of authority, are not equipped to curb the inaccuracies and abuses of algorithms.

Aside from being a distinctive feature of digital labour platforms’ business model, ⁵ automated decision-making has also been widely adopted in almost all sectors in which data represent the underlying infrastructure (e.g., warehouses, home offices, stores, factories, and consulting companies). ⁶ Due to being confronted with an ever-evolving set of potential promises and risks, lawmakers worldwide are pondering whether to step in and seek to better regulate this growing trend. For instance, the European Union (EU) institutions have proposed a panoply of intertwined regulatory measures that display integrated or overlapping characteristics. Yet, the relationship between these initiatives is intricate and contentious. For example, while solely automated decision-making on the basis of personal data is thoroughly regulated by the General Data Protection Regulation (GDPR) and similarly, albeit more narrowly, covered in the context of digital labour platforms by the proposal for a Directive on improving working conditions in platform work (Platform Work Directive [PWD]), the draft AI Regulation classifies some AI-driven practices in the employment context as high risk and then simply defines procedural steps that providers must comply with to ensure their ‘conformity’ with certain requirements. ⁷ A comparable trend of policy activism and stratification can be seen in both the United States (e.g., the NIST Artificial Intelligence Risk Management Framework and the Algorithmic Accountability Act) and Canada (e.g., the Artificial Intelligence and Data Act).

Without engaging with the taxonomical quandary that surrounds the notions under scrutiny, we rely on (still contested) institutional definitions. Here, AI refers to ‘systems that display intelligent behaviour by analysing their environment and taking actions—with some degree of autonomy—to achieve specific goals’. ⁸ In the draft AI Regulation (which is also known as the AI Act), AI systems are defined as software that ‘can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with’ (Article 3). ⁹ The latter definition considers the overarching roles played by AI, such as ‘improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations’ (Recital 3). By contrast, in the proposed PWD, automated decision-making systems are identified by reference to their functions, namely ‘to take or support decisions that significantly affect workers’ working conditions’. ¹⁰ Similarly, algorithmic management can be conceptualised as the use of ‘software to automate organisational functions traditionally carried out by human managers, identified in both platform work and conventional employment settings’. ¹¹

Underpinning most recent regulatory efforts is the increasingly important notion of ‘risk’, which has risen to sit among the prominent organising principles of today's complex reality, despite the discord among experts concerning its meaning. ¹² The ‘risk-based model’ embodies a modern regulatory attitude ¹³ and offers a method for furthering compliance in highly technical sectors in which fully prescriptive regulation is unable to keep pace with new developments. ¹⁴ From the outset, it must be noted that risk functions as a yardstick for the interpretation, ‘calibration’, and application of legal norms, thereby going beyond the classical ‘top-down’ and ‘one-size-fits-all’ methodologies. ¹⁵ Indeed, inspired by product safety legislation, the risk-based approach necessitates a shift towards a ‘granular, scalable’ logic ¹⁶ that is often not compatible with fundamental rights enforcement. ¹⁷ The model's operation is dependent on the context in which it is applied, ¹⁸ and it involves at least two main phases. The first phase (‘assessment’) is devoted to forecasting the likelihood and severity of a certain event, whereas the second phase (‘management’) involves the implementation of coordinated mitigation activities intended to reduce the risk to the point where ‘the harms do not outweigh the benefits and the risk is deemed sufficiently low to be taken’. ¹⁹

This article sketches an overview of recent regulatory attempts to address the different stages of the AI life cycle—that is, the interconnected series of consecutive steps that enable the functioning of such systems. In doing so, it examines whether—and, if so, how—highly fragmented regulatory frameworks can offer reliable safeguards that prevent the harms stemming from AI- and algorithm-driven organisational practices. Moving beyond a labour law perspective and embracing a law and governance approach, we intend to answer the following research questions: Is the widely adopted risk-based methodology fit for purpose? Is it consistent with the actual enforcement of fundamental rights at work, such as privacy, human dignity, equality, and collective rights? We look at developments at the EU level to retrace the consistency and tensions between related tools. Moreover, we also evaluate bills, regulations and soft law tools proposed or adopted in the United States and Canada using a transatlantic (functionalist) comparative approach, ²⁰ underlining the analogies and differences between EU and North American attitudes to risk assessment and management.

The remainder of this article is structured as follows. Section 2 focuses on the notion(s) of ‘risk’ as the common denominator in some EU and North American legal instruments, with the aim being to unpack its various, often nebulous, connotations. We contend that a scalable governance framework, which is partially outsourced to those entities targeted by the relevant regulation, is not appropriate for ensuring compliance with the fundamental rights chiefly impacted by AI- or algorithm-driven practices. In addition to presenting the key provisions of the existing legal schemes, section 3 highlights their interactions as well as the antinomies that exacerbate legal uncertainty and pave the way for regulatory arbitrage. Paradoxically, this frenzied race to regulate AI could slow technological innovation, undermine national frameworks regulating the introduction of workplace technologies, and erode fundamental rights. Finally, Section 4 discusses whether professional settings deserve a dedicated instrument, given both the high stakes involved and the pre-existing power structures. In light of this discussion, the article concludes by stressing the importance of a collective AI governance framework that involves those individuals and communities affected during the design, development and deployment phases.

2. Navigating the ambiguity of ‘risk’ and the pitfalls of risk-based regulation

2.1. Approaches to regulating AI through risk in the EU, United States, and Canada

In this section, we analyse the multiple connotations of the notion of risk in certain EU regulatory instruments (the AI Act, the GDPR, and the PWD, in order of the amount of discussion) and compare them with North American instruments. ²¹

The Explanatory Memorandum drafted by the European Commission explicitly states that the AI Act ‘puts in place a proportionate regulatory system centred on a well-defined risk-based regulatory approach that does not create unnecessary restrictions to trade’. ²² Moreover, ‘legal intervention is tailored to those concrete situations where there is a justified cause for concern or where such concern can reasonably be anticipated in the near future’. The AI Act pursues four main goals (although this succession of priorities is not reflected in the substantive sections of the Act):

guaranteeing that AI systems are safe and respect fundamental rights and values;

ensuring legal certainty to facilitate investment and innovation;

enhancing governance and the effective enforcement of fundamental rights and the safety requirements for AI systems; and

facilitating the development of a single market for lawful, safe, and trustworthy AI, thereby preventing fragmentation and regulatory arbitrage.

The EU Commission did not want to introduce any unnecessary costs that would slow the race to develop AI, although they recognised the need to design effective supervision and enforcement mechanisms in order to avoid infringements of fundamental rights and breaches of safety.

A proactive model both empowers AI ‘developers’ and shifts the burden of compliance (and the associated mandatory reporting) onto them. The AI Act lays down specific rules under Title III—that is, the ‘core’ of the Regulation. ²³ AI systems that ‘create a high risk to the health and safety or fundamental rights of natural persons’ encompass the components of products that are subject to ex-ante conformity assessments conducted by sectoral regulators (e.g., toys, machinery, or medical equipment) and ‘other stand-alone AI systems with mainly fundamental rights implications’ as listed in the updatable yet rigid Annex (while new subcategories can be added, the main categories cannot be altered). AI systems are permitted provided they comply with certain essential requirements and pass the conformity assessments and reviews performed by the ‘providers’, who are considered to be best placed to gauge the risks posed by their AI systems. ²⁴ Less involvement is required on the part of users, ²⁵ who, in the context of work, would almost always be employers. This clearcut distinction does not take into account the dynamic nature of AI development, deployment and monitoring or the possibility of ‘co-production’, whereby users participate in the design of AI systems. ²⁶

The high-risk AI systems referred to in Article 6(2) of the AI Act are made explicit in Annex III. Tellingly, among these ‘highly risky’ systems are those adopted in the context of ‘employment, workers management and access to self-employment’ (point 4) that must undergo the conformity assessment procedure prior to entering the market or being put into service. In particular, reference is made to:

AI systems intended to be used for recruitment or selection of natural persons, notably for advertising vacancies, screening or filtering applications, or evaluating candidates in the course of interviews or tests; and

AI intended to be used for making decisions on promotion and termination of work-related contractual relationships, for task allocation, and for monitoring and evaluating performance and behaviour of persons in such relationships.

In short, while the list of functions is neither comprehensive nor exhaustive, the Annex captures some of the most common uses of AI in the work context—that is, functionally and logically interconnected human resource practices that ‘may appreciably impact future career prospects and livelihoods of these persons’ (Recital 36).

A provider is ‘a natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge’ (Article 3 of the AI Act). ²⁷ It is up to the provider to specify the purpose of an AI system, which then determines its classification and the consequent obligations under Article 16. The classification considers the intended purpose of an AI system and the modalities for which it is used. Despite the aim of delivering a modular and targeted framework, AI technologies are classified in an ‘abstract’ and ‘context-neutral’ manner within the AI Act, ²⁸ with no consideration of case-specific uses. A key limitation of the Regulation is the fact that it fails to consider the multipurpose and adaptive nature of AI systems, which can be easily re-coded and ‘employed for completely different purposes from those for which they were designed’. ²⁹ Aside from the developers’ duty to consider and mitigate the risks stemming from ‘reasonably foreseeable misuse’, ³⁰ this process only barely addresses what experts term ‘function creep’, which involves the progressive widening of the use of systems beyond the purposes for which they were originally intended.

According to Article 9 of the AI Act, the providers of high-risk AI systems must establish, implement, document, and maintain a risk management system throughout the life cycle of a given AI system. In this complex architecture, high-risk AI systems are permitted so long as a ‘continuous iterative process’ is put in place and systematically updated (Article 9(2)). The providers are required to identify and analyse the ‘known and foreseeable risks’ as well as to evaluate the ‘risks that may emerge when the high-risk AI system is used in accordance with its intended purpose and under conditions of reasonably foreseeable misuse’. They must also evaluate other ‘risks based on the analysis of data gathered from the post-market monitoring system’ and, notably, adopt ‘suitable risk management measures’. Within this margin of appreciation, the providers can assume necessary risk mitigation strategies and deploy internal controls based on the verification of the quality management system, examination of the information included in the technical documentation (a form of user transparency), ³¹ and verification of the consistency between the design and development process of the relevant AI system and the technical documentation provided. ³²

The risk-based model is not exclusive to the AI Act. A similar, albeit more open-textured, approach also informs the GDPR. Under Article 35 of the GDPR, ³³ when data processing involving technologies ‘is likely to result in a high risk to the rights and freedoms of natural persons’, ³⁴ the data controller (who is generally the employer) must preventively ‘carry out an assessment of the impact of the envisaged processing operations on the protection of personal data’. Any discrimination stemming from algorithmic management, as one common example in this regard, falls neatly within this risk-centred model. ³⁵ This type of risk assessment is required when personal aspects related to natural persons are systematically and extensively evaluated by means of automated processing, including profiling, that results in decisions with legal or similarly significant effects. According to Article 35(7) of the GDPR, the assessment must include the description of the operations and purposes of the processing, clarification of necessity and proportionality, and elucidation of both the risks faced by the data subjects (workers) and the measures adopted to address those risks and demonstrate compliance with the GDPR. ³⁶

Moreover, given the ‘nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for the rights and freedoms of natural persons’, Article 24(1) of the GDPR imposes on the data controller the duty to ‘implement appropriate technical and organisational measures to ensure and to be able to demonstrate’ that the relevant data processing is performed in accordance with the GDPR. Relatedly, Article 25(1) of the GDPR fleshes out an employer's duty to effectively ‘implement appropriate technical and organisational measures’ and ‘integrate the necessary safeguards into the processing in order to meet the requirements of this Regulation and protect the rights of data subjects’. This must be done after conducting a specific risk assessment, both prior to and during the data processing, which considers ‘the state of the art, the cost of implementation and the nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for rights and freedoms of natural persons posed by the processing’.

The notion of risk is operationalised within the GDPR by means of ‘meta-obligations’ ³⁷ in such a way as to render data controllers (or employers, for the purpose of our analysis) responsible and accountable. ³⁸ The need to fulfil such general requirements is expected to serve as a compliance-enhancing method. As a consequence, provided that all of the GDPR principles are respected, data controllers can modulate their regimes by assessing the likelihood and intensity of the envisaged risk of the related processing operations, rather than having to comply with a highly standardised model. ³⁹ The determination of the level of risk, the resulting salience of putting into place mitigation strategies, and the identification of technical and organisational measures represent part of a ‘knowledge gathering and analysis’ phase that is left to the employers themselves. ⁴⁰ Employers also enjoy a discrete margin of appreciation with regard to the identification of the most appropriate technical and organisational measures (Article 5 of the GDPR).

When reading the fine print, it becomes apparent that a similar scheme is replicated within (the original version of) the proposed Platform Work Directive. In particular, under Article 7 of the PWD, a sui generis algorithmic impact assessment of the risks associated with automated monitoring and decision-making systems when it comes to the occupational safety and health (OSH) of platform workers is introduced. However, the framing of this instrument is indirect—that is, Member States (MS) have a duty to compel digital platforms (in their role as employers or principals) to ‘regularly monitor and evaluate the impact of individual decisions taken or supported by automated monitoring or decision-making systems’ (Article 7(1)). In addition, platforms must put in place a reliable mechanism supported by ‘sufficient human resources’ responsible for compliance with the rules laid down in the national laws transposing the PWD (Article 7(3)). We have commented elsewhere that the key limitation of the approach adopted in the PWD lies in the uncritical acquiescence to the introduction of such highly invasive organisational patterns, which have already proven to undermine workers’ agency and erode the workers’ experience of their job. ⁴¹

When it comes to OSH, digital platforms are required to (i) conduct a risk assessment, ‘in particular as regards possible risks of work-related accidents, psychosocial and ergonomic risks’; (ii) assess whether the current safeguards are suitable for addressing the risks identified, considering the work environment; and (iii) adopt appropriate preventive and protective measures (Article 7(2)). Moreover, automated monitoring and decision-making processes must not increase the pressure or augment the health-related risks faced by platform workers (a similar requirement can be found in California Assembly Bill 701, as discussed below). This is a rather narrowly specified ‘human monitoring’ procedure, which must be read in conjunction with the risk assessment required by the GDPR. Given the inclusion within a set of provisions that (also) apply to self-employed persons performing platform work, it is odd that the rights pertaining to health and safety at work are reserved for ‘workers in view of Union law’ (Recital 40). The second section of the same Article has a broader ambit that also encompasses genuinely self-employed workers, whereas only ‘workers’ fall within the scope of Articles 7(1) and (3).

Across the Atlantic, the National Institute of Standards and Technology (NIST), part of the United States Department of Commerce, takes a similar approach. In March 2022, it introduced a voluntary framework, the NIST Artificial Intelligence Risk Management Framework (AI RMF), ⁴² intended to improve the management of risks stemming from AI and affecting individuals, organisations, and society in general. The framework is designed to deploy a flexible, outcome-focused, cost-effective approach to the design, development, use, and evaluation of AI products, services, and systems. As detailed in the document that launched the multi-stakeholder consultation, ‘managing’ means identifying, assessing, responding to, and communicating the risks of AI. The draft focuses on technical characteristics (accuracy, reliability, robustness and resilience), socio-technical characteristics (explainability, interpretability, privacy, safety, and managing bias) and guiding principles (fairness, accountability, and transparency). ⁴³ It also defines risk as ‘a measure of the extent to which an entity is negatively influenced by a potential circumstance or event [considering] 1) the adverse impacts that could arise if the circumstance or event occurs; and 2) the likelihood of occurrence’ ⁴⁴ and designs a three-part solution based on mapping, measurement, and management. However, despite the abundance of procedural safeguards, the framework lacks binding force.

In addition, while there is nothing akin to the EU's ‘horizontal’ approach, a comparable scheme geared towards mandatory impact assessments regarding AI use is currently making its way through Congress, namely the Algorithmic Accountability Act. The Bill was presented in 2021, ⁴⁵ although no significant progress has been made since then. If approved, it would require the Federal Trade Commission (FTC) to develop regulations requiring large firms (which are defined as those with over USD 50 million in revenue or those that hold data concerning at least one million consumers or consumer devices) to conduct impact assessments in relation to existing and new ‘high-risk automated decision systems’ (including systems that ‘analyze or predict sensitive aspects of [consumers’] lives, such as their work performance, economic situation, health, personal preferences, interests, behavior, location, or movements, [and] alter legal rights of, or otherwise significantly impact, consumers’) and maintain relevant documentation. The systems that would fall within this category are those that may contribute to inaccuracy, bias, or discrimination, or those that facilitate decision-making regarding sensitive aspects of consumers’ lives by evaluating individuals’ behaviour. The Bill partially reproduces the logic of the AI Act and the GDPR, establishing a model that delegates to developers the task of carrying out risk assessment and management practices.

In Canada, further corroborating the suggestion that we are currently living in the golden age of the risk-based approach, the Federal Government introduced Bill C-27 into Parliament in 2022. Bill C-27 includes the Consumer Privacy Protection Act (CPPA) and the Artificial Intelligence and Data Act (AIDA). The AIDA provides that ‘a person responsible for a high-impact system’, as defined within regulations, ‘must, in accordance with [such] regulations, establish measures to identify, assess and mitigate the risks of harm or biased output that could result from the use of the system’. Similar to the EU's AI Act, the AIDA adopts a risk-based approach. It is unclear at this stage if the regulations will eventually define systems that have implications for employment and occupation among the ‘high-impact’ ones, which contrasts with the approach of the EU's AI Act in defining such systems as ‘high-risk’. ⁴⁶

2.2 Questioning the basic presuppositions of the risk-based approach to AI at work

Albeit to varying extents, the novel EU legal tools concerning AI, the American NIST AI RMF, and the Canadian AIDA all pursue the goal of trustworthy AI systems, which accords with the idea that a set of mandatory requirements renders opaque systems more ‘reliable’, thereby fostering end users’ confidence in such instruments. One key difference is that the NIST AI RFM has a non-binding nature. Additionally, there are also minor differences among the risk-based approaches incorporated into the EU legislation. The GDPR and the PWD are hybrid frameworks. ⁴⁷ They reflect an open-ended and bottom-up exercise that does not excessively constrain the room to manoeuvre of data controllers and digital labour platforms. While the AI Act model is rather more rigid in its classification of areas, ⁴⁸ it provides a ‘fast lane’ reserved for AI providers, who are assigned a certain degree of subjectivity despite the mandatory conditions, which contrasts with the strict national regulation of issues such as workplace data collection and processing on the one hand and decision-making on the other.

Based on the above overview, it is clear that, despite its elusive and highly contested nature, ⁴⁹ the risk-based approach is widespread in policymaking and predicated on the replacement of the ‘formal legality and enforcement of individual rights’ with a model of ‘self-regulation for managing innovation in uncertain scenarios’. ⁵⁰ When dealing with highly volatile matters, policymakers tend to prefer the use of adaptive techniques that are ‘calibrated’ to their targets in an attempt to avoid excessive constraints and paralysing effects on strategic and capital-intensive industries. ⁵¹ Such models are also conceived as ‘assum[ing] a technology will be adopted despite its harms’, ⁵² which is questionable. These models partially shift the responsibility onto those creating the potential risk-event and the (lower-end) regulatory prerogative onto decentralised agencies believed to be better positioned when it comes to gathering scientific evidence and ensuring contextual enforcement. ⁵³ They are also consistent with the rules and principles of data protection, ⁵⁴ which favour anticipatory and collaborative measures.

What should be said about the appropriateness of this method? The abandonment of prescriptive frameworks is alarming, especially when such systems affect human agency and dignity as they do in the context of work, among others. The danger is that non-waivable rules and principles in the workplace (from autonomy to respect for private life, decent wages and working time to non-discrimination, equality to freedom of expression and association) ⁵⁵ will end up being sacrificed on the altar of a nebulous notion of ‘compliance 2.0’ comprised of cosmetic audits, vague checklists and courtesy toolkits. ⁵⁶ As we detail in a forthcoming study, there is a stark discrepancy between a quantitative model of compliance, which is ‘directed toward numerical assessments of the probability and severity of tangible harm’, ⁵⁷ and the logic of (immeasurable and unconditional) constitutional rights that cannot be toned down. ⁵⁸ Any attempt to deliver risk mitigation strategies cannot overlook the ‘sensitivity’ of work environments, where an extra layer of substantive and procedural precautions is already required by national laws, ⁵⁹ which often mandate the involvement of those who are subject to decisions. To this must be added the sheer lack of consideration of the social, collective, and human rights implications of this model, ⁶⁰ a point that we shall return to in the final section of this article. ⁶¹

3. Transatlantic insights: Legal transfers and headlong rushes

3.1 More (regulation) is less (protection)? The increasingly patchy EU framework

To paint as accurate a picture as possible, it must be noted that the EU institutions are both championing innovative regulation and grappling with its potential unintended consequences. The accretion of partially intersecting and diverging measures is not limited to the risk assessment models documented in the previous section. Researchers have begun to explore the startling ‘overlaps, gaps and inconsistencies’ in the ‘patchwork’ of regulation, including between the newly proposed AI Act and existing legislation, ⁶² namely the GDPR and national systems governing electronic monitoring within work environments. To offer just one example, it is unclear if the classification of some AI systems as ‘high-risk’ results in the automatic triggering of the GDPR provisions that address high-risk processing. In other words, does the AI Act's classification reinforce the need to perform the impact assessment required by the GDPR with regard to AI systems used in relation to recruitment, selection, evaluation, monitoring, and decision-making based on personal data? And what about the antinomy between the GDPR's ban on solely automated decision-making and the unencumbered tolerance for the very same practice that informs the proposed PWD?

Given the unbalanced information and bargaining power between workers and employers, the introduction of new technologies that offer monitoring as well as data collection and processing capabilities is tightly regulated in the workplace. Workers and their representatives must be informed and consulted. In countries such as Germany, workers also wield co-determination powers. Article 88 of the GDPR allows Member States, ‘by law or collective agreements’, to introduce ‘more specific rules to ensure the protection of the rights and freedoms in respect of the processing of employees’ personal data in the employment context’. ⁶³ This provision aims at fostering responsive solutions to the emergence of new technologies and practices that may significantly affect workers’ rights while also providing an opportunity to bring worker representatives (and, in some instances, data protection authorities [DPAs]) to the table for the purpose of involvement or co-determination. ⁶⁴ Similarly, Article 20 of the PWD allows Member States to apply or introduce more favourable regulations for workers. The ‘horizontal regulatory approach’ adopted by the AI Act seems to underestimate these work-related exceptions.

The drafters of the AI Act are well aware of the potential clash between supranational and domestic regulations. Indeed, it has been clarified that the fact that ‘an AI system is classified as high risk under [the AI Act] should not be interpreted as indicating that the use of the system is necessarily lawful under other acts of Union law or under national law compatible with Union law, such as on the protection of personal data […]’ (Recital 31). ⁶⁵ Despite this laudable attempt to position the proposed AI Act within a more complex and varied web of rules, little has been said about the way in which such cross-reference can be concretely operationalised. The same is true for the scope of this caveat. The protection of personal data is indicated to be only an example of the EU and national laws that could be superseded by the AI Act. What are the other areas in which potential mismatches between the AI Act and other pieces of EU law or national law compatible with EU law could arise? Assigning the task of creatively interpreting this Recital to the national authorities is hazardous. ⁶⁶

Both the legal basis of the AI Act (Article 114 of the Treaty on the Functioning of the European Union [TFEU]) and its entire conceptualisation lean towards liberalising the production and marketing of AI systems in the EU, provided such systems comply with the standards set out in the Act. As discussed extensively elsewhere, ⁶⁷ these standards are inadequate when it comes to curbing algorithmic management systems, which are increasingly common in today's world of work, as they completely neglect the role of the workers’ representatives and trade unions in regulating the introduction of technological tools at work. Therefore, the liberalisation thrust - and the legal basis - underpinning this initiative risk surmounting any domestic regulations, including work-related ones, that mandate higher standards of protection. ⁶⁸ As a result, fear exists that the AI Act would act as a ‘ceiling’ rather than a ‘floor’ of protection, an outcome that would not be unprecedented in the field of EU employment and labour legislation. Similar situations have notoriously occurred with regard to other legal provisions with a ‘liberalising’ aim, which were interpreted by the Court of Justice of the EU in the ‘Laval Quartet’ in such a way as to trump (fundamental) collective labour rights. ⁶⁹

Hence, more protective laws, including those established at the national level under Article 88 of the GDPR or Article 20 of the PWD, risk being watered down or quashed altogether if read as incompatible with the maximum harmonisation aims of the Regulation. The AI Act could be invoked before the Court of Justice of the EU in an effort to unsettle national frameworks providing for the involvement of the social partners prior to the introduction of any technological tool able to monitor workers’ performance. This is particularly relevant when AI applications are embedded within ordinary tools already used in the workplace to protect business assets, assess work performance, or flag deviant behaviours (e.g., the mundane productivity tracking tools offered by collaborative software). While most ordinary monitoring devices, such as closed-circuit television, typically have to pass at least an information and consultation phase prior to being introduced in professional environments—and are sometimes also subject to administrative authorisation—the model envisaged by the AI Act could displace all such procedural protections. These due-process-like safeguards could, in fact, be interpreted as exorbitant and disproportionate relative to the rules provided by the AI Act and, therefore, be viewed as hampering the free provision of the AI-related services that the instrument aims to promote in accordance with the EU's market liberalisation goals.

An evident laissez-faire approach prevails within the draft AI Act. ⁷⁰ The providers of AI systems merely have to comply with a ‘set of horizontal mandatory requirements for trustworthy AI’ listed in Chapter 2 of the Draft. ⁷¹ A ‘practically unachievable’ precondition ⁷² is the use of ‘sufficiently’ high-quality datasets in which the training, validation and testing are relevant, representative, free from error, and complete. The requirements also include:

the establishment, implementation, and documentation of a risk management system (Article 9);

‘appropriate data governance and management practices’ in relation to training, validation, and testing datasets (Article 10);

package-insert-like documentation proving compliance with current rules prior to commercialisation or entry into service (Article 11);

automatic recording of certain crucial information (Article 12);

transparency and ‘interpretability’ of procedures (Article 13);

human oversight by natural persons so as to minimise risks to health, safety, and fundamental rights (Article 14); and

‘an appropriate level of accuracy, robustness, and cybersecurity’ (Article 15).

In practice, however, the requirements are mainly based on certification via the ex-ante ‘conformity assessment procedures’ conducted by the providers themselves or, in a limited number of cases (e.g., remote biometric identification), by external standard-setting bodies. ⁷³

When compared with the GDPR schemes, the AI Act's model is both overinclusive and toothless, at least when it comes to AI systems used for human resource management purposes. While shrouded in an illusion of simplicity and cost-effectiveness, ⁷⁴ it is overinclusive due to the number of specific duties required to fulfil its techno-determinist goals, preventing those individuals affected from having a meaningful voice. One unintended consequence of the massive list of compliance requirements could be an excess of uncertainty, coupled with both high compliance costs for businesses and almost impossible enforcement for workers, trade unions, DPAs, and labour administrations. This could stifle innovation to the advantage of the big players.

This model is also much less robust than that described in the (technology-agnostic) Article 22 of the GDPR, which bans solely automated decision-making with legally impactful consequences. ⁷⁵ Although this prohibition is undermined by a significant carve-out (i.e., it does not apply when this type of processing is deemed ‘necessary for entering into, or performance of, a contract between the data subject and a data controller’, which could well be the case in relation to employment-related applications of automated decisions), section 3 requires the employer to ‘implement suitable measures to safeguard the data subject's rights and freedoms and legitimate interests’, which is coupled with a non-exhaustive list of safeguards such as the right to obtain human intervention, to express a point of view and to contest a decision. ⁷⁶ A purposive interpretation of Article 22 would militate in favour of the existence of the right ‘to obtain an explanation of the decision’. ⁷⁷ Nothing analogous can be found within the AI Act.

Ensuring fairness, transparency, and accountability in terms of algorithmic management in the context of platform work is among the three specific objectives contributing to the overarching goal of the proposed PWD, ⁷⁸ namely the improvement of the working conditions and social rights of people who work through digital labour platforms. For the first time, automated monitoring and decision-making are identified as possible sources of jeopardy for platform workers, due to contributing to both the precariousness of their working conditions and the diminished quality of their engagement. ⁷⁹ If read in conjunction with the GDPR and the proposed AI Act, the PWD could lead to an unexpected situation in which persons performing platform work would, at least on paper, be protected to a much greater extent than ‘ordinary’ workers and data subjects (both employed and self-employed ones). At this stage, it is difficult to foresee whether the ‘more focused’ scope of the proposed PWD would steer it away from the deregulatory impact of the AI Act. It must be stressed, however, that both texts are still under discussion, now entering the final stages of the legislative procedure.

Chapter III of the proposed PWD introduces protection in relation to key managerial prerogatives executed through technological tools and software. Moreover, Article 6 mandates that platform workers be informed about the existence and specific scope of ‘(a) automated monitoring systems which are used to monitor, supervise or evaluate the work performance of platform workers through electronic means’ and ‘(b) automated decision-making systems which are used to take or support decisions that significantly affect those platform workers’ working conditions’. ⁸⁰ With regard to the systems included in (b), platform workers must be informed about the criteria used to make a decision, the ‘weight’ of each criterion, and ‘the grounds for decisions to restrict, suspend or terminate the platform worker's account, to refuse the remuneration for work, on the platform worker's contractual status’ and ‘any decision with similar effects’.

Under Article 8, platform workers have the right to receive a written explanation of how such decisions were reached. This provision covers ‘any decision taken or supported by an automated decision-making system that significantly affects the platform worker's working conditions’ (emphasis added). The reference to the ancillary role of algorithmic tools gives the provision a broader scope than Article 22 of the GDPR (which is limited to decisions solely based on automated processing). ⁸¹ While the GDPR mandates ‘suitable measures to safeguard the data subject's rights and freedoms and legitimate interests’, the PWD requires a human review of significant decisions upon request by an affected person who has the right to contest a platform's practices. In addition, platforms must provide a written statement concerning semi- or fully automated decisions that have critical effects in areas such as the restriction, suspension or termination of accounts, denial of remuneration for work done, and contractual status. Moreover, workers can request the revision of a decision if the provided explanation does not prove persuasive or they feel hindered in relation to their rights. The platform is obligated to provide a detailed reply within a reasonable timeframe, ‘without undue delay and in any event within one week of receipt of the request’ (Article 8(2)).

The PWD also introduces robust information and consultation duties vis-à-vis workers’ representatives regarding the introduction and alteration of automated monitoring and decision-making systems under Article 9—that is, a model that dwarfs any algorithmic impact assessment. However, the provision cannot be read as a full expression of the right to ‘negotiate the algorithm’, ⁸² nor does it cover genuinely self-employed workers, who are left to fend for themselves. ⁸³ We have strongly decried the fact that automated monitoring and decision-making are seen as inherent to the platform business model, a view that subscribes to the common rhetoric of its natural ‘inevitability’. Nonetheless, the PWD allows collective actors to learn about algorithmic systems before they are implemented and offer inputs concerning the adoption and revision of such systems. It has been demonstrated that the collective dimension is precisely where the social harms precipitated by algorithms reveal their strongest impacts. ⁸⁴ In this respect, the existence of ‘individualised’ and ‘remedial’ data rights represents a blunt instrument because it does not prevent abuses from occurring, nor does it actually empower workers who lack the technological savvy, means and assistance necessary to engage in processes aimed at rebalancing information asymmetries. ⁸⁵

3.2 United States: pressure not to lag behind and preliminary steps

There is a great deal of buzz regarding the percolation of EU regulatory instruments into other jurisdictions due to their extraterritorial effects. This phenomenon is referred to as the ‘Brussels effect’, which reflects the capability of ‘first mover’ EU institutions to exercise power or influence over (the rest of) the world by means of legal norms, standards, and sanctions. ⁸⁶ Although this soft authority concerns the ‘spontaneous’ compliance of companies interested in avoiding the costs associated with multiple regimes and standards, in the United States several acts and bills partially emulate the contents or, at least, the purposes of EU tools such as the GDPR. One key reason for this trend towards homologation is the desire on the part of certain policymakers not to be left behind by EU institutional activism in terms of the governance of technologies and ‘digital constitutionalism’ (i.e., the embedding of indefectible principles and values within the digital domain). ⁸⁷ The EU acts as a ‘regulatory entrepreneur’ due to its commitment to conceptualising and operationalising a ‘European approach’ that combines the ability to reap the benefits of technological innovation with the need to safeguard fundamental rights and values. ⁸⁸ However, the concrete results of this ambitious programme are mixed and do not translate the aspiration for a stronger social dimension into practice, as documented in the preceding section.

Recent developments are bringing the United States ‘closer’ to the EU in terms of the guardrails for AI tools. ⁸⁹ The most prominent examples of this regulatory cross-fertilisation are the California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA), two ‘prototypes’ that may soon be replicated in other states. The CCPA (which came into effect in 2020) gives consumers more control over the personal data that companies collect about them. The previous ‘employee-employer exemption’ expired at the end of 2022. As a landmark law, it secures new-generation privacy rights for Californian consumers and workers, including (i) the right to know about the personal information a business collects about them and how it is used, sold, and shared; (ii) the right to delete personal information collected from them (with some exceptions); (iii) the right to opt out of the sale of their personal information; and (iv) the right to allege discrimination in relation to the exercise of their CCPA rights.

Approved in 2020 and entered into force in 2023, the CPRA represents one of the first attempts to secure privacy and data rights in the United States. It aims to reinforce the provisions of the CCPA. It must be noted that the CPRA significantly expands worker privacy rights by providing notice duties, provisions on the correction of inaccurate ‘personal information’ and limits to the use and divulgation of sensitive personal data. Many commentators have drawn a parallel between this legal instrument and the GDPR. ⁹⁰

In the wake of these important developments (which have together led to the newly dubbed ‘California effect’), ⁹¹ other states have adopted similar models, including the Colorado Data Privacy Act, the Connecticut Data Privacy Act, the Utah Consumer Privacy Act, and the Virginia Consumer Protection Act. There are also some narrower instruments, such as the Illinois Artificial Intelligence Video Interview Act, which has been imposing disclosure, transparency, and erasure duties onto employers that use AI systems to analyse interviews for recruitment purposes since January 2020. Similarly, since October 2020 in Maryland, a new law prohibits employers from using facial recognition technology during pre-employment job interviews without first obtaining the candidate's consent. ⁹² Employers must inform applicants that AI is being used and disclose how the system works and which variables are under scrutiny. In addition, New York City's law 1894-A requires all AI-powered hiring tools to be audited for bias. ⁹³ While this law does not enshrine actionable individual rights, it compels employers to disclose the adoption of any AI instruments used to automate recruitment procedures and conduct annual audits on them, with the results needing to be made publicly available. ⁹⁴

In California, Assembly Bill 701 (AB 701) entered into force in January 2022. It is designed to regulate the use of quotas within logistics warehouses. Certain large companies in the industry are required to disclose allocations and pace-of-work standards to each employee upon hiring. Moreover, a ‘written description of each quota to which the employee is subject to, including the quantified number of tasks to be performed or materials to be produced or handled, within a defined period, and any potential adverse employment action that may result from failure to meet the quota’ must be shared. Employees can disregard any quotas that prevent compliance with meal or rest periods, the use of bathroom facilities, or health and safety laws. ⁹⁵ Another important development in this regard is the California Workplace Technology Accountability Act (AB 1651). ⁹⁶ This proposed Act would confer on workers the right to be informed of, review, correct, and secure data collected from them by the employer. It would also enshrine the principles of data minimisation and purpose limitation, thereby limiting the ability of employers to use data beyond the indicated purposes. The impact of these provisions is compounded by the introduction of limitations on the collection and use of data obtained through electronic monitoring as well as certain redlines concerning the purposes and effects of automated decision-making. Significantly, the Act also requires employers that adopt hiring or management technologies to draft and publish a related impact assessment document.

Relatedly, the state of New York has advanced a more targeted Warehouse Worker Protection Act, which is known as the ‘Amazon Warehouse Act’. Its key purpose is to make ‘productivity quotas’ more transparent within warehouses—that is, workers (and regulators) must be provided with documentation concerning their quotas and notified about any shift in the company's expectations with regard to the pace of work and deliverables. The Act bans the use of excessively stringent quotas that prevent workers from taking breaks for personal reasons. While this focus on quotas is suited to the perception of this instrument as regulating algorithms at work, its overarching purpose is actually to ensure compliance with health and safety regulations. Several news reports have exposed the harsh model whereby workers are reprimanded or even fired for their ‘time off-task’ (i.e., the time they spend on activities other than their legally required 30-minute lunch break). This pressure has also been identified as a source of the higher-than-normal injury and illness rates among Amazon workers. ⁹⁷ Workers must receive information about their quotas within the first month of being hired and have the right to access both current and previous productivity data (up to a maximum of three months). Additionally, they are empowered to pursue a court injunction against any detrimental practices, whereas an excessively high injury rate triggers the possibility of a state investigation.

In October 2022, the White House Office of Science and Technology Policy published the Blueprint for an AI Bill of Rights. Despite its resonant title, the proposed AI Bill of Rights is a white paper, meaning that the included rights are not legally enforceable and are mostly addressed to the Federal Government. The outcome of an input-gathering process involving stakeholders and human rights advocates, the key principles of the Bill are the right to scrutiny over how data is used, the right to opt out of automated decision-making, the right to live free from ineffective or unsafe algorithms, the right to be informed of the fact that AI is making a decision about them, and the right to not be discriminated against by algorithms. Practically concomitantly, General Counsel Jennifer Abruzzo published a memo on behalf of the National Labor Relations Board (NLRB) ⁹⁸ proclaiming the intention to focus on the electronic monitoring of workers which results in chilling effects on the organising and exercising of collective bargaining rights. ⁹⁹ The document stresses the importance of enforcement to the agency's lawyers, who prosecute unfair labour practice cases, and the need to adopt a new framework ‘to adapt the [NLRA] to changing patterns of industrial life’. ¹⁰⁰ Significantly, given their extensive pervasiveness both at work and in private lives, algorithmic management tools and surveillance technologies are identified as systems that potentially impair collective rights.

3.3 Canada: the right to be informed (and not much more)

As mentioned above, in 2022, the Canadian Federal Government tabled Bill C-27, which includes legislation concerning consumer privacy protection (the CPPA) and AI and data (the AIDA). The most striking feature of this ‘package’ concerns the provisions regarding employment included within the CPPA. The proposed Act would apply ‘to every organization in respect of personal information that…is about an employee of, or an applicant for employment with, the organization and that the organization collects, uses or discloses in connection with the operation of a federal work, undertaking or business’. Moreover, section 23 provides that ‘an organization may collect, use or disclose an individual's personal information without their knowledge or consent if it was produced by the individual in the course of their employment, business or profession and the collection, use or disclosure is consistent with the purposes for which the information was produced’ (emphasis added). In addition, section 24 allows organisations to ‘use or disclose an individual's personal information’ without their consent if ‘the collection, use or disclosure is necessary to establish, manage or terminate an employment relationship between the organization and the individual’, provided that ‘the organization has informed the individual that the personal information will be or may be collected, used or disclosed for those purposes’.

Consequently, in federally regulated work (which includes, among others, the banking, mail, and radio and television broadcasting sectors), employers would have unfettered authority to introduce and use electronic and algorithmic management and monitoring systems, with individual employees having no recourse if they were informed about the existence and use of such systems. In unionised workplaces, employers would arguably still be required to negotiate the introduction of such systems under sections 51–55 of the Canada Labour Code. The lack of any substantial protection in all other circumstances is, however, extremely concerning. At the provincial level, the only significant legislation in this regard is Ontario's Working for Workers Act of 2022. Its approach is not dissimilar to that of the CPPA. It mandates that businesses with more than 25 employees adopt a written policy outlining whether they monitor employees electronically, in what fashion and for what purposes, albeit without attaching any substantial protection to these procedural provisions. ¹⁰¹

4. Towards ‘participatory AI governance’?

Our analysis has elucidated the intricacies of the current piecemeal legislation regulating AI systems for automated decision-making in work environments. We have underlined the weaknesses of the existing instruments and presented the perils associated with the risk-based approach. This jigsaw of provisions has two opposite poles: the PWD and the AI Act. The proposed PWD fleshes out credible safeguards for platform workers who are exposed to algorithmic management. Yet, the related set of provisions do not extend beyond the boundaries of platform work, prompting scholars, commentators, and Members of the European Parliament ¹⁰² to call for a more ‘universalistic’ approach, ¹⁰³ as such practices pose enormous challenges to national and EU labour protection systems. The PWD could serve as an ‘experiment’ ¹⁰⁴ able to ‘pave the way for a broader approach to the use of [AI] in the labour market in the near future’. ¹⁰⁵ This shows the genuine ambition of the EU Commission and markedly contrasts with the AI Act's potentially liberalising effects, which represent a significant threat to the working conditions and labour rights of ‘logged-in’ workers. In light of this, it seems all the more urgent to address the tension between the protections enshrined within Chapter III of the proposed PWD and the minimalist intervention of the AI Act.

Given the purposes, ¹⁰⁶ limited scope, and specific legal bases of the PWD, coupled with the option for Member States to design a more strongly protective framework that is enshrined in Article 20, it is safe to suggest that the proposed PWD should be read as a lex specialis to both the ‘horizontal’ AI Act and, to a certain extent at least, the GDPR. Alternative interpretations are plausible, although they would have implicitly abrogative effects on the provisions of the PWD, something that would seem counterintuitive to the purposes of instruments adopted by the same legislative bodies during the same period. ¹⁰⁷ However, this unsolved conflict brings us back to the untenable choice to limit the scope of application of the provisions regulating algorithmic decision-making (at the individual and collective levels) laid down within Chapter III of the PWD to only platform workers. While it is not clear whether genuine self-employed platform workers will be included within the final version of the text, leaving outside the scope of its protection all workers who are not engaged by platforms reduces the impact of these promising provisions, thereby furthering the segmentation of data protection schemes.

In addition, in conjunction with Article 88 of the GDPR, the PWD expressly allows the introduction and application of more vigorous levels of protection by Member States. This provision should, therefore, be used to ‘re-empower’ at the EU level initiatives such as the Spanish Ley Rider, which sets out collective transparency rights with regard to the logic, metrics, and parameters computed by AI tools adopted by companies in all industries and sectors. ¹⁰⁸ It could also be used to strengthen the German Co-Determination Act concerning algorithmic management as well as the national provisions concerning technological surveillance that have long existed in some Member States. The very narrow scope of the proposed PWD leaves us wondering if the choice to introduce ambitious information and consultation rights of platform workers’ representatives (Article 9) in a ‘sectoral’ instrument was appropriate. Several reasons have been advanced in the preparatory documents in support of such a move, although the prominence and recrudescence of algorithmic decision-making well beyond the context of the gig economy render the PWD an idiosyncratic model that is easy to circumvent and difficult to enforce given the ‘compliance entrepreneurship’ applied by platforms. ¹⁰⁹

The AI Act has shortcomings that need to be addressed. In chasing the mirage of harmonisation, the proposed Regulation may end up lowering the bar of protection, despite purporting to fight abuses with an extra layer of procedural burdens whose enforcement is left to the regulatees, whose role may be reduced to ‘rubber-stamping’. To overcome the ‘who-controls-the-controllers’ dilemma triggered by the pre-emptive conformity assessment that the providers of high-risk AI systems must conduct, a possible solution would be shifting the burden of proof onto the providers of AI used for employment purposes. They should be able to demonstrate the ‘innocuousness’ and ‘non-toxicity’ of such systems before placing them on the market by involving the workers affected by AI systems and their representatives. ¹¹⁰ Otherwise, such systems would be considered ‘unlawful by default’. ¹¹¹ Moreover, AI-enabled managerial tools pose a number of risks to workers’ rights well beyond the realm of privacy. Thus, full consistency must be ensured with the EU Charter of Fundamental Rights and the secondary EU legislation concerning consumer protection, equality and non-discrimination, and health and safety. To foster legal certainty, the introduction of an explicit provision preventing any national labour and employment regulation from being ‘dismembered’ under the AI regulation would be useful.

Another pitfall to be avoided is the ‘individualised’ understanding of the related rights and principles, ¹¹² an issue that also concerns the PWD due to its exclusion of persons who perform platform work outside the framework of employment relationships from the collective aspects of the rights to information and consultation with regard to automated decision-making systems. The proposed PWD singles out Article 16(2) of the TFEU as the legal basis for its AI-related provisions, which allows for the adoption of rules ‘relating to the protection of individuals with regard to the processing of personal data’ in order to secure informational self-determination. This wording does not seem to presuppose any distinction based on contractual statuses. A growing body of literature is questioning the atomistic mobilisation of data rights. ¹¹³ This shortcoming is exacerbated by the prevalence of risk-based approaches that typically iron out individual differences and allow any profiles that deviate from the ‘average man’ to suffer ‘the consequences of a policy that treats them as invisible and their needs of little worth’. ¹¹⁴ Here, algorithms and AI tools operate at the level of groups, communities, and populations, particularly in professional contexts. ¹¹⁵ Thus, collective rights represent a way forward, even though practical hurdles such as decreased unionisation rates, scarce collective bargaining coverage, and lack of digital literacy could render them weaker.

The emerging consensus among experts is that the labour-related category included in the list of high-risk AI systems should be entirely scrapped from the AI Act. ¹¹⁶ In a similar vein, the narrow provisions concerning algorithmic management that cover platform workers leave a great deal to be desired in terms of their scope of application. The now-pervasive role of AI constitutes a challenge in almost all socio-economic areas and, therefore, requires interventions capable of overcoming the logic of watertight regulatory silos. Professional relationships are always unbalanced when it comes to bargaining power and information asymmetries. This condition is exacerbated by opaque data-driven tools when they are used without collective scrutiny. Hence, it will be vital to ensure that the EU social acquis is enforced and strengthened in the legislative instruments that are currently under discussion. ¹¹⁷ At the same time, it would be short-sighted to assume that the existing frameworks are sufficient. While the organisational structures and strategies facilitated by AI tools are certainly not novel, their scale, scope and volume are genuinely unprecedented. A more fruitful way to make fundamental rights work would be the adoption of a dedicated EU instrument regulating technology in workplaces and providing a non-waivable level of protection.

Until recently, the introduction of productivity-enhancing technological systems in work contexts was focused on business assets owned by the employer, such as machinery and goods, which sometimes resulted in significant indirect effects on workers (e.g., the introduction of assembly lines impacted both tasks and pace). These changes were often considered part of employers’ managerial prerogatives and, therefore, protected under the freedom to conduct business, even if, in certain jurisdictions, co-determination and information and consultation rights ‘tempered’ the unilateral exercise of such prerogatives. It is questionable, however, if the same conceptual apparatus and legal framework centred on the freedom to conduct business can be borrowed for technologies whose primary object is workers’ physical body and intellectual conduct, as in the case of AI-enabled surveillance and management systems. These systems target people, not goods or assets. This suggests that the primary thrust of any regulation of such systems should be centred on the protection of fundamental rights, not on economic freedoms and techno-deterministic assumptions whereby anything technically possible should be allowed.

The impacts of AI and (labour) algorithms on the nature, organisation, and quality of work are rarely advantageous. ¹¹⁸ Several successes have been achieved thanks to strategic litigation, with trade unions having mobilised data protection, transparency and non-discrimination rights and acted as collective claimants before national DPAs and labour courts. ¹¹⁹ Nevertheless, such achievements cannot compensate for the imbalance between informational and bargaining powers, as they are issued on a case-by-case basis and often only bind those parties to the relevant legal action. Thus, concrete advancements in the field are slow to materialise, which makes the case for both targeted new regulations and the recalibration of existing rules more urgent. ¹²⁰ In searching for a better solution to a pressing problem than new instruments such as the AI Act or the North American instruments, and in line with the pledge made by the General Counsel for the NLRB, we conclude with a call to advance ‘participatory AI governance’, namely bottom-up approaches that promote multi-stakeholder involvement in order to ensure that due process and fundamental rights are respected in the work context.

In light of the high stakes involved, the promotion of compliance cannot be left solely to the developers and adopters of such technologies. Even assuming that all ‘conformity boxes’ are ticked ¹²¹ and, therefore, that AI tools receive a ‘compliance green light’, power gaps remain wide open because the extent of managerial prerogatives is inordinately intensified when such tools are adopted in workplaces. Given the limitations of current regulatory frameworks and the uncertainty brought about by partially overlapping models, and while enforcing current regulations remains a crucial priority for authorities, it is time to envision a comprehensive supranational instrument that regulates the use of technologies that perform monitoring, decisional, and disciplinary functions in professional ecosystems. A meaningful shift towards human-centric models requires collective codetermination and, importantly, co-design of AI- and algorithm-based systems throughout their entire life cycle, whereby representative bodies are involved to ensure that innovations are deployed in such a way as to not impinge on fundamental labour rights.