Algorithm Registers: A Box-Ticking Exercise or Meaningful Tool for Transparency?

Abstract

Governments increasingly implement algorithm registers—openly accessible overviews of algorithms used in decision-making—to promote transparency and accountability. However, the contribution of these registers is questioned: some scholars claim that governments only disclose politically non-sensitive algorithms and provide information with limited usefulness for accountability purposes. In contrast, a more optimistic view is also put forward: registers help organizations to be transparent to the public, which increases public trust in algorithm use. In response to this academic debate, this paper aims to provide a theoretical understanding and an empirical mapping of the different factors that shape algorithm registers and their implications. I conducted in-depth interviews with 27 respondents, both developers (public organizations) and key users (oversight authorities and societal watchdogs) of algorithm registers in the Netherlands, and I analyzed 33 policy documents. The findings nuance the current criticisms. While I find that public organizations indeed selectively disclose information and registers are currently not found useful by societal watchdogs and oversight authorities, there are also dynamics that could contribute to more responsible use of algorithms by public organizations, such as a potential disciplining effect of registers. This study highlights the importance of going beyond the rhetoric on algorithm registers, by establishing a deep understanding of the indirect and unexpected dynamics they create. The paper concludes that algorithm registers are currently not a meaningful tool for transparency, but a meaningful box-ticking exercise for public organizations.

Key Points for Practitioners

Algorithm registration can create organizational awareness and learning about algorithms, fostering a disciplining effect wherein public organizations critically evaluate their algorithmic processes, identify risks, and formulate mitigation strategies.

Users of algorithm registers should be cautious in their interpretations of algorithm registrations, as these may only present a selection of (information about) the algorithms in a public organization.

Government organizations should consider reevaluating the goals of algorithm registers, shifting the focus from providing transparency about algorithms for citizens to providing transparency for intermediaries, such as oversight authorities and societal watchdogs.

1. Introduction

Scholars and societal watchdogs are concerned about the increasing use of algorithms in government decisions and services. A core criticism is that algorithms lack accountability and transparency (Busuioc et al., 2023). In response, (local) governments all over the world are experimenting with setting up algorithm registers, and the European Union recently implemented the Artificial Intelligence (AI) Act, mandating the registration of high-risk algorithms (Wörsdörfer, 2023). An algorithm register is a database or record-keeping system that shows the deployment and use of algorithms within an organization. It typically includes information about the type of algorithm, its purpose, the data it uses, and any potential risks associated with its deployment (Haataja et al., 2020). These registers could offer insight into the specific algorithms used for public tasks to intermediaries, including NGOs, journalists, independent experts and regulators (Grimmelikhuijsen & Meijer, 2022), thereby easing the burden on individual citizens.

Despite the hope that algorithm registers will solve the issue of algorithmic opacity, there is a scarcity of empirical research about what algorithm registers are, and what these registers yield. Most of the existing research on algorithm registers has a conceptual approach, rather than presenting actual data (Cath & Jansen, 2022; Floridi, 2020). The few scholars discussing this new phenomenon have contrasting opinions. There are “optimists” who argue that algorithm registers help both government organizations and the public to comprehend the societal impact of algorithms, thereby increasing public trust in algorithm use by governments (Floridi, 2020). “Pessimists”, on the other hand, argue that disclosure of algorithms through these registers is limited, because public organizations have substantial discretion regarding the information they communicate about an algorithm (Cath & Jansen, 2022).

Beyond the few conceptual studies, little is known about what algorithm registers are and how they can be designed. This is important to know, especially considering the requirement outlined in the AI Act to register high-risk algorithms (Wörsdörfer, 2023). Therefore, this paper seeks to empirically investigate the current use of algorithm registers by answering the following research question: What is the nature of algorithm registers, and what are their implications? In doing so, we can complement the conceptual studies, and generate new concepts and theories about the promises and limitations of algorithm registers for governments and relevant stakeholders.

To investigate the nature and implications of algorithm registers, I applied two different methods. First, I conducted in-depth, semi-structured interviews with 27 respondents: developers (civil servants responsible for algorithm registers) and users (oversight authorities and societal watchdogs) of algorithm registers. Second, I conducted a comprehensive analysis of 33 policy documents to explore considerations underlying the design choices for algorithm registers and their implications. This mixed-methods approach allows for an in-depth investigation of the nature of algorithm registers and their perceived implications.

This research was conducted in the Netherlands. The Dutch context is interesting because government organizations have embraced digitalization and they are pioneers in openly disclosing the utilization of algorithms in the public sector (Floridi, 2020). The Netherlands and Finland were the first countries to publish an algorithm register for public organizations to address transparency concerns and promote the responsible use of algorithms (Floridi, 2020). Several other countries increasingly recognize the importance of such registers. For instance, a collaborative effort among nine major European cities, including Barcelona, Brussels, Sofia, and Mannheim, has resulted in the development of an Algorithmic Transparency Standard—a shared data schema for algorithm registers that is validated, open-source, publicly available, and ready for implementation. ¹ Finally, algorithm registers are not limited to Europe. Cities like New York and Toronto have expressed their intention to establish such registers in the near future ² and the city of San Jose in California (United States) has already published several algorithms in their municipal algorithm register. ³

This study makes three significant contributions to the literature on and the field of AI governance and transparency. This paper sheds light on how public organizations sometimes partially, or even selectively, disclose information about algorithms. This practice highlights how formal transparency requirements may seem adequate in theory but often fail to offer real insights into the practical realities of algorithmic decision-making, showing a form of decoupling in the context of algorithmic accountability. At the same time, the study reveals a disciplining effect within public organizations, through comprehensive engagement in algorithm registration. This process compels critical reflection on algorithm use, illustrating the relational aspect of transparency, emphasizing mutual understanding and learning among stakeholders involved in the deployment and use of algorithmic decision-making processes. Finally, it offers a conceptual contribution by providing a nuanced mapping of the different factors that play a role in shaping algorithm registers and their broader societal and internal implications. The detailed insights into algorithm registers provide a deeper understanding how they potentially shape transparency, accountability, and trust in government organizations. As such, the research serves as a valuable resource for practitioners and researchers, facilitating the establishment of effective algorithm registers globally.

2. Research Context: The Netherlands as Forerunner

This exploratory qualitative study was conducted in the Netherlands, a leading country in the adoption of algorithm registers. To understand the context in which this study took place, two important elements should be described. First, the Netherlands is an early adopter of new technologies, and there is widespread use of algorithms in digital public services (Floridi, 2020). The Netherlands ranks among the top 5 countries in the DiGix index, a multidimensional index of digitization, showcasing a robust digital infrastructure and progressive approach to technology integration (Cámara, 2022).

Second, there have been a few major scandals related to the use of algorithms by the government in the Netherlands. One of the most well-known is the childcare benefits scandal, also known as the “toeslagenaffaire”, which involved the wrongful accusation of thousands of parents of fraudulently claiming childcare benefits between 2013 and 2019. Many of these parents, often from immigrant backgrounds, were forced to repay large sums of money, leading to severe financial and personal distress (Konaté & Pali, 2023). The scandal was intensified by the use of algorithms within the Dutch Tax Authority, which contributed to discriminatory practices and aggressive enforcement policies (Alon-Barkat & Busuioc, 2023). This crisis led to widespread public anger, government apologies, and the resignation of the entire Dutch cabinet in January 2021, sparking calls for greater transparency and accountability in the use of governmental algorithms (Bouwmeester, 2023).

One approach to enhancing such transparency is through the implementation of an algorithm register. In 2020, the City of Amsterdam launched one of the first algorithm registers in the world, followed by a national algorithm register in 2022 that has been launched by the Ministry of the Interior and Kingdom Relations in the Netherlands. Public organizations can decide whether to create their own register or use the national register. Within the Dutch context, the national algorithm register thus coexists alongside decentralized algorithm registers of public organizations. Figure 1 displays the homepage of the national algorithm register, offering an illustration of its appearance.

Figure 1.

Homepage of the national algorithm register of the Ministry of the Interior and Kingdom Relations (The Netherlands). ⁴

Notably, at the time of the research, there were no regulations stipulating the design of or content requirements for these registers. This lack of standardization led to a broad diversity in algorithm registers, rendering the Netherlands a very suitable context for studying the various approaches and challenges associated with algorithm registers. Although the Dutch context is unique in terms of the widespread use of algorithms by the government and the significant call for transparency following several scandals, the global attention to algorithm transparency is also rapidly increasing (Watson & Nations, 2019). There are various forms, ways and shapes to establish transparency about algorithms beyond an algorithm register, and this study provides interesting results in those respects as well.

3. Algorithm Registers: Nature and Implications

Scholarly attention toward the transparency and accountability of algorithms has intensified over the past few years. Several recent studies have shown that providing transparency about algorithmic decisions or recommendations can increase public trust in these algorithms (e.g., Grimmelikhuijsen, 2023; Nieuwenhuizen et al., 2025; Schiff et al., 2022). Much of this current research focuses on algorithmic transparency towards individual citizens. While valuable, this individual approach to algorithmic transparency requires significant effort from individuals to interpret and critically assess algorithmic decision-making. In addition, this approach neglects the broader institutional context in which they operate (Grimmelikhuijsen & Meijer, 2022; Nieuwenhuizen, 2025).

A broader organizational or institutional approach is required to increase accountability (De Bruijn et al., 2022; Grimmelikhuijsen & Meijer, 2022). In such an approach, not only information about specific algorithms should be presented, but also about their usage, governance and evaluation, and not just to citizens but also to intermediaries, such as NGOs, journalists, independent experts and regulators (Grimmelikhuijsen & Meijer, 2022). This could lead to more public trust in how governments embed algorithms in their organization (Nieuwenhuizen, 2025). Algorithm registers have been proposed to realize such an approach, yet there is still limited knowledge about these registers.

The first section focuses on the definitions of algorithm registers, their intended goals, and underlying design considerations. The second section discusses the potential implications of algorithm registers. These sections rely on conceptual studies on algorithm registers and a few empirical studies. This literature review establishes the foundation for developing sensitizing concepts that will guide this qualitative study.

3.1 What are Definitions, Goals and Design Considerations Regarding Algorithm Registers?

3.1.1 Definitions

Algorithm registers have emerged as a critical instrument in the pursuit of effective and responsible integration of artificial intelligence (AI) within public organizations (Floridi, 2020). But what are these registers? Murad (2021, p. 16) is one of the first to define an algorithm register as “a log of algorithmic decision-making systems used by a public authority that have some level of direct impact on its citizens”. This includes not only technical specifications but also governance structures that regulate their application. This definition emphasizes the dual role of algorithm registers in shedding light on both the algorithms themselves and the surrounding decision-making processes. Cath and Jansen (2022) further underscore the significance of algorithm registers in providing transparency about decision-making mechanisms that rely on algorithms. They stress that algorithm registers offer insights into the rationale behind algorithmic decisions as well as potential biases inherent in such technologies. Algorithm registers, at their core, thus serve as comprehensive databases that catalog detailed information about the algorithms employed by public organizations. Cath and Jansen (2022) also include the broader context of how these algorithms are utilized in decision-making processes within the public sector. I start with these broad notions of algorithm registers. Then, I aim to get a better understanding of how algorithm registers are defined in practice and to what extent this aligns with the theoretical definition.

3.1.2 Goals

Although the scientific studies on algorithm registers are of a conceptual nature, several students have empirically investigated the design, objectives or effects of these registers (Bottenbley, 2023; de Meijer, 2023; Murad, 2021). They illustrate that algorithm registers serve multiple purposes, primarily focusing on promoting transparency, offering avenues for sanctions and redress, and fostering knowledge sharing and learning in the public sector.

The primary objective of algorithm registers is to promote transparency about the utilization of algorithms (Murad, 2021). This entails making the inner workings of algorithms and the decision-making processes behind them transparent to stakeholders. This allows stakeholders to gain insights into the rationale and potential biases embedded within algorithmic decision-making (Kaminski, 2020). Transparency can be categorized into two levels: first- and second-order. First-order transparency primarily focuses on unveiling the technical aspects of an algorithmic system, encompassing its design, implementation, and functioning. This allows stakeholders to understand the logic of the system, ideally exposing any biases and negative impacts, if any (Kaminski, 2020). Second-order transparency goes beyond the technicalities of algorithms and delves into the governance structures and decision-making processes surrounding algorithms. The purpose is to ensure that those responsible for governing algorithms are transparent and accountable for their actions. Second-order transparency acts as a safeguard, preventing power abuse and promoting responsible AI governance (Kaminski, 2020).

Second, building upon the first goal, algorithm registers could play a significant role in providing opportunities for sanctions and redress in the case of misuse (Busuioc, 2021). By maintaining a comprehensive record of algorithm deployments and their associated governance structures, algorithm registers can enable the identification of instances where algorithms are utilized unethically or harmfully (Cath & Jansen, 2022). This not only serves as a deterrent against misconduct, but also offers a mechanism for citizens and stakeholders to seek remedies and hold public organizations accountable for any wrongdoings (Busuioc, 2021).

Third, algorithm registers can foster knowledge exchange and learning among public organizations. These registers could serve as comprehensive repositories of information concerning algorithm implementation, facilitating public organizations at all levels to gain insights and learn from each other's experiences (Murad, 2021). This collaborative approach aims to prevent redundancy, promote efficiency, and encourage widespread adoption of best practices in the domain of AI governance (Floridi, 2020). I will use these three initial objectives from the literature to explore existing goals and determine if any additional goals arise.

3.1.3 Design Considerations

When creating algorithm registers, there are several aspects to think about regarding how they should look and work. This section describes the key considerations drawn from existing research. The first decision is to determine the unit of disclosure. Organizations must choose what they define as an algorithm, whether to register all algorithms in use, or focus solely on those deemed high-risk (Murad, 2021). Cath and Jansen (2022) argue that the unit of disclosure is inherently a political choice, impacting the register's comprehensiveness and reflecting the organization's values and priorities in fostering transparency and responsible algorithm use.

A second important consideration in the design of an algorithm register is determining its intended audience. These registers can cater to distinct user groups, including citizens, public-interest representatives, or government and policy officials (Floridi, 2020; Murad, 2021). Each group has different needs and levels of technological sophistication: citizens want to know how algorithms have impacted specific government decisions. Public interest representatives, including researchers, investigative journalists, and advocacy groups seek information about algorithms to safeguard public or stakeholder interests (Murad, 2021). Public servants and policymakers constitute another user group, as registers offer insights into which algorithms in government can help improve policy formulation and decision-making (Floridi, 2020; Murad, 2021).

A third decision concerns the type of information, the relevant disclosures, to be made transparent in the algorithm register. These disclosures, however, are not one-size-fits-all but rather intricately linked to the intended audience and objectives of the register. To avoid a “false sense of transparency,” it is vital to recognize that sharing complex technical details, such as source code and raw data, with the public is often impractical (Murad, 2021). By contrast, governments can decide only to disclose information relevant to users, such as how data input leads to a specific recommendation or decision (Cheng et al., 2019).

Finally, organizations should decide on the disclosure modality, which relates to the presentation of information and compatibility of data (Murad, 2021). Designing the interface requires practical consideration of accessibility factors, such as language selection and font size, and easy navigation and location of relevant information for different target audiences. According to Murad (2021), an interactive interface with layered information is the best approach to address the diverse needs of stakeholder groups. This involves presenting basic information that is accessible to all user categories upfront. More technical information can then be presented in specific fields to address the information requirements of other intended user groups. These four design steps will serve as a lens that structures the data gathering on design considerations for algorithm registers.

3.2 What are Potential Implications of Algorithm Registers?

Algorithm registers are generally expected to have several positive impacts. Foremost, algorithmic transparency carries the promise of enhanced accountability and citizen trust (Busuioc, 2021; Floridi, 2020). The idea is that by offering a comprehensive and accessible repository of information on algorithms and their utilization, these registers lay the groundwork for citizens to scrutinize, evaluate, and ultimately trust the ethical and responsible use of algorithms in government operations (Haataja et al., 2020; Seppälä et al., 2021). Secondly, algorithm registers could play an important role in promoting the responsible use of algorithms within organizations, aligning the register closely with ethical principles and regulatory frameworks (Harish et al., 2022). As Murad (2021) suggests, these registers serve as a “cornerstone” to ensure that algorithms are employed in a responsible and ethical manner within the public sector. Overall, the promise of algorithm registers is their ability to enhance transparency and accountability, foster citizen trust, and promote responsible algorithm use within government organizations.

On the other hand, algorithm registers also face fundamental challenges that may prevent such positive outcomes. First, there is a risk that registers create what has been called “ethics theater”, where the importance of algorithm registers is exaggerated, potentially leading to the creation of false narratives about the risks and benefits of algorithms (Cath & Jansen, 2022). Public organizations might establish algorithm registers to merely appear ethical and transparent, essentially using them as a facade to project responsible algorithm use while sidestepping deeper ethical and practical issues surrounding algorithms. A second criticism concerns the absence of transparency obligations for algorithm registers (Busuioc et al., 2023). While the recently adopted AI Act includes a registration requirement for high-risk algorithms, it remains unclear how EU member states will implement this transparency obligation. In the absence of (legal) requirements regarding the adoption and filling of algorithm registers, these registers can become a token gesture rather than a robust tool to ensure transparency and accountability. These discussed potential positive and negative implications serve as sensitizing concepts in the empirical research (Bowen, 2020).

In sum, while expectations are high, scholars also criticize algorithm registers as potentially ineffective and even risky. So far, however, there have been hardly any empirical accounts on the nature of algorithm registers and their implications. Therefore, this article presents an in-depth empirical account to map the characteristics of algorithm registers and their perceived implications.

4. Methods

The context in which this study has been conducted, is extensively described in the “Research context” section. The next sections make the choices related data collection and analysis explicit, following the recommendations of Ashworth et al. (2019) to be transparent in qualitative research reporting.

4.1 Data Collection

This study involves in-depth semi-structured interviews with 27 respondents (developers and users of algorithm registers) and an analysis of 33 relevant policy documents.  The data collection took place between May 2023 and January 2024. The next section begins with a discussion of the semi-structured interviews, followed by a review of the document analysis.

This research seeks to explore the extent to which the discussed conceptual frameworks regarding algorithm registers are reflected in practice. Therefore, the literature section serves as a starting point for the topics covered in the interviews. To accommodate potential new findings related to definitions, goals, and design considerations of algorithm registers, semi-structured interviews were considered most suitable. This approach allowed for deviations from the initial questions if respondents introduced new insights (Bryman, 2016). Further details on the emergence of new topics are provided in the “Data analysis” section.

The guiding questions for the interviews can be found in the supplementary materials (Annexes B and C). Each interview was conducted by the author and lasted for an average of one hour. Eleven respondents were interviewed in their workplace, while 16 respondents were interviewed online using MS Teams, according to their preferences.

Recordings were made of each interview with the permission of the interviewees. The recordings were transcribed and then anonymized. The transcription process was conducted in two steps. Initially, the transcription software Amberscript was employed to automatically generate a verbatim transcript. This software, provided by the author's institution, adheres fully to GDPR regulations, ensuring the highest level of data protection. Subsequently, either the author or a research assistant listened to the interview and made necessary corrections to the generated transcript.

To gain insights into algorithm registers from different perspectives, a purposive sampling strategy for selecting respondents was used in this study (Robinson, 2014). The author identified two categories of respondents that were crucial for understanding the current practices around algorithm registers. First, the developers of these registers: public organizations. Second, important users of algorithm registers: oversight authorities and societal watchdogs. Table 1 presents an overview of the respondents. The author gained access to the field through two methods: by requesting colleague-researchers to facilitate contact with individuals from the selected organizations, and by searching for suitable respondents from the selected organizations on LinkedIn and directly messaging them to invite them to participate in the research.

Table 1.

Overview of Respondents.

Category	Type of Organization	Number of Respondents
Public organization (algorithm register)	Ministry of the Interior and Kingdom Relations (Ministerie van Binnenlandse Zaken en Koninkrijksrelaties)	2
	Employees Insurance Agency (Uitvoeringsinstituut Werknemersverzekeringen)	1
	Social Insurance Bank (Sociale Verzekeringsbank)	1
	City of Amsterdam (Gemeente Amsterdam)	1
	City of Utrecht (Gemeente Utrecht)	1
Oversight authority	Netherlands Court of Audit (Algemene Rekenkamer)	1
	(Municipal) Court of Audit Rotterdam (Rekenkamer Rotterdam)	2
	Data Protection Authority (Autoriteit Persoonsgegevens)	1
	National Ombudsman (Nationale Ombudsman)	2
	(Municipal) Ombudsman Metropolis Amsterdam (Ombudsman Metropool Amsterdam)	3
	National Inspectorate of Justice and Security (Inspectie Justitie en Veiligheid)	1
	Netherlands Institute for Human Rights (College voor de Rechten van de Mens)	1
Societal watchdogs	Association of Dutch Municipalities (Vereniging van Nederlandse Gemeenten)	2
	Expert citizen	1
	Investigative journalism	3
	Non-governmental organization (NGO)	4
Total		27

The author chose the public organizations by using two criteria. First, the organization needed to be at the forefront of the registration process, with multiple algorithm registrations in their register. This criterion was essential to enable the author to inquire about various stages of the design process and to explore its implications. Second, the author aimed for diversity within the category of frontrunners. This meant differences in 1) the design of the register, and 2) the type of organization responsible for the register (ministry, administrative agency, and municipality). This diversity in the type of organization and algorithm registers allows for a comprehensive examination of best practices, challenges, and variations in the implementation of algorithm registers across different contexts.

The following five registers met these criteria: the algorithm register of the Ministry of the Interior and Kingdom Relations, the Netherlands Employees Insurance Agency (“UWV”), the Dutch Social Insurance Bank (“SVB”), the Municipality of Amsterdam, and the Municipality of Utrecht. Although all are considered frontrunners, these organizations vary in their types and approaches to algorithm registration. For instance, the Municipality of Amsterdam extensively registered a few algorithms through an interactive webpage. In contrast, the Municipality of Utrecht stated that they registered all deployed algorithms, but provided only basic information about all these algorithms in an Excel sheet. The interviewed respondents held partial or full responsibility for the algorithm register in their respective organizations, with job titles ranging from public managers to policy advisors. Throughout this study, this group will be called public managers.

The oversight authorities were chosen based on their formal and informal roles in overseeing public organizations and their use of algorithms. Each authority approaches its mandate uniquely; for example, the Court of Audit scrutinizes the legality and efficiency of Dutch government expenditures, whereas the Data Protection Authority supervises the application of the General Data Protection Regulation. Despite these diverse focuses, all seven selected authorities share a common task: they monitor aspects of the work of public organizations that are progressively becoming more intertwined with algorithms.

The term ‘societal watchdogs’ in this study refers to organizations, groups, or individuals within society that monitor or oversee certain aspects of social, political, or ethical behavior. This includes entities such as investigative journalism and civil society organizations (Karadimitriou et al., 2022; Trägårdh et al., 2013). Societal watchdogs play a role in keeping a check on government organizations to ensure transparency, accountability, and adherence to societal norms or values. Four NGOs and two investigative journalists were identified as relevant societal watchdogs for inclusion in this study due to the policy documents, news articles and statements they had written about transparency about public algorithms in the Netherlands. During the interviews, another journalist, an advocacy organization for Dutch municipalities and a critical citizen were proposed by respondents as important experts regarding the topic of algorithm registers. This resulted in interviews with 10 respondents from nine societal watchdogs.

The 33 documents used in this study were policy documents addressing the design, use or implications of algorithm registers. Initially, relevant public policy documents from all organizations listed in Table 1 were gathered, totaling 24 documents. Subsequently, all interviewees were consulted to identify any potentially overlooked relevant documents, resulting in the inclusion of an additional nine documents. In the results section, passages from documents are linked to “(DX)”, where X represents the document number as outlined in the supplementary materials (Annex A).

4.2 Data Analysis

The interviews and documents were coded and analyzed based on the two themes in the theoretical framework (Boeije & Bleijenbergh, 2019). First, the algorithm registers itself: how do stakeholders perceive these registers, what are their goals, and what are the design considerations? Second, the implications: what are positive and negative perceived implications of algorithm registers?

The coding process started by coding interviews. Five interviews (24%) were coded by the author and two other researchers to ensure inter-coder reliability in the use and expansion of the code tree (Bryman, 2016). A detailed description of the coding process can be found in the supplementary materials (Annex D). In assessing inter-coder reliability, the Kappa scores were computed for three coder pairs. The agreement between coders A and B yielded a Kappa score of 0.85. For coders A and C, the Kappa score was 0.77. Coders B and C demonstrated an agreement with a Kappa score of 0.88. The overall inter-coder reliability, represented by the average Kappa score, was computed as 0.83, with the calculation taking into account the weighted average based on the number of codes in each comparison. The author coded the remaining 76% of the interviews and all documents.

The qualitative data analysis software NVivo 14 was used for coding. The coding tree can be found in the supplementary materials (Annexes E and F). Only themes that were mentioned by six or more different sources (interviews or documents) were included in the coding scheme, to limit the analysis to the important findings and ensure a focused exploration of the key outcomes. After coding the data on algorithm registers and their implications, the author identified several patterns in the data. Subsequently, the author collaborated with three senior researchers, experts in the field of responsible and trustworthy use of algorithms in government, to review and make necessary modifications to the identified patterns based on the coding scheme.

5. Results

This results section addresses the two leading research questions: What empirical insights can we gain about 1) the nature of algorithm registers, and 2) their implications?

5.1 Nature of Algorithm Registers

The insights gathered from interviews and documents regarding algorithm registers will be presented in a structured manner through three steps. Firstly, I discuss how public organizations, oversight authorities and societal watchdogs perceive algorithm registers. Secondly, the focus shifts to identifying the goals that algorithm registers intend to achieve. The third step involves a thorough examination of four important design considerations related to algorithm registers.

5.1.1 Perspectives

The analyses revealed three perspectives regarding what an algorithm register represents: a starting point, an endpoint, or a governance mechanism. In a public organization, the chosen perspective significantly dictates the objectives and the design of the register. Simultaneously, the perspective on algorithm registers plays a key role in defining the expectations of oversight authorities and societal watchdogs, outlining what they anticipate a register to encompass.

The starting point perspective views the algorithm register as a basic reference or initial stage in the process. It primarily serves as a repository, listing algorithms with minimal details about them. The emphasis is on providing stakeholders with an overview of algorithms in use, without delving deeply into their specifics. The objectives and design of the register are influenced by the need for simplicity and accessibility, focusing on a straightforward catalog of deployed algorithms. Most oversight authorities and a few public organizations adhered to this perspective. A public manager illustrates this view on the register:

It is the table of contents of your book and the back, and people who want to read it, buy it and explore further.

Conversely, the end point perspective positions the algorithm register as the last step in the algorithm lifecycle. Here, transparency and accountability take center stage. The register becomes a means for the public and relevant parties to access detailed information about algorithms, emphasizing the inner workings and characteristics. Respondents, mostly public organizations themselves, express the wish for publishing algorithms in a register to become a standard and obligatory part of a public organization's service or product process when algorithms are used. This would require providing updates to the algorithm register when changes occur in these processes. This perspective underscores the importance of openness in algorithmic processes, serving as a tool for scrutiny and oversight.

Finally, the governance mechanism perspective sees the algorithm register as a tool for managing and regulating algorithm usage within a public organization. Both societal watchdogs as well as several public organizations perceived algorithm registers in this manner. In addition to extensive information about algorithms, this perspective incorporates processes and policies for updating, reviewing, and assessing algorithms. The register becomes an integral part of an organization's broader governance framework, aiming to ensure compliance, monitor performance, and mitigate risks associated with algorithmic processes. Some public organizations mention taking various measures to handle algorithms more responsibly and consciously, such as designing an algorithm policy for the organization or establishing an internal ethical committee that evaluates algorithms, as this quote from an organizational policy document of a public organization illustrates:

The SVB [Social Insurance Bank] establishes a quality management system for the deployment of the algorithm, encompassing its development, use, and impact. This involves considering the context of the process in which the algorithm will be embedded and determining the measures necessary to ensure a proper alignment. For example, it is important to assess whether the knowledge level of the employees who will be working with the algorithm aligns well with how the algorithm is intended to be used. Effective quality management for algorithm deployment requires attention to both the technical aspects of implementing this technology and handling data, as well as consideration for the ethical, organizational, and human aspects of algorithm use. (D30)

5.1.2 Goals

Following from the different perspectives on what algorithm registers are, there are also different views on the goal of these registers. Firstly, many public organizations believe that using algorithm registers can make how their internal algorithms work, more transparent, ultimately aiming to build trust in the government. A policy paper from the Ministry of the Interior and Kingdom Relations outlines a goal tree for their algorithm register, identifying seven key objectives. The top priority is enhancing trust in the government:

The government is there, among other things, to provide social added value. The government can only do this effectively if there is trust in the government and people feel heard and involved. The algorithm register should contribute to improving trust in the government. (D18)

Secondly, transparency serves not only to enhance trust but also as an important objective in itself. Respondents indicate several types of transparency an algorithm register should aim for. This distinction closely aligns with the previously discussed difference between first-order and second-order transparency. On the one hand, respondents that favor first-order transparency primarily focus on disclosing the technical aspects of an algorithmic system itself, encompassing its design, implementation, and functioning. According to these respondents, this type of transparency aims to understand the rules and logic of the algorithm. This level of transparency is instrumental in comprehending which algorithms are implemented and how civil servants utilize them in their work. On the other hand, respondents that aim for second-order transparency argue that algorithm registers should disclose the governance structures and decision-making processes surrounding algorithms. The following quote from an investigation report by a municipal court of audit illustrates this perspective:

For a proper safeguarding of the ethics and quality of algorithms, insight into the usage and nature of these algorithms is necessary, especially since existing GDPR legislation is insufficient. The initial version of an algorithm register does not yet provide this insight adequately. As a result, there is no understanding of where ethical risks may arise and what corresponding control measures need to be implemented. The nature of the algorithms determines the severity of the control measures. (D28)

Respondents that aim for this type of transparency with an algorithm register explain that its purpose is to ensure that those responsible for governing algorithms are transparent and accountable for their actions.

Finally, respondents have identified additional goals (in policy documents), reflecting downstream effects of transparency. These include enhancing government accountability, promoting responsible algorithm usage, creating a dialogue with society about algorithms, fostering knowledge exchange among public organizations, and empowering citizens by strengthening their position relative to the government.

5.1.3 Design Considerations

Next to more strategically laden considerations about the perspectives on and goals of registers, I found important differences in the operationalization of the design of algorithm registers. Four important design considerations guide public organizations in the process of instituting an algorithm register.

First, a public organization must decide on what they want to be transparent about, known as the unit of disclosure. This decision is influenced by several choices. Public organizations must define what an algorithm is to determine which data applications are included in the register. The interpretation of an algorithm varies among organizations, with some adopting a narrow definition, while others opt for a broader scope, as this public manager illustrates:

We do not want only self-learning algorithms or only high-impact algorithms, but rather a very broad spectrum. Well, even an Excel file with just one formula behind it, so to speak, is already a set of rules, and we consider it as an algorithm.

The debate further centers around the choice between including all algorithms a public organization deploys versus focusing solely on high-risk or high-impact algorithms in the register. Although many public organizations express a preference in policy documents for only registering algorithms with a significant impact on citizens, the lack of clear definitions of what these high-risk or high-impact algorithms are, often leads oversight authorities and societal watchdogs to advocate for the inclusion of all algorithms in the register. In situations where public organizations include all algorithms in the register, it is often observed by stakeholders that there is limited information provided about these algorithms. The interviews and documents show a trade-off between the level of completeness (from zero to all algorithms) and the depth of the information (from no to all information) disclosed. Lastly, there is a consideration regarding the relation between an external register for the public and an internal register for the organization. While understanding the necessity for non-disclosure in the realm of justice and security, oversight authorities and societal watchdogs argue that for most public organizations, there are no justified reasons for limited transparency, as explained by this NGO researcher:

What is the reason to keep something non-public and not disclose anything about it at all? I generally find that quite objectionable. Of course, there may be a reason why, for example, the police's wiretapping room works with certain algorithms that are not disclosed. […] However, when it comes to the municipality, I don't quite see why the municipality of Amsterdam would have algorithms that need to be kept secret. […] So, if you say, yes, this can be very painful if it comes out, well, that might be precisely the reason it should be made public.

A second consideration is the audience for the register. Respondents identify various intended users, with ‘the citizen’ consistently ranked as the primary audience. A policy officer from an NGO briefly summarizes this argument:

For people who don't know much about algorithms but are interested in how decision-making affects them. So that they can see at a glance: oh yes, here an algorithm was used, I was selected because I meet XYZ criteria, and that algorithm is trained in this way, these were the risks, and they are mitigated in this way; so the most basic information.

In addition to citizens, journalists, NGOs, businesses, researchers, legislators, and oversight authorities are also mentioned as intended users. Some organizations distinguish in policy documents between users without technical backgrounds and those with varying degrees of technical knowledge. It is important to note that the intended users do not completely align with the signals organizations receive about the usage of their registers. When public organizations with an algorithm register were asked by the researcher about who provides feedback or inquiries about the register, they typically mention journalists, NGOs, and researchers. Employees of public organizations indicate that they rarely, if ever, receive questions from citizens, although they cannot precisely trace the actual visitors to the registers.

Thirdly, organizations decide what type of information is made transparent, also referred to as the relevant disclosures. Due to different perspectives on what a register is, what goals it should achieve, and who the intended audience is, there is a discrepancy between what public organizations offer and what significant users, such as oversight authorities and societal watchdogs, expect. This mismatch between demand and supply is reflected in what public organizations publish—the current relevant disclosures—which, exceptions aside, mainly consist of information about the technical functioning of an algorithm. However, some of the users interviewed in this study indicated that they wish to gain insight into identified risks and mitigation strategies applied to specific algorithms and, if possible, the training and input data and the source code, so that they can replicate the algorithm. Additionally, users indicate that a relevant disclosure is information on how technology functions in its context. This oversight authority policy officer explains why this is important:

It is mainly about: what are you going to do with it afterwards? And those human components, I think, played a significant role in the Childcare Benefits Scandal as well, because what do civil servants do with the information they receive? And what you do not want is for a civil servant to lean back and accept everything the system produces as the truth. You also do not want the system to constantly crash because various peculiarities and exceptions are made. So, it is almost impossible to separate the use of an algorithm from how a bureaucracy deals with it. They are so closely intertwined, and in that sense, the algorithm register does not actually say much now. It is much more interesting to know: what happens in-house, behind the scenes?

A final design consideration relates to how the information is presented, i.e., the disclosure modality. Here, organizations make three choices: regarding the format, language, and level of differentiation of information. In the Netherlands, there is a national algorithm register where all government organizations are allowed to list their algorithms, but organizations are also free to create their own register. This leads to a significant degree of differentiation in terms of format, language, and information provided. For all three choices, the intended end user is a crucial factor. If a public organization primarily focuses on citizens, respondents indicate that, concerning the format, more effort is generally put into creating an accessible, interactive website rather than an Excel list. The language is also adjusted accordingly, as explained by this public manager:

And then you have an extensive discussion with the communication department about what understandable language is. So, we have to present that in B1, Dutch level B1, on the website, so a lot of time was actually spent on that.

On the other hand, if the citizen is not the primary intended user, and the register is mainly intended for journalists and oversight authorities, a format is chosen that allows for easy comparison of data, such as an Excel file or a website with filtering capabilities. In this case, technical language and jargon are employed, providing detailed explanations of the characteristics of the algorithms. Finally, there is also an intermediate option, offering differentiation of information. In this case, a format is chosen where users without technical knowledge, such as many citizens, can see information about algorithms in simple language in an accessible manner. Users who want more detailed information, such as journalists, can click through and open other tabs with additional information, such as the legal basis or the data sources used.

Table 2.

Overview of Characteristics of Algorithm Registers.

Perspective	Starting point
	End point
	Governance mechanism
Goals	Trust
	Transparency
	Downstream effects of transparency (e.g., accountability, responsible use of algorithms, knowledge exchange, dialogue about algorithms, etc.)
Design considerations	Unit of disclosure - Definition of algorithm - Choice which algorithms in register (all vs. high-risk algorithms) - Internal vs. external register
	Audience - Intended users (citizens) - Actual users (societal watchdogs, oversight authorities and other public organizations)
	Relevant disclosures - Technical information - Potential risks and mitigation strategies - How algorithms are used by civil servants
	Disclosure modality - Format - Language - Differentiation in type of information

Table 2 offers a comprehensive overview, mapping out the factors that shape algorithm registers. While the table suggests that there are different distinct characteristics of algorithm registers, it is important to highlight that many of these characteristics are interconnected. The perspective a public organization holds towards a register has consequences for both the goals they intend to achieve with it and the design considerations involved. For example, an organization viewing the register as a starting point will aim to meet minimal transparency requirements. This means that such a register will provide limited information about algorithms, requiring interested parties to seek additional details if they wish to know more. Nevertheless, all public organizations had one goal in common. They explicitly stated that the overarching goal of the algorithm register is to strengthen citizen trust in algorithm use by public organizations. For external parties, such as journalists or NGOs, transparency, rather than trust, is a goal in itself. To fulfill this goal, different standards for information are necessary, including greater detail, completeness, and second-order transparency, where the algorithm's risks and mitigation strategies are explicitly explained.

5.2 Implications

The in-depth analysis of interviews and documents revealed that several challenges led to negative implications of algorithm registers. These challenges can be broadly categorized into regulatory and oversight issues, transparency concerns, conceptual issues, resource constraints, and user-related challenges.

One of the regulatory and oversight challenges is the absence of comprehensive legislation governing algorithm registration. The voluntary nature of this task undermines the impact of algorithm registers, leading to incomplete and inaccurate representations within these registers. Furthermore, the lack of an overseeing organization dedicated to ensuring the accuracy and completeness of registrations fosters perverse effects. Several respondents mention that examining similar registers, like the record of processing activities under the General Data Protection Regulation, reveals that regulatory frameworks alone are insufficient to prompt organizations to diligently register their data processing activities. Active monitoring and oversight are necessary for the register to evolve into a meaningful tool. Furthermore, for some public organizations, this regulatory and oversight vacuum fosters a “checklist mentality”, wherein there is a tendency to view the registration process merely as a checklist to be ticked off, as this NGO researcher explains:

And now we often see that there is a desire to have it, but the means are not provided, and the commitment to do it well and also maintain it properly is lacking. Because it also takes time to keep it up to date, and then you end up with it becoming a kind of token gesture.

Consequently, these organizations typically register algorithms only once, lacking incentives for subsequent updates or comprehensive field completion. The perception of the register as a mere checklist thus results in sporadic updates and incomplete information. Another example of a perverse effect is that frontrunner organizations, early adopters of algorithm registers, are under intense scrutiny. Societal watchdogs and oversight authorities closely follow and observe their algorithm registrations. This way, intensified scrutiny and media attention of existing algorithm registers act as obstacles for organizations considering starting algorithm registration, leading to a collective action problem.

In addition, transparency challenges significantly hinder the potential value that algorithm registers may have. Incomplete registrations and (essential) information being withheld by public organizations lead to “deceptive transparency”. Societal watchdogs and oversight authorities explain that, in many cases, public organizations may present a facade of openness while omitting crucial details or providing vague descriptions about algorithms. An inspector from an oversight authority illustrates this:

At some point you can imagine that it is, of course, not pleasant if what you are doing as a civil servant seems to be leading to very high risks. So, those descriptions for citizens, are indeed, concealing is not the right word, but yes, they are described very generally.

Furthermore, respondents indicate that there are many inconsistencies in the way public organizations register their algorithms. It is, therefore, difficult to make meaningful comparisons, between organizations or algorithms, about the information that is made transparent.

Conceptual challenges further impede the value of algorithm registers. The lack of a clear goal for these registers introduces ambiguity, with varying objectives that are often challenging to integrate within a single register. For instance, one organization outlines eight different goals for its algorithm register, encompassing goals such as enhancing trust in government, increasing explainable decision-making, and empowering both citizens and businesses. Additionally, the absence of clear definitions for what an algorithm is, and specifically what “high-risk algorithms” or “high-impact algorithms” are, contributes to confusion regarding what should be registered and leads to inconsistencies in registering within and across public organizations. Finally, some public organizations suggest that the registration process motivates them to adapt rather than deactivate algorithms. The discussions that emerge in the registration process are merely about how to align an organization's algorithms with expected norms, rather than engaging in a critical discussion about their necessity. This public manager illustrates how unusual it is to deactivate algorithms:

It is, I must honestly say, my experience that no one ever says: “this is not ethical” or “we won't do it”.

Resource challenges, such as the time and financial investment required for organizations to register algorithms, are perceived as negative implications of algorithm registers. The registration process, which can take from a few weeks to one or even several years for one algorithm, requires not only considerable time but also substantial financial costs. Many individuals are involved in completing such registrations. In certain cases, third-party reluctance to cooperate in the registration process further extends the time and financial investment required for the registration process.

User challenges, lastly, become visible as algorithm registers in their current form do not appear to yield the intended benefits for its primary intended user, namely, the citizen. Societal watchdogs and oversight authorities explain this using two reasons. Firstly, the language employed exceeds the comprehensibility level of the general population, as very technical terms and jargon are used in describing algorithms, despite efforts to use simple, understandable language. Moreover, suboptimal marketing strategies by certain public organizations contribute to a lack of visibility of algorithm registers, hampering citizens’ access to these registers, as described in a municipal policy document:

Additionally, the algorithm register is not sufficiently accessible for residents and businesses, as it cannot be accessed through the Municipality of Utrecht's website www.utrecht.nl. (D31)

This lack of visibility and awareness regarding algorithm registers has also been confirmed by public organizations. These organizations mentioned that they had surveyed citizens or their so-called “customer panel” to assess awareness of their algorithm registers, with the results showing that most people were unfamiliar with them. Additionally, some public managers noted that users could submit questions about algorithms or the register through their website. However, they mentioned that inquiries came exclusively from journalists and researchers, not from ordinary citizens, further highlighting the general lack of public awareness of these registers.

Secondly, there is no connection between individual decisions and algorithm registers, as was mentioned in several interviews and documents. Individual decisions often fail to indicate the involvement of an algorithm in the decision-making process, leaving citizens unaware of its existence. Several oversight authorities and societal watchdogs advocate for public organizations to inform citizens when algorithms are part of the process. In these individual decisions, a reference to the algorithm register could be made for those citizens seeking more information about the algorithm that affected the decision.

In addition to these challenges, the results also highlight several perceived positive implications of algorithm registers. Distinct external positive implications emerge when considering other users, such as oversight authorities and societal watchdogs, interacting with the register. First, interested parties, including societal watchdogs and oversight authorities, gain valuable insights into the algorithms deployed by public organizations. This way, the register raises awareness about public algorithms, providing an overview of algorithmic applications within these organizations. Second, respondents acknowledge the positive aspect of the register in conveying ethical considerations related to algorithms. When filled in completely and accurately, algorithm registers could offer stakeholders not only a list of deployed algorithms but also insights into an organization's approach to responsible algorithmic decision-making. Third, according to oversight authorities and societal watchdogs, one of the most significant positive implications is that the register serves as a crucial starting point for further investigations. The information, or its absence, acts as a point of departure for in-depth examinations, particularly when potential risks are identified that may require further mitigation, as this journalist explains:

And then for journalists and like other societal watchdogs, I mean, it gives us the mechanisms to, you know, more quickly identify whether there is a system that may, you know, need to be investigated or held accountable, because right now we all spend a ton of time just trying to figure out the basics of how one of these systems works.

Furthermore, the implementation of an algorithm register brings about three significant internal positive implications for public organizations. First, the process of establishing and filling an algorithm register generates awareness among civil servants about the use of algorithms within their organization. This is particularly important since many civil servants were previously unaware of the presence of (certain) algorithms in their work processes. Second, the registration process serves as a catalyst for organizational learning. As mentioned before, the Netherlands is a pioneering country in terms of algorithm registers, leading to a lot of experimentation and trial and error in the process of embedding algorithm registers in organizations. From creating an inventory of all algorithms in an organization to publishing them in understandable language on a website, organizations are doing this for the first time and cannot build on acknowledged best practices. This means that organizations are undergoing a significant learning process in dealing with this process. For example, public organizations increasingly employ mechanisms like e-learnings to educate employees about the risks and responsible usage of algorithms. Feedback from society further contributes to organizational learning by shedding light on potential risks or weaknesses in their algorithmic processes. Third, the completion of various information fields in an algorithm register introduces a disciplining effect within public organizations. This policy officer of the Ministry explains:

And now you have to go through it all again and discuss it with each other once more. […] then you actually have the conversation: do we really want to use this at all? Because you have to be able to justify it. What is your legal basis? Why are you doing this? What are the risks? How do you manage them? And if you can't answer those questions, well, then you might need to think twice. In that sense, it also has a purifying effect. By going through this exercise, you may decide not to do something anymore.

The registration process nudges organizations to critically assess their algorithmic decision-making processes, identify associated risks, develop mitigation strategies, and ensure adherence to existing rules and regulations such as the General Data Protection Regulation. Algorithm registers, therefore, not only enhance internal awareness but also foster a culture of continuous learning and improved decision-making within public organizations.

Table 3.

Overview of Implications of Algorithm Registers.

Negative implications	Regulatory and oversight challenges - Voluntary nature leads to incompleteness - No incentive to keep it up-to-date (checklist mentality) - Collective action problem
	Transparency challenges - Deceptive transparency - Inconsistencies in registrations hinder meaningful comparisons between algorithms or organizations
	Conceptual challenges - Lack of clear goal algorithm register - Lack of clear definition of (high-risk) algorithm - Adapting instead of deactivating algorithms
	Resource challenges - Large financial and time investment
	User challenges - Mismatch between algorithm register and intended users due to type of language used and visibility of algorithm register - Lack of connection between individual decisions to citizens and algorithm registers
Positive implications	External - Awareness within society about algorithms used by public organizations - Starting point for investigation
	Internal - Awareness within public organization about algorithms - Organizational learning - Disciplining effect

Table 3 summarizes the most important implications of algorithm registers. The results indicate that there are still many challenges that limit the extent of the positive implications. This finding aligns with the concerns raised by Cath and Jansen (2022) that algorithm registers have limited value for external stakeholders. It remains questionable whether the predefined goals, such as increasing public trust, will indeed be achieved through these registers. However, there are also surprising positive implications, which mostly benefit public organizations themselves. They experienced increased organizational awareness, opportunities for learning, and a disciplining effect from algorithm registers, aligning with some of the positive expectations of Floridi (2020). Over time, these internal positive implications could potentially contribute to the overarching goal of enhancing public trust.

6. Conclusion and Discussion

The main research question of this study was what the nature of algorithm registers is and what their implications are. Based on data from in-depth interviews with 27 respondents and 33 policy documents, we can provide an answer.

Experiences from the Netherlands, a leading country in adopting public algorithm registers, show that a great diversity in algorithm registers exists. This is primarily attributed to varying perspectives on what an algorithm register is: a starting point, end point or governance mechanism, and to various design choices. This becomes visible in the unit of disclosure (e.g., registering all algorithms versus only high risk algorithms), the audience (e.g., citizens as intended users versus oversight authorities and societal watchdogs as actual users), relevant disclosures (e.g., technical information versus how civil servants use algorithms), and the disclosure modality (e.g., interactive webpage versus excel list) of an algorithm register. Despite these differences, a common goal unites all registers: the cultivation of citizen trust through transparency in the deployment of algorithms by public organizations.

However, the findings regarding the implications of algorithm registers showed that it is unlikely that the goal of enhancing public trust will be achieved. At first glance, the register appears to be mainly a paper reality. It seems to be a checklist, with no incentive to complete all fields or maintain up-to-date information. Societal watchdogs and oversight authorities indicate that it merely offers a glimpse into the inner workings of public organizations. Nevertheless, results of this study also show a hidden value.

The process of registering algorithms fosters organizational awareness and learning, as it brings to light pertinent questions and concerns related to an organization's use of algorithms. This could lead to a disciplining effect, as the registration process nudges organizations to critically assess their algorithmic decision-making processes, identify associated risks, develop mitigation strategies, and ensure adherence to existing rules and regulations. Thus, while capturing complex phenomena, such as algorithms, within a tool, like an algorithm register, is challenging, it can hold significance for an organization (Noordegraaf, 2008).

This research offers a nuanced mapping of the different factors that play a role in shaping algorithm registers. While earlier research mainly discusses algorithm registers in the context of their (non)existence (Cath & Jansen, 2022; Floridi, 2020), this study demonstrates that such categorization is not a simple dichotomous choice. The nuanced mapping of algorithm registers’ characteristics reveals that there are many difficult choices to make regarding what an algorithm register is, what objectives it tries to reach and how it should be designed. To create a useful and valuable algorithm register, these choices must be aligned. For example, if one sees an algorithm register as a starting point for oversight authorities and societal watchdogs, the primary goal should not be to increase citizen trust directly. Instead, the goal of the register should be to foster accountability, and design considerations should be tailored accordingly. However, because of the developmental (trial and error) phase in which many public organizations find themselves regarding the implementation of algorithm registers, choices are mostly taken out of pragmatic reasons and not so much out of alignment reasons. The nuanced mapping in this study provides an overview of the different choices, which can help organizations critically consider how they want to align the different characteristics of their algorithm registers.

In sum, considering the primary goal of the register—building trust—initially, it does not lead to an increase in trust by stakeholders using the register. However, eventually, it could contribute to the responsible use of algorithms through its learning and disciplining effect, thereby fostering better algorithmic governance and potentially enhancing public trust. To understand the potential of algorithm registers, we need to look for indirect and unexpected dynamics, instead of direct and linear effects outcomes such as greater transparency, accountability and trust. As public organizations are the ones primarily reaping the benefits, algorithm registers are (currently) not a meaningful tool for transparency, but a meaningful box-ticking exercise.

6.1 Limitations and Directions for Future Research

The field of algorithm registers in the Netherlands is undergoing rapid development, with frequent modifications to registers occurring weekly. This study, conducted between May 2023 and January 2024, reflects this evolution. For instance, the last interview in January 2024 revealed respondents presenting well-founded arguments for certain design considerations. This contrasts with the initial interviews in May 2023, where design choices were largely based on intuition. This shift can be attributed to extensive inter-organizational discussions and learning from best practices within the field. Looking ahead, future research could explore how the line of reasoning, specifically regarding the nature of algorithm registers, changes over the course of the next couple of years, providing valuable insights into the ongoing evolution of algorithm registers.

It is important to note that the nature and implications of algorithm registers are discussed separately but are not independent of each other. This study offers several indications about the interconnectedness of these aspects. For example, an organization's perspective on an algorithm register has implications for its eventual structure and the information it discloses, thereby influencing the outcomes of the register. Future research should examine how the nature and implications of algorithm registers are interconnected. Conducting a quantitative study involving numerous public organizations that register algorithms or employing a process tracing study to assess whether certain design considerations lead to distinct implications would be approaches to enrich the discourse around algorithm registers.

6.2 Theoretical Contributions

This study contributes to the ongoing debate about the need for a broader organizational and institutional approach to transparency in the use of algorithms by public organizations (Grimmelikhuijsen & Meijer, 2022). The findings regarding the implications of algorithm registers show a nuanced reality: arguments of both transparency optimists and pessimists find support to an extent, aligning with earlier research that emphasizes transparency is not a silver bullet in fostering citizen trust (Grimmelikhuijsen & Meijer, 2014).

On the one hand, public organizations indeed selectively disclose information, which can come across as a false sense of transparency (Cath & Jansen, 2022). This selective approach, also known as “openwashing”, creates an impression of openness without presenting a complete or accurate picture of the organization's actions (Heimstädt, 2017). This phenomenon of selective disclosure and openwashing exemplifies decoupling in accountability literature (Power, 1997), where formal transparency requirements may appear to fulfill their intended function on paper but fail to provide genuine insight into the operational realities of algorithmic decision-making. Additionally, in line with earlier research, this study has shown that transparency about algorithms in a register could normalize their use, decontextualizing and depoliticizing the discussion by focusing on creating conditions for deployment rather than critically debating whether algorithms should be deployed at all (Cath & Jansen, 2022; Wang, 2022). This could create a perception that algorithms are inherently neutral tools, potentially masking their complex socio-political consequences. Moreover, the assumption that citizens, or even developers and civil servants using the algorithm, fully comprehend the disclosed information may be overly optimistic. Therefore, this study advocates that transparency intermediaries like oversight authorities and societal watchdogs should be the targeted audience, emphasizing the need for a critical audience in achieving algorithmic accountability (Kemper & Kolkman, 2019).

On the other hand, this study identifies a disciplining effect demonstrating that the more civil servants engage comprehensively in registering algorithms, the more thoroughly the use of algorithms is analyzed, leading to a more responsible use of algorithms in public organizations. Regardless of specific outcomes or practical benefits, transparency through algorithm registers is considered valuable because it contributes to the responsible use of algorithms in public organizations, enhancing the legitimacy of technological solutions within a democratic framework (Loi & Spielkamp, 2021). The act of making something transparent forces organizations to discuss and critically evaluate their use of algorithms. This shows the relational aspect of transparency that could lead to mutual understanding and learning (Ananny & Crawford, 2018; Valentinov et al., 2019) rather than a descriptive type of transparency that involves one-sided algorithm disclosure (e.g., Kempeneer, 2021).

Moreover, these insights align with previous research on the use of measurement tools and accountability mechanisms (e.g., Kempeneer & Van Dooren, 2021; Meijer, 2007; Noordegraaf, 2008), that demonstrates that the tools themselves are not inherently valuable. Rather, the requirement for organizations to reflect on the practices they need to measure or account for, and to make these practices explicit, leads to improved internal processes. Similarly, an algorithm register provides a framework for reflecting on how and why algorithms are used by the government. This research demonstrates that the value of such a transparency tool lies in the discussion about algorithms in use. Transparency tools in other forms and formats might have the same relational and learning effect.

6.3 Practical Recommendations

This research is of practical relevance given the growing attention and adoption of algorithm registers worldwide and the recent enactment of the AI Act which mandates registration of high-risk algorithms for EU member states. By investigating algorithm registers’ characteristics and implications, this research informs the discourse on AI governance, offering two practical recommendations for future developments regarding algorithm registers.

I recommend public organizations to reassess the target audience for algorithm registers. This suggestion stems from the findings revealing a lack of awareness among citizens regarding these registers. One potential adjustment entails shifting the focus of the register from ensuring transparency for citizens to providing transparency for intermediaries. These intermediaries can leverage the register's information for accountability purposes (Grimmelikhuijsen & Meijer, 2022). However, should the primary objective remain citizen-centric, efforts should be directed towards establishing a clear connection between individual decisions involving algorithms, such as speeding tickets, and the register. This allows citizens to navigate and monitor algorithmic decision-making processes that affect them.

In addition, I advise regulators to prioritize establishing mechanisms for robust oversight, as this study indicates that the effectiveness of regulation regarding algorithm registration (AI Act) relies on this critical foundation (see also Enqvist, 2023; Laux, 2023; Tutt, 2017). The findings showed that experiences with similar registers, such as the register of data processing activities under the GDPR, highlight that legislation alone is insufficient. Active oversight is necessary for comprehensive and up-to-date registrations. Oversight authorities responsible for monitoring these registers must be adequately resourced to effectively investigate algorithm registrations.

Supplemental Material

sj-docx-1-ipo-10.1177_15701255241297107 - Supplemental material for Algorithm Registers: A Box-Ticking Exercise or Meaningful Tool for Transparency?

Supplemental material, sj-docx-1-ipo-10.1177_15701255241297107 for Algorithm Registers: A Box-Ticking Exercise or Meaningful Tool for Transparency? by Esther Nieuwenhuizen in Information Polity