Co-creating research at The AI,media,and democracy lab : Reflections on the role of academia in collaborations with media partners

Abstract

This commentary explores academia’s role in co-creating research with media partners, focusing on the distinct roles and challenges that each stakeholder brings to such partnerships. Starting from the perspective of the AI, Media, and Democracy Lab, and building on the Ethical, Legal, and Societal Aspects (ELSA) approach, we share key learnings from 3 years of collaborations with (media) partners. We conclude that navigating dual roles, expectations, output alignment, and a process of knowledge sharing are important requirements for academics and (media) partners to adequately co-create research and insights. We also argue that these key lessons do not always square with how academic research is organized and funded. We underscore that changes in funding structures and the way academic research is assessed can further facilitate the co-creation of research between academic research and projects in the media sector.

Introduction

Academia and the media sector have been collaborating for decades, from co-development partnerships over strategic collaborations and startup-like spin-offs (Westlund and Krumsvik, 2014). Over time, these different constellations between academia and the media - here understood as news outlets and similar organizations that disseminate factual information to the public (Nielsen et al., 2023) - have been integral to the development of both sectors, potentially steering where research is going, as well as how the media sector is innovating (Diakopoulos et al., 2024; Meier, 2007).

The adoption of increasingly complex and opaque technologies and tools underscores the importance of academia-industry collaborations and how these collaborations further our understanding of what these technologies mean for both sectors, as well as the societies they are inherently part of (Schützeneder et al., 2022). In 2024, as AI becomes more integrated into media workflows—automating tasks like content generation, editorial decision-making, and audience moderation—there is yet another shared goal of media and academia to understand how these technologies are developed, on which data they are trained on, and how they are being used in the news ecosystem (Diakopoulos et al., 2024). The automation and augmentation of AI systems confront the media sector, among others, with ethical and legal questions and dilemmas like “how should AI be developed and implemented responsibly?”, “Is it ok to use synthetic content in reporting?”, “Is the use of AI in the media ‘high risk’ and must it be regulated or even banned?” (Cools and Diakopoulos, 2024; Diakopoulos et al., 2024). From the perspective of The AI, Media and Democracy Lab (AIMD Lab) at the University of Amsterdam, we want to share some of our experiences and what methodologies and collaborations we use to find answers to some of the critical questions and ethical dilemmas mentioned above (AI, Media, and Democracy Lab, 2024).

After almost 3 years of collaborating with these partners, we first describe how the Ethical, Legal, and Societal Aspects (ELSA) approach is practiced at the AIMD lab since its foundation in 2021. The lab approaches the responsible development of AI technologies not only as a technical but also a deeply societal, ethical and legal question (Netherlands AI Coalition, 2021). Second, we briefly outline the main types of academia-media sector collaborations within the Lab. Third, we offer some key learnings that are central to how we think about how our academic work potentially impacts journalistic practice and innovation, while also advancing research. Among these learnings, we want to underscore that academia-media sector collaborations require expectation management, output alignment, and a process of knowledge sharing. In doing so, we hope that the “AIMD Lab approach” in this commentary will contribute to the broader discussion of AI ethics and media innovation by offering a model for other interdisciplinary labs, research initiatives and university projects. Lastly, we want to argue that the current structure of academic funding and research assessment does not always support such collaborations, and can even make them more difficult. Research grants are often geared toward detailed and predetermined research questions and methods which sometimes contrasts with the more flexible co-creation approach necessary to facilitate collaboration with media partners and accommodate academic work to the production cycle of AI innovations in the media.

In some disciplines, cooperation with industry and societal partners is primarily seen as valorization and may not always result in publications in peer-reviewed high-ranking journals, making these collaborations less attractive particularly for untenured researchers. In the worst case, there are concerns about academic independence and integrity. We acknowledge existing collaboration programs, such as the Recognition and Rewards Program facilitated by the Dutch Research Council (NWO) ¹ and the broader sector of academia, but we argue that revising additional funding structures and assessment criteria to accommodate the unique demands of academia-media collaborations should bridge this gap. This shift in funding structures and assessment criteria would allow for more meaningful co-creation of research that balances academic rigor and theory with practice, and this way ultimately can result in new research and insights for both academia and society against the backdrop of a broader notion of impact and collaboration.

Adopting the ELSA lab approach at the AIMD lab

The ELSA framework ideally proposes that ethical and societal considerations, such as fairness, transparency, and accountability but also the effects of technology on society, are integrated into the development and implementation of AI systems, and that academics work together and across disciplines with regulators, civil society and companies. Building on the Dutch AI Coalition report on “ELSA Labs for innovation in AI”, we describe three distinct, yet interconnected, factors of how we navigate and learn from the ELSA Lab approach at the AIMD Lab. Important to note is that there is not one ELSA Lab approach, and that we learn from this approach through methodologies, processes, guidelines, and tools which further broadens the different ways in which we understand societal, ethical and legal aspects of technology and innovation.

The first factor concerns human-centric AI design and the importance of capturing human and public values. The AIMD Lab tries to learn to operationalise the ELSA framework to create AI systems that prioritize human and public values, such as fairness, diversity, transparency, and accountability (e.g., for some research papers of the AIMD, see Cools and Diakopoulos, 2023; El Ali et al., 2024; Kieslich et al., 2024; Seipp, 2023; van Drunen et al., 2023). This “human factor in technologies” is crucial, as these values are often left somewhat unaddressed or prioritized over other metrics that are easier to measure like engagement (Møller, 2022). Addressing or capturing these human and public values is important for media organizations and civil society but are often hard to pin down (Vrijenhoek et al., 2024). For example, the AIMD Lab offers specific methodologies, interventions, and workshops that can help media organizations and civil society to substantiate a process of how to integrate values like fairness, diversity, transparency, and accountability in their AI systems. This collaboration is often iterative, as these values are not always quantifiable, but require a more qualitative approach that goes beyond the technical input and also incorporates regulatory guidance and societal perspectives, fostering a more holistic approach to AI development and implementation which could potentially make these organizations more societally ready to develop and implement human-centered AI systems. This approach is the result of years of normative, empirical and computer science research into alternative evaluation metrics.

The second factor concerns the creation of a productive and integrative knowledge loop. The lab brings together fundamental research in social sciences and computer science through the involvement of the University of Amsterdam and the Centrum Wiskunde & Informatica, applied sciences in the form of the Amsterdam University of Applied Sciences (AUAS) and partnering Applied Science Universities in Utrecht and Rotterdam as well as societal partners (for an overview of our partners and projects, see AI, Media and Democracy Lab, 2024). Testing and applying our theoretical methods and concepts in practice with societal partners allows us to translate our insights into applied practice but also further refine our research and validate theoretical findings. In other words: the ultimate goal is to create an integrative knowledge loop and find ways in which knowledge sharing and sector-cooperations benefit practice AND result in new and better theoretical and applied research. Doing so, the ELSA lab approach moves beyond binary understandings of research and valorization of research, but acknowledges that understanding how technologies work and affect society and societal actors ultimately also can result in richer research.

The third factor concerns collaboration and impact with internal and external stakeholders. ELSA’s emphasis on collaboration aligns with the AIMD Lab’s commitment to collaborate with diverse stakeholders, ensuring AI’s positive impact on society. At the lab, we have a multidisciplinary team of researchers with backgrounds that include, but are not limited to political science, humanities, psychology, computer science, law, and communication science. Each discipline contributes in its own way to this collaborative process and on a more abstract level we approach projects where we establish dynamic collaboration among internal lab members, as well as with external stakeholders. In doing so, we are aiming to capture the complexity of AI processes, guidelines, and tools, which is critical if these want to serve public interests. In practice, representatives of the lab’s partners are often invited to meetings and presentations at the Institute for Advanced Studies (IAS) in Amsterdam where the core members of the lab convene every Tuesday. The materiality of a shared space contributes directly to the collaborative processes of past, current and future research projects. Some of the lab’s cooperation partners include media organizations (BBC, DPG Media, NPO, RTL), societal stakeholders (City of Amsterdam, NemoKennislink, Algorithm Watch), and regulatory bodies (Dutch Regulator for the Media, Dutch Ministries, Council of Europe, European Commission).

Types of collaboration at the AIMD lab

There are several types of collaborations and constellations possible between academia and the media sector (for a more comprehensive overview, see Inpart, 2022). Understanding these types is key to understanding the opportunities and challenges of these collaborations which could hopefully lead to more effective partnerships between academic researchers and media organizations. We identify the most prevalent constellations from the perspective of the AIMD Lab which are (1) co-creation and co-development partnerships, and (2) feedback and transfer of knowledge collaborations (AI, Media and Democracy Lab, 2024).

The first type, namely the co-creation and co-development partnerships is one of the most common forms of collaboration at the AIMD lab which centers around creating new methodologies, processes, empirical insights, guidelines or tools. At the AIMD lab, these collaborations could start with a simple meeting between the different partners. The overarching interests are explored in the first meeting, and the different partners provide input that is clustered in a shared one- or two-page document. Bridging these different perspectives of the partners requires a reciprocal understanding of what constitutes relevance and value for each partner and how their individual objectives can be met within the framework of collaboration. The duration of such partnerships can run from a month to a couple of years, and these collaborations could lead to outputs that include but are not limited to internal research reports, blog posts, guidelines, joint events, events, presentations and peer reviewed publications (e.g., see projects, AI, Media and Democracy Lab, 2024). Often, this will include a multimodal output strategy, where research could result in a visualization, a protocol, a blog and an academic publication, to accommodate the mores and information preferences of different target audiences.

In the second type of collaboration, the focus is on knowledge transfer sessions where academia, the media sector, and potential other partners exchange input and insights. At the AIMD Lab, these sessions typically involve researchers from academic institutions sharing their latest findings with media professionals and other partners. Conversely, media partners of the lab like BBC (UK’s public broadcaster), DPG Media (one of the main Dutch-Belgian Media conglomerates) or NPO (The Netherlands’ public broadcaster) might also share practical needs and insights from the ground that could inform (future) academic research. For example, partners of the AIMD Lab could have questions like: “How to translate values like diversity into a media partners’ recommender system?” or “How to draft your organizations’ guidelines on responsible AI?”. At the AIMD lab, this specific partnership tends to be more short-term and in line with the ELSA-lab approach which acknowledges that AI is not only a technical but also deeply societal, ethical and legal question. The majority of these sessions only require one or two meetings to transfer knowledge and explore potential insights for every partner.

Key learnings to further substantiate and intensify academia-media sector research

After 3 years of research and outreach at the AIMD Lab, we emphasize three key learnings that came forth of our academia-media sector collaborations, both from co-creation and co-development partnerships, as well as feedback and transfer of knowledge collaborations. In addition, we will give some examples that illustrate these learnings and why we believe which practices are (not) successful. All in all, we have realized that media organizations see academic partners as increasingly valuable collaborators, as academia can facilitate a more “neutral ground” to discuss specific methodologies, processes, guidelines, and tools, serve as a sparring partner and contribute much needed expertise.

Navigating dual roles and expectations

One of the most important aspects of academia-media collaboration is the recognition that both sectors play dual roles which also requires continuous expectation management. Academia operates as both a center of knowledge generation and a contributor to public education, while the media sector is a “business with a public interest”, balancing profitability with their role as watchdogs and purveyors of truth in society (McQuail, 1993, 2003). Because of these shared roles and goals, it becomes clear why collaboration between academia and the media sector could be beneficial, but there are important differences, too. For example, academia must understand the commercial pressures that some media organizations face, such as audience retention, time constraints, content monetization, and competition for attention in an overcrowded digital landscape (Cornia et al., 2020). Similarly, media organizations must recognize academia’s need for research integrity, academic independence and the importance of the proper application of scientific methods.

To give one example: For a co-creation and co-development partnership between researchers at the AIMD Lab and a media organization, we first drafted a one- or two-page document with potential mutual interests. This forces us to step outside of our “comfort zone”, as we give media partners an idea of the types of projects we are interested in, and what data we would need to execute them. After this document has been shared with the media partner(s), there was time to provide suggestions, which resulted in some back and forth of whether the questions that we as academics were interested in resonated with the media partner, and vice versa, as well as what is needed to conduct the research also in a scientifically correct way.

In the end, we decided to investigate the perceptions of modular journalism, because there was a common interest between researchers at the AIMD Lab and the media organization. Modular journalism is a type of journalism that is determined by different format and contents (e.g., podcasts, videos, summaries etc.) which could potentially decrease trust in news, which is both of interest of the lab, as well as the media organization. As the media partner already had a prototype of this modular journalism, we decided to conduct focus groups to explore these perceptions. After having reached agreement, other common challenges of industry-academia publications had to be navigated, including but are not limited questions of how to reach a joint data sharing agreement, aligning different expectations and priorities (e.g., which role is taken up by whom? Who will be the main point of contact during the project?) and the division of responsibility (e.g., which partner(s) pays for the collection of the data? Who will be the owner of the data?) and under which conditions we would be able to publish. Having a forum to discuss these challenges in the AIMD Lab is helpful, also because these specific roles and responsibilities are not always clear-cut.

For the project on modular journalism, we have encountered some delays which came forth from these different roles and responsibilities. For example, we had some difficulties recruiting participants for the focus groups as it was unclear who was recruiting these participants. Similarly, the one- or two-page document with mutual interests had also been altered throughout the project which complicated the execution of the project. This example demonstrates the need to understand and also explicate the dual roles of researchers and industry partners, and that the ultimate research design is a result of mutual negotiation and, importantly, the will to reach a compromise between the different positions. We believe that this versatile approach can create a continuum of brainstorming and learning rather than a strict division between impact and research.

Navigating different outputs

Collaboration between academia and the media sector involves a range of stakeholders, each with their own outputs. Academic institutions typically prioritize peer-reviewed research, while media organizations focus on producing timely, engaging content and outputs that resonate with their audience. The key to managing these differing outputs is to potentially make them explicit at the start of the project. The process itself—of iterating on ideas, and navigating challenges—often generates value, we believe, even before the final outputs are delivered.

For a specific partnership between the AIMD Lab and a media organization, we were approached by the latter with a request on how to visualize and measure diversity in their content through a dashboard. Although these visualizations and the dashboard in itself might not be sufficient for a lengthy academic publication, there was an agreement that the researchers at the lab could later use the data for more in-depth studies. By providing regular updates, such as interim reports and visualizations, and agreeing on concrete deliverables like the diversity dashboard, both sides’ goals are addressed.

After the one- or two-pager was drafted for the diversity dashboard project, we at the AIMD Lab decided to come up with more short-form outputs like a slide deck and a report with some analysis of their content which highlighted some of the preliminary findings for the media organization. From the perspective of the lab, delivering these different outputs might be challenging, as they could generalize some of the findings for the media partner at the cost of academic depth and precision, specifically when it comes to the topic of diversity. Also, we realized that, at the outset of the project, it is imperative that these outputs are discussed, as well as potential authorship, and to what extent there is a need to “proofread” the output. In the end, we decided to still write a full paper on the project, which was not part of the initial output arrangement of the one- or two-pager. Before the paper went into peer-review, it was read by the media partner and some feedback was provided. In this case, all went well and our ability to publish was not affected but it could also have gone the other way, that the media partner objected. One of the lessons from this cooperation was the need to discuss the pathway to a publication about the results from the onset, also to avoid discontent and undesirable surprises.

A constructive collaboration at the AIMD Lab, like the one mentioned above, could imply that we attempt to establish the assessment of the potential outputs at the outset of a partnership, sometimes in combination with a data sharing agreement. This is, however, not always the case, as the project can change, as well as the outputs. We conclude that communication and a collaboration grounded in trust and mutual respect is critical. Despite this dynamic process, and the uncertainty regarding timelines and output it can bring about, we always try to agree on some concrete deliverables when the one- or two-page document is drafted. This allows us to predetermine specific outputs that can balance media sector needs with academic work, as well as the necessary leeway to still adjust these outputs when the during or after the projects has concluded.

Navigating a process of knowledge sharing: Moving beyond fixed timelines

A third key learning in academia-media sector collaboration is the need for flexibility in the research process. Traditional end-to-end models of knowledge production often depict research partnerships as linear processes, where academia generates knowledge to be transferred to industry at the project’s conclusion. However, in the fast-evolving media landscape, this model can be ineffective or simply unrealistic. A successful collaboration benefits from a non-linear approach, allowing for ongoing updates or adjustments to specific questions, methodologies, and timelines throughout the research process. Rather than waiting until the end of a project to disseminate findings, partners can engage in iterative feedback loops and continuous dialogue.

What sets the AIMD Lab collaboration apart from traditional research projects is the funding model, which is geared toward achieving collaborative goals rather than being tied to specific project durations. This flexibility allows us to adapt, change and pivot the purpose of the research in real-time as opportunities arise, but also work in shorter research sprints. Thus, flexibility from academia is crucial to accommodate the media partner’s reality, and vice versa, including the potential postponement of deliverables as circumstances evolve. Particularly in an area that is dynamic and volatile as digital technology and AI, this extra flexibility also allows us to realise research opportunities that we could have not anticipated when we wrote the original research proposal for the lab.

For a specific transfer of knowledge request, a media organization approached the AIMD Lab for feedback on their guidelines for the responsible use of AI. The lab has specific expertise on the legal and ethical frameworks regarding AI, which goes from procurement best practices over insights of the AI Act and the Digital Services Act, to the practical knowledge on how other media organizations are developing and implementing AI (policies). For this project, some members of the lab were consulted by a so-called “AI taskforce” of the media organization that was responsible for drafting these guidelines. In the end, there were four meetings with the taskforce, although the initial idea was to only schedule one meeting. As the leadership of the media organization decided to speed up the drafting process of the guidelines, we needed to be more flexible at the lab to have these conversations.

A constructive collaboration at the AIMD Lab could imply that specific stakeholders are only consulted once, just as other partners want to invest more time in longer-lasting projects or knowledge transfer sessions. By navigating this process of knowledge sharing, we have learnt that media partners often see the AIMD Lab as a neutral ground where specific topics can be discussed. Specifically for these knowledge sharing sessions, we even more believe in this non-linear approach of adaptation, change, and pivoting, as it might ensure that different stakeholders are continually learning and adapting, resulting in a more dynamic and responsive collaboration.

Going forward: Rethinking funding programs for academia-media sector research

Research collaborations between academia and the media sector are increasingly essential due to the rapidly evolving media landscape, the growing complexity of, and the increasing dependency on technologies, the need for diverse sets of expertise and limited funding for media innovation. As technologies like AI become integral to media workflows and processes, collaborative research can address the challenges and opportunities in the form of academia-media collaborations that include but are not limited to co-creation and co-development partnerships, feedback and transfer of knowledge collaborations. Although we acknowledge the many existing collaboration programs between academia and the sector, such as the ELSA program (NWO, 2024), we argue from the standpoint of the AIMD Lab, that there is still a need for more of these programs and for the success of such collaborations to be more embedded in traditional evaluations of research outside of these specific initiatives.

Current academic funding structures and programs often focus on pre-defined research projects with predetermined (and sometimes pre-registered) outcomes, and we argue that more flexible forms of funding that allow for hybrid partnerships and collaborations could steer innovative collaboration between academia and the media sector forward. This could include funding mechanisms that allow for adaptive project scopes, flexible funding based on milestones, and the inclusion (and compensation) of media and societal partners as active contributors to the research process. In addition, we believe that these collaborations should be intensified with a strong emphasis on two-way exchange of knowledge, rather than a one-directional and more “extractive” way of collecting data from either academia’s or the media sectors’ side. By accommodating the evolving nature of these non-linear collaborations, funding structures can and should better support the development of innovative and impactful research projects going forward, as well as creating time and space for mutual learning.

From the media perspective, embracing collaborations with academic institutions can lead to more projects, tools, processes, and policy guidance. Media organizations that actively engage with researchers can potentially benefit from higher-quality content, increased credibility, and innovative approaches to storytelling. To fully realize these benefits, the media sector needs to develop internal incentives that promote, enable and reward collaboration with academic researchers. This could include recognizing research collaborations in performance reviews, providing resources for joint projects, and fostering a culture that values research partnerships as integral to the organization’s success.

Going forward, it is crucial to address the incentive structures within both academia and the media sector to better support and value collaborative research. To better align academic incentives with collaborative research, institutions and funding bodies should recognize and reward contributions to stakeholder partnerships on par with traditional scholarly outputs, as A1 peer-reviewed publications are often still regarded as the prime currency of academia. This could involve integrating collaborative projects into performance evaluations, granting awards for impactful research, and providing dedicated funding streams for academia-media collaborations. A special note on team science: because of the technological and societal complexity of digital innovation, team science and bringing together researchers from different disciplines is gaining in importance. Team science can be understood as a concept that is broader than interdisciplinary collaboration, and has become more common - and supported - in some disciplines than in others, and involves a lot of invisible work and commitment to do well (Hall et al., 2018). Again, creating the right incentive and reward structures is critical along all disciplines. Finally, it is important to note that not every team member needs to engage in all types of research and collaboration; instead, a well-rounded team can encompass diverse expertise to effectively serve various ends, thereby enhancing the overall impact of the partnership.

In conclusion, we hope that other interdisciplinary labs, research initiatives and university projects can learn from the AIMD Lab learnings and challenges. Additionally, we believe that academia, the media sector and other partners have yet only started exploring what the ELSA-lab approach means. From the lab’s perspective, we believe the ELSA-lab approach can foster interdisciplinary collaboration, stimulate ethical innovation, and potentially provide a valuable framework for addressing societal challenges through responsible research and development. At the same time, we also acknowledge that this academia-media sector collaborations remain complex, especially at a time when there is increasing (big tech) platform dependency, media concentration and media distrust. Despite these challenges, we do hope that our insights will help to navigate the dual roles of academia and the media sector, foster better discussions on expectation management and output alignment, and better accommodate a process of knowledge sharing.