Increasing Engagement in Regulatory Science: Reflections from the Field of Risk Assessment

Abstract

While the demands for greater engagement in science in general and regulatory science in particular have been steadily increasing, we still face limited understanding of the empirical resonance of these demands. Against this context, this paper presents findings from a recent study of a potential participatory opening of the German Federal Institute for Risk Assessment (“Bundesinstitut für Riskobewertung” [BfR]), a prominent regulatory scientific organization in the field of risk governance. Drawing upon quantitative surveys of the public and selected professional experts as well as in-depth qualitative expert interviews, we identify a general support for greater engagement in science-based risk assessment. However, we also find significant contestation concerning its potential enactment and its normative and strategic merit. Underlying these contestations, we identify the persistence of a normal view of science and decisionist understanding of risk assessment, which create conflicting legitimacy demands for BfR and other regulatory scientific organizations. Together with concerns about imbalances in the power to participate, especially in highly specialized engagement processes, these pose significant challenges for the institutionalization of more participatory practices.

Keywords

engagement participation risk assessment consumer protection regulatory science risk governance food safety

Introduction

For centuries, science has continuously expanded its influence. Especially in the last decades, this has included an increasing “expertising” of Western democracies where scientists are called upon to provide knowledge as a legitimacy base for policy decisions (Jasanoff 2011; Liberatore and Funtowicz 2003; Millstone 2009). To accommodate this trend and to distinguish the activities of such science from traditional science usually conducted within the realm of universities, Jasanoff (1994) coined the term “regulatory” science. Sitting at the “boundary” (Guston 2001) of science and politics, the work of regulatory scientific organizations reaches from scientific activities located within academic settings over technical and bureaucratic work to regulatory lobbying and the taking of key decisions (Irwin et al. 1997; Bach and Doehler 2012). One such key activity is the assessment of risks. This is particularly so as policy-making systems in early developed (e.g. European Commission 2000) but increasingly also emerging economies, such as China (Standing Committee of the National People’s Congress of the People’s Republic of China 2015), have been reorganized along the so-called Red Book or decisionist (Millstone 2007, 2009; Weingart 1999) risk analysis model. This means that policy decisions about products or materials are supposed to be based on the independent advice of scientific experts who assess the risks of those products for humans or the environment.

Following the bovine spongiform encephalopathy (BSE) crisis, the European food sector has been at the forefront of the reorganization along the risk analysis model with separate risk assessment organizations setup at European level and within many member states. Next to the creation of an “independent” European Food Safety Authority (The European Parliament and the Council of the European Union 2002, Art. 18), especially Germany followed the red-book model closely by establishing an “explicitly independent” (von Wedel 2001) Federal Institute for Risk Assessment (Bundesinstitut für Risikobewertung, BfR). The proclaimed aim of BfR and other food risk assessment organizations is to use science to “protect consumers, animals and the environment from food-related risks” (European Food Safety Authority [EFSA] 2020). Accordingly, and following terminology used by the Food and Agriculture Organization of the United Nations (FAO) and the US Food and Drug Administration (US FDA), BfR and others refer to themselves as being involved in “science-based consumer protection.”

Indeed, the scientific results of FDA, EFSA, BfR, and other regulatory scientific organizations tend to have significant societal influence as they enable to make public, legally binding decisions about technologies, substances, or other products (Demortain 2017). Given these “enormous decisions and stakes” (Demortain 2017, 141), it arguably becomes crucial, from a prescriptive perspective, to ask how regulatory scientific organizations gain their legitimacy. Even more so from a descriptive perspective as regulatory agencies find themselves caught in ever-greater controversies. The assessments of pesticides, gene technologies, or endocrine disruptors are just some examples of the fading societal support regulatory organizations are facing. As Demortain (2017, 142) summarizes: “Not only are the results of regulatory science—the particular assessments of that risk or that technology—controversial. Its constitutive standards, concepts, and protocols are now frequently the object of critique.”

Greater engagement¹ in science in general and regulatory science in particular has been put forward as a potential panacea for addressing this rising challenge (Stilgoe, Lock, and Wilsdon 2014; Chilvers 2008; Felt and Wynne 2007; Jasanoff 2003; Upham and Dendler 2015). Looking at the wider debate, though, the “participatory turn” has not been without criticism. On a prescriptive level, many question the representativeness and wider emancipatory and deliberative merits of current engagement methods. On a descriptive level, we find a number of studies attesting to a continuous high trust in science, limited interest in engagement, and remaining calls for independent, “truth-speaking” expertise (e.g., TNS Opinion & Social 2013, 2014). As a result, a participatory opening may threaten the very societal support it is trying to achieve. Scholars such as Lövbrand, Pielke, and Beck (2011, 477), therefore, call for scholars to open their engagement calls up “to empirical contestation.” Our paper contributes to this debate by presenting the empirical reactions to a (potential) greater participatory opening of BfR, one of the largest regulatory scientific organizations in the field of (food) risk governance.

Our results show general support for greater engagement in science-based risk assessment. However, we also find significant contestation concerning its normative and strategic relevance as well as its potential enactment. Undergirding these contestations, we identify the persistence of a “normal” (Funtowicz and Ravetz 1993) view of science and “decisionist” (Millstone 2007, 2009; Weingart 1999) understanding of risk analysis. These create conflicting legitimacy demands for BfR and other regulatory scientific organizations working at the boundaries of different social worlds.

It is important to note that our paper does not fundamentally question the prescriptive merit of public engagement. It rather points to continuing challenges on a descriptive level in order to facilitate its deeper institutionalization. To do so, the subsequent section will first review the characteristics and main argumentative bases for a participatory opening of scientific processes in general and regulatory science, including science-based (food) risk assessment, in particular. We will then introduce the empirical focus and methodological approach of our study before presenting its main insights. The last section summarizes, reflects, and concludes pointing to remaining research needs.

Literature Review

In a definition generalized from the US FDA, Moghissi et al. (2017, 2-3) describe regulatory science as a “scientific discipline consisting of the development and application of scientific methods, tools, approaches, and other relevant processes derived from various scientific disciplines used in regulatory and other policy processes.” Unlike academic science, which tends to be directed at the advancement of knowledge in an open-ended fashion, regulatory science is directed at closure (mainly through synthesizing knowledge) often including predictions produced under considerable time pressure (Bach and Doehler 2012; Irwin et al. 1997; Jasanoff 1994).

Across contexts but especially in the field of food, scientific advice produced under these conditions has gained considerable influence. At the core of this rising influence has been the reorganization along the orthodox interpretation of the “decisionist” or “Red Book” risk analysis model (Millstone 2007, 2009; Weingart 1999). This means that policy decisions about products, substances, or other materials are supposed to be based on the advice of scientific experts (mainly from the natural sciences) who assess (usually in quantitative form) the likelihood and severity of adverse effect(s) that may occur to humans or the environment upon exposure. While the former is framed as a political, value-driven decision and is usually referred to as “risk management,” the latter should be based on an independent and objective scientific process and is usually referred to as “risk assessment.” Often, red-book or decisionist models are discursively mixed with technocratic language, that is, a verbal presentation of policies being solely based on pure scientific facts.

Underlying the technocratic but largely also the red-book model is a “modern” (Liberatore and Funtowicz 2003) or “normal” (Funtowicz and Ravetz 1993) understanding of science, which assumes an objective scientific process with no major disputes around its framing questions, empirical evidence and findings. To achieve these undisputed “facts” or “truths,” science is supposed to apply a mix of professionalization, specialization, and independent organization (Lengwiler 2008; Felt and Wynne 2007; Beck 2007; Jasanoff 2003).

Over the years, more and more actors have challenged the decisionist model and its underlying modern understanding of science. This concerned especially the field of new and emerging technologies where uncertainties rendered the production of undisputed facts increasingly difficult. At first, these problems were addressed through the initiation of more scientific studies or the use of scenario and foresight techniques (Liberatore and Funtowicz 2003; Ely et al. 2009b; International Risk Governance Council (IRGC) 2012; Lofstedt and Bouder 2017). More recently, we find calls for a move toward a “co-dynamic” model of risk analysis. At the heart of this model is a fundamental questioning of the clear separation between scientific risk assessment and nonscientific risk management and a demand for a participatory opening of risk assessment toward other, nonscientific audiences (Millstone 2007, 2009).

Such calls for more public engagement in regulatory science go hand in hand with a more general “participatory turn” in science and (Western) society as a whole. Latour (1998), for example, calls for a transition from a culture of science, characterized by certainty, detachment, and objectivity, toward a culture of research, characterized by uncertainty, involvement, and openness. Looking across literatures, three main lines of argument undergird these calls.

Most dominantly, we find a “substantive” (Fiorino 1990) argument where engagement is seen as a way to extend scientific knowledge with nonprofessionalized elements including “real-world circumstances of application” (Irwin et al. 1997, 18). Especially for systemic (IRDG 2012) or “mega-risks” (Lövbrand, Pielke, and Beck 2011) that are embedded in the larger context of societal, financial, ecological, and economic consequences, there is arguably a need for a “hermeneutic shift” where assessment also focuses on the value-driven discourses surrounding technical and other societal developments (Bidwell 2009; Grunwald 2015; Rittel and Webber 1973). In doing so, uncertainties can be reduced and the application of precautionary measures improved (Liberatore and Funtowicz 2003). Additional perceived benefits include capacity building and contributions to the resilience of participating communities (Renn 2016). In other words, from a substantive perspective, greater engagement in science can improve the outcome for society, for example, in terms of public health or broader human well-being (Stirling 2008). We therefore refer to this line of argumentation as “pragmatic” base for greater engagement in science.

Less prominently, actors, mainly within sociology of science and technology studies and, to lesser extent, political science started to question dominant scientific legitimacy claims by naming unstated aspirations, value judgments, and “framing assumptions” as hidden shapers in the social construction of regulatory science and science in general (Bidwell 2009; Stirling and Scoones 2009; Millstone 2009; Ely et al. 2009b; Demortain 2017; Jasanoff 1994). Within this social construction process, regulatory scientific organizations are said to “disenfranchise” (Demortain 2017, 144) certain sets of knowledge. This includes countries that lack access to technologies of quantification, consumers that struggle to frame their concerns as strictly scientific, and environmental and health activists, even if they work to put their claims through in scientific terms (Demortain 2017; Winickoff and Bushey 2009). Through moves to “postnormal” (Funtowicz and Ravetz 1993) or “responsible” (Beck 2007) science, more diverse perspectives and ways of knowing are supposed to be included especially in framing the influential role of (regulatory) science (Stirling and Scoones 2009; Beck 2007; Liberatore and Funtowicz 2003; Millstone 2009). We refer to this line of argumentation as the “normative” (Fiorino 1990) base for greater engagement in science as it is usually shaped by prescriptive ideals of democracy such as inclusiveness or representativeness.

Next to pragmatic and normative achievements, a “participatory turn” in science is said to result in a number of “instrumental” goals (Fiorino 1990). Most importantly, scholars (e.g., Funtowicz and Ravetz 1993; Felt and Wynne 2007; Frewer 1999; Rittel and Webber 1973) and actors from the political realm (e.g., The European Parliament and The European Council 2013) see it as a strategic means to increase the (empirically described) legitimacy and reputation of scientific organizations. This is, mainly, to help address diminishing public trust in science and increase the credibility and public acceptance of new technologies and science-based policy decisions. In other words, from an instrumental perspective, greater engagement in science can improve the outcome for an individual organization, policy, or other social entity. Because of its resonance with discussions in the organizational studies and wider management literature, we refer to this line of argumentation as a “strategic” base for greater engagement in science.

Both normative and strategic objectives have not been without criticism. In terms of its normative goals, a number of authors have questioned the extent to which current participatory science transcends the epistemic authority of experts to the benefit of lay people or rather restrict participation to more mundane tasks removed from individual concerns or public controversies (Bogner 2012; Chilvers 2008; Dickel and Franzen 2016). For some, it may even enable implementers to silence objections and press ahead with chosen paths by “pick[ing] and choos[ing]” those inputs that are considered most useful (Walls, Rowe, and Frewer 2011, 255). Others are unsure about the representativeness of current participatory methods pointing to a common domination of participatory processes by the better educated and more powerful (Ely et al. 2009a; Cleemput et al. 2015; Briggs 2009). Particularly in the field of regulatory science, there has been rising criticism of undue influence, especially from the corporate realm (Demortain 2017). Depending on the form of institutionalization, participatory openings could reinforce such criticism.

Also, questions have emerged whether more engagement in science can or even needs to increase public support and trust. If one follows current surveys, the evidence is mixed. On the one hand, several surveys report a prominent fear of the negative side effects associated with developments in science and technology as well as public interest in greater engagement between science and the public (e.g., TNS Opinion & Social 2013; Bundesamt für Strahlenschutz (BfS) and Bundesinstitut für Risikobewertung (BfR) 2013; Inner City Fund [ICF] and Growth from Knowledge [GfK] 2018). On the other hand, we find remaining high trust in the positive contributions of science and the responsible behavior of scientists and science in general (TNS Opinion & Social 2014, 2013; ICF and GfK 2018). Some surveys also suggest limited interest in and effects of greater engagement. When European citizens were asked, for example, about the level of involvement citizens should have in decisions made about science and technology, the most common response was that citizens should be consulted with only 12 percent agreeing that citizens should participate and have an active role in such decisions (TNS Opinion & Social 2013). The same study also found limited evidence for a relationship between feeling informed about developments in science and technology and thinking that the influence of science on society is positive (TNS Opinion & Social 2013, 2014).

Against this background, the last years have seen more and more concerns that current participatory calls may in fact sideline the “manifold ways […] where science and technology are implicitly trusted, taken-for-granted, depended-on, and enthusiastically embraced” (Felt and Wynne 2007, 11). Some even argue that greater engagement in science may threaten the very legitimacy and societal support it is trying to achieve (Lövbrand, Pielke, and Beck 2011). Multiple authors have described how regulatory scientific organizations in particular act as “boundary” (Guston 2001) or “hybrid” (Gulbrandsen 2011; Parker and Crona 2012) organizations that draw their legitimacy from different, partly conflicting social spheres (Jasanoff 1994; Demortain 2017; Winickoff and Bushey 2009; Korinek and Veit 2015; Rothstein 2013; Millstone 2007; Parker and Crona 2012). Detailing this competition for research institutes, Gulbrandsen (2011) characterizes four cultures: an academic culture emphasizing notions such as independence, purity, and academic freedom; an industrial culture emphasizing values like efficiency and innovation; a bureaucratic culture founded on orderly rules, hierarchy, rationality; and a civil culture that can most easily be characterized by protest movements against the previous three cultures. Therefore, even though the knowledge regulatory scientific organizations produce is distinct from academic science, and even though regulatory scientific organizations often face particularly strong civic protest and calls for more participation, notions such as objectivity, neutrality, and independence are still crucial legitimacy bases (Jasanoff 2011; Demortain 2017). To illustrate this point, we can turn back to Moghissi et al. (2017, 3) who emphasize in their description of regulatory science how the exclusion of “nonscientific issues such as societal objective, ideology, and faith” are an important and often violated requirement for regulatory science.

It is against this challenging context that BfR and other regulatory scientific organizations are trying to meet calls for greater public engagement. In the next section, we will shed more detailed light on this context before we outline our methods to address the need to open current calls for greater engagement “to empirical contestation” (Lövbrand, Pielke, and Beck 2011, 477)

Empirical Context and Method

Context

Following the BSE crisis, the food sector emerged as a key field for new regulatory scientific organizations. Inspired by the decisionist red-book model, decision makers tried to restore public confidence by reorganizing the European food safety system with a clearer separation between “political” risk management and “scientific” risk assessment. On the European level, they created EFSA to “take on the role of an independent scientific point of reference in risk assessment” (The European Parliament and the Council of the European Union 2002, Art. 18). Similar moves followed in many member states with Germany implementing one of the strictest interpretations of the red-book model in Europe and, indeed, worldwide (Millstone 2009). In 2002, German decision makers fully separated the joint food risk assessment and management organization, the Federal Institute for Health Protection of Consumers and Veterinary Medicine, into the Federal Office for Consumer Protection and Food Safety, which became responsible for risk management, and the German Federal Institute for Risk Assessment (BfR), which became responsible for risk assessment. Explicitly set up as a “scientific” organization that is “free of external influences,” BfR joint a larger group of governmental research agencies (GRAs) to prepare “independent” expert reports on product safety (including food, feed, and substances) and communicate those to politicians, consumers, media, businesses, and other groups (von Wedel 2001). Representing the most explicit organization of regulatory science in Germany, GRAs are usually under the direct oversight of a ministry (Bach and Doehler 2012; Korinek and Veit 2015). In the case of BfR, ministerial oversight is provided by the Ministry of Food and Agriculture, for which BfR provides most of its risk assessments. While GRAs tend to accept these “political prerogatives” (Bach and Doehler 2012, 13), most of them, including the BfR, rhetorically insist on being independent.

A second approach in the reorganization of the European food sector was to move consumer interest and the protection of consumer health more center stage (Holm and Halkier 2009). Risk communication joint risk assessment and risk management to ensure an “interactive exchange of information and opinions throughout the risk analysis process […] among risk assessors, risk managers, consumers, feed and food businesses, the academic community and other interested parties” (Regulation No. 178/2002 on the general principles and requirements of food law Art. 3). To achieve that aim, EFSA shall supplement consultations and other engagement measures run by the European Commission with “effective contacts with consumer representatives, producer representatives, processors and any other interested parties” (Regulation No. 178/2002 on the general principles and requirements of food law Art. 42). While EFSA used to implement this requirement primarily through consultative platforms, it put in place a new strategy in 2016. According to this new strategy, it will now reach stakeholders through a combination of a permanent annual stakeholder forum and stakeholder bureau as well as targeted platforms, including scientific colloquia, discussion groups, roundtables, communication labs, and information sessions (EFSA 2016). In Germany, a separate “Risk Communication” department was created within BfR. The aim of this department is to ensure the “participatory dialogue with various stakeholders” (http://www.bfr.bund.de/en/risk_communication-1834.html) through public perception research, traditional communication measures, and more dialogue oriented formats including scientific advisory boards, commissions, and public events (Boel and Hensel 2009).

Over the past years, this threefold model of risk assessment, management, and communication has been increasingly questioned. Scholars (e.g., Barker et al. 2010; Briggs 2009; Ely et al. 2009b; Koenig et al. 2010), but also the European Commission, Parliament, and Council (European Commission 2003; DG Sante 2017; The European Parliament and the European Council 2019) as well as international (Millstone 2009), supranational (EFSA 2016) and national (e.g., Wittkowski 2014) risk governance organizations, are calling for a greater participatory opening of the whole risk analysis process. Resonating with arguments set out in the Literature Review section, the current head of the European Food Safety Agency, for example, points out that “science is no longer seen as a source of enlightenment” but as an elite endeavor that contributes to rather than addresses current risks. To meet this challenge, risk assessment organizations have to deeply engage with their stakeholders (Url 2016). Similar statements can be found with BfR (Wittkowski 2014). Indeed, both organizations are currently investigating potential ways for a greater participatory opening, including the here reported study on “stakeholder management and public engagement in science-based consumer protection” at BfR. In the following, we will detail the methods used for this study before we will present and reflect upon its main results in the Findings section.

Method

To investigate the reactions to a potential greater participatory opening of the BfR risk assessment process, the authors followed a “mixed methods approach” (Modell 2009) combining qualitative interviews and observations with a quantitative survey. As part of a larger study commissioned by BfR and conducted by a third party, ² the survey asked the general public as well as professional experts from politics, science, the media, and relevant associations how important they considered BfR’s engagement (“Einbindung”) with the following groups: science, consumers, public authorities, media, business, politics, and environmental and other nongovernmental organizations (NGOs). The results were contrasted with an additional question on the perceived enacted engagement (“tatsächliche Einbindung”) of these groups. Participants who previously reported a lack of knowledge of BfR received a short introduction into its main tasks. The general public was represented by a sample of 1,000 German-speaking respondents aged fourteen and above living in Germany. The four expert groups were each made up of around 100 representatives, with associations divided equally into consumer and business associations. For each group, an Internet search was used to identify individuals with professional expertise in the areas of consumer protection, nutrition, health, environment, and agriculture. Following a successful pretest with thirty members of the public and ten professional experts, the public survey was conducted via computer-assisted telephone interviews between May17 and June 6, 2016. The expert survey was conducted between of May 18 and July 14, 2016, also using telephone. Both were statistically analyzed using SPSS (Version 21).

To gain deeper insights, we triangulated the survey material with forty-one in-depth qualitative interviews (King 2006) with individuals that had (a) previous experience with BfR engagement processes and (b) a level of seniority that offered them “elite” (Vaughan 2011) insights into wider developments in the field. By selecting experts from across governmental, supragovernmental, nongovernmental, corporate, academic, and media organizations, we aimed to explore diverse societal perspectives. The interviews were mainly in German, two in English. All text fragments that were relevant for publication purposes were translated into English. Each interview followed a semi-structured interview guide inquiring about interviewees’ general opinion on stakeholder engagement in the context of science-based consumer protection; perceptions of current BfR engagement processes; potential (other) means for engagement including a potential stakeholder board and stakeholder consultations; interest in and openness to be engaged in such a process; and important criteria for the successful implementation of such engagement processes. If interviewees agreed, the interviews were recorded and subsequently transcribed. In some cases, interviewees requested to be recorded only in the form of notes or to speak “off the record” (see Table 1 for a list of all interviewees including their position, links to different societal groups, and form of recording). Working as a BfR employee throughout the research, these data were further supplemented with observations and several informal conversations carried out during diverse BfR engagement events (e.g., the 2016 Green Week; BfR science fora, symposia, and other public events; BfR commission meetings). Following a critical realist approach (Danermark et al. 2002), the content of all notes and transcripts was inductively coded within the software MAXQDA (Version 12).

Table 1.

List of Interviewees.

Actor Group	Date	Format	Recording	Language	Position	Code
Scientific organization	January 22, 2016	In person	Yes	German	Working level	4
	January 28, 2016	Telephone	Yes	German	2. Leadership level	5
	March 01, 2017	Telephone	Yes	German	3. Leadership level	28
	November 16, 2016	Telephone	Yes	German	1. Leadership level	17
	November 18, 2016	In person	Yes	German	1. Leadership level	18
	January 06, 2017	Telephone	Yes	German	1. Leadership level	22
International authorities	January 17, 2016	Telephone	Yes	English	2. Leadership level	24
International authorities	September 12, 2016	Telephone	Yes	English	3. Leadership level	15
National authorities	May 22, 2017	In person	Yes	German	4. Leadership level	33
	July 10, 2017	In person	Yes	German	2. Leadership level	36
	January 25, 2017	In person	Yes	German	Working level	25
	May 25, 2017	Telephone	Yes	German	3. Leadership level	35
Political organization	April 20, 2017	In Person	Notes	German	Working level	31
	March 01, 2017	Telephone	Yes	German	Working level	27
	April 20, 2017	In Person	Yes	German	Working level	30
Corporate organization	February 10, 2016	In person	Yes	German	2. Leadership level	8
	January 16, 2016	In person	Notes	German	Working level	Notes B
	January 14, 2016	In person	Yes	German	1. & 2. Leadership level	1
	February 10, 2016	In person	Yes	German	2. Leadership Level	9
	December 07, 2016	In person	Yes	German	2. Leadership level	21
	January 27, 2017	In person	Notes	German	2. Leadership level	26
	February10, 2016	Telephone	Yes	German	2. Leadership level	10
	April 22, 2016	Telephone	Yes	German	3. Leadership level	14
	March 01, 2016	In person	Yes	German	2. Leadership level	12
	January 21, 2016	In person	Notes	German	Working level	Notes C
	Email response		Yes	German	N/A	Notes D
	March 15, 2016	In person	Notes	German	Working level	Notes A
NGO (consumer protection)	February 01, 2016	In person	Yes	German	3. Leadership Level	6
	March 09, 2017	In Person	Yes	German	2. Leadership level	29
	September, 2016	In Person	Yes	German	2. Leadership level	16
NGO (other)	February 25, 2016	Telephone	Yes	German	2. Leadership level	11
	March 18, 2016	In person	Notes	German	Working level	13
	January 18, 2016	In person	Yes	German	1. Leadership level	3
	May 22, 2017	In Person	Yes	German	2. Leadership level	34
	April 24, 2017	In person	Yes	German	Working level	32
	July 28, 2017	In Person	Yes	German	Working level	37
	February 02, 2016	In person	Yes	German	2. Leadership level	7
Media	December 07, 2016	Telephone	Yes	German	N/A	20
	January 17, 2016	Telephone	Yes	German	N/A	23
	November 30, 2016	In person	Yes	German	N/A	19
	January 15, 2016	In person	Yes	German	N/A	2

Note: NGO = nongovernmental organization.

The following sections will present and contrast the main outcomes of the qualitative and quantitative analyses. While relevant criteria are reported elsewhere, the focus here will be on reporting the reactions to a (potential) greater participatory opening of science-based consumer protection in general and the BfR risk assessment process in particular (General Reactions section), including relevant groups for (Relevant Groups section) and potential forms (Form of Engagement section) of engagement.

Findings

General Reactions

Our survey shows a general support for a participatory opening of BfR with over 80 percent of the public considering the engagement of scientists, consumers, and public authorities in the work of BfR to be important or very important. As shown in Figure 1, the importance of NGO, media, corporate, and political participation was rated slightly lower at 72 percent, 60 percent, 59 percent, and 55 percent. Among the professional experts the engagement of scientists (94 percent) and public authorities (87 percent) was considered most important with the engagement of consumers, and NGOs (70 percent), political actors (62 percent), media (55 percent), and corporations (56 percent) also supported but at lower scales (Figure 2). Comparing across expert groups, experts, especially from the corporate but also from the scientific realm, considered engagement of most groups comparatively less important than the average. Greater importance was attributed by experts from the media realm and, to less extent, by experts from consumer associations.

Figure 1.

How important do you consider the engagement of the following groups in the work of BfR? In percent of all those that provided a response | Science (n = 950), consumers (n = 953), public authorities (n = 942), nongovernmental organizations (n = 935), media (n = 953), business (n = 945), politics (n = 947) | Divergent to 100 ≙ answer “partly.” BfR = Bundesinstitut für Riskobewertung.

Figure 2.

Our interviewees supported this impression of a general support for public engagement in science-based risk assessment with interviewees making associations with a dominant trend (Interview 7, NGO) you have everywhere (Interview 10, corporate organization) and that is therefore also interesting for BfR (Interview 9, corporate organization).

Most dominantly, we found a pragmatic base for this support where engagement was seen as a way to facilitate greater knowledge exchange. Especially within the corporate realm, interviewees emphasized the importance for BfR to see the realities in practice (Interview 26, corporate organization). But also scientists acknowledged the practical relevance with one interviewee explaining:

It is often the question: […] what are the conditions in practice that lead to certain by-products being added that can lead to problems. (Interview 4, science)

There were also advocates, especially in the NGO and political realm, for greater exchange around normative perspectives (Interview 32, NGO), morals (Interview 30, political organization), risk perceptions (Interview 31, political organization), and broader impacts on the environment (Interview 34, NGO).

Overall, normative arguments were less prominent but did appear with one interviewee, for example, reasoning:

In a democratic society it is important that things, that afterwards have societal relevance, are not planned and implemented only by expert committees but that stakeholders are engaged early on. (Interview 11, NGO)

Consistent with some of the literature reviewed above, a few interviewees demanded greater engagement in research framing and agenda setting. Especially within the NGO realm, this was partly based on a fundamental questioning of scientific legitimacy claims:

Science likes to see itself as objective, but which questions are asked or how is often very random but has big implications. How research funding is distributed has for example great implications. Our science expert likes to say to the respective committees that they ask the wrong questions. (Interview 32, NGO)

Another NGO representative argued similarly:

Finding someone from a university that has never received external funding from somewhere and answers questions purely academically and, in inverted comma, neutral is in my opinion not completely possible. […] And then I have the opinion that it is better to make explicit those positions so that it can be evaluated from outside. And not to exclude everything under the cover of science. (Interview 11, NGO)

However, the majority of our interviewees, especially from the corporate, scientific, and political realm, showed a more normal view of science and a decisionist understanding of the risk analysis process. One interviewee from the corporate realm, for example, emphasized how the “most important thing for BfR […] is scientific neutrality” (Interview 21, corporate organization). Risk assessment needs to be “clearly separated […from the…] political aspects stakeholders raise […in order to provide…] clear assessment as a base for political decisions” (Interview 9, corporate organization, emphasis added).

Relatedly, the strategic relevance of greater public engagement emerged to be quite contested. On the one hand, a number of interviewees linked greater engagement to organizational promotion. The relevance for building trust, acceptance, and understanding found supporters as well with one representative from the scientific realm, for example, arguing, “if I create the possibility for engagement I can create acceptance” (Interview 5). An NGO representative explained:

Trust you have if you have the feeling you can check. And that it is transparent. Otherwise you can just hope that everything is fine. (Interview 34, NGO)

Another NGO representative made connections to the consumer, reporting,

Otherwise the consumer is […] given the feeling he can’t do anything anyway. He is put in a helpless position and that leads to political frustration and all that. (Interview 13, NGO)

On the other hand, many interviewees discussed potential negative strategic implications, for example, if actors publicly jump out off participatory processes (Interview 32, NGO). Linked to the previously mentioned persisting normal view of science, many were concerned that a participatory opening could have negative impacts on public perception as BfR “derives its reputation from being a scientific organization” (Interview 19, media). As one interviewee put it:

We trust the BfR because we think it is the institution that is independent from the government. If we don’t have that we can close the shop. (Interview 1, corporate organization)

In summary, our data suggest a general support for engagement in science-based risk assessment mainly based on pragmatic grounds. Both normative and strategic arguments emerged to be more contested with the majority of interviewees showing a more normal view of science that should be protected from undue influences. We will return to this prominent concern in the next sections.

Relevant Groups

Next to requesting our participants to rate the importance for BfR to engage different actor groups, our survey asked respondents to evaluate the perceived enacted engagement of these groups. The largest discrepancy emerged regarding consumers: despite their perceived importance, only 28 percent of the members of the public and 18 percent of the professional experts considered consumers to have great or very great influence on the work of BfR; 46 percent of the members of the public and 48 percent of professional experts perceived consumers to have little or no influence (Figures 3 & 4). NGOs, too, were considered to have relatively less impact by both professional experts and members of the public. A different picture emerged for the engagement of corporate actors who were associated with having great or very great influence on BfR work by 63 percent of the questioned members of the public, followed by scientists (55 percent), media (52 percent), political actors (49 percent), and public authorities (44 percent). Among the professional experts 45 percent attributed a great or very great impact to the business sector with greater acknowledgement especially of the influence of scientists (71 percent) but also political actors (56 percent) and public authorities (50 percent). Media was considered to have great or very great impact by 30 percent of the professional experts.

Figure 3.

How big do you think is the actual influence of the following groups on the work of BfR? In percent of all those that provided a response, n = 410 | Divergent to 100 ≙ answer “partly,” “no response,” “don’t know.” BfR = Bundesinstitut für Riskobewertung.

Figure 4.

How big do you think is the actual influence of the following groups on the work of BfR? In percent of all those that provided a response | Science (n = 910), consumers (n = 919), public authorities (n = 904), nongovernmental organizations (n = 896), media (n = 915), business (n = 914), politics (n = 911) | Divergent to 100 ≙ answer “partly.” BfR = Bundesinstitut für Riskobewertung.

Our interviews as well as previous studies on the topic (e.g., Ely et al. 2009a; IRGC 2012) support the resulting impression of a need for greater engagement of consumers. One interviewee from the media realm, for example, calls for BfR to “modify its direction and really address the consumer as well” (Interview 2, media). For many, the current approach of BfR (Wittkowski 2014), but also other organizations like EFSA (2016) to engage consumers mainly through consumer associations, is not sufficient. In the words of one interviewee: those associations can “give a specific consumer opinion. But if you really speak to end-consumers that is not really congruent” (Interview 26, corporate organization). The cited statement resonates with authors such as Krabbenborg (2013, 53) who criticizes treating civil society organizations as “voice of civil society.”

Looking across our interviews, we also find more skeptical views of the role of consumers. Resonating with long-standing scholarly discussions (Barker et al. 2010; Chilvers 2008; Fiorino 1990), one NGO interviewee (no. 32) puzzled to us how there is no single consumer that could represent a generalizable input for the whole group. Consumers may therefore need participation formats that differ from those of other stakeholder groups. Other major concerns included potential problems around limited knowledge resources, especially when it comes to highly specialized debates prominent at organizations like BfR. As one interviewee explains:

If I see for example how highly scientific the discussions are within the BfR commissions. I really have to question which consumer is able to participate in that. (Interview 10, corporate organization)

Similar concerns emerged for NGOs. Although many interviewees supported NGO participation considering their relevant knowledge and overall societal power, one NGO interviewee explained:

Many things we simply cannot attend to. You have to imagine that at our headquarters we have 40, 50 specialists. Those specialists have to work on many different topics. We usually have to cut down because it just gets too much. (Interview 32, NGO)

Another interviewee from a major NGO even argues:

If the societal development is modern governance in the widest sense as in we get everyone around the table so we get all the opinions to get the best outcome—we do not believe in that based on our experience. […] You likely have in every meeting representatives from industry that have the ability to take the time and prepare with own studies and research. NGOs will not be able to do that […] We just have an asymmetry in weapons. (Interview 29 NGO)

In addition to perceived power imbalances, interviewees from the non-NGO realm pointed to potential conflicting interests. In their view, NGOs draw upon creation and selling of fear to maintain their funding base. Engaging in participatory solution finding can arguably detriment this crucial agenda. Interestingly, some NGOs partially agreed with such statements with one NGO representative acknowledging:

NGOs sometimes need to simplify and overrate things to create the necessary audience. I can understand that some scientists then say they cannot agree with these statements anymore and develop an opposing position. But NGOs are also under a lot of pressure to take clear positions. (Interview 32, NGO)

Another argues:

Solving the problem we clearly do not see as our task. We do not have the set up for that. Our task is to thematize issues, take positions, initiate discussions. […] In such advisory committees it is often very detailed. […] Does the donor that finances us want us to sit there? Does he want us to spend donations on it? (Interview 37, NGO)

In terms of corporate engagement, interviewees, especially from the NGO realm, mirrored the impressions from the public survey and the wider literature (e.g., Demortain 2017; Michaels and Wagner 2003) where current engagement seemed to be considered to be too extensive. Looking at the interviews in more depth, this criticism mainly focused on corporate participation in BfR advisory committees as well as the use of industry studies during approval processes.³ At the same time, interviewees across societal groups, including some NGOs, argued that “especially agribusinesses belong to the spectrum. They are the ones affected” (Interview 18, science). As already mentioned in the General Reactions section, many interviewees pointed to relevant corporate data (Interview 10, corporate organization), know-how (Interview 6, NGO), and competency (Interview 11, NGO) concerning the assessment of risks. This holds especially for increasingly globalized supply chains where national authorities have limited reach.

A similarly ambiguous picture emerged around the engagement of political organizations and public authorities with interviewees demanding greater engagement while simultaneously criticizing too much political influence on BfR’s work. For one interviewee, for example, BfR is clearly “seen as part of the governmental apparatus” (Interview 34, NGO), while another explains how the setup of BfR “necessarily implies closeness to politics” (Interview 30, political organization).

Concerning the media, a number of interviewees supported the survey demand for engagement, pointing to the knowledge and influence media can have especially in terms of reaching the consumer (Interview 22, scientific organization). Interestingly, media representatives themselves showed diverse opinions about their engagement. While some indeed demanded direct and active participation, others saw the media more as indirect (Interview 19, media) transmitters of information. Similar to NGOs, some media representatives voiced concern about potential negative impacts on perceptions of their own independence. Another interviewee acknowledged how media representation can change the dynamic of participatory processes, in particular risking making participants less comfortable to speak openly—an issue that was supported by interviewees outside the media realm.

Form of Engagement

Engagement can be enacted through a variety of different mechanisms reaching from one-way provision of information, over consultation on predetermined topics, to two-way dialogue or deeper forms of participation where the participants are actively involved in decision-making processes (Rowe and Frewer 2005; Arnstein 1969). Figure 5 shows that for the professional experts we questioned information provision was considered most relevant for BfR (84 percent), followed by consultations and surveys (80 percent), a fixed advisory board (69 percent), dialogue (66 percent), and participation in risk assessment decisions (50 percent). Across all formats, experts from the corporate realm showed the lowest support with particularly low support for participation in risk assessment decisions (Table 2).

Figure 5.

What mechanisms should be used to engage the most important actors in the work of BfR? In percent of all questioned, n = 410 | multiple selection possible. BfR = Bundesinstitut für Riskobewertung.

Table 2.

What Mechanisms Should Be Used to Engage the Most Important Actors in the Work of BfR (Divided by Expert Groups)?

Engagegment Mechanism	Expert Groups					Total
Engagegment Mechanism	Politics (Percent)	Media (Percent)	Consumer associations (Percent)	Business associations (Percent)	Science	Total
Information	84.0	80.8	94.0	74.1	89.2	84.4
Consultation and surveys	79.0	75.0	96.0	70.4	81.4	79.5
Fixed advisory board	68.0	72.1	74.0	64.8	66.7	69.0
Dialog	62.0	77.9	76.0	44.4	64.7	66.1
Participation in risk assessment decisions	48.0	61.5	46.0	35.2	50.0	50.0
Other	14.0	7.7	18.0	5.6	12.7	11.5

Note: In percent of all questioned, politics (n = 100), science (n = 102), media (n = 104), business associations (n = 54), consumer associations (n = 50) | multiple selection possible.

The emerging picture of demand for more communication measures was supported within our interviews, adding that such communication should try to relate as much as possible to current discourses. In line with the survey, the value of consultations, an advisory board, and participation in decision-making was more contested. In terms of consultations, some valued them to explore different positions (Interview 2, media) and receive input (Interview 13, NGO) on specific questions (Interview, 18, science) as it enables the participants to “research a topic […] and provide something that is well thought about” (Interview 11, NGO). Linking back to the persisting modern worldview, several interviewees saw benefits in the limited impact consultations usually have on decision-making, considering that BfR should be an objective authority (Interview 9, corporate organization). For others, this same point was a major reason for objection. Across interviewees, there seemed to be frustration with consultations often having unclear results and influence (Interview 26, corporate organization) while absorbing significant time resources from both consulting and participating organizations. One interviewee explains:

One problem of consultations is that they suggest a public but say in the end: this perspective does not fit our pattern. As a result, the report stays more or less the same. (Interview 19, media)

Another major point of concern evolved around the need to find consensus across often very contrasting contributions while not allowing for dialogue to reach such consensus. Instead, consensus usually needs to be established by the consulting organizations, which tend to attribute the same weight to each contribution regardless whether “United Nations are commenting or the average person from the street” (Interview 11, NGO).

Mirroring previous findings (e.g., Renn 2009), the reactions to a potential stakeholder advisory board were similarly ambivalent. We found some interviewees praising advisory boards for their ability to establish dialogue (Interview 6, NGO) to work on things in-depth (Interview 32, NGO). Others were more critical, again connected to potential negative effects on the reputation of BfR within a predominantly modern worldview. Many suspected that participants in advisory boards, even more than in consultations, will demand impact on decision-making processes. For interviewees, especially those from the scientific, corporate, and political realms, such demands are likely to conflict with the independence (Interview 28, science) of a neutral scientific institution (Interview 21, corporate organization). Another prominent issue concerned the question of how the members of an advisory board could be chosen. Interviewees criticized how advisory boards tend to be appointed according to who can gain public approval. When discussing how such a selection should take place instead, we found a variety of interpretations. Many interviewees followed a pragmatic line of argumentation calling to include those that can contribute from a technical or scientific perspective and who share constructive and productive attitude (e.g., Interview 28, science). The latter statement links to prominent concerns about the potential for conflict. Next to frequently described conflicts between NGO and corporate actors, there may also be conflicts between public (scientific and other) authorities, especially in case of overlapping responsibilities. This was raised by representatives from political organizations but also acknowledged by public authorities themselves. We also found some normative arguments in the discussion of selection procedures, with some interviewees calling for just (Interview 6, NGO), balanced (Interview 32, NGO), or equal (Interview 2, Media) selection. While these notions were partly interpreted in terms of representativeness, there were also aspirations for equality and inclusiveness, especially considering previously described resource imbalances. However, several interviewees from the NGO as well as media realm refused when questioned whether they would be willing to participate in a potential advisory board:

Our experience is that if we want to take clear positions and be independent it is better if we don’t do this as a campaign organization. (Interview, 37 NGO)

Interviewees and some of the survey experts also raised their own suggestions for potential engagement formats. Most dominantly, this included the request for smaller formats that address particular actors (Interview 5, science) on specific topics (Interview 13, NGO). While larger events that reach a wider audience (Interview 26, corporate organization) and enable everyone to say something (Interview 1, corporate organization) found some supporters as well, many argued that they are often just about exchanging already known positions. Smaller formats, such as workshops or roundtables, can arguably facilitate a more open dialogue and exchange around different ways of thinking (Interview 2, media). For some, especially from the corporate realm, engagement should even be bilateral (Interview 9, corporate organization) with one interviewee explaining:

If media and NGOs participate there can’t be open communication […] So better single conversations with individual groups. (Interview 26, corporate organization)

Overall and across interviewees, there was agreement that the exact form of engagement should be contingent on the stakeholder but also the topic in question. This resonates with previous authors (Barker et al. 2010; Ely et al. 2009b; IRGC 2012; Stirling and Scoones 2009) who argued that (risk) organizations need to apply a combination of methods with the type and extent of engagement depending on the type of risk involved, particularly concerning its associated complexity, uncertainty, and ambiguity.

Discussion and Conclusion

Reacting to rising pressure on science in general and regulatory science in particular to open scientific processes up for greater public engagement, this study mobilized a combination of public and expert survey and interviews to explore the “empirical mandate” (Lövbrand, Pielke, and Beck 2011) for this pressure. More precisely, we studied the empirical reactions to a potential greater participatory opening of the BfR, a prominent regulatory scientific organization in the field of (food) risk governance. Our results suggest a general support for greater engagement in science-based risk assessment. This support mainly bases on pragmatic arguments while the normative and the strategic merit brings to the fore considerable contestation. Strategic aspects are especially debated as public engagement is seen as a potential detriment for the independence-based reputation and (empirical) legitimacy of BfR. This holds particularly for deeper forms of engagement where participants get actively involved in risk assessment decisions, for example, through an advisory board.

Looking behind these contestations and resonating with previous findings (Bickerstaff et al. 2010; Chilvers 2008; Korinek and Veit 2015; Gustafsson 2019), we can see that despite prominent participatory rhetoric, a normal, or modern view of science and decisionist understanding of risk analysis persists. Particularly actors from the scientific, political, and corporate realm seem to hold strong interests in maintaining an image of independent, objective, and reliable scientific processes. Resonating with Stirling and Scoones (2009) decisionist risk assessment continues to be judged more rigorous and robust than are the procedures associated with participation. The latter are often perceived to be overtly normative and hence a matter for risk management rather than risk assessment. Previous authors already described the substantial tensions that can arise—both internally and externally—due to these competing concepts of sound, independent science, and public engagement (Koenig et al. 2010; Small, Güvenç, and Dekay 2014).

Our findings concerning NGO interests are ambivalent and partly contradict the current literature. According to Lofstedt and Bouder (2017), for example, it has been mainly NGOs who have been pushing for a greater participatory opening of the risk assessment process. While some of our NGO interviewees supported such a picture, we also found considerable skepticism. This skepticism emerged not because participation may compromise the independent positioning of scientific organizations but because NGOs positioned themselves to be independent. In addition, several NGO representatives supported concerns about “stakeholder fatigue” (Dreyer and Renn 2009) and the normative merit of current engagement methods. This is in light of the significant time and human and financial resources necessary to participate, especially in highly specialized engagement processes, while the ability to mobilize these resources is unequally distributed across actor groups. Despite widespread support for greater engagement of consumers, similar asymmetry concerns emerged here. This finding connects with broader debates about problems in achieving real empowerment of the public within current participatory turns (Brandt et al. 2013; Briggs 2009; Lengwiler 2008).

We can summarize that while participatory rhetoric is supported on a general level, a number of actor groups hold (different) interests against a deeper enactment of such rhetoric. Looking at the wider discourse, this gives rise to an increasing contestation of different “institutional logics” (Greenwood et al. 2011) where calls for greater participation meet the maintenance of an image of independent, objective, and reliable science and strategically as well as normatively grounded criticism of current and (potential future) engagement practices. Similar to other hybrid or boundary organizations, such as the Intergovernmental Panel on Climate Change (Gustafsson 2019) or the UK Food Stand Standards Agency (Rothstein 2013), this poses significant challenges for BfR in constructing its legitimacy and reputation.

Our findings support literature that has pointed to the challenges regulatory scientific organizations face when moving across the boundaries of different social spheres (Jasanoff 1994; Demortain 2017; Winickoff and Bushey 2009; Korinek and Veit 2015; Rothstein 2013; Millstone 2007). Previous authors argued that to address this challenge boundary organizations need to speak “differently to different audiences” (Guston 2001, 405) or be “all things to all people” (Parker and Crona 2012). However, we arguably also need more discussion on the incommensurability of certain societal demands. This is to help boundary organizations navigate within their difficult social environment but also to facilitate the deeper institutionalization of more participatory practices. To support these discussions, future research should investigate the various colors of current participation logics as well as the deeper causalities underlying their (competing) construction. Especially if science, including regulatory science, is supposed to play a larger role in co-constructing solutions for the “grand challenges” of our times, such investigations are urgently needed.

Footnotes

Acknowledgments

The authors take whole responsibility for the analysis and views expressed in this paper but would like to thank all interview participants for taking the time to share their insights. We also thank Torsten Herold, Deniz Kocak, Natalie Berger, the editorial office, and two anonymous reviewers for their valuable comments.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: Financial support for this research was granted by the German Federal Institute for Risk Assessment.

ORCID iD

Leonie Dendler

Notes

References

Arnstein

S. R.

1969. “A Ladder of Citizen Participation.” Journal of the American Institute of Planners 35 (4): 216–24.

Bach

Doehler

. 2012. “Mandated Science and the Problem of Neutral Expertise: The Case of Governmental Research Agencies.” WZB Discussion Paper SP III 2012-602, WZB, Berlin, Germany.

Barker

G. C.

Bayley

Cassidy

French

Hart

Malakar

P. K.

Maule

Petkov

Shepherd

. 2010. “Can a Participatory Approach Contribute to Food Chain Risk Analysis?” Risk Analysis 30 (5): 766–81.

Beck

2007. Weltrisikogesellschaft: Auf der Suche nach der verlorenen Sicherheit. Frankfurt am Main, Germany: Suhrkamp.

Bickerstaff

Lorenzoni

Jones

Pidgeon

. 2010. “Locating Scientific Citizenship: The Institutional Contexts and Cultures of Public Engagement.” Science, Technology, & Human Values 35 (4): 474–500.

Bidwell

2009. “Is Community-based Participatory Research Postnormal Science?” Science, Technology, & Human Values 34 (6): 741–61.

Boel

G.-F.

Hensel

. 2009. “Tasks and Public Perception of the Federal Institute for Risk Assessment.” Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz 52 (12): 1179–87.

Bogner

2012. “The Paradox of Participation Experiments.” Science, Technology, & Human Values 37 (5): 506–27.

Brandt

Ernst

Gralla

Luederitz

Lang

D. J.

Newig

Reinert

Abson

D. J.

Wehrden von

. 2013. “A Review of Transdisciplinary Research in Sustainability Science.” Land Use 92 (2013): 1–15.

10.

Briggs

2009. “Risk Communication and Stakeholder Participation in the Governance of Systemic Environmental Health Risks.” International Journal of Risk Assessment and Management 13 (3-4): 195–215.

11.

Bundesamt für Strahlenschutz (BfS), Bundesinstitut für Risikobewertung (BfR). 2013. Bürgerbeteiligung im Umwelt- und Gesundheitsschutz: Positionen - Perspektiven - Handlungsfelder.

12.

Chilvers

2008. “Deliberating Competence: Theoretical and Practitioner Perspectives on Effective Participatory Appraisal Practice.” Science, Technology, & Human Values 33 (3): 421–51.

13.

Cleemput

Christiaens

Kohn

Léonard

Daue

Denis

. 2015. “Acceptability and Perceived Benefits and Risks of Public and Patient Involvement in Health Care Policy: A Delphi Survey in Belgian Stakeholders.” Value in Health 18 (4): 477–83.

14.

Danermark

Ekström

Jakobsen

Karlsson

J. C.

. 2002. Explaining Society: Critical Realism in the Social Sciences. London, UK: Routledge.

15.

Demortain

2017. “Expertise, Regulatory Science and the Evaluation of Technology and Risk: Introduction to the Special Issue.” Minerva 55 (2): 139–59.

16.

DG Sante. 2017. “Transparency and Sustainability of the EU Risk Assessment Model in the Food Chain: Ares (2017)6265773.” Accessed April 04, 2018. http://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-6265773.

17.

Dickel

Franzen

. 2016. “The ‘Problem of Extension’ Revisited: New Modes of Digital Participation in Science.” Journal of Science Communication 15 (1): A06_en.

18.

Dreyer

Renn

. 2009. “Introduction.” In Food Safety Governance. Integrating Science, Precaution and Public Involvement, edited by Dreyer

Renn

, 3–10. London, UK: Springer.

19.

EFSA (European Food Safety Authority). 2020. European Food Safety Authority. Accessed February 24, 2020. http://www.efsa.europa.eu/.

20.

EFSA (European Food Safety Authority). 2016. “EFSA Stakeholder Engagement Approach.” Accessed October 12, 2017. https://www.efsa.europa.eu/sites/default/files/EFSA%20Stakeholder%20engagement%20approach_FINAL.pdf.

21.

Ely

Stirling

Dreyer

Renn

Vos

Wendler

. 2009a. “The Need for Change.” In Food Safety Governance. Integrating Science, Precaution and Public Involvement, edited by Dreyer

Renn

, 11–28. London, UK: Springer.

22.

Ely

Stirling

Dreyer

Renn

Vos

Wendler

. 2009b. “Overview of the General Framework.” In Food Safety Governance. Integrating Science, Precaution and Public Involvement, edited by Dreyer

Renn

, 29–46. London, UK: Springer.

23.

European Commission. 2000. The Report of the Scientific Steering Committee’s Working Group on Harmonisation of Risk Assessment Procedures in the Scientific Committees advising the European Commission in the Area of Human and Environmental Health. Accessed November 26, 2015. http://www.bfr.bund.de/cm/343/first_report_on_the_harmonisation_of_risk_assessment_procedures.pdf.

24.

European Commission. 2003. Final Report on Setting the Scientific Frame for the Inclusion of New Quality of Life Concerns in the Risk Assessment Process: Adopted by the Scientific Steering Committee at its Meeting of 10-11 April 2003. Accessed April 01, 2016. http://ec.europa.eu/food/fs/sc/ssc/out362_en.pdf.

25.

The European Parliament, the Council of the European Union. 2002. Regulation No.178/2002 Laying Down the General Principles and Requirements of Food Law, Establishing the European Food Safety Authority and Laying Down Procedures in Matters of Food Safety. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L:2002:031:FULL&from=EN.

26.

The European Parliament, the European Council. 2013. Establishing Horizon 2020—The Framework Programme for Research and Innovation (2014-2020) and Repealing Decision No. 1982/2006/EC: Regulation No.1291/2013. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L:2013:347:FULL&from=EN.

27.

The European Parliament, The European Council. 2019. Regulation (EU) 2019/1381 of the European Parliament and of the Council of 20 June 2019 on the Transparency and Sustainability of the EU Risk Assessment in the Food Chain. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:L:2019:231:FULL&from=EN.

28.

Felt

Wynne

. 2007. Taking European Knowledge Society Seriously: Report of the Expert Group on Science and Governance to the Science, Economy and Society Directorate, Directorate-General for Research, European Commission.

29.

Fiorino

D. J.

1990. “Citizen Participation and Environmental Risk: A Survey of Institutional Mechanisms.” Science, Technology, & Human Values 15 (2), 226–243.

30.

Frewer

1999. “Risk Perception, Social Trust, and Public Participation in Strategic Decision Making: Implications for Emerging Technologies.” Ambio 28 (6): 569–74.

31.

Funtowicz

Ravetz

. 1993. “Science for the Post-normal Age.” Futures 25 (7): 739–55.

32.

Greenwood

Raynard

Kodeih

Micelotta

E. R.

Lounsbury

. 2011. “Institutional Complexity and Organisational Responses.” The Academy of Management Annals 5 (1): 317–71.

33.

Grunwald

2015. “Die hermeneutische Erweiterung der Technikfolgenabschätzung.” Technikfolgenabschätzung—Theorie und Praxis 2 (24): 65–69.

34.

Gulbrandsen

2011. “Research institutes as hybrid organizations: central challenges to their legitimacy.” Policy Sciences 44 (3): 215–30.

35.

Gustafsson

K. M

. 2019. “Learning from the Experiences of the Intergovernmental Panel on Climate Change: Balancing Science and Policy to Enable Trustworthy Knowledge.” Sustainability 11 (23): 6533.

36.

Guston

D. H

. 2001. “Boundary Organizations in Environmental Policy and Science: An Introduction.” Science, Technology, & Human Values 26 (4): 399–408.

37.

Holm

Halkier

. 2009. “EU Food Safety Policy.” European Societies 11 (4): 473–93.

38.

Inner City Fund (ICF) and Growth from Knowledge (GfK). 2018. EU Insights: Consumer Perceptions of Emerging Risks in the Food Chain. Accessed July 21, 2020. https://efsa.onlinelibrary.wiley.com/doi/epdf/10.2903/sp.efsa.2018.EN-1394

39.

International Risk Governance Council (IRGC). 2012. An Introduction to the IRGC Risk Governance Framework. Lausanne, Switzerland: EPFL International Risk Governance Center.

40.

Irwin

Rothstein

Yearley

McCarthy

. 1997. “Regulatory Science—Towards a Sociological Framework.” Futures 29 (1): 17–31.

41.

Jasanoff

1994. The Fifth Branch: Science Advisers as Policymakers. Cambridge, MA: Harvard University Press.

42.

Jasanoff

2003. “Technologies of Humility: Citizen Participation in Governing Science.” Minerva 41 (3): 223–44.

43.

Jasanoff

2011. “The Practices of Objectivity in Regulatory Science.” In Social Knowledge in the Making, edited by Camic

Gross

Lamont

, 307–38. Chicago: University of Chicago Press.

44.

King

2006. “Using Interviews in Qualitative Research.” In Essential Guide to Qualitative Methods in Organizational Research, edited by Cassell

Symon

, 11–22. London, UK: Sage.

45.

Koenig

Kuiper

H. A.

Marvin

H. J. P.

Boon

P. E.

Busk

Cnudde

Cope

davies

H. V.

, et al. 2010. “The Safe Foods Framework for Improved Risk Analysis of Foods.” Food Control 21 (12): 1566–87.

46.

Korinek

R.-L.

Veit

. 2015. “Only Good Fences Keep Good Neighbours! The Institutionalization of Ministry-agency Relationships at the Science-policy Nexus in German Food Safety Policy.” Public Administration 93 (1): 103–20.

47.

Krabbenborg

2013. “DuPont and Environmental Defense Fund Co-constructing a Risk Framework for Nanoscale Materials: An Occasion to Reflect on Interaction Processes in a Joint Inquiry.” NanoEthics 7 (1): 45–54.

48.

Latour

1998. “From the World of Science to the World of Research?” Science 280 (5361): 208–9.

49.

Lengwiler

2008. “Participatory Approaches in Science and Technology: Historical Origins and Current Practices in Critical Perspective.” Science, Technology, & Human Values 33 (2): 186–200.

50.

Liberatore

Funtowicz

. 2003. “‘Democratising’ Expertise, ‘Expertising’ Democracy: What Does This Mean, and Why Bother?” Science and Public Policy 30 (3): 146–50.

51.

Lofstedt

Bouder

. 2017. “Evidence-based Uncertainty Analysis: What Should We Now Do in Europe? A View Point.” Journal of Risk Research 1–20. doi: 10.1080/13669877.2017.1316763.

52.

Lövbrand

Pielke

Beck

. 2011. “A Democracy Paradox in Studies of Science and Technology.” Science, Technology, & Human Values 36 (4): 474–96.

53.

Michaels

Wagner

. 2003. “Disclosure in Regulatory Science.” Science 302 (5653): 2073.

54.

Millstone

2007. “Can Food Safety Policy Making Be Both Scientifically and Democratically Legitimated? If So, How?” Journal of Agricultural and Environmental Ethics 20 (5): 483–508.

55.

Millstone

2009. “Science, Risk and Governance: Radical Rhetorics and the Realities of Reform in Food Safety Governance.” Research Policy 38 (4): 624–36.

56.

Modell

2009. “In Defence of Triangulation: A Critical Realist Approach to Mixed Methods Research in Management Accounting.” Management Accounting Research 20 (3): 208–21.

57.

Moghissi

A. A.

Calderone

R. A.

McBride

D. K.

Geleta

Novak

Spivak

A. S.

Alavi

Estupigan

Koch

. 2017. “Innovation in Regulatory Science: Translational Regulatory Science based on Lessons Learned from Clinical Translational Science.” Journal of Translational Science 3(6): 1–6.

58.

Parker

Crona

. 2012. “On Being All Things to All People: Boundary Organizations and the Contemporary Research University.” Social Studies of Science 42 (2): 262–89.

59.

Renn

2009. “Possible Instruments for Extending Public Participation Beyond the Internet Forum and the Interface Committee.” In Food Safety Governance. Integrating Science, Precaution and Public Involvement, edited by Dreyer

Renn

, 179–94. Berlin, Germany: Springer.

60.

Renn

. 2016. “Inclusive Resilience: A New Approach to Risk Governance.” In IRGC Resource Guide on Resilience, edited by Linkov

Florin

M.-V.

. https://www.irgc.org/risk-governance/resilience/.

61.

Rittel

Webber

. 1973. “Dilemmas in a General Theory of Planning.” Policy Sciences 4 (2): 155–69.

62.

Rothstein

2013. “Domesticating Participation: Participation and the Institutional Rationalities of Science-based Policy-making in the UK Food Standards Agency.” Journal of Risk Research 16 (6): 771–90.

63.

Rowe

Frewer

L. J.

. 2005. “A Typology of Public Engagement Mechanisms.” Science, Technology, & Human Values 30 (2): 251–90.

64.

Small

M. J.

Güvenç

Dekay

M. L.

. 2014. “When Can Scientific Studies Promote Consensus among Conflicting Stakeholders?” Risk Analysis 34 (11): 1978–94.

65.

Standing Committee of the National People’s Congress of the People’s Republic of China. 2015. “Food Safety Law of the People’s Republic of China.” http://www.npc.gov.cn/zgrdw/englishnpc/Law/2011-02/15/content_1620635.htm.

66.

Stilgoe

Lock

S. J.

Wilsdon

. 2014. “Why Should We Promote Public Engagement with Science?” Public Understanding of Science 23 (1): 4–15.

67.

Stirling

2008. “‘Opening Up’ and ‘Closing Down’: Power, Participation, and Pluralism in the Social Appraisal of Technology.” Science, Technology, & Human Values 33 (2): 262–94.

68.

Stirling

Scoones

. 2009. “From Risk Assessment to Knowledge Mapping: Science, Precaution, and Participation in Disease Ecology.” Ecology and Society 14 (2): 14.

69.

TNS Opinion & Social. 2013. Responsible Research and Innovation (RRI), Science and Technology: Special Eurobarometer 401. Accessed November 23, 2015. http://ec.europa.eu/public_opinion/archives/ebs/ebs_401_en.pdf.

70.

TNS Opinion & Social. 2014. Special Eurobarometer 419: Public Perceptions of Science, Research and Innovation. Accessed November 24, 2015. http://ec.europa.eu/public_opinion/archives/ebs/ebs_419_en.pdf.

71.

Upham

P. J.

Dendler

2015. “Scientists as policy actors: a study of the language of biofuel research.” Environmental Science and Policy 47: 137–147.

72.

Url

2016. “Deepening the Dialogue. Science and Policy Making: Towards a New Dialogue.” The 2nd International Network for Government Science Advice Conference, European Commission & International Network for Government Science Advice, September 29–30, Brussels, Belgium.16.

73.

Vaughan

2011. “Elite and Elite-lite Interviewing: Managing our Industrial Legacy.” In Researching Sustainability: A Guide to Social Science Methods, Practice, and Engagement, edited by Franklin

Blyton

, 105–19. London, UK: Earthscan.

74.

von Wedel

2001. “Organisation des gesundheitlichen Verbraucherschutzes.” Schwerpunkt Lebensmittel; Gutachten der Präsidentin des Bundesrechnungshofes als Bundesbeauftragte für Wirtschaftlichkeit in der Verwaltung (Electronic ed.), Bonn.

75.

Walls

Rowe

Frewer

. 2011. “Stakeholder Engagement in Food Risk Management: Evaluation of an Iterated Workshop Approach.” Public Understanding of Science 20 (2): 241–60.

76.

Weingart

1999. “Scientific Expertise and Political Accountability: Paradoxes of Science in Politics.” Science and Public Policy 26 (3): 151–61.

77.

Winickoff

D. E.

Bushey

D. M.

. 2009. “Science and Power in Global Food Regulation: The Rise of the Codex Alimentarius.” Science, Technology, & Human Values 35 (3): 356–81.

78.

Wittkowski

2014. “Elemente einer unabhängigen Politikberatung.” Wie unabhängig kann Wissenschaft sein? Europäische Stakeholderkonferenz, edited by BfR, November 20–21, Berlin, 73–79.