Conflicts of interest in the International Agency for Research on Cancer process of identifying carcinogenic hazards to humans

Abstract

Managing conflicts of interest (COIs) in scientific decision-making is important for minimizing bias and fostering public trust in science. Proper management of COIs has added significance when scientists are making decisions that impact public policy, such as assessing substances for carcinogenicity. The International Agency for Research on Cancer (IARC) organizes expert working groups to identify putative carcinogens and determine whether or not the hazard is likely to present significant potential harm to humans. While IARC has policies for managing COIs, prior professional experience with the substance being assessed is not defined as a COI. Indeed, IARC working group members are chosen based on subject matter expertise, including prior publication on the substance under review. However, a person’s prior experience with a substance poses a significant potential COI by equipping them with strong pre-existing views about the substance’s toxicity and carcinogenicity. To minimize the risk of bias in IARC working groups, participants with voting powers should be independent scientific experts with sufficient professional experience to review carcinogenicity data but with no substantial prior experience with the substance under review. A related IARC practice restricting data review by working groups to selected publications is another significant COI. Instead, all accessible data should be available for consideration by working groups in assessing the carcinogenic hazard of substances. Another recommendation to reduce potential bias would be to reinstate the option of “probably not carcinogenic to humans”.

Introduction

The International Agency for Research on Cancer (IARC) ¹ is a branch of the World Health Organization (WHO) ² tasked with identifying potential cancer hazards for humans. IARC solicits nominations for agents to review from the public, scientists, national health agencies, and other organizations, and publishes the resulting opinions in IARC Monographs. In contrast, the Joint Food and Agriculture Organization (FAO)/World Health Organization Expert Committee on Food Additives (JECFA), ³ and government agencies like the European Food and Safety Authority (EFSA), ⁴ the European Chemicals Agency (ECHA, an agency of the European Union Member States), ⁵ U.S. Environmental Protection Agency (EPA), ⁶ and U.S. Food and Drug Administration (FDA) ⁷ perform comprehensive risk assessments of substances, including those reviewed by IARC, to determine if approved use levels of active substances are associated with cancer and other adverse effects, thereby protecting human health.

“Hazard identification” and “risk assessment” are related concepts but are distinctly different in terms of their implications. Hazard identification, the first step of risk assessment, ⁸ examines whether an agent has the potential to produce harm in humans under any circumstances by any exposure route (e.g., inhalation, ingestion, topical). Risk assessment examines the likelihood that a theoretical hazard poses a threat to humans under “real world” conditions (e.g., exposure level, duration, and route[s] of exposure that occur during every-day life), and it evaluates the exposure threshold of a substance above which it might cause harm. ⁹ Hazard identification and risk assessment involve review of large, complex but incompletely overlapping data sets, so the final interpretations and evidence-based decisions may diverge.

Importantly, the IARC process allows experts with apparent or even genuine conflicts of interest (COI) to serve on a working group. IARC indicates that allowing working group members with COIs to participate “in limited numbers” (where “limited numbers” is not defined) prevents them from unduly influencing the deliberations and ultimate consensus. ¹⁰ Our position is that even one person with a COI, wielding all the privileges of a working group member and strongly advocating for a position at odds with other group members, may influence the review outcome and erode public confidence in the process. A COI is a situation in which a person has an interest that may compromise their judgment or performance of their ethical, legal, or professional obligations. Although most COI policies focus on financial and personal interests, COI declarations should include any professional experiences that create the possibility of bias or mistrust in decision-making (Note 1).

Decision-making apparatus of IARC

Working groups in IARC

The IARC process for identifying carcinogenic hazards employs four independent working groups of subject matter experts that examine distinct data sets: exposure characterization, cancer outcomes in humans (epidemiological studies), cancer in animals (toxicity studies under controlled experimental conditions), and mechanistic evidence (encompassing in silico, in vitro, ex vivo, and in vivo assays). IARC working groups review publications selected by IARC; discuss data; prepare text, tables, and analyses for the monographs; and vote regarding the evidence of carcinogenic hazard for the substance under review. ¹⁰ The IARC Preamble ¹⁰ states that outcomes represent “consensus” perspectives (reflecting majority decisions of voting working group members) that all members agree to support even when it is not everyone’s preferred outcome.

Classification of substances

Since 2019, IARC working groups classify substances into one of four categories: “carcinogenic to humans” (Group 1), “probably carcinogenic to humans” (Group 2A), “possibly carcinogenic to humans” (Group 2B), and “not classifiable as to its carcinogenicity to humans” (Group 3). ¹⁰ In the revised 2019 IARC Preamble, the option of “probably not carcinogenic to humans” (Group 4) was removed.^10,11 This category was formerly available to classify agents that lacked evidence for carcinogenicity in humans and experimental animals. By definition, exclusion of this option imposes a potential bias and indicates that the process is defective in premise and thus ineffective at best.

IARC/WHO Declaration of Interests for working group members

Before invitations are extended to individuals to serve on an IARC working group, each potential participant is expected to complete an IARC/WHO Declaration of Interests (DOI) form to report possible COIs. Examples of self-reported COIs include financial interests such as paid employment, consulting income (including financial compensation as an expert witness), and investments; individual and institutional research support; and non-financial interests such as public statements and positions related to the meeting’s subject. The WHO defines the term “conflict of interest” as “any interest that may affect, or may reasonably be perceived to affect, the expert’s objectivity and independence.” ¹² Interests specifically addressed by this WHO policy are listed in Table 1 and include the individual’s interests and also those of immediate family members.

Table 1.

Conflicts of Interest specifically addressed by World Health Organization policy.

• Employment

• Consulting

• Grants, collaborations, contracts, and other sources of research support

• Honoraria and speaking fees

• Ownership of stock and other investments

• Commercial business interests and intellectual property

• Public statements, such as expert testimony for regulatory or judicial processes

• Unpaid offices or positions related to the subject of the working group meeting

IARC’s assessment of declaration of interests

Once IARC/WHO DOI forms are submitted, IARC assesses the declared interests of potential working group members to determine whether a possible COI may warrant any limitation on an individual’s participation. The IARC Secretariat may conclude that no COI exists, that the interest is irrelevant or insignificant, or that the interest is potentially or clearly significant. ¹² For the latter case, one or more of the following three measures for managing the COI may be applied by the Secretariat: (i) full participation is allowed, with public disclosure of the interest; (ii) partial exclusion is implemented (e.g., the individual is barred from that portion of the IARC work related to the declared interest and also from the corresponding decision-making process); or (iii) total exclusion is mandated. ¹²

IARC Preamble: selection of working group members

The IARC Preamble ¹⁰ makes three points about scientists selected as working group members (Table 2). Despite the IARC assertions to the contrary, this approach is likely to choose individuals who have professional, political, and/or other vested interests that lead to pre-existing, and likely strongly held views regarding the toxicity profile (including carcinogenicity) of a substance, the existence of which may reduce their objectivity. Both support and criticism have been voiced regarding this IARC selection procedure.^13–16

Table 2.

International Agency for Research on Cancer policy on selection of working group members. ¹⁰

1. “IARC selects Working Group members on the basis of expertise related to the subject matter and relevant methodologies, and absence of conflicts of interest.”

2. “Working Group members generally have published research related to the exposure or carcinogenicity of the agents being reviewed and IARC uses literature searches to identify most experts”.

3. “IARC’s reliance on experts with knowledge of the subject matter and/or expertise in methodological assessment is confirmed by decades of experience documenting that there is value in specialized expertise and that the overwhelming majority of Working Group members are committed to the objective evaluation of scientific evidence and not to the narrow advancement of their own research results or a pre-determined outcome.”

Professional experience and potential for bias in IARC

The IARC COI policy does not include professional experience with the substance under review as a COI. Indeed, publications on the substance under review are treated as part of the desirable criteria for selecting working group members. We are of the opinion that experience with the substance being assessed is a COI that should be disclosed and managed appropriately.

Some readers may find this assertion to be counterintuitive because, in other areas of science, experience with a subject is usually not regarded as a COI. For instance, peer reviewers for professional journals and funding agencies are chosen deliberately for their expertise. ¹⁷ From a practical perspective, it is pointless to require peer reviewers to disclose their experience as a COI since almost all researchers who have progressed beyond graduate school possess this type of potential bias. Instead, journal editors generally consider a few specific COIs that should be avoided such as co-authoring publications with at least one of the authors in the past 3 years and being colleagues within the same section/department or similar organizational unit in the past 3 years.

One recent example (from the June 2023 IARC meeting) that we believe has violated this “no conflict of interest” policy was the inclusion of the director of the research institute where the positive animal carcinogenicity studies were performed as a voting member of an IARC working group. ¹⁸ Voting membership presented the clear potential for a COI; consciously or not, their remarks and vote could have been biased toward a determination of ‘sufficient evidence’ of carcinogenicity because that decision would have been in the interest of the reputation and future funding of their testing facility. Therefore, a more appropriate role would have been as an Invited Specialist rather than a voting group member. As another example of potential COIs (also at the June 2023 IARC meeting), 10 of 17 working group members in the exposure, epidemiology, and animal sub-groups for the substance aspartame had previously published (sometimes extensively) on the carcinogenicity of this substance and/or other related substances.

Whether something should be treated as a COI in science depends on the context of the decision being made and the importance of minimizing bias. ¹⁷ In many situations in science, such as reviewing a paper for a journal, a moderate level of subject matter familiarity (bias) is unavoidable and even acceptable. In others, however, such as serving on a committee making drug approval decisions (see discussion below), a high degree of objectivity is required. ¹ We propose that, given the global impact of IARC’s decisions related to public health and environmental regulation and policy, prior professional experience with the substance being investigated constitutes a COI that should be disclosed and managed accordingly.

For an instructive analogy, consider juries, which are charged with rendering impartial verdicts based on the evidence presented in court and not on their biases related to their personal beliefs or prior experiences. Potential jurors may be excluded from serving if they have previous knowledge (i.e., pre-formed opinions) about the individuals or subject of the case. ¹⁹ We argue that IARC working group members should function like jurors who impartially evaluate evidence unsullied by pre-formed opinions.

There are several reasons why professional experience with a substance can be a significant source of bias for IARC working group members. First, working group members who have built a reputation based on publications, research funding, or publicly expressed conclusions regarding a substance may be susceptible to confirmation bias—that is, the tendency to form a judgment based on what one wants or expects to believe based on prior experience with the substance rather than on a review of the currently available evidence alone—and will argue vigorously and cast their votes to enshrine their perspective. ²⁰ Thus, when an IARC working group member has a history of publishing on the carcinogenicity (or the lack thereof) of a substance under review, such prior work represents an established perspective (i.e., “prejudice,” whether overt or covert) with a high potential of impacting the objectivity of their review and final conclusions.

Second, an IARC working group member who has experience with a substance under review may have a professional interest in ensuring that their opinions concerning the substance are endorsed as “valid.” In seeking such verification, particular credence is paid to the support offered by other experts and professionals including scientists and regulators, funding agencies, or even attorneys. Indeed, such affirmation is essential to sustain future research support or income from consulting, serving as an expert witness, or speaking.^21,22 Although IARC policies treat current and previous research support and income as COIs, they do not address a member’s prospects of receiving such benefits in the future.

The failure to treat professional interests as potential COIs is a serious flaw in IARC policy given the nexus between research experience and these professional interests. Scientists who build a publication portfolio on a particular topic (such as the carcinogenicity of a chemical) can enhance their chances of receiving funding, consulting arrangements, and/or scientific accolades not only from government or private industry but also from advocacy groups, which are playing an increasingly influential role in supporting and even directing scientific research. Although scientists have always advocated for various political causes, the connection between science and politics has grown stronger in the last couple of decades as evidenced by the impact of non-governmental organizations on scientific research directions, funding, and policy.^22,23

Third, when several IARC members have professional experience working with a substance under review, their perspectives may have a disproportionate impact on group deliberations. Research has shown that vocal members of groups charged with making business or public policy decisions can dominate the group’s deliberations, resulting in conclusions based on “groupthink” while divergent opinions are discounted or not considered.^24,25

Decision-making apparatuses of other regulatory agencies

Decision-making by IARC (that determines hazard identification) and other entities (that determine risk assessment) varies in some important respects, in terms of both preparation (enrolling participants) and process (discussion rules and voting systems used in decision-making). The conclusions and recommendations of these bodies may therefore also differ from those of IARC. This is because hazard assessment and risk assessment are, though overlapping, different processes with different goals and significance. Importantly, the material available to different bodies varies; IARC only uses select published (and peer reviewed) data whereas some regulatory bodies also use material provided by industry which, because of confidentiality, is not available for traditional 2-3 person peer review and assessment by IARC working groups. Importantly, management of COIs varies dramatically among institutions (see Supplemental Section I for comparison).

Reducing bias in IARC decision-making: the “invited specialist” option

To be effective, IARC working group members need expertise in assessing data concerning toxicity, metabolism, mutagenicity, pathology, carcinogenicity, and other fields of science but do not require expertise related to the specific substance being evaluated. We agree that scientists with professional experience on the substance being assessed can be a valuable resource for IARC deliberations, with appropriate guardrails .

For example, those who have conducted research studies for the substance under review may have tacit knowledge ²⁶ (gained from personal experience with the substance) and could answer questions and clarify methods that cannot easily be gleaned from published articles. To avoid undue bias, such individuals should not be permitted to engage in discussions or evaluate and ratify conclusions resulting from their own work. Accordingly, such subject matter experts should not be IARC working group members with voting (i.e., decision-making) powers but instead should be engaged as non-voting “Invited Specialists”. According to the IARC Preamble, ¹⁰ “Invited Specialists are experts who have critical knowledge and experience but who also have a conflict of interest that warrants exclusion from developing or influencing the evaluations of carcinogenicity. Invited Specialists do not draft any section of the Monograph that pertains to the description or interpretation of cancer data, and they do not participate in the [consensus] evaluations. These experts are invited in limited numbers [emphasis added] when necessary to assist the Working Group by contributing their unique knowledge and experience to the discussions.”

Process of IARC’s selection of data for review and final deliberations

The IARC review process also may be skewed by the choice of data available for review by working group members. According to the Preamble ¹⁰ and Monograph Instructions to Authors, ²⁷ the IARC Secretariat screens the retrieved literature (from the Health Assessment Workspace Collaborative [HAWC] literature search tool) ²⁸ for inclusion based on title and abstract review, according to “pre-defined criteria” (which are not described in the IARC process). The resulting literature searches, “in consultation with a librarian and other technical experts, address whether the agent causes cancer in humans, causes cancer in experimental systems, and/or exhibits key characteristics of established human carcinogens (in humans or in experimental systems)”. Then IARC defines its “principles of systematic review” for inclusion/exclusion in the Preamble. ¹⁰ However, the Preamble states that “[i]nclusion of a study does not imply acceptance of the adequacy of the study design or of the analysis and interpretation of the results”. ¹⁰ In this way, clearly inadequate studies can be included or well-conducted studies (performed under internationally accepted harmonized guidelines and Good Laboratory Practice (GLP) principles^29–31 and evaluated by regulatory agencies as the basis for registration) may be excluded in any review. Using a system that allows selection of inadequate studies or exclusion of well-designed studies is not good scientific practice though this is of less significance in producing what is essentially an opinion about a substance. Good systems exist for establishing the value of systematic reviews,^32–35 and as review articles may be excluded by the IARC process for data selection this omission may ignore many high-quality examinations of carefully characterized data.

IARC typically limits the data available for review to published articles in the public domain, which ignores data contained in unpublished reports (e.g., academic and government research facilities, contract research organizations, and industrial firms). The IARC Preamble ¹⁰ states that “studies may be excluded if they were not about the agent (or a metabolite of the agent), or if they reported no original data on epidemiological or toxicological endpoints (e.g., review articles).” However, there may be publicly available relevant data, publications and/or review articles that outline flaws in study design, conduct, analysis, interpretation, and reporting of the study findings under review by the IARC working groups that could be pertinent to the working group deliberations. To eliminate bias in the process of selecting data for review, an additional update that should be made to the IARC process is to include all toxicological studies relied upon by regulatory agencies in countries that are WHO members. In addition, publications of high scientific merit and quality, including review articles and experimental critiques as well as documents prepared by health authorities and regulatory agencies on the substance being evaluated should be included. Having an independent panel of scientific experts with experience in carcinogenicity assessment rather than IARC staff determine which studies to share with the working group for consideration would also be a worthwhile change. This expanded scope would allow all available data (i.e., both negative and positive carcinogenicity and mechanistic studies, dose response, and reproducibility) to be weighed in reaching decisions.

Conclusions and future perspectives

A firm commitment to scientific integrity and accountability warrants a reconsideration of IARC’s approach to choosing working group members and research data to be considered in assessing carcinogenic hazards. Rather than emphasizing literature searches to identify most working group members who have published research related to the exposure or carcinogenicity of the agents being reviewed, IARC should choose experienced scientists with general knowledge of exposure characterization, cancer in humans, cancer in experimental animals, and mechanistic evidence (depending on the assigned working group). Prior professional experience (including publications) with the substance (or similar substances) under review should be included in the declaration of interests and treated as a potential COI because it creates a risk of pre-judgment (bias) that can adulterate IARC conclusions, and scientists with substantial professional experience related to a substance should be asked to serve as non-voting Invited Specialists, providing context but with limited engagement during data evaluation and discussions. Additionally, published paradigms and accepted methodologies should be used by an independent panel of experts to determine the worth of studies and allow additional data (including review articles and experimental critiques) so that the working group members may draw their own conclusions regarding which data are from well-designed and conducted studies, and thus relevant for consideration during IARC hazard assessments.

The current lack of transparency is another area that should be reevaluated and updated by IARC. Meeting minutes of IARC working groups should be publicly available and reflect the vote totals (but not the votes of individual working group members) supporting the final IARC decision, all deliberations including the basic tenets of divergent opinions, and perspectives of non-voting Invited Specialists. Meeting minutes should also be approved by all working group members. In addition, participants’ curricula vitae and COI declarations regarding financial interests, prior publications by working group members on the substance (or substances of the same use or chemical class) being reviewed, etc. should be readily accessible to the global scientific community, health authorities and regulatory agencies, and the public to further attest that the review was objective.

Although no system of evaluating the carcinogenicity of substances is perfect, minimizing bias and promoting the public’s trust in the system should be paramount. Given the wide implications of IARC decisions, the public and the scientific community deserve detailed knowledge regarding how these decisions are made. Although IARC procedures^10,11 are to “embody principles of scientific rigor, impartial evaluation, transparency, and consistency”, IARC processes and decisions require additional information attesting to impartiality and transparency. IARC has modified its processes for the identification of carcinogenic hazards to humans by interdisciplinary groups in the past ¹¹ and will hopefully continue to do so. Adjustments to further limit COIs among IARC working group members would be a simple and welcome modification fit for this purpose.

Supplemental Material

Supplemental Material - Conflicts of interest in the international agency for research on cancer process of identifying carcinogenic hazards to humans

Supplemental material for Conflicts of interest in the international agency for research on cancer process of identifying carcinogenic hazards to humans by Susan A Elmore, Colin Berry, Brad Bolon, Gary A Boorman, Alys E Bradley, Samuel M Cohen, James E Klaunig, Felix M Kluxen, Robert R Maronpot, Abraham Nyska, Tracey L Papenfuss, Jerold E Rehg, David B Resnik, Ivonne MCM Rietjens, Thomas J Rosol15, Andrew W Suttie, Trenton R Schoeb, Christian Strupp, Bob Thoolen and Klaus Weber in Human & Experimental Toxicology.