Evidence-based causation in toxicology

Abstract

We introduced Evidence-based Toxicology (EBT) in 2005 to address the disparities that exist between the various Weight-of-Evidence (WOE) methods typically applied in the regulatory hazard decision-making arena and urged toxicologists to adopt the evidence-based guidelines long-utilized in medicine (i.e., Evidence-Based Medicine or EBM). This review of the activities leading to the adoption of evidence-based methods and EBT during the last decade demonstrates how fundamental concepts that form EBT, such as the use of systematic reviews to capture and consider all available information, are improving toxicological evaluations performed by various groups and agencies. We reiterate how the EBT framework, a process that provides a method for performing human chemical causation analyses in an objective, transparent and reproducible manner, differs significantly from past and current regulatory WOE approaches. We also discuss why the uncertainties associated with regulatory WOE schemes lead to a definition of the term “risk” that contains unquantifiable uncertainties not present in this term as it is used in epidemiology and medicine. We believe this distinctly different meaning of “risk” should be clearly conveyed to those not familiar with this difference (e.g., the lay public), when theoretical/nomologic risks associated with chemical-induced toxicities are presented outside of regulatory and related scientific parlance.

Causality has been variously discussed, defined, and debated since the early Greek philosophers and has been an important concept in toxicology and all phases of human health advancement. Although this concept has been acknowledged for much of recorded human history, it became a more focal concern in modern times, beginning in the 18th century when Sir Percivall Pott noted that chimney sweeps had a high incidence of scrotal cancer. As medicine and toxicology advanced in the mid-20th century, a set of criteria developed by Bradford Hill has become the accepted standard for causal inference of observational data. ^1,2 But recently this methodology has been improved upon by a more rigorous causation methodology known as evidence-based medicine (EBM). The introduction of evidence-based logic and guidelines quickly became the accepted standard of practice in medicine for reaching scientific decisions about questions of diagnosis, disease management, prevention and treatment, and the efficacy or harm of chemicals or agents used therapeutically and was soon recognized as one of the major milestones in the history of medicine. Seeing the gap between the development and widespread acceptance of evidence-based methods in medicine and the regulatory hazard decision-making process of weight-of-evidence (WOE) schemes used in environmental regulatory toxicology, we introduced evidence-based toxicology (EBT) in 2005. This was a comprehensive framework for performing transparent, auditable, and reproducible causation analyses for chemical exposures that may induce human disease. ³

EBM is the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients. ⁴ Knowledge about current best evidence is gained from a rule-based systematic review of all the applicable medical literature. We applied these same principles to toxicologic causation analysis and called for a parallel method that involved the systematic review and analysis of the scientific literature encompassed in three steps: (1) ask an appropriate question; (2) perform a comprehensive review and assemble the literature that address that specific question; and (3) using a predetermined quality instrument, rate, rank, and analyze the studies in a manner that provides the appropriate evidence to answer the question. Since our initial publication on EBT in 2005 there has been a continued and growing interest in EBT. ^{5 –8} The Johns Hopkins University created an endowed chair in EBT that focused on evidence-based principles to improve and validate testing methods in animals. ⁹ The US Food and Drug Administration adopts the same evidence-based principles for determining the acceptability of health claims. ^10,11 The Society of Toxicology has held several meetings related to EBT at or in conjunction with their annual meetings. ^{12 –15} The first step in any evidence-based analysis, that of producing a comprehensive, systematic review to address specific questions in toxicology, has now gained acceptance in the international toxicology community. ^{16 –19}

However, the term “evidence-based” in toxicology, as in medicine, has led to misunderstandings as scientists always believe they have relied on the available evidence. For example, the EBM, EBT, and Hill criteria causation methodologies we would argue were never a type or variation of the regulatory WOE method as some have erroneously listed it, but like us, Hartung, the other early proponent of EBT methods, have noted that past WOE schemes were the very antithesis of basic EBM/EBT principles and methods. ^20,21 In a similar vein, while our original article stated (and we still accept as true) that human data are the most valid metric to determine human causality, EBT does not call for eliminating the consideration of animal studies. In fact, our publications have consistently argued that when human data are insufficient to answer human causation and human risk questions, the regulatory risk assessment process will derive conservative, health-protective exposure guidelines in the interim. ^{3,22 –24} However, in the interest of providing greater transparency and accuracy in risk communication, we have proposed that regulatory agencies should be more explicit in informing the general public when their decisions do not represent a human-based causation determination but rely only on nonhuman data. In this situation, the possible hazard and associated “risk estimate” are only presumed expressions that contain an unknowable degree of uncertainty. This uncertainty is then used in regulatory risk assessments in a manner that ensures these regulatory guidelines and criteria are protective (but not predictive) of the actual human health outcome from a specific chemical exposure. ^{3,22 –25}

We note, despite recent advances in evidence-based causation methods, recent views discussing causation methodology continue to advocate dated practices and analytical methods to assess causation, ²⁵ for example, blending causation concepts with environmental regulatory acronyms and argot generated decades ago to support specific regulatory needs. So, we would point out that “causation/hazard” classification and human “risk/uncertainty” estimates are two examples where regulatory terminology seems to have ambiguated the medical and epidemiological use of these terms. For example, the default regulatory approach altered the method for identifying true human health hazards (i.e. known risk factors) by combining nomologic (animal data) and epistemic (epidemiology) information in their WOE classifications of the hazard (e.g. carcinogenicity). Figure 1 is a graphical representation of this process.

Figure 1.

The Regulatory WOE Methodologies (Adapted from figures or concepts in the works by Weed, Krimsky, and Adami et al. ^{26 –28} ). WOE: weight-of-evidence.

In regulatory-like WOE approaches, the final decision as to what combination of toxicological and epidemiological data is considered to represent sufficient evidence of causation is decided by the authoritative opinion. Because different combinations of human and animal data are used, the outcome is frequently a nomologic hazard/risk conclusion. The analysis is performed in a nontransparent, unauditable, and post hoc manner. This has led to biased conclusions in the past based on a less than complete acquisition of all available data. ²⁹ Yet the WOE approach has been pervasive in the US Environmental Protection Agency (USEPA) regulatory documents; for example, Lutter et al. report that this terminology was stated 37 times in the USEPA’s 2005 Carcinogen Risk Assessment Guidance. ³⁰ WOE is a term that implies a specific, objective method is being used when that is not always the case. In fact, one of the major limitations of the WOE concept in the past has been that there is no single definition for WOE. ^26,27 When 52 different WOE causation frameworks were systematically analyzed, no two contained the same process elements. ¹⁷ Still, others have noted that the regulatory WOE approach is essentially the antithesis of the evidence-based methods for determining causation, which were first adopted in medicine and then recommended for toxicology. ^3,20,21

Perhaps mindful of the linguistic and structural shortcomings of the past WOE schemes, Rhomberg and coworkers have recently developed a new type of WOE analysis methodology that has a different objective. ^30,31 Its purpose is to identify data gaps in the evidence supporting or contradicting the specified hypothesis. Interestingly, this type of WOE adopts some of the basic components of EBT, as outlined by Guzelian et al., and attempts to move WOE analyses from authoritative opinion/policy decisions toward a more evidence-based type of analysis where the amount and quality of the available evidence support or does not support the hypothesis being evaluated. ³ However, like the past WOE approaches, some decisions may still be derived solely from animal data and, therefore, contain the same species extrapolation uncertainties that apply to former regulatory approaches. Still, its adoption of some EBT causation principles, including better transparency, introduces a marked improvement over the past regulatory WOE assessments.

Our comprehensive framework, calling for the use of evidence-based logic to determine causal inference in toxicology, discussed the steps and procedures for reaching causal opinions consistent with that now applied in medicine. A graphical representation of how causal inference is established with this methodology is shown in Figure 2.

Figure 2.

EBT causation logic. EBT: evidence-based toxicology.

The focus of this approach is the application of a systematic collection of all pertinent available data and an objective methodology that evaluates the epidemiology data using an a priori set of guidelines for rating and ranking the findings of each study. In this manner, the available data itself establish whether there currently exists sufficient evidence to establish a causal linkage or not. As Figure 2 shows, in EBM and EBT, the emphasis is placed on epidemiologic evidence. In causation, the role of animal toxicology is to create/support the single Hill criterion of biologic plausibility. Here animal data may help direct the design of future epidemiology studies, provide mechanistic analyses, or provide for other comparative analyses of available epidemiologic and animal investigations. The push to adopt evidence-based principles to improve animal toxicity testing, initiated by Hartung at the Center for Alternatives to Animal Testing, should also lead to improvement in the use of animal data by leading to a better predictive value of the potential human health effects derived from specific toxicity tests.

Such efforts are needed because animal extrapolations may introduce uncertainties that adversely impact the accuracy of a hazard identification (qualitative extrapolation uncertainty) or its dose-related probability of occurrence (quantitative extrapolation uncertainty). Even the mechanistic/mode-of-action data can be misleading (mechanistic uncertainty) if the animal species (or strain) chosen to study a biologic pathway of interest fails to reflect the actual human pathway. ³¹ In short, the evidence-based method for human causation recognizes and utilizes the helpful insights that animal studies may provide without allowing the uncertainties inherent to each animal-to-man extrapolation to impact other Hill criteria, thereby becoming too pervasive in the overall causal analysis. There are a number of key failures with the application of animal extrapolations that amply illustrate why animal data should be judged cautiously and generally not used as the tipping point to reach a human causation conclusion where the epidemiology evidence is still not sufficient by itself. ^23,25,32,33 In particular, the recent tragedy of hormone replacement therapy is a clear evidence that predicting the human outcome via animal data when the epidemiologic evidence is insufficient can produce disastrous outcomes. In an EBT analysis, toxicology provides guidance to epidemiology, as noted above, but qualitative studies of the actual human response provide the final answers regarding human causation.

The terms “risk” or “risk factors” is another area where regulatory agencies have introduced a broader, less specific meaning that differs from the way these terms are used in medicine and epidemiology. The term risk has three components to it: (1) the causal agent (risk factor); (2) the adverse effect(s) (the hazard); and (3) the probability that exposure to the agent will result in the adverse effect (the numerical risk). These are referred to as: (1) the causal agent component; (2) the hazard (adverse effect) component; and (3) the probability (dose–response) component. Examples of an epistemic or real human risk statement are illustrated below:

There is a 10% risk [probability component] of getting lung cancer [hazard component] for any person with a 20+ pack-year history of cigarette smoking [causal agent component].

Or,

Statistics have shown that the risk of drowning [hazard component] is 1/100,000 [probability component] when swimming without a life jacket [causal agent component].

In contrast, when causality is not known or knowable because of insufficient human data, and answers from animal data are used to close the data gaps, then an unquantifiable amount of uncertainty is introduced into all three components of the risk statement. This reduces a true human risk statement to a hypothetical risk statement. When the causal relationship is not known to be true, both the causal and hazard components are missing and are replaced by hypothetical presumptions of unknown certainty. In turn, the probability component is reduced to a possibility component because one does not have an actual measure of the true human risk when using animal data. Instead, the true probability has been replaced by another uncertain species extrapolation, and so the numerical estimate provided represents just one nomologic possibility of uncertain accuracy. On this issue, we note that a regulatory agency like the USEPA has long recognized that their “risk assessments” or “risk statements” contain presumed or hypothetical components. For example, regarding the toxicity constants used to generate “USEPA-like risk/uncertainty” estimates provided in its Integrated Risk Information System (IRIS), the USEPA notes what many nonscientists fail to understand about these values, that is, they cannot be used to predict the true incidence of human disease. This is because they are not true measures of the human risk, if it is in fact present.

In general IRIS values cannot be validly used to accurately predict the incidence of human disease or the type of effects that chemical exposures have on humans. This is due to the numerous uncertainties involved in risk assessment, including those associated with extrapolations from animal data to humans and from high experimental doses to lower environmental exposure. The organs affected and the type of adverse effect resulting from chemical exposure may differ between study animals and humans. In addition, many factors besides exposures to a chemical influence the occurrence and extent of human disease. ³⁴

Likewise, since 1986, the USEPA has noted that its cancer risk estimates only represent an upper bound estimate of the possible human risk, the true human risk may be a much lower number and in fact may be zero.

These concessions, where the probability component of USEPA risk statements is based on assumed hypotheticals, clearly indicate their use of the term risk is decidedly different from the way this term is applied in epidemiology and medicine. Recognizing that these regulatory risk estimates are protective but not predictive is merely a scientific fact that has been conceded by the agency itself. This feature is something that limits their use but does not undermine their intended purpose, that is, to provide protective exposure guidelines. So, while we have always agreed this approach of using hypothetical presumptions meets the goal of setting health protective exposure guidelines whenever human causation is not knowable, we disagree with calling these uncertain hypothetical characterizations of nomologic possibilities risks. Risk is an epistemic term in medicine, and the regulatory use of nomologic possibilities should more correctly be labeled as such (e.g. “hypothetical risks” or “risk uncertainties” or “nomologic risks”) to differentiate possible/nomologic hazards from those epistemic determinations of known risks and risk factors. In fact, it is for this reason the National Academy of Sciences stated there is no regulatory necessity to corroborate human causation for animal-derived carcinogenic or noncarcinogenic hazards, as the USEPA safety/risk assessment is the same, regardless of whether or not human causation is known. ³⁵

In summary, an understanding of the benefits of applying evidence-based logic to toxicology has improved considerably in the last decade (see Figure 3).

Figure 3.

Acceptance of the evidence-based causation method or its components. ^{5,10,11,16,31,36 –41}

Certain strengths of this methodology, like evidence-based data gathering and review, are now widely adopted in toxicology today. The improvement in medical practice created by EBM is now indisputable; in fact, it has been cited as one of the top 15 milestones in modern medicine. ⁴² So, on this 10th anniversary, we renew our suggestion that toxicologists likewise adopt evidence-based causation logic when addressing causal issues. EBM/EBT provides an objective and transparent method by which causation is determined, and its use would help eliminate the limitations of regulatory-like WOE analyses similar to those lodged more than a decade ago by Ruden. ²⁹ EBT has always required the systematic collection and review of the literature, a process now widely endorsed in toxicology. ³ Adding an a priori study analysis methodology eliminates the variations created by differing authoritative opinions that might otherwise influence the overall causation analysis, improving the reliability and consistency of the final decision. EBM/EBT requires a transparent and auditable decision-making process guided by the availability of good evidence itself. It recognizes the differences between nomologic and epistemic risks, something that does not adversely impact toxicology or regulatory goals, and it would improve the accuracy of the risk statement being communicated to others by identifying which of the three components were based on epistemic evidence versus an uncertain assumption or extrapolation that represents one of the several nomologic possibilities. We resubmit that such changes enhance the causation analyses of adverse effects and anticipate a further transition toward this goal in the years to come.