National Toxicology Program Position Statement on Informed (“Nonblinded”) Analysis in Toxicologic Pathology Evaluation

Abstract

The National Toxicology Program (NTP) uses histopathological evaluation of animal tissues as a key element in its toxicity and carcinogenicity studies. The initial histopathological evaluations are subjected to a rigorous peer review process involving several steps. The NTP peer review process is conducted by multiple, highly trained, and experienced toxicological pathologists employing standardized terminology. In addition, ancillary data, such as body and organ weights and clinical pathology findings, are used to corroborate the diagnoses. The NTP does employ masked analysis to confirm subtle lesions or severity scores, as needed, and during its Pathology Working Groups. The use of masked analysis can have a negative effect on histopathological evaluation because it is important for the pathologist to compare treated groups to the concurrent controls, which would not be possible in a blinded evaluation. Therefore, the NTP supports an informed approach to histopathological evaluation in its toxicity and carcinogenicity studies.

Introduction

The National Toxicology Program (NTP) is an interagency public health partnership of the US National Institutes of Health, Food and Drug Administration, and Centers for Disease Control that characterizes potential hazards of chemical and nonchemical agents of human importance. Animal studies are an important component of a broad portfolio of animal and nonanimal capabilities that are developed and applied in this program. Histopathological evaluations are a key element of our animal study-based hazard assessments. The aim of the histopathological evaluation is to identify and characterize the test article-related changes in tissues and organs, decide their pathologic importance to the overall health of the animal, and predict their relevance to humans.

The NTP supports an informed approach to histopathological evaluation in its toxicity and carcinogenicity studies. Informed (or “open,” “nonblinded,” or “unmasked”) analysis refers to histopathological evaluation with full knowledge of the dose/exposure groups and other information on the study animals. The issue of informed analysis in the evaluation of toxicity and carcinogenicity studies has been a topic of discussion in toxicologic pathology for many years. The point of most contention is that the inherent subjectivity of histopathology assessments has the potential to introduce bias (conscious or unconscious) into the pathology data and, ultimately, the integrated assessment of the study. Histopathology is sometimes described as an “art” suggesting an inherent subjectivity. Although the practice of histopathology is more subjective than many other forms of data collection, significant processes are in place at the NTP to mitigate any bias that might arise by virtue of that subjectivity. Evaluating studies in an open or nonblinded manner is generally considered to be in opposition to the traditional principles of the scientific method, and it may be assumed that the subjectivity of histopathology assessments can affect the reproducibility of a study. The steps taken by the NTP to mitigate the potential for bias while ensuring the integrity and reproducibility of its histopathology data include the use of an independent, multilevel, and rigorous peer review process; highly trained pathologists with no conflicts of interest; international and published standards of terminology; and the application of ancillary data in study interpretation.

The NTP Pathology Review Process

In NTP studies, the supervision of the necropsy and the initial histopathological assessment are performed by the study pathologist at the contract laboratory that conducted the study (the “study laboratory”). The study pathologist is a board-certified veterinary pathologist with experience in toxicologic pathology and has been approved by the NTP to read their studies. The study pathologist performs the initial evaluation of the slides and contributes to the study laboratory report. The data are not considered final, however, until after completion of the NTP peer review process.

Once all the study materials and the approved study laboratory report have been submitted to the NTP, the NTP begins its thorough review of the slides, blocks, pathology tables, and other ancillary information, which includes elements of both quality control and quality assessment (QA). All of the steps in the NTP pathology review process are overseen by an NTP pathologist who is assigned to the study. What follows is a description of the steps in the rigorous peer review process performed at the NTP.

The first step in the pathology review process is a pathology data review (PDR), including the study laboratory report (including any protocol deviations) and the data tables—both the individual animal records and the summary tables. The macroscopic and microscopic findings, clinical signs, body weights, organ weights, and clinical pathology results are reviewed. The necropsy records are examined for entry errors, masses that occurred on an animal in life but were not seen or recorded at necropsy, unaccountable gross observations, potential duplications in terminology or errors in terminology, and any other observation or organ notes that will need to be addressed during the QA review. This review is performed by a pathologist independent of the study pathologist (the PDR pathologist) at a pathology laboratory under contract to the NTP. Subsequently, the PDR pathologist prepares a report and submits it to the NTP. This report contains recommendations for which tissues to review during the pathology QA slide review. All neoplasms are reviewed, whether or not they appear to be treatment related. The report includes organs to be reviewed for all lesions and organs to be reviewed for only specified lesions. The report also includes other lesions that are reviewed only when they have been diagnosed by the study pathologist. This last category is done mainly for terminology changes. Terminology changes are made to a study to standardize the terminology based upon the NTP preferred terminology, such as that found in the online NTP Nonneoplastic Lesion Atlas. The final decision for organs and lesions to be reviewed during the pathology QA process will be made by the NTP head of pathology and the NTP pathologist assigned to the study, along with input from other NTP pathologists and NTP scientists involved in the study.

Following the finalization of the PDR report, an audit of pathology specimens (or APSs) is conducted. This is a quality control step performed by pathologists and technicians at an independent pathology contract laboratory. Residual wet tissue is examined to make sure all potential gross lesions have been trimmed in, that all organs have been properly incised or opened, and that animal identifiers (eg, tail tattoos) match the identification on the wet tissue bags and on the necropsy records. Slides and blocks are examined to make sure they are present and accounted for, to check for complete and accurate labeling, and for proper sectioning of blocks and proper preparation of slides. Typically, for chronic studies, wet tissue, slides, and blocks from a random selection of 10% of the animals in each group are examined during the APS, along with any untrimmed potential gross lesion identified during the PDR process from the review of necropsy records.

After the APS, the slide review is done by the QA pathologist, who serves as the primary peer reviewer, and is an independent contract pathologist who has had no interaction with the study pathologist concerning the study. The QA pathologist will review all the organs and lesions identified in the final PDR report. The QA pathologist, like the study pathologist, evaluates the slides in an open (or nonblinded) manner. After the QA pathologist has reviewed all indicated slides, he or she will prepare a QA report. This report includes a brief summary of the original study results, as well as the results of the histologic peer review by the QA pathologist.

Any differences of opinion that occur between the study pathologist and the QA pathologist are shown to the NTP pathologist assigned to the study, who examines the slides and resolves the differences. In cases when a diagnosis cannot be agreed upon, the slide is set aside to be taken to the Pathology Working Group (PWG) for a resolution. After the NTP pathologist has addressed all the differences of opinion, the QA pathologist and the NTP pathologist decide which slides to take to the PWG.

The purpose of the PWG is to confirm the diagnoses of potential treatment-related effects, to resolve any discrepancies, and to harmonize diagnostic nomenclature. Representative examples of all treatment-related lesions are taken to the PWG, along with any unresolved differences and unusual and uncommon lesions. The PWG typically consists of 6 to 13 toxicologic pathologists and includes the study pathologist (from the study laboratory), the QA pathologist, the NTP pathologist assigned to the study, and pathologists with expertise in the potential target organ. The study pathologist can participate remotely with access to whole slide images. At this time, whole slide images have not been approved for NTP PWGs; therefore, remote participants do not vote at the PWG. The final diagnoses in NTP studies, therefore, represent a consensus between multiple, independent pathologists: the study pathologist, the QA pathologist, the NTP pathologist, and the PWG participants.

In most instances, the QA pathologist also serves as the PWG coordinator. However, the NTP may request that a third independent review be held by a pathologist from a different pathology contract laboratory; in this case, this pathologist would review the findings of the study pathologist and the QA pathologist, resolve differences between the 2, and serve as the PWG coordinator.

Highly Trained Pathologists

The NTP’s unparalleled and robust peer review process is conducted by multiple, highly trained pathologists. This helps ensure the integrity, objectivity, reproducibility, and consistency of histopathology data in NTP studies. Histopathological evaluation in NTP studies is conducted by toxicologic pathologists, who, in general, have received years of training, first as veterinarians, then in residency in veterinary pathology and some in a PhD program, and then, for the majority, by validation of professional standards through achievement of board certification. Additionally, some are recognized as Fellows of the International Academy of Toxicologic Pathologists. The NTP takes steps to assure that none of the pathologists involved in NTP studies have any conflicts of interest with regard to the agent being studied, its manufacturer(s) or any group that supports or opposes its production or use. The multistep peer review process involving multiple, highly trained pathologists, with no conflicts of interest, constitutes a thorough quality control process that ensures the integrity, objectivity, reproducibility, and consistency of histopathology data in NTP studies, making routine blinded evaluations unnecessary.

Consistency in Terminology

Animal study reproducibility is an issue of growing concern in biomedical research. ¹ The NTP mitigates observational bias through an extensive and rigorous peer review process as described above. The reproducibility of histopathology data is assured through the use of defined diagnostic criteria and international and published standards of terminology. The study pathologists, who perform the initial read of all slides in a study; peer review pathologists; and NTP pathologists rely on their extensive training and experience, as well as on published standards for diagnostic criteria and terminology (eg, INHAND [International Harmonization of Nomenclature and Diagnostic Criteria] guidelines for each organ system, standard textbooks for neoplasms, or the online NTP Nonneoplastic Lesion Atlas). ^{2 –4} Standard terminology allows the NTP to make better comparisons between studies read by different pathologists or at different contract laboratories and allows for uniformity of terminology during the comparison of multiple studies, such as for risk assessment/regulatory purposes.

Use of Ancillary Data

In NTP studies, ancillary data, such as organ weights and clinical pathology, serve as correlative end points to histopathology. These ancillary data, which are measured by objective or automated testing methods, provide objective qualification of histopathology data. For example, an observation of diffuse hepatocellular hypertrophy may be supported by an increase in liver weight, renal tubular injury, or glomerular changes might be supported by an increase in urine protein, and cardiomyocyte necrosis might be supported by increased serum troponins. For accuracy of the histopathological findings in a study, it is imperative that the pathologists have access to all information on the study animals, including clinical observations, gross lesions, organ weights, and dose group, when evaluating a study.

Masked Analysis for Histopathology

Masked (or “blinded”) analysis can have a negative effect on the quality of the information obtained from a study and the amount of time required to complete the study. The appropriate interpretation of tissue changes is complex and subjective, requiring the appropriate use of control animals. A potential source of bias in the practice of histopathology is the use of concurrent controls in a study to determine a “baseline” for lesions, especially when changes are subtle (in fact, whether or not the histopathological analysis is informed or not has no effect on the identification of obvious treatment effects). The pathologist compares tissues from treated animals to the concurrent controls in order to separate treatment-related changes from normal biological and even pathobiological variation in our animal models. The extensive peer review process for NTP studies also serves as a check to ensure the reproducibility of data in determining changes beyond normal background variability. Masked evaluation would make the task of separating treatment-related changes from normal variation more difficult. With masked analysis, the pathologist can either record lesions that are clearly outside his or her perception of normal, or record every change with no threshold; both of these approaches decrease the sensitivity of the study to identify potential human health hazards. This decrease in sensitivity could require substantially more animals in order to detect a treatment effect.

Blinding a study would also increase the cost of a study, through increased time and effort, and would increase the likelihood of mistakes occurring during coding and uncoding of the slides. Although cost should not be the defining characteristic of a scientific evaluation to determine public health, it is the stark reality of using public funds that they are limited and must be used in a judicious manner. The data produced by the NTP are sound. It is scrutinized and evaluated by members of the scientific community and is publicly available.

For these reasons, reading a study in an open manner is considered preferable by the NTP. However, in certain situations, masked analysis of a subset of slides is appropriate. When subtle lesions have been identified, or when the severity of a background lesion is slightly increased compared to controls, it is common practice to do a blinded evaluation of a particular lesion or selected slides. Blinding can be used appropriately in reevaluation to verify lesions, ensure uniformity of severity scores, and to aid in determining a no-observable-effect level in specific instances after the pathologist has initially evaluated the slides. This is commonly practiced informally by study and reviewing pathologists in NTP studies; it is sometimes undertaken as a formal part of the pathology review. At the NTP, PWGs are typically conducted in a blinded fashion, where the dose/exposure groups are not known and/or the slide labels are covered. However, in some situations, for specific lesions, blinding the slides is not done.

National Toxicology Program Position on Informed Analysis for Histopathology

In summary, the NTP’s position is that blinded histopathological evaluation is not required in its toxicity or carcinogenicity studies and is, in fact, unnecessary and inappropriate in the initial evaluation of its studies and in the initial stages of the peer review process. Informed analysis is the standard practice for the NTP, as well as the toxicologic pathology industry as a whole, and is in accordance with the recommendations of the Society of Toxicologic Pathology and the American College of Veterinary Pathologists. ^{5 –10} The NTP’s peer review process provides a thorough review of the pathology data by highly trained pathologists using consistent terminology and criteria, and the final diagnoses in NTP studies represent a consensus between multiple, independent pathologists, providing the highest confidence in the NTP’s histopathology data and pathology review process. There is no evidence that the current approaches of informed histopathological evaluation in NTP studies have missed identifying potential hazards of significant public health concern. Instead, current NTP pathology approaches have provided a means for characterizing putative hazards/risks for the public by providing the highest quality data.