Regulatory Forum Opinion Piece*

Background

Histopathology is one of the more subjective diagnostic tools used in human and veterinary medicine, and also one where levels of skill and experience can vary widely from one practitioner to another. This means that when this tool is transposed from its traditional application in disease diagnosis for individual patients to toxicology studies for safety evaluation of a drug or chemical for large populations, additional safeguards are warranted to ensure accuracy of the data and appropriate interpretation in the context of the study data as a whole. Although not a general regulatory requirement, in toxicology studies this is achieved through peer review. Typically, a second pathologist reviews the diagnoses and interpretations of the original study pathologist, with a consensus opinion being expressed in the pathology report. This minimizes the inherent subjectivity in diagnosis, and ensures that at least two practitioners concur on which of these agreed diagnoses are likely effects of treatment.

In the context of pharmaceutical toxicology studies, primary histopathology evaluation is usually done by a pharmaceutical company (sponsor) pathologist or by a contract research organization (CRO) pathologist; peer review is most commonly performed by a sponsor company pathologist (although it may be done by another CRO pathologist). While the expectation is, of course, that the peer reviewer influences the final data and interpretation, regulators have expressed concern that there may be “undue influence” or “bias” from sponsors, such that retention of the pre–peer review data might be desirable. This issue formed the basis of a meeting between representatives of Pharmaceutical Research and Manufacturers of America (PhRMA) and Food and Drug Administration (FDA) in 2012, and also of a panel discussion among industry pathologists and regulators in Japan in 2009 (JSTP 2009).

The proposed solution to these concerns is that the peer review process is made more transparent, as suggested recently in this journal (Englehardt et al. 2013) and directly by regulators within the last few months in a new draft OECD regulatory guidance that states:“ … there is an expectation that the initial review of the pathology data will have been completed before the peer review takes place and if changes are required, these are clearly documented and explained in the final report.”

Although documentation of the peer review process would indeed provide some additional transparency to data changes made following peer review, it is my opinion that need for this additional documentation has not been adequately demonstrated, the benefits are doubtful, and it also has significant drawbacks that could have a net negative impact on the quality of data provided for human risk assessment.

Two sets of pathology data: pre– and post–peer review—how would this be useful in determining undue sponsor influence?

Those not directly involved in peer review may be surprised at how often and how significantly microscopic data can change following peer review. This is similar to other subjective medical screening specialties such as radiology, where numerous studies show that adjudication of radiographs results in significant changes to diagnoses in 20 to 50% of cases (Robinson 1997; Renfrew 1992; Lorentz et al. 1999). The fact that pathology peer reviewers are frequently more experienced than primary pathologists (as recommended by the Society for Toxicologic Pathology; Morton et al. 2010) means that changes as a result of pathology peer review are particularly likely. The less experienced pathologist is more likely to overdiagnose or overinterpret since their perception of the “normal range,” and the repertoire of spontaneous changes in the various species is more limited. In contrast, incidental findings in toxicology studies are more likely to be recognized as such by an experienced pathologist. How regulators, who may not have any pathology expertise themselves, would distinguish these types of discrepancies from genuine “undue influence” is unclear. Many diagnostic changes relate to subtleties of morphology that are not readily documented as a written “reason for change” in an audit trail. In many cases, without submitting the slides for review by a third pathologist, regulators would not be able to determine whether diagnostic changes requested by the sponsor peer reviewer were justified or not. To commission such a review would be time-consuming, expensive, and there would be no guarantee that the opinion of the third pathologist would be more valid than that of either of the previous two. The cost and delay of using larger, formal pathology working groups (PWGs) to resolve these uncertainties would be an option only in rare cases.

A recent example illustrates well an instance where changes to the data may have suggested undue sponsor influence to regulators but review by independent specialists showed that this was not the case. Discrepancies between the CRO pathologist data and final pathology data after sponsor peer review (reclassification of malignant mammary adenocarcinoma to benign fibroadenoma) were cited as grounds for negatively influencing the FDA safety assessment of lorcaserin (FDA 2010). However, subsequent reevaluation of the tumors by an independent PWG confirmed that the original reclassification on sponsor peer review was justified, and if anything had been overconservative, since the PWG made further diagnostic adjustments that downgraded additional tumors from adenocarcinoma to fibroadenoma (Arena Pharmaceuticals 2011).

Evidence of undue sponsor influence via pathology peer review has not been presented

Changes to regulatory requirements that increase cost, time, and have other potential drawbacks should be driven by good evidence that is publicly available. This would allow sponsors both to understand the problem and address the specific concerns immediately on a voluntary basis. In the longer term, it would enable them to provide informed comment on any proposed regulatory changes to address the problem. As illustrated in the lorcaserin example above, peer review often changes data in ways that may appear to serve the interests of the sponsor for valid reasons related to the morphology presented on the slide, so it is important that additional evidence is presented to define regulator concerns before measures are taken to address them.

Potential availability of pre–peer review data to other parties and interpretation by nonexperts

The existence of a second set of pre–peer review pathology “data” in toxicology studies could make it potentially subject to legal Discovery and/or Freedom of Information requests. Having this information in the public domain would lead to challenges from unqualified parties regarding regulatory decisions and pharmaceutical safety generally, reducing public confidence in sponsors and/or regulators. There is often disagreement on medical diagnoses in clinical practice, but a consensus decision is made and treatment proceeds based on this; making professional disagreements available for adjudication by nonspecialists is not considered good medical practice and would be similarly inappropriate for pathology data from toxicology studies.

A sponsor’s interest is in obtaining accurate pathology data, not minimizing findings

It is most definitely not in a sponsor’s long-term interest to use peer review to minimize any real drug findings, since these are much more difficult and costly to address at a later stage of drug development when they appear either in later animal studies or (worst case) in the clinic. Even in the relatively early stages of drug development, there is a high opportunity cost to companies of prolonging the life of compounds with significant toxicology liabilities. Sponsors are well aware of this and most definitely would not welcome any underreporting of anatomic pathology findings. This is demonstrated by the fact that peer review (whether of internal or CRO studies) often adds pathology findings that were not diagnosed in the initial evaluation.

Formal documentation of differences between the primary pathologist and peer reviewer may increase entrenchment of opinion and make consensus harder to reach

The 1:1 nature of peer review, with just a final consensus being published, allows for private professional discussion and creates an optimum environment for arriving at a balanced decision that is driven by the evidence and not unduly influenced by the original positions of the negotiating parties.

Peer review is only one aspect of all kinds of influences on a study pathologist’s final diagnosis which currently go undocumented

Most pathologists take advice on diagnosis or interpretation from one another both within and external to their organizations. This increases the quality of their work and the general experience level of the group. Lesion sharing and discussion is becoming even more widespread now that slides can be scanned and examined digitally by any number of people at remote locations. The pathologist then weighs the opinions that he or she receives and makes a decision based on what he or she considers to be the most convincing evidence. In most cases, this happens prior to a formal peer review and can have a greater influence on a key diagnosis than the peer review itself. Attempting to document this type of interaction (which may well include sponsor pathologists, including the peer review pathologist) would be impractical and kill the process of professional debate and interaction which is ultimately an extremely beneficial influence on the quality of the data. Treating the formal peer review differently makes an artificial distinction and in certain circumstances (such as studies where both primary pathologist and peer reviewer belong to the same company or CRO) may simply increase pre–peer review informal interactions, making the official peer review process a formality.

The susceptibility of pathologists to “undue influence” is probably overestimated

Study pathologists are veterinarians, trained for many years in independent decision making and taking responsibility for clinically critical decisions. After qualifying as veterinarians, they typically have a further 3 to 5 years of residency training prior to board certification. At this stage, they are experienced professionals who (a) have confidence in their opinions and are unlikely to retract what they believe to be scientifically justified as a result of “undue influence” and (b) take pride in their personal record of accuracy. Since they sign the final consensus report as the primary pathologist, it is they who are responsible for any errors revealed upon subsequent animal studies or in the clinic. It is therefore very doubtful that they would allow a sponsor’s wishes to prevail over their professional judgment to any scientifically meaningful extent. Likewise, it is in their interests to accept sponsor peer review advice where they believe it is justified.

In summary, preserving the primary pathologist’s initial report and data generated prior to peer review is most unlikely to shed any light on “undue influence” by sponsors but will very likely show differences in data that will leave regulators with no clear path forward for determining whether the reasons given for those differences are valid. It would potentially distort the process of peer review, lengthen study reporting times, and undermine public confidence in regulatory decisions. In order to justify such a step, there needs to be a clear demonstration that such undue influence was indeed a problem that resulted in poorer quality pathology data being submitted to regulators. Such a case has yet to be made.