Abstract
The Society of Toxicologic Pathology (STP) explored current institutional practices for selecting between non-blinded versus blinded histopathologic evaluation during Good Laboratory Practice (GLP)-compliant, regulatory-type animal toxicity studies using a multi-question survey and STP-wide discussion (held at the 2019 STP annual meeting). Survey responses were received from 107 individuals representing 83 institutions that collectively employ 589 toxicologic pathologists. Most responses came from industry (N = 46, mainly biopharmaceutical or contract research organizations) and consultants (N = 24). For GLP-compliant animal toxicity studies, histopathologic evaluation usually involves initial (primary) non-blinded analysis, with post hoc informal blinded re-examination at the study pathologist’s discretion to confirm subtle findings or establish thresholds. Initial blinded histopathologic evaluation sometimes is chosen by study pathologists to test formal hypotheses and/or by sponsors to address non-pathologist expectations about histopathology data objectivity. Current practice is that a blinded histopathologic evaluation is documented only if formal blinding (ie, using slides with coded labels) is employed, using simple statements without detailed methodology in the study protocol (or an amendment) and/or pathology report. Blinding is not an appropriate strategy for the initial histopathologic evaluation performed during pathology peer reviews of GLP-compliant animal toxicity studies.
This is an opinion article submitted to the Toxicologic Pathology Forum. It represents the views of the author(s). It does not constitute an official position of the Society of Toxicologic Pathology (STP), British Society of Toxicological Pathology (BSTP), or European Society of Toxicologic Pathology (ESTP), and the views expressed might not reflect the best practices recommended by these Societies. This article should not be construed to represent the policies, positions or opinions of their respective organizations, employers, or regulatory agencies.
Keywords
Introduction
The larger scientific community has debated the merits and objections between non-blinded and blinded histopathologic evaluation of tissue sections in animal toxicity studies for decades with respect to the ability of these opposing strategies to yield high-quality data sets that most accurately reflect the “true” response to an experimental manipulation. 1 -7 Since the inception of this debate, the Society of Toxicologic Pathology (STP) has stated unequivocally that nonblinded histopathologic evaluation is the appropriate strategy for animal toxicity studies by both editorial policy of the Society’s journal (note 2), Toxicologic Pathology, 8 -10 and communication as a secondary consideration in multiple STP-sponsored papers regarding toxicologic pathology “best practice” recommendations. 11 -14 A similar stance has been advocated by the American College of Veterinary Pathologists. 15,16 The rationale for use of non-blinded histopathologic evaluation in the toxicologic pathology setting, where the principal purpose of Good Laboratory Practice (GLP)–compliant toxicity studies is hazard identification/safety assessment of novel entities of whatever modality, is that regulatory-type animal toxicity studies (both GLP-compliant and non-GLP-compliant) are expected to be as sensitive as possible in identifying possible findings indicative of potential risk.
Nonetheless, non-pathologists in other scientific disciplines sometimes raise objections over individual and institutional decisions leading to initial histopathologic evaluation of tissue sections from animal toxicity studies utilizing a non-blinded evaluation strategy rather than a blinded approach. Common instances in which blinded histopathologic evaluation is requested by non-pathologists include hypothesis testing, characterization of new animal models and carcinogenicity testing. 3,7 The rationale for this contention is that blinded evaluation limits the bias that is perceived to be inherent in assigning histopathologic diagnoses, which by definition depend to some degree on the individual experience of the pathologist. 4
The divergence between these 2 perspectives has resulted in a recent decision by the STP to formulate “best practice” recommendations for the appropriate use of non-blinded versus blinded histopathologic evaluation for animal toxicity studies. For this purpose, the STP’s Scientific and Regulatory Policy Committee (SRPC) established a Working Group to expound the scientific rationale for selecting non-blinded evaluation versus blinded evaluation as well as to devise suitable procedures for effectively communicating this rationale to other scientists and the general public.
As a first step to accomplishing this STP-sponsored mission, the Working Group solicited feedback from STP members during the second quarter of 2019 regarding their institutions’ existing approaches to selecting and implementing non-blinded versus blinded histopathologic evaluation of tissue sections from GLP-compliant animal toxicity studies. Our opinion piece presents the survey data obtained from this STP-wide review. Since the interpretation of these data represents an opinion and not a formal STP position or policy, primary oversight for this editorial opinion piece through the peer review and publication processes was shifted from SRPC stewardship to the STP’s Toxicologic Pathology Forum, which serves as the STP’s gatekeeper for vetting the appropriateness of opinion manuscripts originating from STP members that are slated for release in Toxicologic Pathology.
Methods
The Working Group sought feedback from STP members through 2 vehicles. The first was a survey regarding extant institutional practices of non-blinded versus blinded histopathologic evaluation for animal toxicity studies. The arrangement and exact wording of the survey questions is provided in the Supplemental Files as Appendix 1. The second format was to hold a public forum at the 2019 STP annual meeting to discuss the possible pros and cons of blinded histopathologic evaluation as applied in industrial hazard assessment/safety assessment settings. Both formats are detailed below.
Content and Format of the STP Members Survey
The survey was assembled by the Working Group, reviewed by the SRPC, and then revised by the Working Group prior to release. The instrument was divided into 3 parts, and was designed to be completed in 15 to 20 minutes to encourage participation.
The first part was to capture demographic data on the responder. This information was used to categorize variations in perspective that might pertain to divergent institutional purposes (eg, hypothesis-driven research vs. GLP-compliant animal toxicity studies) and limit the potential for duplicate responses. This portion of the survey contained 14 questions, 9 regarding the credentials of the responder and 5 evaluating the attributes of the institution. The instructions requested that only one survey be completed for each institution, or for each site for institutions with a global presence but with no coordinated across-site standard operating procedures (SOPs). Answers were recorded using a simple check-box format, so the time anticipated for completing this portion was estimated at 2 to 3 minutes.
The second part of the survey was designed to benchmark current strategies for selecting and implementing non-blinded versus blinded primary (ie, initial) histopathologic evaluation of tissue sections from GLP-compliant animal toxicity studies. This portion consisted of 11 questions and again utilized a check-box format for gathering answers to all but one question that permitted “free text” entry of other relevant comments. The questions examined whether or not and to what extent institutions choose non-blinded versus blinded evaluation, whether or not the choice depends on the type of product being developed, the rationale dictating whether or not a blinded histopathologic evaluation was undertaken, and the methods utilized to perform and document that a blinded evaluation had been conducted. The complexity of the parameters listed as options for each question suggested that this portion of the survey would require 8 to 15 minutes to complete.
The final part of the survey was devised to benchmark non-blinded versus blinded histopathologic evaluation as a strategy for pathology peer review during animal toxicity studies. This fraction contained 5 questions, with the last representing an optional box for “free text” entry. These questions reproduced some of those asked previously with respect to primary histopathologic evaluation, so the anticipated time needed to complete this section was 2 to 5 minutes.
Upon final approval by the SRPC, the survey was formatted by information technology experts at STP headquarters (Reston, Virginia) for release on-line. The software utilized for the project was Survey Monkey (San Mateo, California). The survey was released to STP members in early June 2019 and remained open for a 4-week period. Periodic reminders were given to encourage participation, including 3 online e-mails sent to all STP members who had not previously responded as well as a verbal request delivered during the discussion of this topic at the Town Hall session conducted during the 2019 STP Annual Meeting (see below).
The survey was designed as a descriptive study. Therefore, no formal statistical analysis of the resulting data set was conducted.
Content and Format of the STP Town Hall Meeting
The STP devoted the Town Hall session at its 2019 Annual Meeting to consider the topic “If, When, and How to Undertake Blinded Histopathologic Evaluation.” The incentive for this decision was to obtain additional comments to supplement the formal data set obtained through the online survey. The session was moderated by one Working Group member and coauthor (K.K.) and began with brief (10-minute) introductory talks by 2 other Working Group members and coauthors regarding the potential cons (K.C.) and pros (K.J.) of utilizing blinded histopathologic evaluation to assess tissue sections from animal toxicity studies. These preliminary remarks were followed by an approximately 45-minute discussion among audience members (estimated at 250 attendees) and the speakers. A part of the commentary below gives the major points made during this discussion (collated from handwritten notes recorded during the Town Hall by one coauthor [B.B.]).
Limitations of the Data Set
Several potential shortcomings were recognized by the Working Group in crafting an opinion piece on current non-blinded versus blinded histopathologic evaluation practices as expressed through the online survey and during the face-to-face Town Hall debate. The first consideration, that STP members could be unaware of the survey and Town Hall session as options for providing feedback on this topic, was addressed by substantial advance publicity of both opportunities in multiple venues (e-mails and the Society’s Scope newsletter for both as well as the annual meeting program and intra-session verbal reminders during the meeting for STP members to attend the Town Hall session) as well as intermittent e-mail reminders to complete the questionnaire during the 3-week period for which the online survey was open. Basic concepts regarding non-blinded versus blinded histopathologic evaluation were presented in many fashions, including the introduction to the survey, the details of response options for the survey questions, and the introductory talks that preceded the Town Hall discussion, thereby assuring that responders would have received a suitable overview of the perspectives prior to giving their views. The second possible pitfall, a low response rate among STP members, was not subject to direct intervention by the Working Group except via the publicity campaign promoting awareness of the issue. A third possible drawback is that survey answers provided by responders, who were asked to answer on behalf of their entire institution (or their site for organizations with multiple business locations), might not accurately reflect current practice for today’s toxicologic pathology practitioners. In the Working Group’s view, this prospect is rendered unlikely given that most responders were supervisory pathologists with considerable experience regarding current institutional practices in their respective professional setting (see below). A fourth difficulty is that the data set may be skewed. Particular concerns in this regard were underrepresentation of views from certain practice sectors and the possibility that some responders addressed animal studies in general rather than GLP-compliant animal toxicity studies as intended. Where warranted, the Working Group has attempted to mitigate any such slanting by stratifying the reporting by the different practice sectors (see below). The final consideration is that the data summarized below provide no indication regarding the opinions of those STP members who did not choose to participate in the survey. This absence represents a possible but unavoidable limitation in the results.
Results
The findings are presented separately for the survey and Town Hall debate. The rationale for this format is that the survey data provide quantifiable results of actual institutional practices while the discussion points given for the Town Hall are necessarily individual opinions that have been exposed to further editing for clarity. For the quantitative survey data, the denominator varies among questions since some responders did not provide answers for every query.
Data From the STP Member Survey
The survey was completed by 107 responders collectively representing 83 different institutions employing 589 toxicologic pathologists as of the end of June 2019 (or 46% of the 1280 individuals who were registered as STP members during the time that the survey was open). The discrepancy between the numbers of responders and institutions reflects the receipt of responses from 2 or more sites for 9 institutions.
Demographic information for responders (survey questions 1-7)
The typical responder was a supervisory pathologist with many years (typically 10 or more) of toxicologic pathology experience. This conclusion is reasonable given their position titles (eg, Global or Site Director/Head, Associate Director, Principal/Senior Pathologist, or equivalent). Some responders reported that they had assembled teams of colleagues to develop a group opinion.
Responders were well acquainted with conventional toxicologic pathology practices. A vast majority of responders had undertaken formal biomedical education, supplemented by formal pathology training, and had achieved one or more credentials demonstrating their prowess in the field. Most responders held veterinary medical (N = 99 of 106, 93.4%) or medical (N = 3 of 106, 2.8%) backgrounds. Many also had earned one or more advanced degrees (N = 81 of 106, 76.4%—three-quarters [N = 62] of which were a PhD or equivalent degree) and/or had completed pathology training during a veterinary medical or medical residency or similar practicum (N = 76 of 106, 71.7%). A large majority of responders held 1 or more of the national or multinational professional certifications 17 in either anatomic pathology (N = 86 of 106, 81.1%) and/or toxicologic pathology (N = 24 of 106, 22.6%), and some (N = 19 of 105, 18.1%) also had certifications in toxicology. A large majority (N = 99 of 106, 93.4%) of responders practice as anatomic toxicologic pathologists (ie, in the pathology specialty that is directly responsible for evaluating tissue sections and providing histopathologic diagnoses).
The average years that responders had spent in the field was 20.6 ± 11.1 years (mean ± standard deviation), with a range from 1.5 years to greater than 40 years. A supermajority of responders (N = 86 of 107, 80.4%) had over 10 years of experience, while only 8 (of 106, 7.5%) had 4 or fewer years in toxicologic pathology practice. This latter level of relevant experience is important since 3 to 4 years is the length of time recognized by all global societies of toxicologic pathology (including STP) as being necessary for an entry-level pathologist to achieve sufficient practical experience to be fairly proficient in toxicologic pathology. 17
Taken together, these data indicate that the impressions provided by responders regarding non-blinded versus blinded histopathologic evaluation as practiced in various toxicologic pathology settings accurately represent current thought trends regarding accepted experimental design for this function in animal toxicity studies.
Demographic information for institutions (survey questions 8-14)
The 83 individual institutions covered a broad range of toxicologic pathology practice sectors (Question 10, Table 1). A large majority of responses were received from industry (N = 46 of 83, 55.4%) and private consultants (who typically provide toxicologic pathology expertise to support industry; N = 24 of 83, 28.9%). For industry, most responses (N = 41 of 46, 89.1%) detailed practices in biopharmaceutical companies (N = 29) or contract research organizations (CROs, N = 12), although a few responses were received from agrochemical companies (N = 2 of 46, 4.3%) or medical device companies (N = 3 of 46, 6.5%). As defined by Question 11, companies ranged in size from a single individual to more than 20,000 people worldwide with smaller companies (up to 5,000 total employees) representing a large proportion of the responders (N = 77 of 106, 72.6%) and institutions (64 of 83, 77.1%). With respect to geographic location (Question 13), the bulk of institutions (N = 73 of 83, 88.0%) were located either exclusively in North America or had a global footprint that included a prominent North American presence. Remaining responses were received from institutions sited only in Europe/Russia (N = 9 of 83, 10.8%) or Asia/India (N = 5 of 83, 6.0%). No data were obtained from organizations found only in Africa, Australia/New Zealand, or Central/South America. This geographic distribution is logical given the home locations of most STP members and the global distribution of institutions that utilize toxicologic pathology in their product discovery and development activities.
Institutional Affiliations of Toxicologic Pathologist Responders.
As recorded under Question 8, feedback was received from many sites of 2 multinational CROs (8 and 6 locations, respectively), from 3 sites of a third North American CRO, and from 2 sites each for 2 other multinational CROs and 6 multinational biopharmaceutical companies. All these institutions operate several sites in Europe and/or North America. However, responses from these entities generally appeared to be from discrete reporting units (eg, either an entire site or one of several independent pathology groups at a site), with seemingly little overlap in reporting among the responses from a given institution. Accordingly, the 107 survey responses were treated as independent data streams for the purpose of compiling the results given below.
The majority of responses (N = 52 of 83, 62.7%) were provided by institutions that employ multiple toxicologic pathologists (N = 589 total individuals), but in only 4 cases (for 83 individual institutions, 4.8%) did responses include direct affirmation by the responder that multiple toxicologic pathologists participated together in crafting an institution’s answers to this survey (numerical and free-text entries for Question 12). Thus, the number of individuals who actually contributed to building the current data set was estimated to be approximately 167 (ie, 107 responders plus 60 additional colleagues who helped formulate the institutional response).
Taken together, these demographic characteristics attest that the current data set provides a representative snapshot of current practices used in choosing, implementing, and documenting whether or not a blinded histopathologic evaluation is elected for an animal toxicity study.
Non-blinded versus blinded histopathologic evaluation as a primary strategy (survey questions 15-26)
Data for Question 15 show that institutions select non-blinded over blinded histopathologic evaluation as the strategy for primary (initial) tissue assessment in GLP-compliant animal toxicity studies to address particular experimental objectives. The most common practice is to perform an initial non-blinded examination to identify potential target organs, followed if warranted with a post hoc blinded re-evaluation of target organs only (N = 50 of 107, 46.7%). Sizeable fractions also indicate that their institutions sometimes perform an initial blinded evaluation to address a specific study aim (N = 29 of 107, 27.1%) or that they never perform an initial blinded evaluation (N = 28 of 107, 26.2%). No institution mandates that all animal toxicity studies be conducted using blinded histopathologic evaluation as the default initial approach.
To further explore the use of non-blinded vs. blinded histopathologic evaluation during animal toxicity studies (Table 2), institutions were asked to provide the percentages of animal studies for which a blinded strategy was used by the study pathologist as a predetermined approach for the initial assessment (Question 16) or as a post hoc method for confirming target organs (Question 17). A large majority of the responders (N = 65 of 79, 82.3%) indicated that their institutions never or seldom (≤ 5% of studies) used a blinded approach for the initial histopathologic evaluation. That said, some institutions utilized an initial blinded examination strategy in a substantial minority (up to half, N = 10 of 79, 12.7%) or the majority (N = 4 of 79, 5.1%) of their work. Further data mining indicated that the institutions prone to performing the initial histopathologic evaluation using a blinded strategy mainly were academic and government research facilities or individual consultants engaged in hypothesis-driven research and therefore less likely to be involved regularly with GLP-compliant animal toxicity studies. In contrast, most responders (N = 66 of 68, 97.1%) stated that their institutions used a blinded approach as a post hoc method to confirm target organs in at least some studies, and half (N = 37 of 68, 54.4%) noted that this strategy was used for approximately one-quarter or more of their studies. The modality of the product being evaluated for toxicity (biomolecule, chemical, device, gene therapy, etc) did not impact the decision regarding whether or not to undertake a non-blinded vs. blinded histopathologic evaluation for the primary (initial) assessment or the post hoc re-evaluation to confirm target organs (Question 18).
Use of Blinded Histopathologic Evaluation by Study Pathologists Performing Animal Toxicity Studies.
a Survey questions asked for input regarding Good Laboratory Practice (GLP)–compliant studies.
For Question 19, responders were asked to prioritize potential reasons used in electing to perform a blinded histopathologic evaluation as either an initial assessment or as a post hoc re-evaluation. The data indicate that, of the 8 options offered, several reasons were accorded greater weight by toxicologic pathologists when making a decision to employ a blinded analysis (Table 3). The highest-ranked choice as a whole (calculated as the total number of responses assigning it as the first, second, or third consideration) was to check the incidence and/or severity of a finding in treated animals compared to the background incidence in control animals. This answer was reported by 59 (of 79, 74.7%) as one of the key “top 3” reasons, and it was the single most important parameter given as the first reason for choosing a blinded evaluation (N = 36 of 79, 45.6%). Three other reasons factored strongly in such decisions as indicated by their cumulative rankings of approximately 50%: to minimize observational bias when testing a hypothesis; to define an informative inflection point (eg, “no observed effect level” [NOEL], “no observed adverse effect level” [NOAEL], or comparable threshold); and to confirm potential findings during a post hoc evaluation after an initial non-blinded assessment. Blinding to meet the expectations of non-pathologists with respect to limiting potential diagnostic bias was the next choice across all practice sectors, exhibiting some importance in approximately one-fifth of decisions (N = 17 of 79, 21.5%), while generating unbiased data for statistical analysis typically was not considered to be a critical element in such decisions (N = 4 of 79, 5.1%). Together, these data indicate that the toxicologic pathology community values the sensitivity of non-blinded histopathologic evaluation in detecting and characterizing possible effects of test articles to a greater degree than the truly rigorous objectivity offered by blinded evaluation ab initio followed by statistical analysis as the traditional strategy undertaken in hypothesis-driven experiments.
Principal Reasons for Electing to Perform a Blinded Histopathologic Evaluation for Animal Toxicity Studies.
a The survey question asked responders to rank-order their responses
Responses to Question 20 clearly indicated that toxicologic pathologists regularly utilize informal blinding and are asked to employ formal blinding less frequently (Table 4). Indeed, the current data show that informal blinding (ie, where the pathologist inverts slides to hide their labels and then shuffles them to create a random order of assessment) was the first choice by 59 (of 79, 74.7%) of responders across all practice sectors and ranked more highly as a whole (ie, ranked as the first, second, or third option; N = 72 of 79, 91.1%) than did a strategy involving formal blinding (ie, where the pathologist performs the evaluation using slides with coded labels). Formal blinding is employed in all practice sectors, but relative to the informal approach is used much less often for initial histopathologic evaluation (N = 11 of 79, 13.9%) and post hoc re-evaluation that follows a prior non-blinded analysis (N = 6 of 79, 7.6%).
Common Approaches in Performing a Blinded Histopathologic Evaluation for Animal Toxicity Studies.
a The survey question asked responders to rank-order their responses.
Question 21 probed the extent of data withholding experienced by toxicologic pathologists engaged in performing blinded histopathologic evaluation of tissue sections. Responders were asked to rank their top 3 (of 5) options based on particular categories of information that might be denied to the study pathologist who will perform a blinded assessment (Table 5). Less than half of responders (N = 52) answered this question. Of those who did, a majority worked in settings where the test article characteristics (modality, known mechanisms of action and major activities of the test article class, etc) were provided to the study pathologist, either without (N = 31 of 52, 59.6%) or with (N = 24 of 52, 46.2%) the identities of the control and high-dose groups and known target organs needed to frame the range of normal and test article-related findings. A number of responders indicated that they worked in a venue where all information was withheld from the study pathologist (“blind to all”) in order to assure a randomized (completely blinded) histopathologic evaluation; this option was the most highly ranked response for just under one-quarter (N = 12 of 52, 23.1%) of responders drawn from across all practice sectors (especially academia, which often do not conduct GLP-compliant animal toxicity studies). Based on several free-entry comments for this part of the survey, some of the many nonresponders (N = 55 of 107, 51.4%) chose not to attempt responding to this question because 2 other options—Known Test Article Characteristics, Study Design, and All Dose Groups (ie, information available in standard Study Protocols for GLP-compliant animal toxicity studies) for a blinded initial analysis and Known Test Article Characteristics, Study Design, and Target Organs for the typical post hoc informal re-evaluation—that better reflected their institutions’ actual approach to blinded histopathologic evaluations were not available as potential answers.
Data Withheld From Study Pathologists Tasked With Performing a Blinded Histopathologic Evaluation.
a The survey question asked responders to rank-order their responses.
For Question 22, responders were asked to enumerate which categories of tissue findings are typically subjected to blinded histopathologic evaluation. The vast majority of responders stated that the most compelling reason for a blinded evaluation was to confirm potential target organs identified during the current study during an initial nonblinded assessment (Table 6). This option was the first choice for a supermajority of responders (N = 60 of 76, 78.9%) and was in the top 3 factors for nearly all individuals (N = 71 of 76, 93.4%). The parameter listed as next in importance, evaluation of known target organs (as defined for the specific test article in the current and/or a prior study), was the second choice of many responders (N = 32 of 76, 42.1%) and was one of the top 3 reasons for more than half (N = 43 of 76, 56.6%). A few individuals noted that their institution performed a blinded evaluation of all organs and tissues. Further delving again showed that institutions using a complete blinding strategy chiefly were academic and government research facilities or individual consultants engaged mainly in hypothesis-driven research, not GLP-compliant animal toxicity studies.
Common Findings Subjected to Blinded Histopathologic Evaluation for Animal Toxicity Studies.
a The survey question asked responders to rank-order their responses.
The responsibility regarding whether or not a blinded histopathologic evaluation is warranted (Question 23) may rest with several individuals. This decision is dictated by the study pathologist (first option) or study director (second option) most often rather than being dictated by the institutional management or SOPs (Table 7). Informal discussions with some responders after the survey closed suggest that a study pathologist is fully empowered to make the choice and implement a post hoc blinded re-evaluation to characterize potential target organs, while the study director (ideally in consultation with the study pathologist) is instrumental in making such decisions when a blinded histopathologic assessment is built into a study protocol.
Responsibility for Choosing to Perform a Blinded Histopathologic Evaluation for Animal Toxicity Studies.
a The survey question asked responders to rank-order their responses.
Question 24 assessed 2 related elements: what documentation should be utilized to communicate that a blinded histopathologic evaluation has been undertaken for an animal toxicity study, and when is such documentation necessary. Responders were asked to rank the top 3 (of 6) options regarding how these 2 issues might be addressed (Table 8). More responders indicated their most highly ranked response was not to document (N = 25 of 75, 33.3%) or, if it was documented, to do so only if a formal blinded evaluation of slides with coded labels was done (N = 23 of 75, 30.7%). Three other choices also received higher and comparable total scores, showing that responders usually include them among their top-3 options: in the original study protocol, in a subsequent protocol amendment, or as a statement within the pathology sub-report. Of these, inclusion in the study protocol or pathology sub-report garnered more votes as a first-response option.
Preferred Methods for Documenting a Blinded Histopathologic Evaluation During an Animal Toxicity Study.
a The survey question asked responders to rank-order their responses.
The follow-on Question 25 explored where within a pathology sub-report represented the best places for documenting that a blinded histopathologic evaluation was undertaken. Data on this topic were considered to be of particular relevance to toxicologic pathologists since they, as authors of the pathology sub-report, can designate where they desire to house such statements. Again, responders were asked to rank the top 3 (of 6) options regarding how this message might be communicated (Table 9). In lockstep with Question 24, by far the most common first choice (N = 44 of 74, 59.5%) was not to document that any blinded evaluation has been conducted; this option was selected 3-fold more often than any of the options for providing documentation. Where a blinded evaluation was incorporated in the pathology sub-report, most responders preferred to include a simple declarative statement that such an analysis was done, with no other methodological details.
Means for Documenting a Blinded Histopathologic Evaluation Within a Pathology Sub-Report.
a The survey question asked responders to rank-order their responses.
b “Formal” connotes a blinded evaluation involving slides with coded labels.
c “Informal” means that the study pathologist inverted (or otherwise obscured) the uncoded labels of slides.
Non-blinded versus blinded evaluation as a pathology peer review strategy (Survey Questions 27-31)
Similar to the equivalent query for primary (initial) slide evaluation, institutions may perform non-blinded over blinded histopathologic evaluation as the strategy for initial tissue assessment in pathology peer reviews of GLP-compliant animal toxicity studies to suit particular experimental objectives (Question 27). Just over half of responses (N = 57 of 106, 53.8%) indicated that pathology peer review typically involves a non-blinded histopathologic evaluation. However, a quarter of responders drawn from across all practice sectors noted that their organizations occasionally will conduct a pathology peer review using a blinded strategy for histopathologic evaluation (N = 27 of 106, 25.5%). In such instances, the decision regarding whether or not to perform a blinded pathology peer review appears to depend on the peer review objective (N = 20 of 106, 18.9%). Only 2 (of 106, 1.9%) practitioners, one academician and one consultant, indicated that they always used blinding as the approach for organizing the pathology peer review.
Question 28 assessed the kinds of studies that might be subjected to a blinded pathology peer review. The query was answered by only 10 (of 107, 9.3%) individuals, so trends typical of the toxicologic pathology community as a whole could not be determined. The available responses showed that no particular study type (eg, acute toxicity, chronic toxicity, carcinogenicity) was more likely to be performed using a blinded pathology peer review.
Questions 29 and 30 investigated how many of a hypothetical 100 pathology peer reviews would be done using a blinded histopathologic evaluation for the initial review (Question 29) or at the pathologist’s discretion as a post hoc tool (Question 30). Both queries were completed by a minority of the 107 responders (N = 19, 17.8% for Question 29; N = 20, 18.7% for Question 30). The few responses showed that pathology peer reviews are seldom (<5%) performed by blinded histopathologic evaluation as the initial review strategy, but post hoc blinded analysis is done fairly often (up to 20% of studies) at the discretion of the peer review pathologist to confirm potential target organs.
Free-text entries for the pathology peer review questions (N = 16) revealed a near-consensus that pathology peer review should be non-blinded since the first goal of this practice is to provide a diagnostic cross-check for findings (and severity grades) reported by the study pathologist. A large minority of responses stated that the peer review pathologist has the ability to perform a blinded post hoc re-evaluation if warranted and that such blinded assessments typically are undertaken only for potential target organs or to discriminate subtle variants of contentious findings.
Key town hall talking points on use of non-blinded versus blinded evaluation
The following comments briefly recapitulate the most common threads raised during the Town Hall discussion. The statements have been paraphrased to maintain the grammatical flow, but the conceptual content of discussants’ remarks has not been altered.
In a theoretical sense, a blinded evaluation per se does not guarantee the accuracy of histopathologic diagnoses, nor should bias based on one’s professional/scientific experience be censured a priori as inherently unacceptable. Toxicologic pathologists fuse their version of a common (“objective”) knowledge base—the comprehensive biomedical education—with their unique blueprint of training and practical experience in comparative pathology in generating their diagnoses; therefore, the histopathologic diagnostic process by definition involves some degree of opinion. Blinded histopathologic evaluation cannot eradicate the individual-specific roots responsible for one’s intrinsic assumptions, nor can blinding magically increase the objectivity of histopathologic diagnosis. Furthermore, blinding runs counter to recognized principles of medical/veterinary practice and diagnostic pathology practice where information regarding the clinical history and macroscopic observations (at minimum) is essential when diagnosing anatomic changes seen at the cell and tissue levels. Thus, blinded histopathologic evaluation reduces the sensitivity with which a study pathologist (with or without the assistance of a pathology peer reviewer) identifies subtle or unusual changes that might have significance in predicting an adverse biological response in humans. In addition, blinding removes the ability to utilize historical control data to help define diagnostic criteria for discriminating test article-related and background changes. Access to such historical control data is essential for non-rodent toxicity studies with their small numbers of concurrent control animals.
The most important practical point to toxicologic pathologists is that possession of available background information is indispensable in formulating an accurate histopathologic diagnosis. All toxicologic pathologists inherently possess their own notions based on each individual’s unique background, and proficient microscopic diagnosis and interpretation depends profoundly on having obtained appropriate and sufficient training, qualifications, and experience. Moreover, foreknowledge in the course of a non-blinded histopathologic evaluation is not the same as prejudgment in favor of a preferred outcome. Finally, interpretation of a pathology data set is determined not by the histopathologic diagnoses alone but rather by assessing all qualitative or semi-quantitative microscopic diagnoses (eg, lesion incidences and severity grades) in combination with multiple closely-related pathology end points such as gross lesion dimensions and distributions, organ weights, and clinical pathology values (hematology, serum chemistry, urinalysis, etc). This integrative/correlative approach permits pathologists who might deliver slightly different histopathologic diagnoses based on their individual backgrounds to nonetheless reach a substantively comparable conclusion based on an entire pathology data set.
Several remarks acknowledged that blinded histopathologic evaluation has a place in the toxicologic pathology setting, depending on the objective of the study. Multiple discussants noted that they employ informal blinding routinely as a post hoc device to confirm potential target organs, define a NOAEL or NOEL value, or compare the spectrum of findings in study animals to those in the known range of species- and strain/breed-specific background findings. Several also acknowledged that blinding as the histopathology strategy for the initial evaluation is appropriate when the experiment is designed to test a hypothesis. The take-home message was that toxicologic pathologists support non-blinded histopathologic evaluation for the vast majority of toxicity studies since their purpose is not to prove a particular hypothesis (such as “Is a drug efficacious?”) but rather to identify and characterize potential, often unknown hazards (“Is a drug safe?”) with the utmost sensitivity. The obvious corollary is that the correct design for an animal toxicity study need not, and in fact should not, employ the blind or double-blind designs used for human clinical trials. The exception are animal studies performed for effectiveness (efficacy) under the US FDA Animal Rule, 18 which are designed like clinical trials and included a recommendation for blinded histopathologic evaluation because such studies are undertaken in place of human trials that cannot be done in humans for ethical reasons. Therefore, the consensus among toxicologic pathologists at the Town Hall meeting was that non-blinded histopathologic evaluation is the best way to maximize sensitivity in detecting test article–related effects in animal toxicity studies.
In a practical sense, blinded histopathologic evaluation would be severely limiting if required as a routine practice in toxicologic pathology. Blinded histopathologic examination as a standard practice will heighten the administrative burden (and also cost and time) needed to generate a final set of histopathologic diagnoses, will increase the subjectivity of the data set by removing the ability to use changes in concurrent control animals to refine the diagnoses for their treated counterparts, and thus will reduce the sensitivity for detecting undesirable test article-related effects. 4,19 The experience of discussants was that blinding impacted all these factors by reducing sensitivity (low quality) and lengthening the time (and cost) required to finish an assessment (low speed), which together reduced the value of the pathology data set. Moreover, 1 participant noted that randomization matters less than the order of histopathologic evaluation since diagnoses on features of the current slide are influenced substantially by the 5 most recently viewed slides. 20
Several in the audience suggested that documenting those occasions where a blinded histopathologic evaluation is performed, using a short statement in the pathology sub-report, would be a viable means of indicating to non-pathologists that such a process was in place. If included, discussants suggested that such statements ideally would indicate whether or not the blinding was formal (ie, slides with coded labels) or informal (ie, slides inverted to hide labels and then shuffled). A counter argument to this suggestion was that regulatory-type studies would need more detailed documentation that a blinded evaluation (formal or informal) had been performed.
Summary
The data on present practices for selecting between non-blinded or blinded histopathologic evaluation indicate a remarkable degree of consensus among toxicologic pathologists, and some non-pathologists, who are members of the STP. The pattern closely follows current thinking among industry, contract research organizations, and consultant pathologists, who collectively perform the greater part of animal toxicity studies slated for regulatory review. A very large majority of STP members indicate that hazard identification/safety assessment in regulatory-type, GLP-compliant animal toxicity studies is best addressed utilizing an initial non-blinded (“unblinded”) histopathologic evaluation followed (if necessary) by blinded re-evaluation to discriminate subtle findings and/or set a test article-related effect threshold (eg, the NOEL or NOAEL). Blinded histopathologic evaluation during regulatory-type animal toxicity studies generally is elected by the pathologist, usually is undertaken informally post hoc following an initial non-blinded evaluation, and is not documented in the pathology report. Blinded histopathologic evaluation is seldom chosen as the initial evaluation strategy for a pathology peer review. The rationale for these strategies is that histopathologic diagnoses, while influenced by a pathologist’s individual experience, are interpreted in combination with additional study findings including other pathology data (eg, macroscopic observations, organ weights, clinical pathology endpoints) as well as in-life clinical observations to achieve an integrated assessment of test article-related effects on animal tissues. Taken together, these facts demonstrate that toxicologic pathologists use a well-honed, empirical approach to maximize the value of the histopathologic data set in detecting and characterizing potential hazards, whether or not a blinded evaluation approach is employed.
Supplemental Material
Supplemental Material, sj-docx-1-tpx-10.1177_0192623320920590 - Opinion on Current Use of Non-Blinded Versus Blinded Histopathologic Evaluation in Animal Toxicity Studies
Supplemental Material, sj-docx-1-tpx-10.1177_0192623320920590 for Opinion on Current Use of Non-Blinded Versus Blinded Histopathologic Evaluation in Animal Toxicity Studies by Brad Bolon, Jessica M. Caverly Rae, Karyn Colman, Sabine Francke, Karl Jensen, Kevin Keane, Elizabeth F. McInnes, Kyoko Nakano-Ito, Rick Perry, Evelyne Polack, Karen S. Regan, Annette Romeike, Jamie K. Young and Elizabeth J. Galbreath in Toxicologic Pathology
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
