Abstract
To the Editor:
I read with considerable interest the pair of papers dealing with generation of unbiased histopathology data by T. Holland and C. Holland (2011a, 2011b) in Toxicologic Pathology, volume 39, issue 4, 2011.
The authors correctly indicate in their second (perspective) paper (Holland and Holland 2011b) that the common practice of histological examination of a standard toxicity study is divided into two phases. The first phase is to identify potential or apparent treatment-related changes (in their words “go-look-see”), and the second (sometimes done blinded to treatment or to animal) is to grade or rank these lesions in some way to determine which animals and/or groups are affected. In their first (didactic) paper (Holland and Holland 2011a), they describe several methods of achieving the latter, including: (1) rank ordering individual animals by severity of the subject finding, (2) scoring according to a pre-specified severity grading scheme, (3) identifying those treated animals that have the subject finding to a degree exceeding any in the control group, and (4) side-by-side comparison of pairs of tissues selected from control and treated groups. They also recommend certain statistical procedures that should be applied to the blindly generated data.
During my career I have used each of the methods described by Holland and Holland for specific situations depending on the nature of the finding and the study design. I agree that these are appropriate ways to substantiate a treatment and/or dose relationship. Where I differ is on the use of statistical evaluation of the data. The authors imply that statistical evaluation should be used routinely when blinded data were generated by one of the described methods. I can’t remember a time when I used or felt the need of such mathematical methods. I am not saying statistical analysis should not be used under these circumstances; but let us remember that statistical analysis, even when used for machine-generated hard data, such as clinical pathology or organ weights, is but a tool. It may alert the Study Pathologist and/or Study Director to data that deserve additional consideration. It should be a factor, but not the deciding factor as to whether a given finding is treatment related. In the case of blinded histopathology data, by adopting the process the pathologist has already decided the subject finding is deserving of further attention. Even when derived by blinded evaluation, the data are still to some extent subjective and dependent on the training and experience of the pathologist. By the time the blinded data have been generated, the pathologist will have already spent considerable time thinking about the finding and its consequences. In the end, it is the pathologist’s obligation to use his or her professional judgment to decide which animals and groups are affected.
Having considered the case of blinded evaluation by a semi-objective method in the preceding paragraph, I will add some thoughts regarding the data generated by the initial non-blinded histopathological evaluation of a study. Holland and Holland (2011b) state: It must be stressed that it is only reasonable to use the analytical methods given above if the data are unbiased. This means the data must be gathered with the observer completely blind to the individual’s treatment (among other conditions). There is no need to formally analyze data that has been gathered with knowledge of the treatment group.
I agree with their conclusion, but I would take it a step further. I would say that the initial non-blinded data should NEVER be subjected to formal statistical analysis. We all know the fuzziness that may exist on the slide and the complex thinking used to decide what to include in the database and further fuzziness about grade levels. Having decided what to include in the data, we are faced with a computer-generated incidence table that omits all the fuzziness. It appears to non-pathologists as black and white: “These animals had the finding and these other ones did not.” To apply statistical analysis to such data gives it a further unjustified aura of certainty and precision. I have peer-reviewed studies at laboratories that use computer data capture programs that automatically apply statistical analysis to everything in the incidence table. As a result, the study pathologist may be forced to deal with a lot of junk data in the pathology narrative report. So I say if your computer system has this feature, turn it off. If it can’t be turned off, look for another system.