Restricted accessResearch articleFirst published online 2025-8
Toxicologic Pathology Forum*: Opinion on the Interpretation of Statistical Significance Testing Results From Anatomic and Clinical Pathology Data in Nonclinical Safety Studies
Toxicologic pathologists assess large data sets from nonclinical studies to identify treatment-related effects to assist in predicting human safety hazards. Statistical testing can facilitate data interpretation by highlighting group differences that have a low probability of random occurrence based on a pre-determined P-value cut-off (eg, P < .05). While this method has been used in the interpretation of pathology data for decades, the appropriateness of utilizing statistical testing in this way has been challenged. Here, we discuss common statistical pitfalls in the analysis of toxicologic pathology data, with emphasis on clinical pathology, reaffirming that appropriate use of statistical analysis requires an understanding of (1) the parameters assessed; (2) the inherent strengths and weaknesses of the statistical method used; and (3) that appropriate interpretation of pathology data is based on the pathologist’s expertise. The presence or absence of statistical significance should not supersede expert judgment but should be one of many tools used to reach a conclusion.
AmrheinVGreenlandSMcShaneB. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-307. doi:10.1038/d41586-019-00857-9.
2.
ArndtTKeresztesMOlivierB, et al. Considerations for the identification and conveyance of clinical pathology findings in preclinical toxicity studies: results from the 9th ESTP international expert workshop. Toxicol Pathol. 2024;52(6):319-332. doi:10.1177/01926233241245108.
3.
AulbachAVitskyAArndtT, et al. Interpretative considerations for clinical pathology findings in nonclinical toxicology studies. Vet Clin Pathol. 2019;48(3):383-388. doi:10.1111/vcp.12773.
4.
BailerAJPortierCJ. Effects of treatment-induced mortality and tumor-induced mortality on tests for carcinogenicity in small samples. Biometrics. 1988;44(2):417-431.
5.
BaileySAZidellRHPerryRW. Relationships between organ weight and body/brain weight in the rat: what is the best analytical endpoint?Toxicol Pathol. 2004;32(4):448-466. doi:10.1080/01926230490465874.
6.
BaldeshwilerAM. History of FDA good laboratory practices. Qual Assur J. 2003;7(3):157-161. doi:10.1002/qaj.228.
7.
BenjaminDJBergerJO. Three recommendations for improving the use of p-values. Am Statistician. 2019;73(suppl 1):186-191. doi:10.1080/00031305.2018.1543135.
8.
BetenskyRA. The p-value requires context, not a threshold. Am Statistician. 2019;73(suppl 1):115-117. doi:10.1080/00031305.2018.1529624.
9.
BhardwajSSCamachoFDerrowAFleischerABFeldmanSR. Statistical significance and clinical relevance: the importance of power in clinical trials in dermatology. Arch Dermatol. 2004;140(12):1520-1523. doi:10.1001/archderm.140.12.1520.
10.
BrownsteinNCLouisTAO’HaganAPendergastJ. The role of expert judgment in statistical inference and evidence-based decision-making. Am Stat. 2019;73(1):56-68. doi:10.1080/00031305.2018.1529623.
11.
Calin-JagemanRJCummingG. The new statistics for better science: ask how much, how uncertain, and what else is known. Am Statistician. 2019;73(sup1):271-280. doi:10.1080/00031305.2018.1518266.
12.
ChristleyRM. Statistical significance, power and sample size—what does it all mean?J Small Anim Pract. 2008;49(6):263-263. doi:10.1111/j.1748-5827.2008.00612.x.
Curran-EverettD. Evolution in statistics: p values, statistical significance, kayaks, and walking trees. Adv Physiol Educ. 2020;44(2):221-224. doi:10.1152/advan.00054.2020.
16.
EdwardsAWF. Chapter 67—R.A. Fischer, statistical methods for research workers, first edition (1925). In: Grattan-GuinnessICookeRCorryLCrépelPGuicciardiniN, eds. Landmark Writings in Western Mathematics 1640-1940. Amsterdam, Netherlands: Elsevier Science; 2005:856-870.
17.
European Medicines Agency. Guideline on Repeated Dose Toxicity. Amsterdam, Netherlands: European Medicines Agency; 2008.
18.
European Medicines Agency. ICH Guideline S2 (R1) on Genotoxicity Testing and Data Interpretation for Pharmaceuticals Intended for Human Use. Amsterdam, Netherlands: European Medicines Agency; 2008.
19.
EverdsNESnyderPWBaileyKL, et al. Interpreting stress responses during routine toxicity studies: a review of the biology, impact, and assessment. Toxicol Pathol. 2013;41(4):560-614. doi:10.1177/0192623312466452.
20.
FestingMFW. Extending the statistical analysis and graphical presentation of toxicity test results using standardized effect sizes. Toxicol Pathol. 2014;42(8):1238-1249. doi:10.1177/0192623313517771.
21.
Food and Drug Administration. Redbook 2000: IV.B.3. Pathology Considerations in Toxicity Studies. Silver Spring, MD: Food and Drug Administration; 2007.
22.
Food and Drug Administration. Redbook 2000: IV.B.4. Statistical Considerations in Toxicity Studies. Silver Spring, MD: Food and Drug Administration; 2007.
23.
Food and Drug Administration. Statistical Aspects of the Design, Analysis, and Interpretation of Chronic Rodent Carcinogenicity Studies of Pharmaceuticals. Silver Spring, MD: Food and Drug Administration; 2001.
24.
GadSC. Statistical Methods in Toxicology. 3 Ed. Comprehensive Toxicology III. Amsterdam, Netherlands: Elsevier Inc.; 2017.
25.
GadSRousseauxC. Use and misuse of statistics in the design and interpretation of studies. Handbook of Toxicologic Pathology. Amsterdam, Netherlands: Elsevier Inc.; 2002:327-418.
26.
Gibson-CorleyKNOlivierAKMeyerholzDK. Principles for valid histopathologic scoring in research. Vet Pathol. 2013;50(6):1007-1015. doi:10.1177/0300985813485099.
27.
HallRL. Lies, damn lies, and reference intervals (or hysterical control values for clinical pathology data). Toxicol Pathol. 1997;25(6):647-649. doi:10.1177/019262339702500617.
28.
HallRL. Practical considerations in clinical pathology data interpretation and description: the use of statistics, reference intervals, and severity descriptors. Toxicol Pathol. 2016;45(2):362365. doi:10.1177/0192623316669824.
29.
HallRL. Practical considerations in clinical pathology data interpretation and description. Toxicol Pathol. 2017;45(2):362-365. doi:10.1177/0192623316669824.
30.
HallRLEverdsNE. Factors affecting the interpretation of canine and nonhuman primate clinical pathology. Toxicol Pathol. 2003;31(suppl):6-10. doi:10.1080/01926230390174878.
31.
HanelPHMehlerDM. Beyond reporting statistical significance: identifying informative effect sizes to improve scientific communication. Public Underst Sci. 2019;28(4):468-485. doi:10.1177/0963662519834193.
32.
HardGCKhanKN. A contemporary overview of chronic progressive nephropathy in the laboratory rat, and its significance for human risk assessment. Toxicol Pathol. 2004;32(2):171-180. doi:10.1080/01926230490422574.
33.
HarrKEFlatlandBNabityMFreemanKP. ASVCP guidelines: allowable total error guidelines for biochemistry. Vet Clin Path. 2013;42(4):424436. doi:10.1111/vcp.12101.
34.
HaruyamaESudaMAyukawaY, et al. Testicular development in cynomolgus monkeys. Toxicol Pathol. 2012;40(6):935-942. doi:10.1177/0192623312444619.
35.
HasemanJHajianGCrumpKSelwynMPeaceK. Dual control groups in rodent carcinogenicity studies. Statistical Issues in Drug Research and Development. Boca Raton, FL: CRC Press; 2017:351-362.
HeyseJFRomD. Adjusting for multiplicity of statistical tests in the analysis of carcinogenicity studies. Biom J. 1988;30(8):883-896. doi:10.1002/bimj.4710300802.
38.
HochsterHS. The power of “p”: on overpowered clinical trials and “positive” results. Gastrointest Cancer Res. 2008;2(2):108-109.
Kennedy-ShafferL. Before p < 0.05 to beyond p < 0.05: using history to contextualize p-values and significance testing. Am Stat. 2019;73(suppl 1)82-90. doi:10.1080/00031305.2018.1537891.
41.
KruegerJIHeckPR. Putting the p-value in its place. Am Statistician. 2019;73(sup1):122-128. doi:10.1080/00031305.2018.1470033.
42.
LakensD. The practical alternative to the p value is the correctly used p value. Perspect Psychol Sci. 2021;16(3):639-648. doi:10.1177/1745691620958012.
43.
LeekJTPengRD. Statistics: p values are just the tip of the iceberg. Nature. 2015;520(7549):612. doi:10.1038/520612a.
44.
LieberRL. Statistical significance and statistical power in hypothesis testing. J Orthopaed Res. 1990;8(2):304-309. doi:10.1002/jor.1100080221.
45.
LovellDP. Null hypothesis significance testing and effect sizes: can we “effect” everything. . . or. . . anything?Curr Opin Pharmacol. 2020;51:68-77.
46.
LuusHGMullerFOMeyerBH. Statistical significance versus clinical relevance. Part I. The essential role of the power of a statistical test. South Afr Medical J. 1989;76(10):568-570.
47.
MannPCVahleJKeenanCM, et al. International harmonization of toxicologic pathology nomenclature: an overview and review of basic principles. Toxicol Pathol. 2012;40(4 suppl):7S-13S. doi:10.1177/0192623312438738.
48.
MaschaEJVetterTR. Significance, errors, power, and sample size: the blocking and tackling of statistics. Anesth Analg. 2018;126(2):691-698. doi:10.1213/ane.0000000000002741.
49.
McConnellEESolleveldHASwenbergJABoormanGA. Guidelines for combining neoplasms for evaluation of rodent carcinogenesis studies. J Natl Cancer Inst. 1986;76(2):283-289.
50.
McShaneBBGalDGelmanARobertCTackettJL. Abandon statistical significance. Am Statistician. 2019;73(suppl):235-245. doi:10.1080/00031305.2018.1527253.
51.
NaJYangHBaeSLimK-M. Analysis of statistical methods currently used in toxicology journals. Toxicol Res. 2014;30(3):185-191. doi:10.5487/tr.2014.30.3.185.
National Research Council. Reference Manual on Scientific Evidence: Third Edition. Washington, DC: The National Academies Press; 2011:1034.
54.
OECD. Test no. 407: repeated dose 28-day oral toxicity study in rodents. OECD Guidelines for the Testing of Chemicals, Section 4. Paris, France: OECD Publishing; 2008.
55.
OECD. Test no. 408: repeated dose 90-day oral toxicity study in rodents. OECD Guidelines for the Testing of Chemicals, Section 4. Paris, France: OECD Publishing; 2018.
56.
OECD. Test no. 409: repeated dose 90-day oral toxicity study in non-rodents. OECD Guidelines for the Testing of Chemicals, Section 4. Paris, France: OECD Publishing; 1998.
57.
OECD. Test no. 452: chronic toxicity studies. OECD Guidelines for the Testing of Chemicals, Section 4. Paris, France: OECD Publishing; 2018.
58.
O’HaganA. Expert knowledge elicitation: subjective but scientific. Am Statistician. 2019;73(suppl 1):69-81. doi:10.1080/00031305.2018.1518265.
59.
PackerM. The parable of Schrödinger’s Cat and the illusion of statistical significance in clinical trials. Circulation. 2019;140(10):799-800. doi:10.1161/circulationaha.119.041245.
60.
PetoRPikeMCDayNE, et al. Guidelines for simple, sensitive significance tests for carcinogenic effects in long-term animal experiments. IARC Monogr Eval Carcinog Risk Chem Hum Suppl. 1980(2 suppl):311-426.
61.
PooleALeslieGB. A Practical Approach to Toxicological Investigations. Cambridge: Cambridge University Press; 1989.
62.
RamaiahLHinrichsMJSkubaEVIversonWOEnnulatD. Interpreting and integrating clinical and anatomic pathology results. Toxicol Pathol. 2017;45(1):223-237. doi:10.1177/0192623316677068.
63.
RichterAZinkA. Gehört die statistische Signifikanz aufs Altenteil? [Should statistical significance be retired?]. Z Rheumatol. 2020;79(7):692-695. doi:10.1007/s00393-020-00835-x.
64.
RivasALHoogesteijnAL. Biologically grounded scientific methods: the challenges ahead for combating epidemics. Methods. 2021;195:113-119. doi:10.1016/j.ymeth.2021.09.001.
65.
RonaldLWAllenLSNicoleAL. Moving to a world beyond “ p < 0.05.”Am Statistician. 2019;73:119. doi:10.1080/00031305.2019.1583913.
66.
RousseauxCGGadSC. Statistical assessment of toxicologic pathology studies. Third Ed. Haschek and Rousseaux’s Handbook of Toxicologic Pathology. Cambridge, MA: Academic Press; 2013:893-988.
67.
SchaferKAEighmyJFikesJD, et al. Use of severity grades to characterize histopathologic changes. Toxicol Pathol. 2018;46(3):256-265. doi:10.1177/0192623318761348.
68.
SellersRSMortonDMichaelB, et al. Society of toxicologic pathology position paper: organ weight recommendations for toxicology studies. Toxicol Pathol. 2007;35(5):751-755. doi:10.1080/01926230701595300.
69.
SiskaWGuptaATomlinsonLTripathiNvon BeustB. Recommendations for clinical pathology data generation, interpretation, and reporting in target animal safety studies for veterinary drug development. Int J Toxicol. 2017;36(4):293-302. doi:10.1177/1091581817711876.
70.
SiskaWSchultzeAEEnnulatD, et al. Scientific and regulatory policy committee points to consider: integration of clinical pathology data with anatomic pathology data in nonclinical toxicology studies. Vet Clin Pathol. 2022;51(3):311-329. doi:10.1111/vcp.13167.
71.
Statistical inference in the 21st century: a world beyond p < 0.05. Am Statistician. 2019;73(sup 1):1-401.
72.
TomlinsonLBooneLIRamaiahL, et al. Best practices for veterinary toxicologic clinical pathology, with emphasis on the pharmaceutical and biotechnology industries. Vet Clin Pathol. 2013;42(3):252-269. doi:10.1111/vcp.12059.
73.
TongC. Statistical inference enables bad science; statistical thinking enables good science. Am Statistician. 2019;73(sup1):246-261. doi:10.1080/00031305.2018.1518264.
74.
TripathiNKEverdsNESchultzeAE, et al. Deciphering sources of variability in clinical pathology. Toxicol Pathol. 2017;45(1):90-93. doi:10.1177/0192623316675766.
75.
U.S. Environmental Protection Agency. Evaluation Guidelines for Ecological Toxicity Data in the Open Literature. Washington, DC: U.S. Environmental Protection Agency; 2011.
76.
WanerT. Current statistical approaches to clinical pathological data from toxicological studies. Toxicol Pathol. 1992;20(3-2):477-479. doi:10.1177/0192623392020003203.
77.
WassersteinRLLazarNA. The ASA’s statement on p-values: context, process, and purpose. Am Statistician. 2016;70(2):129-133. doi:10.1080/00031305.2016.1154108.
78.
WhittakerETRobinsonG. The Calculus of Observations: A Treatise on Numerical Mathematics. Frisco, TX: Blackie; 1956.
79.
ZampieriFGCaseyJDShankar- HariMHarrellFEHarhayMO. Using Bayesian methods to augment the interpretation of critical care trials. An overview of theory and example reanalysis of the alveolar recruitment for acute respiratory distress syndrome trial. Am J Resp Crit Care. 2021;203(5):543-552. doi:10.1164/rccm.202006-2381cp.
