The reproducibility of the results from preclinical research rests on many factors, including the selection of appropriate experimental designs for the individual experiments that constitute the investigation. The design of each of these experiments depends on their purpose within the entire investigation and the information to be gained from conducting them. Here, we explain and justify a three-stage strategy comprising a series of different types of experiment, each with a different purpose and design: a pilot study, a hypothesis-generating experiment and a final hypothesis-confirming experiment. Compliance with this three-stage strategy, over the course of an entire investigation, will not only strengthen its reproducibility but, importantly, can save time and other resources, including the total number of animals used.
BallsM (2013) The wisdom of Russell and Burch. 3. Fidelity and discrimination. Altern Lab Animals41: 12–13.
2.
BateSTClarkRA (2014) The Design and Statistical Analysis of Animal Experiments. Cambridge, UK: Cambridge University Press.
3.
BenjaminiY (2020) Selective inference: The silent killer of replicability. Harvard Data Sci Rev2:4. DOI: 10.1162/99608f92.fc62b261.
4.
BenjaminiYDe VeauxREfronB, et al. (2021) ASA president’s task force statement on statistical significance and replicability. Chance34(4): 10-11.
5.
BerkRBrownLBujaA, et al. (2013) Valid post-selection inference. Ann Statist41: 802–837.
6.
ButtonKSIoannidisJPMokryszC, et al. (2013) Power failure: Why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci14: 365–376.
7.
CiaravinoVBrulfertAMillerMW, et al. (1985) Diagnostic ultrasound and sister chromatid exchanges: Failure to reproduce positive findings. Science227: 1349–1351.
8.
ClarkRAShoaibMHewittKN, et al. (2012) A comparison of InVivoStat with other statistical software packages for analysis of data generated from animal experiments. J Psychopharmacol26: 1136–1142.
9.
ClarkIAMonkAMHotchinV, et al. (2020) Does hippocampal volume explain performance differences on hippocampal-dependant tasks?Neuroimage221: 117211.
10.
CummingG (2008) Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspect Psychol Sci 3: 286–300.
11.
DrudeNIGamboaLMDanzigerM, et al. (2021) Improving preclinical studies through replications. Elife10: e62101.
12.
ErringtonTMMathurMSoderbergCK, et al. (2021) Investigating the replicability of preclinical cancer biology. Elife10: e71601.
13.
FestingMF (2003) Principles: The need for better experimental design. Trends Pharmacol Sci24: 341–345.
14.
FestingMFAltmanDG (2002) Guidelines for the design and statistical analysis of experiments using laboratory animals. ILAR J 43: 244–258.
15.
FestingMFOverendPDasRG, et al. (2016) The Design of Animal Experiments: Reducing the Use of Animals in Research Through Better Experimental Design. (Revised and updated edition). London, UK: Sage Publications Ltd.
16.
HeinlCScholman-VéghAMDMellorD, et al. (2022) Declaration of common standards for the preregistration of animal research – speeding up the scientific progress. PNAS Nexus1: pgac016.
17.
HuangWPercie du SertNVollertJ, et al. (2019) General principles of preclinical study design. In: SKKulkarni (ed.) Handbook of Experimental Pharmacology. Berlin, Heidelberg: Springer.
18.
IoannidisJPA (2018) The proposal to lower p value thresholds to .005. JAMA319: 1429–1430.
19.
IoannidisJPA (2019) Retiring statistical significance would give bias a free pass. Nature567: 461.
20.
KarpNASharpeAPhillipsB (2024) Preclinical pilot studies: Five common pitfalls and how to avoid them. Lab Animals58: 481–485.
21.
KimmelmanJMogilJSDirnaglU (2014) Distinguishing between exploratory and confirmatory preclinical research will improve translation. PLoS Biol12: e1001863.
22.
LazicSEClarke-WilliamsCJMunafoMR (2018) What exactly is ‘N’ in cell culture and animal experiments?PLoS Biol16: e2005282.
23.
LeonAC (2004) Multiplicity-adjusted sample size requirements: A strategy to maintain statistical power with Bonferroni adjustments. J Clin Psychiatry65: 1511–1514.
24.
LilleyEStanfordSCKendallDE, et al. (2020) ARRIVE 2.0 and the British Journal of Pharmacology: Updated guidance for 2020. Br J Pharmacol77: 3611–3616.
25.
LimentaniGBRingoMCYeF, et al. (2005) Beyond the t-test: Statistical equivalence testing. Anal Chem77: 221A–226A.
26.
MacleodMMohanS (2019) Reproducibility and rigor in animal-based research. ILAR J (60): 17–23.
27.
MarslandALSalmonPTerryP, et al. (1990) Effects of propranolol on, and noradrenergic correlates of, the response to nonreward. Pharmacol Biochem Behav35: 41–46.
28.
McNuttM (2014) Reproducibility. Science343: 229.
29.
McShaneBBGalDGelmanA, et al. (2019) Abandon statistical significance. Am Statist73: 235–245.
30.
MeadR (1990) The Design of Experiments: Statistical Principles for Practical Applications. Cambridge, UK: Cambridge University Press.
31.
MitchellPJ (2022) Experimental Design and Statistical Analysis for Pharmacology and the Biomedical Sciences. Hoboken, NJ: Wiley Blackwell.
32.
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science349: aac4716.
33.
Percie du SertNBamseyIBateST, et al. (2017) The experimental design assistant. PLoS Biol15: e2003779.
34.
SamsaGSamsaL (2019) A guide to reproducibility in preclinical research. Acad Med94: 47–52.
35.
SchwabSHeldL (2020) Different worlds confirmatory versus exploratory research. Significance17: 8–9.
36.
SenaEvan der WorpHBHowellsD, et al. (2007) How can we improve the pre-clinical development of drugs for stroke?Trends Neurosci30: 433–439.
37.
SenaESVan Der WorpHBBathPM, et al. (2010) Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol8: e1000344.
38.
ShapiroS (2004) Looking to the 21st century: Have we learned from our mistakes, or are we doomed to compound them?Pharmacoepidemiol Drug Saf13: 257–265.
39.
ShawRFestingMFPeersI, et al. (2002) Use of factorial designs to optimize animal experiments and reduce animal use. ILAR J43: 223–232.
40.
StanfordSCAlexanderSCirinoG, et al. (2023) Considering and reporting sex as an experimental variable II: An update on progress in the British Journal of Pharmacology. Br J Pharmacol180: 1191–1196.
41.
StanleyP (2022) Analysis of covariance: A useful tool for the pharmacologist to reduce variation and improve precision using fewer animals. Br J Pharmacol179: 3645–3650.
42.
WassersteinRLSchirmALLazarNA (2019) Moving to a world beyond “p < 0.05”. Am Statist73: 1–19.
43.
XuHLiF (2019) [Retracted] miR-127 aggravates myocardial failure by promoting the TGF-β1/Smad3 signaling. Mol Med Rep20: 2500.
