Permutation Tests Are a Useful Alternative Approach for Statistical Hypothesis Testing in Small Sample Sizes

Abstract

Results interpretation and statistical analysis of animal study data is challenging, since the sample sizes involved are usually very small. The application of frequently used approaches to statistical hypothesis testing, e.g. t-tests or ANOVA methods, rely on specific distributional assumptions being satisfied. It can be hard to reliably assess these assumptions in animal studies with group sizes of usually less than ten animals. Non-parametric analysis methods might be considered as an alternative, but it is well-known that these approaches have lower statistical power in some situations. Following the Three Rs principles, it would be desirable to apply a class of statistical tests that is able to deal with a small number of observations, without the need for specific distributional assumptions. Thus, in this paper, we assess the application of permutation tests which seem to be able to meet both the above requirements. The performance of these permutation tests was compared with standard statistical tests by means of four real-world data examples from animal studies. The results demonstrated that permutation tests have good computational properties, leading to the conclusion that they could be a useful alternative approach when analysing small sample size animal study data for which distributional assumptions may not hold.