The Fisher-Pitman permutation test can detect any type of difference between two samples; hence, a significant Fisher-Pitman permutation test does not necessarily provide evidence for a difference in means. It is possible, however, to test separately for differences in means and variances. Here, we present a recently proposed two-stage procedure to decide whether there are differences in means or variances that can be applied when samples may come from nonnormal distributions with possibly unequal variances.
Get full access to this article
View all access options for this article.
References
1.
BaumgartnerW.WeissP.SchindlerH. (1998) A nonparametric test for the general two-sample problem. Biometrics, 54, 1129–1135.
2.
BehrensW. V. (1929) Ein Beitrag zur Fehlerrechnung bei wenigen Beobachtungen. Landwirtschaftliche Jahrbücher, 68, 807–837.
3.
BerryK. J.MielkeP. W.MielkeH. W. (2002) The Fisher-Pitman permutation test: An attractive alternative to the F test. Psychological Reports, 90, 495–502.
4.
BlairC.ThompsonG. (1992) A distribution-free rank-like test for scale with unequal population locations. Communications in Statistics—Simulation and Computation, 21, 353–371.
5.
BoikR. J. (1987) The Fisher-Pitman permutation test: A non-robust alternative to the normal theory F test when variances are heterogeneous. British Journal of Mathematical and Statistical Psychology, 40, 26–42.
6.
BrunnerE.MunzelU. (2000) The nonparametric Behrens-Fisher problem: Asymptotic theory and a small-sample approximation. Biometrical Journal, 42, 17–25.
7.
FisherR. A. (1936) The coefficient of racial likeness and the future of craniometry. Journal of the Royal Anthropological Institute, 66, 57–63.
8.
FisherR. A. (1939) The comparison of samples with possible unequal variances. Annals of Eugenics, 9, 174–180.
9.
FrancisR. I. C. C.ManlyB. F. J. (2001) Bootstrap calibration to improve the reliability of tests to compare sample means and variances. Environmetrics, 12, 713–729.
10.
GoriaM. N. (1982) A survey of two-sample location-scale problem, asymptotic relative efficiencies of some rank tests. Statistica Neerlandica, 36, 3–13.
11.
HayesA. F. (2000) Randomization tests and the equality of variance assumption when comparing group means. Animal Behaviour, 59, 653–656.
12.
Keller-McNultyS.HigginsJ. J. (1987) Effect of tail weight on power and type-I error of robust permutation tests for location. Communications in Statistics—Simulation and Computation, 16, 17–35.
13.
LeveneH. (1960) Robust tests for equality of variances. In OlkinI.GhuryeS. G.HoeffdingW.MadowW. G.MannH. B. (Eds.), Contributions to probability and statistics. Stanford, CA: Stanford Univer. Press. Pp. 278–292.
14.
ManlyB. F. J. (1997) Randomization, bootstrap and Monte Carlo methods in biology. (2nd ed.) London: Chapman & Hall.
15.
ManlyB. F. J.FrancisR. I. C. C. (1999) Analysis of variance by randomization when variances are unequal. Australian and New Zealand Journal of Statistics, 41, 411–429.
16.
ManlyB. F. J.FrancisR. I. C. C. (2002) Testing for mean and variance differences with samples from distributions that may be non-normal with unequal variances. Journal of Statistical Computation and Simulation, 72, 633–646.
17.
MicceriT. (1989) The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.
18.
MielkeP. W.BerryK. J. (1994) Permutation tests for common location among samples with unequal variances. Journal of Educational and Behavioural Statistics, 19, 217–236.
19.
NeuhäuserM. (2000) An exact two-sample test based on the Baumgartner-Weiß-Schindler statistic and a modification of Lepage's test. Communication in Statistics—Theory and Methods, 29, 67–78.
20.
NeuhäuserM. (2002) Two-sample tests when variances are unequal. Animal Behaviour, 63, 823–825.
21.
NeuhäuserM. (2003) A note on the exact test based on the Baumgartner-Weiss-Schindler statistic in the presence of ties. Computational Statistics and Data Analysis, 42, 561–568.
22.
NeuhäuserM.HothornL. A. (2000) Parametric location-scale and scale trend tests based on Levene's transformation. Computational Statistics and Data Analysis, 33, 189–200.
23.
OgenstadS. (1998) The use of generalized tests in medical research. Journal of Biopharmaceutical Statistics, 8, 497–508.
24.
PitmanE. J. G. (1937) Significance tests which may be applied to samples from any populations. Supplement to the Journal of the Royal Statistical Society, 4, 119–130.
25.
TanizakiH. (1997) Power comparison of non-parametric tests: Small-sample properties from Monte Carlo experiments. Journal of Applied Statistics, 24, 603–632.
26.
Van den BrinkW. P.van den BrinkS. G. J. (1989) A comparison of the power of the t test, Wilcoxon's test, and the approximate permutation test for the two-sample location problem. British Journal of Mathematical and Statistical Psychology, 42, 183–189.
