Permutation tests are based on all possible arrangements of observed data sets. Consequently, such tests yield exact probability values obtained from discrete probability distributions. An exact nondirectional method to combine independent probability values that obey discrete probability distributions is introduced. The exact method is the discrete analog to Fisher's classical method for combining probability values from independent continuous probability distributions. If the combination of probability values includes even one probability value that obeys a sparse discrete probability distribution, then Fisher's classical method may be grossly inadequate.
Get full access to this article
View all access options for this article.
References
1.
BerryK. J.JohnstonJ. E.MielkeP. W.Jr. (2003) Permutation methods for the analysis of matched-pairs experimental designs. Psychological Reports, 92, 1141–1150.
2.
BerryK. J.JohnstonJ. E.MielkeP. W.Jr. (2004) Exact goodness-of-fit tests for unordered equiprobable categories. Perceptual and Motor Skills, 98, 909–919.
3.
BerryK. J.MielkeP. W.Jr.MielkeH. W. (2002) The Fisher-Pitman test: an attractive alternative to the F test. Psychological Reports, 90, 495–502.
4.
EdgingtonE. S. (1995) Randomization tests. (3rd ed.) New York: Marcel Dekker.
5.
EdgingtonE. S.HallerO. (1984) Combining probabilities from discrete probability distributions. Educational and Psychological Measurement, 44, 265–274.
6.
FisherR. A. (1934) Statistical methods for research workers. (5th ed.) Edinburgh, Scot.: Oliver & Boyd.
7.
GoodP. I. (2000) Permutation tests: a practical guide to resampling methods for testing hypotheses. (2nd ed.) New York: Springer-Verlag.
8.
GoodP. I. (2001) Resampling methods: a practical guide to data analysis. (2nd ed.) Boston, MA: Birkhäuser.
9.
GordonM. H.LovelandE. H.CuretonE. E. (1952) An extended table of chi-squared. for two degrees-of-freedom, for use in combining probabilities from independent samples. Psychometrika, 17, 311–316.
10.
KincaidW. M. (1962) The combination of tests based on discrete distributions. Journal of the American Statistical Association, 57, 10–19.
11.
LancasterH. O. (1949) The combination of probabilities arising from data in discrete distributions. Biometrika, 36, 370–382.
12.
LittellR. C.FolksJ. L. (1971) Asymptotic optimality of Fisher's method of combining independent tests. Journal of the American Statistical Association, 66, 802–806.
13.
LittellR. C.FolksJ. L. (1973) Asymptotic optimality of Fisher's method of combining independent tests: II. Journal of the American Statistical Association, 68, 193–194.
14.
LudbrookJ.DudleyH. (1998) Why permutation tests are superior to t and F tests in biomedical research. The American Statistician, 52, 127–132.
15.
LunneborgC. E. (2000) Analysis by resampling: concepts and applications. Pacific Grove, CA: Duxbury.
16.
ManlyB. J. F. (1997) Randomization, bootstrap, and Monte Carlo methods in biology. (2nd ed.) London: Chapman & Hall.
17.
MielkeP. W.Jr. (1972) Asymptotic behavior of two-sample tests based on powers of ranks for detecting scale and location alternatives. Journal of the American Statistical Association, 67, 850–854.
18.
MielkeP. W.Jr.BerryK. J. (2001) Permutation methods: a distance function approach. New York: Springer-Verlag.
19.
MielkeP. W.Jr.BerryK. J.JohnstonJ. E. (2004) Asymptotic log-linear analysis: some cautions concerning sparse frequency tables. Psychological Reports, 94, 19–32.
20.
PearsonE. S. (1950) On questions raised by the combination of tests based on discontinuous distributions. Biometrika, 37, 383–398.
21.
PesarinF. (2001) Multivariate permutation tests: with applications in biostatistics. Chichester, UK: Wiley.
22.
PitmanE. J. G. (1937a) Significance tests which may be applied to samples from any populations. Supplement to the Journal of the Royal Statistical Society, 4, 119–130.
23.
PitmanE. J. G. (1937b) Significance tests which may be applied to samples from any populations: II. Supplement to the Journal of the Royal Statistical Society, 4, 225–232.
24.
PitmanE. J. G. (1938) Significance tests which may be applied to samples from any populations: III. The analysis of variance tests. Biometrika, 29, 322–335.
25.
TrehubA.HeilizerF. (1962) Comments on the testing of combined results. Journal of Clinical Psychology, 18, 329–333.
26.
WallisW. A. (1942) Compounding probabilities from independent significance tests, Econometrica, 10, 229–248.
