When sample observations are not independent, the variance estimate in the denominator of the Student t statistic is altered, inflating the value of the test statistic and resulting in far too many Type I errors. Furthermore, how much the Type I error probability exceeds the nominal significance level is an increasing function of sample size. If N is quite large, in the range of 100 to 200 or larger, small apparently inconsequential correlations that are unknown to a researcher, such as .01 or .02, can have substantial effects and lead to false reports of statistical significance when effect size is zero.
Get full access to this article
View all access options for this article.
References
1.
BerryK. J.MielkeP. W. (2000) A Monte Carlo investigation of the Fisher Z transformation for normal and nonnormal distributions. Psychological Reports, 87, 1101–1114.
2.
CochranW. G. (1947) Some consequences when the assumptions in the analysis of variance are not satisfied. Biometrics, 3, 22–38.
3.
CohenJ. (1988) Statistical power analysis for the behavioral sciences. (2nd ed.) Hillsdale, NJ: Erlbaum.
4.
HayesA. F. (1996) Permutation test is not distribution-free: Testing H0: p = 0. Psychological Methods, 1, 184–198.
5.
KennyD. A.JuddC. M. (1986) Consequences of violating the independence assumption in the analysis of variance. Psychological Bulletin, 99, 422–432.
6.
SchefféH. (1959) The analysis of variance. New York: Wiley.
7.
ToothakerL. E.BanzM.NobleC.CampJ.DavisD. (1983) N=1 designs: The failure of ANOVA-based tests. Journal of Educational Statistics, 8, 289–309.
8.
WalshJ. E. (1947) Concerning the effect of intraclass correlation on certain significance tests. Annals of Mathematical Statistics, 18, 88–96.
9.
ZimmermanD. W.WilliamsR. H.ZumboB. D. (1993) Effect of nonindependence of sample observations on some parametric and nonparametric statistical tests. Communications in Statistics: Simulation and Computation, 22,779–789.
