The widespread practice of substituting an empirical estimate of effect size in the standard sample size formula causes the average power of studies to fall below the nominal level,oftenby a considerable margin. Ordinarily concealed from view in an individual experiment, the power loss becomes visible when studies are examined collectively. To help researchers avoid the power deficit, the expected-power method of sample size determination is described. When information from an earlier study is available, it is recommended that researchers should base sample size calculations on the expected-power formula instead of the traditional power formula.
Get full access to this article
View all access options for this article.
References
1.
Adcock, C. J. (1988). A Bayesian approach to calculating sample sizes. The Statistician, 37, 433-439.
2.
Brown, B. W. , Herson, J., Atkinson, J. N., & Rozell, M. E. (1987). Projection from previous studies: A Bayesian and frequentist compromise. Controlled Clinical Trials, 8, 29-44.
3.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
4.
DasGupta, A. , & Vidakovic, B. (1997). Sample size problems in ANOVA Bayesian point of view. Journal of Statistical Planning and Inference, 65, 335-347.
5.
Gillett, R. (1986). Sample size determination in replication attempts: The standard normal z test. British Journal of Mathematical and Statistical Psychology, 39, 190-207.
6.
Gillett, R. (1995). The expected power of F and t tests conditional on information from an earlier study. British Journal of Mathematical and Statistical Psychology, 48, 371-384.
7.
Gillett, R. (1996). Sample size determination in a chi-squared test given information from a previous study. Journal of Educational and Behavioral Statistics, 21, 230-246.
8.
Gillett, R. (2001). Sample size determination for a t test given a t value from a previous study: A FORTRAN 77 program. Behavior Research Methods, Instruments & Computers, 33, 544-548.
9.
Haase, R. F. , Waechter, D. M., & Solomon, G. S. (1982). How significant is a significant difference? Average effect size of research in counseling psychology. Journal of Counseling Psychology, 29, 58-65.
10.
Hedges, L. V. (1984). Estimation of effect size under nonrandom sampling: The effect of censoring studies yielding insignificant mean differences. Journal of Educational Statistics, 9, 61-85.
11.
Joseph, L. , & Wolfson, D. B. (1997). Interval-based versus decision theoretic criteria for the choice of sample size. The Statistician, 46, 145-149.
12.
Kazdin, A. E. , & Bass, D. (1989). Power to detect differences between alternative treatments in comparative psychotherapy outcomes. Journal of Consulting and Clinical Psychology, 57, 138-147.
13.
Kraemer, H. C. , & Thiemann, S. (1987). How many subjects?Statistical power analysis in research. Newbury Park, CA: Sage.
14.
Lane, D. M. , & Dunlap, W. P. (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical and Statistical Psychology, 31, 107-112.
15.
Murphy, K. R. , & Myors, B. (1998). Statistical power analysis. Mahwah, NJ: Lawrence Erlbaum.
16.
Overall, J. (1969). Classical statistical hypothesis testing within the context of Bayesian theory. Psychological Bulletin, 71, 285-292.
17.
Smith, M. L. (1980). Publication bias and meta-analysis. Evaluation in Education, 4, 22-24.
18.
Spiegelhalter, D. J. , & Freedman, L. S. (1986). A predictive approach to selecting the size of a clinical trial, based on subjective clinical opinion. Statistics in Medicine, 5, 1-13.
19.
Stallard, N. (1998). Sample size determination for phase II clinical trials based on Bayesian decision theory. Biometrics, 54, 279-294.
