We compare the sample size requirements for significance tests and confidence intervals by calculating power of each. The power of confidence interval is defined as the probability of obtaining a short interval width conditional on that the confidence interval includes the parameter of interest. We find that a smaller sample size is required to attain a desired statistical power as compared to comparable power of confidence interval in the two sample independent t test, which is illustrated in an example study that examines the outcome difference between psychotherapy and control in treating depression.
AgrestiA.FinlayB. (2008). Statistical methods for the social sciences (4th ed.). Upper Saddle River, NJ: Prentice Hall.
2.
American Psychological Association. (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
3.
BealS. L. (1989). Sample size determination for confidence intervals on the population mean and on the difference between two population means. Biometrics, 45, 969–977.
4.
BlandJ. M. (2009). The tyranny of power: Is there a better way to calculate sample size?British Medical Journal, 339, 1133–1135.
5.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
6.
CohenJ. (1992). A power primer. Psychological Bulletin, 112, 155–159.
7.
CohenJ. (1994). The Earth is round (p < .05). American Psychologist, 49, 997–1003.
8.
CummingG.FidlerF.LeonardM.KalinowskiP.ChristiansenA.KleinigA.WilsonS. (2007). Statistical reform in psychology: Is anything changing?Psychological Science, 18, 230–232.
9.
HahnG.MeekerW. (1991). Statistical intervals: A guide for practitioners. New York, NY: Wiley.
10.
HsuJ. C. (1988). Sample size computation for designing multiple comparison experiments. Computational Statistics and Data Analysis, 7, 79–91.
11.
HunterJ. E. (1997). Needed: A ban on the significance test. Psychological Science, 8, 3–7.
12.
International Committee of Medical Journal Editors. (1988). Uniform requirements for manuscripts submitted to biomedical journals. Annals of Internal Medicine, 108, 258–265.
13.
Lehmann. (1959). Testing statistical hypotheses. New York, NY: Wiley.
14.
LevinJ. R. (1993). Statistical significance testing from three perspectives. Journal of Experimental Education, 61, 378–382.
15.
LevinJ. R.RobinsonD. H. (1999) Further reflections on hypothesis testing and editorial policy for primary research journals. Educational Psychological Review, 11, 143–155.
16.
LiuX. (2012). Implications of statistical power for confidence intervals. British Journal of Mathematical and Statistical Psychology, 65, 427–437.
17.
RossiJ. S. (1990). Statistical power of psychological research: What have we gained in 20 years?Journal of Consulting and Clinical Psychology, 58, 646–656.
18.
SedlmeierP.GigerenzerG. (1989). Do studies of statistical power have an effect on the power of studies?Psychological Bulletin, 105, 309–316.
19.
SinkC. A.MvududuN. H. (2010). Statistical power, sampling, and effect sizes: Three keys to research relevancy. Counseling Outcome Research and Evaluation1, 1–18.
20.
ThompsonB. (2002). “Statistical,” “practical,” and “clinical”: How many kinds of significance do counselors need to consider?Journal of Counseling and Development, 80, 64–71.
21.
ThompsonB. (2006). Foundations of behavioral statistics: An insight-based approach. New York, NY: Guilford.
22.
WainerH. (1999). One cheer for null hypothesis significance testing. Psychological Methods, 4, 212–213.
23.
WilkinsonL.American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 494–604.
