Kelley compared three methods for setting a confidence interval (CI) around Cohen's standardized mean difference statistic: the noncentral-t-based, percentile (PERC) bootstrap, and biased-corrected and accelerated (BCA) bootstrap methods under three conditions of nonnormality, eight cases of sample size, and six cases of population effect size (ES) magnitude. Kelley recommended the BCA bootstrap method. The authors expand on his investigation by including additional cases of nonnormality. Like Kelley, they find that under many conditions, the BCA bootstrap method works best; however, they also find that in some cases of nonnormality, the method does not control probability coverage. The authors also define a robust parameter for ES and a robust sample statistic, based on trimmed means and Winsorized variances, and cite evidence that coverage probability for this parameter is good over the range of nonnormal distributions investigated when the PERC bootstrap method is used to set CIs for the robust ES.
Algina, J., Keselman, H. J., & Penfield, R. D. (2005a). An alternative to Cohen's standardized mean difference effect size: A robust parameter and confidence interval in the two independentgroupscase. Psychological Methods, 10, 317-328.
2.
Algina, J., Keselman, H. J., & Penfield, R. D. (2005b).Effect sizes and their intervals: The two-groupscorrelated case. Educational and Psychological Measurement, 65, 241-258.
3.
Bradley, J. V. (1978). Robustness?British Journalof Mathematical& Statistical Psychology, 31, 144-152.
4.
Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
5.
Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-574.
6.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
7.
Fleishman, A. I. (1978) A method for simulating non-normal distributions. Psychometrika, 43, 521-532.
8.
Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1986). Robust statistics. New York: John Wiley.
9.
Hoaglin, D. C. (1985). Summarizing shape numerically: The g-and h distributions. In D. C. Hoaglin, F. Mosteller, & J. W. Tukey (Eds.), Exploring data tables, trends, and shapes (pp. 461-513). New York: John Wiley.
10.
Huber, P. J. (1981). Robust statistics. New York: John Wiley.
11.
Kelley, K. (2005). The effects of nonnormal distributions on confidence intervals around the standardized mean difference: Bootstrap and parametric confidence intervals. Educational and Psychological Measurement, 65, 51-69.
12.
SAS Institute. (1999). SAS/IML user's guide, version 8. Cary, NC: Author.
13.
Staudte, R. G., & Sheather, S. J. (1990). Robust estimation and testing. New York: John Wiley.
14.
Steiger, J. H., & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. Harlow, S. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests?(pp. 221-257). Hillsdale, NJ: Lawrence Erlbaum.
15.
Wilcox, R. R. (2003). Applying contemporary statistical techniques. San Diego, CA: Academic Press.
16.
Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.). San Diego, CA: Academic Press.
17.
Wilcox, R. R., & Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8, 254-274.
18.
Yuen, K. K., & Dixon, W. J. (1973). The approximate behaviour and performance of the two-sample trimmed t. Biometrika, 60, 369-374.
