Rosenthal and Rubin introduced a general effect size index, requivalent, for use in meta-analyses of two-group experiments; it employs p values from reports of the original studies to determine an equivalent t test and the corresponding point-biserial correlation coefficient. The present investigation used Monte Carlo—simulated meta-analyses to examine the impact on requivalent effect sizes of research using independent-groups, pooled-variance t tests with that using a less powerful median test. As expected, estimates based on t were higher. These differences were consistent even in the presence of strong variance heterogeneity when data were distributed normally, but not when data were nonnormal. The results suggested that the use of requivalent be confined to combining studies using inferential tests with comparable power and robustness; they also cast doubt on the use of requivalent when data are not distributed normally.
Adams, H.E., Wright, L.W., Jr., & Lohr, B.A. (1996). Is homophobia associated with homosexual arousal?Journal of Abnormal Psychology,105, 440-445.
2.
Algina, J., & Keselman, H.J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63, 537-553.
3.
Becker, B.J. (1994). Combining significance levels. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 215-230). New York: Russell Sage.
4.
Clark-Carter, D. (2005). The importance of considering effect size and statistical power in research. In J. Miles & P. Gilbert (Eds.), A handbook of research methods for clinical and health psychology (pp. 185-192). New York: Oxford University Press.
5.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
6.
Cohen, J. (1992). A power primer. Psychological Bulletin,112, 155-159.
7.
Cohn, L.D., & Becker, B.J. (2003). How meta-analysis increases statistical power . Psychological Methods, 8, 243-253.
8.
Corey, D.M., Dunlap, W.P., & Burke, M.J. (1998). Averaging correlations: Expected values and bias in combined Pearson rs and Fisher's z transformations. Journal of General Psychology, 125, 245-261.
9.
Fedora Project. (2006). Fedora Core 6 [Computer software]. Available at http://fedora.redhat.com
10.
Field, A.P. (2005). Is the meta-analysis of correlation coefficients accurate when population correlations vary?Psychological Methods, 10, 444-467.
11.
Fisher, R.A. (1932). Statistical methods for research workers (4th ed.). London: Oliver & Boyd.
12.
Freidlin, B., & Gastwirth, J.L. (2000). Should the median test be retired from general use?American Statistician, 54, 161-164.
13.
Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
14.
Hedges, L.V., & Pigott, T.D. (2001). The power of statistical tests in meta-analysis . Psychological Methods, 6, 203-217.
15.
Howell, D.C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove, CA: Duxbury.
16.
Howell, D.C. (2004). Fundamental statistics for the behavioral sciences (5th ed.). Belmont, CA: Thomson-Brooks/Cole.
17.
Hsu, L.M. (2005). Some properties of requivalent: A simple effect size indicator. Psychological Methods, 10, 420-427.
18.
Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
19.
Kirk, R.E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
20.
Kirk, R.E. (2003). The importance of effect magnitude. In S. F. Davis (Ed.), Handbook of research methods in experimental psychology (pp. 83-105). Malden, MA: Blackwell Publishing.
21.
Kraemer, H.C. (2005). A simple effect size indicator for two-group comparisons? A comment on requivalent. Psychological Methods , 10, 413-419.
22.
MacCallum, R.C., Zhang, S., Preacher, K.J., & Rucker, D.D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19-40.
23.
Matsumoto, M., & Nishimura, T. (1998). Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator. ACM Transactions on Modeling and Computer Simulation, 8, 3-30.
24.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin,105, 156-166.
25.
Ray, J.W., & Shadish, W.R. (1996). How interchangeable are different estimators of effect size?Journal of Consulting and Clinical Psychology , 64, 1316-1325.
26.
R Development Core Team. (2004). R: A language and environment for statistical computing [Computer software] . Retrieved from http://www.R-project.org
27.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York: Russell Sage.
28.
Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bulletin,118, 183-192.
29.
Rosenthal, R., & Rubin, D.B. (1979). Comparing significance levels of independent studies . Psychological Bulletin,86, 1165-1168.
30.
Rosenthal, R., & Rubin, D.B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology , 74, 166-169.
31.
Rosenthal, R., & Rubin, D.B. (2003). requivalent: A simple effect size indicator . Psychological Methods, 8, 492-496.
32.
Rosnow, R.L., & Rosenthal, R. (2003). Effect sizes for experimenting psychologists. Canadian Journal of Experimental Psychology, 57, 221-237.
33.
Sachs, L. (1984). Applied statistics: A handbook of techniques (2nd ed.). New York: Springer-Verlag.
34.
Schmidt, F.L., & Hunter, J.E. (2003). Meta-analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Research methods in psychology (Vol. 2, pp. 533-554). Hoboken, NJ: John Wiley.
35.
Schulze, R., Holling, H., Grossman, H., Jutting, A., & Brocke, M. (2003). Differences in the results of two meta-analytical approaches. In R. Schulze, H. Holling, & D. Bohning (Eds.), Meta-analysis: New developments and applications in medical and social sciences (pp. 21-39). Ashland, OH: Hogrefe & Huber.
36.
Shadish, W.R., & Haddock, C.K. (1994). Combining estimates of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 261-281). New York: Russell Sage.
37.
Steiger, J.H. (2004). Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9, 164-182.
38.
Tippett, L.H.C. (1931). The method of statistics. London: Williams & Norgate.
39.
Winer, B.J. (1971). Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.
40.
Zimmerman, D.W. (1998). Invalidation of parametric and nonparametric statistical tests by concurrent violation of two assumptions. Journal of Experimental Education, 67, 55-68.
41.
Zimmerman, D.W., & Zumbo, B.D. (1992). Parametric alternatives to the Student t-test under violation of normality and homogeneity of variance. Perceptual and Motor Skills, 74, 835-844.
