The Least Biased d to r pb - r pb to d Conversion Formula Combinations for the Meta-Analysis of Studies Using the Two Independent-Groups Design

Abstract

In a meta-analysis of ds, the utility of converting d to a point-biserial correlation (r_pb), correcting r_pb for study artifacts, and then converting r_pb back to d depends on how accurately the d to r_pb-r_pb to d (dr-rd) method reproduces the original uncorrected ds. This study compared test ds (d_t) converted to dr-rd ds to control ds (d_c) to determine the dr-rd formula combinations with the least bias, defined as the difference between dr-rd ds and d_c. d_t were calculated with available conversion and r_pb correction formulas at various base rates and SD ratios, whereas d_c were calculated without conversion formulas at base rate .50 and SD ratio 1.00. Results showed that the lowest bias formula combinations reproduced d_c at SD ratio 1.00, bias increased as SD ratio decreased, and formula combinations that produced the lowest bias values were, on average, more accurate than formula combinations that produced the next to lowest bias values. Findings provide the lowest bias dr-rd formula combinations for artifact-corrected meta-analyses of studies using the two independent-groups design.

Keywords

Base rate bias effect size meta-analysis

Get full access to this article

View all access options for this article.

References

Aaron

Kromrey

J. D.

Ferron

(1998). Equating r-based and d-based effect size indices: Problems with a commonly recommended formula [Paper presentation]. In Annual Meeting of the Florida educational research association, Orlando, FL, 3 November, 1998. https://eric.ed.gov/?id=ED433353

Becker

(1986). Correcting the point-biserial correlation for attenuation owing to unequal sample size. The Journal of Experimental Education, 55(1), 5–8. https://doi.org/10.1080/00220973.1986.10806427.

Becker

B. J.

(1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology, 41(2), 257–278. https://doi.org/10.1111/j.2044-8317.1988.tb00901.x.

Borenstein

Hedges

Higgins

Rothstein

(2009). Introduction to meta-analysis. Wiley.

Brannick

M. T.

Potter

S. M.

Benitez

Morris

S. B.

(2019). Bias and precision of alternate estimators in meta-analysis: Benefits of blending Schmidt-Hunter and Hedges approaches. Organizational Research Methods, 22(2), 1–25. https://doi.org/10.1177/1094428117741966.

Buchanan

Gillenwaters

Scofield

Valentine

(2019). MOTE: Effect size and confidence interval calculator. https://rdrr.io/cran/MOTE/man/omega.F.html.

Card

(2012). Applied meta-analysis for social science research. Guilford Press.

Carlson

K. D.

Schmidt

F. L.

(1999). Impact of experimental design on effect size: Findings from the research literature on training. Journal of Applied Psychology, 84(6), 851–862. https://doi.org/10.1037/0021-9010.84.6.851.

Cohen

(1965). Some statistical issues in psychological research. In Wolman

B. B.

(Ed.), Handbook of clinical psychology (pp. 95–121). Academic Press.

10.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

11.

Cumming

(2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge.

12.

Delacre

Leys

Mora

Y. L.

Lakens

(2019). Taking parametric assumptions seriously: Arguments for the use of Welch’s F-test instead of the classical F-test in one-way ANOVA. International Review of Social Psychology, 32(1), 1–12. https://doi.org/10.5334/irsp.198.

13.

Doncaster

C. P.

Spake

(2017). Correction for bias in meta-analysis of little-replicated studies. Methods in Ecology and Evolution, 9(3), 634–644. https://doi.org/10.1111/2041-210X.12927.

14.

Ellis

(2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge University Press.

15.

Grissom

R. J.

Kim

J. J.

(2012). Effect sizes for research: Univariate and multivariate applications (2nd ed.) Routledge.

16.

Hays

W. L.

(1994). Statistics for psychologists (5th ed.). Hartcourt Brace.

17.

Hedges

L. V.

(1981). Distribution theory for glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107.

18.

Hedges

L. V.

Olkin

(1985). Statistical methods for meta-analysis. Academic Press.

19.

Hopkins

W. G.

Marshall

S. W.

Batterham

A. M.

Hanin

(2009). Progressive statistics for studies in sports medicine and exercise science. Medicine & Science in Sports & Exercise, 41(1), 3–12. https://doi.org/10.1249/MSS.0b013e31818cb278.

20.

Hunter

Schmidt

(1990). Methods of meta-analysis: Correcting error and bias in research findings. Sage.

21.

Hunter

Schmidt

(2004). Methods of meta-Analysis: Correcting error and bias in research findings (2nd ed.). Sage.

22.

Keppel

Wickens

T. D.

(2004). Design and analysis: A researcher’s handbook (4th ed.). Pearson Prentice Hall.

23.

Keselman

H. J.

Algina

Lix

L. M.

Wilcox

R. R.

Deering

K. N.

(2008). A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. Psychological Methods, 13(2), 110–129. https://doi.apa.org/doi/10.1037/1082-989X.13.2.110.

24.

Keselman

H. J.

Huberty

C. J.

Lix

L. M.

Olejnik

Cribbie

R. A.

Donahue

Kowalchuk

R. K.

Lowman

L. L.

Petoskey

M. D.

Keselman

J. C.

Levin

J. R.

, (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68(3), 350–386. https:doi.org/10.3102/00346543068003350.

25.

Kraemer

H. C.

(2006). Correlation coefficients in medical research: From product moment correlation to the odds ratio. Statistical Methods in Medical Research, 15(6), 525–545. https://doi.org/10.1177/0962280206070650.

26.

Lakens

(2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(863), 1-11. https://doi.org/10.3389/fpsyg.2013.00863.

27.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Sage.

28.

McGrath

R. E.

Meyer

G. J.

(2006). When effect sizes disagree: The case of r and d. Psychological Methods, 11(4), 386–401. https://doi.org/10.1037/1082-989X.11.4.386.

29.

Morris

S. B.

DeShon

R. P.

(2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105–125. https://doi.org/10.1037/1082-989X.7.1.105.

30.

R Core Team . (2021). R: A language and environment for statistical computing. R foundation for statistical computing. https://www.R-project.org/.

31.

Rosenthal

(1991). Meta-analytic procedures for social research. Applied social research methods series (Vol 6). Sage.

32.

Rosenthal

Rosnow

R. L.

Rubin

D. B.

(2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge University Press.

33.

Ruscio

(2008). A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods, 13(1), 19–30. https://doi.org/10.1037/1082-989X.13.1.19.

34.

Sawilowsky

S. S.

(2009). New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 8(2), 597–599. https://doi.org/10.22237/jmasm/1257035100.

35.

Schmidt

Hunter

(2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Sage.

36.

Schulze

(2004). Meta-analysis: A comparison of approaches. Hogrefe & Huber.

37.

Tukey

(1977). Exploratory data analysis. Addison-Wesley.

38.

Wiernik

B. M.

Dahlke

J. A.

(2020). Obtaining unbiased results in meta-analysis: The importance of correcting for statistical artefacts. Advances in Methods and Practices in Psychological Science, 3(1), 94–123. https://doi.org/10.1177/2515245919885611.