Comparing Alternative Corrections for Bias in the Bias-Corrected Bootstrap Test of Mediation

Abstract

Although the bias-corrected (BC) bootstrap is an often-recommended method for testing mediation due to its higher statistical power relative to other tests, it has also been found to have elevated Type I error rates with small sample sizes. Under limitations for participant recruitment, obtaining a larger sample size is not always feasible. Thus, this study examines whether using alternative corrections for bias in the BC bootstrap test of mediation for small sample sizes can achieve equal levels of statistical power without the associated increase in Type I error. A simulation study was conducted to compare Efron and Tibshirani’s original correction for bias, z ₀, to six alternative corrections for bias: (a) mean, (b–e) Winsorized mean with 10%, 20%, 30%, and 40% trimming in each tail, and (f) medcouple (robust skewness measure). Most variation in Type I error (given a medium effect size of one regression slope and zero for the other slope) and power (small effect size in both regression slopes) was found with small sample sizes. Recommendations for applied researchers are made based on the results. An empirical example using data from the ATLAS drug prevention intervention study is presented to illustrate these results. Limitations and future directions are discussed.

Keywords

bias-corrected bootstrap tests of mediation corrections for bias indirect effects Winsorized means

Get full access to this article

View all access options for this article.

References

Bates

Eddelbuettal

(2013). Fast and elegant numerical linear algebra using the RcppEigen Package. Journal of Statistical Software, 52(5), 1–24. http://www.jstatsoft.org/v52/i05/

Bollen

K. A.

Stine

(1990). Direct and indirect effects: Classical and bootstrap estimates of variability. Sociological Methodology, 20, 115–140. https://doi.org/10.2307/271084

Bradley

J. V.

(1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144–152. https://doi.org/10.1111/j.2044-8317.1978.tb00581.x

Brys

Hubert

Struyf

(2003). A comparison of some new measures of skewness. In Dutter

Filzmoser

Gather

Rousseeuw

P. J.

(Eds.), Developments in robust statistics (pp. 98–113). Physica. https://doi.org/10.1007/978-3-642-57338-5_8

Brys

Hubert

Struyf

(2004). A robust measure of skewness. Journal of Computational and Graphical Statistics, 13(4), 996–1017. https://doi.org/10.1198/106186004X12632

Cheung

M. W. L.

(2007). Comparison of approaches to constructing confidence intervals for mediating effects using structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 14(2), 227–246. https://doi.org/10.1080/10705510709336745

Cohen

(1988). Statistical power analyses for the behavioral sciences (2nd ed.). Erlbaum.

Cox

M. G.

Kisbu-Sakarya

Miočević

MacKinnon

D. P.

(2013). Sensitivity plots for confounder bias in the single mediator model. Evaluation Review, 37(5), 405–431. https://doi.org/10.1177/0193841X14524576

Efron

Tibshirani

R. J.

(1993). An introduction to the bootstrap. Chapman & Hall.

10.

Fritz

M. S.

Kenny

D. A.

MacKinnon

D. P.

(2016). The combined effects of measurement error and omitting confounders in the single-mediator model. Multivariate Behavioral Research, 51, 681–697. https://doi.org/10.1080/00273171.2016.1224154

11.

Fritz

M. S.

MacKinnon

D. P.

(2007). Required sample size to detect the mediated effect. Psychological Science, 18(3), 233–239. https://doi.org/10.1111/j.1467-9280.2007.01882.x

12.

Fritz

M. S.

Taylor

A. B.

MacKinnon

D. P.

(2012). Explanation of two anomalous results in statistical mediation analysis. Multivariate Behavioral Research, 47(1), 61–87. https://doi.org/10.1080/00273171.2012.640596

13.

Fuller-Rowell

T. E.

Curtis

D. S.

El-Sheikh

Duke

A. M.

Ryff

C. D.

Zgierska

A. E.

(2017). Racial discrimination mediates race differences in sleep problems: A longitudinal analysis. Cultural Diversity and Ethnic Minority Psychology, 23(2), 165–173. https://doi.org/10.1037/cdp0000104

14.

Goldberg

Elliot

Clarke

G. N.

MacKinnon

D. P.

Moe

Zoref

Green

Wolf

S. L.

Greffrath

Miller

D. J.

Lapin

(1996). Effects of a multidimensional anabolic steroid prevention intervention: The Adolescents Training and Learning to Avoid Steroids (ATLAS) program. Journal of the American Medical Association, 276(19), 1555–1562. https://doi.org/10.1001/jama.1996.03540190027025

15.

Hayes

A. F.

Scharkow

(2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24(10), 1918–1927. https://doi.org/10.1177/0956797613480187

16.

Imai

Keele

Tingley

(2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334. https://doi.org/10.1037/a0020761

17.

James

L. R.

Brett

J. M.

(1984). Mediators, moderators, and tests for mediation. Journal of Applied Psychology, 69(2), 307–321. https://doi.org/10.1037/0021-9010.69.2.307

18.

Jorgensen

T. D.

Pornprasertmanit

Schoemann

A. M.

Rosseel

(2019). semTools: Useful tools for structural equation modeling. R package Version 0.5-2. https://CRAN.R-project.org/package=semTools

19.

Judd

C. M.

Kenny

D. A.

(1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5(5), 602–619. https://doi.org/10.1177/0193841X8100500502

20.

Kenny

D. A.

Kashy

Bolger

(1998). Data analysis in social psychology. In Gilbert

Fiske

Lindzey

(Eds.), Handbook of social psychology (4th ed., pp. 233–265). McGraw-Hill.

21.

Kisbu-Sakarya

MacKinnon

D. P

Miočević

(2014). The distribution of the product explains normal theory mediation confidence interval estimation. Multivariate Behavioral Research, 49(3), 261–268. https://doi.org/10.1080/00273171.2014.903162

22.

Lix

L. M.

Keselman

H. J.

(1998). To trim or not to trim: Tests of location equality under heteroscedasticity and nonnormality. Educational and Psychological Measurement, 58(3), 409–429. https://doi.org/10.1177/0013164498058003004

23.

Lomnicki

Z. A.

(1967). On the distribution of products of random variables. Journal of the Royal Statistical Society: Series B, 29(3), 513–524. https://doi.org/10.1111/j.2517-6161.1967.tb00713.x

24.

Lundgren

Dahl

Hayes

S. C.

(2008). Evaluation of mediators of change in the treatment of epilepsy with acceptance and commitment therapy. Journal of Behavioral Medicine, 31, 225–235. http://doi.org/10.1007/s10865-008-9151-x

25.

MacKinnon

D. P.

Dwyer

J. H.

(1993). Estimating mediating effects in prevention studies. Evaluation Review, 17, 144–158. https://doi.org/10.1177/0193841X9301700202

26.

MacKinnon

D. P.

Fairchild

A. J.

Fritz

M. S.

(2007). Mediation analysis. Annual Review of Psychology, 58, 593–614. https://doi.org/10.1146/annurev.psych.58.110405.085542

27.

MacKinnon

D. P.

Fritz

M. S.

Williams

Lockwood

C. M.

(2007). Distribution of the product confidence limits for the indirect effect: Program PRODCLIN. Behavior Research Methods, 39(3), 384–389. https://doi.org/10.3758/BF03193007

28.

MacKinnon

D. P.

Goldberg

Clarke

G. N.

Elliot

D. L.

Cheong

Lapin

Moe

E. L.

Krull

J. L.

(2001). Mediating mechanisms in a program to reduce intentions to use anabolic steroids and improve exercise self-efficacy and dietary behavior. Prevention Science, 2(1), 15–28. https://doi.org/10.1023/A:1010082828000

29.

MacKinnon

D. P.

Lockwood

C. M.

Hoffman

J. M.

West

S. G.

Sheets

(2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7(1), 83–104. https://doi.org/10.1037//1082-989X.7.1.83

30.

MacKinnon

D. P.

Lockwood

C. M.

Williams

(2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39(1), 99–128. https://doi.org/10.1207/s15327906mbr3901_4

31.

MacKinnon

D. P.

Valente

M. J.

Gonzalez

(2020). The correspondence between causal and traditional mediation analysis: The link is the mediator by treatment interaction. Prevention Science, 21, 147–157. https://doi.org/10.1007/s11121-019-01076-4

32.

MacKinnon

D. P.

Warsi

Dwyer

J. H.

(1995). A simulation study of mediated effect measures. Multivariate Behavioral Research, 30(1), 41–62. https://doi.org/10.1207/s15327906mbr3001_3

33.

Mallinckrodt

Abraham

W. T.

Wei

Russell

D. W.

(2006). Advances in testing the statistical significance of mediation effects. Journal of Counseling Psychology, 53(3), 372–378. https://doi.org/10.1037/0022-0167.53.3.372

34.

Maruska

Hansen

Hanewinkel

Isensee

(2016). The role of substance-specific skills and cognitions in the effectiveness of a school-based prevention program on smoking incidence. Evaluation & the Health Professions, 39(3), 336–355. https://doi.org/10.1177/0163278715588825

35.

McManus

Surawy

Muse

Vazquez-Montes

Williams

J. M. G.

(2012). A randomized clinical trial of mindfulness-based cognitive therapy versus unrestricted services for health anxiety (hypochondriasis). Journal of Consulting and Clinical Psychology, 80(5), 817–828. https://doi.org/10.1037/a0028782

36.

Meeker

W. Q.

Jr Cornwell

L. W.

Aroian

L. A.

(1981). Selected tables in mathematical statistics, volume VII: The product of two normally distributed random variables. American Mathematical Society.

37.

Pearl

(2001). Direct and indirect effects. In Breese

Koller

(Eds.), Proceedings of the 17th conference on uncertainty in artificial intelligence (pp. 411–420). Morgan Kaufmann.

38.

Preacher

K. J.

Hayes

A. F.

(2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 36(4), 717–731. https://doi.org/10.3758/BF03206553

39.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. https://doi.org/10.3758/BRM.40.3.879

40.

Preacher

K. J.

Selig

J. P.

(2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6(2), 77–89. https://doi.org/10.1080/19312458.2012.679848

41.

R Core Team. (2019). R: A language and environment for statistical computing (Version 3.6.2) [Computer Software]. R Foundation for Statistical Computing. https://www.R- project.org/

42.

Revelle

(2018). psych: Procedures for personality and psychological research. Northwestern University. https://CRAN.R-project.org/package=psychVersion=1.8.12

43.

Segaert

Hubert

Rousseeuw

Raymaekers

(2019). mrfDepth: Depth measures in multivariate, regression and functional settings. R package Version 1.0.11. https://CRAN.R-project.org/package=mrfDepth

44.

Sella

Sader

Lolliot

Kadosh

R. C.

(2016). Basic and advanced numerical performances relate to mathematical expertise but are fully mediated by visuospatial skills. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(9), 1458–1472. http://dx.doi.org/10.1037/xlm0000249

45.

Shrout

P. E.

Bolger

(2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422–445. https://doi.org/10.1037/1082-989X.7.4.422

46.

Sobel

M. E.

(1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312. http://dx.doi.org/10.2307/270723

47.

Springer

M. D.

Thompson

W. E.

(1966). The distribution of products of independent random variables. SIAM Journal on Applied Mathematics, 14(3), 511–526. https://doi.org/10.1137/0114046

48.

Stone

C. A.

Sobel

M. E.

(1990). The robustness of estimates of total indirect effects in covariance structure models estimated by maximum likelihood. Psychometrika, 55, 337–352. https://doi.org/10.1007/BF02295291

49.

Tallman

B. A.

Altmaier

Garcia

(2007). Finding benefit from cancer. Journal of Counseling Psychology, 54(4), 481–487. https://doi.org/10.1037/0022-0167.54.4.481

50.

Tingley

Yamamoto

Hirose

Keele

Imai

(2014). Mediation: R package for causal mediation analysis. Journal of Statistical Software, 59(5), 1–38. https://doi.org/10.18637/jss.v059.i05

51.

VanderWeele

T. J.

(2010). Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology, 21, 540–551. https://doi.org/10.1097/EDE.0b013e3181df191c

52.

Wickham

(2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag.

53.

Wickham

(2017). tidyverse: Easily install and load the ‘Tidyverse’. R package version 1.2.1. https://CRAN.R-project.org/package=tidyverse

54.

Wilcox

R. R.

(1995). ANOVA: The practical importance of heterscedastic methods, using trimmed means versus means, and designing simulation studies. British Journal of Mathematical and Statistical Psychology, 48, 99–114. https://doi.org/10.1111/j.2044-8317.1995.tb01052.x

55.

Wilcox

R. R.

Keselman

H. J.

Kowalchuk

R. K.

(1998). Can tests for treatment group equality be improved?: The bootstrap and trimmed means conjecture. British Journal of Mathematical and Statistical Psychology, 51, 123–134. https://doi.org/10.1111/j.2044-8317.1998.tb00670.x

56.

Williams

MacKinnon

D. P.

(2008). Resampling and distribution of the product methods for testing indirect effects in complex models. Structural Equation Modeling: A Multidisciplinary Journal, 15(1), 23–51. https://doi.org/10.1080/10705510701758166

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.01 MB