Mediation Analysis and Warranted Inferences in Media and Communication Research: Examining Research Design in Communication Journals From 1996 to 2017

Abstract

The number of studies employing mediation analysis has increased exponentially in the past two decades. Focusing on research design, this study examines 387 articles in the Journal of Communication, Human Communication Research, Communication Research, Journalism & Mass Communication Quarterly, and Media Psychology between 1996 and 2017. Findings show that while most studies report statistically significant indirect effects, they are inadequate to make causal inferences. Authors also often infer that they uncovered the “true” mediator(s) while alternative models and mediators are rarely acknowledged. Future studies should pay more attention to the role of research design and its implications for making causal inferences.

Keywords

mediation analysis indirect effect research design causal inference content analysis

Get full access to this article

View all access options for this article.

References

Appel

Schreiner

Haffmans

M.-B.

Richter

(2019). The mediating role of event-congruent emotions in narrative persuasion. Poetics, 77, 101385. https://doi.org/10.1016/j.poetic.2019.101385

Arroyo

Segrin

Harwood

(2014). Appearance-related communication mediates the link between self-objectification and health and well-being outcomes. Human Communication Research, 40, 463–482. https://doi.org/10.1111/hcre.12036

Balmas

(2014). When fake news becomes real. Communication Research, 41, 430–454. https://doi.org/10.1177/0093650212453600

Bartsch

Schneider

F. M.

(2014). Entertainment and politics revisited: How non-escapist forms of entertainment can stimulate political interest and information seeking. Journal of Communication, 64, 369–396. https://doi.org/10.1111/jcom.12095

Bélanger

J. J.

Schumpe

B. M.

Nociti

Moyano

Dandeneau

Chamberland

P. E.

Vallerand

R. J.

(2019). Passion and moral disengagement: Different pathways to political activism. Journal of Personality, 87, 1234–1249. https://doi.org/10.1111/jopy.12470

Boukes

Boomgaarden

H. G.

Moorman

de Vreese

C. H.

(2015). At odds: Laughing and thinking? The appreciation, processing, and persuasiveness of political satire. Journal of Communication, 65, 721–744. https://doi.org/10.1111/jcom.12173

Bullock

H. E.

Harlow

L. L.

Mulaik

S. A.

(1994). Causation issues in structural equation modeling research. Structural Equation Modeling: A Multidisciplinary Journal, 1, 253–267. https://doi.org/10.1080/10705519409539977

Bullock

J. G.

Green

D. P.

S. E.

(2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98, 550–558. https://doi.org/10.1037/a0018933

Bullock

J. G.

S. E.

Druckman

J. N.

Green

D. P.

Kuklinski

J. H.

Lupia

(2011). Mediation analysis is harder than it looks. In Druckman

J. N.

Green

Lupia

Kuklinski

J. H.

(Eds.), Cambridge handbook of experimental political science (pp. 508–522). Cambridge University Press. https://doi.org/10.1017/cbo9780511921452.035

10.

Chan

(2018). Reluctance to talk about politics in face-to-face and Facebook settings: Examining the impact of fear of isolation, willingness to self-censor, and peer network characteristics. Mass Communication and Society, 21, 1–23. https://doi.org/10.1080/15205436.2017.1358819

11.

Chan

Chen

H.-T.

Lee

F. L. F.

(2017). Examining the roles of mobile and social media in political participation: A cross-national analysis of three Asian societies using a communication mediation approach. New Media & Society, 19, 2003–2021. https://doi.org/10.1177/1461444816653190

12.

Chia

S. C.

(2010). How social influence mediates media effects on adolescents’ materialism. Communication Research, 37, 400–419. https://doi.org/10.1177/0093650210362463

13.

de Graaf

. (2014). The effectiveness of adaptation of the protagonist in narrative impact: Similarity influences health beliefs through self-referencing. Human Communication Research, 40, 73–90. https://doi.org/10.1111/hcre.12015

14.

Dienlin

Johannes

Bowman

N. D.

Masur

P. K.

Engesser

Kümpel

A. S.

… de Vreese

(2020). An agenda for open science in communication. Journal of Communication. Advance online publication. https://doi.org/10.1093/joc/jqz052

15.

Fiedler

Harris

Schott

(2018). Unwarranted inferences from statistical mediation tests—An analysis of articles published in 2015. Journal of Experimental Social Psychology, 75, 95–102. https://doi.org/10.1016/j.jesp.2017.11.008

16.

Fiedler

Schott

Meiser

(2011). What mediation analysis can (not) do. Journal of Experimental Social Psychology, 47, 1231–1236. https://doi.org/10.1016/j.jesp.2011.05.007

17.

Gabbiadini

Riva

Andrighetto

Volpato

Bushman

B. J.

(2016). Acting like a tough guy: Violent-sexist video games, identification with game characters, masculine beliefs, & empathy for female violence victims. PLOS ONE, 11, Article e0152121. https://doi.org/10.1371/journal.pone.0152121

18.

Gunther

A. C.

(1998). The persuasive press inference. Communication Research, 25, 486–504. https://doi.org/10.1177/009365098025005002

19.

Harwood

Qadar

Chen

C.-Y.

(2016). Harmonious contact: Stories about intergroup musical collaboration improve intergroup attitudes. Journal of Communication, 66, 937–959. https://doi.org/10.1111/jcom.12261

20.

Hayes

A. F.

(2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). Guilford Press.

21.

Hayes

A. F.

Krippendorff

(2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–89. https://doi.org/10.1080/19312450709336664

22.

Holbert

R. L.

Park

(2020). Conceptualizing, organizing, and positing moderation in communication research. Communication Theory, 30, 227–246. https://doi.org/10.1093/ct/qtz006

23.

Holbert

R. L.

Shah

D. V.

Kwak

(2004). Fear, authority, and justice: Crime-related TV viewing and endorsements of capital punishment and gun ownership. Journalism & Mass Communication Quarterly, 81, 343–363. https://doi.org/10.1177/107769900408100208

24.

Holbert

R. L.

Stephenson

M. T.

(2003). The importance of indirect effects in media effects research: Testing for mediation in structural equation modeling. Journal of Broadcasting & Electronic Media, 47, 556–572. https://doi.org/10.1207/s15506878jobem4704_5

25.

Hutchens

M. J.

Hmielowski

J. D.

Pinkleton

B. E.

Beam

M. A.

(2016). A spiral of skepticism? The relationship between citizens’ involvement with campaign information to their skepticism and political knowledge. Journalism & Mass Communication Quarterly, 93, 1073–1090. https://doi.org/10.1177/1077699016654439

26.

Imai

Keele

Yamamoto

(2010). Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science, 25, 51–71. https://doi.org/10.1214/10-sts321

27.

Johnson

B. K.

Slater

M. D.

Silver

N. A.

Ewoldsen

D. R.

(2016). Entertainment and expanding boundaries of the self: Relief from the constraints of the everyday. Journal of Communication, 66, 386–408. https://doi.org/10.1111/jcom.12228

28.

Kam

J. A.

(2011). The effects of language brokering frequency and feelings on Mexican-heritage youth’s mental health and risky behaviors. Journal of Communication, 61, 455–475. https://doi.org/10.1111/j.1460-2466.2011.01552.x

29.

Kline

R. B.

(2015). The mediation myth. Basic and Applied Social Psychology, 37, 202–213. https://doi.org/10.1080/01973533.2015.1049349

30.

Knobloch-Westerwick

Lavis

S. M.

(2017). Selecting serious or satirical, supporting or stirring news? Selective exposure to partisan versus mockery news online videos. Journal of Communication, 67, 54–81. https://doi.org/10.1111/jcom.12271

31.

Lecheler

Bos

Vliegenthart

(2015). The mediating role of emotions: News framing effects on opinions about immigration. Journalism & Mass Communication Quarterly, 92, 812–838. https://doi.org/10.1177/1077699015596338

32.

Martinot

Redersdorff

Guimond

Dif

(2016). Ingroup versus outgroup comparisons and self-esteem: The role of group status and ingroup identification. Personality and Social Psychology Bulletin, 28, 1586–1600. https://doi.org/10.1177/014616702237585

33.

Mathieu

J. E.

DeShon

R. P.

Bergh

D. D.

(2008). Mediational inferences in organizational research. Organizational Research Methods, 11, 203–223. https://doi.org/10.1177/1094428107310089

34.

McGregor

Haji

Kang

S.-J.

(2008). Can ingroup affirmation relieve outgroup derogation? Journal of Experimental Social Psychology, 44, 1395–1401. https://doi.org/10.1016/j.jesp.2008.06.001

35.

Mellado

Georgiou

Nah

(2020). Advancing journalism and communication research: New concepts, theories, and pathways. Journalism & Mass Communication Quarterly, 97, 333–341. https://doi.org/10.1177/1077699020917204

36.

Men

L. R.

Muralidharan

(2017). Understanding social media peer communication and organization–public relationships. Journalism & Mass Communication Quarterly, 94, 81–101. https://doi.org/10.1177/1077699016674187

37.

Morgan

Shanahan

(2017). Television and the cultivation of authoritarianism: A return visit from an unexpected friend. Journal of Communication, 67, 424–444. https://doi.org/10.1111/jcom.12297

38.

Neuman

W. R.

Guggenheim

(2011). The evolution of media effects theory: A six-stage model of cumulative research. Communication Theory, 21, 169–196. https://doi.org/10.1111/j.1468-2885.2011.01381.x

39.

Paek

H.-J.

Gunther

A. C.

(2007). How peer proximity moderates indirect media influence on adolescent smoking. Communication Research, 34, 407–432. https://doi.org/10.1177/0093650207302785

40.

Palomares

N. A.

(2013). When and how goals are contagious in social interaction. Human Communication Research, 39(1), 74–100. https://doi.org/10.1111/j.1468-2958.2012.01439.x

41.

Pirlott

A. G.

MacKinnon

D. P.

(2016). Design approaches to experimental mediation. Journal of Experimental Social Psychology, 66, 29–38. https://doi.org/10.1016/j.jesp.2015.09.012

42.

Pogge

Smith

C. T.

(2020). Dealing with government dysfunction: Perceived electoral system brokenness explains the effects of high and low perceived polarization on support for fixes. Journal of Experimental Social Psychology, 87, 103914. https://doi.org/10.1016/j.jesp.2019.103914

43.

Powell

T. E.

Boomgaarden

H. G.

De Swert

de Vreese

C. H.

(2015). A clearer picture: The contribution of visuals and text to framing effects. Journal of Communication, 65, 997–1017. https://doi.org/10.1111/jcom.12184

44.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891.

45.

Preacher

K. J.

Rucker

D. D.

Hayes

A. F.

(2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–227. https://doi.org/10.1080/00273170701341316

46.

Saleem

Prot

Anderson

C. A.

Lemieux

A. F.

(2017). Exposure to Muslims in media and support for public policies harming Muslims. Communication Research, 44, 841–869. https://doi.org/10.1177/0093650215619214

47.

Saylors

Trafimow

(2020). Why the increasing use of complex causal models is a problem: On the danger sophisticated theoretical narratives pose to truth. Organizational Research Methods. Advance online publication. https://doi.org/10.1177/1094428119893452

48.

Schmuck

Matthes

Paul

F. H.

(2017). Negative stereotypical portrayals of Muslims in right-wing populist campaigns: Perceived discrimination, social identity threats, and hostility among young Muslim adults. Journal of Communication, 67, 610–634. https://doi.org/10.1111/jcom.12313

49.

Shah

D. V.

Cho

Nah

Gotlieb

M. R.

Hwang

Lee

N.-J.

Scholl

R. M.

McLeod

D. M.

(2007). Campaign ads, online messaging, and participation: Extending the communication mediation model. Journal of Communication, 57, 676–703.

50.

Shehata

Amnå

(2019). The development of political interest among adolescents: A communication mediation approach using five waves of panel data. Communication Research, 46, 1055–1077. https://doi.org/10.1177/0093650217714360

51.

Slater

M. D.

Gleason

L. S.

(2012). Contributing to theory and knowledge in quantitative communication science. Communication Methods and Measures, 6, 215–236. https://doi.org/10.1080/19312458.2012.732626

52.

Slater

M. D.

Hayes

A. F.

Ford

V. L.

(2007). Examining the moderating and mediating roles of news exposure and attention on adolescent judgments of alcohol-related risks. Communication Research, 34, 355–381. https://doi.org/10.1177/0093650207302783

53.

Smith

E. R.

(2012). Editorial. Journal of Personality and Social Psychology, 102, 1–3. https://doi.org/10.1037/a0026676

54.

Sotirovic

McLeod

J. M.

(2001). Values, communication behavior, and political participation. Political Communication, 18, 273–300. https://doi.org/10.1080/10584600152400347

55.

Spencer

S. J.

Zanna

M. P.

Fong

G. T.

(2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes. Journal of Personality and Social Psychology, 89, 845–851. https://doi.org/10.1037/0022-3514.89.6.845

56.

Stone-Romero

E. F.

Rosopa

P. J.

(2007). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11, 326–352. https://doi.org/10.1177/1094428107300342

57.

Theiss

J. A.

Knobloch

L. K.

(2014). Relational turbulence and the post-deployment transition. Communication Research, 41, 27–51. https://doi.org/10.1177/0093650211429285

58.

Valkenburg

P. M.

Peter

Walther

J. B.

(2016). Media effects: Theory and research. Annual Review of Psychology, 67, 315–338. https://doi.org/10.1146/annurev-psych-122414-033608

59.

van Koningsbruggen

G. M.

Harris

P. R.

Smits

A. J.

Schüz

Scholz

Cooke

. (2016). Self-affirmation before exposure to health communications promotes intentions and health behavior change by increasing anticipated regret. Communication Research, 43, 1027–1044. https://doi.org/10.1177/0093650214555180

60.

van Oosten

J. M. F.

Peter

Valkenburg

P. M

. (2015). The influence of sexual music videos on adolescents’ misogynistic beliefs. Communication Research, 42, 986–1008. https://doi.org/10.1177/0093650214565893

61.

Walter

Cody

M. J.

Ball-Rokeach

S. J.

(2018). The ebb and flow of communication research: Seven decades of publication trends and research priorities. Journal of Communication, 68, 424–440. https://doi.org/10.1093/joc/jqx015

62.

Wojcieszak

Garrett

R. K.

(2018). Social identity, selective exposure, and affective polarization: How priming national identity shapes attitudes toward immigrants via news selection. Human Communication Research, 44, 247–273. https://doi.org/10.1093/hcr/hqx010

63.

Yanovitzky

Greene

(2009). Quantitative methods and causal inference in media effects research. In Nabi

R. L.

Oliver

M. B.

(Eds.), The SAGE handbook of media processes and effects (pp. 35–52). SAGE.