With the increase of online journalism, embedded multimedia stories have become more popular. Yet, little is known about the cognitive and affective effects this journalistic format may have on the audience. This experimental study compares the effects of embedded multimedia, traditional multimedia, and text-only format on readers’ knowledge gain, emotional reactions, and narrative transportation. Overall, the effects are substantially less pronounced than expected. The audiences’ emotional reactions and narrative transportation do not depend on modality, whereas knowledge gain is slightly decreased by multimodality. The theoretical, practical, and methodological implications of these limited effects are discussed.
BaekY. M.WojcieszakM. E. (2009). Don’t expect too much! Learning from late-night comedy and knowledge item difficulty. Communication Research, 36, 783-809. doi:10.1177/0093650209346805
2.
BarryA. M. (1997). Visual intelligence: Perception, image, and manipulation in visual communication. Albany: State University of New York Press.
3.
BaumM. A. (2003). Soft news and political knowledge: Evidence of absence or absence of evidence?Political Communication, 20, 173-190. doi:10.1080/10584600390211181
4.
BerenbaumH.FujitaF.PfenningJ. (1995). Consistency, specificity, and correlates of negative emotions. Journal of Personality and Social Psychology, 68, 342-352. doi:10.1037/0022-3514.68.2.342
5.
BilandzicH.BusselleR. W. (2008). Transportation and transportability in the cultivation of genre-consistent attitudes and estimates. Journal of Communication, 58, 508-529. doi:10.1111/j.1460-2466.2008.00397.x
6.
BlessH.MackieD. M.SchwarzN. (1992). Mood effects on attitude judgments: Independent effects of mood before and after message elaboration. Journal of Personality and Social Psychology, 63, 585-595. doi:10.1037/0022-3514.63.4.585
7.
BobrowD.NormanD. (1975). Some principles of memory schemata. In BobrowD.CollinsA. (Eds.), Representation and understanding: Studies in cognitive science (pp. 131-149). New York, NY: Academic Press.
8.
BodenhausenG.SheppardL.KramerG. (1994). Negative affect and social judgment: The differential impact of anger and sadness. European Journal of Social Psychology, 24, 45-62. doi:10.1002/ejsp.2420240104
9.
BraderT. (2005). Striking a responsive chord: How campaign ads motivate and persuade voters by appealing to emotions. American Journal of Political Science, 49, 388-405. doi:10.1111/j.0092-5853.2005.00130.x
10.
BraselS. A.GipsJ. (2011). Media multitasking behavior: Concurrent television and computer usage. Cyberpsychology, Behavior, and Social Networking, 14, 527-534. doi:10.1089/cyber.2010.0350
11.
Delli CarpiniM. (2004). Mediating democratic engagement: The impact of communications on citizens’ involvement in political and civic life. In KaidL. L. (Ed.), Handbook of political communication research (pp. 357-394). Mahwah, NJ: Lawrence Erlbaum.
12.
DetenberB. H.SimonsR. F.BennettG. G.Jr. (1998). Roll ’em!: The effects of picture motion on emotional responses. Journal of Broadcasting & Electronic Media, 42, 113-127. doi:10.1080/08838159809364437
13.
DeuzeM. (2004). What is multimedia journalism?Journalism Studies, 5, 139-152. doi:10.1080/1461670042000211131
14.
DowlingD.VoganT. (2014). Can we “Snowfall” this? Digital longform and the race for the tablet market. Digital Journalism, 3, 1-16. doi:10.1080/21670811.2014.930250
15.
DurkinS.WakefieldM. (2008). Interrupting a narrative transportation experience: Program placement effects on responses to antismoking advertising. Journal of Health Communication, 13, 667-680. doi:10.1080/10810730802412248
16.
DwyerF. M. (1978). Strategies for improving visual learning: A hand book for the effective selection, design, and use of visualized materials. State College, PA: Learning Services.
17.
EvelandW. P.Jr.ScheufeleD. A. (2000). Connecting news media use with gaps in knowledge and participation. Political Communication, 17, 215-237. doi:10.1080/105846000414250
FrijdaN. H.KuipersP.ter SchureE. (1989). Relations among emotion, appraisal, and emotional action readiness. Journal of Personality and Social Psychology, 57, 212-228. doi:10.1037/0022-3514.57.2.212
20.
GraberD. A. (1996). Say it with pictures. The ANNALS of the American Academy of Political and Social Science, 546, 85-96.
GreenM. C.BrockT. C. (2000). The role of transportation in the persuasiveness of public narratives. Journal of Personality and Social Psychology, 79, 701-721. doi:10.1037/0022-3514.79.5.701
23.
GreenM. C.BrockT. C.KaufmanG. F. (2004). Understanding media enjoyment: The role of narrative engagement into narrative worlds. Communication Theory, 14, 311-327. doi:10.1111/j.1468-2885.2004.tb00317.x
24.
GreenM. C.KassS.CarreyJ.HerzigB.FeeneyR.SabiniJ. (2008). Transportation across media: Repeated exposure to print and film. Media Psychology, 11, 512-539. doi:10.1080/15213260802492000
HarperC. (2005). Journalism in a digital age. Health, 34(27), 29-38.
27.
HayesA. F. (2013). Conditional process modeling: Using structural equation modeling to examine contingent causal processes. In HancockG. R.MuellerR. O. (Eds.), Structural equation modeling: A second course (2nd ed., pp. 217-264). Greenwich, CT: Information Age.
28.
HernandezR.RueJ. (2015). The principles of multimedia journalism: Packaging digital news. New York, NY: Routledge.
29.
HillC. A. (2004). The psychology of rhetorical images. In HillC. A.HelmersM. H. (Eds.), Defining visual rhetorics (pp. 25-40). Mahwah, NJ: Lawrence Erlbaum.
30.
HuddyL.FeldmanS.CasseseE. (2007). On the distinct political effects of anxiety and anger. In NeumanW. R.MarcusG. E.CriglerA. N.MacKuenM. (Eds.), The affect effect: Dynamics of emotion in political thinking and behavior (pp. 202-230). Chicago, IL: University of Chicago Press.
31.
HutchingsV. L.ValentinoN. A.PhilpotT. S.WhiteI. K. (2006). Racial cues in campaign news: The effects of candidate strategies on group activation and political attentiveness among African Americans. In RedlawskD. P. (Ed.), Feeling politics: Emotion in political information processing (pp. 165-186). New York, NY: Palgrave Macmillan.
32.
JacobsonS. (2010). Emerging models of multimedia journalism: A content analysis of multimedia packages published on nytimes.com. Atlantic Journal of Communication, 18(2), 63-78. doi:10.1080/15456870903554882
33.
JacobsonS. (2012). Transcoding the news: An investigation into multimedia journalism published on nytimes.com 2000-2008. New Media & Society, 14, 867-885. doi:10.1177/1461444811431864
KwakN. (1999). Revisiting the knowledge gap hypothesis education, motivation, and media use. Communication Research, 26, 385-413. doi:10.1177/009365099026004002
37.
LaMarreH. L.LandrevilleK. D. (2009). When is fiction as good as fact? Comparing the influence of documentary and historical reenactment films on engagement, affect, issue interest, and learning. Mass Communication and Society, 12, 537-555. doi:10.1080/15205430903237915
38.
LangA. (2000). The limited capacity model of mediated message processing. Journal of Communication, 50, 46-70. doi:10.1111/j.1460-2466.2000.tb02833.x
39.
Larrondo UretaA. (2011). The potential of web-only feature stories: A case study of Spanish media sites. Journalism Studies, 12, 188-204. doi:10.1080/1461670X.2010.491300
40.
LevieW. H. (1987). Research on pictures: A guide to the literature. In WillowsD. M.HoughtonH. A. (Eds.), The psychology of illustration (pp. 1-50). New York, NY: Springer-Verlag.
41.
LevieW. H.LentzR. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30, 195-232. doi:10.1007/BF02765184
MarcusG. E.MacKuenM. B. (1993). Anxiety, enthusiasm, and the vote: The emotional underpinnings of learning and involvement during presidential campaigns. American Political Science Review, 87, 672-685. doi:10.2307/2938743
44.
MarcusG. E.NeumanW. R.MacKuenM. (2000). Affective intelligence and political judgment. Chicago, IL: University of Chicago Press.
45.
MooreD. M.BurtonJ. K.MyersR. J. (2004). Multiple-channel communication: The theoretical and research foundations of multimedia. In JonassenD. H. (Ed) Handbook of research for educational communications and technology (pp. 979-1005). Mahwah, NJ: Lawrence Erlbaum.
46.
MoyP.XenosM. A.HussainM. M. (2013). News and political entertainment effects on democratic citizenship. The International Encyclopedia of Media Studies. Advance online publication. doi:10.1002/9781444361506.wbiems126
47.
MurphyS. T.FrankL. B.MoranM. B.Patnoe-WoodleyP. (2011). Involved, transported, or emotional? Exploring the determinants of change in knowledge, attitudes, and behavior in entertainment-education. Journal of Communication, 61, 407-431. doi:10.1111/j.1460-2466.2011.01554.x
48.
NabiR. L. (1999). A cognitive-functional model for the effects of discrete negative emotions on information processing, attitude change, and recall. Communication Theory, 9, 292-320. doi:10.1111/j.1468-2885.1999.tb00172.x
49.
NabiR. L. (2003). The framing effects of emotion: Can discrete emotions influence information recall and policy preference?Communication Research, 30, 224-247.
50.
NoyesJ. M.GarlandK. J. (2008). Computer-vs. paper-based tasks: Are they equivalent?Ergonomics, 51, 1352-1375.
51.
OliverM. B.DillardJ. P.BaeK.TamulD. J. (2012). The effect of narrative news format on empathy for stigmatized groups. Journalism & Mass Communication Quarterly, 89, 205-224. doi:10.1177/1077699012439020
52.
PaivioA. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue canadienne de psychologie, 45, 255-287. doi:10.1037/h0084295
PouliotL.CowenP. S. (2007). Does perceived realism really matter in media effects?Media Psychology, 9, 241-259. doi:10.1080/15213260701285819
56.
PriorM. (2005). News vs. entertainment: How increasing media choice widens gaps in political knowledge and turnout. American Journal of Political Science, 49, 577-592. doi:10.1111/j.1540-5907.2005.00143.x
SalomonG. (1984). Television is “easy” and print is “tough”: The differential investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology, 76, 647-658. doi:10.1037/0022-0663.76.4.647
60.
SchillD. (2012). The visual image and the political image: A review of visual communication research in the field of political communication. Review of Communication, 12, 118-142. doi:10.1080/15358593.2011.653504
61.
SegelE.HeerJ. (2010). Narrative visualization: Telling stories with data. IEEE Transactions on Visualization and Computer Graphics, 16, 1139-1148. doi:10.1109/TVCG.2010.179
62.
SeverinW. (1967). Another look at cue summation. AV Communication Review, 15, 233-245. doi:10.1007/BF02768608
63.
SlaterM. D.RounerD.LongM. (2006). Television dramas and support for controversial public policies: Effects and mechanisms. Journal of Communication, 56, 235-252. doi:10.1111/j.1460-2466.2006.00017.x
64.
SonJ.ReeseS. D.DavieW. R. (1987). Effects of visual-verbal redundancy and recaps on television news learning. Journal of Broadcasting & Electronic Media, 31, 207-216. doi:10.1080/08838158709386658
SteensenS. (2009). Online feature journalism: A clash of discourses. Journalism Practice, 3, 13-29. doi:10.1080/17512780802560716
67.
SteensenS. (2010). Back to the feature: Online journalism as innovation, transformation and practice (Doctoral dissertation). University of Oslo, Norway.
68.
SundarS. S. (2000). Multimedia effects on processing and perception of online news: A study of picture, audio, and video downloads. Journalism & Mass Communication Quarterly, 77, 480-499. doi:10.1177/107769900007700302
69.
TiedensZ. (2001). The effect of anger on the hostile inferences of aggressive and nonaggressive people: Specific emotions, cognitive processing, and chronic accessibility. Motivation and Emotion, 25, 233-251. doi:10.1023/A:1012224507488
70.
Trauzettel-KlosinskiS.DietzK. (2012). Standardized assessment of reading performance: The new international reading speed texts IReST. Investigative Ophthalmology & Visual Science, 53, 5452-5461. doi:10.1167/iovs.11-8284
71.
ValentinoN. A.BanksA. J.HutchingsV. L.DavisA. K. (2009). Selective exposure in the internet age: The interaction between anxiety and information utility. Political Psychology, 30, 591-613. doi:10.1111/j.1467-9221.2009.00716.x
72.
ValentinoN. A.BraderT.GroenendykE. W.GregorowiczK.HutchingsV. L. (2011). Election night’s alright for fighting: The role of emotions in political participation. The Journal of Politics, 73, 156-170. doi:10.1017/S0022381610000939
73.
ValentinoN. A.HutchingsV. L.BanksA. J.DavisA. K. (2008). Is a worried citizen a good citizen? Emotions, political information seeking, and learning via the internet. Political Psychology, 29, 247-273. doi:10.1111/j.1467-9221.2008.00625.x
74.
Walma van der MolenJ. H.KlijnM. E. (2004). Recall of television versus print news: Retesting the semantic overlap hypothesis. Journal of Broadcasting & Electronic Media, 48, 89-107. doi:10.1207/s15506878jobem4801_5
WeberC. (2013). Emotions, campaigns, and political participation. Political Research Quarterly, 66, 414-428. doi:10.1177/1065912912449697
77.
WickensC. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3, 159-177. doi:10.1080/14639220210123806
78.
WojcieszakM. (2009). Three dimensionality: Taxonomy of iconic, linguistic, and audio messages in television news. Television & New Media, 10, 459-481. doi:10.1177/1527476409343798
79.
WojcieszakM.KimN. (2015). How to improve attitudes toward disliked groups: The effects of narrative versus numerical evidence on political persuasion. Communication Research. Advance online publication. doi:10.1177/0093650215618480
80.
YeykelisL.CummingsJ. J.ReevesB. (2014). Multitasking on a single device: Arousal and the frequency, anticipation, and prediction of switching between media content on a computer. Journal of Communication, 64, 167-192. doi:10.1111/jcom.12070
81.
ZiefleM. (1998). Effects of display resolution on visual performance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40, 554-568.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
