Contested Certainty and Credibility: The Effect of Personal Stories and Scientific Evidence in User Comments on News Story Evaluation and Relevance

Abstract

This study examined how user comments influence perceptions of a less-controversial news story. The results of a 2 (argument direction: supporting vs. dissenting comments) × 2 (evidence type: anecdotal vs. scientific evidence referenced in comments) between-subjects factorial design experiment with a no-comments control group (N = 426) showed that comments have independent effects on the evaluation of medical science news stories on perceived relevance, uncertainty, and risk perception. Also, the types of comments interact with participants’ intellectual humility and subjective numeracy. The findings illustrate that comments may have a deleterious impact on audience perception of journalistic stories and scientific issues.

Keywords

science news user comments exemplification perceived uncertainty intellectual humility

Additionally, as increasing access to information via the worldwide web and other outlets allows individuals to range beyond their hometown media, readers will be able to assemble meaning on a grander scale by cobbling together stories about the same topic from a variety of places. It seems almost inevitable that such triangulation will make uncertainty a common take-home message. (Dunwoody, 1999, p. 68)

This profound observation made by Sharon Dunwoody in 1999 is made even more prescient when the date is considered. Dunwoody pinpointed the risk of internet-cultivated uncertainty years before online disrupters like social media, blogging, and filter bubbles created by algorithms went mainstream. Although her prediction was not apocalyptic in tone, the four horsemen of misinformation, conspiracy, denialism, and factual relativism all gain ground in a landscape of uncertainty. They have shattered norms of credibility and evidence to the point that all institutions, including those outside of science and health, are facing dire consequences. Furthermore, Dunwoody’s prediction noted how readers would be able to assemble meaning by triangulating various sources. We argue that user comments and the stories they address are one such example of how meaning and scientific certainty can be contested within the same digital information space. This experiment examines how anecdotal and evidence-based user comments influence reader perception of news about a less-controversial scientific topic: antibiotic resistance.

The very first audience comment section appeared in June 1999, the same year that Dunwoody presaged the spread of uncertainty through reader triangulation. The Fresno Bee added a “forum box” at the end of news stories and saw web traffic increase by 30% within days and remained there (Mostafa, 2001). The attraction of user comments, for people to add their opinion but even more to read the dialogue of others (Ksiazek & Springer, 2018), was a boon to a newspaper industry experiencing revenue losses from audience movement to free online news and the demise of the advertising business model (Deuze & Marjoribanks, 2009; Flew, 2012). As publishers added more comment sections and pushed journalists to manage them, scholars also celebrated user comments as democratic deliberation, even going so far as to admonish journalists for not embracing them (Borger et al., 2013; Thomas, 2021). As Thomas (2021) contends, “scholars were in curious alignment with news organization bosses in putting a normative cart before an empirical horse” (p. 236). Meanwhile, journalists themselves recognized that comments departed both from the journalistic mission and their democratic role in it. Retrospectively, and as more news site comment sections are becoming “digital dodos” (Thomas, 2021), we can see clearly that the adoption and growth of this most prominent form of public participation in journalism spawned new problems in the form of undermining journalistic credibility and sowing seeds of misinformation. By 2022, news organizations have largely dropped the forum on their own web pages, although news shared on social media platforms still has adjacent user comments.

Research has shown that user-generated comments can influence reader opinions about health and science news stories (Anderson et al., 2014; Flemming et al., 2017; Hinnant et al., 2016; Kareklas et al., 2015; Shi et al., 2014; Zhang & Wang, 2019). The current study is original because it examines the influence of comments on perceptions of a science news story that is less controversial—relative to topics like vaccination, smoking, or climate change, many of which have been successfully studied in the framework of comment effects. Although controversial science and health topics do generate a lot of attention and coverage, there are quantifiably more less-controversial topics overall. In choosing the less-controversial topic of antibiotic resistance, methodologically, we are able to better control any independent influence of the news topic on the study outcomes. The goal of this research is to reveal whether comments themselves can exert control over audience reactions and in fact generate polarity about an issue where little to none exists. Additionally, this research is original in that it analyzes the influence of a person’s subjective numeracy and intellectual humility as moderating their reactions. This study engages with exemplification theory and argument direction (supportive/dissenting comments) as they relate to the debatable value of participatory discourse about science and health via journalistic outlets.

Literature Review

Audience Comments: Participatory Paradise or Pandora’s Box?

For years, comment sections were extolled as a digital public sphere that would allow the decentralization and democratization of knowledge (Domingo et al., 2008; Heinonen, 2011; Reich, 2011). However, studies have shown that comment spaces are hubs for incivility (Anderson et al., 2014; Chen, 2017; Santana, 2014), bigotry (Antony & Thomas, 2017), and hate speech (Wilhelm & Joeckel, 2019). Furthermore, comments have played a key role in displacing journalistic authority to nonjournalists as part of the participatory media culture (Deuze & Marjoribanks, 2009; Flew, 2012). In studies on the persuasive power of comments, researchers have established that comments can shape thoughts and attitudes about issues (Lee & Tandoc, 2017), influence future stories (Luo, 2014), and spread misinformation (Glenski et al., 2020). Critically, comments are often not fact-checked before they appear because they are user-generated, not journalist-generated, which relates to their ability to spread misinformation. For the news websites that do moderate comments, there is no one way to do it. Options include pre-moderation, post-moderation, moderation through flagging by users, or a combination (Ksiazek & Springer, 2018). Moderation is typically focused on behavior or tone and not whether the comments are supported by facts. Even on platforms where content moderation focuses on factual support, misinformation can metastasize before and after an original misinformed comment has been removed (Gallo & Cho, 2021).

Although scores of news outlets had comment sections by the year 2007, The New York Times in fact took its first “baby steps” in launching its first comment section with a Science Times article (Hoyt, 2007). It is worth noting that the Times, then and now the nation’s paper of record, industry leader, and strong intermedia agenda-setter (Clark & Illman, 2003; Denham, 2014; Golan, 2006; Meeks, 2013), chose a science article for its first comment section. Only a few years later in 2010, the American Journalism Review urged news sites to stop allowing anonymous commenting (Rieder, 2010), which was the same year that Lee and Jang (2010) found that comments influence reader opinion, independent of the news story that the comments follow. As journalists and researchers have noted, user comments have played a critical role in undermining the scientific enterprise specifically (LaBarre, 2013), overshadowing science news’ quality (Scheufele & Krause, 2019) and the credibility of news (Gierth & Bromme, 2020). More specifically, comment sections on climate stories that challenge the reality of anthropogenic climate change have been shown to serve as echo chambers where users double down on their denialist beliefs through their anonymous comments (Walter et al., 2018).

Comments’ Evidence Type: Anecdotal or Scientific

The effect of comments on reader judgments depends on the content and valence of those user comments. In this study, we examine the anecdotal evidence commenters use to construct their arguments. Exemplification theory predicts that stories from individuals with personal experience of an issue affect outcomes such as risk perception and perceived susceptibility more than statistical evidence, like frequencies (Zillmann, 2006). This research considers the personal stories that commenters tell about their own experiences as a form of anecdotal evidence, which should bring the topic at hand to life for the participant through exemplification (Kim et al., 2012). Anecdotal evidence contrasts with empirical evidence gathered through scientific research.

There has been research on evidence type in comments in non-science stories, such as one that tested reactions to comments about high-controversy topics of immigration and same-sex marriage (Wojcieszak & Kim, 2016). They found that anecdotal or narrative evidence led to greater message acceptance and immersion, whereas numerical evidence did not. Pivoting to science and health news, Flemming et al. (2017) looked at anecdotal versus empirical evidence versus no evidence in comments as they responded to the high or low tentativeness (or uncertainty) of deep brain stimulation research findings as articulated in the science news story. They found that comments using anecdotal evidence to amplify the skepticism about the research findings led to greater perceived scientific tentativeness compared with comments without evidence. Skeptical comments with empirical evidence, on the contrary, did not differ from comments with no evidence. In looking at human papillomavirus (HPV) vaccination, Zhang and Wang (2019) looked at evidence type in comments (narrative vs. descriptive, which included scientific knowledge or statistical data) and valence in comments (in support of or against receiving the HPV vaccine). The narrative (or anecdotal) comments resulted in higher risk perception toward HPV but only among participants who read comments that dissented from the news story that was in support of HPV vaccination. Hinnant et al. (2016) found that anecdotal (vs. scientific) evidence in comments on a climate change news story increased perceptions of credibility and risk perception based on political ideology.

Based on previous research that shows a strong influence of the use of exemplars in comment sections on perceptions of the news story, such as perceived uncertainty, attitudes toward the news, and risk perception, we propose the following hypothesis:

Hypothesis 1 (H1): A medical science story with a comment section of anecdotal (vs. scientific) evidence will lead to (a) enhanced perceived relevance, (b) more perceived credibility, (c) greater story likeability, and (d) stronger willingness to share the story, (e) less perceived uncertainty, and (f) increased risk perception toward antibiotic-resistant infections.

Comments’ Argument Direction: Supportive Versus Dissenting

User comments following stories about science and health frequently either support or dissent from the dominant frame of the news story (Collins & Nerlich, 2015). People are more likely to leave comments on news stories that they disagree with because they want to offer corrective information (Rojas, 2010). Comments that take an oppositional approach to the science in the article can influence how people perceive the news article. Anderson et al. (2014) critically discovered that nasty or uncivil comments that are oppositional in tone, not to the science itself, affected the risk perception of emerging technology. Yeo et al. (2019) found that comment moderation could diminish perceptions of bias, thereby mitigating the “nasty effect” of incivility in commenting. Yeo et al.’s research adapts the idea of oppositionality and turns the focus to what happens when the comments show dissent from the news topic (and not to other people).

Several studies have looked at positive (i.e., pro or supportive) or negative (i.e., contra or dissenting) comments about vaccination. Peter et al. (2014) found user comments that were positive about flu vaccination led to a lower perception of vaccine risk, positive attitudes toward vaccination, and a stronger intention to be vaccinated than negative comments. Another study (Petit et al., 2021) found that negative user comments following a news story (fake vs. real) about vaccination lowered perceived news credibility (compared with those who saw positive user comments), and the finding was even more intense when negative comments followed a real news story. Similarly, a study by Winter et al. (2015) found negative comments influenced the persuasive strength of a news article about cannabis legalization, and generally, comments had more power than a story’s “likes” to influence personal attitudes. Negative comments have also been shown to reduce willingness to share a news article, perceived credibility, and agreement with and attitude toward the article and the ideas in it (Boot et al., 2021). Lastly, an article (Naab et al., 2020) that compared supportive versus critical comments after a news story about superfoods (appearing in either a quality newspaper or a sensationalist tabloid) found that critical comments decreased the news credibility. The effect of diminished credibility after reading negative comments was consistent across high- and low-quality news outlets (Naab et al., 2020).

Notably, journalistic articles about new scientific research often present the consensus on the issue and offer a background that supports the consensus instead of sourcing contradictory points, which is more typical in political news. This means that comment support or dissent from the news story and the science itself are one and the same in the current study. Based on the research about argument direction in comments and how critical or negative comments have shown consistent patterns in diminishing how participants perceived credibility and experienced persuasion from the news article and the science it covers, this study proposes a second hypothesis:

Hypothesis 2 (H2): A medical science story with supportive (vs. dissenting) comments will lead to (a) enhanced perceived relevance, (b) more perceived credibility, c) greater story likeability, (d) stronger willingness to share the story, (e) less perceived uncertainty, and (f) increased risk perception toward antibiotic-resistant infections.

Additionally, it is likely that interaction effects will emerge between evidence type and argument direction given that evidence type may amplify the difference between supportive and dissenting comments. Specifically, it is anticipated that anecdotal evidence would result in stronger attitudes toward the story and the issue, aligning with the argument direction compared with scientific evidence. However, because we are using a less-controversial news story for the stimuli, research questions are more appropriate:

Research Question 1 (RQ1): How do evidence type and argument direction jointly affect (a) perceived story relevance, (b) perceived story credibility, (c) story likeability, (d) willingness to share the story, (e) perceived uncertainty, and (f) risk perception toward antibiotic-resistant infections?

Research Question 2 (RQ2): Do the combined effects of evidence type and argument direction of a comment section perform better than a story without a comment section in terms of increasing (a) perceived story relevance, (b) perceived story credibility, (c) story likeability, (d) willingness to share the story, (e) reducing perceived uncertainty, and (f) increasing risk perception toward antibiotic-resistant infections?

Intellectual Humility and Subjective Numeracy

Scholars have looked at the need for cognition as a moderating variable that contributes to how responsive people are to user comments (Hinnant et al., 2016; Lee et al., 2021; Winter & Krämer, 2016). In this study, we explore intellectual humility and subjective numeracy, two other variables that could influence how people process and are influenced by anecdotal or scientific evidence. These individual difference variables are novel in user comments research, and, in comparison with need for cognition, they offer more specificity and nuance to our understanding of how people process communication. The goal of including additional moderators is to better articulate how individual differences in openness to and confidence in processing evidence may help explain the impact of contradictory arguments in comments on perceived uncertainty.

Intellectual humility, a personality trait, refers to the extent to which someone accepts that their beliefs may not be accurate (Leary et al., 2017). High intellectual humility has been associated with several outcomes relevant to processing comments, including the capacity to accept changing opinions or information from public figures (Leary et al., 2017), more general knowledge and less likelihood of claiming knowledge one does not have (Krumrei-Mancuso et al., 2020), and more openness to understanding others’ opinions (Porter & Schumann, 2018). Those with higher intellectual humility can allow for more divergence without abandoning a belief as the trait is associated with greater tolerance of ambiguity (Leary et al., 2017). However, it is unknown how intellectual ability affects the integration of two divergent opinions, as with user comments that challenge a science new story. Therefore, we pose the following research question:

Research Question 3 (RQ3): How does intellectual humility moderate the combined effects of evidence type and argument direction on (a) perceived relevance, (b) perceived credibility, (c) story likeability, (d) willingness to share the story, (e) perceived uncertainty, and (f) risk perception toward antibiotic-resistant infections?

Subjective numeracy is defined as the perceived ability to understand statistics and perform math-related tasks as well as a preference for numeric information in the news and other contexts. Those with lower subjective numeracy also have less understanding of health information and a lower ability to make medical decisions (Fraenkel et al., 2017). In one survey (McNaughton et al., 2011), nearly half the U.S. population (44%) had numeracy skills at the basic level or below. Because subjective numeracy captures individuals’ confidence in their mathematical abilities as well as their preference for the information presented numerically (McNaughton et al., 2011), it is potentially a useful moderator for understanding differences in how people are influenced by scientific evidence in comments. As subjective numeracy has not yet been explored in relation to user-generated content, and because we were looking at influences of scientific evidence that typically references numbers, we pose the following research question:

Research Question 4 (RQ4): How does subjective numeracy moderate the combined effects of evidence type and argument direction on (a) perceived relevance, (b) perceived credibility, (c) story likeability, (d) willingness to share the story, (e) perceived uncertainty, and (f) risk perception toward antibiotic-resistant infections?

Method

Design

The experiment used a 2 (argument direction: supporting comments vs. dissenting comments) × 2 (evidence type: anecdotal evidence vs. scientific evidence referenced in comments) between-subjects factorial design with a control group without comments. A priori power analysis indicated a sample size of 269 for 80% power to detect a medium-sized effect (f = .25) at α = .05.

Participants

The sample recruited by the Qualtrics panel included 454 American adults. Raw data were screened for univariate and multivariate outliers and deleted from 11 participants who completed the study in fewer than 5 minutes, 11 participants who completed the study in more than 40 minutes, and 6 multivariate outliers (Mahalanobi’s distance chi-square test p < .001) based on all dependent variables and covariates. Univariate outliers were kept because the skewness and kurtosis values for each variable were between −1.96 and 1.96, which are considered acceptable to prove normal univariate distribution (West et al., 1995).

Data from the remaining 426 participants were included (M_age = 45.54, SD = 16.16, range: 18–83). Among participants, 125 (29.3%) were male, 300 (70.4%) were female, and 1 (0.2%) preferred not to answer. Most participants were White (n = 350; 82.2%); 32 (7.5%) were Black or African American; 24 (5.6%) were Asian; 14 (3.3%) were Latino or Hispanic; 5 (1.2%) selected “other”; and 1 (0.2%) chose not to report. Among the 426 U.S. adults, 82 read the comments that supported the story content and referenced anecdotal evidence in its support, 87 read the comments that dissented from the story content and referenced anecdotal evidence in its dissent, 86 read the comments supported the story content and referenced scientific evidence in its support, 86 read the comments dissent from the story content and referenced scientific evidence in its dissent, and 85 read the story without any comments (control condition). Respondents were equally distributed across all five conditions in terms of demographics.¹

Stimuli/Procedure

Story Design

Participants all viewed the same news story with either no comments (control) or one of four sets of comments. The news story appeared as a wire story bylined by the Associated Press and published on the website of The Daily Times (a fictional outlet designed to appear similar to The New York Times website). The use of a wire-service story being shared on a fictional news outlet was meant to diminish perceived partisanship or preconceived notions among participants (Fico & Freedman, 2004; Straubhaar et al., 2013). The news story, about how the over-prescription of antibiotics can lead to antibiotic resistance, was adapted from an article that appeared in the Washington Post (Sun, 2016). A graphic designer used InDesign software to create the experimental stimuli, and a professional journalist reviewed the article text for style and content.

Story Topic

There are several reasons why this research used the topic of antibiotic resistance resulting from overprescription as an example of a story that is perceived as less controversial than others to the public. First, a separate research project underway confirms that people perceive the topic of antibiotic resistance (M = 3.49) generally as significantly less controversial than genetically modified organisms (M = 5.04) and climate change (M = 5.08) (ps < .001) (Saffran et al., 2021). Second, in our treatment of the topic in the experimental stimuli, the topic is framed as being about the causal relationship between the overprescription of antibiotics and antibiotic resistance. Even if there is controversy among medical practitioners and researchers about other facets of antibiotic resistance, such as whether people need to finish their course of antibiotics, the findings about the overuse of antibiotics leading to resistance are consistent and not contested (Zaman et al., 2017). Third, knowledge about the topic generally is scarce among members of the public (Zaman et al., 2017). A scan of commentary on social media channels and YouTube revealed little public discussion, except for antibiotic use in livestock, which is a separate topic.

Comments

Across the four conditions, each of the six comments had constant gender-neutral handles, topics, and length (see Appendix A in supplementary online materials for examples of dissenting/scientific, supportive/anecdotal, and control). One of the six comments was a neutral “filler” in order to increase ecological validity by not having all of the comments be on-topic. Only six comments were included, so they would fit on one page and still be legible.

For the anecdotal condition, commenters referred to the evidence of over-prescription and antibiotic resistance from their own experience, friends or family, or local stories. For the scientific evidence condition, commenters referred to the evidence based on studies, scientists, statistics, the Centers for Disease Control and Prevention, graphs, research, and experts. Both conditions were based on the typical verbiage of online commenters.

In addition to pretesting the stimuli with a focus on the manipulation questions (Amazon MTurk, N = 120), we also included a manipulation check to ensure that the manipulation was working in the formal study. At the end of the posttest questionnaire, three questions asked participants whether the original story’s main point was antibiotics being overprescribed (Yes = 1; No = 0), M = 0.96, SD = 0.20, test value = .5, t(426) = 46.91, p < .001, whether the commenters in the story they had read supported (1) or dissented (0) with the story, M_support = .94; M_dissent = .57; t(248.16) = 8.91, p < .001, and whether they used anecdotal (1) or scientific (0) evidence to support their position, M_anecdotal = .79; M_scientific = .58; t(329.62) = 4.29, p < .001. These post-manipulation check results showed participants could distinguish between supporting and dissenting comments and scientific and anecdotal evidence in the study.

Before viewing the stimuli, participants answered questions about their intellectual humility, health information orientation, and subjective numeracy. After viewing the stimuli, participants were asked to give feedback about the message, perceived relevance of the story, perceived story credibility, liking and sharing the story, perceived uncertainty about the overprescription of antibiotics, their perception of risk from antibiotic-resistant infections, and their demographics (e.g., age, gender, political ideology). In total, participants spent an average of 11.54 min on the study.

Dependent Variables

Perceived Story Relevance

To assess the perceived relevance of the story, participants were asked to rate four statements on a 5-point Likert-type scale (1 = strongly disagree; 5 = strongly agree), including, “I found this story interesting,” “I found this story engaging,” and “I found this story personally relevant.” The four items were averaged to create a single scale with a high score representing higher perceived relevance (Cronbach’s α = .84, M = 4.06, SD = 0.76). The scale was created based on the review of previous studies in which perceived relevance was defined as the perception of relevance of a message is related to a person’s goals, values, and interests (Anghelcev & Sar, 2011; Celsi & Olson, 1988).

Perceived Story Credibility

To determine how participants evaluated the credibility of the story, a five-item credibility index (Mayo & Leshner, 2000; Meyer, 1988) was used. Credibility was measured using a six-point semantic differential scale with the following items: “fair/unfair,” (reversed) “biased/unbiased,” “accurate/inaccurate,” “can be trusted/can’t be trusted,” and “tells the whole story/doesn’t tell the whole story.” The items were averaged to create a single scale with high scores representing higher story credibility (Cronbach’s α = .83, M = 4.50, SD = 0.95).

Story Likeability

Story likeability was measured using a single question, “How much did you like this story?” (1 = not at all; 6 = very much; M = 4.74, SD = 1.14).

Willingness to Share

Participants’ willingness to share the story was measured using a single question, “How likely would you be to share this story?” (1 = very unlikely; 6 = very likely; M = 4.31, SD = 1.51).

Perceived Uncertainty

To assess perceived uncertainty about the overprescription of antibiotics, participants were asked to rate three statements on a five-point scale (1 = strongly disagree; 5 = strongly agree) adapted from Spence et al.’s (2012) climate change uncertainty scale, such as: “I am uncertain that antibiotics are really being overprescribed,” and “It is uncertain what the effects of overprescribing antibiotics will be.” The items were averaged to create a single scale in which lower scores indicate more certainty about antibiotic overprescription; higher scores indicate more uncertainty (Cronbach’s α = .77, M = 2.71, SD = 1.08).

Risk Perception

Participants’ risk perception with regard to the impact of antibiotic-resistant infections was assessed with 10 questions adapted from a climate change risk perception index (Smith & Leiserowitz, 2012) with questions such as “How concerned are you about the impact of antibiotic-resistant infections on you personally?” Because the items were on different point scales, all items were transformed to one-point scales,² then averaged (Cronbach’s α = .91, M = 0.64, SD = 0.22).

Moderators

Intellectual Humility

Intellectual humility refers to the extent to which people accept that their beliefs might be incorrect (Leary et al., 2017). Drawing on Leary et al. (2017), participants’ intellectual humility was assessed using six items, such as “I question my own opinions, positions, and viewpoints because they could be wrong” and “In the face of conflicting evidence, I am open to changing my opinions.” Participants were asked to indicate how much each statement was true of them on a five-point scale (1 = not at all like me; 5 = very much like me). The items were averaged to create a single scale in which high values reflect high intellectual humility (Cronbach’s α = .83, M = 3.85, SD = 0.67).

Subjective Numeracy

Subjective numeracy refers to an individual’s perceived ability to understand and the individual’s preferences for dealing with numbers, which was assessed with eight questions adapted from Fagerlin et al. (2007), including “How good are you at working with fractions?” and “How helpful do you find tables and graphs that are parts of a story?” Because the items were on different point scales, all items were transformed to one-point scales, then averaged to create a single scale (Cronbach’s α = .81, M = 0.70, SD = 0.18). See Appendix B (in supplementary online materials) for all measures with associated items.

Data Analysis

Data were analyzed in SPSS25 using a series of analyses of variance (ANOVAs). First, we conducted two-way ANOVAs to examine the main effects of evidence type and argument direction and their interaction on each of the six outcome variables (i.e., H1, H2, and RQ1). In comparing the combined effects of evidence type and argument direction of a comment section with no comment section, the four experimental groups (i.e., support/anecdotal, dissent/anecdotal, support/scientific, and dissent/scientific) and the control group were treated as five groups in order to include the variance associated with the control condition into the mean square error estimate (Elaad, 2013; RQ2). All follow-up pairwise comparisons were adjusted using Least Squares Difference. Furthermore, the PROCESS macro (Model 1: simple moderation model with one moderator; Hayes, 2012) was used to investigate whether intellectual humility and subjective numeracy moderated the effects of the message on outcome variables (RQ3 and RQ4). We chose Model 1 because we only examined one moderator at a time (either intellectual humility or subjective numeracy). Mean-centered intellectual humility and subjective numeracy were used in moderation analyses.

Results

Main Effects (H1 and H2)

H1 proposed that a medical science story with a comment section of anecdotal (vs. scientific) evidence would lead to (a) enhanced perceived relevance, (b) more perceived credibility, (c) greater story likeability, (d) stronger willingness to share the story, (e) less perceived uncertainty, and (f) increased risk perception toward antibiotic-resistant infections. The main effect of evidence type was statistically significant on perceived credibility, F(1, 340) = 5.76, p = .02, partial η² = .02, and story likeability, F(1, 340) = 7.08, p = .01, partial η² = .02. Specifically, the antibiotic resistance story with anecdotal comments was rated significantly more credible and likable than one with scientific comments. However, evidence type of comments did not significantly affect perceived relevance, F(1, 340) = 2.82, p = .09, willingness to share, F(1, 340) = 2.56, p = .11, perceived uncertainty, F(1, 340) = 3.59, p = .06, or risk perception, F(1, 340) = 1.11, p = .29. Thus, H1b and H1c were supported; H1a and H1d-f were not supported. See Table 1 for the means and standard deviations of all three outcome variables for each evidence-type condition.

Table 1.

Means (M) and Standard Deviations (SD) of All Outcome Variables for Each Evidence Type.

Evidence type	Perceived relevance		Perceived credibility		Story likeability		Willingness to share		Perceived uncertainty		Risk perception
Evidence type	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
Anecdotal (n = 169)	4.11	0.81	4.57	1.00	4.85	1.14	4.37	1.53	2.64	1.15	.65	.23
Scientific (n = 172)	3.97	0.77	4.32	0.96	4.52	1.20	4.12	1.56	2.86	1.04	.63	.22

H2 proposed that a medical science story with supportive (vs. dissenting) comments would lead to (a) enhanced perceived relevance, (b) more perceived credibility, (c) greater story likeability, (d) stronger willingness to share the story, (e) less perceived uncertainty, and (f) increased risk perception toward antibiotic-resistant infections. The main effect of argument direction was statistically significant on perceived relevance, F(1, 340) = 4.96, p = .03, partial η² = .01, perceived credibility, F(1, 340) = 9.59, p = .002, partial η² = .03, story likeability, F(1, 340) = 10.17, p = .002, partial η² = .03, willingness to share, F(1, 340) = 11.36, p < .001, partial η² = .03, and risk perception, F(1, 340) = 6.34, p = .01, partial η² = .02. Specifically, the antibiotic resistance story with supportive comments was rated significantly more relevant, credible, and likable than one with dissenting comments. Accordingly, participants were more willing to share the antibiotic resistance story with supportive comments and had greater risk perceptions toward the impact of antibiotic-resistant infections. However, the argument direction of comments did not significantly influence perceived uncertainty, F(1, 340) = .15, p = .70. Thus, H2a–d and H2f were supported; H1e was not supported. See Table 2 for the means and standard deviations of all six outcome variables for each argument direction condition.

Table 2.

Means (M) and Standard Deviations (SD) of All Outcome Variables for Each Argument Direction.

Argument direction	Perceived relevance		Perceived credibility		Story likeability		Willingness to share		Perceived uncertainty		Risk perception
Argument direction	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
Supportive (n = 168)	4.14	0.76	4.61	0.99	4.88	1.1	4.52	1.48	2.73	1.14	0.67	0.21
Dissenting (n = 173)	3.95	0.81	4.28	0.96	4.49	1.22	3.97	1.57	2.77	1.06	0.61	0.24

Interaction of Evidence Type and Argument Direction (RQ1)

There was no interaction of evidence type and argument direction on any of the outcome variables (ps > .05).

Combined Effects of Comments Versus No Comments (RQ2)

RQ2 examined whether a medical science story with a comment section (of any evidence type or argument direction) performed better than a story without a comment section in terms of increasing (a) perceived story relevance, (b) perceived story credibility, (c) story likeability, (d) willingness to share the story, and (e) reducing perceived uncertainty and f) increasing risk perception toward antibiotic-resistant infections? The different types of comments (including no comments) exerted significantly different effects on perceived story relevance, F(4, 421) = 2.60, p = .04, η² = .02, story credibility, F(4, 421) = 6.25, p < .001, partial η² = .06, story likeability, F(4, 421) = 6.04, p < .001, η² = .05, willingness to share the story, F(4, 421) = 4.47, p = .002, η² = .04, and perceived uncertainty, F(4, 421) = 2.21, p = .068. However, different comments did not lead to significantly different influences on risk perceptions toward antibiotic-resistant infections, F(4,421) = 1.93, p = .10. See Table 3 for the means and standard deviations of all six outcome variables for each comment condition.

Table 3.

Means and Standard Deviations of Outcome Variables in Each Condition.

Condition	Perceived relevance		Perceived credibility		Story likeability		Willingness to share		Perceived uncertainty		Risk perception
Condition	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
Anecdotal + supportive (n = 82)	4.25	0.68	4.67	1.03	5.01	1.04	4.65	1.40	2.53	1.15	.68	.20
Anecdotal + dissenting (n = 87)	3.98	0.90	4.47	0.97	4.69	1.21	4.11	1.62	2.75	1.14	.62	.25
Scientific + supportive (n = 86)	4.03	0.81	4.54	0.95	4.76	1.16	4.41	1.54	2.93	1.11	.66	.22
Scientific + dissenting (n = 86)	3.92	0.72	4.10	0.92	4.28	1.20	3.83	1.52	2.80	0.97	.60	.22
No comments (n = 85)	4.14	0.59	4.74	0.76	4.99	0.91	4.58	1.34	2.53	0.99	.63	.22

Specifically, for RQ2a, the antibiotic resistance story with supportive anecdotal comments was perceived as significantly more relevant than one with dissenting scientific comments (p = .004) and one with dissenting anecdotal comments (p = .02) but not significantly more than one without a comment section (p = .35).

For RQ2b, the antibiotic resistance story with dissenting scientific comments was perceived as significantly less credible than all other conditions (for dissenting anecdotal comments, p = .01; for supportive anecdotal comments, p < .001; for supportive scientific comments, p = .002; for no comments, p < .001). However, the differences were not significant among the other four groups.

For RQ2c, the antibiotic resistance story with dissenting scientific comments was less likable than all other conditions (for dissenting anecdotal comments, p = .02; for supportive anecdotal comments, p < .001; for supportive scientific comments, p = .01; for no comment section, p < .001). The differences were not significant among other groups.

For RQ2d, participants who viewed the antibiotic resistance story with dissenting scientific comments were less willing to share the story than those who viewed the story with supportive anecdotal comments (p < .001), supportive scientific comments (p = .01), and without comments (p = .001). In addition, participants who viewed the story with dissenting anecdotal comments were significantly less willing to share the story than those who viewed the story with supportive anecdotal comments (p = .02) and no comments (p = .04).

For RQ2e, the antibiotic resistance story with supportive anecdotal comments (p = .02) and the story without comments (p = .02) led to significantly higher perceived certainty (lower perceived uncertainty score) about the overprescription of antibiotics than the supportive scientific comments.

Moderation Effects (RQ3 and RQ4)

Intellectual Humility

RQ3 investigated how intellectual humility moderated the effects of evidence type and argument direction on (a) perceived relevance, (b) perceived credibility, (c) story likeability, (d) willingness to share the story, (e) perceived uncertainty, and (f) risk perception toward antibiotic-resistant infections. To answer this question, first, we examined how individual intellectual humility moderated the effect of evidence type on the outcomes. There was no significant interaction between evidence type and intellectual humility on any of the six outcome variables.

Second, we examined how individual intellectual humility moderated the effect of argument direction on the outcomes. There was a significant interaction between argument direction and intellectual humility on the perceived relevance of the antibiotic resistance story (RQ3a), B = −.27, SE = .11, t(337) = − 2.49, p = .01, 95% confidence interval [CI]: [−.48, −.06], which is shown in Figure 1. For participants with low intellectual humility, those exposed to supportive comments (M = 3.79) perceived significantly greater story relevance than those exposed to dissenting comments (M = 3.48), intellectual humility 1SD below the M: B = .31, SE = .10, t(337) = 3.07, p = .002, 95% CI [.11, .51]. However, for participants with moderate and high intellectual humility, exposure to supportive or dissenting comments did not significantly differ regarding perceived story relevance (moderate: p = .07; high: p = .65, respectively).

Figure 1.

Moderation Effect of Intellectual Humility on the Relationship Between Argument Direction and Perceived Relevance.

Furthermore, we explored the moderation effect of individual intellectual humility on the combined effects of evidence type and argument direction on the outcome variables. There was a significant interaction between combined comment types and intellectual humility on the perceived relevance of the antibiotic resistance story (RQ3a), B = .46, SE = .14, t(416) = 3.23, p = .001, 95% CI [.18, .74], which is shown in Figure 2. For participants with low intellectual humility, those exposed to the story with dissenting anecdotal comments perceived the story as significantly less relevant (M = 3.49) than those exposed to the story without comments (M = 3.92), intellectual humility 1SD below the M: B = −.43, SE = .13, t(416) = -3.25, p = .001, 95% CI [−.69, −.17]. However, for participants with moderate and high intellectual humility, exposure to the story with dissenting anecdotal comments did not produce significant differences in perceived story relevance compared with exposure to the story without comments (moderate: p = .20; high: p = .20, respectively).

Figure 2.

Moderation Effect of Intellectual Humility on the Combined Effects Evidence Type and Argument Direction on Perceived Relevance.

Subjective Numeracy

RQ4 investigated how subjective numeracy moderated the effects of evidence type and argument direction on (a) perceived relevance, (b) perceived credibility, (c) story likeability, (d) willingness to share the story, (e) perceived uncertainty, and (f) risk perception toward antibiotic-resistant infections. In line with RQ3, we first examined how individual subjective numeracy moderated the effect of evidence type on the six outcomes and then tested how it moderated the impact of argument direction on the outcomes, and the results showed no significant interaction.

Thus, we further explored the moderation effect of subjective numeracy on the combined effects of evidence type and argument direction on the outcome variables. There were significant interaction effects on the perceived relevance of the story (RQ4a), B = 1.26, SE = .56, t(416) = 2.26, p = .02, 95% CI [.16, 2.36], which is shown in Figure 3. For participants with low subjective numeracy, those exposed to the antibiotic resistance story with dissenting anecdotal comments perceived the story as significantly less relevant (M = 3.65) than those exposed to the story without comments (M = 3.95), subjective numeracy 1SD below the M: B = -.31, SE = .15, t(416) = -2.01, p = .045, 95% CI [−.60, −.01]. However, for participants with moderate and high subjective numeracy, exposure to the story with dissenting anecdotal comments and the story without comments did not significantly differ in perceived story relevance (moderate: p = .49; high: p = .29, respectively).

Figure 3.

Moderation Effect of Subjective Numeracy on the Combined Effects Evidence Type and Argument Direction on Perceived Relevance.

Discussion

Two key conclusions emerge from this study. First, compared with other types of comments, dissenting comments with scientific evidence had an adverse influence on measures of story credibility, relevance, likeability, and willingness to share the story. This suggests that comments challenging a story’s scientific conclusions with statements that have the veneer of scientific credibility, like allusions to conflicting studies for instance, have the most significant and negative impact on perceptions of the story. Although supportive, anecdotal comments after the story led to the most desirable impact on participants’ perceptions, this superiority was not statistically different from seeing a story without a comment section. Second, individual difference variables moderated the effects of comments. People with lower intellectual humility and lower subjective numeracy were more likely to be influenced by dissenting, anecdotal comments, and they also perceived the story as less personally relevant. Taken together, user comments that dissent from the point of the story may negatively influence people’s perception of the story and the medical science contained therein.

To explore the findings further, anecdotal evidence, regardless of the argument direction, cultivated more credibility and likeability for the story. This partially echoed the findings of Wojcieszak and Kim (2016) who found anecdotal or narrative evidence led to greater message acceptance and immersion and to Hinnant et al. (2016), who saw improved credibility resulting from anecdotal comments (among conservative audience members on a climate change story). And supportive evidence, regardless of evidence type, enhanced perceived relevance and credibility, increased story likeability and willingness to share the story, and increased risk perception toward antibiotic-resistant infections, which is consistent with one of the main points of the medical news story about antibiotic resistance. That supportive comments would lead to more perceived credibility of the news story is similar to the findings of Petit et al. (2021) and Naab et al. (2020). Overall, the supportive comments led to generally positive outcomes for the perceptions of the story, which would be consistent with the journalistic goal of having a story resonate with readers. That said, supportive evidence alone did not reduce perceived uncertainty about the issue.

When looking at the combined effects of evidence type and argument direction, participants who saw comments that were supportive of the story and marshaled anecdotal evidence experienced increased feelings of perceived relevance of the story (when compared with dissenting stories), and they experienced less uncertainty compared with those who saw comments that were supportive and with scientific evidence. These combined effects again give credence to exemplification theory that argues personal narratives increase immersion in a story as well as acceptance and likeability of a message (Kim et al., 2012; Wojcieszak & Kim, 2016). Our findings add context to this theoretical claim by demonstrating that anecdotal evidence is most influential for audience responses when it supports the argument being made in the news story. This finding aligns with how exemplars are typically used in medical news as supportive anecdotal evidence that demonstrates the human impact of scientific findings or health issues (Kim et al., 2012). However, personal stories that diverge from the main argument of the news story may be viewed with more suspicion or discounted.

On the flip side, participants who saw dissenting scientific comments perceived the story to be less credible and less likable than all the other conditions and were less likely to want to share the story (than those who saw supportive comments). This suggests that evidence that sounds scientific can be influential even from a source that lacks expert credibility, like a commenter on a news story. This finding has implications for deliberation and debate on controversial medical or scientific issues as well. In debates about vaccine efficacy, for instance, people often refer to “doing their own research” when they raise questions or claims that challenge the scientific consensus. Our findings suggest that these vague appeals to contradictory scientific evidence can have an impact on perceptions of news credibility even when the claims are not supported by specific sources. Given its prominence in social media misinformation, future research should explore this rhetorical strategy in more depth.

Lastly, similar to previous need-for-cognition research, just as people with low need-for-cognition are more susceptible to variations in messaging (Hinnant et al., 2016), participants with low intellectual humility and low subjective numeracy were more vulnerable to experiencing the influences of evidence type and argument direction. This suggests that people with less confidence in their ability to assess evidence and with less belief that their opinions can be changed are more likely to be influenced by personal stories rather than scientific evidence. While this may seem counterintuitive in the case of intellectual humility, our findings suggest that what the scale is measuring is not actually resistance to attitude change but instead less awareness that one’s attitude can be changed, in this case by the dissenting personal stories from the anecdotal comments. Future research should explore further how these individual differences compare and which measures hold the most connection to lived experiences.

Future Research, Limitations, and Implications

Future research could also examine how the existence of comment moderation, which has shown promise in some research (Yeo et al., 2019), could influence perceptions of a journalistic story and scientific topic. The employment of the experimental approach increases a certain level of artificiality in this study. As respondents received payment for reading the posts, their responses and attention levels may vary from those who view them outside the experimental context. Additionally, although our separate study confirmed the less-controversial feature of the antibiotic-resistant topic, it would be more compelling if we could include the preexisting attitude toward the topic as a control variable in the analysis. Lastly, this study only considered the comment section with the simplest interface design. However, more and more social media comment sections allow users to comment on other users’ comments. Future studies should benefit from examining how those interactive user comments influence the readers’ attitudes and perceptions. Additionally, our controlled experiment only tested the pure effect of supportive and dissenting comments. In contrast, the comment sections often include a mix of supportive and dissenting, scientific and anecdotal comments in the real world. Thus, future studies should advance the understanding of the effects of a mix of supportive and dissenting comments through online field experiments. Lastly, we only used single-item scales to measure story likeability and willingness to share, which might not capture the full meaning of a construct. For example, sharing a story on social media is different from sharing a story with friends and family. Future studies should consider using multi-item scales to increase the measurement validity.

Although typically having a control condition perform in the same way as certain manipulated variables on some scales would be a cause for dismay, we contend this helps to prove a broader point. The fact that having no comments was as positive at times for relevance and scientific certainty as having anecdotal supporting comments further demonstrates the importance of dropping comment sections from news websites. Supportive comments are nowhere near as common as critical comments (Rojas, 2010), and so the continued presence of largely negative comment sections serves to reduce likeability and credibility, neither of which are good for journalism or the scientific enterprise. Currently, social media sites offer a forum for most of the commenting on news stories. Section 230, which protects technology companies from liability for user content, is currently under scrutiny by the Supreme Court of the United States. The legal protections afforded to social media platforms (and other hosts of user content) could be pared back, which would likely change the format of comment sections because the platform owners could be held legally responsible for user-generated content. Perhaps a reimagining of comment sections would meet some of the originally forecasted ideal outcomes that would serve democratic discourse and participation, which have thus far been elusive.

Our findings have some implications for the practice of science and health journalism. While online comment sections on news websites are less common than they once were, many news consumers now get their news from social media platforms, where user comments and misinformation are rampant. The short posts and conversational tone on social media platforms like Facebook and Twitter may also further decrease the credibility gap between news posts and comments. Journalists and news outlets using social media to promote their stories should be aware of how rhetorical strategies like suggesting dissenting scientific evidence can sow doubt among audiences and also lead to decreased credibility for the story and perhaps the news outlet, even when the story does not cover a hot button or controversial issue. An increasing number of studies have found that some forms of correction can be effective at decreasing the impact of misinformation. Future research should explore whether prebunking from journalists either in the story or in the comments could counter the impact of dissenting comments that attempt to discredit scientific consensus.

Our study intentionally used a less-controversial medical science issue to determine how audiences respond to comments when they are less emotionally involved in the topic and may also have opinions that are more amenable to change. Less-controversial issues like antibiotic resistance still have potentially significant effects on individual and public health given that doubt about scientific consensus could lead individuals not to comply with treatment recommendations. Our findings may also apply to controversial issues, but it is worth noting that user comments on online platforms may also be part of the process by which less-controversial issues become controversial. For example, inconsistent public health recommendations in the news on whether surgical masks should be worn by all to prevent the spread of COVID, refusal of public officials to wear masks, as well as online misinformation and differing opinions about the effects of masks on child development led to a previously noncontroversial scientific consensus on preventing infection becoming hotly debated. Therefore, our findings may help explain not just the effects of different types of comments on perceptions of less-controversial science stories but also how contradictory information in user-generated comments can decrease trust in news media among some audience members, potentially leading to increased polarization and controversy.

More than 20 years after predicting the growth in uncertainty that would be cultivated by readers triangulating sources of information online to arrive at meaning, Dunwoody wrote at the beginning of the COVID-19 pandemic “the internet and social media have played a major role in exacerbating uncertainty perceptions” and “this avalanche of inconsistent, sometimes misleading information can dramatically increase perceptions of uncertainty” (Dunwoody, 2020, p. 472). This study adds an important and novel dimension to understanding democratic discourse in health and science communication via journalistic outlets. Although audience participation and feedback to the journalistic process were once heralded as a transformative way to get citizens involved with important social matters, evidence continues to grow that user participation can be deleterious to understanding and believability, particularly with health and science content. Additionally, this study extended ideas from previous research to a less-controversial topic, ideally limiting the independent influences exerted by a story topic. The fact that participants had significant reactions to an issue that is not polarized makes the problem of comments more alarming.

Supplemental Material

sj-docx-1-scx-10.1177_10755470221150503 – Supplemental material for Contested Certainty and Credibility: The Effect of Personal Stories and Scientific Evidence in User Comments on News Story Evaluation and Relevance

Supplemental material, sj-docx-1-scx-10.1177_10755470221150503 for Contested Certainty and Credibility: The Effect of Personal Stories and Scientific Evidence in User Comments on News Story Evaluation and Relevance by Amanda Hinnant, Sisi Hu, Yoorim Hong and Rachel Young in Science Communication

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors thank the Donald W. Reynolds Journalism Institute (RJI) at the Missouri School of Journalism for its financial support.

ORCID iDs

Amanda Hinnant

Sisi Hu

Yoorim Hong

Rachel Young

Supplemental Material

Supplemental material for this article is available online at .

Notes

Author Biographies

Amanda Hinnant is an associate professor of Journalism Studies at the University of Missouri. She was recognized as the 2021 Outstanding Woman in Journalism and Mass Communication Education by AEJMC’s Commission on the Status of Women. Hinnant’s research focuses on media sociology, health and science communication, and narrative persuasion. Her research has been published in PLOS One, Climatic Change, Health Communication, Journalism Studies, and Communication Research.

Sisi Hu is an assistant professor of advertising at the University of Arkansas School of Journalism and Strategic Media. Hu’s research focuses on strategic health and science communication. Topics she has recently examined include narrative persuasion and message strategies to reduce psychological reactance and promote desired health behavior. Her research has been published in Health Communication, PLOS ONE, and Health Education Research.

Yoorim Hong is a PhD candidate at the Missouri School of Journalism. Her research interests focus on how organizations’ and/or the public’s message features (content and structure) interact with audience characteristics to elicit cognitive and attitudinal responses, and in turn, behavioral outcomes in the context of public relations, corporate social responsibility, and crisis and risk communication.

Rachel Young is an associate professor in the School of Journalism and Mass Communication at the University of Iowa. She studies health media, with a current focus on personal health stories that reinforce or counteract stigma. Her research has been published in Mass Communication and Society, Health Communication, Journalism Studies, and Health Promotion and Practice. She has won awards and grant support for her community-engaged teaching and research.

References

Anderson

A. A.

Brossard

Scheufele

D. A.

Xenos

M. A.

Ladwig

(2014). The “Nasty Effect”: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19(3), 373–387.

Anghelcev

Sar

(2011). The influence of pre-existing audience mood and message relevance on the effectiveness of health PSAs: Differential effects by message type. Journalism & Mass Communication Quarterly, 88(3), 481–501.

Antony

M. G.

Thomas

R. J.

(2017). “Stop sending your kids across our border:” Discursively constructing the unaccompanied youth migrant. Journal of International and Intercultural Communication, 10(1), 4–24.

Boot

A. B.

Dijkstra

Zwaan

R. A.

(2021). The processing and evaluation of news content on social media is influenced by peer-user commentary. Humanities and Social Sciences Communications, 8(1), Article 209.

Borger

Van Hoof

Costera Meijer

Sanders

(2013). Constructing participatory journalism as a scholarly object: A genealogical analysis. Digital Journalism, 1(1), 117–134.

Celsi

R. L.

Olson

J. C.

(1988). The role of involvement in attention and comprehension processes. Journal of Consumer Research, 15(2), 210–224.

Chen

G. M.

(2017). Online incivility and public debate: Nasty talk. Springer.

Clark

Illman

D. L.

(2003). Content analysis of New York Times coverage of space issues for the year 2000. Science Communication, 25(1), 14–38.

Collins

Nerlich

(2015). Examining user comments for deliberative democracy: A corpus-driven analysis of the climate change debate online. Environmental Communication, 9(2), 189–207.

10.

Denham

B. E.

(2014). Intermedia attribute agenda setting in the New York Times: The case of animal abuse in US horse racing. Journalism & Mass Communication Quarterly, 91(1), 17–37.

11.

Deuze

Marjoribanks

(2009). Newswork. Journalism, 10, 555–561.

12.

Dunwoody

(1999). Scientists, journalists, and the meaning of uncertainty. In Friedman

S. M.

Dunwoody

Rogers

C. L.

(Eds.), Communicating uncertainty: Media coverage of new and controversial science (pp. 59–79). Routledge.

13.

Domingo

Quandt

Heinonen

Paulussen

Singer

J. B.

Vujnovic

(2008). Participatory journalism practices in the media and beyond: An international comparative study of initiatives in online newspapers. Journalism Practice, 2(3), 326–342.

14.

Dunwoody

(2020). Science journalism and pandemic uncertainty. Media and Communication, 8(2), 471–474.

15.

Elaad

(2013). Effects of goal-and task-oriented motivation in the guilty action test. International Journal of Psychophysiology, 88(1), 82–90.

16.

Fagerlin

Zikmund-Fisher

B. J.

Ubel

P. A.

Jankovic

Derry

H. A.

Smith

D. M.

(2007). Measuring numeracy without a Math Test: Development of the Subjective Numeracy Scale. Medical Decision Making, 27(5), 672–680.

17.

Fico

Freedman

(2004). Bureau, wire reporters write more balanced stories. Newspaper Research Journal, 25(2), 44–57.

18.

Flemming

Cress

Kimmerle

(2017). Processing the scientific tentativeness of medical research: An experimental study on the effects of research news and user comments in online media. Science Communication, 39(6), 745–770.

19.

Flew

(2012, October 12). The digital transformation of 21st century news journalism [Conference session]. Conference on Digital Media and Journalism, Ming Chuan University, Taipei.

20.

Fraenkel

Stolar

Swift

Street

R. L.

Chowdhary

Peters

(2017). Subjective numeracy and the influence of order and amount of audible information on perceived medication value. Medical Decision Making, 37(3), 230–238.

21.

Gallo

J. A.

Cho

C. Y.

(2021). Social media: Misinformation and content moderation issues for congress (R46662). Congressional Research Service Report.

22.

Gierth

Bromme

(2020). Attacking science on social media: How user comments affect perceived trustworthiness and credibility. Public Understanding of Science, 29(2), 230–247.

23.

Glenski

Volkova

Kumar

(2020). User engagement with digital deception. In Shu

Wang

Lee

Liu

(Eds.), Disinformation, misinformation, and fake news in social media (pp. 39–61). Springer.

24.

Golan

(2006). Inter-media agenda setting and global news coverage: Assessing the influence of the New York Times on three network television evening news programs. Journalism Studies, 7(2), 323–333.

25.

Han

Pei

Kamber

(2011). Data mining: Concepts and techniques. Elsevier.

26.

Hayes

A. F.

(2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling. Retrieved from www.afhayes.com/public/process2012.pdf

27.

Hinnant

Subramanian

Young

(2016). User comments on climate stories: Impacts of anecdotal vs. scientific evidence. Climatic Change, 138(3–4), 411–424.

28.

Hoyt

(2007). Civil discourse, meet the internet. The New York Times, 4.

29.

Kareklas

Muehling

D. D.

Weber

T. J.

(2015). Reexamining health messages in the digital age: A fresh look at source credibility effects. Journal of Advertising, 44(2), 88–104.

30.

Kim

H. S.

Bigman

C. A.

Leader

A. E.

Lerman

Cappella

J. N.

(2012). Narrative health communication and behavior change: The influence of exemplars in the news on intention to quit smoking. Journal of Communication, 62(3), 473–492.

31.

Krumrei-Mancuso

E. J.

Haggard

M. C.

LaBouff

J. P.

Rowatt

W. C.

(2020). Links between intellectual humility and acquiring knowledge. The Journal of Positive Psychology, 15(2), 155–170.

32.

Ksiazek

T. B.

Springer

(2018). User comments in digital journalism: Current research and future directions. In Eldridge

Franklin

(Eds.), The Routledge handbook of developments in digital journalism studies (pp. 475–486). Routledge.

33.

LaBarre

(2013). Why we’re shutting off our comments. Popular Science, 24, 2013–2009.

34.

Leary

M. R.

Diebels

K. J.

Davisson

E. K.

Jongman-Sereno

K. P.

Isherwood

J. C.

Raimi

K. T.

Deffler

S. A.

Hoyle

R. H.

(2017). Cognitive and interpersonal features of intellectual humility. Personality and Social Psychology Bulletin, 43(6), 793–813.

35.

Lee

E.-J.

Jang

Y. J.

(2010). What do others’ reactions to news on internet portal sites tell us? Effects of presentation format and readers’ need for cognition on reality perception. Communication Research, 37(6), 825–846.

36.

Lee

E.-J.

Jang

Y. J.

Chung

(2021). When and how user comments affect news readers’ personal opinion: Perceived public opinion and perceived news position as mediators. Digital Journalism, 9(1), 42–63.

37.

Lee

E.-J.

Tandoc

E. C.

(2017). When news meets the audience: How audience feedback online affects news production and consumption. Human Communication Research, 43(4), 436–449.

38.

Luo

(2014). The Internet and agenda setting in China: The influence of online public opinion on media coverage and government policy. International Journal of Communication, 8, 1289–1312.

39.

Mayo

Leshner

(2000). Assessing the credibility of computer-assisted reporting. Newspaper Research Journal, 21(4), 68–82.

40.

McNaughton

Wallston

K. A.

Rothman

R. L.

Marcovitz

D. E.

Storrow

A. B.

(2011). Short, subjective measures of numeracy and general health literacy in an adult emergency department. Academic Emergency Medicine, 18(11), 1148–1155.

41.

Meeks

(2013). All the gender that’s fit to print: How the New York Times covered Hillary Clinton and Sarah Palin in 2008. Journalism & Mass Communication Quarterly, 90(3), 520–539.

42.

Meyer

(1988). Defining and measuring credibility of newspapers: Developing an index. Journalism & Mass Communication Quarterly, 65(3), 567–574.

43.

Mostafa

(2001, June 11). Sites integrate discussion with news. Editor and Publisher.

44.

Naab

T. K.

Heinbach

Ziegele

Grasberger

M.-T.

(2020). Comments and credibility: How critical user comments decrease perceived news article credibility. Journalism Studies, 21(6), 783–801.

45.

Peter

Rossmann

Keyling

(2014). Exemplification 2.0: Roles of direct and indirect social information in conveying health messages through social network sites. Journal of Media Psychology, 26(1), 19–28.

46.

Petit

Millet

Ali

Sun

(2021). Can we stop the spread of false information on vaccination? How online comments on vaccination news affect readers’ credibility assessments and sharing behaviors. Science Communication, 43(4), 407–434.

47.

Porter

Schumann

(2018). Intellectual humility and openness to the opposing view. Self and Identity, 17(2), 139–162.

48.

Reich

(2011). User comments: The transformation of participatory space. In Singer

J. B.

Hermida

Domingo

Heinonen

Paulussen

Quandt

Reich

Vujnovic

(eds.), Participatory journalism: Guarding the open gates at online newspapers (pp. 96–117). Wiley-Blackwell.

49.

Rieder

(2010). No comment: It’s time for news sites to stop allowing anonymous online comments. American Journalism Review, 32(2), 2–3.

50.

Rojas

(2010). “Corrective” actions in the public sphere: How perceptions of media and media effects shape political behaviors. International Journal of Public Opinion Research, 22(3), 343–363.

51.

Saffran

Hinnant

(2021). Controversial vs. less-controversial science: Reactions to narratives. University of Missouri. [Unupublished manuscript].

52.

Santana

A. D.

(2014). Virtuous or vitriolic: The effect of anonymity on civility in online newspaper reader comment boards. Journalism Practice, 8(1), 18–33.

53.

Scheufele

D. A.

Krause

N. M.

(2019). Science audiences, misinformation, and fake news. Proceedings of the National Academy of Sciences, 116(16), 7662–7669.

54.

Shi

Messaris

Cappella

J. N.

(2014). Effects of online comments on smokers’ perception of antismoking public service announcements. Journal of Computer-Mediated Communication, 19(4), 975–990.

55.

Smith

Leiserowitz

(2012). The rise of global warming skepticism: Exploring affective image associations in the United States over time. Risk Analysis, 32(6), 1021–1032.

56.

Spence

Poortinga

Pidgeon

(2012). The psychological distance of climate change. Risk Analysis, 32(6), 957–972.

57.

Straubhaar

LaRose

Davenport

(2013). Media now: Understanding media, culture, and technology. Cengage Learning.

58.

Sun

L. H.

(2016, May 3). 1 in 3 antibiotics prescribed in U.S. are unnecessary, major study finds. The Washington Post.

59.

Thomas

R. J.

(2021). Audience comments and the civic space that rarely was. Journal of Media Ethics, 36(4), 235–236.

60.

Walter

Brüggemann

Engesser

(2018). Echo chambers of denial: Explaining user comments on climate change. Environmental Communication, 12(2), 204–217.

61.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with nonnormal variables: Problems and remedies. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Sage Publications, Inc.

62.

Wilhelm

Joeckel

(2019). Gendered morality and backlash effects in online discussions: An experimental study on how users respond to hate speech comments against women and sexual minorities. Sex Roles, 80(7), 381–392.

63.

Winter

Brückner

Krämer

N. C.

(2015). They came, they liked, they commented: Social influence on Facebook news channels. Cyberpsychology, Behavior, and Social Networking, 18(8), 431–436.

64.

Winter

Krämer

N. C.

(2016). Who’s right: The author or the audience? Effects of user comments and ratings on the perception of online science articles. De Gruyter Mouton.

65.

Wojcieszak

Kim

(2016). How to improve attitudes toward disliked groups: The effects of narrative versus numerical evidence on political persuasion. Communication Research, 43(6), 785–809.

66.

Yeo

S. K.

Y.-F.

Scheufele

D. A.

Brossard

Xenos

M. A.

Corley

E. A.

(2019). The effect of comment moderation on perceived bias in science news. Information, Communication & Society, 22(1), 129–146.

67.

Zaman

S. B.

Hussain

M. A.

Nye

Mehta

Mamun

K. T.

Hossain

(2017). A review on antibiotic resistance: Alarm bells are ringing. Cureus, 9(6), Article e1403.

68.

Zhang

Wang

(2019). The failure of news coverage supportive of human papillomavirus vaccination: The investigation of the effects of online comments on female college students’ vaccination intention. Vaccine, 37(38), 5681–5687.

69.

Zillmann

(2006). Exemplification effects in the promotion of safety and health. Journal of Communication, 56, S221–S237.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.17 MB