Trusting the Facts: The Role of Framing,News Media as a (Trusted) Source,and Opinion Resonance for Perceived Truth in Statistical Statements

Abstract

Scholars have raised concerns that on many issues, citizens are reluctant to trust factual evidence and statistics. One factor that has been shown to impact the perceived truth in statistics is how they are presented, where negatively framed statistics are perceived as truer than positive. This study explores when this bias applies and not. Results from a survey experiment confirm the presence of a negativity bias in truth perceptions, but also that effects are heterogeneous and moderated by, in particular, the recipients’ preexisting opinions. These findings provide valuable information to public actors responsible for disseminating factual information to diverse publics.

Keywords

equivalence framing perceived truth in statistical statements negativity bias media trust opinion resonance

Research indicates that skepticism toward scientific evidence—even when experts and the scientific community largely agree—is widespread on many issues (Kraft et al., 2015). These findings raise concerns, as rejections of factual evidence can have important ramifications for people’s behavior and society at large. For example, if people consider health statistics to be noncredible, this may lead to lower vaccination coverage (Wagner et al., 2020) and facilitate the spread of contagious diseases (Horne et al., 2015), and if people doubt the credibility of climate statistics, this may reduce their engagement in actions needed to prevent irreversible climate disasters (e.g., Masud et al., 2015). A better understanding of the mechanisms explaining the extent to which citizens trust scientific evidence is thus imperative.

A key question in this context concerns the role of framing and valence when explaining people’s reactions to statistical information. On one hand, some studies show that valence framing can influence the perceived truth in statistics, where people tend to perceive statistical statements as truer if they highlight the positive share of something (e.g., “40% of the world’s ecosystems are intact”) than if highlighting the negative counterpart (“60% of the world’s ecosystems are damaged”; Hilbig, 2009, 2012a, 2012b; Koch & Peter, 2017). While these studies indicate “positive” effects of “framing the negative” on the perceived credibility of the information, a vast body of research on related outcomes suggests the opposite. For example, research has shown that when the positive share is highlighted, people tend to rate the framed object more favorably and express more positive attitudes than when the negative equivalent is highlighted (Bigman et al., 2010; Gamliel & Peer, 2010)—something which indirectly implicates that they also trust the (positively framed) statistics to be true. Hence, the role of valence for citizens’ perceptions and reactions to statistical facts is not clear.

Another key but largely unresolved question is what factors can moderate framing effects when it comes to the perceived truth in statistics. In related research on communication effects on attitudes and behavior, two moderators stand out as important: a credible source and prior opinions. Specifically, this research suggests that the effects of message valence attenuate when the information challenges people’s existing views (e.g., Fatemi et al., 2008), and when it is provided by a perceived credible source (e.g., Druckman, 2001; Kahan et al., 2010).¹ Whether these factors also moderate the effects of equivalence framing on the perceived truth in facts is, however, unclear.

The questions regarding mechanisms are important not only from an academic but also from a societal perspective. For example, while negativity may serve as a cue to credibility, it can have unfavorable consequences such as political cynicism (Soroka & McAdams, 2015) and distrust in societal institutions. If, for example, citizens are presented with the proportion of crimes that are not cleared (rather than the proportion of cleared crimes), this may lead to distrust in the police (Andreescu & Keeling, 2012), and if people are presented with the proportion of people experiencing vaccine side effects rather than the proportion of people who do not, this can create mistrust in health authorities (Tafuri et al., 2014). Thus, different methods may be needed to formulate information when communicating with citizens, and, especially when highlighting the negative is not desirable, knowledge of factors that can reduce skepticism to positively framed facts should be greatly useful. If the conditional role of source credibility and prior views that are seen in research of, for example, behavioral intentions applies also to people’s perceived truth in the statistics, actors responsible for communicating factual information may be able to increase the public’s trust in the same by tailoring the messages to different groups. For example, they could disseminate the information through channels trusted by different target groups (Vraga & Bode, 2017) and formulate the messages in ways that would not come across as (too) threatening to the values of individuals whose beliefs could be strongly challenged by a particular frame (Garret & Weeks, 2013; Lewandowsky et al., 2012).

Against this background, the purpose of this study is to investigate the role of (trust in) the source and issue-specific opinions when people evaluate the truth in positively versus negatively framed statistical statements. Specifically, we investigate (a) whether the effects of equivalence framing on people’s perceptions of the truth in statistics are attenuated by information that the statements come from news media (the source through which most people receive information about the society, Mitchell et al., 2016), and if so, (b) whether this effect is conditional on the recipients’ trust in news media. Second, we investigate whether the effects attenuate when the valence of the frames challenges the recipient’s existing views. For analyses, we use data from a survey experiment with Swedish citizens in which we varied the framing of logically equivalent statistics (“framing the positive” vs. “framing the negative”) and the source of the statistical statements (news media vs. no source), complemented with premeasures of the participants’ trust in news media and their issue-specific opinions.

Equivalence Framing and Its Effects

Broadly speaking, equivalence framing is concerned with how the presentation of logically equivalent information affects people’s responses to this information (Cacciatore et al., 2016; Levin, 1987; Levin et al., 1998; Tversky & Kahneman, 1981). This can entail the alteration of logically equivalent words or phrases, such as replacing the word “welfare” with the phrase “assistance to the poor” (Cacciatore et al., 2016), but also the alteration of the valence of the same piece of fact by accentuating a positive or negative equivalent (Levin, 1987; Levin et al., 1998; Tversky & Kahneman, 1981). In this article, we focus on the manipulation of the valence of equivalent information, which is henceforth what we refer to when we say equivalence framing. In the context of equivalence framing, negative and positive equivalents of facts refer to two counterparts of a property that have respectively negative and positive connotations (e.g., unemployed–employed, damaged–intact), and equivalence framing means the presentation of this property in terms of the share that belongs to either one or the other of these counterparts (Levin et al., 1998).

Consider, for example, that the clearance rate of crimes in society is 40%. This fact can be presented as such, “40% of all crimes cleared,” but can also be presented in terms of its negative counterpart (60% of crimes not cleared). Presuming that people perceive cleared crimes as positive and noncleared crimes as negative, the first statement would be considered a positive framing, and the latter a negative framing of the statistics. Equivalence framing is restricted to manipulating the valence of logically equivalent information and whether a fact is perceived positive or negative in absolute terms is not of concern. What matters is whether the fact presented has a negative and positive counterpart that can be selectively highlighted (e.g., Levin et al., 1998; Tversky & Kahneman, 1981). This form of framing should be distinguished from “emphasis framing,” which refers to the manipulation of the content of a communication and can vary both valence and which sets of facts, considerations, and arguments are presented (Cacciatore et al., 2016; D’Angelo et al., 2019).

The type(s) of equivalence framing that targets the manipulation of the valence of facts was originally developed to address the role of valence for risk preferences, particularly how presenting safe and risky options in terms of gains and losses (e.g., “40% lives saved” vs. “60% lives lost”) could affect people’s willingness to take a risk (Tversky & Kahneman, 1981). Since then, the concept has been developed into three specific subtypes of (equiv)valence frames—risky-choice-, goal-, and attribute framing—that differ in terms of what is framed and what outcomes are measured (for a typology, see Levin et al., 1998). While risky choice frames were developed in research on risk psychology, goal framing is common in health communication research and is concerned with how the framing of the consequences of an action or behavior influences the likelihood of adopting that behavior. The variation here concerns whether the action is presented in terms of its benefits (e.g., the probability of detecting breast cancer if participating in mammography screenings) compared with the risks of not taking the same action (the probability of not detecting breast cancer), and its effects are estimated by comparing the likelihood that the recipients adopt the behavior or not.

Attribute framing, finally, which is the type of equivalence framing we target in this study, refers to the framing of single attributes or characteristics of an object by highlighting the share of a population that either exhibits the negative or positive equivalent of this attribute (Levin et al., 1998). Research of this type of framing is typically concerned with how valence affects the attitudes to the framed object and effects are measured by comparing “favorability ratings,” such as the rating of a policy proposal or consumer product on a scale from “bad–good,” “unfavorable–favorable,” or “unacceptable–acceptable.” A typical attribute framing study is Levin (1987), which showed that people evaluated a piece of steak more favorably when presented in terms of percent “lean” compared with percent “fat.” Other studies have yielded similar effects, for example, that people evaluated vaccines more positively when described in terms of the share of successful cases compared with the share of unsuccessful cases (Bigman et al., 2010), and that they evaluated health care allocations to be fairer when these were framed as delivering resources to certain patients compared with not delivering resources to other patients (Gamliel & Peer, 2010). These effects of attribute framing have been explained in that highlighting the positive and negative share of an attribute activates positively (and negatively) encoded information in people’s associative memory that they, often unconsciously, apply in the process when they evaluate the framed object (Levin, 1987; Levin et al., 1998).

When effects of equivalence framing are assessed, be it based on discrete choices as in studies of risky choice, likelihood ratios as in goal framing, or favorability ratings as in attribute framing, they are estimated as the difference in outcome from each frame. In research on attribute framing, this means that the effect is estimated by subtracting the object evaluation from one (e.g., the negative) frame from that of the other (the positive frame; Levin et al., 1998). Because the effects are evaluated in relative terms, a statistical fact can be perceived negatively both under a positive and under a negative frame and yet be influenced by valence—if one of the frames yields even more negative ratings than the other.

The vast majority of attribute framing studies have targeted its effects on people’s favorability of the framed objects. Overall, this research suggests that statements that highlight the positive equivalent of a piece of fact yield higher favorability ratings than statements that are framing the negative (Koch & Peter, 2017; Levin et al., 1998), supporting the idea that valence frames are activating corresponding positive and negative associations in the evaluation process (e.g., Levin, 1987). While most research on attribute framing thus suggests a “positive” effect on attitudes when statistics are presented in a positive light, the small, but highly relevant research on the effects on perceived truth in the statistical statements themselves yields the opposite results. Here, research so far suggests that negative framing produces the most “favorable evaluations,” that is, negatively framed statistical statements are perceived to be truer than their positive counterparts (Hilbig, 2009, 2012a, 2012b; Koch & Peter, 2017).

Equivalence Framing and Perceived Truth in Statistical Statements

The finding that people tend to perceive negatively framed statistical statements as truer than their positive counterparts has been explained, theoretically, in that people have acculturated “a learned association between negative valence and subjective validity” via news media consumption (Hilbig, 2009, p. 985). This “learning hypothesis” draws on the widespread finding of news media being characterized by negativity bias and predominantly reporting on the negative aspects of events (e.g., Damstra & Boukes, 2021; Soroka & McAdams, 2015). Because most people receive most of their information about societal affairs from news media reports (Mitchell et al., 2016; Shehata & Strömbäck, 2014), and considering that news media are a generally highly trusted source in many societies (Hanitzsch et al., 2018), the hypothesis is that people learn that trustworthy information tends to be negative (Hilbig, 2009; Koch & Peter, 2017). Thus, while (positive and negative) valence framing may influence attitudes by evoking corresponding (positive and negative) associations, the effect of (negative) framing on perceived truth in statements is believed to occur by evoking associations with source credibility, in particular, the credibility of news media. Furthermore, people may simultaneously cultivate an association between positivity and untrustworthy information (Koch & Peter, 2017). Advertisers and politicians, for example, typically present their products and programs in a one-sided positive light. Because messages like these, which aim at persuading rather than informing, are generally perceived to be less trustworthy (e.g., McGraw et al., 2002), the association between negativity and perceived truth in information may be reinforced (Koch & Peter, 2017).

Based on the above propositions, we expect that all else equal, people will rate a statement that presents the share of a negative of a statistical fact as truer than a statement that presents the share of its positive, logically equivalent, counterpart:

H1: Negatively framed statistical statements will yield higher truth ratings than positively framed statistical statements.

Conditional Effects of News Media as a (Trusted) Source

As discussed, an underlying assumption in the existing literature is that negatively framed statistics are perceived truer than positively framed ones because people have learned to associate negative information with a perceived reliable source—news media. This assumption presupposes that people (a) use information about the source as a cue to whether a statistical statement can be considered true and (b) that people find news media to be a trustworthy source. In an experimental study, Koch and Peter (2017) found that when respondents were asked to guess whether statistical statements came from a news report or a political speech, positively framed statements were more likely to be judged as coming from a political speech, whereas negatively framed statements were more likely to be judged as coming from a news report. These findings support the hypothesis that people have learned to associate negative information with news reporting. However, to the best of our knowledge, no studies have investigated directly whether being informed that news media are the source can moderate framing effects on the perceived truth in statistical statements. Neither do we know of any studies in which heterogeneous effects among individuals who trust and do not trust news media have been explored.

Whether there are such conditional effects of trust in news media as the source of a statement is important to study for several reasons. First, if trusted sources of information (like news media) directly or indirectly lead people to trust predominantly negatively framed information, this can have negative consequences for society in terms of increased political cynicism (Soroka & McAdams, 2015) and negative judgments of political representatives (Naurin et al., 2019). Second, while news media seem to remain a generally highly trusted source in many societies (Hanitzsch et al., 2018; Weniger et al., 2019), studies suggest that media trust can be quite volatile (Gronke & Cook, 2007; Hanitzsch et al., 2018) and that trust in news media has become increasingly polarized between political groups (Barthel & Mitchell, 2017). These findings suggest that the relationship between negative framing and truth perceptions may change, at least among certain segments of the public. Studying the role of news media as a source, and of news media trust, in the association between framing and perceived truth can inform these important debates.

Furthermore, related research on emphasis framing has shown that trust in the source can condition framing effects (Druckman, 2001) and research on the correction of misperceptions has found that people are more likely to correct misperceptions when they receive factually correct information from a credible source (Vraga & Bode, 2017). By investigating whether these effects extend to people’s perceptions of the truth in statistical statements, we can improve the understanding of the nature of equivalence framing effects and how they may potentially be mitigated.

In this article, we expect that being informed that news media are the source of a statistical statement will reduce the effects of valence as a cue to truth in the statement. Given that people have come to trust facts that are negatively framed because they associate it with a (trusted) source—news media—negative framing could be seen as an indirect cue to source credibility, especially useful when information on the source is absent. When a person is informed that a statement is disseminated by news media, however, the perceived credibility of news media as a source may serve as an assurance of the truth in the facts conveyed, making the (negative) valence cue less “needed.” Hence, we formulate the following hypothesis:

H2a: If people are informed that news media are the source of a statistical statement, the “positive” effect of negative (vs. positive) framing on perceived truth in the statement will be weaker than when not informed about a source.

While we expect that being informed that news media are the source of a statistical statement will reduce the impact of framing on truth perceptions, H2a should be refined, taking into account the individual’s level of trust in the news media. Even if, on an aggregate level, news media trust is quite high, for people with lower levels of trust, being informed that news media are the source of a statement should provide a rather weak cue to credibility, thereby reducing its moderating impact on framing effects. Hence, we hypothesize that

H2b: The moderating effects of being informed that news media are the source will be weaker for individuals with lower levels of news media trust compared with individuals with higher levels of trust.

Conditional Effects of Existing Opinions

Besides (trust in) news media as the source of a statement, we expect that people’s existing opinions on the issue will moderate the framing effects. Most communication research that studies prior opinions as a moderator draws on a theoretical framework derived from the psychology of information processing, which suggests that people are “dissonance-averse” (Festinger, 1957) and strive to come to conclusions that confirm their existing opinions when processing information (Kunda, 1990). People are often biased information processers and have an easier time accepting information that aligns with their own opinions while being more likely to counter-argue and reject information that challenges them. In support of this framework, earlier studies of equivalence framing found no effects on behavioral intentions when the frames pertained to topics on which people already held strong beliefs (abortion; Marteau, 1989), and that positive framing even had inverse (negative) effects on support for a policy among respondents who had strong negative prior attitudes on the issue (estate taxes; Fatemi et al., 2008). Similarly, related studies of emphasis framing have shown that people’s preexisting beliefs can moderate framing effects on policy support (Brewer, 2002; Shen & Hatfield-Edwards, 2005) and voting intentions (Barker, 2005; Schemer et al., 2012), with weaker effects among individuals whose preexisting beliefs are challenged by the frames. Furthermore, research on misperceptions has shown that corrective messages are less efficient in reducing misperceptions if the correct information contradicts people’s political preferences (e.g., Nyhan & Reifler, 2010).

Based on the psychological theories that, when processing information, people will be motivated to accept information that confirms their existing beliefs and reject information that contradicts them, and previous findings supporting those expectations, we anticipate that although people, on an aggregate level, may consider negatively framed statistics truer than positive, these effects will be weaker among individuals whose existing opinions are challenged by the (negative) frame. Consider, for example, a person who believes that society is becoming more violent and wants tougher interventions to stop crime. If this person is exposed to statistics on violent crimes and is motivated to “confirm” his or her existing belief that society is becoming more violent, then she or he should be more likely to find a negatively framed statement saying that, for example, “3% of violent crimes result in death” to be true, than a positively framed statement saying that “97% of violent crimes are not resulting in death.” For a person who holds the opposite views on violence (i.e., believes that society is becoming less violent and wants to maintain the status quo), the difference in perceived truth between the positively and negatively framed version of the statistics should be weaker. Hence, we formulate the following hypothesis:

H3: The “positive” effects of negative (vs. positive) framing on the perceived truth in statistical statements will be weaker among individuals whose issue-specific opinions are challenged by the negative frame.

Method and Design

The hypotheses were tested through a survey experiment, fielded between October 26 and November 19, 2020. The experiment was embedded in a standing web survey panel with members of the Swedish public, administered by the Laboratory of Opinion Research (LORE) at the University of Gothenburg in Sweden. A subsample of 5,228 self-recruited members of the panel—prestratified on educational level, sex, and age to resemble the Swedish population—was invited, of which 2,909 participated in the study, giving a net participation rate of 53% (AAPOR RRR5 = 50%). When deciding on the sample size, we aimed to maximize statistical power while avoiding respondent fatigue. Considering these factors, we arrived at a sample of approximately 3,000 respondents and 15 statistical statements, giving us about 40,000 units of analyses in total.² This large number of observations should minimize the risk of failing to detect substantially important effects; however, it also enables the detection of potentially very minor effects at a high statistical confidence level. Hence, it is important to pay attention to the substantive effect sizes.

To be included, the participants needed to be at least 18 years of age. A breakdown of the sample on demographic variables is presented in Online Appendix A. The study was approved by the Swedish National Ethics Review Board in September 2020 (Approval ID: 2020-03603) and a pre-analysis plan was registered before the data were inspected by any of the researchers, in March 2021 (Lindgren et al., 2021).

The experiment had a 2 × 2 factorial mixed-subject design, with one within-subjects factor (framing) with two levels (positive vs. negative) and one between-subjects factor (source) with two levels (news media source vs. no source).

Treatments

To measure framing effects, we manipulated the framing of 15 statistical statements on three salient societal issues: “climate change,” “vaccination,” and “immigration,” to create formally equivalent positively and negatively framed versions of each statement. All statements were factually correct and drawn from official sources such as the United Nations, Swedish Statistics, and World Health Organization. The framing condition was randomized within-subject and for each statement, leading to 15 repeated trials in total for each respondent. It was counterbalanced by randomizing the order in which the statements appeared to the subjects.³

In line with previous literature focusing on the type of equivalence framing that targets the manipulation of valence (e.g., Hilbig, 2009; Koch & Peter, 2017; Levin et al., 1998), we define equivalence framing as the manipulation of the valence of information by presenting the share of a population that belongs to either a positive or a negative equivalent of the same property. The factual statements were thus chosen based on the rationale that the statistics presented should have one “positive” and one “negative” equivalent that could be respectively highlighted. To exemplify, on the climate domain, one statement presented statistics on the number of species that are threatened by extinction. Here, we consider “being under threat of extinction” to be negative and “not being under threat of extinction” as its logical positive counterpart. Hence, the negative version of the statement read, “Today, about 25 percent of the world’s animal and plant species are under threat of extinction,” and the positive counterpart, “Today, about 75 percent of the world’s animal and plant species are under no threat of extinction.” English translations of the statistical statements can be found in Online Appendix B.

To reiterate, equivalence framing refers to the presentation of the same facts by logical counterparts; hence, a frame can be considered negative (or positive) regardless of how the statistics are viewed in absolute terms (for example, most people may perceive an unemployment rate of 60% as quite negative both when presented as such and when presented in terms of its positive counterpart, 40% employed). While the framing effects are assessed only in relative terms, however, it should be acknowledged that if a statistical estimate is viewed extremely negatively, there may be a ceiling effect where the negative connotations of the estimate will render the negative frame redundant. The fact that we use a within-subject design and average out the framing effects over 15 statistical statements that to varying degrees may be perceived as positive and negative in absolute terms should ease the risk of underestimating the effects due to such ceiling effects. Another aspect that should be considered here is that, although news media tend to be negatively biased in the reporting on and framing of events (e.g., Damstra & Boukes, 2021; Soroka & McAdams, 2015), some issues may be more predisposed to negative framing in the news than others; for example, news media may be more likely to highlight the negative when presenting information on climate change, whereas they may be more likely to frame information on vaccines neutrally or even in positive terms. The risk that we find effects that depend heavily on how media tend to frame a particular issue should be eased by the inclusion of statements on several different issue domains.

We manipulated the source by informing some respondents that the statistical statements came from “news media,” whereas the others received the statements with source omitted (control group). This factor was manipulated between subjects and occurred in two places: in the introduction to the study and in the questions measuring perceived truth after each statement. In the introduction, one group was informed that they would be presented statistical statements (control group) and the other saw the same information but with news media added as the source: “You will now see a series of statistical statements [presented by news media] regarding some societal issues.” In the questions measuring perceived truth, which were repeated after each statement, the control group consistently saw questions without a source, and the treatment group consistently saw questions with news media added as the source: “To what extent do you agree with the following statements [presented by news media]?”

When considering the operationalization of the news media factor, some choices should be discussed. First, we use the generic term news media rather than quoting specific media outlets. This operationalization fits theoretically with the learning hypothesis, which stipulates that (trust in) news media as a general phenomenon has created the association between negativity and perceived trustworthiness of information (e.g., Hilbig, 2009); however, it does not account for the possibility that some people yet differentiate between specific outlets and have more confidence in some. While this may be a significant issue in countries like the United States, where news media are politically polarized (Daniller et al., 2017), it should be less of a risk in countries with less polarized media systems like Sweden. In Sweden, the polarization of news media is low and the difference between traditional news media and political alternative media is rather clear (all major media are for example traditional news media, while political alternative media are largely confined to an online presence, Nord & Grusell, 2021). General news media trust is also quite high and stable in Sweden (Andersson & Martinsson, 2021) and perceptions of the societal value of different traditional news outlets are substantially correlated (Andersson, 2021). Based on this, we assume that Swedes, in general, understand “news media” as referring to traditional news media and that trust in one news outlet correlates with trust in others.

We also chose to only vary whether “news media” were referred to as the source (not for example the original source quoted in the media reports) and to compare the effects with a group that received no source information. This design offered an effective and internally valid way to investigate the part of the learning hypothesis that targets the association between news media, negativity, and credibility. However, it should be acknowledged that alternative designs, including more diverse sources, and/or designs where sources that could be associated with positive framing (such as politicians, Koch & Peter, 2017) were used as the reference point, could yield stronger effects of the news media cue. This should be considered when evaluating the findings of this study.

Measures

To estimate the perceived truth in the statistical statements, we asked participants to, for each statement, indicate to what extent they agreed with three items (previously used by Koch & Peter, 2017): “I believe that the statistics are credible,” “I have trust in the statistics,” and “I suppose the statistics are correct.” Respondents answered on 7-point scales ranging from 1 = do not agree at all to 7 = completely agree, with a designated midpoint of 4 = neither agree nor disagree. The responses to the three items were summed in additive indices and normalized to range from 0 = low perceived truth to 1 = high perceived truth. Mean values for the 15 indices ranged between M = .53 (SD = .30) and M = .66 (SD = .25), and Cronbach’s alpha ranged between .96 and .98.

To investigate the moderating role of trust in the source (news media) for framing effects (H2b), we used a multi-item question tapping into trust in news media proposed by Strömbäck et al. (2020): “Generally speaking, to what extent do you agree or disagree with the following statements about the news media in Sweden?” with the items being “The news media. . .” “. . . are fair when covering the news,” “. . . are unbiased when covering the news,” “. . . tell the whole story when covering the news,” “. . . are accurate when covering the news,” and “. . . separate facts from opinions when covering the news.” Respondents answered each item on a 7-point scale ranging from 1 = do not agree at all to 7 = completely agree. For our analyses, the items were summed in an additive index and normalized to range from 0 = low trust in news media to 1 = high trust in news media (Cronbach’s α = .95, M = .55, SD = .27).

To investigate the moderating role of issue-specific opinions (“opinion resonance”; H3), we needed measures that indicate whether a participant’s preexisting opinion on the issue domains covered would align with or be challenged by the valence of, respectively, the positive and negative frames. For this, we used two randomized, multi-item questions capturing participants’ views on the three issue domains. First, we asked a question addressing attitudes to environmental, vaccine, and immigration policies, starting with, “What do you think about the following political proposals?” For each item, respondents answered on a 7-point scale with designated endpoints of 1 = very bad proposal and 7 = very good proposal, and a designated midpoint of 4 = neither good nor bad proposal. Second, we asked a question tapping into beliefs on the same issue domains: “Thinking about society today, to what extent do you believe that the following statements are true or false?” with response options of 1 = very certain it’s false, 2 = somewhat certain it’s false, 3 = uncertain whether it’s true or false, 4 = somewhat certain it’s true, and 5 = very certain it’s true. Both question batteries included two items tapping into each issue domain (including some fill-outs about other issues), giving four items in total per issue domain.⁴ The answers were used to create indices comprising opinions on each domain. The items included in the respective index were coded based on the logic that a negative frame should correspond with a low value on the opinion variable and a positive frame should correspond with a high value. For example, in the case of vaccines, a negatively framed version of a statement such as “the latest version of the human papillomavirus (HPV) vaccine does not protect against viral strains that cause about 10% of all cervical cancers” should resonate with the opinions of a respondent with low values on the vaccine index (believe that vaccines are bad, against pro-vaccine policies), whereas it should challenge the views of an individual with high values (believe that vaccines are good, support pro-vaccine policies). All indices were normalized to range from 0 = “high resonance with negative framing” to 1 = “high resonance with positive framing.”⁵ For the analyses, new variables were created where respondents were assigned a fixed value for “climate opinion,” “vaccine opinion,” and “immigration opinion.” These variables were used to match the issue-specific opinion measures with the measures of truth perceptions on corresponding issue domains.

All questions addressing the moderators were asked in the same survey, before the experimental manipulations to ensure their independence.

Analytic Strategies

For analyses, we use linear mixed-effects models with restricted maximum likelihood as the estimation procedure, and the Kenward–Roger approximation to compute degrees of freedom. We have two manipulated independent variables, the within-subject dichotomous variable “framing,” and the between-subject dichotomous variable “source.” The dependent variable is the “perceived truth index.” The first moderating variable is the media trust index and the second is the issue opinion indices, for which we use grand-mean-centered versions. To account for nonindependence in the truth ratings caused by subjects, we include a by-subject random intercept and a by-subject random slope for the within-subject predictor framing. Because all items are rated multiple times, we also include a by-item random intercept to account for nonindependence caused by items. Whenever the models yield statistically significant interactions (cutoff value p < .05), we assessed the marginal effects of framing at different values of the variables included in the interaction terms. We had no exclusion criteria and data from subjects with partially missing data were included in the analyses.

Results

Results from the analyses are presented in Table 1, which displays the fixed parts of the mixed-effects regression models. Starting with H1 (Model 1), we found that, in support of the hypothesis, positively framed statistical statements were perceived as less true than negatively framed statements (B = −.10, SE = .00, p < .001). These results replicate findings from previous studies, indicating a negativity bias when people evaluate the truth in statistical statements.

Table 1.

Effects of Equivalence Framing, News Media as Source, Media Trust, and Issue-Specific Opinions on Perceived Truth in Statistical Statements.

Predictor variable	Model 1		Model 2		Model 3		Model 4		Model 5		Model 6^a
Predictor variable	B (SE)	β	B (SE)	β	B (SE)	β	B (SE)	β	B (SE)	β	B (SE)	β
Frame (ref = neg)	−.10*** (.00)	−.19	−.11*** (.00)	−.20	−.11*** (.00)	−.20	−.10*** (.00)	−.19	−.11*** (.00)	−.20	−.11*** (.00)	−.21
Source (ref = no source)			.01* (.01)	.02	.01* (.01)	.02			.01* (.01)	.02	.01* (.01)	.02
Frame × Source			.01* (.01)	.02	.01* (.01)	.02			.01* (.01)	.02	.01* (.01)	.02
Media trust					.18*** (.02)	.18					.20*** (.02)	.19
Frame × Media Trust					.04** (.01)	.03					.01 (.01)	.01
Source × Media Trust					.06** (.02)	.04					.06** (.02)	.04
Frame × Source × Media Trust					−.01 (.02)	−.00					.00 (.02)	.00
Opinion							−.08*** (.01)	−.07	−.09*** (.01)	−.08	−.11*** (.01)	−.09
Frame × Opinion							.30*** (.01)	.17	.31*** (.01)	.18	.32*** (.01)	.19
Source × Opinion									.02 (.01)	.01	.01 (.02)	.01
Frame × Source × Opinion									−.02 (.02)	−.01	−.01 (.02)	.00
Media Trust × Opinion											−.15*** (.03)	−.04
Frame × Media Trust × Opinion											.20*** (.05)	.04
Source × Media Trust × Opinion											−.05 (.05)	−.01
Frame × Source × Media Trust × Opinion											.07 (.07)	.01
Issue (ref = climate)
Integration											−.01*** (.00)	−.02
Vaccines											.04*** (.00)	.07
Constant	.66*** (.00)		.65*** (.00)		.65*** (.00)		.66*** (.00)		.65*** (.00)		.57*** (.04)
Observations (Subjects)	39,811 (2,726)		39,811 (2,726)		39,811 (2,726)		39,811 (2,726)		39,811 (2,726)		39,811 (2,726)

Note. Linear mixed-effects regression, with restricted maximum likelihood as estimation procedure. Entries are unstandardized beta coefficients (B) with standard errors (SE) in parentheses, and standardized beta coefficients (β), fixed portion of effects. All models include a by-subject random intercept for framing, a by-subject random slope for framing, and a by-item random intercept for framing (see Online Appendix C for the formal equations). “Opinion resonance” range from 0 = “high resonance with negative frame” to 1 = “high resonance with positive frame,” and “Media trust” ranges from 0 = “low trust” to 1 = “high trust.” Continuous predictor variables are grand-mean-centered.

Model 6 includes a variable indicating which issue domain the truth perceptions items belong to, as well as educational level, age, sex, political interest, and ideological left-right predispositions for control.

p < .05. **p < .01. ***p < .001.

Turning to H2a, we expected that if people are informed that news media are the source of a statistical statement, the positive effect of negative framing on perceived truth will attenuate. The results (Model 2, Table 1) show a slight positive effect of source on perceived truth (B = .01, SE = .01, p < .05), and, as expected, a positive interaction between framing and source (B = .01, SE = .01, p < .05). The interaction term indicates that the (positive) effect of being exposed to negative relative to positive framing differs due to the source condition, with the effects being smaller for those who receive information that news media are the source. Pairwise comparisons of the predicted marginal effects of being exposed to negative and positive framing show that the negativity bias when evaluating statistics is present both among individuals not exposed (difference = −.11, SE = .00, p < .001) and among those exposed to a source (difference = −.10, SE = .00, p < .001; for all pairwise comparisons, see Online Appendix D). Because the interaction term was significant, the analyses support H2a, that the negativity bias in evaluating the truth in statistics can be attenuated by informing recipients that the news media delivered the information. However, because the difference in truth ratings was quite small (standardized β = .02), the results suggest that being informed that news media are the source has only a limited impact on the “negativity bias” when people evaluate the truth in statistical statements. The marginal effects of framing on perceived truth in the statements for individuals presented and not presented with the news media source are illustrated in Figure 1.

Figure 1.

Marginal effects of framing on perceived truth in statistical statements when exposed to source and not.

While the results suggest only minor moderating effects of the media source, the effects may be larger for individuals with higher levels of media trust. To investigate this possibility, we calculated a third model including the respondents’ self-estimated media trust and a three-way interaction term for Framing × Source × Media Trust. We expected that the moderating effects of being informed that news media were the source would be weaker for individuals with lower levels of media trust than for individuals with higher levels of trust (H2b). The results (Model 3, Table 1) reveal that media trust has a positive effect on the perceived truth in the statements when source and framing are held constant at their base (negative frame, no source; B = .18, SE = .02, p < .001). They also reveal a positive two-way interaction between media trust and framing (B = .04, SE = .01, p < .01) and between media trust and source (B = .06, SE = .02, p < .01). In contrast to our expectation, there is, however, no three-way interaction between framing, source, and media trust (B = −.01, SE = .02, p > .05); hence, H2b is not supported. Together with the fact that we only found a minor main moderating effect of the source cue, these findings lead us to conclude that learning that news media are the source makes little to reduce the tendency to trust negatively framed statements more than positive ones in substantive terms.

Our final hypothesis (H3) stipulated that the positive effects of negative framing on the perceived truth in statistical statements will be weaker among individuals whose issue opinions are challenged by the negative frames. The results are presented in Model 4, Table 1. Looking at the table, we can first see that people’s issue opinions have a negative effect on the participants’ perceptions of the statements, where individuals whose opinions resonate with the positive framing are less likely to perceive the statements to be true than those whose opinions resonate with the negative framing (B = −.08, SE = .01, p < .001). Second, and more important, the results yield a positive interaction between framing and opinions (B = .30, SE = .01, p < .001). In line with our expectations, the interaction term indicates that the more the respondents’ opinions resonate with the positive frames, the more likely they become to perceive the positively framed statements to be true⁶ and this interaction effect is noteworthy larger (standardized β = .17) than the one for source.

To scrutinize the interaction further, we estimated the predicted marginal effects of framing on perceived truth for individuals with different levels of resonance with the negative and positive frames. The results, which are plotted in Figure 2, illustrate how the differences in perceived truth when exposed to negative versus positive framing shrink as individuals’ opinions become more resonant with the positive frames.

Figure 2.

Marginal effects of framing on perceived truth in statistical statements at levels of opinion resonance.

Pairwise comparisons of the predicted marginal effects of framing (see Table 2) confirm that the differences in truth ratings between negative and positive framing are substantively larger for individuals whose preexisting opinions resonate with the negative frames (10th percentile on opinion indices: difference = −.19, SE = .00, p < .001), than for individuals whose opinions resonate with the positive framing (90th percentile: difference = −.02, SE = .00, p < .001). In support of H3, this confirms that the tendency to trust negatively framed versions of statistical statements more than positive ones declines when the negative frame is contradictive of the recipient’s existing opinions. However, it should also be noted that, even among individuals with issue opinions that can be considered strongly challenged by the negative framing (90th percentile on opinion indices), those statements are rated truer than the positive ones. This underscores that negativity bias is a powerful mechanism in explaining people’s trust in statistical information.

Table 2.

Differences in Predicted Marginal Effects of Framing on Perceived Truth in Statistical Statements at Levels of Opinion Resonance.

Level of opinion resonance^a	Comparison treatment groups			Difference	SE	Z
10th percentile	Positive frame	vs.	Negative frame	−.19	.00	−47.59***
25th percentile	Positive frame	vs.	Negative frame	−.15	.00	−48.61***
50th percentile	Positive frame	vs.	Negative frame	−.10	.00	−39.17***
75th percentile	Positive frame	vs.	Negative frame	−.06	.00	−19.33***
90th percentile	Positive frame	vs.	Negative frame	−.02	.00	−5.08***

Lower percentiles on opinion resonance indicate resonance with negative framing; higher percentiles indicate resonance with positive framing.

***

p < .001.

Before concluding, we performed an exploratory probing of truth perceptions, including all predictors simultaneously as well as “issue-domain” and some demographic variables for control. The results, which are presented in Table 1 (Model 6), confirm those found when testing the hypotheses in separate models and with control variables omitted. Interesting to note here is that while including the issue domain did not change the main results, the issue domain was associated with the truth ratings, where statements on immigration were less likely, and statements about vaccines more likely, to be perceived as true compared with climate change statements. As we discussed in the “Method and Design” section, climate change may be framed more one-sidedly negative than vaccines. Familiarity with positive framing of vaccines in the news could then perhaps explain the higher credibility of the vaccine statements relative to those on climate change. Although explaining these findings is beyond the scope of this study, they could be explored in the future to find out the extent to which framing effects may be sensitive to the topic and why.

Concluding Discussion

To conclude, this study offers the following key takeaways. First, the study confirms earlier findings that all else being equal, negatively framed statistical statements are perceived as truer than their positive counterparts. Second, it offers evidence that those framing effects may be moderated by, in particular, people’s preexisting opinions. Specifically, we found that the more the participants’ prior opinions resonated with a positive framing of the statistical statements, the less effective were the negative framing in increasing perceived truth in the statements. Third, the study shows that news media as the source only to a limited degree moderated the framing effects and that individuals with lower levels of media trust were about equally susceptible to this cue as those with higher levels of trust. These findings suggest that being informed that news media are the source of a statistical statement has little substantive power in reducing tendencies to trust negatively framed versions more than positive ones.

A possible explanation for the limited effects of the media cue relates to the relatively unconscious nature of framing effects. While previous research has found that trust in news media plays a moderating role for relatively conscious media effects like agenda-setting effects (Tsfati, 2003), such effects are weaker for less conscious effects such as framing and priming (Miller & Krosnick, 2000; for similar discussion on the role of trust in the context of misinformation retraction, see Stubenvoll & Matthes, 2022). Our findings that framing effects (while marginally sensitive to naming media as a source) are not moderated by media trust align with such findings. Another aspect to consider is that our source manipulation was relatively weak. First, we referred to “news media” as a generic term instead of specific news outlets. Because the traditional media in Sweden are characterized by low polarization and the difference between traditional news media and political alternative media is quite clear (Nord & Grusell, 2021), we assume that our respondents understood the term news media as referring to traditional news media. It should nevertheless be noted that studies of more specific news outlets could result in more pronounced effects, as could a study in contexts where the media system is more polarized. Second, we estimated the effects of the media cue by comparing it with claims where the source was omitted. This design offered an effective way to investigate the hypothesis that people use negative framing as an indirect cue to fact credibility by associating it with a perceived credible source (news media). However, one should bear in mind that a comparison with other (allegedly less) credible sources (such as politicians, Koch & Peter, 2017) could yield larger effects than those we see here. This question lies outside the scope of this article but may be explored in future research. Such future studies could also include follow-up questions to respondents regarding what source they perceived to be behind the statements, to ensure that they absorbed this information correctly.

In addition to the above, two further limitations of the study should be noted. First, the experiment has limited external validity in the sense that news media typically attribute statistical statements to a primary source (e.g., an expert report) and embed the information in news stories that provide context to the same. The second limitation concerns possible priming effects. To ensure the independence of the moderators, we asked respondents those questions before the experiment. While measuring the moderators a priori is vital for valid tests of their effects on the primary relationship, however, this comes with the risk that the respondents are led to think about these issues in a way that may affect their response to the experiment. This risk could be addressed in future research by measuring the moderators in an independent, preexperimental survey.

While the limited external validity of this experiment must be considered, it offers initial insights into how the negativity bias in perceptions of statistical evidence can be moderated by real-life factors. These findings can inform future research examining how to communicate factual information to diverse publics. While the study confirmed that negative framing was more effective in producing credibility when accounting for the moderators, as noted in the introduction, there are situations when a negative framing can be undesirable, such as when it may lead to the avoidance of certain behavior and political distrust (Bigman et al., 2010; Soroka & McAdams, 2015; Tafuri et al., 2014). As the negativity bias in truth perceptions was clearly reduced among individuals with opinions that resonated with a positive framing, the potential losses in credibility by using positive framing may be offset by the positive effects it can have on other aspects like compliance with recommended measures (e.g., Bigman et al., 2010), something which could make positive framing when communicating with these groups advisable. However, when communicating information to individuals whose beliefs are threatened by a positive framing—the ones most likely to display a “negativity bias” when evaluating the statements’ credibility—additional strategies that could alleviate rejection of facts if framed positively may be considered. There are different ways in which this could be done. Research on motivated reasoning suggests that affirming people’s values in relation to a counter-attitudinal message can reduce message rejection and lead people to express more accurate factual beliefs (Garrett & Weeks, 2013; Lewandowsky et al., 2012). Statistical information could thus be embedded in a message that confirms a group’s values and/or commends the recipients for considering alternative perspectives. For example, when health authorities are to inform about the effectiveness of a vaccine to vaccine skeptics, those most at risk of questioning the truth of vaccine information formulated in terms of its effectiveness (relative to its ineffectiveness), they should make sure to respond to their concerns and confirm the legitimacy of the same. They may also consider formulating the message so that the recipients can distance themselves from a self-focused perspective (Lewandowsky et al., 2012), for example, by presenting what effects the vaccine can have on global health rather than on the individual.

In conclusion, the present study offers insights into the processes underlying skepticism toward statistical evidence and the power and limitations of equivalence framing in these processes. While the findings suggest that the effects of a negative framing were considerably weaker among individuals whose prior views were challenged by these frames, it should be reiterated that the negativity bias was never fully eradicated even among these individuals. Hence, while the study provides ground for future research on the possibilities of tailoring factual messages to groups with different viewpoints, it also serves as a reminder that negativity is a crucial mechanism for explaining perceived truth in facts. More research on how this type of bias can be reduced is therefore imperative. For example, studies of the effects of communicating information through various sources (beyond news media) can be of great value, as can studies of how people with different beliefs can be approached to become less reluctant to accept information that is presented in a positive way.

Supplemental Material

sj-docx-1-jmq-10.1177_10776990221117117 – Supplemental material for Trusting the Facts: The Role of Framing, News Media as a (Trusted) Source, and Opinion Resonance for Perceived Truth in Statistical Statements

Supplemental material, sj-docx-1-jmq-10.1177_10776990221117117 for Trusting the Facts: The Role of Framing, News Media as a (Trusted) Source, and Opinion Resonance for Perceived Truth in Statistical Statements by Elina Lindgren, Torun Lindholm, Rens Vliegenthart, Hajo G. Boomgaarden, Alyt Damstra, Jesper Strömbäck and Yariv Tsfati in Journalism & Mass Communication Quarterly

Footnotes

Acknowledgements

The authors gratefully acknowledge funding from Riksbankens Jubileumsfond (RJ) for the advancements of the Humanities and Social Sciences (Grant No: M18-0310:1). The authors are also grateful to the editors and the anonymous reviewers of Journalism & Mass Communication Quarterly for their feedback and comments on the manuscript, and to the Laboratory of Opinion Research (LORE) at University of Gothenburg, for their assistance with data collection.

Authors’ Note

A pre-analysis plan is available at the project’s Open Science Framework page (), and data can be shared upon request to the corresponding author.

Data Availability Statement

Data used for the analyses can be obtained from the corresponding author upon request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is part of the research project ‘Knowledge Resistance: Causes, Consequences, Cures,’ funded by Riksbankens Jubileumsfond for the Advancement of the Humanities and Social Sciences.

ORCID iDs

Elina Lindgren

Jesper Strömbäck

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Elina Lindgren is a researcher in political communication at the Department of Journalism, Media and Communication, University of Gothenburg. She received her PhD in political science from the Department of Political Science at the University of Gothenburg in 2018. Her research interests are in political communication, public opinion, and public knowledge.

Torun Lindholm is professor in social psychology and deputy head of the Department of Psychology at Stockholm University, Sweden. She received her PhD from the Department of Psychology at Stockholm University in 1999. Her research interests focus on topics in social and cognitive psychology.

Rens Vliegenthart is a professor of media and society and scientific director of the Amsterdam School of Communication Research (ASCoR) at the University of Amsterdam. He received his PhD in social sciences from the Vrije Universiteit in Amsterdam in 2007. His research focuses on the role of media in society, and specifically, the way media and politics interact.

Hajo G. Boomgaarden is professor of empirical social science methods at the Department of Communication at the University of Vienna. He received his PhD from the Faculty of Social and Behavioural Sciences at University of Amsterdam in 2007. His research focus is on various aspects of political communication and on computational methods for communication science.

Alyt Damstra is a postdoctoral researcher at the ASCoR at the University of Amsterdam. She received her PhD from the Faculty of Social and Behavioural Sciences at University of Amsterdam in 2020. Her research agenda includes media effects on political processes and public opinion.

Jesper Strömbäck is professor in journalism and political communication at the Department of Journalism, Media and Communication, University of Gothenburg. He received his PhD in journalism from the Department of Journalism, Media and Communication at Stockholm University in 2001. His research focuses on political communication, political journalism, and media effects.

Yariv Tsfati is professor at the Department of Communication, University of Haifa. He received his PhD from the Annenberg School for Communication at the University of Pennsylvania in 2001. He studies political communication with a focus on audience perceptions of news media and their implications.

References

Andersson

(2021). Mediers samhällsvärde och oberoende spelar stor roll för förtroendet [Media’s societal value and independence play a major role in trust]. In Andersson

Carlander

Grusell

Öhberg

(Eds.), Ingen anledning till oro (?) [No cause for concern (?)] (pp. 57–73). The SOM Institute, University of Gothenburg.

Andersson

Martinsson

(2021). Swedish trends 1986–2020. The SOM Institute.

Andreescu

Keeling

(2012). Explaining the public distrust of police in the newest European Union countries. International Journal of Police Science & Management, 14(3), 219–245. https://doi.org/10.1350/ijps.2012.14.3.278

Barker

(2005). Values, frames, and persuasion in presidential nomination campaigns. Political Behavior, 27(4), 375–394. https://doi.org/10.1007/s11109-005-8145-4

Barthel

Mitchell

(2017). Americans’ attitudes about the news media deeply divided along partisan lines. Pew Research Center. http://www.journalism.org/2017/05/10/americans-attitudes-about-the-news-media-deeply-divided-along-partisan-lines/

Bigman

Cappella

Hornik

(2010). Effective or ineffective: Attribute framing and the human papillomavirus (HPV) vaccine. Patient Education and Counseling, 81, 70–76. https://doi.org/10.1016/j.pec.2010.08.014

Brewer

(2002). Framing, value words, and citizens’ explanations of their issue opinions. Political Communication, 19(3), 303–316. https://doi.org/10.1080/01957470290055510

Cacciatore

M. A.

Iyengar

Scheufele

(2016). The end of framing as we know it. . . and the future of media effects. Mass Communication and Society, 19(1), 7–23.

Damstra

Boukes

(2021). The economy, the news, and the public: A longitudinal study of the impact of economic news on economic evaluations and expectations. Communication Research, 48(1), 26–50. https://doi.org/10.1177/0093650217750971

10.

D’Angelo

Lule

Neuman

Rodriguez

Dimitrova

Carragee

(2019). Beyond framing: A forum for framing researchers. Journalism & Mass Communication Quarterly, 96(1), 12–30. https://doi.org/10.1177/1077699018825004

11.

Daniller

Allen

Tallevi

Mutz

(2017). Measuring trust in the press in a changing media environment. Communication Methods and Measures, 11(1), 76–85.

12.

Druckman

(2001). On the limits of framing effects: Who can frame? Journal of Politics, 63(4), 1041–1066. https://doi.org/10.1111/0022-3816.00100

13.

Fatemi

Hasseldine

Hite

(2008). Resisting framing effects: The importance of prior attitude on estate tax preferences. Journal of the American Taxation Association, 30(1), 101–121. https://doi.org/10.2308/jata.2008.30.1.101

14.

Festinger

(1957). A theory of cognitive dissonance (Vol. 2). Stanford University Press.

15.

Field

Wright

(2011). A primer on using multilevel models in clinical and experimental psychopathology research. Journal of Experimental Psychopathology, 2(2), 271–293.

16.

Gamliel

Peer

(2010). Attribute framing affects the perceived fairness of health care allocation principles. Judgment and Decision Making, 5(1), 11–20. https://ideas.repec.org/a/jdm/journl/v5y2010i1p11-20.html

17.

Garrett

Weeks

(2013, February 23–27). The promise and peril of real-time corrections to political misperceptions. In Proceedings of the 2013 conference on computer supported cooperative work. https://doi.org/10.1145/2441776.2441895

18.

Gronke

Cook

(2007). Disdaining the media: The American public’s changing attitudes toward the news. Political Communication, 24(3), 259–281. https://doi.org/10.1080/10584600701471591

19.

Hanitzsch

Van Dalen

Steindl

(2018). Caught in the nexus: A comparative and longitudinal analysis of public trust in the press. The International Journal of Press/Politics, 23(1), 3–23. https://doi.org/10.1177/1940161217740695

20.

Hilbig

(2009). Sad, thus true: Negativity bias in judgments of truth. Journal of Experimental Social Psychology, 45(4), 983–986. https://doi.org/10.1016/j.jesp.2009.04.012

21.

Hilbig

(2012a). Good things don’t come easy (to mind). Experimental Psychology, 59(1), 38–46. https://doi.org/10.1027/1618-3169/a000124

22.

Hilbig

(2012b). How framing statistical statements affects subjective veracity: Validation and application of a multinomial model for judgments of truth. Cognition, 125(1), 37–48. https://doi.org/10.1016/j.cognition.2012.06.009

23.

Horne

Powell

Hummel

Holyoak

(2015). Countering antivaccination attitudes. Proceedings of the National Academy of Sciences of the United States of America, 112(33), 10321–10324. https://doi.org/10.1073/pnas.1504019112

24.

Kahan

Braman

Cohen

Gastil

Slovic

(2010). Who fears the HPV vaccine, who doesn’t, and why? An experimental study of the mechanisms of cultural cognition. Law and Human Behavior, 34(6), 501–516. https://doi.org/10.1007/s10979-009-9201-0

25.

Koch

Peter

(2017). Effects of equivalence framing on the perceived truth of political messages and the trustworthiness of politicians. Public Opinion Quarterly, 81(4), 847–865. https://doi.org/10.1093/poq/nfx019

26.

Kraft

Lodge

Taber

(2015). Why people “don’t trust the evidence”: Motivated reasoning and scientific beliefs. The ANNALS of the American Academy of Political and Social Science, 658(1), 121–133. https://doi.org/10.1177/0002716214554758

27.

Kunda

(1990). The case for motivated reasoning. Psychological Bulletin, 108(3), 480–498. https://doi.org/10.1037/0033-2909.108.3.480

28.

Levin

(1987). Associative effects of information framing. Bulletin of the Psychonomic Society, 25(2), 85–86. https://doi.org/10.3758/BF03330291

29.

Levin

Schneider

Gaeth

(1998). All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76(2), 149–188. https://doi.org/10.1006/obhd.1998.2804

30.

Lewandowsky

Ecker

Seifert

Schwarz

Cook

(2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131.

31.

Lindgren

Lindholm

Vliegenthart

Boomgaarden

H. G.

Damstra

Strömbäck

Tsfati

(2021, March 11). OSF-prereg-euqivalence. Center for Open Science. https://osf.io/85mtf

32.

Marteau

(1989). Framing of information: Its influence upon decisions of doctors and patients. British Journal of Social Psychology, 28(1), 89–94. https://doi.org/10.1111/j.2044-8309.1989.tb00849.x

33.

Masud

Akhtar

Afroz

Al-Amin

Kari

(2015). Pro-environmental behavior and public understanding of climate change. Mitigation and Adaptation Strategies for Global Change, 20(4), 591–600. https://doi.org/10.1007/s11027-013-9509-4

34.

McGraw

Lodge

Jones

(2002). The pandering politicians of suspicious minds. Journal of Politics, 64(2), 362–383. https://doi.org/10.1111/1468-2508.00130

35.

Miller

Krosnick

(2000). News media impact on the ingredients of presidential evaluations: Politically knowledgeable citizens are guided by a trusted source. American Journal of Political Science, 44(2), 301–315. https://doi.org/10.2307/2669312

36.

Mitchell

Gottfried

Barthel

Shearer

(2016). The modern news consumer: News attitudes and practices in the digital era. Pew Research Center. https://apo.org.au/node/65498

37.

Naurin

Soroka

Markwat

(2019). Asymmetric accountability: An experimental investigation of biases in evaluations of governments’ election pledges. Comparative Political Studies, 52(13–14), 2207–2234. https://doi.org/10.1177/0010414019830740

38.

Nord

Grusell

(2021). Media and politics in Sweden. In Skogerbø

Ihlen

Ø.

Kristensen

Nord

(Eds.), Power, communication and politics in the Nordic countries (pp. 113–132). Nordicom.

39.

Nyhan

Reifler

(2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330. https://doi.org/10.1007/s11109-010-9112-2

40.

Schemer

Wirth

Matthes

(2012). Value resonance and value framing effects on voting intentions in direct-democratic campaigns. American Behavioral Scientist, 56(3), 334–352. https://doi.org/10.1177/0002764211426329

41.

Shehata

Strömbäck

(2014). Mediation of political realities: Media as crucial sources of information. In Esser

Strömbäck

(Eds.), Mediatization of politics: Understanding the transformation of western democracies (pp. 93–113). Palgrave Macmillan.

42.

Shen

Hatfield-Edwards

(2005). Economic individualism, humanitarianism, and welfare reform: A value-based account of framing effects. Journal of Communication, 55(4), 795–809. https://doi.org/10.1111/j.1460-2466.2005.tb03023.x

43.

Soroka

McAdams

(2015). News, politics, and negativity. Political Communication, 32(1), 1–22. https://doi.org/10.1080/10584609.2014.881942

44.

Strömbäck

Tsfati

Boomgaarden

Damstra

Lindgren

Vliegenthart

Lindholm

(2020). News media trust and its impact on media use: Toward a framework for future research. Annals of the International Communication Association, 44(2), 139–156. https://doi.org/10.1080/23808985.2020.1755338

45.

Stubenvoll

Matthes

(2022). Why retractions of numerical misinformation fail: The anchoring effect of inaccurate numbers in the news. Journalism & Mass Communication Quarterly, 99, 368–389. https://doi.org/10776990211021800

46.

Tafuri

Gallone

Cappelli

Martinelli

Prato

Germinario

(2014). Addressing the anti-vaccination movement and the role of HCWs. Vaccine, 32(38), 4860–4865. https://doi.org/10.1016/j.vaccine.2013.11.006

47.

Tsfati

(2003). Does audience skepticism of the media matter in agenda setting? Journal of Broadcasting & Electronic Media, 47(2), 157–176. https://doi.org/10.1207/s15506878jobem4702_1

48.

Tversky

Kahneman

(1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683

49.

Vraga

Bode

(2017). Using expert sources to correct health misinformation in social media. Science Communication, 39(5), 621–645. https://doi.org/10.1177/1075547017731776

50.

Wagner

Prentice

Saad-Roy

Yang

Grenfell

Levin

Laxminarayan

(2020). Economic and behavioral influencers of vaccination and antimicrobial use. Frontiers in Public Health, 8(975), 1–16. https://doi.org/10.3389/fpubh.2020.614113

51.

Weniger

Bigley

Rhodes

(2019). Trust in the press as an institution. International Journal of Innovative Studies in Sociology and Humanities, 4(9), 6–20.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB