Abstract
Integrating the theories of confirmation bias, narrative persuasion, and presentation order, this study examines how the format of disconfirmation and confirmation messages and the presentation order of these messages in online media change the opinions of individuals who initially supported in-person classes for Fall 2020. A 2 (confirmation message: narrative vs. non-narrative) × 2 (disconfirmation message: narrative vs. non-narrative) × 2 (order: disconfirmation message first vs. second) between-subjects online experiment found that delivering a disconfirmation message after a confirmation message was more effective in eliciting attitude and belief change than delivering it prior to a confirmation message. Moreover, such a recency effect was more pronounced when the disconfirmation message was non-narrative, and the confirmation message was in a narrative format. The theoretical and practical implications for message design and placement in a competitive information environment were discussed.
The proliferation of information on the internet has created a highly competitive environment where individuals encounter messages from diverse sources with conflicting opinions and agendas (Niederdeppe et al., 2015). This phenomenon is particularly evident today, where public debates surrounding various topics, such as COVID-19, have become highly polarized. Recent research has found that 75% of U.S. adults have received conflicting information about COVID-19 (Nagler et al., 2020), some of which may align with their preexisting attitudes while others challenge their viewpoints. While exposure to conflicting views can enhance the recognition and understanding of diverse perspectives necessary for a functional deliberative democracy (Price et al., 2002), it can also lead to attitude ambivalence and inaction (Mutz, 2002). Paradoxically, exposure to divergent views may even reinforce existing opinion polarization (Bail et al., 2018). These outcomes arise from the distinct ways people process attitude-consistent and attitude-inconsistent information.
To simplify information acquisition and maintain a consonant state, individuals heavily rely on their pre-existing values and attitudes, a phenomenon known as confirmation bias (Festinger, 1962; Talluri et al., 2018). While confirmation bias minimizes cognitive dissonance, it may hinder individuals from arriving at an accurate understanding of reality, particularly when their prior opinions are based on misconceptions (W. Hart et al., 2009). Confirmation bias and the subsequent selective information acquisition and processing contribute to echo chambers at the micro-level and audience fragmentation and opinion polarization at the macro-level, diminishing the potential positive effects of information diversity facilitated by a high-choice media and opinion market (Porten-Cheé & Eilders, 2019).
This study explores strategies that potentially mitigate confirmation bias. Narrative persuasion, the attempt to influence others through storytelling, is known for its advantages in generating less reactance and covertly changing attitudes (Dal Cin et al., 2004; Kreuter et al., 2007; Ratcliff & Sun, 2020). Thus, narratives have been advocated as a potential tool to effectively deliver disconfirming information, defined as information inconsistent with preexisting values and beliefs. While prior studies have suggested that cognitive elaboration of disconfirmation messages can reduce confirmation bias (Hernandez & Preston, 2013; Knobloch-Westerwick et al., 2020), existing research has primarily focused on how confirmation bias influences subsequent information processing, specifically confirmation (i.e., attitude-consistent information) or disconfirmation information (i.e., attitude-inconsistent information) alone. Less attention has been given to how exposure to both confirmation and disconfirmation messages simultaneously influences persuasion outcomes and how the order of these messages moderates their combined effects. The interaction between confirmation and disconfirmation messages can lead to complex and unpredictable results. While confirmation bias may lead individuals to process subsequent information in a way that supports their existing beliefs and disregards disconfirming information, the format of the information, and the order in which the information is presented—whether a confirmation message is followed by a disconfirmation message or vice versa—can lead to different experiences of cognitive dissonance, such as affecting message perceptions, levels of elaboration and information recall (Crano, 1977; Han et al., 2020; Haugtvedt & Wegener, 1994).
Taken together, by integrating theories of confirmation bias, narrative persuasion, and message order, we aim to examine the influence of narrative (vs. non-narrative) format of disconfirmation and confirmation messages, as well as the message presentation order, on reducing confirmation bias and changing opinions. It is not uncommon to encounter these factors in real-life scenarios particularly when browsing through social media. Individuals often come across posts related to contentious issues like vaccination, alternative medicine, or climate change. It is frequent to encounter messages both confirming and disconfirming their initial positions. How do they feel when they encounter a confirmation message that supports their opinion first, followed by a disconfirmation message later? Do they tend to feel angrier and more resistant, or calmer to process the disconfirmation message? Does the order matter? Do they react differently if they read a disconfirmation message that challenges them first and then a confirmation message? Can they recall the same amount of information in these two situations? Moreover, what if one message is an emotional and compelling story and the other one presents statistics and arguments? Investigating the interplay of these factors can provide valuable guidance for practitioners and communicators who seek to navigate the competitive information environment on the internet. They can utilize this knowledge to optimize the arrangement and presentation of persuasive messages through content curating and delivering algorithms, ultimately enhancing the ability of strategic communication campaigns to effectively shape public attitudes and foster meaningful discourses.
The current study investigates the interplay of message format and order by testing how online media messages regarding in-person instruction during the pandemic can counter individuals’ confirmation bias. After the United States declared COVID-19 a national emergency on March 13, 2020, schools and businesses were closed. Many debates have taken place on whether schools should be reopened for the health of society. In May, some states started multiple phases of reopening, but caused a surge in new cases in June. In late August and early September 2020, school reopening for Fall 2020 has been a contested issue since the massive lockdown administered in Spring 2020 affecting approximately 50.8 million K–12 public students (Education Week, 2020a). The highly debated issue of school reopening during the COVID-19 pandemic serves as a relevant context due to the polarized public opinions surrounding this topic. By investigating how individuals process and respond to confirmation and disconfirmation messages, we seek to provide valuable insights into effective communication strategies in a highly competitive information environment.
Literature review
Confirmation bias
Confirmation bias refers to the tendency to seek out or interpret evidence in agreement with one’s prior beliefs to achieve cognitive resonance (Nickerson, 1998). Festinger’s cognitive dissonance theory (Festinger, 1962) can be considered the theoretical foundation for the confirmation bias. It states that dissonance is psychologically uncomfortable, and consequently, people attempt to suppress their “post-decisional dissonance” (Talluri et al., 2018). When people acquire and interpret information, they are deeply guided by this motivation of minimizing dissonance. Recent evidence suggests information consumers prefer attitude-congruent information and select to expose themselves to such information (Knobloch-Westerwick et al., 2020). Some researchers suggest confirmation bias can be explained by selective attention, such as weighing evidence more heavily if it supports one’s already held beliefs. It is found that our brains are more sensitive to subsequent stimuli that are consistent with the initial stimulus and are more efficient in processing such stimuli (Talluri et al., 2018).
Research regarding whether confirmation bias is a ubiquitous phenomenon shows mixed results. Some researchers argued that confirmation bias is universal (Zhou & Shen, 2021); others have found that not everyone is susceptible to confirmation bias in information selection (Knobloch-Westerwick & Kleinman, 2012). For example, Knobloch-Westerwick and Kleinman (2012) found that some people did not exhibit confirmation bias when consuming election news. For example, individuals who perceived high levels of information utility (i.e., a perceived value of information acquired), such as someone who believed their party was likely to lose and expected a change in government, were likely to override confirmation bias in information exposure (Knobloch-Westerwick & Kleinman, 2012). The study of factors that enhance or reduce confirmation bias has thus garnered more scholarly attention (Hernandez & Preston, 2013; Knobloch-Westerwick et al., 2020).
One crucial question that has triggered most scholarly interests is how to overcome confirmation bias. Previous research finds that confirmation bias may be mitigated by the need for cognition (Knobloch-Westerwick et al., 2020), or induced disfluency in information processing (Hernandez & Preston, 2013). These studies suggest that those who desire or use more cognitive resources to process the information are less likely to exhibit confirmation bias. In other words, enhanced cognition appears to help counter confirmation bias. If people are motivated to process the information comprehensively, they are more likely to resist the urge to rely on the consistency heuristic. By employing the consistency heuristic, individuals tend to simplify information processing, resulting in a more favorable evaluation of information that aligns with their pre-existing attitudes, and reinforce confirmation bias. Hernandez and Preston (2013) noted that induced disfluency can help people develop more analytical processing of information and reduce confirmation bias. However, cognitive overload would not aid the reduction of confirmation bias, as individuals require adequate cognitive resources to overcome such biases.
Contributing to this line of research, we aim to identify messaging strategies that can be used to reduce the influence of confirmation bias. In particular, we examine the effects of message format, (i.e., the narrative vs. non-narrative disconfirmation message) and message presentation order (i.e., the disconfirmation message presented before or after the confirmation message) in facilitating attitude change by enhancing cognitive elaboration on the disconfirmation message that would reduce confirmation bias.
Effects of narrative versus non-narrative disconfirmation messages
Recent persuasion research has paid a great deal of attention to the potential of narratives to serve as persuasive devices (Krause & Rucker, 2020; Ratcliff & Sun, 2020; Shen et al., 2015). Narratives refer to stories with “an identifiable beginning, middle, and end that provides information about scene, characters, and conflict; raises unanswered questions or unresolved conflict; and provides resolution” (Hinyard & Kreuter, 2007, p. 778). Synthesizing empirical studies on narrative effects, a meta-analysis suggests that persuasive information presented in a story format has a small but significant persuasive advantage over messages in a non-story format (Shen et al., 2015). This is linked to the unique features of narratives and the ways in which individuals process and engage with narratives.
Different from non-narrative information, narratives include two defining features: story characters and a story plot portraying how story events are carried out by characters and causally connected (Tal-Or & Cohen, 2016). As the persuasive message of a narrative is typically conveyed through the causal connection between story events, instead of being explicitly stated, narratives are less likely to be perceived as persuasive attacks than non-narrative messages (Dal Cin et al., 2004). Consequently, individuals may generate fewer counterarguments and negative emotions as they process narratives. Moreover, involvement with the story events and the characters may also reduce resistance to persuasion (Moyer-Gusé & Nabi, 2010). Contrary to non-narrative processing, individuals focus less on evaluating factual accuracy but more on a mental simulation of the story events and a coherent story world during narrative exposure (Slater & Rouner, 2002). This process, often referred to as transportation, is characterized by a melding of cognitive, affective, and imaginative attention that can attenuate negative cognitions toward a message (Green & Brock, 2000). Meanwhile, individuals may also devote their cognitive resources to identification—a process in which they take story characters’ perspectives and experience their emotions (Cohen, 2001), thus being less likely to counterargue against the message. In accordance with these proposed psychological mechanisms, meta-analytic evidence reveals that narrative engagement, such as transportation or identification, is negatively associated with resistance, and that narratives do trigger less resistance than non-narrative messages (Ratcliff & Sun, 2020).
The advantage of narratives in reducing audience resistance suggests that they are uniquely positioned to deliver counter-attitudinal information. Under the influence of confirmation bias, individuals often exhibit a defensive tendency and are motivated to critically analyze information when it substantially challenges their pre-existing attitudes, worldviews, or self-concept (Carpenter, 2019; Ling, 2020). As a result, they tend to reject the information by discrediting its arguments or disparaging the source. Given this phenomenon, narratives may be ideally suited to the delivery of counter-attitudinal information. The implicit persuasive intent and the highly immersive experience of narrative engagement may help attenuate motivated reasoning and negative cognitions toward a message (Dal Cin et al., 2004). For example, Slater et al. (2006) found that exposure to narrative TV programs regarding the death penalty increased viewers’ policy support and nullified the influence of their prior ideology.
Some empirical studies, however, have revealed a different pattern. Ecker et al. (2020) conducted three experiments examining the effect of correction format and found that narrative correction in general was no more effective than non-narrative correction. More importantly, when narratives were employed to correct worldview-consistent misperceptions, participants indicated greater reliance on misinformation two days following correction than their counterparts who received non-narrative correctives. Huang and Wang (2022) also found that non-narrative correctives delivered by a social contact on Facebook were more effective in changing e-cigarette-related attitudes following misinformation exposure than narrative correctives. Furthermore, narrative correctives were rated as less credible than non-narrative correctives.
Advocacy contradicting the misinformation likely activated an accuracy-driven mind-set. Under this mind-set, individuals considered narrative evidence as being anecdotal, less objective, and less generalizable (Dahlstrom, 2014). Another explanation may be related to the meta-cognitive experience during narrative processing. In particular, as a fundamental way of human communication, narratives are more easily processed than non-narrative information (Chang, 2009). Although processing fluency is typically considered a cognitive advantage of narratives in communicating complex information (Bullock et al., 2021), prior research has suggested that greater ease in information processing may lead to undesirable consequences when narratives are used to persuade individuals with opposite prior attitudes (Walter et al., 2020). Differently, experiencing disfluency may help minimize the impact of confirmation bias when individuals encounter counterattitudinal messages by promoting careful information processing (Hernandez & Preston, 2013). Given these, narratives’ power in challenging preexisting beliefs may be limited and even backfire among individuals with opposing prior attitudes.
Due to the mixed empirical findings and different theoretical explanations, we propose a research question examining the influence of narratives opposing in-person classes in Fall 2020 among individuals who have initially supported in-person classes. In addition to testing message effects on attitudes like many other studies in this area, we also investigate perceived importance of beliefs and normative perceptions supporting in-person classes as persuasion outcomes. The perceived importance of beliefs can help us understand the beliefs weighted to form an attitude and has been adopted in theoretical models such as the theory of reasoned action (Fishbein & Ajzen, 2009). Dominant reasons favoring or against in-person classes for Fall 2020 in response to the COVID-19 pandemic include the risk for students and teachers getting or spreading the coronavirus, parents being unable to work if their children are at home, students falling behind academically, students missing appropriate social/emotional development, and the financial burden on school districts to follow public health guidelines (Pew Research Center, 2020b). It is known that support for in-person or online instruction at the time depended on the relative importance individuals assigned to these factors. Normative perceptions also play a significant role in the adoption of health recommendations (Fishbein & Ajzen, 2009). Recent evidence indicates that normative perceptions are predictive of changes in attitudes toward following prevention measures, especially during the initial phases of the COVID-19 pandemic (Latkin et al., 2022; Young & Goldstein, 2021):
Research Question 1 (RQ1). Will exposure to the narrative disconfirmation message opposing in-person classes lead to more disconfirmation message-consistent responses than exposure to the non-narrative disconfirmation message, including less (a) favorable post attitudes, (b) perceived importance of beliefs, and (c) normative perceptions supporting in-person classes?
Presentation order
Message presentation order may also influence how disconfirmation messages are processed and received. Recency and primacy effects are proposed to theorize the order effect. When people encounter two communications presenting conflicting ideas, if their judgment is more influenced by the first communication received, a primacy effect happens; if their judgment is more affected by the second communication received, a recency effect takes place (Crano, 1977; Haugtvedt & Wegener, 1994). Researchers have long been interested in determining when a primacy or recency effect occurs. Past studies have examined whether issue familiarity, levels of issue controversy and interest are more likely to activate one effect than another (Lana, 1961, 1963a, 1963b) However, earlier research has obtained mixed results, merely suggesting the level of cognitive elaboration may have an impact on message order effect (Haugtvedt & Wegener, 1994).
From the attitude strength perspective, Haugtvedt and Wegener (1994) argued that if a greater attitude strength is involved when processing the first communication, the attitude change following the second communication will then be “an index of strength of the attitude following the first message” (p. 207). However, if a weaker attitude strength is engaged as individuals process the first communication, people are more likely to be influenced by the second communication and subsequently better recall it. In their study, attitude strength is operationalized as the level of elaboration following the elaboration likelihood model (Petty & Cacioppo, 1986). It is worth noting that memory-based judgment, which is the reliance on memorizing message content to form an attitude, is a characteristic of the low-elaboration rather than the high-elaboration mode (Haugtvedt & Wegener, 1994). In their own experiments, Haugtvedt and Wegener (1994) found that a primacy effect was salient in situations fostering high levels of message elaboration while a recency effect occurred in situations fostering low levels of message elaboration.
Recent empirical studies on conflicting information have largely supported the recency effect, showing that the second communication often has more influence on attitudes and judgments. For instance, a study examined how consumers reacted to negative news coverage of an organization and a corrective message from the organization (Han et al., 2020). It is found that the correction received following negative news coverage can successfully restore the organization’s unfavorable image resulting from the negative coverage, indicating support for the recency effect (Han et al., 2020). Similarly, Vraga et al. (2020) found that fact-focused corrections successfully corrected misperceptions if they were provided after exposure to misinformation but not preceding it. Dai and colleagues (2021) discovered that correction after than before misinformation exposure not only resulted in the most accurate recall but also maintained such effect for at least one week when correction in this order was presented with a debiasing message that reconciled why misinformation appeared at first (Dai et al., 2021). These findings may suggest that people rely on the correction materials to generate attitudes, consistent with a memory-judgment relationship that is typical of a low cognitive elaboration mode, as argued by Haugtvedt and Wegener (1984). A recent meta-analysis on misinformation concluded that debunking, which means correction after misinformation exposure, is more effective than prebunking, defined as correction before misinformation exposure (Walter & Murphy, 2018). We could not conclude that people for sure were in a low-elaboration mode in these studies which evidenced the greater influence of the second communication. But these studies seemed to confirm a common theme that when people are not specially intervened to increase processing of the conflicting messages, a recency effect will occur (Haugtvedt & Wegener, 1994). Thus, we expect that the disconfirmation message would be more effective if it is delivered as the second communication than the first communication:
Hypothesis 1 (H1). Disconfirmation messages delivered as the second communication will be more effective to generate disconfirmation message-consistent a) post attitudes, b) perceived importance of beliefs, and c) normative perceptions than when delivered as the first communication.
As prior literature suggests, narrative and non-narrative persuasion have their own advantages. Narrative messages may be especially effective to deliver a disconfirmation message with an opposing view as it potentially reduces counterarguing (Dal Cin et al., 2004). Non-narrative messages may be uniquely effective to counter confirmation bias as they tend to be more difficult to be processed (Walter et al., 2020). The effect of message format may be moderated by the order of presentation. Since people may be more susceptible to the recency effect when they are not specially intervened to increase attention and elaboration (Haugtvedt & Wegener, 1994; Vraga et al., 2020; Walter & Murphy, 2018), we hypothesize that the message format of the disconfirmation message, when delivered as the first communication, will not elicit much difference on message outcomes as the first message may not result in effective message retention and attitude change. In contrast, when delivered as the second communication, the message format of the disconfirmation message may cause significant differences, as people often rely more on the second message to form judgments. We propose RQ2 to address this possible moderation effect between disconfirmation message and presentation order:
Research Question 2 (RQ2). Will the different effect between narrative and non-narrative disconfirmation message on (a) post attitudes, (b) perceived importance of beliefs, and (c) normative perceptions be moderated by message order, such that the effect of message format will be more pronounced when the disconfirmation message is delivered as the second than the first communication?
Confirmation message, disconfirmation message and presentation order
The ability of a disconfirmation message to counter confirmation bias may depend not only on its own message format and order of presentation, but also on the format of the confirmation message. Confirmation messages in the current study function to maintain people’s prior attitudes. The format of confirmation messages may also play a role influencing the ultimate attitudinal changes. Persuasion research has shown that narrative and non-narrative communication have their unique strengths. Narrative communication is emotionally engaging and helps establish strong attitudes by facilitating mental simulations (Green & Brock, 2000; Slater & Rouner, 2002). Non-narrative communication appeals to people’s accuracy-driven mind-set, and non-narrative evidence is thought of as more generalizable and credible (Dahlstrom, 2014; Huang & Wang, 2022). In maintaining one’s existing beliefs and attitudes, the strengths of these two forms of communication may persist across purposes. For example, the narrative confirmation message might be considered more emotionally resonant and coherent, providing a strong effect facilitating quick information processing and showing advantages to maintain individuals’ prior attitudes. Or the non-narrative confirmation message might be perceived as more credible and well-reasoned, showing advantages to maintain one’s prior attitudes. Due to the different possibilities, we ask whether there might be a three-way interaction that the message format of confirmation message also interacts with the other two factors.
Research Question 3 (RQ3). Will there be a three-way interaction between the confirmation message format (narrative vs. non-narrative), disconfirmation message format (narrative vs. non-narrative) and message presentation order on (a) post attitudes, (b) perceived importance, and (c) normative perceptions?
Method
Study design and participants
A 2 (confirmation message format: narrative vs. non-narrative) × 2 (disconfirmation message format: narrative vs. non-narrative) × 2 (order: disconfirmation message first vs. second) between-subjects online experiment was conducted in September 2020. U.S. adults were recruited from Mechanical Turk. A filter question asked participants how they viewed the risks and benefits of having in-person classes in Fall 2020. Given the study’s interest in individuals who supported in-person classes in Fall 2020, participants who responded “a lot more risks than benefits” or “more risks than benefits” were filtered out and those who responded “more benefits than risks” or “a lot more benefits than risks” proceeded to complete the study. To avoid sensitizing participants, this filter question was masked by nine similar questions regarding other health and social issues (e.g., flu vaccination, e-cigarette use, climate change, etc.).
In total, 191 complete responses were collected. Attention check questions (e.g., “please select strongly agree in this line to show your attention.”) were embedded in the questionnaire to test whether participants carefully read through the questions without straight-lining. Eight participants were excluded due to failure to pass attention checks. The final sample included 183 participants (64.5% males; Mage = 36.89, SDage = 10.60). More than half of the participants identified themselves as Caucasians (55.7%), followed by African Americans (29.5%), Hispanics (7.7%), and Native Americans (6.0%). Other participants identified themselves as multiracial or others (1.1%). Most participants indicated that they had not been officially diagnosed with COVID-19 (73.8%), whereas 25.1% had contracted the virus and 1.1% were waiting for the results.
Statistical power
Post hoc power analysis using G*Power 3.1 revealed that the final sample provided power of .27, .92, and >.99 for effect sizes of f = .10, .25, and .40, respectively. Therefore, the study possessed sufficient power to detect medium and large effects.
Issue context
Our study focused on studying the effects of disconfirmation messages, including its format, the format of the competitive messages (confirmation messages), and order in the context of favoring or disfavoring in-person instruction for Fall 2020. Pew Research Center (2020b) reported that Americans were divided on the type of instruction K-12 schools should provide in that fall: by August 2020, 19% supported in-person learning, 28% supported online learning, 36% supported hybrid and 16% were not sure. In October 2020, poll results suggested that students received online instruction only (46%) for Fall 2020 far exceeded those who received a mix of in-person and online instruction (23%) and in-person instruction only (20%) (Pew Research Center, 2020a).
While the question of whether K–12 schools should reopen for in-person learning in Fall 2020 was a contentious issue with debate and disagreement, most states in the U.S. eventually postponed mandates for in-person learning (Education Week, 2020b). In other words, individuals who previously supported in-person classes had to adapt to the reality that opposing in-person classes had become the mainstream opinion at the time. The messages advocating against in-person instruction and favoring online classes served as disconfirmation messages for them. By examining attitudinal changes among those who initially supported in-person instruction, we had the opportunity to study how these disconfirmation messages influenced dissidents to gain public support for early COVID-19 prevention measures. As we collected data in September 2020, attitudinal changes were more noticeable among those who initially supported in-person instruction compared to those who initially disapproved.
Selection of stimulus messages
Stimulus messages were chosen from 18 existing news articles related to decisions about in-person classes in Fall 2020 from reputable media sites such as CNN, New York Times, Wall Street Journals, and so on. Specifically, this collection of articles included five narratives against, five non-narratives against, four narratives in favor of and four non-narratives in favor of in-person instruction. All messages were edited to be around 400 words. A pretest was conducted with 68 participants recruited from Mechanical Turk who read and rated the messages in terms of its message stance (i.e., the extent to which the messages were against/in favor of in-person classes in Fall 2020). Messages were then selected based on the pretest results. We also ensured to match the focus of the message between the narrative and non-narrative messages.
For instance, a few recurring frames were found in news articles against in-person classes, including those emphasizing teachers’ health risks (teacher focus), the logistical difficulties of handling in-person classes, particularly schools and school districts’ lack of effective plans and supplies to follow social distancing (logistics focus), and students’ health risks (student focus). Corresponding to these frames, the articles in-favor of in-person classes often employed frames such as teachers’ responsibilities and obligations (teacher focus), the logistical difficulties of handling online classes, particularly parents’ struggles with not being able to go back to work (logistics focus), and students’ learning loss and development risk (student focus). By examining the mean differences in participants’ ratings of article stance, we selected the articles with the largest mean difference between in-favor and against articles and ensured that the confirmation messages and disconfirmation messages selected had a similar focus. Some articles, for example, those that emphasized students’ health risks (against in-person classes) and students’ learning loss and development risks (in favor of in-person classes), were not selected because the mean differences in ratings of article stance were relatively small.
Experimental manipulations
The format of the confirmation and disconfirmation messages was operationalized following the definition of narratives (Green & Brock, 2000). The narrative messages contained story characters and a detailed plot outlining the characters’ experiences, thoughts, and actions. Differently, the non-narrative messages delivered arguments primarily through statistics, facts, or didactic information. To ensure that the effects found were not associated with the content of a particular message, two stimulus messages were prepared for each message treatment (i.e., two narrative confirmation messages, two non-narrative confirmation messages; two narrative disconfirmation messages, and two non-narrative disconfirmation messages). In the experiment, each participant viewed two messages: one from the four confirmation messages supporting in-person classes and one from the four disconfirmation messages opposing in-person classes.
Two narratives featured stories of individuals who were in favor of in-person classes. One story was about a working mom who decided to send her third-grade son to a private school as it offered in-person learning in the fall semester. Another story explained how school closures during the pandemic adversely impacted a couple’s marriage and career. The two non-narrative messages supporting in-person classes discussed the necessity of in-person classes, with one focusing on the struggle of working parents and the other highlighting the responsibilities of teachers for the functioning of society.
As for the messages opposing in-person classes, one narrative was about a high school teacher who resigned to keep her family members safe during the pandemic, and the other story was about a kindergarten teacher who was worried about the health risks in the classroom. The two non-narrative messages discussed the reasons for postponing in-person classes. One message focused on the possible health risks presented to teachers, parents, and young children that might result from a rush to reopen; another message discussed why big school districts were not ready for a reopen in the fall due to the prevalence of infection and the logistic constraints for implementing safety measures.
Message presentation order. Half of the participants read a message opposing in-person classes first, followed by a message supporting in-person classes; the other half first viewed a message supporting in-person classes and then a message opposing in-person classes.
Procedure
After participants indicated their consent, they answered the filter question regarding their prior attitude toward in-person classes in Fall 2020. Eligible participants were then randomly assigned to one of eight experimental conditions and viewed two messages, with one supporting in-person classes and one opposing in-person classes. Message format and presentation order varied depending on their condition. Following their exposure to both messages, the outcome variables and demographics were measured. Finally, participants were debriefed and compensated. The whole experiment took about 10 to 15 minutes.
Measurement
Post-exposure attitudes toward in-person classes were measured using four semantic-differential items (Huang & Wang, 2022). On a seven-point scale, participants indicated that after reading the messages, they felt the idea of in-person classes in Fall 2020 was harmful/beneficial, foolish/wise, bad/good, unfavorable/favorable (M = 5.47, SD = 1.28, Cronbach’s α = .89).
Perceived importance of beliefs supporting in-person classes were measured by asking participants rate how important the following beliefs were on a 7-point scale when forming their opinion. These include: “parents will not be able to work if their children are at home,” “students will fall behind academically without in-person instruction,” “students will miss out on social interactions with their peers if campuses are closed,” “teachers are under the obligation to educate students,” and “education systems are essential to the functioning of society” (M = 5.48, SD = 0.84, Cronbach’s α = .72).
Normative perceptions were assessed using two semantic differential items (Kim et al., 2015) on a seven-point scale. Participants indicated that after reading the messages, they believed a vast majority of Americans “do not support/support” and “disapprove/approve” in-person classes in Fall 2020 (M = 5.70, SD = 1.15, r = .66).
Control variable
Participants also indicated their political orientation from 1 (quite conservative) to 5 (quite liberal). The mean was slightly toward the conservative end (M = 2.56, SD = 1.16). Given the strong influence of political ideology on COVID-related issues (P. S. Hart et al., 2020), political orientation was statistically controlled in all analyses.
Results
Manipulation checks
Participants rated on a 7-point scale if the message they read (a) presents a person’s story (1 = strongly disagree, 7 = strongly agree) and (b) is against/in favor of in-person classes for Fall 2020. Participants who read the narrative confirmation message (M = 5.34, SD = 1.66) in favor of in-person classes rated the message more a story than those who read the non-narrative counterpart (M = 4.46, SD = 1.99), F(1, 181) = 10.62, p < .01. Those who read the narrative disconfirmation message (M = 5.10, SD = 1.69) opposing in-person classes also rated the message more a story than those who read the non-narrative counterpart (M = 4.32, SD = 2.04), F(1, 181) = 7.65, p < . 01. Participants also rated the valence of the messages accordingly. They rated a confirmation message more supportive to in-person classes (M = 5.35, SD = 1.30) than a disconfirmation message (M = 4.81, SD = 1.82), F(1, 181) = 12.35, p < . 01.
Hypotheses testing
Analysis of covariance (ANCOVA) tests were conducted to examine the effects of disconfirmation message format, confirmation message format, message presentation order, and their interactions on post attitudes, perceived belief importance, and normative perceptions with political orientation as a covariate variable. RQ1 asked whether there is a main effect of disconfirmation message format on post attitudes, perceived belief importance, and normative perceptions. As shown in Table 1, a main effect of format of disconfirmation message was not found on post attitudes, F(1, 174) = 1.54, p = .22, perceived belief importance, F(1, 174) = 0.63, p = .43, and normative perceptions, F(1, 174) = 0.99, p = .32.
ANCOVA results on post attitudes, perceived belief importance and normative perceptions.
Results that are statistically significant (p < .05) and approaching statistical significance (p < .10) were bolded.
H1 proposed a main effect of presentation order such that disconfirmation message presented later than earlier was more effective. ANCOVA tests revealed a marginally significant effect of presentation order on post attitudes, F(1, 174) = 3.20, p = .08, and perceived belief importance, F(1, 174) = 3.07, p = .08, but not on normative perceptions, F(1, 174) = 1.73, p = .19. Participants who read the disconfirmation message second reported greater disconfirmation message consistent attitudes which is less favorable attitudes toward in-person classes (M = 5.31, SD = 1.30), than those who read the disconfirmation message first (M = 5.63, SD = 1.23). The same pattern was found on perceived belief importance. Those who read the disconfirmation second (M = 5.38, SD = .86) reported a lower level of perceived importance of the beliefs supporting in-person classes than those who read the disconfirmation first (M = 5.58, SD = .82). H1 was partially supported.
RQ2 asked whether message presentation order will moderate the effect of disconfirmation message format. The two-way interaction effect of disconfirmation message format and presentation order was significant on post attitudes, F(1, 174) = 7.68, p < .01, perceived belief importance, F(1, 174) = 5.41, p < .05, and normative perceptions, F(1, 174) = 5.63, p < .05. When delivered as the second message, the non-narrative disconfirmation message was more effective than the narrative to lower the favorability toward in-person classes (Non-narrative: M = 5.01, SD = 1.34 vs. Narrative: M = 5.68, SD = 1.16, p < .01), perceived importance of beliefs favoring in-person classes (Non-narrative: M = 5.22, SD = .85 vs. Narrative: M = 5.58, SD = .84, p < .01) and normative perceptions (Non-narrative: M = 5.38, SD = 1.24 vs. Narrative: M = 5.85, SD = 1.11, p < .05). When participants encountered the disconfirmation message first, the non-narrative and narrative disconfirmation message did not show significant differences on these outcome variables.
RQ3 asked whether there will be a significant three-way interaction of disconfirmation message format, confirmation message format, and message presentation order. The three-way interaction was nonsignificant on post attitudes, F(1, 174) = 1.01, p = .32, or normative perceptions, F(1, 174) = 1.01, p = .32. A significant three-way interaction effect was found on perceived belief importance, F(1, 174) = 4.48, p < .05 (Figure 1). When the confirmation message was narrative, the above-mentioned two-way interaction pattern of disconfirmation message and message presentation order was found, such that the non-narrative disconfirmation message delivered second was more effective to lower the perceived importance of beliefs favoring in-person classes than the narrative disconfirmation message (Non-narrative: M = 5.00, SD = .97 vs. Narrative: M = 5.66, SD = .88). However, when the confirmation message was non-narrative, there was not a particularly effective disconfirmation message and message order combination to counter a non-narrative confirmation message.
Three-way interaction on perceived belief importance.
Discussion
The present study extends the research on confirmation bias, narrative persuasion and presentation order by examining the factors that make a disconfirmation message more effective to reduce the influence of confirmation bias. We expand the narrative and non-narrative persuasion literature by considering the attitudinal valence of the message. We found that the disconfirmation message was more effective when it was delivered as the second communication than as the first communication to generate message-consistent responses, including reducing favorable attitudes toward participants’ initial position and reducing perceived importance of beliefs supporting their initial position. Further, significant two-way interactions were found: the non-narrative disconfirmation message was more effective than the narrative disconfirmation message to generate message-consistent responses when it was delivered as the second communication. When the disconfirmation message was delivered as the first communication, message format did not make a significant difference. In addition to the message format and presentation order of the disconfirmation message, we also considered the format of the confirmation message, which was the competing communication, that maintained people’s initial attitudes. We found that the advantages of the subsequent non-narrative disconfirmation message over the narrative disconfirmation message was only evident when countering a narrative confirmation message delivered before it. In other words, when a confirmation message was narrative and delivered first, it was not very effective to maintain people’s initial attitudes and defend the attacks from the subsequent non-narrative disconfirmation message.
We found the disconfirmation message presented after than before the confirmation message made the disconfirmation message more effective to change attitudes, belief importance and normative perceptions away from people’s initial positions. These findings on the main effect of the presentation order lend support to the recency effect. Since we measured the perceived importance of beliefs supporting the in-person classes, which were the arguments used to create the confirmation message, the findings also show that the disconfirmation message delivered second was able to counter the arguments of the confirmation message delivered first. In other words, due to the effect of order, the arguments of the first message become less impactful. The weakened impact of the first message and the heightened influence of the second message imply that participants, despite initially opposing in-person classes, may have had a weak attitude and did not engage with the conflicting messages in a highly motivated processing state (Haugtvedt & Wegener, 1994). In other words, even though these individuals held a prior stance that contradicted the mass prevention measures of postponing in-person instruction and reopening, their attitudes were likely not strong or even ambivalent. For this reason, they could not engage in the processing mode where they systematically processed the first communication and the processing of the first communication dominated the processing of the subsequent communication. Given how the pandemic progressed, their views were likely to be influenced by memorable disconfirmation communication that they encountered most recently. This perhaps suggests that to change people’s attitudes, the disconfirmation messages need to be repeated and be available as the most recent communication that occurred to the less-than-strong-minded individuals.
This finding does not conflict with inoculation research, which suggests the effectiveness of forewarning (Niederdeppe et al., 2015). Conceptually, inoculation messages warn of a possible threat that is likely to happen and some even address the logical fallacy of the threat. In comparison, our research presents conflicting information, each with its own arguments. It does not lead people through the process of receiving a warning and then having that warning confirmed. While inoculation theory involves a pre-warning that primes the audience to resist future attempts at persuasion, our study focused on the immediate impact of message sequencing without such a forewarning component. The psychological processes at play may indeed differ: while inoculation prepares the audience for resistance, the recency effect indicates a natural cognitive bias toward the most recently presented information without necessarily involving a resistance mechanism.
The presentation order also moderated the effect of disconfirmation message format. Only when delivered after the confirmation message, the non-narrative disconfirmation message was shown to be more effective than the narrative disconfirmation message to influence post attitudes, perceived importance of beliefs, and normative perceptions. According to our manipulation check results, the message format difference was successfully recognized regardless of the order of presentation. However, the format only influenced the attitudinal outcomes when delivered as the most recent communication. This indicated not only the recency effect, but also that the disconfirmation message would have been processed minimally if presented first. If the disconfirmation message was delivered first, participants started the experiment with their initially supportive attitudes toward in-person classes. They then read a disconfirmation message opposing in-person classes, followed by a confirmation message supporting in-person classes. With this presentation order, the disconfirmation message might be discounted as noise and an unnecessary diversion, and minimally processed. When the disconfirmation message was presented second, the noticeable differences in message format had an effect on attitudes. The non-narrative disconfirmation message was shown to be more effective than the narrative disconfirmation message. This is in line with the previous empirical evidence that the non-narrative message was seen as more credible and presenting a more generalizable conclusion (Huang & Wang, 2022; Vraga et al., 2020). It is also possible that the non-narrative disconfirmation message introduced a greater level of processing disfluency, a subjective feeling of difficulty in processing, than the narrative disconfirmation message as prior research has found (Bullock et al., 2021). While processing disfluency is often associated with negative affective feelings in persuasion research, such processing disfluency may enhance cognitive elaboration of the disconfirmation message in the context of processing counterattitudinal information. As prior studies have suggested, such elaboration may be able to help counter the confirmation bias introduced by people’s initial attitudes and then enhanced by the confirmation message people read first (Hernandez & Preston, 2013). Regrettably, processing disfluency was not measured in the current study and should be addressed in future research.
In addition, the results revealed a three-way interaction. Specifically, when the confirmation message was presented first in a narrative form, it was ineffective in countering the subsequent attacks of a non-narrative disconfirmation message. In addition to the effectiveness of the most recently presented non-narrative disconfirmation message in triggering cognitive elaboration that countered confirmation bias, this finding also suggests the weakness of narrative confirmation messages in maintaining people’s prior attitudes. Faced with a non-narrative counter message, narrative messages may be comparatively weaker and lose in direct competition. This finding aligns with previous research indicating that narrative messages are more impactful when supported by other factors, such as endorsement from a popular source (Huang & Wang, 2022), and that narratives alone may exhibit limited persuasive effects. Recognizing the limitations of relying solely on narratives has practical implications for campaign strategies.
Implications
The findings suggest that during a pandemic, when a mass prevention policy is being debated, there is an opportunity to persuade those who initially hold opposing but ambivalent attitudes, particularly considering the uncertainty prevailing in such contexts. To persuade them, the practitioners need to ensure that messages countering dissidents’ initial attitudes are presented later than earlier, and are consistently accessible to these individuals mentally, to have an impact. Easier-to-process narrative messages may not work well in these situations as they may not be effective in countering confirmation bias. People are more motivated to process information effortfully so they can engage in deeper processing of the conflicting information presented. Narrative messages should be cautiously used as they may not effectively counter confirmation bias or maintain initial attitudes. If narratives have to be employed, other communication factors should be utilized to enhance their effects.
Limitations and future studies
All research has limitations, and our project is no exception. We developed our hypotheses based on the level of elaboration people may produce on the disconfirmation message which theoretically determines effectiveness of disconfirmation messages in countering confirmation bias. Although our results support our hypotheses, processing disfluency and cognitive elaboration on the disconfirmation message were regrettably not measured. One challenge was that when we provided two conflicting messages, one confirming and one disconfirming people’s prior attitudes, the assessment of elaboration on each message can be only accurate if immediately given after exposure to each message. However, such measures may sensitize participants about the hypotheses and prime attitudinal responses and therefore were not implemented. Future research should directly test processing disfluency and cognitive elaboration as the mediating mechanisms possibly through a longitudinal design.
Moreover, as the post hoc power analysis indicated, although the study sample provided adequate power to detect medium and large effects, it was limited in testing small effects. This may partially explain why the effect patterns we found on attitudes, perceived belief importance, and normative perceptions were not consistent at times. Future investigations may benefit from replicating the study design with a larger sample size. Our decision to use modified real news articles from reputable news organizations was deliberate, aimed at enhancing the study’s external validity and ensuring that our findings reflect the complexity of real-world media consumption. Although we attempted to match the messages in terms of length and content focus, this approach might unavoidably introduce variability in the details of the messages that could confound the findings. We acknowledge this as a limitation of the study. In addition, since participants read a mixture of messages that could be either narrative, non-narrative, or both, it was challenging to measure narrative engagement processes induced by each message, as it involves repeating the same sets of measurement questions following the exposure to each message. We chose not to include these measures to avoid participants’ fatigue and potential response bias resulting from the repetition of measurement questions (Kuo & Hirshman, 1996). Consequently, we were not able to examine how these narrative engagement processes influence persuasion. We look forward to future improvements in enhancing both internal and external validity and studying the underlying mechanisms.
In addition, since this study examines the influence of disconfirmation messages, confirmation messages, and presentation order on individuals who initially supported in-person classes, we specifically recruited participants who were supportive of in-person classes to participate. Future research will encompass a broader population, including individuals with attitudes covering the full spectrum of initial positions. By doing so, we can ascertain whether the observed patterns persist across participants with both supportive and unsupportive initial attitudes, confirming or disconfirming the majority viewpoint considering the progression of the external environment. To fulfill the objectives of the present study, the messages utilized in the present study were clearly valenced based on preconceptions. A future study may replace one-sided messages with two-sided messages that merit arguments from both sides, or messages featuring uncertainty or being less conclusive. These messages may induce a greater level of processing disfluency and likely help reduce the influence of confirmation bias.
Conclusion
The current study explores strategies that potentially mitigate confirmation bias by integrating research in narrative persuasion, confirmation bias and presentation order. The study found that delivering a disconfirmation message after a confirmation message was more effective in eliciting attitude and belief change, away from individuals’ initial position. The recency effect was more pronounced when the disconfirmation message was non-narrative, and the confirmation message was in a narrative format. This pattern of results underscores the nuanced interplay between message format and presentation order, suggesting that the strategic presentation of information could serve as a powerful tool in counteracting confirmation bias. Future research may expand upon these insights to further optimize communication strategies for more informed public decision-making.
