The role of motivated science reception and numeracy in the context of the COVID-19 pandemic

Abstract

The successful management of the COVID-19 pandemic depends on individuals accepting the current state of research and adhering to the preventive behaviors that follow from it. However, the processing of scientific results is not always rational, but influenced by prior attitudes as well as the ability to understand statistical data. Against this background, this study investigated the role of motivated reasoning and numeracy in the context of the current pandemic. To this end, participants (N = 417; US sample) evaluated two fictitious studies, one indicating that mask mandates in schools are an effective intervention to contain the spread of SARS-CoV-2, and one indicating that mask mandates in schools are counterproductive. Participants evaluated the studies in line with their prior attitude toward mask mandates. In addition, higher numeracy was associated with decreased bias, demonstrating that the ability to reason with numbers can lead to more accurate processing of statistical information.

Keywords

COVID-19 pandemic motivated reasoning motivated science reception numeracy

The COVID-19 pandemic poses a great challenge—not only to scientists and politicians, doctors and health-care workers, but also to the public. First, because the current situation is associated with financial insecurity, increased caregiving burden, confinement-related stress, and reduced well-being for many people (Prime et al., 2020; Vindegaard and Benros, 2020). Second, because containing the pandemic requires large-scale behavior change on the individual level, such as social distancing and wearing masks (Habersaat et al., 2020; Haug et al., 2020; Van Bavel et al., 2020). In other words, the successful management of the COVID-19 pandemic depends on individuals understanding the seriousness of the situation and adhering to the necessary preventive behaviors.

Crucially, the public debate is highly politicized in many countries. For instance, it has been demonstrated that the pandemic has become a deeply partisan issue in the United States and that people’s attitudes toward the pandemic influence their preventive behavior and ultimately health outcomes (Druckman et al., 2021; Gollwitzer et al., 2020). As these studies seem to suggest indirectly, at least some people do not follow “the unforced force of the better argument” (Habermas, 1996: 305), but tend to align their interpretation of the available evidence with their prior beliefs. To make matters even more complicated, much of the public debate is centered around numbers. To be able to follow the debate and to develop an informed opinion, people need to understand statistical concepts such as “exponential growth,” “test positive rate,” “excess mortality,” and “reproduction number”—all of which can easily be misinterpreted (e.g. Adam, 2020).

To better understand this situation from a psychological perspective, at least the following two factors require careful investigation: motivated reasoning and numeracy. Motivated reasoning denotes the observation that human information processing is not always rational and objective, but influenced by the individual’s motives, goals, and attitudes (Kruglanski, 1996; Kunda, 1990). Simply put, “[p]eople are more likely to arrive at those conclusions that they want to arrive at” (Kunda, 1990: 495). In the context of the evaluation of scientific results, motivated reasoning has also been described as motivated science reception or—when referring to the unwillingness to accept well-established findings in particular—as the motivated rejection of science (Hornsey, 2020; Hornsey and Fielding, 2017; Lewandowsky and Oberauer, 2016; Rothmund et al., 2017). One potential mechanism underlying motivated science reception (and motivated reasoning in general) is that information, which is incongruent with one’s attitudes, is evaluated more critically compared to congruent information (Dawson et al., 2002; Edwards and Smith, 1996). It has also been shown that incongruent information can trigger the active search for evidence that contradicts the incongruent information (Kunda, 1987). Motivated science reception has been demonstrated for many different topics, ranging from capital punishment (Lord et al., 1979), gun control (Kahan et al., 2017b; Washburn and Skitka, 2018), and climate change (Hart and Nisbet, 2012) to nanotechnology (Kahan et al., 2009), pacifism (Bender et al., 2016), and gaming (Nauroth et al., 2014, 2015).

Thus, it seems likely to assume that motivated reasoning also plays a role in the context of the COVID-19 pandemic. At first glance, one may suspect that motivated reasoning is only prevalent among those who embrace conspiracy theories and who may ultimately endanger the containment of the pandemic because they are less willing to follow the evidence-based guidelines (Bierwiaczonek et al., 2020; Romer and Jamieson, 2020). Based on the accounts of motivated reasoning described earlier, however, it seems more plausible to assume that motivated reasoning can be found across the entire attitude spectrum; those who strongly support the policies taken to contain the spread of SARS-CoV-2 should overestimate the effectiveness of these policies, while people who strongly disagree with the policies should underestimate their effectiveness.

Nevertheless, it is important to note that the degree to which people engage in motivated reasoning is not equally distributed across the population. For instance, it is often claimed that numeracy, the ability to understand and to reason with numbers is associated with making more informed decisions and achieving better life outcomes (for reviews, see Garcia-Retamero et al., 2019; Reyna et al., 2009). Indeed, it has been demonstrated that higher numeracy leads to an improved understanding of health risks (Davids et al., 2004; Gurmankin et al., 2004; Rolison et al., 2020; Shoots-Reinhard et al., 2020), increased financial literacy (Ghazal et al., 2014; Pachur and Galesic, 2013; Skagerlund et al., 2018; Traczyk et al., 2018), and ultimately to better physical and mental health (Apter et al., 2009; Garcia-Retamero et al., 2015; Huizinga et al., 2008).

At the same time, there are also several studies demonstrating that higher numeracy can lead to increased motivated reasoning, especially when the matters at stake are linked to one’s (political) identity and worldview (Drummond and Fischhoff, 2017; Kahan et al., 2012, 2017a, 2017b; Nurse and Grant, 2020). This finding, which may seem counterintuitive at first, is explained by the idea that highly numerate people can use their skills to rationalize their interpretations of existing evidence so that they align with their attitudes. Note, however, that not all studies that investigated motivated reasoning in the context of politically polarizing issues found such an effect (Connor et al., 2020; Lind et al., 2018; Persson et al., 2021; Tappin et al., 2020a). Thus, the relationship between numeracy and the degree of motivated reasoning may depend on various contextual factors. Hence, it is an open question how numeracy is related to motivated reasoning in the context of the current pandemic.

Against this background, it is of crucial interest to understand whether and how motivated reasoning and numeracy influence the evaluation of scientific information in the context of the COVID-19 pandemic. This study had two goals: First, we sought to test the hypothesis that motivated reasoning plays a role in the evaluation of scientific evidence on the effectiveness of a policy adopted to slow down the spread of SARS-CoV-2. We chose to focus on mask mandates, because attitudes on mask mandates have become polarized in the public (e.g. Gollwitzer et al., 2020), while at the same time, current scientific evidence clearly supports the effectiveness of wearing masks (Howard et al., 2021; Peeples, 2020). Second, we sought to investigate the relationship between numeracy and motivated reasoning, that is, whether numeracy decreases or increases the biased evaluation of scientific evidence.

To this end, we first conducted a prescreening to recruit participants across the entire attitude spectrum, ranging from people who strongly oppose to people who strongly support mask mandates in public (Session 1). A subset of these participants was invited to take part in the main study (Session 2). In the main study, participants were asked to evaluate the results of two (fictitious) studies on the effectiveness of a mask mandate. One study showed that mask mandates in schools are an effective intervention to slow down the spread of SARS-CoV-2, while the other showed that mask mandates in schools are counterproductive. Based on participants’ answers, we calculated an individual bias score and related this bias score to both the participants’ attitude toward mask mandates and their numeracy.

1. Method

Participants

As the main analysis consisted of a linear regression (see below), we aimed for a final sample in the main study of at least 300 participants (Schönbrodt and Perugini, 2013). Participants were recruited through Prolific (www.prolific.co). Participants provided informed consent. The local ethics committee approved the study. All data exclusions, manipulations, and measures are reported. Data and materials can be downloaded at osf.io/8nj2a.

Session 1: prescreening

The study was programmed on SoSciSurvey (Leiner, 2019). In Session 1, participants were prescreened for their attitudes toward mask mandates. Participation was only possible for Prolific users with an approval rate of at least 95% who were fluent in English and currently living in the United States. After recruiting the first 350 participants, the number of people in the sample who opposed mask mandates was very low (6.9%). As there were far more people opposing mask mandates among Republicans (29.7%) than among Democrats (0.5%), further data were collected from Republicans only. As we wanted participants to be equally distributed across the entire attitude spectrum, data collection in Session 1 was continued until there were at least 85 participants on each of the seven points of the scale that measured the attitude toward mask mandates in public (see below). In total, 1665 participants completed Session 1.

After providing informed consent, participants answered questions regarding their attitude toward mask mandates in public and nine other corona policies on a 7-point Likert-type scale (ranging from 1 = “extremely oppose” to 7 = “extremely favor”). The items were presented in random order. Next, participants completed a questionnaire on the satisfaction and frustration of basic psychological needs.¹ Finally, participants answered demographic questions regarding age, gender, ethnicity, political orientation, and education. After completing the prescreening, participants were informed about the purpose of the study and compensated with £0.40.² Session 1 lasted about 3 minutes. Data collection took place from 4 to 12 February 2021.

Session 2: main study

To account for exclusions and potential dropout between sessions, we invited 85 participants of each of the seven scale points of the mask attitude item from Session 1. In case Session 1 data included more than 85 participants on one of the seven points of the scale, we randomly drew a subset of participants. From the total of 595 participants that were invited to participate in Session 2, 488 participants opened the study link and 465 participants completed the study. Again, participants provided informed consent. Participants who did not pass all attention checks (see below) were excluded from the analysis, resulting in a final sample of 417 participants (M_age = 38.05, SD = 13.39, 18–75 years; 222 male, 194 female, 1 diverse). In terms of political orientation, 331 participants identified as Republicans, 24 as Democrats, 54 as Independents, and eight as “Other” (see Supplemental material for information about ethnicity and education). Participants were compensated with £1.35 upon study completion. Participants were debriefed after completing Session 2. More specifically, participants were informed that the two studies were fictitious and that the existing scientific evidence clearly indicates that wearing masks is an effective way to slow down the spread of SARS-CoV-2. The main study lasted about 10 minutes. Data collection took place from 13 to 17 February 2021, that is, at a point during the pandemic when mask mandates in the United States were publicly debated and still in place.

Materials

Attitude toward mask mandates

Attitudes toward mask mandates were measured with the same attitude item as in Session 1 (“What is your attitude toward mask mandates in public?”). In addition, we measured attitude strength as a potential moderator of the relationship between the attitude and motivated reasoning (for an overview of research on attitude strength, see Howe and Krosnick, 2017). Following Pomerantz et al. (1995), we assessed three facets of attitude strength (all items were measured on 7-point Likert-type scales ranging from 1 to 7): attitude certainty (“How certain are you of your views about mask mandates?” “How sure are you that your opinion about mask mandates is right?”), attitude importance (“To you personally, how important is the issue of mask mandates?” “Personally, how much do you care about the issue of mask mandates?”), and ego-involvement (“How central is your attitude toward mask mandates to your self-image?” “How representative of your values is your attitude toward mask mandates?”). The internal consistency of the 6-item attitude strength scale was satisfactory, Cronbach’s α = .79.

Studies on the effectiveness of mask mandates

Participants evaluated two fictitious studies on the effectiveness of mask mandates in schools (see Figure 1): one study demonstrating that mask mandates in schools are effective (pro-mask study) and one study demonstrating that mask mandates in schools are counterproductive (anti-mask study). The results of the studies were presented in two-by-two contingency tables adapted from Kahan et al. (2017b). As the study outcome was manipulated within-subjects to enable the calculation of an individual bias score, we created a second set of numbers in addition to the one used in the original study. The numbers in the contingency tables were generated so that the Binomial Effect Size Display (BESD; Rosenthal and Rubin, 1982) was equivalent across conditions (BESD = 8.8% in the pro-mask studies, BESD = −8.8% in the anti-mask studies; calculated using the tool provided by Lenhard and Lenhard, 2016). Furthermore, all contingency tables showed that there were more schools in which the numbers of infections increased (77%) than schools in which the numbers of infections decreased (23%). Thus, the overall trend was held constant across the two sets of numbers.

Figure 1.

Study conditions.

Most importantly, the numbers in the contingency tables were designed in a way so that superficial processing (i.e. comparing absolute numbers between two cells instead of ratios) easily leads to the wrong interpretation (see also Kahan et al., 2017b). Take the pro-mask study in Stimulus Set A, for instance: When looking at the absolute numbers in the upper left and lower left cell only, there are more schools with a mask mandate (223) than schools without a mask mandate (107) that show an increase in the number of infections, which can lead to the wrong conclusion that a mask mandate is counterproductive. In fact, however, there is an increase in the number of infections in 223 out of 298 schools with a mask mandate (74.8%) compared to an increase in the number of infections in 107 out of 128 schools without a mask mandate (83.6%), indicating that a mask mandate is an overall successful intervention.

Exactly the opposite is true for the anti-mask study in Stimulus Set A. Here, there are fewer schools with a mask mandate (97) than schools without a mask mandate (205) that show an increase in infections, which can lead to the wrong conclusion that a mask mandate is effective. In fact, however, there is an increase in the number of infections in 97 out of 116 schools with a mask mandate (83.6%) compared to an increase in the number of infections in 205 out of 274 schools without a mask mandate (74.8%), indicating that a mask mandate is an overall counterproductive intervention.

Participants were asked to indicate what can be concluded from the presented studies on a 6-point Likert-type scale (ranging from 1 = “A mask mandate in schools lead to an increase in the number of infections” to 6 = “A mask mandate leads to a decrease in the number of infections”). Note that participants may legitimately differ regarding their estimation of the strength of the evidence provided by the studies (i.e. response options 4–6 are correct in the case of the pro-mask study and response options 1–3 are correct in the case of the anti-mask study). As the ratios in the pro-mask and the anti-mask study are exactly reversed and the BESD is equivalent (see above), however, an unbiased observer should show a symmetric response pattern (e.g. tick response option “4” when evaluating the pro-mask study and option “3” when evaluating the anti-mask study). Differently put, an asymmetric response pattern indicates a bias (see Figure 2).

Figure 2.

Bias scores.

Based on this assumption, two dependent variables can be calculated. The anti-mask/pro-mask bias score is the difference between the response to the pro-mask study and the reversely coded response to the anti-mask study. A positive value indicates a pro-mask bias, while a negative value indicates an anti-mask bias. That is, the anti-mask/pro-mask bias score indicates both the direction and the degree of the bias. Note that a zero anti-mask/pro-mask bias score does not necessarily indicate a correct evaluation of the studies, as this score can also result from a symmetric, but incorrect evaluation of the studies. By taking the absolute value of the pro-mask bias score, one can calculate an absolute bias score, which indicates the degree of the bias irrespective of the direction. In addition, one can calculate the study evaluation accuracy, that is, whether a study is evaluated correctly (1) or not (0).

Numeracy

Numeracy was measured using the 8-item numeracy scale developed by Weller et al. (2013). The scale includes items such as “In the Big Bucks Lottery, the chances of winning a $10.00 prize are 1%. What is your best guess about how many people would win a $10.00 prize if 1,000 people each buy a single ticket from Big Bucks?” The numeracy score for each participant equals the number of correct responses, ranging from zero to eight.

Information-seeking behavior

For the purpose of exploratory analyses, information-seeking behavior during the pandemic was measured using two self-generated items (for details, see Online material).

Attention checks

The study included four attention check items (for details, see Online material). Two items were intermixed with the questions on the attitude toward mask mandates. Two items in the form of multiple-choice questions were presented at the end of the study to ensure that participants had read the instructions carefully.

Procedure

First, participants answered the questions about their attitude toward mask mandates in public. Next, participants were told that they would be presented with the results of two scientific studies investigating the effect of mask mandates in schools on the spread of the new coronavirus (for detailed instructions, see Online material). Moreover, participants were told that the studies were conducted in two different states of the United States—and that it will be their task to evaluate whether the studies support the conclusion that a mask mandate in schools is an overall effective intervention or not. Whether participants first evaluated the pro-mask study or the anti-mask study was counterbalanced across participants. Which of the two sets of numbers was used for the pro-mask study and for the anti-mask study was counterbalanced across participants. The two counterbalancing factors did not influence the results (see Online Supplemental material). After evaluating the two studies, participants completed the numeracy scale and answered the questions regarding their information-seeking behavior. Finally, participants responded to the two multiple-choice attention check items.

2. Results

Study evaluation accuracy

Overall, study evaluation accuracy rates were low, confirming the validity of our stimulus material as posing difficult statistical problems. In particular, only 46% of participants evaluated the pro-mask study correctly (i.e. selected 4, 5, or 6 on the 6-point Likert-type scale), and only 39% of participants evaluated the anti-mask study correctly (i.e. selected 1, 2, or 3).

Did participants engage in motivated reasoning?

To investigate whether participants engaged in motivated reasoning, we analyzed two dependent variables: the anti-mask/pro-mask bias score and the absolute bias score.

Anti-mask/pro-mask bias score

Motivated reasoning would be indicated by a significant positive relationship between participants’ anti-mask/pro-mask attitude and their anti-mask/pro-mask bias score (for zero-order correlations, see Table 1). Moreover, the relationship between the anti-mask/pro-mask attitude and the anti-mask/pro-mask bias score should increase with higher attitude strength. To test these hypotheses, we conducted a moderated regression analysis using PROCESS for SPSS (Hayes, 2018). The criterion was the anti-mask/pro-mask bias score. The predictors were the anti-mask/pro-mask attitude (mean-centered), attitude strength (i.e. the mean of the six attitude strength items, mean-centered), and the product term of the two variables. As expected, the anti-mask/pro-mask attitude significantly predicted the anti-mask/pro-mask bias score, b = .32, 95% CI [0.21, 0.43], t(413) = 5.77, p < .001, β = .30. This relationship was significantly moderated by attitude strength, b = .09, 95% CI [0.004, 0.174], t(413) = 2.05, p = .041, ΔR² = .009. Conditional effects analysis revealed that the relationship between attitude and bias was significant at all tested levels of attitude strength, yet the association increased with attitude strength (see Figure 3a), b = .21, 95% CI [0.03, 0.39], t(413) = 2.35, p = .019, β = .20 at −1 SD below the mean; b = .32, 95% CI [0.21, 0.43], t(413) = 5.77, p < .001, β = .30 at the mean, b = .43, 95% CI [0.31, 0.54], t(413) = 7.09, p < .001, β = .40 at +1 SD above the mean. The main effect of attitude strength was not a significant predictor of bias, b = −.04, 95% CI [−0.21, 0.14], t(413) = −0.43, p = .67, β = −.02. Taken together, the findings suggest that participants evaluated the study results consistent with their prior attitudes: Participants with a pro-mask attitude overestimated the evidence for the effectiveness of mask mandates, while participants with an anti-mask attitude underestimated the evidence for the effectiveness of mask mandates. This effect was more pronounced for participants with stronger attitudes toward mask mandates, that is, for participants who were more certain about their attitude toward mask mandates and to whom this attitude was more important.

Table 1.

Descriptive statistics and zero-order correlations.

Variable	M	SD	1	2	3	4	5	6
1. Anti-mask/pro-mask attitude	4.18	2.09	–
2. Attitude strength	4.67	1.21	0.29**	–
3. Numeracy	5.20	1.76	0.07	0.06	–
4. Pro-mask study evaluation	3.26	1.64	0.25**	0.04	0.03	–
5. Anti-mask study evaluation	3.93	1.45	0.23**	0.08	−0.21**	0.02	–
6. Anti-mask/pro-mask bias	0.19	2.21	0.34**	0.08	−0.12*	0.76**	0.67**	–
7. Absolute bias	1.58	1.56	0.01	0.12*	−0.14**	0.17**	−0.05	0.09

SD: standard deviation, M: mean.

Scale range of the variables: anti-mask/pro-mask attitude: 1–7; attitude strength: 1–7; numeracy: 0–8; pro-mask study evaluation: 1–6; anti-mask study evaluation: 1–6; anti-mask/pro-mask bias: −5–5; absolute bias: 0–5.

p < .05, **p < .001.

Figure 3.

Anti-mask/pro-mask bias and attitude. (a) Depicts the anti-mask/pro-mask bias score regressed on anti-mask/pro-mask attitude at three levels of attitude strength (−1 SD below the mean, mean, +1 SD above the mean). (b) Depicts the anti-mask/pro-mask bias score regressed on anti-mask/pro-mask attitude at three levels of numeracy (−1 SD below the mean, mean, +1 SD above the mean).

Absolute bias score

The absolute bias score represents the degree of bias, irrespective of the direction. Analyzing the absolute bias score sheds light on the question whether participants with anti-mask and pro-mask attitudes differ in their degree of bias. We regressed absolute bias scores on anti-mask/pro-mask attitudes (mean-centered), attitude strength (mean-centered), and the product term of these variables. The anti-mask/pro-mask attitude was not significantly related to the absolute bias score, b = −.03, 95% CI [−0.11, 0.05], t(413) = −0.69, p = .49, β = −.04. Attitude strength was significantly associated with the absolute bias score, b = .17, 95% CI [0.04, 0.29], t(413) = 2.52, p = .012, β = .13. The interaction term was not significant, b = .02, 95% CI [−0.05, 0.08], t(413) = 0.49, p = .62, ΔR² = .001. Together, the results suggest that the degree of bias did not differ between participants with anti-mask and pro-mask attitudes. In other words, participants with a pro-mask attitude were not more biased than participants with an anti-mask attitude (and vice versa). Yet, the degree of bias increased with attitude strength, independent of attitude direction. That is, the stronger the participants’ attitude was toward mask mandates, the stronger their bias.

What is the role of numeracy?

As previous research yielded inconsistent results with respect to the role of numeracy, we tested two alternative hypotheses, that is, higher numeracy is related to increased motivated reasoning versus higher numeracy is related to less bias and more accurate study evaluation. To fully investigate these alternative hypotheses, we analyzed three dependent variables, the anti-mask/pro-mask bias score, the absolute bias score, and study evaluation accuracy.

Anti-mask/pro-mask bias score

To test how numeracy is related to motivated reasoning, we regressed the anti-mask/pro-mask bias score on anti-mask/pro-mask attitude (mean-centered), numeracy (mean-centered), and the product term of attitude and numeracy. Replicating the previous analysis, the anti-mask/pro-mask attitude significantly predicted the anti-mask/pro-mask bias score, b = .36, 95% CI [0.27, 0.46], t(413) = 7.46, p < .001, β = .34. Furthermore, higher numeracy was reversely related to the anti-mask/pro-mask bias score, b = −.18, 95% CI [−0.30, −0.07], t(413) = −3.15, p = .002, β = −.14. Most importantly, the interaction between attitude and numeracy was not significant, b = −.05, 95% CI [−0.10, 0.01], t(413) = −1.74, p = .082, ΔR² = .006. Inspection of the simple slopes (Figure 3b) shows that, if anything, the relationship between attitude and bias was stronger at lower levels of numeracy. Thus, the results do not support the hypothesis that higher numeracy is related to increased motivated reasoning.

Absolute bias score

To investigate whether numeracy is related to a lower degree of bias, we regressed absolute bias scores on numeracy (mean-centered), anti-mask/pro-mask attitudes (mean-centered), and the product term of numeracy and attitudes. Higher numeracy was reversely related to absolute bias scores, b = −.13, 95% CI [−0.21, −0.04], t(413) = −2.95, p = .003, β = −.14. Neither mask attitudes, b = .01, 95% CI [−0.06, 0.09], t(413) = 0.37, p = .71, β = .02, nor the interaction of mask attitudes and numeracy, b = −.01, 95% CI [−0.06, 0.03], t(413) = −0.71, p = .48, ΔR² = .001, were significant predictors. Thus, the results show that higher numeracy was related to decreased bias (i.e. to decreased motivated reasoning).

Study evaluation accuracy

As the absence of bias does not necessarily indicate correct study evaluation (see “Method” section), we finally investigated study evaluation accuracy. In particular, we examined the influence of mask attitude and numeracy (and their interaction) on the likelihood that the study results were interpreted incorrectly (0) or correctly (1). We ran two logistic regression analyses with pro-mask study evaluation accuracy or anti-mask study evaluation accuracy as the criterion.

The likelihood that participants evaluated the results of the pro-mask study correctly (i.e. indicated that it showed evidence in support of mask mandates) was predicted by mask attitude, OR = 1.20, 95% CI [1.09, 1.33], p < .001. The more strongly participants favored masks, the higher the likelihood they interpreted the results of the pro-mask study correctly. Numeracy was unrelated to evaluation accuracy, OR = 1.03, 95% CI [0.92, 1.16], p = .56, and there was no interaction between both predictors, OR = 0.98, 95% CI [0.93, 1.04], p = .48. That is, numeracy did not significantly influence whether the pro-mask study was interpreted correctly.

Regarding the results of the anti-mask study, the likelihood that participants evaluated the study correctly (i.e. indicated that it showed evidence that mask mandates are counterproductive) was predicted by mask attitude, OR = 0.81, 95% CI [0.73, 0.90], p < .001, and numeracy, OR = 1.26, 95% CI [1.12, 1.42], p < .001. No interaction between both factors was observed, OR = 1.02, 95% CI [0.96, 1.08], p = .55. The more strongly participants opposed masks, the higher the likelihood they interpreted the results of the anti-mask study correctly. Higher numeracy increased the likelihood that participants correctly interpreted the anti-mask study.

The relative frequency of correct answers as a function of numeracy can be found in Figure 4.

Figure 4.

Relative frequency of correct answers as a function of numeracy (quartile split).

3. Discussion

This study had two goals. First, to test the hypothesis that motivated reasoning plays a role in the evaluation of scientific evidence in the context of the COVID-19 pandemic. Second, to investigate whether numeracy decreases or increases the degree of bias.

Regarding the first goal, results clearly show that participants engaged in motivated reasoning when evaluating studies on the effectiveness of mask mandates in schools. In particular, the more strongly participants opposed a mask mandate in public, the more strongly they exhibited a bias toward interpreting the studies as showing that a mask mandate in schools is counterproductive. Conversely, the more strongly participants favored a mask mandate in public, the more strongly they exhibited a bias toward interpreting the studies as showing that a mask mandate in schools is an effective intervention. This pattern was even more pronounced among participants who reported a higher attitude strength, that is, among participants who were more certain about their attitude toward mask mandates and to whom this attitude was more important. Note, however, that this interaction effect was quite small and accounted for less than 1% of uniquely explained variance. In short, this study strengthens the assumption that motivated reasoning is a basic cognitive mechanism that can be found across the entire attitude spectrum (Kruglanski, 1996; Kunda, 1990).

One potential limitation may be that most of the participants in this study were Republicans, confining our results to this group of people. However, previous research has demonstrated that both Democrats and Republicans are susceptible to biased information-seeking (e.g. Kahan et al., 2017b; Washburn and Skitka, 2018), making it likely that our findings also apply to the general population. Nevertheless, future research could address this question in more detail (for a discussion of similarities and differences between Democrats and Republicans, see Ditto et al., 2019 as well as Baron and Jost, 2019). Importantly, the concentration on Republicans in this study may even be an advantage. As the number of people opposing corona policies is higher among Republicans than among Democrats (see the data from the first 350 participants in Session 1; see also Druckman et al., 2021; Gollwitzer et al., 2020), recruiting a random sample of participants across the entire political spectrum would have led to Republicans being more prominent among people opposing mask mandates and Democrats being more prominent among people supporting mask mandates. Such a distribution would result in a confound between political orientation and mask attitudes. By recruiting (almost) only Republicans, political orientation was held constant across the sample, rendering it unlikely that the results were driven by differences in political orientation rather than attitudes toward mask mandates (c.f. Tappin et al., 2020b).

Regarding the second goal (i.e. examining the role of numeracy), our data show that the ability to understand and reason with numbers is associated with decreased bias. Thus, our results are in line with previous studies suggesting that numeracy can lead to better decisions even in the context of highly politicized issues (Connor et al., 2020; Lind et al., 2018; Tappin et al., 2020a)—and contradict those studies which have found increased motivated reasoning among highly numerate people (Drummond and Fischhoff, 2017; Kahan et al., 2012, 2017a, 2017b; Nurse and Grant, 2020). This study revealed two more interesting findings concerning numeracy.

First and somewhat surprisingly, numeracy was related to study evaluation accuracy on the anti-mask, but not on the pro-mask study. More specifically, the pro-mask study was evaluated correctly by about 50% of participants, irrespective of their numerical ability. Conversely, the anti-mask study was evaluated correctly by about 50% of the higher numerate participants, but only by about 30% of the lower numerate participants. As most research reported in scientific journals supports the conclusion that wearing masks can help to contain the spread of SARS-CoV-2 (see Peeples, 2020), one may speculate that participants expected to be presented with studies demonstrating that mask mandates are effective. As a consequence, participants, especially those low in numeracy, may have based their responses—at least to some extent—on this expectation. Importantly, such an expectation-based response bias would cause wrong answers to the anti-mask study, but not to the pro-mask study, which could explain the pattern observed in this study. However, this potential explanation deserves further investigation.

Second, evaluation accuracy regarding both studies was relatively low across the entire sample, although participants were quite highly educated (62% held at least a bachelor’s degree; for details, see Supplemental material). More specifically, the majority of the participants evaluated the studies incorrectly—and even among the highly numerate participants, only 50% gave the correct answers. Hence, analyzing contingency tables indeed posed a difficult statistical problem to participants. This is potentially problematic, as a lot of pandemic- and health-related information is—explicitly or implicitly—based on contingency tables (e.g. sensitivity and specificity of diagnostic tests, vaccine efficacy). Moreover, properly understanding the complex models and metrics, which are used to communicate the current state of the pandemic, arguably requires more sophisticated statistical knowledge than the ability to compare the ratios of contingency tables. Taken together, this makes the participants’ poor performance in this study even more worrying (see also Paakkari & Okan, 2020).

Thus, this study also has important practical implications. In the short run, it seems important to communicate scientific results in the context of COVID-19 in a way that can easily be understood—especially because the successful management of the pandemic requires behavior change at the individual level (Habersaat et al., 2020; Haug et al., 2020; Van Bavel et al., 2020). Potential strategies to do so may include supporting numeric information with visual representations (Garcia-Retamero and Galesic, 2009; for an overview, see Peters, 2012). In the long run, it will be crucial to foster statistical education in order to reduce innumeracy in the population (Fong et al., 1986; Gigerenzer et al., 2007). Apart from these practical considerations, this study contributes to understanding the psychological mechanisms underlying polarized discussions about the appropriateness of different policies in the current pandemic.

Supplemental Material

sj-pdf-1-pus-10.1177_09636625211047974 – Supplemental material for The role of motivated science reception and numeracy in the context of the COVID-19 pandemic

Supplemental material, sj-pdf-1-pus-10.1177_09636625211047974 for The role of motivated science reception and numeracy in the context of the COVID-19 pandemic by Fabian Hutmacher, Regina Reichardt and Markus Appel in Public Understanding of Science

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Fabian Hutmacher

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Fabian Hutmacher is a postdoctoral researcher at the Human-Computer-Media Institute at the University of Würzburg, Germany. Among other things, he is interested in information processing and memory.

Regina Reichardt is a lecturer at the Department of Psychology at the University of Regensburg, Germany, with research interests in social cognition and educational psychology.

Markus Appel is a professor (Psychology of Communication and New Media) at the University of Würzburg, Germany. Among other things, he is interested in how different forms and formats of information shape our view of the world.

References

Adam

(2020) A guide to R—The pandemic’s misunderstood metric. Nature 583: 346–348.

Apter

Wang

Bogen

Bennett

Jennings

García

, et al. (2009) Linking numeracy and asthma-related quality of life. Patient Education and Counseling 75(3): 386–391.

Baron

Jost

(2019) False equivalence: Are liberals and conservatives in the United States equally biased? Perspectives on Psychological Science 14(2): 292–303.

Bender

Rothmund

Nauroth

Gollwitzer

(2016) How moral threat shapes laypersons’ engagement with science. Personality and Social Psychology Bulletin 42(12): 1723–1735.

Bierwiaczonek

Kunst

Pich

(2020) Belief in COVID-19 conspiracy theories reduces social distancing over time. Applied Psychology: Health and Well-Being 12(4): 1270–1285.

Connor

Sullivan

Alfano

Tintarev

(2020) Motivated numeracy and active reasoning in a Western European sample. Behavioural Public Policy. Epub ahead of print 5 August. DOI: 10.1017/bpp.2020.32.

Davids

Schapira

McAuliffe

Nattinger

(2004) Predictors of pessimistic breast cancer risk perceptions in a primary care population. Journal of General Internal Medicine 19(4): 310–315.

Dawson

Gilovich

Regan

(2002) Motivated reasoning and performance on the Wason selection task. Personality and Social Psychology Bulletin 28(10): 1379–1387.

Ditto

Liu

Clark

Wojcik

Chen

Grady

, et al. (2019) At least bias is bipartisan: A meta-analytic comparison of partisan bias in liberals and conservatives. Perspectives on Psychological Science 14(2): 273–291.

10.

Druckman

Klar

Krupnikov

Levendusky

Ryan

(2021) Affective polarization, local contexts and public opinion in America. Nature Human Behaviour 5(1): 28–38.

11.

Drummond

Fischhoff

(2017) Individuals with greater science literacy and education have more polarized beliefs on controversial science topics. Proceedings of the National Academy of Sciences of the United States of America 114(36): 9587–9592.

12.

Edwards

Smith

(1996) A disconfirmation bias in the evaluation of arguments. Journal of Personality and Social Psychology 71(1): 5–24.

13.

Fong

Krantz

Nisbett

(1986) The effects of statistical training on thinking about everyday problems. Cognitive Psychology 18(3): 253–292.

14.

Garcia-Retamero

Galesic

(2009) Communicating treatment risk reduction to people with low numeracy skills: A cross-cultural comparison. American Journal of Public Health 99(12): 2196–2202.

15.

Garcia-Retamero

Andrade

Sharit

Ruiz

(2015) Is patients’ numeracy related to physical and mental health? Medical Decision Making 35(4): 501–511.

16.

Garcia-Retamero

Sobkow

Petrova

Garrido

Traczyk

(2019) Numeracy and risk literacy: What have we learned so far? The Spanish Journal of Psychology 22: e10.

17.

Ghazal

Cokely

García-Retamero

(2014) Predicting biases in very highly educated samples: Numeracy and metacognition. Judgment and Decision Making 9(1): 15–34.

18.

Gigerenzer

Gaissmaier

Kurz-Milcke

Schwartz

Woloshin

(2007) Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest 8(2): 53–96.

19.

Gollwitzer

Martel

Brady

Pärnamets

Freedman

Knowles

, et al. (2020) Partisan differences in physical distancing are linked to health outcomes during the COVID-19 pandemic. Nature Human Behaviour 4(11): 1186–1197.

20.

Gurmankin

Baron

Armstrong

(2004) Intended message versus message received in hypothetical physician risk communications: Exploring the gap. Risk Analysis 24(5): 1337–1347.

21.

Habermas

(1996) Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Cambridge, MA: MIT Press.

22.

Habersaat

Betsch

Danchin

Sunstein

Böhm

Falk

, et al. (2020) Ten considerations for effectively managing the COVID-19 transition. Nature Human Behaviour 4(7): 677–687.

23.

Hart

Nisbet

(2012) Boomerang effects in science communication: How motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies. Communication Research 39(6): 701–723.

24.

Haug

Geyrhofer

Londei

Dervic

Desvars-Larrive

Loreto

, et al. (2020) Ranking the effectiveness of worldwide COVID-19 government interventions. Nature Human Behaviour 4(12): 1303–1312.

25.

Hayes

(2018) Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach, 2nd edn. New York: Guilford Press.

26.

Hornsey

(2020) Why facts are not enough: Understanding and managing the motivated rejection of science. Current Directions in Psychological Science 29(6): 583–591.

27.

Hornsey

Fielding

(2017) Attitude roots and Jiu Jitsu persuasion: Understanding and overcoming the motivated rejection of science. American Psychologist 72(5): 459–473.

28.

Howard

Huang

Tufekci

Zdimal

Van der Westhuizen

, et al. (2021) An evidence review of face masks against COVID-19. Proceedings of the National Academy of Sciences of the United States of America 118(4): e2014564118.

29.

Howe

Krosnick

(2017) Attitude strength. Annual Review of Psychology 68(1): 327–351.

30.

Huizinga

Beech

Cavanaugh

Elasy

Rothman

(2008) Low numeracy skills are associated with higher BMI. Obesity 16(8): 1966–1968.

31.

Kahan

Braman

Slovic

Gastil

Cohen

(2009) Cultural cognition of the risks and benefits of nanotechnology. Nature Nanotechnology 4(2): 87–90.

32.

Kahan

Landrum

Carpenter

Helft

Hall Jamieson

(2017a) Science curiosity and political information processing. Political Psychology 38: 179–199.

33.

Kahan

Peters

Dawson

Slovic

(2017b) Motivated numeracy and enlightened self-government. Behavioural Public Policy 1: 54–86.

34.

Kahan

Peters

Wittlin

Slovic

Ouellette

Braman

, et al. (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2(10): 732–735.

35.

Kruglanski

(1996) Motivated social cognition: Principles of the interface. In: Kruglanski

Higgins

(eds) Social Psychology: Handbook of Basic Principles. New York: Guilford Press, pp. 493–520.

36.

Kunda

(1987) Motivated inference: Self-serving generation and evaluation of causal theories. Journal of Personality and Social Psychology 53(4): 636–647.

37.

Kunda

(1990) The case for motivated reasoning. Psychological Bulletin 108(3): 480–498.

38.

Leiner

(2019) Sosci Survey [Computer software] (Version 3.1.06). Available at: https://www.soscisurvey.de

39.

Lenhard

(2016) Berechnung von Effektstärken [Calculation of effect sizes]. Dettelbach: Psychometrica.

40.

Lewandowsky

Oberauer

(2016) Motivated rejection of science. Current Directions in Psychological Science 25(4): 217–222.

41.

Lind

Erlandsson

Västfjäll

Tinghög

(2018) Motivated reasoning when assessing the effects of refugee intake. Behavioural Public Policy. Epub ahead of print 6 December. DOI: 10.1017/bpp.2018.41.

42.

Lord

Ross

Lepper

(1979) Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology 37(11): 2098–2109.

43.

Nauroth

Gollwitzer

Bender

Rothmund

(2014) Gamers against science: The case of the violent video games debate. European Journal of Social Psychology 44(2): 104–116.

44.

Nauroth

Gollwitzer

Bender

Rothmund

(2015) Social identity threat motivates science discrediting online comments. PLoS One 10(2): e0117476.

45.

Nurse

Grant

(2020) I’ll see it when I believe it: Motivated numeracy in perceptions of climate change risk. Environmental Communication 14(2): 184–201.

46.

Paakkari

Okan

(2020) COVID-19: Health literacy is an underestimated problem. The Lancet Public Health 5(5): e249–e250.

47.

Pachur

Galesic

(2013) Strategy selection in risky choice: The impact of numeracy, affect, and cross-cultural differences. Journal of Behavioral Decision Making 26(3): 260–271.

48.

Peeples

(2020) Face masks: What the data say. Nature 586: 186–189.

49.

Persson

Andersson

Koppel

Västfjäll

Tinghög

(2021) A preregistered replication of motivated numeracy. Cognition 214: 104768.

50.

Peters

(2012) Beyond comprehension: The role of numeracy in judgments and decisions. Current Directions in Psychological Science 21(1): 31–35.

51.

Pomerantz

Chaiken

Tordesillas

(1995) Attitude strength and resistance processes. Journal of Personality and Social Psychology 69(3): 408–419.

52.

Prime

Wade

Browne

(2020) Risk and resilience in family well-being during the COVID-19 pandemic. American Psychologist 75(5): 631–643.

53.

Reyna

Nelson

Han

Dieckmann

(2009) How numeracy influences risk comprehension and medical decision making. Psychological Bulletin 135(6): 943–973.

54.

Rolison

Morsanyi

Peters

(2020) Understanding health risk comprehension: The role of math anxiety, subjective numeracy, and objective numeracy. Medical Decision Making 40(2): 222–234.

55.

Romer

Jamieson

(2020) Conspiracy theories as barriers to controlling the spread of COVID-19 in the U.S. Social Science & Medicine 263: 113356.

56.

Rosenthal

Rubin

(1982) A simple general purpose display of magnitude of experimental effect. Journal of Educational Psychology 74(2): 166–169.

57.

Rothmund

Gollwitzer

Nauroth

Bender

(2017) Motivierte Wissenschaftsrezeption [Motivated science reception]. Psychologische Rundschau 68(3): 193–197.

58.

Schönbrodt

Perugini

(2013) At what sample size do correlations stabilize? Journal of Research in Personality 47(5): 609–612.

59.

Shoots-Reinhard

Erford

Romer

Evans

Shoben

Klein

, et al. (2020) Numeracy and memory for risk probabilities and risk outcomes depicted on cigarette warning labels. Health Psychology 39(8): 721–730.

60.

Skagerlund

Lind

Strömbäck

Tinghög

Västfjäll

(2018) Financial literacy and the role of numeracy—How individuals’ attitude and affinity with numbers influence financial literacy. Journal of Behavioral and Experimental Economics 74: 18–25.

61.

Tappin

Pennycook

Rand

(2020a) Bayesian or biased? Analytic thinking and political belief updating. Cognition 204: 104375.

62.

Tappin

Pennycook

Rand

(2020b) Rethinking the link between cognitive sophistication and politically motivated reasoning. Journal of Experimental Psychology: General. Epub ahead of print 29 October. DOI: 10.1037/xge0000974.

63.

Traczyk

Sobkow

Fulawka

Kus

Petrova

García-Retamero

(2018) Numerate decision makers don’t use more effortful strategies unless it pays: A process tracing investigation of skilled and adaptive strategy selection in risky decision making. Judgment and Decision Making 13(4): 372–381.

64.

Van Bavel

Baicker

Boggio

Capraro

Cichocka

Cikara

, et al. (2020) Using social and behavioural science to support COVID-19 pandemic response. Nature Human Behaviour 4(5): 460–471.

65.

Vindegaard

Benros

(2020) COVID-19 pandemic and mental health consequences: Systematic review of the current evidence. Brain, Behavior, and Immunity 89: 531–542.

66.

Washburn

Skitka

(2018) Science denial across the political divide: Liberals and conservatives are similarly motivated to deny attitude-inconsistent science. Social Psychological and Personality Science 9(8): 972–980.

67.

Weller

Dieckmann

Tusler

Mertz

Burns

Peters

(2013) Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. Journal of Behavioral Decision Making 26(2): 198–212.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB