Comment on Kahan and Corbin: Can polarization increase with actively open-minded thinking?

Abstract

Kahan and Corbin suggest that the cognitive capabilities that constitute actively open-minded thinking (AOT) are used in the defense of favored positions, thus leading to polarization concerning global warming. Such a conclusion is contrary to the definition of AOT. I suggest, and dispute, two ways in which such a result could be consistent with the definition: the scale that measures AOT is correlated with other traits that do increase polarization; or AOT may itself not be a general trait, so that conservatives do not apply it to global warming. A third, more plausible, explanation is that part of the main result is based on a floor effect, a statistical artifact. It is also possible that the effect of AOT is exerted primarily on political identification. The substantial correlation between AOT and political liberalism also helps us interpret prior conflicting results concerning politics and cognitive style, especially in view of the total absence of any correlation between politics and the Cognitive Reflection Test in the same data set.

Kahan and Corbin (2016, henceforth KC), examining the determinants of the belief that global warming is real and caused by humans (henceforth GWH), reported an interaction between political conservatism and a measure of actively open-minded thinking (AOT). As AOT increased, liberals were more likely to accept GWH, but conservatives showed no effect. Thus, at higher levels of AOT, liberals and conservatives were farther apart, more polarized. KC took their result to imply that AOT could be used in the service of bolstering prior beliefs, rejecting alternative interpretations.

As the originator of the concept of AOT, I found this interpretation odd, because the very definition of AOT (Baron, 2008) implies that people who score highly on this trait will actively seek reasons why they might be wrong, and, if they find such reasons, they will use them in a way that is fair to alternative conclusions. If AOT is a tool, it cannot be employed in the service of bolstering a favored belief. To say that AOT is employed in the service of bolstering is analogous to saying that the trait of honesty is employed in the service of deceiving others.

One way to make sense of the result is based on the fact that KC did not actually measure AOT. Instead, they used a self-report scale of beliefs about the nature of good thinking. This scale arose from early research showing that these beliefs correlate with measures of actual thinking (Baron, 1991, 1995; Baron et al., in press). This was an optimistic result, because it suggested that thinking could be influenced by changing people’s beliefs about what good thinking is, and, indeed, evidence supports this possibility (Baron et al., in press; Perkins et al., 1986). But surely there is some slippage between our beliefs about what we should do and our practices. Thus, the scale used by KC is not a direct measure of thinking itself. And the scale does correlate with measures of cognitive skills that are not designed to measure AOT. It is possible that these other skills, and not AOT itself, are deployed in the service of bolstering beliefs, thus leading to polarization.

I think this explanation is far fetched at best. It requires two assumptions: (1) the AOT scale correlates strongly with these other cognitive skills, compared to its correlation with the trait it purports to measure; (2) these other skills, in turn, are used for bolstering and thus increase GWH among liberals. Moreover, we can test this hypothesis—which involved mediation of the AOT effect on GWH by cognitive skills—using another measure of the sort of cognitive skills that are likely involved. For this purpose the Cognitive Reflection Test (CRT) (Frederick, 2005), a group of three tricky arithmetic problems, is suitable, and this measure was included in the on-line data. The CRT does correlate with AOT in liberals (those below the mean in conservatism, r = 0.37). The usual test of mediation also requires showing that the mediator (CRT) predicts the dependent variable (GWH) in a regression model that includes the independent variable (AOT) as a predictor. In this model, the standardized coefficient for the CRT is 0.04 for liberals and not significant. Yet, in the same model, AOT does predict GWH (coefficient of 0.23 for liberals, p = 0.000). ¹ It is thus unlikely that the effect of AOT is mediated through its correlation with cognitive skills.

A second way in which the result might make sense is this: the AOT scale could measure AOT quite well, but AOT itself might not be a general trait. People may endorse AOT in a self-report questionnaire about beliefs, and they may behave consistently with it in most domains, but they may have gaps when they are strongly committed to a particular view. This explanation seems most compatible with KC’s own conclusions. Thus, even conservatives who are generally high in the AOT trait do not apply this trait to global warming. Liberals with higher AOT will form beliefs more consistent with the evidence, which favors GWH. Thus, we will get the interaction effect shown in KC’s Figure 3, where GWH rises with AOT for liberals but not for conservatives. Note that this explanation would still not account for increased polarization (rejection of GWH) with increasing AOT in conservatives, but, as I shall discuss, such polarization does not occur.

The assumption here is that GWH would increase with AOT in conservatives if they applied AOT to the issue. But this assumption is itself questionable. Instead, AOT could make them less conservative. Denial of GWH is, arguably, part of conservatism itself.

It may seem surprising in this context that some extreme conservatives with high AOT scores still denied GWH. Note, however, that conservatism is correlated negatively with AOT (a fact I shall discuss later). This correlation implies that high-AOT conservatives are relatively rare. Moreover, both the AOT and conservatism scales must have some error in them. Extreme scores on both scales would tend to regress to the mean if the subjects were tested with different questions designed to measure the same traits. The upshot is that truly extreme conservatives with truly high AOT scores (after correction for regression) may not exist in sufficient numbers to permit any conclusion about the beliefs of high-AOT conservatives. ²

Other results suggest that AOT reduces conservatism itself, that is, it increases liberalism. In an ordinary regression of GWH on AOT, the coefficient (for standardized scores) is 0.247. This declines to 0.115 when liberalism is included in the model. (Of course all these correlations are highly significant, and the results are similar with logistic regression.) Yet the coefficient for liberalism declines only from 0.540 to 0.536 when AOT is included in the model predicting GWH from liberalism. In other words, the effect of AOT on GWH appears to be mediated by its effect on liberalism/conservatism itself. Arguably, denial of GWH is part of what conservatism means, and those who accept GWH are unlikely to think of themselves as extreme conservatives. Moreover, AOT may reduce other beliefs that constitute conservatism, although surely not all of them.

Yet it appears that AOT does affect GWH in liberals, which suggests that it ought to do so in conservatives as well. This result is puzzling. We might think that self-styled liberals would endorse most of the beliefs of other liberals, which include GWH, so AOT would have little effect on GWH in this group. KC interpret their result in terms of the use of AOT for bolstering, but I have already pointed out the problem with this interpretation. An alternative interpretation of the result is that low-AOT and high-AOT liberals differ in other ways that affect GWH. Some evidence in the data supports this interpretation. Loosely put, high-AOT liberals tend to be better educated, more politically involved (hence more likely to register to vote), and less likely to be under-represented minorities (who are statistically less well educated). AOT, in turn, is correlated with education, possibly because of causes in both directions, and other things that correlate with education. Low-AOT liberals count themselves as liberal because they are concerned with issues that affect them directly, such as social injustice, but they do not follow issues such as GWH as closely as the high-AOT, more educated, liberals do.

To test this possibility from the data, I regressed GWH on three measures that could be relevant: education, majority (not an underrepresented minority, i.e., not black and not Hispanic), and registration (to vote), with particular attention to their interaction with liberalism (the negative of conservatism). The interaction assesses whether the effects are greater for liberals, which is what the argument of the last paragraph implies. All three measures showed a substantial and highly significant positive interaction with liberalism in predicting GWH, just as does the measure of AOT. ⁴ Moreover, AOT was predicted by the same three variables. ⁴ Thus, we can understand the effect of AOT among liberals as an artifact arising from the differences, largely in education, between high-AOT and low-AOT liberals.

In sum, one part of the interaction observed by KC (shown in their Figure 3), the increase in GWH with AOT among liberals, may result from the different concepts of “liberal” that are related to AOT itself. Otherwise, it seems difficult to understand why all self-styled liberals are not strong believers in GWH, which is, after all, part of the liberal agenda in American politics, just as opposition to action is part of the conservative agenda. The differences between high- and low-AOT liberals weaken the argument that conservatives should also be affected by AOT. And we need not consider the possibility that higher AOT leads to bolstering of pet beliefs, even if this possibility made sense on its own.

The other part of the observed interaction concerns the lack of effect of AOT among conservatives. I already noted that such an effect may be difficult to see because of the small number of extreme conservatives who are high on the AOT scale. But there is another, more serious, problem: a “floor” effect for GWH in conservatives, which also obscures the effect of any manipulation in this group. ⁵ In a yes/no question that does not allow expression of confidence, conservatives may still say that they do not accept GWH. These negative beliefs could weaken considerably as AOT increases. That is what AOT is supposed to do. According to the theory, it should make people less confident in poorly supported beliefs, even if the beliefs are still considered more likely to be true than false. To see an effect, we would need a continuous measure of strength of belief, such as a probability assigned to some ultimately verifiable proposition. To claim no effect of AOT among extreme conservatives is like claiming no effect of blindfolds on maze learning in an experiment in which the maze is almost never solved, with or without blindfolds.

As discussed by Wagenmakers et al. (2012), the floor-effect problem could be overcome if in fact polarization were present for conservatives as well as liberals, that is, a negative correlation between AOT and GWH in extreme conservatives. However, my examination of the data failed to find any cut-off value for conservatism such that conservatives beyond the cut-off showed a negative correlation between AOT and GWH. In fact, for the most extreme group of conservatives (rounded z score of 2), the correlation was slightly positive, although not significant.

In theory, we could find a measure of GWH that might get off the floor. For example, we could ask people if they were 100% certain that warming was not caused by humans. Or we could use a scale with several questions, and/or a graded response scale for each question instead of a yes/no dichotomy. KC did not have data like these, but they did ask whether the climate was getting warmer (whatever its cause). This question was “easier” for conservatives to agree with, and responses to it were uniformly more positive (43% for those with conservatism greater than 0, as opposed to 23% for GWH, although both questions showed [point biserial] correlations with conservatism of −0.54). In fact, those with conservatism scores greater than 0 showed a significant positive effect of AOT on belief in warming itself (GW, logistic regression coefficient of 0.040, p = .009), while the same group of conservatives showed no significant correlation with AOT on the measure actually used (GWH, coefficient of 0.015). This shows that it is possible to increase sensitivity to AOT by getting up off the floor. ⁶

Another measure in the data was an answer to a question about risk, “How much risk do you believe global warming poses to human health, safety, or prosperity?” on a 0–7 scale in which 0 indicated “no risk at all” and 1 indicated “very low risk”. This question, like the warming question, showed the same interaction between AOT and liberalism, but 59% of the most extreme conservative group (rounded score of 2) answered with categories 0 or 1, so they did not have much room to move lower.

In sum, I see no reason why we need to question the role of AOT from these data. A plausible interpretation of the data is that true polarization does not increase with true AOT. Indeed, under improved definitions of liberalism/conservatism (which distinguished high- and low-AOT liberals), AOT could have little effect within a given political group, in part because it would tend to change membership in the groups themselves. The main point of the KC paper is not supported.

Despite the problems with KC’s main conclusions, it is of some interest that a regression of conservatism on AOT yielded a standardized coefficient of −0.27 (p = .000, which is −0.41 corrected for unreliability), a substantial relationship. A number of studies have claimed to find correlations between some measure of cognitive style and left/right political ideology (e.g. Deppe et al., 2015). These correlations are often small and labile. In general, conservatives are thought to be less reflective, more intuitive. but the results depend on the particular measure of political ideology (see Baron, 2015, linked from Deppe, 2015). Many of these studies use the CRT as a measure of cognitive style. In the present data, the CRT did not correlate at all with liberalism. A regression of conservatism on CRT yielded a standardized coefficient of 0.001.

Baron et al. (2015) argued, on the basis of data, that the CRT and AOT scales measure different styles, which are nonetheless correlated moderately with each other (as also argued by Baron et al., in press). Following Baron (1995), we argued that thinking can be described in terms of two dimensions, amount and direction. We argued that the CRT is largely a measure of amount. Specifically, it works as a predictor because it is sensitive to the trait of reflection/impulsivity: the willingness to take more time in order to be more accurate. The response time to CRT items is essentially as good a predictor of other measures as is the number of correct responses.

People do better on tricky arithmetic problems if they are willing to take more time, possibly by using a more reliable method from the outset rather than making a plausible guess. But, when it comes to controversial political and moral questions, it is all too tempting to spend that extra time thinking of reasons why we are right and the other side is wrong. AOT as a trait counteracts this tendency. People who value AOT feel that they ought to spend time considering the other side.

In sum, if we are looking for measures of cognitive style that predict political ideology along the main dimension of the current American political spectrum, AOT seems more informative than the CRT. Conservatives do seem to be more unreflective, less critical of their own beliefs. Of course this is not to say that every issue will show these effects. “Liberals” or “progressives” undoubtedly have some tribal beliefs that would not stand up very well to critical scrutiny. But KC do find quite a high negative correlation between AOT and self-reported conservatism. This is an important result.