Physiology and political beliefs: A response to Knoll,O’Daniel,and Cusato

Abstract

In a recent paper in this journal, Knoll et al. question three studies from our laboratory. In this response to that paper, we address deficiencies in their “reproduction.” Notably, we demonstrate that their data provide little evidence of a negativity bias among research subjects, suggesting a failure not only to reproduce findings from our earlier studies, but also a failure to find a widely acknowledged universal human physiological response trait. This situation raises a number of questions regarding the data on which their analyses are based. We explore these questions below and speculate that Knoll et al.’s data collection procedures may compromise their ability to speak to the external validity of earlier studies.

Introduction

A recent Research & Politics article by Knoll et al. (2015) raises questions about the generalizability of findings contained in three articles coming out of our laboratory (Dodd et al., 2012; Oxley et al., 2008; Smith et al., 2011). We welcome Knoll et al.’s interest and efforts and readily acknowledge that our previous studies, like all research, could have “gotten it wrong.” The scientific process works best when numerous research teams address a selected topic in order that results can be verified, challenged, extended, or modified. This process is greatly aided when those teams allow open access to their data, and we are happy to report that Knoll et al. responded promptly and professionally to our request for theirs. However, we are unconvinced that Knoll et al.’s “reproduction” study provides grounds for questioning the findings we reported earlier. In this comment, we first discuss negativity bias in general, then Knoll et al.’s specific results, and finally the likely reasons for their null findings.

Negativity bias

Negativity bias refers to the well-established and widely accepted empirical finding that people on average are significantly more sensitive, attentive, and responsive to negative than to positive arousing stimuli. Evidence for this bias appears in a variety of cultural contexts and in the wake of an impressive range of research designs (for a review, see Cacioppo and Berntson, 1994). Further, negativity bias brings with it an obvious and compelling evolutionary logic: ignore a positive stimulus and you may not eat, ignore a negative stimulus and you may get eaten—or eat something that could do you harm.

Even as negativity bias appears to be something of a human universal, individual-level variation in it is known to exist and many hypotheses regarding the correlates of this variation have been offered and tested. The relevant hypothesis here is that heightened negativity bias is positively correlated with conservative political beliefs. Numerous studies along these lines have been published. For example, compared to liberals, conservatives weight negative information more heavily (Shook and Fazio, 2009), remember negative stimuli more readily (Mills et al., 2016), attend to negative stimuli more intently (Castelli and Carraro, 2011), report greater disgust sensitivity (Inbar et al., 2009), are more likely to feel threatened (Bonanno and Jost, 2006), have greater grey matter density in their amygdalas, an area of the brain often associated with emotional responses to negative events (Kanai et al., 2011), and have greater amygdalic activation in response to risk (Schreiber et al., 2013).

Our three studies were intended to contribute to this growing literature on the correlation between negativity bias and political preferences by focusing on a rudimentary physiological rather than cognitive or neural response. Using standard measures of sympathetic nervous system response such as electrodermal activity (EDA; sometimes called skin conductance) we found that individuals with conservative beliefs on particular issues, especially those targeted at either sexual or “protective” behaviors, tend to have greater spikes in their EDA when presented with negative stimuli. Though our physiological results seem perfectly consistent with the large and growing literature on the deeper differences of liberals and conservatives, Knoll et al. were unable to reproduce them in their student sample.

Knoll et al.’s results

Aspects of Knoll et al.’s research design are similar to ours. The appropriate physiological sensors for measuring EDA were attached before 69 research participants were shown numerous images seriatim. Some of these images were threatening such as a criminal holding a gun; some were disgusting such as a toilet covered in vomit and fecal matter; some were positive such as a smiling baby; and some were neutral such as a plain piece of toast.

Given the widely acknowledged existence of a negativity bias, a basic validity check on data collection procedures in this research design is examining whether, as expected, negative images (threatening and disgusting images), elicit a significantly greater EDA response than the neutral or positive images. Knoll et al. do not report such a validity check. When we examine their data, we find no statistically significant difference in EDA response for non-threatening images and either threatening (t(63)=0.63, p=0.53) or disgusting images (t(62)=0.69, p=0.49). In short, their data suggest that on average people are not more responsive to negative than to non-negative stimuli. This finding potentially has broad-reaching implications because it challenges the longstanding empirical record that a negativity bias is a universal human trait. For reasons that are not entirely clear, Knoll et al. skip over this result in order to analyze the correlation of political preferences with the degree of negativity bias where, in contrast to our results, they do indeed fail to find a significant relationship between degree of negativity bias and political beliefs.

One explanation for their null finding may indeed be that the results we reported in earlier studies do not replicate, that is, the relationships we reported are artifacts of small samples, chance, failure to properly account for all causes of EDA fluctuations, some unknown contamination in our subject pool, or some other reason. Like all scientists we accept that identifying robust relationships requires replication and we must be willing to accept the null if results from our laboratory do not “travel.” An alternate explanation, however, is that their data are not reliable enough to address this important question of external validity. Given the absence of an overall negativity bias in their data—that is, their failure to find a relationship with high levels of external validity—it is obviously going to be difficult to demonstrate that individual-level variation in that (non-existent) negativity bias correlates with variation in anything else, including political attitudes. Instead of focusing on our three studies on the association of negativity bias and political preferences, Knoll et al. seem positioned to fry a much bigger fish; namely the existence of negativity bias in the first place. Such a bold claim, however, would require data and procedures that were up to the challenge, and on this point we fear that Knoll et al.’s results may come up short.

Knoll et al.’s procedures

Knoll et al. explicitly state that their study is an attempted “reproduction” of our research rather than an attempted “replication.” This caveat is important as their question wording, photographic stimuli, and sample composition differ significantly from our studies. As they put it, their methods are “an attempt to ‘reproduce’ the same findings in a novel context with slightly varying conditions to determine the extent of their generalizability” (Knoll et al., 2015: 3). They are absolutely correct that making slight deviations in research protocols is vital for assessing the convergent validity of established findings; however, we view their deviations as considerable in magnitude. We will limit our discussion to two major departures—physiological data collection procedures and stimulus selection—that raise substantive questions about comparing their results to those from our studies.

The data for Dodd et al. (2012), Oxley et al. (2008), and Smith et al. (2011) came from one of two data collection periods in the summers of 2007 and 2008. Each sample of roughly 50 non-student participants was drawn from a larger representative sample of 200 people, and only nine individuals participated in both the 2007 and 2008 data collection periods. Each research participant was isolated in a light and temperature controlled laboratory for stimulus presentation (on a computer screen right in front of them) and physiological recording. The idea was to exercise as much control as possible over the environment, thereby maximizing participant attention to the presented stimulus rather than environmental distractions.

Knoll et al.’s approach was significantly different. In their study, subjects did not view stimuli in isolation, nor did they view stimuli on a computer screen immediately in front of them. In their study, participants came in groups of 15–20, were seated at individual computer stations, and were hooked up to the physiological data acquisition equipment. Their stimulus presentation took place not on an individual computer but on a single overhead projection viewed by the entire group in the large room, and there was no indication that this room was light and temperature controlled (important because human physiology responds to ambient light and temperature). These procedures raise concerns about potential confounds—for example, participants may have been influenced by other people in the room or by variations in the distance from the screen rather than by a stimulus presented in isolation. The large number of potential distractions may explain why, in contrast to existing research, Knoll et al. failed to find a significant overall negativity bias in their respondents. In any event, collecting physiological data from large groups of people at the same time is not standard practice in psychophysiological research.

If we set aside concerns that these procedures may introduce noise and/or diminished effects into the data collected, the lack of negativity bias in Knoll et al.’s findings could constitute an interesting contribution to the broader literature on physiology and negativity bias. Previous examinations of negativity bias occur with participants who are isolated from other participants. Is it possible that when people are in groups a noticeable reduction in negativity bias occurs? This possibility is consistent with Knoll et al.’s findings and, if correct, would be an interesting and even valuable addition to our current understanding of the social nature of the human species. Nonetheless, given the deviations from standard procedure, it is impossible to tell if the inability to reproduce the findings from our laboratory is due to a failure of external validity in our studies or a failure of internal validity in Knoll et al.’s studies.

Stimulus choice is another concern. Smith et al. specifically chose three images that captured what is labeled as “core or contamination disgust” (Smith et al., 2011: 3). These images were of a man eating a handful of worms, an emaciated but alive body, and excrement floating in a toilet. These images were chosen because they captured a specific concept. In their reproduction of this study, Knoll et al. used images that likely do not tap the same target concept of core or contamination disgust. For example, images of a bloody wound and a bloodied face (two of the images employed by Knoll et al.) arguably are more likely to provoke a threat (or at least animal reminder disgust) rather than a core disgust response—indeed, they are highly similar to images Knoll et al. used to capture physiological responses to threat (r=0.98, p<0.001). Knoll et al. appear to be measuring different concepts than the Smith et al. study, which would certainly help to explain the lack of similarity in the results.

Conclusion

Moving forward, there are a number of factors that scholars conducting replication and reproduction studies should note. In particular, we encourage the use of non-student samples with participants isolated in light and temperature controlled rooms, who are exposed to stimuli in exactly the same way (e.g. stimulus presentation is always from the same distance). Furthermore, as a validity check we encourage a rigorous process of stimulus selection—that is, pretesting stimuli—to ensure selected stimuli are conceptually appropriate. Finally, we also encourage basic validity checks on physiological data collected during the course of these investigations. For example, it is fairly straightforward to check for something like negativity bias by checking mean differences between negative versus neutral or positive stimuli. If that test confirms negativity bias—a general human trait—it provides prima facie evidence that the data are accurately tapping into well understood physiological responses.

We are in complete agreement with Knoll et al.’s call for more replication and reproduction. Pushing this line of research forward is vitally important, as there is still much work to do in order to understand the degree to which these findings hold across multiple contexts. While confident in the internal validity of our studies, we are well aware that external validity is an issue that can only be addressed by replication, preferably in other laboratories. Our studies used two distinct adult samples with relatively small N’s from a politically diverse but geographically compact area. Do our findings travel beyond that? The only honest answer is that we do not know and cannot know until other laboratories have weighed in. The nature of data collection in the area of physiology sometimes precludes large-N studies for reasons as varied as resource limitations and the time intensive nature of the data collection process. This limitation opens the door for the vagaries of small-N studies to produce non-generalizable results. For that reason, we would like to echo Knoll et al.’s call for further research and thank them for taking the first steps in that direction. However, in this case, their failure to detect even a general negativity bias, probably due to notable departures from standard research practices for collecting physiological data, compromises their ability to speak directly to the external validity of our findings.