Can experience overcome stereotypes in times of terror threat?

Abstract

Research on evaluations of leaders has frequently found that female leaders receive lower ratings in times of national security crisis. However, less is known about countervailing factors. We contend that partisanship and leadership experience in relevant domains are two factors that can counteract the negative effects of terrorist threat on evaluations of female political leaders. To test this expectation, we implemented a national study in 2012 containing terrorist threat and non-threat conditions, and then asked participants to evaluate political leaders. The results show that Republican leaders, including women, are unaffected by terrorist threat; in contrast, Democratic leaders are punished during times of terrorist threat, but this negative effect is smaller for then-Secretary of State Hillary Clinton compared to Nancy Pelosi, who lacks similar experience. In short, Republican partisanship is a strong countervailing factor, while leadership experience in national security more modestly countervails.

Female leaders are often perceived as less desirable when national security crisis looms large (Falk and Kenski, 2006). Yet, some scholarship suggests that certain factors could countervail against such general tendencies. One of these is partisanship: Republican women leaders may be immunized against the negative effects of terrorist threat on evaluations of female leaders (Holman et al., 2011, 2016). Another is relevant leadership experience (Swers, 2007). Recently, discussion of this characteristic has gained traction, as some have suggested that 2016 presidential candidate Hillary Clinton’s experience as Secretary of State might have offset negative turns in the public’s evaluation of her when terrorist threat increased in salience. Does such leadership experience affect how the public evaluates female leaders when considering a context of terrorist threat? How does a countervailing effect for leadership, if any, compare to that provided by Republican partisanship?

We provide an answer to these questions with data from an experiment embedded in a 2012 national online survey. The study randomly assigned individuals to conditions that made international terrorist threat salient or not. Following the treatments (or not in the case of the control group), individuals evaluated Hillary Clinton, Nancy Pelosi, Condoleezza Rice, Sarah Palin, Barack Obama, and Mitt Romney. The leaders were selected to vary by gender, partisanship, and national security experience. The study’s design and timing allow new perspective on how partisanship, gender, and experience combine to affect political evaluations.

Results reveal that those exposed to terrorist threat give lower ratings to each of the three Democratic leaders – Clinton, Pelosi, and Obama – compared to those in the control group or a condition that provides positive news. Conversely, none of the Republican candidates – Rice, Palin, and Romney – suffers lower ratings in the terror threat condition. Thus, we find clear evidence affirming the strong role played by partisanship: Democratic leaders suffer a modest decline in evaluations in times of terrorist threat, while Republican leaders do not. Yet, at the same time, we find that Pelosi suffers a more consistent and greater decrease in evaluations when individuals are exposed to terror threat, compared to Clinton and Obama. This supports the notion that candidate qualifications can counteract at least some of the negative effects experienced by Democratic leaders, in particular female leaders, in times of terrorist threat.

The findings contribute to scholarship on gender and politics, by showing that leadership experience in national security is a factor that carries countervailing weight, though its effect is more moderate than partisanship. Our study is particularly timely, as 2016 was a year in which Hillary Clinton ran as the Democratic presidential nominee and terror threat was a salient issue. While some debated in the popular press over whether Clinton had immunity against the negative effects of terrorist threat on evaluations of female (Democratic) leaders (e.g. Albertson and Gadarian, 2016; Hillin, 2016), our results provide evidence that this is partially the case.

Gender, partisanship, and experience

Gender-oriented socialization tends to lead the public to consider political leadership as associated with men exhibiting masculine characteristics (Diekman et al., 2004) and to be biased toward male leadership (Mo, 2015), especially at higher levels of office (Bos, 2011; Eagly, 2007). This tendency is elevated in times of national security threat (Falk and Kenski, 2006) as individuals look to strong leaders (Merolla and Zechmeister, 2009). Absent other information, many assume that male politicians are more capable of handling issues related to security, such as war, military affairs, foreign policy, terrorism, and crime (e.g. Huddy and Terkildsen, 1993; Holman et al., 2016; but see Schneider and Bos, 2014).

Gender is not the only factor that matters in shaping voter preferences in times of threat; in particular, the partisanship and experiences of a candidate may influence evaluations. In terms of partisanship, Republicans are viewed as stronger on national security and foreign affairs, while Democrats are seen as better at handling welfare and domestic policy issues (Petrocik, 1996; Winter, 2010). As a result, the public may prefer Republicans when terrorism is salient (Merolla and Zechmeister, 2009, 2013). Individual characteristics also matter; political experience may be particularly important in shaping candidate evaluations. For example, the public values incumbency during times of national security threat (Berinsky, 2009) and military or foreign policy experience may shape how voters evaluate leaders (Swers, 2007). The possession of a countervailing trait may partially or entirely counteract a tendency to prefer male leadership when security issues are salient.

Partisanship, in some situations, can trump gender cues in candidate evaluations (Hayes, 2011). Holman et al. (2011, 2016) argue that because of the tendency for the public to assign greater competency to the Republican party when it comes to national security issues, Republican affiliation has the potential to mitigate against the tendency for the public to devalue certain female leaders in times of security threat. In an experimental study (2011) in which terrorism is primed, they find a modest positive effect of a terror threat on evaluations of Condoleezza Rice, but a negative effect on evaluations of Hillary Clinton (prior to her experience as Secretary of State). In later work, through the use of fictitious candidates, they more robustly show that Republican female leaders do not suffer when terrorism is salient, while Democratic female leaders do (Holman et al. 2016).

Candidate characteristics such as experience may also counteract gender stereotypes. Considering normal times, Mo (2015) finds that information on candidate qualifications can decrease tendencies to devalue female leadership. ¹ Holman et al. (2011) argue that the positive effect of priming terrorism on evaluations of Rice may have been due not only to her partisanship, but also to her experience in national security. In fact, Swers (2007) finds that a record of military experience can mitigate negative national security-related gender stereotypes about female Senators. And Bauer (2016) shows that female candidates can emphasize masculine competencies to overcome stereotypes. Theoretically, experience in national security could immunize female political leaders against the public’s tendency to privilege male leadership in times of national security threat.

When it comes to contexts characterized by terrorist threat, can leadership experience in national security counteract the tendency among the public to devalue female politicians? If so, how does this countervailing effect compare to that of politician (Republican) partisanship? We provide answers via a study designed to assess how the intersection of gender, partisanship, and foreign policy experience affects evaluations of political leaders when terrorism is salient.

Experimental design and method

The online study was fielded from August 15 to August 28, 2012 to a US sample drawn to approximate the national adult population in regard to gender, age, region, race and ethnicity, and income. The study was conducted by IPSOS International using their proprietary opt-in panel and a Qualtrics platform. Respondents were invited to participate in the study according to quotas set to match the national population and were balanced across conditions (see Tables I and II in the appendix).

Upon responding to an invitation to participate in the study, participants were asked to opt into the research project. ² Pre-treatment, all respondents answered questions about demographic and socioeconomic characteristics and political predispositions and were then randomly assigned into a control or one of the treatment groups. The survey experiment randomly assigned participants to either a control condition or a treatment group that read a news story. Three of the treatments were terrorist threat conditions, which were identical except that two of these news stories ended with a (slightly different) reminder of core democratic values, and there was a good times treatment. The treatments were short (~400–500 words) news stories with a similar structure and text (see appendix for full text of treatments). The material for the stories was drawn from actual reports, but edited into a mock news story; this approach has been validated in previous research (Albertson and Gadarian, 2015; Merolla and Zechmeister, 2009).

The Good Times treatment began with a statement that the country is “headed toward a time of increased well-being.” It referred to positive trends in areas such as education, the environment, and health in the country and the world. The first paragraph ended with a note that a “majority” of citizens report “moderate to high levels of life satisfaction.” The next four paragraphs focused on positive information about education, the environment, advances in science, and general health and welfare.

Each of the three terrorism treatments included a news story focused on the threat of international terrorism, and varied only with respect to the last paragraph. The different versions were included in order to test an expectation about the influence of reminders of democratic values on individuals’ policy preferences and related attitudes under terrorist threat. ³ We collapse the conditions in our primary analysis, but then also present results separately for each terrorism condition.

The first paragraph referenced warnings that the country is “on the brink of experiencing a major terrorist attack,” and placed this in the context of increased vulnerability to terrorism around the world and noted that “the majority of Americans are somewhat or very worried about the possibility of a violent terrorist attack.” The next paragraph discussed the increased danger posed by terrorism and referenced the 2008 Mumbai, India attack by Al Qaeda. The third paragraph referenced Al Qaeda’s intentions to “carry out a lethal series of bombings across multiple countries.” The treatment continued by referencing the risk of biological and chemical weapons, and a statement by a public official about the lethal intentions of terrorists.

After the treatment, all subjects were presented with a set of questions about their political attitudes and preferences. As a manipulation check, we assessed the extent to which the terror threat conditions increased negative emotions and decreased positive emotions relative to the control group and good times conditions, and find that the treatments had these expected effects (see Table III in the appendix).

Key variables and expectations

Does Hillary Clinton’s national security experience protect her from the negative effects of priming terrorism found in scholarship conducted prior to her tenure as Secretary of State? ⁴ To assess the relevance of partisanship, gender, and experience, we examine how respondents rate Clinton, Nancy Pelosi, Barack Obama, Condoleezza Rice, Sarah Palin, and Mitt Romney on a standard feeling thermometer. All of these leaders were politically salient at the time of the study, except Condoleezza Rice, whom we include to mirror Holman et al. (2011). Table 1 highlights our expectations based on the characteristics of each leader. We believe that these leaders were similarly well known at the time of this research. A very small portion of respondents on national surveys responded that they “don’t know” or “never heard of” the leaders in our experiment. In addition (see Appendix Table IV), Pelosi was better known than other congressional leaders.

Table 1.

Leader characteristics and expected effects of priming terrorism on candidate evaluations.

Leader	Gender	Party	Foreign Policy Experience
Hillary Clinton	Female (−)*	Democrat (−)	Secretary of State (+)
Nancy Pelosi	Female (−)	Democrat (−)	Little relevant (−)
Barack Obama	Male (+)	Democrat (−)	Commander in Chief (+)
Condoleezza Rice	Female (−)	Republican (+)	Secretary of State (+)
Sarah Palin	Female (−)	Republican (+)	Little relevant (−)
Mitt Romney	Male (+)	Republican (+)	Little relevant (−)

Note: *Expected effect in parentheses refers to the expected effect of the terrorism threat conditions (as compared to the control condition and good times condition) on the feeling thermometer.

We use a feeling thermometer because it provides a summary judgment of a political figure; it also is a scale that can be easily applied across a range of political leaders. Feeling thermometers provide more reliable and accurate measures than shorter scales (Alwin, 1997) and are widely used in political science and experimental research (Holman et al., 2011; Kam and Kinder, 2007). In an assessment of the validity of the feeling thermometers in this study, we find very similar levels when we compared the average feeling thermometer data in our control condition to other national studies (see Appendix Table V).

Leaders whose combination of characteristics have all negative (positive) signs should have lower (higher) evaluations in the terror conditions compared to the control and good times conditions. ⁵ In these cases we use one-tailed hypothesis tests (Pelosi). Existing literature does not indicate how much weight individuals give to each characteristic, so it is an open question as to whether, for example, foreign policy experience completely washes away any negative effect of terrorism on evaluations of Hillary Clinton or whether it only diminishes that negative effect. Because of this, in these cases, we use two-tailed hypothesis tests.

Key findings

The results demonstrate the dominance of political party and the importance of gender and experience in shaping how voters respond to terrorism threats and evaluate candidates. In Figure 1, we plot mean feelings toward each candidate for the control condition, good times condition, and the aggregated terrorism condition. Of all the leaders evaluated, Nancy Pelosi is the most negatively affected by exposure to the terrorism condition. Mean evaluations of Pelosi are 43.98 in the control condition, and are significantly lower, 39.55, in the combined terrorism condition (p=0.04). The same pattern holds if we compare evaluations to the good times condition (p=0.04) (see appendix Table VI). This is consistent with expectations, in that Nancy Pelosi does not have a single characteristic (gender, party, or foreign policy experience) that would advantage her when terrorism is made salient.

Figure 1.

Feeling thermometer evaluations of leaders in Control, Good Times, and combined Terrorism conditions.

Among the Democratic political leaders with at least one characteristic that may counter the weakness of the party on terrorism, we find that Barack Obama also has lower mean evaluations in the combined terrorism condition, 54.12, compared to the control, 57.8, and the good times condition, 57, although the effect is smaller and is not significant; though if a one-tailed test is used the effect would be significant relative to the control group (p=0.09). Hillary Clinton also suffers when terrorism is primed, but not as much as Pelosi, with mean feelings of 58.4 in the control condition, and 59 in the good times condition, compared to 55.46 in the combined terrorism condition. While these differences are not significant using a two-tailed test, using a one-tailed test the difference between the terrorism and good times condition is significant at p=0.07. These findings also held when we ran the analysis using seemingly unrelated estimation, and the negative coefficient on the terrorism treatment in the Pelosi model was statistically larger than either Clinton’s or Obama’s (see Appendix Table VIII). ⁶

We disaggregate the terrorism conditions in Table 2 and find that two out of the three (Terror No Reminder, Terror Reminder Partisan) have a significant, negative effect on feelings toward Pelosi, compared to the control group and good times conditions, while the other terrorism condition (Terror Reminder Center) is just outside of conventional significance for both comparisons (p=0.12). For Obama and Clinton, only one of the conditions (the Terror No Reminder condition for Obama and the Terror Reminder Center condition for Clinton) has a significant, negative effect relative to the control group (as well as the good times condition for Clinton). The more mixed pattern of results across conditions for Obama and Clinton provides evidence that characteristics such as experience in foreign policy can partially cushion against the negative effects of priming terrorism on Democratic leaders. That we find similar effects for Clinton and Obama may indicate that leadership experience is more relevant than gender in combating party weakness on the issue.

Table 2.

Feeling thermometer ratings across Control, Good Times, and Terror conditions for Democratic leaders.

Condition	Clinton		Pelosi		Obama
	Therm	p-value	Therm	p-value	Therm	p-value
Control	58.4		43.98		57.8
Good Times	58.98	0.83	44.09	0.51	56.95	0.79
Terror (overall)	55.46	0.21	39.55	0.04	54.12	0.18
Terror – No Reminder	56.15	0.42	39.34	0.07	51.9	0.07
Terror – Reminder Center	53.29	0.07	40.35	0.12	54.25	0.28
Terror – Reminder Partisan	57	0.63	38.91	0.06	56.46	0.68

Note: Therm is the average thermometer rating for each leader in each condition. P-value is from a difference in means test between the control and each terror treatment. One-tailed test is reported for Pelosi and two-tailed for Clinton and Obama. The t-test results between Good Times and each treatment are available in the appendix.

Turning to the Republican leaders, we find that the estimated difference in feelings between the control group and aggregated terrorism condition is positive. However, none of the differences is statistically significant. That we find similar effects for all three Republican leaders suggests that the party reputation is strong enough to trump characteristics that might disadvantage these particular leaders (such as gender and lack of foreign policy experience). Holman et al. (2011) found a positive significant difference between a good times baseline condition and a terrorism condition for Condoleezza Rice in a past study, and the weaker effects here could be due to the diminished salience of Rice’s experience in foreign policy over time. Merolla and Zechmeister (2009, 2013) find that Republican incumbents benefit when terrorist threat is primed, and none of the Republican leaders in this sample is an incumbent. If we disaggregate the terrorism conditions (see Table 3), none of the individual conditions help – or harm – any of the Republicans. In the appendix, we also provide an evaluation of whether individuals react differently to the treatment depending on their gender and partisanship (see Tables X to XIII), and find some evidence that the negative effects of the terrorism condition for Pelosi are primarily due to male respondents.

Table 3.

Feeling thermometer ratings across Control, Good Times, and Terror conditions for Republican leaders.

Condition	Rice		Palin		Romney
	Therm	p-value	Therm	p-value	Therm	p-value
Control	56.52		40.86		44.70
Good Times	56.95	0.87	41.63	0.82	44.81	0.97
Terror (overall)	58.43	0.38	42.54	0.52	45.33	0.80
Terror – No Reminder	59.58	0.24	43.90	0.33	46.80	0.49
Terror – Reminder Core	57.17	0.81	41.71	0.79	44.98	0.93
Terror – Reminder Partisan	58.47	0.48	41.86	0.77	44.06	0.84

Note: Therm is the average thermometer rating for each leader in each condition. P-value is from a difference in means test between the control and each terror treatment. Two-tailed tests reported. The t-test results between Good Times and each treatment are available in the appendix.

Conclusion

Throughout the 2016 US presidential primary and general election campaigns, Donald Trump often attacked Hillary Clinton as “weak and ineffective.” Clinton often fired back, noting Trump’s lack of experience and stating he is “temperamentally unfit” to be the commander in chief. At the same time, she positioned herself as the candidate with foreign policy experience in the election, repeatedly touting her credentials and familiarity with addressing issues associated with terrorism.

Our results suggest that Clinton’s experience may have shielded her from some of the negative effects of gender and party stereotypes that harm Democratic female leaders. At the same time, the salience of terrorism was not necessarily to her advantage, as she does not get a boost in evaluations when the issue is primed; rather, in our study, evaluations of Clinton still suffered some decline in some cases. Our study did not actively highlight candidates, or the cases they make on the stump. It could be that actively highlighting experience in the face of terrorist threat more fully immunized Clinton in the 2016 campaign. Furthermore, an open question is whether, and to what extent, evaluations of Donald Trump increase when terrorism is salient. The advantage found for Republican males in past work did not extend in this study to Mitt Romney, a candidate with little foreign policy experience. Taken together, the results in this study suggest that the dynamics of context and candidate traits in a campaign like the US 2016 presidential race – for example, the extent to which Clinton successfully primed her experience and Trump could hang his hat on the Republican label and build a tough image while attempting to undercut Clinton with accusations of weakness – are of consequence for impressions of leaders and, by extension, electoral outcomes, especially to the degree that terrorism is a salient issue. More generally, the results provide support for the notion that (Republican) partisanship can play a strong countervailing role in deflecting lower evaluations for female leaders in times of terror threat, while they also provide evidence that candidate experience matters to some, but not to an overwhelming, degree.