Abstract
The human rights movement has spent considerable energy developing and promoting the adoption of both international and domestic legal prohibitions against torture. Empirical scholarship testing the effectiveness of these prohibitions using observational data, however, has produced mixed results. In this paper, we explore one possible mechanism through which these prohibitions may be effective: dampening public support for torture. Specifically, we conducted a survey experiment to explore the impact of international and constitutional law on public support for torture. We found that a bare majority of respondents in our control group support the use of torture, and that presenting respondents with arguments that this practice violates international law or constitutional law did not produce a statistically significant decrease in support. These findings are consistent with prior research suggesting, even in democracies, that legal prohibitions on torture have been ineffective.
Introduction
One of the primary goals of the modern human rights movement has been stopping governments from torturing their citizens. In order to achieve this goal, considerable effort has been spent developing and promoting the adoption of both international and domestic legal prohibitions of torture. The efforts to promote both international and domestic torture bans have largely been successful. In fact, by 2011, 156 countries had become party to the Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment (CAT), and 157 countries had included torture prohibitions in their constitutions. 1
The empirical scholarship testing the effectiveness of these international and domestic torture prohibitions, however, has produced mixed results. For example, studies testing the impact of the CAT have alternatively found that ratification of the treaty is associated with increased torture rates (Hafner-Burton and Tsutsui, 2005; Hathaway, 2004; Hill, 2010; Hollyer and Rosendorff, 2011; Neumayer, 2005; Vreeland, 2008), has had no effect on torture rates (Lupu, 2013; Powell and Staton, 2009), and even that it is associated with decreases in torture rates (Fariss, 2014, 2015). Similarly, studies testing the impact of constitutional torture prohibitions have both found that they have had no effect on torture rates (Chilton and Versteeg, 2015; Keith et al., 2009; Melton, 2013) and that they are associated with decreases in state repression (Hill and Jones, 2014).
Part of the reason that this line of scholarship has produced conflicting results is that there are a number of substantial barriers to inference that make it difficult to assess the effectiveness of these measures with observational data alone. One of these barriers is that countries’ decisions to adopt international (Lupu, 2013) and domestic (Chilton and Versteeg, 2015) torture prohibitions are endogenous to their rights practices. Moreover, recent research has suggested that a great deal of the data has been used to measure rates of torture is biased because reporting standards have changed over time (Fariss, 2014). Finally, even if it were possible to overcome these barriers to inference, it would still be difficult to disaggregate the impact of international and domestic torture bans because countries frequently adopt both forms of legal prohibitions (Chilton and Tingley, 2013).
Given these hurdles that observational research designs face, one increasingly common research strategy has been to use experimental methods to test the mechanisms that have been theorized as ways that legal restrictions may alter government behavior. More specifically, experiments have been used to test whether explicit references to international legal agreements results in changes in public support for policies that would violate these agreements (e.g. Chilton, 2014, 2015; McEntire et al., 2015; Putnam and Shapiro, 2009; Tomz, 2008; Wallace, 2013, 2014). For torture, it has been suggested that democracy is one of the mechanisms through which torture prohibitions might matter (Simmons, 2009). Specifically, it is possible that when the public is told that torture violates the country’s legal commitments, this increases public disapproval of these practices. Although experimental methods have their own limitations, they do make it possible to design studies with high degrees of internal validity, and, as a result, provide a valuable supplement to the evidence that can be produced by observational studies.
In order to comparatively test the effectiveness of both international and domestic prohibitions on torture, we have conducted what we believe to be the first survey experiment that simultaneously tests the effect of exposing respondents to arguments about the status of international law and constitutional law. More specifically, we fielded a survey experiment to a nationally representative sample of adults in the United States that asked respondents whether they supported the use of torture while randomly exposing some respondents to arguments that torture violates international law or constitutional law (or both). We also built on the limited prior research that has explored why arguments about international law change public opinion (Chilton, 2014), and asked the respondents a series of six questions designed to test the mechanisms that may have lead our treatments to change respondents’ views.
Our results are consistent with recent polling that suggests that a bare majority of Americans support the use of torture. What is more, we found that presenting respondents with the argument that torture violates international law did not produce a statistically significant decrease in support. Presenting respondents with the argument that torture violates the constitution, however, lowered support for torture by 4.6 percentage points (although the effect falls just short of conventional levels of statistical significance). That said, the difference between our international law and constitutional law treatments was not itself statistically significant. We do find a more substantial effect for the sub-group of our respondents that identified as Democrats. Yet, overall, our findings suggest that information on the status of neither international law nor constitutional law has a substantial effect on public support for the use of torture.
Research design
Subject recruitment
We administered our experiment to a nationally representative sample of 2,159 US adults in October 2014. The experiment was administered online, and Survey Sampling International (SSI) recruited the respondents. SSI is a research firm based in the United States that conducts surveys for corporate and academic research. Our sample was specifically designed to be nationally representative of the US adult population based on gender, age, race, and census region. 2
Experimental design
We designed our experiment based on a survey used by Wallace (2013) to explore the effect of commitments to international law on public support for the use of torture. Wallace’s survey used a vignette to explore whether respondents expressed lower support for using torture to interrogate captured enemy combatants when they were randomly assigned to a group that was told that torture would violate international law and was prohibited by treaties the United States had signed. The strengths of Wallace’s vignette are that it was designed to: (1) present a realistic scenario; (2) not bias support of torture in any direction; (3) avoid inflammatory language; and (4) present a neutral stance of the effectiveness of torture (Wallace, 2013: 117–118).
Although our survey used Wallace’s research as a starting point, we made a number of substantial changes to his experiment. 3 Since Wallace’s goal was to assess the effects of international law on public opinion, his vignette focused on the use of torture to interrogate enemy combatants. For his purposes, the interrogation of enemy combatants provided a realistic scenario where respondents could be told that the use of torture was prohibited by international law. Since our goal is to compare the relative effectiveness of international law and constitutional law, we needed a scenario that was clearly prohibited by both international law and the American constitution. The torture of enemy combatants, however, is not clearly prohibited by the constitution. 4 As a result, our vignette did not stipulate that the potential victims of torture where combatants from an opposing side. Instead, our vignette focused on a scenario where torture would plausibly be prohibited by both international and domestic law. 5
More specifically, we started our vignette by presenting all respondents with the following scenario: “Throughout history, people have plotted to overthrow or sabotage the government, and have resorted to the use of violent means to do so. Occasionally, the military captures people that are conspiring to overthrow or sabotage the government through violent means.” The vignette then proceeded to say that: “[t]hese individuals may have information of interest about the conspiracy, such as the location of other conspirators or plans for future attacks. Some government officials believe interrogating these people through a variety of methods is a useful way to obtain this information.”
We then introduced the possibility of torture by saying that: “[t]he interrogation methods could involve torture, meaning they would cause severe pain or suffering to the people they are used on.” Moreover, we added that “[t]he information may, or may not, be accurate or relevant to the conspiracy.” After this, the respondents were randomly presented with one of four treatment conditions. The four treatments were as follows:
The respondents in the Control group were not presented with an argument about the legal status of the use of torture. The inclusion of this control group gives us a baseline to compare the subsequent treatment groups against. 7 The respondents in the Int’l Law group were presented with the argument that the use of these methods violates international law. The wording of this treatment condition is identical to the international law treatment used by Wallace (2013). The respondents in the Con. Law group were presented with the argument that the use of these methods violates the constitution. This treatment condition was designed to be as similar to the Int’l Law group as possible (for example, the treatments include the same number of words, tone, and sentence structure). Finally, respondents in the Combined group were presented with both arguments. Offering some respondents both arguments allows us to evaluate whether there is an “additive” or “substitution” effect to being given both arguments (Chilton, 2015; Tomz, 2008).
After being presented with the vignette and one of the four randomly assigned treatments, respondents were asked: “[t]o what extent do you agree or disagree with the following statement: The United States should use interrogation methods involving torture on people that have plotted to overthrow or sabotage the government.” The respondents were then asked to provide an answer on a six-point scale that included the following options: (1) “strongly disagree”; (2) “disagree”; (3) “lean towards disagree”; (4) “lean towards agree”; (5) “agree”; and (6) “strongly agree.” Following Wallace (2013), we converted these responses into a binary variable for easier interpretation. Our results are substantively similar, however, when using the full six-point response variable as the dependent variable. 8
Balance
In addition to presenting respondents with the vignette described above, we also asked respondents a series of demographic questions. 9 Doing so not only allows us to confirm that our respondents are nationally representative based on their gender, age, race, and census region, but also allows us to ensure that the demographic characteristics of the respondents in each of our treatment groups were balanced. To do so, we followed Chaudoin (2014) and used the balance test from Hansen and Bowers (2008). Using this test, the overall χ2 statistic and the associated overall p-values for each group are: Control, 11.10 (p = 0.35); Int’l Law, 3.47 (p = 0.97); Con. Law, 9.02 (p = 0.53); and Combined, 4.95 (p = 0.90). Thus, we did not find any evidence of demographic imbalance across our four treatment groups.
Results
Primary results
Figure 1 presents the primary results of our experiment. 10 Since we did not find any evidence of imbalance across our randomly assigned treatments, Figure 1 simply reports the mean responses—and 90% confidence intervals (CIs)—for each of the four treatment groups. 11 All of our results, however, are substantively the same when estimated using regressions that control for the demographic characteristics of the respondents. 12
Primary results (mean response and 90% CIs).
As Figure 1 shows, respondents in the Control group were on balance more likely to agree than disagree with the use of torture. For the control group, the mean level of support for using torture against people that have plotted to overthrow the government is 51.1% (90% CI: 47.4%, 54.7%). This number closely tracks the recent public polling on the use of torture: a 2011 Pew survey found that 53% of Americans said that torture could often or sometimes be justified, 13 and a 2013 Associated Press-NORC Center for Public Affairs Research survey found that 50% of Americans said that torture could often or sometimes be justified. 14
Respondents in the Int’l Law group had a mean response of 49.2% (90% CI: 45.6%, 52.7%). Although the respondents given the international law treatment did express lower support for the use of torture than the control group, the difference between the two groups is not statistically significant (p = 0.54). The respondents in the Con. Law group had a mean response of 46.5% (90% CI: 43.0%, 50.0%). Although the difference between the Control group and the Con. Law group is not statistically significant at the 0.1 level using the binary response variable (p = 0.14), it is statistically significant when using the full six-point response variable, albeit only at the 0.1 level (p = 0.08). That said, the difference in support between the Int’l Law group and the Con. Law group is not statistically different (p = 0.38).
Figure 1 also reveals that the respondents in the Combined group had a mean response of 50.2% (90% CI: 46.7%, 53.7%). Interestingly, this is only slightly lower than the mean response for the Control group (p = 0.77). Moreover, the Combined group—whose members were presented with both the international law and constitutional law treatments—were more supportive of torture than the respondents that were given either just the international law treatment or the constitutional law treatment. Since the differences between the Combined group and the other treatments are substantially small and far from statistically significant, 15 we believe that it would be inappropriate to put much weight on them.
It is still interesting, however, that the respondents that received the Combined treatment were more likely to support torture than the respondents receiving either treatment individually. Given the evidence from Tomz (2008) and Chilton (2015) that arguments about the status of international law had an additive effect when combined with non-legal arguments, this is a surprising result that likely cannot simply be explained by the fact that the respondents that were presented with the combined treatment were less likely to read their slightly longer vignette. One possible explanation for our results is that these previous studies did not test combining two legal arguments, and instead combined arguments about international law with economic or moral arguments. It is possible that informing respondents that government officials are advocating for using interrogation techniques including torture despite the fact that it would violate international law and the constitution sends a signal that these officials have particularly good reasons for wanting to use torture in that instance. More research will have to be conducted, however, to verify the higher support for rights violations when using combined legal treatments and to test the theory we have suggested.
Taken together, these results do not reveal any statistically significant differences between our four treatment groups. Given these null results, it is reasonable to calculate how large a difference between the control group and a treatment group would have been needed to find a statistically significant result. With our samples sizes and standard deviations, it would have taken an effect size of roughly 5.4% to find a difference that was statistically significant at the 0.1 level and an effect size of roughly 6.0% to find a difference that was statistically significant at the 0.05 level. To put this in perspective, the level of support for torture was 4.6% lower for our Con. Law group compared to the Control group (which, as previously noted, fell just short of statistical significance at the 0.1 level).
Results by party identification
We also examined whether the political affiliations of the respondents influenced their responses. 16 The results broken out by respondents that identified themselves as either Democrats or Republicans are presented in Figure 2. 17
Results by party identification (mean response and 90% CIs).
There are three things worth nothing about our results based on party identification. First, Democrats in all four treatment groups were consistently less likely than Republicans to support the use of torture: the mean response for Democrats was 47.8% (90% CI: 45.1%, 50.6%), whereas the mean response for Republicans was 61.7% (90% CI: 58.1%, 65.2%). This is consistent with both public polling showing that Republicans are more supportive of torture 18 and the finding in Wallace (2013) that conservatives were more likely to support torture than liberals.
Second, the international law and constitutional law treatments had an effect on support for torture among Democrats but not Republicans. For Democrats, both the Int’l Law group and Con. Law group had a lower mean response than the Control group that is statistically significant at the 0.05 level. The Control group for Democrats had a mean response of 55.4% (90% CI: 49.5%, 61.3%), while the Int’l Law group had a mean response of 43.9% (90% CI: 38.4%, 49.4%) and the Con. Law group had a mean response of 43.9% (90% CI: 38.6%, 49.1%). For Republicans, however, the results for all four treatment conditions are nearly identical. This result is consistent with the finding in Wallace (2013) that the international law treatment lowered support for torture among liberals, but had almost no effect on support for torture among conservatives.
Third, it is puzzling that our results by party identification are consistent with Wallace (2013), but that our overall results are not consistent with Wallace’s finding that presenting respondents with arguments that torture violates international law statistically significantly lowers support for torture. We explored three possibilities for this inconsistency. First, we considered whether this inconsistency is due to the composition of our samples. Wallace’s sample had 26% self-reported liberals, while our sample had 41% self-reported Democrats. We do not think that this cannot explain the inconsistency, because our larger proportion of Democrats should bias us towards finding statistically significant results in the full sample. Second, we considered whether this inconsistency emerges because our treatment groups are not balanced based on party identification. Re-conducting the balance test from Hansen and Bowers (2008) while including variables for Democrats, Republicans, and Independents, however, does not reveal any evidence that our treatment groups are not balanced. 19 Third, we considered whether this inconsistency arises because of differences between the responses of moderates in Wallace’s sample and the responses of independents in our sample. Wallace’s replication data reveals that moderates presented with his international law treatment were 7.5% less likely to support the use of torture than respondents in his control group. 20 In contrast, in our sample independents presented with the international law treatment were nearly 10% more likely to support the use of torture than the control group, and independents presented with the constitutional law treatment were 0.04% more likely to support the use of torture than the Control group. This suggests that the inconsistencies between our overall results and Wallace (2013) are due to the independents responding negatively or neutrally to our law treatments. Future research should explore whether this is due to changes in the views of independents over time, the differences in our vignettes, or idiosyncrasies in the composition of our samples.
Mechanism questions
We also attempted to test the mechanisms that may lead arguments about international law or constitutional law to change public support for torture. To our knowledge, the only previous study that has directly explored possible mechanisms for why arguments about international change public opinion was Chilton (2014). After conducting an experiment on how arguments about international law change support for solitary confinement, Chilton (2014) asked respondents questions designed to test reasons why the international law argument changed respondents’ views. The results suggested that the respondents that received the argument about international law were more likely to believe that it was important that the US treatment of prisoners conform to international standards.
After respondents read our vignette, we also asked” to read: we also asked a series of six additional questions to assess the mechanisms through which our international law and constitutional law treatments may influence public support for the use of torture. Those questions are reported in Table 1. We randomized the order respondents were asked these six questions, and for each we asked for responses on a five-point scale (with 5 representing strong agreement and 1 representing strong disagreement). The responses to these six questions are presented in Figure 3.
Mechanism questions.
Results to mechanism questions (mean response and 90% CIs).
Overall, the results in Figure 3 indicate that the treatments that respondents received while reading our vignette had essentially no influence on the subsequent responses to the six mechanism questions we asked. In fact, the differences in mean responses for the Control group and the Int’l Law group were not statistically significant at the 0.1 level for a single one of the six mechanism questions. This is perhaps unsurprising given that, as the previous results sections revealed, the responses for the Control group and the Int’l Law group for our primary experiment were not different in a statistically significant way.
There were, however, statistically significant differences between the Control group and Con. Law group for two of the six questions. The respondents who received the constitutional law treatment were less likely to think that torture produces Valuable information (p = 0.03) and less likely to think that the United States interrogation practices should conform to International standards (p = 0.01). One explanation for why respondents in the Con. Law group may believe that torture is less likely to produce valuable information than respondents in the control group is that respondents may believe that these interrogation techniques would not have been constitutionally prohibited if they produce important information. In other words, the constitutional law argument may prime respondents to think that torture is less useful. One explanation for why respondents in the Con. Law group are less likely to think that United States’ interrogation practices should conform to International Standards than respondents in the Control group is that the respondents may think that international standards are less relevant after they have been told about domestic, constitutional standards that restrict torture. For example, if the respondents in the Con. Law group thought that the constitution had stricter limits on torture than international law, they would be less likely to think that the United States’ interrogation practices should “always conform” to international standards.
Although these are small differences for only two of the six mechanisms questions, it does provide some additional—but admittedly very weak—evidence that information on constitutional law has a slightly larger effect on support for torture than information on international law. That said, like with our primary results, the difference in mean responses between the Int’l Law group and Con. Law group for both the “International standards” and “Valuable information” questions were substantively small and far from statistically significant.
Conclusion
Our experiment was designed to explore how information about international law and constitutional law impacts public support for the use of torture. While we found no evidence that information about international law changed public opinion, we found some weak evidence that information about constitutional law did. Specifically, using a binary response scale, we found that presenting respondents with the argument that constitutional law prohibits torture lowered public support for torture by 4.6 percentage points. The effect, however, falls outside conventional levels of statistical significance. That said, although the argument about constitutional law had a slightly larger effect than the argument about international law, there was not a statistically significantly difference between these two arguments. In general, our findings suggest that presenting respondents with the arguments that international law and constitutional law prohibits torture has little effect on public support for its use. 21
It is worth noting that there are several limitations to our research design that may limit the generalizability of our results. First, since respondents may already be aware that international law and constitutional law prohibit torture, our experiment cannot directly test the effect that international and constitutional torture prohibitions have on public opinion. Instead, our experiment can only test the effect that presenting respondents with arguments about the status of these sources of law has on public support for torture. As a result, despite the fact that our international law and constitutional law treatments had little effect, it may be the case that both international and domestic torture prohibitions do have an impact on public support for torture, but that we simply cannot measure it because our control group is also aware of the prohibitions. Second, we only tested one vignette on a sample of respondents from one country. It may be the case that arguments about the status of international and constitutional law would have a different effect in other scenarios, or that respondents from other countries would have responded differently than the sample we recruited in the United States.
Despite those limitations, our findings do help to explain existing findings from observational studies on the effectiveness of torture prohibitions. Our finding is consistent with the observational studies that suggest that CAT ratification (e.g. Lupu, 2013; Powell and Staton, 2009) and constitutional torture prohibitions (e.g. Chilton and Versteeg, 2015; Keith et al., 2009) do not reduce torture. Our research is also consistent with studies that suggest that even democratic governments are likely to engage in torture—especially in the face of terror threats or violent dissent (Conrad and Moore, 2010; Davenport et al., 2007; Goderis and Versteeg, 2012; Rejali, 2007; Wantchekon and Healy, 1999). While our findings do not directly prove that lack of popular support indeed causes legal torture prohibitions to fail, they suggest that this is at least a factor that contributes to their ineffectiveness.
Footnotes
Acknowledgements
We would like to thank Geoffrey Wallace, Greg Mitchell, and Aziz Huq for their helpful comments and suggestions. We thank Alec Knight for excellent research assistance.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Carnegie Corporation of New York Grant
The open access article processing charge (APC) for this article was waived due to a grant awarded to Research & Politics from Carnegie Corporation of New York under its ‘Bridging the Gap’ initiative.