Abstract
Peffley and Hurwitz’s article “Persuasion and resistance: Race and the death penalty in America” is an influential study demonstrating the effects of race on death penalty attitudes. White respondents were found to increase their approval for capital punishment when informed that it disproportionately affects African-Americans. We present results from two studies, including one conducted on a nationally representative sample, that fail to find support for this finding. Our first study, which was conducted on Amazon Mechanical Turk, consists of an exact replication as well as an additional manipulation that strengthens the treatment by adding information about a specific black (versus a white) defendant to the stimulus. However, we fail to elicit the backlash effect found in the original study using either manipulation despite having nearly three times the sample size. These findings are mirrored by replication data from a Time-sharing Experiments for the Social Sciences survey that closely replicates Peffley and Hurwitz’s race framing treatment. The results from these studies suggest that the relationship between racial stimuli and death penalty support has changed since the original study, that racial backlash effects in this policy domain are not as robust as previously assumed, or both.
Keywords
Peffley and Hurwitz’s article (2007) (hereafter PH) helps define how political science understands the role of race in public opinion toward the death penalty. Building on other studies showing the effect of racial attitudes on issue opinions (e.g. Bobo, 1997; Dawson, 1994; Kinder and Winter, 2001; Sanchez, 2006; Schuman et al., 1998), PH show that racial stimuli can have a substantial effect on whites’ support for capital punishment. Strikingly, stating that the death penalty is disproportionately applied to African-Americans induced a 12 percentage point increase in support for capital punishment among whites (Peffley and Hurwitz, 2007: 1002). This backlash effect has been widely cited in political science (Knoll et al., 2010; Nyhan and Reifler, 2010; Weber and Thornton, 2012; Wedeking, 2010) and has also influenced research in criminal justice (Pickett et al., 2012; Ramirez, 2013; Unnever and Cullen, 2009), law (Glaser et al., 2015; Haney Lopez, 2010; Unah, 2009), sociology (Savelsberg and King, 2011), and communication (Roh et al., 2015).
However, a very similar manipulation conducted in 2000 on a nationally representative sample found no effect on white support for the death penalty (Bobo and Johnson, 2004: 158–161), though one study did find greater punitiveness when the prison population was described as having a greater percentage of black inmates (Hetey and Eberhardt, 2014). Despite the significant influence of the original result, few subsequent studies have attempted to reproduce the effect in the years since its publication. 1 It is especially valuable to revisit this finding, which relies on data from 2001, given recent changes in the politics of race and crime. Death penalty support has ebbed somewhat in recent years after a marked decline in the 1990s and early 2000s (Gallup, 2017; Shirley and Gelman, 2014). Moreover, the Obama years were marked by seeming changes in the politics of race (Tesler, 2016; Tesler and Sears, 2010), including high-profile controversies over the role of race in the criminal justice system after the deaths of figures such as Michael Brown and Eric Garner. While Peffley et al. (forthcoming) find evidence of a backlash effect in a study conducted during Obama’s tenure in office, the effect is limited to racially conservative white respondents of a single state (Washington). This limited evidence of white backlash contrasts with the more general finding in PH, which finds that the racial frame unconditionally increased support for the death penalty among a nationally representative sample of whites. Given these differing findings and persistent survey evidence suggesting that race continues to play a prominent role in public attitudes toward criminal justice (Hutchings, 2015), understanding the effects of race on death penalty opinion may be more relevant today than ever.
Further, there are important methodological reasons to replicate the PH finding. As other social sciences have found, replicating surprising findings can increase their scientific validity. The continued study of any phenomenon by multiple research teams is necessary for firmly establishing important empirical results (Open Science Collaboration, 2015). The value of replication may be particularly important in psychological studies given recent concerns about reproducibility (Klein et al., 2014). For these reasons, revisiting PH is scientifically valuable as well as substantively interesting.
In this note, we report the results of a replication and extension of PH to assess how racial framing affects whites’ attitudes on the death penalty in contemporary America. Specifically, we conduct an exact replication of the written racial frame introduced in PH and add an experimental manipulation of a prototypical defendant’s race that was intended to strengthen the effect of the racial frame. However, despite our larger sample size of white respondents and the added treatment condition, we do not observe evidence of a white backlash effect. We also do not observe a backlash effect in a replication of PH conducted on a nationally representative sample. The consistency of these null effects may result from changes since 2001 in the effects of racial stimuli on white attitudes about the death penalty or their willingness to express those attitudes in a survey context. Across two separate studies and six separate experimental manipulations, none of our analyses revealed statistically significant framing effects on death penalty support. Moreover, none of our subgroup analyses revealed statistically significant heterogeneous treatment effects among specific demographic groups, nor among those with measured psychological traits.
It is also possible that the white backlash effect is moderated by other factors that distinguish our study from the original, including survey mode and sampling method. Both studies we examine were conducted as online survey experiments, whereas the original PH data were produced via a random-digit telephone survey. Perhaps the presence of an interviewer on the other end of a telephone line conditioned responses in ways that online surveys do not. On the other hand, both our convenience sample of online respondents and a nationally representative sample failed to provide support for the original findings contained in PH, suggesting that the sampling method is not the most likely explanation. In any case, given the large size of the effect in the original study and our failure to find evidence of such an effect among two separate samples, our results suggest that there are important scope conditions that limit the white backlash phenomenon, suggesting the need for further studies to better understand its generalizability.
The original Peffley and Hurwitz (2007) study
The original PH study collected survey responses from the 2000–2001 National Race and Crime Survey, a national random-digit telephone survey of approximately 600 white and 600 black respondents. Embedded within the survey was an experiment that varied the frame in which the death penalty question was posed. The baseline experimental condition provided no frame to respondents and merely asked, “Do you favor or oppose the death penalty for persons convicted of murder?” This question was evaluated on a four-point scale that assessed whether the respondent strongly (or somewhat) favored (or opposed) the death penalty. The racial frame condition presented respondents with a statement claiming, “Some people say that the death penalty is unfair because most of the people who are executed are African-Americans” before asking about the respondent’s support for the death penalty. The experiment also included an innocence frame in which respondents heard the statement “Some people say that the death penalty is unfair because too many innocent people are being executed” prior to being asked about their support for capital punishment.
PH present their key finding as a table of differences in support for the death penalty across experimental groups and races (reproduced here as Table 1). In addition to finding statistically significant racial differences in how respondents reacted to each frame, the study finds statistically significant framing effects for both treatments among black respondents and for the racial treatment among white respondents. Among black respondents, the racial frame reduced support for the death penalty by 12 percentage points and the innocent frame reduced support by 16 percentage points relative to the baseline condition. However, white respondents given the racial frame instead reported 12 percentage points higher approval for the practice than those in the baseline condition—a backlash effect. 2
Death penalty support by race and treatment group (Peffley and Hurwitz, 2007).
The experiment also randomly manipulated the source of the argument as either “some people” or “FBI statistics show that,” which had no discernible influence on support for the death penalty.
Difference between baseline and argument condition is statistically significant (≤.05).
Difference in treatment effect by race of respondent is statistically significant (≤.05)
Note: Statistical significance was computed by estimating an ordered probit equation for the pooled data that regressed support for the death penalty on the frame (baseline, innocence, or racial), a dummy for race of respondent, and race × argument interactions.
Study 1: Replication and extension
Experimental design
We replicate the PH design by employing identical phrasing for the race frame manipulation and the outcome measure (we omit the innocence frame due to its lack of an effect on white respondents). We also extend the original design by including an additional experimental manipulation in order to elicit what we anticipated would be a more pronounced backlash effect. This treatment occurs prior to the administration of PH’s textual frame and consists of introducing a specific defendant accused of murder. Respondents are provided with the mugshot of either a white or black male and accompanying text identifying the individual as facing capital murder charges for shooting a police officer (e.g., “Marvin Guy [or Henry Magee, for the white male] faced capital murder charges for shooting a police officer during a SWAT raid”). 3 The control group sees neither the photograph nor the accompanying text. The specific stimuli used in each condition are provided in Table 2 below. The defendant manipulation presumably reinforces the racial element of capital punishment in two respects. First, it leverages the online survey medium by using photographs, a feature absent from the 2001 telephone survey in the original PH study. The photographs allow the respondents to focus their attention on an individual who can act as a prototypical example of others who face the death penalty. Second, the text accompanying each photograph informs the respondent of the defendant’s crime. Prior research suggests that survey respondents respond in systematically different ways to questions about capital punishment given the context of the crime and details about the offenders (Burgason and Pazzani, 2014).
Experimental conditions.
Following the experimental manipulation, each respondent is asked about their support for the death penalty on a four-point Likert scale ranging from strongly oppose to strongly favor. This measure, which serves as our dependent variable, is identical to the one in PH. (See the Supplementary Appendix for the full text of the instrument and full-size treatment photographs.)
Sample characteristics
Study 1 was conducted online in February 2016 among 2134 respondents in the United States of America recruited from Amazon Mechanical Turk (AMT). 4 Among these respondents, 1653 identified as white, 106 identified as black, and 375 identified as some other race. The subsequent analysis relies solely on the white respondents. 5 In general, our sample of whites skews younger, more educated, and more liberal than a nationally representative sample. Approximately half (47%) identify as female. With respect to age, 37% of respondents are 18–29 years old, 58% are 30–59, and 5% are aged 60+. About 35% of respondents have not completed a college degree; 49% attained some form of undergraduate academic degree; the remaining 15% possess a master’s degree or higher. Ideologically, 29% identify as conservative, 51% as liberal, and 19% as moderate (not leaning in either direction). Similarly, the partisan identification of respondents is 29% Republican (including leaners), 55% Democrat (including leaners), and 15% independent or something else (not leaning toward either major party). These demographic characteristics are balanced across experimental groups (see Supplementary Appendix Table S13).
As with all Amazon Mechanical Turk surveys, this study does not draw upon a nationally representative sample. However, previous studies have broadly established the validity of Mechanical Turk samples for survey experiments (Berinsky et al., 2012). By now, it has been proven effective in providing high-quality participant pools for studies in social psychology (Behrend et al., 2011; Chandler et al., 2014; Summerville and Chartier, 2012), cognitive psychology (Goodman et al., 2013; Paolacci et al., 2010; Sprouse, 2011), and political science (Clifford and Jerit, 2014; Krupnikov and Levine, 2014; Mullinix et al., 2015). Most notably, several previous AMT experiments have successfully elicited racial framing effects using experiments among respondents recruited from Mechanical Turk (Callaghan and Olson, 2017; Mullinix et al., 2015; Shen and LaBouff, 2016). AMT is thus well-suited to this experimental context, though we acknowledge that it underrepresents Republicans and political conservatives, both of whom may be more responsive to racial frames than their liberal/Democratic counterparts (Peffley and Hurwitz, 2007; Peffley et al., forthcoming). 6
Experimental results
We observe no evidence of white backlash in our replication and extension of PH. Figure 1 shows the percentage of white respondents who support the death penalty across experimental treatments, while Table 3 shows the results of an ordered probit model analyzing death penalty support by treatment condition. As these results demonstrate, there are no statistically significant differences across any of these conditions. 7 These results are also consistent when estimated using OLS models (Supplementary Appendix Table S1) and an ordered probit model controlling for all covariates used in the original PH model (Supplementary Appendix Table S3).

Death penalty support by experimental condition.
Treatment effects among whites.
Coefficients and estimated cutpoints from an ordered probit regression (white respondents only); standard errors in parentheses.
Consider first the treatment condition with the race frame but no photograph or information about a specific defendant. This treatment exactly replicates the racial condition in the original PH experiment. Unlike the original experiment, exposure to the racial frame did not increase support for the death penalty among white respondents relative to the control condition. Telling white respondents that the death penalty is disproportionately administered to African-Americans had no significant effect on their views toward the policy (ordered probit coefficient = 0.032, p = 0.723).
These null results are consistent even when the race of a prototypical defendant was manipulated. The effect of the race frame did not change significantly among respondents who received information about the name, crime committed, and picture of a black defendant (coefficient = −0.037, p = 0.775) or a white defendant convicted of capital murder (coefficient = −0.044, p = 0.730). 8 Moreover, the inability of the race frame to elicit a white backlash result remained consistently null even when conditioning on respondents’ characteristics that might otherwise cause heterogeneity in responses to treatment. 9
Study 2: Additional replication analysis
Experimental design
To verify the empirical results of our experiment, we analyze data collected as part of a related study by Jardina and Piston (2017a). 10 The survey, which was fielded in June 2016 by GfK on its nationally representative Knowledge Panel and archived online by Time-Sharing Experiments in the Social Sciences (TESS) (Jardina and Piston, 2017b), included a replication of the control and racial frame conditions and the outcome measure from Peffley and Hurwitz (2007). 11 The TESS data contains responses from 2034 white Americans, whose demographic characteristics more closely mirror the white population than the respondents recruited for Study 1 on AMT. We analyze data from the 661 who served as a control group for a separate framing experiment within the survey, which avoids exposure to any additional treatments that might confound the relationship between death penalty support and the racial frame. Of the 661 respondents considered in this analysis, roughly 50% identified as female, 13% are aged 18–29, 47% are 30–59 years of age, and 40% are 60 years old or older. In terms of education, nearly 54% have not completed a college degree, while 31% have completed some form of undergraduate education. Thirty-nine percent of respondents identify as ideologically conservative, 26% as ideologically liberal, and 34% as moderate (not leaning in either direction). Fifty-one percent identify with the Republican party (including leaners), 46% identify as Democrats (including leaners), and 3% identify as independent or something else. These characteristics are balanced between conditions (see Supplementary Appendix Table S14).
Experimental results
Consistent with Study 1, the TESS data provides no evidence of a relationship between death penalty support and the racial frame. Figure 2 presents the percentage of respondents who support the death penalty across experimental groups. 12 Table 4 displays the coefficients of an ordered probit regression that models death penalty support as a function of treatment. 13 Both illustrate that the racial frame has no statistically or substantively significant effect. As in Study 1, support for the death penalty did not change measurably among white respondents who received a message indicating that the application of the death penalty disproportionately affects African-Americans (ordered probit coefficient = −0.074, p = 0.377). We again also find that the effect of the race frame is not moderated by racial resentment or other potential correlates of racial conservatism. 14

Death penalty support by experimental condition: Time-Sharing Experiments in the Social Sciences data.
Treatment effects among whites: Time-Sharing Experiments in the Social Sciences data.
Coefficients and estimated cutpoints from an ordered probit regression (white respondents only); standard errors in parentheses.
Conclusion
Though perceptions of race undoubtedly influence citizens’ attitudes on many issues, our results suggest that additional research is required to understand how racial frames affect whites’ opinions on capital punishment. The inability of these studies to elicit the white backlash effect found in Peffley and Hurwitz (2007) using the original treatment or a variant that might be expected to enhance the racialized frame of the death penalty suggests that it may be a false positive or subject to previously unknown scope conditions. It is also possible that perceptions of race and the death penalty changed in the 15 years between the original telephone survey in PH and our replication. Though our data do not indicate major changes in overall support for the death penalty since PH’s 2001 sample, our inability to find consistent backlash effects among any subgroup across two studies offers evidence of systematic differences in how race frames the death penalty. It may be that politics under Presidents Obama and Trump has become so racialized that racial primes are increasingly ineffective (Valentino et al., forthcoming). Alternatively, perceptions on the issue may have changed; fewer Americans may now be implicitly sympathetic to the disproportionate application of the death penalty by race. Finally, our failure to find any treatment effect could reflect a difference in survey mode (online versus telephone) or sample population (AMT and GfK versus a random-digit dial probability sample). To resolve these questions, further studies are needed using different populations, survey modes, treatment stimuli, and pre-treatment measures of potential moderators (e.g., need to evaluate) to better establish where, when, and for whom racial frames increase support for the death penalty among whites.
Footnotes
Acknowledgements
We are grateful to Jon Hurwitz, Ashley Jardina, Mark Peffley, and Spencer Piston for providing replication data and to Peffley, Piston, and L.J. Zigerell for valuable comments. All errors are our own.
Correction (June 2025):
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplementary material
The supplementary files are available at http://journals.sagepub.com/doi/suppl/10.1177/2053168017751250. The replication files are available at: ![]()
Notes
Carnegie Corporation of New York Grant
This publication was made possible (in part) by a grant from Carnegie Corporation of New York. The statements made and views expressed are solely the responsibility of the author.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
