Abstract
Although the U.S. Supreme Court goes to great lengths to avoid the “political thicket,” it is sometimes unwittingly pulled in. We employ several experimental treatments—each of which is composed of real behaviors that took place during the Trump impeachment trial—to understand the impact of the trial on attitudes about the Court. We find that Chief Justice Roberts’ presence and behaviors during the trial failed to legitimize the proceeding and may have even harmed views of the Court. Treatments involving Roberts’ actions decreased willingness to accept Court decisions and, in some cases, negatively impacted perceived legitimacy. We also find that criticisms of the Chief Justice by Senators decreased decision acceptance. These findings clarify both the bounds of the institution’s legitimizing power and the tenuous nature of public support in times of greater Court politicization by outside actors.
Chief Justice John Roberts has long been eminently concerned with the U.S. Supreme Court’s legitimacy. Some suggest he is “motivated by fear that he and his colleagues will end up looking like politicians in robes” (Greenhouse 2019). Given Roberts’ consternation about the Court’s legitimacy, it was almost certainly to his chagrin to have been constitutionally thrust into the political thicket when tasked with presiding over the first impeachment trial of President Donald Trump. This paper asks two questions predicated on Roberts’ brief foray into the land of polarized politics. First, did Roberts’ participation affect the Court, either helping or harming its reputation? Second, were Senators’ comments about the Court made during the trial able to influence views of the Court? To find out, we surveyed 1561 U.S. adults in March 2020 (directly after the conclusion of the trial). We utilized genuine events from the impeachment trial—three pertaining to Roberts and one criticism of the Court—as the basis for treatments in a series of experimental tests of the questions posed above. We ask whether these events had any influence on diffuse support for the Supreme Court and acceptance of the Court’s decisions.
We find that Roberts’ concerns about political perceptions of the judiciary were warranted in the context of impeachment. Despite the belief among the media that the chief justice “presided admirably” and “avoided any hint of partiality” (Savage 2020), we find that exposure to certain fixtures of the trial proved harmful to perceptions of the Court, at least in the short term. Both decision acceptance and diffuse support are decreased by experimental treatments involving Roberts; partisanship conditions reactions to various treatments in the expected ways. Additionally, criticisms of the Court’s legitimacy and Roberts’ impartiality during the trial prove capable of influencing decision acceptance. While both President Trump and the Senate ostensibly escaped the trial unscathed, this may not be true of the Supreme Court. Our results indicate that the off-bench behaviors of the justices still reflect on the institution, and the positivity bias that usually shields the Court’s reputation may not be protective in all situations.
Impeachment, Roberts, and Court support
During Trump’s impeachment trial, Chief Justice Roberts—who has long sought to paint the federal judiciary as a nonpartisan institution—was stuck somewhere between the potential consequences of inaction (perceived favoritism of allowing Senate Republicans to operate an in-name-only trial) and the consequences of action (an unelected judge playing a role in the potential removal of the president). The press indicated that the impeachment proceedings “will be full of peril for [Roberts’] reputation and that of his court” (Liptak 2020).
While Roberts was generally reserved during the proceedings, he did have several attention-grabbing moments. We suspect these moments may have negatively impacted support for the Court among the mass public. This is largely because the Court’s public esteem is vulnerable in polarized political situations; when pitted against partisan stimuli, it is the Court that suffers (e.g., Armaly, Forthcoming; Bartels and Johnston, 2020). For instance, Armaly (2018) shows that emotional attachments to political figures purported to criticize the Court can influence diffuse support. Similarly, Clark and Kastellec (2015) find that individuals are more supportive of Court curbing when their party is said to propose it. During the impeachment trial, any of Roberts’ (in)actions could have been viewed as aiding a particular political side. As such, we expect that Roberts’ actions, no matter how unbiased they appear, ultimately harmed public perceptions of the Supreme Court.
Moreover, though the Court is successful at building and maintaining support in the judicial context (e.g., Baird 2001; Salamone 2013), it is less able to do so without the traditional trappings of Supreme Court procedures. Indeed, broad views of the political world—such as differential emotional reactions to partisan stimuli—impact attitudes regarding the Court (Armaly and Enders forthcoming). Such elements were surely present during the impeachment. We suspect that Roberts’ actions—in the absence of what typically protects the Court (e.g., Gibson et al, 2014)—may have been perceived as biased or partisan, and therefore influence public attitudes about the Court.
There is specific precedent to expect that the off-bench, non-judicial behavior of a single justice can influence public perceptions of the Court. Krewson (2019) uses both survey and field experimental evidence to demonstrate that public speeches by the justices prove capable of altering views of that justice, the role of law in judicial decision-making, and the Court itself. Carrington and French (2021) find that reactions to a controversial Senate confirmation hearing condition attitudes toward the Court. Thus, it seems that justices (and nominees) are perceived as representing the Court even during off-bench behavior. We expect the same is true of Roberts’ off-bench behavior at the impeachment hearing.
We additionally consider whether the Court was implicated in any of the trial proceedings that did not involve Roberts, himself. Though the Court appears to be vulnerable to direct attacks from in-group political figures (Armaly 2020; Nelson and Gibson 2019), the impeachment trial offered a unique opportunity to study how comments on the judiciary by elected officials influence public perceptions of the Supreme Court in the presence of a legitimizing figure (i.e., the black-robed Roberts, himself). We more carefully elaborate on our specific expectations below.
Data and experimental treatments
To answer our questions on the role impeachment played on public Court attitudes, we surveyed 1561 people using Amazon’s Mechanical Turk (MTurk) in March 2020, shortly after the conclusion of the impeachment trial. We are not particularly interested in estimating public opinion about the impeachment. Rather, we wish to describe the effects of related newsworthy events on public opinion (i.e., estimate experimental treatment effects). MTurk is a useful and appropriate platform for fielding such a study (Berinsky et al, 2012). Indeed, MTurk is utilized in studies estimating treatment effects regarding opinions about the Court (e.g., Christenson and Glick 2015). We describe the demographic characteristics of our sample in the appendix, and speak to the potential pitfalls of utilizing MTurk in the discussion.
Summary of treatment vignettes.
A Note on Research Design and Generalizability
We first wish to clarify our research design vis-à-vis treatments, hypotheses, and the questions we are asking of our data. Though we are employing experimental methods, we rely on a specific focusing event (i.e., the impeachment). In a sense, we did not generate the treatments that we use; they are directly culled from the observed behaviors of salient political actors. Generating unique treatments just to investigate, for instance, the breaking point of legitimacy in the context of an impeachment trial strikes us as a rather useless activity given the infrequency of impeachments and the few observations available for guidance. That is, experimentation via unique treatments would spawn results of questionable validity, especially external validity. While we may not be able to apply our results to other impeachments (they are incredibly rare), we expect that they do shed light on how the behavior of justices outside of the Court context ultimately reflects on the judiciary. Thus, this paper contributes to the broader literature on the effects of the off-bench behavior of the justices (e.g., Krewson 2019).
Specifically, we want to know whether various aspects of the impeachment trial harmed the Court. Given the existing literature on support for the Supreme Court (e.g., Armaly forthcoming; Clark and Kastellec 2015), we have reason to believe that any focusing event that inherently blends partisan stimuli/institutions and a nonpartisan institution has a strong potential to prove harmful for that nonpartisan institution (e.g., the Supreme Court). This may be particularly true in one of the most contentious political situations imaginable. More broadly, we want to know whether these non-judicial events bear on the Court. In a sense, our hypotheses are natural extensions of our treatments, which are straightforward descriptions of a real world event. We treat people with information most likely to impact the Court, given the existing literature (e.g., Nicholson and Hansford 2014). Our central question, then, is: Does this information impact the Court?
We cannot say with any certainty that the impeachment trial events used as experimental treatments did, in fact, impact public opinion. Indeed, we cannot ascertain when individuals were treated—they may have watched the impeachment trial live on television, seen news reports/snippets at day’s end, or exposed first during our experiment. Thus, claims about our treatments impacting public opinion would be difficult to make, likely requiring representative panel data gathered directly before and after the trial. Instead, we are more equipped, as a function of our research design, to describe the potential ways people may have been impacted by these events if they were paying attention in real time, or were hearing the information after the event. Still, because existing research highlights how non-judicial events can impact the Court (Carrington and French 2021; Krewson 2019), we have reason to believe that what we uncover in the context of impeachment may apply to other, non-impeachment events. Finally, while experimental effects can deviate in magnitude from “real world” effects (see Barabas and Jerit 2010), it seems reasonable to expect that we have at least uncovered meaningful directional effects. We consider this further in the discussion section.
The influence of Roberts’ behavior on the Court
We first examine the role Roberts’ behavior played on two outcomes pertinent to public opinion regarding the judiciary. Decision acceptance, or the acquiescence component of support for the Court (Gibson and Caldeira 1995), is measured using a four-point, decisions “definitely should not be accepted” (1) to “definitely should be accepted” (4) scale. This item, utilized in previous studies of Court attitudes (e.g., Armaly forthcoming), measures willingness to consider the Court’s opinions the final word on legal issues, but does not account for institutional independence attitudes (Gibson et al., 2005). We also employ the legitimacy battery popularized by Gibson et al. (2003) to measure diffuse support, or support for an autonomous judiciary. The two forms of support are conceptually distinct from one another, and represent short-term versus long-term Court attitudes. In this sense, legitimacy represents the “harder” test, as it is the more obdurate form of support that “flows from those who are sympathetic to the function of the Court” (Caldeira and Gibson 1992, 649). We expect that Roberts’ behavior will have a negative impact on both attitudes simply because the Court fares poorly in the partisan context (Armaly, Forthcoming; Bartels and Johnston, 2020; Clark and Kastellec, 2015). Nevertheless, our main expectation is to uncover heterogeneous treatment effects based on partisanship. Finally, we expect different effects in terms of magnitude (i.e., per existing evidence, we believe the effects on acceptance will be stronger than on legitimacy), but not in terms of direction.
Consistent with expectations, each of the three Roberts treatments—admonishment, failure to recall the GOP Senators, and refusing to name the whistleblower—decreases decision acceptance to a statistically significant degree. Acceptance is around 12.5% lower than for control respondents.
1
For diffuse support, only Roberts breaking protocol significantly impacted legitimacy. However, all point estimates were directionally consistent with the decision acceptance analyses: if anything, Roberts’ presence and behavior reduced support for the Court (though not to a significant degree). Figure 1 displays these effects. Treatment effects of Roberts’ actions.
Because of the intensely partisan nature of the impeachment trial, we expect to observe heterogeneous treatment effects. Democrats may have applauded the Chief’s refusal to name the whistleblower, thus conferring more legitimacy on the Court. Republicans—who wanted the whistleblower unmasked—may have punished the Court for Robert’s refusal. For both outcomes, we expect that Democrats and Republicans will both decrease acquiescence and support in the admonishment treatment (after all, Roberts admonished both sides), Democrats (Republicans) will decrease (increase) acceptance and support in the protocol treatment (as Republicans benefitted), and Democrats (Republicans) will increase (decrease) acceptance and support in the whistleblower treatment (as Democrats wished for the identity to remain secret, and Republicans did not). Figure 2 plots treatment effects for Democrats and Republicans, separately, across both dependent variables and treatments.
2
Marginal treatment effects, by partisanship.
Beginning with decision acceptance, in the top row, Democratic respondents appear to reduce their willingness to accept Supreme Court decisions in both the admonishment (left panel) and breaking protocol (center panel) treatments. The effects are negative and statistically significant for each. Republicans, however, are not impacted as greatly as Democrats by these treatments. Still, as is the case with average treatment effects, all point estimates are negative, meaning exposure to Roberts’ actions does not seem to help the Supreme Court. Regarding the whistleblower treatment, neither estimate reaches traditional levels of statistical significance; still, the average treatment effect reported above is negative and significant. Altogether, it seems that Roberts’ actions during the impeachment trial translated to less willingness to accept the Court’s decisions, particularly among Democrats.
Regarding diffuse support, in the bottom row, we observe slightly more meaningful heterogeneity. In both the admonish and whistleblower treatments, Democratic respondents increase diffuse support (though not quite at a statistically significant level) and Republican respondents decrease support. This surely contributes to the null average treatment effects shown above. In the breaking protocol treatment, Republican respondents report less diffuse support than their Democratic counterparts. This is admittedly counter-intuitive, as failure to recall absent Republican Senators is a decision Republican self-identifiers should, seemingly, support. Nevertheless, exposure to Roberts’ behavior meaningfully impacts the legitimacy on which the Court relies. Finally, the whistleblower treatment worked in the expected direction, across partisanship. Republican respondents, who preferred the whistleblower receive scrutiny during the trial, negatively react to Roberts’ expressed refusal to out the individual.
All in all, we observe meaningful reactions to three of Roberts’ noteworthy actions at the impeachment trial in many cases. Although these reactions are not universal across various assessments of the Court, we do see expected heterogeneity across partisanship. Democrats find themselves expressing less willingness to accept decisions than their untreated counterparts, and Republicans are more willing to make fundamental alterations to the institution than their untreated counterparts (in certain instances). While we do not expect (and have not witnessed) a full rebuke of the institution, the Court was likely not unscathed after the impeachment as a function of Roberts’ participation. Of course, memories of these events may not have persisted for very long after the trial. However, reactions to political stimuli are often less about specific recall and more about summary evaluations impacted by various events (Lodge et al, 1995). While democratic norms can rebuild support for the Court (Gibson and Nelson 2015), any event that threatens the Court’s political capital is concerning.
The influence of Court criticism during impeachment
Next, we examine the influence of Senator Warren’s question regarding the legitimacy of the Supreme Court. Direct attacks on the Supreme Court can reduce support for the institution among the mass public (Armaly 2020; Nelson and Gibson 2019), as does partisan strife (Armaly and Enders forthcoming). Often, specific criticisms are levied at the Court itself. Here, our treatment is less targeted at the Court, instead focusing on whether the Chief Justice’s role and Republican Senators’ behavior in the impeachment trial might prove problematic in an era when support for the Court is decreasing. We expect that commentary such as Warren’s will prime considerations of the Court that do not evoke the positivity that frequently underlies evaluations of the judiciary (Gibson and Caldeira 2009). As such, despite Roberts’ best efforts to “…preserve his image and the image of the court as nonpartisan” (Savage 2020), one unintended consequence of the impeachment trial may have been reduced support for the Supreme Court, regardless of what Roberts did (not) do.
Importantly, the question is never attributed to Senator Warren, or any particular senator. The purpose of not attributing the question was to mitigate the influence of partisanship and affect toward the political figure, both of which condition receptivity to criticisms of the Court (Clark and Kastellec 2015; Armaly 2020; respectively). Instead, respondents are merely told that Chief Justice Roberts read the following question aloud: At a time when large majorities of Americans have lost faith in government, does the fact that the Chief Justice is presiding over an impeachment trial in which Republican Senators have, thus far, refused to allow witnesses or evidence contribute to the loss of legitimacy of the Chief Justice, the Supreme Court, and the Constitution?
While it is reasonably understood that this criticism likely does not come from a Republican, respondents may still struggle to determine whether the source is credible. For this reason, we do not predict there will be differential response to Warren’s criticism across partisanship, and present average treatment effects in Figure 3.
3
Information presented in the appendix shows that the effects presented here are not heterogeneous across partisanship, indicating that Warren’s criticism of the Court played a general role. Effect of Senator Warren’s question.
Warren’s criticism seems to have negatively impacted the Court, although diffuse support—perhaps the most important evaluation of the Court—is unmoved. Still, individuals exposed to Warren’s commentary are less accepting of judicial decisions. It appears that some actions of senators during the impeachment trial may have been sufficient to harm the Court, regardless of Roberts’ actions during the trial. While this appears to be good news for the judiciary, continued disappointment with the institution can translate to broader, more enduring discontent (Baird 2001). A nearly 8% decrease in decision acceptance is fairly robust, and signals the ability of outside actors to influence views of the Court.
We note that these effects reflect reactions to a single element of the impeachment trial; as discussed above, any consumer of impeachment news likely would have seen several such instances. In other studies of criticism of the Court (e.g. Armaly 2018), survey respondents are not necessarily asked to consider such criticism in a context where the Court is immediately implicated. Here, it is stated that Roberts was required to read Senator Warren’s question aloud. While we did not describe Roberts’ silent expression of disappointment, 4 or other senators’ efforts to praise Roberts, there is nevertheless a legitimizing symbol inherent in our treatment—Chief Justice Roberts himself. This, perhaps, should have helped fortify support for the Court. Yet, we see the opposite: the Court is harmed, to some degree, by Warren’s criticism. This constitutes supportive evidence that there is little Roberts, or the Court, could have done to stave off the effects of such criticism, especially in such a polarized environment.
Discussion
Altogether, our analyses reveal a negative impact of the (first) impeachment trial of Donald Trump on support for the Supreme Court, in some instances conditional on partisanship. While Roberts’ presence at the impeachment trial could have altered peoples’ opinions about Trump and the impeachment by lending legitimacy to a deeply partisan event, we observe the opposite. The impeachment trial seems more likely to have delegitimized the Court than legitimized it. Moreover, the queries of other political actors, such as Warren, about the legitimacy of the Court seem capable of impacting orientations toward the Court by simply invoking its legitimacy as a non-partisan/ideological branch of government.
That said, diffuse support for the Court was significantly impacted by the behavior of Roberts in only a few cases, and the effects are weaker than those on decision acceptance. This is sensible, as legitimacy is a more obdurate, long-term form of support. Generally, we expect fewer disruptions to legitimacy than more fleeting reactions to the judiciary. Still, that we uncover effects on diffuse support from the impeachment proceedings at all is an important discovery, and accords with a growing body of evidence that the Court’s political capital is not free from harm (e.g., Armaly 2018; Clark and Kastellec 2015). On the one hand, that we do not see effects on legitimacy in all instances is encouraging for the Court and congruent with the theoretical nature of legitimacy as a stable, scarcely flappable force. On the other hand, that we did observe statistically significant effects with respect to single treatments should be disconcerting. Indeed, each of the treatments is composed of real, externally valid events that took place during the impeachment trial. Anyone paying attention to the trial could have been exposed to most, if not all, of them. Thus, our analyses fail to capture possible cumulative effects of exposure to Roberts’ behavior, or that of Senators or other media personalities that may have sought to further politicize the trial or the Court.
Our study is not without limitations. First and foremost, the use of MTurk presents some challenges; “turkers” are less demographically representative (Buhrmester et al., 2011) and less diverse (Paolacci and Chandler 2014) than a random or quota-based sample. Nevertheless, Huff and Tingley (2015) show that many political characteristics of MTurk workers are similar to those on larger, more representative surveys (e.g., the CCES). Still, we encourage appropriate caution in generalizing our findings. One additional limitation concerns the timing of our survey relative to the impeachment. Since the trial had been completed by the time the experiment was fielded (necessarily so), some attitudes may have hardened since the real world stakes had disappeared (i.e., Trump was not going to be removed from office). Although this, in our view, is a worthy sacrifice for externally valid treatments, it is a potential limitation nonetheless. We also encourage future work to investigate the potential for shorter-term evaluations of the Court to impact diffuse support in the long term, particularly with polarization from extrajudicial outsiders (see Armaly and Enders forthcoming). Just as negative affect toward political figures can “spill over” into other domains (see Tesler 2012), so too might short-term evaluations of the Court contaminate the “reservoir of goodwill.”
Finally, as is the case with all survey experiments, effects may not match those found in the real world (Barabas and Jerit 2010). While we believe that our experimental effects are likely muted relative to the conjoint effect of witnessing many of the events we use as treatments, it is still possible that the influence of the impeachment on Court support waned as memory faded. Yet, any perturbations in support are noteworthy, particularly when off-bench, non-judicial behavior in a partisan setting is the culprit. Even if the effects persist for a small subset of respondents or Court-observers, additional legitimacy influencing events in the future may pose the same threat.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
