Abstract
The interpersonal dynamics of political elites remain somewhat shrouded within extant literature. In this paper, we propose a new measure of “appeasement,” using analyses of Supreme Court opinions. Appeasement operationalizes the level of strategic concession-making that individual justices display in their responses to majority opinions. Analyzing a corpus of over 9,000 non-majority opinions between 1946 and 2011, we use a machine learning model to detect the number of concessions versus non-concessions made by individual justices. This ratio of concessions to non-concessions is used to create our appeasement measure. We suggest that appeasement varies as a result of interpersonal relationships, external pressures, and personal characteristics of justices. Conciliatory speech is a tool that may be wielded by political elites in all manner of political rhetoric. As such, appeasement represents an important potential avenue for categorizing and demystifying the interpersonal dynamics of governmental institutions.
Introduction
In moments of profound political division, Supreme Court justices adopt sharply divergent rhetorical strategies, sometimes explicitly confronting their colleagues. In the landmark case Dobbs v. Jackson Women’s Health Organization (2022), Justices Breyer, Sotomayor, and Kagan wrote a fiery dissent, criticizing the majority for discarding decades of abortion-rights precedent. The dissenters sounded alarm bells about the erosion of democratic norms, stating that either “the mass of the majority’s opinion is hypocrisy, or additional constitutional rights are under threat” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 5). They condemned the ruling as an abandonment of judicial responsibility, declaring that “today, the proclivities of individuals rule,” criticizing the majority’s selective historical analysis as mere “window dressing” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 14). Most pointedly, they argue that the decision leaves women with “no rights to speak of,” as well as emphasizing the tangible harm inflicted: “the majority says [each state can] address abortion as it pleases. That is cold comfort…for the poor woman who cannot get the money to fly to a distant state for a procedure” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 3). The dissenting opinion’s intense rhetoric vividly illustrates a clear point of fracture within the Court, by directly targeting the majority writers for their shortcomings.
By contrast, Chief Justice Roberts, concurring in judgment, tempers the tone of discussion through his restrained agreement with the majority. While supporting a narrower ruling on the outcome, Roberts cautions against the “dramatic and consequential ruling” which overturns precedent, noting that it was “unnecessary to decide the case before us” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 2). Prior to doing so, Roberts explicitly highlighted the strengths of the majority’s reasoning, calling it “thoughtful and thorough” and that he “agree[s] with the Court that the viability line established by Roe and Casey should be discarded” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 1). Even so, Roberts repeatedly emphasizes that “none of this…requires that we also take the dramatic step of altogether eliminating the abortion right first recognized in Roe” (Dobbs v. Jackson Women’s Health Organization, 2022, p. 5). His concurrence offers criticism, but also represents a subtle attempt to stabilize perceptions of the Court. Roberts carefully acknowledges his points of agreement with the majority, while clearly underscoring that his colleagues’ radical departure was neither requested nor required (Dobbs v. Jackson Women’s Health Organization, 2022).
This is just one example of the ways that political elites, including Supreme Court justices, take different approaches in the language they use to address their colleagues in the public sphere. In separate political moments, individual elites have varying strategic concerns, indicating their normative sensitivities that are increasingly consequential in the modern political climate. In this paper, we introduce a measure of appeasement that quantifies some of these sensitivities. In the judicial context, the measure captures the extent to which justices strategically concede aspects of the majority’s reasoning, even as they articulate disagreement in their broader concurrence or dissent. Importantly, appeasement is distinct from measures of linguistic tone, such as disagreeable rhetoric, which capture the presence of negative or harsh language (Zilis & Wedeking, 2020). Rather than focusing on the level of affect in legal speech, appeasement captures the prevalence with which justices grant credence to opposing arguments in their process of legal reasoning. We demonstrate that appeasement is an individual difference that corresponds to how political elites orient their differing perspectives and personalities, how they interact with institutional norms, and how they react to internal and external pressures. Specifically, we examine whether appeasement increases as majority coalitions broaden, how it varies in response to issue salience, whether it declines as a function of a justice’s ideological extremity, and whether it captures a distinct rhetorical strategy relative to disagreeable rhetoric.
We focus on the language used by Supreme Court justices in non-majority decisions to test our measurement. The Supreme Court provides a useful setting for this analysis, as justices must communicate disagreement in a highly institutionalized and publicly visible context, where their language is carefully constructed to address multiple audiences, including their colleagues, the legal community, and the public. While we use the Supreme Court as our case study of appeasement, we suggest that appeasatory language is found beyond the confines of judicial opinions (Ceaser et al., 1981; Rhodes, 2014). Specifically, the institutionalization and subtleties of legal arguments make it an ideal case for measuring intentional, appeasing language. This measurement aims to detect the extent to which elites are willing to validate the point of view of their opponents, if only to enhance the strength of their own arguments or the institution as a whole. Wherever there is the possibility for disagreement, there will also be the possibility to appease the other side. However, certain elites will tend towards recognizing and taking advantage of these opportunities, by bonding with or chiding their opposition. Those who wish to maintain a norm of understanding and respect between opposing sides, or merely the appearance of it, will deploy more appeasatory language.
Appeasement can be found in many other facets of the political world. For example, it can be observed in debates on the floor in Congress, where opposing sides advance their positions while engaging with one another. Despite the fact that full debates are rarely watched by the broader public, members of Congress use clips of their floor speeches on social media and to communicate with their constituents (Straus & Glassman, 2016). Representatives deploy bipartisan rhetoric to dampen negative impressions of opposition voters, even if they lack genuine opposition support (Westwood, 2022). Furthermore, congresspersons demonstrate different rhetorical styles depending on the colleague or issue being addressed (Miller & Sutherland, 2023). Therefore, the decision for a political elite to use appeasatory language may speak to their ability to balance their ideological commitments with their other long-term strategic considerations. Furthermore, evidence that an elite recognizes the opportunity to appease may indicate important psychological differences in how they evaluate their opposition and the structure of institutions as a whole.
Understanding these dynamics is particularly important in an era of heightened political polarization (Iyengar et al., 2012, 2019). Literature suggests that partisan identity is a powerful biasing force, which can lead to social and perceptual biases (Gift & Gift, 2015; Mitchelitch, 2015; Nicholson et al., 2016), dehumanization (Martherus et al., 2021), and potentially even support for violence or extreme political actions (Holliday et al., 2024; Kalmoe & Mason, 2022). Although extant research focuses on the presence of negative political rhetoric (Külz et al., 2023; Zilis & Wedeking, 2020), less work examines the circumstances where political elites may temper the language of their disagreements. Furthermore, existing measures of negative rhetoric may not be sufficiently sensitive to rhetorical strategies like appeasement, as they rely on bag-of-words approaches to analyze their text data (Külz et al., 2023; Zilis & Wedeking, 2020), rather than advanced machine learning models, which can capture rhetorical nuances within disagreement. In this sense, studying appeasement could illuminate the conditions that increase the likelihood of political elites working together, promoting collegiality, and stabilizing their institutions, even though they disagree with their opponents.
Our measure of appeasement is conceptually distinct from existing measures of negative rhetoric, particularly “disagreeable rhetoric” as developed by Zilis and Wedeking (2020). Disagreeable rhetoric captures the extent to which justices utilize harsh, negative, or emotionally charged language, often as a signal of issue importance or institutional conflict (Zilis & Wedeking, 2020). By contrast, appeasement captures a different dimension of rhetorical strategy—the extent to which justices validate or concede aspects of opposing arguments within their reasoning. Importantly, we do not perceive these measures as inherently opposed. While disagreeable language measures the harshness present in individual words, appeasement captures the broader rhetorical strategy present in a snippet of text. Therefore, some cases measured as “appeasement” may rate higher in disagreeable rhetoric, while others may rate lower in disagreeable rhetoric. For example, a justice may use harsh, negative language to criticize the consequences of a decision, while still conceding that aspects of the majority’s legal reasoning are sound. This distinction motivates our empirical comparison of appeasement with disagreeable rhetoric, which examines whether appeasement captures unique dimensions of judicial rhetoric that existing measures do not.
Appeasement provides researchers with a way to analyze a key rhetorical tool that certain political elites may tend towards selecting. In the Supreme Court, justices carefully craft their words to match the ethos they wish to project. They do so without the same electoral incentives of Congress that might drive them toward behaving as either a partisan or a moderate (Davis & Hitt, 2024). Legal text is complex and planned, with many opportunities to consider the downstream implications of words (Goldstein, 1992). As such, the dynamics of the Supreme Court provides a valuable setting for analyzing how political elites strategically structure disagreement under institutional constraints. Furthermore, the complexity of identifying appeasement within dense legal text serves as an especially robust test of our qualitative coding scheme.
Example of Concession Versus Non-concession
Using cases from 1946 to 2011 terms, 2 we analyze changes in appeasement by examining internal dynamics in the Court, external pressures on the Court, and personal characteristics of justices. In particular, we find that ideological extremity is associated with lower levels of appeasement, which suggests that appeasement may reveal strategic dynamics that tone-based measures do not. We examine markers that suggest that appeasement is a measure that is particularly attuned to the strategic context of the text, rather than just the words that justices use.
Appeasement in Judicial Opinions
Supreme Court justices themselves acknowledge the complexity of tailoring their messages across different opinions. At the Notre Dame Law Review Federal Courts Symposium in January 2023, Justice Kavanaugh revealed that when writing his opinions, he “doesn’t have just one audience in mind” (Notre Dame Law Review, 2023). During his keynote Q&A session, Justice Kavanaugh answered a question regarding who he attempts to reach with his opinions: Question: “When you’re writing a judicial opinion, who are you considering as your relevant audience, is it your colleagues, is it the lower courts, the legal academy, or the public at large? Who are you focused on when you are writing an opinion?” Kavanaugh: “All of the above… I want the losing party to understand why we disagreed, why I disagreed with them. And they aren’t going to be happy with the decision, by definition, but I want them to read it and say, “Well, okay. They at least treated our arguments with respect and fairly.” So that’s number one. I think over time that systemically if you treat that party well at oral arguments and in the opinion, over time that builds respect, I think, for what we’re doing for the rule of law” (Notre Dame Law Review, 2023, 33:35).
Justice Kavanaugh further notes that his “goal is to try to show respect for the other judges and other parties” and that he learned this behavior clerking for Justice Kennedy (Notre Dame Law Review, 2023, p. 34:27). These responses illustrate that justices have to balance their audiences, often addressing many groups at the same time. For some justices, demonstrating respect and fairness are important aspects of maintaining the approval of these audiences.
Building on perspectives such as Justice Kavanaugh’s, we argue that judicial opinions provide a particularly useful setting for examining appeasement. In this paper, we apply appeasement to non-majority opinions, to measure the interpersonal tact displayed by each justice as they spar with one another over ideological disputes. Written opinions are the primary avenue through which justices reach their many audiences (Baum, 2009). In non-majority opinions, justices must simultaneously challenge the majority’s reasoning while justifying their own position, creating opportunities to incorporate or reject elements of opposing arguments. Justices invariably differ in their conciliatory language, with some utilizing it as a rhetorical tool, depending on various considerations. In our measure, those who have higher rates of these conciliatory elements have a higher level of appeasement, demonstrating their commitment to portraying themselves as evenhanded to their audiences.
When it comes to judicial appeasement, prior work suggests that justices vary in how they engage with opposing arguments in their opinions (Mishler & Sheehan, 1996). Some justices may be more inclined to incorporate concessions into their reasoning, reflecting differences in how they approach the task of writing non-majority opinions. These choices are shaped in part by audience considerations, as justices are aware that their opinions are read by colleagues, the legal community, and the broader public. The way audiences influence opinion-writing can vary substantially across justices (Baum, 2009). In this sense, appeasement captures the extent to which justices incorporate and engage with opposing arguments in response to these audience pressures, regardless of the ultimate purpose such concessions serve.
We test several expectations regarding the use of appeasement in judicial opinions. First, if appeasement reflects strategic behavior within the Court, we expect it to increase in prevalence as majority coalitions broaden, as incentives for maintaining collegiality and institutional legitimacy would be heightened. Second, if appeasement is sensitive to external pressures, we expect it to vary in response to changes in the Court’s broader political environment, including measures of issue salience and public attention to the Court. Third, if appeasement captures an individual-level rhetorical strategy, we expect significant variations among the justices, including a negative relationship with ideological extremity. Finally, we compare appeasement to existing measures of judicial rhetoric, particularly disagreeable rhetoric (Zilis & Wedeking, 2020), examining whether appeasement tracks similar patterns or instead captures a distinct dimension of how justices structure their disagreements.
What Influences Appeasement
Employing appeasatory language could be the result of behind the scenes negotiations to maintain a collegial workplace, despite differences in ideology. Collegiality promotes consensus, which tempers disagreements, reduces dissents, and may cause others to join future majority opinions (Hazelton et al., 2023). Each of these characteristics may also be captured by appeasement. While acknowledging that the public already assumes justices vote along partisan lines (Gibson & Caldeira, 2011), individual justices may attempt to avoid the appearance of being overtly political in non-salient issues, or try to curb allegations of bias if they are behaving in an ideological manner. In this sense, justices utilize their words to communicate to audiences far beyond the involved parties in the case.
In addition to personal audiences, some justices may have the desire to appear evenhanded to maintain the legitimacy of the Court. Given that the Court does not possess the enforcement power needed to implement their decisions, much of the Supreme Court’s power derives from its legitimacy. Justice Scalia (1998) suggests that dissents “augment rather than diminish the prestige of the Court,” explaining that “when history demonstrates that one of the Court’s decisions has been a truly horrendous mistake, it is comforting…to look back and realize that at least some of the justices saw the danger clearly and gave voice, often eloquent voice, to their concern” (Scalia, 1998, p. 19). As in our prior example, Chief Justice Roberts may have been acting in a manner that would best serve to maintain legitimacy of the Court (Fettig & Benesh, 2016). Individual-level perceptions of the Court’s legitimacy rely primarily on the individual’s ideological agreement with the Court (Bartels & Johnston, 2013). If the agreement with the Court turns to disagreement, legitimacy is negatively impacted. Following recent decisions, including Dobbs v. Jackson Women’s Health Organization (2022), it appears that the Court has come under greater scrutiny. The Court has lost legitimacy following this decision and it has lingered (Gibson, 2024). While the loss of legitimacy has shown staying power, it does not mean all is lost. As Gibson (2024) explains, the connections of the institution to democratic values has the potential to help mend the wound caused by salient decisions. Making appeals to these institutional values through appeasatory language might serve a valuable role in restoring the legitimacy of the Court for motivated justices.
Other internal strategic considerations include the way that justices bargain with one another in the drafts of opinions. Bargaining on the U.S. Supreme Court is a critical part of the decision-making process for justices. For some justices, they “don’t negotiate, [they] accommodate” (Perry, 1991: 144). In order for the majority to issue an opinion, five justices must engage in the “collegial game” and work with one another to craft an opinion (Maltzman et al., 2000). This necessitates that bargaining and accommodation take place. Evidence suggests that justices are influenced by both strategic and non-strategic factors (Wahlbeck et al., 1998). Some of these strategic considerations include the composition of the majority, their ideological distance from the other justices, and the specific position that the majority takes.
Going further, Spriggs et al. (1999) describe how justices engage in a cost-benefit analysis when deciding whether to join the majority, suggesting that these decisions are shaped by strategic considerations. Factors such as disagreement with the opinion, history of cooperation, and salience of the case are all factors influencing their decision to join opinions. At the same time, specific opinion authors also have an influence on the content of majority opinions (Lax & Cameron, 2007; Lax & Rader, 2015). Perry (1991) notes that, even before the opinion-writing stage, dissents from denial of certiorari are attempts by justices to communicate to one another or to make a public statement. In these situations, the justices are already addressing their fellow justices, the public, lawyers, and politicians (Perry, 1991).
The tone and style of communication that justices take may enhance the seriousness of their message to their colleagues, successors, or the public at large (Zilis & Wedeking, 2020). Justice Brennan suggests that: “very real tensions sometimes emerge when one confronts a colleague with a dissent. After all, collegiality is important; unanimity does have value; feelings must be respected” (Brennan Jr, 1985: 429). We are motivated to discover if certain justices are more likely to offer up concessions to portray themselves as fair and impartial, solidifying their perspectives in future political discourse. Due to the secretive nature of Supreme Court decision-making process, it is nearly impossible to measure collegiality of the justices when they are coming to a decision. However, we are able to see and quantify the final product that they release to the general public, in the opinions that they write.
Orienting a Measure of Appeasement
The work of Zilis and Wedeking (2020) is closely related to our measure. This paper examines the use of disagreeable rhetoric as a means for understanding how the justices interact with one another. Deploying disagreeable rhetoric signals the level of personal importance of an issue to a particular justice and reflects the nature of the institution (Zilis & Wedeking, 2020). Our measure is different from that of Zilis and Wedeking (2020) because while appeasement contains aspects of disagreement, it also measures the extent to which justices are willing to strategically voice agreements with each other, even as they distinguish themselves from the majority. Furthermore, our model nuances this dictionary method by analyzing the specific context of words using a machine learning model. Zilis and Wedeking (2020) examine the way that disagreeable rhetoric is deployed across entire opinions, while our measure zooms in on the way that justices use language when addressing their colleagues. Disagreeable rhetoric detects when justices’ opinions contain harsh words, but our measure of appeasement examines the strategies behind how justices phrase their disagreements.
Understanding to what extent justices use appeasing language is important, as it can help explain judicial opinions, attitudes, and behaviors in a way that is orthogonal to a purely ideological model (e.g., Lewis & Rose, 2014; Martin & Quinn, 2002; Segal & Cover, 1989; Segal & Spaeth, 1996). This measure elaborates on differences in temperament and personal orientation that individual justices bring into the Court, which could be relevant to a plethora of outcomes for judicial behavior. Justices balance the interpersonal relationships they have with their fellow justices and the external pressures they face to maintain institutional legitimacy with both the public and fellow branches of government (e.g., Braman & Nelson, 2007; Epstein & Knight, 2013; Hall, 2014; Hinkle, 2015; Maltzman et al., 2000; Mark & Zilis, 2018; Richards & Herbert, 2002; Segal et al., 2011). Measuring the impacts of these pressures through appeasement can improve understanding of the factors that go into judicial opinion construction, and may even be used to predict how certain justices break from their ideological peers.
As an exploratory hypothesis for our analysis, we choose to focus on the concept of ideological extremity and median justices, using Martin and Quinn’s (2002) scores. The swing or median justice is the justice who tends to be the crucial fifth vote in a case (Enns & Wohlfarth, 2013). There has been substantial work on the role of the median justice in the Supreme Court—a role that might contribute to a justice’s desire to appear more evenhanded. Accordingly, Brenner et al. (1989) find that the marginal justice shifting to non-majority opinions is an indication of ideological closeness to one member of the opposing side. Those who are more ideologically extreme may seek to “pull down” and sabotage in non-majority opinions, while those who are more centrist may value the norm of civility, even as they disagree. Extant research suggests that social considerations such as these complicate the ideological landscape, conditioning the effects of disagreement (Hazelton et al., 2023). We suggest that an individual’s sensitivity to these considerations should be strongly related to how willing a political elite is to appease the opposing side. As well as serving as an exploratory way of orienting our measure, this ideological lens may also provide important future directions for utilizing this model. Ideological distance is a metric that cuts across all manner of partisan institutions and, as such, a measure that extends from this construct will have a variety of uses.
In our measurement of our construct, we narrow our focus to published, non-majority opinions written by Supreme Court justices. Because non-majority opinions provide the opportunity for individual justices to open a dialogue with a more popular perspective (Corley et al., 2023), they demonstrate a justice’s baseline for tactfully wording their disagreements. Going further, a benefit of analyzing non-majority opinions is that they are frequently written by individual justices, and allow us to analyze appeasement through the lens of individual characteristics. 3 Aggregated across many opinions, we may gain further insights into the temporal dynamics of the Court, along individual, Court-level, and public opinion dimensions. Non-majority decisions serve to weaken or temper the strength and authority of the majority opinion (Corley, 2010; Ray, 1990). Often, these opinions represent a strategic decision to pursue policy preferences within the bounds of institutional and strategic constraints (Wahlbeck et al., 1999). Therefore, these opinions represent a key arena for justices to utilize the tool of appeasement. We do not intend for this measure to be used as a method of determining a justice’s approach to the law or how they reason through the legal aspects of the case. Rather, this measure captures the manner in which interactions between political elites are performed in the public forum.
Measuring Appeasement
To measure appeasement, we utilize a supervised machine learning model, focusing on published regular concurrences, special concurrences, and dissents written by all the justices from 1946 to 2011. 4 Our analyses are limited because our accessible data only covers this time frame. However, the modern Supreme Court emerged within this time period, and prior eras of the Court are generally recognized as hiding outward disagreement. Temporal analysis of a large portion of the 20th and 21st centuries still allows for a healthy, longitudinal analysis of the Court, and allows us to determine variations between individual justices and Court iterations dependent on context and individual characteristics.
We choose to look at non-majority opinions due to the inherent tension between these opinions and the majority. Within these tensions, mentions across more friendly concurrences and more scathing dissents allow us to examine a wide set of data to make broad claims about judicial temperaments. Specifically, these cases allow us to measure how justices structure their interactions with their colleagues when they do not support the outcome or reasoning of a case. Because these opinions represent a path in which the justices are expected to express their discontent with a majority decision, any deviation from this norm may be viewed as utilizing the tool of appeasement. However, in other political contexts, appeasement may be commonplace in the speech of majority parties as well. In an examination of collegiality within the federal courts of appeals, Nash (2022) utilizes a similar method for coding dissenting opinions. However, our measure differs from this method, because we are not making the claim that appeasement inherently implies collegiality. A justice might employ appeasing language to assuage their colleagues, but it is not always the case that their audience is only their colleagues (Baum, 2009).
For our modeling process, the text data of the non-majority opinions is gathered into 9,049 separate files from an extant opinion-text database (Rice, 2017), and then merged to add metadata from the Supreme Court Database (Spaeth et al., 2021). It is apparent in the data that words surrounding mentions of the Court contain the majority of concession data, as this is where a justice addresses the Court directly. As such, we create data points within each non-majority opinion by pulling out a window of words around cases where a justice explicitly mentions the majority opinion or the Court at large. More specifically, 50 word (or larger) windows of text around mentions of “the Court,” “this Court,” “the majority,” and “this majority.” We create these windows by expanding an initial 50 word window by an additional 50 words for each nested occurrence of the mention terms—sometimes expanding the text window to as many as 300 total words. Though most cases only include 50 words, this format allows our human and machine raters more context where necessary and reduces overlapping windows within our sample, which might otherwise bias the training of our machine learning model. These data points represent our sample set for both hand and machine coding of how justices address the majority opinion.
In total, this process creates 60,637 occurrences of mentions of the majority from non-majority opinions between 1946 and 2011. We hand-code a sample of 2,500 of these statements as either a concession, or a non-concession. In this case a “concession” represents any instance where the author validates or concedes strengths of the majority argument, regardless of tone. A “non-concession” refers to every other mention of the majority. We have a second hand-coder rate 100 of these cases with 97% agreement, suggesting a high level of reliability. However, mentions of the majority that refer to a past iteration of “the Court” still need to be cleaned from this dataset. To remove these irrelevant observations, we code this same sample of 2,500 occurrences for if they referred to the current majority. Again validating our coding scheme, our second coder rates 100 of these cases with 93% agreement. These variables are then used to train supervised Bidirectional Encoder Representations from Transformer models (BERT).
BERT is a Large Language Model (LLM) with the ability to understand the context of words within sentences, enabling nuanced classifications. This is particularly important for a complex classification task, such as analyzing legal text. In particular, the BERT model we utilize, LEGAL-BERT, is pre-trained on a variety of legal text databases, and designed to understand legal jargon (Chalkidis et al., 2020). For our outcome of current majority versus not current majority, this model operates with a recall rate of 96.6%, an overall rating of 96.1% accuracy, and an F1 score of 96.9%, as measured by a validation sample of 20% of the hand-coded cases. This subsequent model is used to cull the number of cases from 60,637 to 34,112 cases that refer to the current majority opinion. These 34,112 cases serve as the cleaned data set for our LEGAL-BERT classifier, which machine-codes concessions or non-concessions.
In a separate LEGAL-BERT structure, a machine learning model is trained from our cases that are hand-coded for the presence of concessions or non-concessions within mentions of the majority. Coding a test sample of 20% of hand-coded cases, this LLM operates with a concession recall rate of 92.3%, an overall accuracy rating of 98.6%, and an F1 score of 94.1% in the final model. Extant research suggests that basic, argumentative versus non-argumentative classification of legal texts may have roughly a 73% accuracy rate (Palau & Moens, 2009). Other research summarizes legal text with an F1 score ranging from 64% to 79% (Goncalves & Quaresma, 2005)—understandable given the complexity and density of legal jargon. Considering this, our classifier outperforms our expectations, suggesting that the LEGAL-BERT model is highly attuned to picking up the context-specific coding of our concession versus non-concession model.
To calculate the appeasement score for individual cases, we create a ratio of concessions to non-concessions. The higher the ratio of concessions to non-concessions, the more appeasement a justice demonstrates in a single opinion. For broader measures of appeasement, such as by year, or for an entire justice’s catalog of opinions, we take the number of concessions over the total number of yearly non-majority opinions. This gives us an average amount of concessions per non-majority opinion. Though our ratings necessarily fluctuate given an individual justice’s propensity for non-majority opinions, the richness of this database still allows for strong inferences. 5
In validating our measure, we examine several outcomes. First, we select individual justices, and compare their theoretically expected versus observed levels of appeasement. Next, we examine potential strategic considerations, such as the role of Chief Justice, which we suspected would positively influence appeasement. Then, we compare our measure alongside Zilis and Wedeking’s (2020) measure of disagreeable rhetoric, examining convergent and divergent characteristics. Finally, we analyze an exploratory hypothesis and extension of appeasement—that ideologically moderate political elites would tend towards utilizing this tool more frequently than those who are ideologically extreme (as measured by Martin-Quinn scores).
Results
We begin by examining variations in appeasement across individual justices, focusing on whether the measure captures meaningful differences in how justices structure disagreement. Figure 1 displays the appeasement rate for Justice “Wild Bill” William O. Douglas over time. Justice Douglas is known for his stubbornness and strong opinions throughout his tenure on the bench, writing the greatest number of both majority and dissenting opinions, and for being threatened with impeachment more than any member of the Court (Murphy, 2003). In our data, his appeasement levels are consistently lower across his many non-majority opinions—among the lowest in our dataset. This broadly conforms with our expectations for what appeasement measures, as the careful strategy of appeasement is the antithesis of plain-talking and blunt argumentation, like what would be anticipated from Justice Douglas. Justice Douglas’ versus other justices’ appeasement
We contrast this trend to Justice Rehnquist’s appeasement levels as shown in Figure 2. As the only justice in our dataset to be promoted from Associate Justice to Chief Justice, Rehnquist represents a unique case for understanding our appeasement measure. From his appointment as an Associate Justice in 1972 until his promotion to Chief Justice in 1986, we can see a stable trend of fairly typical appeasement behavior. However, following his appointment to Chief Justice, his pattern of appeasatory behavior begins to change. The graph becomes more volatile, as Justice Rehnquist authors fewer non-majority opinions, though it also reaches new highs. This is likely due to new concerns as Chief Justice (Badas, 2021), particularly towards the end of his term, when Rehnquist begins to contemplate his legacy. Rehnquist’s pattern may also represent the task of maintaining the institutional legitimacy of the Court, a burden that largely rests on the Chief Justice. Because of his new role, he may not have as much time or drive to antagonize the majority. We can see that—whenever Rehnquist’s graph does not touch zero—his appeasatory behavior is nearly always higher than his pre-Chief Justice levels of appeasement. Both Douglas’ and Rehnquist’s level of appeasement over time suggest that the model is picking up on trends that make intuitive sense. Justice Rehnquist’s versus other justices’ appeasement
Looking at individual cases with the lowest levels of appeasement, we observe further evidence that our measure of appeasement is capturing variations in conciliatory speech. Justice Brennan’s lone dissent in the case Francis v. Henderson (1976) represents one of the lowest appeasement scores in our dataset. In this year, Justice Brennan is the justice farthest left in his Martin-Quinn score, also being furthest away ideologically from the majority in this specific case. Known for supporting individual rights (Eisler, 1993), he takes great issue with what he sees as the majority’s restrictions on federal habeas corpus jurisdictions related to Fay v. Noia (1963). Additionally, Justice Brennan is known for favoring criminal and minority defendants (Eisler, 1993). In this specific case, the petitioner is protesting the exclusion of African Americans from the grand jury that indicted him (Francis v. Henderson, 1976). This dissent is rife with harsh statements against the majority opinion, including that “the Court, in addition to failing to supply any justifications for this requirement, fails to supply any content to it” and that such “oversights” are especially ironic in light of the Court’s recent admonition that “[o]ur institutional duty is to follow until changed the law as it now is, not as some members of the Court might wish it to be” (Francis v. Henderson, 1976, p. 547). Considering personal characteristics about Justice Brennan, as well as his ideological place on this Court as the furthest left-wing justice, it is understandable that we observe his harsh condemnation of the majority opinion, with less regard for rhetorical tact.
Examining cases with the highest levels of appeasement, we see that the characteristics of non-majority opinions diverge from cases like Brennan’s dissent in Francis v. Henderson (1976). Justice Blackmun’s concurring opinion in United States v. Ward (1980) is first different as it is a concurrence rather than a dissent—with concurring opinions intuitively tending to rate higher in levels of appeasement than dissenting opinions. Unlike Justice Brennan, who is widely regarded as a staunch liberal, Justice Blackmun’s trajectory on the Court demonstrates a more complex ideological evolution. Initially appointed by Richard Nixon and expected to be a conservative vote, Justice Blackmun became increasingly aligned with the Court’s liberal wing over time (Greenhouse, 2007). In United States v. Ward (1980), Justice Blackmun wrote separately to emphasize nuanced points about the statutory framework, while largely agreeing with the majority. In one such concession, Justice Blackmun states that “in the absence of evidence that excessive penalties actually have been assessed, I would be inclined to regard their likelihood as remote. For these reasons I agree with the Court that only the fifth Mendoza-Martinez factor whether the behavior to which the sanction applies is already a crime.” Justice Blackmun’s tactful approach, carefully balancing agreement with subtle critiques, reflects characteristics of appeasement (United States v. Ward, 1980, p. 257). It is noteworthy that this case dealt with a relatively technical issue under the Federal Water Pollution Control Act, which may have allowed for more measured and cooperative rhetoric than highly salient or ideologically charged cases (United States v. Ward, 1980). Cases like this highlight how appeasement captures not just rhetorical strategy, but also broader ideological dynamics and potential movement within the Court, as Justice Blackmun ultimately became positioned as one of the more liberal justices on the Court (Greenhouse, 2007).
Measure Comparison and Extension
To further validate our measure, we analyze it alongside an extant measure of the harshness of rhetoric in the Supreme Court. Zilis and Wedeking’s (2020) paper serves as an excellent point of comparison. Their measure operationalizes disagreeable rhetoric, which they describe as consisting of “language with harsh, unpleasant, or negative connotations” (Zilis & Wedeking, 2020, p. 1). Their disagreeable rhetoric measure is constructed using a factor analysis score of both the “nasty” and “unpleasant” subdictionaries from the Dictionary of Affect in Language (DAL), and the “negative emotion” category from the Linguistic Inquiry and Word Count (LIWC) package (Zilis & Wedeking, 2020). Our appeasement score has a predominantly negative relationship with the outcomes of their disagreeable rhetoric measure, though Zilis and Wedeking (2020) use a dictionary approach, while we utilize context-sensitive machine learning architecture.
Comparison of Measurement Outcomes
Note. *p
For Zilis and Wedeking’s (2020) first hypothesis, we did not find a significant relationship between personal issue importance and appeasement (t = −1.384, p = 0.167, Figure 3). Though non-significant, this variable is negatively associated with disagreeable rhetoric in this context. In situations that tend to stoke inflammatory language, disagreeable rhetoric and appeasement represent separate constructs, with the disagreeable rhetoric measure operating along a different dimension from appeasement. However, as we demonstrate later, appeasement may be more effective at capturing deeper contextual differences and strategic considerations within non-majority opinions—illustrated through individual case-level examples. Appeasement, disagreeable rhetoric, and personal issue importance
For Zilis and Wedeking’s (2020) second hypothesis of disagreeable rhetoric increasing as the majority coalition size decreases, there is a strong significant result for appeasement. Specifically, as the majority coalition size increases, ratings of appeasement tend to greatly increase (t = 10.07, p < 0.001, Figure 4), even more so than disagreeable rhetoric tends to decrease (t = −3.544, p < 0.001; Table 2 measures are standardized). This result confirms that, not only are appeasement and disagreeable rhetoric negatively associated in relation to these hypotheses, but appeasement may be a measure that is particularly sensitive to strategic dynamics, such as considerations about the shape and size of the majority coalition. Appeasement, disagreeable rhetoric, and majority vote coalition
Pertaining to our exploratory hypotheses relating to ideology, we examine the variable of “ideological accommodation” in Zilis and Wedeking’s (2020) dataset. This variable is created by the average standard deviation of Martin-Quinn scores (Martin & Quinn, 2002), to calculate the breadth of different ideologies accommodated in the majority opinion. Results suggest that appeasement significantly increases as the majority coalition broadens (t = 4.920, p < 0.001), while disagreeable rhetoric decreases (t = −4.818, p < 0.001). These results (shown in Figure 5) provide further evidence that appeasement represents a tactful response to the majority—with opinion writers being more appeasatory in cases where the majority represents a more ideologically diverse coalition of justices. This is likely because justices have to contend with colleagues who are more closely oriented to themselves, and may also need to work strategically to rebut a more complex argument from the majority. Appeasement, disagreeable rhetoric, and ideological accommodation
Appeasement’s ability to capture ideological considerations is further explained by the variable’s relationship to Martin-Quinn scores—measured as these scores’ absolute value, which demonstrates ideological distance from the center. Our exploratory hypothesis is that a justice’s own ideological extremity might lower their tendency to be appeasatory. Our results suggest that this is the case, with more extreme justices rating significantly lower in appeasement (t = −2.953, p = 0.003; Figure 6), despite no significant relationship between Martin-Quinn and disagreeable rhetoric (t = −1.192, p = 0.234). These results suggest a noteworthy break between ours and Zilis and Wedeking’s (2020) measures. Our measure appears to be more strategically and ideologically attuned, whereas Zilis and Wedeking’s (2020) may be more effective at analyzing rhetorical tendencies within specific cases that are personally important to justices. In the next section, we discuss potential reasons for this divergence. Appeasement, disagreeable rhetoric, and ideology
Appeasement Versus Disagreeable Rhetoric, Case-by-Case
To study potential causes for divergence between the appeasement and disagreeable rhetoric measures, we examine examples of individual cases where the measures are aligned or opposed. Evidence suggests our measure may differ from Zilis and Wedeking’s (2020) by focusing on the strategic considerations for how justices direct their language towards one another, rather than the way in which they use language when they are highly invested. As shown above, it is likely that our appeasement score tends to be negatively associated with many outcomes related to disagreeable rhetoric. However, we find that our model is sensitive to certain instances more than others, due to differences in methodology and operationalization. As a result, there are cases where the Zilis and Wedeking (2020) measure rates an opinion as highly disagreeable, but our model labels it as more appeasatory.
The first case we examine is Justice Stevens’ dissenting opinion in McMillan v. Pennsylvania (1986). This is an opinion which our appeasement measure rates high in appeasement (1.59, standardized), while Zilis and Wedeking’s (2020) measure rates it high in disagreeable rhetoric (2.29, standardized). This case examines a Pennsylvania statute that creates mandatory sentencing of five years imprisonment if the defendant could be seen carrying a firearm during the offense, without needing to prove this sentencing factor beyond a reasonable doubt. The Court upholds that visible possession of a firearm is a sentencing factor and not an element of crimes, following from the Pennsylvania Act (McMillan v. Pennsylvania, 1986).
In his dissenting opinion, Justice Stevens makes clear concessions to the majority. He states “It is true, as the Court points out, that the applicability of the reasonable doubt standard…has always been dependent on how a State defines the offense that is charged in any given case” (McMillan v. Pennsylvania, 1986, p. 98). At another point, he makes another clear concession, stating that “It is true, as the Court points out, that the enhanced punishment is within the range that was authorized for any aggravated assault.” (McMillan v. Pennsylvania, 1986: 103). Throughout this case, Justice Stevens demonstrates an evenhanded approach—calmly restating the Court’s arguments, and acknowledging when their points are logically valid, if not sound. Here, our measure of appeasement demonstrates its novelty in measuring Justice Stevens’ cogent dissenting style.
Another dissent where our measure diverged sharply from Zilis and Wedeking’s (2020) measure is Justice Powell’s dissent in Coker v. Georgia (1977). Our measure rated this opinion as high in appeasement (2.15; standardized), while Zilis and Wedeking’s (2020) measure rated it as high in disagreeable rhetoric (4.26, standardized). Despite the high score of disagreeable rhetoric, we find evidence that Justice Powell’s partial concurrence represents a relatively measured tone (and thus high score of appeasement). Justice Powell explains his reasoning carefully, stating: “I concur in the judgment of the Court on the facts of this case and also in the plurality’s reasoning supporting the view, that ordinarily death is disproportionate punishment for the crime of raping an adult woman. Although rape invariably is a reprehensible crime, there is no indication that petitioner’s offense was committed with excessive brutality” (Coker v. Georgia, 1977: 585).
It may be the case that Zilis and Wedeking’s (2020) measure rates Justice Stevens’ and Justice Powell’s dissenting opinions higher in harsh rhetoric because each case covers topics including the death penalty, violence, and rape—which likely appear prominently in their negative terms dictionary. Other examples of cases that Zilis and Wedeking’s measure rates highly disagreeable, that our measure rates as highly appeasatory, include non-majority opinions in Victor v. Nebraska (1994) (Ginsburg dissent) and Estelle v. Gamble (1976) (Stevens dissent). Like Coker v. Georgia (1977) these cases deal with violent topic matters such as “cruel and unusual punishment,” “wanton infliction of pain,” “murder,” and “death sentences”—which may lead them to receive extremely high disagreeable rhetoric scores. We suspect these cases may rate highly on Zilis and Wedeking’s (2020) measure because of the topic matter they deal with, rather than the justice’s style of argument construction. This is a shortcoming of word-counting models, which may not be mitigated with a machine learning model such as our measure of appeasement.
Conclusions
The ways that opposing justices communicate with the majority can be used to instruct the Court and the public, in a way that can have lasting consequences in the legal sphere—especially during periods of institutional turbulence. As Justice Kavanaugh stated, there are many factors that influence the way that justices state their opinions (Notre Dame Law Review, 2023). In this paper we find empirical support for Justice Kavanaugh’s assertion that justices tailor their opinions to their many audiences, including other justices. Particularly, we argue for the use of the construct of appeasement to aid understanding of how political elites weigh strategic, ideological considerations. This finding highlights the value of examining how justices structure disagreements, rather than focusing solely on the tone of their rhetoric. Ideological tensions are applicable to all branches of government, and represent an important future route for applying this appeasement measure. However, as a preliminary case study, our analyses show that this construct can be successfully operationalized even in extremely dense and complex legal text.
This measurement has a range of potential applications. Because judicial appeasement directly measures how the justices interact with each other, this may allow us to examine how varying levels of appeasement are received. A construct like appeasement may aid in predicting justices who change ideologically over time, have warmer relations to their colleagues, and are more likely to coalesce to majority demands. Understanding these dynamics is particularly topical in this political moment, as partisans scrutinize the emergence of potential swing justices, including Justice Amy Coney Barrett (Marimow, 2025). Outside of legal scholarship, appeasement may be even more pertinent as a construct, given the broader dynamics of congressional voting behavior, increased congressional responsiveness to constituencies, and changing democratic norms. As there are debates in both chambers of Congress, there will necessarily be concessions and non-concessions over the course of debate. Indeed, the audiences and goals of members of Congress might be even more salient than that of the Supreme Court, as there is an electoral threat. In this context, these factors may allow for additional extensions of the theory, examining additional strategic considerations within rhetoric-rich environments.
At its core, appeasement measures how political elites shape their linguistic strategy to appeal to an audience. It is a measure of how willing or unwilling they are to temper their own political and ideological preferences to maintain a sense of evenhandedness and legitimacy. These decisions are made by all politicians and political elites, whether they sit on the highest court of the land or the local school board. Future applications of appeasement should explore these non-judicial avenues in greater detail. Examples of these applications include floor speeches in Congress, presidential rallies and debates, political campaign ads, and any other way that political elites message through language. Appeasement extends our understanding of political rhetoric, operationalizing a key tool that political elites use to shape how they direct their arguments towards their colleagues and the public.
Footnotes
Acknowledgements
The authors are grateful for the continued support and feedback from Paul Collins, Douglas Rice, Elli Menounou, Gabriel Vergara, and Justin Burnworth.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.