Does Party Identification Matter for Deliberation? Evidence from the Poland Speaks Experiment

Abstract

Deliberation among the public appears wanting, even in many of the world’s established democracies. This apparent lack of mutually respectful conversation among citizens about politics involving a give-and-take of reasons is often ascribed to growing affective polarisation. The more the citizens come to think of each other as belonging to opposing groups, the less likely it allegedly becomes that they will show respect towards each other or exchange arguments while talking politics. However, the empirical support for this common supposition remains tentative, as prior research suffers from potential endogeneity problems and selection bias. To address these limitations, we introduce a novel experimental design involving an imagined conversation on refugee policy in Poland. Our experimental test shows that, on average, participants’ inclination to deliberate did not significantly differ based on whether they imagined talking to someone from an ingroup or to someone from an outgroup instead. Our findings thereby suggest that the relationship between group identification and public deliberation might not be as straightforward as is often assumed. At least in some contexts, a lack of mutual group identification does not spell disaster for deliberation.

Introduction

Public deliberation is often considered a key ingredient to making democracy work but it equally often seems in short supply. Political theorists and pundits routinely suggest that affective polarisation is a root cause of the apparent lack of public deliberation, that is, mutually respectful conversation among citizens about politics involving a give-and-take of reasons (e.g. Dryzek et al., 2019). The argument is generally that the more the citizens come to think of each other as belonging to opposing groups (e.g. as members of different political parties), the less likely it becomes that they will show respect towards each other or exchange arguments while talking politics. Public deliberation, then, is thought to require a common group identity that ‘[holds] citizens together or [makes] them feel that they are engaged in a common political enterprise’ (O’Flynn, 2006: 5).

There is ample observational evidence that when citizens identify with the same group they become inclined to deal with their political differences through deliberation. For example, the literature on social movements suggests that group identification correlates positively with support for deliberation (Della Porta, 2013) and that deliberation tends to subside precisely once a movement’s groupishness declines because factions emerge (Min, 2015) or, for instance, a movement expands and comes to include ‘newcomers’ (Polletta, 2002). Similarly, studies of town (hall) meetings and deliberative forums also tend to report a positive relationship between group identification, on the one hand, and deliberation, on the other hand (Cossart et al., 2019; Mendelberg and Oleske, 2000). Myers (2022) shows, for example, that social identification in Rural Climate Dialogues in Minnesota promoted deliberation, and in a large-scale study of 19 deliberative forums Niemeyer et al. (2024) demonstrate that reasoning tends to improve when participants have bonded prior to political discussion. ¹

However, despite the consistency of the finding that mutual group identification correlates positively with public deliberation, the observational nature of prior studies gives rise to two potential endogeneity problems: omitted variable bias and reverse causality. Regarding the former, it often remains difficult to ascertain to what degree it is mutual group identification that drives public deliberation or a third variable such as group composition instead. Regarding the latter, it is not obvious that mutual group identification enhances public deliberation rather than vice versa. Deliberation is commonly expected to reduce animosity between groups and there is empirical evidence suggesting that when citizens from different linguistic or partisan groups deliberate together they become more appreciative of the outgroup (Caluwaerts and Reuchamps, 2014; Fishkin et al., 2021).

In addition, most empirics on group bonding and subsequent identification, on the one hand, and public deliberation, on the other hand, come from structured settings like deliberative forums and town hall meetings that tend to have elaborate discussion rules and facilitation in place to enhance deliberation (Tanasoca, 2020). ² This may have introduced selection bias in that we observe a positive correlation between a shared group identity and public deliberation only because settings with deliberation-enabling rules and/or norms are generally selected for empirical inquiry.

We address these limitations by means of a pre-registered survey experiment on the discussion of the Polish government’s policy regarding the acceptance of refugees coming from the Middle East (N = 626). We introduce a novel experimental design – what we call the ‘Poland Speaks’ experiment – in which citizens are asked to imagine a political conversation lacking any specific discursive rules or facilitation. By manipulating the group membership of participants’ interlocutor, it provides a first experimental test of the common supposition that mutual group identification makes deliberation tick.

Hypotheses about Deliberation

We define deliberation as talk involving mutual respect and reasoning about (potential) political decisions. On the one hand, deliberation involves the production and evaluation of reasons as to why something should (not) be done (Mercier and Landemore, 2012). Deliberation in this sense moves beyond mere claim making, requiring speakers to justify the claims they make, and it moves beyond a mere consumption of arguments, requiring speakers to evaluate critically the claims and reasons they receive. On the other hand, it involves showing ‘respect for the integrity of all involved’ (Rosenberg, 2014) in the form of civil communication and attentive listening to what others have to say (Bächtiger and Parkinson, 2019). This excludes from the definition talk in which those involved ‘ignore or dismiss or patronize [others’] statements and expressions’ (Young, 2000: 55).

Based on the extant literature, we expect that people’s propensity to deliberate will increase (decrease) when they receive a cue that their interlocutor is a member of their ingroup (an outgroup). Given that parties are arguably the ‘prime example of (. . .) coalitional affiliation’ in mass democracies (Boyer, 2018: 42), we operationalised group membership and identification in terms of political party affiliation. As deliberation matters most in situations of opinion diversity (Thompson, 2008), we study our expectations in a context of disagreement:

• H1: When in disagreement with someone else, a cue of ingroup identification increases a person’s propensity to deliberate.

• H2: When in disagreement with someone else, a cue of outgroup identification decreases a person’s propensity to deliberate.

The Poland Speaks Experiment

To test our hypotheses, we ran an online survey experiment in Poland in November 2022: the Poland Speaks experiment. We pre-registered the experimental design, materials and hypotheses on the Open Science Framework at https://osf.io/y7rtx (for deviations, see Supplementary Information 7). We relied on the company Dynata to administer the survey. The sample (N = 626) is roughly representative of the population in terms of gender and age categories but generally higher educated in terms of having completed tertiary education (+19.30%), more likely to have voted at the parliamentary elections of 2019 (+26.76) and less likely to have voted for Law and Justice (–16.11%; Supplementary Information 2).

Experimental Design

Following recent research (Eveland et al., 2020), the experiment relies on an imagined discursive interaction. We asked participants to imagine that they are having a conversation with someone else about a particularly salient and divisive policy issue: the Polish government’s policy regarding the acceptance of refugees coming from the Middle East. We specifically asked them to imagine that they were having this conversation in the context of a fictitious My Country Talks event we called ‘Poland Speaks’. My Country Talks (https://www.mycountrytalks.org) works much like a dating app, with an algorithm linking up citizens with different political views for one-on-one conversations about pressing political issues in their country. Figure 1 displays the English translation of the introductory vignette to Poland Speaks (see Supplementary Information 3 for original).

Figure 1.

Introductory Vignette (Translation).

The introductory vignette also included the manipulation. It consisted of a cue of group membership in terms of political party affiliation. ³ There were three between-subject conditions. Participants’ conversation partner either supported their favourite political party (ingroup condition) or their least favourite party instead (outgroup condition) based on their answer to the question ‘Below is a list of the 12 biggest political parties in Poland. Please select your most favorite party as well as your least favorite party’. ⁴ In a control condition, there was no information on the partner’s party affiliation. A series of checks shows that the manipulation worked as intended: participants in the ingroup condition thought they had more in common and identified more strongly with their partner than did participants in the outgroup condition (Supplementary Information 4).

Figure 2 shows the logic of the experiment. It consisted of participants viewing two videos in which their conversation partner voiced an argument and then answering several questions about how they would respond. Just like My Country Talks brings together citizens with diverging political views to talk politics, the conversation partner always voiced an argument going against participants’ expressed opinion on the refugee policy of the Polish government, based on their answer to the question ‘Some people say that the Polish government should receive more refugees from countries in the Middle East. Do you, on balance, AGREE or DISAGREE with this view?’ (cf. Jackman and Sniderman, 2006). One of the arguments related to potential cultural consequences of refugees coming to Poland, the other to potential economic consequences. We randomised the order of the arguments. For details on the video stimuli, see Supplementary Information 5.

Figure 2.

Flowchart of the Experimental Design.

Dependent Variables

To measure participants’ propensity to deliberate, we asked participants after each video to rate four potential replies (Nr. 1–4) in terms of how likely it would be for them to respond in the respective way to their conversation partner on a scale from 1 (very unlikely) to 7 (very likely). The replies were always displayed in a random order (see Supplementary Information 6 for the replies rated by participants).

We selected the replies from what participants had written down in response to the same two videos in a pilot study, meaning that the presented options were realistic statements previously expressed by another Polish citizen (see Supplementary Information 1). Following our conceptualisation of deliberation as communication displaying reasoning and mutual respect, the replies were purposefully selected to reflect all possible combinations of a presence or absence of reasoning, on the one hand, and respectful language, on the other. We made sure the replies always related to the respective video being viewed by the participant (i.e. in favour of or against the cultural or the economic argument) and maintained consistency in the replies presented to participants in favour of and against the government’s refugee policy by minimally rephrasing the original reply where necessary.

The dependent variables are based on likelihood ratings provided by the participants after each of the videos (Supplementary Information 6). Deliberation is the score on the reply (Nr. 4) displaying both characteristics of deliberation (i.e. reasoning and respect), subtracted by the score on the reply (Nr. 1) showing no signs of deliberation. It thereby represents participants’ expressed likelihood to engage in deliberation rather than to refrain from it altogether (M = 0.95, SD = 2.16 (cultural argument); M = 0.88, SD = 2.02 (economic argument)). Reasoning is the average of the two replies including a form of reasoning (Nr. 2 and 4) subtracted by that of the two replies lacking reasoning (Nr. 1 and 3; M = 0.45, SD = 1.56 (cultural argument); M = 0.14, SD = 1.33 (economic argument)). Respect is the average of the two replies lacking disrespectful language (Nr. 3–4) subtracted by the average of the other two replies that did include disrespectful language (Nr. 1–2; M = 0.50, SD = 1.56 (cultural argument); M = 0.74, SD = 1.48 (economic argument)).

In addition, we measured respect by monitoring whether participants correctly answered a multiple-choice question about what their conversation partner had said at the very end of the survey to gauge whether they had listened to her (Listening; on recall as a measure of listening, see Scudder, 2021). It is coded as 1 for correct recall and as 0 otherwise (M = 0.58, SD = 0.49).

As scores on the dependent variables do not strongly correlate across the two videos (Figure A4), we conduct the analyses separately for each video.

Results

Figure 3 summarises the results (for details, see Supplementary Information 8). We do not find statistically significant effects of the group membership cues on Deliberation for either of the two arguments. We also do not find any significant effect of the experimental manipulation on Reasoning, Respect or Listening (Tables A8-A9). Additional analyses suggest that we can be confident that group membership cues (here: party identity) did not have the large effect on participants’ propensity to deliberate that one would expect to find based on prior observational studies (Supplementary Information 8).

Figure 3.

Main Results by Argument Type. (a) Cultural argument. (b) Economic argument.

Conclusion

Theories of deliberation and observational evidence have long suggested that the less citizens identify themselves with the same group, the less likely they are to exchange arguments and show respect to each other when talking politics. This suggests that affective polarisation, with citizens sorting themselves into mutually opposing groups, is a key cause of the apparent scarcity of public deliberation in many democracies today. Here, we have tested this claim by means of a novel experimental design: the Poland Speaks experiment. Contrary to received wisdom, we find robust evidence that, on average, participants’ inclination to deliberate did not significantly differ based on whether they imagined talking to someone from a partisan ingroup or to someone from a partisan outgroup instead.

Nevertheless, our findings are limited in scope, based on imagined discussions on refugee policy in a My Country Talks event in Poland. Importantly, while we tried to simulate the interactive nature of political discussions by using two video-reply sequences (see Figure 2), participants did not actually speak or receive a reply to what they might have said. This may have affected their propensity to deliberate, especially because it meant they did not have to worry about potential social (e.g. group exclusion) or discursive sanctions (e.g. insults). Ways forward in this respect include the use of a pre-programmed chatbot to interact with participants (Kim et al., 2021) or the organisation of real-life, unfacilitated discussions between participants (Strandberg et al., 2019). Furthermore, ideally, a series of experiments will test to what degree our findings replicate across a range of different ingroup/outgroup manipulations (beyond partisan group identity). Finally, as prior research has documented individual-level differences in people’s propensity to deliberate (e.g. Gerber et al., 2018), larger and/or targeted samples could help to examine the potential heterogeneity of group identification effects based on differences in individual-level characteristics such as gender.

Supplemental Material

sj-docx-1-psw-10.1177_14789299241245609 – Supplemental material for Does Party Identification Matter for Deliberation? Evidence from the Poland Speaks Experiment

Supplemental material, sj-docx-1-psw-10.1177_14789299241245609 for Does Party Identification Matter for Deliberation? Evidence from the Poland Speaks Experiment by Ramon van der Does and Honorata Mazepus in Political Studies Review