Abstract
We suggest a new democratic decision-making (or recommendation-making) device for divided societies that may be added to the democratic toolkit. Imaginative Policy Surveys in divided societies seek to combine the advantages of conventional attitude surveys (ability to generalise to the wider population) with some of the advantages of deliberative mini-publics (citizens learn about policy options and consider the perspective of members of the ethno-national out-group). Imaginative Policy Surveys consist of a conventional survey design with two added features: videos providing information and arguments and an imagined policy dialogue with an out-group member. We test the feasibility, effect and perceived legitimacy of Imaginative Policy Surveys in the deeply divided context of Northern Ireland, drawing on evidence from two experiments. We conclude that Imaginative Policy Surveys in divided societies are feasible, have a slight positive pro-compromise effect, and are perceived to be a legitimate decision-making mechanism to an equal extent by rival ethno-national groups.
Introduction
We present a novel way of facilitating considered, compromise-oriented, citizen decision-making (or recommendation-making) on contentious issues in polities riven by deep division. An Imaginative Policy Survey (IPS) in a divided society is a conventional attitude survey of a representative sample of the population, with added design features to enable detailed policy consideration by participants prior to indicating their preferred policy option on a given divisive issue. The added design features include participants watching video presentations (providing background information and arguments from both sides of the divisive issue) and participants engaging in an imagined policy dialogue with an out-group member (to promote engagement with alternative viewpoints held by members of the rival religious or ethno-national group).
An IPS combines the strength of the large-N survey approach to attitude measurement (which seeks to generalise precisely from the sample of survey respondents to the wider population) with some of the strength of the deliberative mini-public approach to attitude formation (which seeks to ensure that citizens are informed about policy proposals, understand the arguments in favour and against, and engage with perspectives different from their own). This combination of strengths enables an IPS to produce a statistically defensible precise estimate of what the entire population’s preferred policy option would be if it had engaged in the same level of consideration of the policy issue that the sample of survey respondents had. We argue that the IPS approach is a useful one to add to the democratic toolkit, especially for divided places that are prone to inter-party gridlock on policy issues and could benefit from legitimate ways of delegating recommendation-making or decision-making to citizens to avoid system breakdown.
We begin by outlining the main features of an IPS in a divided society. We then briefly describe our case study of Northern Ireland. We then address, in the Northern Ireland context, three empirical questions relating to the feasibility, effect and perceived legitimacy of an IPS. First, how easy or difficult is it to successfully conduct an IPS? We assess the real-world feasibility of an IPS by describing how we organised an IPS in Northern Ireland on the issue of flag display. Specifically, we report the quality of participant engagement with the simulated policy discussion (imagined dialogue) with an out-group member, as well as engagement with the video features. Second, what substantive effect, if any, does an IPS have on attitudes towards a contentious issue in a deeply divided place? We examine whether an IPS encourages compromise outcomes. If so, which aspects of our design features are responsible for generating a pro-compromise effect: video provision of information and arguments or imagined dialogue? We do this by reporting the results of an experiment we embedded in our IPS on flag display. Third, if an IPS is used to enable citizens to make recommendations or decisions on divisive issues that have gridlocked the political system, how would the general public react? The real-world plausibility of any novel system of citizen-based decision-making rests in significant part on whether the general public find the process reasonable and acceptable. We assess the perceived legitimacy of IPSs by reporting the results of an experiment on citizens’ perceptions of the legitimacy of IPS decision-making.
Questions one and two (relating to feasibility and effect) are addressed in Study One which examines an IPS on flag display with an embedded experimental design, and question three (relating to perceived legitimacy) is addressed in Study Two, which is a subsequent distinct experiment assessing citizens’ perceived legitimacy of IPS-based decision-making. In short, with respect to feasibility, we find that an IPS is reasonably straightforward to operationalise and implement in a divided society, although we identify that a significant proportion of participants did not positively engage in the imagined dialogue exercise (instead either neutrally or negatively engaging). In terms of impact on attitudes, we observe a limited pro-compromise effect of engaging in the IPS, and this effect is substantively noteworthy among those who did positively engage in the imagined dialogue. In terms of perceived legitimacy, we find that the general public positively evaluate IPS decision-making. Importantly, there are approximately equal levels of perceived legitimacy across the different ethno-national groups, hence avoiding the problem of a democratic mechanism being supported by one group in a divided society and regarded with suspicion by the other group. We tease out, in our concluding section, the implications of our findings for the potential of, and limitations of, IPSs in Northern Ireland and other deeply divided places.
IPSs in divided societies
Conventional opinion polls and attitude surveys are a valuable way of generating precise estimates of a population’s view on a particular policy issue. But few would argue that survey evidence should be used to directly make policy recommendations or policy decisions. This is mainly because views on policy issues expressed by respondents in opinion polls are likely to be only minimally considered; they represent an instant or ‘off the cuff’ reaction to a question (Table 1). In a direct response to this problem of limited citizen consideration of policy issues in surveys and opinion polls, some deliberative democrats advocate deliberative mini-publics, such as deliberative polls and citizens’ assemblies, which are designed to enable citizens to make policy recommendations after carefully considering a policy issue (Farrell et al. 2019; Fishkin, 2009; Renwick et al. 2018; Warren and Pearse, 2008). Such mini-publics vary in terms of scale and duration, among other design choices (see Setälä and Smith, 2018). In most cases, the participating citizens are provided with presentations by experts and advocates in which information on the issue is offered and different arguments for or against different policy options are articulated. Participating citizens also engage in facilitated deliberation with each other, in the context of small group discussion of the relative merits of different proposals. Participants finally vote on the issue, choosing one or other of the different policy options. A downside of deliberative mini-publics (whether citizens’ assemblies, deliberative polls, or another variant) is that the resulting estimates of opinion in favour or against a particular proposal are much more difficult to generalise than estimates from a systematic opinion poll or survey (Table 1, row 2). In part this is because the number of participants is much smaller, often consisting of approximately 100 or 150 randomly selected citizens in citizens’ assemblies (Setälä and Smith (2018: 301) or up to 200 or so in deliberative polls (e.g. Fishkin, 2018: 323–324), in contrast to the typical number of participants of 1000 upon which opinion polls and surveys are based. Additionally, the mode of deliberation (face-to-face discussion) used in deliberative polling and citizens’ assemblies involves the participants affecting and influencing each other. From a statistical standpoint, this undermines the credibility of the sample as comprising participants who are independent of each other, hence impeding generalisation from the randomly selected participants to the wider population.
Characteristics of an Imaginative Policy Survey in a Divided Society compared with opinion polls and mini-publics.
Here, we suggest maintaining the virtues of the conventional atomised large-N sample used in opinion polls and injecting into the conventional survey methodology some additional features which seek to enhance citizen consideration (Table 1, row three). In line with deliberative polling and citizens’ assemblies, we regard consideration as requiring (at least) three features. Citizens should be provided with information explaining the background context of the divisive issue in question. Citizens should also be provided with balanced arguments on both sides of the debate. Additionally, citizens should be facilitated in discussing their own opinion with someone who holds different views, in order to increase appreciation of different perspectives.
How can such provision of information and arguments, and citizen consideration of the issues, be achieved in the context of a conventional survey in a divided society? We suggest two distinct additions to the conventional attitude survey design. First, to provide respondents with information and arguments, we build on the work of Steven Kull’s Programme for Public Consultation (PCC) at the University of Maryland. PCC conduct nationally representative surveys that are focused on specific policy issues such as: increased federal funding for Internal Revenue Service tax enforcement; family and medical leave; and proposals to reform solitary confinement. On each issue, survey respondents are provided with a description of a specific policy proposal. Respondents are then provided with a number of arguments in favour and against the proposal and are asked to evaluate these arguments. They are then asked to indicate how acceptable or unacceptable they find the policy proposal, and finally to indicate whether they support or oppose the proposal (Kull, 2022). PCC argue that conducting a public consultation survey that focuses on a detailed examination of a specific policy issue is a significant improvement upon conventional opinion polls because of the greater detail that can be provided and greater consideration that can be given to the issue by respondents (Kull, 2022). Similar to Kull’s approach, we provide survey participants with a detailed description of a divisive policy question, and the possible policy options and arguments in favour and against each policy option. Instead of providing this description in written format, as Kull does, we enhance accessibility by provision via an accessible video that participants watch.
Our second addition to the conventional survey design seeks to enable citizens in deeply divided places to engage with the perspectives of ethno-national out-group members. Here, we build on the work on imagined contact in social psychology (Crisp and Turner, 2009). Our use of an atomised individual-level survey prohibits the kind of interactive small group discussion that characterises deliberative polls and citizens’ assemblies. Hence, rather than real discussion we apply the imagined contact approach, and we script guidelines for an imagined discussion that participants are invited to have with an out-group member. The resulting imagined policy dialogue seeks to achieve at least some of the value of real group-based discussion, but in a way consistent with individualised response to an atomised large-N survey.
The aim of group-based discussion in conventional citizens’ assembly and deliberative polling exercises is to allow for a discursive interchange between citizens after they have become informed about an issue via presentations by experts and advocates on both sides of the argument (Smith, 2009). Ideally, in the group-based discussion, there is a courteous, respectful and considered debate about the relative merits of different policy options. The discussion would be characterised by a to-and-fro of citizens’ contributions to the debate in which citizens take on board the views of other citizens while articulating and defending (or possibly changing in the light of the discussion) their own positions.
We propose that each IPS participant be asked to imagine that they are discussing the divisive policy issue with another person from the religious or ethno-national out-group, who is briefly described and holds viewpoints different from the participant. The participant is asked to imagine what (for example) ‘Frank’ would think about policy option A, what the participant would say in response to Frank, how Frank would in turn respond, and what the participant would say back in turn. The same occurs with respect to policy option B (and possibly options C, D, etc., depending upon the context). Essentially, this mental simulation task seeks to operationalise as closely as possible the kind of dialogue that would ideally occur in a considered exchange where one is able to understand, articulate and reflect upon the perspectives and views of a different other and respond in an engaged and constructive dialogue.
We suggest that this imagined dialogue approach is particularly suited to the deeply divided context in which the achievement of compromise on contentious issues is valuable for system stability. The rationale for imagined contact is that it can provide the benefits of inter-group interaction without any of the risks that may be associated with face-to-face antagonism, especially in deeply divided societies (Dessel and Rogge, 2008). In a meta-analysis of over 70 studies, Miles and Crisp (2014) found strong evidence in support of imagined contact leading to more conciliatory attitudes. Furthermore, in a specifically political application of imagined contact in the USA, Warner and Villamil (2017) found that imagined contact had a politically depolarising effect: imagined contact reduced negative affect towards the political out-group, and indirectly reduced the attribution of malevolence to the out-group and the acceptance of political violence. This is important in our context of a deeply divided place where reaching inter-group decisions is difficult and often leads to system-destabilising gridlock due to a reluctance to compromise. The achievement of compromise and agreement is not typically a criterion by which the procedural quality of decision-making is assessed. However, in a deeply divided place that is prone to policy gridlock, compromise as an outcome may often be a necessary component of political progress.
While our imagined dialogue approach may appear radically different from the face-to-face real discussions that occur in a deliberative mini-public, we suggest that our proposed approach may achieve at least some of what a real-world discussion seeks to achieve in a deliberative democracy exercise. For example, Owen and Smith (2015: 219) argue that a core feature of the process of deliberative democracy is that it ‘involves citizens reflectively taking up each other’s standpoints’. Our imagined dialogue approach directly invites participants to articulate and respond to an alternative perspective from an ethno-national out-group member. Furthermore, Goodin (2000: 83) suggests that ‘deliberation . . . is less a matter of making people “conversationally present” and more a matter of making them “imaginatively present” in the minds of the deliberators’. Our imagined policy dialogue with an out-group member seeks to achieve exactly this.
We are not suggesting that an IPS can capture anywhere near the full detail and complexity of a real-world deliberative poll or citizens’ assembly. However, we do suggest that the provision of videos with information and argument, and the engagement in an imagined policy dialogue makes an IPS qualitatively different from a conventional survey or opinion poll. An IPS seeks to combine the strengths of conventional surveys (to credibly make precise generalisations) with at least some of the strengths of deliberative polling or citizen assemblies (to achieve at least a basic level of informed consideration of the issue).
We suggest that it is the ability to make a precise generalisation about the informed consideration of the participating sample that is the real policy-relevant strength of an IPS. If a majority of participants in a conventional survey sample supports policy X, it is difficult to argue that that policy should be implemented given (as described earlier) the arguably ill-considered or semi-considered nature of the responses. If a majority of the 100 people in a deliberative mini-public support policy X, it is difficult to argue that that policy should be implemented if we are unsure how effectively we can infer from the 100 to the wider population. However, if a majority of participants in the IPS agree with policy X, it is possible to argue (as with any properly conducted survey) that a majority in the wider population would also support policy X if they had engaged in the same level of learning and consideration. Thus, an IPS seeks to capture accurately and precisely what the considered view of the whole population would be.
Divided places that suffer from policy gridlock due to polarised opinion on divisive issues could benefit from a depolarising compromise-oriented citizen decision-making device that can make precise estimates of considered public opinion. Northern Ireland is one such divided place. We now describe the case and then empirically demonstrate the extent to which an IPS in the Northern Ireland case is feasible, has a compromise-oriented effect, and is seen by opposing ethno-national groups as legitimate.
Northern Ireland case study
Northern Ireland is a deeply divided place with a system of consociational power-sharing that has many benefits but is particularly prone to gridlock. Northern Ireland is hence a useful case for considering novel ways of engaging citizens in influencing decision-making to overcome elite policy division. The 1998 Belfast / Good Friday Agreement established a power-sharing system with election by proportional representation and distribution of ministerial portfolios among the parties from both communities along d’Hondt lines, and mutual vetoes to protect community rights by the ethno-national nationalist and unionist designated groupings in the devolved parliament (Assembly) (O’Leary, 2019). Fully functioning power-sharing has been sporadic since 1998 and parties have disagreed on many issues: same-sex marriage, language rights, flag display, and addressing the legacy of conflict. Power-sharing collapsed for three years, from January 2017 to January 2020 (see Haughey, 2020), and collapsed again in February 2022. The system’s continued vulnerability to policy disagreement and gridlock, which then threatens the entire system, means that consideration of democratic mechanisms to overcome dispute on contentious issues remains salient, and hence provides a valuable context in which to test IPSs in divided societies.
In Northern Ireland there have been many quality surveys on a range of policy issues (Hayes and McAllister (2013) for an overview) and there have also been conventional deliberative mini-publics, including citizens’ assemblies and a deliberative poll (Garry et al., 2020; Luskin et al. 2014; Pow and Garry, 2019). An IPS in Northern Ireland will seek to combine the precision of population surveys and the consideration of deliberative mini-publics. We begin our empirical examination of IPSs in Northern Ireland by focusing on their feasibility and effect.
Study One: Assessing feasibility and effect
Study design
We conduct an IPS with an embedded experiment in order to examine the feasibility and effect of an IPS. We collected data from a representative sample of just over 1000 Northern Ireland citizens, drawn from randomly selected sampling points across the region and representative of the Northern Ireland population as a whole with respect to core socio-demographic characteristics of age, gender, and social class. The data were gathered by Ipsos-Mori, a well-established market research company with a strong track record in generating representative samples of the population. All data were collected in the participants’ own homes (see online Appendix A1 for sampling details).
The substantive subject of our IPS was the contentious issue of flag display in Northern Ireland. The argument over when and where the Union Flag (of the United Kingdom) can be displayed has led to sustained controversial disagreement between Protestant unionist supporters and Catholic nationalists opposed to display. In order to facilitate citizens becoming more informed about the issue of flag display we wrote a script that we believed summarised, in a fair-minded and balanced way, historical and recent controversies on the flag issue. The script was generated in consultation with the lead author of a report on the most recent flag crisis, a major qualitative study of the range of perspectives on the issue (Nolan et al., 2014). We worked with an advertising company to create a professionally produced and edited video with Paul Nolan presenting this script to camera, using auto-cue and with key bullet points at times emphasised in text on screen. This ‘information video’ was shot outside against a pleasant green and streetscape backdrop with Paul Nolan sitting in relaxed manner on a small wall in smart but not overly formal dress. The effect we sought was serious and objective, yet relaxed and engaging (see online Appendix B for example screenshot and online Appendix C for full text.) The video length was two minutes and 56 seconds.
In a similar way, we made a ‘perspectives video’ (four minutes and 12 seconds in length) that focused in turn on each of the distinct policy options: the Union Flag should be flown all the time from public buildings, the Union Flag should be flown on designated days only from public buildings (the compromise option), or the Union Flag should never be flown at all on public buildings. In relation to each option the speaker – again, Paul Nolan – articulated the typical (anti-Union flag) nationalist response to the option, the typical (pro-Union Flag) unionist response to the option, and the typical ‘pro-designated days’ compromise option. Again, it was face-to-camera, using auto-cue, with key bullet points at times emphasised on screen. (See online Appendix D for full text.)
In addition to providing information and arguments via video, we also scripted a mental simulation task for our participants in which they were invited to discuss each policy option with an imagined other from the ethno-national out-group. We presented Protestant respondents with an imagined other who was a Catholic, named Declan, with a hard-line nationalist anti-Union Flag position, and we presented Catholic respondents with an imagined other who was a Protestant, named Andrew, with a hard-line unionist pro-Union Flag position. (The small number of respondents who were neither Catholic nor Protestant were randomly allocated to one or other ‘imagined other’ scenario). (See Appendix E for full imagined dialogue guide.) Each participant was asked to write down the imagined conversation that they engaged in. In order to assess the feasibility of this novel imagined dialogue approach, we report (in the next section) the extent to which participants positively engaged in the mental simulation exercise.
In order to be able to assess the effect of the IPS on attitudes, the following randomly assigned experimental groups were specified in the study. 1 In total, 20 % (randomly assigned) of the citizens in our representative sample engaged in all three aspects of what may be termed the ‘full package’ of consideration – that is, they observed the information video, they observed the perspectives video, and they engaged in the mental simulation task. An additional 20 % (randomly assigned) of the citizens in our sample did not engage in any deliberation (i.e. they did not view either of the videos and did not engage in the mental simulation task). Our first experimental comparison will be between these two randomly assigned groups (i.e. between these two experimental conditions). We expect that those participants in the full three-component deliberation condition will have views on the outcome variable (extent of agreement with the compromise option on flag display) that are more pro-compromise than those in the ‘no consideration’ control group condition.
Furthermore, if there is an observed effect, we wish to disentangle which of the component parts of our IPS is causing the effect. We hence include in our experimental design a number of other treatment groups that allow us analytical leverage in terms of identifying the possible independent influence of (a) the information video, (b) the perspectives video and (c) engaging in the imagined dialogue (see Table 2 for the list of experimental conditions). Specifically, we include a treatment group in which participants watched the information video only (condition B), we include a treatment group in which the participants watched the information video and the perspectives video only (condition C), and we include a treatment group in which participants watched the information video and engaged in the imagined dialogue (condition D). Comparing these three treatments groups (conditions, B, C and D) with the control group (condition A) allows us to parse out the effects of the different component parts of our full deliberative package and assess whether imagined dialogue plays a demonstrable role in driving any impact that the full deliberative package has on opinion.
Experimental conditions.
Feasibility: Engaging in the imagined dialogue task?
We are particularly interested here in examining the extent to which participants effectively engaged with two of the features of our study: the video and the imagined policy dialogue. In relation to the videos this was not problematic. The participants were invited to watch the videos on the laptop computer of the interviewer, and no problems emerged in terms of participants cooperating.
The more challenging feasibility assessment relates to the novel feature of imagined policy dialogue. One may be sceptical as to whether such an exercise would be effectively engaged with by participants. Hence, we examined the textual recording of the participants’ engagement in imagined dialogue in order to assess the quality of engagement. Specifically, we generated and implemented a simple coding frame that distinguished three types of engagement: co-operative (52.3 %), disengaged (30.8 %), and antagonistic (16.9 %) to identify an objective, or etic, assessment of the form of deliberative interchange between the participant and the imagined other (see online Appendix G for full coding details). Similarly, we coded (see online Appendix H for full coding details) the manner in which the participant concluded the imagined deliberative interchange into one of three groups: amicable (57.1 %), neutral (22.1 %), or antagonistic (20.6 %) in order to assess the subjective, or emic, appreciation of the quality of the contact as perceived or characterised by the participant. To identify the proportion of participants who fully and positively engaged, we created a dichotomous variable that distinguishes between participants (coded 1) who engaged in the highest quality deliberative manner with the mental simulation task (i.e. they were coded into the most positive category in both aspects of the dialogue engagement – the interchange and the conclusion (43.1 %)) and those who did not (coded 0; 56.9 %).
Illustrative examples of imagined dialogue coded as co-operative, disengaged, or antagonistic are reported in online Appendices I, J and K, respectively. Common features of co-operative imagined dialogue are the constructive articulation, by the (real) deliberating participant, of the views of the imagined other as well as the participant’s own views, the respectful acknowledgement of differences, and the absence of animosity in the exchange of views. Disengaged dialogue is characterised by ‘don’t know’ responses and an absence of an articulation of either the views of the other or the self. Antagonistic imagined dialogue is characterised by portrayal of the imagined other as unreasonable or irrational and discrediting the views of the other. Examples of how participants engaged with the conclusion of the imagined deliberation, ranged from the courteous amicable ending (e.g. ‘I think we might conclude that we are from different backgrounds and obviously see things differently and agree to differ’), the relieved neutral ending (e.g. ‘Happy that it’s over. I just want to avoid confrontation’) to the antagonistic whereby the imagined other is negatively portrayed (e.g. ‘He is an idiot’).
Overall, participants cooperated in terms of watching the videos and engaging in the imagined policy dialogue. We observe variation in the quality of the engagement in the imagined dialogue. To assess whether this variation has an impact on the effect of the imagined dialogue, we will utilise this variation in the analysis of effect, in addition to seeking to observe aggregate effects. In other words, as part of our examination of effect, below, we will assess whether effect varies by quality of engagement in the imagined policy dialogue.
Effect: Increase support for compromise option?
Aggregate effects
Does watching the information and perspective videos and engaging in imagined dialogue change people’s views, and if so in a moderate or more hard-line direction? In condition E citizens received the full deliberative materials: they watched the information video, then the perspective video, engaged in the imagined dialogue, and answered our main outcome variable question (to what extent would do you agree with the designated days option (the compromise option))? In condition A, the control group, citizens only answered the outcome variable. Our first test is comparing these two groups to assess whether respondents in the ‘full consideration’ condition were more in favour of the compromise option. As illustrated in Figure 1 (comparison of condition E with the control group), we found that they were to a statistically significant degree. The E column in the graph shows that participants who were randomly assigned to the treatment group (engaging in all three elements of the deliberative package) have views that are almost half a unit (on a seven-point scale) more in favour of the compromise (‘designated days’) option than participants who were randomly assigned to the control group who did not engage with any of the three elements of the deliberative package.
Extent of support for pro-compromise position on flag display under different deliberation conditions compared with the control group (condition A) of no deliberation (95% confidence intervals).
Is there a distinct effect of imagined dialogue? The other three treatment conditions included in the experiment (conditions B, C and D) are compared with the control group, and as shown in Figure 1 the only one of these three conditions that is statistically significantly different from the control group of no deliberation is the condition containing the imagined dialogue component. Specifically, engaging in imagined dialogue after receiving information (condition D) yields a more pro-compromise position compared with the control group. In contrast, engaging with the perspectives video after receiving information (condition C) does not (and simply receiving information – condition B – does not lead to a more pro-compromise position than the control group).
In a further analysis designed to assess the distinct role of the imagined dialogue component, we focused on the four experimental groups that were given a treatment (groups B, C, D and E). We conducted a 2 × 2 ANOVA on conditions B–E in order to isolate the effect of the imagined dialogue task from that of watching the perspective-taking video, as well as to determine any interaction between these two factors. We found a significant, though small, main effect of imagined dialogue on agreement with the compromise option, F(1,794) = 4.37, p = .037, partial η2 = 0.005, such that those undertaking the imagined dialogue task were statistically more likely to endorse this option (M = 4.76, SD = 2.19) than those who did not (M = 4.44, SD = 2.20). There was a non-significant main effect of perspective information, such that receiving the perspective video did not have a significant effect upon responses on the compromise option, F(1,794) = 0.76, p = .383. The interaction between imagined dialogue task and watching the perspective video was also non-significant, F(2,794) = .502, p = .479. In other words, while the imagined dialogue task does increase participants’ likelihood of giving a positive response on the compromise option, the provision of the perspective-taking video does not, and neither does this video serve to enhance or inhibit the effect of the imagined dialogue task.
Do effects vary by quality of engagement in the imagined policy dialogue?
We also conduct a further analysis in which we examine whether the quality of engagement in the imagined dialogue task had an effect on attitudes. In this analysis we include only those participants who engaged in the imagined dialogue exercise (i.e. participants in experimental conditions D and E). In Table 3 we report the results of a linear regression in which extent of support for the compromise position on flag display is the dependent variable. Our key predictor variable is the binary variable representing quality of engagement with the mental simulation exercise (described earlier: high-quality engagement = 1; low-quality engagement = 0). We find that participants who demonstrate high-quality deliberative engagement in the imagined dialogue mental simulation task are more likely than those who do not to be in favour of designated days (the compromise option), and the size of the effect of high-quality deliberation is almost two-thirds of a unit on a seven-point scale. In this analysis we are not comparing experimental groups, but rather individual-level differences between participants in terms of the deliberative quality of their engagement. 2 Hence, we include a number of control variables that are likely to be related to the outcome variable and also possibly to the predisposition of participants to engage deliberatively in the mental simulation task, in order to control for possible spurious effects. These controls are being a supporter of the Alliance party (the most cross-community, or moderate party in the system) and being ideologically moderate (self-describing as ‘neither nationalist nor unionist’ rather than adopting either a very or fairly strong unionist or nationalist stance), as well as age, social class and gender (and also whether the participant is in experimental group D or E). Hence, deliberatively engaging in the act of imagined dialogue has a pro-moderate effect on attitudes to flag display.
Predicting support for pro-compromise (pro-designated days) position on flag display (OLS Regression).
Discussion of feasibility and effect
Overall, this empirical study has achieved a number of aims. First, it has performed the simple but important task of showing our suggested novel IPS in action in an illustrative highly challenging real-world context. We have hence demonstrated that our approach is feasible. It is indeed possible to show a large representative sample of a population specially designed movie clips and achieve their engagement (albeit with varying degrees of deliberative quality) in a mental simulation exercise in imagined dialogue. If this is possible on a sensitive polarising issue in a deeply divided place such as Northern Ireland, then it is arguably feasible and possible in other divided contexts. Second, we have shown a small but identifiable effect of our informed imagined dialogue deliberative package (i.e. observing both the information and argumentation video clip and engaging in imagined dialogue) on attitudes in a pro-compromise direction, we have also shown a distinct effect of the imagined dialogue component of the deliberative package, and we have shown that the effect is stronger for those participants who positively engage in the mental simulation task.
Study Two: Assessing perceived legitimacy
Any proposed novel mechanism for involving citizens in a divided place in the decision-making process on contentious issues should enjoy reasonably high levels of support in the general population in order for the proposed mechanism to be regarded as legitimate (or potentially legitimate). Also, the levels of perceived legitimacy in the population should be approximately equal in the different ethno-national groups to avoid the perception that a decision-making mechanism in some way favours one group rather than another. Therefore, we experimentally examine the level of perceived legitimacy that an IPS attracts among the general public. We focus in particular on the most novel element of our design: the imagined dialogue. We examine whether considering an issue via imagined dialogue is seen as legitimate – specifically, whether it is seen as at least as legitimate as the more conventional face-to-face dialogue approach. Essentially, we ask: to what extent do the public support decision-making by an IPS which is based on imagined dialogue, compared with decision-making by a conventional mini-public (in this case, a citizens’ assembly) which is based on real dialogue? Here, we experimentally manipulate the arguably most controversial feature of our suggested IPS approach (imagined rather than real dialogue) in order to assess relative levels of perceived legitimacy.
Study design
The study was conducted on a sample of 272 adults living in Northern Ireland, recruited from an online opt-in panel of over 10,000 members. 3 Respondents were incentivised to participate by earning reward points that can be redeemed in regular prize draws. The profile of the non-random sample was largely male (80 %). Some 44 % of respondents came from a Protestant community background, 37 % from a Catholic background, and 18 % did not identify their community background with either of the two religious groups. Data collection took place from 3 to 6 June 2018.
Respondents were provided with some basic background information relating to the concept of a citizens’ assembly in a text-based vignette. The manipulated text is shown in italics.
In Northern Ireland, there are some issues on which elected politicians are divided over the way forward. On these issues, it has been proposed that a citizens’ assembly could decide on the way forward instead. The citizens’ assembly would consist of a group of ordinary citizens that is representative of the whole population of Northern Ireland. Members would be randomly chosen, in the same way that legal juries are selected. The citizens’ assembly would consider an issue in [{
Participants in the citizens’ assembly will be assumed to have no prior knowledge of the issue in question. They will all be provided with background information and presented with arguments from both sides of the issue.
Participants will then talk about the issue with each other, including with members from a different community. These discussions will take place in small groups of about ten participants, facilitated by a neutral chairperson. This will allow participants the opportunity to consider other perspectives as well as their own. They will be asked to try and think about common ground on the issue. }
Participants will then independently spend time thinking about the issue. They will be asked to imagine that they are having a conversation with a person from another community about the issue. This will allow participants the opportunity to consider other perspectives as well as their own. They will be asked to try and think about common ground on the issue. } ]
After learning about the issue, members of the citizens’ assembly would then be asked to vote on it. What a majority of these people decided in the vote would be seen as the decision on the issue and would be implemented.
Respondents were presented with a simple infographic underneath the text-based vignette of each experimental condition (see Figure 2). These visual supplements were intended to summarise and reinforce the manipulations as clearly and effectively as possible. As with the structure of the text in each vignette, the three infographics contain consistent representations of the first and last stages of decision-making, with variation in the representation of the intermediate stage (where applicable).
Infographics from each of the three experimental conditions. Group 1: Information only. Group 2: Information + real dialogue. Group 3: Information + imagined dialogue.
The first phase of receiving information is summarised with the presentation of evidence both for and against a given policy. This emphasises the balanced nature of the information provided to members. The positioning of the presenter behind a lectern is intended to convey the one-way, top-down flow of information, that is, that members were purely receiving, not discussing, the information. In contrast, the second infographic emphasises that citizens’ assembly members would engage in face-to-face discussion in small groups. While the physical attributes of the citizens were kept deliberately vague, at a minimum the depiction conveys some basic diversity around the table; the members are somehow different from one another.
In the final condition, a combination of thoughtful pose and thought bubbles around two individuals is used to depict the more abstract concept of internal reflection via an imagined conversation. This was a more challenging infographic to design, but the thought bubble, together with the hand gestures, are generally understood to represent the internal act of ‘thinking’. The depiction of the members facing in opposite directions emphasises their mutual independence, but the fact that there are two is to convey the dialogical aspect of the process. The last stage of each infographic underlines that members will take a collective decision by a formal vote.
The composition of the citizens’ assembly (a representative random sample of citizens) and the nature of the decision (binding) were held constant across the three conditions. Drawing on data from a manipulation check embedded in the questionnaire, we demonstrate that respondents successfully recall the phases of decision-making mentioned in the vignette (online Appendix A2). Immediately after reading the text, respondents were asked to evaluate the decision-making process described. The outcome variable of perceived legitimacy was measured using a multi-item scale (online Appendix A3).
Results
As illustrated in Figure 3, we find no difference in perceived legitimacy across the three conditions, F(2, 253) = .59, p = .55. In Group 1, the process involving simply the presentation of information ahead of citizens taking a decision was taken received a mean legitimacy score of 3.43 (SD = 1.65) on the 0-6 multi-item scale. The presence of real dialogue in Group 2 produced a similar score (M = 3.20, SD = 1.68), as did the presence of internal imagined dialogue in Group 3 (M = 3.46, SD = 1.64). Therefore, neither the formal addition of deliberation to the decision-making process, nor the distinction between its real and imagined forms, had an overall effect on the perceived legitimacy of decision-making. This suggests that the general public are just as supportive of an IPS as a conventional citizens’ assembly.
Mean legitimacy scores for each process.
We also conducted a two-way ANOVA to investigate whether perceived legitimacy levels in each experimental condition varied by community background. It revealed a statistically significant interaction, F(4, 244) = 2.84, p = .03. Pairwise comparisons for each simple main effect revealed no significant differences in the perceived legitimacy of decision-making in Groups 1 and 3 between Catholic, Protestant and other respondents. However, in Group 2 perceived legitimacy scores were significantly higher among Catholics than among Protestants, 1.40 (95% CI, .36 to 2.44), p < .01. In this condition perceived legitimacy scores were also higher among respondents identifying with neither of the two traditional backgrounds compared with Protestant respondents, 1.71 (95% CI, .61 to 2.80), p < .01. These differences are illustrated in Figure 4. Therefore, our main conclusion from these results is that conventional citizens’ assemblies based on real discussion attract different levels of support depending on an individual’s community background – a perceptions gap – whereas an IPS based on imagined discussion attracts broadly similar levels of support. 4
Mean legitimacy scores for each process, by community background.
Conclusion
Deeply divided places are challenging to govern, with frequent policy crises or gridlock, due to an unwillingness or inability of each side to compromise. If it is deemed reasonable to supplement elite-level decision-making (or lack of decision-making) with citizen engagement, the question arises as to exactly how to operationalise this. Here, we have advocated a novel vehicle – an IPS – that meets what we regard as two crucial criteria. An IPS can with statistical precision identify whether a majority of the whole population supports a particular policy option. And the supported policy option will be the result of a significantly more informed and reflective process than a typical opinion poll. Our approach seeks to achieve the best of both worlds: the precision of the conventional survey with at least some of the deliberative value of a deliberative mini-public (such as a citizens’ assembly or deliberative poll). This resulting hybrid is road tested on the difficult terrain of Northern Ireland and found to be broadly feasible and have at least some pro-compromise effect (compared with a conventional survey). Furthermore, the approach commands a similar level of perceived legitimacy compared with a conventional mini-public and, importantly, it attracts support in approximately equal measure from people of different community backgrounds.
We encourage further work on successfully engaging participants in the imagined dialogue task. Is non-positive engagement a function of question form or personal predisposition? If the former, question tweaks may illicit greater positive uptake. If the latter, that is a more fundamental problem and prompts investigation of the individual-level traits that prohibit (or enable) effective imaginative engagement. Levels of non-positive engagement may also vary by political context. At times of heightened tension, thinking of the out-group may possibly elicit a more negative response. We also encourage further work on identifying the feasible limits on participant engagement with video components. In the demonstrated experiment, very brief videos were used. Could much more substantive videos lasting several hours be effectively used? We also encourage further work demonstrating the extent to which the video materials we present are perceived to be balanced and fair.
Supplemental Material
sj-docx-1-ips-10.1177_01925121221125049 – Supplemental material for Imaginative policy surveys in divided societies: Feasibility, effect and perceived legitimacy
Supplemental material, sj-docx-1-ips-10.1177_01925121221125049 for Imaginative policy surveys in divided societies: Feasibility, effect and perceived legitimacy by John Garry, James Pow, Clifford Stevenson and Peter Stone in International Political Science Review
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Economic and Social Research Council (ESRC) of the UK: ‘Randomly Selected Politicians: Transforming Democracy in the Post-Conflict Context’ (ES/M000257/1).
Supplemental material
Supplemental material for this article is available online.
Notes
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
