Measuring Epistemic Deliberation on Polarized Issues: The Case of Abortion Provision in Ireland

Abstract

This paper compares the debate quality in the plenary sessions of an Irish Citizens’ Assembly and an Irish parliamentary committee to assess the epistemic effects of public deliberation on a contentious subject: abortion. The unusual occurrence of a similar process of detailed discussion on the same topic in different institutions at around the same time (in 2016–2017) allows us to compare the deliberative capacities of these institutions and thus contribute to discussions on the appropriateness of an increasingly debated democratic reform: assigning political offices by lot. We suggest that the epistemic effect of deliberation on abortion should facilitate nuanced multi-layered discussion that is both ‘deeper’ in being based on multi-faceted arguments and ‘wider’ in terms of a more accommodative view. We anticipate that these effects should be more pronounced in the more deliberative, less polarized, environment of a citizens’ assembly rather than in a parliamentary committee. The analysis deploys the psychological concept of ‘cognitive complexity’. We find that members of the Citizens’ Assembly demonstrate a deeper cognitively complex grasp of the subject matter. In contrast, experts and parliamentarians tend to adjust their mode of delivery at a parliamentary committee reflecting the conflictual and strategic aspects of political debates in such a forum.

Introduction

Deliberative democracy initially conceived at a highly abstract and theoretical level (e.g. Cohen, 1986; Elster, 1998; Habermas, 1997; Luhmann and Habermas, 1971; Rawls, 1993) has been increasingly used for a variety of real-world policy issues (e.g. Parkinson, 2006; Smith, 2005, 2012; Elstub, 2010, Elstub and McLaverty 2014; Farrell and Suiter, 2019; Harris, 2019). In all of these approaches, there is an assumption that within the deliberative system both politicians and citizens can under certain circumstances deliberate well. Bächtiger and Beste (2017: 106) argue that both politicians and citizens have the capacity to approach the ideals of deliberation as envisaged in Habermasian rational discourse if the ‘institutions are appropriate’. Yet this is contested. Shapiro’s (1999) ground breaking piece argued that deliberative studies fail to sufficiently take into account interest and power among political elites. Deliberative scholars such as Fishkin and Luskin (2005) worry that political elites will focus on negotiation over deliberation, thereby undermining the power of the better argument. For others the potential of politicians to do so is limited to the relatively rarefied confines of consensus systems on uncontentious topics (Bächtiger and Hangartner, 2010; Steiner, 2004; Wyss et al., 2015); the possibility of deliberation is seen as especially unlikely in Westminster systems. Some scholars are concerned about the ability of citizens to deliberate; for example, Achen et al. (2017) view citizens as simply inherently incapable of deliberating, while Rosenberg (2014: 115) argues that participants who attend a deliberation do not, in fact, engage in the give and take of the discussion’.

In terms of the potential of citizens to engage in epistemic deliberation empirical evidence points to a deliberative potential, especially when appropriate institutions are utilized, such as deliberative mini-publics. Various research teams have found deliberative and epistemic benefits such as more evidence-based reasoning, or improved citizen preferences through, for example, change of mind or less polarized positions (for a review see Carpini et al., 2004; Mendelberg, 2002; Setälä and Smith, 2018). Furthermore, normative benefits are often grounded in the various psychological mechanisms at play when people deliberate, many of which can also have an important role in determining the outcome of deliberation. Such normative criteria used to measure the quality of the deliberation vary including reaching of meta-consensus (Dryzek and Niemeyer, 2006; Niemeyer and Dryzek, 2007), increased respect between the participants (Gutmann and Thompson, 1996; Steenbergen et al., 2003; Schneiderhan and Khan, 2008), changing of opinion (Suiter et al., 2016), increased coherence between beliefs (Gastil and Dillard, 1999; Suiter and Reidy, 2020), the substantive quality of the outcome, and, importantly, reasoning (Mercier and Landemore, 2012; Suiter et al., 2020). Thus, it seems that empirical research shows that both politicians and citizens have at least the capacity to deliberate when institutions are appropriate.

However, much of the evidence is particular and relies on specific minipublics or specific parliamentary debates on specific topics at specific points of time. We simply do not know how the quality of deliberation varies across citizen and legislative institutions. This is largely because, to our knowledge, the comparative study of the deliberative quality of debate among citizens and among politicians in the same period on the same subject has not been possible to date. It is important to do so because otherwise while we can demonstrate that citizens can deliberate, we cannot tell whether their deliberation is different (either inferior or superior) to the deliberation of elected representatives. This becomes ever more important as deliberation expands across institutions and as it scales up, in particular given the role of both minipublics and parliaments, as different types of deliberative fora, within the wider deliberative system. It can aid our understanding as to how each can contribute to the whole.

Responding to Landemore’s (2017: 286) question, ‘Under what conditions do democratic procedures tend to produce better decisions?’, this paper adopts a comparative perspective to compare the debate quality in the plenary sessions of an Irish citizens’ assembly and an Irish parliamentary committee on the contentious issue of abortion. The unusual occurrence of a similar process of detailed discussion on the same topic in different institutions at around the same time allows us to make real comparisons between the deliberative capacities of these fora. To the best of our knowledge, this hasn’t been done before and it has the potential to contribute to existing scholarship on epistemic quality within different types of deliberative fora. The Irish case, is also particularly interesting when we consider that the Government opted for the Citizens Assembly’s suggested wording for the subsequent referendum over that recommended by the parliamentary committee, arguably adding to the legitimacy of the official governmental process that lead to the referendum on repealing the 8th amendment. Indeed, we find that the epistemic affect of a mini-public’s deliberation on abortion facilitates nuanced multi-layered discussion that is both ‘deeper’ in being based on multi-faceted arguments and ‘wider’ in terms of a more accommodative view than a similar discussion at a parliamentary committee.

The structure of the paper is as follows. We start by setting out the context of Ireland’s abortion debate and the role played by the Citizens’ Assembly of 2016–2018. We then discuss the nature of deliberation in terms of its epistemic features and particularly relating to the treatment of complex subjects such as abortion. Subsequently, we present our case that epistemic standards allow us to judge whether a given deliberative forum is associated with better or worse outcomes from an epistemically normative point of view. We set out the merits of our preferred measurement proxy – Cognitive Complexity – and its operationalization, and describe the data. We present our empirical results, with a reflection on the capturing of cognitive capacity in deliberative debates, followed by some concluding remarks on the normative implications of our findings.

The Context: The Irish Citizens’ Assembly and the Abortion Question

This Citizens’ Assembly was Ireland’s second government-sponsored deliberative mini-public (Farrell et al., 2017, 2019; Suiter et al., 2016). The Assembly was tasked with making important recommendations on a number of policy areas and on whether to propose a referendum should the subject be on a matter of constitutional change. It was composed of 99 randomly selected citizen members and an independent chair. The first issue it dealt with was the possible removal of a controversial 1983 amendment to the Constitution, the so-called ‘Eighth Amendment of the Constitution’, which gave explicit recognition to the right to life of an unborn child, effectively introducing a constitutional ban on abortion in Ireland. A debate on this was something that Irish parliamentarians had largely avoided for 30 years (Reidy, 2020: 2311). The possible repealing or replacing of Article 40.3.3, was considered by the Assembly over a period of five weekends from November 2016 to April 2017.

The Citizens’ Assembly followed the same modus operandi as its antecedent Ireland’s Convention on the Constitution (Farrell et al., 2017; Suiter et al., 2016). Expert presentations with briefing papers were circulated days in advance. Many of the expert presentations were designed to be as objective as possible; though on occasion there were experts with differing perspectives pitched against each other. ¹ There were also presentations by advocacy groups and personal testimonials by a number of women. The presentations were followed by question and answer sessions and then by small group roundtable discussion. Importantly, the small group roundtable discussions were facilitated, a factor which is known to improve deliberative outcomes. A vote was held at the end of the process with the members voting overwhelmingly to replace the article with a new provision explicitly authorizing the Irish Parliament to legislate for abortion and, of great surprise to watching pundits, for a very substantial liberalization of abortion provision.

The Assembly report was submitted to the Irish Parliament, a Westminster-style parliament, which convened a special committee to consider it. The topics discussed by the parliamentary committee were thus similar to those discussed at the Citizens’ Assembly and broadly broke down as follows: constitutional legal issues, constitutional human rights issues, healthcare issues, crisis pregnancy, risks to health, risks to mental health, termination related to rape, termination related to medical issues, and committee business. ² Some committee business was held in private session and some in public. There were 21 members of the committee from both Houses of Parliament: 10 women and 10 men, as well as a female chair. The main conclusion was to amend the Constitution to amend or replace Article 40.3.3 with a new article. The referendum was held on 25 May 2018 with a near record turnout of 64%. The electorate voted to repeal the 8th Amendment by a majority of 66% to 34%, a result noted for its similarity to that of the Citizens’ Assembly vote. Thus, this case is one which allows for the comparison of citizens’ and politicians’ deliberation in two institutions, a mini-public and a Westminster-style parliamentary committee on a contentious issue, abortion.

What Is ‘Good’ Deliberation?

A difficulty in marrying the theoretical and empirical literatures in deliberative democracy has been agreeing the concept of what is ‘good’ deliberation. Arguably much of the focus has tended to be on its procedural merits, which while important does not allow a full exploration of the democratic potential of deliberation. This focus on procedure by empirical deliberative scholars allows them to avoid, in so far as possible, poor decisions based on poor heuristics or biases (Fishkin, 2009) and to move beyond the ‘fact of disagreement’ (Landemore, 2017: 277) to acknowledging intrinsic properties. Of course, this is not to argue that the procedural aspects are unimportant (see Bächtiger et al., 2005, 2014; Gastil et al., 2008; Wojcieszak, 2014; Wojcieszak and Price, 2012). Nonetheless, a procedural focus does not always give due regard to epistemic democracy as a framework that allows for more nuanced and sophisticated approaches to the quality of discourse and participation in deliberation. Among these is the integration of political psychology insights by, for example, Mercier and Landemore (2012) and Brundidge et al. (2014) who have placed the cognitive activity of reasoning centrally, both in terms of the content of the deliberative exchange and the manner in which they are generated.

Following this line of reasoning we can argue that the functioning of reasoning and thus of legitimation and justification in deliberation is social; indeed, more to the point, it can be argumentative in the sense that it enables individuals to argue with each other given they do not blindly trust what others say. This chimes to some extent with the rational concept of trust based on ‘encapsulated interest’ (Hardin, 2001, 2006a) where trust is defined as a relationship between three actions: (1) we must trust you to (2) serve our interests broadly conceived (3) about this issue (Hardin, 2006b). In deliberation then we should expect this, namely that the epistemic vigilance of the public may involve listening to experts and the opinions of others before seeking coherence with our own thoughts. In our understanding, the deliberative model starts from the assumption that political equality is best expressed by a willingness to offer reasons for our preferences that others can accept. Thus, deliberation is a process whereby initial preferences are at least open to the potential of being transformed to take account of others’ preference. And if we enter into a democratic process in a spirit that recognizes that other citizens have equal standing with ourselves, we shall be ready to moderate our claims. This willingness to moderate our views is what results in some uncertainty and tentativeness in deliberation. The mere existence of this openness to moderation of our beliefs means we would expect some uncertainty and tentativeness in deliberation, allowing a gravitation towards ‘true’ beliefs. Thus, epistemically successful deliberation requires the respectful exchange of information, with some degree of contingency, in an inclusive, equal forum that is arguably descriptively and discursively representative.

Moreover, it can be argued that disagreement is a prerequisite for a healthy deliberative process to the extent it ensures a diversity of experiences, preferences and perspectives. Gutmann and Thompson (1996) suggest that when citizens disagree morally, they should continue to deliberate together to reach mutually acceptable decisions. Their proposal is not to remove conflict but to focus on the norm of mutual respect, whereby through a process of deliberation the gap between participants’ differences is reduced and they agree to disagree on those that remain. Deliberative democrats have ‘rarely endorsed consensus as an aspiration for real-world decision-making’ (Curato et al., 2017: 31), recognizing that a focus on consensus may restrict the scope for discourse (Young, 2000). More recently theorists claim that deliberation should strive for meta-consensus (Curato et al., 2017; Niemeyer and Dryzek, 2007) – an outcome of authentic deliberation that is defined as agreement on the domain of reasons and considerations relating to the issue at hand as well as the nature of the available choices (Dryzek and Niemeyer, 2006). In this view some disagreement, or agreeing to disagree or non-convergence, may still be epistemically valuable (see more detailed reasoning in (Fishkin, 2018; Luskin et al., 2017)).

Thus, it may be that epistemically sound deliberation should strive for both depth and breadth in its processes – in other words epistemic standards. Evaluating these standards allows us to judge whether a given deliberative process produces better or worse deliberative or epistemic outcomes. And we can assess these standards regardless of whether we are able to know if the process resulted in participants reaching their truer beliefs (Cohen, 1986; Estlund, 1998; Fishkin, 2018). The problem, then, is how to evaluate what is epistemically sound.

The maturing of deliberative democracy as an empirical research programme has coincided with the development of more diverse and sophisticated methods for testing its key claims (for overviews see Bächtiger, 2018; Beste, 2013; Black et al., 2011; Neblo, 2007; Thompson, 2008). This has included a growing interest in getting inside the ‘black box’ of the deliberative process itself, and measuring the epistemic quality of the deliberation (Bächtiger and Parkinson, 2019), which have included a myriad of different approaches, including: the Discourse Quality Index (DQI) developed by Steenbergen et al. (2003); Niemeyer and Dryzek’s (2007) index of intersubjective consistency; Wyss et al.’s (2015) ‘cognitive complexity’ index; or more recent efforts by Friberg-Fernros and Schaffer (2017) to measure what they refer to as adequate support for given conclusions, and Knops’s (2017) consideration of validity and scope.

Of these the most common deployed framework to evaluate what is epistemically sound is the DQI (Steiner, 2004) where justifications of positions are evaluated, although even this has come in for some criticism that it falls short of assessing whether reasons adequately support conclusions (Friberg-Fernros and Schaffer, 2017). Other more recent approaches suggest useful avenues, such as Friberg-Fernros and Schaffer (2017), who propose an assessment of the epistemic quality of democratic decision-making by evaluating whether it is adequately supported by reason, or Knops (2017), who proposes to examine the validity and scope of arguments. However, neither is yet operationalised within a quantitative framework. In order to capture the individual level reasoning we follow Wyss et al’.s (2015) use of the psychological concept of cognitive complexity (Suedfeld and Tetlock, 1992; Tetlock, 1983). This measures the degree to which an individual perceives, distinguishes and integrates topical dimensions and thus captures both epistemic and accommodative dimensions of political argumentation (Wyss et al., 2015). It thereby captures an individual’s capacity to accommodate different goals or values in deliberative institutions (Gruenfeld and Preston, 2000). High scores on some dimensions indicate a subject’s capacity to accommodate conflicting goals or values in problem-solving situations (Gruenfeld and Preston, 2000). Thus, there are strong conceptual links between cognitive complexity and deliberative ideals (Brundidge et al., 2014) such as epistemic advances and respectful dialogue as key components of democratic quality (see Mansbridge et al., 2012). This approach will also facilitate comparability with data/research emerging from other cases and countries.

Further, cognitive complexity (CC) is particularly suitable for our analysis of the difficult debate over abortion in Ireland in advance of the 2018 vote, which was based on moral and religious beliefs as well as on scientific understandings. Given that this approach is based on a psychological characteristic (or variable) that indicates the complexity and/or simplicity of a person’s framing and perceptual skills, it allows scope to measure the complexity of the arguments used to assess the relative merits and trade-offs associated with different options (Abe, 2011, 2012; Gruenfeld and Preston, 2000; Beste and Wyss, 2014; Brundidge et al., 2014). As Owens and Wedeking (2011) argue, it is important to deploy a dimension that measures the degree to which an individual accepts uncertainty, tentativeness and contingency. And it is especially important in this instance given the nuances in the abortion debate. As Habermas (1990) emphasizes, the development of toleration of contingencies of debate is a singular benefit of deliberation. In common with Beste and Wyss (2014) and Wyss et al. (2015) we find that these contingencies can be captured with a cognitive complexity measure consisting of two broad dimensions, which, following Brundidge et al. (2014), we call differentiation (breadth) and integration (depth). Differentiation refers to the quantity of viewpoints and dimensions considered by a speaker and thus provides an insight into the breadth of perspectives discussed. However, it is also characterized by differentiating or occasionally negative words and phrases which can lend to a rather negative tone. Integration measures the depth of a speaker’s understanding. Consequently, CC represents an important marker of the epistemic quality of debate; by the same token, it also tracks the willingness of actors to integrate and accommodate other viewpoints and to strive for agreement – a feature which Steiner (2004) dubbed a consociational ‘spirit of accommodation’.

Our case involves the discussion of legislating for abortion across two separate forums; a citizens’ assembly (with citizen members) and a parliamentary committee (with parliamentarians). The Citizens’ Assembly was a specifically non-partisan deliberative forum, with members recruited through sortition, and discussion managed by facilitators following specific norms of respect and tolerance of opposing views agreed with the Chair. Indeed, it was set up in order to maximize the chances of a consociational spirit of accommodation (Steiner, 2009). The parliamentary committee was established to debate the report from the Citizens’ Assembly, to reach consensus across partisan lines, and to make a recommendation on whether to go ahead with putting the decision to the people in a referendum and if so to outline potential legislative provisions. The parliamentary committee was a specifically partisan venue following norms of oppositional argument rather than accommodation, which may be exacerbated when debating contentious topics. In addition, parliamentarians tended to actually debate and to discuss within this committee rather than simply reading out statements pre-prepared by aides. The subject matter being tested is a paradigmatic one: studies of deliberative quality in parliamentary debates on abortion reveal lower levels of justification and respect when compared with less polarized issues (Bächtiger et al., 2005; Bächtiger and Steenbergen, 2004).

Given the space afforded to deliberation on the issue at both the parliamentary committee (which took place over 9 months) and at the Citizens’ Assembly (which, as we have seen, had five meetings), we expect that both politicians and citizens’ will be capable of deliberative discourse. However, there may be reasons to think the capacity in both fora will differ. Bächtiger and Beste (2017) give five reasons why politicians may find it difficult to reach deliberative ideals: first the nature of politics is conflictual (Mansbridge, 1983); second, it is difficult to differentiate between deliberative and strategic actions; third the norms of parliamentary debate often value reasoned argument over respect; fourth deliberation is found in rather rare conditions such as consociational parliaments; and fifth it can challenge norms of responsiveness and accountability.

In our case specifically, the members of the parliamentary committee were tasked with considering the Citizens’ Assembly’s recommendations. It was a cross-party committee: thus we might expect some partisan and adversarial tactics to play out as well as the strategic use of speech for possible electoral gain (see Quirk et al., 2018; Steiner, 2004). We might expect that politicians will have grappled with the issue at other times and thus be more likely to express more certainty and less tentativeness than others with less experience of deliberating it.

The members of the Citizens’ Assembly were randomly selected: that plus the norms of deliberation and the role of facilitators should act to dampen partisan tactics. We inquire in particular into the differences between the ordinary citizen members on the one hand and the parliamentary members on the other. We expect that the overall deliberative quality will be higher in the Citizens’ Assembly. Thus, our expectation is that the environment created in a deliberative mini-public as opposed to a parliamentary committee will be more conducive to deliberation among its members.

Thus, we expect:

H1: Deliberative quality among Members should be higher in the Citizens’ Assembly than in the parliamentary committee, as measured in terms of cognitive complexity.

In order for deliberation to happen it is necessary to bring public judgement to bear on expertise (Moore, 2016). In particular when dealing with contentious and complex policy options, such as abortion, it is clear that the inclusion of expertise and evidence in public debates is essential (Goodin, 2008), and that the information provided to the participants has the greatest influence on their opinions (Goodin and Niemeyer, 2003; Thompson et al., 2015). Further this should enhance the epistemic dimension of the deliberation and the resulting policy decision (Brown, 2014; Elstub, 2006).

However, little research has looked at the inclusion of experts or witnesses in mini-publics, on the types of information they provide or how it is presented (Fischer, 2009; Roberts et al., 2020). This paper cannot delve into that debate, but nonetheless we agree with Brown (2014) that the expertise must be embedded in a culture of public scrutiny and that particularly on topics of high public salience citizens must be able to assess competing expert claims if democratic legitimacy is to be ensured. In this regard it is important that experts use language that enables citizens to determine the values underlying policy and to assess conflicting evidence. Gutmann and Thompson (1996: 74) argue that in order to inform deliberation, experts must translate their knowledge into ordinary language. While the organizers of the Assembly did insist on plain English presentations and statements the experts were also likely to discuss their subject matter in a non-emotive manner characterized by wide and deep understanding and as required by their professional norms. Within such an environment citizens and experts have a shared deliberative orientation (Brown, 2008) as opposed to a decision making orientation within interest representation and legislative bodies. Thus, just as government advisory committees have the task of improving the epistemic quality of the decisions reached by government officials, interest groups, and individual citizens (Brown, 2008), so do experts in mini-publics. Nonetheless, we can expect that the experts are likely to adjust their mode of delivery at a parliamentary committee to one more suited to a decision making representative particularly given the conflictual and strategic aspects of political debates structure (for the reasons set out by Bächtiger and Beste (2017) that we discussed above) rather than a deliberative body of citizens. Thus we would expect:

H2: Experts should display higher levels of deliberative quality in the Citizens’ Assembly than in the parliament committee, as measured in terms of cognitive complexity.

Data and Methodology

Following Wyss et al. (2015) we capture cognitive complexity by using a Linguistic Inquiry and Word Count (LIWC) dictionary-based measure (Tausczik and Pennebaker, 2010). This approach has yielded consistent results for psychometric analyses (Brundidge et al., 2014; Owens and Wedeking, 2011). To our knowledge, this is the first attempt to capture the quality of deliberation beyond parliament by such a means. LIWC’s strategy is to treat different words as equivalent on syntactic or semantic grounds. Equivalence classes are defined by Pennebaker et al. (2007) according to semantic categories, with a dictionary of language and psychological processes, which for example, treat ‘hate’, ‘dislike’, and ‘fear’ as members of an equivalent class labelled ‘negative emotion’ (Benoit and Herzog, 2015).

We transcribed discussions from all 23 Citizens’ Assembly Question and Answer (QandA) sessions on The Eighth Amendment of the Constitution. ³ We also transcribed 25 QandA sessions over some 15 days of hearings (which ran from 10 September—13 December 2017) in the parliamentary committee. ⁴ Prior to entry into the LIWC programme, contributions were segmented to each individual speech act. This became the unit of observation in subsequent analysis. In its analysis of psychometric traits, the programme pre-defines certain ‘function word’ and ‘cognitive process’ dimensions which are then specified in the analysis. ⁵

LIWC creates a score for each dimension based on the frequency with which words occur that are attributed in the LIWC dictionary to that dimension. ⁶ For example, the Sixletter dimension includes all words of more than six letters; the Insight dimension includes words such as ‘think’ and ‘consider’; and the Negations dimension includes words such as ‘no’ and ‘never’. Following the approach of Owens and Wedeking (2011), we equate: Cognitive Complexity = Sixletter + Causation + Insight + Tentative + Conjunction + Discrepancy–Certainty–Differentiation–Negation. For a fuller discussion of this equation and its components see; Beste and Wyss (2014), Wyss et al. (2015). ⁷

Each case equates to an individual speech act, and variables represent scores for each dimension. The initial dataset comprised 3,981 cases, with 860 cases from Citizens’ Assembly members and 3,121 from the parliament committee members. When a speech act is short (in Word Count) LIWC generates a 0 score for many dimensions. These 0 scores would distort factor and regression analysis, as they correlate and indeed their correlation is simply indicative of a lack of words to analyse rather than a dimension of the speech act. To avoid this, cases (i.e. speech acts) with less than 15 words were excluded from subsequent statistical analysis, leaving a dataset of n = 861.

We next conduct factor and reliability analysis on the variables mentioned in our equation above. ⁸ Factor analysis (Principal Axis Factoring with Direct Oblimin rotation) on the nine variables does not detect one single cognitive complexity factor in our data, rather it is evident that Tentative and Insight cluster on Factor 1 (eigenvalue 1.12, variance explained 12.44%), while Differentiation and Negation both cluster on Factor 2 (eigenvalue 1.94, variance explained 21.63%). In the Citizens’ Assembly data these were also the only four variables for which extracted communalities were above .47. Correlation tests indicate that coefficients for these two clusters are the only ones approaching medium size: Tentative and Insight (r = .30***); Negation and Differentiation (r = .21**).

Turning first to the first cluster, the Tentativeness dimension counts words like ‘maybe’, ‘fairly’, and ‘perhaps’ and measures the level of tentativeness that each text or decision maker shows. Tentativeness is theorized to be how hesitant or unsure one is about something. The Insight dimension captures the degree to which individuals vary in how much each is able to discern a more in-depth understanding of a subject or its underlying nature. The second cluster includes the Negations dimension, with words like ‘never’, ‘neither’, ‘cannot’, which is theorized to measure to what extent an individual acknowledges the absence or presence of something that is positive or affirmative; while the Differentiation dimension is theorized to capture how distinct or separate one sees concepts and ideas. People use exclusion words to help make distinctions, especially when determining whether something does or does not belong in a category.

According to Owens and Wedeking’s composition the use of both Differentiation and Negation are indicative of lower complexity. They maintain that ‘increased amounts of inclusiveness in speech are associated with higher levels of cognitive complexity’, while ‘increased amounts of exclusiveness in speech are associated with lower levels of cognitive complexity’ (Owens and Wedeking, 2011: 1056). The same is true for Negation: they find that increased usage of those words are indicative of a speaker’s desire to overcome uncertainty by excluding possibilities by means of denying. Thus, from a normative point of view increased amounts of Differentiation and Negation in speech are associated with lower levels of cognitive complexity (the interpretation followed by Wyss et al. (2015) and Owens and Wedeking (2011)).

We argue that these clusters relate to either the depth (integration) or breadth (width) of discussion (see Figure 1). The first component, CC1 (Insight and Tentative), unambiguously points to a deeper discussion involving a combination of insights indicative of higher cognitive complexity (with words such as ‘consider’, ‘wonder’) and likelihood of accommodation (words such as ‘understanding’, or ‘feeling’), combined with tentative views associated with words such as ‘ambiguity’, ‘probably’ and ‘options’. In contrast, the second component CC2 (Negation and Differentiation) amounts to a narrowing of the debate. The negation terms are of course negative and excluding with words such as ‘no’, ‘cannot’ and ‘neither’ (for an indicative list see Table A3 in Appendix 1). A similar if more nuanced impact is achieved with differentiation composed of words such as ‘whereas’, ‘versus’ and ‘haven’t’. Standardized versions of Insight and Tentative were combined to create an additive scale variable for CC1, and standardized versions of Negation and Differentiation were combined to create an additive scale variable for CC2.

Figure 1.

Dimension Clusters for Cognitive Complexity.

Analysis

We first compared the overall cognitive complexity between the Citizens’ Assembly and the parliamentary committee (see Figure 2). The CC1 mean for all speech acts was higher in the Citizens’ Assembly (M = 0.54, standard deviation = 1.60) than in the parliamentary committee (M = -0.15, standard deviation = 1.51). In contrast the CC2 mean for all speech acts was lower in the Citizens’ Assembly (M = -0.20, standard deviation = 1.47) than in the parliamentary committee (M = 0.06, standard deviation = 1.83).

Figure 2.

CC1 and CC2 Scores for all Participants in the Citizens’ Assembly and the Oireachtas Committee.

It is notable that the debate quality is higher in the Assembly than the parliamentary committee. In terms of CC1, which reflects insight and tentativeness, we can see that it is higher at the Assembly than at the committee (see the left hand box in Figure 2), which would seem to reflect higher levels of deliberative capacity, facilitation and strong norms towards respect. By contrast, when the debate switches to the more partisan party political environment of the parliamentary committee, we can see that the debate narrows with negation and differentiation (CC2) being higher there than in the Citizens’ Assembly (see right hand box).

We next turn to analysis of the individual members’ contributions in each forum (citizens in the Citizens’ Assembly, and parliamentarians in the parliamentary committee). When looking at members’ speech acts (see Figure 3), CC1 was higher in the Citizens’ Assembly (M = 0.45, standard deviation = 1.45) than in the parliamentary committee (M = -0.14, standard deviation = 1.49). And again in common with the overall pattern we find that CC2 was lower in the Citizens’ Assembly discussion (M = -0.37, standard deviation = 1.30) than in the parliamentary committee (M = 0.01, Standard deviation = 1.83). In short, H1 is supported: citizen members (at the Citizens’ Assembly) display higher levels of cognitive complexity than parliamentarians (at the parliamentary committee). But it is important to note that both do deliberate and that the parliamentary committee is not devoid of all deliberative norms.

Figure 3.

CC1 and CC2 Scores for Members in the Citizens’ Assembly (Citizens) and the Oireachtas Committee (Parliamentarians).

Finally, we examine the experts, which largely included ethicists, constitutional lawyers, genealogists and obstetricians (see Figure 4). In general, many of the same experts spoke at both events. When looking at experts speech acts alone, CC1 was higher in the Citizens’ Assembly (M = 0.55, standard deviation = 1.38) than in the parliamentary committee (M = -0.12, standard deviation = 1.33). When looking at CC2, experts CC2 was slightly lower in the Citizens’ Assembly (M = 0.25, standard deviation = 1.47) than in the parliamentary committee (M = 0.27, standard deviation = 1.70). Thus, H2 is supported: experts display higher levels of cognitive complexity when addressing the Citizens’ Assembly than the Oireachtas Committee.

Figure 4.

CC1 and CC2 Scores for All Experts in Citizens’ Assembly and Oireachtas Committee.

Conclusion

Comparing the debate quality on abortion in the Irish Citizens’ Assembly and a joint parliamentary committee, this paper drew on the concept of cognitive complexity using the LIWC dictionary to analyse the epistemic impact of deliberation. The results highlight the importance of considering this impact. While we have known that deliberation leads to epistemic improvements, we can see that the improvements may be more substantial than had previously been thought. In this case at least the randomly selected citizens appear to have reached higher epistemic standards than their parliamentarians, at least in terms of considering multi-faceted arguments. Overall, this study finds higher levels of integration and lower levels of differentiation among Citizens’ Assembly members than parliamentary committee members. This pattern was consistent in all sessions.

This finding of higher cognitive complexity in the Citizens’ Assembly than in the parliamentary committee is consistent for all types of speakers including members, and experts. For the experts, it would seem that they may respond to this different environment and to the representative decision making logic of the parliamentary committee. Thus, the non-partisan and less adversarial approach in the Citizens’ Assembly appears to have allowed the experts to discuss their expertise in more depth there than at the parliamentary committee. This could of course also be due to parliamentary committees having become more expert-informed processes (which the experts would be aware of) while Citizens’ Assembly processes are still very new and unfamiliar. And, of course for many experts the parliamentary process was their second time presenting on the issue, which may have also increased familiarity. Nonetheless, we believe that the results are indicative of the increased deliberative potential at the Citizens’ Assembly. This suggest that at least from a deliberative quality point of view there is reason to argue that deliberative mini-publics should be part of a wider deliberative and representative system.

It appears that even on a polarizing and divisive topic such as abortion ordinary citizens can engage with the issue and with complex multi-faceted arguments. They also tend towards being accommodative rather than narrowing the debate. And even more encouragingly they may behave in this way to a greater extent than parliamentarians debating the same topic. It is thus possible that the additional features of mini-public deliberation that are not present at parliamentary committees – for example, facilitation and norms of respect – may have a positive impact. More will need to be done in order to test this in other contexts both geographical and contextual. Our findings also raise interesting questions about expert engagement in deliberative processes that may warrant further research.

In terms of its empirical approach, this paper provided another application of Owens and Wedeking’s (2011) measure of cognitive complexity. The finding here, through PAF analysis, that the dimensions of their cognitive complexity measure do not load on a single factor reinforces the importance of assessing and reporting construct reliability. In this case, the dimension loadings on CC1 and CC2 align with existing cognitive complexity distinctions between depth and differentiation. However, this is something to assess on a case-by-case basis and invites further study into the reliability of the measure when applied to different case contexts. Overall, these results are encouraging for deliberation in mini-publics and highlight the epistemic contributions of citizens’ assemblies in the institutional division of labour within the wider deliberative system.