Applying argumentation to structure and visualize multi-dimensional opinion spaces

Abstract

This paper presents OpMAP: a tool for visualizing large scale, multi-dimensional opinion spaces as geographic maps. OpMAP represents opinions as labelings on a structured deductive argumentation framework. It uses probabilistic degrees of justification and Bayesian coherence measures to calculate how strongly any two opinions cohere with each other. The opinion sample is, accordingly, represented as a weighted graph, a so-called opinion graph, with opinion vectors serving as nodes and coherence values as edge weights. OpMAP partitions the nodes of the opinion graph by using clustering methods. Finally, the graph is visualized as a geographic map using a method based on a particular (e.g., force-directed) layout.

1. Introduction

This paper presents OpMAP: a tool which applies formal argumentation theory to structure and visualize large scale, multi-dimensional opinion spaces as geographic maps. The purpose of this paper is to provide a proof of concept. It describes the design of the tool and applies it to a simple illustrative example.

To understand how people judge a policy proposal, it is important to know what they think about other related issues. For example, one’s stance towards brexit will typically depend on one’s views about immigration, globalization, assessment of Britain’s economy, international cooperation, and a variety of further matters. Taking such dependencies into account, different persons’ views about brexit can be construed as high-dimensional opinion vectors. OpMAP visualizes high-dimensional opinion samples as 2D maps. This visualization is supposed to provide structural insights into the opinion sample, to help individual persons to locate themselves in the opinion landscape, and to serve as base layer for visualizing additional information, e.g. as overlays.

The theoretical background and basic outline of OpMAP can be summarized in five steps:

• A simplified structured argumentation framework is used to represent the relations between the various issues which are considered in the policy survey, resp. in the opinion data.

• An opinion vector is modeled as a labeling on the structured argumentation framework.

• The structured argumentation framework allows one to calculate the extent to which any two opinion vectors, i.e. labelings, cohere with each other. The opinion sample is represented as a weighted graph, with opinion vectors serving as nodes and coherence values as edge weights.

• The nodes of the opinion graph are partitioned by using clustering methods.

• The graph is visualized as a map using a method based on a particular (e.g., force-directed) layout.

The paper proceeds as follows: The remainder of this introductory section reviews related work on argument visualization, visualization of opinion data, as well as formal argumentation theory and graph drawing. Section 2 describes the main theoretical components of OpMAP, it is organized in accordance with the five steps specified above. Following the same structure, Section 3 develops the proof of concept by means of a simple case – a stylized reconstruction of the debate about eating meat – which serves to illustrate OpMAP’s different components as well as the theoretical framework it builds upon. Finally, Section 4 identifies challenges for future research; it concludes this paper by setting forth a list of general criteria for evaluating OpMAP and assessing the quality of visualizations generated with this tool.

1.1. Visualization tools for argument analysis

Visualizing argumentative reasoning nearly always aims at depicting the structure of argumentation [31]. Visual devices such as debate graphs, argument maps, and inference diagrams lay out the structure of arguments or debates with varying degrees of detail and scope. Software tools such as Lasad [24], OVA [22], Rationale [33], Carneades [16], Debategraph.org [8], or Argunet [39] allow one to display, or to collaboratively construct such a map.

Argument maps, which are the state-of-the-art for presenting the results of complex argumentative analyses, are no doubt useful visual devices for experts, argument analysts and critical thinking instructors [18,43]. But there is more to a debate than its arguments and their interrelations – namely the way proponents, stakeholders, or citizens assess the arguments and claims. What a debate’s proponents believe is no less important for decision-making than the debate’s argumentative structure. Against this background, this paper suggests to explore a visualization tool that complements argument maps: opinion maps. Rather than visualizing the argumentative structure (of a debate) itself; opinion maps visualize the argumentatively structured opinion landscape of a debate. OpMAP can hence be understood as a novel species in the “ecosystem of argumentation technologies” [28,34].

1.2. Visualization tools for opinion analysis

While charts (timelines, pie charts, bar charts etc.) seem to be the standard device for visualizing opinion data, three more specific visualization methods that are related to the OpMAP concept can be identified: (1) geographic mapping of regional opinion data; (2) mapping opinions in a topic space; (3) social network visualization.

Re (1): If a survey yields opinion data for different geographic regions (for example, in the U.S., counties or states), this data can be visualized by coloring an actual geographic map accordingly. Presenting the degree of public ascent to key findings of climate science across the U.S. (different map for each finding), the Yale Climate Opinion Maps [20] illustrate this approach. Unlike these visualizations, however, OpMAP does not color a real map; it rather constructs a geographic-like map fully on its own; moreover, OpMAP visualizes not only one belief dimension but complex opinion vectors.

Re (2): ConToVi is a digital research instrument for policy and discourse analysis [10]. One of its central features is the so-called topic space, a radial representation of all topics that are touched upon in a discussion. Speakers and their individual utterances can be mapped into that topic space according to how strongly they relate to the different topics. So, the topic space from ConToVi is a 2D representation of the opinion landscape of a debate. However, unlike OpMAP, this opinion space is not argumentatively structured at all; it is merely based on results from topic analysis.

Re (3): It is standard to represent social networks as graphs, to carry out cluster analysis on these graphs and to draw these graphs (typically by means of force-directed methods) [36]. OpMAP builds on a particular kind of social network, which represent agents only in terms of their complex belief states and which determines the relations between agents just with regard to argumentative relations between their belief states. OpMAP therefore uses methods from social network analysis to study and visualize argumentative networks and epistemic communities [46].

1.3. Theoretical background

OpMAP builds on a variety of well-studied formal models and algorithms. In particular, it construes policy debates as deductive argumentation [2–4,6], i.e., as arguments whose inferences are necessarily truth-preserving. This requires that arguments which instantiate standard, non-deductive schemes for practical reasoning [29,45] are reconstructed as deductively valid, e.g. by adding a corresponding general principle as connecting premiss [25]. Likewise, defeasible arguments can be represented as deductive inferences by adding the defeasible inference principle as premiss. Yet, formally speaking, deductive argumentation frameworks are a special case of structured argumentation frameworks such as ASPIC+ [32]. There are seemingly no principal reasons why the tool presented in this article shouldn’t be generalizable to structured argumentation frameworks, which allow for deductive as well as defeasible arguments.

Moreover, OpMAP adapts and uses Bayesian measures of coherence [9,12,38] in conjunction with probabilistic degrees of justification defined on complex deductive argumentation [5].

Finally, OpMAP relies upon – and might ultimately have to improve – the so-called GMap algorithm [15] in order to cluster and draw the weighted opinion graphs.

2. Opinion map

This section describes the main theoretical components of OpMAP.

2.1. Simple structured argumentation framework

The sentence pool S is supposed to contain all natural-language sentences that are relevant to a debate. We assume that two natural-language sentences can contradict each other or be equivalent – relations which we represent in our simple model. Formally, S is a set of sentences which is closed under negation (i.e., $\mathit{s}\in \mathit{S}⟹\neg \mathit{s}\in \mathit{S}$ , stipulating that $\mathit{s}=\neg \neg \mathit{s}$ ). Thus, S can be represented as a set of unnegated sentences ${\mathit{S}}^{+}$ conjoined with their negations ${\mathit{S}}^{-}$ . Let $\mathit{N}:=|\mathit{S}|\mathrm{/}2$ .

Arguments are inferences from premisses to a conclusion, where both premisses and conclusions are sentences in S. Formally, an argument a on a sentence pool S is a tuple $⟨{\mathit{P}}_{\mathit{a}}\mathrm{,}{\mathit{c}}_{\mathit{a}}⟩$ with ${\mathit{P}}_{\mathit{a}}\subseteq \mathit{S}$ and ${\mathit{c}}_{\mathit{a}}\in \mathit{S}$ , where ${\mathit{P}}_{\mathit{a}}$ denotes its set of premisses, and ${\mathit{c}}_{\mathit{a}}$ its conclusion. It is assumed that all arguments represent deductively valid inferences.

A set of arguments A constructed on a sentence pool S represents a simple structured argumentation framework (SSAF), $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ . SSAFs can be visualized as argument maps, where support and attack relations between arguments are defined in the following way: argument a supports (attacks) argument b if a’s conclusion is equivalent (contradictory) to one of b’s premisses.

2.2. Multi-dimensional opinions in the structured AF

The opinions of an agent in a debate – i.e., her position – is modeled as a partial truth-value assignment to the sentences that appear in the debate.

Definition 1 (Position).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and ${\mathit{S}}_{\mathit{D}}\subseteq \mathit{S}$ . A position $\mathcal{A}$ with domain ${\mathit{S}}_{\mathit{D}}$ is a truth-value assignment to sentences in ${\mathit{S}}_{\mathit{D}}$ , $$\begin{array}{c}\mathcal{A}:{\mathit{S}}_{\mathit{D}}\to \{\mathrm{True}\mathrm{,}\mathrm{False}\}.\end{array}$$

A position $\mathcal{A}$ extends a position $\mathcal{B}$ iff $\mathcal{A}\supset \mathcal{B}$ .

A position $\mathcal{A}$ with domain ${\mathit{S}}_{\mathit{D}}$ is called

complete iff ${\mathit{S}}_{\mathit{D}}=\mathit{S}$ ,

partial iff ${\mathit{S}}_{\mathit{D}}\subset \mathit{S}$ ,

atomic iff $|{\mathit{S}}_{\mathit{D}}|=1$ , and

tautological iff ${\mathit{S}}_{\mathit{D}}=\varnothing$ .

The tautological position is denoted with ⊤.

Equivalently, a position can be expressed as a two-valued index-function over S or a three-valued index function over ${\mathit{S}}^{+}$ . Positions can also be identified with subsets of S. Positions are minimally consistent iff they don’t assign the same truth-value to two contradictory sentences, that is:

Definition 2 (Minimally consistent position).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF. A complete position $\mathcal{A}$ on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ is minimally consistent iff $$\begin{array}{c}\forall \mathit{p}\in \mathit{S}:\mathcal{A}\mathrm{(}\mathit{p}\mathrm{)}\ne \mathcal{A}\mathrm{(}\neg \mathit{p}\mathrm{)}.\end{array}$$

A partial position $\mathcal{B}$ on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ is minimally consistent iff there exists a complete and minimally consistent position which extends $\mathcal{B}$ .

A consistent position, besides being minimally consistent, accounts for the inferential relations between sentences that are established by the various arguments in the SSAF.

Definition 3 (Consistent position).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF. A complete position $\mathcal{A}$ on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ is consistent iff it is minimally consistent and $$\begin{array}{c}\forall \mathit{a}\in \mathit{A}:\text{If }\mathrm{(}\forall \mathit{p}\in {\mathit{P}}_{\mathit{a}}:\mathcal{A}\mathrm{(}\mathit{p}\mathrm{)}=\mathrm{True}\mathrm{)}\mathrm{,}\text{then }\mathrm{(}\mathcal{A}\mathrm{(}{\mathit{c}}_{\mathit{a}}\mathrm{)}=\mathrm{True}\mathrm{)}.\end{array}$$

A partial position $\mathcal{B}$ on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ is consistent iff there exists a complete and consistent position which extends $\mathcal{B}$ .

Finally, we define:

Definition 4 (Mutually consistent position).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF. Two partial positions $\mathcal{A}\mathrm{,}\mathcal{B}$ on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ are mutually consistent iff there exists a complete and consistent position which extends $\mathcal{A}$ and which extends $\mathcal{B}$ .

2.3. Mutual agreement between belief states

An opinion landscape is given through a set of positions plus information about how strongly any two positions agree / disagree with each other. But how can one measure mutual agreement / disagreement? Such a formal measure should meet the following desiderata:

(ArgAgre)

It represents how well two positions fit together given the arguments of the debate.

(IncoDis)

It is able to distinguish different degrees of disagreement for mutually inconsistent positions.

(EquiMax)

It assigns maximal degree of mutual agreement to two identical positions.

Desideratum (ArgAgre) excludes standard distance measures – such as, e.g., Hamming distance, or edit distance, which work on vector representations of positions only – as suitable measures of mutual agreement. That is because such standard distance measures ignore argumentative relations between the sentences.

Bayesian epistemology is arguably the most influential and advanced research program in contemporary epistemology [41]. It therefore seems promising to rely on Bayesian epistemology in constructing a measure of argumentative agreement between positions. This requires one, however, to introduce a probabilistic measure of degree of justification in the first place, which the following three definitions accomplish [5].

Definition 5 (Set of all complete positions which extend a position).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathcal{A}$ be a position. ${\mathrm{\Gamma }}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}$ shall denote the set of all consistent and complete positions on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ . Then $$\begin{array}{l}{\mathrm{\Gamma }}_{\mathcal{A}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}:=\{\mathcal{X}\in {\mathrm{\Gamma }}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}:\mathcal{X}\supset \mathcal{A}\}\mathrm{,}\\ {\mathrm{\Gamma }}_{\neg \mathcal{A}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}:=\{\mathcal{X}\in {\mathrm{\Gamma }}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}:\mathcal{X}\not\supset \mathcal{A}\}.\end{array}$$

The spread of a single position $\mathcal{A}$ refers to the number of consistent and complete positions that extend $\mathcal{A}$ . More generally, the spread of a sequence of positions $\mathrm{(}\neg \mathrm{)}{\mathcal{A}}_1\mathrm{,}\dots \mathrm{,}\mathrm{(}\neg \mathrm{)}{\mathcal{A}}_{\mathit{n}}$ is the number of consistent and complete positions that simultaneously extend the unnegated positions and do not extend the negated ones. Formally:

Definition 6 (Spread of positions).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF. Then the spread of the tautological position, or total spread, is defined as $$\begin{array}{rcl}{\mathit{\sigma }}_{\top }& :=\mathit{\sigma }& :=\left|{\mathrm{\Gamma }}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}\right|.\end{array}$$ Moreover, let ${\mathcal{A}}_1\mathrm{,}\dots \mathrm{,}{\mathcal{A}}_{\mathit{n}}$ be n positions on $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ . And the spread of the sequence of (possibly negated) positions is given by $$\begin{array}{rcl}{\mathit{\sigma }}_{\mathrm{(}\neg \mathrm{)}{\mathcal{A}}_1\mathrm{,}\dots \mathrm{,}\mathrm{(}\neg \mathrm{)}{\mathcal{A}}_{\mathit{n}}}& :=& \left|{\displaystyle \bigcap }_{\mathit{i}=1}^{\mathit{n}}{\mathrm{\Gamma }}_{\mathrm{(}\neg \mathrm{)}{\mathcal{A}}_{\mathit{i}}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}\right|.\end{array}$$

The degree of justification of a position is simply its spread, divided by the tautological position’s spread.

Definition 7 (Degree of justification).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathcal{A}$ , $\mathcal{B}$ be two positions. The degree of justification of $\mathcal{A}$ in $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ is defined as $$\begin{array}{c}\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{A}\mathrm{)}:={\mathit{\sigma }}_{\mathcal{A}}\mathrm{/}\mathit{\sigma }.\end{array}$$ Assuming that $\mathcal{B}$ is consistent, the partial degree of entailment of $\mathcal{A}$ by $\mathcal{B}$ in $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ can be defined as $$\begin{array}{c}\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{A}|\mathcal{B}\mathrm{)}:={\mathit{\sigma }}_{\mathcal{A}\mathrm{,}\mathcal{B}}\mathrm{/}{\mathit{\sigma }}_{\mathcal{B}}.\end{array}$$

As degrees of justification are probabilities (i.e., satisfy the probability axioms), they enable one to import the formal concepts and mathematical results of Bayesian epistemology into the deductive argumentative framework [5]. In particular, a range of Bayesian coherence measures [9,12,38] are now available for setting up an argumentative measure of mutual agreement/disagreement between any two positions.

Desideratum (IncoDis), according to which a suitable agreement measure is able to assign different degrees of disagreement to pairs of mutually inconsistent positions, rules out simple coherence measures, such as the naive Shogenji measure [38]. The question which Bayesian coherence measures are best equipped to deal with inconsistent sets of statements has been discussed extensively in the literature [38,40]. For the purpose of this article, a particular so-called mutual support measure is chosen as starting point [12]; this measure, in turn, relies on the Kemeny–Oppenheim measure of confirmation (see below). These choices seem to account for desideratum (ArgAgre) in a natural way, because mutual support measures assume that coherence depends on the degree to which various parts of a position support each other, which in turn is a function of the given arguments in the debate. Yet, the use of alternative coherence measures should be explored and assessed in future research.

Now, standard Bayesian coherence measures are supposed to assess the internal coherence of a set of statements, whereas, in this context, a measure of mutual coherence (mutual agreement) between two sets of statements (i.e., two positions) is needed. The following definitions adapt a mutual support measure of coherence for these purposes.

Definition 8 (Mutual coherence).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathcal{A}\mathrm{,}\mathcal{B}$ be two minimally consistent positions (with domains ${\mathit{D}}_{\mathcal{A}}\mathrm{,}{\mathit{D}}_{\mathcal{B}}$ and with size ${\mathit{k}}_{\mathcal{A}}=|{\mathit{D}}_{\mathcal{A}}|\mathrm{,}{\mathit{k}}_{\mathcal{B}}=|{\mathit{D}}_{\mathcal{B}}|$ respectively). Then $$\begin{array}{rcl}MutCoh\mathrm{(}\mathcal{A}\mathrm{,}\mathcal{B}\mathrm{)}& :=& {\displaystyle \frac{1}{2^{{\mathit{k}}_{\mathcal{A}}+1}}}{\displaystyle \sum }_{\mathcal{X}\subseteq \mathcal{A}}Conf\mathrm{(}\mathcal{X}\mathrm{,}\mathcal{B}\mathrm{)}\\ & & +{\displaystyle \frac{1}{2^{{\mathit{k}}_{\mathcal{B}}+1}}}{\displaystyle \sum }_{\mathcal{X}\subseteq \mathcal{B}}Conf\mathrm{(}\mathcal{X}\mathrm{,}\mathcal{A}\mathrm{)}\end{array}$$ where $Conf\mathrm{(}\mathcal{X}\mathrm{,}\mathcal{Y}\mathrm{)}$ is a Bayesian confirmation measure, indicating to which extent $\mathcal{Y}$ confirms $\mathcal{X}$ in $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ .

In calculating mutual coherence, the following confirmation measure is relied upon.

Definition 9 (Kemeny–Oppenheim confirmation).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathcal{X}\mathrm{,}\mathcal{Y}$ be two minimally consistent positions. Then the Kemeny–Oppenheim measure of confirmation of $\mathcal{X}$ by $\mathcal{Y}$ is defined as $$\begin{array}{c}{Conf}^{\mathrm{KO}}\mathrm{(}\mathcal{X}\mathrm{,}\mathcal{Y}\mathrm{)}:=\left\{\begin{array}{cc}\frac{\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{Y}|\mathcal{X}\mathrm{)}-\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{Y}|\neg \mathcal{X}\mathrm{)}}{\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{Y}|\mathcal{X}\mathrm{)}+\mathrm{\text{D}}\mathrm{\text{OJ}}\mathrm{(}\mathcal{Y}|\neg \mathcal{X}\mathrm{)}}\hfill & \mathrm{if}\mathcal{Y}⊭\mathcal{X}\wedge \mathcal{Y}⊭\neg \mathcal{X}\hfill \\ 1\hfill & \mathrm{if}\mathcal{Y}\vDash \mathcal{X}\hfill \\ -1\hfill & \mathrm{if}\mathcal{Y}\vDash \neg \mathcal{X}\mathrm{,}\hfill \end{array}\right.\end{array}$$ where $\mathcal{Y}\vDash \mathcal{X}$ iff ${\mathrm{\Gamma }}_{\mathcal{Y}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}\subseteq {\mathrm{\Gamma }}_{\mathcal{X}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}$ .

Note that, according to these definitions, $MutCoh\mathrm{(}\mathcal{A}\mathrm{,}\mathcal{B}\mathrm{)}$ is maximal (i.e., equals 1) if and only if $\mathcal{A}$ and $\mathcal{B}$ are equivalent (i.e., iff ${\mathrm{\Gamma }}_{\mathcal{A}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}={\mathrm{\Gamma }}_{\mathcal{B}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}$ ). The desideratum (EquiMax) is hence satisfied (see Appendix A for a proof).

The mutual coherence measure can be applied either to original positions in the opinion sample or to their deductive closures (which include all sentences implied by an original position).

Definition 10 (Deductive closure).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathcal{A}$ be a position. The deductive closure $\overline{\mathcal{A}}$ of $\mathcal{A}$ is defined as $$\begin{array}{c}\overline{\mathcal{A}}:={\displaystyle \bigcap }{\mathrm{\Gamma }}_{\mathcal{A}}^{⟨\mathit{A}\mathrm{,}\mathit{S}⟩}.\end{array}$$

This means that MutCoh can spell out mutual agreement in at least two different ways (i.e., with respect to the original or the deductively closed positions). It is up to future research to figure out how this choice affects the analysis and visualization of the opinion landscape. As far as the proof of concept in Section 3 is concerned, MutCoh will be applied to the original positions.

The concept of an opinion graph explicates the metaphor of an opinion landscape and fuses the results of this subsection.

Definition 11 (Opinion graph).

Let $⟨\mathit{A}\mathrm{,}\mathit{S}⟩$ be a SSAF, and let $\mathbf{O}=\{{\mathcal{A}}_1\mathrm{,}\dots \mathrm{,}{\mathcal{A}}_{\mathit{m}}\}$ , be a set of minimally consistent positions. The corresponding opinion graph is the complete undirected and weighted graph on O, where edge weights are given by the mutual coherence MutCoh between the two adjacent position nodes.

2.4. Network analysis

In order to understand how the opinion landscape is structured – i.e., which belief states form clusters, and how big these clusters are, or how influential these belief states are – , network analysis methods are applied to the opinion graph.

When applying network analysis methods to the opinion graph, two basic challenges arise:

1. The opinion graph contains edges with negative weight.

2. The opinion graph is complete.

Most network analysis and clustering methods are per se not able to cope with negative edge weights. Moreover, the internal structure of complete graphs is comparatively hard to discover, and most network analysis algorithms primarily consider sparse graphs.

Both challenges can be addressed by transforming edge weights and filtering edges. In principle, this can be done in many different ways. And it would be worthwhile to investigate these alternatives in the future. In order to come up with a proof of concept, this paper relies on a simple linear transformation of edge weights (namely: add 1); as regards filtering, all edges with a transformed edge weight below 0.2 are removed.

2.5. Graph drawing

Network visualization methods can further enhance one’s understanding of the structure of the opinion graph. The most straight-forward network visualization metaphor represents nodes of the graph as points on the plane and edges as curves connecting the corresponding nodes. So-called force-directed algorithms construct such visualizations; these methods aim at placing connected nodes close to each other (see e.g. [23]). This complies with an intuitive depiction of relation among objects. (Thus, within Gestalt theory of human perception, it has been verified numerous times that humans tend to think about objects as related when the distance between them is relatively small.)

While force-directed methods are very popular in practice, since they produce fairly readable drawings for sparse graphs, dense graphs represent a challenge for these methods. Also, the methods do not allow for negative edge weights. Thus, in order to make standard force-directed algorithms applicable, the edge weights are transformed so that all values are positive, and the low-weighted edges are filtered, as already discussed.

A simple and classical way to show the results of the clustering algorithm on the visualization is to color the nodes according to the cluster they belong to. To emphasize the clustering even more and to explore the human familiarity with geographic maps, Gansner et al. [14] have introduced the algorithm GMap, which extends force-directed algorithms and constructs visualizations resembling a geographic map. In this visualization each cluster is depicted by a polygon (country), the nodes that belong to the cluster are represented by points lying within this polygon, and two polygons that contain many pairs of connected edges share a boundary (i.e., are neighboring countries). It has been verified empirically that humans perform better with this map-resembling type of visualization than with simple colored visualizations when it comes to so-called group-based tasks, which are tasks that can be performed by considering nodes, edges, and clusters. Additionally, the performance for other types of tasks, where only nodes or only nodes and edges have to be considered, does not deteriorate [35]. These findings suggest using the GMap method for visualizing the clustered opinion graph.

Applying the GMap algorithm to the opinion graph poses new challenges for the method which stem from the semantic structure of the data. They will be outlined in Section 4.

3. Proof of concept

This section illustrates the formal components of OpMAP by means of a stylized reconstruction of the debate about eating meat – in short: the veggie debate. It develops, in structural analogy to Section 2, a simple case study, which establishes the proof of concept.

Fig. 1.

Visualization of the stylized reconstruction of the veggie debate as argument map. Besides arguments (filled boxes), the three core claims are also shown.

3.1. Simple structured argumentation framework

In the given simplified argumentative analysis of the veggie debate, three propositions take center stage:

• [Eat what you want]: It is permissible to eat animal products of any origin and in any amount.

• [No mass farming]: It is impermissible to eat meat which has been produced in conventional ways, esp. through mass-farming techniques.

• [Vegetarianism]: It is impermissible to eat meat.

The proposition [Eat what you want] is supported by the argument <Moralizing>. The proposition [No mass farming] is supported by the arguments <Conservation> and <Animal suffering>. The proposition [Vegetarianism] is supported by the arguments <Animal rights> and <Global warming>, it is attacked by the <Culinary argument>. Finally, the argument <Consumer not responsible> attacks a common premiss of the arguments <Conservation>, <Animal suffering>, and <Global warming>. Figure 1 displays the macro structure of the debate as an argument map.

All arguments are reconstructed as premiss-conclusion structures with deductively valid inferences. For instance, the detailed reconstruction of the <Culinary argument> reads,

1. The most delicious meals involve meat.

2. It is permissible to eat the most delicious meals.

3. If it is permissible to eat some meal, then it is permissible to eat whatever is included in that meal.

4. Thus (from 1-3): One may eat meat.

The number of premisses per argument varies, in the debate, from 2 to 6; in total, the sentence pool contains $2\times 26$ sentences. The complete, detailed reconstruction of the illustrative debate is given in Appendix B and additionally available as supplementary online material.

3.2. Multi-dimensional opinions in the structured AF

In future applications of OpMAP, opinions of participants, stakeholders and citizens will be elicited by means of a survey, will be manually reconstructed from a discussion [1], or could possibly be identified by means of stance detection and opinion mining algorithms [7]. In order to demonstrate the feasibility of OpMAP, this subsection, however, simply sets up a small fictitious opinion sample. Table 1 defines and describes seven consistent partial positions in the veggie debate’s SSAF. In terms of core claims, positions 1 and 2 accept [Eat what you want], positions 3, 4 and 5 accept [No mass farming], and positions 6 and 7 accept both [Vegetarianism] and [No mass farming]. In the following, these positions will serve as an illustrative opinion sample.

Table 1

Seven consistent partial positions defined on the veggie debate SSAF. “T”, “F” and blank space stand for true, false and judgement suspension, respectively. Sentences in the sentence pool are referenced and arranged row-wise in accordance with their function (premiss/conclusion) in the various arguments; in case one sentence appears in several arguments, different rows will represent one and the same sentence

		Consistent partial positions
		No. 1	No. 2	No. 3	No. 4	No. 5	No. 6	No. 7
Core claims
[Eat what you want]		T	T
[No mass farming]				T	T	T	T	T
[Vegetarianism]							T	T
Arguments
<Moralizing>	(P1)	T	F
	(P2)	T
	(C)	T	T
<Conservation>	(P1)			T			T
	(P2)			T			T
	(P3)			T			T
	(C)			T	T	T	T	T
<Animal suffering>	(P1)			T	T			T
	(P2)			T	T			T
	(P3)			T	T			T
	(C)			T	T	T	T	T
<Animal rights>	(P1)	T						T
	(P2)						F	T
	(C)						T	T
<Global warming>	(P1)						T
	(P2)						T
	(P3)						T
	(C1)						T
	(P4)						T
	(P5)						T
	(C2)						T	T
<Culinary argument>	(P1)		T		T
	(P2)		T		T
	(P3)		T		T
	(C)		T		T
<Consumer not resp.>	(P1)	T				F
	(P2)	T
	(C)	T

3.3. Mutual agreement between belief states

Mutual coherence, as defined in the Section 2.3, can be calculated for every pair of positions in the opinion sample (Table 1). This yields the edge weights of the corresponding opinion graph. Figure 2 displays the weighted adjacency matrix – visualized as a heat map – of this opinion graph.

3.4. Network analysis

As explained above, the original opinion graph includes edges with negative weights; hence the edge weights are transformed (simply by adding 1); and the graph is optionally filtered by removing all edges with a transformed edge weight below 0.2. For illustrative purposes, different clustering methods available in Mathematica are applied to both the unfiltered as well as the filtered transformed opinion graph. As Table 2 shows, filtering affects the results and tends to improve the ability to find clusters, as expected. Especially the clusters obtained with modularity and centrality make sense in view of the design of the opinion sample.

3.5. Graph drawing

Figure 3 displays the opinion graph and two clusterings found by modularity (with and without filtering). Clusters are highlighted through the use of colors and curves that surround each cluster.

Figure 4 shows the result of the application of the GMap algorithm to the filtered opinion graph with the clustering found by modularity. As can be seen, the small illustrative opinion graph yields a plain and simple visualization, by far more clear and intuitive than the visualization of Fig. 3. In addition, the layout seems to reflect the structure of the opinion landscape in a sensible way. Thus, for example, the far-right position “1” is the only position in the sample which accepts [Eat what you want] because of the <Moralizing> and the <Consumer not responsible> arguments, which are not accepted by any other position in the sample, as can be seen from Table 1.

4. Discussion

This concluding section highlights challenges and directions of future research; it suggests, in particular, criteria for assessing opinion graph drawings which may be used in future investigations.

Fig. 2.

Heatmap representation of the weighted adjacency matrix of the illustrative opinion graph. Shading corresponds, in accordance with the right-hand legend, to the degree of mutual coherence between a given pair of positions.

Table 2

Different clustering methods (modularity, hierarchical, centrality) applied to the unfiltered and filtered illustrative opinion graph. All original edge weights are increased by 1. In the filtered graph, edges with weight below 0.2 are removed. Default Mathematica^TM implementations of the clustering methods are used (version 11.0.1.0)

	modularity	hierarchical	centrality
unfiltered	$\{\{3\mathrm{,}4\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\mathrm{,}\{1\mathrm{,}2\}\}$	$\{\{1\mathrm{,}2\mathrm{,}3\mathrm{,}4\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\}$	$\{\{3\mathrm{,}4\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\mathrm{,}\{1\mathrm{,}2\}\}$
filtered	$\{\{3\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\mathrm{,}\{1\mathrm{,}2\mathrm{,}4\}\}$	$\{\{3\mathrm{,}4\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\mathrm{,}\{1\}\mathrm{,}\{2\}\}$	$\{\{3\mathrm{,}4\mathrm{,}5\mathrm{,}6\mathrm{,}7\}\mathrm{,}\{1\mathrm{,}2\}\}$

Fig. 3.

Visualization of the clustered unfiltered (left) and filtered (right) illustrative opinion graph. Communities are identified with modularity clustering. The clusters are depicted by the color of the nodes and additionally by a curve around each cluster. Edge colors correspond to edge weights. The visualization is constructed by the Community Layout algorithm of Mathematica.

Fig. 4.

Geographic visualization of the filtered illustrative opinion graph; generated with the GMap implementation from http://gmap.cs.arizona.edu/ (date 07/11/17).

The simple case study presented in Section 3 establishes the feasibility of OpMAP and constitutes, accordingly, a proof of concept. A basic follow-up question is, obviously, whether OpMAP scales to larger debates and larger opinion samples.

Large debates and opinion samples pose specific problems for OpMAP’s different components:

• The calculation of mutual coherence values is computationally demanding: To start with, determining a single degree of justification requires the solution of a specific #SAT problem (model counting). When mutual coherence between two positions is calculated, ever new degrees of justification have to be determined for all subsets of the two positions at hand. So, it is an important question for OpMAP whether mutual coherence values can be reliably determined by stochastic algorithms (e.g., Markov Chain Monte Carlo), by heuristics (which may, e.g., explore the structure of the SSAF), or by machine learning methods.

• Under the assumption that opinion graphs are relatively dense, large opinion graphs will be difficult to draw in a meaningful way; especially force-directed layouts, which lie at heart of the GMap algorithm, are prone to yield mere “hair-ball” plots. Future research has to investigate how this issue can be overcome [26].

• Clustering, too, will become more difficult for large opinion graphs. That is because, in large graphs, one may expect more so-called hub nodes. These hub nodes are generated by ‘small’ partial positions, which suspend judgement vis-à-vis most of the sentences of the debate. These positions will cohere strongly with many other positions, which translates, in terms of the graph, into many heavily-weighted edges. Tailored filtering methods or other techniques might be needed to handle such hubs [37].

Preliminary evidence from an ongoing exhibition project suggests that the scalability issues can be overcome [27]. It is left to future work to analyze this large-scale implementation of OpMAP in detail.

Two further challenges do not relate directly to scalability. First, a useful opinion map of a debate will have to display meaningful labels – both for individual positions but also, and more importantly, for the different clusters, or “countries”, in the map. It should be investigated whether existing techniques, e.g. from topic labeling [21], can be applied and adapted to generate opinion cluster labels. Second, some foreseen applications of OpMAP involve dynamic updates of the opinion map. This poses novel challenges for clustering and drawing in terms of stability, i.e., neither the clusters nor the drawing should change completely once a new opinion is inserted into the sample [17,19].

Future research will have to study how OpMAP must be extended or modified in order to cope with the aforementioned challenges. For this sort of investigation, it is important to have explicit normative standards against which different versions and implementations of the OpMAP concept are evaluated.

The analogy between opinion maps and geographic maps suggests the following criteria:

(Meaningful Countries)

Opinions in one “country” (cluster) should cohere with one another much more strongly than with opinions outside that “country”.

(Country Size)

The size (area) of a “country” in the map should correspond to the number of agents whose opinions are located in that “country”.

(Distance)

The Euclidean distance between the locations of any two opinions in the drawing should correspond to the degree of mutual coherence between the two opinions.

The informal description of these criteria allows for different mathematical explications (e.g., rank correlation versus standard correlation). (Meaningful Countries), (Country Size) and (Distance) should be seen as umbrella criteria that organize a broad range of different statistical measures. Such precise statistical measures should be applied, in future research, in a context-sensitive way and in function of the corresponding investigation’s purpose.