This article proposes an automated methodology for the analysis of online political discourse. Drawing from the discourse quality index (DQI) by Steenbergen et al., it applies a machine learning–based quantitative approach to measuring the discourse quality of political discussions online. The DelibAnalysis framework aims to provide an accessible, replicable methodology for the measurement of discourse quality that is both platform and language agnostic. The framework uses a simplified version of the DQI to train a classifier, which can then be used to predict the discourse quality of any non-coded comment in a given political discussion online. The objective of this research is to provide a systematic framework for the automated discourse quality analysis of large datasets and, in applying this framework, to yield insight into the structure and features of political discussions online.
StrandbergKGrönlundK.Online deliberation. In: BächtigerADryzekJSMansbridgeJet al. (eds) The Oxford handbook of deliberative democracy. Oxford: Oxford University Press, 2018, pp. 365–377.
2.
HabermasJ.The structural transformation of the public sphere: an inquiry into a category of Bourgeois society (trans. TBurger). Boston, MA: MIT Press, 1961.
3.
SteenbergenMBächtigerASteinerJ.Measuring political deliberation: a discourse quality index. Comp Eur Polit2003; 1(1): 21–48.
4.
DahlbergL.Re-constructing digital democracy: an outline of four ‘positions’. New Media Soc2011; 13(6): 855–872.
5.
LukensmeyerCJBrighamS.Taking democracy to scale: creating a Town Hall meeting for the twenty-first century. Natl Civ Rev2002; 91(4): 351–366.
6.
DryzekJNiemeyerS.Foundations and frontiers of deliberative governance. Oxford: Oxford University Press, 2010.
7.
BächtigerADryzekJSMansbridgeJet al. (eds.) The Oxford handbook of deliberative democracy. Oxford: Oxford University Press, 2018.
8.
MansbridgeJBohmanJChambersSet al. A systemic approach to deliberative democracy. In: ParkinsonJMansbridgeJ (eds) Deliberative systems: deliberative democracy at the large scale. Cambridge: Cambridge University Press, 2012, pp. 1–26.
9.
GerberMSteinerJBachtigerAet al. Deliberative and non-deliberative persuasion: mechanisms of opinion formation in Europolis. Eur Union Polit2014; 15(3): 410–429.
10.
KingM.A critical assessment of Steenbergen et al’s Discourse Quality Index. Roundhouse: J Crit Theory Pract2009; 1(1): 1–8.
11.
GoldVEl-AssadyMHautli-JaniszAet al. Visual linguistic analysis of political discussions: measuring deliberative quality. Digit Scholarsh Hum2017; 32(1): 141–158.
12.
ChambersS.Deliberative democratic theory. Annu Rev Polit Sci2003; 6: 307–326.
13.
JanssenDKiesR.Online forums and deliberative democracy. Acta Polít2005; 40(3): 317–335.
14.
GrönlundKStrandbergKHimmelroosS.The challenge of deliberative democracy online – a comparison of face-to-face and virtual experiments in citizen deliberation. Inf Polit2009; 14(3): 187–201.
15.
KiesR.Promises and limits of web-deliberation. Basingstoke: Palgrave Macmillan, 2010.
16.
BlackLWWelserHTCosleyDet al. Self-governance through group discussion in Wikipedia: measuring deliberation in online groups. Small Gr Res2001; 42(5): 595–634.
17.
KleinM.Enabling large-scale deliberation using attention-mediation metrics. Comp Support Coop W2012; 21(4–5): 449–473.
18.
DalhbergL.Computer-mediated communication and the public sphere: a critical analysis. J Comp-Mediat Comm2001. Epub ahead of print 1 October 2001. DOI: 10.1111/j.1083-6101.2001.tb00137.x.
19.
Monnoyer-SmithLWojcikS.Technology and the quality of public deliberation: a comparison between on and offline participation. Int J Electron Gov2012; 5(1): 24–49.
20.
Fournier-TombsE.A new architecture of the public sphere: online deliberation at the liberal party of Canada’s 2011 extraordinary convention. Master’s Thesis, University of Toronto, Toronto, ON, Canada, 2014.
21.
HabermasJ.The theory of communicative action, vol. I. Boston, MA: Beacon, 1984.
22.
BorgattiSPMehraABrassDJet al. Network analysis in the social sciences. Science2009; 323(5916): 892–895.
23.
BourdieuP.The forms of capital. Cult Theory: Anthol2011; 1: 81–93.
24.
MorenoJL.Who shall survive? A new approach to the problem of human interrelations. Washington, DC: Nervous and Mental Disease Publishing Co, 1934.
25.
Gonzalez-BailonSKaltenbrunnerABanchsR.The structure of political discussion networks: a model for the analysis of online deliberation. J Inf Technol2010; 25(2): 230–243.
26.
HewittJ.How habitual online practices affect the development of asynchronous discussion threads. J Educ Comput Res2003; 28(1): 31–45.
27.
KatonaZZubcsekPSarvaryM.Network effects and personal influences: the diffusion of an online social network. J Marketing Res2011; 48(3): 425–443.
28.
BlumenstockJE. Size matters: word count as a measure of quality on Wikipedia. In: Proceedings of the 17th international conference on World Wide Web, Beijing, China, 21–25 April 2008, pp. 1095–1096. New York: ACM.
29.
DiakopoulosNNaamanM.Towards quality discourse in online news comments. In: Proceedings of the ACM 2011 conference on computer supported cooperative work, Hangzhou, China, 19–23 March 2011, pp. 133–142. New York: ACM.
30.
LampeCZubePLeeJet al. Crowdsourcing civility: a natural experiment examining the effects of distributed moderation in online forums. Gov Inform Q2014; 31(2): 317–326.
31.
ParkJKlingelSCardieCet al. Facilitative moderation for online participation in eRulemaking. In: 13th annual international conference on digital government research, College Park, MD, 4–7 June 2012, pp. 173–182. New York: ACM.
32.
KilnerPGHoadleyCM. Anonymity options and professional participation in an online community of practice. In: 2005 conference on computer support for collaborative learning, Taipei, China, 30 May–4 June 2005. International Society of the Learning Sciences. New York: ACM.
