Linguistic Explorations of Societies (LES) is an interdisciplinary research project with scholars from the fields of political science, computer science, and computational linguistics. The overarching ambition of LES has been to contribute to the survey-based comparative scholarship by compiling and analyzing online text data within and between languages and countries. To this end, the project has developed an online semantic lexicon, which allows researchers to explore meanings and usages of words in online media across a substantial number of geo-coded languages. The lexicon covers data from approximately 140 language–country combinations and is, to our knowledge, the most extensive free research resource of its kind. Such a resource makes it possible to critically examine survey translations and identify discrepancies in order to modify and improve existing survey methodology, and its unique features further enable Internet researchers to study public debate online from a comparative perspective. In this article, we discuss the social scientific rationale for using online text data as a complement to survey data, and present the natural language processing–based methodology behind the lexicon including its underpinning theory and practical modeling. Finally, we engage in a critical reflection about the challenges of using online text data to gauge public opinion and political behavior across the world.
AntoniakM.MimnoD. (2018). Evaluating the stability of embedding-based word similarities. Transactions of the Association for Computational Linguistics, 6, 107–119. MIT Press.
4.
BaniaminH. M. (2020). Citizens’ inflated perceptions of the extent of democracy in different African countries: Are individuals’ notions of the state an answer to the puzzle?Zeitschrift für Vergleichende Politikwissenschaft, 14(4), 321–343.
5.
BehlingO.LawK. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. SAGE.
6.
BeullensK.LoosveldtG.VandenplasC.StoopI. (2018). Response rates in the European social survey: Increasing, decreasing, or a matter of fieldwork efforts? .Survey Methods: Insights from the Field. https://surveyinsights.org/?p=9673
7.
BolukbasiT.ChangK. W.ZouJ.SaligramaV.KalaiA. (2016). Man is to computer programmer as woman is to homemaker? Debiasing Word Embeddings. arXiv preprint arXiv:1607.06520.
8.
BrattonM.MattesR. (2001). Support for democracy in Africa: Intrinsic or instrumental?British Journal of Political Science, 31(3), 447–474.
9.
BraunM.HarknessJ. (2005). Text and context: Challenges to comparability in survey questions. In Hoffmeyer-ZlotnokJ. H. P.HarknessJ. (Eds.), Methodological aspects in cross-national research. GESIS-ZUMA, 5–10.
10.
DahlbergS.AxelssonS.HolmbergS. (2020). Democracy in context: Using a distributional semantic model to study differences in the usage of democracy across languages and countries. Zeitschrift für Vergleichende Politikwissenschaft, 14(4), 425–459.
11.
DahlbergS.HolmbergS. (2014). Democracy and bureaucracy: How their quality matters for popular satisfaction. West European Politics, 37(3), 515–537.
12.
DahlbergS.LindeJ.HolmbergS. (2015). Democratic discontent in old and new democracies: Assessing the importance of democratic input and governmental output. Political Studies, 63, 18–37.
13.
DaltonR. J.SinT. C.JouW. (2007). Understanding democracy: Data from unlikely places. Journal of Democracy, 18(4), 142–156.
14.
De VriesE.SchoonveldeM.SchumacherG. (2018). No longer lost in translation: Evidence that google translate works for comparative bag-of-words text applications. Political Analysis, 26(4), 417–430.
15.
EastonD. (1975). A re-assessment of the concept of political support. British Journal of Political Science, 5(4), 435–457.
16.
EU General Data Protection Regulation. Regulation (EU) 2016/679 of the European parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). OJ 2016 l 119/1. https://eur-lex.europa.eu/eli/reg/2016/679/oj
17.
FerrínM.KriesiH. (Eds.). (2016). How Europeans view and evaluate democracy. Oxford University Press.
18.
FirthJ. R. (1957). A synopsis of linguistic theory 1930-1955 (in special volume of the Philological Society). Oxford University Press.
19.
FutrellR. (2019). Information-theoretic locality properties of natural language. In Proceedings of the first workshop on quantitative syntax (Quasy, Syntaxfest 2019) (pp. 2–15). https://aclanthology.org/W19-7902/
20.
GarciaD.GarasA.SchweitzerF. (2012). Positive words carry less information than negative words. EPJ Data Science, 1(1), 1–12.
21.
GraveE.BojanowskiP.GuptaP.JoulinA.MikolovT. (2018). Learning word vectors for 157 languages. arXiv preprint arXiv:1802.06893
22.
GyllenstenA. C.EkgrenA.SahlgrenM. (2019, May). R-grams: Unsupervised learning of semantic units in natural language. In Proceedings of the 13th international conference on computational semantics—Student papers (pp. 52–62). https://aclanthology.org/W19-0607.pdf
23.
HjelmR. D.FedorovA.Lavoie-MarchildonS.GrewalK.BachmanP.TrischlerA.BengioY. (2018). Learning deep representations by mutual information estimation and maximization. arXiv preprint. arXiv:1808.06670.
24.
Hoffmeyer-ZlotnikJ. H.HarknessJ. (Eds.) (2005). Methodological aspects in cross-national research. GESIS-ZUMA.
KarlgrenJ.SahlgrenM. (2001). From words to understanding. In Foundations of real-world intelligence (pp. 294–308). CSLI Publications.
27.
KingG.MurrayC. J.SalomonJ. A.TandonA. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.
KudoT.RichardsonJ. (2018). Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint arXiv:1808.06226.
30.
KutuzovA.ØvrelidL.SzymanskiT.VelldalE. (2018). Diachronic word embeddings and semantic shifts: A survey. arXiv preprint arXiv:1806.03537.
31.
LenciA. (2008). Distributional semantics in linguistic and cognitive research. Rivista di Linguistica, 20(1), 1–31.
32.
LenciA.SahlgrenM.JeunieauxP.GyllenstenA. C.MilianiM. (2021). A comprehensive comparative evaluation and analysis of distributional semantic models. arXiv preprint arXiv:2105.09825.
33.
LevyO.GoldbergY. (2014). Dependency-based word embeddings. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2, 302–308.
34.
LindeJ.EkmanJ. (2003). Satisfaction with democracy: A note on a frequently used indicator in comparative politics. European Journal of Political Science Research, 42(3), 391–408.
35.
MellonJ.Prosser. C. (2017). Twitter and Facebook are not representative of the general population: Political attitudes and demographics of British social media users. Research & Politics, 4(3), 205316801772000. https://doi.org/10.1177/2053168017720008
36.
MikolovT.ChenK.CorradoG.DeanJ. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
37.
NorrisP. (Ed.) (1999). Critical citizens: Global support for democratic governance. Oxford University Press.
38.
PierrejeanB.TanguyL. (2018, June). Predicting word embeddings variability. In Proceedings of the seventh joint conference on lexical and computational semantics (pp. 154–159). https://hal.archives-ouvertes.fr/hal-01806467/
RogersA.AnanthakrishnaS. H.RumshiskyA. (2018, August). What’s in your embedding, and how it predicts task performance. In Proceedings of the 27th international conference on computational linguistics (pp. 2690–2703). https://aclanthology.org/C18-1228/
41.
SahlgrenM. (2006). The word-space model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. Department of Linguistics, Stockholm University.
42.
SahlgrenM. (2008). The distributional hypothesis. Rivista di Linguistica, 20(1), 33–53.
43.
SahlgrenM.LenciA. (2016, November). The effects of data size and frequency range on distributional semantic models. In Proceedings of the 2016 conference on empirical methods in natural language processing. https://aclanthology.org/D16-1099/
44.
ŞenelL. K.Utluİ.YücesoyV.KoçA.ÇukurT. (2018, December). Generating semantic similarity atlas for natural languages. In 2018 IEEE spoken language technology workshop (SLT) (pp. 795–799). IEEE. https://ieeexplore.ieee.org/document/8639521
45.
ŞenelL. K.YücesoyV.KoçA.ÇukurT. (2017, July). Measuring cross-lingual semantic similarity across European languages. In 2017 40th international conference on telecommunications and signal processing (TSP) (pp. 359–363). IEEE. https://ieeexplore.ieee.org/document/8076005
46.
SennrichR.HaddowB.BirchA. (2015). Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909.
47.
SharmaR.BhattacharyyaP. (2016, January). High, medium or low? Detecting intensity variation among polar synonyms in WordNet. Proceedings of the 8th global WordNet conference global WordNet association. https://aclanthology.org/2016.gwc-1.54/
48.
SmythJ. D.DillmanD. A.ChristianL. M.McBrideM. (2009). Open-ended questions in web surveys: Can increasing the size of answer boxes and providing extra verbal instructions improve response quality?Public Opinion Quarterly, 73(2), 325–337.
49.
StedmanR. C.ConnellyN. A.HeberleinT. A.DeckerD. J.AllredS. B. (2019). The end of the (research) world as we know it? Understanding and coping with declining response rates to mail surveys. Society & Natural Resources, 32(10), 1139–1154.
50.
TurneyP. D. (2012). Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research, 44, 533–585.
51.
TurneyP. D.PantelP. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37, 141–188.
52.
Van de VijverF. J.ChasiotisA. (2010). Making methods meet: Mixed designs in cross-cultural research. In HarknessJ.Michael BraunM.Brad EdwardsB.JohnsonT. P.LybergL.MohlerP. P.PennellB.SmithT. W. (Eds.), Survey methods in multinational, multiregional, and multicultural contexts (pp.455–473). John Wiley & Sons.
53.
WebberW.MoffatAZobelJ. (2010). A similarity measure for indefinite rankings. ACM Transactions on Information Systems (TOIS), 28(4), 1–38.
54.
WegscheiderC.StarkT. (2020). What drives citizens’ evaluation of democratic performance? The interaction of citizens’ democratic knowledge and institutional level of democracy. Zeitschrift für Vergleichende Politikwissenschaft, 14(4), 345–374.
55.
WelzelC. (2013). Freedom rising: Human empowerment and the quest for emancipation. Cambridge University Press.
56.
WelzelC. (2021). Meanings of democracy: Mapping lay perceptions on scholarly norms. Zeitschrift für Vergleichende Politikwissenschaft, 15(1), 107–118.
Zavala-RojasD.SarisW. E. (2018). Measurement invariance in multilingual survey research: The role of the language of the questionnaire. Social Indicators Research, 140(2), 485–510.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
