A Linguistic Strategy to Measure Negative Affective Polarization Through Text Content

Abstract

French

Three studies developed and validated a linguistic dictionary to measure negative affective polarization in English and Spanish political texts. It captures three dimensions: negative affect, delegitimization, and political context. In the first study, two independent judges evaluated the candidate words, and reliability indicators were calculated, showing acceptable values for short texts (.572 in English, .541 in Spanish) and higher values for larger corpora (.964 in English, .957 in Spanish). The second study tested discriminant validity by comparing negative affective polarization scores in social media comments on politics and entertainment. Results showed significantly higher polarization scores in political content, confirming the dictionary's validity. The third study compared the dictionary to an existing online polarization measure, finding greater coverage and alignment with the construct. Additionally, it was observed that polarization scores were higher in texts containing hate speech compared to those where it was absent. The findings suggest that the dictionary in both languages have strong psychometric properties, making it a valuable tool for analyzing online content, particularly social media comments. It can be used as an independent measure or as input for machine and deep learning models.

Keywords

affective polarization social media psychometrics LIWC

Get full access to this article

View all access options for this article.

References

Al Salman

Kim

Mercado

Ring

Doornberg

Fatehi

Crijns

(2021). Are patient linguistic tones associated with mental health and perceived clinician empathy? The Journal of Bone and Joint Surgery, 103(23), 2181–2189. https://doi.org/10.1016/10.2106/JBJS.21.00124

Anselmi

Colledani

Robusto

(2019). A comparison of classical and modern measures of internal consistency. Frontiers in Psychology, 10, 2714. https://doi.org/10.3389/fpsyg.2019.02714

APA - Asociación Americana de Psicología. (2018). APA Dictionary of Psychology. https://dictionary.apa.org/.

Bail

(2021). Breaking the social media prism. How to make our platforms less polarizing. Princeton University Press Princeton & Oxford. https://doi.org/10.2307/j.ctv18zhdhg

Barreto-Galeano

M. I.

Medina-Arboleda

I. F.

Zambrano-Hernández

Sabucedo-Cameselle

J. M.

Blanco-Abarca

Maurice Lair

(2019). Rhetoric, political ideology and the peace process in Colombia: A Twitter® analysis. Studies in Conflict & Terrorism, 44(11), 920–937. https://doi.org/10.1080/1057610X.2019.1615256

Barreto-Galeano

M. I.

Rico-Revelo

Velandia-Morales

Aguilar-Pardo

Garzón-Velandia

D. C.

Carvajal-Builes

J. C.

Carrillo-Ramírez

Sabucedo-Cameselle

J. M.

(2025). Politización diferenciada de discursos en Twitter ante la COVID-19: Comparación de gobiernos locales en Colombia [Differentiated politicization of discourses on Twitter in response to COVID-19: A comparison of local governments in Colombia]. Revista Latina de Comunicación Social, 83, 01–21. https://doi.org/10.4185/RLCS2025-2307

Bar-Tal

(1990). Causes and consequences of delegitimization: Models of conflict and ethnocentrism. Journal of Social Issues, 46(1), 65–81. https://doi.org/10.1111/j.1540-4560.1990.tb00272.x

Bar-Tal

Hammack

P. L.

(2012). Conflict, delegitimization, and violence. In Tropp

L. R.

(Ed.), The Oxford handbook of intergroup conflict (pp. 29–52). Oxford University Press.

Bilewicz

Soral

(2020). Hate speech epidemic. The dynamic effects of derogatory language on intergroup relations and political radicalization. Political Psychology, 41(Suppl. 1), 3–33. https://doi.org/10.1111/pops.12670

10.

Bliuc

A. M.

Bouguettaya

Felise

K. D.

(2021). Online intergroup polarization across political fault lines: An integrative review. Frontiers in Psychology, 12, 1–15. https://doi.org/10.3389/fpsyg.2021.641215

11.

Borja-Orozco

Barreto

Sabucedo

J. M.

López-López

(2008). Construcción del discurso deslegitimador del adversario: Gobierno y paramilitarismo en Colombia. Universitas Psychologica, 7(2), 571–583.

12.

Boyd

R. L.

Ashokkumar

Seraj

Pennebaker

J. W.

(2022). The development and psychometric properties of LIWC-22. University of Texas at Austin. https://www.liwc.app .

13.

Buckels

E. E.

Trapnell

P. D.

Paulhus

D. L.

(2014). Trolls just want to have fun. Personality and Individual Differences, 67, 97–102. https://doi.org/10.1016/J.PAID.2014.01.016

14.

Castaño-Pulgarín

S. A.

Suárez-Betancur

Vega

L. M.

López

H. M.

(2021). Internet, social media and online hate speech. Systematic review. Aggression and Violent Behavior, 58, 101608. https://doi.org/10.1016/j.avb.2021.101608

15.

Cervone

Augoustinos

Maass

(2021). The language of derogation and hate: Functions, consequences, and reappropriation. Journal of Language and Social Psychology, 40(1), 80–101. https://doi.org/10.1177/0261927X20967394

16.

Cikara

Van Bavel

J. J.

(2014). The neuroscience of intergroup relations: An integrative review. Perspectives on Psychological Science, 9(3), 245–274. https://doi.org/10.1177/1745691614527464

17.

Cohen-Almagor

(2014). Countering hate on the Internet. Jahrbuch Fur Recht Und Ethik, 22, 431–443. https://www.jstor.org/stable/43593801

18.

Demata

(2016). Ideological and language polarization in online political discourse. The white house Facebook page. Lingue e Linguaggi, 17, 83–100. https://doi.org/10.1285/i22390359v17p83

19.

Donohue

Hamilton

(2023). A framework for understanding polarizing language. In Fahnestock

Allen Harris

(Eds.), The Routledge handbook of language and persuasion (pp. 207–223). Routledge, Taylor & Francis Group.

20.

Druckman

J. N.

Levendusky

M. S.

(2019). What do we measure when we measure affective polarization? Public Opinion Quarterly, 83(1), 114–122. https://doi.org/10.1093/poq/nfz003

21.

Gagliardone

Gal

Alves

Martinez

(2015). Countering online hate speech. UNESCO Publishing. https://unesdoc.unesco.org/ark:/48223/pf0000233231 .

22.

Garzón-Velandia

D. C.

Barreto

Medina-Arboleda

I. F.

(2020). Validación de un diccionario de LIWC para identificar emociones intergrupales. Revista Latinoamericana de Psicología, 52, 149–159. https://doi.org/10.14349/rlp.2020.v52.15

23.

Garzón-Velandia

D. C.

Barreto Galeano

M. I.

Sabucedo -Cameselle

J. M.

(2024). What is polarization in the social sciences? A scoping review of reviews. Acta Colombiana de Psicología, 27(2), 89–212. https://doi.org/10.14718/ACP.2024.27.2.11

24.

Gidron

Sheffer

Mor

(2022). Validating the feeling thermometer as a measure of partisan affect in multi-party systems. Electoral Studies, 80, 102542. https://doi.org/10.1016/j.electstud.2022.102542

25.

Iyengar

Lelkes

Levendusky

Malhotra

Westwood

S. J.

(2019). The origins and consequences of affective polarization in the United States. https://doi.org/10.1146/annurev-polisci-051117

26.

Iyengar

Sood

Lelkes

(2012). Affect, not ideology: A social identity perspective on polarization. Public Opinion Quarterly, 76(3), 405–431. https://doi.org/10.1093/poq/nfs038

27.

Iyengar

Westwood

S. J.

(2015). Fear and loathing across party lines: New evidence on group polarization. American Journal of Political Science, 59(3), 690–707. https://doi.org/10.1111/ajps.12152

28.

Jost

J. T.

Baldassarri

D. S.

Druckman

J. N.

(2022). Cognitive–motivational mechanisms of political polarization in social-communicative contexts. Nature Reviews Psychology, 1(2–4), 560–576. https://doi.org/10.1038/s44159-022-00093-5

29.

Jost

J. T.

Sterling

(2020). The language of politics: Ideological differences in congressional communication on social media and the floor of Congress. Social Influence, 15(2–4), 80–103. https://doi.org/10.1080/15534510.2020.1871403

30.

Kangas

(2014). What can software tell us about political candidates? A critical analysis of a computerized method for political discourse. Journal of Language and Politics, 13(1), 77–97. https://doi.org/10.1075/jlp.13.1.04kan

31.

Kim

(2022). How language use on Facebook drives affective polarization [Doctorate dissertation]. University of Texas at Austin. https://repositories.lib.utexas.edu/server/api/core/bitstreams/9b6fd12a-7b63-48c0-b708-93ca7bee0138/content .

32.

Koutsoumpis

Oostrom

J. K.

Holtrop

van Breda

Ghassemi

de Vries

R. E.

(2022). The kernel of truth in text-based personality assessment: a meta-analysis of the relations between the big five and the linguistic inquiry and word count (LIWC). Psychological Bulletin, 148(11-12), 843–868. https://doi.org/10.1037/bul0000381

33.

Kubin

von Sikorski

(2021). The role of (social) media in political polarization: a systematic review. Annals of the International Communication Association, 45(3), 188–206. https://doi.org/10.1080/23808985.2021.1976070

34.

Liang

Y. M.

Tsang

N. L. T.

(2023). Word embedding enrichment for dictionary construction: an example of incivility in Cantonese. Computational Communication Research, 5(1), 1–26. https://doi.org/10.5117/CCR2023.1.10.LIAN

35.

Markowitz

D. M.

(2022). Deceptive (De)humanization: How lying about perceived outgroups is revealed in language. Journal of Language and Social Psychology, 42(2), 135–159. https://doi.org/10.1177/0261927X221117497

36.

McCoy

Press

Somer

Tuncel

(May 5, 2022). Reducing pernicious polarization: A comparative historical analysis of depolarization. Carnegie Endowment for International Peace. https://carnegieendowment.org/ .

37.

McMurtrie

Philipp

Hebden

Williams

(2024). Development and validation of the affective polarization scale. International Review of Social Psychology, 37(1), 11, 1–16. https://doi.org/10.5334/irsp.926

38.

Muñoz

Bellogín

Barba-Rojas

, et al. (2024). Quantifying polarization in online political discourse. EPJ Data Science, 13(1), 39. https://doi.org/10.1140/epjds/s13688-024-00480-3

39.

Nettasinghe

Rao

Jiang

Percus

Lerman

(2024). In-group love, out-group hate: a framework to measure affective polarization via contentious online discussions. arXiv. https://arxiv.org/abs/2412.14414.

40.

Orian Harel

Maoz

Halperin

(2020). A conflict within a conflict: Intragroup ideological polarization and intergroup intractable conflict. Current Opinion in Behavioral Sciences, 34, 52–57. https://doi.org/10.1016/j.cobeha.2019.11.013

41.

Overgaard

C. S. B.

Lukito

Soorholtz

(2024). The manifestation of affective polarization on social media: a cross-platform supervised machine learning approach. Proceedings of the Eighteenth International AAAI Conference on Web and Social Media (ICWSM 2024). https://ojs.aaai.org/index.php/ICWSM/article/view/31380

42.

Pennebaker

J. W.

Boyd

R. L.

Booth

R. J.

Ashokkumar

Francis

M. E.

(2022). Linguistic inquiry and word count (LIWC-22). Pennebaker Conglomerates. https://www.liwc.app .

43.

Pérez

J. M.

Furman

D. A.

Alonso Alemany

Luque

(2022). RoBERTuito: a pre-trained language model for social media text in Spanish. arXiv. https://doi.org/10.48550/arXiv.2111.09453

44.

Piazza

J. A.

(2023). Political polarization and political violence. Security Studies, 32(3), 476–504. https://doi.org/10.1080/09636412.2023.2225780

45.

Prins

S. B.

(2018). Polarization: identifying extreme attitudes [Master thesis]. University of Twente. https://essay.utwente.nl/76420/1/Prins_MA_BMS.pdf

46.

Robinson

R. L.

Navea

Ickes

(2013). Predicting final course performance from students’ written self-introductions: a LIWC analysis. Journal of Language and Social Psychology, 32(4), 469–479. https://doi.org/10.1177/0261927X13476869

47.

Rönkkö

Cho

(2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–14. https://doi.org/10.1177/1094428120968614

48.

Sarsfield

Abuchanab

(2024). Populist storytelling and negative affective polarization: social media evidence from Mexico. Latin American Politics and Society, 66(2), 102–131. https://doi.org/10.1017/lap.2024.7

49.

Serrano-Puche

(2021). Digital disinformation and emotions: exploring the social risks of affective polarization. International Review of Sociology, 31(2), 231–245. https://doi.org/10.1080/03906701.2021.1947953

50.

Shrikant

Giles

Angus

(2021). Language and social psychology approaches to race, racism, and social justice: analyzing the past and revealing ways forward. Journal of Language and Social Psychology, 41(1), 4–28. https://doi.org/10.1177/0261927X211064808

51.

Simchon

Brady

W. J.

Van Bavel

J. J.

(2022). Troll and divide: the language of online polarization. PNAS Nexus, 1(1), 1–12. https://doi.org/10.1093/pnasnexus/pgac019

52.

Tausczik

Y. R.

Pennebaker

J. W.

(2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24–54. https://doi.org/10.1177/0261927X09351676

53.

Tavakol

Dennick

(2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

54.

Tong

Margolin

Chunara

Niederdeppe

Taylor

Dunbar

King

A. J.

(2022). Search term identification methods for computational health communication: Word embedding and network approach for health content on YouTube. JMIR Medical Informatics, 10(8), e37862. https://doi.org/10.2196/37862

55.

Underberg

J. E.

Gollwitzer

Oettingen

Gollwitzer

P. M.

(2020). The best words: linguistic indicators of grandiose narcissism in politics. Journal of Language and Social Psychology, 39(2), 271–281. https://doi.org/10.1177/0261927X19883898

56.

United Nations UN- Economic and Social Commission for Western Asia - UNESCWA. (1991). Symbolic violence. https://www.unescwa.org/sd-glossary/symbolic-violence.

57.

Van Bavel

J. J.

Rathje

Harris

Robertson

Sternisko

(2021). How social media shapes polarization. Trends in Cognitive Sciences, 25(11), 913–916. https://doi.org/10.1016/j.tics.2021.07.013

58.

Vasist

P. N.

Chatterjee

Krishnan

(2023). El impacto polarizador de la desinformación política y el discurso de odio: una narrativa configural entre países. Frente del sistema de infantería https://doi.org/10.1007/s10796-023-10390-w

59.

Voteview. (2025). Senate, 119th Congress (2025-2027). Voteview. https://voteview.com/congress/senate/-1/text.

60.

Yarkoni

Westfall

(2017). Choosing prediction over explanation in psychology: lessons from machine learning. Perspectives on Psychological Science : A Journal of the Association for Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393

61.

Wojcieszak

Casas

(2024). Affective polarization on social media: a machine learning approach. Proceedings of the AAAI International Conference on Web and Social Media (ICWSM 2024). AAAI Press. https://ieeexplore.ieee.org/document/10605784