Attitudes are intertwined with culture and language. But to what extent? Emerging perspectives in attitude research suggest that cultural representations in language are more related to implicitly measured (vs. explicitly measured) attitudes, and that such relationships persist across history and diverse languages. We offer a comprehensive test of these ideas by correlating (a) attitudes toward 55 topics (e.g., Rich/Poor, Dogs/Cats, Love/Money) from ~100,000 U.S. English-speaking participants with (b) representations of those same topics in word embeddings from contemporary English text, 200 years of English books, and 53 non-English languages. Strong and robust relationships emerged between representations in contemporary English and implicitly but not explicitly measured attitudes. Moreover, strong correlations with implicitly measured attitudes persisted across 200 years of books, and most non-English languages. Results provide new insights into the nature of implicitly measured attitudes and how they are intertwined with cultural representations that are relatively hidden in patterns of language across time and place.
AlbarracínD.JohnsonB. T.ZannaM. P. (2005). The handbook of attitudes. In AlbarracinD.JohnsonB. T.ZannaMark P. (Eds.), The handbook of attitudes. Lawrence Erlbaum. https://doi.org/10.4324/9781410612823
2.
AlkaissiH.McfarlaneS. I. (2023). Artificial hallucinations in ChatGPT: Implications in scientific writing. Cureus, 15(2), Article e35179. https://doi.org/10.7759/cureus.35179
3.
AllportG. W. (1954). The nature of prejudice. Addison-Wesley.
4.
BanajiM. R. (2001). Implicit attitudes can be measured. In RoedigerH. L.IIINairneJ. S.NeathI.SurprenantA. M. (Eds.), The nature of remembering: Essays in honor of Robert G. Crowder (pp. 117–150). American Psychological Association. https://doi.org/10.1037/10394-007
5.
BhatiaS.WalasekL. (2023). Predicting implicit attitudes with natural language data. Proceedings of the National Academy of Sciences, 120, Article e2220726120. https://doi.org/10.1073/pnas.2220726120
6.
BolukbasiT.ChangK. W.ZouJ.SaligramaV.KalaiA. (2016). Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Proceedings of the 30th Conference on Neural Information Processing Systems (pp. 4356–4364). New York, NY: Curran Associations Inc.
7.
BrunerE. M. (1956). Cultural transmission and cultural change. Southwestern Journal of Anthropology, 12(2), 191–199. https://www.jstor.org/stable/3629114
8.
CaliskanA.BrysonJ. J.NarayananA. (2016). Semantics derived automatically from language corpora necessarily contain human biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230
9.
CharlesworthT. E. S.BanajiM. R. (2022a). Patterns of implicit and explicit attitudes: IV. change and stability from 2007 to 2020. Psychological Science, 33(9), 1347–1371. https://doi.org/10.1177/09567976221084257
10.
CharlesworthT. E. S.BanajiM. R. (2022b). Word embeddings reveal social group attitudes and stereotypes in large language corpora. In DehghaniM.BoydR. L. (Eds.), Handbook of language analysis in psychology (pp. 594–608). Guilford Publications.
11.
CharlesworthT. E. S.CaliskanA.BanajiM. R. (2022). Historical representations of social groups across 200 years of word embeddings from Google Books. Proceedings of the National Academy of Sciences, 119(28), Article e2121798119. https://doi.org/10.1073/pnas.2121798119
12.
CharlesworthT. E. S.HatzenbuehlerM. L. (2024). Mechanisms Upholding the Persistence of Stigma across 100 Years of Historical Text. Scientific Reports, 14. https://doi.org/https://www.nature.com/articles/s41598-024-61044-z
13.
CharlesworthT. E. S.SanjeevN.HatzenbuehlerM. L.BanajiM. R. (2023). Identifying and predicting stereotype change across 72 groups, four text sources, and historical time (1900-2015): Insights from word embeddings. Journal of Personality and Social Psychology, 125(5), 969–990. https://doi.org/https://doi.org/10.1037/pspa0000354
14.
CrandallC. S.EshlemanA.O’BrienL. (2002). Social norms and the expression and suppression of prejudice: The struggle for internalization. Journal of Personality and Social Psychology, 82(3), 359–378. https://doi.org/10.1037/0022-3514.82.3.359
15.
CunninghamW. A.ZelazoP. D.PackerD. J.Van BavelJ. J. (2007). The iterative reprocessing model: A multilevel framework for attitudes and evaluation. Social Cognition, 25(5), 736–760. https://doi.org/10.1521/soco.2007.25.5.736
16.
De HouwerJ.Teige-MocigembaS.SpruytA.MoorsA. (2009). Implicit measures: A normative analysis and review. Psychological Bulletin, 135(3), 347–368. https://doi.org/10.1037/A0014211
17.
DevineP. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56(1), 5–18. https://doi.org/10.1037/0022-3514.56.1.5
18.
DevineP. G.PlantE. A.AmodioD. M.Harmon-JonesE.VanceS. L. (2002). The regulation of explicit and implicit race bias: The role of motivations to respond without prejudice. Journal of Personality and Social Psychology, 82(5), 835–848. https://doi.org/10.1037/0022-3514.82.5.835
19.
DurkheimE. (1974). Sociology and philosophy. The Free Press.
20.
FineG. A. (1979). Small groups and culture creation: The idioculture of little league baseball teams. American Sociological Review, 44(5), 733–745.
21.
GoodrichB.GabryJ.AliI.BrillemanS. (2020). Package “rstanarm” type package title Bayesian Applied Regression Modeling via Stan. https://mc-stan.org/rstanarm
22.
GraveE.BojanowskiP.GuptaP.JoulinA.MikolovT. (2018). Learning word vectors for 157 languages. https://fasttext.cc/
23.
GreenhillS. J.CurrieT. E.GrayR. D. (2009). Does horizontal transmission invalidate cultural phylogenies?Proceedings of the Royal Society B: Biological Sciences, 276(1665), 2299–2306. https://doi.org/10.1098/RSPB.2008.1944
24.
GreenwaldA. G.BanajiM. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102(1), 4–27. https://doi.org/10.1037/0033-295X.102.1.4
25.
GreenwaldA. G.McGheeD. E.SchwartzJ. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480. https://doi.org/10.1037/0022-3514.74.6.1464
26.
HamiltonW. L.LeskovecJ.JurafskyD. (2016). Diachronic word embeddings reveal statistical laws of semantic change. In 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1489–1501). Association for Computational Linguistics. https://doi.org/10.18653/v1/p16-1141
HusseyI.HughesS.LaiC. K.EbersoleC. R.AxtJ.NosekB. A. (2012). The Attitudes, Identities, and Individual Differences (AIID) study and dataset. https://doi.org/10.17605/OSF.IO/PCJWF
29.
KarpinskiA.HiltonJ. L. (2001). Attitudes and the implicit association test. Journal of Personality and Social Psychology, 81(5), 774–788. https://doi.org/10.1037/0022-3514.81.5.774
30.
KatzD.BralyK. (1933). Racial stereotypes of one hundred college students. Journal of Abnormal and Social Psychology, 28(3), 280–290. https://doi.org/10.1037/h0074049
31.
KruschkeJ. K. (2018). Rejecting or accepting parameter values in Bayesian estimation. Advances in Methods and Practices in Psychological Science, 1(2), 270–280. https://doi.org/10.1177/2515245918771304
32.
KurdiB.CharlesworthT. E. S. (2023). The 3D framework of implicit attitude change: Levels of analysis, sources of change, and timescales. Trends in Psychological Science, 27(8), 745–758. https://doi.org/10.1016/j.tics.2023.05.009
33.
KurdiB.RatliffK. A.CunninghamW. A. (2021). Can the implicit association test serve as a valid measure of automatic cognition? A response to Schimmack (2021). Perspectives on Psychological Science, 16(2), 422–434. https://doi.org/10.1177/1745691620904080
34.
LewisM.LupyanG. (2020). Gender stereotypes are reflected in the distributional structure of 25 languages. Nature Human Behaviour, 4(10), 1021–1028. https://doi.org/10.1038/s41562-020-0918-6
35.
MarsdenP. V.SmithT. W.HoutM. (2020). Tracking US social change over a half-century: The general social survey at fifty. Annual Review of Sociology, 46, 109–134. https://doi.org/10.1146/annurev-soc-121919-054838
36.
MohrJ.BailC. A.FryeM.LenaJ. C.LizardoO.McDonnellT. E.MischeA.TavoryI.WherryF. F. V. (2019). Measuring culture. Columbia University Press. https://doi.org/10.7312/MOHR18028/HTML
NosekB. A. (2005). Moderators of the relationship between implicit and explicit evaluation. Journal of Experimental Psychology: General, 134(4), 565–584. https://doi.org/10.1037/0096-3445.134.4.565
NosekB. A.HansenJ. J. (2008). The associations in our heads belong to us: Searching for attitudes and knowledge in implicit evaluation. Cognition & Emotion, 22(4), 553–594. https://doi.org/10.1080/02699930701438186
41.
OlsonM. A.KendrickR. V. (2011) Origins of Attitudes. In Attitudes and Attitude Change (Eds. CranoWilliam D.PrislinRadmila). (pp. 111–130). New York: Psychology Press.
PayneB. K.VuletichH. A.Brown-IannuzziJ. L. (2019). Historical roots of implicit bias in slavery. Proceedings of the National Academy of Sciences of the United States of America, 116(24), 11693–11698. https://doi.org/10.1073/pnas.1818816116
44.
PayneB. K.VuletichH. A.LundbergK. B. (2017). The bias of crowds: How implicit bias bridges personal and systemic prejudice. Psychological Inquiry, 28(4), 233–248. https://doi.org/10.1080/1047840X.2017.1335568
SchimmackU. (2021). The Implicit Association Test: A method in search of a construct. Perspectives on Psychological Science, 16(2), 396–414. https://doi.org/10.1177/1745691619863798
47.
van BavelJ. J.Jenny XiaoY.CunninghamW. A. (2012). Evaluation is a dynamic process: Moving beyond dual system models. Social and Personality Psychology Compass, 6(6), 438–454. https://doi.org/10.1111/j.1751-9004.2012.00438.x
48.
WarrinerA. B.KupermanV.BrysbaertM. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45(4), 1191–1207. https://doi.org/10.3758/s13428-012-0314-x
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
