People associate certain behaviors with certain social groups. These stereotypical beliefs consist of both accurate and inaccurate associations. Using large-scale, data-driven methods with social media as a context, we isolate stereotypes by using verbal expression. Across four social categories—gender, age, education level, and political orientation—we identify words and phrases that lead people to incorrectly guess the social category of the writer. Although raters often correctly categorize authors, they overestimate the importance of some stereotype-congruent signal. Findings suggest that data-driven approaches might be a valuable and ecologically valid tool for identifying even subtle aspects of stereotypes and highlighting the facets that are exaggerated or misapplied.
BurgerJ. D.HendersonJ.KimG.ZarrellaG. (2011, 7). Discriminating gender on Twitter. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 1301–1309). Stroudsburg, PA: Association for Computational Linguistics.
4.
ChungC.PennebakerJ. W. (2007). The psychological functions of function words. In FiedlerK. (Ed.), Social communication (pp. 343–359). New York, NY: Psychology Press.
5.
DiekmanA. B.EaglyA. H.KulesaP. (2002). Accuracy and bias in stereotypes about the social and political attitudes of women and men. Journal of Experimental Social Psychology, 38, 268–282.
6.
DovidioJ. F.BrighamJ. C.JohnsonB. T.GaertnerS. L. (1996). Stereotyping, prejudice, and discrimination: Another look. In MacraeC. N.StangorC.HewstoneM. (Eds.), Stereotypes and stereotyping (pp. 276–319). New York: Guilford Press.
7.
EaglyA. H. (1995). The science and politics of comparing women and men. American Psychologist, 50, 145–158.
8.
EichstaedtJ. C.KernM. L.TobolskyV.YadenD. B.SchwartzH. A.ParkG.…SeligmanM. E. P. (2016). From hypothesis-testing to hypothesis-generation: A review and quantitative comparison of open and closed-vocabulary approaches for text analysis. Manuscript in preparation.
9.
EliasP.BirchM. (2010). SOC2010: Revision of the standard occupational classification. Economic and Labour Market Review, 4, 48–55.
10.
GreenwaldA. G.BanajiM. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4–27.
11.
HaasA. (1979). Male and female spoken language differences: Stereotypes and evidence. Psychological Bulletin, 86, 616–626.
12.
HuddyL.TerkildsenN. (1993). Gender stereotypes and the perception of male and female candidates. American Journal of Political Science, 37, 119–147.
13.
JuddC. M.ParkB. (1993). Definition and assessment of accuracy in social stereotypes. Psychological Review, 100, 109–128.
14.
JussimL.CainT. R.CrawfordJ. T.HarberK.CohenF. (2009). The unbearable accuracy of stereotypes. In NelsonT. D. (Ed.), Handbook of prejudice, stereotyping, and discrimination (pp. 199–227). New York, NY: Taylor & Francis.
15.
JussimL.CrawfordJ. T.RubinsteinR. S. (2015). Stereotype (in) accuracy in perceptions of groups and individuals. Current Directions in Psychological Science, 24, 490–497.
16.
JussimL.ZannaM. P. (2005). Accuracy in social perception: Criticisms, controversies, criteria, components, and cognitive processes. Advances in Experimental Social Psychology, 37, 1–93.
17.
KernM. L.EichstaedtJ. C.SchwartzH. A.ParkG.UngarL. H.StillwellD. J.…SeligmanM. E. (2014). From “sooo excited!!!” to “so proud”: Using language to study development. Developmental Psychology, 50, 178–188.
18.
KernM. L.ParkG.EichstaedtJ. C.SchwartzH. A.SapM.SmithL.UngarL. H. (2016). Gaining insights from social media language: Methodologies and challenges. Psychological Methods. http://dx.doi.org/10.1037/met0000091
19.
LeeI. A.PreacherK. J. (2013). Calculation for the test of the difference between two dependent correlations with one variable in common [Computer software]. Retrieved December, 2015, fromhttp://quantpsy.org/corrtest/corrtest2.htm
20.
LuiM.BaldwinT. (2012, 7). langid. py: An off-the-shelf language identification tool. Proceedings of the ACL 2012 system demonstrations (pp. 25–30). Association for Computational Linguistics, Jeju, Republic of Korea.
21.
MacCallumR.ZhangS.PreacherK.RuckerD. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–40.
22.
McCauleyC. (1995). Are stereotypes exaggerated? A sampling of racial, gender, academic, occupational, and political stereotypes. In LeeY.JussimL.McCauleyC. R. (Eds.), Stereotype accuracy: Toward appreciating group differences (pp. 215–243). Washington, DC: American Psychological Association.
23.
McCauleyC.StittC. L. (1978). An individual and quantitative measure of stereotypes. Journal of Personality and Social Psychology, 36, 929–940.
24.
Moss-RacusinC. A.DovidioJ. F.BrescollV. L.GrahamM. J.HandelsmanJ. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109, 16474–16479.
25.
MurphyM. C.SteeleC. M.GrossJ. J. (2007). Signaling threat how situational cues affect women in math, science, and engineering settings. Psychological Science, 18, 879–885.
26.
NewmanM. L.GroomC. J.HandelmanL. D.PennebakerJ. W. (2008). Gender differences in language use: An analysis of 14,000 text samples. Discourse Processes, 45, 211–236.
27.
NguyenD. P.GravelR.TrieschniggR. B.MederT. (2013, 7). “How old do you think I am?” A study of language and age. Proceedings of the seventh international AAAI conference on weblogs and social media, ICWSM, Cambridge, MA.
28.
NguyenD. P.TrieschniggR. B.DoğruözA. S.GravelR.TheuneM.MederT.de JongF. M. G. (2014, 8). Why gender and age prediction from tweets is hard: Lessons from a crowdsourcing experiment. Berkeley, CA: Association for Computational Linguistics.
29.
ParkG.SchwartzH. A.EichstaedtJ. C.KernM. L.KosinskiM.StillwellD. J.…SeligmanM. E. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108, 934.
30.
PennacchiottiM.PopescuA. M. (2011). A machine learning approach to Twitter user classification. International Conference on Web and Social Media, 11, 281–288.
31.
PennebakerJ. W.FrancisM. E.BoothR. J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahwah, NJ: Lawrence Erlbaum Associates.
32.
PlantE. A.DevineP. G. (1998). Internal and external motivation to respond without prejudice. Journal of Personality and Social Psychology, 75, 811–832.
33.
Preoţiuc-PietroD.VolkovaS.LamposV.BachrachY.AletrasN. (2015). Studying user income through language, behaviour and affect in social media. PLoS ONE, 10, e0138717.
34.
ProthroE. T.MelikianL. H. (1955). Studies in stereotypes: V. Familiarity and the kernel of truth hypothesis. The Journal of Social Psychology, 41, 3–10.
35.
QuinnP. C.YahrJ.KuhnA.SlaterA. M.PascalisO. (2002). Representation of the gender of human faces by infants: A preference for female. Perception, 31, 1109–1121.
36.
RaoD.PaulM. J.FinkC.YarowskyD.OatesT.CoppersmithG. (2011). Hierarchical Bayesian models for latent attribute detection in social media. International Conference on Web and Social Media, 11, 598–601.
37.
RyanC. (2003). Stereotype accuracy. European Review of Social Psychology, 13, 75–109.
38.
SapM.ParkG.EichstaedtJ. C.KernM. L.StillwellD. J.KosinskiM.…SchwartzH. A. (2014). Developing age and gender predictive lexica over social media. Proceedings of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1146–1151.
39.
SchwartzH. A.EichstaedtJ. C.DziurzynskiL.KernM. L.BlancoE.KosinskiM.…UngarL. H. (2013). Toward personality insights from language exploration in social media. Proceedings of the AAAI Spring Symposium Series: Analyzing Microtext, Stanford, CA.
40.
SchwartzH. A.EichstaedtJ. C.KernM. L.DziurzynskiL.RamonesS. M.AgrawalM.…UngarL. H. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE, 8, e73791.
41.
ShapiroJ. R.WilliamsA. M. (2012). The role of stereotype threats in undermining girls’ and women’s performance and interest in STEM fields. Sex Roles, 66, 175–183.
42.
SimesR. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika, 73, 751–754.
43.
StangorC. (1995). Content and application inaccuracy in social stereotyping. In LeeY.JussimL.McCauleyC. R. (Eds.), Stereotype accuracy: Toward appreciating group differences (pp. 275–292). Washington, DC: American Psychological Association.
44.
SteigerJ. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251.
45.
SylwesterK.PurverM. (2015). Twitter language use reflects psychological differences between Democrats and Republicans. PloS ONE, 10, e0137422.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
