Sage Journals: Discover world-class research

Abstract

We combined established psychological measures with techniques in machine learning to measure changes in gender stereotypes over the course of the 20th century as expressed in large-scale historical natural language data. Although our analysis replicated robust gender biases previously documented in the literature, we found that the strength of these biases has diminished over time. This appears to be driven by changes in gender biases for stereotypically feminine traits (rather than stereotypically masculine traits) and changes in gender biases for personality-related traits (rather than physical traits). Our results illustrate the dynamic nature of stereotypes and show how recent advances in data science can be used to provide a long-term historical analysis of core psychological variables. In terms of practice, these findings may, albeit cautiously, suggest that women and men can be less constrained by prescriptions of feminine traits. Additional online materials for this article are available on PWQ’s website at 10.1177/0361684320977178

Keywords

gender stereotypes big data word embeddings femininity masculinity

Get full access to this article

View all access options for this article.

References

Amanatullah

E. T.

Morris

M. W.

(2010). Negotiating gender roles: Gender differences in assertive negotiating are mediated by women’s fear of backlash and attenuated when negotiating on behalf of others. Journal of Personality and Social Psychology, 98(2), 256–267. https://doi.org/10.1037/a0017094

Amanatullah

E. T.

Tinsley

C. H.

(2013). Punishing female negotiators for asserting too much…or not enough: Exploring why advocacy moderates backlash against assertive female negotiators. Organizational Behavior and Human Decision Processes, 120(1), 110–122. https://doi.org/10.1016/j.obhdp.2012.03.006

Babcock

Recalde

M. P.

Vesterlund

Weingart

(2017). Gender differences in accepting and receiving requests for tasks with low promotability. American Economic Review, 107(3), 714–747. https://doi.org/10.1257/aer.w20141734

Bakan

(1966). The duality of human existence. Addison Wesley.

Bem

S. L.

(1974). The measurement of psychological androgyny. Journal of Consulting and Clinical Psychology, 42(2), 155–161. https://doi.org/10.1037/h0036215

Berdahl

J. L.

Moon

S. H.

(2013). Workplace mistreatment of middle class workers based on sex, parenthood, and caregiving. Journal of Social Issues, 69(2), 341–366. https://doi.org/10.1111/josi.12018

Bhatia

(2017a). Associative judgment and vector space semantics. Psychological Review, 124(1), 1–20. https://doi.org/10.1037/rev0000047

Bhatia

(2017b). The semantic representation of prejudice and stereotypes. Cognition, 164, 46–60. https://doi.org/10.1016/j.cognition.2017.03.016

Bhatia

(2019a). Predicting risk perception: New insights from data science. Management Science, 65(8), 3800–3823. https://doi.org/10.1287/mnsc.2018.3121

10.

Bhatia

(2019b). Semantic processes in preferential decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(4), 627–640. https://doi.org/10.1037/xlm0000618

11.

Bhatia

Goodwin

G. P.

Walasek

(2018). Trait associations for Hillary Clinton and Donald Trump in news media: A computational analysis. Social Psychological and Personality Science, 9(2), 123–130. https://doi.org/10.1177/1948550617751584

12.

Bhatia

Richie

Zou

(2019). Distributed semantic representations for modeling human judgment. Current Opinion in Behavioral Sciences, 29, 31–36. https://doi.org/10.1016/j.cobeha.2019.01.020

13.

Bhatia

Stewart

(2018). Naturalistic multiattribute choice. Cognition, 179, 71–88. https://doi.org/10.1016/j.cognition.2018.05.025

14.

Bhatia

Walasek

(2019). Association and response accuracy in the wild. Memory & Cognition, 47(2), 292–298. https://doi.org/10.3758/s13421-018-0869-6

15.

Blakemore

J. E. O.

(2003). Children’s beliefs about violating gender norms: Boys shouldn’t look like girls, and girls shouldn’t act like boys. Sex Roles, 48(9), 411–419. https://doi.org/10.1023/A:1023574427720

16.

Bowles

H. R.

Babcock

Lai

(2007). Social incentives for gender differences in the propensity to initiate negotiations: Sometimes it does hurt to ask. Organizational Behavior and Human Decision Processes, 103(1), 84–103. https://doi.org/10.1016/j.obhdp.2006.09.001

17.

Buffington

Cerf

Jones

Weinberg

B. A.

(2016). STEM training and early career outcomes of female and male graduate students: Evidence from UMETRICS data linked to the 2010 census. The American Economic Review, 106(5), 333–338. https://doi.org/10.1257/aer.p20161124

18.

Caliskan

Bryson

J. J.

Narayanan

(2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230

19.

Carli

L. L.

(1990). Gender, language, and influence. Journal of Personality and Social Psychology, 59(5), 941–951. https://doi.org/10.1037/0022-3514.59.5.941

20.

Cejka

M. A.

Eagly

A. H.

(1999). Gender-stereotypic images of occupations correspond to the sex segregation of employment. Personality and Social Psychology Bulletin, 25(4), 413–423. https://doi.org/10.1177/0146167299025004002

21.

Centers for Disease Control and Prevention. (2017). Marriage and divorce. https://www.cdc.gov/nchs/fastats/marriage-divorce.htm

22.

Cheryan

Ziegler

S. A.

Montoya

A. K.

Jiang

(2017). Why are some STEM fields more gender balanced than others? Psychological Bulletin, 143(1), 1–35. https://doi.org/10.1037/bul0000052

23.

Coltrane

Miller

E. C.

DeHaan

Stewart

(2013). Fathers and the flexibility stigma. Journal of Social Issues, 69(2), 279–302. https://doi.org/10.1111/josi.12015

24.

Cota

A. A.

Fekken

G. C.

(1988). Dimensionality of the personal attributes questionnaire: An empirical replication. Journal of Social Behavior and Personality, 3(1), 135–140.

25.

Croft

Schmader

Block

(2015). An underexamined inequality: Cultural and psychological barriers to men’s engagement with communal roles. Personality and Social Psychology Review, 19(4), 343–370. https://doi.org/10.1177/1088868314564789

26.

Cronbach

L. J.

Furby

(1970). How we should measure “change”: Or should we? Psychological Bulletin, 74(1), 68–80. https://doi.org/10.1037/h0029382

27.

Deaux

Lewis

L. L.

(1984). Structure of gender stereotypes: Interrelationships among components and gender label. Journal of Personality and Social Psychology, 46(5), 991–1004. https://doi.org/10.1037/0022-3514.46.5.991

28.

Dehghani

Johnson

Hoover

Sagi

Garten

Parmar

N. J.

Vaisey

Iliev

Graham

(2016). Purity homophily in social networks. Journal of Experimental Psychology General, 145(3), 366–375. https://doi.org/10.1037/xge0000139

29.

Diekman

A. B.

Eagly

A. H.

(2000). Stereotypes as dynamic constructs: Women and men of the past, present, and future. Personality and Social Psychology Bulletin, 26(10), 1171–1188. https://doi.org/10.1177/0146167200262001

30.

Donnelly

Twenge

J. M.

(2017). Masculine and feminine traits on the Bem sex-role inventory, 1993–2012: A cross-temporal meta-analysis. Sex Roles, 76(9–10), 556–565. https://doi.org/10.1007/s11199-016-0625-y

31.

Eagly

A. H.

Karau

S. J.

(2002). Role congruity theory of prejudice toward female leaders. Psychological Review, 109(3), 573–598. https://doi.org/10.1037//0033-295X.109.3.573

32.

Eagly

A. H.

Nater

Miller

D. I.

Kaufmann

Sczesny

(2019). Gender stereotypes have changed: A cross-temporal meta-analysis of U.S. public opinion polls from 1946 to 2018. American Psychologist, 75(3), 301–315. https://doi.org/10.1037/amp0000494

33.

Eagly

A. H.

Steffen

V. J.

(1984). Gender stereotypes stem from the distribution of women and men into social roles. Journal of Personality and Social Psychology, 46(4), 735–754. https://doi.org/10.1037/0022-3514.46.4.735

34.

Eagly

A. H.

Wood

(2012). Social role theory. In van Lange

Kruglanski

Higgins

E. T.

(Eds.), Handbook of theories in social psychology (pp. 458–476). Sage. https://doi.org/10.4135/9781446249222.n49

35.

Ellemers

(2018). Gender stereotypes. Annual Review of Psychology, 69(1), 275–298. https://doi.org/10.1146/annurev-psych-122216-011719

36.

Evans

J. St B T.

(2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59(1), 255–278. https://doi.org/10.1146/annurev.psych.59.103006.093629

37.

Firth

J. R.

(1957). Papers in linguistics. Oxford University Press.

38.

Fiske

S. T.

Linville

P. W.

(1980). What does the schema concept buy us? Personality and Social Psychology Bulletin, 6(4), 543–557. https://doi.org/10.1177/014616728064006

39.

Garg

Schiebinger

Jurafsky

Zou

(2018). Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences of the United States of America, 115(16), E3635–E3644. https://doi.org/10.1073/pnas.1720347115

40.

Garten

Hoover

Johnson

K. M.

Boghrati

Iskiwitch

Dehghani

(2018). Dictionaries and distributions: Combining expert knowledge and large scale textual data content analysis. Behavior Research Methods, 50(1), 344–361. https://doi.org/10.3758/s13428-017-0875-9

41.

Goodwin

G. P.

Piazza

Rozin

(2014). Moral character predominates in person perception and evaluation. Journal of Personality and Social Psychology, 106(1), 148–168. https://doi.org/10.1037/a0034726

42.

Griffiths

T. L.

(2015). Manifesto for a new (computational) cognitive evolution. Cognition, 135, 21–23. https://doi.org/10.1016/j.cognition.2014.11.026

43.

Griffiths

T. L.

Steyvers

Tenenbaum

J. B.

(2007). Topics in semantic representation. Psychological Review, 114(2), 211–244. https://doi.org/10.1037/0033-295X.114.2.211

44.

Haines

E. L.

Deaux

Lofaro

(2016). The times they are a-changing…or are they not? A comparison of gender stereotypes, 1983–2014. Psychology of Women Quarterly, 40(3), 353–363. https://doi.org/10.1177/0361684316634081

45.

Harlow

L. L.

Oswald

F. L.

(2016). Big data in psychology: Introduction to the special issue. Psychological Methods, 21(4), 447–457. https://doi.org/10.1037/met0000120

46.

Harris

Z. S.

(1954). Distributional structure. Word, 2, 146–162. https://doi.org/10.1080/00437956.1954.11659520

47.

Healey

M. K.

Kahana

M. J.

(2016). A four-component model of age-related memory change. Psychological Review, 123(1), 23–69. https://doi.org/10.1037/rev0000015

48.

Heilman

M. E.

(2001). Description and prescription: How gender stereotypes prevent women’s ascent up the organizational ladder. Journal of Social Issues, 57(4), 657–674. https://doi.org/10.1111/0022-4537.00234

49.

Heppner

P. P.

(1995). The differential role of instrumentality, expressivity, and social support in predicting problem-solving appraisal in men and women. Sex Roles, 32(1–2), 91–108. https://doi.org/10.1007/BF01544759

50.

Hills

T. T.

Jones

M. N.

Todd

P. M.

(2012). Optimal foraging in semantic memory. Psychological Review, 119(2), 431–440. https://doi.org/10.1037/a0027373

51.

Hochschild

A. R.

Machung

(2012). The second shift: Working parents and the revolution at home. Penguin Books.

52.

Holtzman

Abbey

S. E.

Singer

L. G.

Ross

H. J.

Stewart

D. E.

(2011). Social support and optimism impact depression and caregiver burden among organ transplant candidates and their caregivers. Journal of Heart and Lung Transplantation, 28(2), S219–S219. https://doi.org/10.1016/j.healun.2008.11.449

53.

Hopkins

D. J.

(2018). The exaggerated life of death panels? The limited but real influence of elite rhetoric in the 2009–2010 health care debate. Political Behavior, 40(3), 681–709. https://doi.org/10.1007/s11109-017-9418-4

54.

Jones

M. N.

(2017). Big data in cognitive science. Taylor & Francis.

55.

Jones

M. N.

Kintsch

Mewhort

D. J. K.

(2006). High-dimensional semantic space accounts of priming. Journal of Memory and Language, 55(4), 534–552. https://doi.org/10.1016/j.jml.2006.07.003

56.

Jones

M. N.

Mewhort

D. J. K.

(2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114(1), 1–37. https://doi.org/10.1037/0033-295X.114.1.1

57.

Jones

M. N.

Willits

J. A.

Dennis

(2015). Models of semantic memory. In Busemeyer

J. R.

Wang

Townsend

J. T.

Eidels

(Eds.), Oxford handbook of computational and mathematical psychology (pp. 232–254). Oxford University Press.

58.

Kahneman

(2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697–720. https://doi.org/10.1037/0003-066X.58.9.697

59.

Kessler

J. B.

Low

Sullivan

C. D.

(2019). Incentivized resume rating: Eliciting employer preferences without deception. American Economic Review, 109(11), 3713–3744. https://doi.org/10.1257/aer.20181714

60.

Koenig

A. M.

Eagly

A. H.

(2014). Evidence for the social role theory of stereotype content: Observations of groups’ roles shape stereotypes. Journal of Personality and Social Psychology, 107(3), 371–392. https://doi.org/10.1037/a0037215

61.

Kosinski

Behrend

(2017). Editorial overview: Big data in the behavioral sciences. Current Opinion in Behavioral Sciences, 18, iv-vi. https://doi.org/10.1016/j.cobeha.2017.11.007

62.

Landauer

T. K.

Dumais

S. T.

(1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211–240. https://doi.org/10.1037/0033-295X.104.2.211

63.

Lenci

(2018). Distributional models of word meaning. Annual Review of Linguistics, 4(1), 151–171. https://doi.org/10.1146/annurev-linguistics-030514-125254

64.

Levy

Goldberg

Dagan

(2015). Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3, 211–225. https://doi.org/10.1162/tacl_a_00134

65.

Schloss

Follmer

D. J.

(2017). Speaking two “languages” in America: A semantic space analysis of how presidential candidates and their supporters represent abstract political concepts differently. Behavior Research Methods, 49(5), 1668–1685. https://doi.org/10.3758/s13428-017-0931-5

66.

Lippmann

(1922). The world outside and the pictures in our heads. Public Opinion, 4, 1–22.

67.

Mandera

Keuleers

Brysbaert

(2017). Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting: A review and empirical validation. Journal of Memory and Language, 92, 57–78. https://doi.org/10.1016/j.jml.2016.04.001

68.

Mikolov

Sutskever

Chen

Corrado

Dean

(2013). Distributed representations of words and phrases and their compositionality. In Jordan

M. I.

LeCun

Solla

S. A.

(Eds.), Advances in neural information processing systems (pp. 3111–3119). Cambridge, MA: MIT Press.

69.

Moss-Racusin

C. A.

Dovidio

J. F.

Brescoll

V. L.

Graham

M. J.

Handelsman

(2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences of the United States of America, 109(41), 16474–16479. https://doi.org/10.1073/pnas.1211286109

70.

Nisbett

R. E.

Wilson

T. D.

(1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037//0033-295X.84.3.231

71.

Nosek

B. A.

Banaji

Greenwald

A. G.

(2002). Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1), 101–115. https://doi.org/10.1037//1089-2699.6.1.101

72.

Okahana

Zhou

(2018). Graduate enrollment and degrees: 2007 to 2017. Council of Graduate Schools.

73.

Paolacci

Chandler

(2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188. https://doi.org/10.1177/0963721414531598

74.

Pennington

Socher

Manning

C. D.

(2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (Vol. 14, pp. 1532–1543). Doha, Qatar: Association for Computational Linguistics.

75.

Pereira

Gershman

Ritter

Botvinick

(2016). A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data. Cognitive Neuropsychology, 33(3–4), 175–190. https://doi.org/10.1080/02643294.2016.1176907

76.

Prentice

D. A.

Carranza

(2002). What women and men should be, shouldn’t be, are allowed to be, and don’t have to be: The contents of prescriptive gender stereotypes. Psychology of Women Quarterly, 26(4), 269–281. https://doi.org/10.1111/1471-6402.t01-1-00066

77.

Rudman

L. A.

Glick

(2001). Prescriptive gender stereotypes and backlash toward agentic women. Journal of Social Issues, 57(4), 743–762. https://doi.org/10.1111/0022-4537.00239

78.

Rudman

L. A.

Greenwald

A. G.

McGhee

D. E.

(2001). Implicit self-concept and evaluative implicit gender stereotypes: Self and ingroup share desirable traits. Personality and Social Psychology Bulletin, 27(9), 1164–1178. https://doi.org/10.1177/0146167201279009

79.

Schumann

Ross

(2010). Why women apologize more than men: Gender differences in thresholds for perceiving offensive behavior. Psychological Science, 21(11), 1649–1655. https://doi.org/10.1177/0956797610384150

80.

Schwitzgebel

Jennings

C. D.

(2017). Women in philosophy: Quantitative analyses of specialization, prevalence, visibility, and generational change. Public Affairs Quarterly, 31(2), 83–105.

81.

Sloman

S. A.

(1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119(1), 3–22. https://doi.org/10.1037//0033-2909.119.1.3

82.

Smith

E. R.

DeCoster

J. M.

(2000). Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems. Personality and Social Psychology Review, 4(2), 108–131. https://doi.org/10.1207/S15327957PSPR0402_01

83.

Spence

J. T.

Helmreich

R. L.

(1978). Masculinity & femininity. University of Texas Press.

84.

Spence

J. T.

Helmreich

Stapp

(1975). Ratings of self and peers on sex role attributes and their relation to self-esteem and conceptions of masculinity and femininity. Journal of Personality and Social Psychology, 32(1), 29–39. https://doi.org/10.1037/h0076857

85.

Spencer

S. J.

Steele

C. M.

Quinn

D. M.

(1999). Stereotype threat and women’s math performance. Journal of Experimental Social Psychology, 35(1), 4–28. https://doi.org/10.1006/jesp.1998.1373

86.

Steele

C. M.

Aronson

(1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69(5), 797–811. https://doi.org/10.1037//0022-3514.69.5.797

87.

Strack

Deutsch

(2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review, 8(3), 220–247. https://doi.org/10.1207/s15327957pspr0803_1

88.

Tannen

(1994). Gender and discourse. Oxford University Press.

89.

Turney

P. D.

Pantel

(2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37, 141–188. https://doi.org/10.1613/jair.2934

90.

Twenge

J. M.

(1997). Changes in masculine and feminine traits over time: A meta-analysis. Sex Roles, 36(5), 305–325. https://doi.org/10.1007/BF02766650

91.

Twenge

J. M.

Campbell

W. K.

Gentile

(2012). Male and female pronoun use in U.S. books reflects women’s status, 1900-2008. Sex Roles, 67(9–10), 488. https://doi.org/10.1007/s11199-012-0194-7

92.

U.S. Census Bureau. (2019). Historical marital status tables. https://www.census.gov/data/tables/time-series/demo/families/marital.html

93.

U.S. Department of Labor. (2018). Women of working age. https://www.dol.gov/wb/stats/NEWSTATS/latest/demographics.htm#LF-SecRaceEthnicity

94.

Vazire

(2010). Who knows what about a person? The self–other knowledge asymmetry (SOKA) model. Journal of Personality and Social Psychology, 98(2), 281–300. https://doi.org/10.1037/a0017908

95.

Warner

Ellmann

Boesch

(2018). Women’s leadership gap. https://www.americanprogress.org/issues/women/reports/2018/11/20/461273/womens-leadership-gap-2/

96.

Williams

M. J.

Tiedens

L. Z.

(2016). The subtle suspension of backlash: A meta-analysis of penalties for women’s implicit and explicit dominance behavior. Psychological Bulletin, 142(2), 165–197. http://dx.doi.org/10.1037/bul0000039

97.

Yanico

B. J.

(1985). BSRI scores: Stability over four years for college women. Psychology of Women Quarterly, 9(2), 277–283. https://doi.org/10.1111/j.1471-6402.1985.tb00878.x

98.

Yoder

J. D.

Rice

R. W.

Adams

Priest

R. F.

Prince

H. T.

(1982). Reliability of the attitudes toward women scale (AWS) and the personal attributes questionnaire (PAQ). Sex Roles, 8(6), 651–657. https://doi.org/10.1007/BF00289898

99.

Zessoules

McGrew

Madowitz

(2018). The state of the U.S. labor market for mothers. https://www.americanprogress.org/issues/economy/news/2018/05/30/451414/state-u-s-labor-market-mothers-pre-may-2018-jobs-release/

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

Changes in Gender Stereotypes Over Time: A Computational Analysis

Abstract

Keywords

Get full access to this article

References

Supplementary Material