The potential impact of emerging technologies on democratic representation: Evidence from a field experiment

Abstract

Advances in machine learning have led to the creation natural language models that can mimic human writing style and substance. Here we investigate the challenge that machine-generated content, such as that produced by the model GPT-3, presents to democratic representation by assessing the extent to which machine-generated content can pass as constituent sentiment. We conduct a field experiment in which we send both handwritten and machine-generated letters (a total of 32,398 emails) to 7132 state legislators. We compare legislative response rates for the human versus machine-generated constituency letters to gauge whether language models can approximate inauthentic constituency voices at scale. Legislators were only slightly less likely to respond to artificial intelligence (AI)-generated content than to human-written emails; the 2% difference in response rate was statistically significant but substantively small. Qualitative evidence sheds light on the potential perils that this technology presents for democratic representation, but also suggests potential techniques that legislators might employ to guard against misuses of language models.

Keywords

Astroturfing communication language models politics representation

Get full access to this article

View all access options for this article.

References

Ansolabehere

Gerber

Snyder

Jr (2002) Equal votes, equal money: court-ordered redistricting and public expenditures in the American states. American Political Science Review 96(4): 767–777.

Balinski

Young

(2010) Fair Representation: Meeting the Ideal of One Person, One Vote. Washington, DC: Brookings Institution Press.

Bol

Geschwend

Zittel

, et al. (2020) The importance of personal vote intentions for the responsiveness of legislators: A field experiment. European Journal of Political Research 60(2): 455–473.

Brown

Mann

Ryder

, et al. (2020) Language models are few-shot learners. Available at: https://arxiv.org/abs/2005.14165

Buchanan

Lohn

Musser

, et al. (2021) How language models could change disinformation. Center for Security and Emerging Technology. Available at: https://cset.georgetown.edu/publication/truth-lies-and-automation/

Butler

Broockman

(2011) Do politicians racially discriminate against constituents? A field experiment on state legislators. American Journal of Political Science 55(3): 463–477.

Butler

Dynes

(2016) How politicians discount the opinions of constituents with whom they disagree. American Journal of Political Science 60(4): 975–989.

Butler

Nickerson

(2011) Can learning constituency opinion affect how legislators vote? Results from a field experiment. Quarterly Journal of Political Science 6(1): 55–83.

Butler

Karpowitz

Pope

(2012) A field experiment on legislators home styles: service versus policy. The Journal of Politics 74(2): 474–486.

10.

Campbell

(2003) How Policies Make Citizens: Senior Political Activism and the American Welfare State. Princeton, NJ: Princeton University Press.

11.

Castiglione

Pollak

(2018) Creating Political Presence: The New Politics of Democratic Representation. Chicago, IL: The University of Chicago Press.

12.

Congressional Management Foundation (CMF) (2005) Communicating with Congress: How Capitol Hill Is Coping with Citizen Advocacy. Washington, DC: CMF.

13.

Congressional Management Foundation (CMF) (2011) Communicating with Congress: Perceptions of Citizen Advocacy on Capitol Hill. Partnership for a More Perfect Union. Washington, DC: CMF.

14.

Costa

(2017) How responsive are political elites? A meta-analysis of experiments on public officials. Journal of Experimental Political Science 4(3): 241–254.

15.

Dahl

(1971) Polyarchy: Participation and Opposition. New Haven, CT: Yale University Press.

16.

Dexter

(1956) What do congressmen hear: the mail. Public Opinion Quarterly 20(1): 16–27.

17.

Disch

(2019) Constructivist Turn in Political Representation. Edinburgh: Edinburgh University Press.

18.

Einstein

Glick

(2017) Does race affect access to government services? An experiment exploring street-level bureaucrats and access to public housing. American Journal of Political Science 61(1): 100–116.

19.

Gambini

Fagni

Falchi

, et al. (2022) On pushing DeepFake Tweet Detection capabilities to the limits. In: 14th ACM web science conference, pp. 154–163. Available at: https://dl.acm.org/doi/abs/10.1145/3501247.3531560

20.

Griffin

Newman

(2005) Are Voters Better Represented? Journal of Politics 67(4): 1206–1227.

21.

Groh

(2020) Detect deepfakes: how to counteract misinformation created by AI. MIT Media Lab. Available at: https://www.media.mit.edu/projects/detect-fakes/overview/

22.

Hayden

(2003) The false promise of one person, one vote. Michigan Law Review 102(2): 213–266.

23.

Jakesch

French

, et al. (2019) AI-mediated communication: how the perception that profile text was written by AI affects trustworthiness. In: CHI 2019: proceedings of the 2019 CHI conference on human factors in computing systems, 4–9 May. Available at: https://dl.acm.org/doi/10.1145/3290605.3300469

24.

Kreps

McCain

Brundage

(2022) All the news that’s fit to fabricate: AI-generated text as a tool of media misinformation. Journal of Experimental Political Science 9: 104–117.

25.

Landgrave

(2020) Can we reduce deception in elite field experiments? Evidence from a field experiment with state legislative offices. State Politics & Policy Quarterly 20(4): 489–507.

26.

Lapowsky

(2017) How bots broke the FCC’s public comment system. WIRED, 28 November. Available at: https://www.wired.com/story/bots-broke-fcc-public-comment-system/

27.

Liu

Cui

Liu

, et al. (2021) Natural language inference in context—investigating contextual reasoning over long texts. Proceedings of the AAAI Conference on Artificial Intelligence 35(15): 13388–13396.

28.

Martin

Claibourn

(2013) Citizen participation and congressional responsiveness: new evidence that participation matters. Legislative Studies Quarterly 38(1): 59–81.

29.

Nyhan

Reifler

(2015) The effect of fact-checking on elites: a field experiment on U.S. state legislators. American Journal of Political Science 59(3): 628–640.

30.

Putnam

(1992) Making Democracy Work: Civic Traditions in Modern Italy. Princeton, NJ: Princeton University Press.

31.

Saward

(2008) Representation and democracy: revisions and possibilities. Sociology Compass 2(3): 1000–1013.

32.

Stimson

Mackuen

Erikson

(1995) Dynamic representation. American Political Science Review 89(3): 543–565.

33.

Strom

(2009) Coal group is linked to fake letters on climate bill. The New York Times, 4 August. Available at: https://www.nytimes.com/2009/08/05/us/politics/05charity.html

34.

Ternovski

Kalla

Aronow

(2021) Deepfake warnings for political videos increase disbelief but do not improve discernment: evidence from two experiments. Available at: https://osf.io/dta97/

35.

Tormey

(2015) The End of Representative Politics. New York: John Wiley & Sons.

36.

Zellers

Holtzman

Rashkin

, et al. (2019) Defending against neural fake news. Available at: https://arxiv.org/pdf/1905.12616.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.17 MB