Large language models (LLMs), through their exposure to massive collections of online text, learn to reproduce the perspectives and linguistic styles of diverse social and cultural groups. This capability suggests a powerful social scientific application—the simulation of empirically realistic, culturally situated human subjects. Synthesizing recent research in artificial intelligence and computational social science, we outline a methodological foundation for simulating human subjects and their social interactions. We then identify six characteristics of current models that are likely to impair the realistic simulation of human subjects: bias, uniformity, atemporality, disembodiment, linguistic cultures, and alien intelligence. For each of these areas, we discuss promising approaches for overcoming their associated shortcomings. Given the rate of change of these models, we advocate for an ongoing methodological program for the simulation of human subjects that keeps pace with rapid technical progress, and caution that validation against human subjects data remains essential to ensure simulation accuracy.
AlmaatouqAbdullahGriffithsThomas L.SuchowJordan W.WhitingMark E.EvansJamesWattsDuncan J.. 2022. “Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences.” The Behavioral and Brain Sciences47:1–70.
2.
AnLiZvoleffAlexLiuJianguoAxinnWilliam. 2014. “Agent-Based Modeling in Coupled Human and Natural Systems (CHANS): Lessons from a Comparative Analysis.” Annals of the Association of American Geographers.104(4):723–745.
3.
AnthisJacy ReesePaezEze. 2021. “Moral Circle Expansion: A Promising Strategy to Impact the Far Future.” Futures130(102756):102756.
4.
AnthisJacy ReesePauketatJanet V. T.LadakAliManoliAikaterina. 2024. “What Do People Think about Sentient AI?” arXiv preprint:2407.08867.
ArgyleLisa P.BailChristopher A.BusbyEthan C.GublerJoshua R.HoweThomasRyttingChristopherSorensenTaylorWingateDavid. 2023a. “Leveraging AI for Democratic Discourse: Chat Interventions Can Improve Online Political Conversations at Scale.” Proceedings of the National Academy of Sciences of the United States of America120(41):e2311627120.
8.
ArgyleLisa P.BusbyEthan C.FuldaNancyGublerJoshua R.RyttingChristopherWingateDavid. 2023b. “Out of One, Many: Using Language Models to Simulate Human Samples.” Political Analysis: An Annual Publication of the Methodology Section of the American Political Science Association31(3):337–351.
9.
AsimovIsaac.1950. I, Robot. New York, NY: Gnome Press.
10.
BaiYuntaoKadavathSauravKunduSandipanAskellAmandaKernionJacksonJonesAndyChenAnnaGoldieAnnaMirhoseiniAzaliaMcKinnonCameronChenCarolOlssonCatherineOlahChristopherHernandezDannyDrainDawnGanguliDeepLiDustinTran-JohnsonEliPerezEthanKerrJamieMuellerJaredLadishJeffreyLandauJoshuaNdousseKamalLukosuiteKamileLovittLianeSellittoMichaelElhageNelsonSchieferNicholasMercadoNoemiDasSarmaNovaLasenbyRobertLarsonRobinRingerSamJohnstonScottKravecShaunaShowkSheer ElFortStanislavLanhamTameraTelleen-LawtonTimothyConerlyTomHenighanTomHumeTristanBowmanSamuel R.Hatfield-DoddsZacMannBenAmodeiDarioJosephNicholasMcCandlishSamBrownTomKaplanJared. 2022. “Constitutional AI: Harmlessness from AI Feedback”.
11.
BailChristopher A.2024. “Can Generative AI Improve Social Science?” Proceedings of the National Academy of Sciences of the United States of America121(21):e2314021121.
12.
BaldassarriDeliaBearmanPeter. 2007. “Dynamics of Political Polarization.” American Sociological Review72(5):784–811.
13.
BenderEmily M.GebruTimnitMcMillan-MajorAngelinaShmitchellShmargaret. 2021“On the Dangers of Stochastic Parrots.” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. New York, NY, USA: ACM.
14.
BengioYoshuaLouradourJérômeCollobertRonanWestonJason. 2009. “Curriculum Learning.” In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ‘09, pp. 41–48. New York, NY, USA: Association for Computing Machinery.
15.
BisbeeJamesClintonJoshua D.DorffCassyKenkelBrentonLarsonJennifer M.. 2024. “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models.” Political Analysis: An Annual Publication of the Methodology Section of the American Political Science Association32:1–16.
16.
BoelaertJulienCoavouxSamuelOllionEtiennePetevIvaylo D.PrägPatrick. 2024. “Machine Bias: How Do Generative Language Models Answer Opinion Polls?” SocArXiv.
17.
BommasaniRishiHudsonDrew A.AdeliEhsanAltmanRussAroraSimranvon ArxSydneyBernsteinMichael S.et al.2021. “On the Opportunities and Risks of Foundation Models.” arXiv preprint:2108.07258.
18.
BrickenTrentonTempletonAdlyBatsonJoshuaChenBrianJermynAdamConerlyTomTurnerNicholas L.AnilCemDenisonCarsonAskellAmandaLasenbyRobertWuYifanKravecShaunaSchieferNicholasMaxwellTimJosephNicholasTamkinAlexNguyenKarinaMcLeanBraydenBurkeJosiah E.HumeTristanCarterShanHenighanTomOlahChris. 2023. “Towards Monosemanticity: Decomposing Language Models with Dictionary Learning.” Anthropic. Transformer Circuits Thread 2.
19.
BrownTom B.MannBenjaminRyderNickSubbiahMelanieKaplanJaredDhariwalPrafullaNeelakantanArvindShyamPranavSastryGirishAskellAmandaAgarwalSandhiniHerbert-VossArielKruegerGretchenHenighanTomChildRewonRameshAdityaZieglerDaniel M.WuJeffreyWinterClemensHesseChristopherChenMarkSiglerEricLitwinMateuszGrayScottChessBenjaminClarkJackBernerChristopherMcCandlishSamRadfordAlecSutskeverIlyaAmodeiDario. 2020. “Language Models Are Few-Shot Learners.” Advances in Neural Information Processing Systems33:1877–1901.
20.
BruchElizabethAtwellJon. 2013. “Agent-Based Models in Empirical Social Research.” Sociological Methods & Research44(2):186–221.
21.
CartmillErica A.2022. “Gesture.” Annual Review of Anthropology51(1):455–473.
22.
CeruloKaren A.2018. “Scents and Sensibility: Olfaction, Sense-Making, and Meaning Attribution.” American Sociological Review83(2):361–389.
23.
ChafeW.DanielewiczJ.. 1987 "Properties of Spoken and Written Language." Pp. 1-26 in Comprehending Oral and Written Language. New York, NY: Academic Press.
24.
ChalmersDavid J.1997. The Conscious Mind: In Search of a Fundamental Theory. New York, NY: Oxford University Press.
25.
ChiangTed.2010. The Lifecycle of Software Objects. Burton, MI: Subterranean Press.
26.
CsájiBalázs Csanád. 2001. “Approximation with Artificial Neural Networks.” Faculty of Sciences, Etvs Lornd University, Hungary24(48):7.
27.
CuiZiyanLiNingZhouHuaikang. 2024. “Can AI Replace Human Subjects? A Large-Scale Replication of Psychological Experiments with LLMs.” arXiv preprint:2409.00128.
28.
DaiDamaiSunYutaoDongLiHaoYaruMaShumingSuiZhifangWeiFuru. 2023. “Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers.” arXiv preprint:2212.10559.
29.
DavidsonThomas.2024. “Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research.” Socius: Sociological Research for a Dynamic World10:1–17.
30.
DentellaVittoriaGüntherFritzLeivadaEvelina. 2023. “Systematic Testing of Three Language Models Reveals Low Language Accuracy, Absence of Response Stability, and a Yes-Response Bias.” Proceedings of the National Academy of Sciences of the United States of America120(51):e2309583120.
31.
DettmersTimPagnoniArtidoroHoltzmanAriZettlemoyerLuke. 2023. “QLoRA: Efficient finetuning of quantized LLMs”. Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS ’23) 37(441):10088–10115.
32.
DiMaggioPaulGaripFiliz. 2012. “Network Effects and Social Inequality.” Annual Review of Sociology38(2012):93–118.
33.
DodgeJesseIlharcoGabrielSchwartzRoyFarhadiAliHajishirziHannanehSmithNoah. 2020. “Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping.” arXiv preprint:2002.06305.
34.
DriemanG. H. J.1962. “Differences between Written and Spoken Language: An Exploratory Study.” Acta Psychologica20:36–57.
35.
EntwisleBarbaraVerderyAshtonWilliamsNathalie. 2020. “Climate Change and Migration: New Insights from a Dynamic Model of Out-Migration and Return Migration.” American Journal of Sociology125(6):1469–1512.
36.
EpsteinJoshua.1999. “Agent-Based Computational Models and Generative Social Science.” Complexity4(5):41–60.
37.
EpsteinJoshua M.2006. Generative Social Science: Studies in Agent-Based Computational Modeling. Princeton, NJ: Princeton University Press.
38.
EpsteinJoshua M.AxtellRobert. 1996. Growing Artificial Societies: Social Science from the Bottom Up. Washington, DC: Brookings Institution Press.
39.
FarquharSebastianKossenJannikKuhnLorenzGalYarin. 2024. “Detecting Hallucinations in Large Language Models Using Semantic Entropy.” Nature630(8017):625–630.
40.
FatemiBahareKazemiMehranTsitsulinAntonMalkanKarishmaYimJinyeongPalowitchJohnSeoSungyongHalcrowJonathanPerozziBryan. 2024. “Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning.” arXiv preprint:2406.09170.
41.
FilippasApostolosHortonJohn J.ManningBenjamin S.. 2024. “Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?.” Pp. 614-615 in Proceedings of the 25th ACM Conference on Economics and Computation. New York, NY: ACM.
42.
FowlerJames H.ChristakisNicholas A.. 2008. “Dynamic Spread of Happiness in a Large Social Network: Longitudinal Analysis Over 20 Years in the Framingham Heart Study.” BMJ337:a2338.
43.
GargNikhilSchiebingerLondaJurafskyDanZouJames. 2018. “Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes.” Proceedings of the National Academy of Sciences of the United States of America115(16):E3635–E3644.
44.
GarneloMartaShanahanMurray. 2019. “Reconciling Deep Learning with Symbolic Artificial Intelligence: Representing Objects and Relations.” Current Opinion in Behavioral Sciences29:17–23.
45.
GeZhiqiHuangHongzheZhouMingzeLiJunchengWangGuomingTangSiliangZhuangYueting. 2024. “WorldGPT: Empowering LLM as Multimodal World Model.” pp. 7346–55 in Proceedings of the 32nd ACM International Conference on Multimedia. New York, NY, USA: ACM.
46.
GeieckeFriedrichJaravelXavier. 2024. “Conversations at Scale: Robust AI-Led Interviews with a Simple Open-Source Platform.” SSRN 4974382.
47.
GendronGaëlRožanecJože M.WitbrockMichaelDobbieGillian. 2024. “Counterfactual Causal Inference in Natural Language with Large Language Models.” arXiv preprint:2410.06392.
48.
GinosarShirley.2024. “Behavior Prediction for Interacting Entities from Video Observations.” Presented at the Research at TTIC Seminar, November 8, Chicago IL: Toyota Institute of Technology.
49.
GoffmanErving.1979. Goffman: Gender Advertisements. London, England: Harvard University Press.
50.
GolderScott A.MacyMichael W.. 2011. “Diurnal and Seasonal Mood Vary with Work, Sleep, and Daylength Across Diverse Cultures.” Science333(6051):1878–1881.
51.
GoldsteinJosh A.ChaoJasonGrossmanShelbyStamosAlexTomzMichael. 2024a. “How Persuasive Is AI-Generated Propaganda?” PNAS Nexus3(2):gae034.
52.
GoldsteinArielGrinstein-DabushAvigailSchainMarianoWangHaochengHongZhuoqiaoAubreyBobbiNastaseSamuel A.ZadaZaidHamEricFederAmirGazulaH.BuchnikEliavDoyleW.DevoreS.DuganPatriciaReichartRoiFriedmanDanielBrennerMichaelHassidimAvinatanDevinskyO.FlinkerA.HassonUri. 2024b. “Alignment of Brain Embeddings and Artificial Contextual Embeddings in Natural Language Points to Common Geometric Patterns.” Nature Communications15(1):2768.
53.
GoldsteinArielZadaZaidBuchnikEliavSchainMarianoPriceA.AubreyBobbiNastaseSamuel A.FederAmirEmanuelDotanCohenAlonJansenA.GazulaH.ChoeGinaRaoAditiKimCatherineCastoColtonFandaLoraDoyleW.FriedmanD.DuganPatriciaMelloniL.ReichartRoiDevoreS.FlinkerA.HasenfratzLiatLevyOmerHassidimAvinatanBrennerMichaelMatiasYossiNormanK.DevinskyO.HassonU.. 2022. “Shared Computational Principles for Language Processing in Humans and Deep Language Models.” Nature Neuroscience25(3):369–380.
GurneeWesTegmarkMax. 2023. “Language Models Represent Space and Time.” arXiv preprint:2310.02207.
56.
GururanganSuchinMarasovićAnaSwayamdiptaSwabhaLoKyleBeltagyIzDowneyDougSmithNoah A.. 2020. “Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks.” arXiv preprint:2004.10964.
57.
HallidayMichael A. K.1987 “Spoken and Written Modes of Meaning.” Pp. 55–82 in Comprehending Oral and Written Language. San Diego, CA: Academic Press.
58.
HamiltonWilliam L.LeskovecJureJurafskyDan. 2016. “Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change.” arXiv preprint:1605.09096.
59.
HandaDivijChirmuleAdvaitGajeraBimalBaralChitta. 2024. When “Competency” in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers.” arXiv preprint:2402.10601.
60.
HansenLuke R.2023. “On the Existence of Robot Zombies and Our Ethical Obligations to AI Systems.” Journal of Social Computing4(4):270–274.
HelbingDirkYuWenjian. 2009. “The Outbreak of Cooperation among Success-Driven Individuals Under Noisy Conditions.” Proceedings of the National Academy of Sciences of the United States of America106(10):3680–3685.
HeritageJohnRaymondGeoffrey. 2005. “The Terms of Agreement: Indexing Epistemic Authority and Subordination in Talk-in-Interaction.” Social Psychology Quarterly68(1):15–38.
65.
HillFelixLampinenAndrewSchneiderRosaliaClarkStephenBotvinickMatthewMcClellandJames L.SantoroAdam. 2019. “Environmental Drivers of Systematicity and Generalization in a Situated Agent.” arXiv preprint arXiv:1910.00571.
66.
HimmaKenneth Einar. 2009. “Artificial Agency, Consciousness, and the Criteria for Moral Agency: What Properties Must an Artificial Agent Have to Be a Moral Agent?” Ethics and Information Technology11(1):19–29.
67.
HongZhuoqiaoWangHaochengZadaZaidGazulaHarshvardhanTurnerDavidAubreyBobbiNiekerkenLeonardDoyleWernerDevoreSashaDuganPatriciaFriedmanDanielDevinskyOrrinFlinkerAdeenHassonUriNastaseSamuel A.GoldsteinAriel. 2024. “Scale Matters: Large Language Models with Billions (rather than Millions) of Parameters Better Match Neural Representations of Natural Language.” bioRxiv.org: The Preprint Server for Biology 2024.06. 12.598513.
68.
HornikKurtStinchcombeMaxwellWhiteHalbert. 1989. “Multilayer Feedforward Networks Are Universal Approximators.” Neural Networks: The Official Journal of the International Neural Network Society2(5):359–366.
69.
HuaWenyueFanLizhouLiLingyaoMeiKaiJiJianchaoGeYingqiangHemphillLibbyZhangYongfeng. 2023. “War and Peace (WarAgent): Large Language Model-Based Multi-Agent Simulation of World Wars.” arXiv preprint arXiv:2311.17227.
70.
HuJ. E.ShenYelongWallisPhillipAllen-ZhuZeyuanLiYuanzhiWangSheanChenWeizhu. 2021. “LoRA: Low-Rank Adaptation of Large Language Models.” International Conference on Learning Representationsabs/2106.09685.
71.
HuangGuanxiongWangSai. 2023. “Is Artificial Intelligence More Persuasive than Humans? A Meta-Analysis.” The Journal of Communication73(6):552–562.
72.
HuhMinyoungCheungBrianWangTongzhouIsolaPhillip. 2024. “Position: The Platonic Representation Hypothesis.” In Forty-first International Conference on Machine Learning.openreview.net.
73.
JinHaiboZhouAndyMenkeJoe D.WangHaohan. 2024. “Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters.” Advances in Neural Information Processing Systems37:59408–59435.
74.
JoEunkyungEpsteinDaniel A.JungHyunhoonKimYoung-Ho. 2023. “Understanding the Benefits and Challenges of Deploying Conversational AI Leveraging Large Language Models for Public Health Intervention.” Pp. 1–16 in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM.
75.
JonesCameron R.BergenBenjamin K.. 2025. “Large Language Models Pass the Turing Test.” arXiv preprint arXiv:2503.23674
76.
JumperJohnEvansRichardPritzelAlexanderGreenTimFigurnovMichaelRonnebergerOlafTunyasuvunakoolKathrynBatesRussŽídekAugustinPotapenkoAnnaBridglandAlexMeyerClemensKohlSimon A. A.BallardAndrew J.CowieAndrewRomera-ParedesBernardinoNikolovStanislavJainRishubAdlerJonasBackTrevorPetersenStigReimanDavidClancyEllenZielinskiMichalSteineggerMartinPacholskaMichalinaBerghammerTamasBodensteinSebastianSilverDavidVinyalsOriolSeniorAndrew W.KavukcuogluKorayKohliPushmeetHassabisDemis. 2021. “Highly Accurate Protein Structure Prediction with AlphaFold.” Nature596(7873):583–589.
77.
KahnemanDaniel.2011. Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux.
78.
KaplanJaredMcCandlishSamHenighanTomBrownTom B.ChessBenjaminChildRewonGrayScottRadfordAlecWuJeffreyAmodeiDario. 2020. “Scaling Laws for Neural Language Models.” arXiv preprint:2001.08361
79.
KimKhwanAskinNoahEvansJames A.. 2024. “Disrupted Routines Anticipate Musical Exploration.” Proceedings of the National Academy of Sciences of the United States of America121(6):e2306549121.
80.
KimJunsolEvansJamesScheinAaron. 2024. “Linear Representations of Political Perspectives Emerge in Large Language Models.” In Conference on Neural Information Processing Systems.
81.
KimJunsolLeeByungkyu. 2023. “AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction.” arXiv preprint arXiv:2305.09620.
82.
KozlowskiAustin C.KwonHyunkuEvansJames A.. 2024. “In Silico Sociology: Forecasting COVID-19 Polarization with Large Language Models.” arXiv preprint:2407.11190.
83.
KozlowskiAustin C.TaddyMattEvansJames A.. 2019. “The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings.” American Sociological Review84(5):905–949.
84.
KreuterFraukePresserStanleyTourangeauRoger. 2009. “Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity.” Public Opinion Quarterly72(5):847–865.
85.
LaiShiyangPotterYujinKimJunsolZhuangRichardSongDawnEvansJames. 2024. “Position: Evolving AI Collectives Enhance Human Diversity and Enable Self-Regulation.” In Forty-first International Conference on Machine Learning. openreview.net.
86.
LeeSangukYangKai-QiPengTai-QuanHeoRuthLiuHui. 2024. “Exploring Social Desirability Response Bias in Large Language Models: Evidence from GPT-4 Simulations.”
87.
LiYuanZhangYixuanSunLichao. 2023. “MetaAgents: Simulating Interactions of Human Behaviors for LLM-Based Task-Oriented Coordination via Collaborative Generative Agents.”
88.
LiangYanchengChenDaphneGuptaAbhishekDuSimon S.JaquesNatasha. 2024. “Learning to Cooperate with Humans Using Generative Agents.” arXiv preprint:2411.13934
89.
LiuRuiboYangRuixinJiaChenyanZhangGeZhouDennyDaiAndrew M.YangDiyiVosoughiSoroush. 2023. “Training Socially Aligned Language Models on Simulated Social Interactions.”
90.
LongRobertSeboJeffButlinPatrickFinlinsonKathleenFishKyleHardingJacquelinePfauJacobSimsToniBirchJonathanChalmersDavid. 2024. “Taking AI Welfare Seriously.”
91.
LudwigJensMullainathanSendhilRambachanAshesh. 2024. “Large Language Models: An Applied Econometric Framework.” NBER Working Paper 33344.
92.
MarancaA. R. P.ChungJ.HinckM.WolskyA.EgamiN.StewartB.. 2025. “How to Use Generative AIs for Image Analyses in the Social Sciences: Design-based Supervised Learning.” Sociological Methods and Research54(3).
93.
MartinJohn Levi. 2023. “The Ethico-Political Universe of ChatGPT.” Journal of Social Computing4(1):1–11.
94.
McCullochWarren S.PittsWalter. 1943. “A Logical Calculus of the Ideas Immanent in Nervous Activity.” The Bulletin of Mathematical Biophysics5(4):115–133.
95.
MerchantAmilBatznerSimonSchoenholzSamuel S.AykolMuratahanCheonGowoonCubukEkin Dogus. 2023. “Scaling Deep Learning for Materials Discovery.” Nature624(7990):80–85.
96.
MorningAnn.2008. “Reconstructing Race in Science and Society: Biology Textbooks, 1952–2002.” American Journal of Sociology114(Suppl):S106–S137.
97.
MunkhdalaiTsendsurenFaruquiManaalGopalSiddharth. 2024. “Leave No Context Behind: Efficient Infinite Context Transformers with Infini-Attention.” arXiv preprint:2404.07143.
98.
NaveedHumzaKhanAsad UllahQiuShiSaqibMuhammadAnwarSaeedUsmanMuhammadAkhtarNaveedBarnesNickMianAjmal. 2023. “A Comprehensive Overview of Large Language Models.” arXiv preprint:2307.06435.
99.
NelsonMichelle R.2008. “The Hidden Persuaders: Then and Now.” Journal of Advertising37(1):113–126.
100.
NgEvonneGinosarShiryDarrellTrevorJooHanbyul. 2021. “Body2hands: Learning to Infer 3d Hands from Conversational Gesture Body Dynamics.” 11865-11874.
NgEvonneSubramanianSanjayKleinDanKanazawaAngjooDarrellTrevorGinosarShiry. 2023. “Can Language Models Learn to Listen?” 10083-10093.
103.
NiuZhaoyangZhongGuoqiangYuHui. 2021. “A Review on the Attention Mechanism of Deep Learning.” Neurocomputing452:48–62.
104.
NussbaumMartha C.2009. Frontiers of Justice. London, England: Belknap Press.
105.
NyeMaxwellAndreassenAnders JohanGur-AriGuyMichalewskiHenrykAustinJacobBieberDavidDohanDavidLewkowyczAitorBosmaMaartenLuanDavidSuttonCharlesOdenaAugustus. 2021. Show Your Work: Scratchpads for Intermediate Computation with Language Models.”
106.
OpenAI. 2023. “GPT-4 Technical Report.”
107.
OuyangLongWuJeffJiangXuAlmeidaDiogoWainwrightCarroll L.MishkinPamelaZhangChongAgarwalSandhiniSlamaKatarinaRayAlexSchulmanJohnHiltonJacobKeltonFraserMillerLukeSimensMaddieAskellAmandaWelinderPeterChristianoPaulLeikeJanLoweRyan. 2022. “Training Language Models to Follow Instructions with Human Feedback.”
108.
PanicksseryNinaGabrieliNickSchulzJulianTongMegHubingerEvanTurnerAlexander Matt. 2023. “Steering Llama 2 via Contrastive Activation Addition.”
109.
ParkJoon SungO’brienJoseph CCaiCarrie J.MorrisMeredith RingelLiangPercyBernsteinMichael S.. 2023. “Generative Agents: Interactive Simulacra of Human Behavior.” Pp.1–23 in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ‘23), San Francisco, CA. New York, NY: Association for Computing Machinery.
PearlJudea.2009. Causality. Cambridge, UK: Cambridge University Press.
112.
PiaoJinghuaLuZhihongGaoChenXuFengliSantosFernando P.LiYongEvansJames. 2025. “Emergence of Human-like Polarization among Large Language Model Agents.”
113.
PotterYujiaLaiShiyangEvansJamesSongDawn. 2024a. “Hidden Persuaders: How LLM Political Bias Could Sway Our Elections.” in Empirical Methods in Natural Language Processing.
114.
PotterYujinLaiShiyangKimJunsolEvansJamesSongDawn. 2024b. “Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters.” arXiv preprint:2410.24190
115.
RadfordAlecWuJeffreyChildRewonLuanDavidAmodeiDarioSutskeverIlya. 2019. Language Models Are Unsupervised Multitask Learners. San Francisco, CA: OpenAI.
116.
RafailovRafaelSharmaArchitMitchellEricErmonStefanoManningChristopher D.FinnChelsea. 2023. “Direct Preference Optimization: Your Language Model Is Secretly a Reward Model.” Advances in Neural Information Processing Systems36:53728–53741.
117.
RamanNarunLundyTaylorAmouyalSamuelLevineYoavLeyton-BrownKevinTennenholtzMoshe. 2024. “STEER: Assessing the Economic Rationality of Large Language Models.” arXiv preprint:2402.09552
118.
RosenblattF.1958. “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain.” Psychological Review65(6):386–408.
119.
RubinDonald B.2004. Multiple Imputation for Nonresponse in Surveys. New York, NY: John Wiley & Sons.
120.
SalechaAadeshIrelandMolly E.SubrahmanyaShashankaSedocJoãoUngarLyle H.EichstaedtJohannes C.. 2024. “Large Language Models Show Human-like Social Desirability Biases in Survey Responses.” arXiv preprint:2405.06058.
121.
SanturkarShibaniDurmusEsinLadhakFaisalLeeCinooLiangPercyHashimotoTatsunori. 2023. “Whose Opinions Do Language Models Reflect?.” Pp. 29971–29974inProceedings of the 40th International Conference on Machine Learning. Vol. 202, Proceedings of Machine Learning Research, edited by KrauseA.BrunskillE.ChoK.EngelhardtB.SabatoS.ScarlettJ.. PMLR.
122.
SavcisensGermansEliassi-RadTinaHansenLars KaiHvas MortensenLaustLilleholtLauRogersAnnaZettlerIngoLehmannSune. 2024. “Using Sequences of Life-Events to Predict Human Lives.” Nature Computational Science4(1):43–56.
123.
SchellingThomas C.1978. Micromotives and Macrobehavior. New York, NY: W.W. Norton and Company.
124.
ScherrerNinoShiClaudiaFederAmirBleiDavid. 2024. “Evaluating the Moral Beliefs Encoded in Llms.” Advances in Neural Information Processing Systems36.
125.
ShanahanM.McDonellKyleReynoldsLaria. 2023. “Role Play with Large Language Models.” Nature623:493–498.
126.
SharmaMrinankTongMegKorbakTomaszDuvenaudDavidAskellAmandaBowmanSamuel R.ChengNewtonDurmusEsinHatfield-DoddsZacJohnstonScott R.KravecShaunaMaxwellTimothyMcCandlishSamNdousseKamalRauschOliverSchieferNicholasYanDaZhangMirandaPerezEthan. 2023. “Towards Understanding Sycophancy in Language Models.” arXiv preprint:2310.13548
127.
ShiFengEvansJames. 2023. “Surprising Combinations of Research Contents and Contexts Are Related to Impact and Emerge with Scientific Outsiders from Distant Disciplines.” Nature Communications14(1641). doi: https://doi.org/10.1038/s41467-023-36741-4
128.
SiegelZachary S.KapoorSayashNagdirNityaStroeblBenediktNarayananArvind. 2024. “CORE-Bench: Fostering the Credibility of Published Research through a Computational Reproducibility Agent Benchmark.” arXiv preprint:2409.11363
129.
SkarpelisA. K. M.2023. “Horror Vacui: Racial Misalignment, Symbolic Repair, and Imperial Legitimation in German National Socialist Portrait Photography.” American Journal of Sociology129(2):313–383.
130.
SmithJeffrey A.BurowJessica. 2020. “Using Ego Network Data to Inform Agent-Based Models of Diffusion.” Sociological Methods & Research49(4):1018–1063.
131.
SouratiJamshidEvansJames A.. 2023. “Accelerating Science with Human-Aware Artificial Intelligence.” Nature Human Behaviour7:1682-1696.
SuiPeiqiDuedeEamonWuSophieSoRichard. 2024. “Confabulation: The Surprising Value of Large Language Model Hallucinations.” Pp. 14274–14284 in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA, USA: Association for Computational Linguistics.
134.
SunChiQiuXipengXuYigeHuangXuanjing. 2019. “How to Fine-Tune BERT for Text Classification?” Pp. 194–206 in Chinese Computational Linguistics. Singapore: Springer International Publishing.
135.
SuttonRichard.2019. “The Bitter Lesson.” Incomplete Ideas (Blog)13(1):38.
136.
TeamGemini. 2024. “Gemini 1.5: Unlocking Multimodal Understanding across Millions of Tokens of Context.” arXiv preprint:2403.05530.
137.
TempletonAdlyConerlyTomMarcusJonathanLindseyJackBrickenTrentonChenBrianPearceAdamCitroCraigAmeisenEmmanuelJonesAndyCunninghamHoagyTurnerNicholas L.McDougallCallumMacDiarmidMonteTamkinAlexDurmusEsinHumeTristanMosconiFrancescoFreemanC. DanielSumersTheodore R.ReesEdwardBatsonJoshuaJermynAdamCarterShanOlahChrisHenighanTom. 2024. “Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.” Anthropic.
138.
TörnbergPetterValeevaDiliaraUitermarkJustusBailChristopher. 2023. “Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms.”
139.
VafaKeyonPalikotEmilDuTianyuKanodiaAyushAtheySusanBleiDavid M.. 2022. “CAREER: A Foundation Model for Labor Sequence Data.” arXiv preprint:2202.08370
140.
VaswaniAshishShazeerNoamParmarNikiUszkoreitJakobJonesLlionGomezAidan N.KaiserŁ. UkaszPolosukhinIllia. 2017“Attention is All you Need”. 31st Conference on Neural Information Processing Systems (NIPS 2017) 31:5998–6008.
141.
VicinanzaPaulGoldbergAmirSrivastavaSameer B.. 2023. “A Deep-Learning Model of Prescient Ideas Demonstrates That They Emerge from the Periphery.” PNAS Nexus2(1):gac275.
142.
VincentJames.2016. “Twitter Taught Microsoft’s AI Chatbot to Be a Racist Asshole in Less than a Day.” The Verge, May 24.
143.
VongWai KeenWangWentaoOrhanA. EminLakeBrenden M.. 2024. “Grounded Language Acquisition through the Eyes and Ears of a Single Child.” Science (New York, N.Y.)383(6682):504–511.
144.
Von OswaldJohannesNiklassonEyvindRandazzoEttoreSacramentoJoaoMordvintsevAlexanderZhmoginovAndreyVladymyrovMax. 2023. “Transformers Learn In-Context by Gradient Descent.” Pp. 35151–35174inProceedings of the 40th International Conference on Machine Learning. Vol. 202, Proceedings of Machine Learning Research, edited by KrauseA.BrunskillE.ChoK.EngelhardtB.SabatoS.ScarlettJ.. PMLR.
145.
WangHaoranShuKai. 2023. “Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment.” arXiv preprint:2311.09433.
146.
WeiAlexanderHaghtalabNikaSteinhardtJacob. 2023 “Jailbroken: How Does LLM Safety Training Fail?” Pp. 80079–80110 in Vol. abs/2307.02483, edited by OhA.NaumannT.GlobersonA.SaenkoK.HardtM.LevineS.. Curran Associates, Inc.
147.
WeiJasonWangXuezhiSchuurmansDaleBosmaMaartenChiE.XiaF.LeQuocZhouDenny. 2022. “Chain of Thought Prompting Elicits Reasoning in Large Language Models.” Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS '22).36:24824-24837.
148.
WendlerChrisVeselovskyVeniaminMoneaGiovanniWestRobert. 2024. “Do Llamas Work in English? On the Latent Language of Multilingual Transformers.” In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics.
149.
XieZikai.2024. “Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models.” arXiv preprint:2408.05093.
150.
XieChengxingChenCanyuJiaFeiranYeZiyuLaiShiyangShuKaiGuJindongBibiAdelHuZiniuJurgensDavidEvansJamesTorrPhilipGhanemBernardLiGuohao. 2024. “Can Large Language Model Agents Simulate Human Trust Behavior?” Proceedings of the 38th Annual Conference on Neural Information Processing Systems (NeurIPS '24)38:1–36.
151.
XuFengliLiYongJinDepengLuJianhuaSongChaoming. 2021. “Emergence of Urban Growth Patterns from Human Mobility Behavior.” Nature Computational Science1(12):791–800.
152.
XuNanMaXuezhe. 2024. “LLM the Genius Paradox: A Linguistic and Math Expert’s Struggle with Simple Word-Based Counting Problems.” arXiv preprint:2410.14166.
153.
YuanYouliangJiaoWenxiangWangWenxuanHuangJen-TseHePinjiaShiShumingTuZhaopeng. 2023. “GPT-4 Is Too Smart to Be Safe: Stealthy Chat with LLMs via Cipher.” arXiv preprint:2308.06463
154.
ZhaoBowenBrumbaughZanderWangYizhongHajishirziHannanehSmithNoah A.. 2024. “Set the Clock: Temporal Alignment of Pretrained Language Models.” arXiv preprint:2402.16797
155.
ZhouDiZhangYinxian. 2024. “Political Biases and Inconsistencies in Bilingual GPT Models—The Cases of the U.S. and China.” Scientific Reports14(1):25048.
