Transformer-based models for transfer learning have the potential to achieve high prediction accuracies on text-based supervised learning tasks with relatively few training data instances. These models are thus likely to benefit social scientists that seek to have as accurate as possible text-based measures, but only have limited resources for annotating training data. To enable social scientists to leverage these potential benefits for their research, this article explains how these methods work, why they might be advantageous, and what their limitations are. Additionally, three Transformer-based models for transfer learning, BERT, RoBERTa, and the Longformer, are compared to conventional machine learning algorithms on three applications. Across all evaluated tasks, textual styles, and training data set sizes, the conventional models are consistently outperformed by transfer learning with Transformers, thereby demonstrating the benefits these models can bring to text-based social science research.
AgarwalSushantJabbariShahinAgarwalChiragUpadhyaySohiniWuStevenLakkarajuHimabindu. 2021. “Towards the Unification and Robustness of Perturbation and Gradient Based Explanations.” Proceedings of Machine Learning Research139:110-9.
3.
AkbikAlanBlytheDuncanVollgrafRoland. 2018. “Contextual String Embeddings for Sequence Labeling.” pp. 1638-49 in Proceedings of the 27th International Conference on Computational Linguistics, edited by Emily M. Bender, Leon Derczynski, and Pierre Isabelle. Stroudsburg, PA, USA: Association for Computational Linguistics.
AmsalemEranFogel-DrorYairShenhavShaul R.SheaferTamir. 2020. “Fine-Grained Analysis of Diversity Levels in the News.” Communication Methods and Measures14 (4): 266-84.
9.
AnastasopoulosL. JasonBertelliAnthony M.. 2020. “Understanding Delegation Through Machine Learning: A Method and Application to the European Union.” American Political Science Review114 (1): 291-301.
10.
AnconaMarcoCeoliniEneaÖztireliCengizGrossMarkus. 2018. “Towards Better Understanding of Gradient-Based Attribution Methods for Deep Neural Networks.” in 6th International Conference on Learning Representations (ICLR 2018), edited by Yann LeCun.
11.
AribandiVamsiTayYiSchusterTalRaoJinfengZhengHuaixiu StevenMehtaSanket VaibhavZhuangHongleiTranVinh Q.BahriDaraNiJianmoGuptaJaiHuiKaiRuderSebastianMetzlerDonald. 2022. “ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning.” in International Conference on Learning Representations (ICLR 2022), edited by Katja Hofmann and Alexander Rush.
12.
AßenmacherMatthiasHeumannChristian. 2020. “On the Comparability of Pre-trained Language Models.” in Proceedings of the 5th Swiss Text Analytics Conference (SwissText) & 16th Conference on Natural Language Processing (KONVENS), edited by Sarah Ebling, Don Tuggener, Manuela Hürlimann, Mark Cieliebak, and Martin Volk. CEUR-WS.org.
13.
BaJimmy LeiKirosJamie RyanHintonGeoffrey E.. 2016. “Layer Normalization.” arXiv Preprint. arXiv:1607.06450.
BaevskiAlexeiZhouYuhaoMohamedAbdelrahmanAuliMichael. 2020. “wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations.” Pp. 12449-460 in Advances in Neural Information Processing Systems 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin. Curran Associates, Inc.
16.
BahdanauDzmitryChoKyunghyunBengioYoshua. 2015. “Neural Machine Translation by Jointly Learning to Align and Translate.” in 3rd International Conference on Learning Representations (ICLR 2015), edited by Yoshua Bengio and Yann LeCun.
17.
BapnaAnkurChungYu-anWuNanGulatiAnmolJiaYeClarkJonathan H.JohnsonMelvinRiesaJasonConneauAlexisZhangYu. 2021. “SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training.” arXiv Preprint. arXiv:2110.10329.
18.
BarberáPabloBoydstunAmber E.LinnSuzannaMcMahonRyanNaglerJonathan. 2021. “Automated Text Classification of News Articles: A Practical Guide.” Political Analysis29 (1): 19-42.
19.
BelinkovYonatanGlassJames. 2019. “Analysis Methods in Neural Language Processing: A Survey.” Transactions of the Association for Computational Linguistics7:49-72.
20.
BeltagyIzPetersMatthew E.CohanArman. 2020. “Longformer: The Long-Document Transformer. arXiv Preprint.” arXiv:2004.05150.
21.
BengioYoshuaDucharmeRéjeanVincentPascalJanvinChristian. 2003. “A Neural Probabilistic Language Model.” Journal of Machine Learning Research3:1137-55.
22.
BenoitKen. 2020. “Text as Data: An Overview.” Pp. 461-97 in The SAGE Handbook of Research Methods in Political Science and International Relations, edited by L. Curini and R. Franzese. London: SAGE Publications.
23.
BenoitKennethWatanabeKoheiWangHaiyanNultyPaulObengAdamMüllerStefanMatsuoAkitaka. 2018. “quanteda: An R Package for the Quantitative Analysis of Textual Data.” Journal of Open Source Software 3(30):774.
24.
BojanowskiPiotrGraveEdouardJoulinArmandMikolovTomas. 2017. “Enriching Word Vectors With Subword Information.” Transactions of the Association for Computational Linguistics5:135-46.
25.
BommasaniRishiHudsonDrew A.AdeliEhsanAltmanRussAroraSimranArxSydney vonBernsteinMichael S.BohgJeannetteBosselutAntoineBrunskillEmmaBrynjolfssonErikBuchShyamalCardDallasCastellonRodrigoChatterjiNiladri S.ChenAnnie S.CreelKathleenDavisJared QuincyDemszkyDorottyaDonahueChrisDoumbouyaMoussaDurmusEsinErmonStefanoEtchemendyJohnEthayarajhKawinFei-FeiLiFinnChelseaGaleTrevorGillespieLaurenGoelKaranGoodmanNoah D.GrossmanShelbyGuhaNeelHashimotoTatsunoriHendersonPeterHewittJohnHoDaniel E.HongJennyHsuKyleHuangJingIcardThomasJainSaahilJurafskyDanKalluriPratyushaKaramchetiSiddharthKeelingGeoffKhaniFereshteKhattabOmarKohPang WeiKrassMark S.KrishnaRanjayKuditipudiRohithet al. 2021. “On the Opportunities and Risks of Foundation Models.” arXiv Preprint. arXiv:2108.07258.
26.
BoserBernhard E.GuyonIsabelle M.VapnikVladimir N.1992. “A Training Algorithm for Optimal Margin Classifiers.” Pp. 144-52 in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, edited by David Haussler. New York, NY, USA: Association for Computing Machinery.
BudhwarAdityaKuboiToshihiroDekhtyarAlexKhosmoodFoaad. 2018. “Predicting the Vote Using Legislative Speech.” Pp. 1-10 in Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age, edited by Anneke Zuiderwijk and Charles C. Hinnant. New York, NY, USA: Association for Computing Machinery.
CeronAndreaCuriniLuigiIacusStefano M.2015. “Using Sentiment Analysis to Monitor Electoral Campaigns: Method Matters—Evidence From the United States and Italy.” Social Science Computer Review33 (1): 3-20.
34.
CeronAndreaCuriniLuigiIacusStefano M.PorroGiuseppe. 2014. “Every Tweet Counts? How Sentiment Analysis of Social Media Can Improve Our Knowledge of Citizens’ Political Preferences with an Application to Italy and France.” New Media & Society16 (2): 340-58.
35.
ChangCharlesMastersonMichael. 2020. “Using Word Order in Political Text Classification With Long Short-Term Memory Models.” Political Analysis28 (3): 395-411.
36.
ChenTianqiGuestrinCarlos. 2016. “XGBoost: A Scalable Tree Boosting System.” Pp. 785-94 in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA: Association for Computing Machinery.
37.
ChildRewonGrayScottRadfordAlecSutskeverIlya. 2019. “Generating Long Sequences with Sparse Transformers. arXiv Preprint.” arXiv:1904.10509.
38.
ChoKyunghyunvan MerriënboerBartGulcehreCaglarBahdanauDzmitryBougaresFethiSchwenkHolgerBengioYoshua. 2014. “Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation.” Pp. 1724-34 in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Alessandro Moschitti, Bo Pang, and Walter Daelemans. Stroudsburg, PA, USA: Association for Computational Linguistics.
39.
CholletFrancois. 2020. Deep Learning with Python. New York: Manning Publications.
40.
ClarkKevinKhandelwalUrvashiLevyOmerManningChristopher D.2019. “What Does BERT Look at? An Analysis of BERT’s Attention.” Pp. 276-86 in Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, edited by Tal Linzen, Grzegorz Chrupała, Yonatan Belinkov, and Dieuwke Hupkes. Stroudsburg, PA, USA: Association for Computational Linguistics.
41.
ClarkKevinLuongMinh-ThangLeQuoc V.ManningChristopher D.2020. “ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators.” in 8th International Conference on Learning Representations, ICLR 2020, edited by Alexander Rush. OpenReview.net.
Clevert, Djork-Arné, UnterthinerThomasHochreiterSepp. 2016. “Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs).” in 4th International Conference on Learning Representations, ICLR 2016, edited by Yoshua Bengio and Yann LeCun.
44.
ColleoniElanorRozzaAlessandroArvidssonAdam. 2014. “Echo Chamber or Public Sphere? Predicting Political Orientation and Measuring Political Homophily in Twitter Using Big Data.” Journal of Communication64 (2): 317-32.
45.
ConneauAlexisKhandelwalKartikayGoyalNamanChaudharyVishravWenzekGuillaumeGuzmánFranciscoGraveEdouardOttMyleZettlemoyerLukeStoyanovVeselin. 2020. “Unsupervised Cross-lingual Representation Learning at Scale.” Pp. 8440-51 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Stroudsburg, PA, USA: Association for Computational Linguistics.
DennyMatthew J.SpirlingArthur. 2018. “Text Preprocessing for Unsupervised Learning: Why It Matters, When It Misleads, and What to Do About It.” Political Analysis26 (2): 168-89.
DevlinJacobChangMing-WeiLeeKentonToutanovaKristina. 2018. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” arXiv Preprint. arXiv:1810.04805v1.
53.
DevlinJacobChangMing-WeiLeeKentonToutanovaKristina. 2019. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” Pp. 4171-86 in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, edited by Jill Burstein, Christy Doran, and Thamar Solorio. Stroudsburg, PA, USA: Association for Computational Linguistics.
54.
Di CoccoJessicaMonechiBernardo. 2021. “How Populist are Parties? Measuring Degrees of Populism in Party Manifestos Using Supervised Machine Learning.” Political Analysis pp. 1-17.
55.
DiermeierDanielGodboutJean-FrançoisYuBeiKaufmannStefan. 2011. “Language and Ideology in Congress.” British Journal of Political Science42 (1): 31-55.
56.
D’OrazioVitoLandisSteven T.PalmerGlennSchrodtPhilip. 2014. “Separating the Wheat from the Chaff: Applications of Automated Document Classification Using Support Vector Machines.” Political Analysis22 (2): 224-42.
57.
Doshi-VelezFinaleKimBeen. 2017. “Towards A Rigorous Science of Interpretable Machine Learning.” arXiv Preprint. arXiv:1702.08608.
58.
DosovitskiyAlexeyBeyerLucasKolesnikovAlexanderWeissenbornDirkZhaiXiaohuaUnterthinerThomasDehghaniMostafaMindererMatthiasHeigoldGeorgGellySylvainUszkoreitJakobHoulsbyNeil. 2021. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” in 9th International Conference on Learning Representations (ICLR 2021), edited by Shakir Mohamed.
59.
DuchiJohnHazanEladSingerYoram. 2011. “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.” Journal of Machine Learning Research12:2121-59.
60.
DuthieRoryBudzynskaKatarzyna. 2018. “A Deep Modular RNN Approach for Ethos Mining.” Pp. 4041-47 in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, edited by Jérôme Lang. International Joint Conferences on Artificial Intelligence Organization.
61.
ElmanJeffrey L. 1990. “Finding Structure in Time.” Cognitive Science14 (2): 179-211.
62.
Ennser-JedenastikLaurenzMeyerThomas M.2018. “The Impact of Party Cues on Manual Coding of Political Texts.” Political Science Research and Methods6 (3): 625-33.
63.
FirthJohn R. 1957. Studies in Linguistic Analysis (Philological Society). Publications of the Philological Society.Oxford: Blackwell.
64.
FowlerErika FranklinFranzMichael M.MartinGregory J.PeskowitzZacharyRidoutTravis N.. 2021. “Political Advertising Online and Offline.” American Political Science Review115 (1): 130-49.
GattAlbertKrahmerEmiel. 2018. “Survey of the State of the Art in Natural Language Generation: Core Tasks, Applications and Evaluation.” Journal of Artificial Intelligence Research61 (1): 65-170.
67.
GlavašGoranNanniFedericoPonzettoSimone Paolo. 2017. “Cross-Lingual Classification of Topics in Political Texts.” Pp. 42-46 in Proceedings of the Second Workshop on NLP and Computational Social Science, edited by Dirk Hovy, Svitlana Volkova, David Bamman, David Jurgens, Brendan O’Connor, Oren Tsur, and A. Seza Doğruöz. Stroudsburg, PA, USA: Association for Computational Linguistics.
68.
GoldbergYoav. 2016. “A Primer on Neural Network Models for Natural Language Processing.” Journal of Artificial Intelligence Research57 (1): 345-420.
69.
GoodfellowIanBengioYoshuaCourvilleAaron. 2016. Deep Learning. Cambridge, MA, USA: MIT Press.
GoyalPriyaCaronMathildeLefaudeuxBenjaminXuMinWangPengchaoPaiVivekSinghMannatLiptchinskyVitaliyMisraIshanJoulinArmandBojanowskiPiotr. 2021. “Self-supervised Pretraining of Visual Features in the Wild.” arXiv Preprint. arXiv:2103.01988.
72.
GreeneKevin T.ParkBaekkwanColaresiMichael. 2019. “Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information Effects.” Political Analysis27 (2): 223-30.
73.
GrimmerJustinStewartBrandon M.2013. “Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts.” Political Analysis21 (3): 267-97.
74.
HanRujunGillMichaelSpirlingArthurChoKyunghyun. 2018. “Conditional Word Embedding and Hypothesis Testing via Bayes-by-Backprop.” Pp. 4890-95 in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, edited by Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii. Stroudsburg, PA, USA: Association for Computational Linguistics.
HowardJeremyRuderSebastian. 2018. “Universal Language Model Fine-tuning for Text Classification.” Pp. 328-39 in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Iryna Gurevych and Yusuke Miyao. Stroudsburg, PA, USA: Association for Computational Linguistics.
81.
HuMinghaoPengYuxingHuangZhenLiDongshengLvYiwei. 2019. “Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification.” Pp. 537-46 in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Anna Korhonen, David Traum, and Lluís Màrquez. Stroudsburg, PA, USA: Association for Computational Linguistics.
82.
HunterJohn D. 2007. “Matplotlib: A 2D Graphics Environment.” Computing in Science & Engineering9 (3): 90-5.
83.
IyyerMohitEnnsPeterBoyd-GraberJordanResnikPhilip. 2014. “Political Ideology Detection Using Recursive Neural Networks.” Pp. 1113-22 in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Kristina Toutanova and Hua Wu. Stroudsburg, PA, USA: Association for Computational Linguistics.
84.
JacoviAlonGoldbergYoav. 2020. “Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?” Pp. 4198-205 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Stroudsburg, PA, USA: Association for Computational Linguistics.
85.
JaegleAndrewBorgeaudSebastianAlayracJean-BaptisteDoerschCarlIonescuCatalinDingDavidKoppulaSkandaZoranDanielBrockAndrewShelhamerEvanHenaffOlivier JBotvinickMatthewZissermanAndrewVinyalsOriolCarreiraJoao. 2022. “Perceiver IO: A General Architecture for Structured Inputs & Outputs.” in International Conference on Learning Representations (ICLR 2022), edited by Katja Hofmann and Alexander Rush.
86.
JaegleAndrewGimenoFelixBrockAndyVinyalsOriolZissermanAndrewCarreiraJoao. 2021. “Perceiver: General Perception with Iterative Attention.” Pp. 4651-64 in Proceedings of the 38th International Conference on Machine Learning, edited by Marina Meila and Tong Zhang. PMLR.
87.
JamesGarethWittenDanielaHastieTrevorTibshiraniRobert. 2013. An Introduction to Statistical Learning With Applications in R. New York: Springer.
88.
JawaharGaneshSagotBenÖSeddahDjamé. 2019. “What Does BERT Learn about the Structure of Language?” Pp. 3651-57 in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Anna Korhonen, David Traum, and Lluís Màrquez. Stroudsburg, PA, USA: Association for Computational Linguistics.
JohnsonJustin. 2017. “Derivatives, Backpropagation, and Vectorization.” Lecture Handout, CS231n: Convolutional Neural Networks for Visual Recognition, Stanford University. http://cs231n.stanford.edu/handouts/derivatives.pdf.
91.
KatagiriAzusaMinEric. 2019. “The Credibility of Public and Private Signals: A Document-Based Approach.” American Political Science Review113 (1): 156-72.
KeskarNitish ShirishMudigereDheevatsaNocedalJorgeSmelyanskiyMikhailTangPing Tak Peter. 2017. “On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima.” arXiv Preprint. arXiv:1609.04836.
94.
KimJiseonGriggsEldenKimIn SongOhAlice. 2021. “Learning Bill Similarity with Annotated and Augmented Corpora of Bills.” Pp. 10048-064 in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih. Stroudsburg, PA, USA: Association for Computational Linguistics.
95.
KingmaDiederik P.BaJimmy. 2015. “Adam: A Method for Stochastic Optimization.” in 3rd International Conference on Learning Representations (ICLR 2015), edited by Yoshua Bengio and Yann LeCun.
96.
KipfThomas N.WellingMax. 2017. “Semi-Supervised Classification with Graph Convolutional Networks.” in 5th International Conference on Learning Representations, ICLR 2017, edited by Yoshua Bengio and Yann LeCun. OpenReview.net.
97.
KirkpatrickJamesPascanuRazvanRabinowitzNeilVenessJoelDesjardinsGuillaumeRusuAndrei A.MilanKieranQuanJohnRamalhoTiagoGrabska-BarwinskaAgnieszkaHassabisDemisClopathClaudiaKumaranDharshanHadsellRaia. 2017. “Overcoming Catastrophic Forgetting in Neural Networks.” Proceedings of the National Academy of Sciences114 (13): 3521-6.
98.
KitaevNikitaKaiserŁukaszLevskayaAnselm. 2020. “Reformer: The Efficient Transformer.” arXiv Preprint. arXiv:2001.04451.
99.
KobayashiGoroKuribayashiTatsukiYokoiShoInuiKentaro. 2020. “Attention is Not Only a Weight: Analyzing Transformers with Vector Norms.” Pp. 7057-75 in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu. Stroudsburg, PA, USA: Association for Computational Linguistics.
100.
KokhlikyanNarineMiglaniVivekMartinMiguelWangEdwardAlsallakhBilalReynoldsJonathanMelnikovAlexanderKliushkinaNataliaArayaCarlosYanSiqiReblitz-RichardsonOrion. 2020. “Captum: A Unified and Generic Model Interpretability Library for PyTorch.” arXiv Preprint. arXiv:2009.07896.
101.
KozlowskiAustin C.TaddyMattEvansJames A.2019. “The Geometry of Culture: Analyzing the Meanings of Class Through Word Embeddings.” American Sociological Review84 (5): 905-49.
102.
KwonK. HazelPriniskiJ. HunterChadhaMonica. 2018. “Disentangling User Samples: A Supervised Machine Learning Approach to Proxy-Population Mismatch in Twitter Research.” Communication Methods and Measures12 (2-3): 216-37.
103.
LanZhenzhongChenMingdaGoodmanSebastianGimpelKevinSharmaPiyushSoricutRadu. 2020. “ALBERT: A Lite BERT for Self-supervised Learning of Language Representations.” in 8th International Conference on Learning Representations (ICLR 2020), edited by Alexander Rush. OpenReview.net.
104.
LaverMichaelBenoitKennethGarryJohn. 2003. “Extracting Policy Positions from Political Texts Using Words as Data.” American Political Science Review97 (2): 311-31.
105.
LeQuocMikolovTomas. 2014. “Distributed Representations of Sentences and Documents.” Proceedings of the 31st International Conference on Machine Learning32 (2): 1188-96.
106.
GuillaumeLemaîtreNogueiraFernandoAridasChristos K.2017. “Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning.” Journal of Machine Learning Research18 (17): 1-5.
107.
LewisMikeLiuYinhanGoyalNamanGhazvininejadMarjanMohamedAbdelrahmanLevyOmerStoyanovVeselinZettlemoyerLuke. 2020. “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.” Pp. 7871-80 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Stroudsburg, PA, USA: Association for Computational Linguistics.
108.
LiFei-FeiKrishnaRanjayXuDanfei. 2020a. “CS231n: Convolutional Neural Networks for Visual Recognition—Optimization I. Lecture Notes, Stanford University.” Retrieved July 6, 2020 (https://cs231n.github.io/optimization-1/).
109.
LiFei-FeiKrishnaRanjayXuDanfei. 2020b. “CS231n: Convolutional Neural Networks for Visual Recognition—Optimization II. Lecture Notes, Stanford University.” Retrieved July 6, 2020 (https://cs231n.github.io/optimization-2/).
LoshchilovIlyaHutterFrank. 2019. “Decoupled Weight Decay Regularization.” in 7th International Conference on Learning Representations, ICLR 2019, edited by Tara Sainath. OpenReview.net.
112.
LuoHuaishaoJiLeiShiBotianHuangHaoyangDuanNanLiTianruiLiJasonBhartiTaroonZhouMing. 2020. “UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation.” arXiv Preprint. arXiv:2002.06353.
113.
LuongThangPhamHieuManningChristopher D.2015. “Effective Approaches to Attention-based Neural Machine Translation.” Pp. 1412-21 in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, edited by Lluís Màrquez, Chris Callison-Burch, and Jian Su. Stroudsburg, PA, USA: Association for Computational Linguistics.
114.
MacCartneyBill. 2014, July 16. “Understanding Natural Language Understanding.” [Conference Presentation]. ACM SIGAI Bay Area Chapter Inaugural Meeting, San Mateo, CA, United States. https://nlp.stanford.edu/~wcmac/papers/20140716-UNLU.pdf.
115.
ManningChristopher DRaghavanPrabhakarSchützeHinrich. 2008. Introduction to Information Retrieval. Cambridge: Cambridge University Press.
116.
MartinLouisMullerBenjaminSuárezPedro Javier OrtizDupontYoannRomaryLaurentde la ClergerieÉricSeddahDjaméSagotBenoît. 2020. “CamemBERT: a Tasty French Language Model.” Pp. 7203-19 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Stroudsburg, PA, USA: Association for Computational Linguistics.
117.
MastersDominicLuschiCarlo. 2018. “Revisiting Small Batch Training for Deep Neural Networks.” arXiv Preprint. arXiv:1804.07612.
118.
McCannBryanBradburyJamesXiongCaimingSocherRichard. 2018. “Learned in Translation: Contextualized Word Vectors.” arXiv Preprint. arXiv:1708.00107.
McKinneyWes. 2010. “Data Structures for Statistical Computing in Python.” Pp. 51-56 in Proceedings of the 9th Python in Science Conference (SciPy 2010), edited by Stéfan v. d. Walt and Jarrod Millman.
121.
MeidingerMaximilianAßenmacherMatthias. 2021. “A New Benchmark for NLP in Social Sciences: Evaluating the Usefulness of Pre-trained Language Models for Classifying Open-ended Survey Responses.” Pp. 866-73 in Proceedings of the 13th International Conference on Agents and Artificial Intelligence, ICAART 2021, edited by Ana Paula Rocha, Luc Steels, and H. Jaap v. d. Herik. SCITEPRESS.
122.
MikhaylovSlavaLaverMichaelBenoitKenneth R.2012. “Coder Reliability and Misclassification in the Human Coding of Party Manifestos.” Political Analysis20 (1): 78-91.
123.
MikolovTomasChenKaiCorradoGregDeanJeffrey. 2013a. “Efficient Estimation of Word Representations in Vector Space.” arXiv Preprint. arXiv:1301.3781.
124.
MikolovTomasSutskeverIlyaChenKaiCorradoGregDeanJeffrey. 2013b. “Distributed Representations of Words and Phrases and Their Compositionality.” Pp. 3111-19 in Proceedings of the 26th International Conference on Neural Information Processing Systems – Volume 2, edited by Christopher J. C. Burges, Léon Bottou, Max Welling, Zoubin Ghahramani, and Kilian Q. Weinberger. Red Hook, NY, USA: Curran Associates, Inc.
125.
MillerBlakeLinderFridolinMebaneWalter R.2020. “Active Learning Approaches for Labeling Text: Review and Assessment of the Performance of Active Learning Approaches.” Political Analysis28 (4): 532-51.
126.
MillerTim. 2019. “Explanation in Artificial Intelligence: Insights From the Social Sciences.” Artificial Intelligence267:1-38.
127.
MitchellMargaretAguilarJacquiWilsonTheresaVan DurmeBenjamin. 2013. “Open Domain Targeted Sentiment.” Pp. 1643-54 in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: Association for Computational Linguistics.
128.
MittsTamar. 2019. “From Isolation to Radicalization: Anti-Muslim Hostility and Support for ISIS in the West.” American Political Science Review113 (1): 173-94.
129.
MohammadSaif M.SobhaniParinazKiritchenkoSvetlana. 2017. “Stance and Sentiment in Tweets.” ACM Transactions on Internet Technology17 (3): 26:1-22.
MuchlinskiDavidYangXiaoBirchSarahMacdonaldCraigOunisIadh. 2021. “We Need to Go Deeper: Measuring Electoral Violence Using Convolutional Neural Networks and Social Media.” Political Science Research and Methods9:122-39.
132.
Münchener Digitalisierungszentrum der Bayerischen Staatsbibliothek (dbmdz). 2021. Model Card for bert-base-german-uncased from dbmdz. Retrieved May 19, 2021 (https://huggingface.co/dbmdz/bert-base-german-uncased).
133.
NairVinodHintonGeoffrey E.2010. “Rectified Linear Units Improve Restricted Boltzmann Machines.” Pp. 807-14 in Proceedings of the 27th International Conference on International Conference on Machine Learning, edited by Johannes Fürnkranz and Thorsten Joachims. Madison, WI, USA: Omnipress.
134.
NelsonLaura K.BurkDerekKnudsenMarcelMcCallLeslie. 2021. “The Future of Coding: A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods.” Sociological Methods & Research50 (1): 202-37.
135.
NguyenDat QuocTuan NguyenAnh. 2020. “PhoBERT: Pre-trained Language Models for Vietnamese.” Pp. 1037-42 in Findings of the Association for Computational Linguistics: EMNLP 2020, edited by Trevor Cohn, Yulan He, and Yang Liu. Stroudsburg, PA, USA: Association for Computational Linguistics.
136.
NozzaDeboraBianchiFedericoHovyDirk. 2020. “What the [MASK]? Making Sense of Language-Specific BERT Models.” arXiv Preprint. arXiv:2003.02912.
OliphantTravis E. 2015. A Guide to NumPy.Austin, TX: Continuum Press.
139.
OsnabrüggeMoritzAshElliottMorelliMassimo. 2021. “Cross-Domain Topic Classification for Political Texts.” Political Analysis. pp. 1-22.
140.
PanSinno JialinYangQiang. 2010. “A Survey on Transfer Learning.” IEEE Transactions on Knowledge and Data Engineering22 (10): 1345-59.
141.
PangBoLeeLillianVaithyanathanShivakumar. 2002. “Thumbs Up? Sentiment Classification Using Machine Learning Techniques.” Pp. 79-86 in Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA, USA: Association for Computational Linguistics.
142.
ParkBaekkwanGreeneKevinColaresiMichael. 2020. “Human Rights are (Increasingly) Plural: Learning the Changing Taxonomy of Human Rights from Large-Scale Text Reveals Information Effects.” American Political Science Review114 (3): 888-910.
143.
PaszkeAdamGrossSamMassaFranciscoLererAdamBradburyJamesChananGregoryKilleenTrevorLinZemingGimelsheinNataliaAntigaLucaDesmaisonAlbanKopfAndreasYangEdwardDeVitoZacharyRaisonMartinTejaniAlykhanChilamkurthySasankSteinerBenoitFangLuBaiJunjieChintalaSoumith. 2019. “PyTorch: An Imperative Style, High-Performance Deep Learning Library.” Pp. 8024-35 in Advances in Neural Information Processing Systems 32, edited by Hanna Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d. Alché-Buc, Emily Fox, and Roman Garnett. Red Hook, NY, USA: Curran Associates, Inc.
144.
PedregosaF.VaroquauxG.GramfortA.MichelV.ThirionB.GriselO.BlondelM.PrettenhoferP.WeissR.DubourgV.VanderplasJ.PassosA.CournapeauD.BrucherM.PerrotM.DuchesnayE.2011. “Scikit-learn: Machine Learning in Python.” Journal of Machine Learning Research12:2825-30.
145.
PenningtonJeffreySocherRichardManningChristopher. 2014. “GloVe: Global Vectors for Word Representation.” Pp. 1532-43 in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Alessandro Moschitti, Bo Pang, and Walter Daelemans. Stroudsburg, PA, USA: Association for Computational Linguistics.
146.
PerryPatrick O.BenoitKenneth. 2017. “Scaling Text with the Class Affinity Model.” arXiv Preprint. arXiv:1710.08963.
147.
PetersMatthewNeumannMarkIyyerMohitGardnerMattClarkChristopherLeeKentonZettlemoyerLuke. 2018a. “Deep Contextualized Word Representations.” Pp. 2227-37 in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, edited by Marilyn Walker, Heng Ji, and Amanda Stent. Stroudsburg, PA, USA: Association for Computational Linguistics.
148.
PetersMatthew E.NeumannMarkZettlemoyerLukeYihWen-tau. 2018b. “Dissecting Contextual Word Embeddings: Architecture and Representation.” Pp. 1499-1509 in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, edited by Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii. Stroudsburg, PA, USA: Association for Computational Linguistics.
149.
PetersMatthew E.RuderSebastianSmithNoah A.2019. “To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks.” Pp. 7-14 in Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), edited by Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Johannes Welbl, Alexis Conneau, Xiang Ren, and Marek Rei. Stroudsburg, PA, USA: Association for Computational Linguistics.
150.
PilehvarMohammad TaherCamacho-ColladosJose. 2020. Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning. San Rafael, CA, USA: Morgan & Claypool Publishers.
151.
PilnyAndrewMcAninchKellySloneAmandaMooreKelsey. 2019. “Using Supervised Machine Learning in Automated Content Analysis: An Example Using Relational Uncertainty.” Communication Methods and Measures13 (4): 287-304.
152.
PorterEthanVelezYamil R.2021. “Placebo Selection in Survey Experiments: An Agnostic Approach.” Political Analysis. Pp. 1-14.
R Core Team. 2020. “R: A Language and Environment for Statistical Computing.” Computer Software. Vienna, Austria: R Foundation for Statistical Computing.
155.
RadfordAlecKimJong WookHallacyChrisRameshAdityaGohGabrielAgarwalSandhiniSastryGirishAskellAmandaMishkinPamelaClarkJackKruegerGretchenSutskeverIlya. 2021. “Learning Transferable Visual Models From Natural Language Supervision.” Pp. 8748-63 in Proceedings of the 38th International Conference on Machine Learning, edited by Marina Meila and Tong Zhang. PMLR.
RaffelColinShazeerNoamRobertsAdamLeeKatherineNarangSharanMatenaMichaelZhouYanqiLiWeiLiuPeter J.2020. “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.” Journal of Machine Learning Research21 (140): 1-67.
159.
RameshAdityaPavlovMikhailGohGabrielGrayScottVossChelseaRadfordAlecChenMarkSutskeverIlya. 2021. “Zero-Shot Text-to-Image Generation.” Proceedings of Machine Learning Research139:8821-31.
160.
RameyAdam J.KlinglerJonathan D.HollibaughGary E.2019. “Measuring Elite Personality Using Speech.” Political Science Research and Methods7 (1): 163-84.
RehbeinInesPonzettoSimone PaoloAdendorfAnnaBahnsenOkeStoetzerLukasStuckenschmidtHeiner. 2021a. “Come Hither or Go Away? Recognising Pre-Electoral Coalition Signals in the News.” Pp. 7798-7810 in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih. Stroudsburg, PA, USA: Association for Computational Linguistics.
163.
RehbeinInesRuppenhoferJosefBernauerJulian. 2021b. “Who is We? Disambiguating the Referents of First Person Plural Pronouns in Parliamentary Debates.” Pp. 147-58 in Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021), edited by Kilian Evang, Laura Kallmeyer, Rainer Osswald, Jakub Waszczuk, and Torsten Zesch. Düsseldorf, Germany: KONVENS 2021 Organizers.
164.
ReimersNilsGurevychIryna. 2019. “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.” Pp. 3982-92 in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), edited by Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan. Stroudsburg, PA, USA: Association for Computational Linguistics.
165.
RheaultL.BeelenK.CochraneC.HirstG.2016. “Measuring Emotion in Parliamentary Debates with Automated Textual Analysis.” PLoS ONE11 (12): e0168843.
166.
RheaultLudovicCochraneChristopher. 2020. “Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora.” Political Analysis28 (1): 112-33.
167.
RibeiroMarco TulioWuTongshuangGuestrinCarlosSinghSameer. 2020. “Beyond Accuracy: Behavioral Testing of NLP Models with CheckList.” Pp. 4902-12 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Stroudsburg, PA, USA: Association for Computational Linguistics.
RotheSaschaSchützeHinrich. 2015. “AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes.” Pp. 1793-1803 in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), edited by Chengqing Zong and Michael Strube. Stroudsburg, PA, USA: Association for Computational Linguistics.
RudkowskyElenaHaselmayerMartinWastianMatthiasJennyMarceloEmrichStefanSedlmairMichael. 2018. “More than Bags of Words: Sentiment Analysis with Word Embeddings.” Communication Methods and Measures12 (2-3): 140-57.
178.
RumelhartD. E.HintonG. E.WilliamsR. J. 1986. “Learning Representations by Back-Propagating Errors.” Nature323:533-6.
179.
RustPhillipPfeifferJonasVulićIvanRuderSebastianGurevychIryna. 2021. “How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.” Pp. 3118-35 in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), edited by Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli. Stroudsburg, PA, USA: Association for Computational Linguistics.
180.
SchusterM.NakajimaK.2012. “Japanese and Korean Voice Search.” Pp. 5149-52 in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York, NY, USA: Institute of Electrical and Electronics Engineers (IEEE).
181.
SchützeHinrich. 1998. “Automatic Word Sense Discrimination.” Computational Linguistics24 (1): 97-123.
SebőkMiklósKacsukZoltán. 2021. “The Multiclass Classification of Newspaper Articles with Machine Learning: The Hybrid Binary Snowball Approach.” Political Analysis29 (2): 236-49.
186.
SelivanovDimitryBickelManuelWangQing. 2020. “text2vec: Modern Text Mining Framework for R.” Computer Software. http://text2vec.org/.
187.
SennrichRicoHaddowBarryBirchAlexandra. 2016. “Neural Machine Translation of Rare Words with Subword Units.” Pp. 1715-25 in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), edited by Katrin Erk and Noah A. Smith. Stroudsburg, PA, USA: Association for Computational Linguistics.
188.
SlapinJonathan B.KirklandJustin H.2020. ““The Sound of Rebellion: Voting Dissent and Legislative Speech in the UK House of Commons”.” Legislative Studies Quarterly45 (2): 153-76.
189.
SmithNoah A. 2011. Linguistic Structure Prediction. San Rafael, CA, USA: Morgan & Claypool Publishers.
190.
SocherRichardPerelyginAlexWuJeanChuangJasonManningChristopher D.NgAndrewPottsChristopher. 2013. “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank.” Pp. 1631-42 in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, edited by David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard. Stroudsburg, PA, USA: Association for Computational Linguistics.
191.
SongHyunjinTolochkoPetroEberlJakob-MoritzEiseleOlgaGreussingEstherHeidenreichTobiasLindFabienneGalygaSebastianBoomgaardenHajo G.2020. “In Validations We Trust? The Impact of Imperfect Human Annotations as a Gold Standard on the Quality of Validation of Automated Content Analysis.” Political Communication37 (4): 550-72.
192.
SpirlingArthurRodriguezPedro L.2020. “Word Embeddings: What Works, What Doesn’t, and How to Tell the Difference for Applied Research.” Manuscript. http://arthurspirling.org/documents/embed.pdf.
193.
SrivastavaNitishHintonGeoffreyKrizhevskyAlexSutskeverIlyaSalakhutdinovRuslan. 2014. “Dropout: A Simple Way to Prevent Neural Networks From Overfitting.” Journal of Machine Learning Research15 (56): 1929-58.
194.
SunChenMyersAustinVondrickCarlMurphyKevinSchmidCordelia. 2019a. “VideoBERT: A Joint Model for Video and Language Representation Learning.” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV).
195.
SunChiQiuXipengXuYigeHuangXuanjing. 2019b. “How to Fine-Tune BERT for Text Classification?” arXiv Preprint. arXiv:1905.05583.
196.
SutskeverIlyaVinyalsOriolLeQuoc V.2014. ‘Sequence to Sequence Learning with Neural Networks.” Pp. 3104-12 in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, edited by Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger. Cambridge, MA, USA: MIT Press.
197.
TayYiDehghaniMostafaAbnarSamiraShenYikangBahriDaraPhamPhilipRaoJinfengYangLiuRuderSebastianMetzlerDonald. 2021. “Long Range Arena: A Benchmark for Efficient Transformers.” in 9th International Conference on Learning Representations (ICLR 2021), edited by Shakir Mohamed.
198.
TenneyIanDasDipanjanPavlickEllie. 2019a. “BERT Rediscovers the Classical NLP Pipeline.” Pp. 4593-601 in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Anna Korhonen, David Traum, and Lluís Màrquez. Stroudsburg, PA, USA: Association for Computational Linguistics.
199.
TenneyIanXiaPatrickChenBerlinWangAlexPoliakAdamMcCoyR. T.homasKimNajoungDurmeBenjamin VanBowmanSamDasDipanjanPavlickEllie. 2019b. “What Do You Learn From Context? Probing for Sentence Structure in Contextualized Word Representations.” in 7th International Conference on Learning Representations, ICLR 2019, edited by Tara Sainath.
TheocharisYannisBarberàPabloFazekasZoltànPopaSebastian AdrianParnetOlivier. 2016. “A Bad Workman Blames His Tweets: The Consequences of Citizens’ Uncivil Twitter Use When Interacting With Party Candidates.” Journal of Communication66 (6): 1007-31.
TurneyPeter D.PantelPatrick. 2010. “From Frequency to Meaning: Vector Space Models of Semantics.” Journal of Artificial Intelligence Research37:141-88.
Van RossumGuidoDrakeFred L.2009. Python 3 Reference Manual. Scotts Valley, CA: CreateSpace.
209.
VapnikVladimir. 1991. “Principles of Risk Minimization for Learning Theory.” Pp. 831-38 in Advances in Neural Information Processing Systems 4, edited by J. Moody, S. Hanson, and R. P. Lippmann. Morgan-Kaufmann.
210.
VaswaniAshishShazeerNoamParmarNikiUszkoreitJakobJonesLlionGomezAidan NKaiserLukaszPolosukhinIllia. 2017. “Attention is All You Need.” Pp. 5998-6008 in Advances in Neural Information Processing Systems 30, edited by Isabelle Guyon, Ulrike v. Luxburg, Samy Bengio, Hanna Wallach, Rob Fergus, S. Vishwanathan, and Roman Garnett. Red Hook, NY, USA: Curran Associates, Inc.
211.
WangAlexPruksachatkunYadaNangiaNikitaSinghAmanpreetMichaelJulianHillFelixLevyOmerBowmanSamuel. 2019. “SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems 32.” Pp. 3266-80 in Advances in Neural Information Processing Systems, edited by Hanna Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d. Alché-Buc, Emily Fox, and Roman Garnett. Red Hook, NY, USA: Curran Associates, Inc.
212.
WangSinongLiBelinda Z.KhabsaMadianFangHanMaHao. 2020. “Linformer: Self-Attention With Linear Complexity.” arXiv Preprint. arXiv:2006.04768.
213.
WankmüllerSandra. 2022. “A Comparison of Approaches for Imbalanced Classification Problems in the Context of Retrieving Relevant Documents for an Analysis.” arXiv Preprint. arXiv:2205.01600.
214.
WankmüllerSandraHeumannChristian. 2021. “How to Estimate Continuous Sentiments From Texts Using Binary Training Data.” Pp. 182-92 in Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021), edited by Kilian Evang, Laura Kallmeyer, Rainer Osswald, Jakub Waszczuk, and Torsten Zesch. Düsseldorf, Germany: KONVENS 2021 Organizers.
WatanabeKohei. 2021. “Latent Semantic Scaling: A Semisupervised Text Analysis Technique for New Domains and Languages.” Communication Methods and Measures15 (2): 81-102.
217.
WeiJasonBosmaMaartenZhaoVincentGuuKelvinYuAdams WeiLesterBrianDuNanDaiAndrew M.LeQuoc V. 2022. “Finetuned Language Models are Zero-Shot Learners.” in International Conference on Learning Representations (ICLR 2022), edited by Katja Hofmann and Alexander Rush.
218.
WelbersKasperVan AtteveldtWouterBenoitKenneth. 2017. “Text Analysis in R.” Communication Methods and Measures11 (4): 245-65.
219.
WickhamHadley. 2019. “stringr: Simple, Consistent Wrappers for Common String Operations.” Computer Software.
220.
WidmannTobias,WichMaximilian. 2022. “Creating and Comparing Dictionary, Word Embedding, and Transformer-Based Models to Measure Discrete Emotions in German Political Text.” Political Analysis: 1-16.
221.
WilliamsAdinaNangiaNikitaBowmanSamuel. 2018. “A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference.” Pp. 1112-22 in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), edited by Marilyn Walker, Heng Ji, and Amanda Stent. Stroudsburg, PA, USA: Association for Computational Linguistics.
WuPatrick Y.MebaneWalter R.2021. “MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks.” arXiv Preprint. arXiv:2109.11526.
224.
WuYonghuiSchusterMikeChenZhifengLeQuoc V.NorouziMohammadMachereyWolfgangKrikunMaximCaoYuanGaoQinMachereyKlausKlingnerJeffShahApurvaJohnsonMelvinLiuXiaobingKaiserŁukaszGouwsStephanKatoYoshikiyoKudoTakuKazawaHidetoStevensKeithKurianGeorgePatilNishantWangWeiYoungCliffSmithJasonRiesaJasonRudnickAlexVinyalsOriolCorradoGregHughesMacduffDeanJeffrey. 2016. “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation.” arXiv Preprint. arXiv:1609.08144.
XueLintingConstantNoahRobertsAdamKaleMihirAl-RfouRamiSiddhantAdityaBaruaAdityaRaffelColin. 2021. “mT5: A Massively Multilingual Pre-Trained Text-to-Text Transformer.” Pp. 483-98 in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, edited by Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou. Stroudsburg, PA, USA: Association for Computational Linguistics.
227.
ZhilinZhilinxDaiZihangYangYimingCarbonellJaimeSalakhutdinovRuslanLeQuoc V. 2019. “XLNet: Generalized Autoregressive Pretraining for Language Understanding.” Pp. 5753-63 in Advances in Neural Information Processing Systems 32, edited by Hanna Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d. Alché-Buc, Emily Fox, and Roman Garnett. Red Hook, NY, USA: Curran Associates, Inc.
228.
YinPengchengNeubigGrahamYihWen-tauRiedelSebastian. 2020. “TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data.” Pp. 8413-26 in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault. Online: Association for Computational Linguistics.
229.
YinWenpengHayJamaalRothDan. 2019. “Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach.” Pp. 3914-23 in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), edited by Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan. Stroudsburg, PA, USA: Association for Computational Linguistics.
230.
YosinskiJasonCluneJeffBengioYoshuaLipsonHod. 2014. “How Transferable are Features in Deep Neural Networks?” Pp. 3320-28 in Advances in Neural Information Processing Systems 27, edited by Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger. Red Hook, NY, USA: Curran Associates, Inc.
231.
ZaheerManzilGuruganeshGuruDubeyKumar AvinavaAinslieJoshuaAlbertiChrisOntanonSantiagoPhamPhilipRavulaAnirudhWangQifanYangLiAhmedAmr. 2020. “Big Bird: Transformers for Longer Sequences.” Pp. 17283-297 in Advances in Neural Information Processing Systems 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin. Curran Associates, Inc.
232.
ZarrellaGuidoMarshAmy. 2016. “MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection.” Pp. 458-63 in Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), edited by Steven Bethard, Marine Carpuat, Daniel Cer, David Jurgens, Preslav Nakov, and Torsten Zesch. Stroudsburg, PA, USA: Association for Computational Linguistics.
233.
HanZhangPanJennifer. 2019. “CASM: A Deep-Learning Approach for Identifying Collective Action Events With Text and Image Data from Social Media.” Sociological Methodology49 (1): 1-57.
234.
ZhuYukunKirosRyanZemelRichSalakhutdinovRuslanUrtasunRaquelTorralbaAntonioFidlerSanja. 2015. “Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books.” Pp. 19-27 in Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). New York, NY, USA: Institute of Electrical and Electronics Engineers (IEEE).
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
