Automating Research in Business and Technical Communication: Large Language Models as Qualitative Coders

Abstract

The emergence of large language models (LLMs) has disrupted approaches to writing in academic and professional contexts. While much interest has revolved around the ability of LLMs to generate coherent and generically responsible texts with minimal effort and the impact that this will have on writing careers and pedagogy, less attention has been paid to how LLMs can aid writing research. Building from previous research, this study explores the utility of AI text generators to facilitate the qualitative coding research of linguistic data. This study benchmarks five LLM prompting strategies to determine the viability of using LLMs as qualitative coding, not writing, assistants, demonstrating that LLMs can be an effective tool for classifying complex rhetorical expressions and can help business and technical communication researchers quickly produce and test their research designs, enabling them to return insights more quickly and with less initial overhead.