Sage Journals: Discover world-class research

Abstract

This study explores how prompting techniques, especially those integrated with rhetorical analysis results, may improve the effectiveness of artificial intelligence (AI)-generated business communication messages. I conducted an experiment to assess the effectiveness of these prompting techniques in the context of crafting a negative message generated with ChatGPT 3.5 (n = 85). A multiple regression was calculated to explore prompting techniques’ impact on the negative message grades and how each technique influences the message grade. The results (F(4, 80) = 31.84, p < .001), with an adjusted R²= .595, indicate a positive relationship between prompting techniques and the effectiveness of AI-generated messages. This study also identified challenges related to students’ AI literacy. I conclude the study by recommending practical measures on how to incorporate AI into business and professional writing classrooms.

Keywords

ChatGPT prompting techniques prompt engineering rhetorical genre analysis zero-shot few-shot feedback loops AI-assisted writing role prompting AI literacy

Get full access to this article

View all access options for this article.

References

Bašić

Banovac

Kružić

, et al. (2023). ChatGPT-3.5 as writing assistance in students’ essays. Humanities & Social Sciences Communications, 10, 750. https://doi.org/10.1057/s41599-023-02269-7

Bawarshi

A. S.

Reiff

M. J.

(2010). Genre: An introduction to history, theory, research, and pedagogy. Parlor Press; The WAC Clearinghouse. Retrieved from https://wac.colostate.edu/books/referenceguides/bawarshi-reiff/

Cain

(2023). Prompting change: Exploring prompt engineering in large language model AI and its potential to transform education. TechTrends, 68(1), 47–57. https://doi.org/10.1007/s11528-023-00896-0

Chenault

(n.d.). Beginner’s guide: How to incorporate feedback loops in ChatGPT. https://www.aidare.com/beginners-guide-how-to-incorporate-feedback-loops-in-chatgpt/

Clinton

(2023). The complete obsolete guide to generative AI. Manning.

Cohen

(n.d.). Prompt engineering: Automatic application idea to code generator. LinkedIn. https://www.linkedin.com/pulse/prompt-engineering-automatic-application-idea-code-generator-cohen/

Devitt

A. J.

(2004). Writing genres. Southern Illinois University Press.

Dhaduk

(2023). Reinforcement Learning from Human Feedback (RLHF): Benefits, challenges, key components, working. Simform. https://www.simform.com/blog/reinforcement-learning-from-human-feedback/

Dwivedi

Y. K.

Kshetri

Hughes

Slade

E. L.

Jeyaraj

Kar

A. K.

Baabdullah

A. M.

Koohang

Raghavan

Ahuja

Albanna

Albashrawi

M. A.

Al-Busaidi

A. S.

Balakrishnan

Barlette

Basu

Bose

Brooks

Buhalis

…Wright

(2023). “So what if ChatGPT wrote it?” multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642

10.

Eke

D. O.

(2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060

11.

Ekin

(2023). Prompt engineering for ChatGPT: A quick guide to techniques, tips, and best practices. TechRxiv. https://doi.org/10.36227/techrxiv.22683919.v2

12.

Guffey

M. E.

Loewy

(2014). Business communication: Process & product (8th ed.). Cengage Learning.

13.

(2023). ChatGPT sets record for fastest-growing user base. Reuters. https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/

14.

Kong

Zhao

Chen

Qin

Sun

Zhou

(2023). Better zero-shot reasoning with role-play prompting. arXiv preprint arXiv:2308.07702. https://doi.org/10.48550/arXiv.2308.07702

15.

Korzynski

Mazurek

Krzypkowska

Kurasinski

(2023). Artificial intelligence prompt engineering as a new digital competence: Analysis of generative AI technologies such as ChatGPT. Entrepreneurial Business and Economics Review, 11(3), 25–37. https://doi.org/10.15678/EBER.2023.110302

16.

Laquintano

Schnitzler

Vee

(2023). An introduction to teaching with text generation technologies. In Vee

Laquintano

Schnitzler

(Eds.), TextGenEd: Teaching with text generation technologies. WAC Clearinghouse. https://wac.colostate.edu/repository/collections/textgened/

17.

(2023). A practical survey on zero-shot prompt design for in-context learning. In Proceedings of the 14th international conference on recent advances in natural language processing (pp. 641–647). INCOMA Ltd. Shoumen, Bulgaria.

18.

Yang

Wang

(2023). Reinforcement learning with human feedback: Learning dynamic choices via pessimism. arXiv:2305.18438.

19.

Lim

W. M.

Gunasekara

Pallant

J. L.

Pallant

J. I.

Pechenkina

(2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790

20.

Locker

Mackiewicz

Aune

J. E.

Kienzler

(2019). Business and administrative communication (12th ed.). McGraw Hill.

21.

Long

(2023). Large language model guided tree-of-thought. arXiv. https://doi.org/10.48550/arXiv.2305.08291

22.

Miller

C. R.

(1984). Genre as social action. Quarterly Journal of Speech, 70(2), 151–167. https://doi.org/10.1080/00335638409383686

23.

Miller

., Devitt

Gallagher

. (2018). Genre: Permanence and change. Rhetoric Society Quarterly, 48(3), 269–277. https://doi.org/10.1080/02773945.2018.1454194

24.

Mogavi

R. H.

Deng

Kim

J. J.

Zhou

Kwon

Y. D.

Metwally

A. H. S.

Tlili

Bassanelli

Bucchiarone

Gujar

Nacke

L. E.

Hui

(2024). ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptions. Computers in Human Behavior: Artificial Humans, 2(1), 100027. https://doi.org/10.1016/j.chbah.2023.100027

25.

Mollick

E. R.

Mollick

(2022). New modes of learning enabled by AI chatbots: Three methods and assignments. https://ssrn.com/abstract=4300783

26.

Morrison

(2017). Teaching toward the telos of critical thinking: Genre in business communication. Business and Professional Communication Quarterly, 80(4), 460–472. https://doi.org/10.1177/2329490617691967

27.

Mosqueira-Rey

Hernández-Pereira

Alonso-Ríos

Bobes-Bascarán

Fernández-Leal

(2023). Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review, 56, 3005–3054. https://doi.org/10.1007/s10462-022-10246-w

28.

Nazir

Wang

. (2023). A Comprehensive survey of ChatGPT: Advancements, applications, prospects, and challenges.. Meta-radiology, 1(2), 100022. https://doi.org/10.1016/j.metrad.2023.100022

29.

Nourani

King

Ragan

(2020). The role of domain expertise in user trust and the impact of first impressions with intelligent systems. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 8, 112–121. https://doi.org/10.1609/hcomp.v8i1.7469

30.

OpenAI. (n.d.). Prompt engineering. https://platform.openai.com/docs/guides/prompt-engineering

31.

Ouyang

Jiang

Almeida

Wainwright

Mishkin

Zhang

Agarwal

Slama

Ray

Schulman Hilton

Kelton

Miller

Simens

Askell

Welinder

Christiano

Leike

& Lowe

.. (2022). Training language models to follow instructions with human feedback. In S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho, & A Oh (Eds.), NIPS'22: Proceedings of the 36th International Conference on Neural Information Processing Systems (pp. 27730–27744). Red Hook, NY: Red Hook, NY: Curran Associates Inc. https://dl.acm.org/doi/10.5555/3600270.3602281

32.

Paltridge

(2000). Genre knowledge and teaching professional communication. IEEE Transactions on Professional Communication, 43(4), 397–401. https://doi.org/10.1109/47.888814

33.

Prompt Engineering Guide. (n.d.). Zero-shot prompting. https://www.promptingguide.ai/techniques/zeroshot

34.

Ray

P. P.

(2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003

35.

Selzer

(1983). Emphasizing rhetorical principles in business writing . In Halpern

(Ed.), Teaching business writing: Challenges, plans, pedagogy, research (pp. 3–20). American Business Communication Association.

36.

Short

C. E.

Short

J. C.

(2023). The artificially intelligent entrepreneur: ChatGPT, prompt engineering, and entrepreneurial rhetoric creation. Journal of Business Venturing Insights, 19, Article e00388. https://doi.org/10.1016/j.jbvi.2023.e00388

37.

Tardy

(2015). Bending genres, or when is a deviation an innovation? In Artemeva

Freedman

(Eds.), Genre studies around the globe: Beyond the three traditions (pp. 339–365). Inkshed.

38.

Wang

(2021). Genre change in the online context: Responding to negative online reviews and redefining an effective genre construct on Amazon.com. Journal of Business and Technical Communication, 35(3), 297–332. https://doi.org/10.1177/10506519211001113

39.

Wei

Bosma

Zhao

V. Y.

Guu

A. W.

Lester

Dai

A. M.

Q. V.

(2022, April 25-29). Finetuned language models are zero-shot learners. 2022 International Conference on Learning Representations (ICLR), Virtual.

40.

Wei

Wang

Schuurmans

Bosma

Chi

Zhou

(2022). Chain of thought prompting elicits reasoning in large language models.In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, & A. Oh (Eds.), Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS '22) (Article 1800, pp. 24824–24837). Red Hook, NY: Curran Associates Inc. https://dl.acm.org/doi/10.5555/3600270.3602070

41.

White

Hays

Sandborn

Olea

Gilbert

Elnashar

Spencer-Smith

Schmidt

D. C.

(2023). A prompt pattern catalog to enhance prompt engineering with ChatGPT (arXiv:2302.11382). arXiv. https://doi.org/10.48550/arXiv.2302.11382

42.

Wolfram

(2023). What is ChatGPT doing… and why does it work? Stephen Wolfram Writings. http://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work

43.

Zhou

Muresanu

A. I.

Han

Paster

Pitis

Chan

(2023). Large language models are human-level prompt engineers. arXiv. https://doi.org/10.48550/arXiv.2211.01910

Improving ChatGPT's Competency in Generating Effective Business Communication Messages: Integrating Rhetorical Genre Analysis into Prompting Techniques

Abstract

Keywords

Get full access to this article

References