Optimizing AI Social Chatbots for Relational Outcomes: The Effects of Profile Design,Communication Strategies,and Message Framing

Abstract

With more corporations incorporating artificial intelligence (AI) tools like social chatbots into their public communication practices, we explore optimal chatbot designs for organization-public relational outcomes. Our 2 × 2 × 2 factorial web experiment with eight custom-designed chatbots showed that communication strategies (verbal and non-verbal social cues such as emoji, memes, filler words, and response delay) had a significant impact (Cohen’s d = 0.536, standardized coefficient β = 0.438) on chatbot social conversation, the central antecedent to the investigated relational outcomes such as trust in business. Furthermore, the effect of chatbot social conversation is partially mediated by perceived organizational listening, highlighting the importance of listening and its related theories and practices in automated business communication.

Keywords

chatbots organizational listening organization-public relationships automated communication web experiment artificial intelligence

Get full access to this article

View all access options for this article.

References

Adamopoulou

Moussiades

(2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, 100006. https://doi.org/10.1016/j.mlwa.2020.100006

Araujo

(2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051

Ashktorab

Jain

Liao

Q. V.

Weisz

J. D.

(2019). Resilient chatbots: Repair strategy preferences for conversational breakdowns [Conference session]. Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–12). https://doi.org/10.1145/3290605.3300484

Baek

T. H.

Bakpayev

Yoon

Kim

(2022). Smiling AI agents: How anthropomorphism and broad smiles increase charitable giving. International Journal of Advertising, 41(5), 850–867. https://doi.org/10.1080/02650487.2021.2011654

Barcelos

R. H.

Dantas

D. C.

Sénécal

(2018). Watch your tone: How a brand’s tone of voice on social media influences consumer responses. Journal of Interactive Marketing, 41, 60–80. https://doi.org/10.1016/j.intmar.2017.10.001

Baron

R. M.

Kenny

D. A.

(1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037/0022-3514.51.6.1173

Bélisle

Bodur

H. O.

(2010). Avatars as information: Perception of consumers based on their avatars in virtual worlds. Psychology & Marketing, 27(8), 741–765. https://doi.org/10.1002/mar.20354

Borau

Otterbring

Laporte

Fosso Wamba

(2021). The most human bot: Female gendering increases humanness perceptions of bots and acceptance of AI. Psychology & Marketing, 38(7), 1052–1068. https://doi.org/10.1002/mar.21480

Breves

Liebers

Motschenbacher

Reus

(2021). Reducing resistance: The impact of nonfollowers’ and followers’ parasocial relationships with social media influencers on persuasive resistance and advertising effectiveness. Human Communication Research, 47(4), 418–443. https://doi.org/10.1093/hcr/hqab006

10.

Cameron

K. S.

Dutton

J. E.

Quinn

R. E.

(Eds.). (2003). Positive organizational scholarship: Foundations of a new discipline (1st ed). Berrett-Koehler.

11.

Chattaraman

Kwon

W.-S. E.

Gilbert

(2014). Virtual shopping agents: Persona effects for older users. Journal of Research in Interactive Marketing, 8(2), 144–162. https://doi.org/10.1108/JRIM-08-2013-0054

12.

Chen

Z. F.

Y. G.

Men

L. R.

(2023). Effective social media communication for startups in China: Antecedents and outcomes of organization–public dialogic communication. New Media & Society, 25(2), 3613–3640. https://doi.org/10.1177/14614448211051984

13.

Cheng

Jiang

(2022). Customer–brand relationship in the era of artificial intelligence: Understanding the role of chatbot marketing efforts. Journal of Product & Brand Management, 31(2), 252–264. https://doi.org/10.1108/JPBM-05-2020-2907

14.

Colby

K. M.

Weber

Hilf

F. D.

(1971). Artificial paranoia. Artificial Intelligence, 2(1), 1–25. https://doi.org/10.1016/0004-3702(71)90002-6

15.

Crawford

(2009). Following you: Disciplines of listening in social media. Continuum, 23(4), 525–535. https://doi.org/10.1080/10304310903003270

16.

Crijns

Cauberghe

Hudders

Claeys

A.-S.

(2017). How to deal with online consumer comments during a crisis? The impact of personalized organizational responses on organizational reputation. Computers in Human Behavior, 75, 619–631. https://doi.org/10.1016/j.chb.2017.05.046

17.

Croes

E. A. J.

Antheunis

M. L.

(2021). Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot. Journal of Social and Personal Relationships, 38(1), 279–300. https://doi.org/10.1177/0265407520959463

18.

Diederich

Lembcke

T.-B.

Brendel

A. B.

Kolbe

L. M.

(2021). Understanding the impact that response failure has on how users perceive anthropomorphic conversational service agents: Insights from an online experiment. AIS Transactions on Human-Computer Interaction, 13(1), 82–103. https://doi.org/10.17705/1thci.00143

19.

Feine

Morana

Maedche

(2019). Designing a chatbot social cue configuration system [Conference session]. Proceedings of the 40th international conference on information systems (ICIS).

20.

Fransen

M. L.

Verlegh

P. W. J.

Kirmani

Smit

E. G.

(2015). A typology of consumer strategies for resisting advertising, and a review of mechanisms for countering them. International Journal of Advertising, 34(1), 6–16. https://doi.org/10.1080/02650487.2014.995284

21.

Freeman

Maloney

(2021). Body, avatar, and me: The presentation and perception of self in social virtual reality. Proceedings of the ACM on Human-Computer Interaction, 4, 1–27. https://doi.org/10.1145/3432938

22.

Galloway

Swiatek

(2018). Public relations and artificial intelligence: It’s not (just) about robots. Public Relations Review, 44(5), 734–740. https://doi.org/10.1016/j.pubrev.2018.10.008

23.

Gilbert

(2022, April 11). How AI will transform the advertising industry. Ad News. https://www.adnews.com.au/news/how-ai-will-transform-the-advertising-industry

24.

Sundar

S. S.

(2019). Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior, 97, 304–316. https://doi.org/10.1016/j.chb.2019.01.020

25.

Golin

(2004). Trust or consequences: Build trust today or lose your market tomorrow. American Management Association.

26.

Guha

(2018, December 17). Chatbot personalization: Revolutionizing customer experience. Medium. https://medium.datadriveninvestor.com/chatbot-personalization-revolutionizing-customer-experience-dcd130696f0a

27.

Guzman

A. L.

Lewis

S. C.

(2020). Artificial intelligence and communication: A human-machine communication research agenda. New Media & Society, 22(1), 70–86. https://doi.org/10.1177/1461444819858691

28.

Haugeland

I. K. F.

Følstad

Taylor

Bjørkli

C. A.

(2022). Understanding the user experience of customer service chatbots: An experimental study of chatbot interaction design. International Journal of Human-Computer Studies, 161, 102788. https://doi.org/10.1016/j.ijhcs.2022.102788

29.

Hayes

J. L.

Britt

B. C.

Evans

Rush

S. W.

Towery

N. A.

Adamson

A. C.

(2021). Can social media listening platforms’ artificial intelligence be trusted? Examining the accuracy of Crimson Hexagon’s (Now Brandwatch Consumer Research’s) AI-driven analyses. Journal of Advertising, 50(1), 81–91. https://doi.org/10.1080/00913367.2020.1809576

30.

Hayes

J. L.

Golan

Britt

Applequist

(2020). How advertising relevance and consumer–Brand relationship strength limit disclosure effects of native ads on Twitter. International Journal of Advertising, 39(1), 131–165. https://doi.org/10.1080/02650487.2019.1596446

31.

Hancock

Miner

A. S.

(2018). Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, 68(4), 712–733. https://doi.org/10.1093/joc/jqy026

32.

Holtgraves

T. M.

Ross

S. J.

Weywadt

C. R.

Han

T. L.

(2007). Perceiving artificial social agents. Computers in Human Behavior, 23(5), 2163–2174. https://doi.org/10.1016/j.chb.2006.02.017

33.

Hon

L. C.

Grunig

J. E.

(1999). Guidelines for measuring relationships in public relations. Institute for Public Relations.

34.

Hopp

Gangadharbatla

(2016). Novelty effects in augmented reality advertising environments: The influence of exposure time and self-efficacy. Journal of Current Issues & Research in Advertising, 37(2), 113–130. https://doi.org/10.1080/10641734.2016.1171179

35.

Tong

(2023, November 6). OpenAI unveils personalized AI apps as it seeks to expand its ChatGPT consumer business. Reuters. https://www.reuters.com/technology/openai-enables-customized-gpt-bots-offers-cheaper-more-powerful-models-2023-11-06/

36.

Johnson

(2017, April 18). Facebook messenger hits 100,000 bots. Venture Beat. https://venturebeat.com/ai/facebook-messenger-hits-100000-bots/

37.

Kelleher

(2009). Conversational voice, communicated commitment, and public relations outcomes in interactive online communication. Journal of Communication, 59(1), 172–188. https://doi.org/10.1111/j.1460-2466.2008.01410.x

38.

Kim

S. Y.

Schmitt

B. H.

Thalmann

N. M.

(2019). Eliza in the uncanny valley: Anthropomorphizing consumer robots increases their perceived warmth but decreases liking. Marketing Letters, 30(1), 1–12. https://doi.org/10.1007/s11002-019-09485-9

39.

Kirk

(2022, February 9). Personalization and context: AI’s surging role in advertising. Forbes. https://www.forbes.com/sites/forbesbusinessdevelopmentcouncil/2022/02/09/personalization-and-context-ais-surging-role-in-advertising/

40.

Lankton

McKnight

D. H.

Tripp

(2015). Technology, humanness, and trust: Rethinking trust in technology. Journal of the Association for Information Systems, 16(10), 880–918. https://doi.org/10.17705/1jais.00411

41.

Lee

Y.-K.

(2021). Impacts of digital technostress and digital technology self-efficacy on Fintech usage intention of Chinese Gen Z consumers. Sustainability, 13(9), 5077. https://doi.org/10.3390/su13095077

42.

Suh

(2022). Anthropomorphism in AI-enabled technology: A literature review. Electronic Markets, 32(4), 2245–2275. https://doi.org/10.1007/s12525-022-00591-7

43.

(2023). The dark side of ChatGPT: Legal and ethical challenges from stochastic parrots and hallucination. https://doi.org/10.48550/ARXIV.2304.14347

44.

Lippert

S. K.

(2007). Investigating postadoption utilization: An examination into the role of interorganizational and technology trust. IEEE Transactions on Engineering Management, 54(3), 468–483. https://doi.org/10.1109/TEM.2007.900792

45.

Fan

Zhou

(2016). Social presence, trust, and social commerce purchase intention: An empirical research. Computers in Human Behavior, 56, 225–237. https://doi.org/10.1016/j.chb.2015.11.057

46.

McDonald

Kelleher

Lee

Chung

Y. J.

Mueller

Vielledent

Yue

C. A.

(2022). Measuring consumer-perceived humanness of online organizational agents. Computers in Human Behavior, 128, 107092. https://doi.org/10.1016/j.chb.2021.107092

47.

Luger

Sellen

(2016). “Like having a really bad PA”: The gulf between user expectation and experience of conversational agents [Conference session]. Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 5286–5297). https://doi.org/10.1145/2858036.2858288

48.

Macnamara

(2016). Organizational listening: The missing essential in public communication. Peter Lang.

49.

Macnamara

(2018). Toward a theory and practice of organizational listening. International Journal of Listening, 32(1), 1–23. https://doi.org/10.1080/10904018.2017.1375076

50.

Macnamara

(2019). Explicating listening in organization-public communication: Theory, practices, technologies. International Journal of Communication, 13, 5183–5204.

51.

Men

L. R.

Tsai

W.-H. S.

(2015). Infusing social media with humanity: Corporate character, public engagement, and relational outcomes. Public Relations Review, 41(3), 395–403. https://doi.org/10.1016/j.pubrev.2015.02.005

52.

Men

L. R.

Tsai

W.-H. S.

Chen

Z. F.

Y. G.

(2018). Social presence and digital dialogic communication: Engagement lessons from top social CEOs. Journal of Public Relations Research, 30(3), 83–99. https://doi.org/10.1080/1062726X.2018.1498341

53.

Men

L. R.

Zhou

Jin

Thelen

(2023). Shaping corporate character via chatbot social conversation: Impact on organization-public relational outcomes. Public Relations Review, 49(5), 102385. https://doi.org/10.1016/j.pubrev.2023.102385

54.

Men

L. R.

Zhou

Sunny Tsai

W.-H.

(2022). Harnessing the power of chatbot social conversation for organizational listening: The impact on perceived transparency and organization-public relationships. Journal of Public Relations Research, 34(1–2), 20–44. https://doi.org/10.1080/1062726X.2022.2068553

55.

Nass

Isbister

Lee

E.-J.

(2000). Truth is beauty: Researching embodied conversational agents. In Cassell

Sullivan

Prevost

Churchill

(Eds.), Embodied Conversational Agents (pp. 374–402). MIT Press.

56.

Nass

Moon

Green

(1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal of Applied Social Psychology, 27(10), 864–876. https://doi.org/10.1111/j.1559-1816.1997.tb00275.x

57.

Nielsen

Marrone

J. A.

(2018). Humility: Our current understanding of the construct and its role in organizations. International Journal of Management Reviews, 20(4), 805–824. https://doi.org/10.1111/ijmr.12160

58.

Nowak

K. L.

Rauh

(2005). The influence of the avatar on online perceptions of anthropomorphism, androgyny, credibility, homophily, and attraction. Journal of Computer-Mediated Communication, 11(1), 153–178. https://doi.org/10.1111/j.1083-6101.2006.tb00308.x

59.

Owens

B. P.

Hekman

D. R.

(2012). Modeling how to grow: An inductive examination of humble leader behaviors, contingencies, and outcomes. Academy of Management Journal, 55(4), 787–818. https://doi.org/10.5465/amj.2010.0441

60.

Owens

B. P.

Johnson

M. D.

Mitchell

T. R.

(2013). Expressed humility in organizations: Implications for performance, teams, and leadership. Organization Science, 24(5), 1517–1538. https://doi.org/10.1287/orsc.1120.0795

61.

Pina

L. S.

Loureiro

S. M. C.

Rita

Sarmento

E. M.

Bilro

R. G.

Guerreiro

(2019). Analysing consumer-brand engagement through appreciative listening on social network platforms. Journal of Promotion Management, 25(3), 304–313. https://doi.org/10.1080/10496491.2019.1557805

62.

PRovoke Media. (2023). AI in the communications industry 2023. https://www.provokemedia.com/docs/default-source/default-document-library/sp-pr-ai-report_final.pdf

63.

Rappaport

S. D.

(2010). Putting listening to work: The essentials of listening. Journal of Advertising Research, 50(1), 30–41. https://doi.org/10.2501/S0021849910091154

64.

Rourke

Anderson

Garrison

D. R.

Archer

(1999). Assessing social presence in asynchronous text-based computer conferencing. The Journal of Distance Education / Revue de l’ducation Distance, 14(2), 50–71.

65.

Seeger

A.-M.

Pfeiffer

Heinzl

(2021). Texting with humanlike conversational agents: Designing for anthropomorphism. Journal of the Association for Information Systems, 22(4), 931–967. https://doi.org/10.17705/1jais.00685

66.

Short

Williams

Christie

(1976). The social psychology of telecommunications. Wiley.

67.

Shum

(2018). From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering, 19(1), 10–26. https://doi.org/10.1631/FITEE.1700826

68.

Skjuve

Følstad

Fostervold

K. I.

Brandtzaeg

P. B.

(2021). My chatbot companion—A study of human-chatbot relationships. International Journal of Human-Computer Studies, 149, 102601. https://doi.org/10.1016/j.ijhcs.2021.102601

69.

Sundar

S. S.

Bellur

Jia

Kim

H.-S.

(2016). Theoretical importance of contingency in human-computer interaction: Effects of message interactivity on user engagement. Communication Research, 43(5), 595–625. https://doi.org/10.1177/0093650214534962

70.

Sweetser

K. D.

Kelleher

(2016). Communicated commitment and conversational voice: Abbreviated measures of communicative strategies for maintaining organization-public relationships. Journal of Public Relations Research, 28(5–6), 217–231. https://doi.org/10.1080/1062726X.2016.1237359

71.

Tsai

W.-H. S.

Liu

Chuan

C.-H.

(2021). How chatbots’ social presence communication enhances consumer engagement: The mediating role of parasocial interaction and dialogue. Journal of Research in Interactive Marketing, 15(3), 460–482. https://doi.org/10.1108/JRIM-12-2019-0200

72.

Verhagen

van Nes

Feldberg

van Dolen

(2014). Virtual customer service agents: Using social presence and personalization to shape online service encounters. Journal of Computer-Mediated Communication, 19(3), 529–545. https://doi.org/10.1111/jcc4.12066

73.

Wallace

R. S.

(2009). The anatomy of A.L.I.C.E. In Epstein

Roberts

Beber

(Eds.), Parsing the turing test (pp. 181–210). Springer. https://doi.org/10.1007/978-1-4020-6710-5_13

74.

Weizenbaum

(1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168

75.

Wirtz

Patterson

P. G.

Kunz

W. H.

Gruber

V. N.

Paluch

Martins

(2018). Brave new world: Service robots in the frontline. Journal of Service Management, 29(5), 907–931. https://doi.org/10.1108/JOSM-04-2018-0119

76.

Liao

(2020). Explicating cues: A typology for understanding emerging media technologies. Journal of Computer-Mediated Communication, 25(1), 32–43. https://doi.org/10.1093/jcmc/zmz023

77.

Youn

Jin

S. V.

(2021). “In A.I. we trust?” The effects of parasocial interaction and technopian versus luddite ideological views on chatbot-based customer relationship management in the emerging “feeling economy.” Computers in Human Behavior, 119, 106721. https://doi.org/10.1016/j.chb.2021.106721

78.

Zarouali

Van den Broeck

Walrave

Poels

(2018). Predicting consumer responses to a chatbot on Facebook. Cyberpsychology, Behavior, and Social Networking, 21(8), 491–497. https://doi.org/10.1089/cyber.2017.0518

79.

Zhang

Y. J.

Lange

Fukuoka

(2020). Artificial intelligence chatbot behavior change model for designing artificial intelligence chatbots to promote physical activity and a healthy diet. Journal of Medical Internet Research, 22(9), e22845. https://doi.org/10.2196/22845

80.

Zhou

Men

L. R.

Tsai

W.-H. S.

(2023). The power of AI-enabled chatbots as an organizational social listening tool. In Place

K. R.

(Ed.), Organizational listening for strategic communication (1st ed., pp. 63–80). Routledge. https://doi.org/10.4324/9781003273851-6

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB