With more corporations incorporating artificial intelligence (AI) tools like social chatbots into their public communication practices, we explore optimal chatbot designs for organization-public relational outcomes. Our 2 × 2 × 2 factorial web experiment with eight custom-designed chatbots showed that communication strategies (verbal and non-verbal social cues such as emoji, memes, filler words, and response delay) had a significant impact (Cohen’s d = 0.536, standardized coefficient β = 0.438) on chatbot social conversation, the central antecedent to the investigated relational outcomes such as trust in business. Furthermore, the effect of chatbot social conversation is partially mediated by perceived organizational listening, highlighting the importance of listening and its related theories and practices in automated business communication.
AdamopoulouE.MoussiadesL. (2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, 100006. https://doi.org/10.1016/j.mlwa.2020.100006
2.
AraujoT. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
3.
AshktorabZ.JainM.LiaoQ. V.WeiszJ. D. (2019). Resilient chatbots: Repair strategy preferences for conversational breakdowns [Conference session]. Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–12). https://doi.org/10.1145/3290605.3300484
4.
BaekT. H.BakpayevM.YoonS.KimS. (2022). Smiling AI agents: How anthropomorphism and broad smiles increase charitable giving. International Journal of Advertising, 41(5), 850–867. https://doi.org/10.1080/02650487.2021.2011654
5.
BarcelosR. H.DantasD. C.SénécalS. (2018). Watch your tone: How a brand’s tone of voice on social media influences consumer responses. Journal of Interactive Marketing, 41, 60–80. https://doi.org/10.1016/j.intmar.2017.10.001
6.
BaronR. M.KennyD. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037/0022-3514.51.6.1173
7.
BélisleJ.BodurH. O. (2010). Avatars as information: Perception of consumers based on their avatars in virtual worlds. Psychology & Marketing, 27(8), 741–765. https://doi.org/10.1002/mar.20354
8.
BorauS.OtterbringT.LaporteS.Fosso WambaS. (2021). The most human bot: Female gendering increases humanness perceptions of bots and acceptance of AI. Psychology & Marketing, 38(7), 1052–1068. https://doi.org/10.1002/mar.21480
9.
BrevesP.LiebersN.MotschenbacherB.ReusL. (2021). Reducing resistance: The impact of nonfollowers’ and followers’ parasocial relationships with social media influencers on persuasive resistance and advertising effectiveness. Human Communication Research, 47(4), 418–443. https://doi.org/10.1093/hcr/hqab006
10.
CameronK. S.DuttonJ. E.QuinnR. E. (Eds.). (2003). Positive organizational scholarship: Foundations of a new discipline (1st ed). Berrett-Koehler.
11.
ChattaramanV.KwonW.-S. E.GilbertJ.LiY. (2014). Virtual shopping agents: Persona effects for older users. Journal of Research in Interactive Marketing, 8(2), 144–162. https://doi.org/10.1108/JRIM-08-2013-0054
12.
ChenZ. F.JiY. G.MenL. R. (2023). Effective social media communication for startups in China: Antecedents and outcomes of organization–public dialogic communication. New Media & Society, 25(2), 3613–3640. https://doi.org/10.1177/14614448211051984
13.
ChengY.JiangH. (2022). Customer–brand relationship in the era of artificial intelligence: Understanding the role of chatbot marketing efforts. Journal of Product & Brand Management, 31(2), 252–264. https://doi.org/10.1108/JPBM-05-2020-2907
CrijnsH.CaubergheV.HuddersL.ClaeysA.-S. (2017). How to deal with online consumer comments during a crisis? The impact of personalized organizational responses on organizational reputation. Computers in Human Behavior, 75, 619–631. https://doi.org/10.1016/j.chb.2017.05.046
17.
CroesE. A. J.AntheunisM. L. (2021). Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot. Journal of Social and Personal Relationships, 38(1), 279–300. https://doi.org/10.1177/0265407520959463
18.
DiederichS.LembckeT.-B.BrendelA. B.KolbeL. M. (2021). Understanding the impact that response failure has on how users perceive anthropomorphic conversational service agents: Insights from an online experiment. AIS Transactions on Human-Computer Interaction, 13(1), 82–103. https://doi.org/10.17705/1thci.00143
19.
FeineJ.MoranaS.MaedcheA. (2019). Designing a chatbot social cue configuration system [Conference session]. Proceedings of the 40th international conference on information systems (ICIS).
20.
FransenM. L.VerleghP. W. J.KirmaniA.SmitE. G. (2015). A typology of consumer strategies for resisting advertising, and a review of mechanisms for countering them. International Journal of Advertising, 34(1), 6–16. https://doi.org/10.1080/02650487.2014.995284
21.
FreemanG.MaloneyD. (2021). Body, avatar, and me: The presentation and perception of self in social virtual reality. Proceedings of the ACM on Human-Computer Interaction, 4, 1–27. https://doi.org/10.1145/3432938
22.
GallowayC.SwiatekL. (2018). Public relations and artificial intelligence: It’s not (just) about robots. Public Relations Review, 44(5), 734–740. https://doi.org/10.1016/j.pubrev.2018.10.008
GoE.SundarS. S. (2019). Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior, 97, 304–316. https://doi.org/10.1016/j.chb.2019.01.020
25.
GolinA. (2004). Trust or consequences: Build trust today or lose your market tomorrow. American Management Association.
GuzmanA. L.LewisS. C. (2020). Artificial intelligence and communication: A human-machine communication research agenda. New Media & Society, 22(1), 70–86. https://doi.org/10.1177/1461444819858691
28.
HaugelandI. K. F.FølstadA.TaylorC.BjørkliC. A. (2022). Understanding the user experience of customer service chatbots: An experimental study of chatbot interaction design. International Journal of Human-Computer Studies, 161, 102788. https://doi.org/10.1016/j.ijhcs.2022.102788
29.
HayesJ. L.BrittB. C.EvansW.RushS. W.ToweryN. A.AdamsonA. C. (2021). Can social media listening platforms’ artificial intelligence be trusted? Examining the accuracy of Crimson Hexagon’s (Now Brandwatch Consumer Research’s) AI-driven analyses. Journal of Advertising, 50(1), 81–91. https://doi.org/10.1080/00913367.2020.1809576
30.
HayesJ. L.GolanG.BrittB.ApplequistJ. (2020). How advertising relevance and consumer–Brand relationship strength limit disclosure effects of native ads on Twitter. International Journal of Advertising, 39(1), 131–165. https://doi.org/10.1080/02650487.2019.1596446
31.
HoA.HancockJ.MinerA. S. (2018). Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, 68(4), 712–733. https://doi.org/10.1093/joc/jqy026
32.
HoltgravesT. M.RossS. J.WeywadtC. R.HanT. L. (2007). Perceiving artificial social agents. Computers in Human Behavior, 23(5), 2163–2174. https://doi.org/10.1016/j.chb.2006.02.017
33.
HonL. C.GrunigJ. E. (1999). Guidelines for measuring relationships in public relations. Institute for Public Relations.
34.
HoppT.GangadharbatlaH. (2016). Novelty effects in augmented reality advertising environments: The influence of exposure time and self-efficacy. Journal of Current Issues & Research in Advertising, 37(2), 113–130. https://doi.org/10.1080/10641734.2016.1171179
KelleherT. (2009). Conversational voice, communicated commitment, and public relations outcomes in interactive online communication. Journal of Communication, 59(1), 172–188. https://doi.org/10.1111/j.1460-2466.2008.01410.x
38.
KimS. Y.SchmittB. H.ThalmannN. M. (2019). Eliza in the uncanny valley: Anthropomorphizing consumer robots increases their perceived warmth but decreases liking. Marketing Letters, 30(1), 1–12. https://doi.org/10.1007/s11002-019-09485-9
LanktonN.McKnightD. H.TrippJ. (2015). Technology, humanness, and trust: Rethinking trust in technology. Journal of the Association for Information Systems, 16(10), 880–918. https://doi.org/10.17705/1jais.00411
41.
LeeY.-K. (2021). Impacts of digital technostress and digital technology self-efficacy on Fintech usage intention of Chinese Gen Z consumers. Sustainability, 13(9), 5077. https://doi.org/10.3390/su13095077
LippertS. K. (2007). Investigating postadoption utilization: An examination into the role of interorganizational and technology trust. IEEE Transactions on Engineering Management, 54(3), 468–483. https://doi.org/10.1109/TEM.2007.900792
45.
LuB.FanW.ZhouM. (2016). Social presence, trust, and social commerce purchase intention: An empirical research. Computers in Human Behavior, 56, 225–237. https://doi.org/10.1016/j.chb.2015.11.057
46.
LuL.McDonaldC.KelleherT.LeeS.ChungY. J.MuellerS.VielledentM.YueC. A. (2022). Measuring consumer-perceived humanness of online organizational agents. Computers in Human Behavior, 128, 107092. https://doi.org/10.1016/j.chb.2021.107092
47.
LugerE.SellenA. (2016). “Like having a really bad PA”: The gulf between user expectation and experience of conversational agents [Conference session]. Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 5286–5297). https://doi.org/10.1145/2858036.2858288
48.
MacnamaraJ. (2016). Organizational listening: The missing essential in public communication. Peter Lang.
MacnamaraJ. (2019). Explicating listening in organization-public communication: Theory, practices, technologies. International Journal of Communication, 13, 5183–5204.
51.
MenL. R.TsaiW.-H. S. (2015). Infusing social media with humanity: Corporate character, public engagement, and relational outcomes. Public Relations Review, 41(3), 395–403. https://doi.org/10.1016/j.pubrev.2015.02.005
52.
MenL. R.TsaiW.-H. S.ChenZ. F.JiY. G. (2018). Social presence and digital dialogic communication: Engagement lessons from top social CEOs. Journal of Public Relations Research, 30(3), 83–99. https://doi.org/10.1080/1062726X.2018.1498341
53.
MenL. R.ZhouA.JinJ.ThelenP. (2023). Shaping corporate character via chatbot social conversation: Impact on organization-public relational outcomes. Public Relations Review, 49(5), 102385. https://doi.org/10.1016/j.pubrev.2023.102385
54.
MenL. R.ZhouA.Sunny TsaiW.-H. (2022). Harnessing the power of chatbot social conversation for organizational listening: The impact on perceived transparency and organization-public relationships. Journal of Public Relations Research, 34(1–2), 20–44. https://doi.org/10.1080/1062726X.2022.2068553
55.
NassC.IsbisterK.LeeE.-J. (2000). Truth is beauty: Researching embodied conversational agents. In CassellJ.SullivanJ.PrevostS.ChurchillE. (Eds.), Embodied Conversational Agents (pp. 374–402). MIT Press.
56.
NassC.MoonY.GreenN. (1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal of Applied Social Psychology, 27(10), 864–876. https://doi.org/10.1111/j.1559-1816.1997.tb00275.x
57.
NielsenR.MarroneJ. A. (2018). Humility: Our current understanding of the construct and its role in organizations. International Journal of Management Reviews, 20(4), 805–824. https://doi.org/10.1111/ijmr.12160
58.
NowakK. L.RauhC. (2005). The influence of the avatar on online perceptions of anthropomorphism, androgyny, credibility, homophily, and attraction. Journal of Computer-Mediated Communication, 11(1), 153–178. https://doi.org/10.1111/j.1083-6101.2006.tb00308.x
59.
OwensB. P.HekmanD. R. (2012). Modeling how to grow: An inductive examination of humble leader behaviors, contingencies, and outcomes. Academy of Management Journal, 55(4), 787–818. https://doi.org/10.5465/amj.2010.0441
60.
OwensB. P.JohnsonM. D.MitchellT. R. (2013). Expressed humility in organizations: Implications for performance, teams, and leadership. Organization Science, 24(5), 1517–1538. https://doi.org/10.1287/orsc.1120.0795
61.
PinaL. S.LoureiroS. M. C.RitaP.SarmentoE. M.BilroR. G.GuerreiroJ. (2019). Analysing consumer-brand engagement through appreciative listening on social network platforms. Journal of Promotion Management, 25(3), 304–313. https://doi.org/10.1080/10496491.2019.1557805
RappaportS. D. (2010). Putting listening to work: The essentials of listening. Journal of Advertising Research, 50(1), 30–41. https://doi.org/10.2501/S0021849910091154
64.
RourkeL.AndersonT.GarrisonD. R.ArcherW. (1999). Assessing social presence in asynchronous text-based computer conferencing. The Journal of Distance Education / Revue de l’ducation Distance, 14(2), 50–71.
65.
SeegerA.-M.PfeifferJ.HeinzlA. (2021). Texting with humanlike conversational agents: Designing for anthropomorphism. Journal of the Association for Information Systems, 22(4), 931–967. https://doi.org/10.17705/1jais.00685
66.
ShortJ.WilliamsE.ChristieB. (1976). The social psychology of telecommunications. Wiley.
67.
ShumH.HeX.LiD. (2018). From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering, 19(1), 10–26. https://doi.org/10.1631/FITEE.1700826
68.
SkjuveM.FølstadA.FostervoldK. I.BrandtzaegP. B. (2021). My chatbot companion—A study of human-chatbot relationships. International Journal of Human-Computer Studies, 149, 102601. https://doi.org/10.1016/j.ijhcs.2021.102601
69.
SundarS. S.BellurS.OhJ.JiaH.KimH.-S. (2016). Theoretical importance of contingency in human-computer interaction: Effects of message interactivity on user engagement. Communication Research, 43(5), 595–625. https://doi.org/10.1177/0093650214534962
70.
SweetserK. D.KelleherT. (2016). Communicated commitment and conversational voice: Abbreviated measures of communicative strategies for maintaining organization-public relationships. Journal of Public Relations Research, 28(5–6), 217–231. https://doi.org/10.1080/1062726X.2016.1237359
71.
TsaiW.-H. S.LiuY.ChuanC.-H. (2021). How chatbots’ social presence communication enhances consumer engagement: The mediating role of parasocial interaction and dialogue. Journal of Research in Interactive Marketing, 15(3), 460–482. https://doi.org/10.1108/JRIM-12-2019-0200
72.
VerhagenT.van NesJ.FeldbergF.van DolenW. (2014). Virtual customer service agents: Using social presence and personalization to shape online service encounters. Journal of Computer-Mediated Communication, 19(3), 529–545. https://doi.org/10.1111/jcc4.12066
73.
WallaceR. S. (2009). The anatomy of A.L.I.C.E. In EpsteinR.RobertsG.BeberG. (Eds.), Parsing the turing test (pp. 181–210). Springer. https://doi.org/10.1007/978-1-4020-6710-5_13
74.
WeizenbaumJ. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168
75.
WirtzJ.PattersonP. G.KunzW. H.GruberT.LuV. N.PaluchS.MartinsA. (2018). Brave new world: Service robots in the frontline. Journal of Service Management, 29(5), 907–931. https://doi.org/10.1108/JOSM-04-2018-0119
76.
XuK.LiaoT. (2020). Explicating cues: A typology for understanding emerging media technologies. Journal of Computer-Mediated Communication, 25(1), 32–43. https://doi.org/10.1093/jcmc/zmz023
77.
YounS.JinS. V. (2021). “In A.I. we trust?” The effects of parasocial interaction and technopian versus luddite ideological views on chatbot-based customer relationship management in the emerging “feeling economy.”Computers in Human Behavior, 119, 106721. https://doi.org/10.1016/j.chb.2021.106721
78.
ZaroualiB.Van den BroeckE.WalraveM.PoelsK. (2018). Predicting consumer responses to a chatbot on Facebook. Cyberpsychology, Behavior, and Social Networking, 21(8), 491–497. https://doi.org/10.1089/cyber.2017.0518
79.
ZhangJ.OhY. J.LangeP.YuZ.FukuokaY. (2020). Artificial intelligence chatbot behavior change model for designing artificial intelligence chatbots to promote physical activity and a healthy diet. Journal of Medical Internet Research, 22(9), e22845. https://doi.org/10.2196/22845
80.
ZhouA.MenL. R.TsaiW.-H. S. (2023). The power of AI-enabled chatbots as an organizational social listening tool. In PlaceK. R. (Ed.), Organizational listening for strategic communication (1st ed., pp. 63–80). Routledge. https://doi.org/10.4324/9781003273851-6
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
