Firms are exploiting artificial intelligence (AI) coaches to provide training to sales agents and improve their job skills. The authors present several caveats associated with such practices based on a series of randomized field experiments. Experiment 1 shows that the incremental benefit of the AI coach over human managers is heterogeneous across agents in an inverted-U shape: whereas middle-ranked agents improve their performance by the largest amount, both bottom- and top-ranked agents show limited incremental gains. This pattern is driven by a learning-based mechanism in which bottom-ranked agents encounter the most severe information overload problem with the AI versus human coach, while top-ranked agents hold the strongest aversion to the AI relative to a human coach. To alleviate the challenge faced by bottom-ranked agents, Experiment 2 redesigns the AI coach by restricting the training feedback level and shows a significant improvement in agent performance. Experiment 3 reveals that the AI–human coach assemblage outperforms either the AI or human coach alone. This assemblage can harness the hard data skills of the AI coach and soft interpersonal skills of human managers, solving both problems faced by bottom- and top-ranked agents. These findings offer novel insights into AI coaches for researchers and managers alike.
AcemogluDaronRestrepoPascual (2017), “Robots and Jobs: Evidence from US Labor Markets,” Working Paper No. 23285, National Bureau of Economic Research.
2.
AtefiYasharAhearneMichaelIIIJames G MaxhamTodd DonavanD.CarlsonBrad D. (2018), “Does Selective Salesforce Training Work?” Journal of Marketing Research, 55 (5), 722–37.
3.
BessenJames E.GoosMaartenSalomonsAnnaBergeWiljan Van den (2019), “Automatic Reaction: What Happens to Workers at Firms that Automate?” working paper, Boston University.
4.
BrynjolfssonErikHuiXiangLiuMeng (2019), “Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform,” Management Science, 65 (12), 5449–60.
BurtonJason W.SteinMari-KlaraJensenTina Blegind (2020), “A Systematic Review of Algorithm Aversion in Augmented Decision Making,” Journal of Behavioral Decision Making, 33 (2), 220–39.
7.
CasteloNoahBosMaarten W.LehmannDonald R. (2019), “Task-Dependent Algorithm Aversion,” Journal of Marketing Research, 56 (5), 809–25.
8.
ChaikenShelly (1980), “Heuristic Versus Systematic Information Processing and the Use of Source Versus Message Cues in Persuasion,” Journal of Personality and Social Psychology, 39 (5), 752.
9.
ChristiansenTimEvansKenneth R.SchlacterJohn L.WolfeWilliam G. (1996), “Training Differences Between Services and Goods Firms: Impact on Performance, Satisfaction, and Commitment,” Journal of Professional Services Marketing, 15 (1), 47–70.
10.
ChungDoug J.ParkByoung G.KimByungyeon (2019), “The Comprehensive Effects of Salesforce Management: A Dynamic Structural Analysis of Selection, Compensation, and Training,” working paper, Harvard Business School, Harvard University.
DemingDavid J. (2017), “The Growing Importance of Social Skills in the Labor Market,” Quarterly Journal of Economics, 132 (4), 1593–640.
16.
DentonDavid W.KleimanLawrence S. (2011), “Job Tenure as a Moderator of the Relationship Between Autonomy and Satisfaction,” Applied HRM Research, 6 (2), 105–14.
17.
DietvorstBerkeley J. (2015), “Algorithm Aversion,” doctoral dissertation, University of Pennsylvania.
18.
DietvorstBerkeley J.SimmonsJoseph P.MasseyCade (2015), “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err,” Journal of Experimental Psychology: General, 144 (1), 114.
19.
DietvorstBerkeley J.SimmonsJoseph P.MasseyCade (2018), “Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms if They Can (Even Slightly) Modify Them,” Management Science, 64 (3), 1155–70.
20.
DubinskyAlan J. (1996), “Some Assumptions About the Effectiveness of Sales Training,” The Journal of Personal Selling and Sales Management, 16 (3), 67–76.
21.
EastwoodJosephSnookBrentLutherKirk (2012), “What People Want from Their Professionals: Attitudes Toward Decision-Making Strategies,” Journal of Behavioral Decision Making, 25 (5), 458–68.
22.
EdmondsonAmy C.KramerRoderick M.CookKaren S. (2004), “Psychological Safety, Trust, and Learning in Organizations: A Group-Level Lens,” Trust and Distrust in Organizations: Dilemmas and Approaches, 12, 239–72.
23.
FiskeSusan T.TaylorShelley E. (1991), Social Cognition. New York: McGraw-Hill.
24.
FreyCarl BenediktOsborneMichael A. (2017), “The Future of Employment: How Susceptible Are Jobs to Computerisation?” Technological Forecasting and Social Change, 114, 254–80.
JacksonDale E. (1988), Interpersonal Communication for Technically Trained Managers: A Guide to Skills and Techniques. New York: Quorum Books.
27.
JacobyJacob (1974), “Consumer Reaction to Information Displays: Packaging and Advertising,” in Advertising and the Public Interest, DivitaS. F., ed. Chicago: American Marketing Association, 101–18.
LeungEuginaPaolacciGabrielePuntoniStefano (2018), “Man Versus Machine: Resisting Automation in Identity-Based Consumer Behavior,” Journal of Marketing Research, 55 (6), 818–31.
30.
LiJiaLiuMinghuiLiuXuan (2016), “Why Do Employees Resist Knowledge Management Systems? An Empirical Study from the Status Quo Bias and Inertia Perspectives,” Computers in Human Behavior, 65, 189–200.
31.
LoggJennifer M.MinsonJulia A.MooreDon A. (2019), “Algorithm Appreciation: People Prefer Algorithmic to Human Judgment,” Organizational Behavior and Human Decision Processes, 151, 90–103.
32.
LongoniChiaraBonezziAndreaMorewedgeCarey K. (2019), “Resistance to Medical Artificial Intelligence,” Journal of Consumer Research, 46 (4), 629–50.
33.
LuoXuemingTongSiliangFangZhengZheQu (2019), “Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases,” Marketing Science, 38 (6), 937–47.
34.
MartinWarren S.CollinsBen H. (1991), “Sales Technology Applications: Interactive Video Technology in Sales Training: A Case Study,” Journal of Personal Selling & Sales Management, 11 (3), 61–66.
MendeMartinScottMaura L.DoornJenny vanGrewalDhruvShanksIlana (2019), “Service Robots Rising: How Humanoid Robots Influence Service Experiences and Elicit Compensatory Consumer Responses,” Journal of Marketing Research, 56 (4), 535–56.
38.
NewellAllenSimonHerbert Alexander (1972), Human Problem Solving. Englewood Cliffs, NJ: Prentice Hall.
39.
ÖnkalDilekGoodwinPaulThomsonMaryGönülSinanPollockAndrew (2009), “The Relative Influence of Advice from Human Experts and Statistical Methods on Forecast Adjustments,” Journal of Behavioral Decision Making, 22 (4), 390–409.
40.
PuntoniStefanoReczekRebecca WalkerGieslerMarkusBottiSimona (2020), “Consumers and Artificial Intelligence: An Experiential Perspective,” Journal of Marketing, published electronically, https://journals.sagepub.com/doi/full/10.1177/0022242920953847.
41.
RatneshwarSrinivasanChaikenShelly (1991), “Comprehension’s Role in Persuasion: The Case of Its Moderating Effect on the Persuasive Impact of Source Cues,” Journal of Consumer Research, 18 (1), 52–62.
42.
RománSergioRuizSalvadorLuis MunueraJosé (2002), “The Effects of Sales Training on Salesforce Activity,” European Journal of Marketing, 36 (11/12), 1344–66.
SujanHarishWeitzBarton A.KumarNirmalya (1994), “Learning Orientation, Working Smart, and Effective Selling,” Journal of Marketing, 58 (3), 39–52.
51.
SunChenshuoShiZijunLiuXiaoGhoseAnindya (2019), “The Effect of Voice AI on Consumer Purchase and Search Behavior,” working paper, Stern School of Business, New York University.
TverskyAmosKahnemanDaniel (1974), “Judgment Under Uncertainty: Heuristics and Biases,” Science, 185 (4157), 1124–31.
55.
WeitzBarton A.SujanHarishSujanMita (1986), “Knowledge, Motivation, and Adaptive Behavior: A Framework for Improving Selling Effectiveness,” Journal of Marketing, 50 (4), 174–91.
56.
WilsonH. JamesDaughertyPaul R. (2018), “Collaborative Intelligence: Humans and AI Are Joining Forces,” Harvard Business Review, 96 (4), 114–23.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
