The advancement of technology requires a new teaming dynamic between humans and AI/robots. As emotions and affect are critical in forming human-human relationships, they are also expected to have a significant impact on human-AI/robot team dynamics. However, little research has been conducted on the effects of emotions on this relationship and team performance. In this perspective paper, I propose turning HART (human-AI/robot teaming) into HEART (human-emotional AI/robot teaming) and demonstrate its potential effects. To this end, existing frameworks on emotion research are introduced, followed by a brief literature survey on emotional AI/robot research and its potential applications to HART. Finally, further considerations are discussed. I hope this paper can spark lively discussions on a new legitimate framework, HEART, to more accurately understand and predict human behavior and team performance in HART settings.
Get full access to this article
View all access options for this article.
References
1.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.
2.
BarnesJ.FakhrHosseiniM.JeonM.ParkC. H.HowardA. (2017, June). The influence of robot design on acceptance of social robots. In 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) (pp. 51–55).
3.
BarnesJ.FakhrHosseiniS. M.VaseyE.ParkC. H.JeonM. (2020Jan-March). Child-robot theater: Engaging elementary students in informal STEAM education using robots. IEEE Pervasive Computing, 19, 1, 22–31.
4.
BeerJ. M.LilesK. R.WuX.PakalaS. (2017). Affective human– robot interaction. In JeonM. (Ed.). Emotions and Affect in Human Factors and Human-Computer Interaction (pp. 359–381). San Diego, CA: Academic Press.
5.
ChangW. L.ŠabanovicS.HuberL. (2013, March). Use of seal-like robot PARO in sensory group therapy for older adults with dementia. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 101–102). IEEE.
6.
ChrisleyR.ZiemkeT. (2006). Embodiment. Encyclopedia of cognitive science.
7.
CoeckelberghM. (2010). Moral appearances: emotions, robots, and human morality. Ethics and Information Technology, 12(3), 235–241.
8.
DongJ.LawsonE.OlsenJ.JeonM. (2020). Female voice agents in fully autonomous vehicles are not only more likeable and comfortable, but also more competent. Proceedings of the Human Factors and Ergonomics Society’s 2020 International Annual Meeting (HFES2020), Chicago, IL, October5-9.
9.
EysselF.De RuiterL.KuchenbrandtD.BobingerS.HegelF. (2012, March). ‘If you sound like me, you must be more human’: On the interplay of robot and user features on human-robot acceptance and anthropomorphism. In 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 125–126). IEEE.
10.
ForgasJ. P. (1995). Mood and judgment: the affect infusion model (AIM). Psychological bulletin, 117(1), 39–66.
11.
GoetzJ.KieslerS.PowersA. (2003, November). Matching robot appearance and behavior to tasks to improve human-robot cooperation. In The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003. (pp. 55–60).
12.
GrossJ. J. (2015). Emotion regulation: Current status and future prospects. Psychological inquiry, 26(1), 1–26.
13.
HashimotoT.KobayashiH.PolishukA.VernerI. (2013, March). Elementary science lesson delivered by robot. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 133–134). IEEE.
14.
HelanderM. G. (2002, September). Hedonomics-affective human factors design. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 46, No. 12, pp. 978982). Sage CA: Los Angeles, CA: SAGE Publications.
15.
HoqueM.CourgeonM.MartinJ. C.MutluB.PicardR. W. (2013, September). Mach: My automated conversation coach. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing (pp. 697–706).
16.
HosseiniS. M. F.LettingaD.VaseyE.ZhengZ.JeonM.ParkC. H.HowardA. M. (2017). Both “look and feel” matter: Essential factors for robotic companionship. In 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 150–155). IEEE.
17.
JavedH.BurnsR.JeonM.HowardA. M.ParkC. H. (2019). A Robotic Framework to Facilitate Sensory Experiences for Children with Autism Spectrum Disorder: A Preliminary Study. ACM Transactions on Human-Robot Interaction (THRI), 9(1), 1–26.
18.
JeonM. (2017). Emotions and affect in human factors and human– computer interaction: Taxonomy, theories, approaches, and methods. In Emotions and affect in human factors and human-computer interaction (pp. 3–26). Academic Press.
19.
JeonM.RayanI. A. (2011, July). The effect of physical embodiment of an animal robot on affective prosody recognition. In International Conference on Human-Computer Interaction (pp. 523–532). Springer, Berlin, Heidelberg.
20.
JeongS.LoganD. E.GoodwinM. S.GracaS.O’ConnellB.GoodenoughH.. . . PlummerL. (2015). A social robot to mitigate stress, anxiety, and pain in hospital pediatric care. In Proceedings of the Tenth Annual International Conference on Human-Robot Interaction Extended Abstracts (pp. 103–104).
21.
KammersgaardJ. (1988). Four different perspectives on human– computer interaction. International Journal of Man-Machine Studies, 28(4), 343–362.
22.
KishiT.KojimaT.EndoN.DestepheM.OtaniT.JamoneL.. . . TakanishiA. (2013, May). Impression survey of the emotion expression humanoid robot with mental model based dynamic emotions. In 2013 IEEE International Conference on Robotics and Automation (pp. 1663–1668). IEEE.
23.
KuehnJ.HaddadinS. (2016). An artificial robot nervous system to teach robots how to feel pain and reflexively react to potentially damaging contacts. IEEE Robotics and Automation Letters, 2(1), 72–79.
24.
LiuC.ConnK.SarkarN.StoneW. (2008). Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Transactions on Robotics, 24(4), 883–896.
25.
LohseM.HegelF.SwadzbaA.RohlfingK.WachsmuthS.WredeB. (2007, February). What can I do for you? Appearance and application of robots. In Proceedings of AISB (Vol. 7, pp. 121–126).
26.
LoweR.BarakovaE.BillingE.BroekensJ. (2016). Grounding emotions in robots–An introduction to the special issue, Adaptive Behavior, 24(5), 263–266.
27.
McGinnC.TorreI. (2019, March). Can you tell the robot by the voice? An exploratory study on the role of voice in the perception of robots. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 211–221). IEEE.
28.
NagamachiM. (1995). Kansei engineering: a new ergonomic consumer-oriented technology for product development. International Journal of industrial ergonomics, 15(1), 3–11.
29.
NassC.SteuerJ.TauberE. R. (1994, April). Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 72–78).
30.
NormanD. A. (2004). Emotional design: Why we love (or hate) everyday things. Basic Civitas Books.
31.
NunezE.MatsudaS.HirokawaM.SuzukiK. (2015, October). Humanoid robot assisted training for facial expressions recognition based on affective feedback. In International Conference on Social Robotics (pp. 492–501). Springer, Cham.
32.
PicardR. W. (1997). Affective Computing. M.I.T Media Laboratory Perceptual Computing Section Technical Report No. 321.
33.
PrakashA.RogersW. A. (2015). Why some humanoid faces are perceived more positively than others: effects of human-likeness and task. International journal of social robotics, 7(2), 309–331.
34.
Saint-AiméS.Le-PevedicB.DuhautD.ShibataT. (2007, August). EmotiRob: companion robot project. In RO-MAN 2007The 16th IEEE International Symposium on Robot and Human Interactive Communication (pp. 919–924). IEEE.
35.
SeoS. H.GeiskkovitchD.NakaneM.KingC.YoungJ. E. (2015, March). Poor thing! Would you feel sorry for a simulated robot? A comparison of empathy toward a physical and a simulated robot. In 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 125–132).
36.
SeyamaJ. I.NagayamaR. S. (2007). The uncanny valley: Effect of realism on the impression of artificial human faces. Presence: Teleoperators and virtual environments, 16(4), 337–351.
37.
SharmaM.HildebrandtD.NewmanG.YoungJ. E.EskiciogluR. (2013, March). Communicating affect via flight path Exploring use of the Laban Effort System for designing affective locomotion paths. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI)(pp. 293–300).
38.
SmarrC. A.MitznerT. L.BeerJ. M.PrakashA.ChenT. L.KempC. C.RogersW. A. (2014). Domestic robots for older adults: attitudes, preferences, and potential. International journal of social robotics, 6(2), 229–247.
39.
SterlingG. (2020). Roughly 1 in 4 U.S. adults now owns a smart speaker, according to new report. Retrieved in March2021.
40.
ThanapattheerakulT.MaoK.AmorantoJ.ChanJ. H. (2018, December). Emotion in a century: A review of emotion recognition. In Proceedings of the 10th International Conference on Advances in Information Technology (pp. 1–8).
41.
TortaE.WernerF.JohnsonD. O.JuolaJ. F.CuijpersR. H.BazzaniM.. . . BregmanJ. (2014). Evaluation of a small socially-assistive humanoid robot in intelligent homes for the care of the elderly. Journal of Intelligent & Robotic Systems, 76(1), 57–71.
42.
WaltersM. L.SyrdalD. S.DautenhahnK.Te BoekhorstR.KoayK. L. (2008). Avoiding the uncanny valley: robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion. Autonomous Robots, 24(2), 159–178.
43.
WaltersM. L.SyrdalD. S.KoayK. L.DautenhahnK.Te BoekhorstR. (2008). Human approach distances to a mechanical-looking robot with different robot voice styles. In RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication (pp. 707–712).
44.
WhitleyJ. B. E. (2002). Principles of research in behavioral science (2nd Ed.). Boston: McGraw Hill.
