This study investigates the effect of naturalistic visual cues on human avoidance behavior for a potential use in telerobotic user interfaces incorporating mixed-reality environments (e.g., augmented reality).
Background
Telerobotic systems used in hazardous environments require interfaces that draw operators’ attention to potential dangers. Existing means of hazard notification can often distract or induce stress in operators. In the design and implementation of such interfaces, visual semiotics plays a critical role in creating more effective interfaces. Naturalistic visual cues such as Aposematism or Kindchenschema have proven effective to communicate danger or caution in nature, but the application of these cues in visual systems have yet to be thoroughly investigated.
Method
A study was conducted where 40 volunteering participants were asked to control a remote vehicle in a simulated environment. The environment contained a set of neutral and visually augmented obstacles that were designed to provoke avoidance behavior.
Results
The use of visual cues triggered greater avoidance behaviors in participants compared to neutral obstacles. The distance of avoidance was correlated with the type of cue present, with obstacles augmented by Aposematism (Cue A) having a greater participant–obstacle distance than Kindchenschema (Cue K).
Conclusions
This study shows the potential for the incorporation of naturalistic visual cues as a means to designate warning or caution in telerobotic environments.
Applications
The findings can offer practical guidelines for the design of visual cues in telerobotic interfaces. The further incorporation of such cues may reduce operator stress and the amount of human errors in telerobotic operations.
AzumaR.BaillotY.BehringerR.FeinerS.JulierS.MacIntyreB. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21, 34–47.doi:10.1109/38.963459
2.
BackhausN.RosenP. H.ScheidigA.GrossH. M.WischniewskiS. (2018, September). “Somebody help me, please?!” Interaction design framework for needy mobile service robots. In2018 IEEE workshop on advanced robotics and its social impacts (ARSO) (pp.54–61). IEEE.
3.
BaldwinC. L.RunkleR. S. (1967). Biohazards symbol: Development of a biological hazards warning signal. Science, 158, 264–265.doi:10.1126/science.158.3798.264
BellB.HöllererT.FeinerS. (2002). An annotated situation-awareness aid for augmented reality [Paper presentation]. Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology - UIST ’02, Paris, France, 213–223.
6.
BlissJ. P.ActonS. A. (2003). Alarm mistrust in automobiles: How collision alarm reliability affects driving. Applied Ergonomics, 34, 499–509.doi:10.1016/j.apergo.2003.07.003
7.
BlissJ. P.DunnM. C. (2000). Behavioural implications of alarm mistrust as a function of task workload. Ergonomics, 43, 1283–1300.doi:10.1080/001401300421743
8.
CasperJ.MurphyR. R. (2003). Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 33, 367–385.doi:10.1109/TSMCB.2003.811794
9.
ChenJ. Y. C.HaasE. C.BarnesM. J. (2007). Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man and Cybernetics, Part C, 37, 1231–1245.doi:10.1109/TSMCC.2007.905819
10.
ChintamaniK.CaoA.EllisR. D.PandyaA. K. (2010). Improved telemanipulator navigation during display-control misalignments using augmented reality cues. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 40, 29–39.doi:10.1109/TSMCA.2009.2030166
DengZ.JagersandM. (2003). Predictive display system for tele-manipulation using image-based modeling and rendering. Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), 3, 2797–2802.
13.
ElliotA. J.SheldonK. M. (1998). Avoidance personal goals and the personality–illness relationship. Journal of Personality and Social Psychology, 75, 1282–1299.doi:10.1037/0022-3514.75.5.1282
14.
FlavellJ. C.BarrettB. T.BuckleyJ. G.HarrisJ. M.ScallyA. J.BeebeN. B.CruickshankA. G.BennettS. J. (2018). Temporal estimation in prediction motion tasks is biased by a moving destination. Journal of Vision, 18, 1–11.doi:10.1167/18.2.5
15.
FongT.Rochlis ZumbadoJ.CurrieN.MishkinA.AkinD. L. (2013). Space telerobotics. Reviews of Human Factors and Ergonomics, 9, 6–56.doi:10.1177/1557234X13510679
16.
GongR.WangQ.HaiY.ShaoX. (2017). Investigation on factors to influence color emotion and color preference responses. Optik, 136, 71–78.doi:10.1016/j.ijleo.2017.02.026
17.
GreenS. A.BillinghurstM.ChenX.ChaseJ. G. (2008). Human-robot collaboration: A literature review and augmented reality approach in design. International Journal of Advanced Robotic Systems, 5, 1–18.doi:10.5772/5664
18.
GuoQ.MatsushimaA. (2014). 3D tsunami hazard map and models of safe shelters. Journal of Natural Disaster Science, 35, 35–42.doi:10.2328/jnds.35.35
19.
HallE. T. (1966). The hidden dimension. Doubleday.
20.
HannafordB.RosenJ.FriedmanD. W.KingH.RoanP.ChengL.MaJ.KosariS. N.GlozmanD.WhiteL. (2013). Raven-II: An open platform for surgical robotics research. IEEE Transactions on Biomedical Engineering, 60, 954–959.doi:10.1109/TBME.2012.2228858
21.
HashemN. R.HaasD. A.PryorM. W. (2018). Mev photon imaging with robotic sample positioning at a research reactor. Journal of Radioanalytical and Nuclear Chemistry, 318, 599–604.doi:10.1007/s10967-018-6151-3
22.
HopkinV. D. (1995). Situational awareness in air traffic control. Embry-Riddle Aeronautical University Press.
23.
HorswillM. S.McKennaF. P. (2004). Drivers’ hazard perception ability: Situation awareness on the road. InBanburyS.TremblayS. (Eds.), A Cognitive Approach to Situation Awareness (pp.155–175). Ashgate.
24.
ISO 3864-1:2011. (2011). Graphical symbols -- Safety colours and safety signs -- Part 1: Design principles for safety signs and safety markings. https://www.iso.org/standard/51021.html
25.
KarkunP.ChowdhuryA.DharD. (2018). Effect of baby-like product personality on visually perceived pleasure: A study on coffeemakers. InRayG. G.IqbalR.GanguliA. K.KhanzodeV. (Eds.), Ergonomics in caring for people (pp.273–279). Springer. https://doi.org/
26.
KheddarA.ChellaliR.CoiffetP. (2002). Virtual environment–assisted teleoperation. InHaleK. S.StanneyK. M. (Eds.), Human factors and ergonomics. Handbook of virtual environments: Design, implementation, and applications (pp.959–998). CRC Press.
KollingT.BaischS.SchallA.SelicS.RühlS.KimZ.RossbergH. (2016). “What is emotional about emotional robotics?”InTettegahS. Y.GarciaY. E. (Eds.), Emotions, technology, and health (pp.85–103). Academic Press. https://doi.org/10.1016/B978-0-12-801737-1.00005-6
29.
KrotkovE.HackettD.JackelL.PerschbacherM.PippineJ.StraussJ.OrlowskiC. (2018). The DARPA robotics challenge finals: Results and perspectives. InSpenkoM.BuergerS.IagnemmaK. (Eds.), Springer tracts in advanced robotics (Vol. 121, pp.1–26). https://doi.org/10.1007/978-3-319-74666-1_1
30.
LabonteD.BoissyP.MichaudF. (2010). Comparative analysis of 3-D robot teleoperation interfaces with novice users. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 40, 1331–1342.doi:10.1109/TSMCB.2009.2038357
31.
LeshinL. AMahaffyP. RWebsterC. RCabaneMCollPConradP. GArcherP. DAtreyaS. KBrunnerA. EBuchAEigenbrodeJ. LFleschG. JFranzH. BFreissinetCGlavinD. PMcAdamA. CMillerK. EMingD. WMorrisR. VMSL Science Team. (2013). Volatile, isotope, and organic analysis of Martian fines with the Mars curiosity rover. Science, 341, 1238937doi:10.1126/science.1238937
32.
Lev-YadunS. (2009). Aposematic (warning) coloration in plants. InBaluškaF. (Ed.), Plant-environment interactions. Signaling and communication in plants (pp.167–202). Springer. https://doi.org/10.1007/978-3-540-89230-4_10
33.
LinQ.KuoC. (1997). Virtual tele-operation of underwater robots. Proceedings of International Conference on Robotics and Automation, 2, 1022–1027.
34.
LivingstonM. A.RosenblumL. J.JulierS. J.BrownD.BaillotY.SwanJ. E.GabbardJ. L.HixD. (2002). An augmented reality system for military operations in urban terrain. https://apps.dtic.mil/docs/citations/ADA499032
35.
MackenzieA. K.HarrisJ. M. (2014). Characterizing visual attention during driving and non-driving hazard perception tasks in a simulated environment. Proceedings of the Symposium on Eye Tracking Research and Applications - ETRA, 14, 127–130.
36.
MackenzieA. K.HarrisJ. M. (2017). A link between attentional function, effective eye movements, and driving ability. Journal of Experimental Psychology: Human Perception and Performance, 43, 381–394.doi:10.1037/xhp0000297
37.
MappesJ.MarplesN.EndlerJ. (2005). The complex business of survival by aposematism. Trends in Ecology & Evolution, 20, 598–603.doi:10.1016/j.tree.2005.07.011
38.
MarionP.FallonM.DeitsR.ValenzuelaA.D’ArpinoP.IzattC.Tedrake,R. G. (2017). Director: A user interface designed for robot operation with shared autonomy. Journal of Field Robotics, 34, 262–280.doi:10.1002/rob.21681
39.
MieslerL.LederH.HerrmannA. (2011). Isn’t it cute: An evolutionary perspective of baby-schema effects in visual product designs. International Journal of Design, 5, 17–30.
40.
NagataniK.KiribayashiS.OkadaY.OtakeK.YoshidaK.TadokoroS.NishimuraT.YoshidaT.KoyanagiE.FukushimaM.KawatsumaS. (2013). Emergency response to the nuclear accident at the Fukushima Daiichi nuclear power plants using mobile rescue robots. Journal of Field Robotics, 30, 44–63.doi:10.1002/rob.21439
41.
NgaiS. (2005). The cuteness of the avant‐garde. Critical Inquiry, 31, 811–847.doi:10.1086/444516
42.
NöthW. (1994). Opposition at the roots of semiosis. InNöthW. (Ed.), Origins of semiosis: Sign evolution in nature and culture (pp.37–60). De Gruyter Mouton. https://doi.org/10.1515/9783110877502.37
43.
PhanM. T.ThouveninI.FremontV. (2016). Enhancing the driver awareness of pedestrian using augmented reality cues [Conference session]. 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, 1298–1304.
44.
PretloveJ. (1998). Augmenting reality for telerobotics: Unifying real and virtual worlds. Industrial Robot: An International Journal, 25, 401–407.doi:10.1108/01439919810240225
45.
QuigleyM.GerkeyB.SmartW. D. (2015). Programming robots with ROS: A practical introduction to the robot operating system (1st ed.). O’Reilly Media, Inc.
46.
RicksB.NielsenC. W.GoodrichM. A. (2004). Ecological displays for robot interaction: A new perspective. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), 3, 2855–2860.
47.
RothS.CohenL. J. (1986). Approach, avoidance, and coping with stress. American Psychologist, 41, 813–819.doi:10.1037/0003-066X.41.7.813
48.
Schall Jr.M.RuschM.LeeJ.VeceraS.RizzoM. (2010). Attraction without distraction: Effects of augmented reality cues on driver hazard perception. Journal of Vision, 10, 236doi:10.1167/10.7.236
49.
SchallM. C.RuschM. L.LeeJ. D.DawsonJ. D.ThomasG.AksanN.RizzoM. (2013). Augmented reality cues and elderly driver hazard perception. Human Factors: The Journal of the Human Factors and Ergonomics Society, 55, 643–658.doi:10.1177/0018720812462029
50.
SchneirlaT. C. (1959). An evolutionary and developmental theory of biphasic processes underlying approach and withdrawal. InJonesM. R. (Ed.), Nebraska Symposium on motivation (pp.1–42). http://psycnet.apa.org/record/1960-05385-003
51.
ScuratiG. W.GattulloM.FiorentinoM.FerriseF.BordegoniM.UvaA. E. (2018). Converting maintenance actions into standard symbols for augmented reality applications in industry 4.0. Computers in Industry, 98, 68–79.doi:10.1016/j.compind.2018.02.001
52.
SebeokThomas A. (2001). The Swiss pioneer in nonverbal communication studies: Heini Hediger (1908–1992). Legas.
53.
SheridanT. B. (1992). Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments, 1, 120–126.doi:10.1162/pres.1992.1.1.120
54.
SheridanT. B. (2016). Human–robot interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 58, 525–532.doi:10.1177/0018720816644364
55.
SorkinR. D.KantowitzB. H.KantowitzS. C. (1988). Likelihood alarm displays. Human Factors: The Journal of the Human Factors and Ergonomics Society, 30, 445–459.doi:10.1177/001872088803000406
56.
ToobyJ.CosmidesL. (1990). The past explains the present: Emotional adaptations and the structure of ancestral environments. Ethology and Sociobiology, 11, 375–424.
57.
ValnerR.KruusamäeK.PryorM. (2018). TeMoto: Intuitive multi-range telerobotic system with natural gestural and verbal instruction interface. Robotics, 7, 9.doi:10.3390/robotics7010009
58.
WadeA.McCallC.KarapanagiotidisT.SchofieldG.PrestonC.HartleyT.KaestnerM.HornerA.MaloneyR.SmallwoodJ.JefferiesE.BlojM.HarrisJ. (2018). A neuroscientific approach to exploring fundamental questions in VR. Electronic Imaging, 2018, 435-1–435-6.doi:10.2352/ISSN.2470-1173.2018.03.ERVR-435
59.
WallisT. S. A.HorswillM. S. (2007). Using fuzzy signal detection theory to determine why experienced and trained drivers respond faster than novices in a hazard perception test. Accident Analysis & Prevention, 39, 1177–1185.doi:10.1016/j.aap.2007.03.003
60.
YancoH. A.NortonA.OberW.ShaneD.SkinnerA.ViceJ. (2015). Analysis of human-robot interaction at the DARPA robotics challenge trials. Journal of Field Robotics, 32, 420–444.doi:10.1002/rob.21568
61.
YuhJ. (2000). Design and control of autonomous underwater robots: A survey. Autonomous Robots, 8, 7–24.doi:10.1023/A:1008984701078
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
