A major barrier to effective spatial decision-making in human-agent teams is that humans and algorithms use different mechanisms to solve spatial problems, frequently leading them to produce different solutions. Incongruity between algorithm-generated solutions and human spatial mental models results in higher workload in mixed-initiative systems, and potential breakdowns in trust and team situation awareness. Although these performance effects are well-understood, few methods exist for quantifying and comparing human spatial mental models and algorithm-generated solutions. To address these problems, 27 participants completed solutions to 5 spatial planning problems, before and after receiving assistance from 2 navigation algorithms. A novel path mapping and clustering approach provided a means of quantifying consensus in human mental models, and shifts in those mental models after viewing the algorithm-suggested routes. Human solutions clustered into a small number of shared mental models. Individual differences in trust in each algorithm predicted acceptance of that algorithm’s route.
Get full access to this article
View all access options for this article.
References
1.
BrunyéT. T.TaylorH. A. (2008). Extended experience benefits spatial mental model development with route but not survey descriptions. Acta Psychologica, 127, 340-354.
2.
ChenJ. Y. C.BarnesM.J. (2014). Human–agent teaming for multirobot control: A review of human factors issues, IEEE Transactions on Human-Machine Systems, 44(1), 13-29.
3.
ClareA. S. (2013). Modeling real-time human-automation collaborative scheduling of unmanned vehicles. (Unpublished Doctoral Dissertation). Massachusetts Institute of Technology, Cambridge, MA.
4.
ClareA. S.MacbethJ. C.CummingsM. L. (2012). Mixed-initiative strategies for real-time scheduling of multiple unmanned vehicles. In Proceedings of the IEEE American Control Conference, 676-682.
5.
ClareA. S.MaereP. C. P.CummingsM. L. (2012). Assessing operator strategies for real-time replanning of multiple unmanned vehicles. Intelligent Decision Technologies, 6, 221-231.
6.
CummingsM. L.ClareA.HartC. (2010). The role of humanautomation consensus in multiple unmanned vehicle scheduling. Human Factors, 52(2), 348-349.
7.
DryM.LeeM. D.VickersD.HughesP. (2006). Human performance on visually presented traveling salesperson problems with varying numbers of nodes. Journal of Problem Solving, 1, 20-32.
8.
HancockP.A.BillingsD.R.SchaeferK.E. (2011). Can You Trust Your Robot?Ergonomics in Design, 19, 24-29.
9.
JahnG.Johnson-LairdP.N.KnauffM. (2005). Reasoning about consistency with spatial mental models: Hidden and obvious indeterminacy in spatial descriptions. In: FreksaC.KnauffM.Krieg-BrücknerB.NebelB.BarkowskyT. (eds) Spatial cognition IV: Reasoning, action, interaction (pp. 165-180). Berlin, Germany: Springer.
10.
LeeJ.D.SeeK.A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50-80.
11.
LeischF. (2004). FlexMix: A general framework for finite mixture models and latent class regression in R. Journal of Statistical Software, 11(8).
12.
MacGregorJ. N.OrmerodT. C. (1996). Human performance on the Traveling Salesman Problem. Perception & Psychophysics, 58, 527-539.
13.
MillerC. A.ParasuramanR. (2007). Designing for flexible interaction between humans and automation: Delegation interfaces for supervisory control. Human Factors, 49(1), 57-75.
14.
MuellerS. T. (2014). PEBL: The Psychology experiment building language (Version 0.14) [Computer experiment programming language]. Retrieved June2014 from http://pebl.sourceforge.net.
15.
MuellerS. T.PerelmanB. S.TanY.ThanasuanK. (2015). Development of the PEBL Traveling Salesman Problem Computerized Testbed. The Journal of Problem Solving, 8, 4.
16.
MuellerS. T.PerelmanB. S.VeinottE. S. (2015). An optimization approach for mapping and measuring divergence and correspondence between paths. Behavior Research Methods.
17.
OsoskyS.SchusterD.PhillipsE.JentschF.G. (2013). Building Appropriate Trust in Human-Robot Teams. In Proceedings of the 2013 AAAI Spring Symposium Series: Trust and Autonomous Systems, 60-65.
18.
ParasuramanR.RileyV. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2), 230-253.
19.
PerelmanB. S. (2015). A naturalistic computational model of human behavior in navigation and search tasks. (Unpublished Doctoral Dissertation. Michigan Technological University, Houghton, MI.
20.
PerelmanB. S.MuellerS. T. (2015). Identifying mental models of search in a simulated flight task using a pathmapping approach. In Proceedings of the 18th International Symposium on Aviation Psychology.
21.
PerkinsD.RobertsP.FeeneyG. (2003). Missing person behaviour: an aid to the search manager. Newport: Mountain Rescue Council England and Wales.
22.
RothE. M.HansonM. L.HopkinsC.MancusoV.ZachariasG. L. (2004). Human in the loop evaluation of a mixed-initiative system for planning and control of multiple UAV teams. In Proceedings of the HFES Annual Meeting, 48(3), 280-284.
23.
SchaeferK. E.SandersT. L.YordonR. E.BillingsD. R.HancockP. A. (2012). Classification of robot form: factors predicting perceived trustworthiness. In Proceedings of the HFES Annual Meeting, 56(1), 1548-1552.
24.
ShaikhM. T.GoodrichM. A.YiD.HoehneJ. (2016). Interactive multi-objective path planning through a palette-based user interface. In Proceedings of the SPIE Defense + Security, 9837.
25.
TecuciG.BoicuM.CoxM. T. (2007). Seven aspects of mixed-initiative reasoning. AI Magazine, 28(2), 11-18.
26.
TenbrinkT.SeifertI. (2011). Conceptual layers and strategies in tour planning. Cognitive processing, 12(1), 109-125.
27.
TenbrinkT.WienerJ. (2009). The verbalization of multiple strategies in a variant of the traveling salesperson problem. Cognitive processing, 10(2), 143-161.
28.
TverskyB. (1991). Spatial mental models. Psychology of Learning and Motivation, 27, 109-145.
29.
VenablesW. N.RipleyB. D. (2002) Modern Applied Statistics with S. Fourth Edition. New York: Springer.
