In many emerging civilian and military operations, human operators are increasingly being tasked to supervise multiple robotic uninhabited vehicles (UVs) with the support of automation. As 100% automation reliability cannot be assured, it is important to understand the effects of automation imperfection on performance. In addition, adaptive aiding may help counter any adverse effects of static (fixed) automation. Using a high-fidelity multi-UV simulation involving both air and ground vehicles, two experiments examined the effects of automation reliability and adaptive automation on human-system performance with different levels of task load. In Experiment 1, participants performed a reconnaissance mission while assisted with an automatic target recognition (ATR) system whose reliability was low, medium, or high. Overall human-robot team performance was higher than with either human or ATR performance alone. In Experiment 2, participants performed a similar reconnaissance mission with no ATR, static automation, or with adaptive automation keyed to task load. Participant trust and self-confidence were higher and workload was lower for adaptive automation compared with the other conditions. The results show that human-robot teams can benefit from imperfect static automation even in high task load conditions and that adaptive automation can provide additional benefits in trust and workload.
ArciszewskiH. F. R.de GreefT. E.van DelftJ. H. (2009). Adaptive automation in a naval combat management system. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 39(6), 1188–1199.
2.
BaileyN. R.ScerboM. W.FreemanF. G.MikulkaP. J.ScottL. A. (2006). Comparison of a brainbased adaptive system and a manual adaptable system for invoking automation. Human Factors, 48(4), 693.
3.
BarnesM.ParasuramanR.CosenzoK. (2006). Adaptive automation for military robotic systems. In NATO Technical Report RTO-TR-HFM-078: Uninhabited military vehicles. Human factors issues in augmenting the force (pp. 420–440). Brussels, Belgium: NATO.
4.
BillingsC. (1997). Aviation automation: The search for a human-centered approach. Mahwah, NJ: Lawrence Erlbaum.
5.
BillingsC. E.WoodsD. D. (1994). Concerns about adaptive automation in aviation systems. In MoulouaM.ParasuramanR. (Eds.), Human performance in automated systems: Current research and trends (pp. 24–29). Hillsdale, NJ: Lawrence Erlbaum.
ChenJ. Y. C.DurlachP. J.SloanJ. A.BowensL. D. (2008). Human-robot interaction in the context of route reconnaissance missions. Military Psychology, 20(3), 135–149.
8.
ChenJ. Y. C.JoynerC. T. (2006). Concurrent performance of gunner’s and robotic operator’s tasks in a simulated mounted combat system environment (Tech. Rep. ARL-TR-3815). Aberdeen Proving Ground, MD: U.S. Army Research Laboratory. Available at http://www.arl.army.mil/arlreports/2006/ARL-TR-3815.pdf
9.
CookeN.PringleH. L.PedersenH. K.ConnorO. (Eds.). (2006). Human factors of remotely operated vehicles: Advances in human performance and cognitive engineering (Vol. 7; pp. 251–266). Oxford, UK: Elsevier.
10.
CummingsM. L.GuerlainS. (2004, August). Human performance issues in supervisory control of autonomous airborne vehicles. Symposium conducted at AUVSI’s Unmanned Systems North America 2004, Anaheim, CA.
11.
CummingsM. L.GuerlainS. (2007). Developing operator capacity estimates for supervisory control of autonomous vehicles. Human Factors, 49(1), 1–15.
12.
de VisserE. J.ParasuramanR.FreedyA.FreedyE.WeltmanG. (2006). A comprehensive methodology for assessing human-robot team performance for use in training and simulation. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 2639–2643). Santa Monica, CA: Human Factors and Ergonomics Society.
13.
DixonS. R.WickensC. D. (2006). Automation reliability in unmanned aerial vehicle control: A reliance-compliance model of automation dependence in high workload. Human Factors, 48(3), 474–486.
14.
DzindoletM. T.PetersonS. A.PomrankyR. A.PierceL. G.BeckH. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58, 697–718.
15.
EndsleyM. R.KaberD. B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 462–492.
16.
GacyM.DahnD. (2006, March). Commonality of control paradigms for unmanned systems. Paper presented at the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction, Salt Lake City, UT.
17.
GalsterS. M.KnottB. A.BrownR. D. (2006). Managing multiple UAVs: Are we asking the right questions? In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 2639–2643). Santa Monica, CA: Human Factors and Ergonomics Society.
18.
HartS. G.,StavelandL. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In HancockP.A.MeshkatiN. (Eds.), Human mental workload (pp. 139–183). Amsterdam: NorthHolland.
19.
InagakiT. (2003). Adaptive automation: Sharing and trading of control. In HollnagelE. (Ed.), Handbook of cognitive task design (pp. 147–170). Mahwah, NJ: Lawrence Erlbaum.
20.
KaberD. B.EndsleyM. (2004). The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science, 5, 113–153.
21.
KaberD. B.WrightM. C.PrinzelL. P.ClamannM. P. (2005). Adaptive automation of human-machine system information processing functions. Human Factors, 47(4), 730–741.
22.
LeeJ. D.MorayN. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243–1270.
23.
LeeJ. D.SeeK. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80.
24.
LevinthalB. R.WickensC. D. (2006). Management of multiple UAVs with imperfect automation. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 2639–2643). Santa Monica, CA: Human Factors and Ergonomics Society.
25.
LewisM.WangJ.ScerriP. (2006). Teamwork coordination for realistically complex multirobot systems. In NATO RTO-MP-HFM-135 Symposium on Human Factors of Uninhabited Military Vehicles as Force Multipliers (Paper 17). Neuilly-sur-Seine, France: NATO-RTO.
26.
MeyerJ. (2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46, 196–204.
27.
MillerC.ParasuramanR. (2007). Designing for flexible interaction between humans and automation: Delegation interfaces for supervisory control. Human Factors, 49, 57–75.
28.
MitchellD. K.HenthornT. (2005). Soldier workload analysis of the Mounted Combat System (MCS) platoon’s use of unmanned assets (ARL-TR-3476). Aberdeen Proving Ground, MD: Army Research Laboratory.
29.
MoulouaM.GilsonR.HancockP. (2003). Human-centered design of unmanned aerial vehicles. Ergonomics in Design, 11(1), 6–11.
30.
ParasuramanR. (2000). Designing automation for human use: Empirical studies and quantitative models. Ergonomics, 43, 931–951.
31.
ParasuramanR.BahriT.DeatonJ.MorrisonJ.BarnesM. (1992). Theory and design of adaptive automation in aviation systems (Progress Report No. NAWCADWAR-92033-60). Warminster, PA: Naval Air Warfare Center.
32.
ParasuramanR.BarnesM.CosenzoK. (2007). Adaptive automation for human-robot teaming in future command and control systems. International Journal of Command and Control, 1(2), 43–68.
33.
ParasuramanR.CosenzoK.de VisserE. (2009). Adaptive automation for human supervision of multiple uninhabited vehicles: Effects on change detection, situation awareness, and mental workload. Military Psychology, 21, 270–297.
34.
ParasuramanR.GalsterS.SquireP.FurukawaH.MillerC. (2005). A flexible delegation-type interface enhances system performance in human supervision of multiple robots: Empirical studies with RoboFlag. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 35, 481–493.
35.
ParasuramanR.RileyV. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.
36.
ParasuramanR.SheridanT. B.WickensC. D. (2000). A model of types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 30, 286–297.
37.
ParasuramanR.SheridanT.WickensC. (2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2, 140–160.
38.
PrinzelL. J. (2003). Team-centered perspective for adaptive automation design. Washington, DC: National Aeronautics and Space Administration, Langley Research Center.
39.
RoviraE.McGarryK.ParasuramanR. (2007). Effects of imperfect automation on decision making in a simulated command and control task. Human Factors, 49, 76–87.
40.
SarterN.WoodsD.BillingsC. E. (1997). Automation surprises. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (2nd ed., pp. 1926–1943). New York, NY: Wiley.
41.
ScerboM. (2001). Adaptive automation. In KarwowskiW. (Ed.), International encyclopedia of ergonomics and human factors (pp. 1077-1079). London, UK: Taylor & Francis.
42.
SheridanT.ParasuramanR. (2006). Human-automation interaction. Reviews of Human Factors and Ergonomics, 1, 89–129.
43.
SheridanT. B.VerplankW. L. (1978). Human and computer control of undersea teleoperators (Tech. Rep. A556750). Cambridge, MA: MIT Man-Machine Systems Laboratory.
44.
SorkinR. D.WoodsD. D. (1985). Systems with human monitors: A signal detection analysis. Human-Computer Interaction, 1(1), 49–75.
45.
TaylorR. M. (1989). Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design. In Proceedings of the AGARD AMP Symposium on Situational Awareness in Aerospace Operations (CP478). Seuilly-sur Seine, France: NATO AGARD.
46.
TingC.MahfoufM.NassefA.LinkensD. A.PanoutosG.NickelP.HockeyG. (2010). Real-time adaptive automation system based on identification of operator functional state in simulated process control operations. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 40(2), 251–262.
47.
WickensC. D.DixonS. R. (2005). Is there a magic number 7 (to the minus 1)? The benefits of imperfect diagnostic automation: A synthesis of the literature (Tech. Rep. AHFD-05-01/MAAD-05-1). Savoy: University of Illinois, Aviation Human Factors Division.
48.
WickensC. D.DixonS. R. (2006). Workload and automation reliability in UAVs. In CookeN. J.PringleH. L.PedersenH. K.ConnorO. (Eds.), Human factors of remotely operated vehicles (pp. 209–222). Amsterdam, Netherlands: Elsevier.
49.
WienerE. L. (1988). Cockpit automation. In WienerE. L.NagelD. C. (Eds.), Human factors in aviation (pp. 433–461). San Diego, CA: Academic Press.
50.
WilsonG. F.RussellC. A. (2003). Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Human Factors, 45, 635–644.
51.
WilsonG. F.RussellC. A. (2007). Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Human Factors, 49, 1005–1018.
