Participant Characteristics in Human-in-the-Loop Studies with Multiple Unmanned Vehicles including Aircraft

Abstract

The characteristics that a supervisor of multiple autonomous and semi-autonomous systems should possess remain unclear. Determination of these qualities would support job performance as well as recruiting and training. To evaluate the human characteristics currently being considered by human-in-the-loop experiments, a review of the multiple remote vehicle supervision literature was conducted. The human characteristics addressed included: gender, domain relevant experience, working memory, supervisory relevant skills and abilities (e.g., visual skills, spatial ability, attentional control, vigilance), and traits related to multi-tasking (e.g., stress, resilience). The discussion identifies gaps in the current state of the art with respect to the consideration of human characteristics for multi-autonomous and semi-autonomous systems supervision where at least one vehicle is an aircraft.

Get full access to this article

View all access options for this article.

References

Baddeley

A. D.

Hitch

(1974). Working memory. In Bower

G. H.

(Ed.), The Psychology of Learning and Motivation: Advances in Researchand Theory (Vol. 8, pp. 47–89). Academic Press.

Bass

E. J.

Pritchett

A. R.

(2008). Human-automated judge learning: A methodology for examining human interaction with information analysis automation. IEEETransactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 38(4), 759–776.

Bolton

M. L.

Bass

E. J.

(2009). Comparing perceptual judgment and subjective measures of spatial awareness. Applied Ergonomics, 40(4), 597–607.

Breslow

L. A.

Gartenberg

McCurry

J. M.

Trafton

J. G.

(2014). Dynamic operator overload: A model for predicting workload during supervisory control. IEEE Transactions on Human-Machine Systems, 44(1), 30–40.

Bruni

Schurr

Cooke

Riordan

Freeman

(2011). Designing a mixed-initiative decision-support system for multi-UAS mission planning. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 55, pp. 21–25).

Cacace

Finzi

Lippiello

Furci

Mimmo

Marconi

(2016). A control architecture for multiple drones operated via multi-modal interaction in search & rescue mission. In 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) (pp. 233–239).

Calhoun

G. L.

Miller

C. A.

Hughes

T. C.

Draper

M. H.

(2014). UAS sense and avoid system interface design and evaluation. In Proceedings of the Human Factors andErgonomics Society Annual Meeting (Vol. 58, pp. 21–25).

Calhoun

G. L.

Ruff

H. A.

Draper

M. H.

Wright

E. J.

(2011). Automation-level transference effects in simulated multiple unmanned aerial vehicle control. Journal of Cognitive Engineering and Decision Making, 5(1), 55–82.

Calhoun

G. L.

Ruff

H. A.

Spriggs

Murray

L. C.

(2012). Tailored performance-based adaptive levels of automation. Proceedings of the Human Factors andErgonomics Society Annual Meeting, 56, 413–417.

10.

Calhoun

G. L.

Ward

Ruff

H. A.

(2011). Performance-based adaptive automation for supervisory control. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 55, 2059–2063.

11.

Chen

Campbell

Gonzalez

L. F.

Coppin

(2015). Increasing autonomy transparency through capability communication in multiple heterogeneous UAV management. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 2434–2439).

12.

Chien

Lewis

Sycara

Kumru

Liu

(2020). Influence of culture, transparency, trust, and degree of automation on automation use. IEEETransactions on Human-Machine Systems, 50(3), 205–214.

13.

Coyne

Foroughi

Brown

Sibley

(2018). Evaluating decision making in a multi-objective route planning-task. Proceedings of the Human Factors andErgonomics Society Annual Meeting, 62(1), 217–221.

14.

Cummings

M. L.

Clare

Hart

(2010). The role of human-automation consensus in multiple unmanned vehicle scheduling. Human Factors, 52(1), 17–27.

15.

de Visser

Kidwell

Payne

Parker

Brooks

… Parasuraman

(2013). Best of both worlds: Design and evaluation of an adaptive delegation interface. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 255–259.

16.

de Visser

Parasuraman

(2011). Adaptive aiding of human-robot teaming: Effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making, 5(2), 209–231.

17.

de Visser

Shaw

Mohamed-Ameen

Parasuraman

(2010). Modeling human-automation team performance in networked systems: Individual differences in working memory count. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, pp. 1087–1091).

18.

Donmez

Nehme

Cummings

M. L.

(2010). Modeling workload impact in multiple unmanned vehicle supervisory control. IEEETransactions on Systems, Man, and Cybernetics -Part A: Systems and Humans, 40(6), 1180–1190.

19.

Dye

M. W.

Green

C. S.

Bavelier

(2009). The development of attention skills in action video game players. Neuropsychologia, 47(8-9), 1780–1789.

20.

Engle

R. W.

(2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23.

21.

FAA. (2016). FAA Code ofFederal Regulations (14 CFR)Part 107-Small Unmanned Aircraft System. (https://www.ecfr.gov/current/title-14/chapter-I/subchapter-F/part-107)

22.

FAA. (2020). U.S. Civil Airmen Statistics.(https://www.faa.gov/data_research/aviation_data_statistics/civil_airmen_statistics/)

23.

Farmer

Sundberg

N. D.

(1986). Boredom proneness: The development and correlates of a new scale. Journal ofPersonality Assessment, 50(1), 4–17.

24.

Gugerty

Brooks

(2004). Reference-frame misalignment and cardinal direction judgments: group differences and strategies. Journal of Experimental Psychology: Applied, 10(2), 75–88.

25.

Kidwell

Calhoun

G. L.

Ruff

H. A.

Parasuraman

(2012). Adaptable and adaptive automation for supervisory control of multiple autonomous vehicles. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 56, pp. 428–432).

26.

Koh

S. L.

Pfeil

LaViola

J. J.

Jr (2018). Exploring the potential of full body and hand gesture teleoperation of robots inside heterogeneous human-robot teams. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 62, pp. 2008–2012).

27.

Lee

Franchi

Giordano

P. R.

Son

H. I.

Bulthoff

H. H.

(2011). Haptic teleoperation of multiple unmanned aerial vehicles over the internet. In 2011 IEEE International Conference on Robotics andAutomation (pp. 1341–1347).

28.

Lin

Matthews

Wohleber

R. W.

Funke

G. J.

Calhoun

G. L.

Ruff

H. A.

… Chiu

(2020). Overload and automation-dependence in a multi-UAS simulation: Task demand and individual difference factors. Journal of Experimental Psychology: Applied, 26, 218–235.

29.

Matthews

Panganiban

A. R.

Wells

Wohleber

R. W.

Reinerman-Jones

L. E.

(2019). Metacognition, hardiness, and grit as resilience factors in unmanned aerial systems (UAS) operations: a simulation study. Frontiers in Psychology, 10, 1–17.

30.

McKendrick

Shaw

Saqer

de Visser

Parasuraman

(2011). Team performance and communication within networked supervisory control human-machine systems. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 55, pp. 262–266).

31.

Memar

A. H.

Esfahani

E. T.

(2018). Physiological measures for human performance analysis in human-robot teamwork: Case of tele-exploration. IEEE Access, 6, 3694–3705.

32.

Miller

C. A.

Shaw

Emfield

Hamell

deVisser

Parasuraman

Musliner

(2011). Delegating to automation: Performance, complacency and bias effects under non-optimal conditions. In Proceedings of the Human Factors andErgonomics Society Annual Meeting (Vol. 55, pp. 95–99).

33.

Mkrtchyan

A. A.

Macbeth

J. C.

Solovey

E. T.

Ryan

J. C.

Cummings

M. L.

(2012). Using variable-rate alerting to counter boredom in human supervisory control. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56, 1441–1445.

34.

Moacdieh

N. M.

Devlin

S. P.

Jundi

Riggs

S. L.

(2020). Effects of workload and workload transitions on attention allocation in a dual-task environment: Evidence from eye tracking metrics. Journal of Cognitive Engineering and Decision Making, 14(2), 132–151.

35.

Neubauer

Matthews

Langheim

Saxby

(2012). Fatigue and voluntary utilization of automation in simulated driving. Human Factors, 54(5), 734–746.

36.

Prinet

J. C.

Terhune

Sarter

N. B.

(2012). Supporting dynamic replanning in multiple UAV control: A comparison of 3 levels of automation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 56, pp. 423–427).

37.

Roth

Schulte

Schmitt

Brand

(2020). Transparency for a workload-adaptive cognitive agent in a manned-unmanned teaming application. IEEETransactions on Human-Machine Systems, 50(3), 225–233.

38.

Rovira

Cross

Leitch

Bonaceto

(2014). Displaying contextual information reduces the costs of imperfect decision automation in rapid retasking of ISR assets. Human Factors, 56(6), 1036–1049.

39.

Ruff

Calhoun

Frost

Behymer

Bartik

(2018). Comparison of adaptive, adaptable, and hybrid automation for surveillance task completion in a multi-task environment. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 62, pp. 155–159).

40.

Warm

J. S.

Parasuraman

Matthews

(2008). Vigilance requires hard mental work and is stressful. Human Factors, 50(3), 433–441.

41.

Wells

(1994). A multi-dimensional measure of worry: Development and preliminary validation of the Anxious Thoughts Inventory. Anxiety, Stress and Coping, 6(4), 289–299.

42.

Wohleber

R. W.

Matthews

Lin

Szalma

J. L.

Calhoun

G. L.

Funke

G. J.

… Heath

A, R.

(2019). Vigilance and automation dependence in operation of multiple unmanned aerial systems (UAS): a simulation study. Human Factors, 61(3), 488–505.

43.

Wohleber

R. W.

Matthews

Reinerman-Jones

L. E.

Panganiban

A. R.

Scribner

(2015). Individual differences in resilience and affective response during simulated UAV operations. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting(Vol. 59, pp. 751–755).

44.

Wright

J. L.

Chen

J. Y. C.

Barnes

M. J.

(2018). Human-automation interaction for multiple robot control: the effect of varying automation assistance and individual differences on operator performance. Ergonomics, 61(8), 1033–1045.

45.

Wright

J. L.

Quinn

S. A.

Chen

J. Y.

Barnes

M. J.

(2014). Individual differences in human-agent teaming: An analysis of workload and situation awareness through eye movements. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 58, pp. 1410–1414).