Effects of Touch,Voice,and Multimodal Input,and Task Load on Multiple-UAV Monitoring Performance During Simulated Manned-Unmanned Teaming in a Military Helicopter

Abstract

Objective:

We evaluated three interface input methods for a simulated manned-unmanned teaming (MUM-T) supervisory control system designed for Air Mission Commanders (AMCs) in Black Hawk helicopters.

Background:

A key component of the U.S. Army’s vision for unmanned aerial vehicles (UAVs) is to integrate UAVs into manned missions, called MUM-T (Department of Defense, 2010). One application of MUM-T is to provide the AMC of a team of Black Hawk helicopters control of multiple UAVs, offering advanced reconnaissance and real-time intelligence of flight routes and landing zones.

Method:

Participants supervised a (simulated) team of two helicopters and three UAVs while traveling toward a landing zone to deploy ground troops. Participants classified aerial photographs collected by UAVs, monitored instrument warnings, and responded to radio communications. We manipulated interface input modality (touch, voice, multimodal) and task load (number of photographs).

Results:

Compared with voice, touch and multimodal control resulted in better performance on all tasks and resulted in lower subjective workload and greater subjective situation awareness, ps < .05. Participants with higher spatial ability classified more aerial photographs (r = .75) and exhibited shorter response times to instrument warnings (r = −.58) than participants with lower spatial ability.

Conclusion:

Touchscreen and multimodal control were superior to voice control in a supervisory control task that involved monitoring visual displays and communicating on radio channels.

Application:

Although voice control is often considered a more natural and less physically demanding input method, caution is needed when designing visual displays for users sharing common communication channels.

Keywords

supervisory control uninhabited aerial vehicles touchscreens speech user interfaces multimodal displays

Get full access to this article

View all access options for this article.

References

Baber

Morin

Parekh

Cahillane

Houghton

R. J.

(2011). Multimodal control of sensors on multiple simulated unmanned vehicles. Ergonomics, 54, 792–805.

Byers

J. C.

Bittner

A. C.

Hill

S. G.

(1989). Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In Mital

(Ed.), Advances in industrial ergonomics and safety I (pp. 481–485). Philadelphia, PA: Taylor and Francis.

Cahillane

Baber

Morin

(2012). Human factors in UAV. In Angelov

(Ed.), Sense and avoid in UAS: Research and applications (pp. 119–142). West Sussex, UK: Wiley.

Calhoun

G. L.

Draper

M. H.

(2006). Multi-sensory interfaces for remotely operated vehicles. In Cooke

N. J.

Pringle

H. L.

Pedersen

H. K.

Connor

(Eds.), Advances in human performance and cognitive engineering research (Vol. 7, pp.149–163). Oxford, U.K.: Elsevier.

Calhoun

G. L.

Draper

M. H.

(2015). Display and control concepts for multi-UAV applications. In Valavanis

K. P.

Vachtsevanos

G. J.

(Eds.), Handbook of unmanned aerial vehicles (pp. 2443–2473). The Netherlands: Springer.

Calhoun

G. L.

Ruff

H. A.

Behymer

K. J.

Rothwell

C. D.

(2017). Evaluation of interface modality for control of multiple unmanned vehicles. International Conference on Engineering Psychology and Cognitive Ergonomics, 10276, 15–34.

Chen

(2006). Designing human interface in speech technology. New York, NY: Springer.

Chen

J. Y.

Barnes

M. J.

(2012a). Supervisory control of multiple robots in dynamic tasking environments. Ergonomics, 55, 1043–1058.

Chen

J. Y.

Barnes

M. J.

(2012b). Supervisory control of multiple robots: Effects of imperfect automation and individual differences. Human Factors, 54, 157–174.

10.

Chen

J. Y.

Joyner

C. T.

(2009). Concurrent performance of gunner’s and robotics operator’s tasks in a multitasking environment. Military Psychology, 21, 98–113.

11.

Cockburn

Gutwin

Palanque

Deleris

Trask

Coveney

. . . MacLean

(2017). Turbulent touch: Touchscreen input for cockpit flight displays. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 6742–6753. doi:10.1145/3025453.3025584

12.

Department of Defense. (2010). Eyes of the army: U.S. Army roadmap for unmanned aircraft systems 2010–2035. Fort Rucker, AL: U.S. Army UAS Center of Excellence.

13.

Derryberry

Reed

M. A.

(2002). Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology, 111, 225–236.

14.

Draper

Calhoun

Ruff

Williamson

Barry

(2003). Manual versus speech input for unmanned aerial vehicle control station operations. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 47, 109–113.

15.

Gray

W. D.

Sims

C. R.

W. T.

Schoelles

M. J.

(2006). The soft constraints hypothesis: A rational analysis approach to resource allocation for interactive behavior. Psychological Review, 113, 461–482.

16.

Green

D. M.

Swets

J. A.

(1966). Signal detection theory and psychophysics. New York, NY: Wiley.

17.

Hegarty

Waller

(2004). A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence, 32, 175–191.

18.

Jones

D. G.

(2000). Subjective measures of situation awareness. In Endsley

M. R.

Garland

D. J.

(Eds.), Situation awareness: Analysis and measurement (pp. 101–114). Mahwah, NJ: Erlbaum.

19.

Lewis

J. R.

(1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57–78.

20.

McKinley

R. A.

McIntire

L. K.

Funke

M. A.

(2011). Operator selection for unmanned aerial systems: Comparing video game players and pilots. Aviation, Space, and Environmental Medicine, 82, 635–642.

21.

Midkiff

A. H.

Hansman

R. J.

Jr. (1993). Identification of important “party line” informational elements and the implications for situational awareness in the datalink environment. Air Traffic Control Quarterly, 1, 5–30.

22.

Noyes

J. M.

Haas

(2010). Military applications: Human factors aspects of speech-based systems. In Chen

Jokinen

(Eds.), Speech technology: Theory and applications (pp. 251–269). New York, NY: Springer.

23.

Oviatt

(1999). Ten myths of multimodal interaction. Communications of the AMC, 42, 74–81.

24.

Proctor

R. W.

K. P. L.

(2016). Principles for designing interfaces compatible with human information processing. International Journal of Human-Computer Interaction, 32, 2–22.

25.

Taylor

Purman

Schermerhorn

Garcia-Sampedro

Hubal

Crabtree

. . . Spriggs

(2015). Multi-modal interaction for UAS control. Proceedings of SPIE 9468, Unmanned Systems Technology 17, 946802. doi:10.1117/12.2180020.

26.

Thomas

Biswas

Langdon

(2015). State-of-the-art and future concepts for interaction in aircraft cockpits. Proceedings of Universal Access in Human-Computer Interaction: Access to Interaction, 9, 538–549.

27.

Turk

(2014). Multimodal interaction: A review. Pattern Recognition Letters, 36, 189–195.

28.

Wickens

C. D.

(2008). Multiple resources and mental workload. Human Factors, 50, 449–455.

29.

Wickens

C. D.

Carswell

C. M.

(2012). Information processing. In Salvendy

(Ed.), Handbook of human factors and ergonomics (4th ed., pp. 117–161). Hoboken, NJ: Wiley.

30.

Wickens

C. D.

McCarley

J. S.

(2008). Applied attention theory. Boca Raton, FL: CRC Press.

31.

Wickens

C. D.

Vidulich

Sandry-Garza

(1984). Principles of S-C-R compatibility with spatial and verbal tasks: The role of display-control location and voice-interactive display-control interfacing. Human Factors, 26, 533–543.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB

0.13 MB