Objective: This study examined operators' capacity to successfully reallocate highly autonomous in-flight missiles to time-sensitive targets while performing secondary tasks of varying complexity. Background: Regardless of the level of autonomy for unmanned systems, humans will be necessarily involved in the mission planning, higher level operation, and contingency interventions, otherwise known as human supervisory control. As a result, more research is needed that addresses the impact of dynamic decision support systems that support rapid planning and replanning in time-pressured scenarios, particularly on operator workload. Method: A dual screen simulation that allows a single operator the ability to monitor and control 8, 12, or 16 missiles through high level replanning was tested on 42 U.S. Navy personnel. Results: The most significant finding was that when attempting to control 16 missiles, participants' performance on three separate objective performance metrics and their situation awareness were significantly degraded. Conclusion: These results mirror studies of air traffic control that demonstrate a similar decline in performance for controllers managing 17 aircraft as compared with those managing only 10 to 11 aircraft. Moreover, the results suggest that a 70% utilization (percentage busy time) score is a valid threshold for predicting significant performance decay and could be a generalizable metric that can aid in manning predictions. Application: This research is relevant to human supervisory control of networked military and commercial unmanned vehicles in the air, on the ground, and on and under the water.
Get full access to this article
View all access options for this article.
References
1.
Adelman, L. (1991). Experiments, quasi-experiments, and case studies: Areview of empirical methods for evaluating decision support systems. IEEE Transactions on Systems, Man, and Cybernetics, 22, 293-301.
2.
Cummings, M.L. (2003). Designing decision support systems for revolutionary command and control domains. Unpublished doctoral dissertation, University of Virginia, Charlottesville.
3.
Durso, F.T., Hackworth, C.A., Truitt, T.R., Crutchfield, J., Nikolic, D., & Manning, C.A. (1998). Situation awareness as a predictor of performance for en route air traffic controllers. Air Traffic Control Quarterly, 6, 1-20.
4.
Endsley, M. (2000). Direct measurement of situation awareness: Validity and use of SAGAT. In M. Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement (pp. 73-112). Mahwah, NJ: Erlbaum.
5.
Endsley, M., & Rogers, M.D. (1996). Attention distribution and situation awareness in air traffic control. In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting (pp. 82-85). Santa Monica, CA: Human Factors and Ergonomics Society.
6.
Endsley, M., Sollenberger, R.L., Nakata, A., & Stein, E.S. (2000). Situation awareness in air traffic control: Enhanced displays for advanced operations. Atlantic City, NJ: U.S. Department of Transportation.
7.
Galster, S.M., Duley, J.A., Masalonis, A.J., & Parasuraman, R. (2001). Air traffic controller performance and workload under mature free flight: Conflict detection and resolution of aircraft self-separation. International Journal of Aviation Psychology, 11, 71-93.
8.
Hebb, D.O. (1955). Drives and the CNS. Psychological Review, 62, 243-254.
9.
Hilburn, B.G., Bakker, M.W.P., Pekela, W.D., & Parasuraman, R. (1997, October). The effect of free flight on air traffic controller mental workload, monitoring and system performance. Paper presented at the 10th International Conference of European Aerospace Societies , Amsterdam.
10.
Jones, D.G. (2000). Subjective measures of situation awareness. In M. Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement (pp. 113-128). Mahwah, NJ: Erlbaum.
11.
Keppel, G. (1982). Design and analysis: A researcher's handbook. Englewood Cliffs, NJ: Prentice Hall.
12.
Klein, G. (2000). Analysis of situation awareness from critical incident reports. In M. Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement (pp. 51-71). Mahwah, NJ: Erlbaum.
13.
Nichols, T.A., Rogers, W.A., & Fisk, A.D. (2003). Do you know how old your participants are? Ergonomics in Design, 11(3), 22-26.
14.
Rouse, W.B. (1983). Systems engineering models of human-machine interaction . New York: North Holland.
15.
Ruff, H.A., Narayanan, S., & Draper, M.H. (2002). Human interaction with levels of automation and decision-aid fidelity in the supervisory control of multiple simulated unmanned air vehicles. Presence, 11, 335-351.
16.
Santoro, T.P., &Amerson, T.L. (1998, June). Definition and measurement of situation awareness in the submarine attack center. Paper presented at the Command and Control Research and Technology Symposium, Monterey, CA.
17.
Schmidt, D.K. (1978). A queuing analysis of the air traffic controller's workload. IEEE Transactions on Systems, Man, and Cybernetics , 8, 492-498.
18.
Sheridan, T.B. (1992). Telerobotics, automation, and human supervisory control. Cambridge, MA: MIT Press .
19.
Shingledecker, C.A. (1987). In-flight workload assessment using embedded secondary radio communication tasks. In A. Roscoe (Ed.), The Practical Assessment of Pilot Workload (AGARDograph No. 282, pp. 11-14). Neuilly sur Seine, France: Advisory Group for Aerospace Research and Development.
20.
Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Mahwah, NJ : Erlbaum.
21.
Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate statistics (4th ed.). New York: HarperCollins .
22.
Tsang, P., & Wilson, G.F. (1997). Mental Workload. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (2nd ed., pp. 417-489). New York: John Wiley & Sons, Inc.
23.
U.S. Navy. (2002). Tomahawk weapons system (Baseline IV) operational requirements document. Washington, DC: Department of Defense.
24.
Vidulich, M.A. (2000). Testing the sensitivity of situation awareness metrics in interface evaluations. In D. J. Garland (Ed.), Situation awareness analysis and measurement (pp. 227-246). Mahwah, NJ: Erlbaum.
25.
Wickens, C.D., & Hollands, J.G. (2000). Engineering psychology and human performance (3rd ed.). Upper Saddle River, NJ: Prentice-Hall.
26.
Willis, R.A. (2001). Tactical Tomahawk weapon control system user interface analysis and design. Unpublished master's thesis, University of Virginia, Charlottesville.
27.
Yerkes, R.M., & Dodson, J.D. (1908). The relation of strength of stimulus to rapidity of habit-formation. Journal of Comparative Neurology and Psychology , 1, 459-482.
28.
Zacks, R.T., Hasher, L., & Li, K.Z.H. (2000). Human memory. In F. I. M. Craik & T. A. Salthouse (Eds.), The handbook of aging and cognition (pp. 293-357). Mahwah, NJ: Erlbaum.
