Evaluation of 16 Measures of Mental Workload using a Simulated Flight Task Emphasizing Mediational Activity

Abstract

As aircraft and other systems become more automated, a shift is occurring in human operator participation in these systems. This shift is away from manual control and toward activities that tap the higher mental functioning of human operators. Therefore, an experiment was performed in a moving-base flight simulator to assess mediational (cognitive) workload measurement. Specifically, 16 workload estimation techniques were evaluated as to their sensitivity and intrusion in a flight task emphasizing mediational behavior. Task loading, using navigation problems presented on a display, was treated as an independent variable, and workload-measure values were treated as dependent variables. Results indicate that two mediational task measures, two rating scale measures, time estimation, and two eye behavior measures were reliably sensitive to mediational loading. The time estimation measure did, however, intrude on mediational task performance. Several of the remaining measures were completely insensitive to mediational load.

Get full access to this article

View all access options for this article.

References

Berliner, C., Angell, D., and Shearer, D. J. (1964). Behaviors, measures, and instruments for performance evaluation in simulated environments. Paper presented at the Symposium and Workshop on the Quantification of Human Performance, Albuquerque, New Mexico.

Casali, J. G. (1982). A sensitivity/intrusion comparison of mental workload estimation techniques using a simulated flight task emphasizing perceptual piloting behaviors. Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University, Blacksburg, Virginia.

Casali, J. G. , and Wierwille, W. W. (1983). A comparison of rating scale, secondary-task, physiological, and primary-task workload estimation techniques in a simulated flight task emphasizing communications load. Human Factors, 25, 623–641.

Casali, J. G. , and Wierwille, W. W. (1984). On the measurement of pilot perceptual workload: A comparison of assessment techniques addressing sensitivity and intrusion issues. Ergonomics, 27, 1033–1050.

Casali, J. G. , Wierwille, W. W. , and Cordes, R. E. (1983). Respiratory measurement: Overview and new instrumentation. Behavior Research Methods and Instrumentation, 15, 401–405.

Cooper, G. E. , and Harper, R. P., Jr. (1969, April). The use of pilot rating in tire evaluation of aircraft handling qualities (NASA TN-D-5153). Moffett Field, CA: NASA Ames Research Center.

Donnell, M. L. (1979). The application of decision-analytic techniques to the test and evaluation phase of the acquisition of a major air system: Phase III (Technical Report PR79–6–91). McLean, VA: Decisions and Designs, Inc.

Gartner, W. B. , and Murphy, M. R. (1976).Pilot workload and fatigue: A critical survey of concepts and assessment techniques (TN-D-8365). Moffett Field, CA: NASA-Ames Research Center.

Keppel, G. (1973). Design and analysis: A researcher's handbook. Englewood Cliffs, NJ: Prentice-Hall.

10.

Levine, J. M. , Ogden, G. D. , and Eisner, E. J. (1978). Measurement of workload by secondary tasks: A review and annotated bibliography (NAS2–9637). Washington, DC: Advanced Research Resources Organization.

11.

Michon, J. A. (1966). Tapping regularity as a measure of perceptual-motor load. Ergonomics, 9, 401–412.

12.

Moray, N. (Ed.). (1979). Mental workload: Its theory and measurement. New York: Plenum Press.

13.

Moray, N. (1982). Subjective mental workload. Human Factors, 24, 25–40.

14.

Wierwille, W. W. , and Casali, J. G. (1983). A validated rating scale for global mental workload measurement applications. In Proceedings of the Human Factors Society 27th Annual Meeting (pp. 129–133). Santa Monica, CA: Human Factors Society.

15.

Wierwille, W. W. , and Connor, S. A. (1983). Evaluation of 20 workload measures using a psychomotor task in a moving-base aircraft simulator. Human Factors, 25, 1–16.

16.

Wierwille, W. W. , and Williges, B. H. (1980). An annotated bibliography on operator mental workload assessment (SY-27R-80). Patuxent River, MD: Naval Air Test Center.

17.

Wierwille, W. W. , and Williges, R. C. (1978). Survey and analysis of operator workload assessment techniques (Report No. S-78–101). Blacksburg, VA: Systemetrics. Inc.