Four subjective workload scales were compared along four dimensions: sensitivity, operator acceptance, resource requirements, and special procedures. The scales were the Modified Cooper-Harper scale, the National Aeronautics and Space Administration Task Load Index (TLX), the Overall Workload (OW) scale, and the Subjective Workload Assessment Technique. Three U.S. Army systems were studied for potential workload concerns. Data from five different studies on the three systems were compared along the aforementioned four dimensions. Results indicate that all four scales are acceptable tools and are sensitive to different levels of workload. However, TLX and OW are consistently superior when considering sensitivity, as measured by factor validity, and operator acceptance. This research is an example of a systematic approach for examining human factors measurement tools.
Get full access to this article
View all access options for this article.
References
1.
Armstrong Aerospace Medical Research Laboratory. (1987). Subjective workload assessment technique (SWAT): A user's guide.Wright-Patterson Air Force Base: Author.
2.
Bierbaum, C., Szabo, S., and Aldrich, T. (1987). A comprehensive task analysis of the UH-60 mission with crew workload estimates and preliminary decision rules for developing a UH-60 workload prediction model (Tech. Report MDA903-87-C-0523). Ft. Rucker, AL: Anacapa Sciences.
3.
Bittner, A. C. Jr., Byers, J. C., Hill, S. G., Zaklad, A. L., and Christ, R. E. (1989). Generic workload ratings of a mobile air defense system (LOS-F-H). In Proceedings of the Human Factors Society 33rd Annual Meeting (pp. 1476–1480). Santa Monica, CA: Human Factors Society.
4.
Bittner, A. C. Jr., Zaklad, A. L., Dick, A. O., Wherry, R. J., Jr., Herman, E. D., Bulger, J. P., Linton, P. M., Lysaght, R. J., and Dennison, T. W. (1988). Operator workload (OWL) assessment program for the army: Validation and analysis plans for three systems (ATHS, Aquila, LOS-FH) (Tech. Report 2075-3b). Willow Grove, PA: Analytics, Inc.
5.
Byers, J. C., Bittner, A. C. Jr., and Hill, S. G. (1989). Traditional and raw Task Load Index (TLX) correlations: Are paired comparisons necessary? In A. Mital (Ed.), Advances in industrial ergonomics and safety I (pp. 481–485). London: Taylor & Francis.
6.
Byers, J. C., Bittner, A. C. Jr., Hill, S. G., Zaklad, A. L., and Christ, R. E. (1988). Workload assessment of a remotely piloted vehicle (RPV) system. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 1145–1149). Santa Monica, CA: Human Factors Society.
7.
Byers, J. C., Hill, S. G., Zaklad, A. L., and Christ, R. E. (1989). Aquila system report (Tech. Report 2075-4A). Willow Grove, PA: Analytics, Inc.
8.
Cooper, G. E., and Harper, R. P. (1969). The use of pilot ratings in the evaluation of aircraft handling qualities (NASA TN-D-5153). Moffett Field, CA: NASA Ames Research Center.
9.
Dixon, W. J. (Ed.). (1983). BMDP statistical software.Los Angeles: University of California Press.
10.
Hancock, P. A., and Meshkati, N. (Eds.). (1988). Human mental workload.Amsterdam: North-Holland.
11.
Hart, S. G., and Staveland, L. E. (1988). Development of a NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.), Human mental workload (pp. 139–183). Amsterdam: North-Holland.
12.
Hill, S. G., Byers, J. C., Zaklad, A. L., and Christ, R. E. (1989). Subjective workload ratings of the LOS-F-H mobile air defense missile system in a field test environment (Tech. Memo 5). willow Grove, PA: Analytics, Inc.
13.
Hill, S. G., Zaklad, A. L., Bittner, A. C. Jr., Byers, J. C., and Christ, R. E. (1988). Workload assessment of a mobile air defense missile system. In Proceedings of the Human Factors Society 32nd Annual Meeting (pp. 1068–1072). Santa Monica, CA: Human Factors Society.
14.
Hinkley, D. V. (1983). Jackknife methods. In S. Konz, N. L. Johnson, and C. B. Read (Eds.), Encyclopedia of statistical sciences (Vol. 4, pp. 280–287). New York: Wiley.
15.
lavecchia, H. P., Linton, P. M., Bittner, A. C. Jr., and Byers, J. C. (1989). Operator workload in the UH-60 Blackhawk: Crew results vs. TAWL model predictions. In Proceedings of the Human Factors Society 33rd Annual Meeting (pp. 1481–1485). Santa Monica, CA: Human Factors Society.
16.
Iavecchia, H. P., Linton, P. M., Harris, R. M., Zaklad, A. L., and Byers, J. C. (1989). UH-60 system report (Tech. Report 2075-4C). Willow Grove, PA: Analytics, Inc.
17.
Lysaght, R. J., Hill, S. G., Dick, A. O., Plamondon, B. D., Wherry, R. J. Jr., Zaklad, A. L., and Bittner, A. C., Jr. (1989). Operator workload: Comprehensive review and evaluation of operator workload methodologies (ARI Tech. Report 851). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
18.
Moray, N. (Ed.). (1979). Mental workload: Its theory and measurement.New York: Plenum.
19.
O'Donnell, R. D., and Eggemeier, F. T. (1986). Workload assessment methodology. In K. R. Boff, L. Kaufman, and J. P. Thomas (Eds.), Handbook of perception and human performance (pp. 42-1-42–49). New York: Wiley.
20.
Reid, G. B., Shingledecker, C. A., and Eggemeier, F. T. (1981). Application of conjoint measurement to workload scale development. In Proceedings of the Human Factors Society 25th Annual Meeting (pp. 522–525). Santa Monica, CA: Human Factors Society.
21.
Sheridan, T. (1980). Mental workload--What is it? Why bother with it?Human Factors Society Bulletin, 23(2), 1–2.
22.
Vidulich, M. A., and Tsang, P. S. (1985). Assessing subjective workload assessment: A comparison of SWAT and the NASA-Bipolar methods. In Proceedings of the Human Factors Society 29th Annual Meeting (pp. 71–75). Santa Monica, CA: Human Factors Society.
23.
Vidulich, M. A., and Tsang, P. S. (1987). Absolute magnitude estimation and relative judgment approaches to subjective workload assessment. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 1057–1061). Santa Monica, CA: Human Factors Society.
24.
Warr, D., Colle, H., and Reid, G. (1986). A comparative evaluation of two subjective workload measures: The Subjective Workload Assessment Technique and the Modified Cooper-Harper Scale. Paper presented at the Symposium on Psychology in the Department of Defense. U.S. Air Force Academy, Colorado Springs, CO.
25.
Wierwille, W. W., and Casali, J. G. (1983). A validated rating scale for global mental workload measurement application. In Proceedings of the Human Factors Society 27th Annual Meeting (pp. 129–133). Santa Monica, CA: Human Factors Society..
