Assessing Subjective Workload Assessment: A Comparison of SWAT and the NASA-Bipolar Methods

Abstract

Subjective assessments of workload are becoming increasingly important in the evaluation of new systems. Two popular methods were compared in the present investigation: (1) the Subjective Workload Assessment Technique (SWAT) which was developed around the use of conjoint analysis to create interval scales, and (2) a technique under development at NASA that utilizes an individually weighted workload score from a set of nine bipolar ratings. Both methods were applied in a laboratory experiment that required rating a number of single- and dual-task trials of tracking and/or a spatial transformation task. The dual transformation-tracking task results were reviewed. The results for the two assessment techniques were remarkably similar, indicating that the subjective experience of workload is sufficiently robust to be resistant to variations in the measuring technique. Also, both subjective assessment techniques were successful in measuring the differences in task difficulty as indicated by a multivariate analysis of performance. Finally, the specific strengths and weaknesses of each assessment technique were reviewed.

Get full access to this article

View all access options for this article.

References

Collecting Weighted Average Workload Ratings: A Paper-and-Pencil Package. unpublished manuscript. (available from: M. Vidulich, Mail Stop 239–3, NASA-Ames Research Center, Moffett Field, CA, 94035).

Hart

S.G.

Battiste

Lester

P. T.

(1984). Popcorn: A supervisory control simulation for workload and performance research. In Proceedings of the Twentieth Annual Conference on Manual Control (pp. 431–453). Moffett Field, CA: NASA (NASA-CP-2341).

Moray

(1982). Subjective mental workload. Human Factors, 24, 25–40.

Reid

G. B.

Shingledecker

C. A.

Eggemeier

F. T.

(1981). Application of conjoint measurement to workload scale development. In Proceedings of the Human Factors Society 25th Annual Meeting (pp. 522–526). Santa Monica, CA: Human Factors Society.

Sheridan

T. B.

Simpson

R. W.

(1979). Toward the definition and measurement of the mental workload of transport pilots (Tech. Rep. No. FTLR79-4). Cambridge: MIT Flight Transportation Laboratory.

Tsang

P. S.

(1985). Can pilots time-share better than non-pilots. In Proceedings of the Third Symposium on Aviation Psychology. Columbus: Ohio State University Department of Aviation.

Vidulich

M. A.

Tsang

P. S.

(1985). Techniques of subjective workload assessment: A comparison of two methodologies. In Proceedings of the Third Symposium on Aviation Psychology. Columbus: Ohio State University Department of Aviation.