Context Effects in Subjective Mental Workload Ratings

Abstract

The impact of performance context on subjective mental workload ratings was assessed with the Subjective Workload Assessment Technique (SWAT) and the NASA Task Load Index (TLX). In Experiment 1, a strong context effect was demonstrated. A low range of task difficulty produced considerably higher ratings on a common set of difficulty levels than did a high range of task difficulty. In Experiment 2, increasing the participants′ range of experiences during practice eliminated the context effect. We recommend that methods for standardizing context, such as providing experience with the complete difficulty range, be developed for subjective mental workload evaluations. Actual or potential applications of this research include providing methodologies for controlling context effects in practical assessments of mental workload to increase the validity of subjective measures.

Get full access to this article

View all access options for this article.

References

Battig, W. F. , & Montague, W. E. (1969). Category norms for verbal items in 56 categories: A replication and extension of the Connecticut Category Norms. Journal of Experimental Psychology Monographs, 80 (3, Pt. 2).

Colle, H. A. , Amell, J. R. , Ewry, M. E. , & Jenkins, M. L. (1988). Capacity equivalence curves: A double trade-off curve method for equating task performance. Human Factors, 30, 645–656.

Colle, H. A. , & Reid, G. B. (1997). A framework for mental workload research and applications using formal measurement theory. International Journal of Cognitive Ergonomics, 1, 303–313.

Endsley, M. R. , & Kiris, E. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37, 381–394.

Gopher, D. , & Donchin, E. (1986). Workload - An examination of the concept. In K. R. Boff , L. Kaufman , & J. P. Thomas (Eds.), Handbook of perception and human performance: Vol. 2. Cognitive processes and performance (pp. 41/1–41/49). New York: Wiley.

Hart, S. G. , & Staveland, I. E. (1988). Development of NASATLX (Task Load Index): Results of experimental and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139–183). Amsterdam: Elsevier.

Hill, S. G. , Iavecchia, H. P. , Byers, J. C. , Bittner, A. C. , Zaklad, A. L. , & Christ, R. E. (1992). Comparison of four subjective workload rating scales. Human Factors, 34, 429–439.

O'Donnell, R. D. , & Eggemeier, F. T. (1986). Workload assessment methodology. In K. R. Boff , L. Kaufman , & J. P. Thomas (Eds.), Handbook of perception and human performance: Vol. 2. Cognitive processes and performance (pp. 42/1–42/49). New York: Wiley.

Parducci, A. (1963). The range-frequency compromise in judgment. Psychological Monographs, 77(2, Whole No. 565).

10.

Parducci, A. (1965). Category judgment: A range-frequency model. Psychological Review, 72, 407–418.

11.

Parducci, A. , & Wedell, D. H. (1986). The category effect with rating scales: Number of categories, number of stimuli, and method of presentation. Journal of Experimental Psychology: Human Perception and Performance, 12, 496–516.

12.

Reid, G. B. , & Nygren, T. E. (1988). The subjective workload assessment techniques: A scaling procedure for measuring mental workload. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 185–218). Amsterdam: Elsevier.

13.

Wedell, D. H. (1990). Methods for determining the locus of context effects in judgment. In J. P. Caverni , J. M. Fabre , & M. Gonzalez (Eds.), Cognitive biases (pp. 285–302). Amsterdam: Elsevier.

14.

Wedell, D. H. , & Parducci, A. (1985). Category and stimulus effects: A process model for contextual memory in judgment. In G. d'Ydewalle (Ed.), Cognition, information processing, and motivation (Vol. 3, pp. 55–70). Amsterdam: North-Holland.

15.

Wierwille, W. W. , & Eggemeier, F. T. (1993). Recommendations for mental workload measurement in a test and evaluation environment. Human Factors, 35, 263–281.