The aim of this study was to compare the effectiveness of a new index of perceived mental workload, the Multiple Resource Questionnaire (MRQ), with the standard measure of workload used in the study of vigilance, the NASA Task Load Index (NASA-TLX).
Background:
The NASA-TLX has been used extensively to demonstrate that vigilance tasks impose a high level of workload on observers. However, this instrument does not specify the information-processing resources needed for task performance. The MRQ offers a tool to measure the workload associated with vigilance assignments in which such resources can be identified.
Method:
Two experiments were performed in which factors known to influence task demand were varied. Included were the detection of stimulus presence or absence, detecting critical signals by means of successive-type (absolute judgment) and simultaneous-type (comparative judgment) discriminations, and operating under multitask vs. single-task conditions.
Results:
The MRQ paralleled the NASA-TLX in showing that vigilance tasks generally induce high levels of workload and that workload scores are greater in detecting stimulus absence than presence and in making successive as compared to simultaneous-type discriminations. Additionally, the MRQ was more effective than the NASA-TLX in reflecting higher workload in the context of multitask than in single-task conditions. The resource profiles obtained with MRQ fit well with the nature of the vigilance tasks employed, testifying to the scale’s content validity.
Conclusion:
The MRQ may be a meaningful addition to the NASA-TLX for measuring the workload of vigilance assignments.
Application:
By uncovering knowledge representation associated with different tasks, the MRQ may aid in designing operational vigilance displays.
BallardJ. C. (1996). Computerized assessment of sustained attention: A review of factors affecting vigilance performance. Journal of Clinical and Experimental Neuropsychology, 18, 843–863.
3.
BolesD. B. (1998, October). Multiple lateralized spatial processes. Paper presented at the meeting of the Psychonomic Society, Dallas, TX.
4.
BolesD. B.AdairL. P. (2001). Validity of the multiple resources questionnaire (MRQ). In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting (pp. 1795–1799). Santa Monica, CA: Human Factors and Ergonomics Society.
5.
BolesD. B.BurskJ. H.PhillipsJ. B.PerdelwitzJ. R. (2007). Predicting dual-task performance with the multiple resources questionnaire. Human Factors, 49, 32–45.
6.
BolesD. B.DillardM. (in press). The measurement of perceptual resources and workload. In HoffmanR. R.HancockP. A.ScerboM. W.SzalmaJ. L.ParasuramanR. (Eds.), Handbook of applied perception. New York, NY: Cambridge University Press.
7.
BolesD. B.LawM. B. (1998). A simultaneous task comparison of differentiated and undifferentiated hemispheric resource theories. Journal of Experimental Psychology: Human Perception and Performance, 24, 204–215.
8.
CaggianoD. M.ParasuramanR. (2004). The role of memory representation in the vigilance decrement. Psychonomic Bulletin & Review, 11, 932–937.
9.
CarswellC. M.LioC. H.GrantR.KleinM. I.ClarkeD.SealesW. B.StrupS. (2010). Hands-free administration of subjective workload scales: Acceptability in a surgical training environment. Applied Ergonomics, 42, 138–145.
10.
DaviesD. R.ParasuramanR. (1982). The psychology of vigilance. New York, NY: Academic Press.
11.
DillardM. B. (2012). Transfer of training and the role of mental workload in expert performance (Unpublished doctoral dissertation). University of Alabama, Tuscaloosa.
12.
DillardM. B.BolesD. B.BlackS. R. (2012). Bidirectional resources training of simple and complex tasks. In Proceedings of the Human Factors and Ergonomics Society 53th Annual Meeting (pp. 2462–2466). Santa Monica, CA: Human Factors and Ergonomics Society.
13.
FarmerE.BrownsonA. (2003). Review of workload measurement, analysis, and interpretation methods. Report for Eurocontrol Integra Program. Brussels, Belgium: European Organization for the Safety of Air Navigation (Eurocontrol).
14.
HancockP. A. (2013). In search of vigilance. American Psychologist, 68, 97–109.
15.
HartS. G.StavelandL. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In HancockP. A.MeshkatiN. (Eds.), Human mental workload (pp. 139–183). Amsterdam, Netherlands: North-Holland.
16.
HawleyJ. K. (2006, January/February). Patriot fratricides: The human dimension lessons of Operation Iraqi Freedom. Field Artillery, pp. 18–19.
17.
HeltonW. S.HollanderT. D.WarmJ. S.MatthewsG.DemberW. N.WallaartM.. . . HancockP. A. (2005). Signal regularity and the mindlessness model of vigilance. British Journal of Psychology, 96, 249–261.
18.
HollanderT. D.WarmJ. S.MatthewsG.ShockleyK.DemberW. D.WeilerE. M.. . . ScerboM. W. (2004). Feature presence/absence modifies the event rate effect and cerebral hemovelocity in vigilance performance. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp. 1943–1947). Santa Monica, CA: Human Factors and Ergonomics Society.
19.
JohnsonA.ProctorR. W. (2004). Attention: Theory and practice. Thousand Oaks, CA: Sage.
20.
KahnemanD. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice Hall.
21.
KaneM. J.McVayJ. C. (2012). What mind wandering reveals about executive-control abilities and failures. Current Directions in Psychological Science, 21, 348–354.
22.
KleinM. L.RileyM. A.WarmJ. S.MatthewsG. (2005). Perceived mental workload in an endoscopic surgery simulator. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting (pp. 1014–1017). Santa Monica, CA: Human Factors and Ergonomics Society.
23.
KleinM. L.WarmJ. S.RileyM. A.MatthewsG.DoarnC.DonovanJ. F.GaitondeK. (2012). Mental workload and stress perceived by novice operators in the laparoacopic and robotic minimally invasive surgical interfaces. Journal of Endourology, 26, 1089–1094.
24.
LangnerR.EickhoffS. B.SteinbornM. B. (2011). Mental fatigue modulates dynamic adaptation to perceptual demand in speeded detection. PLoS One, 6, 1–10.
25.
MacLeanK. A.AicheleS. R.BridwellD. A.MangunG. R.WojciulikE.SaronC. D. (2009). Interactions between endogenous and exogenous attention during vigilance. Attention, Perception, and Psychophysics, 71, 1042–1058.
26.
MacmillanN. A.CreelmanC. D. (2005). Detection theory: A user’s guide (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
27.
MatthewsG.DaviesD. R.WestermanS. J.StammersR. B. (2000). Human performance: Cognition, stress, and individual differences. East Sussex, UK: Psychology Press.
28.
MaxwellS. E.DelaneyH. D. (2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
29.
McVayJ. C.KaneM. J. (2010). Does mind wondering reflect executive function or executive failure? Comment on Smallwood and Schooler (2006) and Watkins (2008). Psychological Bulletin, 136, 188–197.
MolloyR.ParasuramanR. (1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 311–322.
32.
NachreinerF.HaneckeK. (1992). Vigilance. In SmithA. P.JonesD. M. (Eds.), Handbook of human performance (pp. 262–288). San Diego, CA: Academic Press.
33.
NatsoulasT. (1967). What are perceptual reports about?Psychological Bulletin, 67, 249–272.
34.
NickersonR. S. (1992). Looking ahead: Human factors challenges in a changing world. Mahwah, NJ: Lawrence Erlbaum.
35.
NunnallyJ. C.BernsteinI. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
36.
NygrenT. E. (1991). Psychometric properties of subjective workload measurement techniques: Implications for their use in the assessment of perceived mental workload. Human Factors, 33, 17–33.
37.
O’DonnellR. D.EggemeierF. T. (1986). Workload assessment methodology. In BoffK. R.KaufmanL.ThomasJ. P. (Eds.), Handbook of perception and human performance: Vol. 2. Cognitive processes and performance (pp. 42/1-42/49). New York, NY: Wiley.
38.
ParasuramanR. (1979). Memory load and event rate control sensitivity decrements in sustained attention. Science, 205, 924–927.
39.
ParasuramanR.WarmJ. S.DemberW. N. (1987). Vigilance: Taxonomy and utility. In MarkL. S.WarmJ. S.HustonR. L. (Eds.), Ergonomics and human factors: Recent research (pp. 11–32). New York, NY: Springer-Verlag.
40.
ProctorR. W.VuK.-P. L. (2010). Cumulative knowledge and progress in human factors. Annual Review of Psychology, 61, 623–651.
41.
QuinlanP. T. (2003). Visual feature integration theory: Past, present, and future. Psychological Bulletin, 129, 643–673.
42.
RobertsonI. H.ManlyT.AndradeJ.BaddeleyB. T.YiendJ. (1997). Oops! Performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia, 35, 747–758.
43.
ScerboM. W. (1998). What’s so boring about vigilance? In HoffmanR. R.SherrickM. F.WarmJ. S. (Eds.), Viewing psychology as a whole: The integrative science of William N. Dember (pp. 145–166). Washington, DC: American Psychological Association.
44.
ScerboM. W. (2001). Stress, workload, and boredom in vigilance: A problem and an answer. In HancockP. A.DesmondP. A. (Eds.), Stress workload and fatigue (pp. 267–278). Mahwah, NJ: Lawrence Erlbaum.
45.
SchoenfeldV. S.ScerboM. W. (1999). The effects of search differences for the presence and absence of features on vigilance performance and mental workload. In ScerboM. W.MoulouaM. (Eds.), Automation technology and human performance: Current research and trends (pp. 177–182). Mahwah, NJ: Lawrence Erlbaum.
46.
SchraagenJ. M.ChipmanS. F.ShalinV. L. (Eds.). (2000). Cognitive task analysis. Mahwah, NJ: Lawrence Erlbaum.
47.
SeeJ. E.HoweS. R.WarmJ. S.DemberW. N. (1995). Meta-analysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117, 230–249.
48.
SeeJ. E.WarmJ. S.DemberW. N.HoweS. R. (1997). Vigilance and signal detection theory: An empirical evaluation of five measures of response bias. Human Factors, 39, 14–29.
49.
ShingledeckerC.WeldonD.BehymerK.SimpkinsB.LernerE.WarmJ. S.. . . MurphyJ. S. (2010). Measuring vigilance abilities to enhance combat identification performance. In AndrewsD. H.HerzR. P.WolfM. B. (Eds.), Human factors issues in combat identification (pp. 47–66). Aldershot, UK: Ashgate.
50.
SmitA. S.ElingP. A.CoenenA. M. (2004). Mental effort causes vigilance decrease due to resource depletion. Acta Psychologia, 115, 35–42.
51.
SzalmaJ. L.WarmJ. S.MatthewsG.DemberW. N.WeilerE. M.MeierA.EggemeierF. T. (2004). Effects of sensory modality and task duration on performance, workload, and stress in sustained attention. Human Factors, 46, 219–233.
52.
TreismanA. M.GormicanS. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95, 15–48.
53.
WarmJ. S. (1984). An introduction to vigilance. In WarmJ. S. (Ed.), Sustained attention in human performance (pp. 1–14). Chichester, UK: Wiley.
54.
WarmJ. S. (1993). Vigilance and target detection. In WickensC. D.HueyB. (Eds.), Teams in transition: Workload, stress, and human factors (pp. 139–170). Washington, DC: National Research Council.
55.
WarmJ. S.DemberW. N. (1998). Tests of a vigilance taxonomy. In HoffmanR. R.SherickM. F.WarmJ. S. (Eds.), Viewing psychology as a whole: The integrative science of William N. Dember (pp. 87–112). Washington, DC: American Psychological Association.
56.
WarmJ. S.DemberW. N.HancockP. A. (1996). Vigilance and workload in automated systems. In ParasuramanR.MoulouaM. (Eds.), Automation and human performance: Theory and applications (pp. 183–200). Mahwah, NJ: Lawrence Erlbaum.
57.
WarmJ. S.JerisonH. J. (1984). The psychophysics of vigilance. In WarmJ. S. (Ed.), Sustained attention in human performance (pp. 15–59). Chichester, UK: Wiley.
58.
WarmJ. S.MatthewsG.FinomoreV. S. (2008). Workload and stress in sustained attention. In HancockP. A.SzalmaJ. L. (Eds.), Performance under stress (pp. 115–141). Aldershot, UK: Ashgate.
59.
WarmJ. S.MatthewsG.ParasuramanR. (2009). Cerebral hemodynamics and vigilance performance. Military Psychology, 21, S75–S100.
60.
WarmJ. S.ParasuramanR.MatthewsG. (2008). Vigilance requires hard mental work and is stressful. Human Factors, 50, 433–441.
61.
WickensC. D. (1984). Processing resources in attention. In ParasuramanR.DaviesD. R. (Eds.), Varieties of attention (pp. 63–102). Orlando, FL: Academic Press.
62.
WickensC. D.HollandsJ. G.BanburyS.ParasuramanR. (2013). Engineering psychology and human performance (4th ed.). Boston, MA: Pearson.
63.
WigginsM. W. (2011). Vigilance decrement during a simulated general aviation flight. Applied Cognitive Psychology, 25, 229–235.
