Sage Journals: Discover world-class research

Abstract

Multiple Resource Theory, and one particular instantiation of the theory in the 4-Dimensional Multiple Resource Model have often been invoked as mechanisms to account for dual task interference. Here we present the data and analyses requirements necessary to conclude that the multiple resource model either does or does not predict an increase in the dual task decrement; and in particular we describe the need to consider and control both task difficulty and task priority in comparing conditions that are argued to contrast shared versus separate resources. We then consider one common misconception regarding what the model predicts regarding perfect time sharing, and address the importance of considering alternative theory-based mechanisms of task switching and of confusion and crosstalk, to account for patterns of multi-task interference.

Keywords

Multiple resources Multi-tasking Dual task interference stages of processing

Introduction

Over the last 40 years, since the concept of multiple-resource theory in time-sharing was proposed by Kantowitz and Knight (1976), North (1977), Navon and Gopher (1979), and myself (Wickens, 1976, 1980), and since a particular instantiation of that theory in the multiple resource model was offered (Wickens, 1984), there have been several experiments that have addressed the viability of both the theory and the model. The theory provides the general predictions of how two or more resources behave in accounting for dual task interference; the model specifies the specific identity of those resources. There are other viable multiple resource models that have been developed (e.g., Salvucci & Taatgen, 2008, 2010).

The objective of this paper is to provide the human factors research community with a clear, intuitive explanation of multiple resource model in order to clarify what multiple resources are, and how experimental evidence can be harnessed to provide support for or against the model in terms of its predictions of dual task interference. The reason I undertake this effort now, in spite of several oft-cited past publications about the model (Wickens, 1980, 1984, 2002, 2008, Wickens, Helton et al., 2021), is that over the past decade I have read several papers on the topic, and reviewed numerous manuscripts that have either claimed “disproof” of the multiple resources model, or support for the theory and model that is unwarranted on the basis of the data and analyses provided by the authors.

Figure 1 presents the current instantiation of the model in its “cubical” form. The figure presents in the cube three dichotomous dimensions that are argued to characterize different perceptual modalities, cognitive codes, or processing stage resources, on each side of each dichotomy (the black lines) (modalities is actually a trichotomy, now including tactile, along with the original auditory and visual).

Figure 1.

The 4 dimensional multiple resource model.

The premise of both the theory and model is that when two concurrently performed tasks are asserted by the investigator to use the same resources, where “same” is defined as the same level or category along each of the four dimensions of the model (stages, codes, modalities within perception, and visual channels within the visual perceptual stage), then there will be greater dual task interference, than when the two tasks share separate levels. This is what I refer to as “ the comparison ,” a concept that is critical to the discussion below, and true validation assumes that there are no other confounding differences between the two means being contrasted in the comparison.

Although support for the model shown in Figure 1 comes primarily from performance data in dual task time-sharing studies, it also relies upon converging evidence from neurophysiological studies which validate that the separate resources on each dimension also correspond to separate brain structures (e.g., visual vs. auditory cortex; left and right cerebral hemispheres, Wickens, 2008). As an engineering psychology model I also wish to assure that each dichotomy aligns with important design distinctions that a designer may choose in order to improve multi-tasking performance by the user.

The fundamental multiple resource model that I have proposed (e.g., Wickens, 2008) asserts that there are three relevant factors contributing to this dual task interference (factors, as distinct from the four dichotomous dimensions of the model in Figure 1). These are:

(1) The extent to which the same resources are used in both conditions of the comparison (multiple resource factor).

(2) The amount of resources required, or level of difficulty (cognitive load) of the tasks (Norman & Bobrow, 1975; resource demand factor). Greater difficulty will consume more resources and hence increase the dual task decrement suffered by one or both tasks. Note that the viability of considering resources as a limited commodity is supported by the increasing evidence from neuro-ergonomics studies that oxygen consumption within the brain is both limited and supports cognitive performance (e.g., Ayaz & Dehais, 2019; Hirshfield et al., 2023).

(3) The relative priority of the two time-shared tasks, (resource allocation factor) a variable or emphasis that often leads to the definition of a primary versus a secondary task.

At this point it is important to emphasize that unless factors (2) and (3) are carefully controlled in the selection of tasks and in experimental design and instructions in making the comparison, neither support for, nor evidence against multiple resource theory and the multiple resource model can be conclusively offered.

The Prototypical Experiment

A proper experiment designed to test whether shared versus separate resources result in more dual task interference must include:

● Three baseline conditions:

– Single task performance on each of the two conditions that vary resource structure (e.g., auditory [a] and visual [v] displays). Since this is the task that is M anipulated to make the comparison, we call these conditions Ma1 and Mv1, where the “1” designates “single.”

– The I nterfering task in its single task version. We call this task I1.

– These conditions are represented, in italics, in the top row and leftmost column of Table 1.

● Two dual task conditions sharing the interfering task with each of the two tasks whose structure is varied (the M task). Each dual task condition will in turn generate two performance measures, one for each task. These dual task conditions are shown boldfaced in the two bottom cells of the table. The digit “2” indicates two time-shared tasks.

This 5-condition experimental design will in turn allow the computation of four dual task decrements shown in the four cells of Table 2

Table 1.

Five Conditions of the Experimental Design.

	Ma1	Mv1
I	Ma2 + I2	Mv2 + I2

Table 2.

Calculation of Dual Task Decrement in the Four Cells.

Performance of:	Auditory condition	Visual condition
Manipulate task	Ma1 − Ma2	Mv1 − Mv2
Interfering task	[I1 − I2]\a	[I1 − I2]\v

Note. The “\” indicates “given time shared with {a or v}.” That it is assumed here that for each performance measure, higher is better (e.g., accuracy) and hence the decrement is computed by [single − dual, or single minus dual] a value that is expected to never be negative.

In a prototypical experiment involving a visual interfering task (e.g., driving or flying) we would expect to find that, to the extent that visual versus auditory perception define separate resources, then [Ma1 − Ma2] and [I1 − I2]\a will be smaller than [Mv1 − Mv2] and [i1 − I2\v] for at least one, if not both of the performance measures. If at least one score is smaller and the other is not larger, then this should provide unequivocal evidence in support of the multiple resource model in which perceptual modalities defines resources.

Approach

The approach I have taken is to informally review several articles (published, or unpublished but submitted and reviewed) over the past decade, in order to extract generic examples of how the data intended to either “disprove” multiple resource theory or to validate it, have been inadequate to conclusively do so. I explicitly do NOT cite individual investigators here; rather, this is seen as a guide for future investigations, not a criticism of any particular investigator.

Findings

Given the necessary conditions to validate or invalidate described above, three common errors have occurred in:

1. Measuring the full dual task performance decrement. Investigators have sometimes used the performance decrement in only one of the two time-shared tasks (e.g., M or I) but not both to infer the degree to which this decrement is greater in the shared resource condition than the separate resource condition. However, it is impossible to establish the degree of dual task interference between tasks unless performance in both tasks is assessed. Furthermore, a decrement (single minus dual task performance) must be assessed for both tasks in both of the task load conditions (dual and single), rather than only dual task performance. Thus, the three single task performance measures shown in Table 1 are always required. In the absence of meeting this requirement in experimental design, it is possible that participants may trade off priorities.

Consider for example, a case in which, in the dual task conditions, the visual interfering task (I) is given priority over the visual version of the manipulated task (Mv); but this priority is not imposed in the auditory version condition (Ma) and instead the auditory manipulated task is given priority (the case of intrusive auditory interruptions). Under these circumstances of different task emphasis or priority between conditions, an the observation of greater or equal effect when the visual interfering task is time shared with Ma does not allow inferences to be drawn about resource competition as it could be due to the priority shift. Diagnosis of this case by the researcher cannot be made unless the performance decrements of BOTH tasks in both conditions of the comparison are measured.

2. Difficulty control and difficulty confound. It needs to be established that the task whose resource is manipulated for the comparison (e.g., an auditory vs. a visual interrupting task in a visual driving environment) is not more difficult in its single task form, in the shared relative to the separate resource condition. If it is more difficult, then the finding of greater interference in the shared than the separate condition could be simply due to greater resource demand, not to a shared resource structure.

An example here is with driving (Task I) being time shared with directional guidance (Task M) that is offered through either the auditory or visual channel. Here, suppose the visual guidance is presented on a north-up map as an arrow pointing in the compass direction, N, S, E, or W while the auditory guidance is a command to “turn left” or “turn right.” Here the data might show a smaller decrement with the auditory guidance hence purporting to support the modality dimension of the multiple resource model. But it could just as well be that the smaller decrement results because it is easier to process guidance information when it is offered in an ego-frame of reference (“left-right”), rather than in a world frame (“east-west”).

A subjective workload measure is typically adequate to establish equivalent difficulty or resource demand between the two versions of the M task.

3. Priority control. It should be established, or at least imposed, that the priority instructions given to each task are the same in both of the dual task conditions of the comparison. This can be readily accomplished by either instructions that I or M is the primary (and the other secondary) task or that both are of equal priority.

A violation would obviously be if the participant treats the M task as primary in the auditory condition (Ma2); but the M task as secondary, or of equal priority to the I task, in the visual condition (Mv2).

The issue of priority can be a challenging one to address when this may vary across the two conditions in the comparison, since resource competition can be reflected differently across what may be the two different independent variables (two different tasks)—an “apples versus oranges” comparison. For example, when a tracking task is time-shared with a discrete RT task, consider tracking error versus discrete task RT. A solution to this challenge is to express both decrements in common units, and then simply sum the decrements across the two tasks, as Wickens et al. (1983) have done, to compute an overall interference measure. Common units could be achieved by transforming each decrement into z scores or effect size measures. Alternatively, raw scores could be created by subtracting a minimum value (i.e., the lowest obtained across participants) from a maximum value (the highest obtained), in order to define a scale; and then express each participant score in each condition of the comparison as a point or percentile along this range.

Note that in the three factors addressed above, I have used the case of auditory versus visual resources, because this is the one that appears most prominently in investigations that have addressed or considered the viability of multiple resources. This is in part, because choices of voice versus visual display are such frequent ones confronting interface designers. But similar guidance is also highly relevant for spatial versus linguistic codes of processing, whether at input (i.e., text vs. map display), cognition of mental representation (visual-spatial vs. phonetic: Baddeley, 2003) or response (voice control vs. key press).

Other Issues

Perfect Time-Sharing

In addition to the shortcomings of experimental design and decrement comparison a second misrepresentation of multiple resources occurs when a dual task decrement using separate resources is observed (i.e., not perfect time sharing) and this observation is used as an argument against the multiple resource concept. This argument is based on the investigator’s assumption that use of separate resources will always produce perfect time sharing (zero decrement). But this argument does not take into account that a task pair may compete for common levels of dimensions of multiple resources other than just perceptual modalities. For example, the two tasks may compete for general perceptual resources (interpreting semantic meaning), or for cognitive and/or response resources: competition at all stages should be accounted for (Sarno & Wickens, 1995).

Other Models and Mechanisms

Another issue of relevance here is that a multiple resource model is not the only viable model or mechanism to account for multi-tasking interference (a positive decrement). Two examples are prominent:

First, Attention switching models often assume that it is more time-consuming to switch between modalities than within, a finding that is clearly at odds with the predictions of the multiple resource model (better cross-modal time sharing). But it is of importance here that multiple resource models only apply when it is possible for the two tasks to be performed concurrently, and several circumstances thwart that possibility (Wickens, McCarley, & Gutzwiller, 2022). Indeed, between people within a paradigm, some people may prefer concurrence, and others sequential performance (Brüning & Manzey, 2018).

Second, Cooperation, confusion and crosstalk models, or models of outcome conflict postulate mechanisms of dual task interference that can be operating on performance even as, or while, competition for common resources may be taking place. For example cooperation, can result when two tasks share the same mapping rules (Fracker & Wickens, 1989; Duncan, 1979), as time-sharing performance is facilitated when only a single set of rules need be held in working memory. Here “sameness” facilitates time-sharing, even as sameness of resources hinders it.

Confusion may result when two tasks share the same kind of material; for example, both digits, as when you are engaged in digit data entry while listening to basketball scores. Here material from one task may be confused with, and hence cross-talk, by inadvertently triggering responses for the other (Navon & Miller, 1987). Here, like the multiple resource model, similarity hurts multi-tasking, but does so via a very different mechanism than via the competition for similar resources.

Conclusion

In conclusion, several factors have been identified that must be adequately controlled in order examine the viability of the multiple resource model to predict dual task decrements. Sometimes experimental constraints have prevented full compliance with these factors. While this does not imply that the data are worthless for that comparison, it does require that the conclusions drawn for or against the viability of the model must be offered with appropriate caution.

It is also the case that the existence of alternative theories invites the concern that negative results for the model can be explained by the investigator simply by arguing that “a different model dominates in this case.” This is a legitimate concern and can only be addressed by: (a) conceding that multi-tasking is extremely complex (Salvucci & Taatgen, 2008, 2010), (b) recognizing that multiple mechanisms can be operating simultaneously within a paradigm or context, and (c) invoking alternative mechanisms only when independent reasons indicate that that they should be present. For example, task switching arguments can be invoked if the spatial separation between two visual sources is sufficiently great that both cannot be simultaneously accessed in foveal vision.

Thus, a multiple resource model is not the “holy grail” of multi-task performance prediction. But it is one important tool for doing so; when appropriately applied.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Ayaz

Dehais

(Eds.). (2019). Neuroergonomics: The brain at work and in everyday life. Academic Press.

Baddeley

(2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829–839. https://doi.org/10.1038/nrn1201

Brüning

Manzey

(2018). Flexibility of individual multitasking strategies in task-switching with preview: Are preferences for serial versus overlapping task processing dependent on between-task conflict? Psychological Research, 82(1), 92–108. https://doi.org/10.1007/s00426-017-0924-0

Duncan

(1979). Divided attention: The whole is more than the sum of its parts. Journal of Experimental Psychology: Human Perception and Performance, 5(2), 216–228. https://doi.org/10.1037/0096-1523.5.2.216

Hirshfield

Wickens

Doherty

Spencer

Williams

Hayne

(2023). Toward workload-based adaptive automation: The utility of fNIRS for measuring load in multiple resources in the brain. International Journal of Human-Computer Interaction, 1–28. https://doi.org/10.1080/10447318.2023.2266242

Fracker

M. L.

Wickens

C. D.

(1989). Resources, confusions, and compatibility in dual axis tracking: Displays, controls, and dynamics. Journal of Experimental Psychology: Human Perception and Performance, 15, 80–96.

Kantowitz

B. H.

Knight

J. L.

(1976). Testing tapping timesharing, II: Auditory secondary task. Acta Psychologica, 40(5), 343–362. https://doi.org/10.1016/0001-6918(76)90016-0

Navon

Miller

(1987). Role of outcome conflict in dual-task interference. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 435–448. https://doi.org/10.1037/0096-1523.13.3.435

North

(1977). Task functional demands as factors in dual task performance [Paper presentation]. Proceedings of the 21st Annual Meeting of the Human Factors Society, Santa Monica, CA, United States.

10.

Navon

Gopher

(1979). On the economy of the human processing system. Psychological Review, 86, 254–255.

11.

Norman

D. A.

Bobrow

D. G.

(1975). On data-limited and resource-limited processes. Cognitive Psychology, 7(1), 44–64. https://doi.org/10.1016/0010-0285(75)90004-3

12.

Salvucci

D. D.

Taatgen

N. A.

(2008). Threaded cognition: An integrated theory of concurrent multitasking. Psychological Review, 115(1), 101–130. https://doi.org/10.1037/0033-295X.115.1.101

13.

Salvucci

D. D.

Taatgen

N. A.

(2010). The multitasking mind. Oxford University Press.

14.

Sarno

K. J.

Wickens

C. D.

(1995). The role of multiple resources in predicting time-sharing efficiency: An evaluation of three workload models in a multiple task setting. International Journal of Aviation Psychology, 5(1), 107–130.

15.

Wickens

C. D.

(1976). The effects of divided attention on information processing in tracking. Journal of Experimental Psychology: Human Perception and Performance, 2, 1–13.

16.

Wickens

C. D.

(1980). The structure of attentional resources. In Nickerson

(Ed.), Attention and performance VIII (pp. 239–257). Lawrence Erlbaum.

17.

Wickens

C. D.

(1984). Processing resources in attention. In Parasuraman

Davies

(Eds.), Varieties of attention (pp. 63–101). Academic Press.

18.

Wickens

C. D.

(2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159–177.

19.

Wickens

C. D.

(2008). Multiple resources and mental workload. Human Factors Golden Anniversary Special Issue, 3, 449–455.

20.

Wickens

C. D.

Helton

Hollands

Banbury

(2021). Engineering psychology and human performance (5th ed., Chapter 11). Taylor & Francis.

21.

Wickens

C. D.

McCarley

J. S.

Gutzwiller

R. S.

(2022). Applied attention theory (2nd ed.). Taylor & Francis.

22.

Wickens

C. D.

Sandry

Vidulich

(1983). Compatibility and resource competition between modalities of input, output, and central processing. Human Factors, 25, 227–248.

The Multiple Resource Theory and Model. Some Misconceptions in Data Interpretations

Abstract

Keywords

Introduction

The Prototypical Experiment

Approach

Findings

Other Issues

Perfect Time-Sharing

Other Models and Mechanisms

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

References