Abstract
This research was designed to evaluate the extent to which power functions can predict performance on a task when performance context has been altered. Since power functions reliably describe performance improvements during practice, an assumption implicit in some theories of skill acquisition and transfer is that transfer performance will continue to improve as an extrapolation of the practice power function. In the training phase of Experiment 1, 120 participants practiced solving simple problems from the six-times table. In the transfer phase, these same problems were presented again, intermixed with problems from one of the six conditions differing in various respects to the target problems. With the exception of two of these six conditions, performance on the target problems was slower than was predicted by training phase power function extrapolations. Where the nature of the task was altered, this disruption only occurred in the initial stages of transfer, with performance returning quickly to predicted levels. Where an increased scope of knowledge was required to perform the task, the disruption was more prolonged. Experiment 2 demonstrated that a spacing effect explanation of the results of Experiment 1 was not valid. Results were interpreted as reflecting the effect a change in the conceptual context of a task has on transfer performance. These findings have implications for theories of skill acquisition and transfer that assume transfer performance of established skills will continue to improve according to an extrapolation of the practice power function regardless of the conceptual context of the task.
When time to perform a task is plotted against the amount of practice, a learning curve is typically observed. The shape of this curve is such that improvements in the speed of performance are usually large early in practice, but become progressively smaller as practice continues. Newell and Rosenbloom (1981) suggested that power functions provide the best description of such learning curves and claim that the ubiquity and consistency of power function learning curves mark this phenomenon as a law—known as the power law of learning. 1 One condition for law-like status that was not considered by Newell and Rosenbloom, however, was whether a power function description of a learning curve enables prediction of future performance. Given that performance improvements can be described by a mathematical function, if the task conditions remain consistent, and the motivation of the person performing the task remains constant, then performance should continue to improve according to the function. Therefore, it should be possible to predict future performance by extrapolating the power function that describes past performance. The aim of the current research was to evaluate whether the power law meets this test and to explore some of the implications of this feature of the law.
Explaining the power law of learning is considered a benchmark criterion for evaluating any theory of skill acquisition and transfer (Logan, 1988). Two theories of skill acquisition, the Adaptive Control of Thought-Rational (ACT-R) theory (Anderson, 1982, 1993; Anderson & Lebiere, 1998) and the instance theory of automaticity (Logan, 1988), provide popular accounts of the power law. The ACT-R theory explains skilled performance as the execution of production rules. With practice, productions are developed that relate the occurrence of stimulus conditions and performance goals with the execution of responses. In this way, skilled performance can become reflex-like, with particular conditions (goals and stimuli) automatically invoking particular responses. The ACT-R theory explains the rate of improved performance described by the power law as resulting from a combination of two processes. First, practice leads to refinements in production rules such that fewer steps are required to perform a task, resulting in faster performance. Second, practice leads to strengthening of productions, resulting in productions being accessed and executed faster.
Logan’s (1988) instance theory of automaticity described skilled performance as the execution of specific actions in response to unique stimulus conditions. Each processing episode results in the creation of a representation (an instance) of the stimulus conditions, the goal of processing, the actions executed, and the results of those actions. The instance theory account of the power law states that practice results in the storage of many instances. Practice leads to instances with increasingly shorter retrieval times; however, the probability of shorter retrieval times decreases as practice continues.
Both the ACT-R and instance theories characterize skilled performance as the automatic activation of responses following exposure to particular stimulus conditions. Both theories also state that power functions describe improvements in the speed with which responses are executed. By implication, then, both theories would consider that predictions of the absolute level of performance of a task are possible. That is, if the stimulus conditions and performance goals associated with a task are the same as those encountered during practice, previously acquired productions or instances will be executed in subsequent task performance and in a way that conforms to the rate of improvement described by the power law. Speelman and Kirsner (1993; Speelman, 1995) reported that this is indeed the case. When nothing about a task was changed, a power function that described performance improvements for 288 trials was able to predict the pattern of improvement on a subsequent 288 trials. This observation suggests that the improvement trajectory of skills has some long-term stability.
The successful prediction of future performance on the basis of a power function description of past performance implies that transfer performance may also be predictable. Given that future performance of a task in an old context can be predicted, then it might be possible to predict future performance in a new context. Certainly, the ACT-R and Instance theories imply that if stimulus conditions in a new task context are such that old skills can be executed, then the best prediction of the speed with which those old skills will be executed is determined by extrapolating the power function describing the original improvement of those old skills. In other words, old skills will continue to improve in the context of a new task according to the power function describing the original development of these skills (Speelman & Kirsner, 1993).
Speelman and Kirsner (1993; Speelman, 1995) examined the practical application of this implication, and hence, the theoretical framework underlying the ACT-R and instance theories. They found that transfer performance did not conform to power law predictions because the predictions consistently underestimated the response times. This observation was more closely examined by Speelman and Kirsner (2001) in a task that was essentially a series of arithmetic problems. Over many trials, participants performed the same three calculations during the training and transfer phase, but performed two additional calculations in the transfer phase. Each calculation was constructed in a way that it had to be performed independently, and in a sequence so that “old” problems were completed before “new” problems. According to the theoretical framework of the ACT-R and instance theories, skills developed during training should have transferred completely to the relevant component of the transfer tasks. Speelman and Kirsner (2001) found, however, that reaction time (RT) on the old components of the task was slower at the beginning of transfer than at the end of training, indicating that the presence of the novel task components had in some way affected RT on the old task components. This disruption was only temporary, though, with performance returning to levels predicted by power function extrapolations, suggesting that the change in task context might prompt an adjustment period. This result also challenges the notion that skills have a stable improvement trajectory. The pattern of improvement may be disrupted by changes in task context.
In the second experiment of Speelman and Kirsner’s (2001) study, complexity was manipulated by reversing the training and transfer tasks, with five calculations in the training phase, followed by three calculations in the transfer phase. Although a disruption occurred, its magnitude was not as great as in the first experiment. This led Speelman and Kirsner (2001) to conclude that any change in the task context can cause some disruption, but an increase, as opposed to a decrease, in task complexity leads to the greatest amount of disruption.
It is important to note that as well as altering task complexity, Speelman and Kirsner (2001) simultaneously altered the visual context of the task by changing the number of calculations between the training and transfer phases. In doing so, they may have altered the conceptual context by prompting participants to conceive the task requirements as being different. “Conceptual context” is defined as an internal representation of the typical experimental trial, that influences cognitive processing and memory retrieval by guiding the contents of working memory (see, for example, Peters, Wilson, & Powell, 1976). It is therefore possible that any change in the task environment may prompt a change in the conceptual representation of the task, thus affecting task performance.
The moderating influence of contextual factors is an important consideration in any examination of skill acquisition and performance predictions, because people use context as a frame of reference against which they assess requirements and make task-related decisions (Carlson & Shin, 1996; Müller, 1999; Reder & Klatzky, 1994). This influence is highlighted in a study of Brazilian schoolchildren working as street vendors (Carraher, Carraher, & Schliemann, 1985). When these children were working on the streets, they were able to demonstrate impressive mathematical skills when mentally calculating the total cost of orders that involved different numbers of different objects. In their familiar street environment, they were 98% accurate in their calculations. However, when asked to perform the same tasks in pure mathematical terms (e.g., 5 × 35 = ? as opposed to calculating the total cost of five lemons at 35 cruzeiros), their accuracy dropped to 37%. This drop in accuracy continued to be present even when the tasks were stated as word problems that related directly to their work, with performance improving to only 74%. Although these children obviously possessed the skills to complete the laboratory calculations, they were unable to do so because of a change in the way that they had conceived of the task. That is, a change in the conceptual context of the task meant that they were unable to draw on their well-established skills. This suggests that the perceptual features of the task at hand need to match a person’s internal representation of the task to guide both the identification and application of processing rules to complete the task. Thus, the degree of disparity between the internal representation and external features of the task would be inversely proportional to the degree of transfer performance. This proposition is consistent with Rickard, Healy, and Bourne’s (1994) identical elements model, which holds that skill transfer is a function of the match between an internal abstraction and the perceptual features and required operations of a task. Thus, changing the context of a task, in effect, changes the conceptual representation of the task requirements.
The variability of the practice environment can influence the conceptual context of a task, and hence, can affect both the manner in which skills are initially developed and how they are transferred to novel environments. For example, maintaining a consistent task environment facilitates skill acquisition because people are able to anticipate the next task, and thus have the processing rules available in working memory to execute the task more efficiently (e.g., Carlson & Lundy, 1992; Carlson & Yaure, 1990). If the context of a task were to change (i.e., at transfer), however, then these processing rules may be applied incorrectly. This is the principle underlying the phenomenon known as a mental set, where people adhere to a particular cognitive strategy even though it may be less effective or efficient than another strategy (Woltz, Bell, Kyllonen, & Gardner, 1996). Changing the task context varies the conceptual representation of the task, disrupting the mental set formed as a consequence of practice. Accordingly, this prompts people to engage in strategic processing to re-assess task requirements and load the required processing rules into working memory. This explains why practice in a consistent task environment tends to facilitate the development of skills, but restricts their performance in novel situations (Carlson & Lundy, 1992).
Although a consistent environment benefits skill acquisition, a random task environment hinders this development (Carlson & Yaure, 1990). A continual change in the conceptual context introduces an overhead because people are required to repeatedly appraise task requirements and alternate the relevant processing rules in and out of working memory. Consequently, this switching cost compromises optimal performance during any skill acquisition phase (e.g., Gopher, Armony, & Greenshpan, 2000). In contrast, however, a random task environment facilitates transfer performance (Carlson & Yaure, 1990; Woltz et al., 1996) and increases retention of information (Lee & Magill, 1983). Because the conceptual representation of the task is being continually altered, this means that strategic processes must always be engaged, thus preventing a mental set toward a particular task from being developed or inappropriately applied. Any change in the task environment, therefore, could interfere with the internal (i.e., conceptual) representation of the task and prompt strategic processing. In doing so, this is likely to disrupt a mental set and, so, affect transfer performance, ultimately rendering unreliable any predictions based on learning curve extrapolations. Accordingly, a change in the task’s conceptual representation may have been responsible for the performance disruption observed by Speelman and Kirsner (2001).
Because calculations in Speelman and Kirsner’s (2001) tasks were either added or subtracted to create transfer tasks, a change in the conceptual context may have arisen and thus been responsible for the observed disruption to transfer performance. It is not possible, however, to determine whether this disruption was induced by the change to the visual appearance of the task or a function of the change in perceived complexity, with participants conceiving of the task as requiring alternative processing rules. Consequently, the current research sought to determine the extent to which the disruption is due to varying the degree of conceptual context while controlling visual context.
Experiment 1
The first experiment in this study involved presenting a set of target problems common to both the training and transfer phases one at a time, interspersed with a set of distracter problems in the transfer phase. The target problems were presented in an identical manner in both phases of the experiment, and so there were no changes to the visual context of the task from training to transfer. The conceptual context was altered by varying the nature of the distracter problems. This in turn enabled a manipulation of the processing overhead associated with task switching.
The target problems used in the study involved single-digit multiplication because these problems typically involve simple fact retrieval and reflect robust and long-standing skills (Pesta, Sanders, & Murphy, 1999). To facilitate a conceptual change in the task environment, the study used distracter tasks involving processing rules that varied from subtle to more substantial departures from the target problems. Rickard et al. (1994) stated that the more similar a transfer task is to a training task, the less switching of processing rules is required, and the greater the amount of transfer that will result because problem solution is facilitated by a shared knowledge base. By inference, one would expect that the more removed the distracter problem is from the target problem, in terms of the required processing rules, the greater the disruption to predicted transfer performance because of the overhead introduced by having to alternate these rules in and out of working memory. Conversely, the more similar the distracter problem, the less switching of processing rules would be required. The current study varied the degree of similarity of the processing rules that underlie solution of the target and distracter problems, by selecting distracter conditions that ranged from other arithmetic fact retrieval tasks (or tasks that could be re-cast and then solved by fact retrieval) to algorithmic processing and a combination of algorithmic processing and fact retrieval. There were six distracter conditions: operand change (single-digit multiplication items that were unrelated to the target task), operand reversal (the reversed order of the target items), operation change (the target items presented in a varied format), symbol change (the division equivalent to the target items), double-digit addition (which drew on algorithmic processing), and large multiplication (which involved a combination of memory retrieval and algorithmic processing). Examples of these distracter items are presented in Table 1.
Sample of the Items Used in Experiment 1.
It was predicted that if the transfer disruption observed by Speelman and Kirsner (2001) is simply due to the overhead of processing rule switching induced by a change in a task’s conceptual context, then the disruption should be a function of the degree of departure from the target problem in terms of processing rules. Little or no disruption to RT was expected in the operand change and operand reversal conditions. Because the distracter problems in these conditions could be solved by memory retrieval of multiplication tables, they did not represent substantial departures from the target problems. A greater degree of disruption to RT, however, was expected in the operation change, symbol change, double-digit addition, and large multiplication conditions. The distracter problems in these conditions all required the application of varying degrees of processing rules beyond those of simple memory retrieval of multiplication tables. As such, these conditions involved a greater degree of switching between processing rules because the rules associated with completing the distracter problems varied to those required for completing the target problems. Any disruption resulting from performing the target problems in the context of the distracter problems (i.e., at transfer) would be revealed by performance times being slower than those predicted by the learning curve.
According to the ACT-R theory (Anderson, 1993; Anderson & Lebiere, 1998) and the instance theory of automaticity (Logan, 1988), the best prediction of RT on the target problems in the transfer phase would simply be an extrapolation of the learning curve that described performance on these problems during the training phase. That is, given that the target problems presented in the transfer phase were identical to those presented in the training phase, performance on the transfer target problems should conform to power function predictions. As such, these theories hold that no transfer disruption should be observed. By examining whether transfer performance will continue to improve as an extrapolation of the practice power function, the present study provides a test of an assumption that is implicit in both the ACT-R and instance theories, as well as an aspect of the power law of learning. That is, repeated application of skills, regardless of changes in task context, will push the skills further along a stable improvement trajectory.
Method
Participants
The sample consisted of 120 participants, recruited approximately equally from the Edith Cowan University School of Psychology Volunteers register and from the Western Australia Police Service. Participants had a mean age of 34.99 years (SD = 9.47 years). There were 38 females (M = 33.03 years, SD = 8.92 years) and 82 males (M = 35.90 years, SD = 9.64 years). The mean years of schooling for all participants was 13.22 years (SD = 3.50 years). To ensure that well-developed skills were being examined, only the data of those participants who attained an accuracy level of at least 80% in the training phase were used in the analysis of the results. Two participants failed to meet the required degree of accuracy and were replaced. Demographic information applies only to those participants whose data were included in the analysis. Both experiments reported in this article were approved by the Edith Cowan University Human Research Ethics Committee. All subjects granted their written informed consent to participate in the experiment.
Materials
The selection of the target problems took into account the “problem size effect” under which RT increases as the numerical value of the operand increases (Allen, Ashcraft, & Weber, 1992; Campbell, 1999; LeFevre et al., 1996). For example, 8 × 9 takes longer to perform than 3 × 2. As such, single-digit items from the six-times table were the best candidates for the target problems because they represent the median multiplicand in terms of RT. Multiplication problems containing zero, one, five, or ties (e.g., 6 × 6 = __) were excluded as potential confounds, because they involve rule-based solutions rather than memory retrieval (Campbell, 1987), which was the emphasis of the target problems in this study. The remaining six items were used as target problems and are presented in Table 1.
The distracter conditions followed the same exclusionary rules applied to the test problems, giving six distracter problems in each condition (see Table 1). However, in the case of the double-digit addition and large multiplication conditions, random double-digit numbers greater than or equal to 13 were used (see Table 1 for examples).
Apparatus
The presentation of the experimental tasks and the collection of data were performed by SuperLab Pro Version 2.0 running on either an IBM ThinkPad i1400 laptop computer or a Compaq Presario 1200 laptop computer. A standard 101-key external keyboard was connected to the computer and used to capture participants’ responses.
Procedure
Participants were assigned to conditions as they volunteered: 20 per condition. They were instructed to complete a series of individually presented arithmetic problems as quickly and accurately as possible. After receiving instructions and 10 practice trials (comprised of problems from the five-times table), participants were presented with the training phase of the experiment. All problems were repeated 12 times and presented in a random order to give a total of 72 training trials. The target problems were presented one at a time as a set of six before being repeated again. The transfer phase contained the target problems from the training phase in addition to 72 other problems whose nature depended on the experimental condition to which the participant had been allocated. The new and old problems were presented in a random order.
In each trial, participants were initially presented with an individual problem in the center of the screen and instructed to press the space bar when they had formed the correct answer. Two possible solutions then appeared on either side of the computer screen; one was a correct response, while the other was a table-related error. Table-related errors are responses that are incorrect for the presented problem, but correct for another problem within the given multiplication table (e.g., a table-related error for 6 × 3 = __ would be 24, which corresponds to the answer for another problem in the six-times table). Presenting table-related errors ensured that participants generated, rather than verified, a solution (see Campbell, 1987; LeFevre et al., 1996; Zbrodoff & Logan, 2000). The position of correct answers was counterbalanced across trials between the left and right screen positions.
Participants nominated their response by pressing either the “z” key to select the option on the left side of the screen, or the “/” key to select the option on the right side of the screen. After making their selection, accuracy feedback was provided by presenting “Right” or “Wrong” in the center of the screen for 500 ms, after which the next trial commenced automatically. The transfer phase immediately followed the training phase.
Results
The data were analyzed in blocks of nine trials. This gave a total of eight blocks for the target problems in each phase, as well as eight blocks for distracter problems in the transfer phase, across all six conditions. Mean RT was defined as the elapsed time in milliseconds between initial problem presentation and the left or right button press response. Only correct responses were included in the RT analyses. Accuracy was assessed as the percentage of correct target problems in each block. RT analyses in both phases were mainly performed on the target problems, although one analysis was performed on the data collected on the distracter problems.
Accuracy on target problem performance remained high throughout the experiment (M = 97.64%, SE = 0.26%). A 6 (condition) × 16 (block) mixed-design ANOVA reported no effect of block or condition, demonstrating that accuracy remained constant in each condition and across all trials and was not influenced by the introduction of the distracter problems. This finding supports the study’s premise that the target problems reflect the retrieval of well-established facts from memory.
A 6 (condition) × 8 (block) mixed-design ANOVA performed on the training phase reported a significant effect of block F(7, 798) = 125.99, p < .05, with RT generally decreasing with subsequent trials (see Figure 1). There was no effect of condition, indicating that the rate at which RT decreased with subsequent trials was consistent across all conditions. A one-way ANOVA on the last block of the training phase was not significant, demonstrating that RT at the end of training was comparable across all six conditions. These analyses indicate that the participants in the conditions possessed an equivalent level of multiplication knowledge.
Comparison of observed (points) and predicted (solid lines in inset panel) reaction times for target problems in Experiment 1.
There was a high degree of fit between the observed training RTs and power functions derived for each condition, demonstrating that performance during training conformed to predictions based on the power law of learning. Parameters for these functions and measures of goodness of fit (r2 and root mean square deviation [RMSD]) are presented in Table 2.
Power Function Results for Each Condition in the Training Phase of Experiment 1.
Note. Training RMSD reflects how well the power function fits the training data. Transfer RMSD reflects how well the training power function predicts the transfer data. RMSD = root mean square deviation.
A 2 (phase) × 6 (condition) mixed-design ANOVA on RT for the last block of training and first block of transfer revealed a significant effect of phase, F(1, 114) = 67.96, p < .05, and a significant interaction between phase and condition, F(5, 114) = 4.37, p < .05 (see Figure 1). There was no effect of condition. Tukey’s post hoc analyses of the mean differences between the last block of training and first block of transfer demonstrated that the amount by which performance was slowed as a consequence of introducing the distracter problems was significant (p < .05) in the operation change (M = 361.30 ms), symbol change (M = 343.44 ms), double-digit addition (M = 382.41 ms), and large multiplication conditions (M = 439.40 ms). RT was not disrupted for the operand change (M = 51.44 ms) and operand reversal (M = 65.39 ms) conditions. Post hoc analyses on the interaction indicate that where it occurred, the extent of the disruption was not significantly different across conditions.
To assess the extent to which transfer performance after the distracter problems were introduced could be predicted from training performance, power functions derived from the training phase data were extrapolated a further eight blocks and compared with observed transfer RTs. Transfer performance was considered to have been reasonably predicted on the basis of training performance where extrapolated values passed within the 95% confidence intervals of the transfer RTs (see Figure 1). As with the previous analyses, these figures demonstrate that initial transfer performance was disrupted for the operation change, symbol change, large multiplication, and double-digit addition conditions. However, performance immediately returned to predicted levels in subsequent blocks in the operation change and symbol change conditions. In the case of the double-digit addition and large multiplication conditions, there was a prolonged disruption. Although this prolonged disruption was apparent for only the first two blocks of transfer for the large multiplication condition, it persisted until the final block of transfer for the double-digit addition condition. The poor fit between predicted and observed RTs in these four conditions provides further evidence of a performance disruption. This is indicated by the high RMSD values (see Table 2) in these conditions and represents greater deviation from the predicted values than was the case for the training data.
The analyses demonstrate a consistent disruption to predicted initial transfer performance on the introduction of the distracter problems. In the case of the operation change and symbol change conditions, transfer performance was in accordance with training phase predictions for the remainder of the transfer phase after the initial disruption. There are indications from the double-digit addition and large multiplication conditions, however, that changes in context that involve more than simply memory retrieval can induce a prolonged disruption to expected performance.
A series of t tests were performed on the RT data collected from the distracter problems in the first 10 blocks of the transfer phase. The aim of this analysis was to assess the extent of conceptual change represented by the distracter problems. The greater the conceptual difference between the target and distracter problems, the more likely it would be that RT for the target problems would be faster than for the distracter problems during transfer, especially given that participants had practiced the target problems during training. In the operand change and operand reversal conditions, the only blocks in which RT was significantly faster for the target problems were Blocks 1 and 2 (operand change) and Block 8 (operand reversal). RT for the target problems was far more consistently faster than for the distracter problems in the operation change (Blocks 1-3, 8-10), symbol change (Blocks 1, 3, 10), double-digit addition (Block 1-10), and large multiplication (Blocks 1-10) conditions. These results suggest that there was a far greater conceptual difference between the target and distracter problems in the latter four conditions than in the former two conditions.
Discussion
Experiment 1 demonstrated an immediate performance disruption similar to that noted by Speelman and Kirsner (2001) when the conceptual environment in which the established skills were presented was changed. Consistent with our predictions, changing the conceptual context of the target problems immediately increased RTs in the operation change, symbol change, double-digit addition, and large multiplication conditions. The results support the specific findings of Speelman and Kirsner in concluding that skill performance on a task can indeed be disrupted by the presence of a novel task. Examination of total transfer performance also revealed a prolonged disruption in the large multiplication condition and markedly so in the double-digit addition condition. These results pose a challenge for Anderson’s (1993; Anderson & Lebiere, 1998) ACT-R theory and Logan’s (1988) instance theory of automaticity, which predicted that performance in this situation should continue in accordance with power functions that describe training performance.
Another prediction examined in this study was that increasing the degree of conceptual change would lead to corresponding increases in disruption. Specifically, minimal conceptual change should result in complete transfer (i.e., no disruption to RTs), whereas substantial conceptual change should result in partial or zero transfer (i.e., substantial disruption to RTs). This premise was broadly supported—the introduction of the distracter problems in the operation change, symbol change, double-digit addition, and large multiplication conditions led to an immediate performance disruption, which was not present in the operand change and operand reversal conditions. The predicted degree of conceptual change involved in the transfer conditions was supported by the RT differences on target and distracter problems in the transfer phase, where little difference was observed in the operand change and operand reversal conditions, but a substantial and consistent advantage for target problems was found in the other conditions. Rather than an incremental disruption effect as predicted, however, the amount of conceptual change appeared only to induce a disruption without affecting its magnitude. These findings are consistent with Speelman and Kirsner’s (2001) study, which found that the size of the disruption was not a function of the amount of change in the perceived complexity of the task context. Consequently, it appears that once a threshold of conceptual change is exceeded, a disruption occurs, and that regardless of the extent of conceptual change, when this threshold is exceeded, the magnitude of the disruption remains constant. However, these findings apply only to the initial introduction of the distracter task—they do not generalize across the entire transfer phase because both temporary and prolonged disruptions were observed. This suggests that the extent of conceptual change cannot be used reliably to predict the degree of disruption.
This experiment clearly demonstrates that a change in the conceptual context of a task was sufficient to disrupt transfer performance in several of the conditions. The fact that a disruption resulted from a change in the conceptual context of the overall task in the absence of any change to the visual context of the target task suggests that this may also explain the disruption observed by Speelman and Kirsner (2001). Moreover, this result questions the assumption that if old skills can be executed in the context of new tasks, they will continue to improve as if nothing has changed. In this experiment, the visual appearance of a target problem was identical during the training and transfer phases. Performance accuracy was not affected by the change in task conditions, indicating that the knowledge used during training was also used during transfer. Yet, the application of old knowledge was disrupted during the transfer phase, because of the presence of the distracter problems. Thus, old skills may indeed be executed in new tasks, but the trajectory of their improvement in new tasks does not necessarily follow the trajectory of their past improvement history.
An alternate explanation of the results was proposed by two anonymous reviewers of an earlier version of this article. According to this suggestion, the time spent responding to the distracter problems in the transfer task meant there was time spent away from the target problems. The harder the distracter problem, the more time required to solve the problem, and so the more time spent away from the target problems. The greater the time spent away from performing target problems, the greater the chance that the skills acquired to solve the target problems would decay, with this decay leading to slower performance on the target problems. Certainly, the ACT-R theory has a decay parameter built into its account of the power law of learning, such that time not spent executing a skill results in a reduction of the strength of the skill and so contributes to a slower execution of that skill (Anderson, 1982; Anderson, Fincham, & Douglass, 1999). Thus, according to this explanation of the results of Experiment 1, the disruption observed in the transfer performance of the target problems was due to decay of old skills, rather than a direct effect of a change in context on the execution of the old skills. Delay between processing episodes has been shown to affect performance time of skilled behavior, although the delay that occurs as a result of spacing of the magnitude experienced in Experiment 1 was demonstrated to have little effect (Wilkins & Rawson, 2010). Furthermore, there was no disruption evident in the operand change and operand reversal conditions despite this apparent spacing design feature being present in those conditions too. Nonetheless, Experiment 2 was performed to examine this alternate explanation of the results of Experiment 1.
Experiment 2
A new task was developed to avoid a feature of the design of Experiment 1 whereby the change of task context in the transfer phase introduced a spacing effect (i.e., distracter problems were interspersed between target problems). Each trial in Experiment 2 had two parts. The first part was identical to the problems presented in the training phase of Experiment 1. That is, participants were presented with one of the six-times target problems, and they were asked to pick the correct solution from the two solutions on the screen. In the second part of each trial, they were asked to do something with the solution from the first part. In one phase of the experiment, they were asked to add a number (e.g., two) to the solution. In the other phase, they were asked to subtract a number from the solution. This change from addition to subtraction (or vice versa) in the second part of each trial constituted the change in context from training to transfer. Thus, the new version of the task retained the feature of Experiment 1 whereby old problems (i.e., the six-times problems) were presented in both phases of the experiment in an identical manner, and so conceivably could be solved in an identical manner. The new feature of the task in Experiment 2 (i.e., the second part of each trial) ensured that the spacing feature of the transfer phase in Experiment 1 was present in both phases of Experiment 2. In this way, the change in context and the imposition of a spacing effect were disentangled; the spacing feature was present in both phases and so could not explain any disruption in performance of the six-times problems associated with the change in context.
Method
Participants
The sample consisted of 120 participants, with an age range of 17 to 65 years, and a mean age of 30.86 years (SD = 15.24 years). There were 77 females (M = 29.84 years, SD = 12.34 years) and 43 males (M = 32.77 years, SD = 13.85 years). Fifty-two participants were recruited from the Edith Cowan University (ECU) School of Psychology volunteers list, and 68 participants were recruited from the general public. All participants were offered a ticket in a raffle for a AUS$50 case prize as an incentive for participating. The data from nine participants were not included in the analysis because they did not reach the accuracy criterion of at least 80% in the training phase.
Materials
The six items that were used as target problems in Experiment 1 were used for the same purpose in Experiment 2 (see Table 1). These problems were presented in the first part of each trial, in both the training and transfer phases.
In the second part of each trial, participants were either asked to add or subtract a single-digit number to/from the solution of the first part. These numbers were selected from the integers two to seven. Each of the six-times problems in the first part of a trial was paired with a particular digit in the second part at most 3 times across the experiment.
Apparatus
The presentation of the experimental tasks and the collection of data were performed by SuperLab Pro Version 1.74 running on Macintosh G4 computers. A standard 101-key external keyboard was connected to the computer and used to capture participants’ responses.
Procedure
After receiving instructions and 10 practice trials (comprised of problems from the five-times table), participants were presented with the training phase of the experiment. In the training phase, there were 12 blocks of six trials. The six trials of each block consisted of the six-times problems (Part 1 of each trial) followed by an addition or subtraction problem (Part 2). The trials within each block were presented in a random order. The transfer phase consisted of the same problems as in the training phase, except Part 2 of each trial was different. If a participant had added numbers in Part 2 of their training trials, in transfer they would be asked to subtract numbers. The reverse occurred if a participant subtracted numbers in training. This feature of the design was counterbalanced across participants. The transfer phase consisted of two blocks of six trials (i.e., each of the six-times problems was presented twice).
As in Experiment 1, participants nominated their response in Part 1 of each trial by pressing either the “z” key to select the option on the left side of the screen, or the “/” key to select the option on the right side of the screen. After making their selection, accuracy feedback was provided as part of the instructions for what to do in Part 2 of the trial (see Figure 2). This message was also displayed if the participant made an error on Part 1, and so enabled them to continue to respond to Part 2 in the normal way. Participants responded to Part 2 of the trial using the “z” and “/” keys, and a new screen provided accuracy feedback on that response. Pressing the space bar on the keyboard then initiated the next trial.
Sequence of stimuli presented in a trial in Experiment 2.
In both parts of a trial, positioning of solution options on the screen was balanced within a block, in that for each block the correct option was presented on the left and right sides of the screen for an equal number of the trials. In addition, as both a correct and incorrect option were presented together, whether the incorrect option was less than or greater than the correct option was balanced across trials.
The transfer phase immediately followed the training phase, and participants were given no warning that Part 2 of each trial in transfer would be different to what was experienced during training.
Results
The data were analyzed in blocks of six trials. This gave a total of 12 blocks in the training phase, and two blocks in the transfer phase. Mean RT was defined as the elapsed time between the presentation of Part 1 of a trial and the left or right button press response to the Part 1 problem. Only correct responses were included in the RT analyses. RT analyses in both phases were performed on the target (Part 1) problems only.
There was a high degree of fit between the observed training RTs and the best-fit power function derived to describe the improvement observed in RT, demonstrating that group performance during training conformed to predictions based on the power law of learning. Parameters for these functions and measures of goodness of fit (r2 and RMSD) are presented in Table 3.
Power Function Results in the Training Phase of Experiment 2.
Note. RMSD = root mean square deviation.
The best-fit power function was extrapolated two blocks to predict RT for transfer performance based on the assumption that performance would be unaffected by the change in Part 2 of each trial of this phase. The observed mean RT for both phases and the power function values are presented in Figure 3. It is clear in this figure that the performance of the participants improved substantially during the training phase, but was slowed in the first block of transfer compared with the performance predicted by the power function. In the second transfer block, however, performance had returned to the level predicted by the power function.
Mean reaction times on Part 1 of each trial, averaged across block, in the training and transfer phases of Experiment 2.
The fact that transfer performance quickly returned to a level predicted by extrapolating training performance indicates that the disruption in performance created by changing the nature of Part 2 of each task was only short-lived. To determine the time course of this disruption, the mean RT for each trial in the transfer phase was examined. As can be seen in Figure 4, the only indication of a significant slowing in performance was in Trials 2 and 3 of the transfer phase. That Trial 1 performance was unaffected by the change in context reflects the fact that at the time when participants were solving Part 1 of Transfer Trial 1, the task context had not changed. It was only after participants responded to Part 1 of this trial that they were then presented with the change to Part 2. From Transfer Trial 4 until the end of the transfer phase, performance looks to have returned to pre-disruption levels. Thus, the disruption in the first block of transfer that is obvious in Figure 3 is mainly due to the long RTs in Transfer Trials 2 and 3.
Mean reaction times on Part 1 of each trial in the transfer phase of Experiment 2.
Discussion
A transfer disruption similar to those observed in Experiment 1 was also observed in Experiment 2. This result occurred despite the spacing feature that existed in the transfer phase of Experiment 1 being absent in Experiment 2. Thus, the transfer disruption observed in Experiment 2 cannot be explained by a spacing effect. Furthermore, this result weakens the claim that the transfer disruptions observed in Experiment 1 were due to a spacing effect.
The transfer disruption that was observed in Experiment 2 was very short-lived. Because of the design of the task, the earliest that a transfer disruption could be observed was in the second trial of the transfer phase. The results indicate that it was only in this trial and the next trial that a disruption occurred.
General Discussion
The effect of a change in conceptual context on transfer raises at least two questions. First, what is the mechanism whereby a change in conceptual context can disrupt transfer? Second, why are there two types of disruption, temporary and prolonged? Some possible answers to these questions come from research that examines performance in conditions that require people to apply multiple processing rules.
Research supports the existence of a processing overhead when multiple processing rules are executed in a random order, in comparison with when a single rule is executed in a consistent order (e.g., Carlson & Shin, 1996; Carlson & Yaure, 1990). Under consistent task conditions, the presence of consistent cues aids performance by enabling the processing requirements of the task to be anticipated and loaded into working memory, facilitating an efficient response (Carlson & Yaure, 1990). Müller (1999) claimed that people strive for efficient forms of responding, and he suggests that there is a natural tendency to minimize the processing effort to master a task using cues presented in the environment and that people attempt to search for and engage in strategies that thus improve processing efficiency. Wenger and Carlson (1996) found evidence to indicate that encountering consistent task conditions leads to task sequences being pre-processed before the corresponding information is actually displayed. This is a view shared by Gopher et al. (2000), who suggest that the ability to reduce the cost of shifting from one task to another can be overcome with advanced preparation and having the information already available in working memory. This view suggests that the task requirements can be pre-empted by allowing for the anticipation of more problems of a similar nature (see Carlson & Yaure, 1990; Wenger & Carlson, 1996). In the present study, the training conditions could have encouraged the development of a mental set, which may have involved an expectation of further problems of a similar type. As a result, participants could have anticipated the presentation of each problem by pre-processing components of each problem’s solution, so making them available in working memory prior to the problem being presented (see Woltz et al., 1996). The type of disruption that resulted in transfer can then be understood in terms of a disruption to the mental set that was caused by the presentation of the distracter items. The fact that there was no disruption in the operand change and operand reversal conditions of Experiment 1 suggests that “surprise” at a change in the task context cannot alone be responsible for the disruptions observed in the other conditions. This also supports the proposition that the participants pre-empted the upcoming task by having the processing rules already available in working memory. Furthermore, an element of surprise would not account for the differential disruptions noted between conditions across transfer blocks.
A mental set developed in the context of the training problems involving an expectation that all problems would be simple multiplication problems would only have been applicable to the distracter problems in the operand change and operand reversal conditions of Experiment 1. Certainly, other work suggests that the distracter problems in these conditions would be strongly associated with the corresponding target problems and so would utilize the same knowledge structures (Rickard, 2005). The degree of conceptual similarity between the target and distracter problems in these conditions would probably not have prompted a re-assessment of the task conditions, enabling continued application of pre-processing. As a result, no disruption would be expected in these conditions, and nor was one observed.
The distracter problems presented in the operation change, symbol change, double-digit addition, and large multiplication conditions of Experiment 1 would have required the adoption of strategic processing in the transfer phase. As such, the mental set developed in the training phase would not have been applicable during transfer, prompting a re-assessment of the task requirements. This accounts for the immediate disruption to the target problems observed in the operation change, symbol change, double-digit addition, and large multiplication conditions and is consistent with the a priori predictions that such a disruption should only have occurred in these four conditions.
In the case of the symbol change and operation change conditions, performance returned to predicted levels after the initial introduction of the distracter problems. Participants may quickly have realized that the problems were simply variants of multiplication problems (Campbell, 1999), although this could have taken longer to achieve in the symbol change condition because of the need to switch between × and ÷ symbols throughout the transfer phase. Eventually, in both conditions, it would have been appropriate to engage in the same pre-processing strategies for both the target and distracter problems. It is likely that something similar occurred in Experiment 2, where a change from addition to subtraction (and vice versa) of single digits would require some degree of cognitive reconceptualization for most adults, albeit a small one. In the large multiplication problems of Experiment 1, however, performance was disrupted for the first two blocks of trials of the transfer phase. In this condition, it would have been necessary for participants to apply multiplication rules for the initial stages of a problem, followed by addition rules to arrive at the final answer. The need to coordinate multiplication and addition rules may have meant that the advantages of a pre-processing strategy may not have been apparent as quickly as with the symbol change and operation change conditions. With further practice, participants may have learned that applying multiplication rules was a pre-processing strategy that would enable an efficient response because it would at least initiate a problem solution for both the distracter and target problems. As such, this pre-processing strategy would have been re-engaged to some extent, thus overcoming the disruption, albeit more slowly than in other conditions. In the case of the double-digit addition condition, performance remained disrupted for all except the last block of the transfer phase. The prolonged disruption may be best explained by participants not being able to develop a single pre-processing strategy, because the distracter and test problems required entirely different processing rules. The random presentation of these problems meant that the corresponding rules would have been continually loaded and unloaded from working memory, resulting in the prolonged disruption that was observed.
The present study’s examination of the nature of the disruption revealed that if the novel tasks are such that their influence on old tasks extends beyond the point of their introduction, it is possible for a disruption to occur to performance on the old tasks. Whether an immediate or a prolonged disruption occurs, or both, seems to depend on the qualities of the distracter task used. Where the distracter problems changed the apparent nature of the task requirements, an immediate disruption was observed; where the distracter problems increased the scope of knowledge on which participants had to draw to successfully complete the task, a prolonged disruption occurred. The nature of the task was changed in the case of the operation change, symbol change, double-digit addition, and large multiplication conditions of Experiment 1, and in Experiment 2, because participants needed to apply different strategies to solve the distracter problems rather than simply retrieving multiplication table knowledge.
In the double-digit addition and large multiplication conditions, the scope of knowledge that was required to successfully respond to all tasks needed to include not only knowledge of the six-times table but also the application of algorithms, such as being able to hold the results of subsidiary calculations in memory, perform further calculation, and combine them to arrive at an overall result. This would have increased the burden on working memory, thus introducing an overhead into the overall response process, so leading to a prolonged disruption. In comparison, the nature and scope of the task requirements were changed to a far smaller degree in the case of the operand change and operand reversal conditions, and so neither an immediate nor a prolonged disruption would be expected, which was the case.
Conclusion
This research sought to evaluate whether the power law can be used as a basis for predicting future performance by extrapolating a power function describing past performance. It was found that changing the conceptual context of a task results in a transfer disruption, such that extrapolating training performance underestimates transfer performance times. This result replicates the transfer disruption reported by Speelman and Kirsner (2001) and has clarified the nature of the disruption by demonstrating that the automatic, reflex-like nature of robust skills can apparently be easily disrupted by minor and subtle changes to the context within which these skills are executed (see also, Dishon-Berkovits & Algom, 2000). The current results and those of Speelman and Kirsner provide a challenge to the assumption implicit in theories such as Anderson’s (1993; Anderson & Lebiere, 1998) ACT-R and Logan’s (1988) instance theory that skill performance should continue to improve in accordance with learning curves that describe training performance.
It is important to emphasize, however, that in most conditions of Experiment 1 and Experiment 2, recovery from the transfer disruption was rapid. One interpretation of this result is that component skills are indeed robust in the face of context changes associated with transfer and that it is some meta-level processing involved with adapting to the change in task environment that is the underlying cause of increased performance times early in transfer. This interpretation suggests that the transfer of skills involves the recruitment of established component processes that can apply in the new situation, and the execution time of these processes reflects their recent learning histories, but the recruitment process can impose an overhead on overall performance time (Speelman & Kirsner, 2005; Taatgen, 2013). Although the mechanism underlying this recruitment process is unclear at present, the transfer disruptions reported here and in previous research provide an important constraint for theories of skill acquisition and transfer, particularly because it is unlikely that skills can ever be applied in isolation to any conceptual influences.
Footnotes
This research was reported at the 2013 Annual Meeting of the Psychonomics Society in Toronto, November 2013.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research and/or authorship of this article.