The objective was to conduct research synthesis for the U.S. Army on the effectiveness of two error prevention training strategies (training wheels and scaffolding) on the transfer of training.
Background:
Motivated as part of an ongoing program of research on training effectiveness, the current work presents some of the program’s research into the effects on transfer of error prevention strategies during training from a cognitive load perspective. Based on cognitive load theory, two training strategies were hypothesized to reduce intrinsic load by supporting learners early in acquisition during schema development.
Method:
A transfer ratio and Hedges’ g were used in the two meta-analyses conducted on transfer studies employing the two training strategies. Moderators relevant to cognitive load theory and specific to the implemented strategies were examined. The transfer ratio was the ratio of treatment transfer performance to control transfer. Hedges’ g was used in comparing treatment and control group standardized mean differences. Both effect sizes were analyzed with versions of sample weighted fixed effect models.
Results:
Analysis of the training wheels strategy suggests a transfer benefit. The observed benefit was strongest when the training wheels were a worked example coupled with a principle-based prompt. Analysis of the scaffolding data also suggests a transfer benefit for the strategy.
Conclusion:
Both training wheels and scaffolding demonstrated positive transfer as training strategies. As error prevention techniques, both support the intrinsic load–reducing implications of cognitive load theory.
Application:
The findings are applicable to the development of instructional design guidelines in professional skill-based organizations such as the military.
*AzevedoR.CromleyJ. G.SeibertD. (2004). Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia?Contemporary Educational Psychology, 29, 344–370. doi:10.1016/j.cedpsych.2003.09.002
2.
*AzevedoR.CromleyJ. G.WintersF. I.MoosD. C.GreeneJ. A. (2005). Adaptive human scaffolding facilitates adolescents’ self-regulated learning with hypermedia. Instructional Science, 33, 381–412. doi:10.1007/s11251-005-1273-8
3.
*AzevedoR.SeibertD.GuthrieJ. T.CromleyJ. G.WangH. Y.TronM. (2002, April). How do students regulate their learning of complex systems with hypermedia? Paper presented at the meeting of the American Educational Research Association, New Orleans, LA.
4.
*BannertM. (2000). The effects of training wheels and self-learning materials in software training. Journal of Computer Assisted Learning, 16, 336–346.
5.
BerkR. (2007a). The powerful seductions alchemy. Journal of Experimental Criminology, 3, 293–297.
6.
BerkR. (2007b). Statistical inference and meta-analysis. Journal of Experimental Criminology, 3, 247–270.
7.
*BixlerB. A. (2007). The effects of scaffolding student’s problem-solving process via question prompts on problem solving and intrinsic motivation in an online learning environment (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
8.
BorensteinM.HedgesL.HigginsJ.RothsteinH. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley.
9.
BriggsD. C. (2005). Meta-analysis: A case study. Evaluation Review, 29, 87–127.
10.
CarolanT.McDermottP. L.HutchinsS.WickensC. D.BelanichJ. (2011, November-December). Investigating the impact of training on performance. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, FL.
11.
*CarrollJ. M. (1984). Blocking learner error states in a training-wheels system. Human Factors, 26, 377–389.
12.
CarrollJ. M. (1990). The Nurnberg funnel: Designing minimalist instruction for practical computer skills. Cambridge, MA: MIT.
13.
CarrollJ. M.CarrithersC. (1984a). Blocking learner error states in a training-wheels system. Human Factors, 26, 377–389.
14.
CarrollJ. M.CarrithersC. (1984b). Training wheels in a user interface. Communications of the ACM, 27, 800–806.
15.
*CarrollW. M. (1994). Using worked examples as an instructional support in the algebra classroom. Journal of Educational Psychology, 86, 360–367.
16.
*ChenC. H.BradshawA. C. (2007). The effect of web-based question prompts on scaffolding knowledge integration and ill-structured problem solving. Journal of Research on Technology in Education, 39, 359–375.
17.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
18.
CraikF. I. M.LockhartR. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671–684.
19.
*CrippenK. J.EarlB. L. (2007). The impact of web-based worked examples and self-explanation on performance, problem solving, and self-efficacy. Computers & Education, 49, 809–821.
20.
GlassG. V. (1999, July). Meta-analysis at 25. Paper presented to Office of Special Education Programs Research Project Directors’ Conference, U.S. Department of Education, Washington DC.
21.
*HalabiA. K.TuovinenJ. E.FarleyA. A. (2005). Empirical evidence on the relative efficiency of worked examples versus problem-solving exercises in accounting principles instruction. Issues in Accounting Education, 20, 21–32.
22.
HilbertT. S.RenklA. (2009). Learning how to use a computer-based concept-mapping tool: Self-explaining examples helps. Computers in Human Behavior, 25, 267–274.
23.
*HilbertT. S.RenklA.KesslerS.ReissK. (2008). Learning to prove in geometry: Learning from heuristic examples and how it can be supported. Learning and Instruction, 18, 54–65.
24.
*HohnR. L.MoraesI. (1997-1998). Use of rule-based elaboration of worked examples to promote the acquisition of programming plans. Journal of Computer Information Systems, 35–40.
25.
KalyugaS. (2011). Cognitive load theory: How many types of load does it really need?Educational Psychology Review, 23, 1–19.
26.
*KaoM. T.LehmanJ. D. (1997, March). Scaffolding in a computer-based constructivist environment for teaching statistics to college learners. Paper presented at the meeting of the American Educational Research Association, Chicago, IL.
27.
KeithN.FreseM. (2008). Effectiveness of error management training: A meta-analysis. Journal of Applied Psychology, 93, 59–69.
28.
KirschnerF.KesterL.CorbalanG. (2011). Cognitive load theory and multimedia learning, task characteristics and learning engagement: The current state of the art. Computers in Human Behavior, 27, 1–4.
29.
KirschnerP. A.AyresP.ChandlerP. (2011). Contemporary cognitive load theory research: The good, the bad and the ugly. Computers in Human Behavior, 27, 99–105.
30.
*LangJ. (2007). The effect of presenting worked examples for problem solving in a computer game (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
31.
*LeutnerD. (2000). Double-fading support: A training approach to complex software systems. Journal of Computer Assisted Learning, 16, 347–357.
32.
LipseyM. (2007). Unjustified inferences about meta-analysis. Journal of Experimental Criminology, 3, 271–279.
33.
LipseyM.WilsonD. (2001). Practical meta-analysis. Applied Social Research Methods series (Vol. 49). Thousand Oaks, CA: Sage.
34.
*LoringD. H. (2003). Effects of worked examples and algebra problem solving skill on error and cognitive load (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
35.
LuS.WickensC. D.SarterN.SebokA. (2011). Informing the design of multimodal displays: A meta-analysis of empirical studies comparing auditory and tactile interruptions. In Proceedings of the Human Factors and Ergonomics Society 55th Annual Meeting (pp. 1170–1174). Santa Monica, CA: Human Factors and Ergonomics Society.
36.
*MahanW. L. (2007). Effect of prior knowledge and principle-based and procedure-based instructional strategies on students’ transfer of principles (Master’s thesis). Retrieved from ProQuest Digital Dissertation and Theses.
37.
*McLarenB. M.LimS.KoedingerK. R. (2008). When and how often should worked examples be given to students? New results and a summary of the current state of research. In LoveB. C.McRaeK.SloutskyV. M. (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 2176–2181). Austin, TX: Cognitive Science Society.
38.
*MoraesI. (1995). Use of rule-based elaboration of worked examples to promote the acquisition of programming plans (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
39.
*OndrusekA. L. (1999). The effects of varied problem-solving models in facilitating knowledge acquisition in online searching (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
40.
PaasF.RenklA.SwellerJ. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1–4.
41.
PaasF.van GogT. (2006). Optimising worked example instruction: Different ways to increase germane cognitive load. Learning and Instruction, 16, 87–91.
42.
PaasF.van GogT. (2009). Principles for designing effective and efficient training for complex cognitive skills. In DursoF. (Ed.), Reviews of human factors and ergonomics (Vol. 5, pp. 166–194). Santa Monica, CA: Human Factors and Ergonomics Society.
43.
PaasF.van GogT.SwellerJ. (2010). Cognitive load theory: New conceptualizations, specifications, and integrated research perspectives. Educational Psychology Review, 22(2), 115–121.
44.
*PaasF. G. W.van MerriënboerJ. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.
45.
PeaR. D. (2004). The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. Journal of the Learning Sciences, 13, 423–451.
46.
*PetsangsriS. (2002). The effects of embedded scaffolding strategy in a cognitive flexibility-based computer learning environment. In Proceedings of the International Conference on Computers in Education (pp. 75–79). Washington, DC: IEEE.
47.
*PodolefskyN. S. (2008). Analogical scaffolding: Making meaning in physics through representation and analogy (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
48.
ReyG.BuchwaldF. (2011). The expertise reversal effect: Cognitive load and motivational explanations. Journal of Experimental Psychology: Applied, 17, 33–48.
49.
RoedigerH. L.KarpickeJ. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210.
50.
RosenthalR. (1991). Meta-analytic procedures for social research (Rev. ed.). Beverly Hills, CA: Sage.
51.
*RourkeA.SwellerJ. (2009). The worked-example effect using ill-defined problems: Learning to recognize designers’ styles. Learning and Instruction, 19, 185–199.
52.
SchneiderW. (1985). Training high-performance skills: Fallacies and guidelines. Human Factors, 27, 285–300.
53.
ShadishW. (2007). A world without meta-analysis. Journal of Experimental Criminology, 3, 281–291.
54.
*ShenC. Y.O’NeilH. (2006, April). The effectiveness of worked examples in a game-based learning environment. Paper presented at the meeting of the American Educational Research Association, San Francisco, CA.
55.
ShenC.-Y.TsaiH.-C. (2009). Design principles of worked examples: A review of the empirical studies. Journal of Instructional Psychology, 36, 238–244.
56.
SitzmannT.ElyK.BellB. S.BauerK. (2010). The effects of technical difficulties on learning and attrition during online training. Journal of Experimental Psychology: Applied, 16, 281–292.
57.
SlameckaN. J.GrafP. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604.
58.
SwellerJ. (1988). Cognitive load during problem-solving: Effects on learning. Cognitive Science, 12, 257–285.
59.
*TookeyK. R. (1994). Arithmetic schema and strategy instruction (Doctoral dissertation). Retrieved from ProQuest Digital Dissertation and Theses.
60.
*Van GervenP. W. M.PaasF. G. W. C.van MerriënboerJ. J. G.SchmidtH. G. (2002). Cognitive load theory and aging: Effects of worked examples on training efficiency. Learning and Instruction, 12, 87–105.
61.
*van GogT.JarodzkaH.ScheiterK.GerjetsP.PaasF. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25, 785–791.
62.
van GogT.PaasF.SwellerJ. (2010). Cognitive load theory: Advances in research on worked examples, animations, and cognitive load measurement. Educational Psychology Review, 22, 375–378.
63.
*van GogT.PaasF.van MerriënboerJ. J. G. (2006). Effects of process-oriented worked examples on troubleshooting transfer performance. Learning and Instruction, 16, 154–164.
64.
van GogT.RummelN. (2010). Example-based learning: Integrating cognitive and social-cognitive research perspectives. Educational Psychology Review, 22(2), 155–174.
65.
van MerriënboerJ. J. G.SwellerJ. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17, 147–177.
66.
VanLehnK. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46, 197–221.
67.
*WardM.SwellerJ. (1990). Structuring effective worked examples. Cognition and Instruction, 7, 1–39.
68.
WeinsteinY.McDermottK. B.RoedigerH. L. (2010). A comparison of study strategies for passages: Rereading, answering questions, and generating questions. Journal of Experimental Psychology: Applied, 16, 308–316.
69.
WickensC. D. (1998). Common sense statistics. Ergonomics in Design, 6(4), 18–22.
70.
WickensC. D.HutchinsS.CarolanT.CummingJ. (2011). Investigating the impact of training on transfer: A meta-analytic approach. In Proceedings of the Human Factors and Ergonomics Society 55th Annual Meeting (pp. 2138–2142). Santa Monica, CA: Human Factors and Ergonomics Society.
71.
WickensC. D.HutchinsS.CarolanT.CummingJ. (2012a). Attention and cognitive resource load in training strategies. In HealyA. F.BourneL. E. Jr. (Eds.), Training cognition: Optimizing efficiency, durability, and generalizability (pp. 67–88). New York, NY: Psychology Press.
72.
WickensC. D.HutchinsS.CarolanT.CummingJ. (2012b). Effectiveness of part task training and increasing difficulty training strategies: A meta-analysis approach. Human Factors. Advance online publication.
73.
WickensC. D.PrinetJ.HutchinsS.SarterN.SebokA. (2011). Auditory-visual redundancy in vehicle control interruptions: Two meta-analyses. In Proceedings of the Human Factors and Ergonomics Society 55th Annual Meeting (pp. 1155–1159). Santa Monica, CA: Human Factors and Ergonomics Society.
74.
WightmanD.LinternG. (1985). Part task training for tracking and manual control. Human Factors, 27, 267–284.
