While virtual, augmented, and mixed reality technologies are being used for military medical training and beyond, these component technologies are oftentimes utilized in isolation. eXtended Reality (XR) combines these immersive form factors to support a continuum of virtual training capabilities to include full immersion, augmented overlays that provide multimodal cues to personalize instruction, and physical models to support embodiment and practice of psychomotor skills. When combined, XR technologies provide a multi-faceted training paradigm in which the whole is greater than the sum of the constituent capabilities in isolation. When XR applications are adaptive, and thus vary operational stressors, complexity, learner assistance, and fidelity as a function of trainee proficiency, substantial gains in training efficacy are expected. This paper describes a continuum of XR technologies and how they can be coupled with numerous adaptation strategies and supportive artificial intelligence (AI) techniques to realize personalized, competency-based training solutions that accelerate time to proficiency. Application of this training continuum is demonstrated through a Tactical Combat Casualty Care training use case. Such AI-enabled XR training solutions have the potential to support the military in meeting their growing training demands across military domains and applications, and to provide the right training at the right time.
Deployed Medicine. Tactical combat casualty care collection. Joint Trauma System (JTS)/Committee on Tactical Combat Casualty Care (CoTCCC), https://deployedmedicine.com/market/11 (2019, accessed January 2020).
2.
Office of the Under Secretary of Defense. DoD instruction 1322.24: medical readiness training (MRT). Department of Defense, https://www.deployedmedicine.com/market/11 (2018, accessed January 2020).
3.
MilgramPKishinoF.A taxonomy of mixed reality visual displays. IEICE T Inf Syst1994; 77: 1321–1329.
4.
Angel-UrdinolaDFCastillo-CastroCHoyosA. Meta-analysis assessing the effects of virtual reality training on student learning and skills development. Policy Research Working paper no. 9587, March2021. Education Global Practice, World Bank Group, https://openknowledge.worldbank.org/handle/10986/35299
5.
KyawBMSaxenaNPosadzkiP, et al. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res2019; 21: e12959.
6.
HaqueSSrinivasanS.A meta-analysis of the training effectiveness of virtual reality surgical simulators. IEEE T Inf Technol B2006; 10: 51–58.
7.
GarzónJAcevedoJ.Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educ Res Rev2019; 27: 244–260.
8.
CohenJ.Statistical power analysis for the behavioral sciences. 2nd ed.Mahwah, NJ: Lawrence Erlbaum Associates, 1988.
9.
Cannon-BowersJABowersCProcciK.Optimizing learning in surgical simulations: guidelines from the science of learning and human performance. Surg Clin North Am2010; 90: 583–603.
10.
CohnJVStanneyKMDarkenRP, et al. Training evaluation of virtual environments. In: BakerELDickiesonJWulfeckW, et al. (eds) Assessment of problem solving using simulations. Mahwah, NJ: Lawrence Erlbaum Associates, 2007, pp. 81–105.
11.
DreyfusSE.The five-stage model of adult skill acquisition. B Sci Technol Soc2004; 24: 177–181.
12.
MayerREMorenoR.Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during multimedia learning. In: PlassJLMorenoRBrunkenR (eds) Cognitive load theory. New York: Cambridge University Press, 2010, pp. 131–153.
13.
SpeicherMHallBDNebelingM.What is mixed reality?. In: Proceedings of the 2019 CHI conference on human factors in computing systems (CHI’2019), Glasgow, 4–9 May 2019. New York: ACM.
14.
ParongJMayerRE.Cognitive and affective processes for learning science in immersive virtual reality. J Comput Assist Lear2021; 37: 226–241.
15.
CassavaughNDKramerAF.Transfer of computer-based training to simulated driving in older adults. Appl Ergon2009; 40: 943–952.
16.
Johnson-GlenbergMC.Immersive VR and education: embodied design principles that include gesture and hand controls. Front Robot AI2018; 5: 81.
17.
CisekPKalaskaJF.Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci2010; 33: 269–298.
18.
RochlenLRLevineRTaitAR.First-person point-of-view-augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc2017; 12: 57–62.
19.
ThompsonPHudsonAJ.Getting “SMART” on shock treatment: an evidence-based mnemonic acronym for the initial management of hemorrhage in trauma. J Spec Oper Med2019; 19: 62–65.
20.
RyokaiKAgoginoA.Off the paved paths: exploring nature with a mobile augmented reality learning tool. Int J Mobile Hum Comput Interact2013; 5: 21–49.
21.
KochmarEVuDDBelferR, et al. Automated personalized feedback improves learning gains in an intelligent tutoring system. In: BittencourtIICukurovaMMuldnerK, et al. (eds) Artificial intelligence in education (AIED2020), vol. 12164 (Lecture notes in computer science). Cham: Springer, 2020, pp. 140–146.
22.
LevacDEHuberMESternadD.Learning and transfer of complex motor skills in virtual reality: a perspective review. J Neuroeng Rehabil2019; 16: 121.
23.
AndreattaPBHillardMKrainLP.The impact of stress factors in simulation-based laparoscopic training. Surgery2010; 147: 631–639.
24.
TonhäuserCBükerL.Determinants of transfer of training: a comprehensive literature review. Int J Res Vocat Educ Train2016; 3: 127–165.
25.
AggarwalRGrantcharovTMoorthyK, et al. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg2006; 191: 128–133.
26.
SharmaNDohertyIDongC.Adaptive learning in medical education: the final piece of technology enhanced learning?Ulster Med J2017; 86: 198–200.
27.
PerezRSSkinnerAChatelierP.Lessons learned from intelligent tutoring research for simulation. In: O’NeilHFBakerELPerezRS (eds) Using games and simulations for teaching and assessment: key issues. New York: Routledge/Taylor & Francis, 2016, pp. 308–337.
28.
TaylorDLYeungMBashetAZ.Personalized and adaptive learning. In: RyooJWinkelmannK (eds) Innovative learning environments in STEM higher education (SpringerBriefs in statistics). Cham: Springer, 2021, pp. 17–34.
FletcherJD.What do sharable instructional objects have to do with intelligent tutoring systems, and vice versa? IDA document no. D-2661, 1September2001. Alexandria, VA: Institute for Defense Analyses.
31.
BloomBS.The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educ Researcher1984; 13: 4–16.
32.
DurlachPJRayJM.Designing adaptive instructional environments: insights from empirical evidence. Technical report no. 1297, 1October2011. Orlando, FL: United States Army Research Institute for the Behavioral and Social Sciences.
33.
GoldbergBSSinatraASottilareR, et al. Instructional management for adaptive training and education in support of the US Army learning model–research outline. Technical report no. ARL-SR-0345, November2015. Aberdeen Proving Ground, MD: United States Army Research Laboratory.
34.
BernardRMBorokhovskiESchmidRM, et al. Twenty-first century adaptive teaching and individualized learning operationalized as specific blends of student-centered instructional events: a systematic review and meta-analysis. Campbell Syst Rev2019; 15: e1017.
35.
AlegreFMolinerLMarotoA, et al. Peer tutoring and mathematics in secondary education: literature review, effect sizes, moderators, and implications for practice. Heliyon2019; 5: e02491.
36.
VanLehnK.The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ Psychol2011; 46: 197–221.
37.
XuZWijekumarKRamirezG, et al. The effectiveness of intelligent tutoring systems on K-12 students’ reading comprehension: a meta-analysis. Br J Educ Technol2019; 50: 3119–3137.
KoedingerKRCorbettACPerfettiC.The Knowledge-Learning-Instruction (KLI) framework: bridging the science-practice chasm to enhance robust student learning. Cogn Sci2012; 36: 757–798.
40.
SiuKCBestBJKimJW, et al. Adaptive virtual reality training to optimize military medical skills acquisition and retention. Mil Med2016; 181: 214–220.
41.
ReissleinJReissleinMSeelingP.Comparing static fading with adaptive fading to independent problem solving: the impact on the achievement and attitudes of high school students learning electrical circuit analysis. J Eng Educ2006; 95: 217–226.
42.
NicolDJMacfarlane-DickD.Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Stud High Educ2006; 31: 199–218.
43.
TangKSChengDLMiE, et al. Augmented reality in medical education: a systematic review. Can Med Educ J2020; 11: e81–e96.
44.
KularbphettongKKedsiributPRoonrakwitP.Developing an adaptive web-based intelligent tutoring system using mastery learning technique. Procd Soc Behv2015; 191: 686–691.
45.
MirchiNBissonnetteVYilmazR, et al. The Virtual Operative Assistant: an explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLoS One2020; 15: e0229596.
46.
MarraffinoMDSchroederBLFrauliniNW, et al. Adapting training in real time: an empirical test of adaptive difficulty schedules. Mil Psychol2021; 33: 136–151.
47.
PelánekR.Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Model User: Adap2017; 27: 313–350.
48.
FancsaliSENixonTRitterS. Optimal and worst-case performance of mastery learning assessment with Bayesian knowledge tracing. In: Proceedings of the 6th international conference on educational data mining (ed D’MelloSKCalvoRAOlneyA), Memphis, TN: International Educational Data Mining Society, 6–9July2013, pp. 33–42.
49.
ZhangJShiXKingI, et al. Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th international World Wide Web conference committee (IW3C2), Perth, WA, Australia, 3–7 April 2017, pp. 765–774. New York: ACM.
50.
SapountziABhulaiSCorneliszI, et al. Dynamic knowledge tracing models for large-scale adaptive learning environments. Int J Adv Intel Syst2019; 12: 93–110.
51.
BisgaardCHRubakSLMRodtSA, et al. The effects of graduate competency-based education and mastery learning on patient care and return on investment: a narrative review of basic anesthetic procedures. BMC Med Educ2018; 18: 154.
52.
IssaNSalzmanDHAdlerM.Mastery learning for clinical emergencies. In: McGaghieWBarsukJWayneD (eds) Comprehensive healthcare simulation: mastery learning in health professions education (Comprehensive healthcare simulation). Cham: Springer, 2020, pp. 259–269.
53.
AtkinsonRKRenklAMerrillMM.Transitioning from studying examples to solving problems: effects of self-explanation prompts and fading worked-out steps. J Educ Psychol2003; 95: 774–783.
54.
FloresRInanFA.Examining the impact of adaptively faded worked examples on student learning outcomes. J Interact Learn Res2014; 25: 467–485.
55.
SaldenRJCMAlevenVSchwonkeR, et al. The expertise reversal effect and worked examples in tutored problem solving. Instr Sci2010; 38: 289–307.
56.
RenklA. The worked examples principle in multimedia learning. In: MayerRE (ed.) The Cambridge handbook of multimedia learning. 2nd ed.New York: Cambridge University Press, 2014, pp. 391–412.
57.
O’RourkeEOButlerETolentinoAD, et al. Automatic generation of problems and explanations for an intelligent algebra tutor. In: IsotaniSMillánEOganA, et al. (eds) Artificial intelligence in education (AIED2019), vol. 11625 (Lecture notes in computer science). Cham: Springer, 2019, pp. 383–395.
58.
RoessinghJJPoppingaGVan OijenJ, et al. Application of artificial intelligence to adaptive instruction—combining the concepts. In: Proceedings of HCII 2019 (ed SottilareRSchwarzJ; Lecture notes in computer science), Orlando, FL, 26–31July2019, vol. 11597, pp. 1–15. Cham: Springer.
59.
LeppinkJDuvivierR.Twelve tips for medical curriculum design from a cognitive load theory perspective. Med Teach2016; 38: 669–674.
60.
DyerJ-OHudonAMontpetit-TourangeauK, et al. Example-based learning: comparing the effects of additionally providing three different integrative learning activities on physiotherapy intervention knowledge. BMC Med Educ2015; 15: 37.
61.
WisniewskiBZiererKHattieJ.The power of feedback revisited: a meta-analysis of educational feedback research. Front Psychol2020; 10: 3087.
62.
SadlerDR.Formative assessment: revisiting the territory. Assess Educ Princ Pol Pract1998; 5: 77–84.
63.
LefroyJWatlingCTeunissenP, et al. Guidelines: the do’s, don’ts and don’t knows of feedback for clinical education. Perspect Med Educ2015; 4: 284–299.
64.
SargeantJMannKSinclairD, et al. Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Adv Health Sci Educ Theory Pract2008; 13: 275–288.
65.
LampropoulosGKeramopoulosEDiamantarasK.Enhancing the functionality of augmented reality using deep learning, semantic web and knowledge graphs: a review. Vis Inform2020; 4: 32–42.
66.
HaysMJGlickARLaneHC.Leveraging cognitive science and artificial intelligence to save lives. In: IsotaniSMillánEOganA, et al. (eds) Artificial intelligence in education (AIED2019), vol. 11626 (Lecture notes in computer science). Cham: Springer, 2019, pp. 386–391.
67.
ChanKSZaryN.Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ2019; 5: e13930.
68.
GerardLMatukCMcElhaneyK, et al. Automated, adaptive guidance for K-12 education. Educ Res Rev2015; 15: 41–58.
69.
GerupJSoerensenCBDieckmannP.Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ2020; 11: 1–18.
70.
WeibelNGasquesDJohnsonJ, et al. ARTEMIS: mixed-reality environment for immersive surgical telementoring. In: Proceedings of the CHI’20 extended abstracts, Honolulu, HI, 25–30 April 2020. New York: ACM.
71.
Rojas-MuñozECabreraMELinC, et al. The System for Telementoring with Augmented Reality (STAR): a head-mounted display to improve surgical coaching and confidence in remote areas. Surgery2020; 167: 724–731.
72.
AndersenDPopescuVCabreraME, et al. Medical telementoring using an augmented reality transparent display. Surgery2016; 159: 1646–1653.
73.
VeraAMRussoMMohsinA, et al. Augmented reality telementoring (ART) platform: a randomized controlled trial to assess the efficacy of a new surgical education technology. Surg Endosc2014; 28: 3467–3472.
74.
WangSParsonsMStone-McLeanJ, et al. Augmented reality as a telemedicine platform for remote procedural training. Sensors2017; 17: 2294.
75.
ChenPCGadepalliKMacDonaldR, et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis. Nat Med2019; 25: 1453–1457.
76.
MarcusHJPrattPHughes-HallettA, et al. Comparative effectiveness and safety of image guidance systems in surgery: a preclinical randomised study. Lancet2015; 385: S64.
77.
BisraKLiuQNesbitJC, et al. Inducing self-explanation: a meta-analysis. Educ Psychol Rev2018; 30: 703–725.
78.
VanLehnKJonesRM. What mediates the self-explanation effect? Knowledge gaps, schemas or analogies? In: PolsonM (ed.) Proceedings of the fifteenth annual conference of the cognitive science society. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993, pp. 1034–1039.
79.
ChamberlandMMamedeS.Self-explanation, an instructional strategy to foster clinical reasoning in medical students. Health Prof Educ2015; 1: 24–33.
80.
WylieRChiMTH. The self-explanation principle in multimedia learning. In: MayerRE (ed.) The Cambridge handbook of multimedia learning. 2nd ed.New York: Cambridge University Press, 2014, pp. 413–432.
81.
ChamberlandMMamedeSSt-OngeC, et al. Self-explanation in learning clinical reasoning: the added value of examples and prompts. Med Educ2015; 49: 193–202.
82.
ChamberlandMSetrakianJSt-OngeC, et al. Does providing the correct diagnosis as feedback after self-explanation improve medical students diagnostic performance?BMC Med Educ2019; 19: 194.
WylieRShengMMitamuraT, et al. Effects of adaptive prompted self-explanation on robust learning of second language grammar. In: BiswasGBullSKayJ, et al. (eds) Artificial intelligence in education (AIED2011), vol. 6738 (Lecture notes in computer science). Berlin; Heidelberg: Springer, 2011, pp. 588–590.
85.
GilYSelmanB. A 20-year community roadmap for artificial intelligence research in the US. Washington, DC: Computing Community Consortium (CCC) and Association for the Advancement of Artificial Intelligence (AAAI).
86.
BrusilovskyP.Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for meta-adaptation. Br J Educ Technol2003; 34: 487–497.
87.
SantoianniFCiasulloA.Adaptive design for educational hypermedia environments and bio-educational adaptive design for 3D virtual learning environments. Res Educ Media2018; 10: 30–41.
88.
KohanNSoltani ArabshahiKMojtahedzadehR, et al. Self- directed learning barriers in a virtual environment: a qualitative study. J Adv Med Educ Prof2017; 5: 116–123.
89.
MolinaCATheodoreNAhmedAK, et al. Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study. J Neurosurg Spine2019; 31: 139–146.
90.
AzimiEWinklerATuckerE, et al. Can mixed-reality improve the training of medical procedures?Annu Int Conf IEEE Eng Med Biol Soc2018; 2018: 4065–4068.
91.
HashimotoDARosmanGRusD, et al. Artificial intelligence in surgery: promises and perils. Ann Surg2018; 268: 70–76.
92.
SchwonkeRHauserSNuäcklesM, et al. Enhancing computer-supported writing of learning protocols by adaptive prompts. Comput Hum Behav2006; 22: 77–92.
93.
MedinaMSCastleberryANPerskyAM.Strategies for improving learner metacognition in health professional education. Am J Pharm Educ2017; 81: 78.
94.
Feyzi-BehnaghRAzevedoRLegowskiE, et al. Metacognitive scaffolds improve self-judgments of accuracy in a medical intelligent tutoring system. Instr Sci2014; 42: 159–181.
95.
Van CampenhoutR. Supporting metacognitive learning strategies through an adaptive application. In: SottilareRASchwarzJ (eds) Adaptive instructional systems, vol. 12214 (Lecture notes in computer science). Cham: Springer, 2020, pp. 218–227.
96.
TaubMAzevedoRBouchetF, et al. Can the use of cognitive and metacognitive self-regulated learning strategies be predicted by learners’ levels of prior knowledge in hypermedia-learning environments?Comput Hum Behav2014; 39: 356–367.
97.
BouchetFHarleyJMAzevedoR.Evaluating adaptive pedagogical agents’ prompting strategies effect on students’ emotions. In: VassilevaJAzevedoR (eds) Intelligent tutoring systems, vol. 10858 (Lecture notes in computer science). Cham: Springer, 2018, pp. 33–43.
98.
LiuMShumSBMantzouraniE, et al. Evaluating machine learning approaches to classify pharmacy students’ reflective statements. In: IsotaniSMillánEOganA, et al. (eds) Artificial intelligence in education (AIED2019), vol. 11625 (Lecture notes in computer science). Cham: Springer, 2019, pp. 220–230.
99.
ConatiCVanLehnK.Toward computer-based support of meta-cognitive skills: a computational framework to coach self-explanation. Int J Artif Intell Educ2000; 11: 389–415.
100.
CunninghamPMatusovichHMHunterD-AN, et al. Board # 28 : Beginning to Understand Student Indicators of Metacognition. In: 2017 ASEE Annual Conference & Exposition, Columbus, Ohio, June, ASEE Conferences, 2017. https://peer.asee.org/27820 (accessed 6 December 2021).
101.
HongWHVadiveluJDanielEG, et al. Thinking about thinking: changes in first-year medical students’ metacognition and its relation to performance. Med Educ Online2015; 20: 27561.
102.
DavisDAMazmanianPEFordisM, et al. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA2006; 296: 1094–1102.
103.
AugustinM.How to learn effectively in medical school: test yourself, learn actively, and repeat in intervals. Yale J Biol Med2014; 87: 207–212.
104.
CepedaNJPashlerHVulE, et al. Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol Bull2006; 132: 354–380.
105.
StormBCBjorkRAStormJC.Optimizing retrieval as a learning event: when and why expanding retrieval practice enhances long-term retention. Mem Cognit2010; 38: 244–253.
106.
TabibianBUpadhyayUDeA, et al. Enhancing human learning via spaced repetition optimization. P Natl Acad Sci USA2019; 116: 3988–3993.
107.
KhajahMMLindseyRVMozerMC.Maximizing students’ retention via spaced review: practical guidance from computational models of memory. Top Cogn Sci2014; 6: 157–169.
KerfootBPDeWolfWCMasserBA, et al. Spaced education improves the retention of clinical knowledge by medical students: a randomised controlled trial. Med Educ2007; 41: 23–31.
CorbettATAndersonJR. Knowledge tracing: modeling the acquisition of procedural knowledge. User Model User: Adap1994; 4: 253–278.
112.
JohnsJMahadevanSWoolfB.Estimating student proficiency using an item response theory model. In: Proceedings of the international conference on intelligent tutoring systems (Lecture notes in computer science), Jhongli, Taiwan, 26–30 June 2006, vol. 879, pp. 473–480. Berlin; Heidelberg: Springer.
