Purpose. We analyzed the literature to determine (1) the surgically relevant applications for which head-mounted display (HMD) use is reported; (2) the types of HMD most commonly reported; and (3) the surgical specialties in which HMD use is reported. Methods. The PubMed, Embase, Cochrane Library, and Web of Science databases were searched through August 27, 2017, for publications describing HMD use during surgically relevant applications. We identified 120 relevant English-language, non-opinion publications for inclusion. HMD types were categorized as “heads-up” (nontransparent HMD display and direct visualization of the real environment), “see-through” (visualization of the HMD display overlaid on the real environment), or “non–see-through” (visualization of only the nontransparent HMD display). Results. HMDs were used for image guidance and augmented reality (70 publications), data display (63 publications), communication (34 publications), and education/training (18 publications). See-through HMDs were described in 55 publications, heads-up HMDs in 41 publications, and non–see-through HMDs in 27 publications. Google Glass, a see-through HMD, was the most frequently used model, reported in 32 publications. The specialties with the highest frequency of published HMD use were urology (20 publications), neurosurgery (17 publications), and unspecified surgical specialty (20 publications). Conclusion. Image guidance and augmented reality were the most commonly reported applications for which HMDs were used. See-through HMDs were the most commonly reported type used in surgically relevant applications. Urology and neurosurgery were the specialties with greatest published HMD use.
QianLBarthelAJohnsonA, et al. Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display. Int J Comput Assist Radiol Surg. 2017;12:901-910.
6.
Ottawa Hospital Research Institute; University of Oxford. PRISMA: transparent reporting of systematic reviews and meta-analyses. http://www.prisma-statement.org/. Accessed June 11, 2019.
7.
FuchsHLivingstonMARaskarR, et al. Augmented reality visualization for laparoscopic surgery. In: WellsWMColchesterADelpS, eds. Medical Image Computing and Computer-Assisted Intervention—MICCAI’98. Lecture Notes in Computer Science, Vol 1496. Berlin, Germany: Springer; 1998:934-943.
8.
LinDWBushRWEarleDBSeymourNE. Performance and ergonomic characteristics of expert surgeons using a face-mounted display during virtual reality-simulated laparoscopic surgery: an electromyographically based study. Surg Endosc. 2007;21:1135-1141.
9.
PrendergastCJRyderBAAbodeelyAMuratoreCSCrawfordGPLuksFI. Surgical performance with head-mounted displays in laparoscopic surgery. J Laparoendosc Adv Surg Tech A. 2009;19(suppl 1):S237-S240.
10.
SudraGSpeidelSFritzDMüller-StichBPGuttCDillmanR. MEDIASSIST—MEDIcal ASSistance for intraoperative skill transfer in minimally invasive surgery using augmented reality. Proc SPIE. 2007;6509:65091O-1-9.
11.
EnislidisGWagnerAPloderOEwersR. Computed intraoperative navigation guidance—a preliminary report on a new technique. Br J Oral Maxillofac Surg. 1997;35:271-274.
12.
WagnerAMillesiWWatzingerF, et al. Clinical experience with interactive teleconsultation and teleassistance in craniomaxillofacial surgical procedures. J Oral Maxillofac Surg. 1999;57:1413-1418.
13.
WagnerARasseMMillesiWEwersR. Virtual reality for orthognathic surgery: the augmented reality environment concept. J Oral Maxillofac Surg. 1997;55:456-463.
14.
KanekoNTsunodaMMitsuhashiM, et al. Ultrasound-guided fine-needle aspiration in the neck region using an optical see-through head-mounted display: a randomized controlled trial. J Ultrasound Med. 2017;36:2071-2077.
15.
KanekoNSatoMTakeshimaTSeharaYWatanabeE. Ultrasound-guided central venous catheterization using an optical see-through head-mounted display: a pilot study. J Clin Ultrasound. 2016;44:487-491.
16.
MaruyamaKWatanabeESaitoK, et al. P04.17 Novel augmented reality-based neuronavigation using smart glasses and clinical application to brain tumor surgery. Neuro Oncol. 2017;19(suppl 3):iii43-iii44.
17.
ShinoharaK. Prospects and problems of smart glasses as tools for surgery. In: DuffyVLightnerN, eds. Advances in Human Factors and Ergonomics in Healthcare. Cham, Switzerland: Springer; 2017:39-45.
18.
ArmstrongDGRankinTMGiovincoNAMillsJLMatsuokaY. A heads-up display for diabetic limb salvage surgery: a view through the google looking glass. J Diabetes Sci Technol. 2014;8:951-956.
19.
BaldwinACMallidiHRBaldwinJC, et al. Through the looking glass: real-time video using “Smart” technology provides enhanced intraoperative logistics. World J Surg. 2016;40:242-244.
20.
BorgmannHSocarrasMRSalemJ, et al. Feasibility and safety of augmented reality-assisted urological surgery using smartglass. World J Urol. 2017;35:967-972.
21.
BrewerZEFannHCOgdenWDBurdonTASheikhAY. Inheriting the learner’s view: a Google glass-based wearable computing platform for improving surgical trainee performance. J Surg Educ. 2016;73:682-688.
22.
ChimentiPCMittenDJ. Google glass as an alternative to standard fluoroscopic visualization for percutaneous fixation of hand fractures: a pilot study. Plast Reconstr Surg. 2015;136:328-330.
23.
DattaNMacQueenITSchroederAD, et al. Wearable technology for global surgical teleproctoring. J Surg Educ. 2015;72:1290-1295.
24.
DiazRYoonJChenRQuinones-HinojosaAWharenRKomotarR. Real-time video-streaming to surgical loupe mounted head-up display for navigated meningioma resection [published online April 30, 2017]. Turk Neurosurg. doi:10.5137/1019-5149.JTN.20388-17.1
25.
DickeyRMSrikishenNLipshultzLISpiessPECarrionREHakkyTS. Augmented reality assisted surgery: a urologic training tool. Asian J Androl. 2016;18:732-734.
26.
FunkSLeeDH. Optical head-mounted computer display for education, research, and documentation in hand surgery. J Hand Surg Am. 2016;41:150-153.
27.
GolabMRBreedonPJVloeberghsM. A wearable headset for monitoring electromyography responses within spinal surgery. Eur Spine J. 2016;25:3214-3219.
28.
GuptaSBoehmeJManserK, et al. Does wearable medical technology with video recording capability add value to on-call surgical evaluations?Surg Innov. 2016;23:498-504.
29.
HashimotoDAPhitayakornRFernandez-del CastilloCMeirelesO. A blinded assessment of video quality in wearable technology for telementoring in open surgery: the Google Glass experience. Surg Endosc. 2016;30:372-378.
30.
IqbalMHAydinALowdonA, et al. The effectiveness of Google Glass as a vital signs monitor in surgery: a simulation study. Int J Surg. 2016;36(pt A):293-7.
31.
KunshanHYaminMJinzuoY, et al. A novel wireless wearable fluorescence image-guided surgery system. Conf Proc IEEE Eng Med Biol Soc. 2016;2016:5208-5211.
32.
LeeCKKimYLeeNKimBKimDYiS. Feasibility study of utilization of action camera, GoPro Hero 4, Google Glass, and Panasonic HX-A100 in spine surgery. Spine (Phila Pa 1976). 2017;42:275-280.
33.
LiebertCAZayedMAAalamiOTranJLauJN. Novel use of Google Glass for procedural wireless vital sign monitoring. Surg Innov. 2016;23:366-373.
34.
Martinez-MillanaABayo-MontonJLLizondoAFernandez-LlatasCTraverV. Evaluation of Google Glass technical limitations on their integration in medical systems. Sensors (Basel). 2016;16:E2142.
35.
MentisHMRahimATheodoreP. Crafting the image in surgical telemedicine. Presented at: The 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing; San Francisco, CA; February 27-March 2, 2016.
36.
MoshtaghiOKelleyKSArmstrongWBGhavamiYGuJDjalilianHR. Using Google Glass to solve communication and surgical education challenges in the operating room. Laryngoscope. 2015;125:2295-2297.
37.
MuenstererOJLacherMZoellerCBronsteinMKublerJ. Google Glass in pediatric surgery: an exploratory study. Int J Surg. 2014;12:281-289.
38.
NakhlaJKobetsADe la Garza RamosR, et al. Use of Google Glass to enhance surgical education of neurosurgery residents: “proof-of-concept” study. World Neurosurg. 2017;98:711-714.
39.
OpolskiMPDebskiABoruckiBA, et al. First-in-man computed tomography-guided percutaneous revascularization of coronary chronic total occlusion using a wearable computer: proof of concept. Can J Cardiol. 2016;32:829.e11-e13.
40.
PedenRGMercerRTathamAJ. The use of head-mounted display eyeglasses for teaching surgical skills: a prospective randomised study. Int J Surg. 2016;34:169-173.
41.
PonceBAMenendezMEOladejiLOFrybergerCTDantuluriPK. Emerging technology in surgical education: combining real-time augmented reality and wearable computing devices. Orthopedics. 2014;37:751-757.
42.
PorrasJLKhalidSRootBKKhanISSingerRJ. Point-of-view recording devices for intraoperative neurosurgical video capture. Front Surg. 2016;3:57.
43.
RahimyEGargSJ. Google Glass for recording scleral buckling surgery. JAMA Ophthalmol. 2015;133:710-711.
44.
SahyouniRMoshtaghiOTranDK, et al. Assessment of Google Glass as an adjunct in neurological surgery. Surg Neurol Int. 2017;8:68.
45.
SchaerRSalaminFDel ToroOAJAtzoriMMullerHWidmerA. Live ECG readings using Google Glass in emergency situations. Conf Proc IEEE Eng Med Biol Soc. 2015;2015:315-318.
46.
ShaoPDingHWangJ, et al. Designing a wearable navigation system for image-guided cancer resection surgery. Ann Biomed Eng. 2014;42:2228-2237.
47.
SinkinJCRahmanOFNahabedianMY. Google Glass in the operating room: the plastic surgeon’s perspective. Plast Reconstr Surg. 2016;138:298-302.
48.
YoonJWChenREReFaeyK, et al. Technical feasibility and safety of image-guided parieto-occipital ventricular catheter placement with the assistance of a wearable head-up display. Int J Med Robot. 2017;13(4). doi:10.1002/rcs.1836
49.
ZhangZPeiJWangD, et al. A wearable goggle navigation system for dual-mode optical and ultrasound localization of suspicious lesions: validation studies using tissue-simulating phantoms and an ex vivo human breast tissue model. PLoS One. 2016;11:e0157854.
50.
ChenXXuLWangY, et al. Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J Biomed Inform. 2015;55:124-131.
51.
BirkfellnerWFiglMMatulaC, et al. Computer-enhanced stereoscopic vision in a head-mounted operating binocular. Phys Med Biol. 2003;48:N49-N57.
52.
BirkfellnerWFiglMMatulaC, et al. Stereoscopic visualization in the varioscope AR: a see-through head mounted display for surgical navigation. Proc SPIE. 2002;4681:425-435.
53.
BirkfellnerWFiglMHuberK, et al. Calibration of projection parameters in the Varioscope AR, a head-mounted display for augmented reality visualization in image-guided therapy. Proc SPIE. 2001;4319:471-480.
54.
BirkfellnerWHuberKWatzingerF, et al. Development of the varioscope AR: a see-through HMD for computer-aided surgery. Presented at: Proceedings of IEEE and ACM International Symposium on Augmented Reality (ISAR 2000); Munich, Germany; October 5-6, 2000.
55.
WanschitzFBirkfellnerWFiglM, et al. Computer-enhanced stereoscopic vision in a head-mounted display for oral implant surgery. Clin Oral Implants Res. 2002;13:610-616.
56.
MarschollekMBarthelCBehrendsMSchmeerRMeyenburg-AltwargIBeckerM. Smart glasses in nursing training—redundant gadget or precious tool? A pilot study. Stud Health Technol Inform. 2016;225:377-381.
57.
HollowayRL. Registration error analysis for augmented reality. Presence (Camb). 1997;6:413-432.
58.
KaticDSpenglerPBodenstedtS, et al. A system for context-aware intraoperative augmented reality in dental implant surgery. Int J Comput Assist Radiol Surg. 2015;10:101-108.
59.
SalbTBurgertOGockelTGieslerBDillmannR. Comparison of tracking techniques for intraoperative presentation of medical data using a see-through head-mounted display. Stud Health Technol Inform. 2001;81:443-445.
60.
SchneiderEBartlKDeraTBoningGWagnerPBrandtT. Documentation and teaching of surgery with an eye movement driven head-mounted camera: see what the surgeon sees and does. Stud Health Technol Inform. 2006;119:486-490.
61.
WagnerAPloderOEnislidisGTruppeMEwersR. Image-guided surgery. Int J Oral Maxillofac Surg. 1996;25:147-151.
StewartJBillinghurstM. A wearable navigation display can improve attentiveness to the surgical field. Int J Comput Assist Radiol Surg. 2016;11:1193-1200.
64.
YoshidaSFukuiNSaitoKFujiiYKageyamaYKiharaK. Novel image monitoring system using a head-mounted display for assistants in da Vinci surgery. Int J Urol. 2015;22:520-521.
65.
BadialiGFerrariVCutoloF, et al. Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Craniomaxillofac Surg. 2014;42:1970-1976.
66.
HuLWangMSongZ. A convenient method of video see-through augmented reality based on image-guided surgery system. Paper presented at: Seventh International Conference on Internet Computing for Engineering and Science; Shanghai, China; September 20-22, 2013.
67.
RamanathanRSalamancaJIMandhaniA, et al. Does 3-dimensional (3-D) visualization improve the quality of assistance during robotic radical prostatectomy?World J Urol. 2009;27:95-99.
68.
OrtegaGWolffABaumgaertnerMKendoffD. Usefulness of a head mounted monitor device for viewing intraoperative fluoroscopy during orthopaedic procedures. Arch Orthop Trauma Surg. 2008;128:1123-1126.
69.
UdaniADHarrisonTKHowardSK, et al. Preliminary study of ergonomic behavior during simulated ultrasound-guided regional anesthesia using a head-mounted display. J Ultrasound Med. 2012;31:1277-1280.
70.
RyndinILukasewyczSHoffmanNNakibNABestSMongaM. Impact of heads-up display imaging on endoscopic task performance. J Endourol. 2005;19:964-967.
71.
WongBJFLeeJPDuganFMMacArthurCJ. Head mounted display (HMD) for use in functional endoscopic sinus surgery. Proc SPIE. 1995;2396:337-344.
72.
FujiiYKiharaKYoshidaSIshiokaJMatsuokaYNumaoNSaitoK. A three-dimensional head-mounted display system (RoboSurgeon system) for gasless laparoendoscopic single-port partial cystectomy. Wideochir Inne Tech Maloinwazyjne. 2014;9:638-643.
73.
ItoMKiharaKYoshidaS, et al. Patient’s self-monitoring of transurethral surgical images using a head-mounted display. Urol Case Rep. 2015;3:27-29.
74.
KiharaKSaitoKKomaiYFujiiY. Integrated image monitoring system using head-mounted display for gasless single-port clampless partial nephrectomy. Wideochir Inne Tech Maloinwazyjne. 2014;9:634-637.
75.
Martinez-ToldosJJFernandez-MartinezCNavarro-NavarroA. Experience using a 3D head-mounted display system in ophthalmic surgery. Retina. 2017;37:1419-1421.
76.
MatsuokaYKiharaKKawashimaKFujiiY. Integrated image navigation system using head-mounted display in “RoboSurgeon” endoscopic radical prostatectomy. Wideochir Inne Tech Maloinwazyjne. 2014;9:613-618.
77.
MorisawaTKidaHKusumiFOkinagaSOhanaM. Endoscopic submucosal dissection using head-mounted display. Gastroenterology. 2015;149:290-291.e1.
78.
NakayamaTNumaoNYoshidaS, et al. A novel interactive educational system in the operating room—the IE system. BMC Med Educ. 2016;16:44.
79.
YoshidaSKiharaKTakeshitaH, et al. Head-mounted display for a personal integrated image monitoring system: ureteral stent placement. Urol Int. 2015;94:117-120.
80.
YoshidaSSasakiASatoC, et al. A novel approach to surgical instructions for scrub nurses by using see-through-type head-mounted display. Comput Inform Nurs. 2015;33:335-338.
81.
YoshidaSKiharaKTakeshitaHFujiiY. Instructive head-mounted display system: pointing device using a vision-based finger tracking technique applied to surgical education. Wideochir Inne Tech Maloinwazyjne. 2014;9:449-452.
82.
CutoloFCarboneMParchiPDFerrariVLisantiMFerrariM. Application of a new wearable augmented reality video see-through display to aid percutaneous procedures in spine surgery. In: DePaolisLMongelliA, eds. Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2016 Lecture Notes in Computer Science, Vol 9769. Cham, Switzerland: Springer; 2016:43-54.
83.
KiharaKFujiiYMasudaH, et al. New three-dimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: procedures for gasless single-port radical nephrectomy. Int J Urol. 2012;19:886-889.
84.
LinYKYauHTWangICZhengCChungKH. A novel dental implant guided surgery based on integration of surgical template and augmented reality. Clin Implant Dent Relat Res. 2015;17:543-553.
85.
YoshidaSKiharaKTakeshitaHFujiiY. A head-mounted display-based personal integrated-image monitoring system for transurethral resection of the prostate. Wideochir Inne Tech Maloinwazyjne. 2014;9:644-649.
86.
IshiokaJKiharaKHiguchiS, et al. New head-mounted display system applied to endoscopic management of upper urinary tract carcinomas. Int Braz J Urol. 2014;40:842-845.
87.
CalvanoCJMoranMETackettLDReddyPPBoyleKEPankratovMM. New visualization techniques for in utero surgery: amnioscopy with a three-dimensional head-mounted display and a computer-controlled endoscope. J Endourol. 1998;12:407-410.
88.
ChenJCMoffittKLevyML. Head-mounted display system for microneurosurgery. Stereotact Funct Neurosurg. 1997;68(1-4 pt 1):25-32.
89.
HerronDMLantisJC2ndMaykelJBasuCSchwaitzbergSD. The 3-D monitor and head-mounted display. A quantitative evaluation of advanced laparoscopic viewing technologies. Surg Endosc. 1999;13:751-755.
YoshidaSItoMTatokoroM, et al. Multitask imaging monitor for surgical navigation: combination of touchless interface and head-mounted display. Urol Int. 2017;98:486-488.
92.
LevyMLDayJDAlbuquerqueFSchumakerGGiannottaSLMcCombJG. Heads-up intraoperative endoscopic imaging: a prospective evaluation of techniques and limitations. Neurosurgery. 1997;40:526-531.
93.
AustinJW. Evaluation of a “heads-up” display for cardiopulmonary bypass. J Extra Corpor Technol. 2000;32:49-53.
94.
EhlersJPSrivastavaSKFeilerDNoonanAIRollinsAMTaoYK. Integrative advances for OCT-guided ophthalmic surgery and intraoperative OCT: microscope integration, surgical instrumentation, and heads-up display surgeon feedback. PLoS One. 2014;9:e105224.
95.
EljamelMS. Frameless stereotactic neurosurgery: two steps towards the Holy Grail of surgical navigation. Stereotact Funct Neurosurg. 1999;72:125-128.
96.
GeisWP. Head-mounted video monitor for global visual access in mini-invasive surgery. An initial report. Surg Endosc. 1996;10:768-770.
97.
HeacockGMarshallJ. The use of non-immersive virtual reality in ophthalmic diagnosis and its potential for laser surgery. Lasers Light Ophthalmol. 1994;6:135-141.
98.
ParsonsJRollandJP. A non-intrusive display technique for providing real-time data within a surgeons critical area of interest. Stud Health Technol Inform. 1998;50:246-251.
99.
ShenLCarrasco-ZevallosOKellerB, et al. Novel microscope-integrated stereoscopic heads-up display for intrasurgical optical coherence tomography. Biomed Opt Express. 2016;7:1711-1726.
100.
YoshidaTInoueHKumagaiYIwaiT. Image-navigated endoscopic surgery: introduction of the face-mounted display system with picture-in-picture capabilities. Hepatogastroenterology. 2000;47:375-377.
101.
TanakaHNakamuraHTamakiENariaiTHirakawaK. Brain surgery simulation system using VR technique and improvement of presence. Stud Health Technol Inform. 1998;50:150-154.
102.
LinQXuZLiB, et al. Immersive virtual reality for visualization of abdominal CT. Proc SPIE Int Soc Opt Eng. 2013;8673.
103.
MinnichDJSchellSR. Evaluation of face-mounted binocular video display for laparoscopy: outcomes of psychometric skills testing and surgeon satisfaction. J Laparoendosc Adv Surg Tech A. 2003;13:333-338.
104.
YoshidaTInoueHIwaiT. A new face-mounted display system: pilot trial of clinical application in therapeutic endoscopy. Endoscopy. 2000;32:974-976.
105.
TraubJStefanPHeiningSM, et al. Hybrid navigation interface for orthopedic and trauma surgery. Med Image Comput Comput Assist Interv. 2006;9(pt 1):373-380.
106.
BeuchatATaubSSabyJD, et al. Cybertools improve reaction time in open heart surgery. Eur J Cardiothorac Surg. 2005;27:266-269.
107.
RosenthalMStateALeeJ, et al. Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med Image Anal. 2002;6:313-320.
108.
SalbTBurgertOGockelT, et al. Risk reduction in craniofacial surgery using computer-based modeling and intraoperative immersion. Stud Health Technol Inform. 2002;85:441-447.
109.
SudraGMarmullaRSalbT, et al. First clinical tests with the augmented reality system INPRES. Stud Health Technol Inform. 2005;111:532-537.
110.
SudraGMarmullaRSalbT, et al. Technical experience from clinical studies with INPRES and a concept for a miniature augmented reality system. Proc SPIE. 2005;5744:748-755.
111.
ThomsenMNLangRD. An experimental comparison of 3-dimensional and 2-dimensional endoscopic systems in a model. Arthroscopy. 2004;20:419-423.
112.
SankaranarayananGLiBManserK, et al. Face and construct validation of a next generation virtual reality (Gen2-VR) surgical simulator. Surg Endosc. 2016;30:979-985.
113.
MondalSBGaoSZhuN, et al. Binocular Goggle augmented imaging and navigation system provides real-time fluorescence image guidance for tumor resection and sentinel lymph node mapping. Sci Rep. 2015;5:12117.
114.
AzarFSPerrinNKhameneAVogtSSauerF. User performance analysis of different image-based navigation systems for needle placement procedures. Proc SPIE. 2004;5367:110-121.
115.
BichlmeierCOckertBHeiningSMAhmadiANavabN. Stepping into the operating theater: ARAV—augmented reality aided vertebroplasty. IEEE International Symposium on Mixed and Augmented Reality. 2008:165-166.
116.
BichlmeierCHeiningSMRustaeeMNavabN. Virtually extended surgical drilling device: virtual mirror for navigated spine surgery. Med Image Comput Comput Assist Interv. 2007;10(pt 1):434-441.
117.
GaoMKChenYMLiuQHuangCLiZYZhangDH. Three-dimensional path planning and guidance of leg vascular based on improved ant colony algorithm in augmented reality. J Med Syst. 2015;39:133.
118.
KimBH. Lightless cataract surgery using a near-infrared operating microscope. J Cataract Refract Surg. 2006;32:1683-1690.
RhienmoraPGajanananKHaddawyP, et al. Haptic augmented reality dental trainer with automatic performance assessment. Paper presented at: Proceedings of the 15th International Conference on Intelligent User Interfaces; Hong Kong, China; February 7-10, 2010.
121.
SielhorstTBichlmeierCHeiningSMNavabN. Depth perception—a major issue in medical AR: evaluation study by twenty surgeons. Med Image Comput Comput Assist Interv. 2006;9(pt 1):364-372.
122.
WackerFKVogtSKhameneA, et al. MR image-guided needle biopsies with a combination of augmented reality and MRI: a pilot study in phantoms and animals. Int Congr Ser. 2005;1281:424-428.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
