Sage Journals: Discover world-class research

Abstract

Background

In recent years, numerous innovative yet challenging surgeries, such as minimally invasive procedures, have introduced an overwhelming amount of new technologies, increasing the cognitive load for surgeons and potentially diluting their attention. Cognitive support technologies (CSTs) have been in development to reduce surgeons’ cognitive load and minimize errors. Despite its huge demands, it still lacks a systematic review.

Methods

Literature was searched up until May 21st, 2021. Pubmed, Web of Science, and IEEExplore. Studies that aimed at reducing the cognitive load of surgeons were included. Additionally, studies that contained an experimental trial with real patients and real surgeons were prioritized, although phantom and animal studies were also included. Major outcomes that were assessed included surgical error, anatomical localization accuracy, total procedural time, and patient outcome.

Results

A total of 37 studies were included. Overall, the implementation of CSTs had better surgical performance than the traditional methods. Most studies reported decreased error rate and increased efficiency. In terms of accuracy, most CSTs had over 90% accuracy in identifying anatomical markers with an error margin below 5 mm. Most studies reported a decrease in surgical time, although some were statistically insignificant.

Discussion

CSTs have been shown to reduce the mental workload of surgeons. However, the limited ergonomic design of current CSTs has hindered their widespread use in the clinical setting. Overall, more clinical data on actual patients is needed to provide concrete evidence before the ubiquitous implementation of CSTs.

Keywords

cognitive support surgery performance mental workload anatomical accuracy patient outcomes

Introduction

Modern surgery has come a long way. From the first successful open surgery to the laparoscopic and robotic-assisted procedures standard in today’s operating room, the success of modern surgery is no anomaly; it is deeply entwined with the development of surgical technology. Ranging from aseptic techniques to intra-operative imaging, technology has always been an effective support for surgeons.¹ Novel surgical procedures such as minimally invasive surgery (MIS) also benefit the patients, as it is often associated with lower patient morbidity, infection, and a shorter hospital stay.^2-4 However, it is important to highlight that surgical techniques today have increased in complexity compared to previous decades. Procedures like endoscopic or laparoscopic surgery have restricted the surgeon’s visual field to 2-D instead of the traditional 3-D view in open surgeries. The disparity in the visual field introduces problems such as image-reality incompatibility, loss of depth perception, and abnormal haptic feedback; additionally, novel surgical techniques also require new machines to be introduced to the operating room (OR), which may lead to extra staff to monitor the devices. The mental strain caused by MIS and excessive signals present in the working environment of the surgeon may lead to cognitive overload, which causes surgeon performance to decrease⁵; this induces surgical errors while also extending operational time.^6,7 Another potential cause to surgical errors involves communication. Irrelevant conversations within the OR are associated with poorer team collaboration and performance,^8,9 while also extending operational time; equipment-related distractions is associated with higher stress and cognitive load.⁸ Compared to open surgery, communication in MIS is more unclear and more frequently directed at equipment instead of the procedure, likely due to the increased equipment and staff used to support the MIS.¹⁰. Parker et al¹¹ reported a longer duration of open transforaminal lumbar interbody fusion surgery when comparing MIS to standard open surgery. Experiments that mimicked both MIS and open surgery environments reported a decreased efficiency in task completion and an increased mental workload.¹² Accompanying the increased cognitive load are increased probability of adverse events. In the top 559 hospitals located in the US, close to 81.7% of intraoperative errors are related to surgeons’ cognition being overwhelmed, each offense could cause up to 125,000$ in penalty. Thus, fatigue-related events, such as retained foreign objects or wrong surgical sites would be significantly reduced if surgeons received cognitive support.¹³ To counteract the increased cognitive load of surgeons and to reduce surgical errors, cognitive support technology (CST) has developed since the early 2000s and has seen significant growth in recent years to aid the decision-making of physicians and surgeons. However, despite its importance, technology in this field lacks a systematic review and assessment of the impacts on surgical performance. We aim to review the current status and trends of CST in surgery. CSTs covered in this review pertains to information presentation for diagnostic support and visualization for enhancing performance. Specifically, we include the software and hardware employed for data presentation, analysis in surgery, and impact on surgeon performance and patient outcome.

Method

Search Criteria

The inclusion criteria include a list of terms that can be categorized into three major groups: cognitive support, surgery, and digital reality. A figure of an example search in PubMed is provided. Related terms are connected with Boolean operators; terms in a column are connected with the “OR” operator, while terms in a row are connected with the “AND” operator (Figure 1). A CST is a technology that reaches a high level of innovation and technological complexity, often involving programming or robotic design aimed at reducing cognitive load of surgeons. Exclusion criteria were set to be checklist articles because an overwhelming number of checklists was included in the literature search. In this review, a checklist defined as a digital or non-digital element used to verify aspects of the surgery in the OR. Surgical checklists were often specific to a large variety of surgeries which required a large amount of background information. There were also additional checklists assessing other checklists which were too complicated for the purpose of this review. Literature included in this review was gathered through a literature search in medical databases, including the Web of Science, PubMed, and IEEExplore, articles published until May 21, 2021 were included. An effort was made to search for clinical trials or randomized controlled trials, but a few exceptional model and architectural articles are included to provide a framework understanding of the topic. The article search and selection process was completed by a single reviewer (ZSZ). Mendeley reference manager (Relx, New York, USA) was used to organize the references. Studies were included in this review if it targeted the peri-operative phase in surgery, contained the development software or device that aimed to reduce surgeons’ cognitive load, and experimental trials that assessed the efficacy of the technology. Studies that only included a phantom trial, an animal trial, or used synthetic patients were included. Articles were first screened by title and abstract assessments. The remaining articles then underwent full-text analysis to further evaluate relevant articles.

Figure 1.

Advance search criteria for PubMed.

Data Analysis

A single reviewer (ZSZ) examined key characteristics of each article including data source, development status, experiment type, sample size, phase of surgery, and the type of procedure. Full-text articles selected were organized with Microsoft Excel (Microsoft Corporation, WA, USA). Papers were further stratified into different subgroups by the parameters they measured; clinical outcomes, patient outcomes, and OR time variations were all taken into consideration. Discussion of the advantages and disadvantages of CSTs were gathered from the discussion and conclusion of the articles. Interpretation of the study results was based on the result section of each paper. A secondary reviewer (YW) verified the accuracy of the content. In events where the article cannot be classified into a set group, or parameters on the article was unclear, then a discussion was to occur with two other reviewers (BZ, WL) until an agreement was made.

Results

Overall, 481 citations were found throughout the literature search; after electronically removing 22 duplicates using Endnote (Clarivate, London), 459 remaining articles underwent further screening. Two independent reviewers (ZSZ, LW) screened the citations based on title and abstract, 369 articles were removed, leaving 35 for full-text review. During full-text review, 14 articles were removed: seven lacked quantitative data; six were overview articles; one was a checklist. An additional 19 articles were found through relevant searching, summing the total number of articles included in this review to be 40 articles. Each article had at least one relevant data describing cognitive load reduction. The complete article evaluation process is outlined as Figure 2.

Figure 2.

Flow chat of the article selection process.

Data Source

CSTs use input data from a few different varieties to generate 3-D images for enhanced visualization or to assist patient diagnoses. Medical imaging databases such as XNAT are developed for imaging upload and information extraction.¹⁴ For all the papers in this review, an overwhelming 23 (58%) CSTs used X-ray-based images such as computed tomography (CT) as their primary data source. 6 (15%) CSTs adopted MRI as their data source while another 6(15%) CSTs used ultrasound; however, they are not as common as the CT scans due to a variety of different reasons relating to costs and accuracy. 2(6%) CST uses other types of input such as live videos with manual markers with its application in nursing, and physiological parameters when analyzing cognitive load.¹⁵ 7(18%) CST did not provide specific sources of their input data, which may be due to their employment of previously established data from other sources (Table 1).

Table 1.

Characteristics of Included Studies.

Study	Data Source	Type of Development	Type of Experiment	Sample Size (n) (Staff; Patient)	Phase of Surgery	Procedure
M¨arz et al (2015)	Non-specific	Software development	Development	(NS; 184)	Pre-operative	Hepatectomy
Schoch et al (2016)	MRI	Software development	Development	(NS; 624)	Pre-operative	Cardiac surgery
Thienphrapa et al (2019)	CT	Software development	Development	(NS; NS)	Intra-operative	Video-assisted thoracoscopic surgery
Willekes et al (2008)	CT	Hardware development	User tests	(NS; 2)	Intra-operative	Video-assisted thoracic surgery
Hansen et al (2008)	Ultrasound	Software+hardware development	Development	(NS; NS)	Intra-operative	Hepatectomy
Lang et al (2005)	MRI or CT	Software development	Clinical tests	(2; 25)	Pre-operative	Hepatectomy
Hagenah et al (2018)	Ultrasound	Software development	Development	(NS; 10)	Pre-operative	Cardiac surgery
Chi et al (2018)	CT	Software+hardware development	User tests	(1; 6)	Intra-operative	Cardiac surgery
Abhari et al (2015)	MRI	Software+hardware development	User tests	(21; NS)	Pre-operative	Neurosurgery
Hagenah et al (2019)	Ultrasound	Software development	Development	(NS; 24)	Pre-operative	Cardiac surgery
Simoes et al (2013)	CT	Software+hardware development	Development	(NS; NS)	Pre-operative	Robot-assisted laparoscopic surgery
Hayashi et al (2017)	CT	Software development	Clinical tests	(3; 26)	Pre-operative	Laparoscopic gastrectomy
Azagury et al (2012)	CT	Software+hardware development	Clinical tests	(3; 3)	Intra-operative	Natural orifice transluminal
						Endoscopic surgery
Baumhauer et al (2008)	CT	Software development	User test	(1; 3)	Intra-operative	Laparoscopic partial nephrectomy
Birkfellner et al (2003)	CT	Software+hardware development	Development	(NS; NS)	Intra-operative	Skull base surgery
Collins et al (2014)	MRI	Software development	User tests	(NS; 1)	Intra-operative	Laparoscopic myomectomy
Conrad et al (2016)	CT	Software+hardware development	User tests	(NS; 1)	Intra-operative	Laparoscopic rescue of failed portal vein embolization
Debarba et al (2010)	CT	Software development	Development	(NS; 4)	Pre-operative	Hepatectomy
Haouchine et al (2013)	CT	Software development	Development	(NS; 2)	Intra-operative	Liver minimally invasive surgery
Marzano et al (2013)	CT	Software+hardware development	User tests	(NS; 1)	Intra-operative	Pancreatico-duodenectomy
Osorio et al (2010)	CT	Software development	Clinical tests	(NS; 24)	Intra-operative	Laparoscopic surgery
^aLinxweiler et al (2020)	CT	Software development	Clinical tests	(2; 100)	Intra-operative	Sinus-related surgery
Ntourakis et al (2015)	CT	Software+hardware development	Clinical tests	(NS; 3)	Intra-operative	Hepatectomy
Ogawa et al (2019)	CT	Software+hardware development	Clinical tests	(1; 41)	Intra-operative	Total hip arthroplasty
Buck et al (2005)	MRI or CT	Software development	Development	(NS; 39)	Pre-operative + Intra-operative	Cardiac surgery
Dias et al (2020)	Live video	Hardware development	User tests	(45; NS)	Intra-operative	Intubation
Dilley et al (2019)	Non-specific	Software development	User tests	(36; NS)	Intra-operative	Laparoscopic cholecystectomy
Dixon et al (2013)	CT	Software+hardware development	Clinical tests	(33; 1)	Intra-operative	Endoscopic transnasal skull base navigation
^aGrasso et al (2015)	CT	Software+hardware development	Clinical tests	(3; 180)	Intra-operative	Lung biopsie
Invernizzi et al (2020)	Non-specific	Software development	Clinical tests	(3; 30)	Pre-operative	Breast cancer related lymphedema diagnosis
Jeon et al (2014)	Ultrasound	Software+hardware development	User tests	(20; NS)	Intra-operative	Cardiac surgery
Marcus et al (2015)	CT	Software+hardware development	User tests	(50; 1)	Intra-operative	Laparoscopic neurosurgery
Diana et al (2017)	MRI	Software development	Clinical tests	(10; 58)	Pre-operative + Intra-operative	Gallbladder surgery
Rosenthal et al (2005)	Ultrasound	Software+hardware development	User tests	(1; NS)	Intra-operative	Breast biopsy
Ru¨ger et al (2020)	Ultrasound	Software+hardware development	User tests	(20; 1)	Intra-operative	Ultrasound-guided needle placements
Wei et al (2016)	CT	Software development	Clinical tests	(1; 40)	Intra-operative	Percutaneous kyphoplasty
Wilson et al (2013)	Non-specific	Software+hardware development	Clinical tests	(34; 16)	Intra-operative	Tension pneumothorax decompression
Patel et al (2023)	Non-specific	Hardware development	User tests	(15; NS)	Intra-operative	Robotic suturing
Barragan et al (2022)	Physio Signal+ Live video	Software development	User tests	(8; NS)	Intra-operative	Robotic surgery
^aRompianesi et al (2023)	Non-specific	Hardware development	Clinical test	(61; NS)	Pre-operative+intra-operative	Hepatectomy

Development indicates the lack of human participants; user test indicates the incorporation of surgeons but no patient participants; clinical test indicates the participation of both surgeons and patients are involved in the study.

^aLarge sample size studies.

Phase of Surgery

Different CSTs act on three main phases of surgery: preoperative, intraoperative, and post-operative. Preoperative includes all planning activities before the onset of the surgery; intraoperative refers to surgical performance during an operation. CSTs categorized to the different phases of surgery act as interventions or visual assistance to the surgeon. 12 (32%) CSTs are responsible for preoperative interventions and 27 (73%) CSTs are responsible for intraoperative interventions. No CST was found to assist in patient rehabilitation after the surgery had been completed.

Types of Surgery

CSTs are present in a wide variety of procedures. General surgery, thoracic surgery, neurosurgery, urology surgery, otolaryngology surgery (ENT), and orthopedic surgery are all applicable for CST use. While it is difficult to categorize them into specific groups, it is certainly clear that there is an emphasis on MIS (n = 11) (Table 1).

Surgical Performance Outcome

A total of 26 (65%) literature reported their findings numerically. Out of the 26, 7 studies reported their findings in a percentage or an error distance, 16 studies reported their results as a success rate in comparison with a control group, and 3 studies included both methods in their quantitative approach. Of the 10 studies with a numerical value for their error rate (Table 2). 10 studies measured errors related to the software output (labeling error, registration error, or staining error), while 2 studies measured the error in surgeon’s performance. While most studies reported accuracy above 90% while keeping range of error are smaller than 5 mm, Rüger et al reported an error exceeding 5 mm, and Thienphrapa et al reported as low as an 82% labelling accuracy for anatomical structures.^16,17 19 articles reported their results as a comparison with a control group (Table 3). 17 (89%) studies reported at least one positive outcome associated with CST uses, including reduced error, increased successful trials, increased efficiency, decreased redirection frequency of surgical tools, increased recognition of pathology, and improved patient outcomes (Table 3). 2 studies reported the difference between CST group and controls were statistically insignificant.^18,19 2 studies reported at least 1 negative impact associated with CST use, which were decreased pathology recognition and image quality.^20,21 Interestingly, contradictive reports were observed between 2 studies; Dixon reported a decrease in recognition while Marcus reported an increase in recognition.^20,22 Additionally, 2 studies involving a surgical predictive tool reported numerical values assessing the efficacy of their algorithm^14,23 (Table 4).

Table 2.

Characteristics of Studies Reporting CSTs Performance as Error Rates.

Study	Parameters Measured	Results	Anatomical Location	Testing Frequency (per Participant)
Rosenthal et al (2002)	Operational error	1.62 mm	Breast	50
Birkfellner et al (2003)	Average fiducial registration error	0.9 mm (max≤ 1.2 mm)	Skull	3
Buck et al (2005)	Registration error	1.04±.45 mm (max≤2 mm)	Heart	NS
Baumhauer et al (2008)	Target visualization error	≥.49 mm	Kidney	3
Baumhauer et al (2008)	Endoscope registration accuracy (±1px)	≥97%	Kidney	3
Debarba et al (2010)	Volume estimate error	1.96±.96% (max≤3.49%)	Liver	NS
Haouchine et al (2013)	Registration error	3.91 mm (max≤4.4 mm)	Liver	NS
Marzano et al (2013)	Registration error	2.8 mm	Pancreas/Duodenum	1
Conrad et al (2016)	Registration accuracy	≤5 mm	Liver	1
Thienphrapa et al (2019)	Anatomical labeling accuracy (≤1 mm)	92%	Lung	NS
Thienphrapa et al (2019)	Anatomical registration accuracy	83% to 99%	Lung	NS
Ruger et al (2020)	Operational error	5.0 mm±2.8 mm	General	4
Barragan et al (2022)	Cognitive load detection accuracy	78%	General	1
*Rompianesi et al (2023)	Staining success rate	57.1%	Liver	1

Table 3.

Characteristics of Studies Reporting Surgeons’ Performances With the Employment of CSTs.

Study	Parameters Measured	Results	Sample size	P value
Birkfellner et al (2003)	Target localization rate	⇓ error	1	N/A
Ogawa et al (2019)	Angle registration accuracy	N.S*	41	N/A
Chi et al (2018)	Root mean square error	⇓ error	6	P < .05
Abhari et al (2015)	Rotational and translational error	⇓ error	21	P < .005
Azagury et al (2010)	Failure rate	⇓ error	3	P = 0·002
^aLinxweiler et al (2020)	Registration error	⇓ error (resident)	100	P < .0001
^aLinxweiler et al (2020)	Registration error	⇓ error (senior physician)	100	P < .019
Dias et al (2020)	Error rate	⇓ error	45	P < .001
Dias et al (2020)	Success rate	⇑ successful trial	45	P < .001
Dilley et al (2019)	Perforation error	⇓ error	36	P = .02
Dilley et al (2019)	Efficiency	⇑ efficiency	36	P = .0004
Dixon et al (2013)	Recognition of unexpected findings	⇓ recognition	31	P < .041
Dixon et al (2013)	Target accuracy	⇓ error	33	P < .001
Grasso et al (2012)	Complications	N.S	180	N.S
Jeon et al (2012)	# Of redirections	⇓ redirections	20	P < .001
Marcus et al (2015)	Recognition of unexpected findings	⇑ recognition	50	P = .025
Marcus et al (2015)	Depth perception	⇑ perception	50	P = .015
Diana et al (2017)	Image quality	⇓ quality	10	P < .0001
Rosenthal et al (2002)	Operational error	⇓ error	50	P < .02
Ruger et al (2020)	Operational error	⇓ error	20	P < .022
Wilson et al (2013)	Success rate	⇑ success	34	P < .008
Wei et al (2019)	Operational success	⇑ patient outcome	50	P < .05
Patel et al (2022)	Knot tensile strength	⇑ knot tensile strength	15	P < .001
Barragan et al (2022)	Tool changing event	⇓ tool change		P < .01
Barragan et al (2022)	Clutching time	⇓ clutching time		P < .01

N.S indicates no statistically significant change. ⇑ indicates increases/improvement in a given category compared to control. ⇓ indicates decreases in a given category compared to control.

^aLarge sample size studies.

Table 4.

Characteristics of Studies Involving Surgical Predictions.

Study	Contribution	Results	Significance
Marz et al (2015)	Patient data driven diagnostic and treatment aid	70 matching patients;38±15 assertions per patient84475 patient-related observations	Assists physician in diagnosis, individualizing treatment and provide follow-up for patients that require liver resections
Schoch et al (2016)	Patient data driven cardiac prosthetic selection aid	55% elimination of unsuitable annuloplasty ring	Increased automation and range of selections for annuloplasty ring

Patient Outcome

9 studies reported patient outcomes after the use of CST in their corresponding surgery (Table 5). 5 (56%) studies reported a minor or no post-operative complications. 2 (22%) studies observed slight bleeding in CST groups.^24,25 3 studies reported serious either intraoperative or post-complication.^21,25 Ntourakis reported a patient with a urinary tract infection and another patient with additional metastasis in CST groups. Diana reported intraoperative conversion from a MIS to an open surgery in the CST group.^21,25

Table 5.

Characteristics of Studies Reporting Patients’ Outcomes With the Employment of CSTs.

Study	Patient Outcomes Measured	Conclusion	Patient Sample Size
Willekes et al (2008)	Post operative complications	No complications	2
Lang et al (2005)	% of patients with complete tumor removal	94.1% patient with complete tumor removal	25
^aLinxweiler et al (2020)	Post operative bleeding	Slight bleeding in experimental group	100
^aLinxweiler et al (2020)	Pain intensity	Reduced pain experimental group	100
Ntourakis et al (2015)	Operative blood loss	0 - 50 mL	3
	Negative surgical margin	.7 - 17 mm	3
	Post operative outcome	1 patient had a urinary tract infection1 patient had additional metastasis	3
Ogawa et al (2019)	Post operative complications	No complications	41
	Pain or numbness	No complications	41
	Scarring	Minor complications	41
^aGrasso et al (2015)	Procedural complications	No significant complications	180
Diana et al (2017)	Surgical complications	2 cases were converted to open surgery;1 case was converted to laparoscopic surgery	58
Wei et al (2016)	Post operative complication	No complications	40
^aRompianesi et al (2023)	Post operative complication	19 cases of complications (25%)	61
	Length of hospital stay	5
	Operative blood loss	150 mL

^aLarge sample size studies.

OR Time Variation

18 studies reported an OR time difference (Table 6). 14 (78%) studies reported a trend in decreasing OR time, while 7 (38%) studies are statistically significant (P < .05). 4 (22%) studies reported an increase in OR time, 3 (18%) are statistically significant. Due to the inherent differences between the included procedures, the operational time between each procedure dramatically varies. To standardize the change in operational time, the differences are converted into a percentage change. Overall, there has been a 27.5% decrease in surgical time with the use of CST in comparison to traditional methods (mean: 27.5%; std: 41.8%) (Figure 3). Although the general pattern of CST use reduces procedural time, it is worth highlighting that robotics procedures tends to have longer operational time than their open surgery counterpart.

Table 6.

Characteristics of Studies Reporting Operation Time Changes With the Employment of CSTs.

Study	Results	Sample size	P value
Chi et al (2018)	⇑	6	P < .05
Abhari et al (2015)	⇓	21	P < .05
Azagury et al (2010)	⇓	3	P < .027
Marzano et al (2010)	⇓	1	N.S
^aLinxweiler et al (2020)	⇑ (senior physicians)	100	P = .3350
^aLinxweiler et al (2020)	⇑ (resident)	100	P = .1934
Ogawa et al (2020)	⇓	41	P = .80
Dias et al (2020)	⇓	45	P < .001
Dilley et al (2019)	⇑	36	P ≤ .01
Dixon et al (2013)	⇓	33	P = .153
Grasso et al (2012)	⇓	180	P < .001
Jeon et al (2012)	⇓	20	P = .0002
Marcus et al (2015)	⇓	50	P = .007
Diana et al (2017)	⇓ (NIR-C vs control)	10	P = .00000003
Diana et al (2017)	⇓ (AR vs control)	10	P = .000013
Ruger et al (2020)	⇓	20	P = .211
Wei et al (2019)	⇓	40	P < .05
Wilson et al (2013)	⇑	34	N.S
Patel et al (2022)	⇓	15	NS
Barragan et al (2022)	⇓	8	P < .01

N.S indicates no statistically significant change. ⇑ indicates increases/improvement in each category compared to control. ⇓ indicates decreases in each category compared to control.

^aLarge sample size studies.

Figure 3.

Procedure duration change with the implementation of CSTs.

Discussion

Medical imaging such as X-rays, CT scans, MRIs, and ultrasound data are widely used in surgical planning and intraoperative guidance. Augmented or mixed reality technologies utilize these images to create visual 3-D models for visual enhancement. However, tissue deformation during surgery can cause inaccuracies in these models. In actual surgery, target registrations are primarily done during the end-exhalation plateau phase of the patient’s respiratory cycle to ensure accurate markers for anatomical structures.²⁶ The margin of error for surgery registration ranges from 1 mm to 15 mm.^27,28

Most CSTs are developed for specific types of surgeries, such as hepatic (ablation) cardiac surgery, vascular surgery with cannulation and catheterization technologies, laparoscopic surgery, lung surgery, and arthroplasty. One of the most important functions of CSTs for surgeons is visual assistance. Clinically, one of the procedures that considerably benefit from visual aid is liver resection, because portal blood vessels may be difficult to visualize under the liver, residual liver volume is hard to estimate, and there may be hidden tumours that are not detected initially.^25,26,29,30 Patient-specific virtual reality anatomical models may be built through preoperative or intra-operative images depending on data availability.^26,30,31 During planning, surgeons may freely rotate the model to detect unexpected tumours and adjust the resection plane accordingly; based on the resection plane, the software returns the residual liver volume as well as security margins around the liver for surgeons to analyze the validity of the plan.^26,30 Patient-specific digital models can be generated to support valve replacement in heart surgeries.^32,33 Trajectory optimization done by machine learning can reduce the risks in cannulation and catheterization procedures.³⁴ In laparoscopic gastrectomy, CSTs serve to recommend ideal port locations for increased tool maneuverability.³⁵ Current CSTs developed for lung biopsy assist the operator by providing a real-time augmented reality video displaying the position of the needle and surrounding tissue.¹⁹ Regardless of the type of operation, CSTs aim to reduce surgeons' cognitive load and decrease surgical complications. Preoperative and intra-operative cognitive support technologies (CSTs) have been developed to reduce the cognitive load of surgeons during surgical procedures. Preoperative CSTs simplify anatomical complexities, aid in prosthetic selection, and provide visualization; the major intended role of preoperative CSTs is to aid the surgeon accurately and efficiently in patient diagnosis and surgical planning. For example, in mitral valve replacement surgery, an algorithm was created to reduce commercial mitral valve options based on patient-specific heart dimensions and medical history.¹⁴ In neurosurgery planning, a CST was developed to create a dynamic 3D model of a patient's brain and pathology, providing recommendations for surgical procedures.^22,36 CSTs should progress from cognitive support for individual surgeons to support for the entire surgical team, utilizing virtual and augmented reality to improve communication and decision-making efficiency. During the intraoperative phase, the consequences of overloading the surgeon’s cognition are often more serious, leading to poorer adverse patient outcomes. Intraoperative cognitive overload primarily comes from visual disturbances, unexpected events, and variable environmental factors.^24,37-39 Thus, the primary role of intraoperative CSTs is to alleviate these problems and to stabilize the surgeon’s cognitive workload. , For example, to address the problem of intraoperative visual disturbances in endoscopic-related surgeries, CSTs that provide a 3D visualization of an endoscope tip and software that allows for navigation aids were often prioritized; this type of assistance is especially beneficial for surgeries that contain a trans-nasal component because navigation is difficult in this subset of surgeries.^24,37,38 In variable and intense environments like the battlefield, Auditory guidance headsets with micro-speaker have been developed to treat needle decompression of tension pneumothoraxes for combat medics.³⁹ Robotic innovations like the Da Vinci robot and the Xi robot are being implemented in an increasing amount of surgical specialties. Innovations in surgical robotics have shown to improve motion stability and decrease tissue destruction.^40,41 Simple CSTs, such as tattooing with methylene markers, have also been developed to aid in target localization.⁴² Robotic surgery can also be combined with intraoperative fluorescent markers to provide surgeons with improved visualization of the anatomical structures around the resection plane and tumour location.⁴³ To assess the cognition levels of surgeons during robotic surgery, machine learning-based CSTs are able to analyze physiological parameters such as electroencephalogram and eye behaviour to detect moments of high cognitive load; in response to cognitive overload, the semi-autonomous program may take control of simple tasks such as blood suction and irrigation.⁴⁴ More invasive neurostimulation techniques were also tested; transcranial direct current stimulation on surgeons completing a knot typing task in robotic surgery was reported to have increased knot tensile strength.⁴⁵ Cognitive surgical technologies (CSTs) incorporating augmented reality (AR) and image guidance have shown promising results in improving precision and reducing errors in various surgical procedures. Similar to other visual aid CSTs, AR-based CSTs also rely on radiology images to create a patient-specific 3-D model. Intraoperatively, anatomical landmarks around the surgical site and security margin are labelled to the surgeons to aid their incisions; clarifications are given to surgeons around complex anatomical regions, and some CSTs provide procedural instruction during the operation. Studies have reported increased accuracy rates in AR or voice-guided surgery, with decreased error rates of up to 10%.^{15,16,24,27,34,36,37,46,47} AR-based CSTs have been effective in treating hepatocellular carcinoma, neonatal intubation, laparoscopic nephrectomy, and needle biopsies, resulting in increased success rates and reduced error rates.^15,19,29,48 However, in some procedures like total hip arthroplasty and breast cancer-related lymphedema diagnoses, the implementation of CSTs did not result in significant differences.^18,49 The presence of CST may also influence procedure time, with increased distraction or unfamiliarity with the device potentially reducing efficiency.^24,34,39,46 However, there is insufficient evidence available on intraoperative CSTs to support the conclusion that it significantly lowers cognitive load and boosts performance, future research should focus on this area. With more development, CSTs may be crucial in reducing the cognitive load of surgeons during surgical procedures, improving communication and decision-making efficiency, and ensuring patient safety. The development of CSTs should continue to focus on reducing procedure complexity, reducing error rates, and aiding intraoperative decision-making to improve surgical outcomes. Current CSTs have limitations that hinder their widespread use in the OR. One major problem is the inadequacy of existing VR and AR technology to handle tissue deformation and organ fluctuation, requiring the intervention of expert computer scientists during surgery.^25,28,38 Additionally, there is a lack of randomized controlled trials and sufficient clinical data due to the disconnection between medical staff and computer scientists, leading to outsourcing clinical trials to actual hospitals after CST development. Furthermore, CSTs often lack ergonomic design for effective surgical implementation, affecting navigation, reducing the field of view, and other ergonomics-related problems.^16,36 Surgeons also have a preference for traditional techniques, and CSTs are often expensive, limiting their use in routine procedures. Learning curve problems and the potential for attentional blindness also pose challenges to CST use. CSTs may be more suitable for complex procedures requiring cognitive assistance, and future development should prioritize ergonomic design to improve usability and maneuverability. Overall, CSTs require further research and development to improve their effectiveness and gain wider acceptance in the ORs.

Conclusion

This review suggests that CSTs have shown promising results in improving surgical performance and reducing errors by increasing surgeons' visual fields. CSTs are currently in development for a wide variety of surgery specialties, and its purpose are often multi-dimensional, aiding surgeons in different areas before and during the procedure. Some advantages of CST, such as accuracy-based parameters, are supported with more evidence than other. However, several challenges still need to be addressed, including inaccuracies in accounting for tissue deformation, a lack of clinical data, particularly patient data, and ergonomic issues with bulky equipment. When developing these CSTs, adequate consideration must also be given to the possibility that the addition of CSTs would fatigue the surgical operator and cause them to regress in performance. Despite these limitations, the potential benefits of CSTs make them a promising area for further development and research, in today’s innovative age, improvements in hardware, software and design could overcome these technological challenges, furthering improving the medical field with interdisciplinary efforts.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Zhong Shi Zhang has been funded by Alberta Innovate Summer Studentship for this research.

ORCID iDs

Zhong Shi Zhang

Bin Zheng

References

Owens

Stoessel

. Surgical site infections: epidemiology, microbiology and prevention. J Hosp Infect. 2008;70(Suppl 2):3-10. doi:10.1016/S0195-6701(08)60017-1.

Coccolini

Catena

Pisano

, et al. Open versus laparoscopic cholecystectomy in acute cholecystitis. Systematic review and meta-analysis. Int J Surg. 2015;18:196-204. doi:10.1016/j.ijsu.2015.04.083.

Martínez-Pérez

de’Angelis

Brunetti

, et al. Laparoscopic vs. open surgery for the treatment of iatrogenic colonoscopic perforations: a systematic review and meta-analysis. World J Emerg Surg. 2017;12:8. doi:10.1186/s13017-017-0121-x.

Nicolás

Czerwonko

Ardiles

, et al. Laparoscopic vs open liver resection for metastatic colorectal cancer: analysis of surgical margin status and survival. Langenbecks Arch Surg. 2022;407(3):1113-1119. doi:10.1007/s00423-021-02396-2.

Mazur

Mosaly

Jackson

, et al. Quantitative assessment of workload and stressors in clinical radiation oncology. Int J Radiat Oncol. 2012;83(5):e571-e576. doi:10.1016/j.ijrobp.2012.01.063.

Lacy

García-Valdecasas

Delgado

, et al. Laparoscopy-assisted colectomy versus open colectomy for treatment of non-metastatic colon cancer: a randomised trial. Lancet Lond Engl. 2002;359(9325):2224-2229. doi:10.1016/S0140-6736(02)09290-5.

van Bergen

Kunert

Bessell

Buess

. Comparative study of two-dimensional and three-dimensional vision systems for minimally invasive surgery. Surg Endosc. 1998;12(7):948-954. doi:10.1007/s004649900754.

Wheelock

Suliman

Wharton

, et al. The impact of operating room distractions on stress, workload, and teamwork. Ann Surg. 2015;261(6):1079-1084. doi:10.1097/SLA.0000000000001051.

Mcmullan

Urwin

Gates

Sunderland

Westbrook

. Are operating room distractions, interruptions and disruptions associated with performance and patient safety? A systematic review and meta-analysis. Int J Qual Health Care. 2021;33(2):mzab068. doi:10.1093/intqhc/mzab068.

10.

Sevdalis

Wong

HWL

Arora

, et al. Quantitative analysis of intraoperative communication in open and laparoscopic surgery. Surg Endosc. 2012;26(10):2931-2938. doi:10.1007/s00464-012-2287-3.

11.

Parker

Mendenhall

Shau

, et al. Minimally invasive versus open transforaminal lumbar interbody fusion for degenerative spondylolisthesis: comparative effectiveness and cost-utility analysis. World Neurosurg. 2014;82(1-2):230-238. doi:10.1016/j.wneu.2013.01.041.

12.

Berguer

Smith

Chung

. Performing laparoscopic surgery is significantly more stressful for the surgeon than open surgery. Surg Endosc. 2001;15(10):1204-1207. doi:10.1007/s004640080030.

13.

Cohen

Lui

Zheng

, et al. Rates of serious surgical errors in California and plans to prevent recurrence. JAMA Netw Open. 2021;4(5):e217058. doi:10.1001/jamanetworkopen.2021.7058.

14.

Schoch

Philipp

Weller

, et al. In: Webster

Yaniv

, eds. Cognitive tools pipeline for assistance of mitral valve surgery. Washington, DC: NASA; 2016

15.

Dias

Greenberg

Goldberg

Fisher

Tanaka

. Augmented reality–assisted video laryngoscopy and simulated neonatal intubations: a pilot study. Pediatrics. 2021;147(3):e2020005009. doi:10.1542/peds.2020-005009.

16.

Rüger

Feufel

Moosburner

Özbek

Pratschke

Sauer

. Ultrasound in augmented reality: a mixed-methods evaluation of head-mounted displays in image-guided interventions. Int J Comput Assist Radiol Surg. 2020;15(11):1895-1905. doi:10.1007/s11548-020-02236-6.

17.

Thienphrapa

Bydlon

Chen

, et al. Interactive endoscopy: a next-generation, streamlined user interface for lung surgery navigation. In: Shen

Liu

Peters

, eds. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Vol 11768. Lecture Notes in Computer Science. Berlin, Germany: Springer International Publishing; 2019:83-91. doi:10.1007/978-3-030-32254-0_10.

18.

Ogawa

Kurosaka

Sato

Hirasawa

Matsubara

Tsukada

. Does an augmented reality-based portable navigation system improve the accuracy of acetabular component orientation during tha? A randomized controlled trial. Clin Orthop. 2020;478(5):935-943. doi:10.1097/CORR.0000000000001083.

19.

Grasso

Luppi

Cazzato

, et al. Percutaneous computed tomography-guided lung biopsies: preliminary results using an augmented reality navigation system. Tumori. 2012;98(6):775-782. doi:10.1177/030089161209800616.

20.

Dixon

Daly

Chan

Vescan

Witterick

Irish

. Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc. 2013;27(2):454-461. doi:10.1007/s00464-012-2457-3.

21.

Diana

Soler

Agnus

, et al. Prospective evaluation of precision multimodal gallbladder surgery navigation: virtual reality, near-infrared fluorescence, and X-ray-based intraoperative cholangiography. Ann Surg. 2017;266(5):890-897. doi:10.1097/SLA.0000000000002400.

22.

Marcus

Pratt

Hughes-Hallett

, et al. Comparative effectiveness and safety of image guidance systems in neurosurgery: a preclinical randomized study. J Neurosurg. 2015;123(2):307-313. doi:10.3171/2014.10.JNS141662.

23.

März

Hafezi

Weller

, et al. Toward knowledge-based liver surgery: holistic information processing for surgical decision support. Int J Comput Assist Radiol Surg. 2015;10(6):749-759. doi:10.1007/s11548-015-1187-0.

24.

Linxweiler

Pillong

Kopanja

, et al. Augmented reality‐enhanced navigation in endoscopic sinus surgery: a prospective, randomized, controlled clinical trial. Laryngoscope Investig Otolaryngol. 2020;5(4):621-629. doi:10.1002/lio2.436.

25.

Ntourakis

Memeo

Soler

Marescaux

Mutter

Pessaux

. Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg. 2016;40(2):419-426. doi:10.1007/s00268-015-3229-8.

26.

Hansen

Köhn

Schlichting

, et al. Intraoperative modification of resection plans for liver surgery. Int J Comput Assist Radiol Surg. 2008;3(3-4):291-297. doi:10.1007/s11548-008-0161-5.

27.

Birkfellner

Figl

Matula

, et al. Computer-enhanced stereoscopic vision in a head-mounted operating binocular. Phys Med Biol. 2003;48(3):N49-57. doi:10.1088/0031-9155/48/3/402.

28.

Haouchine

Dequidt

Peterlik

Kerrien

Berger

Cotin

. Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery. In: 2013 IEEE international symposium on mixed and augmented reality (ISMAR), Adelaide, Australia, 1-4 October 2013

29.

Lang

Radtke

Hindennach

, et al. Impact of virtual tumor resection and computer-assisted risk analysis on operation planning and intraoperative strategy in major hepatic resection. Arch Surg. 2005;140(7):629-638. doi:10.1001/archsurg.140.7.629

30.

Debarba

Zanchet

Fracaro

Maciel

Kalil

. Efficient liver surgery planning in 3D based on functional segment classification and volumetric information. Annu Int Conf IEEE Eng Med Biol Soc. 2010;2010:4797-4800. doi:10.1109/IEMBS.2010.5628026.

31.

De Buck

Maes

Ector

, et al. An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures. IEEE Trans Med Imaging. 2005;24(11):1512-1524. doi:10.1109/TMI.2005.857661.

32.

Hagenah

Evers

Scharfschwerdt

Schweikard

Ernst

. A Support Vector Regression-Based Data-Driven Leaflet Modeling Approach for Personalized Aortic Valve Prosthesis Development. Lübeck, Germany: University of Lubeck; 2018

33.

Hagenah

Mehdi

Ernst

. Generating healthy aortic root geometries from ultrasound images of the individual pathological morphology using deep convolutional autoencoders. In: 2019 Computing in Cardiology (CinC), Singapore, 08-11 September 2019

34.

Chi

Liu

Abdelaziz

MEMK

, et al. Trajectory optimization of robot-assisted endovascular catheterization with reinforcement learning. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain, 01-05 October 2018

35.

Hayashi

Misawa

Mori

. Optimal port placement planning method for laparoscopic gastrectomy. Int J Comput Assist Radiol Surg. 2017;12(10):1677-1684. doi:10.1007/s11548-017-1548-y.

36.

Abhari

Baxter

JSH

Chen

ECS

, et al. Training for planning tumour resection: augmented reality and human factors. IEEE Trans Biomed Eng. 2015;62(6):1466-1477. doi:10.1109/TBME.2014.2385874.

37.

Azagury

Ryou

Shaikh

, et al. Real-time computed tomography-based augmented reality for natural orifice transluminal endoscopic surgery navigation. Br J Surg. 2012;99(9):1246-1253. doi:10.1002/bjs.8838.

38.

Marzano

Piardi

Soler

, et al. Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg. 2013;17(11):1980-1983. doi:10.1007/s11605-013-2307-1.

39.

Wilson

Doswell

Fashola

, et al. Using augmented reality as a clinical support tool to assist combat medics in the treatment of tension pneumothoraces. Mil Med. 2013;178(9):981-985. doi:10.7205/MILMED-D-13-00074.

40.

Trastulli

Desiderio

Lin

, et al. Open vs robotic gastrectomy with D2 lymphadenectomy: a propensity score-matched analysis on 1469 patients from the IMIGASTRIC prospective database. Langenbecks Arch Surg. 2023;408(1):302. doi:10.1007/s00423-023-03032-x.

41.

Toesca

Sangalli

Maisonneuve

, et al. A randomized trial of robotic mastectomy versus open surgery in women with breast cancer or BrCA mutation. Ann Surg. 2022;276(1):11-19. doi:10.1097/SLA.0000000000004969.

42.

Willekes

Boutros

Goldfarb

. VATS intraoperative tattooing to facilitate solitary pulmonary nodule resection. J Cardiothorac Surg. 2008;3(1):13. doi:10.1186/1749-8090-3-13.

43.

Rompianesi

Pegoraro

Ramaci

Ceresa

Montalti

Troisi

. Preoperative planning and intraoperative real-time navigation with indocyanine green fluorescence in robotic liver surgery. Langenbecks Arch Surg. 2023;408(1):292. doi:10.1007/s00423-023-03024-x.

44.

Barragan

Yang

Wachs

. A neurotechnological aid for semi-autonomous suction in robotic-assisted surgery. Sci Rep. 2022;12(1):4504. doi:10.1038/s41598-022-08063-w.

45.

Patel

Suwa

Kinross

, et al. Neuroenhancement of surgeons during robotic suturing. Surg Endosc. 2022;36(7):4803-4814. doi:10.1007/s00464-021-08823-1.

46.

Dilley

JWR

Hughes-Hallett

Pratt

, et al. Perfect registration leads to imperfect performance: a randomized trial of multimodal intraoperative image guidance. Ann Surg. 2019;269(2):236-242. doi:10.1097/SLA.0000000000002793.

47.

Rosenthal

State

Lee

, et al. Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med Image Anal. 2002;6(3):313-320. doi:10.1016/S1361-8415(02)00088-9.

48.

Baumhauer

Simpfendörfer

Müller-Stich

, et al. Soft tissue navigation for laparoscopic partial nephrectomy. Int J Comput Assist Radiol Surg. 2008;3(3-4):307-314. doi:10.1007/s11548-008-0216-7.

49.

Invernizzi

Runza

De Sire

, et al. Integrating augmented reality tools in breast cancer related lymphedema prognostication and diagnosis. J Vis Exp. 2020:156:60093. doi:10.3791/60093.

A Review of Cognitive Support Systems in the Operating Room

Abstract

Background

Methods

Results

Discussion

Keywords

Introduction

Method

Search Criteria

Data Analysis

Results

Data Source

Phase of Surgery

Types of Surgery

Surgical Performance Outcome

Patient Outcome

OR Time Variation

Discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iDs

References