Abstract
Introduction
Centralization of oncological care results in a growing demand for specialized consultations and referrals. Improved telemedicine solutions are needed to facilitate access to specialist care and select patients eligible for referral. The purpose of this quality improvement initiative was to optimize transmural care for patients suffering from colorectal cancer liver metastases through implementation of an online expert panel.
Methods
A digital communication platform was developed to share medical data, including high-quality diagnostic imaging of patients suffering from colorectal cancer liver metastases. Feasibility of local treatment strategies was assessed by a panel of liver specialists to select patients for referral. After implementation, an observational cohort study was conducted to evaluate quality improvement in transmural care using revised Standards for Quality Improvement Reporting Excellence guidelines.
Results
From September 2016–September 2018, eight hospitals were connected to the platform, covering a population of 3 m. In total, 123 cases were assessed, of which 54 (43.9%) were prevented from needless physical referral. Assessment of treatment strategy by an online expert panel significantly reduced the average lead time during multidisciplinary team meetings from 3.73 min to 2.12 min per patient (p < 0.01).
Conclusions
Implementation of an online expert panel is an innovative, accessible and user-friendly way to provide cancer-specific expertise to regional hospitals. E-consultation of such panels may result in more efficient multidisciplinary team meetings and prevent fragile patients from needless referral. Sustainability of these panels however is subject to structural financial compensation, so a cost-effectiveness analysis is warranted.
Keywords
Introduction
Centralization and specialization in oncological care result in a growing demand for specialized consultations and referrals. 1 , 2 With exponential increase in medical knowledge and technical advances, regional hospitals may lack cancer-specific expertise and need to refer patients to specialized cancer centres. 3 However, which patients to refer is not always clear, resulting in potential undertreatment or unnecessary referrals of patients with cancer.4–10
For patients suffering from colorectal cancer liver metastases (CRLMs), local treatment comprising of resection and local ablation is the only potentially curative treatment.11–13 Most patients with CRLMs are diagnosed in regional hospitals specialized in colorectal surgery, while local treatment of CRLMs is exclusively performed in dedicated liver centres. 7 , 14 To select patients for referral, feasibility of local treatment strategies is therefore often assessed during multidisciplinary team (MDT) meetings using videoconference facilities.
Several barriers, however, have been identified for optimal decision-making during these meetings, including technical issues and time pressure.15–18 Poor image quality during videoconferencing can hamper proper assessment of treatment strategy. Moreover, these meetings are often time-consuming and put a strain on both specialists and hospital resources. Quality of care can be affected by non-attendance at these meetings, as these meetings need to be scheduled between clinical duties. 16 , 17 , 19 , 20
Other telemedicine solutions, like written digital requests and e-consultation of fully virtual tumour boards, have been developed to facilitate access to specialist cancer care, in a non-simultaneous manner.21–23 E-consultation of multiple experts, including liver surgeons and interventional radiologists, is however encouraged as interobserver variability with regard to feasibility of local treatment strategies is common, even between liver specialists.24–26 Virtual tumour boards, which interact asynchronously in contrast to MDTs using audiovisual teleconferencing for real-time communication, may therefore be more suitable. 23
As the urgency of an improved strategy to make multidisciplinary liver expertise accessible in regional hospitals appeared evident, a quality improvement initiative was rolled out to optimize transmural care for patients suffering from CRLMs. A virtual tumour board (also known as online expert panel) was implemented to consult multiple liver specialists in order to provide cancer-specific expertise and select patients for referral.
Methods
Setting
Treatment strategy of patients suffering from CRLMs is regularly evaluated during weekly MDT meetings to discuss patients with (a history of) colorectal cancer in hospitals specialized in colorectal surgery. Several specialists, including at least one diagnostic radiologist, colorectal surgeon, medical oncologist, radiation oncologist, gastroenterologist and pathologist, attend these meetings in person, while in cases of non-liver centres a liver surgeon is consulted through videoconferencing to assess feasibility of local treatment strategies prior to patient referral. In the North Western region of the Netherlands, liver surgeons from three dedicated liver centres: two teaching hospitals and an academic liver centre, can be consulted during these meetings. Proper selection of patients eligible for local treatment of CRLMs is important, not only for patients’ anticipation, but also for management of patient flow, as local treatment is solely performed in dedicated liver centres, while systemic treatment is preferably offered in regional hospitals, closer to home.
Project
In collaboration with referring partners, a regional online expert panel of liver specialists was designed to improve access to specialist care, as part of the Moving towards Regional Oncology Networks programme of the Dutch Ministry of Health, Welfare and Sport. Hereby, a multidisciplinary team specialized in local treatment strategies for CRLMs could be consulted on demand, on every working day of the week. The primary objective of this quality improvement initiative was to provide multidisciplinary liver expertise in non-liver centres, in order to select patients eligible for referral. Other objectives were enhancement of timely communication, medical knowledge exchange and equity of care.
Preparation
From January 2016, safety and privacy aspects were examined of various Web-based viewers by a data protection officer. In this pilot phase, EVOCS (Fysicon BV, part of Canon Medical Systems Corporation) was chosen as the online portal with a pay-per-view license to connect healthcare professionals in the referral region. This secured platform was previously used by radiologists and neurologists to reduce ‘door-to-needle’ time in patients with acute cerebrovascular accidents, also known as StrokeNet. 27 To make the platform accessible and user-friendly for healthcare professionals in oncological care, capacity and performance were increased to support exchange of large Digital Imaging and Communications in Medicine (DICOM) files, like magnetic resonance imaging (MRI), positron emission tomography (PET) and computed tomography (CT) scans. Other features, like contrast enhancement and simultaneous display of two image series or modalities to enable tumour response assessment within the platform were realized (see Figure 1). To make DICOM files directly retrievable, Picture Archiving and Communication Systems (PACS) were linked to the platform. Moreover, two chat rooms were developed, so the sender was able to contact the members of the expert panel, and treatment strategy could be discussed within the panel asynchronously. A notification of a new message was sent to the panelists by mail to enable timely communication.

Online platform to assess feasibility of local treatment strategies in patients suffering from colorectal cancer liver metastases (CRLMs).
Implementation
From September 2016, the platform was implemented in a specialized colorectal cancer centre and an academic liver centre to tackle initial teething problems in the workflow. Radiologists and liver surgeons from the academic centre were connected to the platform. Personalized log-ins with accompanying passwords were supplied. Medical staff were trained to upload medical information to the platform after the patient’s informed consent was obtained. Focus groups were organized with patients, healthcare professionals, lawyers and liability insurance representatives to determine requirements in the use of online expert panels according to the current legislation. 28
Expansion
From January 2017–September 2018, healthcare professionals from the remaining general hospitals in our referral region were connected to the platform. Moreover, the expert panel expanded with expertise of an interventional radiologist and liver surgeons from a second and third liver centre in the region. In cooperation with the supplier of electronic patient’s files, an import order was developed to download DICOM files from the platform into the PACS system of the academic centre. Hereby, a revision of diagnostic imaging was organized semi-automatically.
Design
An observational cohort study was conducted. The revised Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines were used. 29 Approval of the Institutional Ethical Review Board of VU University Medical Centre was granted, with a waiver of informed consent (METc VUmc 2019.600).
Data collection
From September 2016–September 2019, data regarding patient and tumour characteristics, diagnostic imaging, treatment advice and motives to follow or waive panel advice were prospectively collected by the project coordinator to monitor quality of care and patient referral. Data were extracted from the platform, standardized reports and electronic patient files. From May–October 2018, all MDT meetings discussing patients with CRLMs through videoconferencing were monitored in Amsterdam UMC, location VUmc, the academic liver centre to determine the added value of an online expert panel in transmural care, defined as a reduction in lead time per patient. The presence of extrahepatic metastases, attendance of liver experts, treatment advice and lead time per patient were prospectively registered.
Data analysis
Patients discussed by the expert panel were compared with those not discussed by the expert panel prior to the MDT meeting. Statistical analysis of data was performed using IBM SPSS Statistics 22.0 software. A Mann-Whitney U test was used to compare the difference in average lead time between groups. A p-value of <0.05 was considered statistically significant.
Results
Over a two-year period, all eight hospitals in the described referral region were connected to the platform, covering a population of 3 m. Four liver surgeons from two teaching hospitals and three liver surgeons from the academic liver centre participated in the panel, as well as a diagnostic radiologist specialized in liver pathology and an interventional radiologist from the academic centre.
Workflow
Relevant medical information, including medical history, medical examination, diagnostic imaging with official radiology and pathology reports were uploaded to the platform by case managers, nurse practitioners or medical doctors after informed consent. Diagnostic imaging was retrieved through the patient or social security number. A notification was automatically sent by mail to inform the panel members about a new request. Diagnostic imaging was reviewed by a diagnostic or interventional radiologist. The official radiology report was revised, if needed. Subsequently, the interventional radiologist and two liver surgeons, selected from an academic and general teaching hospital based on availability by a member of the project staff, independently assessed feasibility of local treatment of CRLMs. If no consensus was reached, a third liver surgeon was asked to reassess treatment strategy. Panel advice included one of the following options: (a) local treatment is feasible, (b) local treatment is not feasible or (c) discussion in a dedicated liver MDT meeting with expertise of a medical oncologist, radiation oncologist and/or nuclear radiologist is needed. If patients were deemed eligible for local treatment of CRLMs, based on technical and oncological aspects, treatment strategy was specified and referral was requested. Within two weekdays, reports were sent out via the platform to facilitate local MDTs in conducting a personalized treatment plan for every patient with CRLMs in the referral region (Figure 2).

Infographic of the workflow of an online expert panel in transmural care for patients suffering from colorectal cancer liver metastases (CRLMs) (source: www.oncologienetwerken.nl). MDT: multidisciplinary team.
Panel advice
Over the described timeline, the online expert panel discussed 123 cases (i.e. 104 unique patients). A total of 245 cross-sectional imaging examinations, including 69 (28.2%) contrast-enhanced (ce) MRI scans, 82 (33.5%) ce CT scans and 94 (38.4%) 18F-fluorodeoxyglucose (FDG) PET-CT scans were assessed.A median of two examinations (range 1–11) were reviewed per case. In 117/123 (95.1%) cases, panel advice was sent out in time. Needless referral was prevented in 15 (12.2%) cases, as the suspicion for (recurrent) CRLMs was not confirmed after revision of diagnostic imaging. In one of these cases, additional diagnostic imaging was performed in a regional hospital opposed to panel advice, where the presence of metastatic disease was confirmed. Patient and tumour characteristics of the remaining 108 cases are presented in Table 1. The majority of patients had (a history of) synchronous liver metastases and presented with bilobar disease. The online expert panel deemed local treatment of CRLMs feasible in 66 (61.1%) cases. However, based on oncological aspects (i.e. disease and tumour-related characteristics), systemic treatment was advised in 19 of these cases, resulting in 47 (43.5%) patients eligible for local treatment strategies. In 31/39 (79.5%) patients not considered eligible for local treatment of CRLMs, the panel suggested participation in ongoing clinical trials. Overall, panel advice was followed in 104 (96.3%) cases. Reasons to differ from the personalized treatment plan were: more extensive disease after additional diagnostic imaging (n = 3) and start of systemic induction therapy to reduce the procedural risk by downsizing tumour load (n = 1).
Characteristics of patients with colorectal cancer liver metastases (CRLMs) discussed by the online expert panel between September 2016–2019.
Lead time
From May–October 2018, 776 patients with (a history of) colorectal cancer were discussed during videoconference with the academic liver centre to assess treatment strategy, of which 194 (25.0%) presented with liver metastases. One hundred and fifteen patients presented with liver-only metastases, of which 12 (10.4%) patients were previously assessed by the online expert panel. Remarkably, in 48 (41.7%) patients with liver-only metastases, liver expertise was not sufficient during these MDT meetings, which resulted in a delay in diagnosis or treatment plan. Assessment of treatment strategy by the online expert panel prior to MDT meetings reduced lead time from 3.73 min to 2.12 min per patient (p < 0.01).
Discussion
Implementation of an online expert panel appeared to be a feasible tool to provide non-liver centres with multidisciplinary liver expertise to select patients for referral. The digital communication platform was easily implemented in several hospitals within the referral region. E-consultation of the expert panel prior to MDT meetings reduced average lead time per patient. Moreover, patients not eligible for local treatment strategies were averted from needless referral, resulting in improved transmural care for patients suffering from CRLMs.
Liver expertise was considered insufficient to recommend treatment in 41.7% of patients with liver-only metastases during regular MDT meetings in the participating hospitals. In theory, these patients could have benefited from consultation of this panel. Moreover, e-consultation of an online expert panel prevented needless referral in 43.9% of cases, as presence of liver metastases was not confirmed after revision of diagnostic imaging or systemic treatment in the consulting centre was advised. Of these patients, just one patient turned out to be undertreated, since liver metastases were diagnosed on additional diagnostic imaging in the referring hospital. In addition, 43.5% of patients were deemed eligible for upfront local treatment strategies, as compared to ∼20% in previously published population-based studies. 9 , 30 However, reduced undertreatment could not be concluded on the basis of the current study, as the panel was consulted in a selection of patients and no direct comparison with standard of care was performed.
In addition, the current study showed that triage of patients by the online expert panel resulted in more efficient MDT meetings and psychological, financial and time burdens associated with needless referral were spared. Moreover, high compliance with treatment advice was observed, which suggests reliability of this panel. Uncertainty regarding eligibility for potential curative treatment may be reduced, as multiple liver specialists assess feasibility of local treatment strategies, potentially resulting in less need for second opinions.
To our knowledge, this is the first study that assessed feasibility of an online multidisciplinary expert panel in transmural care, even though expert panels have been used as part of clinical trials. 24 , 25 Different laws and legislation exist for medical practice according to standard healthcare versus research. Additionally, privacy regulations have been strengthened worldwide. 15 , 28 , 31 , 32 Despite these regulations, we have shown that implementation of a fully virtual panel is feasible in clinical practice.
Several forms of hybrid MDTs have, however, been studied, mostly using videoconferencing for reasons of convenience or geography. 23 ,33–40 These meetings require multisite coordination with available technology at all sites. 39 Moreover, expertise is often not available on demand, as these (bi-) weekly meetings need to be scheduled ahead to ensure simultaneous presence of key members. All in all, e-consultation of specialists has been widely adopted to improve access to specialist care and hereby reduce regular visits. 22 ,41–52 However, studies reporting on direct effects of e-consultation on patient outcome are scarce. 41 , 53 , 54 Where most studies focused on consultation of a single specialist by primary care providers, we aimed to optimize collaboration between secondary and tertiary care professionals, as the value of specialist decision-making in treatment strategy for patients suffering from CRLMs has been shown. 10 , 55 Implementation of e-consultation services with multiple sites involved was mostly successful in the USA and Canada, where a nationally unified electronic health record is employed. 21 , 43 , 56 In the Netherlands, no unified electronic health record exists. Each hospital manages their own electronic health record system. Therefore, a fully digital workflow was developed to enable hospitals in the referral region to consult the panel, regardless of electronic health record system. Moreover, multiple experts from various dedicated liver centres could hereby deliberate on treatment strategy. Consequently, medical knowledge exchange and equity of care were promoted. The downside however was that the platform was not integrated in the electronic health record system, which may hamper the workflow and upscaling in the future.
An extensive description of the implementation and workflow of the online expert panel was reported for reproducible purposes so, to interpret our findings accordingly, several limitations of this project need to be addressed. First, ‘face-to-face’ interaction with live discussion between care providers was not possible during assessment of treatment strategies by the online expert panel. The value of interaction during MDT meetings has been reported, even though social factors may affect decision-making in these meetings negatively. 23 ,57–59 We therefore believe that e-consultation of an online expert panel appears exceptionally appropriate for technical assessment or triage of patients, in contrast to discussions with regard to eligibility based on patient’s fitness or preferences. Additionally, it remained important to reason why potential curative treatment was (not) deemed feasible in patients with CRLMs, as care-givers needed to be informed, but also to reduce the knowledge gap between the consulting centre and the expert panel. 60 Moreover, a relatively small cohort study was conducted to evaluate the benefit of this panel. No pre-implementation measurements were obtained, which hampered proper assessment of improvement of care. Direct effects on patient outcome were difficult to quantify, as randomized-controlled or blind case re-evaluation were not considered ethical and would interfere with the workflow. Nevertheless, lead time per patient during regular MDT meetings was monitored to determine the added value of this panel with regard to the current battle with time constraints. Consultation of the online expert panel happened, however, in a non-random selection of patients. Findings in the present study are therefore subject to selection bias. In addition, the satisfaction of users was not surveyed. The platform and workflow were gradually adjusted based on feedback of users during implementation of the panel. Last, no assessment of cost-effectiveness was performed, as the feasibility and value of an online expert panel was assessed in addition to MDT meetings. However, we hypothesize that implementation of online expert panels may result in a cost reduction in oncological care, as these panels may replace (a part of) regional MDTs in the nearby future, particularly as current MDT meetings are struggling with time pressure.
Regional and national expert panels are currently rolled out for patients with other malignancies, like brain tumours, advanced rectal tumours and pancoast tumours, to reduce lead time from diagnosis to treatment and ultimately improve patient outcome. Considerations to consult the panel need to be further clarified. To make online expert panels sustainable in oncological care, financial compensation is needed from healthcare insurance agencies for this kind of medical service, like a second opinion. 21 , 61 A cost-effectiveness analysis, based on the quadruple aim framework, will help to develop this business model. 53 Moreover, besides the considerations of care providers to consult an online expert panel, we are looking into patient perspectives towards e-consultation of an expert panel with regard to potential privacy concerns.
In conclusion, we have shown that implementation of an online expert panel is feasible in clinical practice. E-consultation of an online expert panel is an innovative, accessible and user-friendly way to provide regional MDTs with cancer-specific expertise and improve transmural care. Direct effects of e-consultation of an online expert panel on patient outcome however need to be further clarified. Based on the findings of the current study, we advocate a prominent role for online expert panels in transmural care for patients suffering from CRLMs.
Footnotes
Acknowledgements
This article was produced on behalf of the following collaborators: Joris JA Joosten, Steven J Oosterling, Ronald JCLM Vuylsteke, Jan-Willem D de Waard and Jan Hein TM van Waesberghe. The authors would like to thank the other members of the project team: ECJ van Kessel, LS Rieter, J Bullens, M van den Heuvel, FE Schuurmans and SLJ Delissen, from Amsterdam UMC, location VUmc, Amsterdam, and LCM Dol, DC Ruijter and C Plasschaert from Northwest Clinics, Alkmaar, AA van Zweeden, from Amstelland Hospital, Amstelveen, S Muller-Hartemink and M Corver from Zaans Medical Center, Zaandam, CCM Keijzer-Steur and SAC Breg from Dijklander Hospital, Hoorn, and P Dienaar and P Hoekstra from Spaarne Gasthuis, Hoofddorp, for their contribution to the project. The following author contributions were made: TH, BZ and GK contributed to the conception and design of the project. All authors contributed to data acquisition. Data analysis, data interpretation and article drafting were performed by TH. MS, MM, WS, BZ and GK revised the manuscript critically for important intellectual content. All authors gave final approval of the manuscript to be submitted.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: The authors declare that there is no conflict of interest, besides the governmental funding of the project.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by The Netherlands Organisation for Health Research and Development (ZonMw) (grant: Citrien Fund), as part of the Moving towards Regional Oncology Networks programme.
