Abstract
Liver vessel segmentation from routinely performed medical imaging is a useful tool for diagnosis, treatment planning and delivery, and prognosis evaluation for many diseases, particularly liver cancer. A precise representation of liver anatomy is crucial to define the extent of the disease and, when suitable, the consequent resective or ablative procedure, in order to guarantee a radical treatment without sacrificing an excessive volume of healthy liver. Once mainly performed manually, with notable cost in terms of time and human energies, vessel segmentation is currently realized through the application of artificial intelligence (AI), which has gained increased interest and development of the field. Many different AI-driven models adopted for this aim have been described and can be grouped into different categories: thresholding methods, edge- and region-based methods, model-based methods, and machine learning models. The latter includes neural network and deep learning models that now represent the principal algorithms exploited for vessel segmentation. The present narrative review describes how liver vessel segmentation can be realized through AI models, with a summary of model results in terms of accuracy, and an overview on the future progress of this topic.
Keywords
Introduction
Blood vessel segmentation through artificial intelligence (AI) has gained increased interest over recent decades as it enables the acquisition of organ vessel three-dimensional (3D) reconstructions that are crucial for the diagnosis, treatment delivery, and prognosis evaluation of many different diseases, including liver diseases, such as liver cancer. Medical imaging, including ultrasonography, computed tomography (CT), and magnetic resonance (MR), is routinely performed in patients with chronic liver disease or suspected liver malignancy to identify and characterize liver neoplasms. Liver CT and MR can easily detect liver lesions and identify their benign or malignant nature with elevated accuracy. In the case of hepatocellular carcinoma, the observation of some atypical characteristics after multiphase contrast-enhanced CT or MR imaging may even be used to establish a full diagnosis, avoiding the need of invasive procedures, such as liver biopsy. 1 Moreover, preoperative imaging is essential for planning an intervention when liver tumors are eligible for surgical or ablative procedures. Definition of the vascular anatomy and subsequent liver segmentation allow a radical oncological resection or ablation of the tumor without sacrificing an excessive amount of healthy parenchyma, thus avoiding the risk of post-procedural liver failure and making successive possible interventions feasible. In addition, liver MR and CT provide a reference for evaluating the effectiveness of treatments as they can be used to assess the presence of residual cancer.
Three-dimensional reconstruction of the liver and its anatomical components has been shown to have many useful applications. Firstly, reproducing a 3D image of the liver gives physicians more accurate information concerning tumor anatomy and its relationship with the surrounding vessels, as well as tumor volume and postoperative future liver remnant, compared with classic two-dimensional (2D) images. 2 Liver resection margins are one of the most important factors affecting tumor recurrence rate and overall survival in both primary liver cancer and liver metastatic disease.3,4 For example, resection margins in the case of liver metastases from colorectal cancer deeply affect outcomes depending on whether the resection is complete or not and, in the latter case, whether there is residual tumor burden on the liver parenchyma or on a vessel. 5 A precise definition of the liver and its vascular anatomy facilitates a surgical strategy that will obtain the best possible oncological results, reducing risks from intraoperative complications associated with unexpected anatomical variations. In addition, other invasive procedures, such as percutaneous ablation, need a perfect knowledge of the liver anatomy to achieve the best possible results and reduce the risk of injury to adjacent organs. 6 Furthermore, patients may also benefit from 3D reconstructions as they increase patient involvement, helping to better understand the disease, principles of treatments, and possible post-procedural complications. 7
Blood vessel segmentation of the liver from 2D images is the hardest process to perform due to the anatomic characteristics of liver vessels. The liver arteries and double venous liver system constitute a thick tree that defines the functional segmentation of the parenchyma. 8 In addition to anatomic complexity, technical issues, such as low contrast images or high noise, make liver vascular segmentation challenging, often resulting in the creation of improper connections between different vascular systems. 9 Manual segmentation of the hepatic vessels is time consuming as it demands a slice-by-slice analysis, and it is strongly operator dependent. 10 For this reason, researchers have attempted to create semi-automated and automated vessel segmentation processes, with the aim of obtaining quick and thorough reconstructions. The published literature is full of attempts to find a rapid and highly accurate method of performing liver vascular segmentation. 11 However, many limitations remain to be overcome and no fully automatic, repeatable and reliable method of segmentation is currently available for clinical practice.
The number of published articles on organ segmentation is growing constantly. In his research, 12 Dakua reviewed state of the art segmentation algorithms for different organs, while Ansari et al. 13 reviewed different segmentation models that have been specifically proposed for liver disease (and hepatocellular liver cancer in particular) diagnosis and treatment.
Although AI-driven tools are constantly being developed with the aim of facilitating the diagnostic and therapeutic process for many diseases, the effective advantages of this technology are yet to be proven. As an example, the role of semi-automatic and automatic detection tools based on medical imaging has been evaluated concerning their reliability in the diagnostic process compared with the manual procedure. In the research by Akhtar et al, 14 a simulation to evaluate how models could measure tumor volumes based on the time of disease relapse after surgery showed that both semi-automatic and automatic algorithms tend to underperform manual measure. Conversely, manual segmentation is extremely time consuming, particularly when an elevated number of anatomical details, such as liver vessels, is provided. Although there are many reports of liver parenchyma and tumor segmentation, few have also focused on semi-automatic and automatic models to perform liver vessel segmentation.
The present narrative review aims to provide the medical community with an overview and vulgarization of the technical aspects of developing and testing the main methods used for liver vascular segmentation and how they have recently evolved, with a highlight on new insights provided by AI-driven methods, and discussion of future perspectives.
Methods
This narrative review was produced by analyzing manuscripts from the PubMed, Embase, Scopus, Google Scholar, and Cochrane Library databases, published between 1 January 2010 and 31 December 2023, that were identified using the following search terms: ‘(liver) AND (segmentation) AND ((vessel) OR (vascular) OR (portal vein) OR (hepatic vein)) AND ((artificial intelligence) OR (thresholding) OR (region) OR (edge) OR (model) OR (machine learning))’.
Liver vascular anatomy
The liver has a peculiar vascular anatomy composed of one main artery and two venous systems, the portal vein and the hepatic veins. The arterial vascular anatomy depends on the proper hepatic artery, which is the terminal artery of the common hepatic artery, coming from the coeliac trunk, bringing oxygenated blood to the liver. In the hepatic hilum, the proper hepatic artery bifurcates into the right and left hepatic arteries, which further divide in the liver parenchyma to give the sectorial and segmental arteries that vascularize each independent liver functional unit. 15 This modal anatomy of the hepatic artery concerns only about 75% of individuals, as the left and right hepatic arteries can sometimes originate from other branches of the coeliac trunk or from the superior mesenteric artery. 16
Blood that comes from the foregut, midgut, and hindgut drains to the liver via the portal vein. In the lesser omentum, the portal vein divides into the right and left portal veins, which follow the same artery distribution to liver sectors and segments. Although the anatomy of the proximal portion of the portal vein is constant, the intra-parenchymal divisions of the portal vein can vary between individuals. 17
Finally, the hepatic veins collect blood from the hepatic sinusoids to drain into the inferior cava vein. Three hepatic veins (right, middle, and left) separate the liver into four sectors (right posterior, right anterior, left medial, and left lateral). 18
Thanks to this peculiar anatomical vascular system, the liver can perform metabolic, storage, digestive, and detoxification functions in cooperation with other organs, notably the gastrointestinal system. What is absorbed by the bowel is delivered to the liver through the portal vein tree and, once the hepatocytes have fulfilled their purposes, the metabolites are expelled in the blood circulation through the hepatic veins. Meanwhile, the hepatic artery supplies the liver with the sources of energy that it needs to carry out its functions. As they are involved in different roles, a deficiency occurring in only one of the three connected vascular systems will affect liver function. 19
Imaging techniques and methods for liver vascular segmentation
Liver vascular segmentation is realized by exploiting the images obtained with the most common routine techniques for diagnosis and follow-up of liver disease: ultrasonography, CT, and MR imaging. Ultrasonography is usually performed as a first-level screening technique, which is then followed by CT scan in case of doubt of pathological findings. Liver MR imaging is usually performed to complete diagnosis and liver lesions characterization when CT scan is insufficient. Hence, CT with or without contrast-enhancement remains the most frequently used imaging technique for liver disease diagnosis and follow-up. This is corroborated by a recent systematic review, 11 stating that among 31 studies reporting liver vascular segmentation, only 2 and 3 reported vessel segmentation models exploiting ultrasound and MR images, respectively.
Systems used for vascular segmentation can be distinguished depending on whether the intervention of humans is requested during processing or not, and on the method adopted for processing. Semi-automated vascular segmentation needs human input to undertake and pursue vessel reconstruction, thus reintroducing the limits and bias of manual processing. Conversely, automated vascular segmentation fully relies on computer elaboration without human intervention. Vascular segmentation is usually performed following several steps, including: medical image pre-processing to enhance vascular structures; vessel segmentation using specific methods; and post-processing operation to improve segmentation results. Many methods have been adopted for the task of vessels segmentation, alone or in combination, such as: thresholding, region-based, model-based, tracking models and machine learning models (MLMs). In particular, the most common tools reported in the literature for liver vessel segmentation rely on model-based methods, such as active contour models (ACMs) and tracking models (TMs), as well as MLMs. While ACMs and TMs rely on matching a deformable model to the object of interest using a rule-based approach, MLMs are a subset of AI that are able to develop their own algorithm to solve a problem. Characteristics of the selected studies concerning the model and the imaging technique exploited for liver vessel segmentation are summarized in Table 1. A flow-chart depicting different phases of the segmentation process, from radiological images to the final liver vessels reconstruction, is shown in Figure 1.
Models and imaging techniques employed for liver vessel segmentation in selected published studies.
RBM, region-based model; ACM, active contour model; TM, tracking model; MLM, machine learning model; CT, computed tomography; MR, magnetic resonance; US, ultrasonography; FP, false positive; FN, false negative; TP, true positive; PPV, positive predictive value; DC, Dice similarity coefficient; Acc, accuracy; Se, sensitivity; Sp, specificity; VOE, volumetric overlap error. aLiver veins with or without hepatic arteries are segmented but not distinguished into arterial, portal and hepatic branches.
Flow-chart depicting image processing for liver vascular segmentation.
In the following section, a brief description of the principles of these segmentation models is provided, with examples from relevant publications.
Thresholding methods
In thresholding models, the pixels of a gray image (e.g., the image from a CT scan) are compared with specified thresholds so that any of the pixels is assigned to a different category, such as black or white, thus separating background and foreground objects.
In the setting of liver vascular segmentation, the complexity of the image prevents the possibility of performing segmentation through thresholding alone, but these models have been used in combination with other algorithms to complete the task. Soler et al. 20 exploited a thresholding algorithm to extrapolate the liver mask (the portion of the image including the liver) from CT scans to further apply filters that allow the anatomical segmentation of liver parenchyma, tumors, and portal and hepatic veins. In the research by Yang et al., 21 thresholding was used for both liver parenchyma and liver vein extractions after applying an algorithm capable of identifying seed points from which liver and vessel segmentations begin; this model is further discussed in the TMs section. Foruzan et al. 22 used thresholding to reduce the impact on image segmentation of a common hindrance, such as image noise.
Region-based and edge-based methods
Region-based models perform segmentation by regrouping pixels based on their intrinsic properties, such as intensity, density, and color. These models contrast with edge-based methods whose main property is to identify boundaries or edges within an image based on rapid intensity changes. Region-based models perform well when the processed images have distinct boundaries and well-defined properties.
Regarding thresholding, region-based and edge-based models are often combined together or with other models, particularly MLMs, in a unique function in order to perform liver and liver vessel segmentation. In the model proposed by Zeng et al, 23 the Authors specifically focus on liver veins and exploit an intensity gradient analysis to identify portal and hepatic vein centerline seeds, followed by vessel segmentation performed through the consecutive application of the fast marching method (a region-based method) and the graph cut method (a model-based method) to build the shape of liver vessels. Then, portal and hepatic veins are distinguished by the distance of a vessel from a portal or hepatic centerline voxel. Through this method, the Authors obtained 97.7% accuracy, and 79.8% and 98.6% sensitivity and specificity, respectively, for liver vein segmentation. Although the Authors tested their model on a clinical CT-scan dataset, very little information is given concerning the characteristics of the individuals in the imaging dataset and the images themselves.
Model-based methods
Model-based methods have been developed with the aim of overcoming some of the principal issues faced during the segmentation process due to low-quality images. Lack of contrast and the presence of artifacts complicate the task of identifying boundaries with edge-based and region-based models. In manual segmentation, despite using low-quality images, the expert can identify organ edges thanks to the a priori knowledge of its shape and appearance. Through model-based methods, the aim is to use an algorithm to convert information about the object of interest. 24
In model-based methods, the model is manually or automatically located at the object to be segmented in the image, then the shape and appearance of the model are optimized to overlap the object boundaries. Regarding previously mentioned methods, in liver segmentation, model-based methods are often used with other methods to fulfill the task, thus overcoming the limits of each single method.25,26
Among model-based methods, ACMs are the most commonly adopted in medical imaging segmentation when focusing on liver vessels.
Active contour models, also called ‘snakes’, were first introduced to outline objects from a 2D digital image. 27 The ACM algorithm creates a round line that, through an iterative process, shrinks or expands itself to progressively adapt its form to overlap the object edges. This happens using the gradient magnitude of the image, which is a characteristic of each pixel revealing how quickly the image is changing in a specific region, typically where the edges of an object can be found. To help the snake overlap the object’s edges, some constraints are added to the algorithm, forcing the contour line to be smooth and its points to be elastic and moving at the same pace through iterations. The contour can move toward the object edge following many principles, of which the most common in vascular segmentation is energy minimization. ACMs transform the image segmentation issue into an energy minimization problem by minimizing an integration of the internal energy of a curve and its external energy. The internal energy comes from the continuity and smoothness of a curve and the external energy is derived from the edge map of an object. Based on the same principle, other ACMs have been then developed that are able to extract objects by exploiting region properties or prior geometric knowledge rather than edges. Other methods have been tested on segmentation but with negative results in tracking objects made by branches through multiple slices, as in the case of vessels.
One of the first examples of the application of an ACM for liver vessel segmentation, called vascular ACM, was reported by Shang et al. 28 The Authors adopted a region competition-based ACM to perform wide liver arteries extraction, then introduced an additional term that drives the snake to overlap the vessel region from the central line, to draw out thin vessels having a weak contrast with the background. Finally, they used the minimal principal curvature, which is a measure of how the surface bends by different amounts in different directions from a specific point, to avoid branch rupture after multiple iterations, thus keeping the topology of vessels intact. No preprocessing information was provided. The model was applied to 20 3D contrast-enhanced CT scans and compared with previously elaborated ACMs; furthermore, an expert radiologist performed a manual segmentation on six randomly chosen cases to evaluate the reliability of this model. This automated vessel segmentation procedure permitted extraction of the main vascular branches with over 95% sensitivity, while for more peripheral branches, the sensitivity was 90% (6th to 8th order branches) and 81% (9th or higher order branches). Overall detection sensitivity was 91% with 3% false detection rate versus manual segmentation. To obtain accurate segmentations, 300 iterations were required, and mean time for 3D reconstruction was 8 minutes, which was longer than the alternative ACM used as comparison, but still reasonable.
Chung et al. 29 also described an automated vessel segmentation method based on ACM for liver portal and hepatic veins. In their research, they initially focused on a combination of preprocessing steps to enhance the vascular tree and reduce image noise on the liver region of 55 contrast-enhanced CT scans. The algorithm was trained on five CT scans and then tested on the remaining 50. Then, vessel segmentation was performed using an ACM exploiting both region and edges information. In addition, the newly proposed model was compared with vessel segmentation obtained by other existing ACMs, including the vascular ACM described by Shang et al., 28 after applying the same preprocessing methods. An expert radiologist manually identified all branches of the vascular tree of the 50 test datasets to set a term for model comparison. The proposed algorithm performed the best in localizing branches, as well as in false positive (0.015 ± 0.011) and negative error (0.072 ± 0.032) in identifying branches. Concerning computational performance, the mean time needed for the full segmentation process was 257 seconds, which was longer than the time necessary for the comparative models. With these two models, the Authors provided a tool capable of producing a satisfactory segmentation of peripheral vessels, the identification of which is crucial in clinical application, for example when the definition of the correct anatomical location of a lesion is needed.
Other imaging techniques, such as contrast-enhanced liver MR, may also be exploited to produce vessel segmentation with ACM, as reported by Lu et al. 30 In their method, preprocessing was performed through manual selection of vascular tree seeds, comprising the points of new branch departure, followed by clustering of K-means to classify seeds as relating to the same vessel tree. After that, vessel segmentation was performed with an ACM using the energy minimization principle. Here again, the region of interest extracted through AI was compared with manual expert radiologist segmentation evaluating false positive rate, true positive rate, and Dice similarity coefficient. The evaluated method performed the best, with a false positive rate of 76.23% and true positive rate of 78.43%.
Although ACMs are useful for vascular liver segmentation and may be applied in various imaging techniques, they present some limitations. First, good initialization of the snake is critical to precisely define the objects edges, particularly in the case of small objects or noisy images. Automated methods do not always precisely fulfill this task and semi-automated methods are more time consuming and rely on operator ability and experience. Initial contour definition is a fundamental point in this method, representing one of the most critical issues. In addition, small image features are often overlooked in the energy minimization technique, and when images are large, the method becomes slow. Also, to obtain high accuracy, convergence of regions of interest plays a key role; however, higher accuracy needs tighter convergence criteria, which exponentially increases computational time.
Tracking models (TMs)
Tracking models produce segmentation starting with a seed point that is chosen in a vessel of interest, and then adding subsequent points that are found based on image-derived data. Seeds can be identified manually or automatically through preprocessing. Once seeds are placed, two principal approaches are carried out to produce vessel segmentation: the model-based and the minimum cost path methods. The model-based approach exploits predefined 3D models, usually tubular, that are adapted on the specific 2D image to build vessel segmentation between seeds using gradient or intensity image features. In contrast, the minimum cost path approach provides vascular reconstruction by identifying the shortest path between two points of the same vessel based on image characteristics.
In the semi-automatic method described by Yang et al. 21 for portal and hepatic vein segmentation, automatic identification of the seed points is realized by computing average vessel voxel intensity and assigning the seeds to vessels when voxel intensity is within the established range. Starting from the identified seed points, with a region growing method, neighboring voxels are searched and added to the extracted vessels if the voxels are within the given threshold intervals. Then, depending on the voxel intensity threshold adopted, different extraction results are provided to the user to select the most appropriate one. Finally, the portal and hepatic vein branches that are connected are automatically identified and separated manually. The accuracy of vessel segmentation with this method was compared with the manual segmentation realized by an expert radiologist on 10 CT scans of the public dataset SLIVER07. 31 No false positive errors were found, while some false negative errors were identified in distal branches due to small vessel diameter and low contrast. In all 10 reconstructions, no aberrant portal and hepatic vein connection was found. Mean time for single segmentation was 2 ± 0.4 minutes. This proposed method stands out for its impressive accuracy; however, it demands an expert intervention that affects its serviceability.
Sangsefidi et al. 32 proposed an alternative TM based on the minimum cost method. After applying image filters to enhance liver vessels in the preprocessing phase, the statistical parameters of vessel and liver voxels are estimated to eliminate perivascular voxels not pertaining to liver vessels after thresholding. Based on these data, liver vascular skeletonization is performed. Starting from this binary vascular structure, segmentation is then realized using a graph-cut algorithm that is based on an energy minimization process. This model was initially tested on 50 contrast-enhanced CT scans comprising healthy and pathologic liver images coming from two different public datasets, and then on seven contrast-enhanced pathologic liver CT scans using manual segmentation performed by an experienced radiologist as a reference. The Dice index with the proposed model was 0.74 ± 0.12. Another similar algorithm based on TMs has been recently proposed by Kazami et al. 33 The portal and hepatic vein voxel sources in the image are identified through a method called template matching, which consists of finding small parts of an image that match a template image. Then, the center voxels of portal and hepatic veins are traced and added starting from the source voxels. Edge weights, which measure the cost to travel the distance between two points, are defined by this algorithm depending on the characteristics of a pair of voxels: the more similar the pair of voxels, the lower the edge weight. Following the minimum cost path method, the track made of vessel center voxels is reconstructed, minimizing the sum of edge weights. The algorithm was trained on 110, and tested on 46, contrast-enhanced CT scans from living liver donor candidates. Sensitivity, specificity, and Dice similarity coefficient were 0.58, 0.98, and 0.69, respectively. Missing branches and branch misclassification was evaluated using labels previously defined by liver surgeons. Misclassification was 0.8–14.6% for first to fourth-degree branches and missing branches for the same branches’ order span were 0–39.4% for the portal vein system. For the hepatic veins, the misclassification range was 0–6.5%, while missing branches were 0–27.3%. Compared with the model described by Yang et al., 21 those described by Sangsefidi et al. 32 and Kazami et al. 33 have the merit of proposing systems that substantially reduce human intervention, although being less accurate.
One of the most frequently reported limitations of TMs is the identification of incomplete blood vessels, which may be due to intensity inhomogeneity, noise, and also the presence of liver pathologies. Similar to ACMs, seeds from which the model generates segmentation through tracking can be inaccurate when performed in an automated manner and become time-consuming when performed manually.
Machine learning models (MLMs)
The central role of MLMs for liver parenchyma, liver cancer, and liver vessels segmentation has been described, along with their coexisting limitations and possible future directions. 34
Artificial intelligence and MLMs have brought new opportunities to enhance segmentation of the vascular system.35,36 In supervised MLMs, imaging features to produce vascular segmentation are extracted from a training image dataset containing the characteristics of interest (e.g., intensity and gradient) with their corresponding labels. Once the model is trained, it can be tested on a test image dataset containing the same already categorized features to assess model accuracy. Conversely, unsupervised MLMs exploit the statistical distribution of some features of the input data to build a model able to give a label without knowing the ground-truth category. Then, it is tested on an a priori labels-known dataset to evaluate accuracy. 11
In MLMs applied to medical imaging analysis and segmentation, training is usually performed through an artificial neural network (ANN). These models represent a subset of MLMs and the main core of deep learning. ANNs comprise node layers: an input layer, one or more hidden layers, and an output layer. Input nodes receive raw data, while output nodes provide the expected output. Every node is a regression model with an input, a weight, a threshold, and an output. Once an input layer is determined, weights are assigned. These weights help determine the importance of any given variable, with larger ones contributing more significantly to the output compared with other inputs. All inputs are then multiplied by their respective weights and then summed. Subsequently, the output is passed through an activation function, which determines the output. This results in the output of one node becoming the input of the next node. This process of passing data from one layer to the next layer in one single direction defines this neural network as a feedforward network. 37 When at least one layer of the network uses a mathematical operation called convolution, the ANN is named a convolutional neural network (CNN), which is specifically designed to process pixel data and is used in image recognition and processing. When data can also flow backward and not only in one direction, the neural network is called recurrent. 38 The term deep learning, when referring to neural networks, include those networks composed of three or more layers. 37 The definitions of the most common terms concerning AI are summarized in Table 2.
Definition of terms related to artificial intelligence.
One of the first applications of deep learning in liver vessel segmentation was reported by Zeng et al. 39 After preprocessing, including the application of multiple filters to enhance the vessels of six portal-phase CT scans, the unsupervised learning machine algorithm was applied to train the ANN to obtain parameters and weights to recognize the portal vein structure. The number of layers is selected balancing training time and segmentation sensitivity. The Authors compared this model with different ACMs and the support vector machine algorithm, a supervised machine learning algorithm. The proposed algorithm showed 98.1% accuracy and 74.2% sensitivity, which was better than the other evaluated algorithms. Specificity (99.3%) was also higher than the other methods, except for the geodesic ACM. In this attempt, although it represents one of the first experiences of using ANN and the model was shaped to reduce training time as much as possible, the superiority of AI in vessel segmentation was shown versus previous models, without the need of human intervention in the segmentation process.
Other investigators have implemented and refined the use of ANN for liver vessel segmentation, introducing new features to the network structure. For example, Ibragimov et al. 40 described the first example of portal vein segmentation using a MLM based on CNN. The model was trained on 72 3D contrast-enhanced CT scans of patients scheduled for liver stereotactic body radiotherapy. The CNN segmentation was refined by associating the Markov Random Fields, which is a graphical model of joint probability distribution, permitting the reduction of artifacts and isolated segmentation regions. The Authors compared portal vein segmentation obtained with their algorithm, adopted in both an automated and semi-automated fashion, with manual segmentation produced by an experienced radiologist. When the algorithm was applied after manual region of interest identification, the Dice similarity coefficient was 0.83, while for automated segmentation it was decreased at 0.7.
In 2015, the University of Fribourg created a specific ANN for biomedical image segmentations called U-Net, 41 comprising a fully CNN with the aim of producing precise segmentations with a limited training dataset. Huang et al. 42 used the U-Net algorithm on the 3Dircadb and SLIVER07 public datasets, exploiting 3D CT scans for portal vein segmentation. The 3D U-Net algorithm was implemented with a Dice loss function that benefits from Dice similarity coefficient calculation to reduce voxel misclassification. The Dice similarity coefficient and sensitivity with this method were 0.675 and 74.3%, respectively, which raised to 0.753 and 76.7% upon postprocessing refinement by an expert surgeon. To overcome the problem of the possible absence of 3D images, Kitrungrotsakul et al. 43 proposed a multipath CNN algorithm consisting of three separate CNNs that extract features of the sagittal, coronal, and transverse planes of a contrast-enhanced CT scan. As the layers of the different networks are fully connected, the network can gather information from another axis, keeping the size of the network smaller than those exploiting 3D images. Training and testing were performed on two publicly available datasets, 3Dircadb and VASCUSYNTH, 44 using the leave-one-out method. The proposed multipath 2D CNN model was compared with a single-path 2D CNN model, a 3D CNN model and other ACMs and TMs previously described in the literature. With a Dice similarity coefficient of 0.903, the proposed model was shown to be the most accurate. Based on this research, adding convolution to at least one of the layers of the neural network improves segmentation accuracy using both 2D and 3D images, the latter not always being available in routine clinical practice.
Another example of full CNN application in liver vessel segmentation is provided by Mishra et al., 45 using ultrasound images. The network architecture consists of a series of convolutional layers whose outputs are passed through a leaky rectified linear unit, which is an activation function that, in the case of increased noise and outliers in data, helps avoid discarding potentially important information. Auxiliary layers enrich the architecture and focus on boundary definition and object (vessel) region discrimination from the background (liver parenchyma). A fusion layer at the end of the network combines auxiliary layer outputs in order to select the best features for generating the segmentation and use the remaining features for refinements. Due to the process duration, which is stated to be about 25 hours, training was performed on 10 images followed by cross-validation on the complete dataset of 350 liver ultrasound images. This model was then compared with the Frangi filter and the U-Net algorithm,41,46 and showed a slightly increased Dice similarity coefficient (0.91 versus 0.88 and 0.87, respectively). Although ultrasound imaging is always affected by the user-dependent nature of the exam, the possibility of performing accurate liver vessel segmentation through ultrasound images is an important innovation, as it is non-radiant and easily available. The cross-validation performed by the Authors on a large image dataset improves the reliability of the results.
As an alternative to CNN models, Nazir et al. 47 described the first experience of applying a cascade incremental learning model for portal vein and hepatic artery segmentation. Cascading consists of a particular subtype of ensemble learning, which is a subtype of machine learning using multiple learning algorithms to obtain better predictive performance than using one type of algorithm alone. In cascading, the input is used to assign an initial classifier; all the information contained in the output of a given classifier is then used as additional information for the next classifier in the cascade. The algorithm was trained and tested on 1000 contrast-enhanced CT scans from a clinical dataset, and on 30 contrast-enhanced CT scans of the public dataset SLIVER07. The accuracy of the automated vascular segmentation compared with manual segmentation was 98.9% and it took 20 seconds to perform on a single CT scan. Mixing the advantages of different learning algorithms allows the building of an MLM with accuracy that is comparable to human segmentation, but with minimal computational time and no human intervention.
Finally, a completely different procedure based on fuzzy connectedness was employed by Zhang et al. 48 for liver vessel segmentation. The principle of fuzzy connectedness is to manage the problem of image heterogeneity, artifacts, and noise defining how the image elements hang together spatially in spite of focusing on their gradation of intensities. In defining objects in a given image, the strength of connectedness between every pair of image elements is considered, which in turn is determined by considering all possible connecting paths between the pair. Again, training and testing was performed on contrast-enhanced CT scans from publicly available datasets, showing 96.4% and 96.8% accuracy and 0.673 and 0.714 Dice similarity coefficient for 3Dircadb and SLIVER07 datasets, respectively.
CNN models that have been specifically conceived to analyze images have progressively gained a primary role for vessel segmentation, with different variants of these models described in the published literature. Kazami et al, 33 whose TM for liver vessel segmentation has been described above, also developed a deep learning model based on a 3D CNN to extract vessels from the image, detect the center voxel, and perform vascular tree reconstruction. Compared with the TM, the CNN model had a significantly increased sensitivity (84% versus 58%) and Dice similarity coefficient (0.9 versus 0.68), while no difference was found in terms of specificity (97% versus 98%) for vessel recognition. The deep learning algorithm also performed better in portal and hepatic vein misclassification and missing rate. Consequently, the reconstruction coming from the CNN model also demanded an inferior manual correction time in order to produce the final segmentation. Yan et al. 49 proposed another CNN-based model applying an attention-guided mechanism to improve vessel extraction from CT-scans. This mechanism uses a combination of different image features to identify the vessel structure before extraction. The Authors reported a 0.904 Dice similarity coefficient when they applied their model to the 3Dircadb dataset, which was higher than that reported with other CNN-based models.42,43 CNNs can also exploit MR images without contrast to produce liver vessel segmentation. 50 However, reported Dice similarity coefficients for portal and hepatic vein segmentations were 0.634 and 0.532, respectively, 50 which is inferior to that obtained with the other described deep learning methods. This was probably due to the fact that the Authors based their segmentation on single-phase images without contrast-enhancement. A similar attempt was reported by Oh et al., 51 whose CNN model used hepatobiliary phase liver MR images to perform liver, mass, portal vein, hepatic vein, and bile duct segmentation. Dice similarity coefficients for reconstruction of portal and hepatic veins were slightly increased versus the previous model (0.61 and 0.70, respectively). Although the Authors assert that the segmentation generated with this algorithm has an important role for preoperative planning, no evaluation of the model under this topic is reported in the study. MLMs are promising techniques for semi- and fully automated vessel segmentation and their use in clinical practice for liver segmentation has been reported in different settings and exploiting different medical image types.52,53 The high malleability and possibility of incremental learning of ANNs make them very attractive for this purpose. As reported above, the accuracy of vessel segmentation performed with MLMs, when explicitly compared, is higher than ACMs and TMs. However, some pitfalls concerning the effectiveness of MLMs remain. The existing machine-learning techniques are primarily suitable for datasets having less variety in input images. When the dataset has low variability, MLMs are suitable for rapid training. When more features need to be analyzed by the model with variable settings, the principal solution is to increase the number of filters and layers and newly train the model, which can be a highly time-consuming and computationally expansive process. Furthermore, another main challenge of ANNs is represented by the so-called ‘black box’. Compared with other MLMs, such as decision trees, it can be very difficult to understand the cause of misinterpretation when an erroneous output is obtained from the given input.
Evaluation metrics
There is wide heterogeneity concerning the evaluation measures used to assess the above proposed models. 54 The most commonly used measures are briefly described here (TP, true positive; FP, false positive; TN, true negative; and FN, false negative):
Accuracy, calculated as TP + TN/TP +TN + FP + FN, describes how close a set of prediction measurements are to the true value.
Sensitivity, calculated as TP/TP + FN, describes the positive detection capabilities for pixel classification.
Specificity, calculated as TN/TN + FP, evaluates the capabilities for correctly identifying true negative pixel classes.
Dice similarity coefficient (also known as Dice score, Dice index or F1 score), calculated as 2TP/2TP + FP + FN, is a statistical tool to gauge the similarity of two samples.
Clinical applications
The potential clinical applications of liver vessel segmentation in the treatment of liver diseases are vast, especially, but not only, for liver cancer. As mentioned above, the vascular anatomy of the liver has to be extensively examined before any surgical or ablative procedure to identify the precise localization of the lesions and their relation to the vessels, in order to perform a radical treatment without sacrificing an excessive amount of non-pathologic liver tissue. When dealing with primary or metastatic liver cancer, different treatments are recommended depending on the histological type of malignancy, either considering the type of treatment or its technical aspects. Knowing the precise tumor location and its relationship with adjacent structures, mainly liver vessels, facilitates both the surgical procedure and the whole disease management in a specific patient. A precise segmentation informs the surgeon what type of resection will be safest and associated with the best oncological results. Moreover, if there is a chance that residual tumor may remain on vascular structures that cannot be resected, a complementary radiological or systemic treatment could be suggested to improve the patient’s recurrence-free survival. In addition, a 3D reconstruction that shows more advanced tumor disease than was suspected on standard 2D imaging may allow the surgeon to address a more appropriate treatment with the patient. Furthermore, the same observations can be made concerning all radiological treatments (e.g., transarterial chemoembolization, radioembolization, or microwave/radiofrequency ablation, etc.), for which anatomy definition is crucial to estimate the risk and benefit of the procedure, as well as treatment efficacy.
Models conceived for the clinical application of liver and liver-vessel segmentation can be regrouped into those developed for diagnosis and treatment planning and those elaborated for treatment monitoring.
Diagnosis, treatment planning and monitoring, prognosis prediction and follow-up
Although one of the main objectives of vascular segmentation is to help in decision-making and prepare interventional procedures for liver disease, most published studies on the topic have reported the model characteristics and measures of accuracy, but most did not systematically report or evaluate practical cases of how they could be advantageously exploited.
Few examples of the clinical application of liver vascular segmentation models have been published. For example, Nazir et al. 47 developed their cascade incremental learning model to compare the portal vein and hepatic artery anatomy of living liver donors with liver transplant candidates. The Authors proposed an algorithm to automatically match two patients based on possible vascular variants identified through automated segmentation from the perspective of liver transplantation, in order to reduce donor postoperative liver failure and to maximize recipient benefits, avoiding suboptimal pairing. Concerning liver cancer treatment, Kock et al. 55 developed a deep learning ANN to produce portal vein, hepatic vein, arterial, liver tissue and tumor segmentation. The algorithm was developed on contrast-enhanced CT scans taken from internal clinical datasets, then 2D images and 3D reconstructions were examined by expert radiologists to assess if the automated segmentation tool produced a representation of liver anatomy that would aid transarterial radioembolization. In their experience, 61.11% of 3D reconstructions performed with ANN could be used for procedure planning versus 75% of manual segmentations. Ibragimov et al. 40 intended the above-described CNN algorithm to be used for stereotactic body radiotherapy of liver malignancies. Although the model was trained and tested on contrast-enhanced CT scans of patients scheduled for this procedure, no evaluation of the results of segmentation was performed in this perspective.
Nevertheless, many experiences of 3D reconstruction use have been reported, including liver vessel segmentation for surgical planning, residents training, and patient education.7,56–59 A recent systematic review and meta-analysis on the advantages of 3D reconstruction for preoperative planning of laparoscopic liver resections showed a reduction in operative time and estimated blood loss for those patients whose preoperative planning was completed with 3D liver reconstruction, but no difference in terms of hospital length-of-stay and post-operative complications. 60 Despite these results, it is difficult to conclude the advantages of 3D reconstructions, as most of the included studies were retrospective with small cohorts, and heterogeneous, including patients with different clinical characteristics, liver diseases and interventions. 60 Most published studies reported the experience and satisfaction of the user on 3D liver segmentation applied to various invasive procedures, but objective data on the clinical benefits and patient outcomes were scarcely reported.
Other investigators have reported the potential interest in combining liver vascular segmentation with other anatomic characterization to enhance preoperative disease evaluation and surgical planning. Ye et al. 61 evaluated the advantages of preoperative 3D liver reconstructions and indocyanine green excretion test to precisely assess liver vascular and biliary anatomy to perform liver surgery. They reported that patients who benefit from these two technologies had reduced operative time and positive tumoral margin rate. Bijlstra et al. 62 proposed a liver segmentation model integrated into the cockpit of a robotic surgical device, allowing the surgeon to examine 3D liver reconstructions while performing robotic resections with the aid of intraoperative ultrasonography and indocyanine green technology. Again, the efficacy of this technology was evaluated through a satisfaction questionnaire completed by the liver surgeon, but no other assessment of the clinical benefits to patients was provided.
The most practical application of segmentation, including vascular but also parenchymal reconstruction, to real clinical situations, are reported by Soler et al. 63 and Quero et al. 64 These Authors describe virtual reality tools that are capable of superposing a 3D reconstruction of organ inner structures to the patient. Reconstructions are made with AI-based algorithms, whose characteristics are not detailed in the manuscripts, from patients’ own imaging exams. This results in the possibility for a surgeon to identify vascular structures, lesions, and other anatomical features while observing the organ’s surface when operating. These tools remain under development to overcome some limitations, including reconstruction inaccuracy, imprecision in virtual image superposition to the patient in the clinical situation, and tool usability, but have already been shown to improve surgeons’ technical decisions in real-life settings.
The clinical application of liver segmentation has also been explored in many other fields, from diagnosis to treatment monitoring, follow-up, and outcome prediction. Researchers have produced tools, mainly based on ANN and deep learning, that are capable of performing automatic segmentation of the liver and its tumors, 65 and of predicting with high accuracy their histologic nature,66,67 thus allowing a precise anatomic definition and histological characterization without invasive procedures. MLMs have also been adopted for different aspects of patient management after treatment. Feeding ANN models with imaging data from patients who received treatment for liver cancer permits the production of tools that are capable of evaluating treatment efficacy, 68 helping to identify disease-recurrence during follow-up,69,70 or predicting its probabilities based on clinical, biochemical, radiological, or pathological data.71,72 In the present narrative review, such models were not discussed, as vascular segmentation is not performed for these aims.
Education
Several studies have suggested that developing methods to enhance liver segmentation might also be helpful for both surgical trainees and patients to better understand the surgical procedures. Cheng et al. 59 gave the following to three groups of residents: the 3D virtual reconstruction, the 3D printing model, or the 2D contrast-enhanced CT scan of three patients who underwent laparoscopic hepatectomy for hepatocellular carcinoma. Each resident was then asked to complete a questionnaire on tumor location, identification of the relationship between the tumor and liver vessels, and surgical planning. The groups who exploited the 3D virtual reconstruction and the 3D printing model obtained better scores than those who had 2D images. A study by Giehl-Brown et al. 7 focused on the results of providing 3D reconstructions to patients. The general comprehension and satisfaction regarding the presurgical physical examination of 20 patients undergoing liver resection, for whom an individual specific 3D model of the liver was produced, were compared with 20 patients scheduled for liver surgery for whom no liver model was constructed. Patients who received a 3D liver model were found to be globally more satisfied and to have a better understanding of the surgical procedures.
Limits and perspectives
New and updated vessel segmentation models are being trained, tested, and evaluated continuously due to their noticeable potential applications in the clinical and pedagogical settings. However, obtaining a highly accurate, fast, and fully automated algorithm capable of performing liver vascular segmentation remains challenging with numerous interfering issues.
The published methods were trained and tested on different private image datasets that can hardly be clearly evaluated and compared; a method that is highly effective on one dataset may be inaccurate when applied to other images with slightly different characteristics. The proposed published models were frequently trained and tested on the same dataset, without external validation. The method for manual vascular segmentation, which is commonly used to define the ground truth and evaluate the efficacy of AI-methods, was not systematically well-described or reported. In addition, liver vascular segmentation methods were mostly trained and tested on non-pathologic liver images, while the applications of AI-based reconstructions would mainly target and intend to be used in images from patients with chronic liver disease and/or cancer.
The lack of easily accessible publicly available data, along with the difficulty in terms of resources and time costs to produce new imaging datasets containing masks for all the different anatomical structures, represents an important limitation for the development of liver vessels segmentation models that follow a robust validation process. These datasets, each with its peculiar characteristics, are both used for developing and testing, or for validating the proposed segmentation models. Although many public datasets contain liver images, 34 few have the information required for liver vessel segmentation.
The MIDAS-LT was the first published publicly available dataset containing liver images with vessel information. However, only four CT-scans containing livers and liver tumors are available. The same limitation concerns the 3D-IRCADb database, composed of 22 CT scans of livers and liver tumors, only a minority of which also contain vessel information. The main public source for liver images that also contain vessel labels is the MSDC-T8 dataset, which is composed of 303 training CT scans and 140 test CT scans, and is by far the widest imaging database containing liver, tumor, and vessels mask images.
Other well-known imaging datasets, such as SLIVER07 (20 training and 10 test CT scans), LiTS (130 training and 70 test CT scans), CHAOS (40 CT scan and 120 MR images), and TGCA-LIHC (237 CT, MR, and positron emission tomography scans) are open source and available for model development.
An additional limitation of the different reported models is the heterogeneity of research methodology, with various metrics used to test the performances and evaluate the accuracy of the algorithms. The Dice similarity coefficient is frequently but not always reported in this research, while other measures, such as accuracy, sensitivity, specificity, true and false positive rate, and misclassification rate are inconsistently reported, making direct comparison between different studies hard to perform.42,47 Several pitfalls common to research on AI and image analysis have been described regarding the inadequate use of metrics for model validation. For example, inappropriate use of metrics that are not suitable for the problem category being analyzed, the choice of metrics with specific properties that do not completely meet the requirements of the dataset and/or of the segmentation method, or the poor application of the metric to an image or a dataset. 73 The issue of model validation is made even more complex considering that the techniques are usually compared with manual segmentations performed by radiologists or surgeons whose experience is rarely directly and precisely detailed and assessed. 33 Moreover, further accuracy estimation is made comparing the new proposed model’s performance with one or more previously described segmentation models. However, models used for comparisons are not usually the same, and the reason why one is preferred to another is often unclear.29,42,43
The constant development of imaging techniques makes new clinical images more detailed, with a higher resolution and less noise. However, in daily clinical practice, CT scans and MR imaging are performed with different techniques and contrast protocols, increasing the challenges in producing a comprehensive model. Moreover, some patient-associated factors, such as movement or prosthetic artifacts, can affect the segmentation accuracy. To overcome these constraints and obtain accurate reconstructions, a huge amount of data is necessary, thus increasing the computational cost for segmentation. New expedients to improve segmentation performance are often reported in the literature, from model structure modifications to the use of multi-phasic input with the aim of exploiting most of the information contained in a single exam. 74 On the other hand, technical development is providing researchers with more computational power to apply their models. Deep learning algorithms, in particular, are becoming strongly popular as a consequence of these phenomena. In parallel, many new publicly available image datasets are being developed to allow researchers to test their algorithms with objective methods. Research on automated or semi-automated vessel segmentation has been difficult for health professionals to access due to its technicity, while on the other hand, it is necessary to better evaluate and show evidence on how these tools could be easily applied to improve diagnosis, treatment, and follow-up of patients. To date, the literature on AI-driven liver vessel segmentation and its clinical application have been mainly addressed separately, where only the technical aspects of the algorithm generated, or the clinical use of the algorithms, were reported. It would be of real interest to investigate both aspects simultaneously in future research, as it would provide robust and transparent technical information and help health professionals to better assess and foresee future applications for clinical practice.
Conclusion
Artificial intelligence-enhanced liver vessel segmentation offers great potential for the management of patients with liver disease, by improving the anatomical characterization to help decision making and planning therapeutic interventions. Invasive procedures on the liver can benefit from the accurate definition of the relationship between vessels and lesions in order to obtain the best results, reducing intraoperative and postoperative complications. New models are being developed to obtain highly accurate reconstructions, mainly based on ANN. However, many limitations regarding how segmentation models are trained, tested, and validated remain to be overcome, and their potential benefits for clinical practice need to be evaluated. Collecting imaging data within an international, multicentric, open-source dataset may facilitate the task of creating a comprehensive segmentation model. Transdisciplinary teams composed of engineers, radiologists, hepatologists, oncologists, and surgeons would allow for defining, building and testing highly performing tools that could improve the care provided to patients with liver disease.
Footnotes
Author contributions
AC, writing – original draft, investigation; FL, investigation, writing – review and editing; BS, validation, visualization; AI, validation, visualization; HD, supervision; JR, project administration, conceptualization, writing – review and editing.
Declaration of conflicting interest
The Authors declare that there is no conflict of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.