Sage Journals: Discover world-class research

Abstract

Exponential development in artificial intelligence or deep learning technology has resulted in more trials to systematically determine the pathological diagnoses using whole slide images (WSIs) in clinical and nonclinical studies. In this study, we applied Mask Regions with Convolution Neural Network (Mask R-CNN), a deep learning model that uses instance segmentation, to detect hepatic fibrosis induced by N-nitrosodimethylamine (NDMA) in Sprague-Dawley rats. From 51 WSIs, we collected 2011 cropped images with hepatic fibrosis annotations. Training and detection of hepatic fibrosis via artificial intelligence methods was performed using Tensorflow 2.1.0, powered by an NVIDIA 2080 Ti GPU. From the test process using tile images, 95% of model accuracy was verified. In addition, we validated the model to determine whether the predictions by the trained model can reflect the scoring system by the pathologists at the WSI level. The validation was conducted by comparing the model predictions in 18 WSIs at 20× and 10× magnifications with ground truth annotations and board-certified pathologists. Predictions at 20× showed a high correlation with ground truth (R ² = 0.9660) and a good correlation with the average fibrosis rank by pathologists (R ² = 0.8887). Therefore, the Mask R-CNN algorithm is a useful tool for detecting and quantifying pathological findings in nonclinical studies.

Keywords

deep learning Mask R-CNN liver fibrosis NASH steatohepatitis NDMA cirrhosis

Introduction

With the development of medical imaging techniques over the past few decades, research has been actively conducted on diagnosis and prediction in the clinical field, using the data derived from imaging.¹ Artificial intelligence (AI) methods, including traditional machine learning and deep learning, have offered opportunities to apply medical imaging data, such as radiological and histopathological data, in clinical prediction to reduce the human effort in diagnostics.² Since the advent of deep learning, which uses convolutional neural networks (CNNs), computer vision techniques have enabled breakthrough developments in accuracy, which could not be accomplished by traditional machine learning.^3,4

There are 2 main tasks in image analysis where deep learning can be used. One is image classification, which treats each image as an identical category. The other is object detection, which refers to object localization and recognition. In addition to object detection, segmentation classifies the categories of images using pixel-level prediction.⁵ Image segmentation has been widely used in the medical field, even before the advent of deep learning, using machine learning methods to detect and track medical instruments in surgical operations,⁶ to analyze brains and their tumors from magnetic resonance imaging (MRI),⁷ and to visualize colon crypts.⁸ After the introduction of deep learning, the implementation of a fully connected network⁹ and deconvolution¹⁰ improved accuracy in more challenging segmentation techniques, such as instance segmentation. Instance segmentation can classify each pixel into its category and perform instance-wise segmentation simultaneously; therefore, it is trained to detect each object in an image. This progress has been applied successfully to pixel-level detection of pathological findings observed in highly magnified whole slide images (WSIs), such as those of mitosis, which can be an important indicator of tumor progression and/or malignancy¹¹; moreover, it has been used to discriminate nuclei between normal and pathological lesion.¹² Mask Regions with Convolution Neural Network (Mask R-CNN) is an instance segmentation algorithm. This algorithm performs the combined task of object detection, where the goal is to classify individual objects and localize each object using a bounding box (bbox), and semantic segmentation, where the goal is to classify each pixel into a fixed set of categories without differentiating object instances.¹³ The Mask R-CNN algorithm has been recently applied to medical image data to detect lung nodules (using computed tomography images),¹⁴ glomerulosclerosis (using images obtained from MRI),¹⁵ breast cancer (using images obtained from ultrasound data),¹⁶ and nuclei within cancer biopsies.¹⁷

An application of AI for pathological analysis in nonclinical fields of medicine has recently been introduced. Previous studies applied the image classification method to analyze whether deep learning can achieve the pathologist’s grading accuracy in liver fibrosis using the rodent nonalcoholic steatohepatitis model¹⁸ via a classification algorithm; moreover, these studies attempted to quantify hepatic fibrosis using segmentation in picrosirius red-stained slides.¹⁹ Hepatic fibrosis occurs because of an abnormal and repeated tissue repair response generated as a result of multifactorial chronic liver injury, and its pathogenesis is associated with elevated levels of reactive oxidative stress caused by the metabolism and detoxification of drugs in the liver.²⁰ N-Nitrosodimethylamine (NDMA), a well-known carcinogen, has been administered to rats to induce hepatic fibrosis; this has been a good and reproducible animal model to investigate the early events involved in the pathogenesis of human liver fibrosis. This model has also been used to screen antifibrotic agents that could reverse fibrosis and arrest the progression of liver fibrosis to cirrhosis.²¹ Although two segmentation algorithms (U-net²² and Mask R-CNN²³) have been applied to biological samples, neither of the techniques has been applied to hematoxylin and eosin (H&E)-stained slides to detect rodent hepatic fibrosis. In a recent study on the detection of skin lesions and immune cells in immunofluorescence images, Mask R-CNN exhibited more precise segmentation than U-net.^24,25 Therefore, in the present study, we conducted instance segmentation of hepatic fibrosis via Mask R-CNN using the NDMA-induced hepatic fibrosis animal model to test whether instance segmentation by deep learning can be implemented for the detection of lesions for preclinical diagnosis.

Materials and Methods

Animal Treatment

Five-week-old male Sprague-Dawley (SD) rats (Crl: CD) were obtained from Orient Bio, Inc (Republic of Korea) and acclimatized for 5 days prior to study initiation. During the studies, the animals were kept in a room with controlled conditions (temperature 23 ± 3 °C and relative humidity at 30% to 70% on a 12-hour light/12-hour dark cycle with 150-300 lux, ventilation 10-20 cycles/h). A standard rat and mouse pellet diet (Lab Diet #5002 Lab Diet #5053 PMI Nutrition International; irradiated by γ-ray) was provided to the animals ad libitum (except where investigation requires otherwise), and microbial monitoring for diet was performed. The animals had ad libitum access to filtered, ultraviolet light-irradiated municipal tap water. The drinking water was analyzed every 6 months for specified contaminants in the Daejeon Regional Institute of Health and Environment. This experiment was approved by the Institutional Animal Care and Use Committee and conducted in an Assessment and Accreditation of Laboratory Animal Care accredited facility. Test animals were chosen randomly for the control and hepatic fibrosis groups and kept for 7 days before administering the drug.

NDMA was purchased from Wako (147-03781, 99.5% purity). NDMA (10 mg) was dissolved in 10 mL (13.5 mM) of sterile distilled water just before administration. To induce hepatic fibrosis, 10 mg/kg of NDMA was administrated to 6- to 7-week-old SD rats via intraperitoneal injection, 3 times a week for 3 weeks (total of 9 doses). After chemical administration, the test animals were killed using isoflurane, and the livers were collected in 10% neutral buffered formalin. After tissue collection, H&E staining was performed using paraffin-embedded left lateral and median lobes of the liver, and the sections were used for digital archiving.

Data Preparation

Whole slide images of the induced hepatic fibrosis liver sections were scanned using an Aperio Scanscope XT with a 20× objective using bright field illumination. The scan resolution was 0.5 µm per pixel and was saved as TIFF stripes with JPEG2000 image compression. The data preparation for hepatic fibrosis instance segmentation is described in Supplementary Figure 1. The WSIs at 10× magnification were cropped into 448 × 448 pixels of tile images, and all the cropped images were assessed by a pathologist to confirm that the images had apparent hepatic fibrosis. All the labeling was performed using the VGG Image annotator 2.0.1.0 (Visual Geometry Group, Oxford University, United Kingdom), and the annotation information was saved as a json file. Hepatic fibrosis was labeled along the borderline of the lesions, and the annotation lines were drawn as close to the lesion as possible to ensure distinction from normal cells. Blood vessels located inside the lesions were largely excluded for more accurate detection of the lesion. A total of 2,011 image tiles were obtained from 51 WSIs, and the lesions on these images were labeled for the training and testing of Mask R-CNN. The annotated image tiles were divided in the ratio of 7:2:1 for training, validation, and test data sets using the train_test split function embedded in the scikit-learn package. To increase the quality of the data set, the training data set was confirmed once again before the training, and data augmentation was conducted for improving training, as proven in the previous studies.^26
–28 The training data set was augmented 8 times using a combination of image augmentation techniques (reverse, rotation, and brightness). A total of 10,848 images were used for training, and 437 and 218 images were used for validation and testing, respectively.

Structure of Mask R-CNN Algorithm

All the procedures related to algorithm training, including data distribution, were performed using an open-source framework for machine learning (Tensorflow 2.1.0,²⁹ using Keras 2.4.3 backend) powered by an NVIDIA RTX 2080ti 11G GPU. Matterport Mask R-CNN 2.1 package was used for training, and its requirements were met in this study.^13,30 It is based on the feature pyramid network and ResNet101 backbone and generates bboxs and segmentation masks for each object in the image. Mask R-CNN was developed based on the object detection algorithm “Faster R-CNN.” Faster R-CNN, which inspired Mask R-CNN, consists of 2 stages. The first stage, called the region proposal network (RPN), proposes candidate object bboxs. The second stage extracts features using region of interest pooling (RoIPool) from each candidate box and performs classification and bbox regression. In Mask R-CNN, RoIPool is replaced with RoIAlign to predict pixel-accurate masks. The RoIPool first quantizes a floating number region of interest (RoI) to the discrete granularity of the feature map; however, this quantization may introduce misalignments between the RoI and the extract features, negatively affecting prediction of pixel-accurate masks.¹³ RoIAlign uses bilinear interpolation³¹ to compute the exact values of the input features at 4 regularly sampled locations in each RoI bin and aggregates the result using max pooling. Unlike Faster R-CNN, the Mask R-CNN architecture has a 2-stage procedure that includes RPN and RoIAlign while simultaneously predicting the class and box offset, and a binary mask prediction network is added for the binary mask for each RoI (Figure 1).

Figure 1.

The network architecture of Mask R-CNN and segmentation example of hepatic fibrosis.

Training of Hepatic Fibrosis and Metrics for Model Performance

Hyperparameters

A total of 11,285 images were used to train the model and to validate rat hepatic fibrosis. The hyperparameters used during the training are described in Table 1. All the configurations, except the 5 parameters, were set as default defined by the Matterport package. The 5 parameters that were not set as default were customized to fit the hepatic fibrosis data set. Four images were simultaneously analyzed using IMAGE_PER_GPU, and 2 GPUs were used during the training. The image size was determined as 448 × 448 by IMAGE_MAX_DIM and IMAGE_MIN_DIM according to the tile image size. The threshold of instance classification accuracy, DETECTION_MIN_CONFIDENCE, was adjusted to 0.5.

Table 1.

Hyperparameters used for training Mask R-CNN algorithm.

Hyper parameters	Values
IMAGE_PER_GPU	4
GPU_COUNT	2
STEPS_PER_EPOCH	1000
IMAGE_MAX_DIM	448
IMAGE_MIN_DIM	448
DETECTION_MIN_CONFIDENCE	0.5
LEARNING_RATE	0.001
LEARNING_MOMENTUM	0.9
WEIGHT_DECAY	0.0001

Loss

The calculation of training and validation losses was performed as follows. Class (label), mask, and bbox losses observed during the training and validation were serialized using the tf.summary module and visualized by a tensorBoard. Class loss was calculated as the multiclass cross-entropy loss. The mask network only predicts whether each pixel belongs to the class by the sigmoid; therefore, the loss for the mask was determined by binary cross-entropy. In the case of bbox, a smooth L1 loss, which calculates the error between the prediction and ground truth, was used. Finally, Mask R-CNN loss (total loss) was calculated as the sum of losses including sparse softmax cross-entropy for label, smooth L1 loss for bbox, and binary cross-entropy loss for mask.

Metrics for model performance

To verify the model performance, mean average precision (mAP) was used. The mAP can be derived from the intersection of the union (IoU), precision, and recall values. The IoU value calculates the ratio of the overlaid area of the union of the predictions and the ground truth. In general, the true positive (TP) of each object detection is defined by an IoU value greater than 0.5. The precision or recall is determined as follows: the former is calculated by the ratio of TPs to all ground truths, and the latter is calculated by the ratio of TPs to total detections. Finally, the average precision (AP) was estimated by calculating the area below the precision-recall curves (Supplementary Figure 2) of each object in an image, and the mean AP was determined by the mean value of the sum of all the APs. Therefore, a higher mAP indicated a more accurate model. In this study, we used transformed mAP, which assumes an mAP value of 0 if there is any misprediction found in an image. From this transformation, we could analyze the error cases in more detail to investigate the cause of the correct and incorrect predictions as well as to evaluate the model performance more strictly.

Testing the Model Performance in WSIs

To test the algorithm performance in real-world data, we conducted a prediction test of the algorithm at the WSI level. Eighteen WSIs that were not used during the training were prepared for the test, consisting of 6 WSIs from the control group and 12 WSIs from the test group. The WSIs were scanned with a 20× objective using bright field illumination. Before the test, the fibrosis lesion including the connective tissue of each WSI was annotated by a board-certified pathologist as the ground truth to be compared with the prediction of the algorithm. After annotation, the annotated region area was calculated and transformed into a percentage of the liver section area. The WSI annotation and calculation of the annotated area were conducted using Aperio Image Scope version 12.4.0 (Leica Biosystems).

The prediction was performed using two magnification-scale images (10× and 20×). Each magnification image of WSI was divided into 448 × 448 pixels of tile images, and fibrosis was inferred using the trained algorithm. After the prediction ended, the cropped images with the prediction masks were merged into a WSI. The areas of prediction masks were calculated and transformed into the percentage of the liver section area to validate how well the algorithm predicted hepatic fibrosis compared to the ground truth data.

In addition, to test whether the algorithm can represent the pathologists’ diagnoses, the hepatic fibrosis rate and rank by the algorithm were compared to the pathologists’ grades. Four board-certified pathologists graded each WSI according to the Ishak hepatic scoring system.³² The grades of the sample WSIs ranged from grades 0 to 5, and the pathologists also determined the rank of the fibrosis for each WSI. Finally, the average value of grade and rank by pathologists was compared.

Statistical Analyses

To prove that the algorithm can represent the pathologists’ diagnoses, the hepatic fibrosis rate and rank by the algorithm were compared to the average grades and ranks of pathologists’. Linear regression was applied for the correlations between the algorithm and pathologists. Statistical analyses were performed using GraphPad Prism (version 5.01 for Windows, GraphPad Software, www.graphpad.com).

Results

Training and Test of Hepatic Fibrosis by Mask R-CNN

To train the mask R-CNN network to identify hepatic fibrosis, a total of 10,848 annotated tile images, including the augmented samples, were used. During the training, total losses, including class, mask, and bbox, showed 2 times the elevation at epochs 60 and 120 (Supplementary Figure 3A). The incensement of the loss at these 2 points was due to the changing layer inherent in the algorithm for the effective training of the algorithm. In the validation case, the total loss steadily increased until the end. The last loss value, 3.5, is related to the validation mAP of 0.96 (Supplementary Figure 3B).

The mAP value of 0.95 was calculated using the testing results of 218 tile images with annotations. The mAP calculated here is a transformed value for stricter analysis and evaluation as the original calculated value of mAP was 0.96. Therefore, the model performance obtained from this study is outstanding. The algorithm successfully distinguished between the lesions and normal liver cells using image tiles and blood vessels. The predicted hepatic fibrosis lesions were comparable to the annotated lesions (Figure 2).

Figure 2.

Instance segmentation test results for hepatic fibrosis trained using the Mask R-CNN model. The model performance showed 95% of mAP. The left column shows the original image tiles before annotation, the middle column shows annotated images outlined with a yellow line. The right column shows the predicted area of fibrosis determined by Mask R-CNN. Dashed lines were drawn using the left, top, and right bottom points of predictions, and segmentations were depicted in red. mAP indicates mean average precision; Mask R-CNN, Mask Regions with Convolution Neural Network.

In addition to high accuracy, we identified 5% error cases from the test. We confirmed all the cases of errors to obtain a higher accuracy for further studies. We found 4 patterns from these errors and defined them as type 1 to 4 errors. Type 1 and 2 errors were defined according to the transformed mAP definition that the detection results were not matching the ground truth annotation. Type 1 error: the correct lesion was detected but 2 lesions were recognized as 1. Type 2 error is the opposite case of type 1 error, wherein the correct lesion was detected but 1 lesion was recognized as 2 (Figure 3). Half of the errors belonged to these 2 types. Type 1 and type 2 errors can be considered as correct predictions; however, this was not reflected in mAP.

Figure 3.

Type 1 and type 2 errors observed from the test. These errors can be assumed as correct detection cases.

Type 3 and 4 errors were classified according to the minimum detection confidence assigned in the hyperparameters when operating the algorithm. The minimum detection confidence is a threshold for accuracy. We set this parameter as 0.5 to detect as many lesions as possible and the test results in which confidence was below 0.5 are classified as errors. Type 3 errors comprise incorrect predictions, compared to the ground truth, owing to confusing labels; this was the second most frequent type of error in the test (Supplementary Figure 4). Confusing label means that the annotations did not perfectly cover the lesions in an image or were confused with the normal cells; therefore, they needed to be confirmed. This result implies that exact and precise lesion annotation is the most important factor for higher accuracy of the algorithm test. Type 4 errors were caused by complex labels and were a result of the failure of the algorithm to detect the exact area of the lesion at the confidence level (Supplementary Figure 4). However, even the pathologists had difficulty separating the lesion from the normal hepatic cells in these original image tiles; therefore, it would be fastidious for the algorithm. On including the type 1 and type 2 errors, which were comparable with the ground truth, the accuracy might be estimated as 0.97. It is known that the human error level for classifying an object is 5%. If we supplement the metric calculation method that classifies type 1 and 2 errors as correct, the model performance would be comparable to human error.

Model Accuracy at the Whole Slide Level

To validate the model performance at the whole slide level, we tested the trained algorithm to predict hepatic fibrosis from 18 WSIs, which were independent of the data set. The test was operated at 2 magnification scales, 10× and 20×. The results showed that the model prediction at 20× was more similar to the ground truth determined by a board-certified pathologist, although the model was trained based on the images from 10× magnification. The predictions from the 20× scale detected more lesions that were not predicted in the 10× scale (Figure 4, yellow boxes). The fibrosis rate inferred by the model at the 20× scale showed a high correlation with the ground truth (Figure 5A, R ² = 0.9660). In the case of prediction results at the 20× scale, the fibrosis rank coincided with the ground truth (Figure 5B, R ² = 0.9834).

Figure 4.

The ground truth annotation by a board-certified pathologist (top) and WSI level detection results of fibrosis lesion from two magnification scales images by Mask R-CNN (middle and bottom). The differences between the predictions and ground truth were marked by yellow boxes. Mask R-CNN indicates Mask Regions with Convolution Neural Network; WSI, whole slide image.

Figure 5.

Correlations between fibrosis rate of ground truth and its value inferred by the algorithm (A). Correlations between fibrosis rank of ground truth and inferred rank by the algorithm (B). The red and blue dots represent 20× and 10× magnification, respectively.

After validating the algorithm with the ground truth, we tested the possibility of the algorithm representing the pathologists’ diagnoses. The two parameters, fibrosis ratio and rank by the algorithm, were compared with the average grade and rank determined by pathologists using linear regression. The sample WSIs’ grades ranged from 0 to 5, although they were differently assigned according to the pathologists. Thus, using the average grades of each slide, we analyzed the hepatic fibrosis rate distribution according to the grades before comparing the correlations between the pathologists and the trained algorithm. The result showed that the algorithm tended to predict a greater fibrosis rate than the ground truth, however, the increscent trends according to the grades were similar (Supplementary Figure 5). The correlations between the average grades of the pathologist and the fibrosis ratio determined by the algorithm were lower than those between the algorithm and the ground truth (Figure 6A). The linear regression coefficients between the two magnification scales showed that 20× was rated higher than 10×. The correlations of the rank inferred by the algorithm and the average rank assigned by pathologists had better linear regression coefficients as compared to the grades (Figure 6B). We supposed that the minor numerical differences in the fibrosis ratios between the slides made the pathologists grade them differently. This might have caused the low correlations between the model and the ground truth.

Figure 6.

Correlations between average grade of pathologists and fibrosis rate inferred by the algorithm (A). Correlations between average ranks by pathologists and inferred rank by the algorithm (B). The red and blue dots represent 20× and 10× magnification, respectively.

Discussion

Few studies have investigated hepatic fibrosis in animal models using deep learning. Heinemann et al¹⁸ used a classification algorithm that determined the grade (0-4) of hepatic fibrosis lesions based on tile images. The classification accuracy for fibrosis was 86.3%, and Cohen κ, which indicates the agreement between the deep learning-based approach and the ground truth, was 0.81. This result indicated the possibility of implementing deep learning in preclinical studies; however, the accuracy was not satisfactory. A recent study showed automated quantification of liver fibrosis in mice using a segmentation algorithm, U-net, with two magnifications (10× and 40×) of picrosirius red-stained slide images.¹⁹ The study showed a relatively high F1 score (0.8775), which is a value for evaluating model performance, similar to mAP, and a higher correlation with pathologists’ diagnoses at high magnification than with low magnification. Unlike previous studies, we conducted pixel-level detection of hepatic fibrosis in H&E-stained slides obtained from SD rats using the deep-learning instance segmentation algorithm, Mask R-CNN. The trained algorithm exhibited an accuracy of 0.95, which is the highest value compared to the previous studies that aimed to detect hepatic fibrosis during the test. Considering type 1 and type 2 errors, which were comparable with the ground truth, the accuracy might be estimated as 0.97. Thus, our model exhibits the highest accuracy for detecting hepatic fibrosis in rodents.

In addition to the model accuracy test for tile images, we intended to determine the actual accuracy at the WSI level. A total of 18 slides that were independent of the tile image data set were used for the WSI test. Each slide had ground truth annotated by a board-certified pathologist, and the model performance was validated by comparing the prediction results with the ground truth. The predicted fibrosis ratio calculated by the trained model showed a high correlation with that of the ground truth, and slightly higher regression coefficients were observed at 20× magnification than at 10× magnification scale, although the model was trained based on the images from 10× magnification. This might be because highly magnified images would be useful to detect more lesions than images of low magnification. A few studies have used deep learning algorithms to quantify histopathological changes in structures such as ovarian follicles³³ and fatty vacuoles,³⁴ which are of typical circular shapes that are relatively easier to detect. The strong correlations of fibrosis ratio between the value predicted by the trained model and the ground truth in this study showed the possibility of implementing deep learning techniques to quantify atypically shaped lesions. In addition, the present study showed reliable correlations between the predicted fibro-ratio rank and the average fibrosis rank by pathologists at the WSI level. For prediction of grade or rank, 10× prediction could be acceptable since a slight difference was observed in linear regression coefficients between the two magnifications. However, the objective of pixel segmentation is precise detection and localization of an object, and our data also showed more precise detection at 20× (Figure 4); therefore, we suggest that 20× prediction would be better for future research. Although we used the average rank because of the variation in grades assigned by pathologists due to pathologists’ subjectivity and broad limitation among the grades, the correlation of almost 0.89 is an encouraging result for the implementation of deep-learning techniques in the field of diagnosis for toxicological pathology.

Segmentation using AI networks is a growing methodology in pathology, not only for clinical but also for nonclinical studies. Here, we showed the implementation of pixel-level detection of hepatic fibrosis in H&E-stained slide images, the most commonly used staining method, via the Mask R-CNN network. The model performance observed from the test was encouraging. If complemented by the method for error calculation that covers the errors almost close to the ground truth (type 1 and 2 errors), it might be useful to reduce the errors and improve the model performance. In addition, assigning a ground truth annotation at the WSI level allowed us to confirm the model performance at the slide level, which is inevitable for actual application in nonclinical studies. Although annotation in a slide is laborious, we believe that determining the ground truth at the WSI level and comparing the predicted lesions to the ground truth would be a useful method for model validation. In summary, our study indicates that Mask R-CNN could be a powerful tool to quantify and validate atypical lesions, such as hepatic fibrosis, at the WSI level and is also useful for nonclinical toxicologic pathology diagnosis.

Supplemental Material

sj-doc-1-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-doc-1-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Supplemental Material

sj-tif-1-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-tif-1-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Supplemental Material

sj-tif-2-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-tif-2-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Supplemental Material

sj-tif-3-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-tif-3-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Supplemental Material

sj-tif-4-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-tif-4-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Supplemental Material

sj-tif-5-tpx-10.1177_01926233211057128 – Supplemental material for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats

Supplemental material, sj-tif-5-tpx-10.1177_01926233211057128 for Implementation and Practice of Deep Learning-Based Instance Segmentation Algorithm for Quantification of Hepatic Fibrosis at Whole Slide Level in Sprague-Dawley Rats by Ji-Hee Hwang, Hyun-Ji Kim, Heejin Park, Byoung-Seok Lee, Hwa-Young Son, Yong-Bum Kim, Sang-Yeop Jun, Jong-Hyun Park, Jaeku Lee and Jae-Woo Cho in Toxicologic Pathology

Footnotes

Declaration of Conflicting Interests

The author(s) declared no real, perceived, or potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant (20183MFDS411) from Ministry of Food and Drug Safety in 2021.

ORCID iD

Ji-Hee Hwang

Supplemental Material

Supplemental material for this article is available online.

References

Brody

. Medical imaging. Nature. 2013;502(7473):S81. doi:10.1038/502s81a

Mahmood

Owais

Noh

, et al. Accurate Segmentation of Nuclear Regions with Multi-Organ Histopathology Images Using Artificial Intelligence for Cancer Diagnosis in Personalized Medicine. J Pers Med. 2021;11(6):515. doi:10.3390/jpm11060515

Russakovsky

Deng

, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115(3):211–252. doi:10.1007/s11263-015-0816-y

LeCun

Bengio

Hinton

. Deep learning. Nature. 2015;521(7553):436–444. doi:10.1038/nature14539

Liu

Deng

Yang

. Recent progress in semantic image segmentation. Artif Intell Rev. 2019;52:1089–1106. https://doi.org/10.1007/s10462-018-9641-3

Wei

G-Q

Arbter

Hirzinger

. Automatic tracking of laparoscopic instruments by color coding. Lect Notes Comput Sci. 1997;1204:357–366. doi:10.1007/bfb0029257

Moon

Bullitt

van Leemput

Gerig

.Automatic Brain and Tumor Segmentation. In: Dohi

Kikinis

, eds. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2002. Lect Notes Comput Sci. Springer. 2002;2488:372–379. doi:10.1007/3-540-45786-0_46

Cohen

Rivlin

Shimshoni

Sabo

. Memory based active contour algorithm using pixel-level classified images for colon crypt segmentation. Comput Med Imaging Graph. 2015;43:150–164. doi:10.1016/j.compmedimag.2014.12.006

Long

Shelhamer

Darrell

. Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015:3431–3440. doi:10.1109/cvpr.2015.7298965

10.

Noh

Hong

Han

. Learning Deconvolution Network for Semantic Segmentation. 2015 IEEE International Conference on Computer Vision (ICCV). 2015:1520–1528. doi:10.1109/iccv.2015.178

11.

Dodballapur

Song

Huang

Chen

Chrzanowski

Cai

. Mask-driven mitosis detection in histopathology images. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). 2019:1855–1859. doi:10.1109/isbi.2019.8759164

12.

Ronneberger

Fischer

Brox

U-Net: Convolutional Networks for Biomedical Image Segmentation. Lect Notes Comput Sci. 2015;9351:234–241. doi:10.1007/978-3-319-24574-4_28

13.

Gkioxari

Dollár

Girshick

. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV); October 22-29, 2017; Venice, Italy:2980–2988, doi:10.1109/ICCV.2017.322

14.

Liu

Dong

. Segmentation of Lung Nodule in CT Images Based on Mask R-CNN. 2018 9th International Conference on Awareness Science and Technology (iCAST). 2018:1–6. doi:10.1109/icawst.2018.8517248

15.

Altini

Cascarano

Brunetti

, et al. A deep learning instance segmentation approach for global glomerulosclerosis assessment in donor kidney biopsies. Electronics. 2020;9(11):1768. doi:10.3390/electronics9111768

16.

Chiao

J-Y

Chen

K-Y

Liao

KY-K

Hsieh

P-H

Zhang

Huang

T-C

. Detection and classification the breast tumors using mask R-CNN on sonograms. Medicine. 2019;98(19):e15200. doi:10.1097/md.0000000000015200

17.

Naylor

Lae

Reyal

Walter

. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Med Imaging. 2019;38(2):448–459. doi:10.1109/tmi.2018.2865709

18.

Heinemann

Birk

Stierstorfer

. Deep learning enables pathologist-like scoring of NASH models. Sci Rep. 2019;9(1):1–10. doi:10.1038/s41598-019-54904-6

19.

Ramot

Deshpande

Morello

Michieli

Shlomov

Nyska

. Microscope-Based automated quantification of liver fibrosis in mice using a deep learning Algorithm. Toxicol Pathol. 2021;49(5):1126–1133. doi:10.1177/01926233211003866

20.

George

Tsuchishima

Tsutsumi

. Metabolism of N-nitrosodimethylamine, methylation ofmacromolecules, and development of hepatic fibrosis in rodent models. J Mol Med. 2020;98(9):1203–1213. doi:10.1007/s00109-020-01950-7

21.

George

Tsuchishima

Tsutsumi

. Molecular mechanisms in the pathogenesis of N-nitrosodimethylamine induced hepatic fibrosis. Cell Death Disease. 2019;10(1):1–9. doi:10.1038/s41419-018-1272-8

22.

Ronneberger

Fischer

Brox

. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab

Hornegger

Wells

Frangi

, eds. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lect Notes Comput Sci. Springer. 2015:9351. doi:10.1007/978-3-319-24574-4_28.

23.

Gkioxari

Dollar

Girshick

. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV) . 2017:2980–2988. doi:10.1109/ICCV.2017.322.

24.

Durkee

Abraham

Fuhrman

Clark

Giger

. Comparing Mask R-CNN and U-Net architectures for robust automatic segmentation of immune cells in immunofluorescence images of Lupus Nephritis biopsies. Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XIX. SPIE 11647. 2021;11647:116470X. doi:10.1117/12.2577785

25.

Alfaro

Fonseca

Albornoz

Martínez

Ramrez

. A Brief Analysis of U-Net and Mask R-CNN for Skin Lesion Segmentation. 2019 IEEE International Work Conference on Bioinspired Intelligence (IWOBI). 2019:000123–000126. doi:10.1109/IWOBI47054.2019.9114436

26.

Bini

. Artificial intelligence, machine learning, deep learning, and cognitive computing: what do these terms mean and how will they impact health care? J Arthroplasty. 2018;33(8):2358–2361. doi:10.1016/j.arth.2018.02.067

27.

Bertram

Aubreville

Marzahl

Maier

Klopfleisch

. A large-scale dataset for mitotic figure assessment on whole slide images of canine cutaneous mast cell tumor. Sci Data. 2019;6(1):274. doi:10.1038/s41597-019-0290-4

28.

Bertram

Aubreville

Gurtner

, et al. Computerized calculation of mitotic count distribution in canine cutaneous mast cell tumor sections: mitotic count is area dependent. Vet Pathol. 2020;57(2):214–226. doi:10.1177/0300985819890686

29.

Abadi

Agarwal

Barham

, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv. 2016. 1603.04467.

30.

Matterport/Mask_RCNN. GitHub. https://github.com/matterport/Mask_RCNN. Published 2019. Accessed February 10, 2021.

31.

Jaderberg

Simonyan

Zisserman

Kavukcuoglu

. Spatial transformer networks. In: NIPS, ed. Advances in Neural Information Processing Systems. Curran Associates, Inc; 2015;28. https://proceedings.neurips.cc/paper/2015/file/33ceb07bf4eeb3da587e268d663aba1a-Paper.pdf.

32.

Ishak

Baptista

Bianchi

, et al. Histological grading and staging of chronic hepatitis. J Hepatol. 1995;22(6):696–699. doi:10.1016/0168-8278(95)80226-6

33.

Carboni

Marxfeld

Tuoken

, et al. A workflow for the performance of the differential ovarian follicle count using deep neuronal networks. Toxicol Pathol. 2021;49(4):843–850. doi:10.1177/0192623320969130

34.

Ramot

Zandani

Madar

Deshmukh

Nyska

. Utilization of a deep learning Algorithm for microscope-based fatty vacuole quantification in a fatty liver model in mice. Toxicol Pathol. 2020;48(5):702–707. doi:10.1177/0192623320926478

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

4.55 MB

2.62 MB

1.55 MB

31.46 MB

0.76 MB