Sage Journals: Discover world-class research

Abstract

Many eye-related diseases will lead to blindness or worse when it is lack of treatment in the early stages of the disease. Retinal vessel is important for doctors to detect eye diseases, even though the increase of some thin vessels may also mean the occurrence of certain diseases. Therefore, automatic retinal vessel segmentation is of great help to doctors in diagnosing diseases. In this paper, an automatic vessel segmentation method is proposed for retinal image, which is based on support vector machine combining multi-scale feature fusion model and B-COSFIRE filter response. Firstly, the inverted green channel image is enhanced by B-COSFIRE filter to strengthen bar-like vessel structures. Then the features are extracted by means of line operator in a multiresolution way, namely that each filtered image is down-sampled to cover a wider area, hence each sampled pixels can obtain not only the global but also local information. Then the final obtained features from three scales together along the depth direction are combined to train the SVM model. Finally, we use the classifier model to predict blood vessels. The proposed algorithm is evaluated on the public available fundus images datasets (DRIVE: Precision = 0.8657, Se = 0.7088, Sp = 0.9660 and ACC = 0.9900; STARE: Precision = 0.8782, Se = 0.6189, Sp = 0.9908 and ACC = 0.9494). The experiment results show that our proposed algorithm has effects on retinal vessels segmentation.

Keywords

Retinal Vessel Segmentation B-COSFIRE Filter Support Vector Machine Multi-Scale Manner

Introduction

Automatic retinal blood vessels segmentation is good for the detection of vascular diseases such as cardiovascular and cerebrovascular diseases, diabetes, and ophthalmological diseases.^1,2 But manual vessel segmentation for retinal image is time consuming and laborious tasks. Therefore, the automatic retinal vessel segmentation provides a convenient way to assist doctors' analysis and follow-up treatment.^3–5

Many automatic retinal vessel segmentation methods^5–7 have been proposed. The current retinal vascular segmentation methods are mainly divided into two categories: unsupervised methods and supervised methods.

The unsupervised methods mainly utilize the features to segment retinal blood vessels without training sample. Sahin et al.⁸ proposed a multi-scale segmentation algorithm combining hybrid region merging with watershed segmentation. Liu et al.⁹ proposed a threshold segmentation method based on principal component, which use singular value decomposition (SVD) algorithm to accurately locate the principal component and reduce the synthesis strength through threshold segmentation. Pin et al.¹⁰ utilized wavelet decomposition and multi-scale region-growing to derive regions of interest for accurate location. Dong et al.¹¹ utilized a novel sub-Markov random walk algorithm with prior label to iron out slender objectssegmentation. Shafiullah et al.¹² concatenated local and global information to do intensity inhomogeneous image segmentation. Montuoro et al.¹³ proposed a surface segmentation model based on graph theory algorithm. Saroj et al.¹⁴ used principal component analysis (PCA) method to transformoriginal color image into gray-scale image and designed Fréchet matched filter to select best parameters.

Compared with unsupervised methods, supervised learning methods are to learn vessel features from a plenty of labeled training images or to train vessel pixel classifier model for vessel segmentation. The existing supervised methods can be divided into deep learning related methods^15–19 and traditional machine learning methods. Hinton et al.²⁰ formally proposed the concept of deep learning in 2006 and the deep learning method immediately aroused great repercussions in the academic circle. Since Alexnet²¹ won the famous Imagenet image recognition competition in 2012, the deep learning algorithm has stood out in the world competition and once again attracted the attention of academia and industry in the field of deep learning. Deep learning algorithms have increasingly been developed. In deep learning related methods, deep convolutional neural network had superior performance on a variety of tasks in image processing and computer vision. Ronneberger et al.²² proposed a u-shaped deep learning network called U-net, which only need a small amount of image to train the model, and it can obtain good segmentation accuracy. Some similar networks have been proposed one after another, such as V-Net²³, Res-U-Net¹⁸, U-Net++²⁴. Ma et al.²⁵ proposed a multi-task segmention method that can segment blood vessels while also distinguishing arteries and veins. Liskowski et al.⁷ used the deep learning method to do multi-pixel classifification, which can train large samples. These methods can achieve satisfactory performance, but they need a highly configured running environment. Many traditional machine learning methods work well for medical image processing tasks. For instance, the support vector machine (SVM)is extensively used in retinal blood vessel segmentation owing to its great performance. Ricci et al.²⁶ proposed a retinal vessels segmentation method with SVM. Aiping et al.²⁷ proposed a novel pixel-wise SVM classifier to extract multiple morphological characteristics of tumor nests and segment tumor nests and stroma from images. Wang et al.²⁸ proposed a method which possessed corrected morphological transformation and fractal dimension. Li et al.²⁹ designed a new deep network to segment retinal blood vessels through cross-modality data conversion, which has strong induction ability. M. Liu et al.³⁰ proposed a multi-template method based on relational induction to obtain important structural imformation in order to maximize performance.

In this paper, a supervised learning method based on the SVM is proposed, which combines B-COSFIRE (the Combination of Shifted Filter Responses with B standing for bar, which is an abstraction for a vessel) filter and multi-scale features fusion. Firstly, the B-COSFIRE filter is used for the retinal image to increase the contrast information of vascular structure and non-vascular areas. Secondly, considering the particularity of retinal vascular structure, the image features are extracted by improved line operator and the image is convoluted with 12 rotated versions of 2D matched filter impulse response. And the maximum response relative to angle is reserved. The multi-scale features fusion is used to obtain not only the global but also local information. Then the training dataset is composed of multi-scale feature vectors and filtered images to train the SVM classifier. Finally, vascular classification results for retinal vessel images are obtained by SVM classifier. And the image binarization operation is done by post processing operation for obtaining final vessel tree.

Method

Our goal here is to obtain satisfactory vessel segmentation result by SVM method³² for retinal image. In our proposed method, we adopt two stages retinal blood vessel segmentation method: train stage and test stage. There are four main processes. (1) Image enhancement. This is a preprocessing operation to increase the contrast information of vascular structure and non-vascular areas by B-COSFIRE filter for inverted green channel image. (2) Multi-scale features fusion. The features are extracted by means of line operator from fifiltered image in a multiresolution way to obtain not only the global but also local information. (3) Train SVM classifier. The filtered image and the corresponding extracted features are combined as the train data to train a SVM classifier. (4) Application of SVM classifier. The final segmentation image is obtained by SVM classifier from training stage. And the test data is done by using the same operation as the training stage. It includes image enhancement, multi-scale features fusion and train SVM classifier in training stage. It includes the image enhancement, multi-scale features fusion and application of SVM classifier in testing stage. The schematic illustration of the proposed segmentation process is showed in Figure 1, and we will elaborate on our proposed supervised learning method hereinafter.

Figure 1.

The schematic illustration of the proposed segmentation process: (1) Image enhancement, (2) Multi-scale features fusion, (3) Train SVM classifier, and (4) Application SVM classifier.

Train stage

Image enhancement

Considering the linear characteristics of retinal blood vessels, two line operators are utilized to calculate the feature vector instead of one line operator. Firstly, we used the all-green channel image throughout the training and testing process considering the noise of red channel and blue channel. The results of image prepocessing are shown in Figure 2. Then the inversion method is used to increase the contrast information of vascular structure and non-vascular areas. Then B-COSFIRE filter^31,33,34 is used for distinguishing blood vessels from non-vascular information, which is a kind of two line detectors and effective to detect strip structures.

Figure 2.

The results of image preprocessing. From left to right, (a) is original image, (b) is the retinal image of the extracted green channel part, (c) is the inversion image after extracting green channel, respectively.

The filter is defined by the product of a set of Gaussian of differential (DoG) filter responses to achieve B-COSFIRE response. For a retinal image $I$ , $(x, y)$ is the intensity in pixel point $p$ . And the filtering formula is defined as follow:

\begin{matrix} Do G_{σ} (x, y) = \frac{1}{2 π σ^{2}} \exp (\frac{x^{2} + y^{2}}{- 2 σ^{2}}) - \frac{1}{2 π {(0.5 σ)}^{2}} \exp (\frac{x^{2} + y^{2}}{- 2 {(0.5 σ)}^{2}}) \end{matrix}

(1)

Here,

σ

is the standard deviation of the Gaussian function. The convolution formula of DoG filter is defined as follow:

C_{σ} (x, y) \overset{d e f}{=} {| I * D o G_{σ} |}^{+}

(2)

C_{σ} (x, y)

is obtained by convolution of the inverted green channel image and the DoG filter with kernel function

Do G_{σ} (x - x^{'}, y - y^{'})

(x^{'}, y^{'})

is the intensity in pixel point

q

. Figure 3 shows the B-COSFIRE’s selective configuration for vertical bars. The support area center is symbolized by a cross sign “1”. The rest of points represent the points of interest where the strongest DoG response is obtained along a concentric circle. As shown in Figure 4, the black dot is DoG filter support center and each gray circle represents a DoG filter. The B-COSFIRE combines each group responses of DoG filters by multiplication. Fuzzy DoG response and shift operation are added to increase the fault tolerance of B-COSFIRE. The fuzzy operation is to calculate the maximum weighted threshold response of the DoG filter. The shift operation is to offset each fuzzy DoG response. According to different vessel directions, 12 directions of the rotating filter are set to select the maximum value. In principle, the B-COSFIRE cannot get a response at the end of the strip (or vessel). However, the B-COSFIRE can also obtain the information of vascular endings owing to the noise background in the retinal image. Then another B-COSFIRE filter is added to tack the problem. Therefore, the final feature map

I^{'}

as shown in Figure 5 is obtained by two B-COSFIRE responses.

Figure 3.

B-COSFIRE’s selective configuration.³¹

Figure 4.

Schematic diagram of B-COSFIRE.

Figure 5.

The results of image enhancement with B-COSFIRE for two images from datasets DRIVE and STARE. And (a) is originalimage from DRIVE, (b) is (a) corresponding fifiltered image, (c) is original image from STARE, (d) is (c) corresponding fifiltered image.

Multi-Scale Feature Fusion

Linear feature extraction is done by linear operator for the feature map images. These features are extracted by means of line operator in a multi-resolution way, namely that each filtered image is down-sampled to cover a wider area, hence each sampled pixels can obtain not only the global but also local information.

For calculating the corresponding line strength²⁶ to each pixel on the feature map, we first calculate the average gray value of point in different directions to obtain the line intensity $S (x, y)$ of each target pixel point $(x, y)$ $.$ When calculating the average gray value of point $(x, y)$ , we extract patches of different sizes according to different scales. For instance, when the scale is 15, the patch size is 15*15. The others are 7*7 and 23*23, respectively. And 12 directions are used to evaluate the line intensity of black point pixels (the angular resolution is 15 $^{\circ}$ ) and find out the direction of the maximum average gray level of point $(x, y)$ by the value of the maximum average gray level. If the selected line is parallel to the blood vessel, the line intensity will reach the maximum value as shown in Figure 6(a). Otherwise, the selected line would overlap with the blood vessel and the intensity of corresponding line will decrease. Then the vascular pixels and non-vascular pixels are distinguished by calculating line strength.

Figure 6.

Linear feature extraction. (a) is the selected line (15 gray point pixels) of blood vessel, (b) is orthogonal line (3 gray point pixels).

$L (x, y)$ and $N (x, y)$ respectively represent the maximum average gray value, the average gray value in the square window that the size is equal to $L$ . The calculation formula of line strength is:

S (x, y) = L (x, y) - N (x, y)

(3)

The orthogonal line is also considered to distinguish the large bright vessels and dark thin vessels since they have similar intensity, which is showed in Figure 6(b). The selected orthogonal line is 3 pixels and its average gray level is represented by

L_{0} (x, y)

. By subtracting the average intensity of the square window again, the orthogonal line intensity can be obtained:

S_{0} (x, y) = L_{0} (x, y) - N (x, y)

(4)

Next, for reducing errors of feature extraction in the optic disc, a feature vector near

V = [S (x, y), S_{0} (x, y), I^{'} (x, y)]

is constructed. Where,

S (x, y)

S_{0} (x, y)

and

I^{'} (x, y)

represent line intensity, orthogonal line intensity, and gray value of the pixel point

p

in feature map, respectively.

Generally, smaller scale sampling can observe the details of the image while larger scale sampling can grasp the overall characteristics of the image. In the single scale segmentation, the line operator of length 15 pixels performs best. Compared with single scale sampling, multi-scale sampling can obtain the global and local information. We chose a lot of different scales and the three scales of 7, 15 and 23 works best. Then the final obtained features from three scales together along the depth direction with the feature map image is integrated to train the SVM classifier.

Train SVM classifier

The feature map of linear features extracted from three different scales and extracted gray image from original image are used for fusion operation, and the feature map obtained is used as the input of SVM classifier for image segmentation. Usually, SVM³² is used for settling the problem of binary classification of data. N samples are input and the label of output samples is represented by ${+ 1, - 1}$ . Taking the training samples as the research object, a hyperplane $W^{T} X + b = 0$ is found in the feature space of the samples to effectively separate the labels $+ 1$ and $- 1$ of the two types of samples.

Test stage

For each test image data, the image enhancing and multi-scale features fusion are done as same as the train stage. Then the SVM classifier is applied for obtaining the final vessel tree map, which is obtained from the train stage.

Experiments

Dataset and Parameter Setting

The public dataset DRIVE⁴⁰ (The Digital Retinal Images for Vessel Extraction) and STARE⁴¹ (Structured Analysis of the Retinal) are used for evaluation. There are 40 retinal images in DRIVE dataset and evenly divided as training set and testing set respectively. In STARE dataset, we use first 14 images to train the method and 6 images for testing. All retinal images are manually segmented by two experts and the manual segmentation images from the first expert in DRIVE and STARE dataset are adopted to evaluate the performance of the our method.

And for multiresolution manner, the number of multi-scale is 3 and the patch size is 7*7, 15*15 and 23*23 for each scale since the best performance can be obtained when the patch size is 15*15.

Evaluation Criteria

The segmentation performance of our proposed method in different procedures is compared with several state-of-the-art blood vessel segmentation methods. And four evaluation criteria are used for evaluating the segmentation performance, including Accuracy (ACC), Precision, Sensitivity (Se) and Specificity (Sp). They are defined as follow:

Accuracy = \frac{T P + T N}{T P + T N + F P + F N}

(5)

Sensitivity = \frac{T P}{T P + T N}

(6)

Precision = \frac{T P}{T P + F P}

(7)

Specificity = \frac{T N}{T N + F P}

(8)

Where, TP as true positive, FP as false positive, TN as true negative and FN as false negative by given binary segmentation map and manual annotation. The contribution of the image enhance manner and the multiresolution manner are also evaluated by the four evaluation criteria.

Experiment and Analysis

First, the performance of the proposed method on DRIVE and STARE is as shown in Tables 1 and 2, and four evaluation criteria for each image and their mean and standard deviation (mean $\pm$ std %) are calculated. The best results are shown in bold for each evaluation criterion. In DRIVE dataset, the average Precision, Sensitivity(Se), Specificity(Sp) and Accuracy(ACC) are 0.8657, 0.7088, 0.9900 and 0.9660, and in STARE dataset, the average Precision, Sensitivity(Se), Specificity(Sp) and Accuracy(ACC) are 0.8782, 0.6189, 0.9908 and 0.9494, respectively. Image enhancement for original retinal image increases the retinal vascular structure information and assists the subsequent segmentation. The multiresolution manner helps each sampled pixels obtain not only the global but also local information.

Table 1.

The performance for the proposed method on DRIVE dataset. The best results are shown in bold.

Testing pictures	Precision	Sensitivity	Specificity	Accuracy
1	0.8480	0.7364	0.9876	0.9654
2	0.9404	0.6912	0.9950	0.9645
3	0.8632	0.7195	0.9889	0.9649
4	0.9203	0.6755	0.994	0.9664
5	0.9125	0.7135	0.9940	0.9712
6	0.8850	0.6589	0.9912	0.9603
7	0.8328	0.7403	0.9885	0.9708
8	0.7535	0.7421	0.9827	0.9667
9	0.8581	0.6783	0.9901	0.9650
10	0.8467	0.7389	0.9897	0.9717
11	0.8705	0.7098	0.9904	0.9671
12	0.8263	0.7580	0.9861	0.9678
13	0.9181	0.6216	0.9936	0.9561
14	0.8141	0.7682	0.9858	0.9696
15	0.8120	0.6769	0.9874	0.9642
16	0.8851	0.7272	0.9912	0.9687
17	0.8080	0.7729	0.9853	0.9696
18	0.9056	0.7119	0.9925	0.9667
19	0.9198	0.6905	0.9934	0.9633
20	0.8944	0.6451	0.9922	0.9601
Mean $\pm$ std	0.8657 $\pm$ 0.047	0.7088 $\pm$ 0.1591	0.9900 $\pm$ 0.2158	0.9660 $\pm$ 0.2106

Table 2.

The performance for the proposed method on STARE dataset. The best results are shown in bold.

Testing pictures	Precision	Sensitivity	Specificity	Accuracy
15	0.9253	0.6426	0.9926	0.9487
16	0.9640	0.481	0.9968	0.9182
17	0.9372	0.6107	0.9938	0.9436
18	0.8984	0.6990	0.9948	0.9765
19	0.7820	0.6860	0.9872	0.9683
20	0.7623	0.5932	0.9795	0.9410
Mean $\pm$ std	0.8782 $\pm$ 0.085	0.6189 $\pm$ 0.078	0.9908 $\pm$ 0.006	0.9494 $\pm$ 0.021

Then, we compared the performance of different procedures in our method with several state-of-the-art blood vessels segmentation methods by four evaluation criteria on DRIVE dataset: (1) our method without BC and MCM is single scale without B-COSFIRE, (2) our method without MCM is single scale with B-COSFIRE, (3) our method combines multi-scale and B-COSFIRE. We provided the vascular segmentation results of DRIVR dataset, as shown in Table 3, Figure 7 and Figure 8. And we also compared the performance of our best method with several state-of-the-art blood vessels segmentation methods by four evaluation criteria on STARE dataset. The segmentation results of retinal vessels in STARE dataset are shown in Table 4. It is shown that our method is effective for improving the performance including precision, specificity and accuracy by using B-COSFIRE operation and multiresolution manner.

Table 3.

The comparison of experimental results between different other methods and our proposed method with and without B-COSFIRE(BC), with and without multiresolution manner(MSM) on DRIVE dataset. The best results are shown in bold.

Algorithm	Precision	Sensitivity	Specificity	Accuracy
YAN³⁵	-	0.7631	0.9820	0.9538
Fu³⁶	-	0.7603	-	0.9523
Orlando³⁷	0.7854	0.7897	0.9684	-
Budai, A³⁸	-	0.644	0.987	0.9572
Chakraborti³⁹	-	0.7205	0.9579	0.9370
Our method w/o BC and MSM	0.8409	0.615	0.9882	0.9554
Our method w/o MSM	0.8572	0.6933	0.9889	0.9628
Our method	0.8657	0.7088	0.9900	0.9660

Table 4.

The comparison of experimental results between different other methods and our proposed method with B-COSFIRE(BC) and multiresolution manner(MSM) on STARE dataset. The best results are shown in bold.

Algorithm	Precision	Sensitivity	Specificity	Accuracy
YAN³⁵	-	0.7735	0.9857	0.9636
Fu³⁶	-	0.7412	-	0.9585
Orlando³⁷	0.7740	0.7680	0.9738	-
Budai, A³⁸	-	0.58	0.982	0.9339
Chakraborti³⁹	-	0.6786	0.9586	-
our method	0.8782	0.6189	0.9908	0.9494

The exemplar results of vessel segmentation are given in Figure 7. For the first row, it is the original retinal fundus image, the manual annotation, the segmented result, and the highlighted differences between the manual annotation and the segmented result, respectively. And for the second row, they are local enlarged results from the first row images. For the highlighted differences image, the red color pixels indicate that vessels are correctly classified as blood vessel pixels, while green color pixels indicate that background pixels are incorrectly classified as blood vessel pixels and the blue color pixels indicate that vessels pixels are incorrectly classified as background pixels.

Figure 7.

Exemplar result of vessel segmentation on DRIVE. For the first row, from left to right: the retinal image, the manual annotation,the segmented result using our proposed method, and the highlighted differences between manual annotation and the segmented image using proposed method, respectively. For the second row, they are the locally enlarged results from the images of the fifirst row.

The segmentation results are shown in Figure 8 and Figure 10. It shows segmentation results with our different procedures compared with the ground-truth, which is selected by the best and worst segmentation map among all segmentation evaluation criteria for test images in the DRIVE database. In Figure 8, column (a) are original retinal images, column (b) are manual segmentation images, column (c) are the segmentation images using single scale without B-COSFIRE filtering, column (d) are the segmentation images using single scale after B-COSFIRE filtering, respectively. Column (e) in Figure 8 are the segmentation images after B-COSFIRE filtering with multiresolution manner. And in column (e), the first row is the test image with the highest precision and specificity in the DRIVE testset, which are 0.9404 and 0.9950 respectively. And the test images with the highest sensitivity and accuracy criteria are displayed in the second and third lines, with sensitivity and accuracy of 0.7729 and 0.9717 respectively. The fourth row is the test image with lowest precision and specificity, which are 0.7585 and 0.9827 respectively. And the last row is the test image with lowest sensitivity and accuracy criteria, which are 0.6216 and 0.9561 respectively. It is shown that, (1) the perfomance of the proposed procedure is gradually improved as more procedures are trained with additional features which are obtained by multiresolution manner. (2) The B-COSFIRE filtering can improve the performance for the vessel segmentation. And in Figure 9, it shows segmentation results with our best model procedure 3 compared with the ground-truth for test images in STARE dataset. From left to right in Figure 9, column (a) are original retinal images, column (b) are manual segmentation images, column (c) are the segmented images by our proposed method, column (d) are the highlighted differences between manual annotation and the segmented image using our proposed method, respectively.

Figure 8.

The vessel segmentation results for retinal images in DRIVE dataset. From left to right, (a) is original image, (b) is manual segmentation image, (c) is our method without BC and MCM, (d) is our method without MCM, (e) is our method, respectively.

Figure 9.

The vessel segmentation results for retinal images in STARE dataset. From left to right, (a) is original image, (b) is manual segmentation image, (c) is the segmented image by our proposed method, (d) is the highlighted differences between manual annotation and the segmented image using our proposed method, respectively.

Figure 10.

The comparison with different procedure in DRIVE dataset. Evaluation criteria are precision, sensitivity, specificity and accuracy, respectively.

Conclusion

In this paper, we propose an automatic vessel segmentation method for retinal fundus image, which is founded on support vector machine combining multi-scale feature fusion model and B-COSFIRE filter response. Firstly, the inverted green channel part of images are extracted and the B-COSFIRE filter was utilized to strengthen bar-like vessel structures. Then the features are extracted by line operator in a multiresolution manner. Then the final obtained features from three scales together along the depth direction is combined to train the SVM classifier model. Finally, the classifier is utilized to predict the vessel map. It is shown that our proposed procedure have effect on achieving higher performance from experimental results. And the decrease of sensitivity is mainly due to the omission of small blood vessels, which will be the key point of the future work. The effect of our method to strengthen the sensitivity in retinal fundus images which have lesions is scant. And deep learning algorithms, especially U-Net,²² and U-shaped improved network algorthms, have been proven to be effective way to to segment medical image in recent years. In the future, we will investigate and study how to improve U-shaped networks instead of SVM to train more efficient models for better retinal images segmentation in the future work.

Footnotes

Acknowledgement

Han Zhou and Hongtao Xu checked the sentence errors and formatting in the revision process of the paper. Changcai Yang and Riqing Chen put forward and guided me through many opinions on the later stage of the thesis.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

This work is supported by the Natural Science Foundation of Fujian Province (Grant No. 2020J01573), National Natural Science Foundation of China (Grant No. 61701117, 61802064) and the Fund of Cloud Computing and Big Data for Smart Agriculture (Grant No. 117-612014063), the Research Foundation for Education Bureau Young Scientists of Fujian Province (Grant No. JAT180748), the subject of Education Science Planning of Fujian Province in 2020 (FJJKCG20-178) and Fuzhou Science and Technology Project (Grant No. 2019-SG-7).

ORCID iDs

Ting Zhang

Jun Li

Lifang Wei

References

Cheng

, et al. Structure-preserving guided retinal image filtering and its application for optic disk analysis. IEEE Trans Med Imaging 2018; 37(11): 2536–2546. DOI: 10.1109/TMI.2018.2838550.

Ikram

Janssen

JAMJL

Roos

AME

, et al. Retinal vessel diameters and risk of impaired fasting glucose or diabetes. Diabetes 2006; 55(2): 506–510. URL http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=20050501&lang=zh-cn&site=ehost-live.

Mou

Zhao

Chen

, et al. Cs-net: Channel and spatial attention network for curvilinear structure segmentation. In Shen

Liu

Peters

, et al. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham: Springer International Publishing. ISBN 978-3-030-32239-7, pp. 721–730.

Sevik

Köse

Berber

, et al. Identification of suitable fundus images using automated quality assessment methods. J Biomed Opt 2014; 19(4): 46006.

Zhao

Zheng

Liu

, et al. Automatic 2-d/3-d vessel enhancement in multiple modality images using a weighted symmetry filter. IEEE Trans Med Imaging 2018; 37(2): 438–450. DOI: 10.1109/TMI.2017.2756073.

Yan

Yang

Cheng

KTT

. A three-stage deep learning model for accurate retinal vessel segmentation. IEEE J Biomed Health Inform 2018; PP: 1–1.

Liskowski

Krawiec

. Segmenting retinal blood vessels with deep neural networks. IEEE Trans Med Imaging 2016; 35(11): 2369–2380. DOI: 10.1109/TMI.2016.2546227.

Sahin

Ulusoy

. Automatic multi-scale segmentation of high spatial resolution satellite images using watersheds. In 2013 IEEE International Geoscience and Remote Sensing Symposium-IGARSS. pp. 2505–2508. 10.1109/IGARSS.2013.6723330.

Liu

. Vessel segmentation using principal component based threshold algorithm. In 2019 Chinese Control And Decision Conference (CCDC). pp. 2739–2741. 10.1109/CCDC.2019.8832506.

10.

Wang

, et al. Automatic cell nuclei segmentation and classification of breast cancer histopathology images. Signal Processing 2016; 122: 1–13. DOI: https://doi.org/10.1016/j.sigpro.2015.11.011. URL http://www.sciencedirect.com/science/article/pii/S01651684150%03916.

11.

Dong

Shen

Shao

, et al. Sub-markov random walk for image segmentation. IEEE Trans Image Process 2016; 25(2): 516–527. DOI: 10.1109/TIP.2015.2505184.

12.

Soomro

Munir

Choi

. Hybrid two-stage active contour method with region and edge information for intensity inhomogeneous image segmentation. PLoS ONE 2018; 13(1): 1–20. DOI: 10.1371/journal.pone.0191827. URL https://doi.org/10.1371/journal.pone.0191827.

13.

Montuoro

Waldstein

Gerendas

, et al. Joint retinal layer and fluid segmentation in oct scans of eyes with severe macular edema using unsupervised representation and auto-context. Biomed Opt Express 2017; 8(3): 1874–1888. DOI: 10.1364/BOE.8.001874. URL http://www.osapublishing.org/boe/abstract.cfm?URI=boe-8-3-187%4.

14.

Saroj

Kumar

Singh

. Fréchet pdf based matched filter approach for retinal blood vessels segmentation. Comput Methods Programs Biomed 2020; 194: 105490. https://doi.org/10.1016/j.cmpb.2020.105490. URL http://www.sciencedirect.com/science/article/pii/S01692607193%16761.

15.

Cheng

, et al. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 2019; 38(10): 2281–2292. DOI: 10.1109/TMI.2019.2903562.

16.

Shelhamer

Long

Darrell

. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 2017; 39(4): 640–651. DOI: 10.1109/TPAMI.2016.2572683. URL https://doi.org/10.1109/TPAMI.2016.2572683.

17.

Russakovsky

Deng

, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015; 115(3): 211–252.

18.

Xiao

Lian

Luo

, et al.: Weighted res-unet for high-quality retina vessel segmentation. In 2018 9th International Conference on Information Technology in Medicine and Education (ITME). pp. 327–331. 10.1109/ITME.2018.00080.

19.

Tian

Chen

Yang

, et al.:Variational pansharpening by exploiting cartoon-texture similarities. IEEE Transactions on Geoscience and Remote Sensing to be published, DOI: 10.1109/TGRS.2020.3048257.

20.

Hinton

E et

. Reducing the dimensionality of data with neural networks. Science 2006; 313(5786): 504–507.

21.

Krizhevsky

Sutskever

Hinton

. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012; 25(2): 1106–1114.

22.

Ronneberger

Fischer

Brox

. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention.

23.

Çiçek

Abdulkadir

Lienkamp

, et al. 3d u-net: Learning dense volumetric segmentation from sparse annotation. In Ourselin

Joskowicz

Sabuncu

, et al. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Cham: Springer International Publishing. ISBN 978-3-319-46723-8, pp. 424–432.

24.

Zhou

Rahman Siddiquee

Tajbakhsh

. et al. Unet++: A nested u-net architecture for medical image segmentation. In Stoyanov

Taylor

Carneiro

, et al. (eds.) Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer International Publishing. ISBN 978-3-030-00889-5, pp. 3–11.

25.

. et al. Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification. In Shen

Liu

Peters

, et al. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham: Springer International Publishing. ISBN 978-3-030-32239-7, pp. 769–778.

26.

Ricci

Perfetti

. Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging 2007; 26(10): 1357–1365. DOI: 10.1109/TMI.2007.898551.

27.

Aiping

Jiamei

Linwei

, et al. Segmentation of hematoxylin-eosin stained breast cancer histopathological images based on pixel-wise svm classifier. ence China 2015; (09): 92105-092105.

28.

Wang

. Retinal vessel segmentation approach based on corrected morphological transformation and fractal dimension. IET Image Processing 2019; 13(9): 2538–2547. URL https://digital-library.theiet.org/content/journals/10.1049/i%et-ipr.2018.5636.

29.

Feng

Xie

, et al. A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans Med Imaging 2016; 35(1): 109–118. DOI: 10.1109/TMI.2015.2457891.

30.

Liu

Zhang

Shen

. Relationship induced multi-template learning for diagnosis of alzheimer’s disease and mild cognitive impairment. IEEE Trans Med Imaging 2016; 35(6): 1463–1474. DOI: 10.1109/TMI.2016.2515021.

31.

Azzopardi

Strisciuglio

Vento

, et al. Trainable cosfire filters for vessel delineation with application to retinal images. Med Image Anal 2015; 19(1): 46–57. DOI: 10.1016/j.media.2014.08.002. URL https://doi.org/10.1016/j.media.2014.08.002.

32.

Cortes

Vapnik

. Support-vector networks. Mach Learn 1995; 20(3): 273–297.

33.

Memari

Ramli

Bin Saripan

, et al. Supervised retinal vessel segmentation from color fundus images based on matched filtering and adaboost classifier. PLoS ONE 2017; 12(12): e0188939. 10.1371/journal.pone.0188939. URL https://europepmc.org/articles/PMC5724901.

34.

Strisciuglio

Azzopardi

Vento

, et al. Supervised vessel delineation in retinal fundus images with the automatic selection of b-cosfire filters. Machine Vision Applications 2016; 27(8): 1137–1149.

35.

Yan

Yang

Cheng

. A three-stage deep learning model for accurate retinal vessel segmentation. IEEE J Biomed Health Inform 2019; 23(4): 1427–1436. DOI: 10.1109/JBHI.2018.2872813.

36.

Lin

. et al. Deepvessel: Retinal vessel segmentation via deep learning and conditional random field. pp. 132–139. 10.1007/978-3-319-46723-8˙16.

37.

Orlando

Prokofyeva

Blaschko

. A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Transactions on Biomedical Engineering 2017; 64(1): 16–27. DOI: 10.1109/TBME.2016.2535311.

38.

Budai

Bock

Maier

, et al. Robust vessel segmentation in fundus images. Int J Biomed Imaging 2013; 2013(6): 154860.

39.

Chakraborti

Jha

Chowdhury

, et al. A self-adaptive matched filter for retinal blood vessel detection. Mach Vis Appl 2015; 26(1): 55–68.

40.

Staal

Abramoff

Niemeijer

, et al. Ridge-based vessel segmentation in color images of the retina. IEEE Trans Med Imaging 2004; 23(4): 501–509. DOI: 10.1109/TMI.2004.825627.

41.

Hoover, Adam, Kouznetsova et al. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging 2000; 19(3): 203–210.

Learning based multi-scale feature fusion for retinal blood vessels segmentation

Abstract

Keywords

Introduction

Method

Train stage

Image enhancement

Multi-Scale Feature Fusion

Train SVM classifier

Test stage

Experiments

Dataset and Parameter Setting

Evaluation Criteria

Experiment and Analysis

Conclusion

Footnotes

Acknowledgement

Declaration of conflicting interests

Funding

ORCID iDs

References