Retinal blood vessel segmentation using saliency detection model and region optimization

Abstract

In this paper, we present an algorithm for the effective segmentation of retinal blood vessels in vessel quantization for assessing the risk of cerebrovascular diseases. Given that the vessel is the highlight of the fundus image and has a characteristic texture, we adopt color and texture as the saliency features for vessel extraction combined with region optimization. The optimal thresholding can be obtained through the gray histogram thresholding method to segment the vessel. Moreover, morphological operators are applied to preserve the remaining small vessels considering the loss of small vessels. Experiments are designed to evaluate the performance of the proposed models with more than 94% accuracy. Experimental results reveal that the blood vessel can be effectively detected by applying our method on the retinal images.

Keywords

Retinal vessel segmentation saliency feature texture saliency color saliency region optimization

Introduction

Automated vessel segmentation is the prerequisite and the main step in the analysis of fundus images and computer-aided diagnosis of retinal diseases. Retinal vessel can provide many information such as thickness, tortuosity, branch, microaneurysm, and so on.^1,2 The characteristics of the vessel can help to observe the progress of the disease.³ Quantitative measurement is good for doctors to monitor the progress of systemic treatment. Vessel segmentation is also the main step for analyzing the structure of fundus images. For example, OD detection should depend on the results of vessel segmentation.

Various vessel segmentation algorithms have been proposed by domestic and foreign researchers, which can be divided into two categories.⁴ The first category is the unsupervised method, which includes vessel tracking, matched filter, morphological processing, and deformable model.^{7–10,20–22} Vessel tracking may be influenced by the vessel crossings, bifurcations and is terminated when the contrast between the vessels and the background is weak. Kar and Maity⁵ proposed a combined algorithm adopted curvelet transform, matched filtering, and Laplacian of Gaussian filter. Chaudhuri et al.⁶ presented a novel self-adaptive matched filter based on Gaussian shape modeling of vessel cross-sectional profile of the retinal for detecting retinal blood vessels. The accuracy of the approach based on deformable models is low when the vessels and the background are low in contrast. In addition, Zhao et al.⁹ proposed retinal vessel segmentation based on level sets and region growth. The second category is the supervised method, which is based on pixel classification, such as neural networks. Zhu et al.¹⁰ presented an ensemble retinal vessel segmentation based on supervised learning. Fu et al.¹¹ proposed a novel method based on deep learning and random field for retinal vessel segmentation. Usually, supervised methods are extremely time-consuming because a training procedure is indispensable to determine the parameters in these models using ground truth composed by hand-labeled vessel segmentation results.

As a result of the disadvantages of the above algorithms, a novel algorithm based on visual saliency is proposed to segment the retinal vessel. Given the differentiation in color, brightness, and texture feature between regions of retinal vessel and background in fundus images, a saliency model is used to highlight the vessel region in fundus images. Using saliency characteristic of the underlying data can highlight the vessel in images.

In this paper, a visual saliency detection method is used to enhance the contrast between vessel region and background region in fundus images, which is beneficial in extracting vessel region from fundus images. Firstly, guided filtering is used to enhance the contrast between vessel and background region in fundus images. Secondly, color and texture information are used as the saliency features while a region optimization method is proposed for the extraction of vessel regions. Thirdly, thresholding method based on gray histogram is used to obtain the binary image. Finally, morphological operators are applied to preserve the remaining small vessels. Figure 1 shows the flow chart of the proposed technique.

Figure 1.

Flow chart of the proposed technique.

Method

Contrast enhancement based on guided filtering

The fundus image is important in the diagnosis of chronic diseases, such as diabetes and hypertension. The image shows nonuniform brightness and contrast because of the influence of lighting, reflection, perspective, and equipment parameters in image acquisition. Such characteristics can affect the diagnosis result. Image enhancement is thus necessary because the processed images must be matched or merged after processing the fundus images. In this study, we adopt guided filtering¹² to enhance the image. Guided filtering is a linear variable filtering process. The local information on the algorithm is considered, which can keep smoothing edges and limit the amplification of the noise. Guided filtering has a very good enhancing effect, especially for the dark image. The output image can be obtained as follows

q_{i} = \sum_{j} W_{ij} (I) p_{j}

(1)

where I is the guided image, p is the input image, q is the output image, and I is equal to p.

$W_{ij}$ is the filtering kernel function, which can be obtained as follows

W_{ij} (I) = \frac{1}{| ω |^{2}} \sum_{k : (i, j) \in ω_{k}} [1 + \frac{(I_{i} - μ_{k}) (I_{j} - μ_{k})}{σ_{k}^{2} + ɛ}]

(2)

where

ω_{k}

is the kernel window;

| ω |

is the pixel number of the window;

μ_{k}

and

σ_{k}^{2}

are the mean and variance within the window of I, ɛ is the smoothing factor. In this algorithm, the parameter of

| ω |

is 16 and the parameter of ɛ is 0.01. Figure 2 shows the enhanced images.

Figure 2.

The processed image: (a) the original image; (b) the green channel; (c) the guided filtering image.

Construction of saliency detection model

Option of color spaces

Color is an important underlying characteristic in analyzing the saliency of an image. Human vision is sensitive to red, green, yellow, and blue colors. CIELab is close to the visual space. Thus, we choose the RGB and CIELab color spaces.

Texture feature of the retinal image

Vessels have a characteristic texture. Texture can reflect the visual feature of the homogeneous phenomena in images. Texture feature does not rely on color or brightness and can reflect the property of the gray change, color distribution, and luminance distribution in images. Thus, we choose texture as the saliency feature. Given that the 2D Gabor filter can capture the local structure corresponding to the scale, spatial position, and direction choice, this filter is adopted to extract the texture feature. The function of Gabor filter is presented in the following equation¹³

\begin{matrix} g (x, y) = \frac{1}{2 π σ_{x}^{2} σ_{y}^{2}} exp (- \frac{1}{2} (\frac{x^{2}}{σ_{x}^{2}} + \frac{y^{2}}{σ_{y}^{2}}) \\ + 2 π j (x cos θ + y sin θ)) \end{matrix}

(3)

The Fourier transform is provided in the following equation

G (u, v) = exp [- \frac{1}{2} (\frac{{(u - W)}^{2}}{σ_{u}^{2}} + \frac{v^{2}}{σ_{v}^{2}})]

(4)

where

σ_{u} = \frac{1}{2 π σ_{x}}

σ_{v} = \frac{1}{2 π σ_{y}}

; W is the complex modulated frequency of the Gaussian function.

G (u, v)

is the Fourier transform of

g (x, y)

To effectively describe the texture information for distinction, the feature information must be accurately extracted. Thus, the design of Gabor filter is important. Specifically, its frequency must not overlap and the entire area must be considered. The position of Gabor filter is decided by orientation and scale. Given that vessels have different orientations, Gabor filter must also have different orientations with responses to 0°, 30°, 60°, 90°, 120°, and 150°. Small scale can be used to detect tiny vessels and large scale can be used to detect the main vessels. Thus, five different scales, 3, 5, 7, 9, 11 are employed. In this study, we use different orientations and scales for describing the texture of the green channel.

The filtered results using the above-mentioned five scales in each orientation are merged into one by taking the maximum of each scale. The maximum response is shown in Figure 3.

Figure 3.

Texture feature of the retinal images: (a) the response in 0°; (b) the response in 30°; (c) the response in 60°; (d) the response in 90°; (e) the response in 120°; (f) the response in 150°.

Color feature of the retinal image

The pixel saliency is the sum of color Gaussian distance between each sensing unit and other units in the neighborhood based on the contrast. Given the global and local features, the neighborhood can be adjusted according to the position of the pixel in the image. The equation can be obtained as follows^13–15

S (x, y) = exp (\frac{d}{σ^{2}}) \frac{1}{A} \sum_{(s, t) \in U} | I (x, y) - I (s, t) |

(5)

where U is the maximal rectangle neighborhood around the pixel (x, y); w and h are the width and length of the image, respectively. A is the area of U, and d is the ratio of A and the area of the image. d can be adjusted according to the distance between the pixel (x,y) and the center of the image. Hence, the farther the distance, the smaller the value. σ is the control parameter and can adjust the effect on the feature from d. The bigger the σ, the larger the effect of weights in the central space. σ² is 0.4 in this algorithm. The color feature images are shown in Figure 4.

Figure 4.

Color feature of the retinal images: (a) color feature in R channel; (b) color feature in G channel; (c) color feature in B channel; (d) color feature in L channel; (e) color feature in a channel; (f) color feature in b channel.

Fusion of saliency feature

The 2D Gabor transformation is determined through different orientations and scales. The saliency feature can be extracted after transformation. Image fusion is an important step in saliency detection. Information fusion methods are commonly used in saliency detection model and involve multiplication, average, square, maximum, and logarithm. Considering that the Gabor filter is multi-oriented, the filter orientation must be chosen appropriately. The maximum response corresponds to the vessel orientation. Thus, we choose the maximum in the same coordinate point of different Gabor transformations in synthesizing one image as the saliency feature image.¹³ The color of different channels is weighted according to the 2D entropy. The texture and color features are fused according to the 2D entropy,¹⁶ which is shown in Figure 5.

Figure 5.

Fusion of the saliency feature: (a) texture saliency feature; (b) color saliency feature; (c) fusion image.

Region optimization

The saliency image is gray and the scope value is limited. The contrast intensity of the saliency in gray image is not strong enough. The contrast can be enhanced through region optimization, which can result in the difference between high saliency value and low saliency value. After that, the higher saliency of the image will become higher and the lower saliency of the image will become lower.

The function of optimization is described as follows¹⁷

S_{final} (p) = {\begin{matrix} - k \cdot exp (δ (- S (p) - midval)) + midval - 1 \\ whereS (p) < midval \\ k \cdot exp (δ (S (p) - midval)) + midval - 1 \\ whereS (p) \geq midval \end{matrix}

(6)

where S(p) is the saliency value before processing; midval is found to be between 0 and 255; k and δ are the controlling factors that are confirmed through experiments. When 40 ≤ midval ≤ 120, δ is the log function. When 120 < midval ≤ 160, δ is the exponential function.

The saliency value S_final can be obtained through the piecewise function. The graph is shown in Figure 6. The region optimization of the saliency image is shown in Figure 7.

Figure 6.

Piecewise function.

Figure 7.

Region optimization of the saliency image.

Gray histogram thresholding segmentation based on areas

The saliency feature image must be segmented by the threshold. The choice of threshold will directly affect the result of the segmentation. 1D threshold segmentation has poor adaptability and is sensitive to noise, while 2D threshold segmentation has high computing complexity. To solve the above-mentioned problems, we adopt a novel threshold segmentation algorithm, which is adaptive and less distracted by noise. The novel algorithm is described as follows.

Improved 1D histogram algorithm based on areas

The histogram is computed according to the relation between f(m, n) and g(m, n).The algorithm adopts the mixture statistics in counting the histogram, which can be expressed as follows¹⁸

\begin{matrix} f_{mix} (m, n) = {\begin{matrix} f (m, n) & if | f (m, n) - g (m, n) \leq t_{1} | \\ g (m, n) & if | f (m, n) - g (m, n) \geq t_{1} | \end{matrix} \end{matrix}

(7)

where (m, n) denotes the pixel, f(m, n) denotes the pixel value, g(m, n) denotes the neighborhood of 5 × 5, and z is the gray level; hence, z = 0, 1, 2, … , L-1 (L = 256).

Thus, the constructing equation of the histogram is described as follows

h_{mix} (m, n) = \frac{1}{M * N} \sum_{m, n} δ (f_{mix} (m, n) = z)

(8)

where the pixels in the first area are the normal pixels and the weak edge pixels, the pixels in the second area are the noise pixels, and the pixels in the third area are the strong edge pixels. t₁ equals 40 in this algorithm. The area based on gray histogram is shown in Figure 8.

Figure 8.

The area based on gray histogram.

Threshold segmentation based on Otsu

After obtaining the new histogram, the Otsu segmentation method is adopted to obtain the saliency image segmentation. The segmentation result is shown in Figure 9.

Figure 9.

The segmentation image.

Repairing small vessels

Some small vessels can be lost after the segmentation based on the gray histogram thresholding method. To avoid the loss of small vessels, we adopt the following method: if R(m, n) equals 0 and one of the pixel values in eight neighbor fields is equal to 1, then the pixel (m,n) can be recognized as the vessel. This operation is equivalent to dilation. The result is shown in Figure 10.

Figure 10.

The retinal vessel segmentation image.

Experimental results and analysis

Data collection

The proposed algorithm is evaluated using DRIVE database and some clinical fundus images. DRIVE database is established from a diabetic retinopathy screening program in the Netherlands. About 400 diabetic subjects between 25 and 90 years were chosen. The retinal images with 565 × 584 resolution were captured using a Cannon CR5 nonmydriatic 3CDD camera with a 45° field of view. The site provides hand-labeled data from two graders, which can be used to evaluate the algorithm's performance.

The proposed algorithm is implemented in the operating system of Microsoft Windows 8, with intele31231v3 CPU and 32G memory. The development environment is Matlab2013.

Evaluation methodology

To evaluate the algorithm, we calculate the true positive (TP) ratio (TPR), false positive (FP) ratio (FPR), and accuracy. We assume that TP and true negative show the correct vessel pixels and background pixels, respectively. This assumption is consistent with the judgment of the ophthalmologist. Furthermore, FP and false negative show the incorrect vessel pixels and background pixels, respectively. This deduction is inconsistent with the judgment of the ophthalmologist. The evaluation equation is as follows

Sn = TP / (TP + FN)

(9)

Spec = TN / (FP + TN)

(10)

ACC = (TP + TN) / (TP + FP + TN + FN)

(11)

Results and analysis

All images were processed by the proposed algorithm. The vessel segmentation results are shown in Figure 11.

Figure 11.

Retinal vessel segmented result of DRIVE database: (a) original retinal image; (b) segmented results by Maji et al.²³; (c) segmented results by Wang et al.¹⁹; (d) segmented results by the proposed algorithm; (e) segmented results by the expert.

The proposed algorithm is compared with the existing retinal vessel segmentation algorithms and the manual labeling in the DRIVE database. In Figure 11, the first column shows the original image. The second column indicates the result reported by Maji et al.²³ The third column presents the findings of Wang et al.¹⁹ The fourth column shows the result of the proposed algorithm. The final column is the result of manual labeling. As shown in Figure 11, there are some discontinuous points existing in the results by Maji et al.²³ Some tiny vessels are lost in the results by Wang et al.¹⁹ and Maji et al.,²³ which is important for the analysis of fundus images. There are many tiny vessels in the results of the proposed algorithm and it is seen that the continuity of the vessel network is quite good.

The red areas in Figure 11 are the dark areas in the fundus images. The segmentation results are not very good and a plenty of tiny vessels are lost by adopting the algorithm by Maji et al.²³ But the dark areas can be adjusted by guided filtering and the clear vessel network can be obtained through the saliency model based on region optimization in this paper.

Table 1 shows the performance comparison of our method with other approaches in terms of Spec, Sn, and accuracy. As shown in the table, the performance of our method is better than some existing methods. But the accuracy of the proposed algorithm is lower than the result by the supervised algorithm. A 39-dimensional feature vector is extracted and the sampled set is first treated by the classification and regression tree as a weak classifier and then strengthened by a trained AdaBoost-based classifier as a strong classifier in Zhu et al.¹⁰ The algorithm in Fu et al.¹¹ is based on the deep learning. The training set is not needed in the proposed algorithm, which is better for clinical diagnosis.

Table 1.

Comparison of performance between the recent studies.

Method	Spec	Sn	ACC
Zhu et al.¹⁰	0.9607	0.7528	0.9820
Fu et al.¹¹	–	0.7603	0.9523
Wang et al.¹⁹	–	0.7527	0.9457
Maji et al.²³	–	–	0.9327
Our method	0.9861	0.7967	0.9465

Conclusion

Segmentation of retinal vessel plays an important role in the diagnostic procedure of retinopathy. Due to the difficulties in the retinal vessel segmentation, a novel method for automated vessel segmentation is proposed in this paper. Considering the improved segmentation, the retinal image contrast is enhanced by guided filtering. By applying the saliency feature combined with region optimization, the vessel in the image can be highlighted. Furthermore, the saliency image is segmented through the gray histogram thresholding segmentation based on areas. To avoid the loss of small vessels, we adopt morphological operation to repair small vessels. The experimental results show that the proposed method can accurately segment retinal vessels and effectively obtain their connectivity. It is suitable for computer-aided eye disease diagnosis and evaluation using fundus images.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of Fujian Province, China (No. 2016J0129), by the Educational Commission of Fujian Province of China (No. JAT160070).

References

Wong

. Is retinal photography useful in the measurement of stroke risk? Lancet Neurol 2004; 3: 179–183.

Goto

Kimoto

Katsuki

et al.

Pathological studies on the intracerebral and retinal arteries in cerebrovascular and non cerebrovascular diseases. Stroke 1975; 6: 263–269.

Ikram

Ong

Cheung

et al.

Retinal vascular caliber measurements: Clinical significance, current knowledge and future perspectives. Ophthalmologia 2013; 229: 125–136.

Srinidhi

Aparna

Rajan

. Recent advancements in retinal vessel segmentation. J Med Syst 2017; 41: 70.

Kar

Maity

. Blood vessel extraction and optic disc removal using curvelet transform and kernel fuzzy c-means. Comput Biol Med 2016; 70: 174–189.

Chaudhuri

Chatterjee

Katz

et al.

Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Trans Med Imag 1989; 8: 263–269.

Vese

Chan

. A multiphase level set framework for image segmentation using the Mumford and Shah model. Int J Comput Vision 2002; 50: 271–293.

Sum

Cheung

PYS

. Vessel extraction under non-uniform illumination: A level set approach. IEEE Trans Biomed Eng 2008; 55: 358–360.

Zhao

Wang

et al.

Retinal vessels segmentation based on level set and region growing. Pattern Recog 2014; 47: 2437–2446.

10.

Zhu

Zou

Cui

et al.

An ensemble retinal vessel segmentation based on supervised learning in Fundus Images. Chinese J Electron 2016; 25: 503–511.

11.

Fu HZ, Xu YW, Kee Wong DW, et al. Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. In: IEEE international symposium on biomedical imaging, Prague, Czech Republic, 2016, pp. 698–701.

12.

He CM, Sun J and Tang X. Guided image filtering. In: Proceedings of the 11th European conference on computer vision. Heraklion, Crete, Greece, 2010, pp. 1–14.

13.

Zhang Y-b, Chen Z, Zhang F, et al. Visual saliency detection based on color and texture features. Appl Res Comput 2015; 32: 284–287.

14.

Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, 2009, pp. 1597–1604. Miami, Florida, USA: CVPR.

15.

Achanta R and Susstrunk S. Saliency detection using maximum symmetric surround. In: Proceedings of the 17th IEEE international conference on image processing, 2010, pp. 2653–2656. Hong Kong: ICIP.

16.

Zhang

Y-b

Han

J-w

Guo

. Saliency detection by combining spatial and spectral information. Optic Lett 2013; 38: 1987–1989.

17.

Wang Y. The research of application and methods of the image visual saliency detection. Dissertation, South China University of Technology, PR China, 2015.

18.

Wang QW. Study on image histogram feature and application. Dissertation, University of Science and Technology of China, PR China, 2014.

19.

Wang

X-H

Zhao

Y-Q

Liao

et al.

Automatic segmentation for retinal vessel based on multi-scale 2D Gabor wavelet. Acta Automat Sin 2015; 41: 970–980.

20.

Lei

Fisher

Wang

. Retinal vessel segmentation using multi-scale textons derived from keypoints. Comput Med Imag Graph 2015; 45: 47–56.

21.

Kaba D, Salazar-Gonzalez AG, Li YM, et al. Segmentation of retinal blood vessel using Gaussian mixture modes and expectation maximization. In: Proceedings of the 2nd international conference on health information science. Berlin, Heidelberg: Springer, 2013, pp. 105–112.

22.

Wang

Lin

et al.

Retinal vessel segmentation using multi-wavelet kernels and multi-scale hierarchical decomposition. Pattern Recog 2013; 46: 703–715.

23.

Maji D, Santara A, Ghosh S, et al. Deep neural network and random forest hybrid architecture for learning to detect retinal vessels in Fundus images. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milano, Italy, 25–29 August 2015.