The detections of retinopathy symptoms and tractional retinal detachment

Abstract

Since the symptoms of the diabetic retinopathy, macular degeneration, and many other eye diseases are often associated with the bright spots, dark spots, and vascular abnormalities in retinal images, this study proposes a retinopathy image segmentation (system) to extract the bright spots, dark spots, and blood vessels from retinal images. Moreover, since a high proportion of diabetic patients suffer from the diabetic retinopathy with the most severe one to be the retinal detachment, the retinal image-based tractional retinal detachment diagnosis system is proposed, as well. Two genetic-based parameter detectors are given to provide the most appropriate values for the parameters used by the retinopathy image segmentation system and the tractional retinal detachment diagnosis system. The experimental results illustrate that the retinopathy image segmentation system and the tractional retinal detachment diagnosis system can provide expressive results in segmenting bright spots, dark spots, blood vessels, and in diagnosing tractional retinal detachment. Both systems are helpful for the ophthalmologists in diagnosing eye diseases.

Keywords

Retinal image image segmentation tractional retinal detachment genetic algorithm

Introduction

The World Health Organization (WHO)¹ reported in 2004 that approximately 285 million people worldwide are visually impaired, 39 million of whom are blind and 246 million have low vision. Therefore, vision problems have become a crucial health concern worldwide. The WHO also reported that diabetic retinopathy and age-related macular degeneration are critical diseases resulting in vision loss.²

Many eye diseases are chronic conditions such as macular degeneration, glaucoma, and diabetic retinopathy; however, the patients are frequently unaware of their chronic eye diseases although they have already lost some vision. Early diagnosis of the abnormalities can decrease the probability of visual loss. For example, a high proportion of diabetic patients develop retinopathy, which may cause visual impairment or blindness, substantially influencing patients’ quality of life. Therefore, if diabetic patients with retinopathy can be diagnosed at an early stage, the risk of blindness can be reduced.

Because endocrinologists who treat diabetes possess less knowledge of eye diseases than ophthalmologists, consultations for diabetic patients with ophthalmologists are typically arranged through hospital referrals or document deliveries. However, this process is time-consuming and expensive because of complex procedures and human factors, which may delay treatment and exacerbate the disease.

Ophthalmologists typically interpret retinal images; however, image interpretation may contain errors caused by human influence, resulting from dissimilarities in the diagnosis experience of ophthalmologists. Moreover, the prevalence of retinopathy increases year by year, increasing the number of patients who receive fundus photography. Ophthalmologists require a large amount of time to interpret the vast amount of retinal image data, causing workload increase.

Therefore, it is necessary to develop an image interpretation system to automatically recognize the symptoms of retinopathy. When the retinal abnormality of a patient is detected, the patient must receive treatment to control disease progression to reduce the incidence of blindness. Thus, establishing an automatic interpretation system for symptom recognition is important.

In a retinopathy image, the white or bright yellow area signifies the following symptoms:

Hard exudates, which are composed of the deposition of protein and lipid materials, marked by the arrow in Figure 1(a).

Cotton wool spots, which are gray or bright spots similar to cotton wool or fluff with irregular shapes and unclear boundaries, denoted by arrow 1 in Figure 1(b).

Figure 1.

Symptoms of retinopathy: (a) hard exudates, (b) cotton wool spots and capillary hemangioma.

In addition, the dark red areas indicate the following symptoms:

Capillary hemangiomata, appearing as tiny bumps with colors ranging from red to purple.

Hemorrhages, expressed as small petechiae with circular or irregular shapes, denoted by arrow 2 in Figure 1(b).

Microvascular abnormalities or hyperplasia: microvessels are distorted or exhibit uneven thickness (Figure 2).

Figure 2.

Microvascular abnormalities or hyperplasia.

Tractional retinal detachment occurs when scar tissue on the retina’s surface contracts and causes the retina to pull away from the retinal pigment epithelium. Diabetes can lead to issues with the retinal vascular system and cause scar tissue in the eye that could cause detachment. Tractional retinal detachment may cause blindness in serious cases (Figure 3).

Figure 3.

Tractional retinal detachment.

However, the retinal images obtained from fundus photography frequently exhibit uneven brightness, as shown in Figure 4(a). The area denoted by the arrow exhibits noise interference, which may influence the diagnosis process and results. Additionally, images may be overly blurry and fail to provide a clear symptom boundary. Therefore, the areas denoting specific symptoms in an image cannot be accurately segmented for further recognition (Figure 4(b)). When an image is captured, lighting can be adjusted to produce images with varying brightness; thus, retinal images even for the same symptom and the same patient may differ in color and brightness. For example, part of the image in Figure 4(c) is brighter than optic disk and part of the image in Figure 4(d) is very dark. Therefore, segmenting the optic disk from a retinopathy image is difficult.

Figure 4.

Retinal images: (a) brightness of image is uneven, (b) the optic disk is indistinct, (c) part of the image is very bright, and (d) part of the image is very dark.

After a retinal image is converted into a gray-level image, the symptoms visible in the gray-level image exhibit varying levels of brightness. For example, hard exudates and cotton wool spots are brighter than capillary hemangiomata, hemorrhages, and blood vessels. Therefore, the areas of bright spots, dark spots, or vessels in the image should be targeted and segmented for image detection to facilitate physicians’ symptom recognition.

This study proposes a retinopathy image segmentation (RIS) system to correctly segment the regions of bright spots, dark spots, and vessels from a retinal image. The RIS system can precisely segment the regions of bright spots, dark spots, and vessels from a retinal image. It is indifferent to noise in the image. This study also proposes a retinal-image-based tractional retinal detachment diagnosis (TRDD) system to automatically diagnose tractional retinal detachment in patients.

RIS system

One of the aims in this study is to segment the bright spots, dark spots, and blood vessels from retinal images helping physicians identify the symptoms of retinal abnormalities. Hence, the bright spots, dark spots, and blood vessels are the regions of interest (ROI) which the RIS system expects to segment. The RIS system comprises three stages:

Preprocessing: extract the retinal area from a retinal image;

Bright-spot segmentation: segment the bright spots from the segmented retinal area;

Dark-spot and vessel segmentation: segment the dark regions from the segmented retinal area (including dark spots and blood vessels), and then separate dark spots and blood vessels in the segmented dark regions.

Image preprocessing

First, the RIS system transforms a color retinal image I into a gray-level image I_g as follows

I_{g} (x, y) = 0.299 \times I_{R} (x, y) + 0.587 \times I_{G} (x, y) + 0.114 \times I_{B} (x, y)

(1)

where I_R (x, y), I_G (x, y), and I_B (x, y) are the red, green, and blue (RGB) color components of image I, respectively.³Figure 5(b) is the gray-level image of that in Figure 5(a).

Figure 5.

(a) Original image, (b) gray-level image, (c) result obtained by Otsu’s thresholding method, and (d) result obtained by opening operation.

Because the background of a retinopathy image does not encompass ROI—bright spots, dark spots, and blood vessels, the RIS system then removes the background of the retinal image I_g. In the RIS system, Otsu’s⁴ thresholding method is applied to I_g to derive the optimal threshold T^* and generate a binary image I_b by

I_{b} (x, y) = {\begin{matrix} 0, I_{g} (x, y) < T^{*} \\ 1, I_{g} (x, y) \geq T^{*} \end{matrix}

(2)

where 0 and 1 denote black and white pixels in I_b, respectively. The image in Figure 5(b) is then converted into a binary image shown in Figure 5(c).

The biggest white region in I_b (see Figure 5(c)) is considered to be the retinal region. Since a retinal image typically contains texts, the binary image I_b may include undesired bright text spots. Therefore, a structured element with 3 × 3 1-bit pixels is used, and opening operation (including two erosion operations and then two dilation operations)⁵ is conducted on I_b to remove the text spots and the black holes in the retinal region, as shown in Figure 5(d). The image in Figure 5(d) is called a mask image I_m.

On a retinal color image, RGB three-color components are, respectively, considered to three gray-level images, namely, I_R, I_G, and I_B (i.e. Figure 6), among which I_G exhibits the largest contrast in bright spots, dark spots, blood vessels, and others in the retinal region. Hence, the RIS system intends to extract bright spots, dark spots, and blood vessels from I_G.

Figure 6.

A (a) color image and its (b) I_R, (c) I_G, and (d) I_B.

To facilitate the segmentation of the bright spots, dark spots, and blood vessels from I_G, the RIS system stretches the contrast of the retinal region in I_G to the range of 0 to 255 and creates a new gray-level image $I_{G}^{'}$ by

I_{G}^{'} (x, y) = {\begin{matrix} \frac{I_{G} (x, y) - Mi n_{G}}{Ma x_{G} - Mi n_{G}} \times 255, & if I_{m} (x, y) = 1 \\ 0, & otherwise \end{matrix}

(3)

where Min_G and Max_G are the minimal and maximal gray levels of the retinal region in I_G. Figure 7(a) shows the $I_{G}^{'}$ of I_G in Figure 6(c).

Figure 7.

(a) The $I_{G}^{'}$ of I_G in Figure 6(c), (b) the extracted bright spots with n_p = 81 and k_bp = 2, and (c) the extracted bright spots after removed noise with T_bA = 5.

Bright-spot segmentation

This subsection describes the bright-spot segmentation in detail from the retinal region in $I_{G}^{'}$ . The bright-spot segmentation stage comprises three steps: positive cross filtering (PCF), bright-spot segmentation, and noise elimination. The RIS system uses PCF to highlight the bright areas and suppress the dark areas in $I_{G}^{'}$ , then extracts the bright spots and eliminates noise from the segmented bright spots.

A retinal image may be taken with various lighting angles which may result in uneven brightness on the image. For example, in Figure 7(a), some bright spots are located in the darker area while others are located in the brighter area. Only using a global threshold cannot be effectively used to extract the bright spots from the image. In this study, PCF is proposed to solve this problem.

Let S_p be a set consisting of 2 × n_p − 1 pixels $I_{G}^{'} (x, y)$ , $I_{G}^{'} (x + 1, y)$ , $I_{G}^{'} (x - 1, y)$ , $I_{G}^{'} (x + 2, y)$ , $I_{G}^{'} (x - 2, y)$ , …, $I_{G}^{'} (x + ⌊ n_{p} / 2 ⌋, y)$ , $I_{G}^{'} (x - ⌊ n_{p} / 2 ⌋, y)$ , $I_{G}^{'} (x, y + 1)$ , $I_{G}^{'} (x, y - 1)$ , $I_{G}^{'} (x, y + 2)$ , $I_{G}^{'} (x, y - 2)$ , …, $I_{G}^{'} (x, y + ⌊ n_{p} / 2 ⌋)$ , $I_{G}^{'} (x, y - ⌊ n_{p} / 2 ⌋)$ Also let m_p be the average gray level of the pixels in S_p. To highlight the bright spots, PCF converts $I_{G}^{'}$ into a new gray-level image I_P as follows

I_{P} (x, y) = {\begin{matrix} {I^{'}}_{G} (x, y) - m_{p}, & if {I^{'}}_{G} (x, y) > m_{p} \\ 0, & otherwise \end{matrix}

(4)

The gray levels of I_P are then stretched to the range of 0–255. After that, the RIS system creates a binary image I_bp by

I_{b p} (x, y) = {\begin{matrix} 1, i f I_{P} (x, y) > T_{b p}, \\ 0, otherwise \end{matrix} and T_{b p} = μ_{b p} + k_{b p} \times σ_{b p}

(5)

where k_bp is a given constant; µ_bp and σ_bp are the average and standard deviation of the gray levels in I_P. The 1-bits in I_bp indicate that their related pixels in I_P and $I_{G}^{'}$ are in bright spots.

There may be noise (indicated by red arrows) in Figure 7(b) displaying the bright spots segmented from Figure 7(b) with n_p = 81 and k_bp = 2. Assume A_i is the area of the ith bright spot in I_bp, G_i is the average gray level of the spot in I_P, and $\bar{G}$ is the average gray level of all the bright spots in I_P. If $(A_{i} \leq T_{bA})$ and $(G_{i} \leq \bar{G})$ , the RIS system considers the bright spot as noise and removes it, where T_bA is a given threshold. Figure 7(c) demonstrates the result of bright spots segmented from Figure 7(b) with T_bA = 5 after removing noise.

Dark-spot and vessel segmentation

This subsection explicates the segmentation of dark spots and vessels from the retinal region of $I_{G}^{'}$ . This stage provides negative cross filtering (NCF) to strengthen the low gray-level areas in the retinal region of $I_{G}^{'}$ , and uses run-length enhancement to prevent the contours of dark spots and vessels from being fragmented. After that, it segments dark regions from the retinal region of $I_{G}^{'}$ and then separates vessels from the segmented dark regions.

In this stage, a 3 × 3 mean filter⁵ is first used to eliminate noise in $I_{G}^{'}$ . Figure 8(a) displays the image obtained by moving noise from the image in Figure 7(a). Next, in this study, a NCF is presented to enhance the dark-spot areas on image $I_{G}^{'}$ . Similarly, let S_n be a set consisting of 2 × n_p − 1 pixels $I_{G}^{'} (x, y)$ , $I_{G}^{'} (x + 1, y)$ , $I_{G}^{'} (x - 1, y)$ , $I_{G}^{'} (x + 2, y)$ , $I_{G}^{'} (x - 2, y)$ , …, $I_{G}^{'} (x + ⌊ n_{p} / 2 ⌋, y)$ , $I_{G}^{'} (x - ⌊ n_{p} / 2 ⌋, y)$ , $I_{G}^{'} (x, y + 1)$ , $I_{G}^{'} (x, y - 1)$ , $I_{G}^{'} (x, y + 2)$ , $I_{G}^{'} (x, y - 2)$ , …, $I_{G}^{'} (x, y + ⌊ n_{p} / 2 ⌋)$ , $I_{G}^{'} (x, y - ⌊ n_{p} / 2 ⌋)$ . Also let m_p be the average gray level of the pixels in S_p. To highlight the dark spots, NCF converts $I_{G}^{'}$ into a new gray-level image I_N as follows

I_{N} (x, y) = {\begin{matrix} m_{p} - I_{G}^{'} (x, y), & if I_{G}^{'} (x, y) < m_{p} \\ 0, & otherwise \end{matrix}

(6)

Figure 8.

The processing procedure of dark area segmentation: (a) the image $I_{G}^{'}$ after moving noise from the image in Figure 7(a), (b) the image obtained by NCF on the image in Figure 8(a) with n_n = 81, (c) the image I_R obtained by RLE method on the image in Figure 8(b) with n_R = 21, and (d) the dark regions in $I_{G}^{'}$ indicated in I_bd.

The gray levels of I_N are then stretched to the range of 0–255. Figure 8(b) demonstrates the image I_N obtained by NCF on the image in Figure 8(a) with n_n = 81.

Some noise may appear and the contours of blood vessels may be disconnected in image I_N (see the image in Figure 8(c)). The RIS system uses run-length enhancer⁶ to solve above problems. Let W_R be an n_R × n_R window in I_N and I_N (x, y) the central pixel of W_R. Assume that L_i is a line segment on W_R and passes through I_N (x, y), P_h₁, and P_h₂, and the included angle between L_i and X-axis is 22.5 × h° for h = 0, 1, …, 7, where P_h₁ and P_h₂ are two pixels located on the boundary of W_R.

Figure 9 shows the W_R and eight line segments L₀, L₁, …, L₇. If the gray level of I_N (x, y) is less than the average gray level of all the pixels in W_R, the run-length enhancer assigns ${MIN}_{h = 0}^{7} ((1 / n_{R}) \sum_{i = 1}^{n_{R}} c_{hi})$ to the gray level of I_R (x, y); else it sets the gray level of I_R (x, y) to ${MAX}_{h = 0}^{7} ((1 / n_{R}) \sum_{i = 1}^{n_{R}} c_{hi})$ , where c_hi is the gray level of the ith pixel on L_h. Then, the gray level of I_R is stretched to the range of 0–255. Figure 8(c) illustrates the result obtained by running the run-length enhancer on the image in Figure 8(b).

Figure 9.

The W_R and eight line segments L₀, L₁, …, L₇.

After that, the RIS system generates a binary image I_bd by

I_{b d} (x, y) = {\begin{matrix} 1, if I_{R} (x, y) > T_{b d}, \\ 0, otherwise \end{matrix} and T_{b d} = μ_{b d} + k_{b d} \times σ_{b d}

(7)

where k_bd is a given constant; µ_bd and σ_bd are the average and standard deviation of the gray levels in I_R. The 1-bits in I_bd indicate that their related pixels in I_R and $I_{G}^{'}$ are in dark spots. The RIS system considers the region with area less than T_ad as noise and removes it. Figure 8(d) shows the I_bd obtained from the image in Figure 8(c) with k_bd = 2 and T_ad = 10.

In image I_bd, the bright regions indicate the dark spots and blood vessels in $I_{G}^{'}$ . In $I_{G}^{'}$ , dark spots typically appear as irregularly shaped blocky areas, and blood vessels appear as stripes. Let D_i be the longest distance between any two pixels on the contour of the ith bright region R_i in $I_{G}^{'}$ , $A_{i}^{'} = π (D_{i} / 2)^{2}$ , and A_i the area of R_i. R_i is regarded as a blood vessel if $(A_{i} / A_{i}^{'}) < T_{v}$ , and otherwise R_i is considered to be a dark spot, where T_v is a given threshold. Figure 10 displays the results of dark spots and blood vessels segmented from image I_bd in Figure 8(d) where T_v = 0.3.

Figure 10.

The segmentation results of dark spots and vessels from image I_bd in Figure 8(d) with T_v = 0.3: (a) the dark parts segmented and (b) the vessels segmented.

Genetic-based parameter detector–retinopathy image segmentation method

In this subsection, a genetic algorithm, called the genetic-based parameter detector–retinopathy image segmentation (GBPD-RIS) method, is used to determine the appropriate values of the parameters used in the RIS system. The segmentation results of the RIS system are considerably influenced by the parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v, n_p, k_bp, and T_bA which affect the effectiveness of bright-spot segmentation; n_n, n_R, k_bd, T_ad, and T_v decide the segmentation results of dark spots and blood vessels. The GBPD-RIS system will employ accumulated historical data to derive the most suitable values of parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v.

In the accumulated historical data, ground truth are the contours of bright spots, dark spots, and vessels manually drawn by ophthalmologists. Precision and Recall scores⁷ are often used to measure the image segmentation results. True positive (TP) is the number of pixels which are considered to the pixels in ROI by the segmentation method and the ground truth. False positive (FP) is the number of pixels which are considered to the pixels in ROI by the segmentation method but not in the ground truth. False negative (FN) is the number of pixels which are considered to the pixels in ROI by the segmentation method but it is not. Precision and recall are then defined as: $Precision = (TP / TP + FP)$ , and $Recall = (TP / TP + FN)$ . F-score (F) combining precision and recall into one single measure is the harmonic mean of precision and recall: $F = (2 \times precision \times recall) / (precision + recall)$ .

In the GBPD-RIS method, each chromosome is divided into eight substrings s_p, s_bp, s_bA, s_n, s_R, s_bd, s_ad, and s_v, respectively, describing the values of parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v. When given a chromosome Ch, the values of the parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v depicted by Ch are defined as

n_{p} = 3 + 2 \times v_{p}

k_{bp} = 0.1 \times v_{bp}

T_{bA} = 1 \times v_{bA}

n_{n} = 3 + 2 \times v_{n}

n_{R} = 3 + 2 \times v_{R}

k_{bd} = 0.1 \times v_{R}

T_{ad} = 1 + 1 \times v_{R}

T_{v} = 0.1 \times v_{v}

Then, the values of the parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v are used by the RIS system to segment the bright spots, dark spots, and blood vessels based on the accumulated historical data, and the obtained F-score of the segmentation is regarded as the fitness of Ch.

Initially, the GBPD-RIS method randomly generates N_g chromosomes, called initial chromosomes. Subsequently, the GBPD-RIS method alternatively performs mutation, crossover, and selection operations to evolve the optimal solution. In the mutation operation, a new chromosome is created for each initial chromosome Ch by randomly selecting a bit b from each substring. The selected bit b is then replaced by ¬b, where ¬ represents the bitwise operator NOT.

In the crossover operation, N_g pairs of chromosomes are randomly selected from initial chromosomes. Let n_g be the number of bits in a chromosome and M_g a binary string with n_g bits. For each selected chromosome pair Ch₁ and Ch₂, the GBPD-RIS method randomly selects $⌈ (n_{g} / 2) ⌉$ bits from M_g and sets the selected bits to 1-bits and others to 0-bits in M_g. The GBPD-RIS method then creates a new chromosome Ch by $Ch = (C h_{1} \land M_{g}) \lor (C h_{2} \land \neg M_{g})$ , where ∧ stands for the bitwise operator AND, and ∨ denotes the bitwise operator OR.

In the selection operation, the GBPD-RIS method selects N_g chromosomes with best fitness as new initial chromosomes for next run from N_g initial chromosomes, N_g chromosomes created by mutation operation, and N_g chromosomes generated by crossover operation. The GBPD-RIS method repeatedly alternatively executes mutation, crossover, and selection operations until the fitness of the reserved initial chromosomes is very closed or the number of runs is equal to a given maximal number MAX_NO_RUN of runs.

TRDD system

The second purpose of this study is to develop a TRDD system to automatically diagnose tractional retinal detachment disease based on patient’s retinal image. The TRDD system contains three steps: gray-level co-occurrence matrix (GLCM) analysis, bilayer feature matching (BFM), and GBPD-TRDD method. In the GLCM analysis step, the red and green components of the retinal image (i.e. the mages in Figure 11) are used to create co-occurrence matrixes. BFM is then conducted. Specifically, for each retinal image, GLCM is employed to calculate four features: contrast, dissimilarity, correlation, and inverse difference moment. To increase the recognition rate of the TRDD system, K-means clustering⁸ is conducted repeatedly to categorize an image set into small classes. The number of small classes will be determined by a genetic-based algorithm, which is also adopted to determine the most suitable parameter values for increasing the recognition rate of the TRDD system.

Figure 11.

The red, green, and blue color component images of a retinal image: (a) original color image, (b) red color component image I_R, (c) green color component image I_G, and (d) blue color component image I_B.

GLCM analysis

Co-occurrence matrix (GLCM) is a frequently used texture analysis method in image processing and pattern recognition.⁹ This study will take it to describe the characteristics of a gray-level image I₀. A co-occurrence matrix is the conditional joint probabilities of all pair-wise combinations of the gray levels in I₀ for given two parameters inter-pixel distance (δ) and orientation (θ). The co-occurrence matrix can be defined as: M_c (i, j) = {P (i, j) | (δ, θ)}, where P (i, j) (the co-occurrence probability between gray levels i and j) is defined as

P (i, j) = \frac{C_{ij}}{\sum_{k = 1}^{G} C_{ik}}

(8)

where C_ij is the occurrences of gray levels i and j in I₀ based on a certain (δ, θ) pair; and G is the quantized number of gray levels. The sum in the denominator thus represents the total number of gray-level pairs (i, j) within I₀. The four statistics in Table 1 are often applied to describe the texture features of an image based on the co-occurrence probabilities.¹⁰

Table 1.

Co-occurrence texture statistics.

Feature	Formula
Contrast	$\sum_{i = 0}^{255} (P (i, j) \times {(j - i)}^{2})$
Dissimilarity	$\sum_{i = 0}^{255} (P (i, j) \times \| j - i \|)$
Correlation	$\sum_{i = 0}^{255} (P (i, j) \times (i - μ_{x}) \times (i - μ_{y})) / (σ_{x} σ_{y})$
Inverse difference moment	$\sum_{i = 0}^{255} P (i, j) / (1 + {(j - i)}^{2})$

For a color retinal image I_RGB, the red and green color components can clearly indicate the symptoms of retinal detachment (see Figure 11). Let I_R and I_G be the images formed, respectively, by the red and green color components of I_RGB. The TRDD system will apply the co-occurrence statistics of I_R and I_G to describe the texture characteristics of I_RGB and then diagnoses the tractional retinal detachment disease based on the extracted co-occurrence texture statistics with δ = 1, and θ = 0°.

BFM

The TRDD system divides the accumulated historical retinal images into two sets: one consisting of the retinal images taken from the patients with tractional retinal detachment, and the other containing the retinal images taken from healthy normal persons. Let S_d be the set comprising the retinal images taken from the patients with tractional retinal detachment and S_n be the set containing the retinal images taken from healthy normal persons. Also let f_c, f_d, f_r, and f_i are the contrast, dissimilarity, correlation, and inverse difference moment features of an image in a retinal image set, and c_c, c_d, c_r, and c_i be, respectively, the averages of f_c, f_d, f_r, and f_i of all the images in the set. The c_c, c_d, c_r, and c_i are generally employed to depict the characteristics of the images in the set. We call (c_c, c_d, c_r, and c_i) the class centroid of the image set. If the content of the images in a set is quite different, c_c, c_d, c_r, and c_i cannot precisely depict the characteristics of the images in the set. This study hence provides BFM method to solve this problem.

Figure 12(a) shows two data sets S₁ and S₂. C₁ (resp. C₂) is the class centroid of S₁ (resp. S₂). Assume there is a quite difference in within-class variances between S₁ and S₂; a datum X is in set S₁ but far away from class centroid C₁, and the within-class variance of S₁ is much greater than the within-class variance of S₂. X will be misclassified into S₂ since the distance between X and C₁ is much longer than the distance between X and C₂.

Figure 12.

Example for the bilayer feature matching method: (a) an input data x is exactly around the boundary of two classes C₁ and C₂, and (b) two classes C₁ and C₂, and their subclasses.

To deal with the problem mentioned above, the BFM method categorizes all the data in one set into some small classes. When given a datum X, the BFM method calculates the distance between X and each small class. If the distance d_ci of X and small class c is minimal, the BFM method considers that X is in the set which class c belong to. In Figure 12(b), S₁ is partitioned into four small classes, the class centroids of which are C₁₁, C₁₂, C₁₃, and C₁₄; S₁ is separated into two small classes, the class centroids of which are C₂₁ and C₂₂. Since X is most closed to C₁₂, X is in S₁.

K-means clustering algorithm⁸ is one of the commonly used clustering methods. The BFM method applies K-means clustering algorithm to classify the images in S_d into n_d small classes and the images in S_n into n_n small classes according to their contrast, dissimilarity, correlation, and inverse difference moment features. Let C_nc = (c_ncc, c_ncd, c_ncr, c_nci) be, respectively, the averages of the contrast, dissimilarity, correlation, and inverse difference moment features of all the images in small class c of S_n, and C_dc = (c_dcc, c_dcd, c_dcr, c_dci) be the averages of the contrast, dissimilarity, correlation, and inverse difference moment features of all the images in small class c of S_d. When given a retinal image with the features X = (f_c, f_d, f_r, f_i) of contrast, dissimilarity, correlation, and inverse difference moment, the BFM method assigns X to set h, where

h = ARG (\underset{s = {n, d}}{MIN} ({MIN}_{c = 1}^{n_{s}} (d (X, C_{sc}))))

(9)

where n_s is the number of small classes in set S_s.

In the K-means clustering algorithm and the BFM method, the distance d (X, C_sc) between data X and class centroid C_sc is defined as

d (X, C_{sc}) = \sum_{k = {c, d, r . i}} (w_{k} \times {| f_{k} - c_{sck} |}^{r_{k}})

(10)

where w_k and r_k are the given constants.

GBPD-TRDD method

The performance of the TRDD system is considerably affected by the parameters n_n, n_d, w_c, w_d, w_r, w_i, r_c, r_d, r_r, and r_i. In this study, a genetic-based parameter detector (GBPD-TRDD method) is proposed to determine the optimal parameters which are used in the TRDD system. Similarly to the GBPD-RIS method, in the GBPD-TRDD method, each chromosome is divided into 10 substrings $s_{n_{n}}$ , $s_{n_{d}}$ , $s_{w_{c}}$ , $s_{w_{d}}$ , $s_{w_{r}}$ , $s_{w_{i}}$ , $s_{r_{c}}$ , $s_{r_{d}}$ , $s_{r_{r}}$ , and $s_{r_{i}}$ which, respectively, describes the values of the parameters n_n, n_d, w_c, w_d, w_r, w_i, r_c, r_d, r_r, and r_i. When given a chromosome Ch, the values of the parameters depicted by the chromosome are defined as

\begin{matrix} n_{n} = 1 + v_{n_{n}} \\ n_{d} = 1 + v_{n_{d}} \\ w_{c} = \frac{v_{w_{c}}}{v_{w_{c}} + v_{w_{d}} + v_{w_{r}} + v_{w_{i}}} \\ w_{d} = \frac{v_{w_{d}}}{v_{w_{c}} + v_{w_{d}} + v_{w_{r}} + v_{w_{i}}} \\ w_{r} = \frac{v_{w_{r}}}{v_{w_{c}} + v_{w_{d}} + v_{w_{r}} + v_{w_{i}}} \\ w_{i} = \frac{v_{w_{i}}}{v_{w_{c}} + v_{w_{d}} + v_{w_{r}} + v_{w_{i}}} \\ r_{c} = v_{r_{c}} \times 0.05 \\ r_{d} = v_{r_{d}} \times 0.05 \\ r_{r} = v_{r_{r}} \times 0.05 \\ r_{i} = v_{r_{i}} \times 0.05 \end{matrix}

(11)

where $v_{n_{n}}$ , $v_{n_{d}}$ , $v_{w_{c}}$ , $v_{w_{d}}$ , $v_{w_{r}}$ , $v_{w_{i}}$ , $v_{r_{c}}$ , $v_{r_{d}}$ , $v_{r_{r}}$ , and $v_{r_{i}}$ are the values described by $s_{n_{n}}$ , $s_{n_{d}}$ , $s_{w_{c}}$ , $s_{w_{d}}$ , $s_{w_{r}}$ , $s_{w_{i}}$ , $s_{r_{c}}$ , $s_{r_{d}}$ , $s_{r_{r}}$ , and $s_{r_{i}}$ . Then, the values of the parameters n_n, n_d, w_c, w_d, w_r, w_i, r_c, r_d, r_r, and r_i described by Ch are used by the TRDD system to diagnose tractional retinal detachment based on the accumulated historical data. The obtained recognition rate (the ratio of the number of images which is correctly recognized by a system to the total number of test images) is used as the fitness of Ch. After that the GBPD-TRDD method repeatedly alternatively executes the mutation, crossover, and selection operations which are the same as those adopted by the GBPD-RIS method until the requirement is satisfied.

Experimental results and discussions

This section is to investigate the performances of the RIS system and the TRDD system by experiments based on the images downloaded from structured analysis of the retina (STARE) retinal image database.¹¹ The STARE database provides the symptom locations marked by professional physicians. Moreover, the retinal images in the database were all taken by the TopCon TRV-50 fundus camera with a 35° field of view. Each image consists of 605 × 700 pixels with eight bits per RGB channel.¹¹

The performances of the RIS system

Experiment 1 is to derive the optimal values for the parameters n_p, k_bp, T_bA, n_n, n_R, k_bd, T_ad, and T_v used by the RIS system via the GBPD-RIS method. In the following experiments in this subsection, 60 images containing bright spots and dark spots downloaded from the STARE database are used as test images. In the first experiment, each of s_p, s_n, and s_R is described by 10 bits, and each of s_bp, s_bA, s_bd, s_ad, and s_v is described by 15 bits. Table 2 shows the most suitable values of the parameters obtained in this experiment. These values will be used by each following experiment in this subsection.

Table 2.

The most suitable values of the parameters used by the retinopathy image segmentation system.

Parameter	n_p	k_bp	T_bA	n_n	n_R	k_bd	T_ad	T_v
Values	81	2	5	81	21	2	10	0.3

In experiment 2, the RIS system and Eddins’¹² watershed method are used to segment the bright and dark spots. Tables 3 and 4 demonstrate the experimental results. Recall, precision, and F-score are used to explain the effectiveness of the segmentation methods. Table 3 explains that the RIS system provides a lower recall than the watershed method does. However, the RIS system gives much higher precision and F-score than the watershed method does. Figure 13 shows the segmentation results for some test images. From Figure 13, one can obviously observe that Eddins’ watershed method loses much more areas of bright spots while the RIS system give a little quantity of bright-spot areas which are not in the round truth. Table 4 explains that the RIS system gets far higher recall, precision, and F-score than Eddins’ watershed method does. Figure 14 shows the segmentation results for some test images. Figure 14 displays that the RIS system is much superior to Eddins’ watershed method in extracting dark spots from a retinal image.

Table 3.

Bright-spot segmentation results.

Method	Precision	Recall	F-score
RIS system	81.68	76.98	79.26
Eddins	21.99	89.95	35.34

RIS: retinopathy image segmentation.

Table 4.

Dark-spot segmentation results.

Method	Precision	Recall	F-score
RIS system	55.60	64.75	57.52
Eddins	0.44	5.56	0.77

RIS: retinopathy image segmentation.

Figure 13.

Bright-spot segmentation: (a) and (b) the ground truth on two image; (c) and (d) the results obtained by the retinopathy image segmentation system; (e) and (f) the results obtained by Eddins’ watershed method.

Figure 14.

Dark-spot segmentation: (a) and (b) the ground truth on two image; (c) and (d) the results obtained by the retinopathy image segmentation system; (e) and (f) the results obtained by Eddins’ watershed method.

Experiment 3 is to extract blood vessels from retinal images by the RIS system, Mendonca’s method,¹³ and Hoovers’ method.¹⁴Table 5 exhibits the experimental results and Figure 15 shows the segmentation results for some test images. The effect of blood vessel segmentation achieved by the RIS system is better than those obtained by Mendoncas’ method and Hoovers’ method.

Table 5.

Blood vessel segmentation results.

Method	Precision	Recall	F-score
RIS system	89.50	93.51	91.46107
Mendonca	73.99	94.79	83.10833
Hoover	71.23	92.69	80.55526

RIS: retinopathy image segmentation.

Figure 15.

Blood vessel segmentation: (a) and (b) the ground truth on two image; (c) and (d) the results obtained by the retinopathy image segmentation system; (e) and (f) the results obtained by Mendonca’s method; (g) and (h) the results obtained by Hoovers’ method.

The performances of the TRDD system

Next experiment is to derive the optimal values for the parameters n_n, n_d, w_c, w_d, w_r, w_i, r_c, r_d, r_r, and r_i used by the TRDD system via the GBPD-TRDD method. In the experiments in this subsection, 53 retinal images downloaded from STARE database were used, including 23 images taken from healthy normal persons and 30 images taken from the patients with tractional retinal detachment are used as test images. In this experiment, each of the substrings $s_{n_{n}}$ and $s_{n_{d}}$ is respected by four bits in a chromosome, and each of the substrings $s_{w_{d}}$ , $s_{w_{r}}$ , $s_{w_{i}}$ , $s_{r_{c}}$ , $s_{r_{d}}$ , $s_{r_{r}}$ , and $s_{r_{i}}$ is respected by 10 bits in a chromosome. Table 6 demonstrates the most suitable values of the parameters obtained in this experiment. These values will be used by the following experiment in this subsection.

Table 6.

The optimal values of the parameters used by the tractional retinal detachment diagnosis system.

n_n	n_d	w_c	w_d	w_r	w_i	r_c	r_d	r_r	r_i
2	2	0.21	0.32	0.26	0.21	3.60	2.35	1.65	3.10

The last experiment is to investigate the performance of the TRDD system based on the 53 test images. The experimental results indicate that the TRDD system achieves an accurate rate of 92% in diagnosing tractional retinal detachment.

Conclusion

The patients suffering from hard exudates and cotton wool spots usually have common symptom bright spots in their retinopathy images, and the ones suffering from capillary hemangiomata, hemorrhages, and vascular abnormality usually have common symptom dark spots in their retinopathy images, and they may lead to vascular malformations. This study hence proposes the RIS system to segment the bright spots, dark spots, and blood vessels in retinal images. Additionally, retinal detachment caused by proliferative diabetic retinopathy is a primary factor causing vision loss. The TRDD system is also developed to automatically diagnose tractional retinal detachment based on retinal images.

In this study, PCF and native cross filtering are presented to eliminate the influence of uneven brightness on the image caused by uneven light exposure. The BFM method is proposed to promote the performance of the TRDD system. The GBPD-RIS and GBPD-TRDD methods are provided to determine the most suitable parameters used in the RIS system and the TRDD system. The RIS system and the TRDD system can give impressive experimental results.

Footnotes

Academic Editor: Stephen D Prior

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

World Health Organization (WHO). Visual impairment and blindness, http://search.who.int/search?q=Visually+impaired&ie=utf8&site=who&client=_en&proxystylesheet=_en&output=xml_no_dtd&oe=utf8&getfields=doctype (accessed 28 June 2012).

Resnikoff

Pascolini

Mariotti

. Global magnitude of visual impairment caused by uncorrected refractive errors in 2004. B World Health Organ 2008; 86: 63–70.

Horiuchi

Hirano

. Colorization algorithm for grayscale image by propagating seed pixels. In: Proceedings of the IEEE international conference on image processing (ICIP), Barcelona, 14–17 September 2003, vol. 1, pp.457–460. New York: IEEE.

Otsu

A threshold selection method from gray-level histograms. IEEE T Syst Man Cyb 1979; 9: 62–66.

Gonzalez

Woods

RE.

Digital image processing. 2nd ed.New York: Prentice Hall, 2002.

Huang

Chen

Chan

. An automatic indirect immunofluorescence based cell segmentation and counting system. Taipei, Taiwan: National Digital Library of Theses and Dissertations in Taiwan, 2010.

Goutte

Gaussier

É.

A probabilistic interpretation of precision, recall and F-score, with implication for valuation, vol. 3408 (Lecture Notes in Computer Science). Berlin: Springer, 2005, pp.345–359.

MacQueen

JB.

Some methods for classification and analysis of multivariate observations. In: Le Cam

Neyman

(eds) Proceedings of 5-th Berkeley symposium on mathematical statistics and probability, vol. 1. Berkeley, CA: University of California Press, 1967, pp.281–297.

Haralick

RM.

Statistical and structural approaches to texture. Proc IEEE 1979; 67: 786–804.

10.

Haralick

Sternberg

Zhuang

XH.

Image analysis using mathematical morphology. IEEE T Pattern Anal 1987; 9: 532–550.

11.

Hoover

STARE database, 2000, http://www.ces.clemson.edu/~ahoover/stare/

12.

Eddins

The watershed transform: strategies for image segmentation. Newsletters—MATLAB news and notes, 2002, http://in.mathworks.com/company/newsletters/articles/the-watershed-transform-strategies-for-image-segmentation.html

13.

Mendonca

Campilho

Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction. IEEE T Med Imaging 2006; 25: 1200–1213.

14.

Hoover

Kouznetsova

Goldbaum

Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE T Med Imaging 2000; 19: 203–211.