Ship infrared image edge detection based on an improved adaptive Canny algorithm

Abstract

Influenced by light reflection and water fog interference, ship infrared images are mostly blurred and have low signal-to-noise ratio. In this paper, an improved adaptive Canny edge detection algorithm for infrared image of ship is proposed, which aims to solve the threshold of the traditional Canny cannot be adjusted automatically and the shortcomings of sensitivity to noise. The contrast limited adaptive histogram equalization algorithm is adopted to enhance the infrared image, the morphological filter replaces the Gauss filter to smooth the image, and the OTSU algorithm is utilized to adjust the high and low thresholds dynamically. The experimental results show that the improved Canny algorithm, which can not only improve the contrast of the image and automatically adjust the threshold but also reduce the background sea clutter and false edges, is an effective edge detection method.

Introduction

The infrared image, mainly describing thermal radiation of target and background, is widely used in the field of ship traffic management. However, most of the ship infrared images are blurred, have low signal-to-noise ratio (SNR), and polluted due to the limitations of detection device. Thus, a suitable edge detection method that can extract the ship’s edges accurately and clearly is essential. The general edge detection methods include differential operator method, template matching method, boundary and curve enhancement technology, wavelet edge detection, edge focusing, texture edge monitoring, neural network edge detection, and so on. Some algorithms process a whole infrared image without considering the prior information of the target region, which may lead to the failure detection of the target from the background. Some algorithms consider the prior information of the target region; however, these algorithms are too complex to meet the needs of the quasi real-time application of infrared reconnaissance. Ying-juan et al. ¹ and Sharifi et al. ² classified various edge detectors, by testing and comparing their advantages and insufficiencies. The traditional Canny algorithm is an effective optimal edge detector. ³ It is susceptible to noise pollution, but its Gauss variance value and dual threshold are not adaptive. ⁴ As a result, too much detailed edges are detected. In order to solve this problem, the OTSU algorithm, ⁵ which is proposed on the basis of the principle of least squares, is adopted to select the threshold automatically. In this article, an adaptive Canny algorithm is proposed by image enhancement and automatic calculation of the gradient adaptive edge detection. The adaptive Canny algorithm is applied to the ship infrared image edge detection and the experiment results show that the ship’s edges are detected more clearly with much less noises.

Traditional Canny algorithm

Three traditional criteria

The three criteria of the Canny edge detection algorithm, proposed by Canny ³ in 1986 based on optimization algorithm, are as follows:

1. SNR criteria. The real edges are detected as much as possible while the false edges are detected as less as possible. Then, the maximum SNR output is obtained.

2. Location accuracy criteria. The actual edges of the target in the image should be consistent with the detected edges.

3. Single edge response criteria. Remove the false edges response to the maximum extent so that the edges in the image can only be detected once. Due to the high detection accuracy, Canny algorithm is widely used.

Traditional Canny algorithm

Generally, there are four steps of the traditional Canny algorithm: Gauss filtering, gradient calculation, non-maximum suppression, and threshold and edge determination. ⁶

First, the Gauss filter, especially the two-dimensional Gauss function, is used to denoise the image. The formula of Gauss filter is shown as follows

$$G(x,y)=\frac{1}{2\pi {\sigma }^2}\exp (-\frac{x^2+y^2}{2{\sigma }^2})$$

(1)

$$I_1(x,y)=G(x,y)*I(x,y)$$

(2)

where $I(x,y)$ is the function of original image, and $\sigma$ is the width of the Gauss filter, which directly determines the effect of filtering. After the convolution filtering, the gradient of the image is calculated. However, the calculation process is slow because the image is made up of discrete points. In order to speed up the calculation, Rafati et al. ⁷ proposed Sobel algorithm to calculate the gradient of the image. In this article, 3 × 3 convolution kernel is used to perform convolution operations on the $X$ direction and $Y$ direction of the image, respectively

$$k_x=\left(\begin{array}{ccc}\hfill -1\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \multicolumn{1}{c}{-2}& \hfill 0\hfill & \hfill 2\hfill \\ \multicolumn{1}{c}{-1}& \hfill 0\hfill & \hfill 1\hfill \end{array}\right),k_y=\left(\begin{array}{ccc}\hfill -1\hfill & \hfill -2\hfill & \hfill -1\hfill \\ \multicolumn{1}{c}{0}& \hfill 0\hfill & \hfill 0\hfill \\ \multicolumn{1}{c}{1}& \hfill 2\hfill & \hfill 1\hfill \end{array}\right)$$

(3)

where $k_x$ and $k_y$ are the convolution kernel of the $X$ direction and $Y$ direction, respectively. After the convolution of the image, the gradient component value of the image in the $X$ direction would be $s_x$ , while $Y$ direction would be $s_y$ . Generally

$$\left|s\right|=\sqrt{s_x^2+s_y^2}$$

(4)

$$\theta =\arctan \frac{s_y}{s_x}$$

(5)

As mentioned above, $s_x$ and $s_y$ are the gradient components of the image along the $X$ and $Y$ directions at $(x,y)$ point, $\left|s\right|$ is the gradient of the point, and $\theta$ is the direction of the gradient of the point.

The gradient vectors should be “non-maximum suppression” after calculating the gradient at each point of the image, that is, to make out all the local maximums to get rid of these non-maximum points of the local.8-adjacency pixels method is used in the Canny edge detection algorithm, that is, the interpolation results were compared with the center point in the 0/45/90/135 direction, if the center point of the gradient is less than the maximum value of the interpolation results, the maximum is set to 0; if the center point is greater than the maximum gradient interpolation results, the maximum is set to 1. The 8-adjacency pixels are shown in Table 1.

Table 1.

8-adjacency pixels.

4	3	2
1	•	1
2	3	4

Finally, the threshold is selected and the edges are connected. The traditional Canny algorithm adopts dual thresholds $\mathrm{Th}$ and $\mathrm{Tl}$ to process binarization of the image that has been non-maximum suppressed. If the gray value of the point is larger than the maximum value, it is labeled as edge point; while if the gray value of the point is smaller than the low threshold $(\mathrm{Tl})$ , it is labeled as non-edge point; for those points whose gray value is between the two thresholds, only if it is connected to the edge point can be labeled as edge point.

Deficiency of the traditional Canny

The Gauss filter is used to smooth the image in the traditional Canny algorithm, in which the parameter $\sigma$ determines not only the coverage width of Gauss function but also the smoothness of the image.^8,9 The larger the $\sigma$ , the better the filtering effect; however, the edge information is prone to be blurred and weakened. The smaller the $\sigma$ , the worse the noise suppression effect, or even no effect.

Furthermore, the high threshold $\mathrm{Th}$ and the low threshold $\mathrm{Tl}$ of Canny algorithm are responsible for removing false edges and connecting edge points, respectively. ⁴ However, the dual thresholds are fixed and are not adaptive to different infrared images and various application scenarios.

Improved Canny algorithm

Most of the ship infrared images are blurred, with noises caused by sea wave, fog, and other light reflection. In order to reduce these interferences effectively, this article adopts contrast limited adaptive histogram equalization (CLAHE) algorithm to enhance the ship infrared image and replaces Gauss filter with morphological filtering. Finally, the improved OTSU method is used to optimize the dual thresholds of gradient to achieve adaptive detection of ship edges in infrared images.

CLAHE algorithm

By adjusting the dynamic range of the images, the histogram equalization (HE) can improve the image contrast and gray tone to enhance the contrast effect. ¹⁰ Because the HE algorithm can make a global dynamic adjustment of the image, it is likely to increase the contrast of the random sea waves and reduce the contrast of the useful signals such as the ship contour, and reduce the gray level, resulting in the details loss of the ship.^11,12 The CLAHE algorithm, ¹³ which equalizes the blocks of the image, can achieve better effects. The blocks of image share adjacent information fully by linear interpolation and can adjust local gray level adaptively. The specific steps of CLAHE algorithm are as follows:

• Step 1: dividing the image into blocks. The blocks do not overlap and are of equal size, and each block contains a number of pixels $M$ . If the image is irregular, then border interpolate is necessary. In this article, we use the “replicate” method to do the marginal adjustment in order to make the image regular, so that the extended image can be divided into several equal blocks.

• Step 2: calculating the histogram of blocks. $h(x)$ represents the histogram of the blocks, and $x$ represents the gray scale with range of $[0,L-1]$ , where $L$ is the gray series.

• Step 3: calculating the shear threshold clipLimit. The shear threshold clipLimit is calculated by the following equation

$$\mathrm{clipLimit}=\frac{M}{L}+\frac{M-M/L}{\mathrm{normClipLimit}}$$

(6)

where $\mathrm{normClipLimit}$ is the contrast enhancement value, which determines the extent of contrast enhancement. If the $\mathrm{normClipLimit}$ is too large, the contrast enhancement would be insufficient, and the details would not be clear either. On the contrary, if the $\mathrm{normClipLimit}$ is too small, the noise would be over-enhanced. In this article, the $\mathrm{normClipLimit}$ is set as 1.5.

• Step 4: redistributing the pixels. For each block, the corresponding clipLimit is used for shearing $h(x)$ , and the sheared pixels are redistributed evenly to each gray level of the histogram according to the following equations

$$\mathrm{totalE}=\sum _{x=0}^{L-1}\left(\max (h\left(x\right)-\mathrm{clipLimit},0)\right)$$

(7)

$$\mathrm{avgBIncr}=\mathrm{totalE}/L$$

(8)

where $\mathrm{totalE}$ is the sum of pixels that exceed $\mathrm{clipLimit}$ , and $\mathrm{avgBIncr}$ refers to the average number of pixels per grayscale in the histogram. This step can be explained clearly with the help of Figure 1.

Figure 1.

Pixels redistribution.

Repeat the above distributing until all the pixels are sheared.

• Step 5: HE for each block.

• Step 6: reconstructing the pixel gray value. The gray values of the pixels in the center of each block are obtained and are taken as the reference points to calculate the gray values of each point in the output image by the bilinear interpolation method.

The original ship infrared image is shown in Figure 2(a), while Figure 2(b) and (c) are the comparison results of HE algorithm and CLAHE algorithm. It can be seen from the figures that the background sea wave and other noise are suppressed greater in Figure 2(c) than in Figure 2(b), and the edges of the ship are clearer, too.

Figure 2.

Experimental results: (a) original infrared image, (b) HE algorithm result, and (c) CLAHE algorithm result.

Mathematical morphology filtering

On the basis of rigorous mathematical theory, the mathematical morphology filter processes the image nonlinearly. ¹⁴ The smoothing process of the image by mathematical morphology is actually an opening-and-closing operation. ¹⁵ Assuming that $L$ is the smoothed image, $I$ is the original image, and $B$ is the structuring elements, the general morphological filtering operators are as follows

$$L_1=(I·B)◯B$$

(9)

$$L_2=(I◯B)·B$$

(10)

where ◯ is an open operator and · is a closed operator. Thus, the smoothed image $L$ is

$$L=\frac{1}{2}[(I·B)◯B+(I◯B)·B]$$

(11)

In this article, the $B$ is a $3\times 4$ matrix, with the elements 1, that is

$$B=\left[\begin{array}{cccc}\hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \multicolumn{1}{c}{1}& \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \multicolumn{1}{c}{1}& \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \end{array}\right]$$

(12)

The tiny and bright details of sea wave can be reduced effectively after mathematical morphology filtering, while the overall gray level distribution and large bright area are not affected.

High and low thresholds

The high and low thresholds of Canny algorithm are usually determined by OTSU algorithm, ⁵ which can obtain the optimal segmentation gray value on the basis of the gray histogram of the image. ⁶ It is noted that the gradient of the background sea wave in the image has the characteristics of the same amplitude, direction, and spatial correlation. In order to filter the false edges introduced by the waves around the ship, this article adaptively determines the high and low thresholds by OTSU algorithm based on the gradient histogram of the image.

The pixels in the image array after non-maximum suppression are divided into two categories: contour pixels and background pixels. Let $W_0$ be the ratio of the number of background pixels to the image after clustering, $U_0$ be the average gradient of the background pixels after clustering, $W_1$ be the ratio of the number of contour pixels to the image after the clustering, and $U_1$ be the average gradient of the contour pixels after clustering. Then, the mean gradient of the whole interval is

$$U=W_0*U_0+W_1*U_1$$

(13)

The cost function is defined as

$$G=W_0*W_1*(U_0-U_1)*(U_0-U_1)$$

(14)

The high threshold $\mathrm{Th}$ is the maximum value of $G$ by traversing all the gradient magnitudes, and the low threshold $\mathrm{Tl}\approx 0.33*\mathrm{Th}$ . As the different images have different gradient magnitudes, the high and low thresholds of Canny algorithm can be adaptively determined according to the image.

Simulation results and discussion

In order to test and verify the effectiveness of the improved Canny algorithm, two ship infrared images with different gradient magnitudes were selected for experiments. The experimental results are shown below.

In the experiments, $\mathrm{Th}$ and $\mathrm{Tl}$ are set as 120 and 40, respectively. We can see from Figure 3(b) and (e) that although the traditional Canny algorithm can detect the edges of the ship, the false edges introduced by sea wave are also detected because of its poor suppression of the background noise. By contrast, the false edges around the ship are suppressed and greatly reduced by improved Canny algorithm shown in Figure 3(c) and (f). The proposed improved Canny algorithm can not only effectively improve the image contrast and reduce the influence of noise but also strengthen the edges and detect real edges.

Figure 3.

Experimental results: (a) first original infrared image, (b) Canny algorithm result, (c) improved Canny algorithm result, (d) second original infrared image, (e) Canny algorithm result, and (f) improved Canny algorithm result.

Conclusion

Infrared radiation has merits such as good adaptability to environment, long transmission distance, and can penetrate fog, haze, water, and so on; therefore, it is suitable for ship management system. However, due to the limitation of the instrument and the complexity of the sea surface, the infrared images also have the problems of ambiguity, low SNR, and sea clutter. Since the traditional Canny algorithm requires artificial setting thresholds and sensitive to noise, this article proposed an improved Canny algorithm for ocean ship infrared image to detect edges adaptively. First, CLAHE algorithm is utilized to enhance the ship infrared image, then mathematical morphology filtering is used for smoothing the image. Finally, the improved OTSU method is used to optimize the high thresholds $\mathrm{Th}$ and set $\mathrm{Tl}\approx 0.33*\mathrm{Th}$ . The experimental results show that the improved Canny edge detection algorithm can automatically set the thresholds, extract the edges, and reduce false edges. It is proved to be an effective method for edge detection with high accuracy and precision.