A surface defects inspection method based on multidirectional gray-level fluctuation

Abstract

Machine vision inspection technology provides an efficient tool for surface defects inspection. However, because of the multiformity of surface defects, the existing machine vision methods for surface defects inspection are limited by application scenarios. In order to improve the versatility of algorithms, and to process various kinds of images more accurately, we propose a new adaptive method for surface defect detection, named neighborhood gray-level difference method using the multidirectional gray-level fluctuation. This method changes thresholds and step values by extracting gray-level-fluctuating condition of images, and then it uses the neighborhood gray-level difference to segment defects from background. Experimental results demonstrate the effectiveness of the proposed method for inspecting different surface defects. Compared with other methods, the proposed method can be applied to inspect various surface defects, and it can provide more accurate defect segmentation results.

Keywords

Defect inspection gray-level fluctuation gray-level difference machine vision misclassification error

Introduction

Surface defects will decrease the quality of products, and at the same time, they will also impair the economic benefits of enterprises. Therefore, in the past decades, a lot of efforts have been made on the surface defects inspection systems. Since the traditional methods like artificial visual inspection are time-consuming and costly, automatic nondestructive inspection systems are urgently needed now.^1,2 Compared with other existing nondestructive inspection methods, such as methods of magnetic testing,³ eddy current⁴ and penetrant testing,⁵ machine vision inspection is widely recognized by many modern enterprises for its higher speed and lower price.⁶ As this method integrates many advanced technologies including image processing, optics, pattern recognition, artificial intelligence, and so on, it has obtained advantages such as being real time, high accuracy rate, and reliability. In recent years, the machine vision inspection technology has become a research hot spot in the field of surface defects inspection.^7,8 Among them, however, the number of algorithms which deal with visible light directly is much smaller,^9
–11 most of defect segmentation algorithms still need to use infrared and laser technology, which will increase the cost of defects inspection.

Image segmentation is the foundation of machine vision applications,¹² and automatic thresholding has been widely used in the machine vision inspection,¹³ which is used to extract objects from the image background. Thresholding techniques can be classified into global thresholding and local thresholding.¹⁴ To divide image pixels into object class or background class, the global methods use one calculated threshold value, while the local thresholding techniques use several different adapted threshold values which are selected according to the local region information.

Among them, Otsu method is a commonly used global thresholding method. Otsu’s method confirms a threshold which makes the between-class variance maximum or the within-class variance minimum. Otsu method can obtain satisfactory results when the objects and background have the similar variance, but it fails when the sizes of objects and image background are greatly different. To solve this problem, a number of improved Otsu methods have been proposed. For example, Ng proposed valley emphasis (VE) using the valley point information in the objective variance.¹³ Fan and Lei took neighborhood information at the valley point into consideration and came up with an improved Otsu method named neighborhood VE (NVE).¹⁵ Yuan et al. used the weighted object variance to revise Otsu method.¹⁶ It is worth mentioning that all these improved Otsu methods have high speed and achieve satisfactory results when the histogram distribution of the image is of bimodal pattern or close to bimodal pattern. But the complexity of environment will affect the histogram distribution of images. For example, nonuniform illumination will make the histogram distribution become unimodal pattern or close to unimodal pattern. In this situation, the effectiveness of these improved Otsu methods may greatly decrease. Therefore, Rosin proposed unimodal thresholding (UT),¹⁷ which is one of the most popular thresholding algorithms having good performance processing images with unimodal distributions. Martínez et al.¹⁸ successfully utilized UT method in industrial applications for surface defect inspection.

The local thresholding methods can segment objects from background more flexibly. The local information in each small region taken into account, local thresholding methods usually get better segmentation results in complex environment, so they can deal with nonuniform illumination images very well. Classic local thresholding methods include Bernsen,¹⁹ Niblack,²⁰ and Sauvola and Pietikäinen.²¹ However, there is no uniform standard to decide the number of blocks to divide images. Researchers usually select the number based on experience, thus the robustness of algorithms may be decreased. Dialectically, local thresholding methods are also limited by the regional information, which often leads to wrong classification.⁷ In conclusion, the existing thresholding methods for automatic inspection usually have low efficiency and narrow application scope.

In addition to those above, some very interesting ways are introduced to analyze surfaces with three-dimensional (3-D) models, which can process images from buildings till the limit of resolution of few microns, and they can be used in inspecting drilling,²² surface reconstruction,²³ laser ranging,²⁴ and so on; these methods also have a great value to further research.

In order to accurately inspect defects, we develop a new automatic method for surface defects inspection, named neighborhood gray-level difference method using multidirectional gray-level fluctuation (NDF). The contributions of this research are summarized as follows:

The proposed method combines the advantages of global characteristics with local characteristics. NDF changes thresholds and step values by extracting gray-level-fluctuating condition of the image, and then it uses the neighborhood gray-level difference to segment defects images and obtain the final results.

Experimental results demonstrate that the proposed method can be applied to various surface defects inspection, such as metal, woods, and walls. At the same time, it provides more accurate results than the existing methods, and we validate it by the misclassification error (ME).

The rest parts of the article are organized as follows. “Problems of the existing methods for surface defects inspection” section introduces the existing thresholding methods. “Proposed method” section introduces the proposed method (NDF) for surface defects inspection in detail. Experiments and analysis are discussed in “Experimental results and analysis” section. Finally, the last section concludes the article and describes future research.

Problems of the existing methods for surface defects inspection

The global thresholding methods

Global thresholding methods select a single threshold value from the information of gray-level histogram. Sezgin and Sankur did a thorough survey in global thresholding techniques and discovered that the Otsu was one of the most commonly used methods.²⁵

Otsu method

An image consists of many pixels, each pixel has a value, named gray-level value q, ranging from 0 to L − 1. The size of the image M × N decides the total number of pixels n. An image can be described as f(x, y), and the value of f(x, y) equals q. Define the number of pixels equal q as n_q . Then, the probability of gray-level q can be calculated by the following formula

\begin{matrix} P_{q} = \frac{n_{q}}{n} & q = {0, 1, 2, \cdot \cdot \cdot, L - 1} \end{matrix}

Now, suppose the selected threshold is $k \in [0, L - 1]$ , then pixels of the image can be divided into two classes: $C_{1} = [0, k] and C_{2} = [k + 1, L - 1]$ . C ₁ and C ₂ are normally corresponding to the pixels of foreground (objects of interest) and the background. Their cumulative probabilities are defined as $P_{1} (k) and P_{2} (k)$ , respectively

P_{1} (k) = \sum_{q = 0}^{q = k} P_{q}

P_{2} (k) = \sum_{q = k + 1}^{q = L - 1} P_{q} = 1 - P_{1}

$m_{1} (k) and m_{2} (k)$ denote the average gray-level values of C ₁ and C ₂, respectively

m_{1} (k) = \sum_{q = 0}^{q = k} q \times \frac{P_{q}}{p_{1} (k)}

m_{2} (k) = \sum_{q = k + 1}^{q = L - 1} q \times \frac{P_{q}}{p_{2} (k)}

The calculation formula of between-class variance $σ_{B}^{2} (k)$ for the image is as follows

σ_{B}^{2} (k) = P_{1} (k) {(m_{1} (k))}^{2} + P_{2} (k) {(m_{2} (k))}^{2}

Then, keep changing the selected threshold $k \in [0, L - 1] until σ_{B}^{2} (k)$ reaches the maximum value. In other words, the optimal threshold (TH) decided by Otsu should maximize the between-class variance.

The improved Otsu methods

Ng¹³ proposed a revised Otsu method named VE by weighting the between-class variance. The weighted value W equals $(1 - P_{k})$ , that means, $σ_{B}^{2} (k)$ was transformed into $[σ_{B}^{2} (k)]^{'}$ .

{\begin{matrix} {[σ_{B}^{2} (k)]}^{^{'}} = (1 - W) \times (P_{1} (k) {(m_{1} (k))}^{2} + P_{2} (k) {(m_{2} (k))}^{2}) \\ W = P_{k} \end{matrix}

Fan and Lei¹⁵ suggested neighborhood information should be taken into consideration and proposed NVE method. He changed the weighted value W according to the sum probability of the neighborhood gray-level values around the gray level k in interval t.

\begin{array}{l} W = \sum_{q = k - t}^{q = k + t} P (q) \\ \begin{matrix}  \end{matrix} \begin{matrix}  \end{matrix} = P (k - t) + P (k - t + 1) + \cdot \cdot \cdot + P (k + t - 1) + P (k + t) \end{array}

Yuan et al.¹⁶ introduced the weighted object variance (WOV) method; in his research, $σ_{B}^{2} (k)$ should be transformed into ${[σ_{B}^{2} (k)]}^{''}$

{[σ_{B}^{2} (k)]}^{″} = P_{1} (k) P_{1} (k) {(m_{1} (k))}^{2} + P_{2} (k) {(m_{2} (k))}^{2}

Otsu methods select threshold according to the histogram of images.²⁶ In an ideal situation, the Otsu optimal threshold should be located at the valley point of the histogram when the histogram of image is of bimodal distribution. For images, which are of unimodal distribution, threshold should be located at the left bottom rim of the single peak. However, histogram can be affected by illumination easily. For those uneven illuminative images, histograms cannot accurately represent the difference between the objects and the background. The lighter portions of defective area may be recognized as background. Similarly, the darker portions without defects may also be recognized as defects.

Problems of the existing Otsu methods

Figure 1 presents the disadvantages of Otsu methods. Original images are showed in Figure 1(a).

Figure 1.

Disadvantage of the Otsu methods in segmenting uneven illuminative images: (a) original images; (b) to (e) the segmentation results of Otsu, VE, NVE, and WOV; (f) to (g) histograms and threshold values of original images, respectively (in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; NVE: neighborhood VE.

The first line of Figure 1 shows a metal surface image with scratches. The Otsu method segments part of background as defects. The result of VE method loses most of the defects. NVE and WOV cannot recognize these scratches at all. Figure 1(f) is the histogram of first original image; it indicates that Figure 1(a) is of unimodal distribution. The results show that Otsu methods cannot provide satisfactory results when the images are of unimodal distribution.

The second line of Figure 1 shows an uneven illuminative image of the first original image. The top part and the left bottom part of the second original image are brighter than other parts, while the right part is darker and the right bottom region is the darkest. Otsu method and VE segment the whole darker right part as defect incorrectly. NVE provides completely wrong result, which segments the whole image as defect. WOV method recognizes the darkest part as object incorrectly. By comparing Figure 1(f) with Figure 1(e), we can discover that illumination has enormous influence on histogram. The experimental results further demonstrate that the effectiveness of Otsu methods may be greatly decreased by nonuniform illumination.

The local thresholding methods

Local thresholding techniques determine the thresholds according to per-pixel neighborhood statistics features. Classic local thresholding methods include Bernsen method, Niblack method, and Sauvola method.

Bernsen method

Bernsen adopts local contrasting technologies. Assume that there is a $(2 w + 1) \times (2 w + 1)$ window and define the gray-level value of current pixel as $f (x, y)$ . The Bernsen threshold $T (x, y)$ equals the average of maximum gray-level value and minimum gray-level value in the window

T (x, y) = \frac{1}{2} (\max_{\begin{array}{l} - w < k < w \\ - w < l < w \end{array}} f (x + k, y + l) + \min_{\begin{array}{l} - w < k < w \\ - w < l < w \end{array}} f (x + k, y + l))

Finally, Bernsen method attains binary image by the following formula

b (x, y) = {\begin{matrix} \begin{matrix} 0 & f (x, y) < T (x, y) \end{matrix} \\ \begin{matrix} 1 & f (x, y) \geq T (x, y) \end{matrix} \end{matrix}

Niblack method and Sauvola method

Niblack and Sauvola obtain thresholds by computing mean value and mean variance of local region. Assume that there is a $(2 w + 1) \times (2 w + 1)$ window and define the gray-level value of current pixel as $f (x, y)$ . The average and standard deviation of pixels in the window are $m (x, y) and s (x, y)$ , respectively.

The threshold $T (x, y)$ of Niblack can be computed as follows

T (x, y) = m (x, y) + k \cdot s (x, y)

where k is a parameter for correcting result, which usually ranges from 0 to 1.

Sauvola improves Niblack by redefining the threshold $T (x, y) as T^{'} (x, y)$

T^{'} (x, y) = m (x, y) \cdot (1 + k \cdot (\frac{s (x, y)}{R} - 1))

where R is a dynamic range parameter, which is usually set to R = 128.

Disadvantages of Bernsen, Niblack, and Sauvola

The above three methods compute statistical property based on a $(2 w + 1) \times (2 w + 1)$ window. The size of the window has great influence on the segmentation results. If the value of w is increased, these local thresholding algorithms may lose the advantage of local properties and miss some details of the image. The processing time will also increase with w. Conversely, if the value of w is too small, which will generate noise into result images. It is hard to select proper values of w for various images automatically, which greatly limits the application of local thresholding methods introduced above. Comparing with global thresholding, local thresholding methods cost more processing time.²⁷ For example, Niblack and Sauvola cost more time because they deal with images based on point-to-point processing method. It is difficult to apply them to real-time surface defects inspection systems.

After analyzing the disadvantages of existing methods for surface defects inspection, we find three difficult challenges to be overcome. These challenges are as follow:

The surface defects show the characteristics of multiformity and complexity. Firstly, there are various materials, such as wood, metal, fabric, the wall, and so on. Secondly, the common defects contain scratches, indentation, cracks, stains, and so on.^28
–30 The diversity of defects makes surface defects inspection more difficult.

Some products are highly reflective, especially products composed of metal.³¹ Even worse, the surface images of some products are often of illumination inequality. Both of them lead to low contrast between defects and background.

Some products are exposed to complex outdoor environments, where defects easily hide in the dirt. It makes defects harder to be found accurately.

In order to overcome these challenges, we propose a new automatic method for surface defects inspection. The proposed method is based on NDF. It can overcome the drawbacks of existing methods and obtain satisfactory results.

Proposed method

The proposed NDF method

Defects on product surface are various, including scratches, holes, cracks, and so on. Therefore, the gray-level values of surface defects images are usually unevenly distributed. Using only a single threshold cannot overcome the influence of variety of defects. Meanwhile, it is certainly not enough to obtain accurate segmentation results relying on the absoluteness of gray-level information merely. Through the analysis above, we propose a new method. The algorithm not only takes local gray-level-fluctuating information into consideration but also takes advantage of global algorithms. The proposed algorithm greatly enhances the effectiveness and accuracy of surface defects inspection.

Four steps are required in this method, as shown in Figure 2.

Step 1. Analyze gray-level fluctuation information of surface images from four directions containing 0°, 45°, 90°, and 135°.

Step 2. Based on the gray-level-fluctuating condition of images, change thresholds and step values automatically.

Step 3. Segment the images according to the thresholds and step values selected by step 2 and use the segmentation algorithm of gray-level difference in local region.

Step 4. Condense these images generated in step 2 into one image to form the final result.

Figure 2.

Processing steps of the proposed method.

Adaptive step values and thresholds for the NDF method

Intuitively, images can be treated as two-dimensional fluctuations. As is shown in Figure 3, protuberances of it are waves of gray level. X-axis and Y-axis represent the directions of height and width of the image, respectively. $f (x, y)$ shows gray-level values. For a single pixel, pixel located at peak of waves has higher gray-level value. Pixel located at the bottom of waves has lower gray-level value. The gray-level values of defect areas are usually higher or lower than other parts, which will reflect in the waves of gray level. In other words, defect areas will increase volatility of images. Before processing images, we should analyze the fluctuating conditions of them.

Figure 3.

Schematic of gray-level-fluctuating condition.

As is shown in Figure 3, the proposed method analyzes gray-level fluctuation in the directions of 0°, 45°, 90° 135° and obtains step values and thresholds automatically.

Extract all one-dimensional (1-D) gray graphs of every direction (0°, 45°, 90°, 135°). The total number of each 1-D gray-graph in every direction is n. F(n) represents each 1-D gray-graph mean gray-level value in every direction

F (n) = \frac{1}{n} \sum_{i = 1}^{n} f (i)

where F(i) is the gray-level value of pixel i. The mean-variance K of 1-D gray-graph pixels group is as follows

K = \frac{\sum_{i = 0}^{n} \sqrt{| F {(i)}^{2} - F {(n)}^{2} |}}{n}

Select the pixels group of every direction (0°, 45°, 90°, 135°), which maximizes K, and then define each direction’s pixels group selected above as principal curves $F_{d} (i)$ ; the mean gray-level value of principal curves $F_{d} (i) is F_{d} (n)$ , where d represents directions including 0°, 45°, 90°, 135°.

The gray-level-fluctuating of 1-D pixels group is abstracted, as shown in Figure 4. Then compute the difference H_d between $F_{d} (i) and F_{d} (n)$ and find the pixel making the difference maximum. For example, in Figure 4, the pixel i₂ has the maximum gray-level difference $F_{d} (n) and \max (H_{d}) = | F_{d} (i_{2}) - F_{d} (n) |$ . Find the contiguous areas, which are all above (or below) the average gray value, and $F_{d} (n)$ marks the adjacent pixels in contiguous area of current pixel i₂. Then define the start and end pixel of the marked points as k and m, respectively. As is shown in Figure 4, the adjacent pixels should be marked as $k \sim m = i_{1} \sim i_{3}$ . Define the length of adjacent pixels marked above as $L_{d} = | k - m | = | i_{3} - i_{1} |$ .

Figure 4.

The gray-level fluctuation of the principal curve F_d (i).

The proposed method selects step value J_d according to L_d

J_{d} = {\begin{matrix} 2 & 0 < L_{d} \leq \frac{2}{3} n \\ 3 & \frac{2}{3} n < L_{d} \leq n \end{matrix}

The threshold of each direction is decided as follows

G_{d} = \frac{\sum_{i = k}^{m} | F_{d} (i) - F_{d} (n) |}{L_{d}}

The step values and thresholds of each direction are decided in this section, which take full advantages of the global characters of the images. Then, the proposed method segments the images according to the neighborhood gray-level difference, which makes use of the advantages of local information. Both of them combine the global characteristics with local characteristics, thus the proposed method provides more accurate results. Segment image in different directions as follows

F (x, y) = {\begin{matrix} \begin{matrix} 1 & | F (x, y) - F (x + i \cdot L, y + j \cdot L) | > G_{d} \end{matrix} \\ \begin{matrix} 0 & | F (x, y) - F (x + i \cdot L, y + j \cdot L) | < G_{d} \end{matrix} \end{matrix}

where $i, j \in {- 1, 0, 1}$ , and they are used to represent different directions by choosing different values. The relationship between i, j and directions is shown in Figure 5. Figure 6 presents the segmentation results of neighborhood gray-level difference in different directions. Figure 6(a) is the original image to be processed.

Figure 5.

Schematic of neighborhood gray-level difference in different directions.

Figure 6.

Segmentation results of neighborhood gray-level difference in different directions: (a) original image, (b) segmentation result T ₀, (c) segmentation result T ₄₅, (d) segmentation result T ₉₀, (e) segmentation result T ₁₃₅, and (f) the final result T (in this article, defects are indicated by black pixels and background is indicated by white pixels).

As in Figure 6(b), segment image in direction 0° and 180° using step value J₀ and threshold G₀ and obtain result T₀;

As in Figure 6(c), segment image in direction 45° and 225° using step value J₄₅ and threshold G₄₅ and obtain result T₄₅;

As in Figure 6(d), segment image in direction 90° and 270° using step value J₉₀ and threshold G₉₀ and obtain result T₉₀;

As in Figure 6(e), segment image in direction 135° and 315° using step value J₁₃₅ and threshold G₁₃₅ and obtain result T₁₃₅.

The final segmentation result is $T = (T_{0} | T_{45} | T_{90} | T_{135})$ , which is shown in Figure 6(f).

Experimental results and analysis

Experiments are conducted in a computer using MATLAB R2010a (MathWorks, Massachusetts, U.S.A.), with Intel Core I7 2.60 GHz CPU and 8 GB memory. The proposed method can be applied to various defects inspection. To verify this, three groups of surface images are chosen for experiments. The first group contains metal surface images. The second tested data are made up of wooden surface images. The last one is used to test the segmentation results for cracks in the wall. In this stage, many researchers use ME to test the effectiveness of segmentation methods.^32

–35 We also adopt ME as the validation criterion to validate the effect of the proposed method.

The ME is computed as follows

ME = 1 - \frac{| B_{O} \cap B_{T} | + | F_{O} \cap F_{T} |}{| B_{O} + F_{O} |} \in [0, 1]

where B_o and F_o are the foreground and background of the ideal segmentation result, respectively. The ideal segmentation results are usually made by researchers themselves. The value of ME can intuitively reflect the accuracy of segmentation. The value of ME ranges from 0 to 1, and the smaller ME represents better results, while bigger ME means lower accuracy.

In this article, we use seven existing methods to compare with the proposed method (NDF), including Otsu, VE, WOV, Bernsen, Niblack, Sauvola, and UT. Among these local thresholding methods, Bernsen computes different thresholds using a 100 × 100 window. Niblack adopts a 11 × 11 window, and Sauvola deals with images using a 80 × 80 window.

Inspection of metal surface

In order to test the effectiveness of NDF for inspecting metal surface, we set five images including common defects: mixed other impurities, rusts, cracks, concaves, and scratches, and most of these images have uneven illumination. Especially, illumination has bigger influence on metal because metal is a highly reflective material. Therefore, illumination is an important factor which cannot be neglected. In order to quantitatively and intuitively compare the results of different methods, we express ME values of different methods for metal image I–metal image VI by histograms in Figure 12(a) to (e).

Figure 7(a) is metal image I with impurities. The Otsu and VE methods define most of the background as defects. Nonuniform illumination makes some area darker than other part of image. Otsu, VE, and UT methods classify the darker area as foreground because the darker area has lower gray-level values. WOV segments all defects as background, while it inspects the darkest pixels as foreground. Bernsen method is also influenced by the nonuniform illumination and classifies part of dark area as objects wrongly. Niblack recognizes most of defects but it generates too much noises. Although it loses some details of defects, Sauvola provides a better result. But it also segments part of background as defects wrongly. For metal image I, the proposed method provides the best result among the other seven methods. Figure 7(b) is the ideal segmentation result of metal image I and Figure 12(a) is the ME histogram of eight methods. Obviously, the proposed method has the smallest ME that is 0.0219. It means the proposed method provides the most accurate result.

Figure 7.

Segmentation results of metal image I with impurities: (a) origin image; (b) ideal segmentation result; (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 8(a) is metal image II with rusts. Its illumination is not homogeneous too. The rusts are of point shape and distributed randomly. Otsu, VE, Bernsen, and UT methods segment most of darker area as object incorrectly. WOV fails to recognize defects and just inspects the darkest part of the image. Niblack method generates too much noise. Sauvola method segments most of the defects, but it is also affected by the illumination and classifies bottom part of metal image II as object improperly. The proposed method recognizes almost all the defects, but it is affected by textures of the image and generates a small amount of noise. Figure 8(b) presents the ideal segmentation result of metal image II. Figure 12(b) shows the ME of eight methods for Figure 8(a). Otsu and VE have the highest ME values, while the proposed method has the smallest one. The proposed method provides the best segmentation result for Figure 8(a).

Figure 8.

Segmentation results of metal image II with rusts: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 9(a) is metal image III with cracks and rough textures. The illumination at the bottom of metal image III is lower than that of other parts. Otsu and VE obtain almost the same results, while they segment the darker bottom as object. Bernsen has the same problem as Otsu and VE. WOV loses a lot of details of defects. Sauvola and UT are affected by the rough textures and generate some noise, while Niblack generates much noise. The proposed method recognizes the border of the cracks. Comparing with Figure 9(b), all the methods make out defects accurately, but most of them generate varying degrees of noise. In Figure 12(c), the highest value of ME is created by Niblack, and the smallest one is generated by the proposed method. The proposed method obtains relatively better result than other methods.

Figure 9.

Segmentation results of metal image III with cracks: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 10(a) is metal image IV with concaves. The gray-level values of defect are not lower than background as most defective images. In this case, global thresholding methods are invalid. As is shown in Figure 10(c) to (e), Otsu, VE, and WOV fail to process metal image IV. Bernsen, Sauvola, and UT recognize nothing of this image. Niblack has too much noise. For this image, only the proposed method segments concave properly. Figures 10(b) and 12(d) further prove the effectiveness of the proposed method for metal image IV.

Figure 10.

Segmentation results of metal image IV with concaves: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Metal image V with scratches is presented in Figure 11(a). Comparing with Figure 11(b), this image has a slight scratch and it contains some textures in shape of oblique lines, which confuses the textures with the true scratches. Otsu segments lots of background into foreground. VE, WOV, Bernsen, and Sauvola fail to inspect Figure 11(a) completely. Niblack recognizes the whole textures of the image. UT segments part of the defects but introduces some noise. The proposed method is affected by the textures and segments some textures as defects. In Figure 12(e), the ME of the proposed method is close to 0, and it still provides the best result.

Figure 11.

Segmentation results of metal image V with scratches: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 12.

Ideal segmentation results and ME of metal images above: (a) to (e) MEs histogram of eight methods for metal surface images above (in this article, defects are indicated by black pixels and background is indicated by white pixels). ME: misclassification error.

Experiments of inspecting the above metal surface images show that the proposed method provides good results for metal images. But it may be affected by the surface textures, which will result in a little noise and false targets, and the false targets mean the false objects wrongly inspected.

Inspection of wooden surface

Common defects on wooden surface include wormholes, concaves, cracks, rots, and so on. We set four images covering all types of defects introduced above (Figures 13(a), 14(a), 15(a), and 16(a)). In this section, effect of the proposed method on wooden surface will be tested completely. The ideal results of original wooden images are shown in Figures 13 to 16(b). The ME of different methods for the provided images is presented in Figure 17(a) to (d).

Figure 13.

Segmentation results of wooden image I with wormholes: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 14.

Segmentation results of wooden image II with concaves: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 15.

Segmentation results of wooden image III with cracks: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 16.

Segmentation results of wooden image IV with rotten areas: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (NDF; in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding; NDF: neighborhood gray-level difference method using multidirectional gray-level fluctuation.

Figure 17.

Ideal segmentation results and ME of wooden images above: (a) to (d) MEs histogram of eight methods for wooden surface images above (in this article, defects are indicated by black pixels and background is indicated by white pixels). ME: misclassification error.

In Figure 13(a), wooden image I contains some cracks and lots of small wormholes. The top of image is dark and some tan plaques are randomly distributed in the image. Otsu and VE classify dark area and tan plaques as defects wrongly. Bernsen also segments plaques into objects. Niblack generates serious noise. UT fails to make out defects. WOV recognizes most holes and all cracks, but it incorrectly identifies some plaques as objects and loses some details of defects. Compared with Figure 17(a), though generating a small amount of noise, the proposed method still provides the most accurate result and generates the smallest ME.

In Figure 14(a), wooden image II has a deep concave in the middle of it. On both sides of the concave, there are some slight cracks. The area around the concave has lower gray-level values than other regions, which increases the challenge of inspection. As predicted, Otsu and Bernern define most of the area around the concave as objects improperly. VE and Sauvola segment part of the area around the concave as defects, too. Niblack and UT obtain almost completely wrong result. WOV fails to make out the crack on the left of the image. The proposed method gets the best result but it generates some noise. Figure 17(b) shows the proposed method has the most correct result compared to Figure 14(b).

Wooden image III in Figure 15(a) is smooth and illuminated uniformly. Otsu, VE, WOV, and Bernsen obtain satisfactory results. Niblack has the worst effect. Under the influence of wooden texture, Sauvola, UT, and the proposed method generate some noise in segmentation results. As is shown in Figure 17(c), the ME values of all methods tested except Niblack method are close to 0.

Figure 16(a) is wooden image IV with a big rotten area in the middle. Apart from Niblack and the proposed method, the rest methods make out the rotten area correctly. UT loses some details of the defect. The proposed method only recognizes the border of the object. It is caused by the segmentation method which the proposed method adopts. Because the object is big, the gray-level values around have no difference with the pixel to be processed. Therefore, the pixels within object cannot be made out by the proposed method. However, the border of the defect is also of great significance to the defect inspection system.

The proposed method has better effect on segmenting small wooden defects. But it can only recognizes the border of big objects.

Inspection of cracks in the wall

Inspection of cracks in the wall also has practical meaning. It can be applied to dam detection and high buildings inspection, which are time-consuming and difficult to conduct. We provide experiments for the detection of cracks in the wall using the proposed method. The data to be tested include three shapes of cracks in the wall.

Wall image I is shown in Figure 18(a). It has a long crack throughout the whole image. The areas around the crack have lighter color. Otsu and VE segment most of the background into object wrongly. UT introduces much noise. WOV loses some details of the defect. Bernsen, Sauvola, and the proposed method make out the crack successfully, but they generate different degrees of noise.

Figure 18.

Segmentation results of wall image I with cracks: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding.

In Figure 19(a), the cracks of wall image II are more complex than other wall images. Otsu, VE, Bernsen, Sauvola, and UT segment some dark areas in background as defects wrongly. Niblack completely fails to segment this image. WOV and the proposed method provide better results. As is shown in Figure 21(a), the ME values of them are ME = 0.0409 and ME = 0.0416, respectively.

Figure 19.

Segmentation results of wall image II with cracks: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding.

Wall image III is shown in Figure 20(a), and the cracks are not obvious. In other words, the gray-level value of defective areas has no obvious difference with the background. The VE method, Niblack, and UT cannot segment wall image III. WOV and Bernsen fail to recognize the crack. Otsu method classifies part of the background as defects wrongly. Sauvola and the proposed method make out the cracks successfully. But Sauvola loses some details. The proposed method provides the best result with ME = 0.0026 (Figure 21(c)).

Figure 20.

Segmentation results of wall image III with cracks: (a) origin image; (b) ideal segmentation result; and (c) to (j) the segmentation results of the Otsu, VE, WOV, Bernsen, Niblack, Sauvola, UT, and the proposed method (in this article, defects are indicated by black pixels and background is indicated by white pixels). VE: valley emphasis; UT: unimodal thresholding.

Figure 21.

Ideal segmentation results and ME of wall images above: (a) to (c) MEs histogram of eight methods for wall images above (in this article, defects are indicated by black pixels and background is indicated by white pixels). ME: misclassification error.

The proposed method has good results in detecting various kinds of cracks in the wall. But it is affected by the texture of the wall surface and it may lead to a little noise.

False positive rates for nondestructive surface images

In a practical application, only the ME value as the evaluation is not enough. Researchers also concern false positive rates (FPRs) for nondestructive surface images. FPR can be described as follows

FPR = \frac{Number of defect-free samples detected as defective}{Total number of defect-free samples}

FPR ranges from 0 to 1; lower FPR value represents more accurate judgment.

Values of FPRs of the eight methods for different tested images are presented in Table 1. In the table, three kinds of nondestructive surface images are tested, including metal images, wooden images, and wall images. A data set with 100 metal nondestructive images, 100 wooden nondestructive images, and 100 wall nondestructive images is constructed to comprehensively test the defects detection results. Otsu and Niblack both have high FPR values in three kinds of tested images. WOV has the lowest FPR values in eight methods, but this method cannot complete the defect detection work in most of the time. The proposed method (neighborhood gray-level difference method using the multidirectional gray-level fluctuation (DNF)) performs not so well, especially in wall images, it has high FPR value. Because wall images usually contain complex textures, the proposed method will be greatly affected by these textures. However, the proposed method can complete the task of defect detection almost every time, and the optimization of the algorithm for different textures is the next step of our research.

Table 1.

Comparison of false positive rates for eight methods.

	Otsu (%)	VE (%)	WOV (%)	Bernsen (%)	Niblack (%)	Sauvola (%)	UT (%)	DNF (%)
Metal	97.4	71.6	20.4	60.0	100	35.3	67.5	44.2
Wood	98.3	35.5	40.6	43.6	100	49.5	90.1	52.1
Wall	99.2	69.3	8.4	74.8	100	72.3	73.2	65.4

VE: valley emphasis; UT: unimodal thresholding.

Conclusion

In this article, we provide a new automatic method based on NDF. It selects four thresholds and step values according to fluctuation of different directions (0°, 45°, 90°, 135°). Then, DNF method segments the image by neighborhood gray-level difference method using the thresholds and step values computed before. Therefore, this proposed method combines the advantages of global properties and local properties. It can obtain more accurate results than the existing methods for surface defects inspection.

The experimental results verify the efficiency of NDF. This new method can be applied to detecting defects in different situations, such as metal surface, wooden surface, and wall surface. The proposed method provides satisfactory results in all these three situations. It can overcome the influence of illumination and the shape of defects. However, this method may generate different degrees of noise under influence of the texture in the surface. Because of the principles of neighborhood gray-level difference method, the proposed method can only segment the borders of big objects. In conclusion, compared with other existing methods, the proposed new method obtains better results for inspecting various material surface defects.

In the future, our efforts can focus on both decreasing the influence of surface texture and improving the validity of recognizing big objects. Besides, we will attempt to combine our method with 3-D models^22
–24 for getting better and more accurate surface analysis results.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (no. 41306089).

References

Yao

Wang

Liu

. Development status of surface defects inspection for metal plants. Mater Rev 2011; 25(2): 238–241.

Sun

Cheng

Wang

. Method for detecting high-speed rail surface defects by photoacoustic signal. Acta Physica Sinica Chinese Edition 2016; 65(3): 38105-038105.

Atzlesberger

Zagar

Cihal

. Sub-surface defect detection in a steel sheet. Meas Sci Technol 2013; 24(8): 084003.

Xie

Xiao

Ya-Peng

. Research on an eddy current sensor used for steel ball surface defect detection. Transducer Microsystem Technol 2012; 11: 012.

Yang

. Non-destructive testing of pressure vessels: penetrant testing. Nondestr Test 2004; 26(7): 359–363.

Feng

Zou

Yan

. Real-time fabric defect detection using accelerated small-scale over-completed dictionary of sparse coding. Int J Adv Robot Syst 2016; 13: 1.

Song

Yan

. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Appl Surf Sci 2013; 285(21): 858–864.

Liu

Chen

. Defect detection system design for steel surface based on machine vision. Manuf Autom 2015; 12: 134–137.

Padilla

Chawla

Silva

L F

. Image analysis of cracks in the weld metal of a wet welded steel joint by three dimensional (3D) X-ray microtomography. Mater Charact 2013; 83: 139–144.

10.

Zhang

Xie

. Optoelectronic inspection of defects for metal cylindrical workpieces. Opt Precision Eng 2014; 22(7): 1871–1876.

11.

. On-line detection technique of tiny surface defects for metal plates and strips based on photometric stereo. J Mech Eng 2013; 49(4): 25–29.

12.

Wang

. Cluster ensemble-based image segmentation. Int J Adv Robot Syst 2013; 10(7): 297.

13.

. Automatic thresholding for defect detection. Pattern Recogn Lett 2006; 27(14): 1644–1649.

14.

Yin

. Image segmentation technology based on thresholding. Mod Electron Tech 2007; 30(23): 107–108.

15.

Fan

Lei

. A modified valley-emphasis method for automatic thresholding. Pattern Recogn Lett 2012; 33(6): 703–708.

16.

Yuan

Peng

. An improved Otsu method using the weighted object variance for defect detection. Appl Surf Sci 2015; 349: 472–484.

17.

Rosin

. Unimodal thresholding. Pattern Recogn 2001; 34(11): 2083–2096.

18.

Martínez

Ortega

García

. An industrial vision system for surface quality inspection of transparent parts. Int J Adv Manuf Tech. 2013; 68(5): 1123–1136.

19.

Bernsen

. Dynamic thresholding of gray-level images. Int Conference on Pattern Recogn 1986; 2: 1251–1255.

20.

Niblack

. An introduction to digital image processing. Denmark: Strandberg Publishing Company 1985.

21.

Sauvola

Pietikäinen

. Adaptive document image binarization. Pattern Recogn 2000; 33(2): 225–236.

22.

Marani

Nitti

Cicirelli

. High-resolution laser scanning for three-dimensional inspection of drilling tools. Adv Mech Eng 2013; 2013(5): 1–13.

23.

Marani

Roselli

Nitti

. A 3D vision system for high resolution surface reconstruction. Sensing Technology (ICST), 2013 Seventh International Conference on. IEEE, 2013: 157–162.

24.

Marani

Reno

Nitti

. A modified iterative closest point algorithm for 3D point cloud registration. Comput Aided Civ Infrastruct Eng 2016; 31(7): 515–534.

25.

Sezgin

Sankur

. Survey over image thresholding techniques and quantitative performance evaluation. J Electron Imaging 2004; 13(1): 146–168.

26.

Liu

Cai

Ling

. Survey on the method of image threshold segmentation. Comput Knowledge Technol 2015; 11(1): 140–142.

27.

Tian

Tan

. Modification of two-dimensional entropic thresholding method and its fast iterative algorithm. Moshi Shibie Yu Rengong Zhineng/Pattern Recogn Artif Intell 2010; 23(1): 127–136.

28.

Qin

. Research on hardware surface defects detection based on image processing techniques. J Eng Des 2011; 18(2): 134–138.

29.

Qin

Liu

Gong

. Testing surface crack defects of sheet metal with ultrasonic lock in thermography. Infrared Laser Eng 2013; 42(5): 1123–1130.

30.

Wang

. Research on defects on the metal surface based on image processing. Liaoning, Shengyang, China: Shenyang Ligong University 2012.

31.

Fang

. Highlight area reconstruction of data matrix barcodes marked on metal surface. Comput Eng Appl 2016; 52(16): 205–209.

32.

Lin

. Tsallis entropy and the long-range correlation in image thresholding. Signal Process 2012; 92: 2931–2939.

33.

Zou

. Image bi-level thresholding based on multiscale gradient multiplication. Int Conference Digital Image Process 2012; 38(4): 853–861.

34.

Rao

Wang

. Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods. Postharvest Biol Technol 2013; 82(4): 59–69.

35.

Beauchemin

. Image thresholding based on semivariance. Pattern Recogn Lett 2013; 34(5): 456–462.