Sage Journals: Discover world-class research

Abstract

At present, computer vision system is widely used for the cleanliness inspection for spinneret holes, but it has a high misjudgment rate for many holes with small dirt. In this paper, a method is proposed to improve the accuracy of the cleanliness inspection. The method has four sequential phases. First, the closed contour curve of a standard hole and its curvature are extracted. Based on double-threshold segmentation of the contour curve, line and arc segments are segmented to generate a closed piecewise curve model. Second, the model is fitted to the closed contour curve L₁ of the hole to be inspected based on a nonlinear least squares principle, and the distance curve that represents the shortest distance between the closed piecewise curve L₂ constructed from the aligned model and L₁ is calculated based on the nearest neighbor search algorithm. Third, the dirt detection curve L₃ is generated from the distance curve weighted by a blended weighting curve. Final, based on a global threshold combining with unevenness elimination of L₃, the cleanliness index is calculated based on the segmentation and location of dirt, and is used to judge whether the hole is qualified or not. The experimental results of four databases demonstrate that, the proposed method provides better performance compared with the traditional method.

Keywords

Spinneret hole cleanliness inspection dirt micro-hole inspection

Introduction

Chemical fibers are mostly made of crystalline polymers. The fused polymer is ejected from the spinneret hole under the conditions of high temperature and high pressure, and the chemical fiber is formed after cooling and crystallization. After a period of use, the spinneret must be cleaned ultrasonically and dried, and it is necessary to inspect whether there is any remaining dirt before the spinneret is reused. Otherwise, the dirt will lead to abnormities of filament performance, which means that the cleanliness inspection of spinneret holes is an important procedure during the quality monitoring process of synthetic fiber production.

In recent years, computer vision technology has been widely used in the textile industry.^1–8 For the research of the automatic inspection of spinneret holes, Yin et al.⁹ and Tan¹⁰ proposed the self-correcting position control system. Tan¹⁰ studied many image processing algorithms for hole focusing and inspection. Komatsu et al.¹¹ proposed a critical dimension measurement of a hoe pattern. Nagasu et al.¹² used low-frequency sound for inspecting micro-holes with diameters of less than 100 µm. Cai et al.¹³ proposed a roundness measurement with the least square circle fitting method for the processing of geometric feature recognition of holes. Zhang et al.¹⁴ extracted the geometrical characteristics of round holes to judge whether them are blocked or not. Yang et al.¹⁵ designed a high-precision calibration model for spinneret inspection system. Yang et al.¹⁶ analyzed the influential factors of the measurement accuracy. Yang et al.¹⁷ acquired the clear image of large spinneret by increasing number of auto-focus. Gu et al.¹⁸ proposed an inspection route planning method based on improved ant colony algorithm. Wang et al.¹⁹ achieved the judgment of block by area threshold method and calculated the shaped degree for non-circular spinneret hole. Chen et al.²⁰ developed a low cost digital spinneret inspection system with opencv toolkits. Chen et al.²¹ developed an inclination measurement by using the imaging based autofocus method. Chen et al.^22,23 established a path generation method for circle-type, rectangle-type and ring-type spinnerets. In the last 10 years, American ASPEX Co. Ltd, Japanese TORAY Co. Ltd, Chinese Academy of Sciences et al. have already developed spinneret automatic inspection systems for industrial applications.

This paper focus on the problem of the cleanliness inspection for the holes, but so far, almost all the proposed method are based on the characteristic indexes such as area and perimeter to determine whether the hole is clean or not. We know dirt inside the hole will cause the index value to be abnormal. However, due to the fluctuation of area and perimeter index values caused by machining errors beyond the sensitivity range of the detection of tiny dirt, this method is only effective for holes with large dirt whose contour is significantly different from that of a normal hole. Therefore, the cleanliness inspection for the holes with tiny dirt, especially for special-shaped holes, has always been a difficult question.

To solve this problem, we proposed the approach based on the template matching error analysis method. Firstly, we extracted the standard contour template of the hole, and defined the distance curve as the error distribution of the alignment between the contour of the standard template and that of the hole to be inspected. Secondly, the dirt detection curve from the distance curve weighted with a blended weighting curve was defined. Finally, the cleanliness index was qualified based on the segmentation and location of dirt on the dirt detection curve. However, if the contour curve is used as a rigid-template aligned to that of the hole to be inspected, especially for complex-shaped holes, the local fitting error between the template and the deformed contour curve caused by machining errors can easily lead to a false-positive dirt identification. Therefore, according to the mechanism of the contour curve of hole mainly based on the combination of line and arc segments, we proposed a closed piecewise curve model with parameterized control points for non-rigid registration to the contour curve of the hole to minimize the errors, which can not only ensure the overall rigidity of the segments in the model, but also avoid their excessive deformation. The proposed method can improve the accuracy of the cleanliness inspection and be suitable for the most shapes of holes with various degrees of dirt distribution.

As Figure 1 shows, our approach includes six steps: (a) Extracted the closed contour curve $L_{1}$ of a standardized hole and $L_{2}$ of the hole to be inspected; (b) Generated a closed piecewise curve model with parameterized control points; (c) Fitted the model to $L_{2}$ based on the non-liner least squares principle; (d) Calculated a distance curve based on the aligned model and $L_{2}$ ; (e) Generated the dirt detection curve from the distance curve weighted with a blended weighting curve; (f) Segmented the dirt from the dirt detection curve and generated the cleanliness index and compared it with predefined criteria to judge whether the hole is qualified or not. Our main contributions are summarized as: (a) a closed piecewise curve model suitable for all the shapes of holes, (b) the generations of the distance curve and dirt detection curve, and (c) the definitions of the model of dirt segmentation and cleanliness index.

Figure 1.

Flow chart of the proposed method.

Image acquisition

Our Image acquisition system includes motor control unit, image acquisition unit, light unit, etc. The motor control unit was adopted with three-axis ADLINK servo motor control system, the acquisition unit was adopted with Basler 500M industrial camera (2592 × 1944 × 8 bit and 15 frames/s) and a 14× zoom lens, and the light unit was adopted with upper LED lighting for focusing and back LED lighting for acquisition. The software was compiled with C++ on a Windows system.

In this paper, four types of small image database were collected. Each database belongs to a same spinneret plate. Among them, DB1 has 36 images with Y-shaped holes (DPI = 1.4594 µm / pixel, 0.5 mm (petal length) × 0.18 mm(petal width), including 26 clean holes), DB2 has 268 images with flat-shaped holes (DPI = 2.3681 µm / pixel, 0.4 mm(petal length) × 0.06 mm (petal width), including 255 clean holes), DB3 has 22 images with cross-shaped holes (DPI = 1.9692 µm / pixel, 0.45 mm(petal length) × 0.1 mm (petal width), including seven clean holes), DB4 has 51 images with round holes (DPI = 2.3681 µm / pixel, 0.18 mm (diameter), including 37 clean holes). The holes in DB1 to DB4 have been identified with clean or dirt by experienced inspectors, and all images in each database have been cropped to the appropriate size. Figure 2(a) shows the working diagram of the system when the upper and back light sources are turned on. Figure 2(b) shows the back and front image of the spinneret plate. Figure 2(c) shows the front and back project lighting image captured from a round hole with sharp-shaped large dirt. Figure 2(d)–(g) show four images of DB1 to DB4 respectively, in which column 1 is the standard clean hole, which is used as the image for model generation, column 2 to 4 are holes with clean, tiny and large dirt, respectively, which are used to analyze the effects of cleanliness inspection lately.

Figure 2.

Image acquisition: (a) acquisition system, (b) images of the spinneret plate, (c) images of round hole with sharp-shaped dirt, and (d)–(g) four images in DB1to DB4 respectively.

Method of cleanliness inspection

Closed contour curve reconstruction

The edge of a clean hole is smooth and it will become irregular if there is dirt in the hole. The brightness of the dirty inside of the hole changes with the size of the dirt and its distance from the edge of the orifice. It is difficult to find a suitable fixed threshold for various dirt segmentation, because even small changes of threshold will lead to a shift of the segmented boundary. However, due to the properties of non-maximum and zero-crossings, the edge detector can locate the edge more precisely than the global thresholding method. We used the canny method²⁴ to extract the hole boundary, in which two thresholds are used to detect strong and weak edges, and the true weak edges are detected more likely and robustly, which is especially suitable for detecting the tiny dirt. Here, we used $[σ_{t h r 1} = 50, σ_{t h r 2} = 200]$ as the low and high gradient magnitude threshold pair to detect hole boundary. In general, the edge points in boundary may be broken, or there may be bifurcations or isolated noise, so we processed it with morphological operations and linked the edge points to form a closed-contour curve by edge tracing algorithm.²⁵ Figure 3 shows the results of reconstructed closed-contour curves, in which (a)–(d) correspond to the closed contour curves of Figure 1(d) to (g) respectively.

Figure 3.

Reconstructed closed contour curve: (a)–(d) correspond to the results of Figure 2(d)–(f).

Model generation

Let $L_{1} = {l_{i}^{1} = (x_{i}^{1}, y_{i}^{1}), i \in [1, m_{1}]}$ be the closed contour curve reconstructed from Figure 3(a1), where $l_{i}^{1}$ is the coordinate of the $i$ -th point in curve, $m_{1}$ is the number of the curve point. We firstly calculated the curvature of $L_{1}$ . According to the characteristic distribution of the curvature, $L_{1}$ was divided into line and arc segments which were connected end-to-end based on double-threshold segmentation of the perimeter curve. The line or arc parameters of its corresponding segment were estimated based on the least squares fitting algorithm,²⁶ and the control point was located depending on the solver of the intersection of each adjacent segment pair. Finally, the model was generated, and the closed piecewise curve was constructed, which was used as an approximate representation of $L_{1}$ .

Calculation of the curvature

Let $l_{i, f}^{1}$ and $l_{i, b}^{1}$ be the farthest point of $l_{i}^{1}$ within the radius $r_{1}$ before and after. Based on $(l_{i}^{1}, l_{i, f}^{1}, l_{i, b}^{1})$ , a circle was fitted using the least squares method²⁶ and its radius was used as the curvature operator $c_{i}$ . The curvature of $L_{1}$ was described by the equation $C = {c_{i}, i \in [1, m_{1}]}$ . Figure 4(a) shows a schematic of curvature calculation, and Figure 4(b) show the curvature curves of $L_{1}$ in Figure 3(a1).

Figure 4.

Examples of the model generation process: (a) illustration of curvature calculation, (b)–(f) the curvature curve, line and arc segmentation, fitted line and arc segments and the reconstructed model generated from Figure 3(a1), respectively, and (g)–(i) the reconstructed model generated from Figure 3(b1), 3(c1), and 3(d1), respectively.

Closed piecewise curve model with parameterized control points

Let $c_{t h r 1}$ and $c_{t h r 2}$ be the curvature threshold for the separation of line and arc segments, respectively, and usually $c_{t h r 2}$ ≪ $c_{t h r 1}$ . Let $g_{t h r}$ be the length threshold for segment fusion. If $c_{i} > c_{t h r 1}$ , $l_{i}^{1}$ belongs to a line segment, shown as the black point in Figure 4(b). If $c_{i} <_c_{t h r 1}$ and $c_{i} > c_{t h r 2}$ , $l_{i}^{1}$ belongs to a large arc segment, shown as the green line in Figure 4(b), otherwise it belongs to a small arc segment, shown as the red line in Figure 4(b). The start and end positions of each segment were used as its control points, and two adjacent segments shared the same one. Compared with the line segment, the arc segment added a middle control point located at the center of the arc which indicated its direction. Let $G = {g_{i}, i \in [1, f_{g}]}$ be the set of segment lengths, where $g_{i}$ denotes the length of each segment and $f_{g}$ denotes the total number of segments. By searching the minimum segment length $g_{k m i n}$ in $G$ , if $g_{k m i n} < g_{t h r}$ , the $k m i n$ -th segment was discarded, and the control point of its two adjacent segments was replaced by the midpoint of the discarded segment. Repeat iteratively for calculation of $G$ and fusion of the segments with minimum length and leave when no conditions are met. If there was only one arc segment after curvature segmentation and segment fusion, it means that the spinneret hole was round. In this case, it was divided into two connected arc segments. The parameterized model was established as follows:

F = {f_{i} = (f_{i, x}, f_{i, y}), i \in [1, f_{n}]}

(1)

P = {p_{i} = (s, e), i \in [1, f_{l}], s \in [1, f_{n}], e \in [1, f_{n}], s \neq e}

(2)

Q = {\begin{cases} q_{i} = (s, c, e), i \in [1, f_{c}], s \in [1, f_{n}], \\ e \in [1, f_{n}], s \neq e, s \neq c, c \neq e \end{cases}}

(3)

\bar{C} = {\bar{c_{i}}, i \in [1, f_{c}]}

(4)

\begin{array}{l} C P C M (F, P, Q) = \sum_{i = 1}^{f_{l}} L I N E (f_{p_{i}, s}, f_{p_{i}, e}) \\ + \sum_{j = 1}^{f_{c}} A R C (f_{q_{j}, s}, f_{q_{j}, c}, f_{q_{j}, e}) \end{array}

(5)

Where $F$ is the set of control points, $P$ is the set of line segments, $Q$ is the set of arc segments, $\bar{C}$ is the set of radii of arc segments, $C P C M$ is the reconstructed curve of the model, $s$ is the number of control points as starting points of segments, $e$ is the number of control points as ending points of segments, $c$ is the number of control points as middle points for arc segments, $f_{n}$ is total number of control points, $f_{l}$ is total number of line segments, and $f_{c}$ is the total number of arc segments. $L I N E (f_{p_{i}, s}, f_{p_{i}, e})$ is the line segment function formed by two points $(f_{p_{i}, s}, f_{p_{i}, e})$ ,²⁷ and $A R C (f_{q_{j}, s}, f_{q_{j}, c}, f_{q_{j}, e})$ is the arc segment function formed by three points $(f_{q_{j}, s}, f_{q_{j}, c}, f_{q_{j}, e})$ , which can be expressed as multiple line segments connected end to end.

Figure 4 shows examples of model generation with parameters set as $r_{1} = 61$ , $c_{t h r 1} = 350$ , $c_{t h r 2} = 150$ , and $g_{t h r} = 40$ . The model generation process for Figure 3(a1) is demonstrated in Figure 4(b)–(f), respectively. Figure 4(c) shows the curvature segmentation result before segment fusion, and the result of the corresponding segment fusion as well as the locations of control points are shown in Figure 4(d). Figure 4(e) shows the least-squares fitting results of each line and arc segment in Figure 4(d). Figure 4(f) shows the results of readjustment of each control point and the reconstructed closed piecewise curve of the model. The CPCM generated from Figure 3(b1)–(d1) is shown in Figure 4(g)–(i), respectively.

Non-rigid registration of the model

Let $L_{2} = {l_{i}^{2} = (x_{i}^{2}, y_{i}^{2}), i \in [1, m_{2}]}$ be the contour curve extracted from Figure 3(a2), where $l_{i}^{2}$ are the coordinates of $i$ -th curve point, $m_{2}$ is the total number of the curve points. The non-rigid registration of the model is based on the use of a non-linear least squares optimization algorithm to minimize the mean distance between the points in $L_{2}$ and their projections on the closed piecewise curve, which was constructed from the model by relocating the positions of control points. Let $\emptyset_{l} (l_{k}^{2}, F, p_{i})$ be the distance function from the $k$ -th point in $L_{2}$ to its projection on the $i$ -th line segment of the model. Let $\emptyset_{c} (l_{k}^{2}, F, q_{j})$ be the distance function from the $k$ -th point in $L_{2}$ to its projection on the $j$ -th arc segment of the model. The objective function for the optimization of the control point set $F$ was defined as follows:

\underset{(F)}{argmin} \sum_{k = 1}^{m_{2}} \min {\begin{cases} \min_{i \in [1, f_{l}]} | \emptyset_{l} (l_{k}^{2}, F, p_{i}) |, \\ \min_{j \in [1, f_{c}]} | \emptyset_{c} (l_{k}^{2}, F, q_{j}) | \end{cases}}

(6)

We used the ceres-solver optimal method (http://ceres-solver.org) to solve equation (5), and the optimized control point set was obtained as $F_{τ}$ . Figure 5 shows the result of non-rigid registration of the model in Figures 3(a3) to Figure 4(f), in which the blue line represents the CPCM of the aligned model and the red line represents $L_{2}$ .

Figure 5.

The non-rigid registration of the model: (a) the result of alignment of the model in Figure 3(a3) to Figure 4(f), (b) the CPCM from the aligned model in (a), and (c) zoomed-in view of (b).

The distance curve

We redefined $L_{3} = {l_{i}^{3} = (x_{i}^{3}, y_{i}^{3}), i \in [1, m_{3}]}$ as the CPCM from the model with $F_{τ}$ , where $l_{i}^{3}$ are the coordinates of the $i$ -th curve point, $m_{3}$ is the total number of the closed curve points. Figure 6(a) shows a schematic representation of distance curve calculation, in which the blue dotted line represents $L_{3}$ , and the red dotted line represents $L_{2}$ . Let $M$ be an $m_{2} \times m_{3}$ matrix initialized with −999, and let $M_{(j, i)} = ‖ l_{i}^{3} - l_{j}^{2} ‖$ be the distance from $l_{i}^{2}$ in $L_{2}$ to its closest point $l_{j}^{3}$ in $L_{3}$ , shown as the blue line in Figure 6(a). Finding the maximum value ${\bar{M}}_{(k, i)} = \max_{j \in [1, m_{2}]} (M_{(j, i)})$ along each column $i$ in $M$ , if ${\bar{M}}_{(k, i)} = - 999$ , indicates that $l_{i}^{3}$ was not searched by any point in $L_{2}$ as the closest point, then the point pairs $(j 1, i 1)$ and $(j 2, i 2)$ that meet the conditions of ${\bar{M}}_{(j 1, i 1)} ≅ - 999$ and ${\bar{M}}_{(j 2, i 2)} ≅ - 999$ were searched from the forward and backward columns of column $i$ in $M$ , and the row $k$ was located via ${\bar{M}}_{(k, i)} = \min_{j \in [j 1, j 2]} (‖ l_{i}^{3} - l_{j}^{2} ‖)$ , which is shown as the green line in Figure 6(a). Finally the distance curve of $L_{3}$ was calculated as follows:

L_{4} = {D (i) = \max_{j \in [1, m_{2}]} (M_{(j, i)}), i \in [1, m_{3}]}

(7)

Note that $L_{4}$ may have many local fluctuations due to the irregular dirt edge. We therefore dilated it with a $1 \times 9$ line structuring element to filter the sawtooth noise. Figure 6 shows the results of distance curve calculation from Figures 3(a2), in which Figure 6(d) shows the illustration of converting $L_{4}$ into the distribution of the 2D distance curve, and the normal direction of the curve points to the enclosed area of the model.

Figure 6.

Distance curve calculation: (a) overview of distance curve calculation with local enlargement, (b) the calculation results of (a), (c) the 1D distance curve L4, (d) the illustration of the 2D distance curve constructed from L4, and (e) the 2D smoothed distance curve of (c).

The dirt detection curve

A smaller curvature often leads to a larger mechanical processing error, which results in a larger error between $L_{2}$ and $L_{3}$ , as shown in Figure 5(c). In order to suppress the error as much as possible, the distance curve $L_{4}$ was weighted based on the distribution of the curvature. The curvature weight of the line segment was set to 1, described by equation (8), where $D_{l}$ is the point set of the $l$ -th line segment in $L_{3}$ . The gradient interpolation method in which a smaller curvature has a smaller weight was adopted for the weighting function of arc segments, described by equation (9), where $D_{c}$ is the point set of the $c$ -th arc segment in $L_{3}$ . Equation (10) describes the mixed curvature weighting function $W_{t 1}$ , in which the coefficient jump of the curvature transition region among the weighted curve $W_{l}$ and $W_{c}$ was smoothed with a Gaussian kernel $g (i)$ , where * represents the convolution operation, and µ1 = 0, $σ_{1}$ = 3.0. Because of the discontinuity in the curvature distribution around the location of the control point between two adjacent segments, as shown in Figure 5(c), nonlinear fitting errors may appear between the location of the optimized control point and the curvature transition area of the contour curve. If the hole was dirty or its deformation was large, the error will cause the dirt curve to appear as a bloated artifact around the location of the control point, and it will be segmented as dirt. Therefore, we used the Gaussian weighting function $W_{t 2}$ to suppress it as given by equations (11) and (12), where $f_{i}$ is the point index of the $f$ -th control point, and $σ_{2} = 2.5$ . Finally, the blended weighting function $W_{t}$ was given by equation (13), and the dirt detection curve $L_{5}$ denoting the distance curve weighted by the blended weighting function was defined by equation (14).

W_{l} (i) = {\begin{matrix} 1, i f i \in D_{l} \\ 0, i f i \notin D_{l} \end{matrix}

(8)

W_{c} (i, \bar{c_{l}}) = {\begin{matrix} \begin{matrix} 1.0, i f \bar{c_{l}} > c_{t h r 1} \\ 0.5 + \frac{\bar{c_{l}} - c_{t h r 2}}{c_{t h r 1} - c_{t h r 2}} 0.5 \\ 0.5, i f \bar{c_{l}} < c_{t h r 2} \end{matrix} & i f i \in D_{c} \\ 0 & i f i \notin D_{c} \end{matrix}

(9)

\begin{array}{l} W_{t 1} (i) = [\sum_{l = 1}^{f_{l}} W_{l} (i) + \sum_{c = 1}^{f_{c}} W_{c} (i, \bar{c_{l}})] * g (i), \\ g (i) = - \exp (- 0.5 {[(i - μ_{1}) / σ_{1}]}^{2}) / \sqrt{2 π} σ_{1} \end{array}

(10)

W_{f} (i) = 0.5 (1 - \exp (- {[\frac{i - f_{i}}{σ_{2}}]}^{2} / 2))

(11)

W_{t 2} (i) = \min_{f \in [1, f_{n}]} W_{f} (i)

(12)

W_{t} (i) = W_{t 1} (i) \cdot W_{t 2} (i)

(13)

L_{5} = {W (i) = W_{t} (i) \cdot D (i), i \in [1, m_{3}]}

(14)

Figure 7 shows the calculation of the dirt detection curves in Figures 6(e). In Figures 7(b), the distance from each point on the blue curve along the normal direction to the red curve is its blended weighting coefficient. From the comparison between Figures 6(e) and 7(c) we can see that the false singularity blob caused by the nonlinear fitting errors is mainly suppressed by the blended weighting function.

Figure 7.

The dirt detection curve: (a) the weighting curves of L4, (b) the 2D blended weighting curves of L4; (c) the 2D weighted dirt detection curves of Figure 6(e), shown as red line.

Dirt segmentation and cleanliness inspection

Generally, the distance curve and the dirt detection curve of a standardized hole are approximately smooth and flat, and the dirt appears as a singularity blob in the detection curve, which can be segmented and located by a global distance threshold. However, due to the machining error of the spinneret, plate deformation and abrasion of the hole after many times of use, local distortion will occur at the contour curve of the hole, which leads to baseline unevenness in the dirt detection curve, as shown in Figures 8(a). This unevenness makes the global thresholding method unsuitable for dirt segmentation. Therefore, we adopted the top-hat transform²⁵ to eliminate the irregularity of the baseline in the dirt detection curve, as shown in Figures 8(b). Figure 8 shows the results of dirt segmentation and location in Figures 7(c). The model of dirt segmentation and its parameter definition are shown in Figure 8(d), where W, H, and S are the width, height, and area of dirt respectively and $D_{t h r}$ is the global threshold value. Let the $i$ -th dirt index be ${W_{i}, H_{i}, S_{i}}, i \in [1, n]$ , where $n$ is the number of dirts. In the production of synthetics fiber, dirt in the spinneret hole can potentially break the filament. For this reason, the relative weighting of dirt with sharp shape (shown as Figure 1(c)) should be increased based on aspect ratio when calculating the cleanliness index. The cleanliness index of hole was calculated as follows:

P = \sum_{i = 1}^{n} S_{i} \cdot [(H_{i} / W_{i}) < 1 ? 1.0 : (H_{i} / W_{i})]

(15)

In Figure 8(e), the cleanliness index of the hole is shown in the center in parentheses, the same as below. Meanwhile, the location of dirt is labeled with the black wireframe, and the dirt parameters are shown as (W, H, S) next to it. Let $P_{t h r}$ be the threshold of cleanliness. If $P > P_{t h r}$ , the hole is unqualified as containing dirt. In this case, the hole will be blown automatically, and then be inspected again. If the hole was still unqualified, it will be cleaned manually. In this paper, $D_{t h r}$ and $P_{t h r}$ were optimized by maximizing the True Recognition Rate (TRR). False Accept Rate (FRR), False Rejection Rate (FRR) and TRR were defined as bellows.

FAR (P_{t h r}, D_{t h r}) = (\sum_{i = 1}^{m_{d}} [\begin{array}{l} (P_{i} (D_{t h r}) \\ < P_{t h r}) ? 1 : 0 \end{array}] / m_{d}) \times 100 %

(16)

FRR (P_{t h r}, D_{t h r}) = (\sum_{j = 1}^{m_{c}} [\begin{array}{l} (P_{j} (D_{t h r}) \\ \geq P_{t h r}) ? 1 : 0 \end{array}] / m_{c}) \times 100 %

(17)

T R R (P_{t h r}, D_{t h r}) = (\begin{array}{l} (\begin{array}{l} \sum_{i = 1}^{m_{d}} [\begin{array}{l} (P_{i} (D_{t h r}) \\ < P_{t h r}) ? 1 : 0 \end{array}] \\ + \sum_{j = 1}^{m_{c}} [\begin{array}{l} (P_{j} (D_{t h r}) \\ < P_{t h r}) ? 1 : 0 \end{array}] \end{array}) \\ / (m_{d} + m_{c}) \end{array}) \times 100 %

(18)

Where $m_{d}$ and $m_{c}$ is the number of dirty and clean holes respectively, and $P_{i} (D_{t h r})$ and $P_{j} (D_{t h r})$ is the cleanliness index of $i$ -th and $j$ -th hole with its dirt segmented by $D_{t h r}$ . The objective function for the optimization of $P_{t h r}$ and $D_{t h r}$ was defined as follows:

\underset{(D_{t h r}, P_{t h r})}{argmax} T R R (P_{t h r} \in [0, 1000], D_{t h r} \in [1.5, 4.0])

(19)

In the factory, $P_{t h r}$ usually be adjusted according to the actual quality monitoring requirements for cleanliness inspection.

Figure 8.

The segmentation and location of dirt: (a) and (b) baseline estimation and its elimination for Figure 7(c), respectively, (c) the 2D curve of (b), (d) the illustration of dirt segmentation and its parameter definition, and (e) the dirt segmentation and labeling for (c).

Results and discussion

We firstly listed part of holes in DB1 to DB4, and aligned every contour curve of clean holes to that of its standard hole. Then we optimized the $D_{t h r}$ and $P_{t h r}$ for each database, and analyzed the results of the dirt inspection of examples in Figure 3. Finally, according to the traditional method, we fitted the multivariate distributions of the area and perimeter of clean and dirty holes with the Multivariate Gaussian Distribution (MGD) (https://en.wikipedia.org/wiki/Multivariate_normal_distribution), and compared the accuracy of its identification with the proposed method.

Distributions of aligned contour curve of clean holes

Part of the clean holes in DB1 to DB4 collected in this paper are shown in Figure 9(a)–(d), and the dirty holes are shown in Figure 10(a)–(d). We aligned the contour curve of the clean hole in each database to that of its corresponding standard hole rigidly, and the deformation of each hole due to machining errors can be visually observed. Figure 11 shows the error distributions between the aligned contour curve of clean holes and that of its standard hole (shown as red line in the enlarged area in Figure 11(a)–(d)).

Figure 9.

Part of clean holes: (a)–(d) corresponds to DB1 to DB4, respectively.

Figure 10.

The holes with dirt: (a)–(d) correspond to DB1 to DB4, respectively.

Figure 11.

The error distribution of the contour curve of clean holes: (a)–(d) correspond to DB1 to DB4 respectively.

Optimization for $D_{t h r}$ and $P_{t h r}$

Figure 12 shows the optimization curves of TRR- $D_{t h r}$ - $P_{t h r}$ in DB1 to DB4. Figure 13 shows the histograms of the cleanliness index $P$ calculated from the optimized $D_{t h r}$ , in which red line represents the optimized $P_{t h r}$ . Table 1 shows the optimized parameters and their TRR, FAR, and FRR for each database. From Table 1 we can see that under the highest TRR optimization goal, all clean holes in each database are correctly identified. However, there are still some holes with tiny dirt within the critical range (see a1, a6 and b12 in Figure 10) or their dirt locate at the top of flap and is suppressed by blend weighting curve (see b9, b13, and c9 in Figure 10), which cause them identified as clean holes. For round holes, the TRR is 100%.

Table 1.

Optimized parameters and the accuracy of identification for the proposed method.

	$D_{t h r}$	$P_{t h r}$	TRR (%)	FAR (%)	False accepted holes in Figure 10
DB1	1.95	21	94.44	20.00	a1, a6
DB2	3.10	10	98.88	23.08	b9, b12, b13
DB3	2.00	133	95.46	6.67	c9
DB4	1.50	4	100.00	0.00

Figure 12.

The optimization curves: (a)–(d) correspond to DB1–DB4 respectively.

Figure 13.

(a)–(d) The histograms of cleanliness indexes in DB1–DB4.

Analysis of the dirt inspection

Figure 14 shows the cleanliness inspection results of holes in Figure 3 with optimized parameters, in which we can see that this method can satisfy the requirements for cleanliness inspection of holes with various degrees of dirt. In Figure 14(a) to (d), the holes are clean identified by manually, and there is no dirt detected except for a small dirt with a very low cleanness index in Figure 14(c), which indicates that there may be errors in subjective evaluation or the rule of judgments for complex-shaped holes may be relaxed. From Figure 14(e)–(g) we can see that, for holes with tiny dirt, the non-rigid registration of the model can accurately relocate the control point to the target position of the contour curve, and the dirt can be recognized correctly both in the line and arc segments. Especially for the round hole in Figure 14(g), the curvature difference between the two arcs in $L_{3}$ is small enough to ensure the curve smoothly overcomes the connection between adjacent segments, which guarantees the generation of a dirt detection curve free from the interference caused by the nonlinear fitting errors around the location of the control points, and the dirt detection curve with baseline elimination can effectively illustrate the distribution of the dirt. For holes with large dirt in Figure 14(h)–(k), the optimization of relocating control points will fail during the non-grid registration process due to too much difference between $L_{1}$ and $L_{3}$ . However, the detection distance between $L_{2}$ and $L_{3}$ is still big enough to reflect the singularity of the dirt, which will only affect the quantitation of the cleanliness index, nor the judgment whether the hole is qualified or not.

Figure 14.

Cleanliness inspection results: (a)–(d) and (h)–(k) correspond to the results of column 2 and column 4 in Figure 3, respectively, (e)–(g) correspond to the results of (c2), (c3), and (c4) in Figure 3, respectively. Column 1: non-rigid registration; Column 2: dirt detection curve; Column 3: dirt segmentation and labeling.

Identification based on the traditional method

We defined the curve length of $L_{2}$ as the value of perimeter, denoted as $x_{p}$ , and the filling area of $L_{2}$ as the value of area, denoted as $x_{s}$ . The probability density function (pdf) of MGD was given as bellows:

f (x) = \frac{\exp (- \frac{1}{2} {(x - μ)}^{T} Σ^{- 1} (x - μ))}{2 π \sqrt{| Σ |}}

(20)

Where $x = {(x_{s}, x_{p})}^{T}$ denoting the multivariate vector, $μ = {(μ_{s}, μ_{p})}^{T}$ denoting its mean vector, and $Σ = [\begin{matrix} σ_{s}^{2} & σ_{s} σ_{p} \\ σ_{p} σ_{s} & σ_{p}^{2} \end{matrix}]$ denoting its covariance matrix. Let $X_{c} = [x_{1}^{c}, x_{2}^{c}, \dots, x_{m_{c}}^{c}]$ and $X_{d} = [x_{1}^{d}, x_{2}^{d}, \dots, x_{m_{d}}^{d}]$ be the vector set of clean and dirt holes, and we fitted equation (20) to them as $f_{c} (x)$ and $f_{d} (x)$ respectively.²⁸ Figure 15 demonstrates the fitting results of DB1 to DB4, in which $f_{c} (x)$ is represented with red curve and $f_{d} (x)$ is represented with blue dash line. The fitted parameters $(μ_{s}, μ_{p})$ and $(σ_{s}, σ_{p})$ of MGD are shown in Table 2. From Figure 15 we can see that all the distributions of $f_{c} (x)$ and $f_{d} (x)$ in DB1 to DB4 are partially overlapping, especially for DB1 and DB2, and the variances of $f_{d} (x)$ are obviously bigger than those of $f_{c} (x)$ . Based on the rule of maximum probability, we redefined FRR, FAR, and TRR as given bellows:

FAR (X_{d}) = (\sum_{i = 1}^{m_{d}} [(\begin{array}{l} f_{d} (x_{i}^{d}) \\ < f_{c} (x_{i}^{d}) \end{array}) ? 1 : 0] / m_{d}) \times 100 %

(21)

FRR (X_{c}) = (\sum_{j = 1}^{m_{c}} [(\begin{array}{l} f_{c} (x_{i}^{c}) \\ < f_{d} (x_{i}^{c}) \end{array}) ? 1 : 0] / m_{c}) \times 100 %

(22)

T R R (X_{c}, X_{d}) = (\begin{array}{l} (\begin{array}{l} \sum_{i = 1}^{m_{d}} [(\begin{array}{l} f_{d} (x_{i}^{d}) \\ \geq f_{c} (x_{i}^{d}) \end{array}) ? 1 : 0] \\ + \sum_{j = 1}^{m_{c}} [(\begin{array}{l} f_{c} (x_{i}^{c}) \\ \geq f_{d} (x_{i}^{c}) \end{array}) ? 1 : 0] \end{array}) \\ / (m_{d} + m_{c}) \end{array}) \times 100 %

(23)

Figure 15.

The fitting results of MGD: (a)–(d) correspond to DB1 to DB4 respectively.

Table 2.

Fitted parameters of MGD.

	Clean		Dirty
	$(μ_{s}, μ_{p})$	$(σ_{s}, σ_{p})$	$(μ_{s}, μ_{p})$	$(σ_{s}, σ_{p})$
DB1	(2,73,834, 1758)	(4574.1, 7.2)	(2,74,663, 1765)	(8033.3, 20.7)
DB2	(24,819, 1093)	(399.8, 62.9)	(24,484, 1071)	(1482.8, 63.8)
DB3	(1,38,055, 1574)	(869.5, 102.9)	(1,33,901, 1628)	(3506.4, 99.2)
DB4	(25,451, 558)	(100.6, 2.1)	(25,018, 573)	(520.2, 27.2)

Table 3 shows the values of FAR, FRR and TRR calculated from equation (21) to (23) in DB1 to DB4. We can see that in DB2 only Figure 10(b6) that blocked nearly in half was identified as dirty hole, and half of the dirty holes in DB1 were mistaken for clean holes, and in DB3 and DB4 the low FAR is due to the low overlapped area of two fitted MGD.

Table 3.

The accuracy of identification based on fitted MGD.

	TRR (%)	FAR (%)	FRR (%)	False accepted holes in Figure 10
DB1	75	50.00	15.38	a1, a3, a5, a9, a10
DB2	89.55	92.31	6.27	b1–b5, b7–b14
DB3	77.27	26.67	14.29	c2, c9, c13, c14
DB4	88.24	35.71	2.70	d4, d9, d10, d12, d14

Comparison of the accuracy between the proposed and traditional method

By comparing Tables 1 and 3, the average values of TRR, FAR, and FRR for the proposed method are 97.2, 12.3, and 0, respectively, while those for the traditional method are 82.5, 51.2, and 9.7. We can see that the performance of the proposed method is advanced and much better than that of the traditional method. In Table 3, high FAR and FRR indicate that it is difficult to identify the hole is clean or dirty using only the parameters of area and perimeter. In actual inspection, high FRR will seriously reduce the work efficiency and high FAR will significantly decrease the product quality. Under the guarantee that FRR is zero, several holes with dirt within the critical range can be ignored. Hence, our proposed method can meet the requirements for cleanliness inspection in factory.

Application for complex-shaped spinneret holes

Here two types of complex-shaped spinneret holes were collected to verify the performance of the proposed method. One is 3C-shaped holes with three unconnected C-shaped holes, and anther is caterpillar-shaped holes, which are composed of continuous arcs and lines. The results of their cleanness inspection were shown in Figures 16 and 17. We can see that, the models of CPCM were correctly reconstructed and aligned to the contour curves of dirty holes, and the tiny dirt was located based on the dirt detection curve and their cleanliness index were given. Since most of the spinneret hole shapes are not more complex than those listed in this paper, it can be inferred that the proposed method can meet the requirements of the cleanliness detection for vast majority of irregular-shaped spinneret holes.

Figure 16.

The cleanliness inspection for 3C-shaped spinneret holes: (a)–(d) correspond to the dirty hole image, the aligned CPCM, the dirt detection curve and labeled cleanliness index.

Figure 17.

The cleanliness inspection for caterpillar-shaped spinneret holes: (a)–(d) correspond to the dirty hole image, the aligned CPCM, the dirt detection curve and labeled cleanliness index.

Conclusion

In this paper, we proposed a method to improve the accuracy of the cleanliness inspection. This method generated a closed piecewise curve model and aligned it to the contour curve of the hole. The cleanliness index was calculated from the distance curve representing the distribution of dirt by the model of dirt segmentation and parameters definition. The outcomes of this research work are as follows:

(a) Four types of small image databases are collected by our acquisition system;

(b) Our experiment produces good performance in cleanliness inspection of spinneret holes;

(c) For round and special-shaped holes, we achieve the accuracy of 0% for FRR and 12.3% for FAR, which provide a much better performance than the traditional method;

(d) Moreover, our method can be extended to the cleanliness inspection for the most shapes of holes and the geometry parameters of hole can be calculated in terms of its closed piecewise curve model in the future.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Shanghai Municipal Science and Technology Commission under Grant No. 18030501400.

ORCID iD

Xu Zengbo

References

Cao

Ruan

Peng

, et al. Large-complex-surface defect detection by hybrid gradient threshold segmentation and image registration. IEEE Access 2018; 6: 36235–36246.

Liu

Wang

, et al. Multistage GAN for fabric defect detection. IEEE Trans Image Process 2020; 29: 3388–3400.

Huang

Wang

, et al. Unsupervised fabric defect detection based on a deep convolutional generative adversarial network. Text Res J 2020; 90(3–4): 247–270.

HS.

Robust image retrieval for lacy and embroidered fabric. Text Res J 2019; 89(13): 2616–2625.

Wang

Ouyang

Gao

, et al. Instrumental evaluation of fabric abrasive wear using 3D surface images. J Text Inst 2017; 108(5): 846–851.

Memarian

Amani-Tehran

Latifi

Rank ordering and image processing methods aided fabric wrinkle evaluation. Fibers Polym 2011; 12(6): 830–835.

Jin

Wang

FM.

Measurement of short fiber contents in raw cotton using dual-beard images. Text Res J 2018; 88(1): 14–26.

Yang

Lin

Chen

WJ.

Using deep principal components analysis-based neural networks for fabric pilling classification. Electronics 2019; 8(5):474.

Yin

Yang

Zhang

Self-correcting position control system by machine vision in spinneret’s auto inspection. J Donghua Univ Nat Sci 2007; 33(1): 117–119.

10.

Tan

ZY.

Research on automated technologies inspecting and machining spinneret for melt-spun nonwoven. Shanghai: Donghua University, 2009, pp.1–3.

11.

Komatsu

Tsubusaki

Motoki

, et al. A fully automated measurement of a hole pattern using image processing technique. J Electron Microsc 1998; 47(5): 471–476.

12.

Nagasu

Eguchi

Itoh

, et al. Micro-hole inspection system using low-frequency sound. J Jpn Soc Exp Mech 2014; 14: 105–109 (special issue).

13.

Cai

Xiong

Liu

Research and development of micro-hole quality detecting system based on machine vision. Adv Mat Res 2013; 694–697: 1902–1905.

14.

Zhang

Feng

WC.

Detecting algorithm of spinneret hole image. J Jilin Univ Sci Ed 2010; 48(5): 835–839.

15.

Yang

Cui

. Study on the micro-measure technology of continuous automatic measurement of spinneret. In: International Conference on Machine Vision and Human-machine Interface, Kaifeng, China, 24–25 April 2010, pp.744–747. New York: IEEE.

16.

Yang

Cui

. Analysis of factors influencing measurement accuracy of spinneret automatic inspection system. In: International Conference on Machine Vision and Human-machine Interface, Kaifeng, China, 24–25 April 2010, pp.722–724. New York: IEEE.

17.

Yang

Cui

Study on the method of automatic inspection the large spinneret. In: International Conference on Electrical and Control Engineering, Wuhan, China, 25–27 June 2010, pp.2162–2165. New York: IEEE.

18.

Tao

Liu

CL.

Route planning for concentric spinneret inspection based on improved ant colony algorithm. J Text Res 2017; 38(11): 124–130.

19.

Wang

Lin

Feng

, et al. Detection method for non-circular spinneret hole. J Jilin Univ Sci Ed 2012; 50(4): 762–766.

20.

Chen

Jywe

Hung

, et al. The development of a low cost digital spinneret inspection system. In: IEEE International Instrument & Measurement Technology Conference Proceedings, Austin, TX, 3–6 May 2010, pp.1003–1007. New York: IEEE.

21.

Chen

Huang

Chang

, et al. The development of inclination measurement by using the imaging based autofocus method. Phys Procedia 2011; 19: 43–48.

22.

Chen

Chang

Hung

, et al. Developing a fast inspection path generation method for an automatic spinneret inspection system. In: IEEE International Instrumentation and Measurement Technology Conference Proceedings, Graz, Austria, 13–16 May 2012, pp.1452–1455. New York: IEEE.

23.

Chen

Hung

Jywe

, et al. The development of an automatic scanning path generation method for the spinneret test. SPIE Symposium on SPIE Optical Engineering + Applications 2010.

24.

Canny

A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986; 8(6): 679–698.

25.

Rafael

Richard

Digital image processing. 3rd ed. Upper Saddle River, NJ: Prentice-Hall, Inc., 2008.

26.

Gary

Adrian

Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., 2008.

27.

Harrington

Computer graphics-a programming approach. New York, NY: McGraw-Hill Book Company, 1987.

28.

McLachlan

David

Finite mixture models. Hoboken, NJ: John Wiley & Sons, Inc., 2000.

A method for inspecting the cleanliness of spinneret holes

Abstract

Keywords

Introduction

Image acquisition

Method of cleanliness inspection

Closed contour curve reconstruction

Model generation

Calculation of the curvature

Closed piecewise curve model with parameterized control points

Non-rigid registration of the model

The distance curve

The dirt detection curve

Dirt segmentation and cleanliness inspection

Results and discussion

Distributions of aligned contour curve of clean holes

Optimization for D t h r and P t h r

Analysis of the dirt inspection

Identification based on the traditional method

Comparison of the accuracy between the proposed and traditional method

Application for complex-shaped spinneret holes

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References

Optimization for $D_{t h r}$ and $P_{t h r}$