Fabric defect fetection via weighted low-rank decomposition and Laplacian regularization

Abstract

Low-rank decomposition models have potential for fabric defect detection, where a feature matrix is decomposed into a low-rank matrix that corresponding to repeated texture structure and a sparse matrix that represent defective regions. Two limitations, however, still exist. First, previous work might fail to detect some large homogeneous defective block. Second, when the background and defective regions are relatively coherent or the texture of fabric image is complex, it is difficult to use previous methods to separate them. To deal with these problems, a new weighted low-rank decomposition model with Laplace regularization (WLRL) is proposed in this paper: (1) a weighted low-rank decomposition model that can decompose the original image into background and defective regions, and (2) a Laplace regularization that can enlarge the distance between the background and the defective regions. The performance of the proposed method WLRL is evaluated on the box- and star-patterned fabric databases, and superior results are shown compared with state-of-the-art methods, that is, 98.70% ACC (accuracy) and 72.83% TPR (true positive rate) for box-patterned fabrics, 99.09% ACC (accuracy) and 83.63% TPR (true positive rate) for star-patterned fabrics.

Introduction

In the fabric industry, fabric production is usually done on weaving and knitting machines. ¹ Therefore, this process always produces various defects, such as oil stain and knotting. Fabric defects are one of the main factors affecting the quality of fabrics. Considering that the loss of fabric profits caused by defects can reach 45%: 65%, ² defect detection is a necessary step in fabric production. At present, defect detection mainly depends on experienced skilled workers, but this method is too subjective and inefficient. Automated visual inspection of fabrics can make up for these shortcomings. ³ Hence, the research on automatic fabric defect detection has far-reaching significance.

Generally, fabric defect detection can be classified into two main categories: automatic defect detection of un-patterned fabrics (e.g. plain and twill fabrics), and automatic defect detection of patterned fabrics (e.g. lattice fabric). There is a line of researches on un-patterned fabrics defect detection, mainly contain Fourier transforms,^4,5 Gabor filter,^6,7 Markov random field,^8,9 wavelet transform,^10,11 support vector machine, ¹² and neural network.^13,14

Patterned fabrics refer to the images of texture units with periodic variations. In 2017, Ng et al. ¹⁵ defined the most basic unit of fabric texture as motif, and the combination of motifs can form a lattice which appears repetitively in patterned fabrics, as shown in Figure 1. Different types of fabrics (e.g. box- and star-patterned) have different sizes and shapes of the lattices. At present, there are many corresponding detection algorithms. For example, in 2005, Ngan et al. ¹⁶ proposed the wavelet preprocessed golden image subtraction (WGIS), but WGIS can only detect large defect blocks, which is invalid for small defects. Tang et al. ¹⁷ proposed the Elo Rating (ER) algorithm in 2016, which is a defect detection method based on the spirit of sportsmanship that is, fair matches between different image blocks. In 2017, Jia et al. ¹⁸ proposed a fabric defect detection method based on Gabor filter, which uses morphological component analysis (MCA) to segment the lattice automatically, and then transform the defect detection problem into lattice similarity problem. This method has achieved superior results, but for different types of defects, the adaptability needs to be further improved. In 2017, Yapi et al. ¹⁹ proposed a learning-based fabric detection method, and the experiment on the standard database showed that the method achieved good detection results. Wang et al. ²⁰ proposed a method of fabric defect detection based on deep feature and low-rank recovery in 2018. The experiment shows that this method has a good detection effect. In 2018, Yi et al. ²¹ proposed a fabric defect detection model based on distance matching function and optimal Gabor filter bank. Experimental results show that the method has short learning time, good robustness, and can quickly and accurately detect fabric defects. In 2020, liu et al. ²² proposed a fabric defect detection method based on deep-feature and low-rank decomposition model. The experimental results show that the low-rank decomposition model can effectively separate defects from the background, and the method has a high detection accuracy.

Figure 1.

Structural analysis of patterned fabrics.

The low-rank decomposition (LR) model can decompose the original image into low-rank components that represent the background of the image and sparse components that corresponding the defective regions. Gao et al. ²³ combine GHOG (Gabor-HOG) with low-rank recovery to detect fabric defects. First, extract the GHOG feature of the image, and then use the low-rank recovery model to detect the defects. PN-RPCA ²⁴ proposed a defect prior to guide the low-rank decomposition model and adds noise terms to reduce the influence of noise. Generally, these LR-based (low-rank decomposition model) defect detection methods assume that a fabric image can be represented as a combination of high visual redundancy part (e.g. background regions) and a sparse part (e.g. defective regions).

Although the previous defect detection methods have achieved promising results, there still exist several problems: (1) previous studies might fail to detect some large homogeneous defective block; and (2) when the background and defective regions are relatively coherent or the texture of fabric image is complex, the previous methods are difficult to separate them. In order to overcome these problems, a new weighted LR model with Laplacian regularization (WLRL) is proposed in this paper which is inspired by the LSG ¹⁸ and SMD. ²⁵ We have enhanced the traditional low-rank decomposition model ²⁶ with two important components.

First, we get more accurate defect prior by lattice segmentation, and construct weighted low-rank decomposition model by defect prior. Second, in order to enhance the detection efficiency, we add Laplacian regularization ²⁵ to the weighted LR model, which is used to enlarge the distance between the background and the defective regions. The regularizer encourages blocks with similar pixels to share similar representations and eventually separates salient regions from the background as much as possible. These attributes enable the proposed WLRL method to exhibit excellent performance in defect detection. In summary, this paper makes the following contributions:

1. By combining the properties of the two techniques, a weighted low-rank decomposition with Laplacian regularization (WLRL) is proposed for lattice-based fabric defect detection.

2. In constructing the weighed low-rank decomposition model the defect prior is introduced to obtain more accurate defects.

3. In the weighted low-rank decomposition model, a Laplacian regularization term is added to enlarge the distance between the background and the defective regions, which makes the proposed method more accurate and robust.

Related work

Lattice segmentation

Lattice segmentation can be classified into two categories based on computational methods ²⁷ : the local feature-based and the global structure-based methods. The former analyzes local features first and then global structure, while the latter is the opposite. Liu et al. ²⁸ perform lattice segmentation based on local features, which has a better detection result for simple textures, but it is difficult to detect complex textures. Chang et al. ²⁹ used the global variance minimization to perform lattice segmentation. First, an ideal lattice size is assumed, and then the sum of the variances of these lattices is calculated. When the sum is approaching the smallest, the optimal lattice size is obtained. Jia et al. ¹⁸ propose a new method, which can perform lattice segmentation without obtaining the corresponding lattice prior knowledge. This paper uses this method for lattice segmentation.

Low-rank decomposition

Candès et al. ²⁶ first proposed the low-rank decomposition model (LR). According to different applications, the low-rank decomposition model can be called robust principal component analysis (RPCA) or robust subspace learning. ³⁰ Bouwmans et al. ³¹ and Lin et al. ³² analyzed in detail the evolution process of LR model and the optimized solution method. Given a feature matrix $F$ of the input image, the LR model can decompose it into a low-rank matrix $B$ (representing the background part of the image) and a sparse matrix $S$ (representing the significant foreground area of the image). Therefore, fabric defect detection can be described as the following questions:

$$\underset{B,S}{\min }({\Vert B\Vert }_{*}+\lambda {\Vert S\Vert }_1)s.t.F=B+S$$

(1)

where the nuclear norm ${\Vert \cdot \Vert }_{*}$ (sum of all singular values in the matrix) is the convex relaxation of the matrix rank function; ${\Vert \cdot \Vert }_1$ represents the $l_1$ norm of the matrix, which is the maximal value in the sum of the matrix column vectors absolute values; $\lambda >0$ represents the weighted factor, which is used to balance the degree of low-rank and sparsity.

At present, the low-rank decomposition model has been successfully applied in signal processing, computer vision and industrial process. For instance, Pu et al. ³³ designs a new structured matrix decomposition method, which decomposes the data matrix of each mode into the superposition of three terms for blind audio-visual location and separation. S. Javed et al. ³⁴ propose a spatiotemporal structured sparse RPCA algorithm for moving objects detection. Li et al. ³⁵ used the low-rank decomposition model to detect the defects of thin-walled structure. The experiment shows that the method has achieved satisfactory results.

The research on different norms and regularization terms in low-rank decomposition is a very active field and their selection is always driven by the application like in background/foreground separation. On the basis of RPCA, Sobral et al. ³⁶ proposed a double-constrained RPCA model called SCM-RPCA, which is used to improve the target foreground detection in the marine scene. The experimental results show that the method has better detection results. Zhou et al. ³⁷ proposed a new graph-regularized Laplace low-rank approximation detecting model (GRLA) for infrared dim target scene. The kernel norm was replaced by non convex Laplacian low-rank regularization, which improved the accuracy of non-uniform background estimation. Shijila et al. ³⁸ used convergence convex optimization algorithm to regularize the total variation (TV) norm in the low-rank decomposition model, which reduced the computational complexity.

The proposed WLRL method

The proposed WLRL method consists of two parts: the acquisition of defect prior and the low-rank decomposition model. First, the defect prior (approximate defective regions) is obtained by lattice segmentation and added into the low-rank decomposition model to construct the weighted low-rank decomposition model. Second, adding Laplacian regularization to enlarge the distance between the backgrounds and defective regions. Last, using the optimal threshold algorithm to segment the saliency map generated by the sparse matrix to locate the defective regions. Figure 2 shows the specific steps.

Figure 2.

The procedure of the proposed WLRL method.

Acquisition of defect prior

One disadvantage of LR model is that it may fail to detect some large homogeneous defective block. In order to avoid this error and improve the detection effect of LR, the defect prior will be introduced in the paper. Defect prior is a penalty matrix used to guide LR, which can improve the detection probability of defects and reduce the detection probability of defect-free regions. The acquisition of defect prior is generally divided into two stages: training stage and testing stage.

Training stage: First, the defect-free fabrics is segmented by lattice segmentation ¹⁸ according to the period size. Then each lattice covers a rectangular area of the original image, and each lattice is a sub-image of the original fabric image. Therefore, some traditional feature extraction^39,40 methods can be used for lattice feature extraction. Here Gabor filter is used for feature extraction. The predefined Gabor filter banks of 2D Gabor transform function are as follows.

$$\begin{array}{l}g\left(x,y\right)=\frac{1}{2{\sigma }_x{\sigma }_y}\exp \left(-\frac{1}{2}\left(\frac{x_{\theta }^2}{{\sigma }_x}+\frac{y_{\theta }^2}{{\sigma }_y}\right)\right)\\ \exp (2\pi i\cdot g_0{x}_{\theta })\end{array}$$

(2)

$$\left[\begin{array}{c}{x}_{\theta }\\ y_{\theta }\end{array}\right]=\left[\begin{array}{cc}cos\theta & -sin\theta \\ sin\theta & cos\theta \end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]$$

(3)

The parameters ${\sigma }_x$ and ${\sigma }_y$ represent the shape factor of the Gauss surface; $\theta$ denotes directions and $g_0$ denotes center frequency. The values of these parameters determine the quality of the filter. In addition, these values are predefined ⁴⁰ and optimized ⁴¹ in fabric defect detection. In this paper, the parameters $\theta$ are defined as 0, π/4, π/2, and 3 π/4.Then the Chebychev distance between different lattices is calculated, and a distance matrix $\mathit{D}$ is constructed. The average distance $\mathit{d}$ is calculated and put into the testing stage as a parameter. Specific details can refer to. ¹⁸

Testing stage: In the training phase, the distance between feature blocks are used to represent the similarity between lattices. Repeat the steps mentioned above to get a feature distance matrix. When the distance is greater than $\mathit{d}$ , it is marked as a defective lattice. Otherwise, it is a defect-free lattice. The steps are summarized in Figure 3.

Figure 3.

The procedure of acquiring the fabric defect prior.

Weighted LR and laplacian regularization

In order to improve the detection accuracy of the LR model for different defect types, based on LR, a weight matrix is added to guide the decomposition process, which makes the model more accurate and robust. Meanwhile, it can avoid failure when the previous studies might fail to detect some large homogeneous defective block. In addition, in order to make the model adapt to different types of defects, we add Laplacian regularization to enlarge the distance between the backgrounds and the defects.

WLR model

Considering that LR may fail to detect some large homogeneous defective block. Therefore, a valid weighting matrix $\mathit{W}$ is constructed and introduced into LR to guide low-rank decomposition. A weighted low-rank decomposition model (WLR) is described as follows:

$$\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_{1}s.t.F=B+S$$

(4)

where the weight matrix $\mathit{W}$ = exp(- $\mathit{P}$ ) and $\mathit{P}$ represents the defect prior. A larger $\mathit{P}(i,j)$ value indicates $\mathit{F}(i,j)$ has a high probability to be a defective pixel. The augmented Lagrange function is defined as follows:

$$\begin{array}{l}ℒ(B,S,Y,\mu )={\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_1\\ +\mathrm{Tr}(Y(F-B-S))+\frac{\mu }{2}{\Vert F-B-S\Vert }_F^2\end{array}$$

(5)

where $\mathit{Y}$ is the Lagrange multiplier; Tr (⋅) represents the trace of the matrix; µ > 0 is a constant, which is the penalty for the violation of the linear constraint; and ${\Vert \cdot \Vert }_F$ denotes the Frobenius norm of a matrix.

Laplacian regularization

When the difference between the backgrounds and defects is not obvious or the texture of fabric image is complex, the backgrounds will be highly correlated with the defects. Therefore, the previous WLR-based method will be difficult to work facing this situation. To solve this problem, a Laplacian regularization ²⁵ is introduced to enlarge the distance between the backgrounds and the defects, so as to distinguish the defective regions from the background. We define Laplacian regularization as follows:

$$\Theta (B,S)=\frac{1}{2}\sum _{i,j}^N{\Vert S_i-S_j\Vert }_2^2{a}_{i.j}=\mathrm{Tr}(S{M}_F{S}^T)$$

(6)

where $\Theta (B,S)$ denotes the interactive regularization term used to increase the distance between subspaces $\mathit{B}$ and $\mathit{S}$ ; ${\mathit{S}}_i$ denotes the $i$ -th column of matrix $\mathit{S}$ ; $a_{i,j}$ denotes the $(i,j)$ )-th entry of an affinity matrix $A=(a_{i,j})\in {ℝ}^{N\times N}$ , which represents the feature similarity of image patches; and $M_F\in {ℝ}^{N\times N}$ denotes a Laplacian matrix. The affinity matrix $A$ is defined as follows:

$$a_{i,j}=\{\begin{array}{l}\exp \left(-\frac{\Vert f_i-f_j^2\Vert }{2{\sigma }^2}\right),if\left(I_i,I_j\right)\in \mathbb{V}\\ 0,\mathrm{otherwise}\end{array}$$

(7)

where $\mathbb{V}$ represents the set of adjacent block pairs which are either neighbors (first-order) or “neighbors of neighbors” (second-order reachable) on the image. The ( $i$ $(i,j)$ )-th entry of the Laplacian matrix $M_F$ is defined as:

$${(M_F)}_{i,j}=\{\begin{array}{l}-a_{i,j},\mathrm{if}i\ne j\\ {\sum }_{j\ne i}{a}_{i,j},\mathrm{otherwise}\end{array}$$

(8)

Essentially, Laplacian regularization enlarges the distance between feature subspaces by smoothing the vectors in $\mathit{S}$ , which is according to the local neighborhood derived from the feature matrix $\mathit{F}$ . This method encourages blocks with similar pixels to be represented in the same way and blocks with different pixels to be represented in different ways.

WLRL model

Based on the WLR, Laplacian regularization is added to construct the WLRL model. The model is defined as follows:

$$\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_1+\beta \Theta (B,S)\mathrm{s}.\mathrm{t}.F=B+S$$

(9)

where the first term is a low-rank restriction, which is used to represent the background of fabrics. The second term is a sparse restriction, which is used to indicate the defective regions of fabrics. The third term is Laplacian regularization to enlarge the distance between the low-rank part and the sparse part, and $\lambda$ , $\beta$ are positive trade-off parameters represent the degree of sparsity and Laplacian interactive regularization terms, respectively. $\Theta (\cdot )$ represents the Laplacian interactive regularization term.

In order to facilitate the solution, the auxiliary variable $\mathit{H}$ is introduced to separate the objective function. Equation 9 can be described as follows:

$$\begin{array}{l}\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_1+\beta Tr(H{M}_F{H}^T)\\ s.t.F=B+S,S=H\end{array}$$

(10)

Optimization

Considering the efficiency and accuracy of the algorithm, the ADM ⁴² is used to solve the convex optimization model WLRL, and the minimum augmented Lagrange function is:

$$\begin{array}{l}ℒ(B,S,H,Y_1,Y_2,\mu )={\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_1+\beta \mathrm{Tr}(H{M}_F{H}^T)\\ +\mathrm{Tr}(Y_1^T(F-B-S))+\mathrm{Tr}(Y_2^T(S-H))\\ +\frac{\mu }{2}({\Vert F-B-S\Vert }_F^2+{\Vert S-H\Vert }_F^2)\end{array}$$

(11)

where ${\mathit{Y}}_1$ and ${\mathit{Y}}_2$ are the Lagrange multipliers, µ > 0 is a constant, which is the penalty for the violation of the linear constraint. In order to solve the Eq. 11, the optimal $\mathit{B}$ , $\mathit{S}$ and $\mathit{H}$ are searched in the paper iteratively. The solution process is summarized as Alg. 1 and is termed as ADM-WLRL. Details of the iteration are as follows.

Algorithm 1: ADM-WLRL

Input: feature matrix $\mathit{F}$ , defect prior $\mathit{W}$ , Laplacian matrix ${\mathit{M}}_F$ and parameters. Output: $\mathit{B},\mathit{S}$ . 1 Initialize B 0 = 0, H0 = 0, Y10 = 0, Y20 = 0, µ0 = 0 and $k=0$ . 2 while not converged( $k$ =0,1) do 3  update $\mathit{B}$ and fix the others by   $B_{k+1}=\underset{B}{\arg \min }ℒ(B,S_k,H_k,Y_1^k,Y_2^k,{\mu }_k)$ 4  update $\mathit{H}$ and fix the others by   $H_{k+1}=\underset{H}{\arg \min }ℒ(B_{k+1},S_k,H,Y_1^k,Y_2^k,{\mu }_k)$ 5  update $S$ and fix the others by   $S_{k+1}=\underset{S}{\arg \min }ℒ(B_{k+1},S,H_{k+1},Y_1^k,Y_2^k,{\mu }_k)$ 6  update the Lagrange multiplier by Eq. (19) and check the con-vergence conditions $F-B_{k+1}-S_{k+1}<\epsilon$ , $S_{k+1}-H_{k+1}<\epsilon$ 7  update the parameter µ by ${\mu }_{k+1}=\min (\rho {\mu }_k,{\mu }_{\max })$ 8  update the iterations $k$ by $k=k+1$ 9 return ${\mathit{B}}_k,{\mathit{S}}_k$ .

Updating $\mathit{B}$ : fix the others and update $\mathit{B}$ by solving the following problem:

$$\begin{array}{l}{B}_{k+1}=\underset{B}{\arg \min }ℒ(B,S_k,H_k,Y_1^k,Y_2^k,{\mu }_k)\\ =\underset{B}{\arg \min }{\Vert B\Vert }_{*}+\mathrm{Tr}((Y_1^k{)}^T(F-B-S_k))\\ +\frac{{\mu }_k}{2}{\Vert F-B-S_k\Vert }_F^2\\ =\underset{B}{\arg \min }\frac{1}{{\mu }_k}{\Vert B\Vert }_{*}+\frac{1}{2}{\Vert B-X_B\Vert }_F^2\end{array}$$

(12)

where $X_B=F-S_k+Y_1^k/{\mu }_k$ . Eq. 12 can be described as:

$$B_{k+1}=U{T}_{1/{\mu }_k}[\Sigma ]V^T,\mathrm{where}(U,\Sigma ,V^T)=svd(X_B)$$

(13)

Note that $T[\cdot ]$ presents Singular Value Thresholding (SVT) ⁴³ operator and $\Sigma$ is the singular value of the matrix of $X_S$ .

Updating $\mathit{H}$ : fix the others and update by solving the following problem:

$$\begin{array}{l}{H}_{k+1}=\underset{H}{\arg \min }ℒ(B_{k+1},S_k,H,Y_1^k,Y_2^k,{\mu }_k)\\ =\underset{H}{\arg \min }\beta \mathrm{Tr}(H{M}_F{H}^T)\\ +\mathrm{Tr}((Y_2^k{)}^T(S_k-H))+\frac{{\mu }_k}{2}{\Vert S_k-H\Vert }_F^2\end{array}$$

(14)

Updating $\mathit{S}$ : fix the others and update $\mathit{S}$ by solving the following problem:

$$\begin{array}{l}{S}_{k+1}=\underset{S}{\arg \min }ℒ(B_{k+1},S,H_{k+1},Y_1^k,Y_2^k,{\mu }_k)\\ =\underset{S}{\arg \min }\lambda \Vert W\cdot S\Vert +\mathrm{Tr}((Y_1^k{)}^T(F-B_{k+1}-S))\\ +\mathrm{Tr}((Y_2^k{)}^T(B-H_{k+1}))\\ +\frac{{\mu }_k}{2}(\Vert F-B_{k+1}-S\Vert +{\Vert S-H_{k+1}\Vert }_F^2)\\ =\underset{S}{\arg \min }\frac{\lambda }{2{\mu }_k}\Vert W\cdot S\Vert +\frac{1}{2}{\Vert S-X_S\Vert }_F^2\end{array}$$

(15)

where $X_s=F-B_{k+1}+H_{k+1}+(Y_1^k+Y_2^k)/{\mu }_k/2$ .

The closed solution of $S_{k+1}$ is:

$$S_{k+1}=\mathrm{shrink}(X_S,\lambda /{\mu }_k)W^{-1}$$

(16)

where shrink(⋅) is a soft threshold operator, expressed as:

$$\mathrm{shrink}(\eta ,x)=sign(x)max\{abs(x)-\eta ,0\}$$

(17)

Updating ${\mathit{Y}}_1$ and ${\mathit{Y}}_2$ : update the Lagrange multipliers ${\mathit{Y}}_1$ and ${\mathit{Y}}_2$ :

$$\begin{array}{l}{Y}_1^{k+1}=Y_1^k+{\mu }_k(F-B_{k+1}-S_{k+1})\\ Y_2^{k+1}=Y_2^k+{\mu }_k(S_{k+1}-H_{k+1})\end{array}$$

(18)

WLRL-based fabric defect detection

By explaining the optimization process in the previous section, the WLRL-based defect detection method can be defined now. The framework for patterned fabric defect detection consists of four steps: image representation, defect prior, low-rank decomposition and generating a saliency map and its segmentation.

Step 1: Image representation

First, Gabor filter is used to preprocess the pattern fabric image to generate Gabor maps. Then the image is segmented into $N$ blocks by using the simple liner iterative clustering (SLIC) ⁴⁴ algorithm, which is expressed as $P=\{P_1,P_2,\dots ,P_N\}$ . If each feature block $P_i$ is represented by the feature vector $f_i$ , the original image can be represented as a feature matrix $F=\{f_1,f_2,\dots ,f_N\}$ , where $F\in {ℝ}^{M\times N}$ , $M$ is the dimension of the features (here $M$ = 53).

Step 2: Defect prior

First, the fabric image is segmented according to the period size by using lattice segmentation. Then, Gabor features of each lattice are extracted, and the Chebyshev distance between each lattice is calculated. By comparing the lattices, the defect prior is obtained, as shown in Figure 3. By adding defect prior, the model has a higher detection rate and robustness.

Step 3: Low-rank decomposition

When $\mathit{W}$ is ready, we use the MATLAB syntax repmat (W,2,M) to perform a dimension transformation to make it consistent with the dimension of $\mathit{F}$ . Then the WLRL model (Eq.(10)) is used to decompose $\mathit{F}$ into a low-rank matrix $\mathit{B}$ and a sparse matrix $\mathit{S}$ . The specific steps are shown in Figure 2.

Step 4: Generating a saliency map and its segmentation

The feature matrix $\mathit{F}$ of fabric image is decomposed into low-rank matrix $\mathit{B}$ corresponding to fabric background and sparse matrix $\mathit{S}$ corresponding to defective regions by the WLRL model. The saliency of each image block is expressed by calculating the $l_1$ norm of each column ${\mathit{S}}_i$ in $\mathit{S}$ , It can be expressed as $\mathrm{Sal}(I_i)={\Vert S_i\Vert }_1$ . The obtained largest Sal( $I_i$ ) indicates that this block is most likely to contain defects. Then denoting $\mathit{S}$ to produce a new saliency map $\widehat{\mathit{S}}$ , $\widehat{S}=g*(S\circ S)$ , where g denotes the Gauss smoothing filter, “∘” denotes Hadamard inner product operator, and “∗” denotes the convolution operation.

Then the saliency map $\widehat{\mathit{S}}$ is transformed into gray image $\mathit{G}$ :

$$G=\frac{\widehat{S}-\min (\widehat{S})}{\max (\widehat{S})-\min (S)}\times 255$$

(19)

Finally, the improved adaptive threshold segmentation algorithm ⁴⁵ is used to segment G, and the final binary detection results are obtained.

Experiments and performance evaluation

To fully evaluate the proposed method, a series of experiments are conducted by using standard datasets, including four state-of-the-art solutions for comparison. In particular, the dataset is provided by the Industrial Automation Research Laboratory from Dept. of Electrical and Electronic Engineering of Hong Kong University, which includes 106 standard datasets of box- and star-patterned, with a size of 256-by-256 in 24-bit depth. The box-patterned database has 30 defect-free images and 26 defective images. The star-patterned database has 25 defect-free images and 25 defective images. There are five types of defects in all the defective images, including Broken end, Hole, Netting multiple, Thick bar and Thin bar. All defective images have corresponding binary ground-truth images (manually labeled images) which has value 1 and 0 corresponding to the defects and background, respectively. The comparison methods in this paper include ER ¹⁷ (2016), PN-RPCA ²⁴ (2016), GHOG-LR ²³ (2016) and LSG ¹⁸ (2017).

In order to further evaluate the effectiveness and applicability of the proposed method, some evaluation indexes are defined, accuracy (ACC), true positive rate (TPR), false positive rate (FPR), positive predictive value (PPV) and negative predictive value (NPV). These five indicators are used as the standard for the evaluation of the detection algorithm. The details of these metrics are provided in. ¹⁷ All the experiments are conducted on a notebook computer with an Intel Core i5-3320 processor and 6GB memory. The algorithm runs in the environment of MATLAB 2016a.

Experimental results and analysis

Component analysis in the proposed model

In order to further analyze the role of the various components of the WLRL model, we compared LR, WLR and WLRL on the same data set.The corresponding objective functions are shown in Table 1. Figure 4 shows the test results of some LR, WLR and WLRL. The 1st row displays original defective images; the 2nd row is corresponding ground-truth (GT) images; the 3rd row displays the results of the LR method; the 4th row exhibits the results of the WLR method; the 5th row shows the WLRL detection results.

Table 1.

Objective functions of different models related to WLRL.

Method	Object function
LR	$\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert S\Vert }_{1}s.t.F=B+S$
WLR	$\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_{1}s.t.F=B+S$
WLRL	$\underset{B,S}{\min }{\Vert B\Vert }_{*}+\lambda {\Vert W\cdot S\Vert }_1+\beta Tr(H{M}_F{H}^T)$ $s.t.F=B+S,S=H$

Figure 4.

Comparative experimental results of LR, WLR, and WLRL on box and star samples.

It can be seen from Figure 4 that WLR increases the detection rate of large homogeneous defects, indicating that the defect prior plays an important role in ensuring the detection of large homogeneous defects. Compared with WLR, the detection effect of WLRL is further improved, and the detection results of small defects are increased, which confirms the effect of Laplace regular term. Compared with LR, WLRL has a significant improvement in detection results. For the fabric images whose foreground and background share similar appearance, WLRL successfully separates the defects from the background, while LR often fail. These results illustrate the robustness of the WLRL model, and confirm the effectiveness of the proposed weighted low-rank decomposition with Laplacian regularization. From the figure we can conclude that (1) through the defect prior, the detection rate of large homogeneous defect regions can be increased, and (2) after adding the Laplace regular term, the experimental results show that the detection results Significant improvement.

Tables 2 and 3 are the comparison table of LR and WLRL test results on box- and star-patterned fabrics. It can be seen from Table 2 that WLRL has the best detection effect, that is, with overall highest ACC and highest TPR. And the total TPR increased by more than 20%, but at the same time, the FPR is inevitably slightly improved. It can be seen from Table 3 that the detection effect of WLRL has been greatly improved, especially for Thin Bar defect type, because WLRL has higher sensitivity to small and fine defects. In particular, the experimental results show that the WLRL can make up for the lack of LR and improve the detection accuracy.

Table 2.

The numerical comparison between LR and WLRL on box-patterned fabric.

Box	ACC	TPR	FPR	PPV	NPV	Method
Broken end(5)	98.13	66.67	1.53	41.74	99.22	LR
	99.21	70.15	1.42	48.90	99.36	WLRL
Hole(5)	98.36	85.32	1.12	32.95	99.87	LR
	98.54	89.61	1.38	33.29	99.91	WLRL
Netting multiple(5)	97.70	53.78	1.72	26.57	99.39	LR
	98.65	73.44	2.03	29.38	99.65	WLRL
Thick bar(6)	96.7	19.94	1.00	31.14	97.63	LR
	98.12	78.85	1.93	48.94	99.66	WLRL
Thin bar(5)	97.54	38.38	1.89	16.50	99.41	LR
	98.98	52.09	1.62	23.37	99.57	WLRL
Overall (26)	97.68	52.82	1.45	29.78	99.16	LR
	98.70	72.83	1.67	36.78	99.63	WLRL

Table 3.

The numerical comparison between LR and WLRL on star-patterned fabric

Star	ACC	TPR	FPR	PPV	NPV	Method
Broken End(5)	97.01	82.43	2.86	16.23	99.85	LR
	99.35	85.20	2.91	16.72	99.89	WLRL
Hole(5)	98.69	87.14	1.26	29.21	99.94	LR
	99.85	92.97	1.12	31.32	99.89	WLRL
Netting Multiple(5)	97.85	69.93	1.47	44.01	99.27	LR
	98.64	88.87	1.17	53.23	99.78	WLRL
Thick Bar(5)	95.08	38.51	2.28	25.42	97.20	LR
	99.12	87.37	0.74	72.51	99.49	WLRL
Thin Bar(5)	96.30	10.22	2.89	3.74	99.13	LR
	98.51	63.72	2.84	21.84	99.12	WLRL
Overall (25)	96.98	57.65	2.15	23.72	99.07	LR
	99.09	83.63	1.75	39.12	99.56	WLRL

Adjusting the parameters of WLRL method

WLRL uses the weight parameter $\lambda$ to adjust the degree of low rank and sparsity. Figure 5 illustrates the detection result of the weight parameter $\lambda$ effect on the box- and star-patterned fabrics. The box- and star-patterned fabrics have undergone the same evaluation processes with the parameter of 0.02:0.1. Figure 5 clearly indicates that $\lambda$ = 0.06 offers the best detection results among the five choices. When the parameters are too large or too small, it will cause false or missed detection.

Figure 5.

Detection results under different parameters $\lambda$ . The first column corresponds to the defective image, the second column is Ground-truth image, and the other column are the test results under different parameters.

Tables 4 and 5 show the TPR and FPR detection results of box- and star-patterned fabrics with different parameter $\lambda$ . For box-patterned fabrics, when $\lambda$ = 0.06 Hole, Netting Multiple and Thick Bar defect types achieve the highest TPR. Although the $\lambda$ = 0.04, Thin Bar achieves the highest TPR, but lower TPR for other types of defects. The average TPR increases from 61.55% to 72.83% when $\lambda$ rises from 0.02 to 0.06. Therefore, $\lambda$ = 0.06 is an optimal choice for box-patterned fabrics because all TPRs and FPRs begin to stable at the levels of 72.83% and 1.68%. For star-patterned fabrics, when $\lambda$ = 0.06, the highest TPR is obtained for Netting Multiple, Thick Bar and Thin Bar defect types, and the lowest FPR is obtained for Thick Bar. When $\lambda$ = 0.04, Broken End and Hole defect types achieve the highest TPR. Although $\lambda$ = 0.1 generates the lowest average FPR 1.63%, it actually misses many defective regions. In contrast, the metric TPR reveals the performance more accurately: $\lambda$ = 0.06 provides an average TPR of 83.63% for star-patterned fabrics, which is the highest among all parameters.

Table 4.

TPR and FPR w. r. t. different parameters for box-patterned fabric.

Parameter $\lambda$	TPR for star-patterned fabrics					FPR for star-patterned fabrics
	0.02	0.04	0.06	0.08	0.1	0.02	0.04	0.06	0.08	0.1
Broken end	81.34	79.89	70.15	60.81	59.34	1.51	1.53	1.42	1.34	1.34
Hole	82.22	88.79	89.61	87.60	81.03	1.10	1.23	1.38	1.38	1.33
Netting multiple	47.25	69.80	73.44	70.66	64.33	1.84	1.88	2.03	1.97	1.85
Thick bar	57.13	56.63	78.85	56.85	57.01	1.34	1.34	1.93	2.10	2.06
Thin bar	39.81	52.64	52.09	50.77	47.91	2.30	1.95	1.62	1.60	1.55
Average TPR/FPR	61.55	69.55	72.83	65.39	61.92	1.62	1.59	1.68	1.68	1.63

Table 5.

TPR and FPR w. r. t. different parameters for star-patterned fabric.

Parameter $\lambda$	TPR for star-patterned fabrics					FPR for star-patterned fabrics
	0.02	0.04	0.06	0.08	0.1	0.02	0.04	0.06	0.08	0.1
Broken end	81.34	93.83	85.20	85.17	66.05	2.38	2.66	2.91	2.17	1.90
Hole	84.46	93.69	92.97	89.88	81.03	1.26	1.16	1.12	1.07	0.97
Netting multiple	76.05	87.96	88.87	83.41	73.78	0.90	1.16	1.17	1.09	1.03
Thick bar	61.40	80.99	87.37	63.53	63.06	1.99	1.58	0.74	1.57	1.76
Thin bar	30.00	54.53	63.72	52.48	50.63	3.24	2.58	2.84	2.30	2.50
Average TPR/FPR	66.65	82.20	83.63	74.89	66.91	1.95	1.83	1.75	1.64	1.63

Comparison with state-of-the-arts

In this section, we test the WLRL on two categories of defective images: box- and star-patterned fabrics. In order to perform an intensive evaluation and clearly understand how accurate detection of the WLRL, we now compare the WLRL with other comparison methods in all respects (ER, PN-RPCA, GHOG-LR, and LSG).

Experimental results

For each defect type in the fabric image database, a defect image is randomly selected. The results of ER, PN-RPCA, GHOG-LR, LSG, and WLRL are illustrated in Figures 6 and 7. Figure 6 depicts the detection results of defective images chosen from box-patterned fabric database. LSG and WLRL detection results are significantly higher than other methods. PN-RPCA has a good detection result for Thick Bar, but the detection result of Hole is not ideal. In detail, GHOG-LR can detect the defective regions roughly but causes a large number of false detection. The main reason is that GHOG feature is not sensitive to box-patterned fabrics. In contrast, WLRL has strong adaptability to different types of defects and can accurately locate the defective regions.

Figure 6.

Box-patterned fabric:Defective sample names (first row), testing images (second row), ground-truth(third row), detected results by ER (fourth row), PN-RPCA (fifth row), GHOG-LR (sixth row), LSG (seventh row), and our proposed WLRL (eighth row).

Figure 7.

Star-patterned fabric:Defective sample names (first row), testing images (second row), ground-truth (third row), detected results by ER (fourth row), PN-RPCA (fifth row), GHOG-LR (sixth row), LSG (seventh row), and our proposed WLRL (eighth row).

Figure 7 depicts the detection results of defective images chosen from the star-patterned fabric database. All methods except ER show relatively accurate detection results, especially WLRL. Although GHOG-LR cannot accurately detect defects of Thin Bar type, it can roughly locate defects of other types, and its performance on star-patterned fabrics is better than box-patterned fabrics. Generally, WLRL shows accurate detection results compared with other methods.

Tables 6 and 7 tabulate the numerical results of each defect type in box- and star-patterned fabrics. The results of the recent fabric detection method ER, PN-RPCA, GHOG-LR, and LSG are compared with the WLRL.

Table 6.

Numerical results for each defect type of box-patterned fabric.

Box pattern	ACC	TPR	FPR	PPV	NPV	Method
Broken end (5)	98.23	10.26	0.69	30.43	95.83	ER
	98.21	69.04	1.19	54.51	99.26	PN-RPCA
	93.49	26.25	5.28	11.37	98.58	GHOG-LR
	99.23	63.16	0.78	89.02	99.34	LSG
	99.21	70.15	1.42	48.9	99.36	WLRL
Hole(5)	99.34	0	0.03	0	97.69	ER
	95.76	54.33	3.89	13.56	99.60	PN-RPCA
	94.21	22.37	5.23	2.74	99.37	GHOG-LR
	98.33	63.27	0.85	73.32	99.26	LSG
	98.54	89.61	1.38	33.29	99.91	WLRL
Netting multiple(5)	99.45	0.15	0.04	4.00	95.81	ER
	95.36	26.70	2.37	17.22	99.11	PN-RPCA
	93.29	26.68	5.88	4.19	99.04	GHOG-LR
	98.43	52.52	0.89	82.11	99.42	LSG
	98.65	73.44	2.03	29.38	99.65	WLRL
Thick bar(6)	98.35	22.76	1.68	42.40	96.93	ER
	97.14	47.68	1.58	59.51	98.64	PN-RPCA
	95.12	39.05	3.52	32.89	98.48	GHOG-LR
	99.34	77.21	0.82	72.23	82.28	LSG
	98.12	78.85	1.93	48.94	99.66	WLRL
Thin bar(5)	99.23	5.84	4.51	2.36	97.68	ER
	96.31	50.19	3.30	18.21	99.56	PN-RPCA
	93.85	54.41	5.78	9.15	99.55	GHOG-LR
	99.54	47.20	1.95	61.32	99.52	LSG
	98.98	52.09	1.62	23.37	99.57	WLRL
Overall(26)	98.92	7.80	1.39	15.84	96.80	ER
	96.55	50.19	2.47	36.36	99.25	PN-RPCA
	93.99	33.95	5.14	9.15	99.55	GHOG-LR
	98.86	60.67	1.06	75.8	96.01	LSG
	98.70	72.83	1.67	36.78	99.63	WLRL

Table 7.

Numerical results for each defect type of star-patterned fabric.

Star pattern	ACC	TPR	FPR	PPV	NPV	Method
Broken end(5)	99.32	8.79	1.16	7.17	99.27	ER
	98.59	76.36	1.11	27.54	99.90	PN-RPCA
	97.40	85.01	2.51	17.73	99.88	GHOG-LR
	99.27	66.35	0.59	66.15	99.82	LSG
	99.35	85.20	2.91	16.72	99.89	WLRL
Hole(5)	99.23	24.47	1.23	11.68	99.54	ER
	97.81	79.86	2.11	31.91	99.90	PN-RPCA
	99.98	95.38	1.70	23.50	99.98	GHOG-LR
	99.78	66.34	0.52	75.41	99.81	LSG
	99.85	92.97	1.12	31.32	99.89	WLRL
Netting multiple(5)	98.24	16.42	0.82	12.61	98.54	ER
	98.06	76.48	1.54	49.37	99.56	PN-RPCA
	99.67	83.74	1.63	45.82	99.67	GHOG-LR
	99.54	50.12	0.34	86.21	99.26	LSG
	98.64	88.87	1.17	53.23	99.78	WLRL
Thick bar(5)	98.53	69.52	1.67	54.52	98.81	ER
	98.12	65.66	0.34	77.49	98.39	PN-RPCA
	97.92	82.36	1.15	60.73	98.97	GHOG-LR
	98.57	77.06	0.79	85.18	99.52	LSG
	99.12	87.37	0.74	72.51	99.49	WLRL
Thin bar(5)	99.27	45.47	2.83	12.50	99.61	ER
	97.36	72.96	1.74	31.34	99.86	PN-RPCA
	97.17	47.41	2.53	16.41	99.50	GHOG-LR
	98.28	47.80	2.37	68.26	99.75	LSG
	98.51	63.72	2.84	21.84	99.12	WLRL
Overall(25)	98.91	32.93	1.54	19.7	99.12	ER
	97.98	74.26	1.55	43.53	99.50	PN-RPCA
	98.42	78.90	1.87	32.83	99.60	GHOG-LR
	99.08	68.49	0.80	76.24	99.63	LSG
	99.09	83.63	1.75	39.12	99.56	WLRL

The WLRL obtains overall results of 98.70% ACC, 72.83% TPR, 1.67% FPR, 36.78% PPV, and 99.63% NPV for the box-patterned fabrics (Table 6). The overall detection effect of WLRL is the best among all methods, that, with overall highest TPR and highest NPV. In all methods, the overall TPR (7.8%) of ER is the lowest, which indicates that it was less adaptable to the box-patterned fabrics. LSG achieves the highest overall PPV (75.8%) and the lowest overall FPR (1.06%), but its overall TPR (60.67%) is lower than WLRL (72.83%). GHOG-LR achieves the highest overall TPR (54.41%) for Thin Bar defect type, but the defect detection results for other types are poor. The ACC of several detection methods is more than 95%. In general, the results show that WLRL has stronger discriminative power to detect defects for box-patterned fabrics.

The WLRL obtains overall results of 99.09% ACC, 83.63% TPR, 1.75% FPR, 39.12% PPV, and 99.56% NPV for the star-patterned fabrics (Table 7). The overall detection effect of WLRL is the best among all methods, that is, with overall highest ACC and TPR. Although for defect type Thin Bar PN-RPCA outperforms WLRL about TPR (72.96%), it has a much worse TPR for other types while WLRL has the highest TPR for other types. LSG is a competitive rival for WLRL, but it has a much worse TPR than WLRL. Generally, the result of WLRL is very satisfactory and encouraging for defect detection on this kind of star-patterned fabrics.

In short, The WLRL compelling performances among all compared methods, both in numerical results and visualization for box-patterned fabrics and star-patterned fabrics.

Running time comparison

The real-time property of fabric algorithm is very important in industrial application. Therefore, the efficiency of the method is tested by using 50 fabric images and the running time is shown in Figure 8 Here x-coordinate represents the index of the fabric images and y- coordinate represents the running time of the method. The comparison methods used here include ER, PN-RPCA, GHOG-LR, and LSG, all of which run on the same device. In order to evaluate the method fairly, the running time does not include the training time. LSG is the most time-consuming of all methods because it needs to acquire cartoon components through morphological component analysis (MAC). Compared with PN-RPCA and GHOG-LR, WLRL is the most time-consuming, because WLRL consumes time in acquiring defect prior. Therefore, the performance of WLRL is better than other methods when considering accuracy.

Figure 8.

The running time of ER, PN-RPCA, GHOG-LR, LSG and WLRL.

Conclusion

This paper presents a novel method for patterned fabric detection called WLRL, which combines weighted low-rank decomposition with Laplace regularization. A defect prior is added to the low-rank decomposition model to construct the weighted low-rank decomposition model, then add the Laplacian regularization to increase the distance between backgrounds and defects. Experiments show that WLRL can efficiently pinpoint the locations of defective objects in patterned fabric images with sharp edges. The average ACC and TPR of box- and star-patterned fabrics are 99.1% and 79.73%, respectively.

The database employed in this research is kindly provided by Industrial Automation Research Laboratory from Department of Electrical and Electronic of Hong Kong University