Sage Journals: Discover world-class research

Abstract

Due to the suboptimal efficiency, accuracy, and increasing costs of manual defect detection in the textile industry, online visual inspection for fabric defects has emerged as an essential and promising research area. However, challenges such as the lack of defective samples and issues with industrial deployment still persist. This paper presents a novel defect detection technique based on deep learning, which primarily comprises two frameworks. First, we design an improved generative adversarial network with an encoder–decoder architecture to address the paucity of requisite defective samples. We use defect-free samples as input to the generator, ensuring that the generated defect samples maintain a similar pattern. We mitigate the vanishing gradient problem using Wasserstein distance as the loss function. Second, we enhance the Single Shot MultiBox Detector network by introducing Inception modules and feature fusion to detect defects across different scales. The AdaBound optimizer is selected to update the model parameters. We compare the proposed approach with other methods on self-generated fabric data sets that are partially produced by our generative adversarial network model. An online defect detection system is proposed to capture fabric images and evaluation in a production environment. Experiments demonstrate the superior performance of the proposed approach, achieving 97.5% accuracy in real time, making it well-suited for application in the industry.

Keywords

Convolutional Neural Network Fabric Defect Detection Generative Adversarial Network Online Detection Single Shot MultiBox Detector

Introduction

Fabric surface inspection is a critical task in the textile industry, whether from the perspective of quality or cost.¹ Typically, this task is carried out manually, which is highly intensive and costly. Moreover, it is hard to set a unified standard as defects are detected subjectively. Therefore, there is a pressing need to replace manual detection with industrial automatic methods, such as machine vision.²

The traditional machine vision methods that use image processing rely on handcrafted features combined with machine learning techniques.³ These can be divided into statistics-based,⁴ spectrum-based,⁵ and model-based⁶ methods. While traditional methods can achieve good detection results, they are sensitive to the lighting environment, and their performance depends on parameter adjustment. In recent years, deep learning methods have performed well in the field of computer vision by capturing complex nonlinear relationship.^3,7 However, deep learning approaches require a large number of fabric defect samples, which are difficult to obtain from the industrial site. In addition, real-time detection should be conducted during production to identify problems promptly, while most detection methods are still offline.

In this paper, we present a novel approach for synthesizing defect samples using a generative adversarial network (GAN) with an encoder–decoder architecture. In contrast to traditional data augmentation methods like random cropping, rotation, and scaling, GANs⁸ have shown great potential in generating high-quality synthetic data that closely resemble the original data. Our approach utilizes defect-free samples as input to the generator, ensuring that the generated defect samples preserve similar patterns. Next, inspired by the Single Shot MultiBox Detector⁹ (SSD) model, we proposed a refined framework to detect defects with greater accuracy. The refined framework fuses the feature maps from different scales, enabling the detection of defects of various sizes. Thus, the entire model primarily comprises two frameworks. Moreover, we design an online inspection system to evaluate the proposed methods in a real production environment. The system consists of several cameras, a light source, and a mechanical platform. The main contributions are summarized as follows:

An improved GAN model with an encoder–decoder architecture is proposed to expand the defect samples variously without corrupting the original textured pattern.

We propose a refined SSD model by introducing a feature pyramid structure to increase the interaction of information from different levels.

We construct an online defect detection system and demonstrated the performance of our methods in a production environment.

Related Work

Previously, traditional machine vision methods based on manually designed feature extractors were able to effectively detect simple fabric defects with better robustness and interpretability. Typical examples include a gray-level co-occurrence matrix,¹⁰ gray histogram back-projection,¹¹ linear filters, and morphological methods.¹² Kang and Zhang¹³ proposed a novel method by integrating the idea of the integral image into the Elo-rating algorithm (IIER), which can quickly detect the defects of various fabric types detect. Liu et al.¹⁴ proposed a fabric method based on template correction and primitive decomposition.

Traditional machine vision methods involve crafting features manually by experts to identify defects, which is time-consuming and often requires domain-specific knowledge. Recently, with the advancement of computer computing ability, deep learning methods have achieved excellent performance in many visual applications, such as classification¹⁵ and detection.^16,17 Deep learning methods can automatically extract features from the input data, eliminating the need for manual feature engineering. Some researchers began to apply proven deep learning architectures like Faster R-CNN¹⁸ and Cascade R-CNN¹⁹ to fabric defect detection as well.^20,21 Jing et al.²² used the nonlinear mapping ability of radial basis function neural network to detect color fabric defects, which can identify more kinds of defects than traditional methods. Wu et al.²³ used the shallow neural network structure method to detect single-color fabric defects, which effectively improves the speed of defect detection. Li et al.²⁴ proposed a novel feature-based attention network with sequential information for fabric defect detection. Li et al.²⁵ proposed a wide and compact convolutional neural network architecture for the detection of a few common fabric defects. Jeyaraj and Nadar²⁶ proposed a computer-aided fabric defect detection and classification algorithm using an advanced learning algorithm. Liu et al.²⁷ proposed a multistage GAN model to synthesize reasonable defects in new defect-free samples, which can deal with fabrics with unseen textures or materials. However, deep learning methods require a large number of defect samples for training, which can be a challenge, especially in textile applications.

The Proposed Method

The steps of the defect detection approach are shown in Figure 1. First, the collected defect and defect-free images will be pre-processed, which will be described in section “Refined SSD Structure.” Second, the improved GAN is used to expand the defect sample set. Then, the real and generated defect samples are used to train the refined SSD model. Finally, fabric defects will be detected by realizing the classification and position based on the trained model.

Figure 1.

Defect detection steps.

Improved GAN Structure

Pipeline

The proposed GAN framework is shown in Figure 2(b).

Figure 2.

Pipeline of GAN framework: (a) the original GAN and (b) the modified GAN.

The generator of the original GAN (Figure 2(a)) is to convert a Gaussian distribution or other prior random distribution into the desired data distribution. Given the prominent textured pattern of the fabric, we have incorporated an additional transformation network before the original generator. Using defect-free samples as the input, the new generator has been devised as an encoder–decoder structure architecture to map the defect-free sample to a latent space of the texture pattern. This enables the generated defect samples to maintain a pattern that closely resembles the actual ones. Meanwhile, the discriminator, composed of a sequence of convolution layers, classifies the input data as real or fake. These two modules are learned in an adversarial and unsupervised manner. To enhance the network’s robustness against noise and corrupted input samples, we add Gaussian noise to the input training samples of the generator. The Network in Network (NIN) module, which significantly enhances the nonlinear performance of the entire network, is introduced to address the problem of mode collapsing. Furthermore, 1 × 1 convolution kernels can help reduce the network parameters, thereby accelerating the generation speed. A more detailed parameterization of our network is provided in Table 1.

Table 1.

The general outline of our generator and discriminator.

Module		Layer	Output size	Parameters
				Kernel	Stride	Padding
		Input	512 × 512 × 1
Generator	Encoder	Conv	256 × 256 × 32	5 × 5	2	2
		Conv	128 × 128 × 64	5 × 5	2	2
		Conv	128 × 128 × 64	3 × 3	1	1
		Conv	64 × 64 × 128	5 × 5	2	2
		Conv	64 × 64 × 128	3 × 3	1	1
		Conv	32 × 32 × 256	3 × 3	2	1
	NIN	Conv	32 × 32 × 128	1 × 1	1	0
	Decoder	Deconv	64 × 64 × 128	4 × 4	2	1
		Conv	64 × 64 × 128	3 × 3	1	1
		Deconv	128 × 128 × 64	4 × 4	2	1
		Conv	128 × 128 × 64	3 × 3	1	1
		Deconv	256 × 256 × 32	4 × 4	2	1
		Deconv	512 × 512 × 16	4 × 4	2	1
		Conv	512 × 512 × 1	1 × 1	1	0
Discriminator		Conv	256 × 256 × 16	5 × 5	2	2
		Conv	128 × 128 × 32	5 × 5	2	2
		Conv	64 × 64 × 64	3 × 3	2	1
		Conv	32 × 32 × 128	3 × 3	2	1
		Conv	16 × 16 × 256	3 × 3	2	1
		Conv	8 × 8 × 512	3 × 3	2	1
		Average pool	1 × 1 × 512	8 × 8	1	0
		FC	1 × 1

NIN: Network in Network; FC: Fully Connected Layer.

Leaky Rectified linear units (Leaky ReLUs), followed by a batch normalization operation at the same time, are applied after each convolution layer as activation functions except for the output layers of both the encoder (including NIN) and the decoder, in which linear activation functions are used.

Loss Function

The GAN loss function is typically formulated as a minimax problem, where the generator $G$ tries to minimize the discriminator’s ability to distinguish between the real and generated images, while the discriminator $D$ tries to maximize them. One way to measure the distance between the generated data distribution $P_{G}$ and the real data distribution $P_{data}$ is to use a divergence metric.

The Jensen-Shannon (JS) divergence is commonly used in GANs.⁸ However, the problem with the JS divergence is that its gradients can become very small or zero when the generated and real data distributions are very different. The Wasserstein distance measures the minimum amount of work that is required to transform one probability distribution into another. It can provide a smooth and continuous gradient, even when the distributions are very different, and has been shown to provide a more meaningful measure of distance between probability distributions.²⁸

The Wasserstein distance assumes that the discriminator is Lipschitz continuous. In this paper, we introduce the Wasserstein distance with gradient penalty²⁹ to enforce the Lipschitz continuity constraint on the discriminator.

V (G, D) = max_{D} {E_{x ~ P_{data}} [D (x)] - E_{x ~ P_{G}} [D (x)]} - λ E_{x ~ P_{penalty}} [{(| | \nabla_{x} D (x) | | - 1)}^{2}]

(1)

where $λ$ is a hyperparameter that controls the strength of the gradient penalty, $\nabla_{x} D (x)$ is the gradient of the discriminator with respect to x. $x ~ P_{penalty}$ is a random interpolation between a real data sample and a generated sample.

The gradient penalty term encourages the discriminator to have a Lipschitz constant of 1, which means that its gradients are bounded by a constant factor. The gradient penalty term penalizes the discriminator if its gradients are too large or too small, which helps to ensure stable training and improve the quality of the generated samples.

Refined SSD Structure

Overall Framework

As shown in Figure 3, we proposed a novel fabric defect detection model based on the improved SSD. As a one-stage method in which the frame detection operation is complete in one step, the detection process can be finished more efficiently while maintaining high accuracy. The proposed model consists of ResNet50-based backbone, Inception modules, Boundary Refinement blocks, and an expansive path.

Figure 3.

Overview of the proposed SSD framework.

ResNet50

We change the backbone of the original SSD from VGG16³⁰ to ResNet50³¹ by duplicating several blocks of it and arranging them in cascade, as shown in the left of Figure 3. The purpose is to improve accuracy.

Inception

To address the issue of varying fabric defect sizes and types, we introduce Inspection modules³² into the SSD network framework. The asymmetric structure of the Inspection module, illustrated in Figure 4(a), can significantly increase the nonlinear degree of the network, effectively extract the fine features of defects in the process of network training, and improve the accuracy of defect detection.

Figure 4.

The details of Inception: (a) and boundary refinement, (b) blocks in Figure 3.

Boundary Refinement

We also employ the Boundary Refinement block³³ as a residual structure to refine the boundary of the defect in the feature map. The details can be referred to in Figure 4(b).

Feature Fusion

To get multi-contextual information from detection, we adopt an expansive feature fusion path (the right part of Figure 3). This allows us to capture more semantic information from deeper layers and potentially fuse it with earlier feature maps. We employ learnable deconvolution layers and summation-based skip connections with the corresponding map filtered by Inception modules for feature fusion.

Optimization

The original SSD model uses Stochastic Gradient Descent (SGD) as an optimizer because the gradient scaling of the algorithm in different dimensions is uniform. The optimization effect is good in the later stage of network training, while the convergence speed is slow in the early stage. Thus, it is difficult to deal with the unbalanced training samples. Another optimizer, Adam, can significantly improve the training speed of the network in the early stage, but it is easy to find the global optimal solution due to the extreme learning rate in the later stage.

In order to solve the above problems, AdaBound³⁴ is adopted:

{\hat{η}}_{t} = Clip (α / \sqrt{V_{t}}, η_{l} (t), η_{u} (t))

(2)

where $α$ is the initial learning rate. $V_{t}$ is the initial momentum. $η_{l}$ initialization value is 0. $η_{u}$ initialization value is infinite. The upper and lower bounds of ${\hat{η}}_{t}$ are limited by clipping $η_{l}$ and $η_{u}$ .

The initial optimization function is similar to Adam and the training speed is fast. The post-optimization function is similar to SGD. The optimization function is stable throughout the whole training process.

Experiment

In this section, we will validate the performance of the proposed detection model through a series of experiments. In addition, we also design an online inspection system to test the proposed methods in a real production environment.

Data Sets

First, we conduct contrast experiments on Tianchi fabric defect data set³⁵ to better test the performance of the proposed detection model. The data set contains 9576 images with a size of 1000 × 2466 pixels, including 5913 defect images and 3663 defect-free images. There are 34 categories of defects, including holes, stains, knotted ends, rough warps, and so on.

Then, we generate sufficient samples on a small, self-built data set and validate the performance of the proposed model on an online detection system. All the images are captured by the system elaborated in Figure 6. It consists of 25 grayscale images of size 4096 × 1024 pixels captured from three different warp fabric structures, as shown in Figure 7(a). The images contain five types of common fabric defects, including dirt (water stains, oil stains, etc.), spots (knots, etc.), creases, warp defects (broken yarn, loose warp, crack ends, etc.), and weft defects (lines, looped weft, etc.).

Metrics

We choose mean Average Precision (mAP), Accuracy (ACC) as metrics to evaluate the location accuracy of the model, which are often utilized in object detection. mAP in patch-wise tasks can be calculated by the following equations:

mAP = \frac{1}{C} \sum_{k = 1}^{C} A P_{k}

(3)

where,

A P_{k} = \int_{0}^{1} P_{k} (R_{k}) d R_{k}

(4)

P_{k} = \frac{T P_{k}}{T P_{k} + F P_{k}}

(5)

R_{k} = \frac{T P_{k}}{T P_{k} + F N_{k}}

where $C$ is the number of defect categories, $T P_{k}$ means the true positive of defect class $k$ , $F P_{k}$ means the false-positive of defect class $k$ , and $F N_{k}$ means the false-negative of defect class $k$ . $AP$ refers to the area enclosed by the P-R curve and the R axis. The comprehensive metric mAP refers to the average of all categories of defects $AP$ .

Considering that in the process of cloth defect detection, we focus more on whether the defect is detected correctly than whether the detected defects can perfectly overlap with the labeled region. Therefore, in the calculation of mAP, we set the threshold value of Intersection over Union (IoU) to 0.5, that is, it is considered that the defect is correctly detected if the IoU between the detected region and the labeled region reaches 0.5.

In addition, we use detection time as a metric to evaluate the detection speed of the model. Frame Per Second (FPS) is used on the Tianchi data set, while the average detection time per meter is used in the online experiments for the processes of data acquisition, transmission, and so on also take time.

Contrast Experiments

Experimental Settings

On the Tianchi data set, we only use defect images for training, for defect images also contain a large number of defect-free samples. We split the data set into a training set and a validation set in the ratio of 9:1, that is, 5321 images for training and 592 images for validation. The experiments were trained and evaluated on a single Nvidia RTXA5000, with batch size 16 and epoch 300.

To show the advantages of the proposed method, we apply several common defect detection approaches, such as Jing et al.’s,³⁶ Liu et al.’s,³⁷ and Zhou et al.’s³⁸ works as comparison methods. The configurations are as follows:

Jin: YOLOv3 (DarkNet53 + three layers of feature pyramid +feature fusion) + SGD + NMS;

Liu: SSD (VGG16 + six layers of feature pyramid) + SGD + NMS;

Zhou: Faster RCNN (ResNet50 + FPN + RPN) + Adam + NMS;

Ours: ResNet50+four layers of feature pyramid + Inception + boundary refinement + feature fusion + AdaBound + NMS.

Results

The performance of different detection methods is given in Table 2. In comparison with mAP, the proposed model has a 33.21% score, which has higher location accuracy than other methods. In terms of detection speed, our method can reach 60 FPS, which is suitable for real-time detection. The proposed method uses multiscale feature fusion with Inception structure, which increases the inference time but has higher accuracy compared to SSD and Faster R-CNN. It can better recognize small defects compared to YOLOv3 due to the fusion of the low-level features.

Table 2.

Comparison on different fabric defect detection methods on Tianchi data set.

Method	mAP	FPS
Jing³⁶	27.42	60.42
Liu³⁷	13.02	112.96
Zhou³⁸	8.28	18.93
Ours	33.21	60.31

mAP: mean Average Precision; FPS: Frame Per Second.

Figure 5 shows the detection results of different detection methods on the Tianchi data set. It can be seen that our method is better at detecting defects at different scales.

Figure 5.

The detection results of different detection methods on the Tianchi data set: (a) Jin, (b) Zhou, (c) Liu, and (d) ours.

Online Detection

In order to evaluate the effectiveness of the proposed methods online, we design a fabric defect detection system.

Hardware System

The components are illustrated in Figure 6. Fabric is transported mechanically on a platform. The light source which can produce uniform white light with adjustable light intensity to project on the fabric is perpendicular above. Images are captured by two cameras based on CMOS sensors. We use 4K line-scan cameras, and the frame grabber is an Xtium-CL MX4 for image acquisition. The total field of view of two cameras is 2 m, while the width of fabrics is 1.5–1.8 m. Therefore, two cameras are sufficient to cover the entire fabric. An encoder is used to trigger the image acquisition. The vertical distance between the lenses and the fabric can be adjusted according to the actual width of the fabric. As the effect of defect detection is easily affected by the image quality, the camera and light source of the detection system are installed independently from the detection platform.

Figure 6.

Fabric detection system: (a) the components of the detection system and (b) detection system physical map.

Experimental Settings

Due to the high resolution and the small number of defective images, we randomly crop all the original images into 512 × 512 pixels. We only crop in the region with defects to ensure that we do not change linear defects into spot defects, and the problem of positive and negative sample imbalance can be mitigated at the same time.

The data set contains 4400 defect samples, among which 4000 images are generated. We randomly divided them into two parts: 4000 for training and 400 for validation.

We use random horizontal flipping, random vertical flipping, random 180° rotation, and random histogram equalization as data augmentation, as the captured images are easily affected by the environment. The results are shown in Figure 7.

Figure 7.

The result of data augmentation: (a) original image and (b) after augmentation.

Defect Generation Results

Given that the generation quality and convergence speed of images with various types of defects may be different, the improved GAN is compared to the original GAN under different kinds of defect images.

To compare the convergence speed of the improved GAN and the original GAN under different defects, for analysis, spots, dirt, and creases are chosen. The performance of the generator depends on the value and stability of the loss function of the discriminator in the training process. When the number of iterations is set to 9000, the numerical changes of the discriminator’s loss function in the process of model training are visualized, and 300 points are taken for mapping, as shown in Figure 8.

Figure 8.

The convergence rate comparison of GAN algorithm: the convergence diagram of (a) spot, (b) dirt, and (c) crease.

The original GAN can converge after about 4000 iterations, while the convergence stability is not as good as that of the improved GAN. Figure 8(b) shows the effect of the original GAN and the improved GAN in dealing with the dirt with complex features. With the increase in iteration times, the discriminator loss function tends to diverge. Figure 8(c) shows the convergence of the original GAN and the improved GAN when dealing with crease defects. The improved GAN has good convergence speed and convergence effect for defects and converges to 0 after 1000 iterations.

Figure 9 shows the outputs from the original improved GAN generator trained after 9000 iterations. Obviously, the defect images generated by the original GAN are fuzzier. Conversely, samples from our approach show more variation and the original texture pattern remains unaltered, which is consistent with the actual situation.

Figure 9.

Image comparison between original GAN and improved GAN. (a) Spot images generated by original GAN. (b) Spot images generated by improved GAN. (c) Dirt images generated by original GAN. (d) Dirt images generated by improved GAN.

Defect Detection Results

Table 3 shows comparison of defect detection effects between SSD and our approach. The results show that the original SSD and our approach can accurately detect only a single type of defect and dirt in the image, but the improved SSD model has a higher confidence score in identifying defects, which is 0.07 and 0.19 points higher, respectively. When there is a mixture of defects and dirt in the image, the original SSD can only identify the dirt, whereas our approach can simultaneously identify the defects and dirt. Our approach can also accurately identify the same image containing multiple defects, which is more common.

Table 3.

Comparison of defect detection effects between SSD and our approach.

SSD	Our approach

SSD: Single Shot MultiBox Detector.

In order to realize online inspection, the defect detection time is especially important. We compare our approach with Faster R-CNN¹⁷ and SSD model in five types of defects: spots, dirt, crease, warp defects, and weft defects. The size of the testing image is cropped into 512 × 512 pixels, and the detection results are shown in Table 4.

Table 4.

Contrast results of three models under five kinds of defects.

Defects	Detection algorithm	Number of images	Accuracy (%)	Average detection time (s)
Spot	Faster R-CNN		98.5	0.090
	SSD	80	90	0.022
	Our approach		98	0.047
Dirt	Faster R-CNN		98.5	0.100
	SSD	80	90	0.024
	Our approach		98.5	0.040
Crease	Faster R-CNN		96	0.093
	SSD	80	86.5	0.023
	Our approach		95	0.048
Warp defects	Faster R-CNN		98.5	0.092
	SSD	80	93	0.021
	Our approach		98	0.049
Weft defects	Faster R-CNN		98.5	0.093
	SSD	80	93.5	0.021
	Our approach		98	0.049

SSD: Single Shot MultiBox Detector.

According to Table 4, our approach has higher accuracy than SSD, and basically has the detection accuracy of the Faster R-CNN, with an average detection accuracy of 97.5%. The detection speed is about twice that of Faster R-CNN, and the speed can reach 0.047s/frame, which can meet the requirements of factory online inspection.

Conclusion

In this paper, we present an online detection method for fabric defects based on deep learning, which mainly consists of two frameworks. First, an improved GAN model with the encoder–decoder structure, whose inputs are noisy defect-free samples to maintain the original texture pattern, is adopted to handle the lack of required training samples. Then, we design a novel extension of SSD with deconvolutions and Inception modules to recognize the defects in multi scale. Our experimental results show that this method is efficient while maintaining high accuracy, which makes it suitable for industrial applications.

Although the speed of detection can meet the requirements of the real detection process, the proposed approach is mainly for plain fabrics. In the future, we will concentrate on sample generation on more complex patterned fabrics, which can be combined with our method to improve detection performance.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Key Research and Development Project (grant number: 2018YFB1308800).

References

Hanbay

Talu

Ozguven

OF.

Fabric defect detection systems and methods—A systematic literature review. Optik 2016; 127: 11960–11973.

Rasheed

Zafar

Rasheed

, et al. Fabric defect detection using computer vision techniques: A comprehensive review. Math Probl Eng 2020; 2020: 24.

Tabernik

Šela

Skvarč

, et al. Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 2020; 31: 759–776.

Ngan

HYT

Yuan

, et al. Patterned fabric inspection and visualization by the method of image decomposition. IEEE Trans Autom Sci Eng 2014; 11: 943–947.

Jia

Chen

Liang

, et al. Fabric defect inspection based on lattice segmentation and Gabor filtering. Neurocomputing 2017; 238: 84–102.

Wang

Huang

XB.

Fabric defect detection based on multiple fractal features and support vector data description. Eng Appl Artif Intell 2009; 22: 224–235.

Wang

Chen

Qiao

, et al. A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 2018; 94: 3465–3471.

Goodfellow

Pouget-Abadie

Mirza

, et al. Generative adversarial networks: An overview. Computer Science, 2014, https://arxiv.org/pdf/1710.07035.pdf

Liu

Anguelov

Erhan

, et al. SSD: Single shot multibox detector. In: 14th European conference on computer vision (ECCV) (ed Leibe

Matas

Sebe

, et al.), Amsterdam, 8–16 October 2016, pp. 21–37. Cham: Springer.

10.

Yuan

Zhang

, et al. A defect detection method for unpatterned fabric based on multidirectional binary patterns and the gray-level co-occurrence matrix. Text Res J 2020; 90: 776–796.

11.

Sun

Zhou

Gao

, et al. A fast fabric defect detection framework for multi-layer convolutional neural network based on histogram back-projection. IEICE Trans Inf Syst 2019; E102D: 2504–2514.

12.

Celik

Dulger

Topalbekiroglu

Fabric defect detection using linear filtering and morphological operations. Indian J Fibre Text Res 2014; 39: 254–259.

13.

Kang

Zhang

EH.

A universal defect detection approach for various types of fabrics based on the Elo-rating algorithm of the integral image. Text Res J 2019; 89: 4766–4793.

14.

Liu

Chang

Liang

, et al. Defect inspection research on fabric based on template correction and primitive decomposition. IET Image Process 2019; 13: 2916–2928.

15.

Celik

Dulger

Topalbekiroglu

Development of a machine vision system: Real-time fabric defect detection and classification with neural networks. J Text Inst 2014; 105: 575–585.

16.

Pamula

Impact of data loss for prediction of traffic flow on an urban road using neural networks. IEEE Trans Intell Transp Syst 2019; 20: 1000–1009.

17.

Sun

ZB.

A graph-based semisupervised deep learning model for PolSAR image classification. IEEE Trans Geosci Remote Sensing 2019; 57: 2116–2132.

18.

Ren

Girshick

, et al. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 2017; 39: 1137–1149.

19.

Cai

Vasconcelos

Cascade R-CNN: Delving into high quality object detection. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June 2018, pp. 6154–6162. New York: IEEE.

20.

Bag of tricks for fabric defect detection based on Cascade R-CNN. Text Res J 2021; 91(5–6): 599–612.

21.

Liu

, et al. Fabric defect detection based on faster R-CNN. In: Yu

Dong

(eds) Ninth international conference on graphic and image processing, Qingdao, China, 10 April 2018, pp. 23–32. Bellingham, WA: Spie-Int Soc Optical Engineering.

22.

Jing

Fan

, et al. Yarn-dyed fabric defect detection based on deep-convolutional neural network. J Text Res 2017; 38: 68–74.

23.

Zhuo

, et al. A fast monochromatic fabric defect fast detection method based on convolutional neural network. J Comput Aided Des Comput Graph 2018; 30: 2262–2270.

24.

Liu

A novel feature-based network with sequential information for textile defect detection. Color Technol 2020; 136: 476–484.

25.

Zhang

Lee

DJ.

Automatic fabric defect detection with a wide-and-compact network. Neurocomputing 2019; 329: 329–338.

26.

Jeyaraj

Nadar

ERS

. Computer vision for automatic detection and classification of fabric defect employing deep learning algorithm. Int J Cloth Sci Technol 2019; 31: 510–521.

27.

Liu

Wang

, et al. Multistage GAN for fabric defect detection. IEEE Trans Image Process 2020; 29: 3388–3400.

28.

Arjovsky

Chintala

Bottou

Wasserstein generative adversarial networks. In: Proceedings of the 34th international conference on machine learning (ed Doina

Yee Whye

), Sydney, NSW, Australia, 6–11 August 2017, pp. 214–223, https://proceedings.mlr.press/v70/arjovsky17a.html

29.

Gulrajani

Ahmed

Arjovsky

, et al. Improved training of wasserstein GANs. In: Advances in neural information processing systems 30: Annual conference on neural information processing systems, Long Beach, CA, 4–9 December 2017.

30.

Simonyan

Zisserman

Very deep convolutional networks for large-scale image recognition. Computer Science, 2014, https://arxiv.org/abs/1409.1556

31.

Zhang

Ren

, et al. Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, NV, 27–30 June 2016, pp. 770–778. New York: IEEE.

32.

Szegedy

Liu

Jia

, et al. Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition, Boston, MA, 7–12 June 2015, pp. 1–9. New York: IEEE.

33.

Peng

Zhang

, et al. Large kernel matters—Improve semantic segmentation by Global Convolutional Network. In: 30th IEEE conference on computer vision and pattern recognition, Honolulu, HI, 21–26 July 2017, pp. 1743–1751. New York: IEEE.

34.

Luo

Xiong

Liu

, et al. Adaptive gradient methods with dynamic bound of learning rate. arXiv [preprint]. arXiv: 190209843 2019.

35.

Fabric defect dataset, 2020, https://tianchi.aliyun.com/dataset/dataDetail?spm=5176.12281978.0.0.6a934926vgTXXq&dataId=79336

36.

Jing

Zhuo

Zhang

, et al. Fabric defect detection using the improved YOLOv3 model. J Eng Fibers Fabr 2020; 15: 1558925020908268.

37.

Liu

, et al. Fabric defects detection based on SSD. In: Proceedings of the 2nd international conference on graphics and signal processing, Sydney, NSW, Australia, 6–8 October 2018, pp. 74–78. New York: Association for Computing Machinery.

38.

Zhou

Jang

Chen

, et al. Exploring Faster RCNN for fabric defect detection. In: 2020 third international conference on artificial intelligence for industries (AI4I), Irvine, CA, 21–23 September 2020, pp. 52–55. New York: IEEE.

Online Fabric Defects Detection Using Convolutional Neural Networks with Two Frameworks

Abstract

Keywords

Introduction

Related Work

The Proposed Method

Improved GAN Structure

Pipeline

Loss Function

Refined SSD Structure

Overall Framework

ResNet50

Inception

Boundary Refinement

Feature Fusion

Optimization

Experiment

Data Sets

Metrics

Contrast Experiments

Experimental Settings

Results

Online Detection

Hardware System

Experimental Settings

Defect Generation Results

Defect Detection Results

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References