Sage Journals: Discover world-class research

Abstract

Deep convolutional neural networks (CNNs) have shown great success in single-class fabric image detection. However, real-world fabric defect images generally contain several types of defects in one image. Accurately recognizing and classifying multi-class fabric defect images is still an unsolved issue due to the complexity of intersected defects, as well as difficulty in distinguishing small-size defects. To address these challenges, this study develops a methodology based on the deep learning feature pyramid networks (FPN) approach to detect multi-class fabric defects. To evaluate the proposed detection model, we built a unique multi-class fabric defects database (DHU-MO1000), where multi-class defect images are generated by industrial monitors from a textile factory. We used the dataset as the benchmark for multi-class defects detection training and testing the FPN. Furthermore, we conducted extensive experimental validations for various design choices. The experimental results show that the model outperformed existing multi-class object detection methods.

Keywords

Computer Vision Convolutional Neural Networks Fabric Defect Detection Multi-Class Object Detection

Introduction

In the modern fabric industry, fabric defect recognition and detection are important for textile industrial quality control. In most textile mills, the visual detection of trained workers is still a critical element in the fabric defect detection process, with low detection efficiency and precision due to psychological factors, different fibers, and many other constraints. Due to the constraints, automated detection based on computers or machines has drawn considerable attention in recent decades.

Typically, single-class fabric defect detection is a special case of multi-class detection. The multi-class object detection of fabric defect images is a more general and practical problem, since the majority of real-world fabric images contain multiple defects. As shown in Fig. 1, for single-label defect images, the fabric defects are roughly aligned with the label. While for multi-label defect images, even with the same label (e.g., brokenpick or felter in Fig. 1), the textile and size of defects are different from those of single-label defects. Also, some defects are too small to be distinguished. Hence, the multi-class object detection of fabric defect images is more difficult than the single-class situation.

Fig. 1

Selected image examples of fabric defects. Foreground fabric defects in single-class images are usually roughly aligned (images in the first row). However, the assumption of fabric defect alignment is not valid for multi-class object images (images in the second and third rows).

At present, there are mainly five types of single-class fabric defect detection algorithms: 1)

Spectral-based methods (e.g., Fourier Transform⁷ and Gabor Transform^10,14,19,33). These methods can accurately detect fabric defects by using image frequency domain information, but local and whole information in the image is difficult to be considered. These approaches need a great deal of calculation to ensure detection precision.

Statistical-based algorithms, which are characterized by the extraction of the eigenvalue and spatial distribution of image gray values (e.g., morphology³² and co-occurrence matrix¹⁸) The advantage of these methods is their speed because of using the gray value characteristics of the image, but they are more susceptible to noise and external interference. In addition, statistical-based algorithms do not easily detect fabric defect features that are not obvious.

Learning-based algorithms (e.g., support vector machines¹⁵ and neural networks^8,13,16) that can further extract the feature information of the defective image. The disadvantage of using these methods is that the dimension of the feature information is low, thus, the detection accuracy is difficult to improve.

Structure-based algorithms, using the texture analysis method.¹ These algorithms can achieve good detection results, but it is difficult to choose the suitable feature extractor for the fabric image where defects are not distinct.

Model-based algorithms, which construct the stochastic models with random variables. The auto-regression model² belongs to these algorithms. In this approach, the parameters can be adjusted to determine whether there are fabric defects. But the problem is that if the selected parameters are improper, the convergence rate is very slow.

Lately, convolutional neural networks (CNNs)¹² have shown promising performance in computer vision applications, such as image detection,^28,31 image restoration,⁹ crowd counting,³⁴ and paleo valley recognition.¹¹ In addition, CNN has also achieved state-of-the-art performance in large-scale single-object image classification.³ With the characteristics of CNN, Girshick et al. designed the region-based convolutional neural network (RCNN),⁵ and obtained the candidate region by the region selection approach. Then, they presented the Fast RCNN⁴ and Faster RCNN²² approaches for object detection. Furthermore, a top-down architecture with lateral connections is proposed for building high-level semantic feature maps at all scales, which is called the feature pyramid networks (FPN).¹⁷ CNN has been widely applied in single-class fabric defect classification,²⁷ fabric pattern generation,²⁶ and detection.²⁸ For instance, Li et al.¹⁶ proposed a Fisher criterion-based stacked denoising auto-encoder (AE) model for detecting deformable patterned textile defects. Mei et al.²⁰ presented an unsupervised learning-based automated approach with a multi-scale convolutional denoising auto-encoder network model, which synthesizes results from multiple pyramid levels and highlights defective regions through the reconstruction residual maps generated by the convolutional denoising auto-encoder networks. Mei et al.²¹ also presented an unsupervised learning approach for automated defect inspection on homogeneous and nonregular textured surfaces.

Meanwhile, many methods^23,30 have also been proposed to address multi-object image detection. However, these approaches cannot detect small-size defects effectively. At present, recognizing and classifying multi-class fabric defect images still remain unsolved, making it a pressing and crucial task for improving the quality of textile products.

In this study, based on the development of deep learning, the multi-class object detection of fabric defect images is studied. Due to the advantages of detecting small targets, the FPN network is used to identify fabric defects. The main contributions of this paper are presented as follows.

A unique fabric database (DHU-MO1000) was created by collecting the fixation data from textile mills.

An FPN was used for the multi-class detection of fabric defect images. To the best of our knowledge, this is the first work on applying the deep learning framework into multi-class fabric defect detection.

Our experiments on DHU-MO1000 demonstrated that the model can obtain better recognition performance than the current state-of-the-art approaches.

The remainder of this paper is organized as follows. FPN is overviewed, multi-class detection of fabric defect images based on FPN is introduced, experiment results and feature analysis are provided, followed by the conclusion.

Overview of FPN

Girshick et al. further presented the FPN based on Faster RCNN.²² The structure of the network consisted of the FPN, a region proposal network (RPN), region of interest (ROI) pooling, and classification and regression.

FPN Architecture

The FPN architecture takes an image of an arbitrary size as the input, and outputs proportionally-sized feature maps at multiple levels. The structure of FPN involves a top-down pathway, lateral connections, and a bottom-up pathway. The top-down pathway can obtain higher resolution features by upsampling semantically stronger, but spatially coarser, feature maps from higher pyramid levels. These features are then enhanced by the bottom-up pathway with lateral connections. Each lateral connection merges features with the same size from the top-down pathway and the bottom-up pathway. The bottom-up pathway executes the feed-forward computation, which computes the feature maps at several scales. The bottom-up pathway completes the feed-forward computation that computes a feature hierarchy. The feature hierarchy consists of feature maps at several scales with a scaling.

As shown in Fig. 2, the building block constructs the top-down feature maps. With a spatially coarser feature map, the block upsamples the spatial resolution by a factor of 2. Then, the upsampled feature maps are integrated with the corresponding bottom-up map by element-wise addition. This process is iterated until the finest resolution is generated. Before the iteration process, the 1 x 1 convolutional layer is attached to produce the coarsest resolution map. Finally, the 3 x 3 convolution operation is appended on each merged map to generate the final feature map, which can reduce the aliasing effect of upsampling. The final set of the feature map is {P₂, P₃, P₄, P₅}.

Fig. 2

A building block illustrating the lateral connection and the top-down pathway, merged by addition.

RPN

The RPN network takes an image as input and outputs a series of object proposals. This process is modeled with a fully convolutional network. To generate region proposals, the RPN network is slid over the convolutional feature map, which is outputed by the last shared convolutional layer. This RPN network takes an n x n spatial window of the input convolutional feature map as input. Each sliding window is mapped to the lower-dimensional feature. Then, the lower-dimensional feature is fed into two fully-connected layers (the box-classification layer and the box-regression layer). Fig. 3 shows the RPN architecture. For each sliding window, we take the center point as the reference point to select nine anchors. In general, the anchors correspond to three kinds of ratio aspects and three scales of the candidate area. For each anchor, the output of the classification layer is a score, which represents the region of interests or the background. The output of the regression layer has four coordinates that indicate the coordinate position of the fabric defect. When the RPN was proposed, the number of candidate regions is significantly reduced. The quality of the candidate boxes is improved. The objective function following the multi-task loss is minimized (Eq. 1).

Fig. 3

Region Proposal Network (RPN).

L ({p_{i}}, {t_{i}}) = \frac{1}{N_{c l s}} \sum_{i} L_{c l s} (p_{i}, p_{i}^{*}) + λ \frac{1}{N_{r e g}} \sum p_{i}^{*} L_{r e g} (t_{i}, t_{i}^{*})

Eq. 1

ROI Pooling Layer

The ROI operation uses max pooling to convert features in any valid area of interest into small feature maps with a fixed spatial range of H x W (e.g., 9 x 9), where H and W each represents a particular layer of the ROI. In particular, the ROI pooling layer is a rectangular window, which is defined by a four-tuple (r, c, h, w) that specifies its height and width (h, w) and its top-left corner (r, c). The max pooling of ROI works by dividing the h x w ROI window into an H x W grid of sub-windows of approximate size h/H x w/W, and then calculates the values in each sub-window into the corresponding output grid cell. As the standard max pooling operation, this pooling is applied independently to each feature map channel.

After the ROI operation, the feature maps and region proposals can be collected through the ROI pooling layer, which is characterized by the non-fixed size of the feature maps. The output of ROI pooling layer is a vector, whose size is channel*w*h, where w and h are the width and height of the feature map, and channel is the dimension of the feature map.

Classification and Regression

In the FPN network, the model can achieve a bounding-box regression by a different manner from previous ROI-based methods. The feature information used for regression is of the same spatial size on the feature maps. To account for varying sizes, a set of k bounding-box regressors are learned. The k regressors do not share weights, and each regressor is responsible for an aspect ratio and one scale. As such, the feature is fixed, and it is still possible to predict boxes of various sizes.

For image classification, classification layers calculate the proposal's class by using the full connection layer and the softmax network. For the bounding box regression, Faster RCNN adopts the parameterizations of the four following coordinates (Eq. 2).²²

\begin{array}{l} t_{x} = (x - x_{a}) / w_{a}, t_{y} = (y - y_{a}) / h_{a} \\ t_{w} = \log (w / w_{a}), t_{h} = \log (h / h_{a}) \\ t_{x}^{*} = (x^{*} - x_{a}) / w_{a}, t_{y}^{*} = (y^{*} - y_{a}) / h_{a} \\ t_{w}^{*} = \log (w^{*} / w_{a}), t_{h}^{*} = \log (h^{*} / h_{a}) \end{array}

Eq. 2

x, y, w, and h represent the box's center coordinates and its width and height. Variables x, x_a, and x* denote the predicted box, anchor box, and ground-truth box, respectively (likewise for y, w, h).

FPN Net-Based Fabric Defect Detection Model

The method developed to identify multi-class defects is comprised of three key steps.

The multi-class fabric defect dataset is collected from textile mills. The quality of the data largely affects the detection result of our algorithm. Hence, we preprocessed the raw data.

FPN net is trained and validated using the preprocessed data.

FPN net is tested on DHU-MO1000, and the classification performance is evaluated and presented. The learning process of the proposed approach is described in detail later.

Data Preparation and Augmentation

In this study, the multi-class fabric defect dataset is collected from the textile mills. Before training the FPN model using the dataset, the original data is preprocessed and augmented by two steps: segmentation of fabric defects and diversification of the images. The original defect images are 1280x1024 pixels, and include stain, irregular texture, and fringe. The first step is to crop and obtain local image blocks, each of which has a size of 320x320 pixels. The second step is to rotate and translate the images so that the model can learn more invariant image features. The range of rotation is from 5° to 20°, and the range of translation is from 0 to 50 pixels. The original fabric defect images are shown in Fig. 4. The images with horizontal and vertical flips are shown in Fig. 5.

Fig. 4

Examples of the original image.

Fig. 5

Horizontal and vertical flips of the images.

FPN Algorithm

In the FPN network, the two most basic operations are convolution and pooling. With a multi-class defect image as the input, the convolutional layer convolutes the feature map of the upper layer to generate the feature maps (Eqs. 3 and 4).

x_{c j}^{l} = f (u_{c j}^{l})

Eq. 3

u_{c j}^{l} = \sum_{i \in M_{i}} x_{c j}^{l - 1} * k_{i j}^{l} + b_{c j}^{l}

Eq. 4

f(·) is the nonlinear activation function, x^lcj represents the feature map of the j-th channel of the l-th convolutional layer, the subscript c is used to distinguish the pooling layer parameters and the convolution layer parameters, u^l_cj is the network activation of the j-th channel of the l-th convolutional layer that can be obtained by summing the convolution operation of the feature map x^l-1_cj, M_j represents the subset of feature maps of the input, k^l_ij denotes the convolutional kernel, b^l_cj is the shared basis, and * denotes the convolution operation. The feature map in the pooling layer can be formulated as in Eq. 5.

u_{p j}^{l} = β_{j}^{l} d o w n (x_{c j}^{l - 1}) + b_{p j}^{l}

Eq. 5

The operation down(·) is the function of the pooling layer, β_l^l represents the weight factor of the j-th channel of the l-th pooling layer, b^l_pj is the shared basis, and u^l_pj is the network activation of the j-th channel of the l-th pooling layer.

In this study, the pre-trained resnet50⁶ model is used for image feature extraction. The FPN architecture can generate multi-dimensional feature representations for an image. The main role of the RPN is to generate region proposals. Then, ROI can convert a feature map with a random input into a fixed-size feature map. The whole learning process is presented in Algorithm 1.

Algorithm 1:

FPN Model for multi-class object fabric defects detection.

Inputs: The original images X; The cross-entropy C.

Output: Weight and bias matrices k^l_ij, b^l_cj, and b^l_pj; the loss L_total; refined bounding box; and detection time T. i ∈ M_j; M_j is the subset of the input feature map; j and l represent the j-th channel of the l-th convolution layer, respectively.

Procedure:

1: Get the data X.

2: Construct the graph. obtain graph G with weights k_i ^l_j.

3: Initialization. Initialize learning rate, batch size, resnet50.ckpt and so on.

4: While iter < max_iters +1

5: Images feature extraction (FPN). Extract multi-class fabric defect features.

6: Region Proposal Network (RPN). Generate the region proposal (RoIs).

7: RoIHead. Classification layers calculate the proposal's regression to get the exact final positions t_x*,t*_y,t_w*, t_h*.

8: The output Output L_total, T, refined bounding box.

9: Save. Parameters k^l_ij, b^l_cj, b^l_pj and graph of session.

10: End while.

Learning the Proposed Networks

A certain number of multi-class fabric images are randomly sampled from the training sets for training. The testing sets are used to evaluate the performance of the proposed approach. The learning process of the FPN model is illustrated separately.

Resnet50 is pretrained by ImageNet dataset to extract the features. A proper initialization is set on the hyper parameters. All new layers are randomly initialized by drawing weights from a zero-mean Gaussian distribution with a standard deviation of 0.01. The stochastic gradient descent with a batch size of one sample is used with 10,000 iterations, the weight decay of 0.0004, the momentum of 0.9, and the gamma of 0.1. The learning rate is initially set as 0.0001. The size of ROI is set as 14. Our model is developed based on the deep learning library Tensorflow 1.2.0 and relevant third-party libraries. We conduct our experiments on a personal computer with 128GB RAM and four Nvidia GeForce GTX 1080 GPUs.

Results and Discussion

To validate the effectiveness of the proposed approach, the experiments are carried out on a self-developed dataset (DHU-MO1000). In addition, the detection performance is also verified on the single-class detection of fabric defect images. The single-class fabric defect image dataset (DHU-SL1000)²⁸ is adopted in this study. The comparison experiments with three state-of-the-art algorithms are also conducted. This section includes three subsections described as follows: 1)

The experiment setup is introduced, including an overview of the dataset and several quantitative indicators.

The detection results are presented and evaluated. The detection performance is compared with the state-of-the-art models.

To further understand the learning process of the model, the features are visualized and analyzed.

Setup

The textile dataset DHU-MO1000 consists of approximately 1000 samples, including 950 defect images and 50 defect-free images. The dataset contains six categories of defects: normal (defect-free), sundries, oilstains, brokenpick, felter, and brokenend. Some typical textile defect samples are shown in Fig. 4. The characteristics of the fabric defect images dataset are shown in Table I. Over 80% of these images belong to multiple classes simultaneously.

Table I.

Characteristics of Multi-Class Fabric Defect Data^a

Label Set	#Image	Label Set	#Image	Label Set	#Image
n	50	be + s	155	f + s + o	5
be	30	bp + s	155	f + bp + o	5
bp	30	f + s	155	s + bp + be	5
f	30	o + bp	155	f + s + be	5
o	30	s + f	155	s + be + o	5
s	30			Total	1000

n: normal, be: brokenend, bp: brokenpick, f: felter, o: oilstains, s: sundries

Several quantitative indicators are selected to evaluate the multi-class detection results, including accuracy, recall, Average Precision (AP) and mean of AP (mAP) in Eq. 6.

recall = \frac{T P}{T P + F N} \times 100 %

Eq. 6

TP and FN refer to the ratios of defective samples that are detected as defective and defect-free, respectively. Following the reference,²⁹ we use AP and mAP in this study.

Detection Results

We evaluate our approach for multi-class fabric defects detection and compare its detection performance with other approaches. In Fig. 6, the detection results for some multi-class fabric defect images are presented. It can be seen from the figure that the FPN model can obtain effective performance in different fabric defect types. Meanwhile, we can see that the model can achieve competitive detection performance even for some very small defects, such as sundries.

Fig. 6

Examples of multi-class fabric defect detection.

Table II shows the recall and AP of multi-class detection for detecting different defect types. It can be seen that the model can obtain good results for detecting the defect brokenpick, but cannot obtain effective results for detecting felter. This phenomenon may be caused by the property and background of a fabric defect. The defect brokenpick is more pronounced relative to the background. The felter is more blurred, making it difficult to be distinguished. The detection performance of the compared methods on DHU-MO1000 is quantified (Table III). The first approach is based on non-locally centralized sparse representation.²⁵

Table II.

Experimental Results of Multi-Class Detection of Various Defect Types

Metrics	Brokenpick	Felter	Sundries	Brokenend	Oilstains
Recall (%)	100	85.00	97.05	85.71	90.91
AP (%)	96.60	48.02	80.73	69.93	82.47

Table III.

Performance Comparison of Different Detection Approaches on Multi-Class Fabric Dataset

Model	Nonlocal Sparse³⁰	Faster RCNN²²	Ours
Number of defect types	5	5	5
mAP (%)	67.81	71.32	75.56

The second approach is Faster RCNN.²² The mAP of Faster RCNN is 71.32%, which is better than nonlocal sparse method. Our FPN model can reach 75.56%, improving the Faster RCNN method by 4.24%. This experiment further demonstrates the superiority of the FPN model.

To verify the efficiency of the proposed model on multi-class fabric defects detection, we also evaluated the testing time. The results are obtained on the GPU with Nvidia GeForceGTX 1080. During the training process, the time cost of training the model is 357 min. Table IV shows the time consumed for the testing process. The average detection time is 0.5 s for each image. From the results, we can see that the proposed method is suitable for multi-class fabric defects detection.

Table IV.

Time of Fabric Defect Detection^a

Label Set	be	bp	be+s	bp+s	f+bp+o	s+bp+be
Test Time (s)	0.515	0.516	0.532	0.548	0.520	0.517
	0.535	0.515	0.555	0.524	0.516	0.523
	0.526	0.520	0.536	0.532	0.530	0.518
	0.529	0.519	0.526	0.512	0.522	0.511
	0.512	0.529	0.520	0.512	0.518	0.521
Average Time (s)	0.523	0.520	0.533	0.526	0.521	0.518

be: brokenend, bp: brokenpick, f: felter, o: oilstains, s: sundries

In addition, it should be recognized that single-class fabric defect detection is a special case of multi-class detection. Here, we also verify the performance of the algorithm on single-class fabric dataset.²⁸ The detection results of the proposed method are compared with the state-of-the-art algorithms. Among the three existing methods, the first one is a fabric defect detection model that uses optimized filters to detect the defect images.²⁴ The second approach is based on non-locally centralized sparse representation.²⁵ The last method is a modified Faster RCNN.²⁸ As shown in Table V, the deep learning algorithms (Faster RCNN and FPN) can achieve better performance than the traditional detection algorithms (such as Gabor Filter and Nonlocal Sparse). Moreover, the proposed FPN model has the best comprehensive performance compared with the other methods. These results prove that our model achieves good performance, not only in the case of the multi-class situation, but also in the single-class fabric image detection.

Table V.

Performance Comparison of Various Detection Approaches on Single-Class Fabric Dataset

Model	Gabor Filter³¹	Nonlocal Sparse³⁰	Faster RCNN²²	Ours
Nums	4	4	6	6
Accuracy (%)	92.3	94.1	95.8	96.6

Feature Analysis

To further understand the learning process of the network, we visualize the extracted feature information of four convolution layers in Fig. 7.

Fig. 7

Visualization of the four convolution layers. The first row is the original images, and the 2^nd-5^th rows represent respectively the feature maps of the 2^nd-5^th convolution layer.

As shown in Fig. 7, the different layers in the network are concerned with the different feature information. It is worth noting that, the model can first learn some low-level features, such as colors and edges. Then, the model can learn more distinguishing and discriminative features, such as fabric texture and defects. The important feature information (defects) can be clearly seen in layer 5. Furthermore, with the different feature information in different layers, the FPN framework for building feature pyramids inside Con-Nets can achieve good feature representations.

Conclusions

In this study, deep learning has been applied to multi-class fabric defect detection. The experimental results demonstrate that the FPN model suits the processing of fabric defect datasets and can achieve effective detection performance, considering the characteristics of multi-class defect images. In future work, the FPN model can be implemented based on other state-of-the-art back-bone networks, such Resnet101 and Resnet152. The back-bone networks can further improve the detection performance of our proposed approach.

References

Mohammed Bennamoun and Adriana Bodnarova Automatic visual inspection and flaw detection in textile materials: Past, present and future. In SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218), Vol. 5, pp 4340–4343, 1998.

Fernand

Fan

Cohen Zhigang

, and Attali

Stephane

IEEE Transactions on Pattern Analysis & Machine Intelligence 1991, 8, 803–808.

Deng

Jia

, Dong

Wei

, Socher

Richard

, Li

Li-Jia

, Li

Kai

, and Fei-Fei

Imagenet: A large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition, pp 248–255, 2009.

Ross Girshick Fast r-cnn. In Proceedings of the IEEE inter-national conference on computer vision, pp 1440–1448, 2015.

Girshick

Ross

, Donahue

Jeff

, Darrell

Trevor

, and Jitendra

Malik

IEEE Transactions on Pattern Analysis & Machine Intelligence 2015, 38 (1), 142–158.

Kaiming

, Zhang

Xiangyu

, Ren

Shaoqing

, and Jian Sun Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778, 2016.

Guang-Hua

, Wang

Qing-Hui

, and Guo-Hui

Zhang

Applied Optics 2015, 54 (10), 2963–2980.

Wei

Bing

, Hao

Kaungrong

, Gao

Lei

, Xue-song

Tang

Information Sciences 2020, 541, 60–74.

Liqing

Huang

and Youshen

Xia

Joint blur kernel estimation and cnn for blind image restoration.

Neurocomputing, 2019. doi: 10.1016/j.neucom.2018.12.083

10.

Jia

Liang

, Chen, Liang

Jiuzhen

, and Zhenjie

Hou

Neurocomputing 2017, 238, 84–102.

11.

Jiang

Zhenjiao

, Mallants

Dirk

, Peeters

Luk

, Gao

Lei

, Soerensen

Camilla

, and Gregoire

Mariethoz

Hydrol. Earth Syst. Sci. 2019, 23 (6), 2561–2580.

12.

Alex

Krizhevsky

Ilya

Sutskever

, and Geoffrey

Hinton Imagenet classification with deep convolutional neural net-works. In Advances in neural information processing systems, pp 1097–1105, 2012.

13.

Wei

Bing

, Hao

Kaungrong

, Gao

Lei

, Xue-song

Tang

Neural Networks 2020, 130, 100–110..

14.

Ajay

Kumar

and Grantham

K. H. Pang

Optical Engineering 2000, 39 (12), 3176–3191.

15.

Wenyu

and Longdi

Cheng

The Journal of the Textile Institute 2014, 105 (2), 163–174.

16.

Yundong

, Zhao

Weigang

, and Jiahao

Pan

IEEE Transactions on Automation Science and Engineering 2016, 14 (2), 1256–1264.

17.

Lin

Tsung-Yi

, Dollar

Piotr

, Girshick

Ross

, He

Kaiming

, Hariharan

Bharath

, and Serge

Belongie

Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125, 2017.

18.

Mak

Kai-Ling

, Peng

, and Ka

Fai Cedric Yiu

Image and Vision Computing 2009, 27 (10), 1585–1592.

19.

Mak

Kai-Ling

, Peng

Pai

, and Ka-Fai

Cedric Yiu

Journal of Industrial & Management Optimization 2012, 8 (2), 325–341.

20.

Mei

Shuang

, Wang

Yudan

, and Guojun

Wen

Sensors 2018, 18 (4), 1064.

21.

Mei

Shuang

, Yang

Hua

, and Zhouping

Yin

IEEE Transactions on Instrumentation and Measurement 2018, 67 (6), 1266–1277.

22.

Ren

Shaoqing

, He

Kaiming

, Girshick

Ross

, and Jian

Sun

Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pp 91–99, 2015.

23.

Rubino

Cosimo

, Crocco

Marco

, and Alessio

Del Bue

IEEE Transactions on Pattern Analysis & Machine Intelligence 2017, 40 (6), 1281–1294.

24.

Tong

, Wai

Keung Wong

, and Kwong

C. K.

Neurocomputing 2016, 173, 1386–1401.

25.

Tong

, Wai

Keung Wong

, and Chun

Kit Kwong

IEEE Access 2017, 5, 5947–5964.

26.

Toyoura

Masahiro

, Igarashi

Tetsuya

, and Xiaoyang

Mao

Generating jacquard fabric pattern with visual impressions.

IEEE Transactions on Industrial Informatics, 2018.

27.

Wei

Bing

, Hao

Kuangrong

, Tang

Xue-song

, and Yongsheng

Ding

Textile Research Journal 2019, 89 (17), 3539–3555.

28.

Wei

Bing

, Hao

Kuangrong

, Tang

Xue-song

, and Lihong

Ren

Fabric defect detection based on faster rcnn. In International Conference on Artificial Intelligence on Textile and Apparel, pp 45–51. Springer, 2018.

29.

Wei

Yunchao

, Xia

Wei

, Lin

Min

, Huang

Junshi

, Ni

Bingbing

, Dong

Jian

, Zhao

Yao

, and Shuicheng

Yan

IEEE Transactions on Pattern Analysis & Machine Intelligence 2015, 38 (9), 1901-1907.

30.

Yao

Chao

, Sun

Pengfei

, Zhi

Ruicong

, and Yanfei

Shen

IEEE Access 2018, 6, 37676–37684.

31.

Zhao

Yudi

, Hao

Kuangrong

, He

Haibo

, Tang

Xuesong

, and Bing

Wei

Neurocomputing 2020, 380, 259–270.

32.

Zhu

Dandan

, Pan

Ruru

, Gao

Weidong

, and Jie

Zhang

Autex Research Journal 2015, 15 (3), 226–232.

33.

Zhu

Qiuping

, Wu

Minyuan

, Li

Jie

, and Dexiang

Deng

Textile Research Journal 2014, 84 (15), 1634–1649.

34.

Zou

Zhikang

, Cheng

, Qu

Xiaoye

, Ji

Shouling

, Guo

Xiaox-iao

, and Pan

Zhou

Neurocomputing 2019, 367, 75–83.

Multi-Class Object Learning with Application to Fabric Defects Detection

Abstract

Keywords

Introduction

Overview of FPN

FPN Architecture

RPN

ROI Pooling Layer

Classification and Regression

FPN Net-Based Fabric Defect Detection Model

Data Preparation and Augmentation

FPN Algorithm

Learning the Proposed Networks

Results and Discussion

Setup

Detection Results

Feature Analysis

Conclusions

References