When deep learning is applied to intelligent textile defect detection, the insufficient training data may result in low accuracy and poor adaptability of varying defect types of the trained defect model. To address the above problem, an enhanced generative adversarial network for data augmentation and improved fabric defect detection was proposed. Firstly, the dataset is preprocessed to generate defect localization maps, which are combined with non-defective fabric images and input into the network for training, which helps to better extract defect features. In addition, by utilizing a Double U-Net network, the fusion of defects and textures is enhanced. Next, random noise and the multi-head attention mechanism are introduced to improve the model’s generalization ability and enhance the realism and diversity of the generated images. Finally, we merge the newly generated defect image data with the original defect data to realize the data enhancement. Comparison experiments were performed using the YOLOv3 object detection model on the training data before and after data enhancement. The experimental results show a significant accuracy improvement for five defect types – float, line, knot, hole, and stain – increasing from 41%, 44%, 38%, 42%, and 41% to 78%, 76%, 72%, 67%, and 64%, respectively.
SongLLiRChenS.Fabric defect detection based on membership degree of regions. IEEE Access2020;
8: 48752–48760.
2.
HanbayKTaluMFÖzgüvenÖF.Fabric defect detection systems and methods—a systematic literature review. Optik2016;
127: 11960–11973.
3.
LiYZhangDLeeDJ.Automatic fabric defect detection with a wide-and-compact network. Neurocomputing2019;
329: 329–338.
4.
ZhaoYHaoKHeH, et al.
A visual long-short-term memory based integrated CNN model for fabric defect image classification. Neurocomputing2020;
380: 259–270.
5.
AlomMZTahaTMYakopcicC, et al.
The history began from alexnet: a comprehensive survey on deep learning approaches. arXiv preprint arXiv:1803.01164, 2018.
6.
OuyangWXuBHouJ, et al.
Fabric defect detection using activation layer embedded convolutional neural network. IEEE Access2019;
7: 70130–70140.
7.
GoodfellowIPouget-AbadieJMirzaM, et al.
Generative adversarial nets. In: Z Ghahramani, M Welling, C Cortes, et al. (eds) Neural Information Processing Systems, Montreal, Canada, 8–11 December 2014, paper no. 1384, p. 27. Massachusetts: MIT Press.
8.
HuGHuangJWangQ, et al.
Unsupervised fabric defect detection based on a deep convolutional generative adversarial network. Text Res J2019;
90: 247–270.
9.
ParkSWHuhJHKimJC.BEGAN v3: avoiding mode collapse in GANs using variational inference. Electronics2020;
9: 688.
10.
DubeyAFuchsJLübkeM, et al. Generative adversial network based extended target detection for automotive MIMO radar. In: 2020 IEEE international radar conference (RADAR), Washington, USA, 28–30 April 2020, pp. 220–225. New York: IEEE.
11.
HuangYWangZ.Fabric defect segmentation method based on deep learning. IEEE Trans Instrum Meas2021;
70: 1–15.
12.
ChenMYuLZhiC, et al.
Improved faster R-CNN for fabric defect detection based on Gabor filter with Genetic Algorithm optimization. Comput Ind2022;
134: 103551.
13.
MaharanaKMondalSNemadeB.A review: data pre-processing and data augmentation techniques. Glob Transit Proc2022;
3: 91–99.
14.
InoueH.Data augmentation by pairing samples for images classification. arxiv preprint arxiv:1801.02929, 2018.
ChadebecCThibeau-SutreEBurgosN, et al.
Data augmentation in high dimensional low sample size setting using a geometry-based variational autoencoder. IEEE Trans Patt Anal Mach Intell2022;
45: 2879–2896.
17.
ZhaoAXA.Learning distributions of transformations from small datasets for applied image synthesis.
Massachusetts Institute of Technology, 2019.
18.
EdeJM.Deep learning in electron microscopy. Mach Learn Sci Technol2021;
2: 011004.
19.
SuárezPLSappaADVintimillaBX. Infrared image colorization based on a triplet DCGAN architecture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Hawaii, USA, 21–26 July 2017, pp. 18–23. New York: IEEE.
20.
HuangGYuanYXuQ, et al. An empirical study on evaluation metrics of generative adversarial networks. In: 6th International Conference on Learning Representations (ICLP), Vancouver, Canada, 30 April–3 May 2018, OpenReview.net.
21.
ChenYHouX. An improvement based on Wasserstein GAN for alleviating mode collapsing. In: 2020 international joint conference on neural networks (IJCNN), Glasgow, UK, 19–24 July 2020, pp. 1–7. New York: IEEE.
22.
ZhaoYTakakiSLuongHT, et al.
Wasserstein GAN and waveform loss-based acoustic model training for multi-speaker text-to-speech synthesis systems using a WaveNet vocoder. IEEE Access2018;
6: 60478–60488.
23.
LeeJLeeH.Improving SSH detection model using IPA time and WGAN-GP. Comput Secur2022;
116: 102672.
24.
WangCCaoYZhangS, et al.
A reconstruction method for missing data in power system measurement based on LSGAN. Front Energ Res2021;
9: 651807.
25.
IsolaPZhuJ-YZhouT, et al. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Hawaii, USA, 21-26 July 2017, pp. 1125–1134. New York: IEEE.
26.
Richardson E, Alaluf Y, Patashnik O, et al. Encoding in style: A stylegan encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), virtually, 19–25 June 2021, pp. 2287–2296. New York: IEEE.
27.
QuYChenYHuangJ, et al. Enhanced pix2pix dehazing network. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). California, USA, 15–20 June 2019, pp. 8160–8168. New York: IEEE.
28.
BlochLBrüngelRFriedrichCM. Boosting efficientnets ensemble performance via pseudo-labels and synthetic images by pix2pixHD for infection and Ischaemia classification in diabetic foot ulcers. In: MH Yap, B Cassidy and C Kendrick (eds) Diabetic foot ulcers grand challenge, Strasbourg, France, 27 September 2021, vol. 13183, pp. 30–49. Switzerland: Springer, Cham.
29.
ZhangZWangJWeiD, et al.
An improved temporal convolutional network with attention mechanism for photovoltaic generation forecasting. Eng Appl Artif Intell2023;
123: 106273.
30.
WangSHFernandesSLZhuZ, et al.
AVNC: attention-based VGG-style network for COVID-19 diagnosis by CBAM. IEEE Sens J2021;
22: 17431–17438.
31.
ShortenCKhoshgoftaarTM.A survey on image data augmentation for deep learning. J Big Data2019;
6: 1–48.
32.
XieHWuZ.A robust fabric defect detection method based on improved RefineDet. Sensors2020;
20: 4260.
33.
WooSParkJLeeJY, et al. CBAM: convolutional block attention module. In: V Ferrari, M Hebert, C Sminchisescu, et al. (eds) European Conference on Computer Vision, Munich, Germany, 8–14 September 2018, vol. 11211, pp. 3–19. Switzerland: Springer, Cham.
34.
HuJShenLSunG.Squeeze-and-excitation networks. IEEE Trans Patt Anal Mach Intell2020;
42: 2011–2023.
35.
ZhengKYangDLiuJ, et al.
Recognition of teachers’ facial expression intensity based on convolutional neural network and attention mechanism. IEEE Access2020;
8: 226437–226444.
36.
Naranjo-AlcazarJPerez-CastanosSZuccarelloP, et al.
Acoustic scene classification with squeeze-excitation residual networks. IEEE Access2020;
8: 112287–112296.
37.
WangSHFernandesSLZhuZ, et al.
AVNC: attention-based VGG-style network for COVID-19 diagnosis by CBAM. IEEE Sens J2021;
22: 17431–17438.
38.
ShortenCKhoshgoftaarTM.A survey on image data augmentation for deep learning. J Big Data2019;
6: 1–48.
GuoQHuangJLongN, et al.
MS-pointer network: abstractive text summary based on multi-head self-attention. IEEE Access2019;
7: 138603–138613.
41.
ZhangZShaoNGaoC, et al.
Mixhead: breaking the low-rank bottleneck in multi-head attention language models. Knowl Based Syst2022;
240: 108075.
42.
RezaSFerreiraMCMachadoJJM, et al.
A multi-head attention-based transformer model for traffic flow forecasting with a comparative analysis to recurrent neural networks. Exp Syst Appl2022;
202: 117275.
43.
ZhongTZhangSZhouF, et al.
Hybrid graph convolutional networks with multi-head attention for location recommendation. World Wide Web2020;
23: 3125–3151.
44.
ParmarNVaswaniAUszkoreitJ, et al. Image transformer. In: N Lawrence (ed) Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018, vol. 80, pp. 4055–4064. New York: PMLR.
45.
WangXGirshickRGuptaA, et al. Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Utah, USA, 18–22 June 2018, pp. 7794–7803. New York: IEEE.
