Patch SVDD(support vector data description)-based channel attention embedding and improvement of classifier

Abstract

Some anomaly detection methods are based on CNN to fuse spatial and channel-wise information together within local receptive fields. However, the correlation between feature channels has not been fully utilized. Channel attention has been shown to model the interdependence between convolution feature channels and improve network representation. It is possible to introduce channel attention into anomaly detection. We attempt to directly embed the SE(Squeeze and Excitation) module into the convolutional layer but reduced anomaly detection performance. Therefore, we propose a lightweight channel attention module C-SE(Current Squeeze and Excitation) suitable for anomaly detection. C-SE module not only improves the representation ability of depth convolutional neural network but also has a significant effect on texture anomaly detection. C-SE module body is constructed by average pooling and maximum pooling branches, which ensure that local salient features of the image are not lost. Then reduce the negative impact of feature calibration through a long connection. In addition, the improvement of classifier plays an important role. Experimental results have shown that the proposed method outperforms the Patch SVDD methods by 3% in image-level AUROC and 0.7% in pixel-level AUROC on the MVTec benchmark. The higher AUROC score and the faster rate of convergence prove the effectiveness of the method.

Keywords

Anomaly detection channel attention feature calibration texture MVTec

Get full access to this article

View all access options for this article.

References

Samet Akcay , Amir Atapour-Abarghouei , Toby Breckon

Ganomaly: Semi-supervised anomaly detection via adversarial training, arXiv preprint arXiv:1805.06725.

Alec Radford , Luke Metz , Soumith Chintala Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv:1511.06434.

Jihun Yi , Sungroh Yoon Patch svdd: Patch-level svdd for anomaly detection and segmentation, In Proceedings of the Asian Conference on Computer Vision, 2020, 1, 3.

Bergmann

, Fauser

, Sattlegger

, Steger

Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4183–4192, 2020, 5.

Jie Hu , Li Shen , Samuel Albanie , Gang Sun , Enhua Wu Squeeze-and-Excitation Networks, arXiv preprint arXiv:1709.01507.

C.-L.

, Sohn

, Yoon

, Pfister

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization, arXiv preprint arXiv:2104.04015, 2021.

Tal Reiss , Niv Cohen , Liron Bergman , Yedid Hoshen PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation, arXiv preprint arXiv:2010.05903.

Defard , Aleksandr Setkov , Angelique Loesch , Romaric Audigier Padim: a patch distribution modeling framework for anomaly detection and localization, In International Conference on Pattern Recognition, pages 475–489. Springer, 2021. 1, 2, 4, 7.

Karsten Roth , Latha Pemula , Joaquin Zepeda , Bernhard Schölkopf , Thomas Brox , Peter Gehler Towards Total Recall in Industrial Anomaly Detection, arXiv preprint arXiv:2106.08265.

10.

Denis Gudovskiy , Shun Ishizaka , Kazuki Kozuka CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows, arXiv preprint arXiv:2107.12571.

11.

Jiawei Yu , Ye Zheng , Xiang Wang , Wei Li , Yushuang Wu , Rui Zhao , Liwei Wu FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows, arXiv preprint arXiv:2111.07677.

12.

Jonathan Pirnay , Keng Chai Inpainting Transformer for Anomaly Detection, arXiv preprint arXiv:2104.13897.

13.

Kaiming He , Xiangyu Zhang , Shaoqing Ren , Jian Sun Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 4, 5, 8

14.

Alexey Dosovitskiy , Lucas Beyer , Alexander Kolesnikov , Dirk Weissenborn , Xiaohua Zhai , Unterthiner , Mostafa Dehghani , Matthias Minderer , Georg Heigold , Sylvain Gelly , Jakob Uszkoreit , Neil Houlsby An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, arXiv preprint arXiv:2010.11929.

15.

Ruff

, Vandermeulen

R.A.

, Görnitz

, Deecke

, Kloft

Deep one-class classification, In International conference on machine learning, (2018), 4393–4402.

16.

Rippel

, Mertens

, Merhof

Modeling the distribution of normal data in pre-trained deep features for anomaly detection, in arXiv, 2005.14140, 2020.

17.

Lee

, Lee

, Shin

A simple unified framework for detecting out-of-distribution samples and adversarial attacks, in NIPS, 2018.

18.

Sabokrou

, Khalooei

, Fathy

, Adeli

Adversarially learned one-class classifier for novelty detection, in CVPR, 2018.

19.

Pidhorskyi

, Almohsen

, Adjeroh

D.A.

, Doretto

Generative probabilistic novelty detection with adversarial autoencoders, in NIPS, 2018.

20.

Dong Gong , Lingqiao Liu , Vuong Le , Budhaditya Saha , Moussa Reda Mansour , Svetha Venkatesh , Anton van den Hengel Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection, arXiv preprint arXiv:1904.02639.

21.

Cohen

, Hoshen

Sub-image anomaly detection with deep pyramid correspondences, in arXiv, 2005.02357, 2020.

22.

Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan Gomez

, Lukasz Kaiser , Illia Polosukhin Attention Is All You Need, arXiv preprint arXiv:1706.03762.

23.

Sanghyun Woo , Jongchan Park , Joon-Young Lee , In So Kweon CBAM: Convolutional Block Attention Module, arXiv preprint arXiv:1807.06521.

24.

Bergmann

, Fauser

, Sattlegger

, Steger

MVTec AD –A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 9592–9600.

25.

Calders

, Jaroszewicz

Efficient auc optimization for classification, in European Conference on Principles of Data Mining and Knowledge Discovery, 2007.

26.

Vitjan Zavrtanik , Matej Kristan , Danijel Skocaj , Reconstruction by inpainting for visual anomaly detection, In Pattern Recognition 112 (2021), 107706, 1, 2, 3, 6, 7.

27.

Kaitai Zhang , Bin Wang , Jay Kuo,

C.-C.

PEDENet: Image Anomaly Localization via Patch Embedding and Density Estimation, arXiv preprint arXiv:2110.15525.

28.

Chaoqin Huang , Haoyan Guan , Aofan Jiang , Ya Zhang , Michael Spratling , Yan-Feng Wang Registration 5 based Few-Shot Anomaly Detection, arXiv preprint arXiv:2207.07361.

29.

Rudolph

, Wandt

, Rosenhahn

Same same but differnet: Semi-supervised defect detection with normalizing flows, In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, 1907–1916.

30.

Danilo Jimenez Rezende , Shakir Mohamed Variational inference with normalizing flows, arXiv preprint arXiv:1505.05770.