Regular detection and cleaning of carbon deposits in automotive engines can help maintain normal vehicle operation and reduce the negative impact on the environment. However, traditional detection methods are unable to meet the needs of large-scale automotive inspection. This study employs deep learning techniques for the intelligent detection of internal engine carbon deposits captured by endoscopes. However, many challenges including large intra-class variance, limited inter-class variance, and small sample sizes in certain categories are induced by the dataset of carbon deposits poses. Traditional data augmentation methods tend to obscure key semantic information, while the classification efficiency of conventional models remains low. In this study, an Adaptive Intra-class Part Swapping (A-INPS) method was introduced to expand the dataset by randomly substituting key discriminative regions within the same class using Grad-CAM. Additionally, a Feature Refinement Attention Module (FRAM)-RepVGG model was proposed, leveraging RepVGG to achieve lossless compression and enhance detection efficiency. The FRAM incorporated in the model enhances focusing and reduces redundant feature extraction in terms of space and channel. The experimental testing results show that the algorithm has an accuracy of 91.59% and a superior detection speed of 57.55FPS on GTX1080Ti, which is more suitable for deployment in real industrial environments.
WangD.Engine carbon deposition detection system and method, 2018.
2.
QianFWangTQianC, et al. Measurement device for determining carbon deposits distribution in cylinder based on knocking sound of internal combustion engine, 2023.
3.
WangJDingWDengJ, et al. Method for detecting and distinguishing early combustion and carbon deposits of engine based on ion current, 2023.
4.
LuMSituBZhaoY, et al. Inside carbon deposits thickness detection device of fuel engine, 2018.
5.
ShangC.Device for monitoring vehicle carbon deposits by using internet-of-things sensor, 2018.
6.
XiaoJJiMLanL.Engine detection method and system, computer equipment and storage medium, 2021.
7.
LiXSongQWuJ, et al. Locally-enriched cross-reconstruction for few-shot fine-grained image classification. IEEE Trans Circuits Syst Video Technol2023; 33(12): 7530–7540.
8.
KayaMBilgeHŞ.Deep metric learning: a survey. Symmetry2019; 11(9): 1066.
9.
YanJLuoLDengC, et al. Adaptive hierarchical similarity metric learning with noisy labels. IEEE Trans Image Process2023; 32: 1245–1256.
10.
UllahICarrión-OjedaDEscaleraS, et al. Meta-album: multi-domain meta-dataset for few-shot image classification. Adv Neural Inf Process Syst2022; 35: 3232–3247.
11.
ZhengZWuHLvL, et al. Iccl: Independent and correlative correspondence learning for few-shot image classification. Knowl Based Syst2023; 266: 110412.
12.
PintelasELivierisIEPintelasP.Adaptive augmentation framework for domain independent few shot learning. Knowl Based Syst2024; 299: 112047.
13.
ZhangHCisseMDauphinY, et al. Mixup: beyond empirical risk minimization. In: International conference on learning representations (ICLR), Vancouver, BC, Canada, 30 April-3 May 2018, pp. 1–13.
14.
YunSHanDOhS, et al. Cutmix: regularization strategy to train strong classifiers with localizable features. In: IEEE/CVF international conference on computer vision (ICCV), Seoul, Korea, 27 October–2 November 2019, pp. 6023–6032.
15.
UddinAMoniraMShinW, et al. Saliencymix: a saliency guided data augmentation strategy for better regularization. In: 9th international conference on learning representations (ICLR). 3–7 May 2021, Online.
16.
HuangSWangXTaoD. Snapmix: semantically proportional mixing for augmenting fine-grained data. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35, 19–21 May 2021, New York, NY, USA, pp. 1628–1636.
17.
ZhangLHuangSLiuW. Intra-class part swapping for fine-grained image classification. In: IEEE/CVF winter conference on applications of computer vision (WACV), Waikoloa, HI, USA, 3–8 January 2021, pp. 3209–3218.
18.
ZhangZCChenZDXieZY, et al. S3mix: Same category same semantics mixing for augmenting fine-grained images. ACM Trans Multimed Comput Commun Appl2024; 20(1): 1–16.
19.
VidalCKollmeyerPChemaliE, et al. Li-ion battery state of charge estimation using long short-term memory recurrent neural network with transfer learning. In: IEEE transportation electrification conference and expo (ITEC), Detroit, USA, 19–21 June 2019, pp. 1–6.
20.
TanYZhaoG.Transfer learning with long short-term memory network for state-of-health prediction of lithium-ion batteries. IRE Trans Ind Electron2020; 67(10): 8723–8731.
21.
DosovitskiyABeyerLKolesnikovA, et al. An image is worth 16x16 words: transformers for image recognition at scale. In: 9th international conference on learning representations (ICLR). 3–7 May 2021. Online.
22.
LiuZLinYCaoY, et al. Swin transformer: hierarchical vision transformer using shifted windows. In: IEEE/CVF international conference on computer vision (ICCV), Montreal, Canada, 10–17 October 2021, pp. 10012–10022.
23.
LiuMZhangCBaiH, et al. Cross-part learning for fine-grained image classification. In: IEEE transactions on image processing, vol. 31, 2021, pp. 748–758.
24.
ZhangKLiuNYuanX, et al. Fine-grained age estimation in the wild with attention LSTM networks. IEEE Trans Circuits Syst Video Technol2020; 30(9): 3140–3152.
25.
LinZJiaJGaoW, et al. Fine-grained visual categorization of butterfly specimens at sub-species level via a convolutional neural network with skip-connections. Neurocomputing2020; 384: 295–313.
26.
LiAKangBChenJ, et al. Global information-assisted fine-grained visual categorization in internet of things. IEEE Internet Things J2023; 10(1): 940–952.
27.
BeheraAWhartonZLiuY, et al. Regional attention network (ran) for head pose and fine-grained gesture recognition. IEEE Trans Affect Comput2023; 14(1): 549–562.
28.
KrizhevskyASutskeverIHintonG. Imagenet classification with deep convolutional neural networks. In: Proceedings of advances in neural information processing systems (NIPS), Lake Tahoe, NV, USA, 3–6 December 2012, pp. 106–1114.
29.
SimonyanKZissermanA. Very deep convolutional networks for large scale image recognition. In: International conference on learning representations (ICLR), San Diego, CA, USA, 7–9 May 2015, pp. 1–14.
30.
SzegedyCLiuWJiaY, et al. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, USA, 7–12 June 2015, pp. 1–9.
31.
TanMLeQ. Efficientnet: rethinking model scaling for convolutional neural networks. In: International conference on machine learning, Long Beach, CA, USA, 9–15 June 2019, pp. 6105–6114.
32.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016.
33.
DingXZhangXMaN, et al. Repvgg: making vgg-style convnets great again. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Nashville, TN, USA, 20–25 June, 2021, pp. 13733–13742.
34.
WangXYangSXiaoY, et al. A vehicle classification model based on deep active learning. Pattern Recognit Lett2023; 171: 84–91.
35.
ZhangQShuJChenC, et al. Optimization of pneumonia ct classification model using repvgg and spatial attention features. Front Med2023; 10: 1233724.
36.
Co SJIS. Engine testing methods, systems, computer equipment and storage media, 2021.
37.
Co SJIS. Automotive digital image testing and diagnosis workstation, 2021.
38.
Co SJIS. An engine cleaning device, 2022.
39.
SelvarajuRCogswellMDasA, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision (ICCV), Venice, Italy, 22–29 October 2017, pp. 618–626.
40.
HouQZhouDFengJ. Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Nashville, TN, USA, 19–25 June 2021, pp. 13708–13717.
41.
LiJWenYHeL. Scconv: spatial and channel reconstruction convolution for feature redundancy. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Vancouver, Canada, 17–24 June 2023, pp. 6153–6162.
