Concurrently detecting multiple objects of interest will yield massive time savings in processing and enable a more streamlined and unified detection system. The ShuttleNet is designed to repeat the encoding–decoding round freely or even endlessly, achieving prodigious successes in terms of simultaneous detection of multiple pavement distresses and surface design features on asphalt pavements. This paper proposes an efficient and improved architecture of ShuttleNet called ShuttleNetV2 for enhanced global modeling and retrieving fine details capabilities. The proposed ShuttleNetV2 represents two major modifications on the original ShuttleNet. On the one hand, the self-attention mechanism is purposefully introduced to capture long-range dependency. On the other hand, ShuttleNetV2 adopts various sampling scales to combine the characteristics of different receptive fields. The experimental results indicate that the recommended architectural variation of the proposed ShuttleNetV2 model yields a mean F-measure of 94.21% and a mean intersection-over-union of 0.8914 on 1500 pairs of testing images. The proposed ShuttleNetV2 outperforms ShuttleNet in detecting nearly all types of pavement patterns. In particular, ShuttleNetV2 efficaciously tackles the tangible limitations of ShuttleNet in detecting giant distresses. Moreover, the ShuttleNetV2 can process an image in roughly 78 ms using modern graphic processing unit devices, which has a promising potential in supporting the real-time detection of multiple pavement distresses and surface design features on asphalt pavements.
YangXLiHYuY, et al. Automatic pixel-level crack detection and measurement using fully convolutional network. Comput-Aided Civil Infrastruct Eng2018; 33(12): 1090–1109.
2.
JangKKimNAnYK. Deep learning–based autonomous concrete crack evaluation through hybrid image scanning. Struct Health Monit2019; 18(5–6): 1722–1737.
3.
ChowJKSuZWuJ, et al. Artificial intelligence-empowered pipeline for image-based inspection of concrete structures. Autom Constr2020; 120: 103372.
4.
ZhangAWangKCLiB, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput-Aided Civil Infrastruct Eng2017; 32(10): 805–819.
5.
ZhangAWangKCFeiY, et al. Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. J Comput Civil Eng2018; 32(5): 04018041.
6.
ZhangAWangKCFeiY, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network. Comput-Aided Civil Infrastruct Eng2019; 34(3): 213–229.
7.
GavilánMBalconesDMarcosO, et al. Adaptive road crack detection system by pavement classification. Sensors2011; 11(10): 9628–9657.
8.
ZhangAWangKCAiC. 3D Shadow modeling for detection of descended patterns on 3D pavement surface. J Comput Civil Eng2017; 31(4): 04017019.
9.
YangFZhangLYuS, et al. Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans Intell Transp Syst2019; 21(4): 1525–1535.
10.
ZouQZhangZLiQ, et al. DeepCrack: learning hierarchical convolutional features for crack detection. IEEE Trans Image Process2018; 28(3): 1498–1512.
11.
HuyanJLiWTigheS, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit2020; 27(8): e2551.
12.
DungCVAnhLD. Autonomous concrete crack detection using deep fully convolutional neural network. Autom Constr2019; 99: 52–58.
13.
GuanJYangXDingL, et al. Automated pixel-level pavement distress detection based on stereo vision and deep learning. Autom Constr2021; 129: 103788.
14.
HuyanJLiWTigheS, et al. Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network. Autom Constr2019; 107: 102946.
15.
HassanSIO’SullivanDMcKeeverS, et al. Detecting patches on road pavement images acquired with 3D laser sensors using object detection and deep learning. In: 17th International joint conference on computer vision, imaging and computer graphics theory and applications (VISI- GRAPP), Online, 6 Feb–8 Feb 2022, pp. 413–420.
16.
WenCSunXLiJ, et al. A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds. ISPRS-J Photogramm Remote Sens2019; 147: 178–192.
17.
KimIBChoJSZiGS, et al. Detection and identification of expansion joint gap of road bridges by machine learning using line-scan camera images. Appl Syst Innov2021; 4(4): 94.
18.
WangWZhangAWangKC, et al. Pavement crack width measurement based on Laplace’s equation for continuity and unambiguity. Comput-Aided Civil Infrastruct Eng2018; 33(2): 110–123.
19.
ZhangAWangKCLiuY, et al. Intelligent pixel-level detection of multiple distresses and surface design features on asphalt pavements. Comput-Aided Civil Infrastruct Eng2022; 37(13): 1654–1673.
20.
ChenLCPapandreouGKokkinosI, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arXiv:1606.00915, 2017.
21.
ChenLCPapandreouGSchroffF, et al. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017.
22.
ChenLCZhuYPapandreouG, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv:1802.02611, 2018.
23.
VaswaniAShazeerNParmarN, et al. Attention is all you need. arXiv:1706.03762, 2017;: 1–15.
24.
DosovitskiyABeyerLKolesnikovA, et al. An image is worth 16×16 words: transformers for image recognition at scale. arXiv:2010.11929, 2020.
25.
WangWXieELiX, et al. Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: 2021 IEEE/CVF international conference on computer vision (ICCV), Montreal, QC, 11 Oct–17 Oct 2021, pp. 568–578.
26.
LiuZLinYCaoY, et al. Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF international conference on computer vision (ICCV), Montreal, QC, 11 Oct–17 Oct 2021, pp. 10012–10022.
27.
TouvronHCordMDouzeM, et al. Training data-efficient image transformers distillation through attention. In: In: International conference on machine learning (ICML), Online, 18 Jul–24 Jul 2021, pp. 10347–10357.
28.
LiJXiaXLiW, et al. Next-ViT: next generation vision transformer for efficient deployment in realistic industrial scenarios. arXiv:2207.05501, 2022.
29.
MaNZhangXZhengHT, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design. arXiv:1807.11164, 2018.
30.
LiRZhengSZhangC, et al. ABCNet: attentive bilateral contextual network for efficient semantic segmentation of fine-resolution remotely sensed imagery. ISPRS-J Photogramm Remote Sens2021; 181: 84–98.
31.
YuWLuoMZhouP, et al. MetaFormer is actually what you need for vision. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), New Orleans, LA, 18 Jun–24 Jun 2022, pp. 10809–10819.
32.
HeKZhangXRenS, et al. 2016. Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, WA, 27 Jun–30 Jun 2016, pp. 770–778.
33.
LinTYGoyalPGirshickR, et al. Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell2017; 42(2): 318–327.
34.
AbadiMAgarwalABarhamP, et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467, 2016.
35.
ShiYCuiLQiZ, et al. Automatic road crack detection using random structured forests. IEEE Trans Intell Transp Syst2016; 17(12): 3434–3445.
36.
HeKChenXXieS, et al. Masked autoencoders are scalable vision learners. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), New Orleans, LA, 18 Jun–24 Jun 2022, pp. 15979–15988.
37.
XieZZhangZCaoY, et al. SimMIM: a simple framework for masked image modeling. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), New Orleans, LA, 18 Jun–24 Jun 2022, pp. 9643–9653.
