In structural health monitoring (SHM), there is an increasing demand for real-time image-based damage detection. Such a technology is essential for minimizing hazard loss caused by delayed emergency response after earthquakes or other natural disasters, or service interruption during structural inspection. Compressive sampling (CS) is a promising solution to achieve such a goal by greatly reducing the power consumption on high-resolution image transmission when using wireless devices. However, conventional CS failed to achieve high enough compression ratios, while existing generative-model-based CS requires laboriously training a high-quality generator with many large-scale images. To overcome such a bottleneck that hinders the practical use of CS in SHM, we propose a multitask CS algorithm that only relies on existing generators trained by low-pixel crack images. By exploiting the new discovery that similar crack images share a similar sparsity pattern in their latent vectors mapped by the generator, our algorithm achieves higher crack detection accuracy and robustness within a much shorter time when using a high data compression ratio. We verify the effectiveness of the proposed CS algorithm using synthetic and real image data. The results demonstrate that this work has moved a step closer toward successful implementation of operational CS-based crack detection systems in real-time SHM.
BaoYChenZWeiS, et al. The state of the art of data science and engineering in structural health monitoring. Engineering2019; 5: 234–242.
2.
SpencerBFHoskereVNarazakiY.Advances in computer vision-based civil infrastructure inspection and monitoring. Engineering2019; 5: 199–222.
3.
AkbarMAQidwaiUJahanshahiMR.An evaluation of image-based structural health monitoring using integrated unmanned aerial vehicle platform. Struct Control Health Monit2018; 26: e2276.
4.
LaHMDinhTHPhamNH, et al. Automated robotic monitoring and inspection of steel structures and bridges. Robotica2018; 37: 947–967.
5.
ZhangLYangFDaniel ZhangY, et al. Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016, pp. 3708–3712. Piscataway, NJ: IEEE.
6.
SchmuggeSJRiceLNguyenNR, et al. Detection of cracks in nuclear power plant using spatial-temporal grouping of local patches. In: 2016 IEEE winter conference on applications of computer vision (WACV), Lake Placid, NY, USA, 07–10 March 2016, pp. 1–7. Piscataway, NJ: IEEE.
7.
ChaY-JChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput-Aided Civ Infrastruct Eng2017; 33: 731–747.
8.
ZouQZhangZLiQ, et al. DeepCrack: learning hierarchical convolutional features for crack detection. IEEE Trans Image Process2019; 28: 1498–1512.
9.
YangXLiHYuY, et al. Automatic pixel-level crack detection and measurement using fully convolutional network. Comput-Aided Civ Infrastruct Eng2018; 33: 1090–1109.
10.
HsiehY-ATsaiYJ.Machine learning for crack detection: review and model performance comparison. J Comput Civil Eng2020; 34: 04020038.
11.
JiangSZhangJ.Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system. Comput-Aided Civ Infrastruct Eng2019; 35: 549–564.
12.
TorokMMGolparvar-FardMKochersbergerKB.Image-based automated 3D crack detection for post-disaster building assessment. J Comput Civil Eng2014; 28: A4014004.
13.
Ghosh MondalTJahanshahiMRWuR, et al. Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance. Struct Control Health Monit2020; 27: e2507.
14.
MunawarHSHammadAWAHaddadA, et al. Image-based crack detection methods: a review. Infrastructures2021; 6: 115.
15.
RensKLWipfTJKlaiberFW.Review of nondestructive evaluation techniques of civil infrastructure. J Perform Constr Facil1997; 11: 152–160.
16.
SolimanMFrangopolDMMondoroA.A probabilistic approach for optimizing inspection, monitoring, and maintenance actions against fatigue of critical ship details. Struct Saf2016; 60: 91–101.
17.
MohanAPoobalS.Crack detection using image processing: a critical review and analysis. Alex Eng J2018; 57: 787–798.
18.
YaoYTungS-TEGlisicB.Crack detection and characterization techniques-an overview. Struct Control Health Monit2014; 21: 1387–1413.
19.
KangDChaY-J.Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Comput-Aided Civ Infrastruct Eng2018; 33: 885–902.
20.
CandesEJRombergJTaoT.Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory2006; 52: 489–509.
21.
CandèsE.Compressive sampling. In Proceedings of the international congress of mathematicians, Madrid, Spain, 22–30 August 2006, pp. 1433–1452. Berlin, Germany: European Mathematical Society (EMS) Press.
22.
DonohoDL.Compressed sensing. IEEE Trans Inf Theory2006; 52: 1289–1306.
23.
JayantN.Signal compression: technology targets and research directions. IEEE J Sel Areas Commun1992; 10: 796–818.
24.
WallaceGK.The JPEG still picture compression standard. IEEE Trans Consum Electron1992; 38: xviii–xxxiv.
25.
MaSZhangXJiaC, et al. Image and video compression with neural networks: a review. IEEE Trans Circuits Syst Video Technol2020; 30: 1683–1698.
26.
OikeYEl GamalA.CMOS image sensor with per-column ΣΔ ADC and programmable compressed sensing. IEEE J Solid-State Circuits2013; 48: 318–328.
DadkhahMDeenMJShiraniS.CMOS image sensor with area-efficient block-based compressive sensing. IEEE Sens J2015; 15: 3699–3710.
29.
ShannonCE.Communication in the presence of noise. Proc IRE1949; 37: 10–21.
30.
BaoYLiHOuJ.Emerging data technology in structural health monitoring: compressive sensing technology. J Civ Struct Health2014; 4: 77–90.
31.
KougioumtzoglouIAPetromichelakisIPsarosAF.Sparse representations and compressive sampling approaches in engineering mechanics: a review of theoretical concepts and diverse applications. Probab Eng Mech2020; 61: 103082.
32.
BaoYBeckJLLiH.Compressive sampling for accelerometer signals in structural health monitoring. Struct Health Monit2010; 10: 235–246.
33.
HuangYBeckJLWuS, et al. Robust bayesian compressive sensing for signals in structural health monitoring. Comput-Aided Civ Infrastruct Eng2013; 29: 160–179.
34.
AkcakayaMTarokhV.Shannon-theoretic limits on noisy compressive sampling. IEEE Trans Inf Theory2010; 56: 492–504.
35.
GilbertACStraussMJTroppJA, et al. One sketch for all: fast algorithms for compressed sensing. In: Proceedings of the thirty-ninth annual ACM symposium on theory of computing, San Diego, CA, USA, June2007, pp. 237–246. New York, NY: Association for Computing Machinery (ACM).
36.
BoraAJalalAPriceE, et al. Compressed sensing using generative models. In: Proceedings of the 34th international conference on machine learning, Sydney, NSW, Australia, July2017, pp. 537–546. JMLR.org.
37.
YangYNagarajaiahS.Robust data transmission and recovery of images by compressed sensing for structural health diagnosis. Struct Control Health Monit2016; 24: e1856.
38.
HuangYZhangHLiH, et al. Recovering compressed images for automatic crack segmentation using generative models. Mech Syst Signal Process2021; 146: 107061.
39.
ZhaoSLiuZLinJ, et al. Differentiable augmentation for data-efficient gan training. In: Proceedings of the 34th international conference on neural information processing systems, Vancouver, BC, Canada, December2020, pp. 7559–7570. Red Hook, NY: Curran Associates Inc.
40.
BaraniukR.Compressive sensing [Lecture Notes]. IEEE Signal Process Mag2007; 24: 118–121.
41.
GoodfellowIPouget-AbadieJMirzaM, et al. Generative adversarial networks. Commun ACM2020; 63: 139–144.
DharMGroverAErmonS.Modeling sparse deviations for compressed sensing using generative models. In: International conference on machine learning, Stockholmsmssan, STO, Sweden, 10–15 July 2018, pp. 1214–1223. JMLR.org.
44.
KilledarVPokalaPKSekhar SeelamantulaC.Sparsity driven latent space sampling for generative prior based compressive sensing. In: 2021–2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), Toronto, ON, Canada, 06–11June2021, pp. 2895–2899. Piscataway, NJ: IEEE.
45.
RadfordAMetzLChintalaS.Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint2015; arXiv:1511.06434.
KarrasTLaineSAittalaM, et al. Analyzing and improving the image quality of StyleGAN. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, USA, 13–19 June 2020, pp. 8107–8116. Piscataway, NJ: IEEE.
49.
KarrasTAittalaMLaineS, et al. Alias-free generative adversarial networks. In: Thirty-fifth conference on neural information processing systems (NeurIPS), Vancouver, BC, Canada, 07–10 December 2021, pp. 852–863. Red Hook, NY: Curran Associates, Inc.
50.
JiSXueYCarinL.Bayesian compressive sensing. IEEE Trans Signal Process2008; 56: 2346–2356.
51.
LiuJJiSYeJ.Multi-task feature learning via efficient l2, 1-norm minimization. In: Proceedings of the 25th conference on uncertainty in artificial intelligence, Montreal, QC, Canada, June2009, pp. 339–348. Arlington, VA: AUAI Press.
52.
JalalALiuLDimakisAG, et al. Robust compressed sensing using generative models. In: Proceedings of the 34th international conference on neural information processing systems, Vancouver, BC, Canada, 06–12 December 2020, pp. 713–727. Red Hook, NY: Curran Associates Inc.
53.
ÖzgenelCFSorguçAG.Performance comparison of pretrained convolutional neural networks on crack detection in buildings. In: Proceedings of the international symposium on automation and robotics in construction, Berlin, Germany, 20–25 July 2018, pp. 1–8. International Association for Automation and Robotics in Construction (IAARC).
54.
LiuYYaoJLuX, et al. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing2019; 338: 139–153.
55.
ZhangXRajanDStoryB.Concrete crack detection using context-aware deep semantic segmentation network. Comput-Aided Civ Infrastruct Eng2019; 34: 951–971.
56.
DaisDBalİESmyrouE, et al. Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning. Autom Constr2021; 125: 103606.
57.
OngJCLauSLIsmadiM-Z, et al. Feature pyramid network with self-guided attention refinement module for crack segmentation. Struct Health Monit2022; 22: 672–688.
58.
LiuZElezzabiAYZhaoHV. Maximum frame rate video acquisition using adaptive compressed sensing. IEEE Trans Circuits Syst Video Technol2011; 21: 1704–1718.
59.
HeuselMRamsauerHUnterthinerT, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of the 31st international conference on neural information processing systems, Long Beach, CA, USA, 04–09 December 2017, pp. 6629–6640. Red Hook, NY: Curran Associates Inc.
60.
PaszkeAGrossSMassaF, et al. PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of the 33rd international conference on neural information processing systems, Vancouver, BC, Canada, December2019, pp. 8026–8037. Red Hook, NY: Curran Associates Inc.
61.
SzegedyCVanhouckeVIoffeS, et al. Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016, pp. 2818–2826. Piscataway, NJ: IEEE.
62.
NieFHuangHCaiX, et al. Efficient and robust feature selection via joint ℓ2, 1-norms minimization. In: Proceedings of the 23rd international conference on neural information processing systems, Vancouver, BC, Canada, 06–09 December 2010, pp. 1813–1821. Red Hook, NY: Curran Associates Inc.
