Circular tubes and ball joints are widely used in large-span grid structures. Bending of circular tubes is a common form of damage in grid structures, posing a potential risk of global safety. Traditional manual inspections are labor and time cost, and usually need high-altitude operation. To address this issue, an automatic bent tube detection method based on computer vision and deep learning is proposed in this article. In the method, ball joints in the image are identified and localized using YOLOv5s object-detection network, and then the tubes are identified by semantic segmentation conducted by a DeepLab v3+ network enhanced with the contextual transformer block. The axes of the bent tubes are extracted from the segmented masks by the skeletonization algorithms. Finally, bent-tube deflections are assessed as the perpendicular distance from the bent-tube axis to the joint anchor box centerline. The proposed framework is validated by eccentric compression tests on three circular tubes with ball joints. It is shown that the average error in deflection assessment by the proposed framework is less than 2 mm, with a maximum error rate of 3.7% which is deemed acceptable in engineering application.
HuybrechtsSTsaiS W.Analysis and behavior of grid structures. Compos Sci Technol1996; 56(9): 1001–1015.
2.
DongSZhaoYXingD.Application and development of modern long-span space structures in China. Front Struct Civ Eng2012; 6: 224–239.
3.
LuoYChenYWanHP, et al. Development of laser-based displacement monitoring system and its application to large-scale spatial structures. J Civ Struct Health Monit2021; 11: 381–395.
4.
WangYCLuoYZSunB, et al. Field measurement system based on a wireless sensor network for the wind load on spatial structures: design, experimental, and field validation. Struct Control Health Monit2018; 25(9): e2192.
5.
LiuHYingJChenZ, et al. Research status of mechanical properties of aluminum alloy grid structure. Structures2024; 61: 105967.
6.
WeiXCFanJSLiuYF, et al. Automated inspection and monitoring of member deformation in grid structures. Comput Aided Civ Infrastruct Eng2022; 37(10): 1277–1297.
7.
LiuYFLiuXGFanJS, et al. Refined safety assessment of steel grid structures with crooked tubular members. Autom Constr2019; 99: 249–264.
8.
YaoTTabyJMoanT.Ultimate strength and post-ultimate strength behavior of damaged tubular members in offshore structures. J Offshore Mech Arct Eng1988; 110(3): 254–262.
9.
GB 50292-2015. Standard for appraisal of reliability of civil buildings. Beijing: China Architecture and Building Press, 2015 (in Chinese).
10.
NadhimEAHonCXiaB, et al. Falls from height in the construction industry: a critical review of the scientific literature. Int J Environ Res Public health2016; 13(7): 638.
11.
Thoft-ChristensenP.Infrastructures and life-cycle cost-benefit analysis. Struct Infrastruct Eng2012; 8(5): 507–516.
12.
ChengJCPWangM. Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom Constr2018; 95: 155–171.
13.
Jin LimHHwangSKimH, et al. Steel bridge corrosion inspection with combined vision and thermographic images. Struct Health Monit2021; 20(6): 3424–3435.
14.
ZhangHWuSHuangY, et al. Robust multitask compressive sampling via deep generative models for crack detection in structural health monitoring. Struct Health Monit2024; 23(3): 1383–1402.
15.
WangWYinQAiC, et al. Automation railway fastener tightness detection based on instance segmentation and monocular depth estimation. Eng Struct2025; 322: 119229.
16.
ScaioniMFengTBarazzettiL, et al. Image-based deformation measurement. Appl Geomat2015; 7: 75–90.
17.
HanYWuGFengD.Vision-based displacement measurement using an unmanned aerial vehicle. Struct Control Health Monit2022; 29(10): e3025.
18.
XuYBrownjohnJKongD.A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge. Struct Control Health Monit2018; 25(5): e2155.
19.
JiangSZhangJGaoC.Bridge deformation measurement using unmanned aerial dual camera and learning-based tracking method. Struct Control Health Monit2023; 2023(1): 4752072.
20.
YangYDornCManciniT, et al. Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification. Mech Syst Signal Process2017; 85: 567–590.
21.
ChengYSuZZhangJ.Mode shape-aided cable force estimation of a double-hanger system using a vision-based monitoring method. Measurement2024; 227: 114214.
22.
RiSYeJToyamaN, et al. Drone-based displacement measurement of infrastructures utilizing phase information. Nat Commun2024; 15(1): 395.
23.
SunHHuangY.Automatic deflection measurement for outdoor steel structure based on digital image correlation and three-stage multi-scale clustering algorithm. Autom Constr2024; 163: 105416.
24.
WangJTLiuYFLiuXG, et al. Photogrammetry-based bending monitoring and load identification of steel truss structures. Adv Struct Eng2023; 26(13): 2543–2561.
25.
HeKGkioxariGDollárP, et al. Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, Venice, Italy, 22–29 October 2017. New York: IEEE, pp. 2961–2969.
26.
ChaYJAliRLewisJ, et al. Deep learning-based structural health monitoring. Autom Constr2024; 161: 105328.
27.
GaoYMosalamKM.Deep transfer learning for image-based structural damage recognition. Comput Aided Civ Infrastruct Eng2018; 33(9): 748–768.
28.
YuSZhangJZhuC, et al. Full-field deformation measurement and cracks detection in speckle scene using the deep learning-aided digital image correlation method. Mech Syst Signal Process2024; 209: 111131.
29.
AthaDJJahanshahiMR.Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct Health Monit2018; 17(5): 1110–1128.
30.
ChaYJChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Infrastruct Eng2018; 33(9): 731–747.
31.
ChoiWChaYJ.SDDNet: Real-time crack segmentation. IEEE Trans Ind Electron2019; 67(9): 8016–8025.
32.
KangDHChaYJ.Efficient attention-based deep encoder and decoder for automatic crack segmentation. Struct Health Monit2022; 21(5): 2190–2205.
33.
ZhaiYNgAHMLuoZ, et al. Dynamic image segmentation and recognition measurement of axial compression experiment based on image clustering and semantic segmentation in RC column with FRP tubes. Measurement2024; 227: 114207.
34.
WaqasAKangDChaYJ.Deep learning-based obstacle-avoiding autonomous UAVs with fiducial marker-based localization for structural health monitoring. Struct Health Monit2024; 23(2): 971–990.
35.
KangDChaYJ.Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Comput Aided Civ Infrastruct Eng2018; 33(10): 885–902.
36.
AliRKangDSuhG, et al. Real-time multiple damage mapping using autonomous UAV and deep faster region-based neural networks for GPS-denied structures. Autom Constr2021; 130: 103831.
37.
ZouZChenKShiZ, et al. Object detection in 20 years: a survey. Proc IEEE2023; 111(3): 257–276.
38.
HorvatMJelečevićLGledecG.A comparative study of YOLOv5 models performance for image localization and classification. In: Central European conference on information and intelligent systems, Dubrovnik, Croatia, 21–23 September 2022. Zagreb: University of Zagreb, pp. 349–356.
39.
WangCYLiaoHYMWuYH, et al. CSPNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 14–19 July 2020. America: IEEE, pp. 390–391.
40.
ParkHYooYSeoG, et al. C3: concentrated-comprehensive convolution and its application to semantic segmentation. arXiv:1812.04920, 2018. doi: 10.48550/arXiv.1812.04920.
41.
HeKZhangXRenS, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transactions on pattern analysis and machine intelligence2015; 37(9): 1904–1916.
42.
ChenLCZhuYPapandreouG, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018. Berlin: Springer, pp. 801–818.
43.
ChoKVan MerriënboerBGulcehreC, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078, 2014. doi: 10.48550/arXiv.1406.1078.
44.
WangJJLiuYFNieX, et al. Deep convolutional neural networks for semantic segmentation of cracks. Struct Control Health Monit2022; 29(1): e2850.
45.
LiYYaoTPanY, et al. Contextual transformer networks for visual recognition. IEEE Trans Pattern Anal Mach Intell2022; 45(2): 1489–1500.
46.
VaswaniA.Attention is all you need. Adv Neural Inf Process Syst2017; 30: 5998–6008.
47.
SaeedKTabędzkiMRybnikM, et al. K3M: a universal algorithm for image skeletonization and a review of thinning techniques. Int J Appl Math Comput Sci2010; 20: 317–335.
48.
ZhangTYSuenCY.A fast parallel algorithm for thinning digital patterns. Commun ACM1984; 27(3): 236–239.
49.
EveringhamMVan GoolLWilliamsCKI, et al. The pascal visual object classes (voc) challenge. Int J Comput Vis2010; 88: 303–338.
50.
UijlingsJRRVan De SandeEAGeversT, et al. Selective search for object recognition. International journal of computer vision2013; 104: 154–171.
51.
LinTYMaireMBelongieS, et al. Microsoft coco: common objects in context. In: Computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, 6–12 September 2014. Berlin: Springer, pp. 740–755.
