Bolt loosening angle detection through arrangement shape-independent perspective transformation and corner point extraction based on semantic segmentation results

Abstract

The identification of loosened bolts is crucial for the early warning of structural damage and maintaining the overall stability of the structure. Most existing two-dimensional computer vision-based bolt loosening detection methods need to rely on the bolt arrangement shape for perspective transformation, to overcome this limitation, this study proposes a bolt loosening angle detection method through arrangement shape-independent perspective transformation and corner point extraction based on semantic segmentation results. First, a dataset of 1748 images containing bolts is collected. Second, a bolt region of interest (RoI) detection is developed to extract the sub-image of the bolt. The results show that the training and validation accuracy of bolt RoI detector are 99.5%, and 99.4%, respectively. Then, the contour corner points are extracted, the automatic and arrangement shape-independent perspective transformation of the image is completed using the planar homography-based image processing algorithm and the regular hexagonal characteristics of the bolt. Finally, automatic estimation of the bolt angle is performed on the corrected bolt image. The proposed method is used to detect bolt images obtained from different shooting distances, perspective angles, and lighting conditions. The results demonstrate that the method can accurately detect the looseness of bolts in bolt connections, and the quantification error is mostly less than 3°, it has the potential for real-time bolt loosening monitoring of multi-type bolted connections.