Cycle-consistency-constrained few-shot learning framework for universal multi-type structural damage segmentation

Abstract

Despite the significant advancements in computer-vision-based structural damage recognition enhanced by deep learning techniques, challenges persist with training convergence, recognition stability, and model generalization for multi-type damage with small-scale datasets. To address these issues, few-shot learning has emerged as a promising solution to achieve universal damage segmentation using limited annotated images. This study proposes a novel cycle-consistency-constrained few-shot segmentation framework tailored for multi-type structural damage recognition. A cycle-consistency-constrained prototype learning paradigm is constructed to enhance the adequate utilization of limited pixel-level annotations, which is leveraged by establishing a bidirectional mutual supervision mechanism between support and query sets. Subsequently, a non-parametric similarity-guided optimization module is incorporated into the high-level latent feature space of image embedding. This module induces a similarity-driven contrast learning process for each pixel of feature maps and learns universal prototypes that condense the abstract semantic context of foreground (i.e., multi-type damage) and background. Furthermore, a synthetic loss function, which comprises mutually supervised segmentation dice loss, metric loss, and contrastive loss, is designed to ensure the bidirectional pixel-level segmentation accuracy, intra-class compactness, and inter-class separability of learned prototypes for multi-type damage. A multi-type structural damage dataset, encompassing concrete crack, steel fatigue crack, concrete spalling, and steel corrosion, is collected to validate the efficacy, necessity, and generalizability of the proposed method through a series of comparative studies and ablation experiments. The results indicate that segmentation accuracies for multi-type structural damage significantly surpass that of directly training a conventional segmentation model, performing significant improvements in average mean intersection-over-union (mIoU) and mean pixel accuracy (mPA) by 11.5% and 9.1%, respectively. In addition, the adaptability of the proposed method for one-shot learning, using only one annotated image for a completely new damage type, is also corroborated by notable increases of average mIoU and mPA by 8.1% and 7.7%.