Sam-Based Instance Segmentation Models for the Automation of Structural Damage Detection

Automating visual inspection for capturing defects based on civil structures appearance is crucial due to its currently labour-intensive and time-consuming nature. An important aspect of automated inspection is image acquisition, which is rapid and cost-effective considering the pervasive developments in both software and hardware computing in recent years. Previous studies largely focused on concrete and asphalt, with limited attention to masonry cracks and a lack of publicly available datasets. In this paper, we address these gaps by introducing the “MCrack1300” dataset, consisting of 1,300 annotated images (640 pixels × 640 pixels), covering bricks, broken bricks, and cracks, for instance segmentation. We evaluate several leading algorithms for benchmarking, and propose two novel, automatically executable methods based on the latest visual large-scale model, the prompt-based Segment Anything Model (SAM). We fine-tune the encoder of SAM using Low-Rank Adaptation (LoRA). The first method involves abandoning the prompt encoder and connecting the SAM’s encoder to other decoders, while the second method introduces a learnable self-generating prompter. We redesign the feature extractor for seamless integration of the two proposed methods with SAM’s encoder. Both proposed methods exceed state-of-the-art performance, surpassing the best benchmark by approximately 3% for all classes and around 6% for cracks specifically. Based on successful detection, we then propose a method based on a monocular camera and the Hough Line Transform to automatically convert images into orthographic projection maps. By incorporating known real sizes of brick units, we accurately estimate crack dimensions, with the results differing by less than 10% from those obtained by laser scanning. Overall, we address important research gaps in masonry crack detection and size estimation by introducing a new dataset, as well as novel SAM-based detection algorithms and monocular photogrammetric methodology, ultimately, offering reliable automated solutions.