PaperlayerView exact revision on arXiv

Source-linked AI summary

SCSegamba: Lightweight Structure-Aware Vision Mamba for Crack Segmentation in Structures

Hui Liu, Chen Jia, Fan Shi, Xu Cheng, Shengyong Chen

arXiv:2503.01113v3cs.CV

TL;DR

Pixel-level crack segmentation must handle varied morphology and texture while remaining computationally efficient. SCSegamba uses a Structure-Aware Visual State Space module with lightweight gated convolution and structure-aware scanning, achieving strong multi-scenario performance with a small model.

  • Problem

    Pixel-level segmentation across diverse crack scenarios requires modeling morphology and texture while balancing segmentation quality against computational resource usage.

  • Method

    SCSegamba combines SAVSS, GBC, SASS, and MFS to capture crack shape and irregular texture cues with low computational cost.

  • Results

    F1 score 0.8390 and mIoU 0.8479 were achieved on the challenging multi-scenario dataset with 18.16G FLOPs and 2.8M parameters.

  • Takeaways & Limitations

    SCSegamba provides a lightweight structure-aware approach for precise crack segmentation across diverse and complex scenarios, including potential edge-device deployment.

Abstract

from arXiv · show

Pixel-level segmentation of structural cracks across various scenarios remains a considerable challenge. Current methods encounter challenges in effectively modeling crack morphology and texture, facing challenges in balancing segmentation quality with low computational resource usage. To overcome these limitations, we propose a lightweight Structure-Aware Vision Mamba Network (SCSegamba), capable of generating high-quality pixel-level segmentation maps by leveraging both the morphological information and texture cues of crack pixels with minimal computational cost. Specifically, we developed a Structure-Aware Visual State Space module (SAVSS), which incorporates a lightweight Gated Bottleneck Convolution (GBC) and a Structure-Aware Scanning Strategy (SASS). The key insight of GBC lies in its effectiveness in modeling the morphological information of cracks, while the SASS enhances the perception of crack topology and texture by strengthening the continuity of semantic information between crack pixels. Experiments on crack benchmark datasets demonstrate that our method outperforms other state-of-the-art (SOTA) methods, achieving the highest performance with only 2.8M parameters. On the multi-scenario dataset, our method reached 0.8390 in F1 score and 0.8479 in mIoU. The code is available at https://github.com/Karl1109/SCSegamba.

1. Introduction

Structural cracks vary substantially across materials and environments, while existing CNN, Transformer, and Mamba approaches struggle to jointly capture crack morphology and texture under computational constraints. SCSegamba addresses this balance with structure-aware scanning and lightweight gated convolution for efficient pixel-level segmentation.

  • Motivation: Material and environmental differences produce substantial variation in crack morphology and visual appearance, complicating pixel-level segmentation across scenarios.This challenge is especially relevant for bitumen pavement, concrete, and metal structures requiring regular monitoring.
  • Existing limitations: CNNs extract local crack features effectively but limited receptive fields hinder broad irregular dependency modeling, causing discontinuous segmentation and weak background suppression.Dilated convolution expands the receptive field but retains restrictive inductive biases.
  • Existing limitations: Transformer methods capture irregular pixel dependencies but incur high memory and computational demands that limit deployment on resource-constrained devices.Attention-based approaches can also sacrifice irregular-dependency and texture modeling when sparsified or linearized.
  • Existing limitations: Mamba-based visual models offer efficient sequence modeling, yet existing VSS blocks and scans inadequately capture irregular crack morphology and texture amid shadows and irrelevant regions.Linear feature processing can limit selective enhancement or suppression of crack features against disturbances.
  • Proposed approach: SCSegamba combines SAVSS, GBC, SASS, and MFS to model crack shape and texture while generating high-quality segmentation maps with low computational resources.GBC captures shape cues and adapts to complex backgrounds, while SASS strengthens semantic continuity; four SAVSS layers yielded optimal performance under complex interference.

2. Related Works

Crack segmentation has progressed from traditional feature extraction to CNNs, Transformers, and visual state-space models, each addressing different feature or efficiency needs. The paper motivates a crack-specific Mamba design because existing visual state-space blocks require stronger shape extraction and directional awareness for irregular crack textures.

  • Traditional methods: Traditional wavelets, percolation models, and k-means methods are straightforward but struggle with background interference and high segmentation accuracy.
  • CNNs and Transformers: CNN-based networks achieve strong crack-segmentation performance, while Transformer variants model long-range dependencies and fine-grained texture cues.Examples include ViT-based VCVNet, Swin-CNN SWT-CNN, and knowledge-distilled TBUNet.
  • Transformer limitations: Transformer self-attention introduces computational complexity that grows quadratically with sequence length, increasing parameter counts and computational demands.
  • Visual state-space models: Mamba and visual state-space models provide efficient long-distance dependency modeling without attention, supporting applications such as instance segmentation and object detection.ViM and VMamba emphasize efficient visual modeling, while PlainMamba uses fixed-width layer stacking.
  • Research gap: Existing visual state-space blocks and scanning strategies are not sufficiently optimized for crack segmentation, motivating stronger shape extraction and directional awareness.The paper identifies these requirements as necessary for capturing intricate crack details and irregular texture cues efficiently.

3. Methodology

SCSegamba combines SAVSS for crack shape and texture extraction with MFS for efficient multi-scale feature processing. Its GBC reduces computational cost, while SASS improves continuity and multi-directional modeling of crack structures.

  • Overall architecture: SCSegamba processes RGB images into patch sequences, extracts four multi-scale feature maps with SAVSS, and produces a refined segmentation output through MFS.The output is a single tensor with spatial dimensions matching the input image.
  • Lightweight GBC: BottConv uses low-rank projections and depthwise convolution to reduce complexity while preserving spatial feature extraction.The complexity decreases from O(fp^2d) to O(f_0p^2d) + O(ff_0), with f_0 < f.
  • Lightweight GBC: GBC combines residual features, gated feature interactions, and repeated BottConv processing to refine fine-grained crack representations.The design preserves basic crack features while dynamically refining detailed regions for more accurate segmentation maps.
  • Structure-aware scanning: SASS uses two parallel snake paths and two diagonal snake paths to preserve semantic and texture continuity across multiple crack directions.The SAVSS implementation uses four layers, and its state-space equations propagate hidden states and integrate multi-directional features.
  • Structure-aware scanning: PAF fuses the original sequence with SS2D features, while residual connections and GBC refine inter-layer outputs within SAVSS.These operations support crack shape and texture detail extraction and feature flow through later stages.
  • Multi-scale feature segmentation: MFS individually processes SAVSS feature maps with MLP operations and dynamic upsampling before aggregating them into a high-quality segmentation map.The feature maps are restored to the original resolution before aggregation.

4. Experiments

Experiments evaluate SCSegamba on four datasets against nine SOTA methods, using segmentation quality, visual comparisons, complexity, and ablations. SCSegamba achieves the strongest reported performance while maintaining low computational and parameter costs.

  • Comparison with SOTA Methods: SCSegamba achieves the best performance across four public datasets compared with nine SOTA methods.On DeepCrack, it exceeds the next-best method by 1.50% in F1 and 1.09% in mIoU; on CrackMap, by 2.06% and 1.65%, respectively.
  • Comparison with SOTA Methods: 2.21% higher F1 and 1.74% higher mIoU than the next-best method are achieved on the multi-scenario TUT dataset.The method suppresses irrelevant noise across complex topologies, metallic backgrounds, turbine blades, and dim underground pipelines.
  • Complexity Analysis: 2.80M parameters and a 37MB model size make SCSegamba smaller than all compared methods.Its FLOPs are only 3.22G higher than Crackmer, while parameters and model size are 52.54% and 13.95% lower than the next-best results.
  • Ablation Studies: 1.57% higher F1 and 1.21% higher mIoU are obtained with the proposed MFS segmentation head than with the second-best head.MFS achieves the best results across all six reported metrics on TUT.
  • Ablation Studies: 1.57% and 1.42% improvements in F1 and mIoU follow from adding GBC, while residual connections improve them by 0.13% and 2.47%.The fully integrated SAVSS achieves the best results across all metrics; using only PAF minimizes complexity but reduces performance.
  • Ablation Studies: 0.30% higher F1 and 0.33% higher mIoU are achieved by SASS than by the diagonal snake strategy under four-path scanning.The comparison uses four different directional scanning paths with otherwise matched conditions.

5. Conclusion

The conclusion presents SCSegamba as a lightweight structure-aware Vision Mamba for precise crack segmentation. Across four datasets, it combines strong segmentation performance with low computational cost, particularly on the multi-scenario dataset.

  • Conclusion: SCSegamba combines SAVSS and MFS to enhance crack shape and texture perception with a low parameter count.SAVSS uses GBC and SASS to capture irregular crack textures across varied structures.
  • Conclusion: 0.8390 F1 and 0.8479 mIoU are achieved on the challenging multi-scenario dataset with 18.16G FLOPs and 2.8M parameters.The conclusion identifies the method as suitable for edge devices and real-world crack detection.
  • Conclusion: Future work will incorporate multimodal cues and further optimize VSS design and scanning strategies.The stated goal is to improve segmentation quality while retaining low computational resources.

7. Details of SASS and Ablation Experiments

SASS aligns multi-directional scanning with crack extension and irregular shape to capture texture and topology. Ablations examine scan direction and path count, with four paths consistently preferred and SASS performing best.

  • SASS Design: SASS combines parallel-snake and diagonal-snake scans to align scanning paths with cracks’ irregular extensions.This design aims to strengthen semantic continuity and capture texture information from multiple directions.
  • Path-Number Ablation: Four scanning paths outperform two across the evaluated scanning strategies on the TUT dataset.The paper attributes this pattern to improved capture of fine crack details and topological cues.
  • Implementation: Algorithm 1 presents the execution process of the proposed SASS.The algorithm is included to clarify how the scanning strategy is implemented.

8. Details of Objective Function and Analysis

The objective combines BCE and Dice losses, with their weighting selected empirically to improve crack-background discrimination. A 1:5 α-to-β ratio produced the best reported performance on the multi-scenario dataset.

  • Objective function: The objective uses combined BCE and Dice losses for segmentation training.BCE uses predicted probabilities against ground-truth labels, while Dice loss measures overlap-related agreement.
  • Loss weighting: A 1:5 α-to-β ratio was selected as optimal after testing multiple hyperparameter settings.The ratio weights the two loss components in equation 16.
  • Loss weighting: 0.65% improvement in F1 and 0.55% improvement in mIoU over the 1:2 ratio was achieved with the 1:5 setting.The comparison was conducted on the multi-scenario dataset.
  • Interpretation: The selected weighting helps distinguish background pixels from the relatively few crack-region pixels.The passage attributes the performance gain to balancing Dice and BCE losses for this class-imbalance setting.

9. Visualisation Comparisons

Visual comparisons show strong fine-crack segmentation in relatively clean scenes and effective interference suppression in diverse noisy scenes. The reported advantages are linked to GBC-based morphology modeling and SAVSS-based topology capture.

  • Clean-scene comparisons: SCSegamba accurately segments intricate fine cracks across Crack500, DeepCrack, and CrackMap visual examples.These datasets mainly contain bitumen, concrete, and brick scenes with limited background noise and varied crack thicknesses.
  • Clean-scene comparisons: Other methods show discontinuities and expanded segmentation regions that do not align with the actual cracks.The comparison highlights weaker continuity and fine-detail segmentation relative to SCSegamba.
  • Noisy-scene comparisons: SCSegamba suppresses irrelevant noise and precisely segments cracks on generator blades and steel pipes in the TUT dataset.The TUT scenes contain diverse conditions and substantial background noise.
  • Noisy-scene comparisons: SAVSS is credited with capturing crack topologies, while CNN-based methods struggle to separate background noise from crack regions.The passage identifies contextual-dependency limitations in methods such as RIND and SFIAN.

10. Additional Analysis

Additional experiments examine SAVSS depth and patch size as trade-offs between segmentation performance and resource efficiency. Four SAVSS layers and patch size 8 were reported as the best settings in their respective studies.

  • SAVSS depth: Four SAVSS layers achieved the best balance between performance and computational requirements.The study selected four layers for SCSegamba after comparing different depths.
  • SAVSS depth: 0.036% higher F1 and 0.21% higher mIoU were obtained with four SAVSS layers than with eight layers.The comparison is reported in the layer-number ablation.
  • Patch size: Patch size 8 produced the best reported performance in the patch-size experiment.Patch size 4 reduced parameters and model size but limited the receptive field.
  • Patch size: 1.16% higher F1 and 1.17% higher mIoU were achieved with patch size 8 than with patch size 4.The comparison evaluates patch sizes used during SAVSS patch embedding.

11. Real-world Deployment Applications

The deployment setup streams road-surface video from an intelligent vehicle to a server for crack segmentation. SCSegamba combines fast inference with continuous, noise-suppressed video segmentation on resource-constrained hardware.

  • Deployment setup: The deployment system transmits vehicle video in real time to a server that produces the final segmentation output.The setup places an intelligent vehicle on an outdoor road surface and remotely controls it from the server terminal.
  • Inference speed: 0.0313 seconds per frame was achieved by SCSegamba on the resource-constrained server.This inference-speed comparison was conducted against other SOTA methods.
  • Video results: SCSegamba generated continuous crack-region maps while suppressing irrelevant noise in video data.The visual comparison reports fewer false positives and more continuous regions than several competing methods.
  • Video results: PlainMamba, MambaIR, and CSMamba produced false positives in some irrelevant noise spots despite continuous segmentation.DTrCNet and CTCrackSeg showed discontinuities, while Crackmer struggled with crack-background distinction.
  • SASS implementation: SASS execution constructs four scan orders, inverse mappings, and direction sequences from the patch matrix dimensions.The algorithm initializes horizontal, vertical, and diagonal traversal procedures before returning O, O inverse, and D.
Loading 2503.01113v3…