PaperlayerView exact revision on arXiv

Source-linked AI summary

DROA-CLIPSeg: Prompt-Guided Thin Crack Segmentation in Low-Light Conditions

Vinita Shukla, Partibha Jangra, Vinayak Pathak, Praful Hambarde, Amit Shukla

arXiv:2610.33635v1cs.CVcs.AI

TL;DR

Low-light pavement crack segmentation remains challenged by reduced contrast and thin, faint structures that existing methods and generic enhancers do not explicitly preserve. DROA-CLIPSeg adapts frozen CLIPSeg with QKV-LoRA and DROA, and the paper reports that it outperforms vision–language baselines on both CrackLingo splits while generalizing zero-shot to MVTec-AD.

  • Problem

    Low-light conditions and thin hairline cracks challenge pavement segmentation, while generic enhancement targets perceptual quality rather than preserving structural cues for downstream segmentation.

    Source: §1 · pp. 1–2

  • Method

    DROA-CLIPSeg adapts frozen CLIPSeg with QKV-LoRA, a Domain–Residual Orthogonal Adapter, and crack-aware text annotations for Crack9k and Crack12k.

    Source: §1 · p. 2

  • Results

    77.4/70.8 F1 and 63.1/54.7 IoU; with ∼1.93 M parameters, DROA-CLIPSeg outperforms all 9 zero-shot and 4 fine-tuned vision–language baselines and generalizes zero-shot to MVTec-AD.

    Source: §6 · p. 15

  • Takeaways & Limitations

    A frozen-backbone, parameter-efficient prompt-guided approach achieves reported gains on both crack benchmarks, with the largest gains on thin cracks, and transfers zero-shot to MVTec-AD.

    Source: §1 · p. 2, §6 · p. 15

Abstract

from arXiv · show

Pavement crack segmentation under low light is degraded by reduced contrast, shadows, headlight glare, joints, stains, and wet seams. Existing crack segmentation methods mostly assume good illumination. Low-light enhancement methods improve visual quality, but they do not explicitly preserve crack topology. To address these limitations, we formulate low-light crack segmentation as a degradation-aware, prompt-grounded structural defect segmentation problem and propose DROA-CLIPSeg, a parameter-efficient framework that adapts a frozen CLIPSeg vision language backbone using two lightweight modules. QKV Low-Rank Adaptation is injected into the Q/K/V projections of the vision and text encoders to improve crack-specific image text alignment without backbone fine-tuning. The proposed Domain Residual Orthogonal Adapter (DROA) decomposes intermediate features into a dominant crack-domain branch, which captures recurring low-light crack patterns, and an orthogonal residual branch, which preserves faint, thin, and fragmented evidence usually suppressed by noise, shadow, or over-smoothing. We also introduce CrackLingo-9K and CrackLingo-12K, text augmented versions of Crack9k and Crack12k with crack-aware prompts. Training only approximately 1.93 M parameters, corresponding to 1.3% of the full model, DROA-CLIPSeg outperforms 9 zero-shot and 4 fine-tuned vision language baselines on both splits, generalises zero-shot to MVTec AD across textures and defect morphologies, and is supported by extensive ablations. Our source code is publicly available at https://github.com/VShuklacode/DROA-CLIPSeg

1 Introduction

Low-light acquisition and thin hairline cracks remain difficult for automated pavement inspection, while existing enhancement and segmentation approaches do not preserve the structural cues needed for these defects. DROA-CLIPSeg addresses this gap with prompt-guided adaptation of frozen CLIPSeg and crack-aware text annotations.

1 Introduction

  • Low light and thin hairline cracks undermine deployment: reduced contrast and noise make cracks blend with shadows, stains, joints, and wet seams, while vision-only models may miss faint cracks or over-predict shadows.Generic enhancers target perceptual quality rather than segmentation cues, and chaining enhancement with segmentation can introduce artefacts at thin structures.Source: §1 · pp. 1–2
  • Prior low-light crack methods use CNN/RGB-only pipelines, depend on enhancement or paired cross-illumination supervision, and have been validated only on small in-domain benchmarks.The introduction identifies the lack of vision–language prompting for low-light thin-crack segmentation as an unaddressed regime.Source: §1 · p. 2
  • DROA-CLIPSeg adapts frozen CLIPSeg with Q/K/V LoRA and a Domain–Residual Orthogonal Adapter that preserves recurring crack appearance alongside faint, fragmented evidence.The adapter’s branches are fused by a learnable energy gate; Crack9k and Crack12k are manually annotated with location, surface, and crack-type descriptions.Source: §1 · p. 2
  • The contributions include training only ∼1.9 M new parameters, crack-aware text annotations, and experiments reporting consistent Dice/IoU gains on both benchmarks, largest on thin-crack subsets.The framework adapts a fully frozen CLIPSeg backbone.Source: §1 · p. 2

Related Work

Prior vision–language segmentation models offer prompt-conditioned masks or pixel-level predictions, but natural-image training leaves them unspecialized for low-light, elongated pavement cracks. DROA is presented as a feature-level complement to weight-based LoRA for retaining faint crack evidence.

2.1 Vision–Language Foundation Models for Segmentation

  • Natural-image-trained vision–language models do not specialize in long, narrow, low-contrast pavement cracks under degraded illumination; concurrent work also indicates visibility-aware adaptation is needed.CLIPSeg [27] generates binary masks from free-form text, matching the paper’s per-prompt crack annotations and providing a compact decoder.Source: §2.1 · p. 3

2.2 Parameter-Efficient Adaptation of Vision Transformers

  • DROA adapts features while LoRA adapts weights: LoRA controls token interactions, whereas DROA gates dominant and orthogonal residual subspaces to preserve faint hairline-crack signals.The passage contrasts this with full fine-tuning, which is expensive and erodes generalization on small targets.Source: §2.2 · p. 3

3 Methodology

DROA-CLIPSeg uses a frozen CLIPSeg backbone with lightweight adaptation modules to turn a road image and crack prompt into a probability map and refined mask. Prompts provide semantic cues, while the architecture adapts and decodes visual and textual representations.

3 Methodology

  • DROA-CLIPSeg combines a frozen CLIPSeg backbone with QKV-LoRA and DROA to adapt thin, low-contrast crack segmentation using few trainable parameters.The input is a road image and natural-language crack prompt; the output is a binary crack segmentation mask.Source: §3 · pp. 3–4

3.1 Overall Architecture

  • The architecture encodes image features and crack-prompt embeddings, adapts both representations, decodes a coarse mask, and refines its spatial quality.The components are frozen CLIP vision and text encoders, QKV-LoRA, DROA, the CLIPSeg decoder, and a refinement head.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 Prompt-Guided Crack Segmentation

  • Crack-specific prompts such as “thin crack,” “longitudinal crack,” or “transverse crack” provide semantic guidance through the text encoder to the segmentation model.The prompt is encoded as a text embedding that conditions the CLIPSeg decoder to focus on crack-like structures rather than general edges or background textures.Source: §3.2 · p. 5

3.3 QKV Low-Rank Adaptation

QKV-LoRA adapts transformer attention without fully fine-tuning CLIPSeg by adding low-rank trainable updates to frozen query, key, and value projections. This is intended to improve continuity across crack fragments and suppress irrelevant structures.

3.3 QKV Low-Rank Adaptation

  • Low-rank trainable updates to query, key, and value projections adapt attention while leaving the original projection matrices frozen.The approach targets narrow, fragmented cracks that can resemble shadows, road markings, or background noise.Source: §3.3 · p. 5
  • Each projection update is factorized using trainable low-rank matrices A and B, with rank r and scaling factor α, instead of updating the full pretrained weight matrix.Only A and B are optimized; the original pretrained weight remains frozen.Source: §3.3 · pp. 5–6
  • QKV-LoRA changes how tokens attend to one another, supporting long-range continuity between disconnected crack fragments and suppression of shadows, stains, and road textures.The implementation places QKV-LoRA in vision-encoder layers and selected text-encoder layers to capture fine crack structures and improve prompt–visual alignment.Source: §3.3 · p. 6

3.4 Domain–Residual Orthogonal Adapter

DROA separates dominant crack-domain features from orthogonal residual evidence, then combines both updates with the original feature. Its gated residual branch complements QKV-LoRA by preserving weak crack evidence after feature transformations.

3.4 Domain–Residual Orthogonal Adapter

  • The dominant branch learns the main crack-related representation, while an orthogonal residual branch targets faint or thin evidence that the dominant branch may ignore.The residual feature is projected away from the dominant crack-domain direction to encourage complementary rather than repeated information.Source: §3.4 · pp. 6–7
  • DROA combines the original feature with dominant and gated residual updates to produce the adapted feature.A learnable energy gate controls the residual branch's contribution.Source: §3.4 · p. 7
  • QKV-LoRA adapts token interactions, whereas DROA adapts which feature information is preserved or emphasized after attention and MLP transformations.The two mechanisms are presented as complementary.Source: §3.4 · p. 7

3.5 Adapter Placement Strategy

Adapter placement varies across the frozen CLIPSeg encoders. Visual adapters span transformer layers, while text adaptation is concentrated in deeper layers to target prompt-relevant semantics.

3.5 Adapter Placement Strategy

  • Vision adapters are applied across transformer layers to capture crack patterns at multiple representation levels, from local texture and edges to semantic crack structures.Early layers preserve local texture and edge information, while deeper layers capture more semantic structure.Source: §3.5 · p. 7
  • Vision QKV-LoRA is inserted in attention projections, and DROA follows both the attention output projection and the MLP block.These placements are specified for a vision transformer layer.Source: §3.5 · p. 7
  • Text adapters are placed mainly in deeper layers because higher layers carry prompt-relevant semantics such as crack type, direction, and severity.The stated aim is to improve task-specific text-image alignment without disturbing general linguistic structure.Source: §3.5 · p. 7

3.6 CLIPSeg Decoder

The decoder conditions a coarse crack mask on adapted image features and a crack prompt. A refinement head and a thin-crack-aware objective address fine structure, continuity, and class imbalance.

3.6 CLIPSeg Decoder

  • The CLIPSeg decoder combines adapted visual features and the text embedding to produce a coarse crack mask, with the text embedding conditioning the prediction on the crack prompt.The prompt helps distinguish crack-like defects from other thin structures such as shadows, lines, or road boundaries.Source: §3.6 · p. 8

3.7 Prompt-Conditioned Refinement Head

  • The refinement head receives the upsampled coarse mask and improves local crack continuity, suppresses isolated false positives, and sharpens thin boundaries.It is added because patch-level encoding and decoder upsampling may lose fine spatial detail.Source: §3.7 · p. 8

3.8 Thin-Crack-Aware Training Objective

  • The training objective combines weighted binary cross-entropy, Dice loss, and structural connectivity supervision to address class imbalance and preserve thin, connected crack structures.The skeleton-aware term encourages centerline preservation and reduces fragmented predictions.Source: §3.8 · pp. 8–9

3.9 Training Strategy

Training keeps the CLIPSeg encoders and decoder frozen while optimizing the adaptation modules and refinement head. The approach is intended to reduce computation and overfitting while preserving pretrained vision-language knowledge.

3.9 Training Strategy

  • The original CLIPSeg vision encoder, text encoder, and decoder remain frozen; training optimizes QKV-LoRA, DROA, the conditional text adapter, and the refinement head.The frozen CLIPSeg parameters are not updated.Source: §3.9 · p. 9
  • The frozen-backbone design is described as reducing computational cost and overfitting on limited crack datasets while preserving CLIPSeg's general visual-language knowledge.These are the stated advantages of the training strategy.Source: §3.9 · p. 9
  • QKV-LoRA modulates token interactions, while DROA decomposes post-attention features into dominant crack-domain and orthogonal residual subspaces that preserve faint thin-crack evidence.The approach combines these adaptations with a prompt-conditioned CLIPSeg decoder for low-light, low-contrast crack segmentation.Source: §3.9 · p. 9

4 Experiments

DROA-CLIPSeg is evaluated on CrackLingo-9K and -12K against zero-shot and fine-tuned baselines, with qualitative and cross-domain tests. The experiments also examine its prompt format and report results on MVTec-AD without fine-tuning.

Tier-1 zero-shot comparison

  • Zero-shot CLIP-family models reach Dice/F1 ≤16% on both splits and precision below 9%, despite competitive image-level accuracy.All baselines use pretrained weights and the same crack prompt without parameter updates.Source: §4 · p. 10, Table 1 · p. 10, Table 2 · p. 11

Experimental setup

  • CrackLingo pairs darkened images with per-sample prompts describing orientation, low-light vision, location, and crack category.The experimental setup darkens each channel using I′=max(I−35,0) and trains 1.93 M parameters, or 1.3% of the approximately 152 M-parameter backbone.Source: §4 · pp. 9–10, Figure 4 · p. 10

Tier-2 fine-tune comparison

  • MaskCLIP-FT is the strongest Tier-2 baseline on CrackLingo-9K, with 50.6 F1 and 33.8 IoU, but remains below DROA-CLIPSeg.MaskCLIP-FT unfreezes 14.8 M parameters; the comparison evaluates fine-tuned baselines on both splits.Source: §4 · p. 11, Table 3 · p. 11

Full method comparison

  • DROA-CLIPSeg achieves the highest F1 and IoU on both CrackLingo splits while training only 1.9 M parameters, outperforming full CLIPSeg fine-tuning, AA-CLIP, and supervised few-shot baselines.The comparison is consolidated in Table 4.Source: §4 · p. 11, Table 4 · p. 12

Qualitative results

  • Across CrackLingo examples, DROA-CLIPSeg produces thin, continuous, ground-truth-aligned centrelines where zero-shot and fine-tuned baselines often collapse or fragment predictions.The extended qualitative sweep includes zero-shot and fine-tuned vision-language models.Source: §4 · p. 12, Figure 5 · p. 12

Cross-domain generalisation

  • A CrackLingo-12K-trained checkpoint localises defects zero-shot across four MVTec-AD categories and textures without MVTec-specific retraining.The evaluated morphologies include elongated defects and a compact puncture.Source: §4 · pp. 12–13

Ablation Study

Ablations test DROA components, prompt content, and broader design choices. The reported results indicate that both adapter branches and the gate matter, and that prompt and training-design choices affect performance.

DROA component ablation

  • Removing the dual path, energy gate, residual branch, or dominant branch drops F1 by 4.7–6.5 points on CrackLingo-12K.The full model attains 77.36 F1 / 63.07 IoU.Source: Ablation Study · p. 15

Text-prompt ablation

  • Stripping location/category information from prompts or using semantically empty text reduces IoU by 9.5 and 21.5 points, respectively.The prompt ablations are evaluated on CrackLingo-12K.Source: Ablation Study · p. 15

Text contribution

  • A wrong-no-crack prompt reduces IoU by 33.7%, while coarse, empty, shuffled, or wrong-orientation prompts each cost ≤1%.The seven-condition study indicates that the model uses the language signal and grounds the category cue on visual evidence.Source: Ablation Study · p. 15

Extensive ablation

  • Across seven design axes, LoRA r=32 and DROA dimension 32–64 perform best; early-, mid-, or late-only adapter placement drops 4–8 DSC.BCE-only training loses 4.8 DSC on thin cracks, and crack-aware prompts outperform bare class names and GPT-generated descriptions.Source: Ablation Study · p. 15

6 Conclusion

DROA-CLIPSeg combines QKV-LoRA, DROA, and a thin-crack-aware objective to adapt frozen CLIPSeg on text-augmented crack benchmarks. It reports gains across benchmarks and zero-shot transfer, while retaining limits under extreme low light and motion blur.

6 Conclusion

  • 77.4/70.8 F1 and 63.1/54.7 IoU; DROA-CLIPSeg outperforms all listed zero-shot and fine-tuned vision-language baselines, full CLIPSeg fine-tuning, AA-CLIP, and supervised ResNet50/101, with largest gains on thin cracks.It trains approximately 1.93 M parameters (1.3%) and generalises zero-shot to MVTec-AD.Source: §6 · p. 15
  • DROA-CLIPSeg adapts frozen CLIPSeg with QKV-LoRA, DROA, and a thin-crack-aware objective on the text-augmented CrackLingo-9K/-12K benchmarks.Source: §6 · p. 15
  • Ablations indicate that the dominant and orthogonal-residual subspaces, learnable energy gate, and prompt semantics each contribute substantially to final performance.Source: §6 · p. 15
  • Performance remains limited under extreme low light and motion blur; future work targets bridge and wall defects and real-time deployment.Source: §6 · p. 15

Related papers

Loading 2610.33635v1…