ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation

Introduction

• Existing vessel methods are often dataset- or modality-specific, while foundational 2D vessel segmentation across anatomies and modalities remains underexplored.Prior methods also lack discriminative vascular appearance features and modeling of modality-specific structures such as branching patterns.Source: Introduction · p. 1
• SAM’s point and box prompts poorly fit elongated, branched vessels, and its generic decoder struggles to preserve thin branches and bifurcations.The decoder also lacks explicit modality-aware vascular representations for fine-grained recovery under appearance variation.Source: Introduction · pp. 1–2
• ReG-SAM adds graph prompt embeddings for modality-aware vascular morphology and vascular prototype embeddings for fine-grained, modality-specific decoding.The prototypes use multi-scale image features with vascular masks, while graph prompts are derived from vascular-mask graphs.Source: Introduction · p. 2
• Reference graph-guided learning samples modality-aware graphs from a vascular database and aligns their embeddings with query-image representations, enabling inference without ground-truth masks.The framework is evaluated on 19 public datasets and is reported to outperform existing SAM-based methods, including methods with manual prompts.Source: Introduction · p. 2

Domain-Specific Vessel Segmentation

• Handcrafted vascular filters require careful tuning and generalize poorly, while deep-learning vessel methods commonly remain tied to individual datasets or modalities.Examples of handcrafted approaches include matched filtering, ridge detection, and Hessian-based vessel filtering.Source: Domain-Specific Vessel Segmentation · p. 2
• To address sparse annotations, some work synthesizes training data, while graph-based approaches derive graphs from masks, features, or image patches.The cited synthesis and graph approaches include Ma et al. 2021; Kim, Oh, and Ye 2022; Lin et al. 2023; Zhang et al. 2024b; Kreitner et al. 2024; Zhang et al. 2024a; Liang et al. 2026; Hao et al. 2026, and Shin et al. 2019; Mishra et al. 2021; Xu and Wu 2024; Ahmed et al. 2026; Jalali, Fateh, and Rezvani 2024; Li et al. 2026.Source: Domain-Specific Vessel Segmentation · p. 2

Foundational Medical Image Segmentation

• Medical foundation segmentors adapt SAM through fine-tuning, but general 2D vessel segmentation remains underexplored and OVS-Net still relies on conventional point prompts.OVS-Net combines a SAM backbone with a convolutional branch but lacks dedicated vascular-morphology modeling.Source: Foundational Medical Image Segmentation · p. 2

Methodology

• ReG-SAM samples modality-matched reference graphs and estimates GPEs and VPEs from the database rather than requiring annotated masks at inference.The modality-organized database supports reference-guided representation learning for query images.Source: Methodology · p. 2

Graph Set Sampling from Vascular Database

• Randomly sampling same-modality graphs preserves reference diversity while learning modality-level priors from scarce vascular annotations.The database stores mask-derived graphs and modality labels; the query image's own graph is excluded from its candidate set.Source: Graph Set Sampling from Vascular… · pp. 2–3

Reference-Guided Graph-Prompt Learning

• RGPL encodes query and reference graphs into embeddings, using the fused reference embedding as the GPE during inference when the query mask is unavailable.The GPE captures modality-aware vascular morphology and guides mask decoding.Source: Reference-Guided Graph-Prompt Learning · p. 3

Reference-Guided Vascular-Prototype Learning

• VPE learning aligns mask-derived, multi-scale CNN vessel prototypes with graph-derived reference prototypes, using the reference prototype at inference.The query prototype is formed by fusing its multi-scale embeddings; training averages query and reference prototypes.Source: Reference-Guided Vascular-Prototype… · pp. 3–4

Vessel-Aware Fine-Grained Decoding

• The decoder adds CNN skip connections and four progressive upsampling stages, then uses VPE-guided attention with residual gating to modulate features.The VPA module applies at the end of each upsampling stage to support fine-grained vessel segmentation.Source: Vessel-Aware Fine-Grained Decoding · p. 4

Loss Function

• Training combines Dice and BCE segmentation losses with L1 embedding alignment and contrastive loss for modality-level consistency.The contrastive term pulls embeddings from the same modality closer and pushes those from different modalities apart.Source: Loss Function · p. 4

Datasets

• The benchmark includes 19 public datasets across six imaging modalities, with ORVS held out as a zero-shot test set.The remaining datasets were merged and split 80:20, yielding 3,349 training and 844 test samples.Source: Datasets · p. 4

Implementation Details

• ReG-SAM uses adapters in each image-encoder transformer block, freezes the original encoder weights, and initializes from pretrained MedSAM-B.Only adapter parameters are optimized for image-encoder adaptation.Source: Implementation Details · p. 4

Compared Methods

• The comparison retrains five benchmark methods and evaluates Dice and IoU, including prompted SAM variants and prompt-free ASPS.The selected methods include SAM-Med2D, MedSAM, SAM-HQ, ASPS, and OVS-Net.Source: Compared Methods · p. 5

Quantitative Results

• ReG-SAM achieves the best overall segmentation performance on both regular and zero-shot test sets, though it does not rank first in every modality.It excels on Fundus, DSA, and OCTA, and Figure 2 shows balanced dataset-wise performance with limited degradation on a few datasets.Source: Quantitative Results · p. 5, Figure 2 · p. 5, Table 2 · p. 6

Qualitative Results

• In challenging branched-vessel regions, ReG-SAM preserves fine structures more effectively than baselines, although some vessel details remain missed.Figure 3 marks challenging regions with green boxes and enlarges them for comparison.Source: Qualitative Results · p. 5, Figure 3 · p. 6

Ablation Studies

• Reducing decoder upsampling stages or removing GPEs, VPEs, or either representation loss degrades performance; removing VPEs causes a larger drop than removing GPEs.Table 3 summarizes the architecture and loss ablations.Source: Table 3 · p. 7, Ablation Studies · p. 7

Comparison on Sampling Strategies

• Random reference-graph sampling outperforms top-k under the tested setting, and k = 3 performs best on both test sets.Random sampling is more stable across k values; the authors attribute this likely to greater diversity in graph combinations.Source: Figure 4 · p. 7, Comparison on Sampling Strategies · p. 7

• Across 19 public vascular datasets, ReG-SAM consistently outperforms existing methods in overall segmentation accuracy and zero-shot generalization.Source: Discussion and Conclusion · p. 7
• A vascular database and reference graph-guided learning schemes estimate GPEs and VPEs from randomly sampled graphs, without ground-truth masks during inference.Source: Discussion and Conclusion · p. 7
• Fewer than 5,000 samples limit the framework’s potential, while low-quality annotations can make accurate predictions appear over-segmented and reduce measured performance.The authors suggest larger, higher-quality, and more diverse datasets could improve ReG-SAM.Source: Discussion and Conclusion · p. 7