PaperlayerView exact revision on arXiv

Source-linked AI summary

VTRL-Med: Progressive Visual Refinement with Topology-Guided Regularization for Medical Image Clustering

Guangyu Meng, Pengfei Gu, Peixian Liang, John P. Lalor, Erin Wolf Chambers, Danny Z. Chen

arXiv:2610.32884v1cs.CV

TL;DR

Label-free medical image clustering offers a way to use large unlabeled datasets, but existing methods may miss structural cues and explicit topology. VTRL-Med adapts SAM, filters pseudo-labels, and progressively refines visual features with PH-based regularization. Across 13 benchmarks, it achieves state-of-the-art label-free clustering performance, improving average accuracy by 4.48% over existing methods.

  • Problem

    Deep clustering may underuse class-agnostic region and boundary cues and lacks explicit encoding of global topological patterns such as connectivity and enclosed structures.

    Source: §1 · p. 2

  • Method

    VTRL-Med adapts SAM, filters pseudo-labels through prototype selection and k-NN consistency, then refines visual features with auxiliary PH-based regularization.

    Source: §1 · p. 2

  • Results

    Across 13 medical image benchmarks, VTRL-Med achieves state-of-the-art label-free performance, improving average accuracy by 4.48% over existing methods.

    Source: §1 · p. 2, §4.1 · p. 6

  • Takeaways & Limitations

    Visual-structural features serve as the primary clustering representation, with PH providing complementary geometric cues for samples ambiguous in visual feature space.

    Source: §1 · p. 2

Abstract

from arXiv · show

Deep clustering offers a promising approach for analyzing large unlabeled medical image datasets, yet prevailing methods still face two limitations: they may neglect class-agnostic structural priors that capture object boundaries and regions, and they do not explicitly encode topological patterns (e.g., connectivity and holes). These sources of information play distinct and complementary roles: visual-structural features provide the primary discriminative representation for clustering, while topological features provide additional global geometric cues for residual visual ambiguities. We introduce VTRL-Med, a progressive visual-structural learning framework with topology-guided regularization for medical image clustering. VTRL-Med first adapts the Segment Anything Model (SAM) to extract image features that retain structural cues inherited from mask-supervised pretraining. It then generates reliable pseudo-labels through a two-step filtering process that combines prototype-based selection with k-nearest neighbor consistency validation. Finally, it progressively refines the visual branch while an auxiliary topology branch, constructed from persistent homology (PH) descriptors, provides additional geometric regularization during training. Across 13 medical image benchmarks, VTRL-Med achieves state-of-the-art clustering performance, improving average accuracy by 4.48% over existing methods. On three representative datasets, topology-guided regularization further improves accuracy by 1.3--4.3% over visual refinement alone.

1. Introduction

Label-free medical image clustering addresses the annotation bottleneck, but existing methods may underuse boundary and region cues and omit explicit topological encoding. VTRL-Med combines SAM-derived visual-structural features, filtered pseudo-labels, and progressive visual refinement regularized by persistent homology, achieving state-of-the-art results across 13 benchmarks.

1. Introduction

  • Label-free clustering can make unlabeled medical images useful for tasks including novel disease subtype discovery, treatment-response cohort stratification, and screening quality control.Supervised methods face costly, time-consuming expert annotation and inter- and intra-observer variability.Source: §1 · p. 1
  • Existing self-supervised methods may under-emphasize class-agnostic region and boundary cues, and deep clustering methods do not explicitly encode global topology.Figure 1 illustrates stronger lesion-structure attention from adapted SAM and a visually similar lesion pair with distinct PH signatures.Source: §1 · p. 1, Figure 1 · p. 2
  • Visual-structural features provide the primary clustering representation, while PH supplies complementary global geometric cues for samples ambiguous in visual feature space.The framework uses topology to regularize rather than replace visual learning.Source: §1 · p. 2
  • VTRL-Med adapts SAM for structural cues, filters pseudo-labels with prototype selection and k-NN consistency, then refines visual features with PH-based regularization.The topology branch is used during training; final cluster assignments come from the refined visual branch.Source: §1 · p. 2
  • VTRL-Med achieves state-of-the-art performance on 13 medical image benchmarks, improving average accuracy by 4.48% over existing methods.Topology-guided regularization further improves accuracy by 1.3–4.3% over visual refinement alone.Source: §1 · p. 2

2. Related Work

Related work covers deep clustering, self-supervised representation learning, and persistent homology in medical imaging. VTRL-Med combines these strands through SAM-derived structural cues, filtered pseudo-labeling, progressive visual refinement, and auxiliary PH regularization.

Deep Clustering

  • Deep clustering jointly learns representations and cluster assignments; recent pseudo-labeling approaches include SCAN [52], SPICE [37], and CDC [22].Medical examples include DeepMCAT [23] and MedTrans; VTRL-Med adds quota-based prototype selection and k-NN consistency filtering followed by topology-guided refinement.Source: §2 · p. 2

Self-Supervised Learning (SSL) for Medical Imaging

  • SSL methods learn representations without labels, while SAM uses mask-supervised pretraining to provide boundary-aware, class-agnostic segmentation cues.Related approaches include contrastive methods SimCLR [9] and MoCo-v3 [10], BYOL [15], and DINO variants [7] [38] [47].Source: §2 · p. 2

Persistent Homology in Medical Imaging

  • Persistent homology tracks births and deaths of connected components and loops across scales, and cubical PH extracts signatures directly from pixel features without explicit segmentation.Prior medical-imaging PH applications include tumor analysis [41], cancer grading, and supervised classification such as PHG-Net [39], predominantly in labeled settings.Source: §2 · pp. 2–3

3. Method

VTRL-Med combines mask-informed visual features, filtered pseudo-labels, and topology-guided training to cluster unlabeled medical images. Its topology branch supplements the visual representation during training; final assignments come from the visual branch alone.

3.1. Stage 1: SAM Adaptation for Visual Features & Mask-Based Structural Priors

  • Stage 1 adapts SAM to medical images while retaining mask-supervised structural cues, producing a domain-adapted encoder for downstream clustering.The student fine-tunes the last four Transformer blocks while freezing earlier layers; adaptation combines self-distillation, feature-distribution regularization, and spatial-structure anchoring.Source: §3.1 · p. 3

3.2. Stage 2: Confidence-and-Consistency Pseudo-Labeling

  • Stage 2 freezes the adapted encoder and trains a clustering head with reliable pseudo-labels selected through prototype-based quotas and k-NN consistency validation.Prototype selection targets coverage, while requiring 70% agreement among 10 nearest neighbors filters locally inconsistent assignments.Source: §3.2 · pp. 3–4

3.3. Stage 3: Progressive Visual Refinement with Topological Regularization

  • Persistent-homology features encode connected components and holes as complementary global geometry, while visual-structural features remain the primary clustering representation.The topology branch extracts top H0 and H1 features and provides training-time regularization; the stated objective combines visual, topology, and regularization losses.Source: §3.3 · pp. 4–5
  • Stage 3 refines the visual branch with an auxiliary topology branch, using bootstrap training before mutual refinement to manage the branches’ different initialization quality.The visual branch starts from adapted SAM features and a trained clustering head, while TopoNet starts from random initialization; at inference, only the visual branch assigns clusters.Source: §3.3 · pp. 4–5

4. Experiments

Across 13 medical image datasets, VTRL-Med outperforms label-free clustering baselines, and ablations examine the roles of its stages and topology branch. Further analyses locate topology corrections near cluster boundaries and report computational costs.

4. Experiments

  • Evaluation covers 13 datasets, with hyperparameters tuned on three datasets and applied unchanged to the other 10; no method uses target-task annotations.Methods are compared over three runs using ACC, NMI, and ARI; paired t-tests assess statistical significance.Source: §4 · pp. 5–6

4.2. Component Analysis and Ablation

  • Removing Stage 1 drops ACC by 30–39%, while removing Stage 3 degrades results by 9–32%; Stage 2 removal has dataset-dependent effects.On PathMNIST, Stage 3 visual refinement contributes +31.2% and the topology branch adds +1.3%; without Stage 3, the head can overfit early pseudo-label errors.Source: §4.2 · p. 6
  • 36.06–80.52% ACC after SAM adaptation exceeds frozen SAM’s 25.13–49.51%, showing that pretrained SAM features alone do not consistently support clustering.On PathMNIST, adapted SAM reaches 80.52% versus 77.21% for SimCLR and 77.53% for MoCo-v3.Source: §4.2 · p. 6

4.1. Main Results

  • VTRL-Med ranks first on 10/13 datasets and improves average ACC by 4.48% over SPICE, the second-best baseline, across 13 benchmarks.Its average ACC/NMI/ARI is 81.51%/62.78%/63.34%; all reported improvements are statistically significant (p<0.02).Source: §4.1 · p. 6, Table 2 · p. 7

4.2. Component Analysis and Ablation

  • With Stage 3 refinement, two-step filtering achieves the highest ACC on all three tested datasets, outperforming FixMatch by 3.3–10.9%, FlexMatch by 2.3–9.1%, and SPICE by 2.1–17.7%.Removing confidence selection drops ACC by 0.4–3.8%, and removing k-NN consistency drops it by 1.0–2.9%.Source: §4.2 · pp. 6–7
  • The topology branch improves ACC by 1.3–4.3%, but fixed high weighting drops performance by 4.5–5.8%; removing Phase A causes 8.0–14.6% drops.Fixed moderate weighting remains competitive with 0.4–2.8% drops; the reported ablation indicates that the topology branch requires direct supervision before mutual refinement.Source: §4.2 · p. 7

4.3. Analysis of Topology-Corrected Predictions

  • Rescued samples lie near cluster boundaries, with 4.3× and 10.3× greater boundary proximity than correctly classified samples on PathMNIST and BloodMNIST, respectively.Topology rescues 123 and degrades 28 samples on PathMNIST, and rescues 38 and degrades 8 on BloodMNIST; Grad-CAM shows attention shifting toward more discriminative structures.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4. Computational Complexity

  • On TissueMNIST, VTRL-Med achieves 4.4× the throughput of MedMamba-B with only 3.29% lower accuracy; topology adds less than 1% throughput overhead.Stage 1 accounts for 72.4% of training time, while persistent-homology feature extraction is offline and takes 46.8 minutes.Source: Table 4 · p. 8, Table 5 · p. 8, §4.4 · pp. 8–9

5. Conclusions

VTRL-Med combines SAM-based structural priors with persistent-homology-based topological regularization for label-free medical image clustering. Across 13 benchmarks, it achieves state-of-the-art label-free performance within 3.29% of supervised accuracy at negligible additional cost.

5. Conclusions

  • Across 13 benchmarks, VTRL-Med achieves state-of-the-art label-free performance within 3.29% of supervised accuracy at negligible additional cost.Source: §5 · p. 9
  • VTRL-Med unifies SAM-based structural priors with persistent-homology-based topological regularization for medical image clustering.Source: §5 · p. 9
  • The experiments confirm complementary roles for structural and topological information, and the approach opens avenues for anatomy-faithful deep medical image clustering.Source: §5 · p. 9

Related papers

Loading 2610.32884v1…