PaperlayerView exact revision on arXiv

Source-linked AI summary

UNIVERS: Learning UNIversal Voxel-wise Embedding via Randomized Synthesis for generalizable one-shot landmark localization

Derong Yu, Qingsong Zhao, Yanbing Yue, Yikai Li, Zhi Zhang, Guoyan Zheng

arXiv:2610.29098v1cs.CVcs.AI

TL;DR

One-shot landmark localization across modalities and varying fields of view is difficult with existing approaches that depend on augmented single-image views, limited real data, or iterative paired-image training. UNIVERS learns voxel-wise embeddings end-to-end from randomized synthetic data and cross-domain contrastive learning. The authors report that it outperforms state-of-the-art methods across three one-shot settings, with particular advantage in cross-modality localization.

  • Problem

    Existing one-shot methods have limited cross-modality generalization and rely on scarce real clinical data or paired multi-modality images with multi-stage iterative training.

    Source: §I · p. 2

  • Method

    UNIVERS learns voxel-wise embeddings end-to-end from shape-template pseudo-data randomized across semantic layout, appearance, FOV, and pose, using cross-domain contrastive learning.

    Source: §I · p. 2

  • Results

    UNIVERS consistently outperforms state-of-the-art one-shot localization methods across intra-dataset, cross-dataset intra-modality, and cross-dataset cross-modality settings.

    Source: §I · p. 2

  • Takeaways & Limitations

    The trained model localizes landmarks in multi-modality volume scans with arbitrary FOVs without retraining or finetuning.

    Source: §I · p. 2

Abstract

from arXiv · show

One-shot landmark localization in volume scans with varying fields of view (FOVs) is challenging. While existing self-supervised learning methods have shown promising performance, they rely on contrastive learning built upon augmented views of the same image, which inherently limits their generalization in cross-modality scenarios. This motivates research on cross-modality embedding learning using paired multi-modality images. However, existing methods suffer from not only limited training data, but also the requirement of multi-stage iterative training. To address these limitations, we propose UNIVERS, an end-to-end trainable framework that learns UNIversal Voxel-wise Embedding via Randomized Synthesis. Our framework leverages diverse pseudo-data synthesized from organ shape templates, with randomization applied to semantic layout, visual appearance, FOV, and spatial pose. Coupled with cross-domain discriminative voxel-wise contrastive learning, this domain randomization strategy forces the network to disregard domain-specific biases and learn features that are robust against variations in imaging contrast, resolution, and spatial context. Consequently, the trained model enables one-shot landmark localization in multi-modality volume scans with arbitrary FOVs. In this study, the efficacy of the proposed method is demonstrated by applying it to the task of localizing the adrenal glands, which are paired, small, and irregular endocrine organs. In particular, comprehensive experiments are conducted on three challenging adrenal gland datasets. The experimental results demonstrate that UNIVERS consistently outperforms the state-of-the-art (SOTA) one-shot landmark localization approaches, with particularly prominent advantages in cross-modality scenarios. The source code is publicly available at https://github.com/Kendra9999/UNIVERS.

I. INTRODUCTION

One-shot landmark localization reduces annotation needs, but current approaches have difficulty generalizing across modalities and domain shifts. UNIVERS trains universal voxel-wise embeddings from randomized synthetic data and outperforms state-of-the-art methods across three one-shot settings.

I. INTRODUCTION

  • Precise adrenal-center localization is an important preprocessing step for downstream tasks, while manual annotation is laborious, costly, and requires clinical expertise.The adrenal glands are small and irregular, making their centers difficult to localize in volumetric images.Source: §I · p. 1
  • Existing methods rely on single-modality augmented views or paired multi-modality images, and may require multi-stage iterative training and substantial real clinical data.The limited domain variation in available data can leave features less robust to shifts in anatomy, contrast, resolution, FOV, and pose.Source: §I · pp. 1–2
  • UNIVERS consistently outperforms state-of-the-art approaches across three one-shot evaluation settings.The settings are intra-dataset, cross-dataset intra-modality, and cross-dataset cross-modality.Source: §I · p. 2
  • UNIVERS trains end-to-end on pseudo-data randomized in semantic layout, appearance, FOV, and spatial pose to learn domain-robust voxel-wise features.At inference, it uses anatomical consistency to generate embeddings for real multi-domain images without retraining or finetuning.Source: §I · p. 2
  • The proposed Cross-domain Discriminative Voxel-wise Contrastive loss uses multi-domain pseudo-patch quadruplets with multiple positives to promote domain-invariant, discriminative embeddings.The framework also proposes a coarse-to-fine localization approach using a single frozen UNIVERS network for multi-modality scans with arbitrary FOVs.Source: §I · p. 2

II. RELATED WORK

Prior one-shot localization methods use image registration or learned voxel embeddings, while domain-randomized synthesis has chiefly served other medical-imaging tasks. UNIVERS applies synthetic domain randomization to discriminative voxel-wise embedding for landmark localization.

A. One-shot landmark localization

  • Embedding-learning approaches match support and query landmarks by voxel-feature similarity, but prior methods are trained within one modality and can generalize poorly across heterogeneous domains.SAM emphasizes negative sampling, UAE-S adds semantic learning, and MedLAM combines anatomical mapping with multi-scale similarity.Source: A. One-shot landmark localization · p. 2
  • UAE-M extends embedding learning to CT-MRI matching, but uses multi-stage iterative training with cross-modality registration to establish voxel correspondences.UNIVERS instead learns domain-invariant embeddings end-to-end from synthetic data generated through domain randomization.Source: A. One-shot landmark localization · pp. 2–3

B. Domain randomization for medical image analysis

  • Domain-randomized synthesis has supported segmentation, registration, and representation learning, but prior methods were not designed for generalizable one-shot landmark localization.Landmark localization requires each voxel embedding to distinguish its anatomical region from other regions.Source: B. Domain randomization for medical… · p. 3

A. Learning universal voxel-wise embedding

  • UNIVERS uses shape templates to create aligned pseudo-image pairs with shared randomized semantic layouts and different appearances, plus patch quadruplets with distinct FOVs and poses.A shared CNN encoder feeds semantic and discriminative decoders trained respectively with CPSC and CDVC losses.Source: A. Learning universal voxel-wise… · p. 3

1) Data generation:

UNIVERS builds training databases from organ shape templates, synthesizing paired images and quadruplets of patches to expose the model to varied appearance and spatial context.

1) Data generation:

  • Randomized image pairs share a semantic layout but differ in appearance, simulating aligned scans from distinct domains.GMM intensity sampling and independently applied intensity augmentations create appearance variation, while shared geometric augmentation preserves alignment.Source: 1) Data generation · p. 3
  • Patch quadruplets are sampled from aligned image pairs to provide patches with varied fields of view and spatial poses.The two pipelines separately simulate FOV variation through random cropping and pose variation through independent rotation, translation, and scaling.Source: 1) Data generation · p. 3
  • Randomly selecting between the cropping and geometric-augmentation pipelines separates spatial-variation sources, avoiding overly destructive combined changes that could hinder reliable voxel correspondences.The image-pair and patch-quadruplet databases are the randomized training inputs shown in Fig. 1(a).Source: 1) Data generation · p. 3, Figure 1 · p. 4

2) Training:

UNIVERS trains semantic and discriminative voxel-wise embeddings with separate decoders and contrastive objectives. Joint optimization combines the two losses, aligning cross-domain semantic features and encouraging discriminative correspondences across pseudo-patches.

2) Training:

  • The DS decoder learns domain-invariant semantic embeddings with CPSC, aligning class prototypes across cross-domain pseudo-image pairs.The loss pulls voxels toward their class prototypes and away from other class prototypes, facilitating organ differentiation.Source: 2) Training · p. 4
  • The DD decoder learns discriminative embeddings with CDVC on multi-domain pseudo-patch quadruplets and uses multiple positive voxels per anchor.Each anchor has three positives from the other patches at the same original-volume location; negatives use the hard and diverse sampling strategy from.Source: 2) Training · pp. 4–5
  • CDVC pulls embeddings of multi-domain positive voxels together while repelling negatives, promoting domain-invariant and discriminative anatomical representations.The loss operates on normalized DD embeddings of anchor, positive, and negative voxels.Source: 2) Training · p. 5
  • The DS and DD decoders are jointly optimized with a weighted sum of CPSC and CDVC losses, where λ balances the terms.Source: 2) Training · p. 5

3) Inference:

At inference, UNIVERS produces a normalized DV embedding for an arbitrary-FOV volume without retraining or finetuning.

3) Inference:

  • UNIVERS concatenates DS and DD decoder outputs and applies ℓ2-normalization to produce the DV embedding for an arbitrary-FOV volume, without retraining or finetuning.Source: 3) Inference · p. 5

B. Application to landmark localization

One-shot localization matches a labeled support landmark to its corresponding point in an unlabeled query using voxel-wise embeddings. For adrenal glands, a coarse-to-fine procedure uses the same frozen UNIVERS network to propagate and refine bilateral centers.

1) One-shot landmark localization:

  • One-shot localization extracts embeddings from a labeled support image and an unlabeled query, then matches corresponding landmarks in the domain-invariant embedding space.The query may differ in modality and/or FOV; matching uses a fixed-point-based method introduced in.Source: 1) One-shot landmark localization · p. 5

2) Coarse-to-fine one-shot adrenal gland localization:

  • The adrenal application uses one frozen UNIVERS network for both coarse and fine localization, accommodating volumes with arbitrary FOVs.Source: 2) Coarse-to-fine one-shot adrenal… · p. 5
  • Coarse localization resamples support and query images to isotropic spacing, then propagates the labeled support centers to the query.Source: 2) Coarse-to-fine one-shot adrenal… · p. 5
  • Fine localization resamples to finer spacing, crops separate left and right patches around support and coarse query centers, and independently matches each patch pair.Crop boundaries avoid left-right overlap and image-boundary exceedance; refined centers are mapped back to the original query coordinates.Source: 2) Coarse-to-fine one-shot adrenal… · p. 5

C. Implementation details

UNIVERS was evaluated across three one-shot settings using three datasets. The implementation synthesizes randomized training pairs, adds online augmentations, and applies a coarse-to-fine adrenal localization framework.

C. Implementation details

  • Offline construction synthesized 120,000 randomized image pairs, with online augmentations including noise, intensity shifts, blurring, deformations, resolution degradation, affine transforms, and axis flipping.Source: C. Implementation details · p. 5
  • Training ran for 50,000 iterations with SGD, momentum 0.9, weight decay 1.0 × 10−4, and an initial learning rate of 0.02.Source: C. Implementation details · p. 5
  • Coarse and fine adrenal localization used isotropic spacings of 2.0 mm and 1.0 mm, respectively, with fine-stage crops of 94 × 125 × 125 voxels.Source: C. Implementation details · p. 5

A. Datasets and evaluation settings

C. Implementation details

  • Fig. 2 depicts the coarse-to-fine framework; its embeddings are reduced to three dimensions by PCA and shown as RGB images.The adrenal centers are projected along the frontal plane for visualization only.Source: Figure 2 · p. 6

1) MAGI:

  • The MAGI dataset contains 644 CT images, with adrenal-center ground truth derived from expert-annotated masks and a 480/52/112 train-validation-test split.The images span varying sizes and spacings.Source: 1) MAGI · p. 6

2) DAP Atlas:

The section defines the evaluation datasets and one-shot protocol, then lists comparison methods and supervised reference bounds.

2) DAP Atlas:

  • DAP Atlas supplies 533 whole-body CT scans; adrenal masks yielded centroid ground-truth points, with 360 training, 41 validation, and 132 test cases.Source: 2) DAP Atlas · p. 6

3) MRISegmentator:

  • MRISegmentator contributes 729 of 780 multi-sequence MRI scans containing both glands, split into 454 training, 44 validation, and 231 test scans.Across all three settings, the same MAGI training sample served as support; query tests came from MAGI, DAP Atlas, or MRISegmentator, and other training and validation data were reserved for upper-bound models.Source: 3) MRISegmentator · p. 6, Figure 3 · p. 6
  • Evaluation uses mean Euclidean distance (MED) and success detection rate (SDR), reported at 10 mm and 20 mm thresholds, with Wilcoxon signed-rank tests at 0.05 significance.Source: 3) MRISegmentator · p. 6

B. Comparison methods

  • Comparisons include registration-based Affine and ConvexAdam, embedding methods MedLAM, SAM, UAE-S, and UAE-M, plus Anatomix and nnLandmark-trained upper- and lower-bound models.The upper bound uses the full training set for the test dataset; the lower bound uses only the single support sample.Source: B. Comparison methods · pp. 6–7

C. Comparison results

UNIVERS outperforms competing one-shot methods in both intra-dataset and cross-dataset intra-modality localization, with reported MED differences statistically significant.

1) Results under the intra-dataset one-shot setting:

2) Results under the cross-dataset intra-modality one-shot setting:

3) Results under the cross-dataset cross-modality one-shot setting:

UNIVERS leads in cross-dataset cross-modality localization. Ablations also show that the proposed patch-generation strategy outperforms single-pipeline and combined augmentation variants.

3) Results under the cross-dataset cross-modality one-shot setting:

  • UNIVERS achieved MED 9.65 mm and SDR10 64.72% in cross-modality testing, versus UAE-M at MED 12.29 mm and SDR10 41.77%.The MED reduction was 2.64 mm, the SDR10 increase was 22.95%, and MED differences from one-shot competitors were statistically significant (p<0.001).Source: 3) Results under the cross-dataset… · p. 8

1) Results on the impact of the patch quadruplet generation strategy:

  • Cropping-only raised MED by 2.04/3.89/2.69 mm, while geometric-augmentation-only raised it by 1.22/4.69/1.38 mm across the three settings.The results indicate that either single pipeline is insufficient to capture the spatial variation in real data for effective embedding learning.Source: 1) Results on the impact of the patch… · pp. 8–9, Table IV · p. 9
  • Against cropping-plus-geometric augmentation, UNIVERS reduced MED by 0.48/2.24/1.76 mm and raised SDR10 by 1.79/9.47/8.88% across the three settings.The authors suggest that combining large FOV shifts with severe deformations may create overly destructive variations that hinder reliable voxel correspondences.Source: 1) Results on the impact of the patch… · p. 9, Table IV · p. 9

2) Results of sub-component ablations on the contrastive learning strategy:

The ablations show that both semantic embedding learning and multi-positive cross-domain contrastive learning contribute to one-shot localization. UNIVERS consistently outperformed state-of-the-art methods across three datasets and three one-shot settings.

2) Results of sub-component ablations on the contrastive learning strategy:

  • Disabling the CPSC loss and DS decoder increased MED by 3.84, 6.25, and 3.57 mm and reduced SDR10mm by 21.87%, 23.86%, and 14.72% across three settings.The authors interpret these degradations as evidence that CPSC-captured high-level semantic learning is vital for precise one-shot localization.Source: 2) Results of sub-component ablations… · p. 9
  • Compared with intra-domain single-positive learning, the multi-positive strategy reduced MED by 12.75 mm and 14.74 mm and improved SDR10mm by 18.18% and 37.66% in the two cross-dataset settings.The two settings are cross-dataset intra-modality and cross-dataset cross-modality, respectively.Source: 2) Results of sub-component ablations… · p. 9
  • Compared with cross-domain single-positive learning, multi-positive learning reduced MED by 1.37, 1.68, and 0.42 mm and improved SDR10mm by 7.14%, 7.95%, and 4.11% across the three settings.The authors attribute the advantage to combining positive samples from intra- and cross-domain patches for more comprehensive supervision.Source: 2) Results of sub-component ablations… · p. 9

V. CONCLUSION

  • UNIVERS consistently outperformed state-of-the-art methods across three challenging datasets and three one-shot settings of increasing difficulty.The conclusion describes the framework as a promising approach for one-shot landmark localization in real clinical applications.Source: §V · pp. 9–10
  • UNIVERS synthesizes pseudo-image pairs and patch quadruplets with randomized appearances and spatial contexts, then learns domain-invariant voxel embeddings for coarse-to-fine adrenal-center localization.The conclusion describes the framework as end-to-end trainable and uses a single frozen UNIVERS network at inference.Source: §V · pp. 9–10

Related papers

Loading 2610.29098v1…