Source-linked AI summary
A General-Purpose Dual-Path Interaction Framework for Multi-View Medical Image Segmentation: Alleviating Data Volume Dependency in Small-Sample Regimes
Shuhuan Wang, Yuanyuan Cui, Pengju Wang, Huimin Guo, Ling Yue, Yin Gu, Yanchao Zhang, Wei Feng, Yunhe Liu, Wei Qian, Mingjun Xu, Shuang Liu, He Ma
TL;DR
Medical image segmentation relies on large annotated datasets, while existing multi-view approaches face constraints including paired-sample requirements. DPIF combines cross-view alignment, shared-feature knowledge transfer, and topological regularization in a plug-and-play framework; across 24 backbones, it improves performance and can match or exceed full-data single-path models using a subset of data.
Problem
Medical segmentation depends on large annotated datasets, and multi-view methods may be architecture-coupled or require strictly paired samples.
Source: §1 · p. 2Method
DPIF is a plug-and-play dual-path framework that aligns unpaired cross-view distributions, transfers knowledge within shared features, and regularizes cross-view topology.
Source: §1 · p. 2, §3.2 · p. 3, §3.2.2 · p. 5Results
Across 24 encoder-decoder backbones, DPIF improved performance and could match or exceed full-data single-path models using a subset of data.
Source: §7 · p. 17Takeaways & Limitations
DPIF is presented as a practical, scalable solution for multi-view medical image segmentation under data-scarce conditions.
Source: §7 · p. 17The current implementation supports only two-view interactions, and its differentiable persistent-homology approximation may miss subtle topological features.
Source: §6 · p. 17
Abstract
from arXiv · showhide
The performance of medical image segmentation depends on a large amount of annotated data, and most segmentation models experience a significant drop in performance in low-sample-size scenarios. Existing semi-supervised and weakly supervised methods rely on specific network architectures, while traditional multi-view-assisted segmentation methods require strictly paired samples, which limits their clinical applications. To address these issues, we propose DPIF, a general-purpose plug-and-play dual-path interaction framework compatible with arbitrary encoder-decoder segmentation networks, which leverages cross-view anatomical complementarity to reduce annotation reliance. DPIF first performs Sinkhorn optimal transport-based cross-view distribution alignment to establish global semantic correspondence without sample-level pairing. It then adopts conditional mutual information-guided feature decoupling to enable bidirectional knowledge transfer only within the shared semantic space, suppressing view-specific noise. Finally, persistent homology-based topological regularization encodes anatomical invariants as unsupervised priors to mitigate overfitting. Experiments were conducted using 24 baseline models on a private multicenter echocardiography dataset and two public datasets. The results show that DPIF-enhanced models outperform single-view baseline models when trained on comparable amounts of data. Notably, most dual-path variants trained on 40\% - 60\% of the data achieve performance comparable to that of single-view models trained on the full dataset. This demonstrates that DPIF provides a general-purpose and scalable solution for data-scarce multi-view medical image segmentation problems.
1 Introduction
Medical segmentation depends on costly pixel-level annotations, and multi-view imaging adds annotation workload despite providing complementary anatomical information. DPIF is proposed as a plug-and-play framework that uses cross-view complementarity to reduce reliance on annotated data.
1 Introduction
- Pixel-level annotation is costly and requires clinical expertise, while annotating each view separately further increases workload and costs.The passage identifies scarce annotations as a bottleneck for clinical translation, particularly for rare diseases.Source: §1 · p. 2
- Multi-view images provide complementary anatomical information, but independent processing and simple fusion can miss cross-view relationships or introduce redundant noise.Existing frameworks are also often coupled to specific backbones, hindering adaptation and cross-scenario deployment.Source: §1 · p. 2
- DPIF uses cross-view anatomical complementarity to reduce reliance on segmented annotations and is designed as a plug-and-play framework for encoder-decoder networks.Its components align unpaired cross-view distributions, transfer knowledge within shared features under conditional mutual information constraints, and use topological consistency as a prior against small-sample overfitting.Source: §1 · p. 2
2 Related WORK
Prior multi-view segmentation approaches fuse views in several ways but face challenges involving view differences, architecture constraints, and paired data. Small-sample approaches address annotation scarcity through semi-supervised, weakly supervised, or few-shot learning, with limitations described for each.
2.1 Multi-view Medical Image Segmentation
- Input-, feature-, and decision-level fusion methods respectively stack images, combine intermediate features, or merge prediction outputs, with trade-offs in spatial differences, view-specific noise, and semantic information.Many schemes remain architecture-constrained, perform poorly with limited data, and require strictly paired multi-view patient data.Source: §2.1 · pp. 2–3
2.2 Small-Sample Medical Image Segmentation
- Semi-supervised methods still require unlabeled data, weak labels impose an accuracy ceiling, and few-shot transfer faces uncertain effectiveness across medical imaging domains.Existing approaches rarely exploit correlations among views or provide universal reusable interaction, despite multi-view images costing less to acquire than single-view annotations.Source: §2.2 · p. 3
3 Method
DPIF connects two independently initialized segmentation branches through feature decoupling, unpaired distribution alignment, and adaptive bidirectional interaction, with topological regularization and a joint loss. Its design aims to exchange shared cross-view semantics while limiting view-specific noise and adapting interaction to segmentation uncertainty.
3.1 Overview
- DPIF uses two independently initialized branches for single-view segmentation, interacts during training, and disables the interaction module at inference.Source: §3.1 · p. 3
3.2 Optimal Transmission Decoupling Interaction Module
- The ODI module connects the segmentation pipelines through feature decoupling, unpaired distribution alignment, and adaptive bidirectional feature enhancement.Source: §3.2 · pp. 3, 5
3.2.1 Two-Layer Feature Decoupling
- Two-layer decoupling separates shared semantics for cross-view exchange from view-specific features excluded from interaction, using conditional mutual-information constraints to limit noise in the shared space.Source: §3.2.1 · p. 5
3.2.2 Sinkhorn-Based Unpaired Distribution Alignment
- Sinkhorn optimal transport aligns shared-feature distributions globally, allowing bidirectional semantic correspondence and knowledge transfer without sample-level pairing.Source: §3.2.2 · pp. 5–6
3.2.3 Adaptive Interaction Weights Based on Segmentation Uncertainty
- Uncertainty-based weights adapt interaction intensity, directing more guidance to the long-axis branch when its confidence is lower than the short-axis branch, and vice versa.Source: §3.2.3 · p. 6
3.3 Cross-View Topological Consistency Regularization Loss
- Persistent-homology regularization uses cross-view topological consistency as an unsupervised anatomical prior to narrow the model’s hypothesis space and alleviate overfitting.Source: §3.3 · pp. 6–7
3.4 Joint Loss Function
- The joint objective combines segmentation, feature-decoupling, topological-consistency, and cross-view knowledge-distillation losses for end-to-end optimization.Source: §3.4 · p. 7
4 Experiment
This section describes the three multi-view datasets, their evaluation protocol, and the training setup. It also identifies the scope of the reported result visuals.
4.1 Datasets
- Evaluation spans PME echocardiography, FLARE2023 liver CT, and MSD hippocampal MRI, with 858, 581, and 172 images per view, respectively.PME includes 429 patients from five clinical centers.Source: §4.1 · p. 8
- All datasets were split by patient; 20% was reserved for independent testing, and the remainder was used for training and validation with five-fold cross-validation.Augmentation was applied only to training data.Source: §4.1 · p. 8, Figure 2 · p. 9
4.2 Evaluation Metrics
- Evaluation reports DSC (overlap), IoU (intersection/union), HD95 (boundary alignment), and ASD (mean boundary-point distance), averaged across five fold-optimal models tested independently.Source: §4.2 · p. 8
4.3 Experimental Setup
- Training used Adam with cosine-annealed learning rates, batch size 2, 100 epochs, and a backbone frozen for the first 30 epochs.The similarity threshold was τ = 0.2, distillation temperature T = 2.0, and loss weights α = 0.1, βs = 0.05, βl = 0.05.Source: §4.3 · p. 8
- Table 1 compares single- and dual-path models on PME long-axis data, while Figure 3 pairs segmentation overlays with bars showing DSC percentage improvements.Source: Table 1 · p. 9, Figure 3 · p. 10
5 Results
Across five results analyses, DPIF-enhanced models improve segmentation performance over single-path and compared multi-view methods, with gains also reported at reduced training-data levels and in clinician review. Ablations identify Sinkhorn alignment as the primary improvement driver, while topological regularization improves boundary metrics.
5.1 Comparison with Single-Path Segmentation Models
- DSC improved by 13.33% on long-axis views and 8.97% on short-axis views on average; the hippocampus long-axis gain reached 19.2%, and RollingUNet improved 57.57%.Average HD95 decreased by 4.27 on long-axis views and 2.38 on short-axis views, although a few models, including SCSEUNet, showed a slight HD95 decrease.Source: §5.1 · p. 10, Table 2 · p. 11, Table 3 · p. 11
5.2 Comparison with Multi-View Segmentation Methods
- DPIF showed a consistent performance advantage across views and datasets; on the liver long-axis view, DP-UMNet improved 5.77% over single-path UMNet and outperformed all compared methods.All other compared methods underperformed DP-UMNet across the three test datasets; most also fell below the single-path UMNet baseline.Source: §5.2 · p. 12, Table 4 · p. 12, Figure 4 · p. 13
5.3 Ablation Studies and Analyses
- Sinkhorn alignment increased DSC by 2% on the long-axis view and 2.24% on the short-axis view, while topological regularization raised long-axis DSC by 1.14% and reduced HD95 by about 0.39.Feature decoupling alone yielded only slight improvement, and adaptive interaction weights alone yielded negligible improvement without distribution alignment.Source: §5.3 · p. 14, Table 5 · p. 14
5.4 Analysis of Different Training Data Usage
- DPIF-enhanced performance increased monotonically with training data, with the largest improvement from 20% to 40%; in most cases it matched full-data single-path accuracy below 60% annotation.DP-UMNet exceeded full-data single-path models at 60% annotation on PME, needed 40% on FLARE2023, and approached the full-data baseline at 60% on hippocampal MRI.Source: §5.4 · pp. 14–15
5.5 Blind Review of Segmentation Results by Clinicians
- In PME short-axis clinician review, DP-UMNet scored 3.49 ± 0.60 and increased cases rated ≥4 from 24.4% to 52.2%.Improvements were most pronounced where baseline clinical scores were weakest; no further significant gains were found for already-high-scoring FLARE2023 and MSD short-axis cases.Source: §5.5 · p. 15, Table 6 · p. 15
6 Discussion
DPIF improves performance on challenging ultrasound and MRI datasets where traditional multi-view methods may underperform single-view baselines, and its data-utilization gains grow with task difficulty. It also serves as an enhancement module for fully supervised multi-view pipelines, although the current implementation is limited to two views and approximate topological regularization.
6 Discussion
- DPIF improves performance on challenging ultrasound and MRI datasets, while traditional multi-view methods can underperform single-view baselines on noisy, unpaired data.The reported gains are attributed particularly to feature alignment and two-layer decoupling, which mitigate noise and differences in interaction targets.Source: §6 · p. 15
- Performance gains increase as task difficulty rises and baseline performance falls, with improvements continuing even at 100% data usage.The discussion identifies low-performing backbones such as RollingUNet as a potential fit for rare-disease settings with scarce annotations.Source: §6 · p. 15
- Figure 5 compares training-data amounts with the model’s full-dataset single-path DSC, shown by the dashed line.Source: Figure 5 · p. 16
- DPIF is presented as an enhancement module for fully supervised multi-view clinical pipelines, not only as a few-shot learning approach.Source: §6 · p. 17
- The implementation supports only two-view interactions, and its differentiable persistent-homology approximation may miss subtle topological features.The discussion identifies extension to three or more views for volumetric 3D segmentation and a more precise differentiable topological solver as needed developments.Source: §6 · p. 17
7 Conclusion
The conclusion presents DPIF as a plug-and-play framework that improves data utilization in multi-view medical image segmentation. Across 24 encoder-decoder backbones, it reports gains over existing multi-view methods and performance comparable to or better than full-data single-path models using a subset of data.
7 Conclusion
- DPIF improves data utilization across 24 encoder-decoder backbone networks in multi-view medical image segmentation.Source: §7 · p. 17
- DPIF outperforms existing multi-view segmentation methods and can match or exceed full-dataset single-path models using a subset of data.Source: §7 · p. 17
- The conclusion characterizes DPIF as a practical, scalable solution for multi-view segmentation under data-scarce conditions.Source: §7 · p. 17