Source-linked AI summary
H-vmunet: High-order Vision Mamba UNet for Medical Image Segmentation
Renkai Wu, Yinghao Liu, Pengchen Liang, Qing Chang
TL;DR
Medical image segmentation needs models that capture remote information without the memory demands of Transformers or the limitations of convolution. This paper proposes H-vmunet, combining higher-order SS2D and Local-SS2D within a UNet-based Vision Mamba framework; it reports strong competitiveness across three datasets and a 67.28% parameter reduction versus VM-UNet.
Problem
CNNs are limited in handling long-range information, while Vision Transformers have low local-feature sensitivity and high memory requirements.
Method
H-vmunet integrates H-SS2D and Local-SS2D through an H-VSS module within the UNet framework for medical image segmentation.
Results
H-vmunet shows strong competitiveness across ISIC2017, Spleen, and CVC-ClinicDB while reducing parameters by 67.28% versus VM-UNet.
Takeaways & Limitations
Higher-order interactions are presented as a way to preserve SS2D’s global receptive field while reducing redundant information and emphasizing local features.
Takeaways & Limitations
The 1-order operation is weak at reducing redundant feature information, motivating the subsequent higher-order design.
Abstract
from arXiv · showhide
In the field of medical image segmentation, variant models based on Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs) as the base modules have been very widely developed and applied. However, CNNs are often limited in their ability to deal with long sequences of information, while the low sensitivity of ViTs to local feature information and the problem of secondary computational complexity limit their development. Recently, the emergence of state-space models (SSMs), especially 2D-selective-scan (SS2D), has had an impact on the longtime dominance of traditional CNNs and ViTs as the foundational modules of visual neural networks. In this paper, we extend the adaptability of SS2D by proposing a High-order Vision Mamba UNet (H-vmunet) for medical image segmentation. Among them, the proposed High-order 2D-selective-scan (H-SS2D) progressively reduces the introduction of redundant information during SS2D operations through higher-order interactions. In addition, the proposed Local-SS2D module improves the learning ability of local features of SS2D at each order of interaction. We conducted comparison and ablation experiments on three publicly available medical image datasets (ISIC2017, Spleen, and CVC-ClinicDB), and the results all demonstrate the strong competitiveness of H-vmunet in medical image segmentation tasks. The code is available from https://github.com/wurenkai/H-vmunet .
1 Introduction
H-vmunet extends SS2D with higher-order interactions and integrates the resulting H-VSS module into UNet for medical image segmentation. The approach targets redundant global information while preserving long-range modeling, and reports reduced parameters with improved performance across three datasets.
- Motivation: CNN-based methods struggle to acquire remote information, while Transformer-based methods usually require substantial memory.
- Motivation: SSMs can capture remote dependencies with parallel training, motivating their use as an alternative visual modeling foundation.
- Contributions: H-SS2D progressively applies higher-order SS2D interactions to preserve a global receptive field while minimizing redundant information.
- Contributions: The H-VSS module is built around H-SS2D, and H-vmunet combines H-VSS with the UNet framework for medical image segmentation.
- Results: 67.28% fewer parameters than VM-UNet, with significantly improved performance across ISIC2017, Spleen, and CVC-ClinicDB.
2 Related work
Medical image segmentation has evolved from traditional methods to deep learning architectures, with U-Net becoming central because its skip connections preserve and fuse features across levels. Recent work introduced Vision Mamba into U-Net, but VM-UNet retained memory and performance problems that motivate H-vmunet.
- 2.1 Image segmentation: Traditional segmentation methods use mathematical image-processing techniques to distinguish regions, but deep learning later became dominant as hardware enabled more complex computation.
- 2.2 Medical image segmentation: U-Net accelerated medical image segmentation by preserving and fusing high-level and low-level features through skip connections.
- 2.2 Medical image segmentation: Medical image segmentation requires stronger lesion-feature learning than natural-scene segmentation and must address complex scale variation.
- 2.2 Medical image segmentation: U-Net variants refine skip connections and use attention to improve the combination of feature information across multiple scales.
- 2.2 Medical image segmentation: VM-UNet introduced pure Vision Mamba into U-Net for medical image segmentation, but still showed excessive memory usage and insufficient performance against current models.
- 2.2 Medical image segmentation: H-vmunet combines higher-order VMamba feature extraction with U-Net to enhance target-feature extraction and reduce redundant feature fusion while using SS2D’s global receptive field.
3 Method
H-vmunet is a six-layer U-shaped medical image segmentation network that integrates H-VSS modules, SS2D, Local-SS2D, and multilevel, multiscale feature fusion. Its high-order scan progressively processes feature channels to retain global interactions while reducing redundant information.
- Architecture: H-vmunet uses a six-layer U-shaped architecture with encoder, decoder, and skip-connection components.Its channel widths are [8,16,32,64,128,256].
- Architecture: Layers 3–6 combine H-VSS modules with 3 × 3 convolutions, using H-VSS orders 2, 3, 4, and 5 respectively.Layers 1 and 2 use standard 3 × 3 convolutions.
- High-order visual state space: H-VSS replaces Transformer self-attention with high-order 2D-selective-scan for spatial interactions of arbitrary order.The module also includes LayerNorm and an MLP around the H-SS2D operation.
- Local-SS2D: Local-SS2D splits features between convolution and SS2D pathways, then concatenates them to combine local and global feature learning.The module uses LayerNorm, a 3 × 3 convolution, SS2D, and concatenation.
- 1-order 2D-selective-scan: H1-SS2D projects features to twice the channel count, applies Local-SS2D to half, then gates the remaining half before another SS2D operation.The gating uses a Hadamard product, followed by linear projection.
- High-order 2D-selective-scan: Higher-order SS2D partitions channels across successive interactions, using Local-SS2D outputs in Hadamard products before each SS2D operation.For order n, the channel count at order k is C_k = C 2^(n−k−1), for 0 ≤ k ≤ n−1.
4 Experiment
Experiments evaluated H-vmunet on three public medical image datasets using standard segmentation metrics and comparisons with established models. The results also examined parameter efficiency and the effects of SS2D design choices and interaction orders.
- Datasets: Experiments used ISIC2017, Spleen, and CVC-ClinicDB datasets to evaluate medical image segmentation performance.The experiments included skin-lesion, spleen, and polyp segmentation datasets.
- Evaluation setup: Performance was assessed using DSC, sensitivity, specificity, and accuracy, with accuracy computed from TP, TN, FP, and FN.The experiments used BceDice loss, 250 training epochs, batch size 8, and AdamW optimization.
- Comparison results: H-vmunet improved over MHorUNet on all three public datasets, indicating that incorporating SS2D into higher-order interactions improved the reported results.Table 1 compared H-vmunet variants with multiple medical segmentation models, including MHorUNet and VM-UNet.
- Efficiency comparison: 67.28% fewer parameters were reported for H-vmunet than VM-UNet, while the reduction relative to MHorUNet was 6.37%.Figure 5 compared parameter counts and memory usage among H-vmunet, MHorUNet, and VM-UNet.
- Ablation experiments: Ablations compared four SS2D-related configurations, including replacements of SS2D and Local-SS2D, with setup 4 representing the proposed H-SS2D.The study also tested SS2D interaction orders [1] [2] [3] and [2] [3] [5] against single identical orders.
5 Conclusion
The paper proposes H-vmunet, a UNet-based medical image segmentation framework that embeds H-SS2D in a higher-order visual state-space module. Across three public datasets, it reports reduced redundancy and local-feature enhancement alongside a 67.28% parameter reduction versus VM-UNet.
- Method: H-SS2D maintains SS2D’s global receptive field while minimizing redundant information through higher-order operations.The method is based on a state-space model and is embedded in the H-VSS module.
- Framework and validation: H-vmunet combines H-VSS with the UNet framework for medical image segmentation and was validated on ISIC2017, Spleen, and CVC-ClinicDB.The proposed model uses higher-order visual state-space processing within a U-shaped segmentation architecture.
- Reported outcome: 67.28% fewer parameters were reported for H-vmunet than for the traditional pure Vision Mamba UNet model VM-UNet.The conclusion also attributes improved local-feature learning and reduced redundant information to H-SS2D.