PaperlayerView exact revision on arXiv

Source-linked AI summary

CiUNet: A Hybrid Swin-CNN UNet for Medical Image Segmentation

Bin Dong, Jinghong Chen

arXiv:2608.22281v1eess.IVcs.CV

TL;DR

Medical image segmentation needs both accurate local-detail recovery and practical deployment properties, while pure Transformer U-Nets have limited local detail capture and interpretability. CiUNet addresses these gaps with dual encoders, XSkip connections, feature fusion, auxiliary supervision, and boundary-aware loss design, and reports extensive Synapse evaluations.

  • Problem

    Pure Transformer-based U-shaped networks can inadequately capture local details and offer limited interpretability for medical image segmentation.

  • Method

    CiUNet combines Swin and lightweight CNN encoders with asymmetric fusion, XSkip connections, auxiliary supervision heads, and dynamically weighted Dice loss.

  • Results

    Extensive experiments evaluate CiUNet on the Synapse multi-organ CT segmentation dataset using Dice and Hausdorff-based measures.

  • Takeaways & Limitations

    The encoder-decoder separation supports local processing of sensitive CT images while transmitting high-level semantic features for cloud-based decoder computation.

  • Takeaways & Limitations

    As a 2D slice-wise framework, CiUNet cannot adequately recover inter-slice context, leaving small organs sensitive to boundary-slice uncertainty.

Abstract

from arXiv · show

Medical image segmentation requires high accuracy and robustness, yet practical commercial deployment also demands privacy preservation and computational efficiency. In this context, the U-Net architecture, which can be inherently decoupled into independent encoder and decoder components, serves as a natural commercial choice. However, pure Transformer-based variants like Swin-UNet often suffer from insufficient local detail capture and limited interpretability. In this paper, we propose a lightweight hybrid architecture built upon the Swin-UNet framework. Our model integrates a parallel CNN encoder to complement the shallow layer reasoning of Swin Transformers with local texture features. To bridge the semantic gap and enhance fine-grained spatial detail recovery, we design an asymmetric feature fusion strategy and introduce cross-layer skip (XSkip) connections that explicitly propagate shallow CNN features into the decoder. We further incorporate novel loss functions and an auxiliary supervision head (Aux-Head) to strengthen training stability, boundary delineation, and intermediate feature interpretability. Extensive experiments on the Synapse multi-organ segmentation dataset demonstrate that our approach achieves state-of-the-art competitive Dice scores and Hausdorff distances, offering an accurate, efficient, and interpretable solution for clinical deployment.

1 Introduction

CiUNet revisits Swin-UNet with a lightweight hybrid design that combines global Transformer reasoning with local CNN features, enhanced skip connections, and auxiliary supervision. Evaluated on Synapse, it targets accurate, efficient, interpretable, and privacy-preserving deployment.

  • The model is evaluated on Synapse, a challenging eight-organ CT benchmark containing 30 scans and 3,779 axial slices with variable anatomy and complex boundaries.
  • CiUNet combines a primary Swin encoder for global context with a parallel lightweight ResNet encoder for complementary high-resolution local features.
  • XSkip selectively aligns and propagates shallow CNN features into deeper decoder layers through spatial attention to recover resolution and sharpen boundaries.
  • Auxiliary segmentation heads provide multi-level gradient guidance that improves training stability, accelerates convergence, and supports intermediate-feature interpretability.
  • UNet encoder-decoder separation is presented as enabling local processing of sensitive CT images while transmitting only high-level semantic features for cloud decoding.
  • 83.72% mean Dice with 31.68M trainable parameters demonstrates the reported balance of segmentation accuracy and model compactness.

2 Related Work

The paper situates CiUNet within a field shaped by the progression from convolutional to hybrid medical image segmentation models. Figure 1 compares CiUNet with other methods using parameter count and DSC on Synapse.

  • Deep learning medical segmentation has progressed from purely convolutional architectures toward advanced hybrid designs.
  • Figure 1 compares CiUNet with other methods according to parameter count and DSC on the Synapse dataset.

2D Convolution-based Segmentation Methods

U-Net established the standard 2D convolutional encoder-decoder pattern by combining semantic features with fine-grained spatial details through long-range skip connections. Later variants optimize feature propagation and skip pathways.

  • U-Net became the de facto standard for 2D medical image segmentation through its symmetric encoder-decoder structure and long-range skip connections.
  • Res-UNet, U-Net++, and UNet3+ refine the U-shaped backbone through deeper propagation, nested dense skips, and full-scale skip connections.

2D Transformer-based Segmentation Methods

Vision Transformers introduced global self-attention to medical image segmentation, while Swin Transformer reduced computational complexity through hierarchical shifted-window attention. Swin-UNet benefits from global correlations but may preserve low-level textures and boundaries poorly.

  • Vision Transformers brought global self-attention from NLP into computer vision, enabling long-range dependency modeling.
  • Swin Transformer makes dense prediction more computationally viable through hierarchical architecture and shifted-window attention.
  • Swin-UNet uses a pure Swin Transformer encoder-decoder to capture long-range semantic correlations in medical segmentation.
  • Pure Transformer architectures can require extensive pre-training and may lose low-level textures, producing blurred boundaries and inaccurate segmentation.

2D Hybrid Segmentation Methods

Hybrid Transformer-CNN approaches combine global reasoning with local precision for medical image segmentation. Recent methods use dual-branch encoders and fusion modules to merge these complementary representations.

  • Hybrid segmentation methods reconcile Transformer-based global reasoning with CNN-based local precision.
  • TransUNet combines CNN-based initial feature extraction with Transformer-based global context encoding.
  • HiFormer uses a double-branch encoder and Double-Level Fusion module to merge Swin Transformer and CNN features.
  • MISSFormer introduces an Enhanced Transformer Block and cross-scale context mechanisms.

3D Segmentation Methods

Three-dimensional segmentation methods exploit volumetric information across consecutive slices, unlike 2D models that process isolated frames. This supports semantic coherence and reduces discontinuities between slices.

  • 3D architectures use full-organ volumetric information rather than isolated 2D frames.
  • 3D U-Net and V-Net improve semantic coherence and reduce inter-slice discontinuity by exploiting volumetric consistency.
  • 3D methods are generally considered important for practical segmentation robustness, particularly for organs requiring volumetric context.

3 Method

CiUNet is a 2D dual-encoder U-shaped network combining Swin Transformer global representations with CNN local features. Its fusion, asymmetric decoder, cross-layer skips, and auxiliary heads address multi-scale feature integration and spatial-detail recovery.

  • Architecture Overview: CiUNet is a dual-encoder U-shaped network for 2D medical image segmentation, combining Swin Transformer and CNN branches.
  • Architecture Overview: The Swin encoder spans Layer1–LayerBN, while the parallel CNN encoder extracts local features from Layer0–Layer3.
  • Architecture Overview: Both feature-extraction branches use frozen initial modules, restricting trainable parameters to subsequent down-sampling layers.
  • Swin Transformer Backbone: Swin-Tiny window and shifted-window attention provide the backbone’s local self-attention, cross-window interaction, and linear computational complexity.
  • Dual Encoder: The dual-encoder design pairs Swin global semantic reasoning with CNN high-resolution local textures.
  • Feature Fusion: A dedicated fusion module dynamically combines same-resolution branch features before decoding, prioritizing Swin semantics and complementing them with CNN spatial information.
  • Decoder: The asymmetric decoder progressively restores resolution through four successive layers, differing from Swin-Unet’s strictly symmetric decoder.
  • Auxiliary Supervision: Aux-Heads on Layer1 and Layer2 project intermediate decoder features to semantic-class logits for auxiliary supervision.

4 Experiments on Synapse Dataset

On the Synapse multi-organ CT benchmark, CiUNet combines multi-level supervision, geometric loss components, and curriculum training with a dual-encoder design. It achieves strong segmentation accuracy and boundary performance, while ablations show benefits from structural upgrades, XSkip connections, and auxiliary supervision.

  • Dataset and evaluation: The Synapse benchmark contains 30 abdominal CT scans, 3,779 axial slices, and annotations for eight organs, evaluated using DSC and HD95.The standard split uses 18 training cases and 12 testing cases.
  • Implementation details: The training objective combines pixel-level classification, region overlap, and boundary alignment through three geometric loss constraints.The implementation uses Focal Loss for class imbalance, dynamically weighted Dice loss for organ contours, and HausdorffERLoss for Hausdorff-distance approximation.
  • Implementation details: Auxiliary supervision heads attach to intermediate decoder maps to visualize deep representations and encourage structurally coherent predictions across feature levels.Each head uses a lightweight residual structure for channel alignment.
  • Results: 83.72% mean DSC and 12.13 mm mean HD95 are achieved on Synapse, with particularly strong DSC for gallbladder and both kidneys.CiUNet reports 79.47% for gallbladder, 91.39% for left kidney, and 91.27% for right kidney, while pancreas and stomach remain weaker than some state-of-the-art methods.
  • Ablation study: At 150 epochs, the upgraded framework reaches 81.40% DSC and 15.43 HD95 versus the Swin-Unet baseline’s 79.13% DSC and 21.55 HD95.At 280 epochs, the upgraded framework reaches 81.47% DSC and 14.68 HD95, indicating only marginal gains from extended training.
  • Ablation study: XSkip improves the 273-epoch model to 83.54% DSC and 11.26 HD95, while auxiliary supervision reaches 83.72% DSC and 12.13mm HD95 at 204 epochs.The auxiliary-supervision configuration already achieves 83.29% DSC and 13.21mm HD95 at 148 epochs.

5 Discussion

The discussion identifies metric bias, 2D limitations for small organs, and non-medical backbone initialization as important boundaries of CiUNet. These limitations motivate false-positive suppression, inter-slice awareness, and in-domain pre-training.

  • Metric bias toward false positives: Dice and HD95 can reward over-segmentation because slightly outward boundaries improve overlap without substantially increasing Hausdorff distance.The authors suggest explicit false-positive suppression in the loss as a refinement direction.
  • Sensitivity to boundary slices in 3D anatomy: The performance gap between 2D and 3D models remains non-negligible for small organs such as the pancreas and gallbladder.Volumetric continuity benefits 3D models, while 2D propagation of shallow features cannot adequately recover inter-slice context.
  • Sensitivity to boundary slices in 3D anatomy: Partial-volume effects and ambiguous boundaries make small organs especially uncertain at the beginning and end of volumetric sequences.The authors identify lightweight inter-slice awareness within a 2D framework as a future challenge.
  • Suboptimal backbone initialization: Swin-Tiny and the parallel ResNet branch are not specifically tailored to medical-image characteristics, making the current backbone initialization inherently suboptimal.The paper proposes joint in-domain medical pre-training for both branches.

6 Conclusion

CiUNet combines Swin Transformer global reasoning with parallel CNN local-texture sensitivity, cross-branch fusion, XSkip connections, and auxiliary supervision. On Synapse, it achieves competitive segmentation performance with fewer trainable parameters than existing 3D methods, balancing accuracy and computational efficiency.

  • 6 Conclusion: CiUNet integrates Swin global reasoning with parallel CNN texture sensitivity to address limitations of pure Transformer U-shaped networks.Cross-branch fusion and XSkip connections mitigate spatial information loss from aggressive down-sampling.
  • 6 Conclusion: Cross-branch fusion and XSkip connections mitigate spatial information loss introduced by aggressive down-sampling.The architecture propagates complementary features into the decoder for structurally coherent representations.
  • 6 Conclusion: Auxiliary supervision heads guide deep feature learning and promote more interpretable, structurally coherent intermediate representations.This supervision complements the hybrid encoder-decoder design.
  • 6 Conclusion: CiUNet achieves competitive Synapse performance with significantly fewer trainable parameters than existing 3D methods.The conclusion frames this as a balance between segmentation accuracy and computational efficiency.
Loading 2608.22281v1…