PaperlayerView exact revision on arXiv

Source-linked AI summary

MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation

Jiacheng Ruan, Suncheng Xiang, Mingye Xie, Ting Liu, Yuzhuo Fu

arXiv:2211.01784v1eess.IVcs.CV

TL;DR

MALUNet addresses the difficulty of deploying increasingly complex segmentation models in resource-limited clinical settings. It combines a six-stage U-shape architecture with four attention modules, achieving a state-of-the-art balance among parameters, computational complexity, and segmentation performance on ISIC2017 and ISIC2018.

  • Problem

    Existing segmentation models often add complex modules that increase parameters and computational demands, limiting use in resource-constrained clinical environments.

  • Method

    MALUNet combines a six-stage U-shape architecture with four attention modules that model global-local features, sample relationships, and multi-stage channel and spatial information.

  • Results

    MALUNet achieves a state-of-the-art balance among parameters, computational complexity, and segmentation performance on the ISIC2017 and ISIC2018 datasets.

  • Takeaways & Limitations

    The study supports lightweight medical image segmentation models that retain competitive performance while using fewer parameters and lower computational complexity.

Abstract

from arXiv · show

Recently, some pioneering works have preferred applying more complex modules to improve segmentation performances. However, it is not friendly for actual clinical environments due to limited computing resources. To address this challenge, we propose a light-weight model to achieve competitive performances for skin lesion segmentation at the lowest cost of parameters and computational complexity so far. Briefly, we propose four modules: (1) DGA consists of dilated convolution and gated attention mechanisms to extract global and local feature information; (2) IEA, which is based on external attention to characterize the overall datasets and enhance the connection between samples; (3) CAB is composed of 1D convolution and fully connected layers to perform a global and local fusion of multi-stage features to generate attention maps at channel axis; (4) SAB, which operates on multi-stage features by a shared 2D convolution to generate attention maps at spatial axis. We combine four modules with our U-shape architecture and obtain a light-weight medical image segmentation model dubbed as MALUNet. Compared with UNet, our model improves the mIoU and DSC metrics by 2.39% and 1.49%, respectively, with a 44x and 166x reduction in the number of parameters and computational complexity. In addition, we conduct comparison experiments on two skin lesion segmentation datasets (ISIC2017 and ISIC2018). Experimental results show that our model achieves state-of-the-art in balancing the number of parameters, computational complexity and segmentation performances. Code is available at https://github.com/JCruan519/MALUNet.

I. INTRODUCTION

MALUNet addresses the tension between segmentation accuracy and deployment constraints by combining four attention modules with a lightweight U-shape architecture. Experiments on ISIC2017 and ISIC2018 evaluate its balance of parameters, computational complexity, and segmentation performance.

  • Motivation: Existing segmentation models often add complex modules that are difficult to deploy on mobile medical devices with limited memory.The introduction also identifies limitations in modeling relationships between samples and fully using multi-stage, multi-scale information.
  • Proposed approach: Four attention modules extract global and local features, characterize whole-dataset sample relationships, and fuse multi-stage information into channel and spatial attention maps.The modules are DGA, IEA, CAB, and SAB.
  • Proposed approach: MALUNet combines the four modules with a U-shape architecture to reduce parameters and computational complexity while maintaining competitive segmentation performance.The contribution describes MALUNet as having the minimum parameter count and lowest computational complexity.
  • Experiments: Extensive experiments on ISIC2017 and ISIC2018 report state-of-the-art balance among parameters, computational complexity, and segmentation performance.The claim concerns the balance of these three criteria across the two datasets.

II. RELATED WORKS

Medical image segmentation has largely evolved from fully convolutional and UNet-based models toward attention and Transformer-enhanced architectures. The paper positions MALUNet as a lightweight alternative because many recent methods improve performance at the cost of parameters and computational complexity.

  • Convolutional and UNet-based models: FCN introduced fully convolutional feature extraction, followed by UNet, an encoder-decoder model that achieved strong medical image segmentation results.UNet uses a U-shape architecture and became the basis for many subsequent medical segmentation models.
  • Convolutional and UNet-based models: UNet variants such as UNet++, Attention-UNet, Res-UNet, and Dense-UNet modify feature connections or attention mechanisms to improve segmentation.The cited passage describes dense skip connections, attention gates, and information-preserving changes.
  • Transformer-enhanced models: Recent methods combine Vision Transformers or their variants with UNet to improve feature extraction and segmentation performance.Examples include Swin-UNet, DS-TransUNet, TransFuse, and UTNetV2, which use Transformer, multiscale, parallel-encoder, or hybrid structures.
  • Lightweight segmentation: Many prior networks improve performance while retaining heavy parameter counts and computational complexity, limiting practical medical deployment.The related-work discussion motivates lightweight segmentation models for practical applications and mobile health.

III. METHODS

The methods section introduces four attention modules and combines them with a U-shape architecture to address the need for global, local, and multi-stage feature information in lightweight segmentation.

  • Overall architecture: MALUNet combines four proposed modules with a U-shape architecture for lightweight medical image segmentation.The modules are DGA, IEA, CAB, and SAB.
  • Motivation: Global information helps locate lesion regions, while local information captures lesion edges and corners for more complete predictions.
  • DGA: The DGA overview specifies a 3 × 3 convolution kernel unless otherwise stated.
  • DGA: DGA uses split dilated convolutions and gated attention to extract global and local feature information.It contains an SDC unit and a GA unit; depthwise separable convolutions use dilation rates 5 and 7 for global information and 1 and 2 for local information.
  • DGA: The DGA feature map is split into four channel-wise parts, processed separately, concatenated, and passed through a convolution.The supplied notation defines the split and dilated depthwise separable convolutions, while sigmoid gating and element-wise multiplication are also used.

B. Inverted External Attention Block

IEA adapts external attention for lightweight medical image segmentation by representing dataset-level information and restoring the input feature dimensions through an expanded intermediate representation.

  • Attention mechanism: IEA uses two memory units to characterize feature information from the entire dataset and strengthen associations between samples.The memory units are parameter-sharing Conv1D operations in the external-attention formulation.
  • Design motivation: IEA is an inverted external attention block designed for a lightweight medical image segmentation model.It is motivated by external attention and the inverted residual module in MobileNetV2.
  • Processing pipeline: For an input X ∈ R^C×H×W, IEA reshapes it to C × HW, expands channels fourfold, restores the dimension, and reconstructs the feature-map shape.A final 1 × 1 convolution and residual addition produce the output feature map.
  • Processing pipeline: IEA expands the feature map to 4C × HW rather than fixing the intermediate channel count at 64.

C. Channel Attention Bridge Block

CAB fuses multi-stage and multi-scale encoder features along the channel axis to generate channel attention maps using complementary local and global fusion operations.

  • Purpose: CAB generates channel attention maps by concatenating features from different stages along the channel axis.The module targets multi-stage and multi-scale information integration for segmenting targets of different sizes.
  • Operations: CAB uses global average pooling, channel-wise concatenation, 1D convolution, fully connected layers, sigmoid activation, and element-wise multiplication.The notation defines t_i as feature maps from different encoder stages and s as the total number of stages.
  • Architecture illustration: The CAB architecture is illustrated in Fig. 5.
  • Fusion design: CAB divides feature fusion into local 1D convolution and global stage-specific fully connected operations.A five-stage model is used to visualize the design.

D. Spatial Attention Bridge Block

SAB fuses multi-stage and multi-scale features spatially to generate stage-specific spatial attention maps through pooled channel summaries and a shared dilated convolution.

  • Purpose: SAB fuses multi-stage and multi-scale information along the spatial axis to generate an attention map for each stage.
  • Processing: For each stage, SAB concatenates channel-wise average- and max-pooled maps before applying a shared dilated convolution and sigmoid activation.The convolution uses dilation rate 3 and kernel size 7.
  • Output: The generated spatial attention map is multiplied element-wise with the original feature map and combined with residual information.

E. MALUNet

MALUNet combines a six-stage U-shape architecture with attention modules that process dataset-level, sample-level, multi-stage, and multi-scale information. Bridge attention features are added to decoder features to reduce semantic differences and sampling-related information loss.

  • MALUNet combines attention modules with a U-shape architecture to form a lightweight medical image segmentation model.
  • The six-stage architecture uses channels {8, 16, 24, 32, 48, 64}, with IEA and DGA arranged in series in Stages 4-6.IEA captures overall dataset features and sample relationships, while DGA acquires global and local information within samples.
  • The SAB architecture is designed as a bridge attention module for processing multi-stage features.
  • Bridge attention modules fuse encoder features from Stages 1-5 and generate spatial and channel attention maps.
  • The resulting bridge features are added to decoder features to reduce encoder-decoder semantic differences and alleviate information loss from sampling.

A. Datasets

The experiments use ISIC2017 and ISIC2018 dermoscopy datasets with segmentation masks, split randomly into training and testing sets. Images are normalized, resized, and augmented before training.

  • ISIC2017 contains 2150 dermoscopy images and ISIC2018 contains 2694 images, each with segmentation mask labels.
  • The datasets are randomly divided at a 7:3 ratio into training and testing sets.ISIC2017 uses 1500 training and 650 testing images; ISIC2018 uses 1886 training and 808 testing images.
  • All images are normalized and resized to 256 × 256, with vertical flips, horizontal flips, and random rotation used for augmentation.
  • The loss uses binary cross-entropy and Dice components weighted by λ1 and λ2, with both weights set to 1 by default.

C. Evaluation Metrics

Segmentation performance is evaluated with overlap, classification, and resource metrics. Comparisons are conducted on ISIC2017 and ISIC2018 using fixed 256 × 256 inputs for parameter and complexity measurements.

  • The evaluation uses mIoU, DSC, Accuracy, Sensitivity, and Specificity to measure segmentation performance.
  • Params measures model parameters in millions, while GFLOPs measures computational complexity.
  • Parameter and GFLOPs measurements use a 256 × 256 input size.
  • The metrics are defined using true positives, false positives, false negatives, and true negatives.
  • Comparative experiments evaluate models on the ISIC2017 dataset, with bold and underlined entries marking the best and second-best results.
  • Comparative experiments evaluate models on the ISIC2018 dataset, with bold and underlined entries marking the best and second-best results.

E. Ablation Studies

Ablation studies establish a compact BasicUNet baseline and evaluate individual and hybrid module configurations on ISIC2018. The final MALUNet configuration is selected by balancing performance, parameters, and computational complexity.

  • Baseline determination: BasicUNet uses six stages, one plain convolution per stage, element-wise addition between encoder and decoder, and channels {8, 16, 24, 32, 48, 64}.
  • Single-module ablation: Single-module ablations train and test each proposed module on ISIC2018, with EA included as a comparison for IEA.
  • Single-module ablation: DGA improves performance while reducing parameters and computational complexity in the single-module ablation.The authors associate this result with simultaneous acquisition of global and local feature information.
  • Single-module ablation: The other three modules improve performance while introducing only a few additional parameters.
  • Hybrid ablation: The four-module hybrid ablation on ISIC2018 compares configurations to determine the final form of MALUNet.
  • Hybrid ablation: BU+IEA+DGA+(B) is selected as MALUNet because the design prioritizes balance among parameters, computational complexity, and performance.BU+DGA+IEA+(B) achieved the best performance, but was not selected under the stated lightweight-design objective.

V. CONCLUSION

The study defines four attention goals and combines them in MALUNet, a light-weight six-stage U-shape model for medical image segmentation. Experiments on ISIC2017 and ISIC2018 support a state-of-the-art balance among model size, computational complexity, and performance, while the authors identify skin-lesion segmentation as the current scope.

  • Conclusion: Four attention modules target global and local features, dataset-wide sample characteristics, and channel- and spatial-level attention maps.The four goals cover feature extraction, inter-sample relationships, and multi-stage, multi-scale feature fusion.
  • Conclusion: The ablation study evaluates the performance of four modules under CAB-first and SAB-first ordering schemes.
  • Conclusion: MALUNet combines the proposed modules with a six-stage U-shape architecture as a light-weight medical image segmentation model.
  • Conclusion: Comparative experiments and visualization results on ISIC2017 and ISIC2018 support state-of-the-art balance among parameters, computational complexity, and segmentation performances.
  • Conclusion: MALUNet is currently proposed only for skin lesion segmentation, with pruning and neural architecture search planned to extend it to other diseases.
Loading 2211.01784v1…