Source-linked AI summary
SGDC: Structurally-Guided Dynamic Convolution for Medical Image Segmentation
Bo Shi, Wei-ping Zhu, M. N. S. Swamy
TL;DR
Pooling-based dynamic convolution can discard high-frequency details needed for precise medical segmentation boundaries. The paper introduces SGDC, guided by an explicitly supervised structure extractor, and reports state-of-the-art results with improved boundary fidelity across four datasets.
Problem
Pooling-based dynamic-kernel generation collapses high-frequency spatial details, motivating a structure-sensitive alternative for fine-grained medical segmentation.
Method
SGDC uses an explicitly supervised SGE branch to provide multi-channel structural guidance for spatially aware kernels and gating signals, while jointly supervising segmentation and boundaries.
Results
SGD-Net achieves state-of-the-art performance on ISIC 2016, PH2, ISIC 2018, and CoNIC, with improved boundary fidelity measured by HD95.
Takeaways & Limitations
Pooling-free structural guidance mitigates over-smoothing and preserves fine-grained structural integrity in medical image segmentation.
Abstract
from arXiv · showhide
Spatially variant dynamic convolution provides a principled approach of integrating spatial adaptivity into deep neural networks. However, mainstream designs in medical segmentation commonly generate dynamic kernels through average pooling, which implicitly collapses high-frequency spatial details into a coarse, spatially-compressed representation, leading to over-smoothed predictions that degrade the fidelity of fine-grained clinical structures. To address this limitation, we propose a novel Structure-Guided Dynamic Convolution (SGDC) mechanism, which leverages an explicitly supervised structure-extraction branch to guide the generation of dynamic kernels and gating signals for structure-aware feature modulation. Specifically, the high-fidelity boundary information from this auxiliary branch is fused with semantic features to enable spatially-precise feature modulation. By replacing context aggregation with pixel-wise structural guidance, the proposed design effectively prevents the information loss introduced by average pooling. Experimental results show that SGDC achieves state-of-the-art performance on ISIC 2016, PH2, ISIC 2018, and CoNIC datasets, delivering superior boundary fidelity by reducing the Hausdorff Distance (HD95) by 2.05, and providing consistent IoU gains of 0.99\%-1.49\% over pooling-based baselines. Moreover, the mechanism exhibits strong potential for extension to other fine-grained, structure-sensitive vision tasks, such as small-object detection, offering a principled solution for preserving structural integrity in medical image analysis. To facilitate reproducibility and encourage further research, the implementation code for both our SGE and SGDC modules has been is publicly released at https://github.com/solstice0621/SGDC.
1. Introduction
Medical image segmentation must balance broad semantic context with high-resolution boundary preservation, but existing guidance and dynamic-convolution strategies can lose fine structural information. SGDC addresses this by using explicitly supervised structural cues to generate spatially precise modulation.
- Motivation: Medical segmentation faces a trade-off between expanding receptive fields for semantic understanding and preserving resolution for precise boundaries.Operations favoring semantic context can compromise the fidelity of intricate structures important for clinical diagnosis.
- Limitations of Existing Guidance: Compressing boundary features into single-channel attention creates an information bottleneck, while simple fusion can smooth high-frequency details.The paper argues that structural fidelity requires interactive modulation rather than simple aggregation.
- Dynamic Convolution: Spatially variant dynamic convolution generates location-specific kernels, but existing methods condition them on local or cross-stage semantic features.These approaches improve local structural modeling through spatial adaptivity.
- Conditioning Limitation: Backbone-derived semantic features are spatially smooth and dominated by low-frequency information, limiting their suitability for boundary refinement.The paper identifies a mismatch between semantically homogeneous conditioning signals and boundary-sensitive kernels.
- Proposed Solution: SGDC replaces pooled semantic conditioning with explicitly supervised structural guidance for generating spatially aware kernels and gating signals.The SGE branch produces multi-channel guidance maps from multi-scale features to supply high-frequency structural cues.
2. Related Works
Dynamic convolution has progressed from globally conditioned kernels to spatially variant designs, while explicit edge supervision supplies structural cues. Both lines retain limitations in localization or fusion, motivating more effective structural integration.
- Dynamic Convolution: Early sample-variant dynamic convolutions generate one kernel set per input image, typically using global average pooling for kernel generation.This global aggregation was effective for classification but provides limited spatial specificity.
- Spatially Variant Methods: Spatially variant methods address localization by conditioning kernels on local neighborhoods or individual pixel features through implicit self-guidance.A second stream uses guidance from other network stages, broadening the conditioning sources.
- Edge Supervision: Explicit edge-supervision methods learn boundary information, but single-channel compression and simple fusion can bottleneck or smooth structural details.The related-work discussion motivates richer mechanisms for transmitting high-frequency boundary cues.
3. Methodology
SGD-Net combines a hierarchical encoder–decoder with an explicitly supervised structure extractor and pooling-free SGDC blocks. Structural guidance drives dynamic kernels and complementary feature processing, while multi-output supervision jointly optimizes segmentation and boundaries.
- Overall Architecture: SGD-Net uses a hierarchical encoder–decoder with a pretrained Res2Net-50 backbone for multi-scale feature extraction.A Transformer encoder variant processes shallow features to capture long-range dependencies with sparse multi-scale sampling.
- Overall Architecture: The network applies deep supervision to segmentation predictions at three decoder scales, optimizing representations across network depths.Coarser auxiliary outputs provide intermediate supervision signals.
- Structure Guidance Extractor: The SGE produces a ground-truth-supervised single-channel edge map and multi-channel structural guidance, separating localization from richer feature guidance.Its outputs are generated from fused deep and shallow features through independent convolutional heads.
- Structure-Guided Dynamic Convolution: SGDC replaces pooling-driven refinement with a lightweight head that uses encoder features and SGE guidance to generate spatially varying kernels and two gates.The fused inputs enter a 1×1 convolution that produces Wdyn, g1, and g2 at each spatial location.
- Structure-Guided Dynamic Convolution: SGDC processes features through complementary dynamic and local branches, merges them residually, and supports detail refinement at fine scales and topology preservation at coarse scales.The dynamic branch uses unfolded local patches and Softmax-normalized spatial weights.
- Loss Function: The composite objective combines multi-scale segmentation supervision with directly supervised edge prediction using hybrid BCE–Dice segmentation loss and edge Dice loss.The structural term is weighted with λ = 3 in the experiments.
4. Experiments and Results
SGD-Net achieves strong segmentation performance across skin-lesion and nuclei datasets, while ablations attribute gains to structure-guided, pooling-free feature modulation and explicitly supervised structural priors.
- State-of-the-art comparisons: SGD-Net achieves state-of-the-art skin-lesion segmentation performance across the evaluated datasets and protocols.The evaluation covers ISIC 2016→PH2 generalization and ISIC 2018 experiments.
- State-of-the-art comparisons: 92.93% Dice and 87.29% IoU on the 2016→PH2 test outperform the best previous method by 0.37% and 0.69%, respectively.On ISIC 2018, the method achieves 91.41% Dice and 84.96% IoU, exceeding several competing architectures.
- State-of-the-art comparisons: 81.61% Dice, 69.46% IoU, and 68.79% PQ rank SGD-Net first across all three CoNIC metrics.These results are obtained with comparable FLOPs and parameter counts.
- Qualitative analysis: Qualitative results show more complete lesion delineation and better-preserved nuclei topology than the baseline, alongside sharper structural guidance maps.The reported visual difference is associated with high-contrast structural responses from SGE and reduced over-smoothing in SGDC.
- Ablation study: The Res2Net + ViT baseline has HD95 32.63, while replacing simple addition with Contmix yields HD95 24.21 in the fusion ablation.The ablation argues that pooling-based aggregation can smooth high-frequency structural details, whereas SGDC avoids pooling through dual-branch fusion.
- Ablation study: Removing explicit boundary guidance still gives Dice 90.37%, while λ = 3 is reported as the optimal structural-loss weight.Fixed operators outperform learnable convolutions, and Sobel is selected for its clean single-response gradient despite Laplacian achieving Dice 92.11%.
5. Conclusion
SGD-Net addresses structural detail loss from average-pooled kernel generation with an average-pooling-free SGDC module guided by auxiliary structural priors. The design improves boundary fidelity and attributes gains to the interaction of deterministic local refinement with adaptive dynamic modulation.
- SGDC removes average pooling from feature modulation to preserve high-frequency structural information.
- SGDC uses structural priors from the auxiliary SGE branch to generate spatially precise kernels that adapt to local structure without sacrificing detail.
- Performance gains arise from the synergistic interaction between deterministic local refinement and adaptive dynamic modulation, with potential beyond medical segmentation.