PaperlayerView exact revision on arXiv

Source-linked AI summary

DualMiT-Net: Local-Global Transformer-Convolutional Fusion for Breast Mass Segmentation in Mammographic Regions of Interest

Alibek Kamiluly, Milana Muratova, Yash Patel, Fan Li

arXiv:2608.15019v1cs.CVcs.AIcs.LG

TL;DR

Breast mass segmentation is difficult because low contrast, noise, overlapping structures, and variable boundaries complicate precise lesion delineation. DualMiT-Net combines focused and wider mammographic views, achieving Dice 0.9375 and IoU 0.8834 while outperforming six standard baselines.

  • Problem

    Breast mass segmentation remains difficult because low contrast, noise, overlapping structures, and reader-varying contours obscure lesion boundaries.

  • Method

    DualMiT-Net fuses a lesion-centered MiT-B5 branch with a wider-context EfficientNet-B5 branch and texture-aware mammographic inputs.

  • Results

    Dice coefficient 0.9375 ± 0.0011 and IoU 0.8834 ± 0.0020 were achieved across three runs, outperforming six standard segmentation baselines.

  • Takeaways & Limitations

    Combining local lesion information with wider breast context provided accurate and consistent mass segmentation under the evaluated CBIS-DDSM protocol.

  • Takeaways & Limitations

    Evaluation was lesion-centered on CBIS-DDSM, so the results do not establish end-to-end full-mammogram performance or generalization across imaging systems.

Abstract

from arXiv · show

Breast mass segmentation is an important step in computer-aided mammography, but it remains difficult because masses can have low contrast, irregular shapes, and boundaries that blend with surrounding breast tissue. To address this problem, we present DualMiT-Net, a dual-branch network that uses both a focused view of the mass and a wider view of the surrounding tissue. The local branch uses a Mix Transformer (MiT-B5) encoder to learn mass shape, texture, and boundary information, while the global branch uses an EfficientNet-B5 encoder to learn surrounding breast context. Features from the two branches are shared at the deeper encoder levels and are then progressively fused in a single decoder. A spatial gate controls how much global information is added during decoding. We also evaluated four input representations and selected a percentile-windowed mammogram combined with a Gabor texture response. The model was trained and evaluated on the mass subset of the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM) using a patient-level split. Across three training runs, DualMiT-Net with exponential moving average weights achieved a mean Dice coefficient of 0.9375 and a mean Intersection over Union of 0.8834. It also achieved better Dice and IoU scores than six standard encoder-decoder baselines trained using the same data and training settings. These results show that combining local mass information with wider breast context can provide accurate and consistent breast mass segmentation.

1. Introduction

Breast mass segmentation is difficult because mammograms often have low contrast, noise, overlapping structures, and reader-dependent contours. DualMiT-Net addresses these challenges with dual local-global views, deep cross-branch fusion, spatially gated decoding, and a texture-aware input representation, achieving strong matched-protocol performance.

  • Motivation: Mammographic masses are difficult to segment because low contrast, noise, overlapping structures, and reader variability obscure reliable boundaries.Reading time constraints further motivate automated assistance in mammographic interpretation.
  • Proposed approach: DualMiT-Net uses a lesion-centered ROI and a wider contextual view to learn mass details and surrounding breast-tissue information.The local branch uses MiT-B5, while the global branch uses EfficientNet-B5.
  • Proposed approach: Deep feature sharing and spatially gated decoding selectively combine local and global features during reconstruction.The architecture exchanges information at deep encoder levels and controls contextual incorporation during decoding.
  • Input and evaluation: A percentile-windowed Gabor representation is evaluated with DualMiT-Net for texture-aware mass segmentation.The representation combines percentile-windowed mammographic information with a Gabor texture response.
  • Input and evaluation: 0.9375 Dice and 0.8834 IoU were achieved while outperforming six standard segmentation baselines under the same experimental protocol.These results are reported for DualMiT-Net with the evaluated texture-aware representation.

2. Related Work

Prior breast mass segmentation methods build on CNN encoder-decoder designs, attention and multi-scale processing, transformer-based global modeling, or complementary mammographic views. DualMiT-Net extends these ideas by using aligned local and global fields of view from the same lesion with separate encoders and gated progressive fusion.

  • CNN-Based Methods: CNN-based encoder-decoder methods, especially U-Net, remain a common foundation for breast mass segmentation and address low contrast, irregular boundaries, and variable mass size and shape.U-Net combines high-level features with spatial information from earlier layers.
  • Attention and Multi-Scale Methods: Attention, multi-scale, adversarial, multi-task, and connected residual approaches target feature selection, varied mass sizes and shapes, or segmentation refinement on CBIS-DDSM.Examples include AUNet, AM-MSP-cGAN, MTLNet, ARF-Net, and Connected-ResUNets.
  • Transformer-Based Methods: Transformer-based methods use self-attention or hybrid designs to capture relationships between distant regions while retaining spatial detail through convolutional or U-Net-like components.SegFormer uses a Mix Transformer encoder to produce multi-scale features with efficient self-attention.
  • Mammographic Context: Mammography-specific transformer and multi-view methods incorporate local-global transformer features or complementary cranio-caudal and mediolateral oblique views for segmentation.YOLO-LOGO and Ma and Peng’s cross-view variational autoencoder were evaluated in mammographic settings including CBIS-DDSM.
  • DualMiT-Net: DualMiT-Net differs by processing two aligned fields of view from the same lesion with separate encoders, deeper feature exchange, progressive single-decoder fusion, and a spatial gate.The local view captures mass shape, texture, and boundary structure, whereas the wider view captures surrounding breast tissue.

3. Materials and Methods

The framework consists of preprocessing followed by dual-branch segmentation using two aligned mammogram inputs: a lesion-centered local view and a wider global view of surrounding breast tissue.

  • Framework overview: The pipeline has two stages: preprocessing and dual-branch segmentation.The proposed framework is illustrated in Fig. 1.
  • Input preparation: Each mammogram is prepared as two aligned inputs, with the local input centered on the lesion and the global input covering surrounding breast tissue.Both inputs are passed to the segmentation network.

3.1. Dataset

The study used the CBIS-DDSM mass subset, retaining cases with valid mammogram–mask correspondences while excluding calcifications and unmatched masks. Data were divided by patient into training, validation, and test subsets, with examples spanning varied breast and lesion characteristics.

  • Dataset composition: CBIS-DDSM provided digitized mammograms with expert binary lesion masks for the experiments.Only mass cases with valid image-mask correspondence were retained; calcification cases and unmatched masks were excluded.
  • Dataset splitting: Patient-level splitting kept all images from each patient within a single subset to prevent related views from crossing dataset partitions.This avoided overly optimistic performance estimates caused by having views from the same patient in both training and test sets.
  • Dataset splitting: 753 image-mask pairs from 480 patients formed training, 91 pairs from 62 patients formed validation, and 92 pairs from 64 patients formed test.Image and mask counts matched within every subset, while some patients contributed multiple mammographic views.
  • Dataset characteristics: Representative cases showed variation in breast appearance, lesion size, lesion location, and mass shape across the dataset.The examples included mammograms with corresponding expert mass masks.

3.2. Preprocessing

Preprocessing creates aligned lesion-centered local and global views with a shared target mask, using percentile normalization and texture-enhanced three-channel inputs. Lesion geometry is standardized before training, but segmentation operates on a known lesion region defined by expert annotations.

  • Aligned lesion views: Each lesion yields a 256 × 256 local crop preserving mass boundaries, a 512 × 512 wider crop retaining surrounding breast tissue, and an aligned target mask.Both views are centered using the lesion position from the expert annotation mask.
  • Intensity normalization: Percentile-based clipping followed by rescaling to [0, 1] produces the normalized mammogram, accommodating intensity variation across digitized screen-film images.Fixed gray-level clipping was avoided because mammogram intensities vary between images.
  • Input representation: The selected three-channel input combines the percentile-windowed mammogram with a normalized Gabor texture response, while repeating the mammogram in the second channel.The Gabor response captures oriented texture patterns relevant to spiculation and mass-margin description, while preserving compatibility with pretrained encoders.
  • Lesion-centered setup: Because expert annotations provide lesion position and extent, the network segments a known lesion region rather than locating lesions in the full mammogram.The paper discusses the interpretive effect of this lesion-centered setup separately.
  • Lesion normalization: PCA-based lesion normalization estimates center, orientation, and long-axis length, then applies the same rotation-and-scaling transform to image and mask.The scale factor is s = Lt/L, enlarging lesions below the target length and reducing larger lesions.

3.3. Network Architecture

DualMiT-Net processes local lesion and global breast-tissue views through separate encoders, shares deep features, and progressively fuses them in a decoder. A spatially gated residual mechanism adds global context while preserving the local segmentation pathway.

  • Dual-branch encoders: DualMiT-Net uses a 256×256 local lesion view with MiT-B5 and a 512×512 global tissue view with EfficientNet-B5, each producing four feature scales.The local branch learns lesion shape, texture, and boundaries, whereas the global branch captures surrounding breast context.
  • Deep feature sharing: The branches share features bidirectionally at their two deepest encoder levels, including global-to-local sharing before decoding.Sharing is restricted to deeper, lower-resolution levels to ease alignment of the two feature streams.
  • Progressive fusion: At each decoder stage, local encoder features remain the main pathway while resampled global features provide additional context alongside preceding decoder features.The decoder reconstructs the mask from the deepest to the highest-resolution feature level.
  • Spatially gated context: The spatial gate controls global-feature contribution independently at each image position, and its linear positional cost avoids a full pairwise attention matrix.A learnable scalar also controls the amount of gated global information added at each decoder level while retaining the local representation through residual fusion.
  • Output head: After contextual fusion, scSE recalibrates decoder channels and spatial locations before a 1 × 1 convolution and sigmoid produce pixel-wise mass probabilities.The final binary mask uses a fixed threshold of τ = 0.50.

3.4. Loss Function

DualMiT-Net uses a weighted combination of BCE and soft Dice loss to provide pixel-level supervision while directly optimizing mask overlap. The weighting favors Dice loss, while BCE stabilizes early optimization through dense pixel-wise gradients.

  • Loss Function: The training objective combines Binary Cross-Entropy (BCE) loss with soft Dice loss, providing pixel-wise supervision and directly rewarding predicted–ground-truth mask overlap.BCE supervises individual pixels, whereas soft Dice evaluates regional overlap using a differentiable prediction map.
  • Loss Function: λBCE = 0.3 and λDice = 0.7 weight the objective toward Dice loss.The Dice term contributes 70% of the weighted objective.
  • Loss Function: The BCE term supplies dense pixel-wise gradients from the first epoch, helping stabilize early optimization.Its independent per-pixel evaluation provides supervision throughout the image.
  • Loss Function: The soft Dice term evaluates the predicted mask as a whole and is insensitive to the proportion of image occupied by background.Its weighting is favored because regional overlap is the primary assessment criterion.

3.5. Evaluation Metrics

Evaluation used complementary overlap, error-type, and boundary-distance metrics on thresholded binary masks. Dice and IoU were primary overlap measures, while HD95 assessed boundary accuracy and all metrics were averaged equally across 92 test images.

  • Metric families: Segmentation quality was assessed with regional-overlap metrics and boundary-distance metrics using TP, FP, FN, and TN definitions.The evaluation separates mask agreement from predicted-versus-ground-truth boundary distance.
  • Overlap metrics: Dice and IoU quantify mask overlap, with IoU producing a lower numerical score than Dice for the same prediction.Dice equals one for exact agreement and zero when masks do not intersect; IoU is treated as stricter because it scores lower for equivalent overlap.
  • Error characterization: Precision, recall, specificity, and pixel accuracy characterize over-segmentation, under-segmentation, background labeling, and overall pixel correctness.Precision and recall are interpreted together, while high pixel accuracy may be misleading because background occupies much of the lesion-centered patch.
  • Boundary metric: HD95 was additionally reported because overlap metrics emphasize mass interiors and are comparatively insensitive to clinically important margin accuracy.HD95 summarizes the 95th percentile of distances between predicted and ground-truth mask boundaries.
  • Evaluation protocol: All models used a fixed threshold of τ = 0.50, and metrics were computed per image then averaged across 92 test images with equal case contribution.The threshold was not tuned on test data because threshold tuning trades precision against recall and would confound comparisons.

3.6. Experimental Protocol

The experimental protocol used standardized AdamW training with learning-rate scheduling, gradient clipping, and early stopping, while evaluating both base and EMA weights. Each configuration was repeated across three seeds, with results summarized across runs and supplemented by bootstrap confidence intervals over test cases.

  • Training protocol: All models trained for up to 150 epochs with batch size four using AdamW, learning rate 5 × 10−5, weight decay 1×10−4, and gradient clipping at 1.0.The learning rate was halved when validation Dice stopped improving, and training ended after 20 consecutive unimproved epochs.
  • EMA evaluation: EMA weights used decay factor β = 0.999 and were evaluated alongside directly optimized base weights.The EMA parameters followed a smoothed trajectory, while both parameter sets were reported together.
  • Reproducibility: Every configuration was trained three times with seeds 42, 43, and 44, with results reported as the mean and standard deviation across runs.Repeating experiments helped distinguish model effects from variation caused by random initialization.
  • Uncertainty analysis: Bootstrap confidence intervals resampled the 92 test images with replacement 10,000 times and used the 2.5th and 97.5th percentiles as interval limits.The intervals characterized variation across test cases, while variation between training runs was reported separately.

4. Results

DualMiT-Net achieved strong segmentation performance, with the window-and-Gabor representation producing the best input results and EMA weights yielding the strongest overall overlap. Ablations and qualitative analyses indicate that performance remains stable, while errors mainly occur near ambiguous lesion boundaries.

  • Overall segmentation performance: 0.9375±0.0011 Dice and 0.8834±0.0020 IoU were achieved by full DualMiT-Net with EMA weights across three training seeds.The full EMA model produced the highest overlap scores among the complete model and two ablation variants.
  • Input representation: 0.9376 Dice and 0.8836 IoU were obtained with windowing plus Gabor filtering and EMA weights, exceeding windowing alone at 0.9363 Dice and 0.8815 IoU.The window-and-Gabor representation also achieved the lowest base-model HD95 of 6.70 pixels.
  • Ablation study: 0.9375 Dice and 0.8834 IoU were achieved by the complete EMA model, compared with 0.9372 without scSE and 0.9371 without G2L sharing.Removing either refinement component caused only small changes while retaining the dual-view encoders and decoder.
  • Ablation study: None of the six paired ablation comparisons reached statistical significance at p < 0.05, with mean differences of 0.0003–0.0006 for Dice and IoU.Per-image distributions were closely grouped, although HD95 varied more for difficult cases.
  • Comparison with baselines: 0.0350 Dice and 0.0594 IoU were the absolute improvements of DualMiT-Net with EMA weights over U-Net with ResNet-50, the strongest baseline.DualMiT-Net achieved 0.9375 Dice and 0.8834 IoU, versus 0.9025 Dice and 0.8240 IoU for the baseline.
  • Comparison with prior methods: 0.9375 Dice and 0.8834 IoU were the highest values among the compared previous methods, exceeding Cross-view VAE at 0.9246 Dice and 0.8720 IoU.Qualitative examples showed close boundary adherence overall, while low contrast and unclear boundaries produced wider errors near lesion margins.

5. Discussion

DualMiT-Net’s performance is attributed primarily to its complete local-global architecture, with refinement modules and the selected windowing-plus-Gabor representation further supporting feature exchange and segmentation quality. Its lesion-centered evaluation remains limited by dependence on known approximate mass locations and difficulties in low-contrast or unclear-boundary cases.

  • Architecture: DualMiT-Net’s dual-view design combines lesion detail with surrounding-breast context through separate encoders and decoder fusion.This architecture produced a clear improvement over the standard segmentation baselines.
  • Performance: 0.9375 Dice coefficient and 0.8834 IoU were achieved by DualMiT-Net with EMA weights.The complete model was retained as the final configuration because it achieved the strongest overall overlap performance.
  • Ablation: Removing scSE or G2L sharing caused only small changes, while the complete local-global architecture provided the core segmentation capability.G2L sharing refines local representations with global context, and scSE recalibrates fused decoder features spatially and channel-wise.
  • Ablation: 0.9282 precision and 0.9482 recall were achieved with EMA weights in the full model.Without scSE, recall increased to 0.9522, while precision decreased to 0.9238 and specificity to 0.8953, indicating greater surrounding-tissue inclusion.
  • Limitations and future work: The evaluation is lesion-centered because local ROIs and orientation-scale normalization use expert annotations, so results do not represent end-to-end full-mammogram detection.Future work includes automatic lesion localization, independent mammography datasets, and improved boundary segmentation for difficult cases.

6. Conclusion

DualMiT-Net is a dual-branch encoder-decoder for breast mass segmentation in lesion-centered mammographic ROIs. It combines local lesion features, wider contextual features, and Gabor-enhanced percentile-windowed mammograms through shared deep features and progressive decoder fusion.

  • Model architecture: DualMiT-Net uses a MiT-B5 encoder for a local lesion view and an EfficientNet-B5 encoder for a wider contextual view.The model is designed for breast mass segmentation in lesion-centered mammographic ROIs.
  • Model architecture: The two branches share features at their deepest encoder levels before progressively fusing them in a single decoder.This architecture integrates local and contextual representations during decoding.
  • Input representation: The selected input representation combines percentile-windowed mammograms with Gabor texture information.This representation was selected for the segmentation model.
Loading 2608.15019v1…