PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis

Jonghun Kim, Junmo Kwon, Hyunjin Park

arXiv:2610.21703v1cs.CV

TL;DR

CE-MRI can clarify tumor boundaries but requires contrast agents that raise safety concerns and add scan time and cost. The paper synthesizes CE-MRI from non-contrast sequences with MoFD, which adaptively combines sequence features and their differences. On UCSF-PDGM and UPenn-GBM, the method outperformed baselines in image fidelity and improved downstream tumor segmentation performance.

  • Problem

    Non-contrast MRI can leave tumor boundaries ambiguous, while CE-MRI requires contrast agents that raise safety concerns and increase acquisition time and cost.

    Source: §1 · p. 2

  • Method

    MoFD synthesizes CE-MRI by dynamically selecting and integrating sequence-specific features and their explicit differences.

    Source: §1 · p. 3

  • Results

    On UCSF-PDGM and UPenn-GBM, the method consistently outperformed baselines in image fidelity and improved downstream tumor segmentation performance.

    Source: §6 · p. 14

  • Takeaways & Limitations

    The synthesized images support improved downstream tumor segmentation performance in the evaluated datasets.

    Source: §6 · p. 14

    Adversarial learning can improve perceptual quality but may produce details or artifacts absent from ground truth, compromising pixel-level fidelity.

    Source: §3.3 · p. 7

Abstract

from arXiv · show

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for accurately diagnosing and segmenting brain tumors. However, the acquisition of CE-MRI requires injecting contrast agents, which poses potential health risks and increases scan costs. Here, we propose a CE-MRI synthesis method from non-contrast sequences, reducing both risk and cost. Unlike existing methods that rely on channel-wise concatenation to fuse multi-sequence information for synthesis, we introduce a Mixture of Feature Difference (MoFD) module. The proposed module uses a dynamic weighting strategy to adaptively select and aggregate informative features from both individual source sequences and explicit differences among them. Quantitative and qualitative evaluations demonstrate that our proposed method outperforms baselines in terms of image fidelity and perceptual quality on the UCSF-PDGM and UPenn-GBM datasets. Our method successfully reconstructs tumor regions and achieves segmentation performance comparable to that obtained with real CE-MRI. Our ablation studies show that although adversarial learning improves perceptual quality, a regression-based variant of our method achieves higher structural fidelity.

1 Introduction

CE-MRI can clarify tumor boundaries that are ambiguous on non-contrast scans, but contrast agents raise safety concerns and increase scan time and cost. The paper proposes MoFD to synthesize CE-MRI by combining sequence-specific features with their differences.

1 Introduction

  • Contrast agents make lesions hyperintense on T1-weighted images, helping identify their location and extent when non-contrast scans leave tumor boundaries ambiguous.Their administration also raises safety concerns, and CE-MRI requires longer acquisition times and higher costs.Source: §1 · pp. 1–2
  • Channel-wise concatenation can overlook distinct sequence contrast characteristics, while explicit feature differences can represent inter-sequence contrast changes for synthesis.T1, T2, and T2-FLAIR differ in how they depict water, lesions, fat, and other tissues.Source: §1 · pp. 2–3
  • MoFD extracts sequence-specific features, computes differences between them, and dynamically weights both kinds of features for selective integration.The paper evaluates regression-based and GAN-based approaches and assesses tumor segmentation on synthesized images using nnU-Net.Source: §1 · p. 3

2 Related Work

Prior CE-MRI synthesis work uses varied translation and fusion approaches, while many methods concatenate modalities and adversarial training can trade fidelity for perceptual quality. The paper builds on feature-difference modeling and dynamic selection to fuse multi-sequence MRI.

2.1 Medical Image Translation for CE-MRI Synthesis

  • Prior CE-MRI synthesis includes concatenation-based methods and GAN-based approaches, with adversarial learning often improving perceptual quality at a cost to fidelity.Examples include full-dose T1ce synthesis from non-contrast or low-dose images and regression-based synthesis from T1, T2, and ADC.Source: §2.1 · pp. 3–4

2.2 Feature Fusion and Difference Modeling

  • Earlier fusion work includes hybrid, task-specific, and difference-weighting strategies; MoFD computes cross-modality feature differences and uses gating to select informative features.The cited difference weighting block uses feature-map differences between baseline and follow-up scans to generate an attention map for changed regions.Source: §2.2 · p. 4
  • The proposed synthesis pipeline uses weight-sharing encoders for T1, T2, and T2-FLAIR, then dynamically weights and aggregates their features and differences before decoding CE-MRI.A softmax function assigns the adaptive weights.Source: Figure 2 · p. 5

3 Method

The method processes each MRI sequence with a shared encoder, then combines sequence-specific features and pairwise differences for CE-MRI synthesis. Structural alignment regularizes anatomy across sequences, while reconstruction and optional adversarial objectives balance fidelity and perceptual realism.

3 Method

  • A shared encoder maps each sequence into a common feature space, preserving modality-specific features while keeping the architecture parameter-efficient.The decoder synthesizes CE-MRI from the encoded sequences.Source: §3 · p. 5

3.1 Mixture of Feature Difference

  • At each scale, MoFD combines individual sequence features with pairwise feature differences and adaptively weights them before passing the aggregate to the decoder.Global average pooling followed by softmax produces the attention weights.Source: §3.1 · pp. 5–6

3.2 Structural Alignment

  • Structural alignment compares sequence features using SSIM's structure component, excluding luminance and contrast terms to preserve anatomical consistency despite intensity variation.The loss is applied to all pairwise sequence-feature combinations and is intended to help preserve patient-specific structure during synthesis.Source: §3.2 · pp. 6–7

3.3 Training Strategy

  • The training objective combines reconstruction and structural-alignment losses; an optional PatchGAN adversarial loss can promote realistic textures but may compromise pixel-level fidelity.The reconstruction loss encourages the prediction to match the ground truth, while the alignment objective regularizes structural consistency.Source: §3.3 · pp. 7–8, Figure 3 · p. 7

4 Experiments

Experiments use two public multi-sequence glioblastoma MRI datasets, a 3D U-Net-based implementation, and image-quality and segmentation metrics. Comparisons span 2D and 3D synthesis models.

4.1 Datasets.

  • UCSF-PDGM and UPenn-GBM each provide T1, T2, T2-FLAIR, and T1ce images, tumor masks, and separate training, validation, and test splits.The UCSF-PDGM split is 210/23/54 subjects; UPenn-GBM is 210/21/52.Source: §4.1 · p. 8

4.2 Implementation details.

  • The model uses a 3D U-Net backbone with MoFD modules in skip connections at all stages; training runs for 500 epochs with batch size 1.Inputs are cropped to 192×192×128; the learning rate is 1e-4.Source: §4.2 · p. 8

4.3 Metrics.

  • Image synthesis is assessed with 3D PSNR and SSIM, perceptual quality with LPIPS, and tumor segmentation with nnU-Net Dice scores for WT, ET, and TC.Source: §4.3 · p. 8

4.4 Comparison methods.

  • Comparisons span 3D and 2D models; Table 1 reports quantitative T1ce performance, while Fig. 4 shows tumor-region synthesis and nnU-Net segmentations, with GAN improving perceptual quality over blurry GAN-free results.Source: §4.4 · p. 8, Figure 4 · p. 9, Table 1 · p. 9

5 Results

Across UCSF-PDGM and UPenn-GBM, the proposed method performs strongly against comparison methods, with regression-based training favoring fidelity and adversarial training favoring perceptual quality. Ablations and feature visualizations support dynamic, multi-sequence feature integration and show that selection varies by network scale.

5.1 Results on the Comparison Methods

  • MoFD w/o GAN achieves the best PSNR and SSIM on UCSF-PDGM, while MoFD w/ GAN has the best LPIPS and second-best PSNR and SSIM; the proposed model also has the best Dice scores for WT, ET, TC, and their average.A similar trend is reported on UPENN-GBM.Source: §5.1 · pp. 9–11, Table 2 · p. 10

5.2 Multi-Sequence Combinations

  • Using T1, T2, and T2-FLAIR together achieves the highest fidelity, perceptual quality, and downstream segmentation scores, though tumor-core segmentation remains limited.T1-only input generally improves image fidelity but gives poor segmentation because T1 provides insufficient tumor contrast information.Source: §5.2 · p. 11, Table 3 · p. 11

5.3 Ablation Study

  • Removing the convolution block leaves feature scales inconsistent without normalization, while weighted sums significantly outperform averaging features.The comparison evaluates MoFD architecture variants.Source: §5.3 · pp. 11–12, Table 4 · p. 12
  • Excluding Lgan improves fidelity, while including it improves perceptual quality but reduces fidelity; Lalign improves both, and regression-based training yields better tumor segmentation than GAN-based training.The authors note that adversarial training risks adding realistic but false details that deviate from ground truth.Source: §5.3 · p. 12, Table 5 · p. 13
  • Selecting all six sequence and difference features performs better across both datasets than selecting a partial subset.The six features comprise three sequence features and three difference maps.Source: §5.3 · p. 12, Table 6 · p. 13

5.4 Analysis of Feature Selection

  • The weighted feature sum resembles the target tumor region; selection is relatively balanced at low scales and increasingly favors T2 and T2-FLAIR at deeper scales.The authors emphasize that low-level features remain important for voxel-level accuracy and fine-grained synthesis.Source: §5.4 · pp. 13–14, Figure 8 · p. 13

6 Conclusion

The conclusion presents MoFD as a way to synthesize T1ce from non-contrast multi-sequence MRI by combining sequence-specific features with pairwise feature differences. Experiments and ablations support its performance and the value of dynamic weighting, with regression-based training better preserving structural fidelity when accurate tumor representation is needed.

6 Conclusion

  • MoFD models sequence-specific features and pairwise feature differences, then adaptively selects complementary information for T1ce synthesis.This replaces reliance on simple channel-wise concatenation.Source: §6 · p. 14
  • Experiments on UCSF-PDGM and UPenn-GBM show that the method outperforms baselines in image fidelity and improves downstream tumor segmentation.Source: §6 · p. 14
  • Ablations indicate that normalization, dynamic weighting, all input sequences, and their difference features benefit synthesis; regression-based training better preserves structural fidelity when accurate tumor representation is required.Source: §6 · p. 14

Related papers

Loading 2610.21703v1…