On the Role of MRI Sequences in Cross-Dataset Generalization for Brain Tumor Segmentation

Henrique Zan Grande; João G. Pitol; Lucas B. Schuck; Rafael V. Serenato; Rayson Laroca; Andre Gustavo Hochuli

Source-linked AI summary

On the Role of MRI Sequences in Cross-Dataset Generalization for Brain Tumor Segmentation

Henrique Zan Grande, João G. Pitol, Lucas B. Schuck, Rafael V. Serenato, Rayson Laroca, Andre Gustavo Hochuli

arXiv:2608.29944v1Aug 30, 2026cs.CV

TL;DR

Cross-dataset brain tumor segmentation remains difficult under domain shift and limited annotations. The study independently evaluates MRI modalities with a controlled ResUNet-based protocol, then tests limited target-domain fine-tuning. T2f/FLAIR provides the strongest cross-dataset performance, while 10–20% target data already approaches upper-bound performance.

Problem
Cross-dataset brain tumor segmentation is challenged by domain shift and limited annotated medical data, motivating study of modality-specific robustness and annotation-efficient adaptation.
Method
The study uses a standardized 2D ResUNet pipeline to train each MRI modality independently, evaluate controlled transfer settings, and fine-tune the best modality with limited target-domain data.
Results
T2f/FLAIR achieves the strongest cross-dataset performance, while fine-tuning with 10–20% of target-domain data already approaches upper-bound performance.
Takeaways & Limitations
Modality choice, data diversity, and lesion characteristics influence generalization, while limited target-domain adaptation can substantially improve performance with reduced annotation and computational costs.

Abstract

from arXiv · show

Brain tumor segmentation in magnetic resonance imaging (MRI) is a critical task for diagnosis and treatment planning. Despite the success of deep learning architectures such as U-Net and its variants, performance degradation across datasets remains a major challenge, particularly under domain shift and limited annotated data. To address this issue, this study systematically evaluates how individual MRI sequences influence model robustness across two well-known datasets. A ResUNet-based framework is employed, where each modality is trained independently to isolate its effect under a controlled cross-dataset evaluation protocol with tumor size stratification, without target-domain training, or with limited domain adaptation. Results show that the T2f/FLAIR sequence achieves the best cross-dataset performance, with Dice scores exceeding 75%. It consistently outperforms other modalities across most tumor size ranges, while multi-sequence training further improves performance. Additionally, even limited target-domain adaptation yields rapid initial gains, reducing the need for extensive annotations and costly retraining. Our source code is publicly available at https://github.com/henrique-zan/brain_tumor_segmentation/.

I. INTRODUCTION

Brain tumor segmentation is clinically important, but domain shift and limited annotations can degrade cross-dataset performance. This study isolates MRI-sequence effects and evaluates modality robustness, generalization gaps, and data-efficient adaptation using a controlled ResUNet pipeline.

Brain tumor segmentation supports diagnosis, surgical planning, and clinical follow-up in medical imaging.
Domain shifts from acquisition protocols, scanners, patient populations, and image characteristics can substantially degrade performance on unseen datasets.
Fewer studies have isolated each MRI sequence under cross-dataset domain shift, despite the potential to identify more transferable representations.
The study asks how modality affects cross-domain degradation, which sequences are most robust, and how target-domain annotation effort relates to improvement.
Each modality is trained and evaluated independently with a standardized 2D ResUNet pipeline, followed by fine-tuning of the best-performing modality using limited target-domain data.
The paper contributes controlled modality-specific robustness evaluation, quantitative generalization-gap analysis, robust-sequence identification, and annotation-budget assessment.

II. LITERATURE REVIEW

Brain tumor segmentation research is dominated by U-Net-based convolutional methods and multimodal fusion, but cross-dataset robustness remains insufficiently addressed. This study targets the unquantified modality-specific contribution under domain shift and limited annotation budgets.

U-Net and its variants dominate MRI brain tumor segmentation, while prior work emphasizes architectural improvements and multimodal fusion.
U-Net combines contextual representation with spatial detail through its encoder-decoder structure and skip connections.
ResUNet variants improve optimization or feature extraction through residual blocks, attention mechanisms, and automated configuration.
Combining T1, T1c, T2, and T2f/FLAIR can improve in-domain segmentation accuracy and robustness over single-sequence inputs.
Individual MRI-sequence contributions to cross-domain robustness remain insufficiently quantified, especially under limited annotation budgets.

III. PROBLEM STATEMENT

The problem is to measure segmentation loss when a single-sequence model transfers between BraTS cohorts, identify more robust sequences, and assess how small labeled target subsets compensate for domain shift. A controlled evaluation uses identical modeling and preprocessing conditions across modalities.

Cross-dataset transfer can lose segmentation performance, motivating comparisons of modality robustness and annotation-efficient compensation.
Every modality uses the same 2D ResUNet architecture, data preparation, and optimization settings for controlled comparison.
The comparison includes in-domain evaluation, direct cross-dataset transfer, joint multi-dataset training, and incremental target-domain fine-tuning.
The experimental design applies uniform partitioning, preprocessing, architecture, training configuration, and evaluation procedures.

A. Datasets

BraTS 2020 and BraTS 2024 provide complementary cohorts for testing robustness under realistic dataset differences. Patient-level partitioning, uniform resizing, tumor-containing slice selection, and conservative augmentation standardize preparation.

BraTS 2020 contains 369 multimodal MRI studies, whereas BraTS 2024 includes approximately 1,350 cases in a larger, more heterogeneous cohort.
Figure 1 presents representative BraTS 2020 cases above BraTS 2024 cases across four MRI sequences and their binary masks.
The datasets differ in patient characteristics, tumor morphology, scanners, and acquisition protocols, creating realistic domain shift.
Patient-level splits assign 70% of patients to training, 15% to validation, and 15% to testing, preventing slice-level leakage across subsets.
Images are resized to 240 × 240 pixels, and 2D slices containing tumor tissue are extracted for each modality.
Training augmentation uses 90◦ rotations and horizontal and vertical flips, excluding aggressive intensity and geometric transformations.

B. Architecture

The study uses a 2D ResUNet with a pretrained ResNet-34 encoder, residual learning, skip connections, and a decoder that produces binary segmentation maps from single-channel MRI slices.

Architecture: The 2D ResUNet uses a pretrained ResNet-34 encoder with residual mappings and skip connections for hierarchical, multi-scale feature extraction.Residual identity mappings support gradient propagation, while skip connections preserve fine-grained spatial information for decoding.
Architecture: The decoder has four upsampling stages that progressively refine concatenated encoder features through transposed convolutions and convolutional blocks.Its channel progression is 512 →256 →128 →64 →64.
Architecture: The network receives a single-channel MRI slice and outputs a binary segmentation map through a final 1 × 1 convolution.Dropout with p = 0.2 is used within convolutional blocks to reduce overfitting in limited-data and cross-dataset settings.
Architecture: Each modality is trained independently with a combined Dice-loss and binary-crossentropy objective under early stopping and adaptive learning-rate reduction.Training used up to 80 epochs, batch size 32, and an initial learning rate of 1×10−3.

IV. EXPERIMENTS AND DISCUSSION

Experiments compare segmentation across BraTS 2020 and BraTS 2024 using multiple training and evaluation protocols, overlap metrics, target-domain fine-tuning, and tumor-size stratification.

Experimental Design: The experiments use BraTS 2020 and BraTS 2024 to compare in-domain, cross-dataset, and multi-domain evaluation protocols.Fine-tuning with increasing fractions of labeled target-domain data addresses the domain-adaptation question.
Reproducibility: The source code, trained weights, models, and experimental artifacts are publicly available for reproducibility.The repository is linked at github.com/henrique-zan/brain_tumor_segmentation/.
Tumor-Size Analysis: Tumor-size quartiles and representative cases are analyzed because Dice is more sensitive to small absolute errors in small lesions.This stratification examines whether modality performance varies with tumor size.
Evaluation Metrics: All comparisons report Dice and Intersection over Union, averaged over tumor-containing test slices.Both metrics quantify agreement between predicted and reference tumor masks.

A. Generalization Evaluation (RQ1/RQ2)

The generalization evaluation progressively increases distributional discrepancy across in-domain, direct cross-dataset, and joint multi-domain protocols. T2F/FLAIR shows the strongest cross-domain robustness, while multi-domain training reduces degradation.

Protocol Design: The evaluation protocols progressively increase distributional discrepancy to assess modality-specific generalization behavior.This design supports controlled comparison of robustness under changing train-test distributions.
Baseline: In-domain training and testing provide an upper-bound reference without domain shift.Training, validation, and test data share the same underlying distribution.
Cross-Dataset: Direct cross-dataset evaluation trains on one dataset and tests on the other without adaptation, using both transfer directions.The protocol introduces differences in acquisition protocols, intensity distributions, and population characteristics.
Multi-Domain Training: Joint training uses balanced 50/50 data from BraTS 2020 and BraTS 2024 to test whether heterogeneous exposure improves generalization.Patient-level holdout splits are preserved to avoid data leakage.
Results: T2F maintains the highest cross-domain performance, up to 76.2 Dice, while T1-based modalities show lower robustness at approximately 49.7 Dice.In-domain Dice values were 77.3–78.3 for FLAIR and 73.3–74.2 for T2W, but cross-domain transfer caused substantial degradation.
Results: Multi-domain training further mitigates degradation, with T2F reaching 75.0 compared with 77.3/78.3 in the corresponding single-domain baselines.The results support more transferable representations for T2-based inputs than for T1-based inputs.

B. Domain Adaptation (RQ3)

Fine-tuning the best-performing T2F/FLAIR modality with limited labeled target-domain data produces rapid gains in cross-domain performance, followed by diminishing returns.

Domain Adaptation: Fine-tuning with only 10–20% of labeled target-domain data already approaches upper-bound performance.Additional target data produces a plateau, indicating diminishing returns.
Domain Adaptation: Limited target-domain adaptation substantially improves generalization while reducing annotation and computational costs.The largest gains occur when adapting BraTS 2020-trained models to BraTS 2024; the reverse direction yields more modest improvements of approximately 5% Dice.

C. Qualitative Analysis

Tumor-size-stratified results show that overlap metrics improve from smaller to larger lesions, with T2f/FLAIR strongest for small tumors and T2w slightly favored for the largest.

Tumor-size stratification: Dice and IoU increase consistently from the smallest tumor quartile (Q1) to the largest (Q4).The quartiles contain equally populated groups, and values in parentheses report mean tumor area.
Modality comparison: T2f/FLAIR performs best for small tumors, indicating higher sensitivity to subtle boundaries.For medium tumors, T2f/FLAIR and T2w are comparable and both outperform T1-based modalities.
Modality comparison: T2w slightly surpasses T2f/FLAIR for large tumors in the upper quartile.For larger volumes overall, the two modalities offer similar performance, with a slight T2w advantage in Q4.
Qualitative examples: Figure 5 presents representative segmentation masks for cases closest to the mean tumor size in each quartile.These masks complement the quantitative tumor-size analysis in Table II.
Qualitative examples: Figure 6 contrasts high-accuracy, intermediate, and failure cases using T2f/FLAIR predictions against ground-truth overlays.The reported Dice scores are 99.1%, 70.1%, and 0.0%, respectively.

V. CONCLUSIONS

The study concludes that MRI modality, training-domain diversity, and tumor size shape cross-dataset generalization. T2f/FLAIR is strongest overall, while limited target-domain fine-tuning reaches near-upper-bound performance efficiently.

Conclusions: T2f/FLAIR provides the best cross-dataset performance and strongest generalization, followed closely by T2w.T1-based modalities exhibit lower robustness under domain shift.
Conclusions: Multi-domain training reduces the generalization gap and brings transfer performance closer to in-domain conditions.The conclusion identifies data diversity as an important factor in cross-dataset performance.
Conclusions: Tumor size significantly affects performance: T2f/FLAIR is more effective for small lesions, while T2w becomes comparable or slightly superior for larger tumors.The modality-size relationship is reported as part of the study’s conclusions.
Conclusions: 10–20% of target-domain data already approaches upper-bound performance during fine-tuning, with limited gains thereafter.This indicates diminishing returns as the annotation fraction increases.
Conclusions: The findings provide practical guidance for modality selection and resource-constrained deployment scenarios.The paper links generalization to modality choice, data diversity, and lesion characteristics.
Future work: Future work includes multimodal and 3D models, independent clinical datasets beyond BraTS, improved small-lesion detection, and foundation-model evaluation.The proposed directions also include improved embedding spaces and adaptive architectures.