Source-linked AI summary
Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation
Mohsen Ghafoorian, Alireza Mehrtash, Tina Kapur, Nico Karssemeijer, Elena Marchiori, Mehran Pesteie, Charles R. G. Guttmann, Frank-Erik de Leeuw, Clare M. Tempany, Bram van Ginneken, Andriy Fedorov, Purang Abolmaesumi, Bram Platel, William M. Wells
TL;DR
MRI protocol differences can make CNNs trained on one domain perform poorly on another, limiting reuse of annotated legacy datasets. This study evaluates transfer-learning adaptation for brain WMH segmentation across domains and compares it with direct application and training from scratch. With two target-domain training images, the adapted network achieved a Dice score of 0.63, while the from-scratch model achieved 0.15.
Problem
MRI acquisition differences change image appearance, while CNNs trained under one protocol may perform poorly on another domain, raising questions about reusing legacy models and data.
Method
The study trains a CNN on legacy MRI data, transfers its weights to the target domain, freezes shallow layers, fine-tunes deeper layers, and compares this with direct transfer and training from scratch.
Results
With two target-domain training images, the adapted model achieved a Dice score of 0.63 versus 0.15 for a model trained from scratch on the same data.
Takeaways & Limitations
Transfer learning substantially outperformed training from scratch when only a small set of target-domain examples was available.
Abstract
from arXiv · showhide
Magnetic Resonance Imaging (MRI) is widely used in routine clinical diagnosis and treatment. However, variations in MRI acquisition protocols result in different appearances of normal and diseased tissue in the images. Convolutional neural networks (CNNs), which have shown to be successful in many medical image analysis tasks, are typically sensitive to the variations in imaging protocols. Therefore, in many cases, networks trained on data acquired with one MRI protocol, do not perform satisfactorily on data acquired with different protocols. This limits the use of models trained with large annotated legacy datasets on a new dataset with a different domain which is often a recurring situation in clinical settings. In this study, we aim to answer the following central questions regarding domain adaptation in medical image analysis: Given a fitted legacy model, 1) How much data from the new domain is required for a decent adaptation of the original network?; and, 2) What portion of the pre-trained model parameters should be retrained given a certain number of the new domain training samples? To address these questions, we conducted extensive experiments in white matter hyperintensity segmentation task. We trained a CNN on legacy MR images of brain and evaluated the performance of the domain-adapted network on the same task with images from a different domain. We then compared the performance of the model to the surrogate scenarios where either the same trained network is used or a new network is trained from scratch on the new dataset.The domain-adapted network tuned only by two training examples achieved a Dice score of 0.63 substantially outperforming a similar network trained on the same set of examples from scratch.
1 Introduction
The study addresses domain adaptation for CNN-based brain WMH segmentation when MRI acquisition protocols create different data domains. It investigates how much target-domain data and how many pretrained layers should be used during adaptation.
- Motivation: CNNs perform well on specific medical imaging domains, but their generalizability to differently distributed test data is often not evaluated.This creates uncertainty about their usability across imaging protocols.
- Background: Transfer learning improves learning of a target predictive function using information from a different source domain or task.The source-trained predictive model is denoted ˜fST(.) after adaptation to the target domain.
- Background: CNNs are expected to learn general visual features in early layers and more abstract task-dependent features in deeper layers.This hierarchy motivates examining which layers should remain frozen or be fine-tuned during domain adaptation.
- Research questions: The study asks how much target-domain data is needed and how many pretrained layers should be fine-tuned for effective adaptation.These questions are especially relevant when costly annotated legacy data coexist with newer datasets acquired after scanner or protocol changes.
- Contribution: The experiments apply transfer learning to adapt models trained on legacy MRI data for brain WMH segmentation.The study aims to answer the domain-adaptation questions through this task.
2 Materials and Method
The experiments use longitudinal MRI data as source and target domains, train a 15-layer patch-based CNN, and adapt source models by freezing shallow layers while fine-tuning deeper ones. Adaptation is compared with direct transfer and target-domain training from scratch across multiple target training-set sizes.
- Dataset: The source domain contains 280 baseline patient acquisitions, while the target domain contains 159 follow-up scans acquired with different MRI parameters.Baseline and follow-up FLAIR scans differ in voxel size and slice gap, while T1 images use the same voxel size.
- Dataset: The study preserves the baseline patient-level partitioning in the follow-up dataset to prevent potential label leakages.This keeps corresponding source and target partitions separated at the patient level.
- Data preparation: Training uses 32×32 FLAIR and T1 patches labeled by their central voxel, with WMH and matched normal-voxel sampling plus y-axis flipping augmentation.The resulting training datasets contain approximately 1.2 million baseline patches and 1.75 million follow-up patches.
- Network architecture: The CNN has 15 layers: 12 convolutional layers with 3×3 filters, three dense layers, and a final softmax layer.Pooling is omitted to preserve spatial feature information needed for segmentation.
- Domain adaptation: Domain adaptation transfers source weights, freezes the shallowest i layers, and fine-tunes the remaining d −i deeper layers using target-domain data.The network depth d is 15 in these experiments, and the same optimization, loss, and regularization procedures are retained.
- Experimental design: The experiments compare direct source-to-target application, target-domain training from scratch, and source-model transfer with fine-tuning.Training sizes range from 2 to 100 target cases, and segmentation quality is measured with Dice score.
3 Results
The results compare target-domain Dice scores across direct application, transfer learning, and training from scratch, while varying target training size and the number of fine-tuned layers.
- 0.76 Dice was achieved on the source domain, whereas direct application of the same model to the target domain yielded 0.005 Dice.
- The comparison included domain-adapted models, a source-trained model without fine-tuning, and networks trained from scratch on varying target-domain sample sizes.
- Target-domain test Dice scores were evaluated as functions of training-set size and the number of abstract layers fine-tuned.
4 Discussion and Conclusions
Domain adaptation substantially improves performance with very small target-domain datasets, while the most effective fine-tuning depth depends on available data.
- 0.63 Dice was achieved by domain adaptation using only two target training images, compared with 0.15 for training from scratch on the same number of images.
- With only a few target training cases, fine-tuning only the last dense layers produced the best results and avoided over-fitting from too many parameters.
- As more target training data became available, fine-tuning shallower representations such as the last convolutional layers became more appropriate.
- Tuning the first few convolutional layers was rarely useful because their characteristics were considered domain-independent.