Deep Learning for identifying radiogenomic associations in breast cancer

• Molecular subtypes are associated with disease progression, so physicians base treatment recommendations on genomic analysis of each patient’s tumor.Source: Introduction · p. 1
• Prior breast-cancer MRI radiogenomics used radiologist-extracted or algorithmic features, but handcrafted features require researchers to anticipate useful tumor characteristics.Examples include texture, shape, and enhancement dynamics.Source: Introduction · p. 1
• The study proposes comparing training from scratch, transfer learning, and off-the-shelf deep features for breast-cancer radiogenomic analysis.Deep learning has shown superiority to handcrafted approaches in automatic feature extraction, image classification, and object detection [17–19].Source: Introduction · p. 1
• Limited medical-imaging samples can cause overfitting when training from scratch, motivating transfer learning or pre-trained feature extraction with a traditional classifier.The authors note that these approaches have worked well on subsets of medical-imaging tasks [20–22].Source: Introduction · p. 2

Patient Population

• 270 of 400 consecutive patients were included after exclusions, reader review, and image-processing checks.The remaining 286 cases were annotated by six breast imagers; 11 were skipped, and five more were excluded from the remaining 275 cases.Source: Patient Population · p. 2
• 90 of the 270 cases were Luminal A, while 180 belonged to the other three subtypes.The cohort comprised consecutive preoperative DCE-MRIs acquired at Duke University Medical Center from September 2007 to June 2009.Source: Patient Population · p. 2

Imaging and Pathology Data

• MRI was acquired on 1.5 T or 3.0 T scanners using a breast coil.Scanners included Signa HDx/HDxt and MAGNETOM Avanto/Trio systems.Source: Imaging and Pathology Data · p. 2
• Each MRI included nonfat-saturated T1-weighted, fat-saturated T2-weighted, and pre-contrast plus three post-contrast fat-suppressed T1-weighted gradient-echo series.Contrast was administered intravenously at 0.2 mL/kg.Source: Imaging and Pathology Data · p. 2
• ER, PR, and HER2-neu status came from the initial breast biopsy pathology report.Source: Imaging and Pathology Data · p. 2

Image Annotation

• Six fellowship-trained breast-imaging radiologists with 6–20 years of experience annotated up to five lesion bounding boxes per case, converted into binary masks.Mask values were 1 inside and 0 outside each bounding box.Source: Image Annotation · p. 2

MR Imaging Pre-processing and Tumor Patch Extraction

• MRIs with differing pixel spacing were registered to the most frequent spatial resolution, with other resolutions scaled by bilinear interpolation.Source: Image Annotation · p. 2
• Three-channel images concatenated the three post-contrast sequences minus the pre-contrast sequence, and square patches of 80–120 pixels were sampled around lesion centers.Each lesion also received four patches generated by random translation and rotation; 44,660 patches were generated for each patch size.Source: Image Annotation · p. 2

Methods Overview

• Off-the-shelf deep features used a natural-image-pretrained network as a feature extractor, then trained an SVM on the extracted features.Thin arrows in Figure 1 indicate feeding images to the networks; thick arrows indicate training.Source: Figure 1 · p. 3
• Transfer learning pre-trained a model on natural images, then fine-tuned it using the study dataset.Figure 1 distinguishes training steps from feeding images to the network using thick and thin arrows, respectively.Source: Figure 1 · p. 3
• Training from scratch directly trained a neural network using the study dataset.The models were trained for 40 epochs using Caffe on a desktop with an NVIDIA GTX 1080 GPU.Source: Methods Overview · p. 2, Figure 1 · p. 3

Training from Scratch

• Four architectures were evaluated: original and reduced GoogleNet, reduced VGGNet, and CIFAR, with GoogleNet chosen for multi-scale lesions and reduced VGGNet for data scarcity.GoogleNet’s Inception module uses different kernel sizes to capture patterns at different scales; VGGNet has more weights and requires more training data.Source: Training from Scratch · p. 3
• GoogleNet_R reduced its first convolution layer and pooling stride, while original GoogleNet used a 0.0001 learning rate rather than the 0.01 default.The learning rate was lowered because the study’s image patches varied less than natural images.Source: Training from Scratch · p. 3
• Reduced VGGNet retained its first two convolution layers and all fully connected layers; network inputs were 64×64×3 for GoogleNet and 80×80×3 for VGG_R and CIFAR.The final layer output dimension was changed from 1000 to 2 for GoogleNet and VGG_R; CIFAR used an output dimension of 2.Source: Training from Scratch · pp. 3–4

Transfer Learning

• The workflow pre-trained GoogleNet and VGGNet on ImageNet, then fine-tuned them on the study dataset.Transfer learning uses a network pre-trained for a different task before fine-tuning it on the target dataset.Source: Transfer Learning · p. 4
• The final fully connected layer was changed from 1000 outputs to 2 and randomly reinitialized; its learning rate was 0.001 versus 0.0001 for other layers.Source: Transfer Learning · p. 4
• Patches were resized to 256×256×3, then randomly cropped each epoch to 224×224×3 inputs for both networks.Source: Transfer Learning · p. 4

Off-the-shelf deep features

• ImageNet-pretrained GoogleNet and VGGNet served as feature extractors, with SVMs trained separately on the extracted features.Separating feature extraction from classification was intended to reduce overfitting while leveraging deep features.Source: Off-the-shelf deep features · p. 4
• The last fully connected layer was the default feature source, while feature maps from other layers were also evaluated.The study considered that shallow layers provide less-specific low-level features, whereas deeper layers provide higher-level but more task-specific features.Source: Off-the-shelf deep features · p. 4
• Feature maps were max pooled across image-plane coordinates to limit vector length, and SVMs with different kernel functions were compared.The authors noted that the limited training sample size could impair SVM performance when feature length exceeded the number of training samples.Source: Off-the-shelf deep features · p. 4

Model Evaluation

• At test time, each patient’s final score was the average of 25 patch scores sampled from five lesion-containing slices.The five slices with the largest lesions were selected, with patches taken from each slice’s four corners and middle region.Source: Model Evaluation · p. 4
• The dataset was divided into 10 folds with approximately balanced positive-to-negative sample ratios, and final AUC was the mean of fold-specific AUCs.Bootstrapping was used to calculate the confidence interval.Source: Model Evaluation · p. 5

Results

• CIFAR had the highest test-set AUC among the four scratch-trained networks, while the larger networks’ training AUCs all exceeded 0.90.The authors interpret this pattern as indicating that larger networks were more prone to overfitting on the small dataset.Source: Results · p. 5, Table 1 · p. 5
• 0.65 test AUC was the best result across tested feature layers, achieved by GoogleNet’s final fully connected layer (FC1).FC1’s test AUC exceeded the other layer results, which ranged from 0.52 to 0.63.Source: Results · p. 6, Table 4
• 0.64 AUC was the best result from a single FC1 feature, slightly below the all-feature polynomial-kernel SVM result, but this estimate was biased and not crossvalidated.The single-feature analysis evaluated 1,000 features.Source: Results · p. 6, Figure 4 · p. 6

Discussion

• 0.65 AUC was highest for pre-trained GoogleNet features with a polynomial-kernel SVM; transfer learning performed below this approach but above training from scratch at 0.58 AUC.The authors interpret the pattern as evidence that overfitting is a major problem when training samples are limited.Source: Discussion · p. 7
• The authors identify pre-training on large medical-image datasets and multi-task learning as potential ways to address data insufficiency and overfitting.They propose that tasks with little training data could benefit from tasks with more abundant data.Source: Discussion · p. 7
• The results had low-to-intermediate prognostic power, and the small case count limited analysis to Luminal A versus all other subtypes.The authors describe the observed predictive value despite the small sample as encouraging, while noting that individual subtype distinctions were not assessed because there were few non-Luminal A cases.Source: Discussion · p. 7
• The authors conclude that deep learning can aid investigation of imaging–tumor relationships in breast cancer and propose repeating the study with a larger dataset.They also identify comparison with traditional machine-learning models using hand-crafted features as future work.Source: Discussion · p. 7