Adversarial training and dilated convolutions for brain MRI segmentation

1 Introduction

• CNNs have been applied to brain MRI tissue segmentation and brain abnormality segmentation.Adversarial segmentation approaches have also been used for prostate MRI [9] and chest X-ray organ segmentation [1].Source: §1 · pp. 1–2
• Dilated convolutions sparsely distribute weights across a larger receptive field while preserving input coverage.They can provide a large receptive field with limited trainable weights and convolutional layers, without subsampling.Source: §1 · p. 1
• GANs use a discriminator to distinguish real from generated images, motivating a generator to produce images that look real.Related adversarial approaches have used discriminators to distinguish domains or manual from generated segmentations.Source: §1 · pp. 1–2
• Adversarial training is applied to brain MRI segmentation using fully convolutional and dilated network architectures.The study evaluates two image sets.Source: §1 · p. 2

2 Materials and Methods

• 35 T1-weighted scans were split into 15 training and 20 test images.They were acquired on a Siemens Vision 1.5T scanner; subject age was 32.9 ± 19.2 years (mean ± SD).Source: Adult subjects · p. 2
• Images were segmented into six classes: WM, cGM, BGT, CB, BS, and lvCSF.These denote white matter, cortical grey matter, basal ganglia and thalami, cerebellum, brain stem, and lateral ventricular cerebrospinal fluid.Source: Adult subjects · p. 2
• The images came from the MICCAI 2012 challenge on multi-atlas labelling.Source: Adult subjects · p. 2

Adult subjects

• 20 scans were split into five training and 15 test images.They were acquired on a Philips Achieva 3T scanner; subject age was 70.5 ± 4.0 years (mean ± SD).Source: Elderly subjects · p. 2
• Images were segmented into seven classes: WM, cGM, BGT, CB, BS, lvCSF, and pCSF.Possible white matter lesions were included in the WM class.Source: Elderly subjects · p. 2
• The images were provided by the MRBrainS13 challenge.Source: Elderly subjects · p. 2

2.2 Network architecture

• Both segmentation architectures feed their outputs to a discriminator that distinguishes generated from manual segmentations.Both are fully convolutional, allowing arbitrarily sized inputs during testing.Source: §2.2 · p. 2
• The dilated network uses 15 convolutional layers of 32 3×3 kernels, yielding a 31×31-voxel receptive field.Training uses a 51×51-voxel input and a 21×21-voxel output.Source: Fully convolutional network · p. 2
• The network has 140,039 trainable parameters for C = 7 classes (6 plus background; adult subjects) and 140,296 for C = 8 classes (7 plus background; elderly subjects).Source: Fully convolutional network · p. 2

Fully convolutional network

• The network uses increasing dilation factors across 3×3 convolutional layers, without subsampling.Its architecture follows Yu et al..Source: Dilated network · p. 3
• A 67×67-voxel receptive field is achieved with seven 3×3 convolutional layers.Training uses an 87×87-voxel input corresponding to a 21×21-voxel output.Source: Dilated network · p. 3
• The network has 56,039 trainable parameters for C = 7 and 56,072 for C = 8.C = 7 applies to adult subjects; C = 8 applies to elderly subjects.Source: Dilated network · p. 3

Dilated network

• The discriminator receives a one-hot or softmax segmentation and a 25×25 image patch.It evaluates real versus generated image–segmentation combinations.Source: Discriminator network · p. 3
• Two image-processing convolutional layers precede concatenation with the segmentation.Source: Discriminator network · p. 3
• The discriminator ends with two output nodes for manual versus generated segmentations.Between concatenation and output are convolutional layers, max pooling, and a 256-node fully connected layer.Source: Discriminator network · p. 3

Discriminator network

• The dilated network has a 67×67 receptive field, compared with 31×31 for the fully convolutional network.Neither network uses subsampling layers.Source: Figure 1 · p. 3
• The figure’s right panel shows adversarial training, with red connections indicating discriminator-loss influence on segmentation-network backpropagation.Source: Figure 1 · p. 3
• Figure 2 shows four test cases ordered by adult FCN, adult DN, elderly FCN, and elderly DN.Colours encode WM, cGM, BGT, CB, BS, lvCSF, and pCSF; arrows mark errors corrected with adversarial training.Source: Figure 2 · p. 4

2.3 Adversarial training

• Segmentation updates ascend adversarial discriminator loss, while cross-entropy updates descend segmentation loss.The discriminator is also updated using manual segmentations.Source: §2.3 · p. 5
• The three update types alternate: segmentation cross-entropy, discriminator loss on manual segmentations, and adversarial loss on images.Source: §2.3 · p. 5
• Separate optimisers and learning rates are used; the discriminator’s smaller learning rate slows its adaptation.This is intended to keep discriminator loss from converging too quickly and preserve its influence on the segmentation network.Source: §2.3 · p. 5
• ReLU units are used throughout, with batch normalisation on all layers and dropout on 1×1 convolution layers.Source: §2.3 · p. 5

3.1 Experiments

• Training used RMSprop, 300-sample minibatches, and five epochs of 50,000 patches per class per image; adversarial updates alternated among Ls, Ld, and La.The reported learning rate was 10−3 for baseline segmentation and 10−3 for segmentation and 10−5 for discriminator updates in the adversarial setup.Source: §3.1 · p. 5

3.2 Evaluation

• 0.67 ±0.04 to 0.91 ±0.03 (fully convolutional) and 0.91 ± 0.03 to 0.92 ± 0.03 (dilated): adult six-class mean Dice improved with adversarial training.The values are means ± standard deviations.Source: Figure 3 · p. 6, §3.2 · p. 6
• 0.80 ± 0.02 to 0.83 ± 0.02 (fully convolutional) and 0.83 ± 0.02 to 0.85 ± 0.01 (dilated): elderly seven-class mean Dice improved with adversarial training.The values are means ± standard deviations.Source: Figure 3 · p. 6, §3.2 · p. 7
• Segmentation components per image decreased with adversarial training across both subject sets and architectures.Adult subjects: 1745 ± 400 to 626 ± 247 (fully convolutional) and 417 ± 152 to 365 ± 122 (dilated); elderly subjects: 926 ± 134 to 692 ± 88 (fully convolutional) and 601 ± 104 to 481 ± 90 (dilated).Source: §3.2 · p. 6
• Paired t-tests found significant Dice improvement for every tissue class in both subject sets and networks except elderly lvCSF with the dilated network.Source: §3.2 · p. 6, Figure 3 · p. 6

• Adversarial training improved segmentation quality and structural consistency, though improvements were usually small and had limited effect on Dice coefficients.Improvements were especially clear for the deeper, more difficult-to-train fully convolutional networks compared with shallower dilated networks.Source: §4 · p. 7
• The additional loss distinguishes real from generated segmentations, capturing inconsistencies that averaged per-voxel loss does not.Source: §4 · p. 7
• The effects of architecture and training choices remain to be evaluated; larger output patches may add discriminative information but risk overly global decisions that reduce local information.Source: §4 · p. 7