Source-linked AI summary
Semi-Supervised Deep Learning for Fully Convolutional Networks
Christoph Baur, Shadi Albarqouni, Nassir Navab
TL;DR
Semi-supervised learning can use unlabeled data, but existing methods had not addressed FCNs, which are used for image segmentation. This paper adapts auxiliary manifold embedding to FCNs with Random Feature Embedding and reports improved generalization on MS lesion segmentation when using angular cosine distance.
Problem
Semi-supervised deep-learning methods had focused on classic CNNs, leaving no known method for FCNs despite their use in image segmentation.
Source: §1 · p. 2Method
The framework adds an auxiliary manifold embedding loss to an FCN and uses Random Feature Embedding to sample pixel representations from labeled and unlabeled data.
Source: §2.1 · p. 3, §2.2 · p. 4Results
The method can improve generalization of existing models on MS lesion segmentation when angular cosine distance is used as the distance metric.
Source: §4 · p. 8Takeaways & Limitations
The authors identify testing the method on other problems and investigating multiple embedding objectives at different FCN layers as follow-up work.
Source: §4 · p. 8The reported experiments concern MS lesion segmentation; the authors call for applying the method to other problems to assess its potential.
Source: §4 · p. 8
Abstract
from arXiv · showhide
Deep learning usually requires large amounts of labeled training data, but annotating data is costly and tedious. The framework of semi-supervised learning provides the means to use both labeled data and arbitrary amounts of unlabeled data for training. Recently, semi-supervised deep learning has been intensively studied for standard CNN architectures. However, Fully Convolutional Networks (FCNs) set the state-of-the-art for many image segmentation tasks. To the best of our knowledge, there is no existing semi-supervised learning method for such FCNs yet. We lift the concept of auxiliary manifold embedding for semi-supervised learning to FCNs with the help of Random Feature Embedding. In our experiments on the challenging task of MS Lesion Segmentation, we leverage the proposed framework for the purpose of domain adaptation and report substantial improvements over the baseline model.
1 Introduction
Medical image segmentation faces scarce labeled data and device-related distribution shifts, while semi-supervised methods had mainly targeted classic CNNs. The paper extends auxiliary manifold embedding to FCNs using Random Feature Embedding and applies it to MS lesion segmentation for domain adaptation.
1 Introduction
- MS lesion segmentation is challenged by scarce ground-truth labels and distribution shifts across MRI devices.Manual annotation is time-consuming and tedious, while unlabeled data can often be provided more readily.Source: §1 · p. 1
- Earlier semi-supervised deep-learning approaches included auxiliary manifold embedding, entropy regularization, ladder networks, and graph embeddings.The introduction describes work by Weston et al., Lee et al., Rasmus et al. [8], and Yang et al. [12].Source: §1 · pp. 1–2
- The paper introduces Random Feature Embedding to extend auxiliary manifold embedding to FCNs and uses it for semi-supervised domain adaptation in MS lesion segmentation.The authors describe this as a novel strategy for FCNs.Source: §1 · p. 2
2 Methodology
The framework trains an FCN using labeled images for segmentation and labeled and unlabeled images for an auxiliary embedding objective. Random Feature Embedding samples pixel representations to make this objective practical for FCNs.
2 Methodology
- The framework uses labeled images to train a modified U-Net with a Dice-Loss segmentation objective and uses labeled and unlabeled images for an auxiliary embedding loss.The input images and label maps have matching spatial dimensions, allowing a label per pixel.Source: §2 · p. 2, §2.1 · p. 3
2.1 Auxiliary Manifold Embedding
- The auxiliary manifold embedding loss draws neighboring representations together and pushes non-neighboring ones apart when they fall within margin m.Its weight λ_l regularizes the loss at hidden layer l; adjacency can be derived from labels or template matching with NCC, similar to.Source: §2.1 · p. 3
- Angular cosine distance is bounded between [0, 1] and is chosen for high-dimensional deep features instead of the typical ℓ2-norm distance.The paper gives easier margin selection and superior performance on high-dimensional feature representations as reasons for this choice.Source: §2.1 · p. 3
2.2 Random Feature Embedding
- Random Feature Embedding samples a limited number of pixel embeddings because comparing embeddings from all pixels in large images is computationally infeasible.The loss remains valid because gradients are propagated only for selected pixels.Source: §2.2 · pp. 3–4
- The sampling strategies balance class representation differently: 50/50 equalizes classes, Distribution-Aware RFE preserves the prior ratio, and 80/20 fixes background and foreground sampling at 80% and 20%.The first two can respectively oversample or undersample a class when classes are unbalanced and the embedding sample is small.Source: §2.2 · p. 4
3 Experiments and Results
The experiments assess semi-supervised domain adaptation for MS lesion segmentation across scanner-defined domains. With a noisy prior, ACD, and 80/20 Random Feature Embedding, models improve over the lower-bound model across target domains.
3 Experiments and Results
- MS lesion experiments use scanner-defined domains A–D, about 6,000 lesion-centered patches per domain, a 7:3 train/validation split, and full-slice testing.The framework is used for domain adaptation to improve a baseline model with data from other domains.Source: §3 · pp. 4–5
3.1 Baseline Models
- Lower-bound AL trains on domain A alone, while upper bounds fine-tune AL with labeled data from domain A and one target domain.The upper bounds represent models trained for domains A and B, A and C, or A and D.Source: §3.1 · pp. 5–6
Proof of Concept
- Across 36 target-B settings, increasing nE brought random embeddings closer to the full distribution and improved ACD segmentation, especially with 80/20 sampling; ℓ2 was poor and unstable.The experiments varied sampling strategy, embedding count, and distance metric; the figure reports domain-B F-scores and embedding-distribution divergence.Source: Figure 3 · p. 6, Proof of Concept · pp. 6–7
3.2 Semi-Supervised Embedding
- Semi-supervised models start from epoch 35 of AL and attach embedding loss to the second-last convolutional layer, whose input-resolution maps avoid downsampling small-lesion priors.This layer choice avoids losing heavily underrepresented lesion pixels when downsampling the prior.Source: §3.2 · p. 6
Real Prior
- With noisy priors, ACD, and 80/20 RFE, models improved over lower-bound AL across domains B–D; Fig. 4 shows fewer false positives in domain C and slight inferiority to its upper bound.The real-prior models used about 4,000 patches from target-domain volumes and labeled training patches from domain A.Source: Figure 4 · p. 7, Real Prior · p. 7
4 Discussion and Conclusion
The paper integrates auxiliary manifold embedding into semi-supervised FCNs through Random Feature Embedding. On MS lesion segmentation, the method improves existing models’ generalization when ACD is used, while further investigation remains open.
4 Discussion and Conclusion
- The paper integrates auxiliary manifold embedding into a semi-supervised deep-learning framework for FCNs, using Random Feature Embedding to sample feature representations for nonlinear embedding.The authors identify multi-layer embedding objectives and evaluation on other problems as directions for future work.Source: §4 · p. 8
- On MS lesion segmentation, the method improved existing models’ generalization when ACD was used.Source: §4 · p. 8
- The method’s full potential remains to be explored through embedding objectives at multiple FCN layers and application to other problems.Source: §4 · p. 8