Source-linked AI summary
Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation
Adrian V. Dalca, John Guttag, Mert R. Sabuncu
TL;DR
The paper addresses biomedical segmentation without paired images and manual segmentations. It learns an anatomical prior from unpaired segmentation data and embeds it in a generative CNN model for unsupervised segmentation. The authors report rapid segmentation across structural brain MRI settings, while noting that interpretation is limited by the lack of prior zero-shot results and modality-specific evaluation constraints.
Problem
CNN-based biomedical segmentation often depends on costly paired image–segmentation data, motivating methods that can operate when such annotations are unavailable.
Method
The method learns an anatomical prior and integrates it with a generative probabilistic model and convolutional networks to compute unsupervised segmentations.
Results
The model enables rapid unsupervised segmentation, with plausible anatomical predictions across T1-weighted and challenging T2-FLAIR brain MRI scans.
Takeaways & Limitations
Anatomical priors can support CNN-based segmentation in settings where segmentation is traditionally not possible or takes hours for a single scan.
Takeaways & Limitations
Interpretation is limited because rapid zero-shot segmentation lacks prior results for comparison, and T2-FLAIR evaluation is visual because no annotation tool was available.
Abstract
from arXiv · showhide
We consider the problem of segmenting a biomedical image into anatomical regions of interest. We specifically address the frequent scenario where we have no paired training data that contains images and their manual segmentations. Instead, we employ unpaired segmentation images to build an anatomical prior. Critically these segmentations can be derived from imaging data from a different dataset and imaging modality than the current task. We introduce a generative probabilistic model that employs the learned prior through a convolutional neural network to compute segmentations in an unsupervised setting. We conducted an empirical analysis of the proposed approach in the context of structural brain MRI segmentation, using a multi-study dataset of more than 14,000 scans. Our results show that an anatomical prior can enable fast unsupervised segmentation which is typically not possible using standard convolutional networks. The integration of anatomical priors can facilitate CNN-based anatomical segmentation in a range of novel clinical problems, where few or no annotations are available and thus standard networks are not trainable. The code is freely available at http://github.com/adalca/neuron.
1. Introduction
The paper addresses biomedical segmentation when paired image–annotation data are unavailable, combining anatomical priors with CNNs to support unsupervised segmentation. It presents a general, computationally efficient strategy that also supports subject-specific segmentation sampling and uncertainty estimates.
- CNN-based biomedical segmentation commonly requires costly paired scans and expert annotations, limiting its use when labeled data are unavailable.Manual delineation can take several expert hours per scan.
- The method integrates a deep anatomical prior with CNN-based segmentation through a theoretically grounded generative model.The prior is characterized with an auto-encoding variational CNN, while an encoder-decoder CNN performs segmentation.
- The approach operates in an unsupervised setting without paired annotations and is presented as general and computationally efficient.It also provides a framework for sampling possible subject-specific segmentations and estimating their uncertainty.
2. Related Work
Prior CNN segmentation methods often depend on annotations and lack explicit anatomical knowledge, while existing prior-based and generative approaches introduce limitations in scope, computation, or supervision. The paper combines generative anatomical modeling, CNNs, and variational inference to target rapid unsupervised segmentation.
- CNN segmentation methods typically require large datasets and augmentation, while lacking anatomical knowledge that could preserve known shapes and locations.This creates difficulties in unsupervised settings and in distinguishing similarly appearing structures with consistent anatomical positions.
- The proposed strategy uses CNN architectures to learn anatomical priors and segment medical images without paired example segmentations.This directly addresses the annotation burden associated with supervised CNN segmentation.
- CRF post-processing captures only local constraints and adds computation, while location features can be network-specific and increase parameter burden.These approaches do not provide the paper’s unified generative treatment of anatomical structure and image formation.
- Shape-prior methods generally address supervised problems through ad-hoc cost functions that correct standard CNN segmentations.Their use of learned shape representations remains tied to ground-truth segmentations in the described setting.
- Classical probabilistic segmentation models combine anatomical priors with appearance likelihoods, but common inference procedures can take several hours per image.The paper draws on these models while incorporating convolutional networks and variational Bayes approximations.
- The paper applies this combined framework to cortical and subcortical brain MRI regions and reports rapid unsupervised segmentation across imaging settings.The related-work context motivates a general approach beyond specialized modality- or disease-specific tools.
3. Generative Model
The generative model represents an MR volume as arising from an anatomical segmentation map, with a latent embedding governing plausible shapes and a label-conditioned appearance model generating voxel intensities. Learned parameters support MAP segmentation of new images.
- The model assumes that a 3D MR volume x is generated from a 3D anatomical segmentation map s.Image and segmentation values are indexed by voxel j.
- A latent variable z embeds anatomical shapes and has a normal prior with mean 0 and identity covariance.This embedding controls the prior distribution over plausible segmentation maps.
- The segmentation map s is drawn from a categorical distribution conditioned on z, with f_j,l giving the probability of label l at voxel j.Thus, the latent embedding determines the possible anatomical shapes represented in s.
- Given s, voxel intensities are generated from a label-conditioned normal appearance model parameterized by anatomical-label means and variances.The joint likelihood factors into the image likelihood pθx|s(x|s) and segmentation prior pθs|z(s|z).
- After learning the model parameters, segmentation for a new image is obtained by maximum a posteriori estimation.The inference target is the segmentation s_i corresponding to the observed image x_i.
4. Learning
The model uses convolutional networks and variational inference to learn anatomical representations, a segmentation prior, and image-conditioned posterior distributions. Inference is feed-forward, while sampling from the learned distributions supports plausible segmentations and uncertainty estimation.
- Variational learning: Because the posterior and marginal segmentation distributions are intractable, the method uses variational approximations, Jensen’s inequality, and a voxel-factorized prior to derive trainable objectives.The approximation qψ(z|s) is modeled as a normal distribution with diagonal covariance, and SGVB with mini-batches provides optimization.
- Learning the generative model: The framework uses convolutional encoders to estimate latent anatomical representations and approximate posterior parameters from images.The image encoder outputs the mean and diagonal covariance of a low-dimensional embedding distribution.
- Learning the generative model: A decoder maps latent variables to segmentation-distribution parameters, while an appearance model maps segmentation probabilities to image-intensity parameters.The appearance model estimates the mean intensity parameters and assumes additive zero-mean Gaussian noise with separately estimated variance.
- Learning the anatomical prior: The anatomical prior is learned independently from unpaired segmentation maps, including maps from other datasets or imaging modalities, without paired image-segmentation training data.Its encoder outputs µz|s and Σz|s, and the prior decoder is used in the full generative model.
- Inference and uncertainty: At inference, the model encodes an image, decodes the latent mean, and selects the maximum segmentation at each voxel using fast feed-forward operations.Sampling latent variables and decoding them instead produces multiple plausible subject-specific segmentations for uncertainty estimation.
- Implementation: The implementation uses a 2D convolutional architecture for 3D images, with five downsampling levels, 3x3 kernels, ELU activations, and a location-specific voxel prior.The location prior is multiplied with the decoder output and discourages extreme decodings, while shape properties remain encoded in the decoder.
5. Experiments
The experiments evaluate unsupervised anatomical segmentation on large T1-weighted and cross-modality T2-FLAIR datasets. The proposed method produces plausible anatomical structures, operates rapidly, and remains effective despite different tissue contrasts and lower-quality images.
- Datasets: The study uses more than 14,000 T1-weighted brain MRI scans from eight publicly available datasets with varied ages, health states, and acquisition details.Scans were resampled to a 256x256x256 grid with 1mm isotropic voxels and cropped to 160x192x224.
- Datasets: A subset of 5,000 annotated images trained the anatomical prior, while the remaining data were treated as unannotated and used for validation.After architecture selection, the test dataset was evaluated by training and testing the model in an unsupervised fashion.
- Datasets: The second dataset contains more than 3800 T2-FLAIR scans with different tissue properties, lower acquisition quality, and 5mm slice spacing.These scans test whether priors learned from T1-weighted data remain useful across modalities; no automatic tool was available for detailed anatomical segmentation there.
- Evaluation: Standard unsupervised U-Net baselines using image MSE or mutual information produced nonsensical segmentations and were omitted from the results.FreeSurfer results were regarded as an optimistic bound for the T1-weighted data, while T2-FLAIR results were evaluated visually.
- Efficiency: At test time, a new subject requires affine registration to a template before the CNN produces a segmentation estimate in less than a few seconds on an NVidia Titan X GPU.The reported runtime covers the registration-plus-inference workflow described for a new subject.
- T1-weighted results: The proposed method estimates approximate T1-weighted anatomical locations and shapes, but smooths fine cortical-folding details that the prior does not easily capture.Figure 6 reports average Dice overlap across several prevalent, larger anatomical regions.
- T2-FLAIR results: Despite lower image quality and different tissue contrasts, the method predicts plausible T2-FLAIR segmentations in challenging unsupervised cases.The T2-FLAIR examples use overlays and outlines across different coronal slices to show variability and task difficulty.
6. Conclusion
The paper introduces a generative probabilistic model that uses a CNN-learned anatomical prior for unsupervised segmentation. It enables rapid inference where segmentation is traditionally impossible or takes hours per scan.
- The model uses a convolutional neural network to learn an anatomical prior for unsupervised segmentation.
- The anatomical prior encourages segmentation maps from a known distribution while requiring generated images to agree with the observed scan.
- The approach enables rapid convolutional-network segmentation in settings where segmentation is traditionally impossible or takes hours for one scan.
- Integrating anatomical priors may facilitate accurate segmentation for novel clinical problems with limited dataset availability.