PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Atlas Segmentation of Biomedical Images: A Survey

Juan Eugenio Iglesias, Mert Rory Sabuncu

arXiv:1412.3421v2cs.CV

TL;DR

Manual biomedical image segmentation is slow, costly, error-prone, and difficult to scale, motivating automatic approaches. This survey traces multi-atlas segmentation (MAS), in which labeled atlases are registered to novel images and their labels propagated or fused, and synthesizes its evolution, applications, trade-offs, and future directions. MAS offers a flexible framework for biomedical segmentation, but registration remains a computational bottleneck and performance is less established beyond brain MRI.

  • Problem

    Manual delineation is slow, costly, error-prone, difficult to reproduce, and unscalable, creating a need for automatic or semi-automatic biomedical image segmentation.

  • Method

    The survey documents MAS development, methods, applications, relationships to alternative approaches, and future directions, covering strategies that use individual labeled atlases rather than a summarized model.

  • Results

    MAS provides a flexible framework for biomedical segmentation that can exploit anatomical variation and application-specific trade-offs, while registration remains its computational bottleneck.

  • Takeaways & Limitations

    MAS is positioned as a promising approach for a wide range of biomedical applications, including longitudinal image analysis where highly accurate segmentation is needed.

  • Takeaways & Limitations

    Most MAS applications have focused on human brain MRI, while registration is less effective in modalities and body parts such as abdominal imaging.

Abstract

from arXiv · show

Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing, Brandt, Menzel and Maurer Jr (2004), Klein, Mensh, Ghosh, Tourville and Hirsch (2005), and Heckemann, Hajnal, Aljabar, Rueckert and Hammers (2006), is becoming one of the most widely-used and successful image segmentation techniques in biomedical applications. By manipulating and utilizing the entire dataset of "atlases" (training images that have been previously labeled, e.g., manually by an expert), rather than some model-based average representation, MAS has the flexibility to better capture anatomical variation, thus offering superior segmentation accuracy. This benefit, however, typically comes at a high computational cost. Recent advancements in computer hardware and image processing software have been instrumental in addressing this challenge and facilitated the wide adoption of MAS. Today, MAS has come a long way and the approach includes a wide array of sophisticated algorithms that employ ideas from machine learning, probabilistic modeling, optimization, and computer vision, among other fields. This paper presents a survey of published MAS algorithms and studies that have applied these methods to various biomedical problems. In writing this survey, we have three distinct aims. Our primary goal is to document how MAS was originally conceived, later evolved, and now relates to alternative methods. Second, this paper is intended to be a detailed reference of past research activity in MAS, which now spans over a decade (2003 - 2014) and entails novel methodological developments and application-specific solutions. Finally, our goal is to also present a perspective on the future of MAS, which, we believe, will be one of the dominant approaches in biomedical image segmentation.

1. Historical Introduction and Background

Biomedical image segmentation assigns biologically meaningful labels to pixels or voxels, but manual delineation is slow, costly, error-prone, and difficult to scale. MAS addresses anatomical variation by retaining and consulting multiple labeled atlases rather than reducing them to a single model.

  • Segmentation labels image pixels or voxels with biologically meaningful structures, tissues, regions of interest, or background.
  • Manual delineation is slow, expensive, error-prone, difficult to reproduce, and unsuitable for many large-scale or time-critical applications.Automatic and semi-automatic methods can speed processing while improving reliability, repeatability, and scalability.
  • Classical atlas-guided segmentation registers an expert-labeled atlas to a novel image and propagates its labels through the resulting spatial mapping.The registration step deforms one image until it resembles the other, establishing correspondence between coordinate systems.
  • A single atlas and deformation model usually cannot capture wide anatomical variation, motivating the use of several atlases for segmentation.Early multi-atlas approaches selected a relevant atlas before applying registration-based segmentation.
  • MAS keeps individual atlases available for segmenting a novel image, commonly registering each atlas, propagating labels, and selecting the most frequent voxel label.This label-fusion rule is commonly called majority voting.
  • The survey organizes MAS methods by their building blocks and reviews applications, future directions, and literature published through the end of 2014.The authors note that some relevant or redundant publications may have been omitted.

2.1. Generation of Atlases

Atlases are labeled training images that underpin MAS, but their quality, representativeness, and acquisition cost constrain performance. Surveyed strategies enrich or filter atlas pools using expert, non-expert, synthetic, and problem-specific sources.

  • Atlases are labeled training images typically produced through meticulous, costly expert annotation using visualization software and anatomical knowledge.
  • Atlas selection can improve accuracy by discarding low-quality or irrelevant training cases, although reducing the atlas pool can also decrease segmentation quality.Problem-specific prior knowledge can guide selection, such as choosing narrow-lumen coronary arteries for neointima segmentation.
  • Synthetic atlases can enrich anatomical variability when available atlases are unrepresentative, but synthesized deformations may be anatomically implausible.One cited strategy uses a principal-component-analysis model to synthesize deformations.
  • Non-expert segmentations can trade annotation quality for atlas quantity, with methods modeling annotator expertise or weighting contributions by self-declared confidence.

2.2. Offline Learning

Offline learning analyzes atlases before the novel image is observed, extracting information that can guide later segmentation or reduce computational cost. Surveyed approaches learn regions of interest, intensity-label models, classifiers, atlas reliability, or image-space structures.

  • Offline learning analyzes atlas data before observing the novel image, unlike classical MAS, which performs little or no prior atlas processing.
  • Learned rough regions of interest can constrain or guide subsequent processing and reduce computational cost during novel-image segmentation.
  • Likelihood models can link image intensities to labels and improve segmentation, but may degrade performance when atlas and novel-image intensities are poorly matched.
  • Atlas reliability can be estimated from cross-registered label agreement, while learned training-image structures support efficient distances, registrations, and label propagation.

2.3. Registration

Registration establishes spatial correspondence between each atlas and the novel image, enabling label propagation but imposing MAS’s main computational burden. Methods vary from deformable transformations to common-coordinate and patch-based strategies.

  • Registration: Registration establishes spatial correspondence by deforming images to optimize alignment while enforcing plausible transformations.Its three components are the deformation model, objective function, and optimizer.
  • Registration: MAS typically computes an independent registration between each atlas and the novel image, using generic intensity-based registration tools.
  • Registration: Common-coordinate approaches co-register atlases offline, then concatenate template-to-novel and atlas-to-template transformations for label propagation.
  • Registration: Patch-based correspondence can improve MAS performance by relaxing one-to-one correspondence or reducing dependence on highly accurate registration.With rough alignment, local-neighborhood searches can remain computationally efficient, and anatomical context can improve patch matches.

2.4. Atlas Selection

Atlas selection reduces the number of atlases used in MAS to address computational cost and potentially exclude irrelevant training images. Selection methods range from basic similarity ranking to manifold-based approaches, while the optimal atlas count remains underexplored.

  • Atlas Selection: Atlas selection improves computational efficiency and may improve accuracy by excluding irrelevant atlases, but its impact depends on the algorithm and application.
  • Atlas Selection: A typical MAS algorithm’s demand is at least linear in the number of utilized atlases, so selecting half may about double speed and halve memory requirements.
  • Atlas Selection: Selection effectiveness depends on registration: registering all atlases can inform selection but provides no registration savings, whereas cheaper selection may be less informed.
  • Atlas Selection: Some methods iterate between segmentation and atlas selection, pruning or adding atlases according to the current segmentation estimate.
  • Atlas Selection: Selection criteria include intensity similarity, metadata, registration consistency, deformation, and learned image-manifold structure.Manifold-based measures add complexity but can outperform standard similarity measures in atlas selection.
  • Atlas Selection: The optimal number of selected atlases remains an overlooked research topic and is commonly chosen heuristically or through empirical sampling strategies.

2.5. Label Propagation

After atlas selection and registration, MAS propagates atlas labels into novel-image coordinates. Nearest-neighbor interpolation is widely used, while signed distance maps and boundary transformations provide alternative representations.

  • Label Propagation: Classical MAS propagates selected atlas labels to novel-image coordinates after establishing spatial correspondence.
  • Label Propagation: Nearest-neighbor interpolation remains one of the most popular propagation strategies, transferring a single label from each atlas to every novel-image voxel.
  • Label Propagation: Signed distance maps assign positive values inside a structure, negative values outside, and magnitudes proportional to the closest boundary distance.They encode uncertainty near label boundaries and relative confidence deeper within regions.
  • Label Propagation: Because signed distance maps are not naturally normalized, they can be converted into label probabilities through mappings such as the logistic mapping.
  • Label Propagation: Some methods transform atlas label boundaries directly instead of applying volumetric warps to the images.

2.6. Online Learning

Online learning extends MAS beyond direct label fusion by using registered atlas–image relationships and current segmentation estimates. Approaches iteratively refine atlas selection, model intensity-label relationships, or expand the atlas set with automatically segmented images.

  • Online Learning: Online-learning MAS methods exploit relationships between registered atlases and novel images to improve performance beyond direct label fusion.
  • Online Learning: Some methods iteratively update atlas selection or registration using the current segmentation estimate, excluding outlier atlases from fusion.
  • Online Learning: Intensity-label models can refine propagated labels through conditional Gaussian models or non-parametric density estimators.This strategy can be counterproductive when atlas and novel-scan intensities are poorly matched.
  • Online Learning: A discriminative approach builds local L1-regularized support vector-machine classifiers from registered-atlas intensities and labels, implicitly performing label fusion.
  • Online Learning: LEAP learns a manifold over training and novel images, automatically segments nearby novel images, adds them to the atlas list, and repeats the procedure.

2.7. Label Fusion

Label fusion combines propagated atlas labels using increasingly sophisticated similarity-, optimization-, patch-, and probabilistic approaches. Methods improve on simple voting but may incur high computational cost and retain unresolved choices about local weighting.

  • Early label fusion used best-atlas selection or majority voting to combine propagated labels.
  • Global weighting based on atlas-to-image similarity evolved toward local and semi-local weighting to model spatially varying registration accuracy.
  • Local fusion weights can use similarity ranks, multi-scale agreement, or correlations among atlases, improving over global weights while leaving the optimal local metric unclear.
  • Patch-based methods estimate voxelwise fusion weights from local image neighborhoods, including multi-channel MRI information, and have produced state-of-the-art accuracy often at high computational cost.
  • Alternative learning-based formulations estimate weights through sparse reconstruction or matrix completion rather than direct patch similarity.
  • Probabilistic and optimization-based formulations encode atlas agreement alongside priors such as spatial or temporal smoothness.

2.8. Post-processing

Label fusion is not always the final MAS output: some methods use it to initialize later segmentation steps, while others apply substantial classifier-, level-set-, graph-, or probabilistic refinement.

  • Post-processing may use label fusion to initialize bounding-box selection, active-contour evolution, or smooth contour fitting.
  • Other methods refine fusion outputs with error-correction classifiers or voxelwise techniques based on level sets, random forests, support vector machines, patches, or graph cuts.
  • Fusion outputs can also provide priors for probabilistic segmentation algorithms or be processed as posterior probability maps rather than hard-thresholded labels.
  • In anomalous regions such as tumors, posterior-map processing has been reported as beneficial.

3. Survey of Applications

MAS has been applied across a broad range of biomedical segmentation problems, with brain MRI as the most prevalent application area and substantial use in other organs, modalities, and populations.

  • Since its initial use in bee-brain confocal microscopy, MAS has been applied to diverse biomedical segmentation problems, especially brain MRI.
  • Brain MRI studies commonly parcellate cortical and subcortical regions in structural T1-weighted images, including whole-brain segmentation.
  • Applications also include brain-MRI preprocessing, tumors, eyes, optic nerves, and fetal, neonatal, infant, and animal imaging.
  • Outside the brain, MAS has been used for prostate, heart, breast, knee, spine, and coronary-OCT segmentation across MRI, CT, X-ray, ultrasound, angiography, and OCT.

4. Discussion and Future Directions

The survey describes MAS as an accurate but computationally demanding framework whose future depends on improved efficiency, training data, robustness, probabilistic modeling, and evaluation across applications.

  • MAS has evolved from registration, label propagation, and majority voting into multi-step systems using optimization, computer vision, machine learning, and probabilistic modeling.
  • Computational demand remains a major shortcoming because processing all atlases requires substantial memory and time, although parallel hardware and GPUs can reduce the burden.
  • Atlas number, quality, and annotation consistency critically affect segmentation accuracy, motivating active learning, quality control, crowdsourcing, and heterogeneous atlas sets.
  • Probabilistic approaches can model missing or heterogeneous labels, variable modalities, uncertainty, and domain knowledge, while registration marginalization offers a high-accuracy alternative at potentially prohibitive cost.
  • Simultaneously segmenting multiple novel images may improve results, particularly for serial scans and longitudinal analysis.
  • Most applications remain in human brain MRI, whereas registration is less effective in settings such as abdominal imaging with sliding organ walls; robust intensity handling is also needed for multi-site studies.
  • No segmentation algorithm is universally optimal; MAS offers varied tradeoffs, but computational efficiency, accuracy, realistic modeling, and proper evaluation remain central considerations.
Loading 1412.3421v2…