Source-linked AI summary
Automated Design of Deep Learning Methods for Biomedical Image Segmentation
Fabian Isensee, Paul F. Jäger, Simon A. A. Kohl, Jens Petersen, Klaus H. Maier-Hein
TL;DR
Biomedical segmentation pipelines require difficult, dataset-specific expert design, limiting out-of-the-box use. nnU-Net automates pipeline adaptation and achieves state-of-the-art performance across most evaluated tasks without manual intervention.
Problem
Designing biomedical segmentation pipelines requires extensive expertise because dataset properties vary substantially and small configuration errors can sharply reduce performance.
Source: §1 · p. 2Method
nnU-Net systematically generates dataset-specific pipeline configurations from standardized dataset properties using explicit, interpretable heuristic rules.
Source: §2 · p. 3, §3 · p. 11Results
nnU-Net establishes state-of-the-art performance on most evaluated tasks and outperforms specialized segmentation pipelines across diverse datasets without manual adaptation.
Source: §1 · p. 3, §2 · p. 6, §3 · p. 11Takeaways & Limitations
nnU-Net provides a standardized, out-of-the-box baseline for 3D segmentation and enables systematic evaluation of methodological modifications across datasets.
Source: §3 · pp. 11–12Automatic adaptation may be suboptimal for tasks requiring domain-specific metrics, unconsidered dataset properties, or specialized preprocessing and loss functions.
Source: §3 · p. 12
Abstract
from arXiv · showhide
Biomedical imaging is a driver of scientific discovery and core component of medical care, currently stimulated by the field of deep learning. While semantic segmentation algorithms enable 3D image analysis and quantification in many applications, the design of respective specialised solutions is non-trivial and highly dependent on dataset properties and hardware conditions. We propose nnU-Net, a deep learning framework that condenses the current domain knowledge and autonomously takes the key decisions required to transfer a basic architecture to different datasets and segmentation tasks. Without manual tuning, nnU-Net surpasses most specialised deep learning pipelines in 19 public international competitions and sets a new state of the art in the majority of the 49 tasks. The results demonstrate a vast hidden potential in the systematic adaptation of deep learning methods to different datasets. We make nnU-Net publicly available as an open-source tool that can effectively be used out-of-the-box, rendering state of the art segmentation accessible to non-experts and catalyzing scientific progress as a framework for automated method design.
1 Introduction
Biomedical image segmentation is essential but difficult to adapt because dataset and hardware conditions vary, while expert-driven design is costly, poorly reproducible, and often suboptimal. nnU-Net addresses this by automatically generating high-quality pipelines from dataset fingerprints using heuristic rules, enabling out-of-the-box segmentation and strong challenge performance without manual intervention.
- Semantic segmentation supports scientific discovery and clinical applications, including diagnostic support, therapy planning, intra-operative assistance, and tumor growth monitoring [28].Source: §1 · pp. 1–2
- Designing segmentation pipelines is difficult because biomedical datasets, hardware, architectures, training procedures, and hyperparameters vary widely, making trial-and-error development expertise-intensive, costly, and hard to reproduce [22].Source: §1 · p. 2
- The authors hypothesize that a suitably designed basic U-Net remains difficult to beat despite approximately 12000 studies citing the 2015 architecture [31].Source: §1 · p. 2
- nnU-Net represents datasets with data fingerprints and pipelines with pipeline fingerprints, then uses heuristic rules to generate high-quality configurations while accounting for hardware constraints.Source: §1 · p. 2
- By applying systematic rules to previously unseen datasets without further optimization or manual adaptation, nnU-Net delivers out-of-the-box state-of-the-art segmentation and serves as an open-source reference method [22].The paper also examines the KiTS challenge to show that task-specific design and configuration can matter more than choosing among architectural extensions.Source: §1 · pp. 2–3
2 Results
Across 49 segmentation tasks, nnU-Net outperformed most specialized pipelines and achieved state-of-the-art performance in 29 tasks. Its automated, dataset-specific configuration and broad evaluation also show that hyperparameter design and multi-dataset testing are more reliable than architectural variation alone.
nnU-Net automatically adapts to any new dataset
- nnU-Net adapts automatically by deriving inferred parameters from dataset fingerprints, testing three U-Net configurations with cross-validation, and selecting the best configuration or ensemble.Heuristic rules handle co-dependent choices such as spacing, patch size, topology, and batch size, while empirical postprocessing is applied when beneficial.Source: §2 · p. 3, Figure 2 · p. 5
nnU-Net handles a wide variety of target structures and image properties
- Across 19 datasets and 49 tasks spanning MRI, CT, and EM, nnU-Net handled diverse organs, tumors, lesions, and cellular structures without task-specific re-optimization.The independent test datasets were used without further optimization after development on the Medical Segmentation Decathlon training sets.Source: Figure 1 · p. 4, §2 · pp. 5–6
nnU-Net outperforms specialized pipelines in a range of diverse tasks
- 29 of 49 target structures achieved a new state of the art, while nnU-Net otherwise performed on par with or close to top leaderboard entries.This occurred despite nnU-Net’s generic design, against pipelines specifically optimized for individual tasks.Source: §2 · p. 6, Figure 3 · p. 7
Details in pipeline configuration have more impact on performance than architectural variations
- On KiTS, top-performing methods used broadly similar U-Net-like architectures, whereas their scores and rankings varied widely, showing that configuration details matter more than architectural extensions.Residual, dense, attention, and dilated-convolution modifications were not necessary for good performance; plain 3D U-Net nnU-Net set a new open-leaderboard state of the art.Source: §2 · pp. 6, 8, Figure 4 · p. 8
Different datasets require different pipeline configurations
- Different datasets require different pipeline configurations because dataset properties interact through image size, patch size, network topology, and other hyperparameters.This dependence explains why settings optimized for one dataset may fail to generalize and require re-optimization elsewhere.Source: §2 · pp. 8–9, Figure 5 · p. 9
Multiple tasks enable robust design decisions
- The original nnU-Net configuration ranked first when aggregating results across ten datasets, while none of nine tested variants improved consistently across all tasks.Ranking volatility demonstrated that individual hyperparameter effects depend on the dataset, making evaluations on too few datasets unreliable for general conclusions.Source: §2 · pp. 9–10, Figure 6 · p. 10
3 Discussion
nnU-Net achieves state-of-the-art biomedical segmentation across diverse tasks by replacing manual pipeline optimization with explicit, interpretable heuristics and requiring no user intervention. Its standardized automation supports reproducible evaluation and strong baselines, while highly domain-specific tasks may still require manual modification.
Overall contribution
- nnU-Net sets a new state of the art in most evaluated tasks, outperforming specialized pipelines without introducing a new architecture, loss function, or training scheme.Its performance instead comes from systematically replacing manual pipeline optimization with explicit heuristic rules and zero user intervention.Source: §3 · p. 11
Interpretation of performance
- Across 19 datasets, performance variation among methods using the same network type indicates that hyperparameter configuration can outweigh architectural modifications and bias baseline comparisons.This supports the view that many proposed architectural extensions may not generalize beyond their original datasets.Source: §3 · p. 11
Research utility
- nnU-Net addresses current research bottlenecks by enabling methodological evaluation across arbitrary datasets and providing a standardized baseline without manual task-specific adaptation.It can therefore support large-scale comparisons and serve as a strong starting point for new 3D segmentation tasks.Source: §3 · pp. 11–12
Relation to automated optimization
- Unlike AutoML and grid search, nnU-Net encodes domain knowledge as inductive biases, avoiding task-specific optimization and reducing computational and data requirements.Heuristics developed from 10 Medical Segmentation Decathlon datasets generalized across the remaining challenge datasets, making hyperparameter selection more systematic and interpretable.Source: §3 · pp. 11–12
Limitations
- nnU-Net may be suboptimal when tasks use domain-specific metrics or dataset properties not covered by its heuristics, requiring extensions or manual modifications.For CREMI synaptic-cleft segmentation, it ranked 6/39, while manual loss-function adaptation and electron-microscopy preprocessing may be needed to exceed state-of-the-art performance.Source: §3 · p. 12
Conclusion
- Overall, nnU-Net generalizes strongly across 49 diverse tasks without manual intervention and offers an out-of-the-box state-of-the-art tool, evaluation framework, and comparison baseline.The discussion identifies hyperparameter optimization as a major difficulty that nnU-Net automates through a systematic procedure.Source: §3 · p. 12
Methods
nnU-Net automatically configures biomedical image-segmentation pipelines from dataset fingerprints, heuristic rules, hardware constraints, and training-time empirical optimization. Its fixed U-Net template is adapted through preprocessing, resolution, topology, memory, cascade, inference, model-selection, and postprocessing rules.
Dataset and pipeline fingerprints
- nnU-Net builds dataset and pipeline fingerprints, then infers essential configuration choices from image properties, modalities, classes, training-set statistics, and hardware constraints.The pipeline fingerprint combines heuristic, blueprint, and empirically optimized parameters.Source: Methods · p. 16
Blueprint parameters
- The architecture uses a common U-Net template with instance normalization, leaky ReLUs, deep supervision, strided convolutions, and capped feature-map widths instead of specialized architectural variants.Most 3D configurations use batch size 2; feature maps start at 32 and are capped at 320 for 3D and 512 for 2D U-Nets.Source: Methods · p. 16
Training schedule
- Training uses 1000 epochs, Nesterov SGD with µ = 0.99 and initial learning rate 0.01, poly decay, combined cross-entropy and Dice loss, foreground oversampling, and on-the-fly augmentation.Samples comprise 66.7% random patches and 33.3% foreground-containing patches, with rotations, scaling, noise, blur, intensity, resolution, gamma, and mirroring augmentations.Source: Methods · pp. 16–17
Inferred parameters
- Preprocessing normalizes intensities by modality, resamples heterogeneous voxel spacings with anisotropy-aware interpolation, and selects target spacing from training-set spacing statistics.Non-CT images use per-image z-scoring, whereas CT uses foreground percentile clipping and global foreground normalization; full-resolution spacing is generally the per-axis median, with a 10th-percentile adjustment for strongly anisotropic data.Source: Methods · pp. 17–18
Adaptation of network topology, patch size and batch size
- Network topology, patch size, and batch size are adapted to resampled image geometry, voxel spacing, and GPU memory by iteratively estimating feature-map memory and reducing patches until the budget is met.The adaptation prioritizes large patches, requires a minimum batch size of 2, adjusts axis-specific downsampling and kernels, and uses remaining memory to enlarge the batch when possible.Source: Methods · pp. 18–19
Configuration of 3D U-Net cascade
- A 3D U-Net cascade is triggered when the full-resolution patch covers less than 12.5% of the median image shape, using a low-resolution contextual model followed by full-resolution refinement.The low-resolution configuration increases spacing until its patch covers more than 25% of the current median image shape, and its prediction guides the second U-Net as an additional input channel.Source: Methods · p. 19
Empirical parameters
- nnU-Net selects one or two U-Net configurations by cross-validated foreground Dice, averages ensemble softmax probabilities, and benchmarks connected-component removal before accepting class-wise postprocessing.Inference uses half-overlapping sliding windows with Gaussian center weighting and mirroring-based test-time augmentation.Source: Methods · pp. 17, 19, 20
Reporting summary
Further information on the research design is available in the Nature Research Reporting Summary linked to the article.
- Further research-design information is provided in the linked Nature Research Reporting Summary.Source: Reporting summary · p. 20
A Dataset details · B nnU-Net Design Principles
The section describes the challenge datasets used in the manuscript and explains that their training-case statistics form the dataset fingerprints. It also introduces nnU-Net’s design principles conceptually, directing readers to the online methods for implementation details.
A Dataset details
- Dataset statistics are computed from the training cases for each dataset.Source: A Dataset details · p. 23
- These training-case statistics provide the basis for the dataset fingerprints shown in Figure 5.Source: A Dataset details · p. 23
- The dataset notes account for multiple annotated examples per training case and small differences between Decathlon Liver and its related dataset.Source: A Dataset details · p. 23
- The manuscript uses multiple challenge datasets whose overview and access references are provided in Table A.1.Source: Table A.1 · p. 23, A Dataset details · p. 23
- Table A.1 summarizes the challenge datasets used throughout the manuscript.Source: Table A.1 · p. 23
B nnU-Net Design Principles
- The nnU-Net design principles are presented as a brief conceptual overview.Source: B nnU-Net Design Principles · p. 24
- Detailed implementation information for these design guidelines is provided in the online methods.Source: B nnU-Net Design Principles · p. 24
B.1 Blueprint Parameters · B.2 Inferred Parameters · B.3 Empirical Parameters
nnU-Net combines a simple U-Net blueprint with dataset-driven parameter inference and validation-based empirical selection. This design adapts architectures, preprocessing, training, and postprocessing to each biomedical segmentation task.
B.1 Blueprint Parameters
- The blueprint uses plain convolutions, instance normalization, Leaky ReLUs, two blocks per resolution stage, strided-convolution downsampling, and transposed-convolution upsampling.The convolution–instance normalization–Leaky ReLU order is used throughout, while alternative downsampling and upsampling methods showed no substantial accuracy disparity.Source: §B.1 · p. 24
- nnU-Net generates three U-Net configurations and automatically selects the best-performing method or ensemble after cross-validation, while the 3D full-resolution U-Net shows overall best performance.Its patch size is limited by GPU memory and may be too small to aggregate sufficient contextual information for large data.Source: §B.1 · p. 24, §B.3 · p. 26
- Training runs for 1000 epochs of 250 iterations using SGD with learning rate 0.01, Nesterov momentum 0.99, polyLR decay, and on-the-fly probabilistic augmentation; shorter training empirically diminishes segmentation performance.Source: §B.1 · p. 24
B.2 Inferred Parameters
- nnU-Net configures parameters on-the-fly from each task’s data fingerprint, adapting network depth, receptive field, anisotropic pooling, and out-of-plane kernels to input size and spacing.Downsampling stops independently when an axis becomes too small, while anisotropic data initially uses in-plane pooling and effectively 2D out-of-plane convolutions.Source: §B.2 · p. 25
- Input patches are made as large as GPU memory permits with batch size 2, follow the median resampled shape, and expand batch size when memory remains.Target spacing uses median training-case spacing for isotropic data and the 10th percentile out-of-plane spacing for anisotropic data, with modality-appropriate interpolation.Source: §B.2 · p. 26
- Preprocessing defaults to per-image z-score normalization, except CT, which uses a foreground-based global scheme determined across training cases.Resampling uses third-order splines for data and linear interpolation for one-hot labels, while anisotropic out-of-plane resampling uses nearest neighbor.Source: §B.2 · p. 26
B.3 Empirical Parameters
- Validation performance determines empirical model choice and postprocessing: nnU-Net selects the best model or ensemble and retains largest-component filtering only when class-wise Dice improves.These decisions cannot be inferred reliably from the dataset fingerprint alone; connected-component analysis exploits tasks where the target structure usually has one instance.Source: §B.3 · p. 26
C Analysis of exemplary nnU-Net-generated pipelines · C.1 ACDC · C.2 LiTS
The exemplary pipelines show nnU-Net adapting preprocessing, resolution, architecture, training, and postprocessing to dataset-specific imaging properties. On ACDC and LiTS, automated configuration selected different normalization and multiscale strategies, with ensembles achieving the highest reported validation scores.
C Analysis of exemplary nnU-Net-generated pipelines
- The pipelines were automatically generated for ACDC and LiTS to illustrate how nnU-Net’s design principles respond to dataset-specific imaging characteristics.Figures C.1 and C.2 summarize the generated architectures for the two datasets.Source: C Analysis of exemplary… · p. 27, §C.1 · p. 27, §C.2 · p. 29
C.1 ACDC
- ACDC cine MRI has sparse, potentially misaligned slices, with a typical image shape of 9 × 237 × 256 voxels at 10 × 1.56 × 1.56 mm spacing.These properties motivate preserving heterogeneous out-of-plane resolution for the 2D model and adapting the 3D model to strong anisotropy.Source: §C.1 · pp. 27–28
- For ACDC, nnU-Net normalizes each MRI scan independently and configures anisotropy-aware 3D processing, using initial in-plane convolutions and pooling before switching to isotropic operations.The 3D target out-of-plane spacing is 5 mm, and in-plane augmentation avoids interpolation across slices; the 3D cascade is omitted because the full-resolution model covers the median image shape.Source: §C.1 · p. 28
- On ACDC, the ensemble of 2D and 3D full-resolution U-Nets achieved the highest score, 0.9228, versus 0.9165 and 0.9181 for the individual configurations.The ensemble was selected for test prediction, with largest-connected-component removal benefiting the right ventricle and left ventricular cavity.Source: §C.1 · p. 28
C.2 LiTS
- LiTS provides 131 annotated abdominal CT training cases with a median shape of 432 × 512 × 512 voxels and spacing of 1 × 0.77 × 0.77 mm.Its nearly isotropic spacing supports full three-dimensional processing and large contextual patches.Source: §C.2 · p. 29
- For LiTS, nnU-Net applies global foreground-based CT normalization, while the 3D U-Net uses isotropic kernels and strides with a 128 × 128 × 128 patch size and batch size 2.The method resamples isotropic and anisotropic images differently and prioritizes patch context over batch size.Source: §C.2 · pp. 29, 31
- LiTS uses a 3D cascade because the full-resolution patch covers only 1/60 of the resampled median image, risking overly zoomed-in predictions.A low-resolution U-Net first produces coarse segmentations, which a full-resolution U-Net refines using their one-hot encoding concatenated to the input.Source: §C.2 · p. 31
- On LiTS, the best two-model combination, combining the low- and full-resolution U-Nets, achieved a Dice score of 0.8111, exceeding every individual configuration.The individual scores were 0.7625 for 2D, 0.8044 for 3D full resolution, 0.7796 for 3D low resolution, and 0.8017 for the cascade full-resolution U-Net.Source: §C.2 · p. 31
D Details on nnU-Net’s Data Augmentation
nnU-Net uses a fixed, CPU-based, on-the-fly augmentation pipeline across datasets, adapting spatial operations to patch dimensionality and anisotropy. It also augments cascade segmentation masks with morphology and connected-component removal.
- Augmentations are computed on the fly by CPU background workers through batchgenerators, with identical pipeline parameters across datasets.The pipeline applies a variety of augmentations during training without dataset-specific parameter changes.Source: D Details on nnU-Net’s Data Augmentation · p. 31
- Spatial rotation, scaling, and low-resolution simulation operate in 3D for standard 3D patches but in 2D for 2D or anisotropic patches.A patch is anisotropic when its largest edge is at least three times its smallest; sampled patches are initially oversized and then center-cropped to reduce boundary padding.Source: D Details on nnU-Net’s Data Augmentation · pp. 31–32
- Augmentations use randomly sampled parameters and predefined probabilities, including joint rotation-scaling, noise, blur, brightness, contrast, low-resolution, gamma, and mirroring.Rotation and scaling are each triggered with probability 0.2, while other operations use operation-specific probabilities; scaling factors range from 0.7 to 1.4 and rotation ranges depend on patch geometry.Source: D Details on nnU-Net’s Data Augmentation · p. 32
- Low-resolution simulation downsamples selected modalities with nearest-neighbor interpolation and restores them with cubic interpolation, remaining 2D for 2D or anisotropic patches.It is applied with probability 0.25 per sample and 0.5 per associated modality, using a downsampling factor sampled from U(1, 2).Source: D Details on nnU-Net’s Data Augmentation · p. 32
- Gamma augmentation applies nonlinear voxelwise intensity transformations with γ ∼U(0.7, 1.5), optionally after intensity inversion, while mirroring occurs with probability 0.5 along all axes.Both transformations rescale intensities to and from their original value range.Source: D Details on nnU-Net’s Data Augmentation · pp. 32–33
- For cascade full-resolution U-Nets, predicted one-hot masks additionally undergo random binary morphology or removal of small connected components.Binary dilation, erosion, opening, or closing uses spherical radii sampled from U(1, 8) with probability 0.4, while components smaller than 15% of patch size are removed with probability 0.2.Source: D Details on nnU-Net’s Data Augmentation · p. 33
E Network Architecture Configuration · F Summary of nnU-Net Challenge Participations · F.1 Challenge Inclusion Criteria
nnU-Net configures dataset-specific architectures through memory-based iterative design and evaluates eligible 3D segmentation challenges using standardized, from-scratch cross-validation and test prediction. Challenges were selected to test robustness across datasets under explicit data, evaluation, and comparability criteria.
F Summary of nnU-Net Challenge Participations
- Manual intervention was sometimes required to format inputs or split related cases, especially multiple time points from the same patient, into mutually exclusive folds.The authors disclose interventions for each dataset and provide implementation guidance with the source code.Source: F Summary of nnU-Net Challenge… · p. 33
- For every dataset, nnU-Net trains applicable 2D, 3D full-resolution, 3D low-resolution, and 3D cascade configurations using five-fold cross-validation from scratch.Training uses only the challenge-provided training data, without pretraining or external training data.Source: F Summary of nnU-Net Challenge… · p. 33
- nnU-Net selects ideal configurations through cross-validation and ensembling, then uses the best configuration to predict the challenge test cases.Generated pipelines and detailed cross-validation and test-set results are provided for each dataset.Source: F Summary of nnU-Net Challenge… · p. 33
E Network Architecture Configuration
- Architecture configuration uses memory-consumption estimates to iteratively determine U-Net designs, completing all configurations in under one second.The workflow accounts for discontinuities caused by pooling and network depth, while low-resolution cascade models must cover at least one-quarter of the median resampled training-case shape.Source: Figure E.1 · p. 34
- Low-resolution cascade architectures are reconfigured separately because anisotropic downsampling and aspect-ratio-dependent patch sizes can make them differ from full-resolution U-Nets.Higher-resolution axes are downsampled first, potentially changing the data aspect ratio relative to full-resolution inputs.Source: Figure E.1 · p. 34
F.1 Challenge Inclusion Criteria
- Challenges were selected to maximize dataset diversity while requiring 3D semantic segmentation, available training cases, separate hidden-ground-truth test cases, and participant-result comparability.Comparability could be established through standardized evaluation, such as an online platform and public leaderboard.Source: §F.1 · p. 34
- The qualifying competitions were selected for nnU-Net evaluation, with CREMI 3 identified as the only biological-domain competition known to meet the stated criteria.The criteria permit images of any size and any 3D imaging modality.Source: §F.1 · pp. 34–35
F.2 Compact Architecture Representation … F.9 Combined Healthy Abdominal Organ Segmentation (CHAOS) (D16)
This section presents a compact, fully reconstructable representation of nnU-Net architectures and documents their manual-intervention-free application across diverse biomedical segmentation challenges, including the CHAOS MRI liver and multiorgan tasks. It also describes dataset-specific handling and reports the resulting CHAOS task scores.
F.2 Compact Architecture Representation
- nnU-Net architectures can be fully reconstructed from lists of convolutional kernel sizes and downsampling strides, avoiding an excessive number of figures.The decoder mirrors the encoder, while strides encode downsampling or transposed-convolution upscaling; auxiliary-loss outputs are added above the two lowest resolutions.Source: §F.2 · pp. 35–36
F.3 Medical Segmentation Decathlon
- The Medical Segmentation Decathlon covers ten tasks selected to represent broad medical-image variability, with separate development and unseen-dataset phases evaluated on test cases.The appendix provides generated configurations and results for BrainTumour, Heart, Liver, Hippocampus, Prostate, Lung, Pancreas, Hepatic Vessel, Spleen, and Colon.Source: §F.3 · p. 36, Table F.1 · p. 37, Table F.2 · p. 37
- The refined nnU-Net achieved first rank on the Medical Segmentation Decathlon, outperforming the initial nnU-Net and other methods that had held state of the art.The competition spans ten segmentation tasks, and nnU-Net was applied to all ten without manual intervention.Source: §F.3 · p. 36
F.4 Multi Atlas Labeling Beyond the Cranial Vault: Abdomen (D11)
- nnU-Net was applied to the BCV abdominal-organ challenge without manual intervention, which contains 30 training and 20 test CT images targeting thirteen organs [12].The reported BCV results use postprocessing for the selected model, and the best ensemble combines the 3D U-Net cascade with the 3D full resolution U-Net.Source: §F.4 · p. 43, Table F.22 · p. 44
F.5 PROMISE12 (D12)
- nnU-Net was applied without manual intervention to PROMISE12, which provides 50 annotated training cases and 30 online-evaluated prostate test cases.The test-set submission achieved an evaluation score of 89.6507; the best ensemble combined the 2D U-Net and 3D full resolution U-Net.Source: §F.5 · p. 44, Table F.24 · p. 45
F.6 The Automatic Cardiac Diagnosis Challenge (ACDC) (D13)
- For ACDC, nnU-Net used patient-exclusive cross-validation splits because each patient contributed enddiastolic and endsystolic images, while otherwise requiring no manual intervention.The challenge contains 100 training and 50 test patients, with three cardiac target structures segmented at two time points.Source: §F.6 · p. 45, Table F.26 · p. 46
F.7 Liver and Liver Tumor Segmentation Challenge (LiTS) (D14)
- nnU-Net was applied without manual intervention to LiTS, which provides 131 annotated training CT images and 70 unannotated test images evaluated online.The best ensemble combined the 3D low resolution U-Net and the 3D full resolution U-Net.Source: §F.7 · p. 46, Table F.28 · p. 47
F.8 Longitudinal multiple sclerosis lesion segmentation challenge (MSLesion) (D15)
- For MSLesion, patient-exclusive cross-validation treated the two expert annotations as separate images, producing 42 training images while ignoring longitudinal scan structure during training and inference.The submission score was 92.874, and the best ensemble combined the 2D U-Net and 3D full resolution U-Net.Source: §F.8 · p. 47, Table F.30 · p. 48
F.9 Combined Healthy Abdominal Organ Segmentation (CHAOS) (D16)
- The study focused on CHAOS Task 3 MRI liver segmentation and Task 5 MRI multiorgan segmentation, whose targets were the liver, spleen, and left and right kidneys.Each task provided 20 training cases, and Task 3 was a liver-only subset of Task 5.Source: §F.9 · p. 48
- nnU-Net used 60 training images by treating T1 in-phase and out-phase scans as separate examples, splitting cross-validation at patient level, and averaging their softmax predictions at inference.The model was trained only for Task 5, with the liver extracted from Task 5 predictions for Task 3.Source: §F.9 · p. 48
- 72.44 was achieved for Task 5 and 75.10 for Task 3 in the CHAOS test submission.The test set was evaluated online; individual-organ Dice scores were not reported.Source: Table F.32 · p. 49
F.10 Kidney and Kidney Tumor Segmentation (KiTS) (D17) · F.11 Segmentation of THoracic Organs at Risk in CT images (SegTHOR) (D18)
nnU-Net was evaluated without manual intervention on the KiTS and SegTHOR CT segmentation challenges, with KiTS using original non-interpolated data and achieving a new open-leaderboard state of the art over the prior manually optimized solution.
F.10 Kidney and Kidney Tumor Segmentation (KiTS) (D17)
- KiTS provided 210 training and 90 test cases for kidney and kidney-tumor segmentation, with both original and common-spacing image geometries.Test predictions were evaluated on the organizers’ online platform.Source: §F.10 · p. 49
- nnU-Net set a new state of the art on the KiTS open leaderboard, outperforming the initial manually optimized solution.The submission was evaluated after additional competitors had entered the open leaderboard, rather than only against the original challenge submissions.Source: §F.10 · p. 50
- For KiTS, nnU-Net used the original non-interpolated images and automatically homogenized voxel spacing without manual intervention.The training data reused a modified dataset because two faulty cases were replaced and four cases suspected of labeling errors were excluded.Source: §F.10 · pp. 49–50
- The KiTS results table reports five-fold cross-validation Dice scores for training cases and online-platform Dice scores for the test set.The test-set kidney score is not directly comparable because the platform evaluates the union of kidney and tumor labels, while nnU-Net evaluates labels independently; the best ensemble combined the 3D U-Net cascade and 3D full-resolution U-Net.Source: Table F.34 · p. 50
F.11 Segmentation of THoracic Organs at Risk in CT images (SegTHOR) (D18)
- SegTHOR evaluates segmentation of the heart, aorta, trachea, and esophagus using 40 training images and 20 test images.Test images were evaluated using the challenge’s online platform.Source: §F.11 · pp. 50–51
- nnU-Net was applied to SegTHOR without manual intervention, and the reported results used five-fold cross-validation for training cases and online-platform evaluation for test cases.The best SegTHOR ensemble combined the 3D U-Net cascade and the 3D full-resolution U-Net.Source: §F.11 · p. 51, Table F.36 · p. 51
F.12 Challenge on Circuit Reconstruction from Electron Microscopy Images (CREMI) (D19)
nnU-Net was applied to CREMI’s synaptic cleft segmentation task using an ensemble of five 3D full-resolution models trained on all three training volumes. The test submission achieved a CREMI score of 74.96, where lower is better.
Challenge summary
- The synaptic cleft task segments cell membranes at synapses in serial-section transmission electron microscopy scans of the Drosophila melanogaster brain.CREMI provides three training volumes and three test volumes, with test evaluation performed online.Source: §F.12 · p. 51
Application of nnU-Net to CREMI
- Because only three training volumes were available, five models were trained on all volumes and ensembled instead of using five-fold cross-validation.This left no validation data, preventing nnU-Net from selecting among three model configurations; only the 3D full-resolution configuration was trained.Source: §F.12 · pp. 51–52
- Each image was normalized independently by subtracting its mean and dividing by its standard deviation.Source: §F.12 · p. 52
Results
- 74.96 CREMI score was achieved on the test submission, with lower scores indicating better performance.The online evaluation reported no Dice score or validation-based performance estimate.Source: §F.12 · p. 52
G Using nnU-Net with limited compute resources
nnU-Net was designed to reduce computational complexity so that running its generated configurations remains manageable for most users and researchers, while offering shortcuts when resources are extremely scarce.
- Reducing computational complexity was a key motivation in nnU-Net’s design.Source: G Using nnU-Net with limited compute… · p. 52
- nnU-Net aims to make running all generated configurations manageable for most users and researchers.Source: G Using nnU-Net with limited compute… · p. 52
- When computational resources are extremely scarce, users can apply shortcuts to reduce the required effort.Source: G Using nnU-Net with limited compute… · p. 52
G.1 Reducing the number of network trainings · G.2 Reduction of GPU memory
nnU-Net offers configurable shortcuts that reduce training time while preserving user choice over accuracy, reliability, and ensemble size. Its default 11GB GPU requirement can also be adapted for smaller hardware.
G.1 Reducing the number of network trainings
- The full nnU-Net pipeline requires 10 or 20 U-Net trainings, each taking several days on one GPU, motivating strategies to reduce computation.The larger count applies when a 3D U-Net cascade is configured.Source: §G.1 · p. 52
- The 3D full resolution U-Net generally performs best and can be selected alone or trained with five-fold cross-validation and ensembling.Users may train one model on all cases or ensemble five cross-validation models for test predictions.Source: §G.1 · p. 52
- Other configurations can outperform the 3D full resolution U-Net in dataset-specific scenarios, but selecting them requires domain knowledge.Highly anisotropic images may favor 2D U-Net, while very large images may favor a cascade when the target requires a large receptive field; these relations are not guaranteed.Source: §G.1 · pp. 52–53
- Running one train-validation split per configuration reduces training to two models without a cascade or eight with a cascade, but weakens estimates and selection reliability.The cascade still requires five low-resolution models and one full-resolution model, and the selected configuration can later be retrained or ensembled with five-fold cross-validation.Source: §G.1 · p. 53
G.2 Reduction of GPU memory
- nnU-Net’s default configuration uses 11GB of GPU memory, a requirement tested extensively on modern deep-learning GPUs.The authors recommend retaining this default because it provides excellent segmentation accuracy.Source: §G.2 · p. 53
- Users can adapt nnU-Net’s network configuration to run on GPUs with less memory, with instructions supplied in the source code.The adaptation is presented as an option for users unable to meet the default 11GB requirement.Source: §G.2 · p. 53
Related papers
This paper cites
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift2015
- PyTorch: An Imperative Style, High-Performance Deep Learning Library2019
- Densely Connected Convolutional Networks2016
- Rethinking the Inception Architecture for Computer Vision2015
- Squeeze-and-Excitation Networks2017