Source-linked AI summary
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Yuyin Zhou, Zhe Li, Song Bai, Chong Wang, Xinlei Chen, Mei Han, Elliot Fishman, Alan Yuille
TL;DR
Partially labeled abdominal CT datasets can misclassify unlabeled organs as background, complicating multi-organ segmentation. PaNN incorporates empirical anatomical organ-size priors through a soft constraint and uses min-max stochastic primal-dual optimization, achieving an average Dice score of 84.97% on MICCAI2015.
Problem
Partially labeled abdominal CT datasets leave unlabeled organs marked as background, creating ambiguity for multi-organ segmentation needed in clinical applications.
Method
PaNN regularizes average predicted organ-size distributions toward empirical statistics from fully labeled data and optimizes the objective through min-max stochastic primal-dual gradients.
Results
84.97% average Dice was achieved on the MICCAI2015 “Multi-Atlas Labeling Beyond the Cranial Vault” abdominal organ-segmentation challenge.
Takeaways & Limitations
PaNN improves segmentation of organs beyond the partially labeled structures, consistent with anatomical organ-size priors contributing to multi-organ segmentation.
Takeaways & Limitations
Direct stochastic-gradient optimization is difficult because the logarithmic loss contains an average whose induced stochastic gradients are intrinsically biased.
Abstract
from arXiv · showhide
Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention. As data annotation requires massive human labor from experienced radiologists, it is common that training data are partially labeled, e.g., pancreas datasets only have the pancreas labeled while leaving the rest marked as background. However, these background labels can be misleading in multi-organ segmentation since the "background" usually contains some other organs of interest. To address the background ambiguity in these partially-labeled datasets, we propose Prior-aware Neural Network (PaNN) via explicitly incorporating anatomical priors on abdominal organ sizes, guiding the training process with domain-specific knowledge. More specifically, PaNN assumes that the average organ size distributions in the abdomen should approximate their empirical distributions, a prior statistics obtained from the fully-labeled dataset. As our training objective is difficult to be directly optimized using stochastic gradient descent [20], we propose to reformulate it in a min-max form and optimize it via the stochastic primal-dual gradient algorithm. PaNN achieves state-of-the-art performance on the MICCAI2015 challenge "Multi-Atlas Labeling Beyond the Cranial Vault", a competition on organ segmentation in the abdomen. We report an average Dice score of 84.97%, surpassing the prior art by a large margin of 3.27%.
1. Introduction
Partially labeled abdominal CT datasets create ambiguous background labels because unlabeled voxels may contain organs of interest. PaNN addresses this challenge by incorporating anatomical organ-size priors into multi-organ segmentation training.
- Abdominal multi-organ segmentation is difficult because organs have complex morphology and low soft-tissue contrast.
- Medical image annotations are expensive and require experienced radiologists plus additional expert checking, limiting high-quality labeled data.
- Partially labeled datasets, such as pancreas datasets, label only selected organs while marking the remainder as background.
- Naive semi-supervised training is misleading because it treats unlabeled pixels or voxels as background and does not exploit anatomical constraints.
- PaNN adds a soft prior-aware loss that regularizes average predicted organ-size distributions toward proportions estimated from fully labeled data.
2. Related Work
Prior medical-image segmentation methods often rely on sufficient labeled data or weak annotations but do not explicitly capture anatomical priors. PaNN instead embeds abdominal-region statistics into the training objective without additional training budget.
- Most advanced semantic-segmentation methods are supervised and therefore require sufficient labeled training data.
- Medical-image annotation is especially difficult because labeling must be performed by experts, making semi- or weakly supervised learning more realistic.
- Existing semi- or weakly supervised approaches use mechanisms such as pseudo-label refinement, adversarial costs, or bounding-box annotations.
- These methods fail to capture anatomical priors, although such priors are particularly relevant because medical structures are constrained in shape, location, and size.
- PaNN learns from partial annotations by embedding abdominal-region statistics in the training objective without additional training budget.
3. Prior-aware Neural Network
PaNN addresses partial supervision by combining fully labeled and partially labeled abdominal CT datasets with an anatomical size prior. It matches predicted organ-size distributions to empirical proportions and optimizes the resulting objective through min-max stochastic primal-dual training.
- 3.1. Partial Supervision: Partial supervision combines a fully labeled dataset with multiple datasets labeling only subsets of abdominal organs.Semi-supervision and full supervision are treated as extreme cases of this framework.
- 3.1. Partial Supervision: EM-like training can use partially labeled data but requires high-quality pseudo-labels and does not explicitly impose anatomical shape or size priors.The partially labeled data contain unlabeled background pixels that require pseudo-label estimation.
- 3.2. Prior-aware Loss: PaNN adds a soft penalty requiring average predicted organ-size distributions to approximate empirical proportions computed from fully labeled data.The prior-aware loss matches distributions through Kullback-Leibler divergence and acts as a soft global constraint.
- 3.2. Prior-aware Loss: The prior distribution q is fixed during training, so its entropy is constant and can be omitted from optimization.The predicted distribution is estimated by averaging pixel or voxel probability vectors across partially labeled datasets.
- 3.3. Derivation: Direct stochastic-gradient minimization is biased because the averaged predicted distribution occurs inside logarithmic loss rather than as a sum of independent terms.This motivates reformulating the KL-divergence objective as an equivalent min-max problem using auxiliary primal and dual variables.
- 3.4. Model Training: Training first initializes network weights on fully labeled data, then alternates pseudo-label estimation, dual-variable ascent, and network-weight descent on both data sources.The auxiliary variables are ν, µ, and YP; Θ is updated with standard stochastic gradient descent after they are fixed.
4. Experiments
Experiments evaluate PaNN on partially labeled abdominal CT datasets using cross-validation, organ-level analyses, qualitative examples, and the MICCAI 2015 challenge. PaNN generally improves segmentation performance over comparison methods, including difficult organs and settings with more partial data.
- Datasets and Evaluation Metric: The evaluation uses 5-fold cross-validation on 30 fully labeled CT cases, with 24 cases for training and 6 for testing in each fold.The fully labeled dataset contains 13 annotated abdominal structures; three partially labeled datasets target the spleen, pancreas, and liver.
- Datasets and Evaluation Metric: Dice-Sørensen Coefficient measures prediction–ground-truth voxel-set similarity, with average DSC reported across 13 anatomical structures and testing cases.The challenge evaluation additionally includes average surface distances and Hausdorff distances.
- Effect of PaNN: 1.28% and 1.69% gains over the naive partially supervised method are reported with ResNet-50 and ResNet-101, while 3D UNet gains 0.45%.PaNN achieves the best result when datasets A, B, and C are combined for both 2D and 3D models.
- Effect of PaNN: 1.56% improvement is obtained by PaNN when increasing partial datasets from one to their union, compared with 0.82% for semi-supervision and 0.51% for partial supervision.The reported ResNet-101 values change from 77.48% to 79.04% for PaNN, suggesting greater tolerance of background ambiguity with more partially labeled data.
- Organ-by-organ Analysis: Statistically significant improvements over the fully supervised method occur for almost every organ, with p < 0.001 for 11 of 13 organs.Reported gains include gallbladder from 67.26% to 72.26%, pancreas from 71.45% to 73.62%, and left adrenal gland from 49.51% to 53.63%.
- Organ-by-organ Analysis: PaNN improves organs beyond the partially labeled target liver, including gallbladder, stomach, IVC, and pancreas.The authors attribute this superiority not only to additional training data but also to incorporating anatomical priors on abdominal organ sizes.
5. Conclusion
PaNN is presented as a way to better use partially-labeled datasets for multi-organ segmentation by addressing background ambiguity with anatomical organ-size priors.
- PaNN regularizes network outputs so abdominal organ-size distributions approximate prior statistics.
A. Summary of Partially-labeled Datasets
The paper compiles partially-labeled datasets to support research on partially-supervised multi-organ segmentation.
- The collected datasets target different organs or subsets of abdominal organs rather than labeling every organ.
- The authors provide these datasets for researchers to explore partial supervision in multi-organ segmentation.
- Table 4 summarizes the partially-labeled datasets and uses N to denote the number of annotated cases.
B. Qualitative Evaluation
Qualitative results show improved segmentation regions across several abdominal organs compared with fully supervised training.
- The qualitative examples show evident improvements for the pancreas, gallbladder, left adrenal gland, stomach, and portal and splenic veins.
C. Generalization to Other Datasets
PaNN is evaluated beyond the main results on additional datasets and qualitative cases, where the reported comparisons favor the proposed method.
- Table 5 reports that PaNN achieves the best performance compared with existing methods on a newly collected high-quality abdominal dataset.
- Figure 5 compares methods using all three partially-labeled datasets A, B, and C with ResNet-101 as the backbone.
- The qualitative comparison presents five cases across six slices and zooms improved regions to show finer details.