Source-linked AI summary
Cell Detection with Star-convex Polygons
Uwe Schmidt, Martin Weigert, Coleman Broaddus, Gene Myers
TL;DR
Crowded-cell microscopy challenges pixel-grouping and bounding-box methods because they can merge bordering cells or suppress valid instances. StarDist predicts star-convex polygons and object probabilities per pixel, selecting instances with non-maximum suppression. The method performs strongly on crowded and fluorescence-microscopy datasets, while its 32-direction parametric shapes are less accurate at the largest IoU thresholds.
Problem
Crowded microscopy images can cause pixel-grouping methods to merge cells and bounding-box methods to suppress valid instances because boxes poorly approximate nuclei.
Method
StarDist predicts a star-convex polygon and object probability for every pixel, then applies non-maximum suppression to obtain cell instances.
Results
StarDist performs well on crowded nuclei and outperforms competing methods on challenging fluorescence microscopy images, including for IoU thresholds τ < 0.75 on DSB2018.
Takeaways & Limitations
Star-convex polygons provide accurate nucleus localization without refinement, while StarDist remains competitive with Mask R-CNN using an order of magnitude fewer parameters.
Takeaways & Limitations
StarDist yields inferior results at the largest synthetic-dataset IoU thresholds because it uses only 32 radial directions rather than per-pixel segmentation.
Abstract
from arXiv · showhide
Automatic detection and segmentation of cells and nuclei in microscopy images is important for many biological applications. Recent successful learning-based approaches include per-pixel cell segmentation with subsequent pixel grouping, or localization of bounding boxes with subsequent shape refinement. In situations of crowded cells, these can be prone to segmentation errors, such as falsely merging bordering cells or suppressing valid cell instances due to the poor approximation with bounding boxes. To overcome these issues, we propose to localize cell nuclei via star-convex polygons, which are a much better shape representation as compared to bounding boxes and thus do not need shape refinement. To that end, we train a convolutional neural network that predicts for every pixel a polygon for the cell instance at that position. We demonstrate the merits of our approach on two synthetic datasets and one challenging dataset of diverse fluorescence microscopy images.
1 Introduction
Crowded-cell microscopy makes both pixel-grouping and bounding-box detection vulnerable to merging distinct cells or suppressing valid instances. StarDist addresses these problems by predicting star-convex polygons that can localize nuclei without shape refinement.
- Pixel classification followed by grouping can fuse bordering but distinct nuclei when only a few pixels are misclassified.
- Bounding-box methods use non-maximum suppression, which can suppress valid nuclei when overlapping boxes poorly represent their shapes.
- StarDist predicts star-convex polygons flexible enough to compete with instance segmentation localization accuracy without refinement.
- The method uses a lightweight U-Net-based neural network and predicts object probabilities and polygon parameters for each pixel.
- Experiments evaluate the approach on two synthetic datasets and a challenging fluorescence-microscopy dataset, including crowded nuclei.
2 Method
StarDist predicts a star-convex polygon and object probability at every pixel, then uses non-maximum suppression to select final object instances. Its training combines object-probability classification with probability-weighted polygon-distance regression.
- For each pixel, the model regresses distances to an object boundary along n predefined radial directions, forming a star-convex polygon.
- The model separately predicts each pixel’s object probability and uses sufficiently probable pixels to generate polygon candidates.
- Greedy non-maximum suppression retains polygons with the highest object probabilities among candidates above a probability threshold.
- Object probability is defined as normalized Euclidean distance to the nearest background pixel, favoring candidates near cell centers during suppression.
- A simple U-Net predicts the object-probability and polygon outputs, with an added convolutional layer separating their feature demands.
- Training uses binary cross-entropy for object probabilities and object-probability-weighted mean absolute error for polygon distances.
3 Experiments
Experiments compare StarDist with U-Net and Mask R-CNN across synthetic and real microscopy datasets using average precision at multiple IoU thresholds. StarDist handles crowded or overlapping cells well and outperforms competing methods on DSB2018, while its parametric shape model is less precise at the highest synthetic-data IoU thresholds.
- Datasets: The evaluation uses three datasets with distinct cell-detection challenges, including synthetic touching objects and real fluorescence microscopy images.Each dataset uses 90% of images for training and 10% for testing; methods share training crops and augmentation.
- Evaluation: Average precision APτ is computed from true positives, false positives, and false negatives defined by an IoU threshold τ.A prediction matches a ground-truth object when their intersection over union exceeds τ.
- Results: On Dataset Toy, StarDist and both U-Net baselines are essentially perfect for τ < 0.7, whereas Mask R-CNN performs substantially worse because overlapping axis-aligned boxes suppress touching objects.The result exposes a limitation of bounding-box-based detection for slanted and touching shapes.
- Results: On TRAgen, U-Net (2 class) has the lowest accuracy because touching cells are erroneously fused, while the other methods achieve almost perfect accuracy at many IoU thresholds.The dataset’s simulated cells have stereotypical size and texture, which may contribute to the strong performance.
- Results: On DSB2018, StarDist outperforms all other methods for τ < 0.75, with Mask R-CNN the next-best method.Observed errors include merged cells for U-Net, bounding-box artifacts for Mask R-CNN, and missing cells across methods.
- Limitations: StarDist is inferior at the largest IoU thresholds on the synthetic datasets because it predicts a shape model from only 32 radial directions rather than per-pixel segmentations.The parametric representation can nevertheless produce complete shape hypotheses for partially visible boundary nuclei.
4 Discussion
The discussion presents star-convex polygons as an effective representation for localizing nuclei in challenging and crowded images. StarDist also offers simpler training and use than Mask R-CNN and may support cell-tracking scenarios requiring diverse or partially visible shape hypotheses.
- Discussion: StarDist localizes cell nuclei accurately under challenging conditions and is especially appealing for images containing very crowded cells.Its errors are described as graceful when it omits a cell or predicts a plausible shape.
- Discussion: Compared with Mask R-CNN, StarDist has an order of magnitude fewer parameters and few hyper-parameters that require little tuning.The authors describe it as simpler to train and use while remaining competitive with Mask R-CNN.
- Discussion: StarDist could benefit cell tracking by retaining multiple diverse polygon hypotheses and plausibly completing shapes for cells partially visible at image boundaries.These properties could help track cells entering or leaving the field of view over time.