Source-linked AI summary
Unsupervised Learning of Cell Instances with Generative Routing Pyramids
Ziwen Liu, Martin Weigert
TL;DR
Microscopy workflows for cell instance identification typically rely on supervised models requiring laborious annotations. This paper introduces an unsupervised routing-pyramid model that learns instance masks and morphological representations from unlabeled images, outperforming evaluated unsupervised baselines while modeling perturbation-associated phenotypes.
Problem
Supervised microscopy segmentation methods require laborious manual annotations, creating recurring dataset costs when specimens, modalities, or acquisition conditions change.
Method
An unsupervised coarse-to-fine routing-pyramid model reconstructs microscopy images while associating pixels with latent sources that yield instance masks and morphological representations.
Results
Across fluorescence and phase-contrast datasets, the method outperforms evaluated unsupervised baselines at IoU 0.5; representations and generated or retrieved cells recover perturbation-associated phenotypes.
Takeaways & Limitations
Recurring visual structure within an experiment can support learning cell instances and morphology without manual annotations.
Takeaways & Limitations
The method requires a separate model for each dataset and assumes compact, similarly appearing objects against a smoother background, leaving cross-sample generalization unexplored.
Abstract
from arXiv · showhide
Identifying and representing object instances such as cells or nuclei is a common task in microscopy image analysis. Established machine learning workflows typically use supervised detection or segmentation followed by feature extraction or classification, which requires manual annotations and treats instance segmentation and cell representation as separate stages. We describe a new unsupervised method for cell instance segmentation and phenotypic classification from unlabeled microscopy images. Our method is based on reconstructing each image using a coarse-to-fine routing pyramid that associates pixels with spatially sparse latent sources. The resulting pixel-to-latent associations yield instance masks, while the source latents encode cell morphology. We demonstrate competitive performance in instance segmentation across diverse cell morphologies and imaging modalities, as well as generative modeling of cellular phenotypes under perturbations. Source code and checkpoints are available at https://github.com/weigertlab/routing-pyramids.
1 Introduction
The paper introduces Generative Routing Pyramids, an object-centric generative model that learns cell instances and representations directly from unlabeled microscopy images. It combines shared appearance modeling with spatial assignments that preserve distinct object identities, producing instance masks and phenotype-oriented latent representations.
- Supervised cell segmentation methods require laborious manual annotation of large training-image datasets, making dataset creation a recurring cost.
- Label-free instance learning must combine shared appearance modeling with spatial assignments that keep visually similar adjacent cells associated with distinct sources.
- Generative Routing Pyramids is trained directly on unlabeled microscopy images and reconstructs each image from candidate objects using a single decoder pass.
- Foreground-presence penalties make the model activate a source only when its shared decoder can reconstruct a recurring object appearance.
- Pixel-to-source ancestries yield instance masks, while corresponding source latents provide object representations for single-cell phenotype analysis.
2 Related Work
Prior cell-instance methods rely on supervised mask-derived targets or unsupervised surrogate objectives, while representation-learning approaches generally separate segmentation from cell representation. The proposed method jointly learns instance segmentation and object representation through a coarse-to-fine generative routing hierarchy without instance annotations.
- Supervised instance segmentation: StarDist regresses radial distances for star-convex polygons, whereas Cellpose predicts pixel-space flows toward object centers; the proposed method uses neither flow targets nor instance annotations.StarDist applies nonmaximum suppression after polygon proposal generation, while Cellpose groups pixels by following flows over multiple steps.
- Unsupervised instance segmentation: Cellulus and CellSeg3D replace mask-derived targets with surrogate objectives, then obtain instances through clustering or post-processing.Cellulus learns dense spatial embeddings and uses mean-shift clustering with morphological filtering; CellSeg3D learns semantic foreground and separates instances with Voronoi-Otsu post-processing.
- Microscopy representation learning: Microscopy representation-learning methods produce dense features or learn from single-cell crops, but treat segmentation and representation learning as distinct stages requiring separate curation and training.Crop-based approaches assume prior detections or masks, while dense pixel- or patch-feature methods still require instance extraction for single-cell analysis.
- Coarse-to-fine generative models: A coarse-to-fine hierarchy defines spatial information flow in a single-image generative model and induces instance segmentation.The resulting spatial flow field is described as analogous to the object center-to-boundary flows of StarDist and Cellpose, but implicitly defined by the image.
3 Method
The method learns cell instances and object representations from unlabeled microscopy images by reconstructing images with a coarse-to-fine routing pyramid. Shared routing weights produce pixel-to-latent associations for instance masks, while foreground latents encode object morphology.
- Latent representation and gating: An encoder infers foreground and background latent grids plus a sparse presence map, and a pyramidal decoder reconstructs the image by routing local information across resolutions.Foreground and background candidates are sampled independently; the presence gate blends them before decoding, coupling which regions use foreground capacity.
- Pyramidal routing: Each decoder site selects among a valid 3 × 3 neighborhood using a masked softmax, defining locally supported transition probabilities on a directed acyclic graph.Routing edges connect destination sites to same-resolution or two-times-coarser parent sites, and each destination distributes unit assignment mass among valid parents.
- Pixel-to-seed associations: Composing layer-wise routing kernels yields a pixel-to-seed association matrix whose rows are full transition distributions from output pixels to latent seed sites.The same learned graph supports coarse-to-fine feature generation and fine-to-coarse ancestry interpretation, with routing flows allowed to split and merge.
- Inference and representations: At inference, posterior means replace sampled latents, thresholded presence sites are grouped by 8-connectivity into object seeds, and the same components define object-level phenotype embeddings.The presence threshold is τs = 0.5 in all experiments; each embedding is a presence-weighted average of foreground posterior means.
4 Experiments
Experiments evaluate annotation-free instance segmentation across diverse cell datasets and test whether object latents capture perturbation-induced phenotypes. Routing Pyramids outperform unsupervised baselines for segmentation and support accurate phenotype classification, generation, and retrieval.
- Instance segmentation: Experiments span three cell-culture datasets covering nuclear and whole-cell targets, fluorescence and phase-contrast imaging, and varied morphology and density.The datasets are Allen nuclear morphology, Fluo-N2DL-HeLa, and PhC-C2DL-PSC.
- Instance segmentation: Routing Pyramids achieve the best panoptic quality among unsupervised methods on all three datasets: 0.867, 0.800, and 0.518 versus Cellulus at 0.787, 0.756, and 0.370.The method is second only to supervised Cellpose-SAM in most settings.
- Phenotypic representation: Object latents form distinct clusters by perturbation dose in U2OS cells treated with LY294002 or wortmannin.PCA and UMAP are applied to embeddings from series-dilution and control images.
- Phenotypic representation: Thresholding average two-component GMM responsibilities at 0.5 recovers positive and negative classes with 100% accuracy for 16 control replicates.The GMM components are assigned identities by comparing their means with held-out positive and negative controls.
- Phenotypic representation: The learned GMM supports instance generation by sampling components and instance retrieval by nearest neighbors to component means, producing samples consistent with perturbation-induced phenotypes.Retrieved samples illustrate distinct morphologies associated with the learned latent clusters.
5 Conclusion
Generative Routing Pyramids learn cell instances and morphological representations from unlabeled microscopy images by reconstructing images through pixel-to-latent routing. The method outperforms evaluated unsupervised baselines, supports phenotype modeling under drug perturbations, and has clear limitations in generalization and scene complexity.
- Contribution: Generative Routing Pyramids learn cell instances and morphological representations from unlabeled 2D microscopy images.The method is an unsupervised object-centric model.
- Method: Pixel-to-latent routing through spatially sparse sources produces instance masks and object embeddings from the same image decomposition.The routing-pyramid decoder reconstructs each image while tracing pixels to latent sources.
- Results: Across fluorescence and phase-contrast datasets, the method outperforms evaluated unsupervised baselines at an IoU threshold of 0.5 with minimal post-processing.This performance is reported across diverse microscopy modalities.
- Results: In a two-channel fluorescence drug perturbation assay, mixture responsibilities align with treatment dose, while generated and retrieved cells recover corresponding phenotypes.These findings use learned object representations for phenotype modeling and retrieval.
- Limitations: The method currently requires a separate model for each dataset, assumes compact similarly appearing objects on smooth backgrounds, and remains unexplored for complex scenes such as tissue imaging.Generalization across diverse samples is left for future work.