Source-linked AI summary
Micro-Net: A unified model for segmentation of various objects in microscopy images
Shan E Ahmed Raza, Linda Cheung, Muhammad Shaban, Simon Graham, David Epstein, Stella Pelengaris, Michael Khan, Nasir M. Rajpoot
TL;DR
Microscopy pipelines need reliable segmentation and localization of structures such as cells, nuclei, and glands across image modalities. The paper introduces Micro-Net, a CNN using multi-resolution inputs, intermediate connections, and max-pooling bypass layers, and reports that it outperforms recently published deep-learning approaches while producing competitive results on public datasets.
Problem
Automated microscopy analysis requires segmentation of structures such as cells, nuclei, and glands across fluorescence and histology images.
Method
Micro-Net is a CNN framework that uses multi-resolution processing and intermediate connections, with extra layers bypassing max-pooling to retain context and localization information.
Results
Micro-Net's qualitative and quantitative results outperform recently published deep-learning approaches and produce competitive results on publicly available datasets.
Takeaways & Limitations
The same Micro-Net architecture can segment cells, nuclei, and glands in fluorescence microscopy and histology images after slight tuning of input parameters.
Takeaways & Limitations
Gland segmentation remains constrained at patch boundaries, where the method misses some glands and needs improvement.
Abstract
from arXiv · showhide
Object segmentation and structure localization are important steps in automated image analysis pipelines for microscopy images. We present a convolution neural network (CNN) based deep learning architecture for segmentation of objects in microscopy images. The proposed network can be used to segment cells, nuclei and glands in fluorescence microscopy and histology images after slight tuning of input parameters. The network trains at multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi-resolution deconvolution filters. The extra convolutional layers which bypass the max-pooling operation allow the network to train for variable input intensities and object size and make it robust to noisy data. We compare our results on publicly available data sets and show that the proposed network outperforms recent deep learning algorithms.
1. Introduction
Micro-Net is proposed as a unified CNN framework for segmenting cells, nuclei, and glands across fluorescence and histology images. Its multi-resolution processing and max-pooling bypass layers preserve context and weak features, supporting robust localization under variable appearance and noise.
- Architecture: The architecture learns image features at multiple input resolutions to improve tissue-component understanding and accommodate variable object sizes and shapes.Multi-resolution training is described as addressing variable intensities and texture.
- Architecture: Additional layers bypass max-pooling to retain weak features that pooling might discard, preserving context and localization across resolutions.The network uses these intermediate connections to retain information from weak boundaries and features.
- Ablation evidence: Ablation results show improved accuracy and loss when Micro-Net retains multi-resolution input and bypass layers compared with Micro-Net–.Micro-Net– removes both design components while keeping the rest of the architecture unchanged.
- Qualitative comparison: On a noisy lung H & E image, U-Net segmented most cellular regions, whereas Micro-Net successfully located nuclei despite high noise.The passages attribute this qualitative difference to learning context at multiple resolutions and handling weaker boundaries.
- Motivation and scope: Micro-Net targets segmentation of nuclei, cells, and glands across histology and fluorescence microscopy modalities.The framework is intended to support different applications through slight retraining or input-parameter tuning.
- Evaluation: The paper reports detailed robustness results under noise and comparative evaluations against state-of-the-art deep-learning methods and challenge datasets.The analysis includes the MICCAI 2017 CPM nuclear-segmentation challenge dataset and publicly available datasets.
2. Data Sets and Challenges
The paper evaluates segmentation across multiplexed fluorescence, tumour histology, and colon histology datasets, each presenting substantial variation in object appearance and structure. These challenges include variable intensity, neighbouring or fused objects, heterogeneous nuclei, and diverse gland morphology.
- Datasets: Three datasets cover cell segmentation in multiplexed fluorescence, nuclear segmentation across four tumour types, and gland segmentation in colon cancer histology.The datasets include fluorescence images, H&E-stained histology images, and the Warwick-QU dataset from the GLaS challenge.
- Datasets: The proposed network is intended to handle diverse datasets and segmentation tasks.The paper demonstrates capability across cell, nuclear, and gland segmentation settings.
- 2.1. Multiplexed Fluorescence Imaging Data: Fluorescence cell segmentation is difficult because cells vary in intensity, shape, size, packing, and separation, while nuclei may be absent or off-centre.Signal-to-noise variation also arises from imaging and experimental conditions, requiring robustness to variable intensity and fused cellular regions.
- 2.3. Gland Segmentation (GLaS) Challenge Data Set: Automatic gland segmentation is difficult because gland texture, size, and structure vary substantially, especially in malignant tissue.The Warwick-QU dataset contains 165 images with expert-pathologist ground truth and includes benign and malignant cases.
3. The Proposed Network
Micro-Net is a multi-resolution CNN architecture for microscopy segmentation that combines downsampling and upsampling paths with bypassed convolutional features. Its design preserves localization and context while supporting variable object sizes, intensities, and noisy inputs.
- Input and overall architecture: Micro-Net accepts fluorescence marker inputs or stain-normalized RGB histology images, followed by input batch normalization.Fluorescence inputs use membrane and nuclear marker images; H&E inputs use stain-normalized RGB images.
- Input and overall architecture: The five-group architecture uses thirteen branches to organize convolution, pooling, resizing, concatenation, and multi-resolution prediction operations.The groups are divided according to their functions and layer/filter sets.
- Downsampling path: Convolution-only bypass features preserve information discarded by max-pooling, helping retain tiny details and features near noisy neighborhoods.The bypass path is motivated by retaining boundary details and other features that do not produce maximum responses.
- Upsampling path: Skip connections from downsampling branches are added during upsampling to improve localization and capture context, while branch outputs double spatial dimensions and halve feature depth.Each upsampling branch combines the preceding branch with a downsampling feature at a closely matching resolution.
- Output generation: Additional deconvolution layers produce segmentation maps matching the input size, eliminating the need for overlap-tile processing and reducing the number of required patches.The architecture therefore removes computational steps associated with producing the desired segmentation output.
- Output generation: Auxiliary outputs from intermediate upsampling branches are combined with the main output through convolution and softmax classification.Three auxiliary feature masks are generated, resized to a common output size, and fed into the main output branch.
- Training strategy: Training uses weighted cross-entropy with higher penalties at merging cell boundaries, while auxiliary-loss contributions decay exponentially with training progress.The total loss is l = l_o + (l_a1 + l_a2 + l_a3)/epoch, where epoch counts completed training passes.
4. Results and Discussion
Experiments evaluated Micro-Net against established segmentation networks across microscopy datasets and noise levels. Micro-Net generally achieved stronger segmentation and robustness, while gland results revealed errors near patch and image boundaries.
- Cell segmentation: On fluorescence cell images, Micro-Net handled variable cell intensities and sizes or shapes better than the compared architectures.FCN8 identified cellular regions without separating individual cells, while U-Net missed cells with weak boundaries and DCAN struggled with smaller endocrine cells.
- Cell segmentation: 3–4%: Micro-Net exceeded state-of-the-art approaches in average Dice, F1 score, object Dice, pixel accuracy, and object Hausdorff.The comparison used post-processed outputs and the listed quantitative measures.
- Ablation: Removing multi-resolution input and bypass layers produced results slightly better than U-Net but worse than Micro-Net, supporting the proposed design changes.The ablated model did not match the full architecture.
- Noise robustness: At increasing noise, Micro-Net retained the top position for F1 score and object Dice, while overall performance remained strongest across the evaluated metrics.The reported metrics included Dice, F1 score, object Dice, pixel accuracy, and object Hausdorff.
- Noise robustness: Object Hausdorff increased by 6 points for Micro-Net, compared with 12 points for DCAN and 55 points for U-Net as noise increased.The smaller increase was reported as evidence of greater stability and noise robustness.
- Noise robustness: Multi-resolution output supported robust shape similarity under noise, although Micro-Net showed the steepest decline in several other metrics.The decline concerned Dice, F1, object Dice, and pixel accuracy; object Hausdorff declined less.
- Gland segmentation: In gland segmentation, Micro-Net achieved competitive results, ranked third by rank-sum criteria, and outperformed some contest and recent deep-learning methods.It ranked best for F1 and object Dice on test A but lower on test B, while object Hausdorff ranked second on both test sets.
5. Conclusions
Micro-Net is presented as a unified deep-learning architecture for segmenting cells, nuclei, and glands across fluorescence and histology images with slight input tuning. Its multi-resolution design retains context and localization, handles weak and variable features, remains robust to fluorescence noise, and achieves competitive results against recent approaches.
- Scope: Micro-Net segments cells, nuclei, and glands in fluorescence and H&E-stained images after slight tuning of input parameters.The framework targets multiple object types and image modalities.
- Architecture: The architecture uses multiple input resolutions and intermediate connections to retain context and localization.Extra convolutional layers bypass max-pooling to preserve information for learning weak features alongside strongly observed features.
- Robustness: Micro-Net is robust to varying noise levels in fluorescence image data.The paper links this robustness to training on weak and variable features through the architecture's additional convolutional pathways.
- Evaluation: Qualitative and quantitative evaluations show that Micro-Net outperforms recently published deep-learning approaches and produces competitive state-of-the-art results.The paper compares the method using publicly available datasets and reports competitive outcomes against state-of-the-art methods.
- Applications: The authors propose extending the algorithm to multiplexed fluorescence images for molecular profiling, cancer grading, and tumour-microenvironment studies.The fluorescence image dataset is intended for public release subject to manuscript publication, while two other datasets are already publicly available.