Source-linked AI summary
Sparse Autoencoder for Unsupervised Nucleus Detection and Representation in Histopathology Images
Le Hou, Vu Nguyen, Dimitris Samaras, Tahsin M. Kurc, Yi Gao, Tianhao Zhao, Joel H. Saltz
TL;DR
Nucleus analysis in histopathology is valuable but costly to scale because expert annotation is labor-intensive and existing pipelines separate detection from representation learning. This paper introduces a crosswise-sparse CAE that jointly detects and represents nuclei without supervision and can be fine-tuned end-to-end. Across four datasets, it reduces state-of-the-art errors by up to 42% and achieves comparable performance with 5% of the fully supervised annotation cost.
Problem
Large-scale nucleus analysis is limited by the labor and expense of expert annotation and by pipelines that separate detection from representation learning.
Method
A crosswise-sparse CAE jointly performs unsupervised nucleus detection and feature representation, with pretrained modules usable for end-to-end supervised fine-tuning.
Results
The method reduces state-of-the-art errors by up to 42% and achieves comparable results using 5% of the training data.
Takeaways & Limitations
The CAE provides a unified unsupervised initialization for supervised nucleus classification and segmentation across four challenging datasets.
Takeaways & Limitations
On one dataset, improvement was less significant because the nucleus images came from a fixed detection method that could not be fine-tuned.
Abstract
from arXiv · showhide
Histopathology images are crucial to the study of complex diseases such as cancer. The histologic characteristics of nuclei play a key role in disease diagnosis, prognosis and analysis. In this work, we propose a sparse Convolutional Autoencoder (CAE) for fully unsupervised, simultaneous nucleus detection and feature extraction in histopathology tissue images. Our CAE detects and encodes nuclei in image patches in tissue images into sparse feature maps that encode both the location and appearance of nuclei. Our CAE is the first unsupervised detection network for computer vision applications. The pretrained nucleus detection and feature extraction modules in our CAE can be fine-tuned for supervised learning in an end-to-end fashion. We evaluate our method on four datasets and reduce the errors of state-of-the-art methods up to 42%. We are able to achieve comparable performance with only 5% of the fully-supervised annotation cost.
1. Introduction
Histopathology nuclear analysis is important but difficult to scale because manual annotation and separate detection pipelines require substantial expert effort. The proposed crosswise-sparse CAE jointly detects and represents nuclei without supervision, then supports end-to-end supervised learning with improved performance and reduced annotation requirements.
- Manual nuclear examination does not scale to large studies, while nuclear size, shape, and chromatin pattern support disease diagnosis and cell-type distinction.
- Existing semi-supervised approaches require expert-labeled data, and nucleus-centered training patches require a separate detection step that often needs tuning.
- The CAE unifies unsupervised nucleus detection and feature representation in one end-to-end trainable network, addressing the absence of unsupervised detection networks for visual applications.
- The model separates background from foreground, represents nucleus locations with sparse feature maps, and encodes each detected nucleus into a feature vector.
- Crosswise sparsity synchronizes foreground activations across feature maps, so reconstruction optimization simultaneously learns nucleus locations and nuclear features.
- 42% maximum error reduction was achieved on classification tasks, while segmentation error was reduced by 20% relative to U-net.
- Comparable results were achieved using 5% of the training data, and the pretrained modules can be fine-tuned for end-to-end supervised learning.
2. Crosswise Sparse CAE
The crosswise sparse CAE reconstructs tissue images as foreground and background, while its sparse foreground maps encode nucleus locations and appearance. A learned detection map synchronizes foreground activation across feature maps, and small receptive fields encourage one encoded vector per nucleus.
- The CAE reconstructs an image as the pixel-wise sum of foreground and background reconstructions, with sparse foreground maps representing detected nuclei and extracted features.Background maps need not be sparse.
- Background is encoded first, and the residual between reconstructed background and input forms the foreground containing nuclei.The method assumes background occupies most of the tissue image and varies at a larger scale than foreground structures.
- Crosswise sparsity requires neurons across all foreground feature maps to remain inactive at most spatial locations, forming a group-sparse representation.Only a few feature-map vectors therefore contribute to foreground reconstruction, encouraging them to represent salient nuclei.
- A binary detection map synchronizes activation across foreground feature maps, while dense feature maps are multiplied element-wise by it to produce sparse foreground maps.The detection map is learned by minimizing reconstruction error, so detection and feature extraction are optimized simultaneously.
- The threshold-based detector targets an expected sparsity rate that can vary across inputs, unlike fixed-k sparsification.The method uses p = 1.6 in experiments; threshold computation uses a percentile and running average, with α = 0.1.
- Reducing encoding neurons’ receptive fields makes each foreground vector cover a region close to one nucleus, supporting one-to-one nucleus-to-vector correspondence.Figure 3 specifies that the receptive field should contain only one nucleus in most cases.
3. Experiments
The method is evaluated through unsupervised reconstruction and downstream classification and segmentation experiments using four datasets, including self-collected and public histopathology data. Example reconstructions show foreground/background decomposition and nucleus detection, while the overall evaluation reports better results than other methods.
- The method is evaluated on four datasets spanning lymphocyte-rich region classification, individual lymphocyte classification, nuclear attribute classification, and nucleus segmentation.The datasets include self-collected lymphocyte data, an existing nuclear shape and attribute dataset, and the MICCAI 2015 challenge dataset.
- The proposed method achieves better results than other methods across the reported evaluations.
- The unsupervised-learning dataset contains 0.5 million unlabeled 100×100-pixel crops from 400 lung adenocarcinoma images in TCGA.Images were collected at 20X with 0.5 microns per pixel.
- Figure 4 displays detection maps and reconstructed foreground and background images, with the final reconstruction formed by pixel-wise summation.The examples illustrate decomposition of input images and nucleus representation in the foreground.
3.2. CAE Architecture
For segmentation, the CAE avoids pooling to preserve the spatial detail of small nuclei, while using reduced input dimensions and fewer feature maps to control computation.
- Classification architecture: All three classification CAEs use the same architecture illustrated in Fig. 2 and Table 1.Batch normalization is applied before leaky ReLU activations in all layers.
- Segmentation architecture: The segmentation CAE eliminates pooling layers because nuclei average around 20 × 20 pixels and pooling can discard important spatial information.This differs from the U-net strategy of preserving detail with skip connections.
- Segmentation architecture: The segmentation configuration uses 40 × 40 inputs and 80 to 200 feature maps to reduce the high computational cost of a pooling-free network.Other segmentation settings remain unchanged from the main CAE configuration.
3.3. CNN Architecture
The supervised CNNs reuse the trained CAE’s learned representations and add task-specific convolutional layers to the foreground and background branches before pooling their features.
- CNN construction: Classification CNNs retain Parts 1–6 of the CAE and initialize those layers with the corresponding pretrained CAE parameters.This transfers the unsupervised representation into the supervised model.
- CNN construction: The foreground branch receives four 1 × 1 convolutional layers, while the background branch receives two 3 × 3 convolutional layers.Each added layer contains 320 convolutional filters.
- CNN construction: Global average pooling is applied to the outputs of both branches before combining their pooled features.
3.4. Learning Details
The study trains the CAE on unlabeled data, initializes supervised CNNs from it, and compares the crosswise-sparse model with conventional sparse, fully supervised, pretrained, and unsparse alternatives.
- Methods tested: CSP-CNN is initialized by the proposed crosswise sparse CAE, using a 1.6% sparsity rate chosen to approximate the average number of nuclei per training image.
- Methods tested: SP-CNN removes the crosswise constraint and nucleus detection branch, while SUP-CNN removes both the background representation and nucleus detection branches.
- Methods tested: The comparison also includes ImageNet-pretrained VGG16, a conventional unsparse CAE-based DEN-CNN, and U-net models for nucleus segmentation.Five U-net variants with different feature-map counts are evaluated, with the best-performing network selected.
- Evaluation: Table 2 reports classification performance using AUROC across three tasks and notes that the 5%-data CSP-CNN fails on the third task because five of 15 classes have fewer than two positive training instances.
3.6. Classifying Lymphocyte-rich Regions
On the lymphocyte-rich region classification task, CSP-CNN achieves the best result and remains superior to SP-CNN even with only 5% of the training data. The comparison attributes the gain to the crosswise constraint and its unsupervised nucleus detection capability.
- Classification results: CSP-CNN achieves the best AUROC on the lymphocyte-rich region dataset.The dataset uses 20,876 training images and 2,480 testing images.
- Classification results: 42%: CSP-CNN reduces the error of the best-performing SP-CNN baseline on this classification task.
- Classification results: With only 5% of the training data, CSP-CNN outperforms SP-CNN on the lymphocyte-rich region task.
- Classification results: The only difference between CSP-CNN and SP-CNN is the crosswise constraint, which enables unsupervised nucleus detection in the CAE.The authors use this comparison to support the importance of crosswise sparsity for performance.
3.7. Classifying Individual Lymphocytes
The proposed method reduced the baseline SP-CNN error by 25% in individual lymphocyte classification, using an end-to-end approach rather than separately tuned feature extraction and classification.
- 25% error reduction was achieved compared with the SP-CNN baseline.The comparison used four train/test splits with averaged results.
3.8. Nuclear Shape and Attribute Classification
On this dataset, the proposed method improved over the state of the art less substantially because the nucleus images came from a fixed detection method that could not be fine-tuned.
- The improvement over the state of the art was less significant than on other datasets.The evaluation used the same 5-fold training and testing separation protocol.
- The fixed nucleus detection method prevented fine-tuning with the proposed method.
3.9. Nucleus Segmentation
The proposed method achieved higher nucleus-segmentation performance than the challenge winner and U-net, while using only 5% of the sliding windows for training.
- The proposed method achieved a significantly higher DICE-average score than the MICCAI challenge winner and U-net.The DICE-average is the average of two versions of the DICE coefficient.
- Pooling layers were removed because nuclei are approximately 20 × 20 pixels and pooling discards important spatial information.
- 0.80 was the highest challenge DICE-average, which the CSP-CNN outperformed using only 5% of sliding windows during training.
4. Conclusions
The paper proposes a crosswise sparse CAE for simultaneous unsupervised nucleus detection and feature extraction, intended to initialize supervised classification or segmentation CNNs in an end-to-end pipeline.
- The crosswise sparse CAE simultaneously performs unsupervised nucleus detection and feature extraction.
- The CAE can initialize CNNs for supervised classification or segmentation, allowing detection, feature extraction, and downstream prediction to be trained end-to-end.
- Experiments on four challenging datasets indicate that crosswise sparsity is essential to state-of-the-art results.