Source-linked AI summary
Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification
Simon Graham, Mostafa Jahanifar, Ayesha Azam, Mohammed Nimir, Yee-Wah Tsang, Katherine Dodd, Emily Hero, Harvir Sahota, Atisha Tank, Ksenija Benes, Noorul Wahab, Fayyaz Minhas, Shan E Ahmed Raza, Hesham El Daly, Kishore Gopalakrishnan, David Snead, Nasir Rajpoot
TL;DR
Computational pathology needs large quantities of accurate nuclear annotations, but manual labelling is costly and fully automatic boundaries can be unreliable. Lizard addresses this gap with a multi-stage, pathologist-in-the-loop pipeline and provides nearly half a million labelled nuclei from H&E-stained colon tissue. The resulting dataset supports nuclear instance segmentation and classification research, while its scope remains bounded by current annotation choices and refinement trade-offs.
Problem
Deep segmentation models require abundant accurate pixel-level annotations, while manual nuclear labelling is time-consuming and automatic boundary generation can reduce annotation reliability.
Method
Lizard uses a multi-stage annotation pipeline with pathologist-in-the-loop refinement and representative-sample concordance assessment.
Results
Nearly half a million labelled nuclei make Lizard the largest known nuclear instance-segmentation and classification dataset in computational pathology.
Takeaways & Limitations
Lizard provides a large colon-tissue resource for developing and benchmarking nuclear segmentation, classification and downstream cell-based models.
Takeaways & Limitations
Skipping boundary refinement can reduce annotation accuracy, and current future work identifies additional nuclear types and non-colonic applications as extensions.
Abstract
from arXiv · showhide
The development of deep segmentation models for computational pathology (CPath) can help foster the investigation of interpretable morphological biomarkers. Yet, there is a major bottleneck in the success of such approaches because supervised deep learning models require an abundance of accurately labelled data. This issue is exacerbated in the field of CPath because the generation of detailed annotations usually demands the input of a pathologist to be able to distinguish between different tissue constructs and nuclei. Manually labelling nuclei may not be a feasible approach for collecting large-scale annotated datasets, especially when a single image region can contain thousands of different cells. However, solely relying on automatic generation of annotations will limit the accuracy and reliability of ground truth. Therefore, to help overcome the above challenges, we propose a multi-stage annotation pipeline to enable the collection of large-scale datasets for histology image analysis, with pathologist-in-the-loop refinement steps. Using this pipeline, we generate the largest known nuclear instance segmentation and classification dataset, containing nearly half a million labelled nuclei in H&E stained colon tissue. We have released the dataset and encourage the research community to utilise it to drive forward the development of downstream cell-based models in CPath.
1. Introduction
Computational pathology needs accurate nuclear localisation to extract interpretable features, but segmentation models require extensive expert-labelled data. Lizard addresses this bottleneck with a multi-stage, pathologist-refined annotation pipeline and a large colon-tissue dataset.
- Motivation: Accurate nuclear localisation enables interpretable morphological features for downstream tasks such as cancer-grade prediction.
- Motivation: Deep segmentation models require abundant pixel-level annotations, making expert labelling a major bottleneck for computational pathology.Accurately delineating object boundaries is time-consuming, and annotation accuracy can trade off against dataset scale.
- Contributions: A multi-stage annotation pipeline collects accurate large-scale instance-segmentation datasets through automated predictions and targeted refinement.
- Contributions: Nearly half a million labelled nuclei make Lizard the largest instance-segmentation and classification dataset in computational pathology.
- Dataset: Lizard includes colon-tissue nuclei from six sources with segmentation masks and labels for epithelial, connective, lymphocyte, plasma, neutrophil and eosinophil cells.The tissue spans normal, inflammatory, dysplastic and cancerous conditions.
2. Related work
Earlier nuclear datasets established segmentation benchmarks but often lacked nuclear-type labels or relied on automatic boundary generation. Lizard combines large scale with individual nuclear class labels and aims to improve annotation reliability.
- Single-category datasets: TNBC, CPM-17 and MoNuSeg provide manually labelled nuclear boundaries but treat nuclei as a single category.
- Class-labelled datasets: Nuclear class prediction supports tumour-microenvironment profiling beyond boundary segmentation alone.
- Class-labelled datasets: CoNSeP and MoNuSAC pair nuclear segmentation masks with class labels, including inflammatory subcategories in MoNuSAC.
- Scalable annotation: PanNuke and the breast-cancer annotation approach scale annotation with semi-automatic or crowdsourced methods, but automatic boundaries may remain inaccurate for difficult nuclei.
- Lizard: Lizard is the largest listed nuclear instance-segmentation dataset that also provides an individual label for each nucleus.
3. The dataset generation pipeline
Lizard’s dataset-generation pipeline uses four sequential stages to combine automated annotation with targeted manual refinement and verification. This design focuses human effort on unsatisfactory boundaries while retaining large-scale processing.
- Pipeline stages: The pipeline first optimises automated segmentation and classification for epithelial, inflammatory and connective-tissue nuclei.
- Pipeline stages: A sequential boundary-refinement stage directs manual attention only to nuclei with unsatisfactory automatic or semi-automatic results.
- Pipeline stages: An automated subtyping stage classifies inflammatory nuclei as lymphocytes, plasma cells, neutrophils or eosinophils.
- Pipeline stages: The final stage verifies the class labels of all nuclei.
CoNSeP
Lizard combines six colon-tissue data sources with automated nuclear labelling and pathologist-guided refinement. Its staged process reduces full manual boundary tracing while retaining an accuracy safeguard through optional manual refinement.
- Data sources: Six data sources—GlaS, CRAG, CoNSeP, DigestPath, PanNuke, and TCGA—provide colon image regions for Lizard.The sources include WSI-derived regions, multi-centre TCGA data, and biopsy samples from four Chinese hospitals.
- Automated initial labelling: HoVer-Net first labels nuclei as epithelial, inflammatory, or connective tissue in the unannotated target dataset.Neoplastic and non-neoplastic epithelial nuclei are grouped together, while connective tissue includes fibroblast, muscle, and endothelial nuclei.
- Iterative refinement: Pathologist-selected 512×512-pixel regions with poor predictions are manually refined, added to training data, and used for iterative model retraining.This targets errors on the dataset being labelled rather than relying only on annotations from the original sources.
- Boundary refinement: Boundary refinement proceeds from a single nucleus point through NuClick-generated boundaries to manual correction of remaining imperfections.The staged approach reserves complete manual delineation for cases where automated or semi-automatic results remain inadequate.
- Boundary refinement: Skipping manual refinement accelerates annotation collection but may reduce annotation accuracy.The pipeline therefore trades annotation effort against final boundary quality when the last refinement stage is omitted.
4. Dataset statistics and baseline results
The dataset’s annotations were assessed against pathologist judgments, and baseline experiments compared segmentation and classification models using cross-validation and an external test set. HoVer-Net achieved the strongest overall performance, while rare inflammatory classes remained more difficult.
- Pathologist assessment: Three pathologists assessed a representative sample of nuclei, and concordance was measured with pairwise Cohen’s kappa and 95% confidence-interval margins of error.The sample contained varied morphological appearances across classes.
- Pathologist assessment: All settings showed substantial agreement, with dataset–pathologist concordance not significantly different from inter-pathologist concordance.The authors interpret this as evidence that annotation disagreement is justified and dataset labels are accurate.
- Dataset statistics: The dataset’s category distributions reflect tissue prevalence, with significantly fewer eosinophils and neutrophils because they mainly occur in acute infective or inflammatory conditions.The source breakdown also notes that DigestPath and CRAG contribute most labelled nuclei, whereas CoNSeP contributes the fewest.
- Experimental setup: The experiments use three patient-wise folds, average results across folds, and evaluate fold-dependent models on TCGA as an external test set.The released splits support reproducible comparisons between methods.
- Experimental setup: U-Net, Micro-Net, and HoVer-Net were trained for nuclear segmentation and classification, with extra output convolutions enabling classification for U-Net and Micro-Net.HoVer-Net is described as a state-of-the-art simultaneous segmentation-and-classification model.
- Evaluation metrics: Panoptic quality combines detection quality and segmentation quality, while multi-class panoptic quality averages class-specific scores.Matching uses an IoU threshold greater than 0.5 to identify true-positive segment pairs.
- Baseline results: HoVer-Net achieved the best performance by a significant margin in both cross-validation and external testing, establishing a strong baseline for Lizard.The authors report competitive results despite using 20× resolution and suggest this resolution may suffice for the evaluated tasks.
5. Conclusions and future directions
Lizard is a large colon-tissue dataset with nearly half a million annotated nuclei, generated through pathologist-refined multi-stage annotation. The authors aim to support accurate, interpretable downstream computational pathology models and plan to expand nuclear categories.
- Lizard contains nearly half a million annotated nuclei with associated class labels, making it the largest known available dataset for nuclear instance segmentation and classification in computational pathology.
- A multi-stage pipeline with substantial pathologist-in-the-loop refinement enabled accurate annotation collection at scale.
- The dataset focuses on colon tissue from diverse colonic conditions to increase the chance of generalisation to unseen examples.
- The released dataset is intended to support accurate and interpretable downstream models for computational analysis of H&E-stained colon tissue.
- Future expansion may add macrophages, epithelial subtypes, and signet ring cells, while inflammatory-cell morphology may support applications beyond colon tissue.
A. Representative sample cluster examples
The representative clusters show lower within-cluster than between-cluster variability, with category-specific differences in nuclear shape, staining, nucleoli, and cytoplasm.
- Clusters contain nuclei with similar appearances, while different clusters within categories show greater morphological variation.Examples include differences in nuclear shape, staining intensity, visible nucleoli, and eosinophilic cytoplasm.
- Connective-tissue-cell clusters differ between long, thin, homogeneously dark nuclei and paler nuclei.
- Epithelial-nuclei clusters differ between round, pale nuclei with visible nucleoli and darker-stained nuclei.
- Eosinophil clusters differ in the prominence of eosinophilic cytoplasm.
B. Baseline results per category
Per-class results show strong performance for common epithelial, lymphocyte, and connective-tissue classes, especially with HoVer-Net, but lower performance for underrepresented neutrophil and eosinophil classes.
- Epithelial, lymphocyte, and connective-tissue classes perform strongly, especially for HoVer-Net.
- Neutrophil and eosinophil panoptic quality is significantly lower than for the other classes.
- HoVer-Net performs slightly better than U-Net and Micro-Net for neutrophils and eosinophils, where dice loss may alleviate class-imbalance effects.
- Eosinophil segmentation may be more challenging because eosinophilic cytoplasm is annotated, whereas most other classes include only the nucleus.
- The reported experiments train on a smaller dataset proportion because of three folds and an external test set; using all data may improve underrepresented-class performance.
C. Concordance statistics per category
Concordance is generally high across nuclear categories, but lymphocyte–plasma-cell confusion is the main disagreement, suggesting a possible future grouping of these chronic inflammatory cells.
- Confusion matrices assess category-level agreement across the settings reported in Table 2, with perfect agreement represented by high main-diagonal values.
- Raters generally show strong agreement, with the best agreement for epithelial cells.
- The main disagreement is between lymphocytes and plasma cells, especially between pathologists and the dataset annotations.
- Because lymphocytes and plasma cells are both chronic inflammatory cells, future work could potentially group them into one category.
- The confusion matrices indicate an upper bound for models trained to distinguish the dataset’s six nuclear categories.