CryoNuSeg: A Dataset for Nuclei Instance Segmentation of Cryosectioned H&E-Stained Histological Images

• Supervised deep-learning segmentation requires fully annotated masks for training and evaluation, while expert annotation is time-consuming and can vary within and between observers.Source: §1 · pp. 1–2
• Earlier H&E nuclei datasets generally omit tissue-processing information, and the authors report no prior explicit analysis of nuclei instance segmentation in FS-derived images.Source: §1 · p. 2
• Frozen sections support rapid clinical decisions, but rapid preparation can degrade image quality and alter nuclear appearance relative to FFPE images.Reported artefacts include condensed nuclear chromatin and ice crystals, which may affect both expert interpretation and algorithm performance.Source: §1 · p. 2
• CryoNuSeg is a fully annotated FS-derived H&E nuclei dataset spanning 10 human organs, with three mark-ups enabling intra- and inter-observer variability analysis.The paper also investigates FS versus FFPE effects on a deep-learning segmentation algorithm and provides a baseline benchmark.Source: §1 · p. 2

2.1. Dataset

• Thirty 512 × 512 patches from 10 organs were selected from TCGA, manually annotated by two experts, and supplemented by a second mark-up from one annotator.The dataset includes conventional masks, border-removed masks, distance maps, and weighted maps; MoNuSeg images were also divided into organ-balanced FS and FFPE subsets.Source: §2.1 · pp. 2, 4

2.2. Nuclei segmentation method

• The baseline model combines U-Net foreground segmentation with distance-map regression, then merges predictions using Gaussian smoothing and watershed processing.Training uses image augmentation; inference removes very small instances and fills holes.Source: §2.2 · p. 4

2.3. Evaluation

• Performance is assessed with Dice, AJI, and PQ, with two-sided Wilcoxon signed-rank tests used to detect statistically significant differences.Dice measures semantic segmentation, while AJI and PQ assess instance segmentation; the paper describes PQ as more robust than AJI.Source: §2.3 · p. 4

2.4. Implementation

• The authors provide Matlab code for mask generation and use Keras for the segmentation model; the accompanying figure displays raw patches alongside manual labels and auxiliary masks.Experiments ran on a workstation with an Intel Core i7-8700 CPU and a TITAN V GPU.Source: §2.4 · pp. 4–5, Figure 2 · p. 5

3. RESULTS

• The protocol-effect experiment trained separate models on MoNuSeg-FS and MoNuSeg-FFPE, then tested both on all CryoNuSeg images.Table 3 reports Dice, AJI, and PQ against Annotator 1’s first-round masks, with p-values comparing the two training sets.Source: §3 · p. 5, Table 3 · p. 6
• Combined MoNuSeg-FS and MoNuSeg-FFPE training yielded average Dice of 79.1 ± 5.2%, AJI of 51.8 ± 6.4%, and PQ of 48.2 ± 7.3% on CryoNuSeg.Compared with MoNuSeg-FS training, p-values were 0.0196, 0.0001, and 0.0001 for Dice, AJI, and PQ; compared with MoNuSeg-FFPE, they were 0.0068, 0.0005, and 0.0001, respectively.Source: §3 · p. 5
• The CryoNuSeg baseline used 10-fold cross-validation, training on nine organs and testing on the remaining organ in each fold.This setup was intended to provide a benchmark for future studies using CryoNuSeg.Source: §3 · p. 7
• 80.2 ± 4.8% average Dice, 52.9 ± 5.7% average AJI, and 49.4 ± 6.5% average PQ were obtained using CryoNuSeg and combined MoNuSeg training in organ-held-out 10-fold cross-validation.Each fold used 27 CryoNuSeg images from nine organs plus the combined MoNuSeg-FS/MoNuSeg-FFPE data for training, and three images from the held-out organ for testing.Source: §3 · p. 7
• The annotation-variability experiments compared Annotator 2’s masks with Annotator 1’s first-round masks and compared Annotator 1’s first and second rounds.Source: §3 · p. 7

4. DISCUSSION

• Wilcoxon tests found no significant Dice, AJI, or PQ difference between MoNuSeg-FS and MoNuSeg-FFPE training, despite large adrenal AJI/PQ differences.The authors call for a larger adrenal-gland training sample to investigate that tissue-specific result.Source: §4 · p. 8
• Combining MoNuSeg-FS and MoNuSeg-FFPE produced a slight, statistically significant PQ improvement, likely related to increasing training images from 15 to 30.The authors state that the training-image increase is a likely explanation, not an isolated protocol effect.Source: §4 · p. 8
• Further work is planned to test tissue fixation and embedding effects on gland or tumour segmentation and WSI classification or grading.The discussion identifies these as other histological image-analysis tasks beyond the investigated segmentation setting.Source: §4 · p. 8
• Ten-fold cross-validation holds out one organ per fold, supporting evaluation on unseen organs and consistent folds for future comparisons.Each of the ten organs serves once as the test organ.Source: §4 · p. 8
• CryoNuSeg training significantly outperformed MoNuSeg-FS and MoNuSeg-FFPE across all six pairwise metric tests.The authors suggest larger training-image counts or differences in train/test annotators could explain the result.Source: §4 · p. 8
• Inter-observer agreement was relatively close by Dice but weaker for AJI and PQ, and inter-observer variability affected performance more than intra-observer variability.Annotator 2 tended to over-segment nuclei; intra-observer comparisons also differed significantly across all three indices.Source: Table 5 · p. 7, Table 6 · p. 7, §4 · pp. 8–9
• Unclear touching-nucleus borders, folded tissue, acquisition artefacts, annotation errors, and fatigue can contribute to observer variability.Removing vague mask areas may partially address these issues but requires extra supervision and time.Source: §4 · p. 9
• The baseline segmentation results were relatively close to manual annotation, although the PQ comparison remained statistically significant.The reported p-values were 0.055 for Dice, 0.052 for AJI, and 0.003 for PQ.Source: §4 · p. 9

5. CONCLUSIONS

• CryoNuSeg is introduced as the first fully manually annotated FS-derived H&E-stained nuclei segmentation dataset.The dataset is publicly available with extensive documentation and sample implementations.Source: §5 · p. 9
• The dataset is intended for developing and comparing nuclei instance segmentation algorithms, with a state-of-the-art DL-based baseline benchmark.The paper also describes the dataset-generation steps.Source: §5 · p. 9
• The paper investigates tissue fixation and embedding effects on segmentation performance, with other histological image-analysis tasks left for future work.The authors identify further comparisons across tasks as planned work.Source: §5 · p. 9