PaperlayerView exact revision on arXiv

Source-linked AI summary

WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma

Chu Han, Xipeng Pan, Lixu Yan, Huan Lin, Bingbing Li, Su Yao, Shanshan Lv, Zhenwei Shi, Jinhai Mai, Jiatai Lin, Bingchao Zhao, Zeyan Xu, Zhizhen Wang, Yumeng Wang, Yuan Zhang, Huihui Wang, Chao Zhu, Chunhui Lin, Lijian Mao, Min Wu, Luwen Duan, Jingsong Zhu, Dong Hu, Zijie Fang, Yang Chen, Yongbing Zhang, Yi Li, Yiwen Zou, Yiduo Yu, Xiaomeng Li, Haiming Li, Yanfen Cui, Guoqiang Han, Yan Xu, Jun Xu, Huihua Yang, Chunming Li, Zhenbing Liu, Cheng Lu, Xin Chen, Changhong Liang, Qingling Zhang, Zaiyi Liu

arXiv:2204.06455v2eess.IVcs.CV

TL;DR

Pixel-level histopathology annotation is costly, motivating WSSS4LUAD to seek LUAD tissue segmentation from patch-level labels. The challenge assembled a pathologist-in-the-loop dataset and evaluated submissions, with the best team achieving 0.8413 mIoU.

  • Problem

    Pixel-level annotations for tissue segmentation require substantial time and resources, motivating more annotation-efficient LUAD approaches.

    Source: §I · p. 2

  • Method

    WSSS4LUAD provides patch-level training labels and pixel-level evaluation annotations for three LUAD tissue classes, prepared through a pathologist-in-the-loop process.

    Source: §I · p. 2, §V · p. 10

  • Results

    0.8413 mIoU was achieved by the best-performing team; 28 teams submitted results with over 1,000 submissions.

    Source: A. Brief Summary · p. 8, §V · p. 10

  • Takeaways & Limitations

    The authors view patch-level weak labels as a feasible complement to pixel-boundary annotation, with potential to alleviate annotation effort.

    Source: §V · p. 11

    Imperfect labels remain inevitable, especially among the pixel-level annotations.

    Source: §V · p. 11

Abstract

from arXiv · show

Lung cancer is the leading cause of cancer death worldwide, and adenocarcinoma (LUAD) is the most common subtype. Exploiting the potential value of the histopathology images can promote precision medicine in oncology. Tissue segmentation is the basic upstream task of histopathology image analysis. Existing deep learning models have achieved superior segmentation performance but require sufficient pixel-level annotations, which is time-consuming and expensive. To enrich the label resources of LUAD and to alleviate the annotation efforts, we organize this challenge WSSS4LUAD to call for the outstanding weakly-supervised semantic segmentation (WSSS) techniques for histopathology images of LUAD. Participants have to design the algorithm to segment tumor epithelial, tumor-associated stroma and normal tissue with only patch-level labels. This challenge includes 10,091 patch-level annotations (the training set) and over 130 million labeled pixels (the validation and test sets), from 87 WSIs (67 from GDPH, 20 from TCGA). All the labels were generated by a pathologist-in-the-loop pipeline with the help of AI models and checked by the label review board. Among 532 registrations, 28 teams submitted the results in the test phase with over 1,000 submissions. Finally, the first place team achieved mIoU of 0.8413 (tumor: 0.8389, stroma: 0.7931, normal: 0.8919). According to the technical reports of the top-tier teams, CAM is still the most popular approach in WSSS. Cutmix data augmentation has been widely adopted to generate more reliable samples. With the success of this challenge, we believe that WSSS approaches with patch-level annotations can be a complement to the traditional pixel annotations while reducing the annotation efforts. The entire dataset has been released to encourage more researches on computational pathology in LUAD and more novel WSSS techniques.

I. INTRODUCTION

LUAD tissue segmentation can support quantitative histopathology analysis, but pixel-level labeling is difficult because WSIs are enormous, tumors are heterogeneous, and expert annotation is demanding. WSSS4LUAD addresses this by inviting methods that segment three tissue classes from patch-level labels while expanding public LUAD labeling resources.

I. INTRODUCTION

  • Histopathology examination of H&E-stained tissue remains the gold standard for LUAD diagnosis and conveys tissue morphology and spatial organization.Experienced pathologists assess tumor progression by reading patterns of interaction between tumor and its microenvironment.Source: §I · pp. 1–2
  • Pixel-level annotation is especially difficult for huge WSIs, heterogeneous tumor morphology, and tissue boundaries that require pathologist expertise.The passage describes tumor–stroma interactions in LUAD as more twisted and complex than in two other cancers.Source: §I · p. 2
  • WSSS4LUAD asks participants to segment tumor epithelial, tumor-associated stroma, and normal tissue using patch-level labels.The challenge also aims to expand public LUAD labeling resources and reduce pathologists’ annotation efforts.Source: §I · p. 2

II. RELATED WORKS

  • The paper briefly reviews recent work on lung cancer histopathology analysis, image segmentation, and annotation-efficient models.Source: §II · p. 2

A. Lung Cancer - Histopathology Image Analysis

Computational pathology uses histopathology images for objective, quantitative assessment, while tissue-level segmentation helps distinguish tumor constituents. The paper situates WSSS4LUAD amid limited lung tissue-label resources and persistent annotation demands.

A. Lung Cancer - Histopathology Image Analysis

  • Lung histopathology supports analyses of diagnosis, molecular status, immune phenotypes, and prognosis, but large-scale tissue annotation can entail substantial workload.One cited team annotated around 2.8 × 10^9 µm^2 across more than 3,000 tumor slides.Source: A. Lung Cancer - Histopathology Image… · pp. 2–3

B. Fully-supervised Histopathology Image Segmentation

C. Annotation Efficient Approaches

Annotation-efficient approaches include active learning and self-supervised learning. The reviewed work describes selecting informative samples for labeling and learning image representations through pretext tasks without human annotations.

1) Active Learning:

  • Active learning lets models select valuable, informative samples for expert labeling, with examples spanning sample selection, transfer learning, self-training, and pathologist-in-the-loop dataset creation.Doyle et al. used selection to alleviate class imbalance in prostate histopathology; Greenwald et al. created a cell dataset with over a million labeled cells.Source: 1) Active Learning · p. 3

2) Self-supervised Learning:

  • Self-supervised learning uses a pretext task without human annotations to learn feature representations for a later task.Source: 2) Self-supervised Learning · p. 3
  • Self-supervised methods include contrastive, predictive, and generative objectives, with examples such as MoCo, SimCLR, BYOL,, image reconstruction [41], and inpainting.Source: 2) Self-supervised Learning · p. 3
  • Histopathology self-supervised methods use tasks such as resolution-sequence prediction, visual-field expansion, progressively harder curricula, and masked autoencoding.The reviewed approaches are described as learning domain-specific features and image characteristics that can alleviate training burden and data requirements.Source: 2) Self-supervised Learning · p. 3

3) Not-so-supervised Learning:

Not-so-supervised approaches reduce dense labeling through patch-level classification, semi-supervised learning, or weak labels. WSSS4LUAD evaluates patch-label methods on dense pixel annotations for three LUAD tissue categories.

3) Not-so-supervised Learning:

  • Patch-level classification simplifies segmentation annotation but sacrifices boundary precision; PDBL is described as reducing training burden and annotations while maintaining promising classification accuracy.The cited patch-classification approach is Kather et al.,, followed by Lin et al. proposing PDBL.Source: 3) Not-so-supervised Learning · p. 4
  • Semi-supervised learning explores labeled–unlabeled semantic connections, but assumes their distributions are sufficiently close, a potential obstacle with heterogeneous tumor morphology.The passage suggests weak labels with greater sample variety may be more effective than dense annotations with limited variety for histopathology segmentation.Source: 3) Not-so-supervised Learning · p. 4
  • Weak-supervision examples include slide-level multiple-instance learning, scribbles, points, and tissue-proportion labels; CAM-based models also generate initial pixel-level pseudo-masks for challenge validation and test sets.The passage notes multiple-instance models segment cancerous versus non-cancerous regions, while the organizers’ CAM model uses progressive dropout attention and multi-layer pseudo-supervision.Source: 3) Not-so-supervised Learning · p. 4

III. CHALLENGE DESCRIPTION

  • WSSS4LUAD trains with patch-level classification labels and evaluates pixel-level segmentation of tumor epithelial, tumor-associated stroma, and normal tissue.Source: §III · p. 4

A. Whole Slide Images Preparation

WSSS4LUAD combines WSIs from GDPH and TCGA with a pathologist-reviewed, AI-assisted process for producing patch-level training labels and pixel-level validation and test labels.

A. Whole Slide Images Preparation

  • 67 GDPH and 20 TCGA slides were scanned or downloaded at 40X, quality-filtered, and kept without stain normalization.The GDPH WSIs could not yet be released because of ethical restrictions.Source: A. Whole Slide Images Preparation · p. 4

B. Annotators Recruitment and Label Review Board

  • One expert, two senior, and eight junior pathologists used training, label review, and overlapping assignments to support consistency.The expert and senior pathologists formed the label review board; labeling criteria were discussed and refined through sample review.Source: B. Annotators Recruitment and Label… · p. 4

C. Pathologist-in-the-loop Labeling Process

  • The pathologist-in-the-loop process is intended to accelerate annotation and reduce manual effort as corrected labels accumulate.The challenge uses this process for both patch-level and pixel-level annotation.Source: C. Pathologist-in-the-loop Labeling… · p. 4, Figure 3 · p. 6
  • For patch-level labels, ResNet38 generates pseudo-labels from about 500 initial patches, which pathologists check and correct before model fine-tuning.High-confidence labels are verified by junior pathologists; uncertain labels are corrected by senior pathologists, and every patch label is verified by at least two pathologists.Source: C. Pathologist-in-the-loop Labeling… · pp. 4, 6
  • For pixel-level labels, MLPS [69] generates heatmaps and pseudo-masks from patch labels, which pathologists refine and the review board confirms.Verified ground truths are added to the training set to fine-tune the model.Source: C. Pathologist-in-the-loop Labeling… · p. 6

D. Dataset Details

The dataset details include the training-label format and magnification of the patches.

D. Dataset Details

  • Training samples use three-digit [Tumor, Stroma, Normal] classification labels, and all patches are under 10× magnification.Source: D. Dataset Details · p. 6

1) Training set:

The training set contains filtered patch samples from GDPH and TCGA, while validation and test sets use pixel-level labels from separate WSIs and manually cropped patches.

1) Training set:

  • 10,091 patches remained after 15,000 random crops from 49 GDPH and 14 TCGA WSIs were filtered for blur, dirt, and ambiguity.Source: 1) Training set · p. 6
  • The largest training-label combination is [1,1,0] (tumor and stroma without normal); some combinations are rare or absent, and small patches may not capture normal tissue alongside tumor or stroma.Source: Figure 4 · p. 7, 1) Training set · p. 7
  • Training filenames encode the slide ID, (x, y) coordinates, and classification labels.Source: 1) Training set · p. 7

2) Validation and test sets:

  • Validation and test sets each use nine GDPH and three TCGA WSIs; the validation set has 40 manually cropped patches and the test set has 80.The validation patches comprise nine large and 31 small patches; the test patches comprise 14 large and 66 small patches.Source: 2) Validation and test sets · p. 7
  • White alveolar background is threshold-defined and excluded from evaluation.Source: 2) Validation and test sets · p. 7

E. Evaluation Metrics

The challenge evaluates segmentation with mIoU under specified data-use and background rules, and reports broad participation and methods from top teams.

E. Evaluation Metrics

  • mIoU excludes white alveolar backgrounds, and participants may train only on the provided data, with ImageNet pretraining allowed for initialization.Source: E. Evaluation Metrics · p. 8

A. Brief Summary

  • 532 participants registered, 28 teams submitted over 1,000 test-phase submissions, and 14 teams exceeded 0.7 mIoU.Source: A. Brief Summary · p. 8

B. Top-tier Methods

  • Lin et al. use a Siamese multi-task model whose classification-task CAM serves as the segmentation pseudo-mask, with ASPP and cross-supervision between the two models.Source: 1) Lin et al. (1st place, Team:… · p. 8

1) Lin et al. (1st place, Team: ChunhuiLin):

Lin et al. combine synthetic pixel labels, Siamese cross-supervision, and spatial post-processing, while Zhu et al. use image splicing to augment data and guide segmentation with CAM-derived pseudo-masks.

1) Lin et al. (1st place, Team: ChunhuiLin):

  • Lin et al. augment single-class patches to rebalance normal tissue and use CutMix to create fake images with pixel-level ground truth.They first replace ImageNet parameters using dataset mean and variance.Source: 1) Lin et al. (1st place, Team:… · p. 8
  • Lin et al. pair CAM-based classification with a segmentation branch, then use two structurally identical Siamese models to cross-supervise from each other's pseudo-masks.The tasks share an encoder but use separate decoders; the team also adds traditional image-processing pseudo-labels to classification training.Source: 1) Lin et al. (1st place, Team:… · p. 8
  • Lin et al. combine Lovasz and cross-entropy losses, apply online hard-negative mining, and ensemble Siamese outputs at inference.The reported ensemble includes ResNet101 and ResNeSt269 with Deeplabv3, and HRNetw48.Source: 1) Lin et al. (1st place, Team:… · p. 8
  • Lin et al. fill false-background pixels with BFS and relabel tumor or stroma as normal when normal tissue exceeds 30% in a 100x100 neighborhood.This post-processing uses the stated prior that normal tissue less often overlaps tumor and stroma.Source: 1) Lin et al. (1st place, Team:… · p. 8

2) Zhu et al. (2nd place, Team: baseline0412):

  • Zhu et al. extract foregrounds from single-class patches and splice cropped regions across classes to create multi-class images and pixel-level ground-truth pairs.They also splice CAM-generated pseudo-masks.Source: 2) Zhu et al. (2nd place, Team:… · pp. 8–9
  • Zhu et al. pair a CAM classification module with a pseudo-mask-guided segmentation network to improve localization because CAMs alone give imprecise boundaries.The model uses image splicing as part of this multi-task approach.Source: 2) Zhu et al. (2nd place, Team:… · p. 9

3) Fang et al. (3rd place, Team: Vison307):

Fang et al. pre-train segmentation on synthesized pixel-labeled patches before using inferred masks to train on the full dataset. Li et al. weight easier examples more heavily and use a two-stage CAM-to-segmentation framework.

3) Fang et al. (3rd place, Team: Vison307):

  • Fang et al. concatenate randomly cropped single-class patches into multi-label patches with pixel-level pseudo-masks, then augment them with rotations, flips, and CutMix.The generated samples pre-train the segmentation model before it infers masks for the remaining multi-category patches; those masks are then used to train on the full dataset, with multiple models ensembled at inference.Source: 3) Fang et al. (3rd place, Team:… · p. 9

4) Li et al. (Team: shichuanyexi):

  • Li et al. normalize the segmentation loss map with softmax(-L) divided by its mean to weight cross-entropy, assigning higher weights dynamically to easy, low-loss examples.The mining strategy emphasizes credible pseudo-mask supervision and is intended to mitigate noise from classification-derived masks.Source: 4) Li et al. (Team: shichuanyexi) · p. 9
  • Li et al. use a two-stage framework: ResNet38 classification generates CAM pseudo-masks, followed by PSPNet segmentation, with multi-scale testing in both stages.Training images are resized to 224; segmentation testing uses resolution 320.Source: 4) Li et al. (Team: shichuanyexi) · p. 9

C. Quantitative and Qualitative Results

The top five teams exceeded 0.80 mIoU, and the top three used CutMix-like augmentation to create pixel labels, outperforming a patch-label-only approach. Qualitative comparisons also highlight annotation noise and the model's sensitivity to small regions.

Quantitative results

  • The top five teams achieved over 0.80 mIoU, and the top three using CutMix-like pixel-label generation outperformed the team relying only on patch-level annotations.The first-place team also achieved the best normal-tissue segmentation by leveraging the stated low-crosstalk prior.Source: C. Quantitative and Qualitative Results · p. 9

Qualitative results

  • The top-ranked approach had the best qualitative performance, while the third-place model's omission of multi-class patches yielded less effective predictions and inaccurate tumor–stroma boundaries.The comparison used patches spanning tissue categories, intraclass variation, and staining styles; Fig. 6 shows masks at image corners and overlaid on the images.Source: C. Quantitative and Qualitative Results · p. 9, Figure 6 · p. 10

Ground-truth limitations

  • Manual ground truth is imperfect and noisy at both pixel and patch levels, and precisely defining every pixel in a large patch or whole slide is difficult.The authors note that models may therefore perform better than expected against these labels.Source: C. Quantitative and Qualitative Results · p. 9

Qualitative results

V. CONCLUSION

WSSS4LUAD assembled patch-level LUAD training labels and pixel-level validation and test annotations to reduce annotation effort and evaluate weakly supervised segmentation. The challenge drew broad participation, while its conclusion highlights annotation noise and the omission of minority tissue classes.

V. CONCLUSION

  • The released dataset comprises 87 LUAD WSIs, 10,091 patch-level training labels, and over 131 million pixel annotations for validation and testing.Labels were prepared by 11 pathologists using a pathologist-in-the-loop strategy.Source: §V · p. 10
  • 0.8413 mIoU was the best team's result; 28 teams submitted results, with over 1,000 submissions from 532 registrations.The authors also report CAM as the most popular WSSS approach among top-tier teams.Source: §V · p. 10
  • Top teams used CAM, while the first-place team combined Siamese cross-supervision with pseudo-mask exchange, cutmix augmentation, and class balancing.Some teams also used multi-task classification and segmentation to learn more generalizable features.Source: §V · pp. 10–11
  • Patch-level training labels took less than a week to prepare, compared with around one and a half months for pixel-level validation and test labels.The authors view patch-level labels as a feasible complement to pixel-boundary drawing for reducing annotation effort.Source: §V · p. 11
  • Labels remain imperfect, and some local model predictions appear to outperform the ground truth, complicating interpretation of model–label disagreements.The authors observe less noise in patch-level than pixel-level labels, while noting that the two labeling tasks differ substantially in difficulty.Source: §V · p. 11, Figure 7 · p. 11
  • The challenge evaluates only three majority tissue classes, leaving minority classes such as necrosis, vessels, and bronchus for future evaluation.The authors note that some omitted classes may be important despite occupying very small portions of whole-slide images.Source: §V · p. 11
Loading 2204.06455v2…