Source-linked AI summary
Colorectal Cancer Segmentation with Adaptive Augmentation and Multi-Resolution Ensemble Models
Ümit Mert Çağlar, Alptekin Temizel
TL;DR
CRC’s burden makes early diagnosis and accurate tumor grading important. This paper presents an automated whole-slide segmentation pipeline and reports F1 scores over 6 points above existing baselines.
Problem
Early CRC diagnosis and accurate tumor grading matter, while specialist shortages limit screening and early detection.
Source: §1 · p. 1Method
Results
Over 6 points: the pipeline surpassed existing baselines in F1 scores.
Source: §6 · p. 6Takeaways & Limitations
The paper presents an automated pipeline for multi-class colorectal cancer tumor-grading segmentation.
Source: §6 · p. 6The study uses images downscaled to 20×, 40×, and 60×; full-resolution images and larger models are left for future work.
Source: §6 · p. 6
Abstract
from arXiv · showhide
Colorectal cancer (CRC) is the second most deadly and third most common cancer, and the leading cause of death among gastrointestinal cancers. Early diagnosis is crucial for the treatment of this cancer and increasing the survival rates. Although CRC is more common in developed regions, its occurrence is also increasing in developing regions as well. CRC diagnosis relies on histopathology assessment post-biopsy. Automated deep learning algorithms can significantly reduce diagnosis time, enhancing efficiency and supporting timely clinical decisions. We present an automated segmentation pipeline for whole-slide histopathology images that labels tumor grades 1-3 and normal mucosa. It utilizes dense prediction transformers with various encoder backbones, overlapping patches, and test-time augmentation. An adaptive augmentation policy, guided by large language models, further improves training. Top models were ensembled via soft voting, and mask refining post-processing steps, Gaussian blurring, morphological closing, and connected components analysis. On a colorectal cancer grade dataset, our method improved the F1 score from 62.92 to 69.84. Code is available here: github.com/caglarmert/ICIP2025
1. INTRODUCTION
The introduction motivates automated CRC diagnosis through the importance of early detection and accurate tumor grading, alongside a shortage of specialists. It presents an automated whole-slide segmentation pipeline and summarizes its experimental, augmentation, ensemble, and post-processing contributions.
Motivation
- CRC is a leading cause of gastrointestinal cancer deaths, and early diagnosis and accurate biopsy-based tumor grading inform prognosis and treatment.The introduction also notes that specialist shortages limit screening and early detection, particularly in low- and middle-income regions.Source: §1 · p. 1
- Transformer-based automated histopathology segmentation is presented as a potential way to accelerate CRC diagnosis amid limited specialist access.Source: §1 · p. 1
2. RELATED WORK
Related work covers deep learning for histopathology classification and segmentation, including CNNs, attention-based models, and transformer approaches. It also discusses encoder-decoder architectures, ensembles, and image processing.
Deep learning for histopathology
- CNNs, residual CNNs, and attention-based models have been applied to histopathological assessment and segmentation.Source: §2 · p. 2
Segmentation architectures
- Medical-image segmentation approaches include single- and multi-scale transformer, CNN, and hybrid models, with encoder-decoder networks described as predominant.Source: §2 · p. 2
Ensembling and image processing
- Related work identifies ensemble methods and image pre- and post-processing, including Gaussian filtering and morphological operations, as approaches used in digital pathology.Source: §2 · p. 2
3. METHODOLOGY
The methodology processes whole-slide images through downscaling, multi-resolution model training, ensembling, and mask refinement. It uses overlapping patches, adaptive augmentation guided by language models, and soft voting.
3. METHODOLOGY
- Whole-slide images are downscaled into smaller slices with varying overlap, then multiple encoder-decoder architectures are trained at different resolutions using language-model-guided adaptive augmentation.Source: §3 · p. 2
3.1 Pre-processing
- 2048×2048 patches are downscaled at 60×, 40×, and 20×, with nearest-neighbor interpolation for masks and bilinear interpolation for images.The patches are saved losslessly; area interpolation is used for output probability maps.Source: §3.1 · p. 2
3.2 Model Architecture and Training
- The experiments vary architectures, encoders, input scalings, patch settings, and loss functions, and apply LLM-feedback adaptive augmentation during training.Evaluated architectures include FCNs, U-Nets, segmentation transformers, and DPTs; backbones include Multi-Axis Vision Transformers, EfficientNets, and residual networks.Source: §3.2 · p. 3
3.3 Model Ensembling
- Ensembling combines top-model predictions through hard voting or soft voting over class probabilities with a class-specific bias favoring tumor classes.The final mask is the argmax of the aggregated probability tensors.Source: §3.3 · p. 3
3.4 Post-Processing
- Mask refinement smooths probability maps with Gaussian filtering, closes small holes and gaps morphologically, and removes small connected regions.Source: §3.4 · p. 3
4. EXPERIMENTAL EVALUATION
On CCTGS, the final ensemble with mask refinement reached a mean F1 of 69.84, above the Swin Transformer baseline; on EBHI-SEG, the approach outperformed baselines across nearly all classes and metrics.
4. EXPERIMENTAL EVALUATION
- 69.84 mean F1 followed soft-biased voting and mask refinement, up from 62.92 for the Swin Transformer baseline on CCTGS.The dataset contains 103 whole-slide images with pixel-wise masks for three tumor grades and normal tissue.Source: §4 · p. 3
- Figure 2 pairs five class-probability heat maps with final segmentation masks, illustrating tumor-versus-background segmentation on a sample image.The overlays show the model attending to different image sections.Source: §4 · p. 3, Figure 2 · p. 4
- On EBHI-SEG, the approach outperformed U-Net, Seg-Net, and MedT across nearly all classes and evaluation metrics.Models trained with adaptive augmentation achieved the highest scores.Source: §4 · p. 4, Table 2 · p. 4
5. ABLATION STUDY
Ablations across 245 models identify effective losses, architectures, augmentation, and ensemble choices; resolution results show comparable performance across tested levels.
5. ABLATION STUDY
- To optimize histopathology image segmentation, the study trained 245 models over 1,500+ hours, testing six loss functions, three optimizers, four architectures with ten encoder backbones, and three input resolutions.Source: §5 · p. 5
- Dice, Tversky, and Jaccard losses outperformed cross-entropy and focal loss, with Dice the most effective loss for F1 optimization.Tversky favored precision over recall, while Jaccard had slightly lower F1 but marginally higher IoU than Tversky.Source: §5 · p. 5, Table 3 · p. 6
- SegFormer and DPT surpassed the literature baseline, and transformer-based architectures with pretrained transformer encoders outperformed traditional U-Net or SegFormer architectures with CNN encoders.The evaluation covered FCN, U-Net, SegFormer, and DPT architectures with varied backbones.Source: §5 · p. 5, Table 3 · p. 6
- Models at 20×, 40×, and 60× downscaling had comparable performance, with 40× offering an intermediate balance of image detail and computational efficiency.The best models at each resolution used DPT with MaxViT-Large; the 40× model used Jaccard loss, while the 20× and 60× models used Dice loss.Source: §5 · p. 5, Table 3 · p. 6
- The LLM-guided adaptive augmentation strategy improved model performance.Its feedback loop updates augmentation using the current policy, evaluation metrics, architecture, and hyperparameters.Source: §5 · p. 5, Table 3 · p. 6
- Soft voting outperformed hard voting, while post-processing marginally improved results without additional training costs.The approach combines ensemble strategies with mask-refinement steps.Source: §5 · p. 5, Table 3 · p. 6
6. CONCLUSIONS
The paper presents a transformer-based pipeline for multi-class colorectal tumor segmentation and reports F1 scores more than six points above existing baselines. Its future work includes broader ensemble and test-time augmentation strategies, as well as full-resolution images and larger models when resources permit.
6. CONCLUSIONS
- The pipeline uses DPT with MaxViT encoders and overlapping patches, with reported F1 scores over six points above existing baselines.The approach targets multi-class tumor grading in colorectal histopathology images.Source: §6 · p. 6
- Test-time augmentation and multi-resolution model predictions are combined by soft voting, followed by post-processing to refine segmentation masks.Different resolutions capture fine and coarse details, producing diverse probability matrices.Source: §6 · p. 6
- Future work includes one-vs-all binary segmentation ensembles, expanded or feedback-loop test-time augmentation, and full-resolution images with larger models when resources allow.The current study used 20×, 40×, and 60× downscaled images, which can lose detail.Source: §6 · p. 6