Source-linked AI summary
GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection
Haoyuan Chen, Chen Li, Ge Wang, Xiaoyan Li, Md Rahaman, Hongzan Sun, Weiming Hu, Yixin Li, Wanli Liu, Changhao Sun, Shiliang Ai, Marcin Grzegorzek
TL;DR
Gastric histopathological diagnosis is time-consuming and subjective, while CNNs have difficulty handling global information and images contain abnormalities at varying scales. The paper proposes GasHis-Transformer and a lightweight variant combining global transformer modeling with local CNN analysis. The reported accuracies are 97.97% and 96.43% on gastric histopathological datasets, with additional accuracies of 88.10% and 84.34% on a lymphoma dataset and H&E and histopathological generalizability, respectively.
Problem
Gastric cancer diagnosis is time-consuming and subjective, and CNN models do not handle global information well for histopathological images containing abnormalities at different scales.
Method
GasHis-Transformer combines a position-encoded transformer for global information with a CNN using local convolution, while LW-GasHis-Transformer reduces model size and training time using Dropconnect.
Results
97.97% and 96.43% accuracies are reported on gastric histopathological datasets, while 88.10% and 84.34% are reported on a lymphoma dataset and H&E and histopathological generalizability, respectively.
Takeaways & Limitations
GasHis-Transformer demonstrates high global detection performance and potential for GHID, with reported generalizability to histopathological staging tasks for other cancers.
Takeaways & Limitations
The paper states that an experiment may lead to degradation of classification performance.
Abstract
from arXiv · showhide
In this paper, a multi-scale visual transformer model, referred as GasHis-Transformer, is proposed for Gastric Histopathological Image Detection (GHID), which enables the automatic global detection of gastric cancer images. GasHis-Transformer model consists of two key modules designed to extract global and local information using a position-encoded transformer model and a convolutional neural network with local convolution, respectively. A publicly available hematoxylin and eosin (H&E) stained gastric histopathological image dataset is used in the experiment. Furthermore, a Dropconnect based lightweight network is proposed to reduce the model size and training time of GasHis-Transformer for clinical applications with improved confidence. Moreover, a series of contrast and extended experiments verify the robustness, extensibility and stability of GasHis-Transformer. In conclusion, GasHis-Transformer demonstrates high global detection performance and shows its significant potential in GHID task.
1. Introduction
Gastric cancer diagnosis relies on H&E microscopy but is time-consuming and subjective. Histopathological images contain abnormalities at different scales, motivating models that capture both global and local information.
- H&E sections are examined by pathologists under a microscope to diagnose gastric cancer.
- The conventional diagnostic process is time-consuming and subjective, making accurate gastric cancer screening and diagnosis difficult.
- Computer-aided diagnosis uses image processing, pattern recognition, machine learning, and computer vision to improve examination efficiency.
- CNN models are widely used in computer vision but do not handle global information well.
- Histopathological images may contain either large or tiny abnormal regions, so detection models need strong global and local information extraction.
- A hybrid model combining CNN and Visual Transformer capabilities is proposed to address these information-extraction requirements.
Gastric Histopathological Image Detection (GHID) tasks, namely GasHis-
GasHis-Transformer combines transformer-based global modeling with CNN-based multi-scale local analysis for gastric histopathological images. Its lightweight variant targets clinical deployment, while experiments report strong detection and transfer performance.
- GasHis-Transformer integrates the global and local information-describing capabilities of Visual Transformers and CNNs.
- The Global Information Module uses a transformer-oriented design to represent a gastric histopathological image as a whole.
- The Local Information Module follows a parallel Inception-V3 structure to obtain multi-scale local details.
- The model introduces multi-scale image analysis to describe details in gastric histopathological images under a microscope.
- Its lightweight module uses quantization and Dropconnect to reduce model parameter size and training time for clinical applications with improved confidence.
- GasHis-Transformer reports good gastric cancer detection and excellent generalization on histopathological staging tasks for other cancers.
2. Related Work
Related GHID work spans traditional machine learning, CNN-based deep learning, attention mechanisms, and Visual Transformer architectures. Visual Transformers are highlighted for modeling image-wide information through attention.
- Traditional GHID research used random forests with 332 global graph features, including mean, variance, skewness, and kurtosis.
- Deep learning methods increasingly address GHID tasks, including improved ResNet-v2 networks with added pooling and convolution layers.
- Related approaches include DeepLab-V3 with ResNet-50, AlexNet, VGG, Inception-V3, ResNet, and Xception networks.
- Attention-based methods such as Non-local+Resnet, CBAM+Resnet, SENet+CNN, GCNet+Resnet, HCRF-AM, and Visual Transformers show strong global detection performance.
- Visual Transformer image-detection models include pure self-attention structures represented by ViT and self-attention structures combined with CNNs such as BoTNet-50, TransMed, and LeViT.
- The main advantage attributed to Visual Transformers is better global image description through attention mechanisms.
3. GasHis-Transformer
GasHis-Transformer combines transformer-based global modeling with convolutional local feature extraction for gastric histopathological image detection. Its design builds on ViT and BoTNet mechanisms, using positional encoding, multi-scale preprocessing, and a lightweight variant for clinical-oriented deployment.
- 3.1. Vision Transformer (ViT): ViT converts 2D images into patch-embedding sequences, adds positional information, and processes them with alternating MHSA and MLP blocks before classification.Layer normalization and residual connections precede the transformer blocks, while an MLP-based fully connected layer produces classification confidence.
- 3.2. BoTNet: BoTNet-50 combines ResNet-50 with MHSA layers to obtain global self-attention on 2D feature maps.It replaces the last three 3 × 3 spatial convolutions in stage c5 while retaining the latter ResNet-50 structure.
- 3.2. BoTNet: BoTNet-50 uses 2D relative position encoding, whereas ViT uses standard 2D patch-sequence position encoding.Relative offsets are represented separately for rows and columns and combined with content-based attention through softmax aggregation.
- 3.2. BoTNet: When input and output are 128-dimensional, convolution over 3 spatial extents has the same computational cost as MHSA over 19 spatial extents.The paper states that MHSA computation is less costly than convolution at the same spatial extent, contributing to lower parameter and computation requirements for BoTNet-50 than ResNet-50.
- 3.3. GasHis-Transformer: GasHis-Transformer detects gastric cancer images using global and local feature-extraction modules, with a lightweight version also proposed.The pipeline uses rotation and mirroring augmentation, image normalization, and training images containing normal and abnormal gastric histopathological samples.
- 3.3. GasHis-Transformer: Image normalization produces a mean of 0 and standard deviation of 1 while preserving global information and modifying pixels to accelerate model convergence.The normalization process is applied to improve image quality and speed training.
4.1. Experimental Settings
The experiments use balanced H&E gastric histopathology data, standardized preprocessing, augmentation, and common optimization and evaluation settings for global detection.
- Dataset: HE-GHI-DS contains 140 normal and 560 abnormal H&E-stained gastric histopathological images.Images are 20× magnified whole-slide regions in TIFF format with 2048×2048-pixel dimensions.
- Preprocessing and Augmentation: Images are horizontally and vertically flipped and rotated by 90, 180, and 270 degrees, expanding each dataset sixfold.All images are resized to 224 × 224 pixels by bilinear interpolation; validation and test augmentation supports multi-scale generalization evaluation.
- Evaluation Metrics: Precision, recall, F1-score, and accuracy are used to evaluate global detection performance.These criteria are defined using true positives, true negatives, false positives, and false negatives.
4.2. Evaluation Results of GasHis-Transformer
GasHis-Transformer achieves strong gastric-image detection, with feature visualizations linking correct recognition to cancerous regions and lumen structures while revealing difficulty with tiny lesions.
- Global Detection Results: 98.55%, 97.38%, 97.97% and 97.97% are GasHis-Transformer’s Pre, Rec, F1 and Acc on the test set, respectively.The confusion matrix reports 407 abnormal and 403 normal images correctly classified.
- Global Detection Results: 95.99%, 96.90%, 96.43% and 96.43% are LW-GasHis-Transformer’s Pre, Rec, F1 and Acc, respectively.It incorrectly reports 13 abnormal images as normal and 17 normal images as abnormal.
- Feature Visualization: GasHis-Transformer features separate most abnormal and normal images in t-SNE space and emphasize cancerous regions and lumen structures, respectively.Most abnormal examples show extensive carcinoma areas, whereas most normal examples contain lumen structures.
- Error Analysis: Similar feature vectors explain some errors, including visually similar intestinal metaplasia and images with small cancerous regions.The model tends to detect large contiguous regions and can struggle with tiny cancerous regions.
- Model Comparisons: GasHis-Transformer and LW-GasHis-Transformer show more stable training and smaller variance than same-scale CNN and attention models.The paper attributes this comparison to their multi-scale feature extraction.
- Normalization: Normalization increases detection metrics overall, improving GasHis-Transformer Pre by 1.11% and Rec by 0.44%.The authors report an increasing trend for Pre, Rec, F1 score, and Acc with normalization.
4.3. Extended Experiment
Extended experiments test GasHis-Transformer on gastrointestinal, breast, and lymphoma histopathology datasets, reporting strong detection or classification performance beyond gastric cancer.
- BreakHis Classification: GasHis-Transformer and LW-GasHis-Transformer outperform traditional CNN and attention models on BreakHis classification.The paper reports better image classification performance for both Transformer variants.
- Cross-dataset Generalization: The Transformer variants perform well on both H&E-stained and IHC-stained histopathological image datasets.The extended experiments cover breast cancer BreakHis and three-class lymphoma IHC-LI-DS classification.
- IHC-LI-DS Classification: On IHC-LI-DS, GasHis-Transformer has the best Rec, F1 and Acc, while LW-GasHis-Transformer has the best Pre among compared models.GasHis-Transformer reaches 82.42% Pre, 83.30% Rec, 83.16% F1, and 82.38% Acc; the supplied passage also reports 83.64%.
4.4. Experimental Environment and Computational Time
The lightweight and full models are evaluated on computational cost, with the lightweight variant reducing training time while retaining rapid inference.
- Training Time: GasHis-Transformer and LW-GasHis-Transformer require 0.86 and 0.67 hours to train, respectively, on 840 images for 75 epochs.The experiments report parameter size and training time across eight deep learning models and three datasets.
- Inference Time: Both models test 840 images in 30 seconds, equivalent to 0.036 seconds per image.This inference timing is reported for GasHis-Transformer and LW-GasHis-Transformer.
5. Conclusion and Future Work
The paper proposes GasHis-Transformer and a lightweight variant for gastric histopathological image detection, combining Transformer-based global modeling with CNN-based local feature extraction. Experiments report strong gastric-cancer classification performance, extended-dataset results, and a small-dataset limitation motivating future domain-adaptation and few-shot-learning work.
- Contributions and Method: GasHis-Transformer combines Transformer-based long-range correlation modeling with CNN-based local-information extraction in a unified image context.The lightweight variant is also proposed for gastric cancer detection in histopathological images.
- Gastric Cancer Detection: 97.97% and 96.43% accuracies were obtained by GasHis-Transformer and LW-GasHis-Transformer, respectively, on a gastric cancer histopathological dataset.The reported results indicate potential for gastric histopathological image detection tasks.
- Extended Experiments: 88.10% and 84.34% accuracies were achieved on breast cancer and lymphoma datasets by GasHis-Transformer, compared with 87.93% and 83.64% for its lightweight variant.The experiments used H&E- and IHC-stained histopathological images and were presented as evidence of generalizability.
- Extended Experiments: The extended experiments are described as demonstrating generalizability across H&E- and IHC-stained histopathological images.The reported datasets include breast cancer and lymphoma classification tasks.
- Future Work: Small datasets may degrade GasHis-Transformer classification performance, motivating future work using domain adaptation and few-shot learning.This limitation is explicitly identified in relation to the extended experiment.