Source-linked AI summary
Accurate Leukocyte Detection Based on Deformable-DETR and Multi-Level Feature Fusion for Aiding Diagnosis of Blood Diseases
Yifei Chen, Chenyan Zhang, Ben Chen, Yiyu Huang, Yifei Sun, Changmiao Wang, Xianjun Fu, Yuxing Dai, Feiwei Qin, Yong Peng, Yu Gao
TL;DR
Manual leukocyte isolation for blood-image analysis is time-consuming and error-prone, motivating automated detection that handles scarce features and scale disparities. MFDS-DETR combines leukocyte-specific multilevel fusion with deformable attention, and the authors report superior results across three datasets while noting the need for larger, more varied validation data.
Problem
Manual leukocyte isolation is time-consuming and susceptible to image-quality and environmental errors, while leukocyte images present scarce features and scale disparities.
Method
MFDS-DETR combines HS-FPN multilevel fusion with deformable self-attention and self- and cross-attention mechanisms across its encoder and decoder.
Results
MFDS-DETR outperforms advanced and baseline leukocyte detection models on the LISC, BCCD, and WBCCD datasets.
Takeaways & Limitations
The reported results support MFDS-DETR’s effectiveness and broad applicability for leukocyte detection across public and private datasets.
Takeaways & Limitations
The authors identify a need for larger and more varied datasets to further validate MFDS-DETR’s robustness and generalizability.
Abstract
from arXiv · showhide
In standard hospital blood tests, the traditional process requires doctors to manually isolate leukocytes from microscopic images of patients' blood using microscopes. These isolated leukocytes are then categorized via automatic leukocyte classifiers to determine the proportion and volume of different types of leukocytes present in the blood samples, aiding disease diagnosis. This methodology is not only time-consuming and labor-intensive, but it also has a high propensity for errors due to factors such as image quality and environmental conditions, which could potentially lead to incorrect subsequent classifications and misdiagnosis. To address these issues, this paper proposes an innovative method of leukocyte detection: the Multi-level Feature Fusion and Deformable Self-attention DETR (MFDS-DETR). To tackle the issue of leukocyte scale disparity, we designed the High-level Screening-feature Fusion Pyramid (HS-FPN), enabling multi-level fusion. This model uses high-level features as weights to filter low-level feature information via a channel attention module and then merges the screened information with the high-level features, thus enhancing the model's feature expression capability. Further, we address the issue of leukocyte feature scarcity by incorporating a multi-scale deformable self-attention module in the encoder and using the self-attention and cross-deformable attention mechanisms in the decoder, which aids in the extraction of the global features of the leukocyte feature maps. The effectiveness, superiority, and generalizability of the proposed MFDS-DETR method are confirmed through comparisons with other cutting-edge leukocyte detection models using the private WBCDD, public LISC and BCCD datasets. Our source code and private WBCCD dataset are available at https://github.com/JustlfC03/MFDS-DETR.
1. Introduction
Leukocyte detection supports blood-disease diagnosis but is hindered by manual isolation, image variability, scarce features, and scale differences. MFDS-DETR addresses these challenges through HS-FPN, deformable attention, new datasets, and multi-dataset evaluation.
- Manual leukocyte isolation is labor-intensive and time-consuming, while image quality and environmental conditions can cause errors affecting subsequent classification.
- Leukocyte detection is challenged by hospital-dependent color variation, limited discernible features, varying magnification, and differences in leukocyte sizes.
- MFDS-DETR introduces HS-FPN, which uses high-level features to filter low-level information through channel attention before feature fusion.
- MFDS-DETR combines multi-scale fusion and deformable self-attention in a backbone, HS-FPN, encoder, and decoder architecture.
- The study labels LISC target frames and develops the WBCCD leukocyte detection dataset for access by other researchers.
- MFDS-DETR outperforms advanced and baseline leukocyte detection models on LISC, BCCD, and WBCCD, supporting its reported effectiveness and broad applicability.
2. Related Work
Prior leukocyte detectors commonly use single-stage or CNN-based approaches, while existing multi-scale fusion methods largely originate from natural-image detection. MFDS-DETR targets their limitations by combining leukocyte-specific fusion with deformable attention for global feature extraction.
- CNN-based leukocyte detectors can struggle to learn global image features because convolutional operators emphasize local information.
- Low-resolution microscopy images and leukocyte characteristics produce insufficient features, while hospital magnification differences and nonuniform leukocyte sizes create scale gaps.
- Existing multi-scale fusion methods use parallel branches or serial connections but are fundamentally designed for natural images.
- MFDS-DETR fuses multiscale features with HS-FPN, uses deformable self-attention for global features, and decodes leukocyte locations and classes.
3.1. Overall Architecture
MFDS-DETR consists of a backbone, HS-FPN, encoder, and decoder. The architecture extracts multiscale leukocyte features, filters low-level information with high-level semantics, and supports global feature extraction and detection.
- MFDS-DETR’s overall architecture comprises the backbone network, HS-FPN, encoder, and decoder.
- The backbone extracts multiscale leukocyte image features for subsequent feature fusion.
- HS-FPN uses channel attention to apply high-level semantic features as weights for filtering low-level features before merging them with high-level features.
3.2. Backbone Network
MFDS-DETR uses an enhanced ResNet-50 backbone for feature extraction. An added convolution block extracts deeper semantic information to address the paucity of features in leukocyte images.
- An enhanced ResNet-50 serves as MFDS-DETR’s backbone network for leukocyte feature extraction.
- The backbone adds a convolution block to the original ResNet-50 to extract deeper semantic information and improve detection effectiveness.
- The added block uses sequential 1 × 1, 3 × 3, and 1 × 1 convolutions to reduce channels, shrink feature maps, and then increase channels.
3.3. High-level Screening-feature Pyramid Networks
HS-FPN addresses leukocyte scale variation by screening low-level features with high-level semantic information before multi-scale fusion. Its feature-selection and feature-fusion components combine complementary localization and semantic cues for detecting subtle leukocyte characteristics.
- HS-FPN targets diameter differences among leukocytes and size changes caused by different microscopes.
- The network contains feature-selection and feature-fusion modules that respectively screen multi-scale maps and integrate high- and low-level information.
- Channel attention converts high-level semantic features into weights that filter low-level features before pointwise fusion.
- The dimensional-matching module uses 1×1 convolution to reduce feature maps at each scale to 256 channels.
- The SFF mechanism combines transposed convolution and bilinear interpolation to restore high-level feature scale for purposeful fusion.
3.4. Deformable Self-attention Module
The deformable self-attention module uses learned offsets and weighted sampling to process multi-scale leukocyte features. Its offset and attention components support scale-aware feature extraction from encoder inputs.
- The module comprises an Offset Module and an Attention Module.
- Offset Module: The Offset Module predicts sampling offsets from query vectors and reference-point coordinates, then obtains sampled features through bilinear interpolation.
- Attention Module: The Attention Module applies Softmax weights to offset outputs, aggregates them, concatenates attention heads, and produces the final vector.
- The multi-level deformable attention module learns offsets across input scales using normalized reference-point positions.
3.5. Encoder and Decoder
The encoder extracts global leukocyte features from multi-level maps, while the decoder relates feature representations to target locations and classes. Both stages use attention-based processing within layered architectures.
- Encoder: The encoder receives multi-level feature maps with spatial-location and scale encoding to extract global leukocyte features.
- Encoder: Each encoder layer combines deformable self-attention with a feed-forward network, using normalized reference points and eight attention heads.
- Decoder: The decoder establishes relationships among detected representations and identifies target locations and classes.
- Decoder: Each decoder layer contains self-attention and cross-attention feature-extraction components alongside feed-forward processing.
3.6. Joint Loss Function
The joint loss combines classification, box-regression, and auxiliary objectives. Focal loss addresses the positive–negative imbalance, while the regression formulation combines GIoU and L1 losses and auxiliary supervision accelerates training.
- The final objective comprises classification loss, box-regression loss, and auxiliary loss.
- Classification Loss: Focal loss addresses the imbalance created by 100 decoder queries versus typically 2 or 3 leukocytes per image.
- Classification Loss: Focal loss uses α for category-proportion compensation and γ=2 to weight difficult and straightforward samples.
- Regression Loss: The regression loss combines GIoU and L1 terms to reduce sensitivity to leukocyte input-image size.
- Auxiliary Loss: Auxiliary loss computes classification and regression losses at each decoder output layer to expedite model convergence.
4. Experiment
The experiments evaluate MFDS-DETR on WBCDD, LISC, and BCCD against established detection models and assess generalization and component contributions. MFDS-DETR achieves strong detection performance, while ablations support the roles of HS-FPN, encoder layers, position encoding, and auxiliary losses.
- Dataset: 364 BCCD images contain leukocytes, red blood cells, and platelets, with dense distributions, adhesion, and occlusion; 292 images were used for training and 72 for testing.BCCD annotations cover both blood cells and platelets, unlike the first two datasets.
- Comparison setup: MFDS-DETR was compared with Faster R-CNN, SSD, RetinaNet, DETR, Deformable DETR, TE-YOLOF, and YOLOv5-ALT across WBCDD, LISC, and BCCD.LISC and BCCD were used to evaluate generalizability, while WBCDD was assembled for this research.
- Comparison results: 79.7% AP and 97.2% AP50 were achieved on WBCDD, improving over Faster R-CNN by 21.5% and 23.5%, respectively.The authors attribute these results to multi-scale and global feature extraction.
- Generalization results: On LISC, MFDS-DETR led in AP and AP75, while on BCCD it achieved optimal results across all reported indicators.Its LISC AP50 was 0.1% below Faster R-CNN and Deformable DETR, and neutrophils were the exception among cell types.
- Ablation studies: Removing the encoder decreased AP by 2.8%, AP50 by 3.2%, and AP75 by 3.0%, while removing spatial or scale encoding reduced AP by 3.1% and 3.4%.Ablations also found that omitting auxiliary loss reduced AP by 4.5%, with GIoU loss more influential than L1 loss.
4.5. Model visualization analysis
The visualization analysis presents MFDS-DETR’s predicted leukocyte categories, locations, and confidence levels against ground-truth boxes, while accompanying tables compare positional codes and loss functions using AP metrics.
- Visual prediction analysis: Figure 11 compares MFDS-DETR predictions with ground-truth categories and target boxes on WBCDD images.Ground-truth boxes are black; other boxes show predicted results and confidence levels.
- Visual prediction analysis: Green, orange, purple, blue, and yellow boxes represent lymphocytes, neutrophils granulocytes, eosinophils, basophils, and monocytes, respectively.
- Ablation comparisons: Table 9 compares AP, AP50, and AP75 across different positional codes.
- Ablation comparisons: Table 10 compares AP, AP50, and AP75 across different loss functions.
- Visual prediction analysis: MFDS-DETR produces high-confidence predictions and precise positioning for all five leukocyte types shown in the WBCDD visualization.
5. Conclusion
The paper concludes that MFDS-DETR combines multi-level feature fusion, deformable attention, and joint losses for leukocyte detection, evaluated through comparisons, ablations, and visualization. Its broader validation remains constrained by dataset size and quality, motivating larger and more varied datasets and continued model adaptation.
- Conclusion: MFDS-DETR comprises a backbone, HS-FPN, encoder, decoder, and joint loss function for leukocyte detection.HS-FPN screens low-level features using high-level feature maps, while deformable attention supports global-feature extraction and target-location learning.
- Conclusion: The joint loss integrates classification, regression, and auxiliary losses, with auxiliary loss computing decoder losses at each layer to expedite convergence.
- Conclusion: Comparison experiments evaluate MFDS-DETR against advanced leukocyte detection models on WBCDD, LISC, and ALL-IDB, with WBCDD ablations examining key components.
- Limitations: The available datasets constrain leukocyte-assay progress through limited scale, dense scenes, adhesion, occlusion, and substandard image quality.The paper releases WBCCD to provide a higher-quality resource for researchers.
- Limitations: Future validation should use larger and more varied datasets, while the model requires ongoing refinement as medical imaging and deep learning advance.