Source-linked AI summary
Applying Faster R-CNN for Object Detection on Malaria Images
Jane Hung, Deepali Ravel, Stefanie C. P. Lopes, Gabriel Rangel, Odailton Amaral Nery, Benoit Malleret, Francois Nosten, Marcus V. G. Lacerda, Marcelo U. Ferreira, Laurent Rénia, Manoj T. Duraisingh, Fabio T. M. Costa, Matthias Marti, Anne E. Carpenter
TL;DR
The paper addresses the limited application of object detection to biological microscopy, where malaria cells are manually inspected amid imaging variation, uncertain classes, scarce annotations, and severe RBC imbalance. It applies ImageNet-pretrained Faster R-CNN, compares it with segmentation and random-forest classification, and reports stronger performance for the two-stage model than the baseline and human annotators on the evaluated cases.
Problem
Automated malaria-cell detection and staging is limited by challenging brightfield images, scarce expert annotations, uncertain classes, and class imbalance dominated by RBCs.
Method
The study fine-tunes ImageNet-pretrained Faster R-CNN and uses a two-stage pipeline that separates RBC-versus-other detection from fine-grained classification.
Results
98% total accuracy was achieved by the two-stage model, compared with 50% for the traditional segmentation-plus-machine-learning baseline and 72% for expert annotators on the reported evaluated cells.
Takeaways & Limitations
Faster R-CNN outperformed the traditional baseline, and the two-stage model exceeded expert annotator accuracy for non-difficult infected cells in this evaluation.
Takeaways & Limitations
Further validation is needed using more reliable ground truth and samples prepared in different laboratories to test robustness.
Abstract
from arXiv · showhide
Deep learning based models have had great success in object detection, but the state of the art models have not yet been widely applied to biological image data. We apply for the first time an object detection model previously used on natural images to identify cells and recognize their stages in brightfield microscopy images of malaria-infected blood. Many micro-organisms like malaria parasites are still studied by expert manual inspection and hand counting. This type of object detection task is challenging due to factors like variations in cell shape, density, and color, and uncertainty of some cell classes. In addition, annotated data useful for training is scarce, and the class distribution is inherently highly imbalanced due to the dominance of uninfected red blood cells. We use Faster Region-based Convolutional Neural Network (Faster R-CNN), one of the top performing object detection models in recent years, pre-trained on ImageNet but fine tuned with our data, and compare it to a baseline, which is based on a traditional approach consisting of cell segmentation, extraction of several single-cell features, and classification using random forests. To conduct our initial study, we collect and label a dataset of 1300 fields of view consisting of around 100,000 individual cells. We demonstrate that Faster R-CNN outperforms our baseline and put the results in context of human performance.
1. Introduction
The paper targets automated cell detection and class recognition in malaria microscopy, where manual expert inspection remains standard and biological image data has seen limited application of state-of-the-art object detectors. It addresses variable imaging conditions, uncertain classes, scarce annotations, and severe class imbalance by applying Faster R-CNN and collecting a large labeled dataset.
- Motivation: Manual inspection of malaria microscopy images remains the gold standard, motivating automated single-cell classification and counting.The intended benefits are faster, more accurate quantitative results without human variability.
- Challenges: Brightfield malaria images vary in illumination, cell shape, density, and color, while some cell classes remain uncertain even to experts.
- Challenges: Training is constrained by scarce expert annotations and an inherently imbalanced class distribution dominated by uninfected RBCs.
- Prior work: Earlier malaria-automation workflows relied on handcrafted intensity, shape, and texture features, but lacked generalizability and reproducibility.The field also lacked standardized image sets and reporting metrics.
- Approach: The study applies Faster R-CNN and introduces a dataset of 1300 microscopy images containing 100,000 individual cells.A traditional segmentation-and-random-forest pipeline serves as the baseline.
2. Data
The dataset comprises ex vivo P. vivax samples from three laboratories and expert-labeled microscopy images covering relevant blood-cell and parasite-stage categories. The data are strongly dominated by RBCs, which account for about 97% of all cells.
- Data sources: Samples came from P. vivax-infected patients in Manaus, Brazil, and Thailand, with Manaus and Thailand used for training and validation and Brazil reserved for testing.
- Annotations: Expert annotation covered RBC, leukocyte, gametocyte, ring, trophozoite, and schizont categories, with unclear cells marked difficult and excluded from training.
- Class distribution: 97% of the approximately 100,000 cells were RBCs, creating a naturally imbalanced object-class distribution.
- Annotations: Figure 1 uses colored annotations to represent different class labels in an example microscopy image.
3. Establish a Baseline with Traditional Method
The traditional baseline begins with cell segmentation, but example outputs show errors when cells are split or merged. These segmentation errors are central to interpreting the baseline approach.
- Segmentation: The baseline illustrates cell segmentation by comparing an example image before and after segmentation.
- Segmentation errors: Some segmentation errors split one cell into multiple detected objects.
- Segmentation errors: Other segmentation errors merge multiple cells into a single detected object.
4. Deep Learning
The deep-learning approach uses Faster R-CNN for object proposals and detection, with a two-stage design separating RBC-versus-other detection from fine-grained classification. Training uses cropped, augmented images and ImageNet pretraining followed by fine-tuning.
- Training data: 448x448 crops augment training examples and reduce training time compared with using full-sized images.Crops were sampled to increase the number of cells represented, up to a maximum of 100 crops per full image.
- Training data: Rotating crops with underrepresented classes increased those cell counts by roughly 4 times, while crops containing only RBCs were removed.
- Faster R-CNN: Faster R-CNN combines a Region Proposal Network with a Fast R-CNN detector that classifies proposed object regions while sharing convolutional layers.
- Two-stage detection and classification: The model first detects bounding boxes and labels objects as RBC or other, then classifies other detections into finer categories using AlexNet features.The second stage uses a 4096-dimensional feature vector for fine-grained classification.
- Training: Deep-learning models were pretrained on ImageNet and fine-tuned on the study’s training data.
- Training: The two-stage model augmented individual cell images using rotations, flips, shifts, color-channel changes, and scale changes.
5. Results and Discussion
The traditional segmentation-plus-classification baseline reached 50% accuracy, while Faster R-CNN achieved 59% in one-stage classification and 98% after separating RBC detection from detailed infected-cell classification. The two-stage model also exceeded expert annotator accuracy on non-difficult infected cells, though further validation and cross-lab robustness testing remain necessary.
- 5.1. Baseline: 50% accuracy was achieved by the traditional segmentation-plus-machine-learning baseline against matched segmentation objects and ground truth.This baseline omits mis-segmentation error, so its accuracy is an overestimate relative to true ground truth.
- 5.2. Deep Learning: One-Stage Classification: 59% accuracy was achieved by one-stage Faster R-CNN classification after disregarding background, RBCs, and difficult cells.The learned features clearly separated RBCs from other classes but did not separate infected stages well.
- 5.3. Deep Learning: Two-Stage Classification: 98% total accuracy was achieved by the two-stage model, significantly improving on one-stage classification.Faster R-CNN first separated RBCs from other cells, while a separate image classifier made detailed classifications of non-RBC detections.
- 5.4. Comparison with Humans: 72% total accuracy was obtained by two expert annotators for non-difficult infected cells, below the two-stage model’s accuracy.The comparison places the model’s performance above human annotator accuracy for cases where humans were confident.
- 5.4. Comparison with Humans: Further validation is needed using more reliable ground truth and samples prepared in a different laboratory.The authors also identify an intended online tool as a future direction.