Source-linked AI summary
Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, Danny Z. Chen
TL;DR
Biomedical image segmentation requires costly expert annotation, and the paper asks which instances should be labeled under limited effort. It combines FCNs with active learning, using FCN-derived uncertainty and similarity within a generalized maximum set cover formulation. On gland and lymph-node ultrasound datasets, the method achieves state-of-the-art segmentation performance using 50% of training data.
Problem
Biomedical segmentation requires costly expert annotation, especially because images can contain many object instances and datasets have limited spatial annotations.
Method
The framework iteratively trains FCNs, estimates uncertainty and similarity, and formulates annotation selection as a generalized maximum set cover problem.
Results
50% of training data achieves state-of-the-art segmentation performance on the 2015 MICCAI Gland Challenge and lymph-node ultrasound datasets.
Takeaways & Limitations
Annotation suggestions direct manual effort toward effective areas while reducing the amount of training data used for state-of-the-art segmentation.
Abstract
from arXiv · showhide
Image segmentation is a fundamental problem in biomedical image analysis. Recent advances in deep learning have achieved promising results on many biomedical image segmentation benchmarks. However, due to large variations in biomedical images (different modalities, image settings, objects, noise, etc), to utilize deep learning on a new application, it usually needs a new set of training data. This can incur a great deal of annotation effort and cost, because only biomedical experts can annotate effectively, and often there are too many instances in images (e.g., cells) to annotate. In this paper, we aim to address the following question: With limited effort (e.g., time) for annotation, what instances should be annotated in order to attain the best performance? We present a deep active learning framework that combines fully convolutional network (FCN) and active learning to significantly reduce annotation effort by making judicious suggestions on the most effective annotation areas. We utilize uncertainty and similarity information provided by FCN and formulate a generalized version of the maximum set cover problem to determine the most representative and uncertain areas for annotation. Extensive experiments using the 2015 MICCAI Gland Challenge dataset and a lymph node ultrasound image segmentation dataset show that, using annotation suggestions by our method, state-of-the-art segmentation performance can be achieved by using only 50% of training data.
1 Introduction
Biomedical image segmentation requires costly expert annotation, motivating active selection of training areas. The paper combines FCNs with active learning to select uncertain, representative areas and reports state-of-the-art performance using 50% of training data.
- Motivation: Biomedical segmentation data are costly to acquire because trained experts must annotate images containing many object instances.Public biomedical datasets may contain as few as 85 spatially annotated MICCAI Gland Challenge images and 30 ISBI EM Challenge images.
- Framework: The proposed framework combines FCNs and active learning to suggest effective annotation areas.It iteratively trains FCNs and uses information extracted from them to choose the next batch of images.
- Framework: The framework uses FCNs to obtain domain-specific descriptors and directly generate segmentation without region proposals.This addresses issues identified for earlier active-learning methods based on pretrained region proposals.
- Selection strategy: Uncertainty and similarity information are combined to formulate a generalized maximum set cover problem for annotation selection.The framework uses these signals to determine which areas are both uncertain and representative.
- Results: Experiments on gland and lymph-node ultrasound datasets show that annotation suggestions outperform random and uncertainty queries.The reported comparison covers both datasets and common query strategies.
- Results: 50% of training data is sufficient for the framework to achieve state-of-the-art segmentation performance.This result is reported across experiments on the 2015 MICCAI Gland Challenge and lymph-node ultrasound datasets.
2 Method
The method combines an FCN with active learning to select annotation areas that are both uncertain and representative. It estimates uncertainty and image similarity from FCN features, then uses a generalized maximum set cover formulation to choose samples.
- The framework combines a new FCN, uncertainty and similarity estimation, and an annotation suggestion algorithm.
- The FCN is designed for faster training and better generality using residual networks, batch normalization, and bottleneck modules.
- Uncertainty estimation: Bootstrapped disagreement estimates FCN uncertainty, which is strongly correlated with testing errors; sample uncertainty is the mean pixel uncertainty.
- Similarity estimation: Similarity is estimated from channel-wise mean features in the FCN encoding layer, producing domain-specific descriptors with shape information.
- Annotation suggestion: The selection procedure first retains the top K uncertain images, then chooses k representative images from that candidate set.
- Annotation suggestion: Representativeness rewards selections that are similar to many unannotated images and cover diverse cases.
- Annotation suggestion: Maximizing representativeness is formulated as a generalized maximum set cover problem and solved with a greedy method.
3 Experiments and Results
Experiments evaluate the framework on gland and lymph node ultrasound segmentation datasets under limited annotation budgets. The method consistently outperforms random and uncertainty-only querying and reaches state-of-the-art performance with 50% of the training data.
- Experiments use the 2015 MICCAI Gland Challenge dataset and a lymph node ultrasound segmentation dataset.The datasets contain 85 and 37 training images, respectively.
- Gland segmentation: The gland experiment compares random query, uncertainty query, and the proposed annotation suggestion under 10%, 30%, and 50% labeled-pixel budgets.
- Gland segmentation: The proposed annotation suggestion is consistently better than random query and uncertainty query for gland segmentation.
- Gland segmentation: 50% of the training data is sufficient for the framework to achieve state-of-the-art gland segmentation performance.
- Lymph node segmentation: Under a 50% budget, the framework outperforms uncertainty querying in all reported lymph node segmentation cases and exceeds the state-of-the-art method.
4 Conclusions
The paper presents a deep active learning framework combining FCNs with an annotation suggestion approach that directs manual effort to effective annotation areas.
- The framework combines fully convolutional networks with active learning for biomedical image segmentation.
- The new FCN model attains state-of-the-art segmentation performance.
- Its annotation suggestion approach directs manual annotation efforts to the most effective annotation areas.