A Survey on Deep Active Learning: Recent Advances and New Frontiers

Dongyuan Li; Zhen Wang; Yankai Chen; Renhe Jiang; Weiping Ding; Manabu Okumura

Source-linked AI summary

A Survey on Deep Active Learning: Recent Advances and New Frontiers

Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, Manabu Okumura

arXiv:2405.00334v2May 1, 2024cs.LG

TL;DR

Deep active learning lacks a sufficiently comprehensive survey despite its goal of achieving strong performance with fewer human-labeled samples. This paper systematically collects and filters DAL studies, defines the task, organizes methods across five perspectives, reviews applications, and analyzes challenges. Its synthesis identifies DAL’s potential for few-shot or one-shot use with large pre-trained models and highlights unresolved issues including cold-start, scalability, noisy data, and unstable performance.

Problem
Recent DAL surveys have limited coverage of advances, emerging learning paradigms, applications, and challenges, motivating a comprehensive synthesis.
Method
The paper collects and filters DAL studies, formalizes the task, summarizes baselines and datasets, and classifies methods by five perspectives before reviewing applications and challenges.
Results
The survey finds that DAL has strong potential for few-shot or one-shot use with large pre-trained models, while reviewed studies report 10–20% labeled data can yield 5–10 times greater fine-tuning efficiency than full-data training.
Takeaways & Limitations
DAL provides a broad framework for reducing annotation requirements across NLP, computer vision, data mining, and other application areas.

Abstract

from arXiv · show

Active learning seeks to achieve strong performance with fewer training samples. It does this by iteratively asking an oracle to label new selected samples in a human-in-the-loop manner. This technique has gained increasing popularity due to its broad applicability, yet its survey papers, especially for deep learning-based active learning (DAL), remain scarce. Therefore, we conduct an advanced and comprehensive survey on DAL. We first introduce reviewed paper collection and filtering. Second, we formally define the DAL task and summarize the most influential baselines and widely used datasets. Third, we systematically provide a taxonomy of DAL methods from five perspectives, including annotation types, query strategies, deep model architectures, learning paradigms, and training processes, and objectively analyze their strengths and weaknesses. Then, we comprehensively summarize main applications of DAL in Natural Language Processing (NLP), Computer Vision (CV), and Data Mining (DM), etc. Finally, we discuss challenges and perspectives after a detailed analysis of current studies. This work aims to serve as a useful and quick guide for researchers in overcoming difficulties in DAL. We hope that this survey will spur further progress in this burgeoning field.

I. INTRODUCTION

This survey addresses the limited coverage of recent deep active learning research by systematically reviewing DAL foundations, methods, applications, and challenges. It organizes the field through paper filtering, formal definitions, a five-perspective taxonomy, application coverage, and challenge analysis.

Motivation and scope: DAL selects informative and representative unlabeled samples for oracle annotation, iteratively adding them to training data to reduce labeling costs while maintaining competitive performance.The pipeline trains on an initial labeled set, queries an unlabeled pool, obtains oracle labels, and retrains or fine-tunes the model.
Motivation and scope: Existing DAL surveys often emphasize specific tasks or domains while providing limited coverage of recent advances, emerging learning paradigms, and challenges.The survey positions itself as a comprehensive review of recent DAL developments and insights.
Survey organization: The survey formalizes DAL, summarizes influential baselines and datasets, and develops a taxonomy spanning annotation types, query strategies, architectures, learning paradigms, and training processes.Its taxonomy also analyzes the strengths and weaknesses of the categorized methods.
Applications and challenges: Applications are reviewed across NLP, computer vision, data mining, and other domains, alongside an analysis of emerging challenges and possible opportunities.The challenge discussion includes pipeline-related issues such as costly annotation, stopping strategies, and cold-start.
Findings and resources: The survey reports that DAL can support few-shot or one-shot use of large pre-trained models, with some studies using 10–20% of labeled data for fine-tuning and achieving 5–10 times greater efficiency than full-data training.These figures are reported as findings from reviewed studies rather than as a new experiment conducted by the survey.
Findings and resources: The authors provide a GitHub repository containing up-to-date DAL papers, code, and datasets to support research review and reuse.The repository accompanies the survey’s collection, taxonomy, application, and challenge coverage.

II. PAPER COLLECTION AND FILTERING

The survey constructs a DAL literature collection by combining keyword searches across multiple databases with manual relevance inspection and additional filtering. The process reveals growing interest in DAL.

Collection: Searches combine three-keyword queries from defined groups across Google Scholar, Scopus, Semantic Scholar, and Web of Science.The keyword groups include “Active Learning,” “Machine Learning,” and “Open-set.”
Filtering: 10,000 collected papers became 3,967 unique papers after duplicate removal, and manual abstract inspection identified 1,273 pertinent articles.The publication range spans January 2013 to March 2023, with up to 200 papers collected per query.
Collection: The publication trend shows growing interest in deep active learning.The trend is presented in Fig. 4 alongside the field’s keywords.

III. DEEP ACTIVE LEARNING

Pool-based DAL iteratively selects informative unlabeled samples, obtains oracle labels, retrains the model, and stops when the labeling budget or target performance is reached. The survey presents this process through a formal definition and Algorithm 1.

Procedure: The procedure initializes a labeled dataset with b samples, trains a model, and repeats acquisition and retraining for up to T iterations.Algorithm 1 specifies batch size b, iteration count T, and query function α as core inputs.
Querying: At iteration i, an acquisition function α uses the current model and unlabeled pool to select batch Q_i of size b.The selected batch is annotated before entering the training set.
Definition: Pool-based DAL selects informative samples from a large unlabeled pool until a performance target or predefined budget is reached.The survey focuses on pool-based methods because most DAL methods belong to this category.
Retraining: An oracle labels Q_i, the labeled samples are added to D_i_train, and the model is retrained on the expanded dataset.This update is repeated until the labeling budget is exhausted or desired performance is reached.

B. Comparisons between Traditional and Deep AL

Deep AL differs from traditional AL because its representations are learned dynamically, requiring representativeness measures to be recomputed during the process. The survey organizes important DAL baselines and datasets while covering major strategy, architecture, and learning-paradigm developments.

Comparisons between Traditional and Deep AL: Deep AL jointly learns feature representations with neural networks, so pairwise distances and similarities for representativeness must be recomputed during selection.Traditional AL generally uses fixed pre-processed features for uncertainty or representativeness calculations.
Baselines: The baseline taxonomy covers uncertainty, diversity, Bayesian, reinforcement-learning, transfer-learning, and semi-supervised approaches.Examples include DBAL, CoreSet, BatchBALD, PAL, AADA, and pseudo-labeling methods.
Pre-trained methods: Pre-trained language models enable DAL-based few-shot or zero-shot learning and can support fine-tuning with 10%∼20% labeled samples.The survey reports comparable or higher performance than full-dataset fine-tuning for one BERT-based text-classification study.
Datasets: The survey’s dataset coverage includes image, text, and audio datasets used in DAL research.Dataset information is summarized in Table II.

IV. TAXONOMY OF DAL

DAL query strategies organize sample selection around annotation types and five principal strategy families, each balancing informativeness, coverage, model impact, or computational cost differently.

Annotation types: Hard, soft, hybrid, explanatory, and random/multi-agent annotations define distinct ways to obtain labels or labeling decisions.
Query strategies: Uncertainty-based methods select ambiguous samples using predictive entropy, confidence, mutual information, or proximity to decision boundaries.
Query strategies: Uncertainty-focused selection can repeatedly choose redundant points, lack robustness to outliers, and generalize poorly across tasks.
Query strategies: Representative-based methods select prototypical samples using density or diversity to cover the unlabeled feature distribution.
Query strategies: Representative-only methods may overlook decision-boundary samples and have nearly quadratic computational complexity with respect to data size.
Query strategies: Influence-based methods estimate a sample’s effect on target-model performance through direct impact metrics, learning policies, or separate estimation models.
Query strategies: Bayesian methods use Bayes’ rule to estimate informativeness or model impact, while hybrid methods combine strategies but leave effective trade-offs unresolved.

C. Model Architecture )

DAL uses diverse model architectures, ranging from traditional statistical models and Bayesian neural networks to recurrent and convolutional neural networks.

Model architectures: Bayesian neural networks combine neural networks with Bayesian inference to quantify uncertainty in outputs and weights.
Model architectures: RNNs model long-term dependencies in sequential data but are seldom combined with DAL because they require large labeled datasets.
Model architectures: CNNs are feedforward networks that extract features through convolutional operations.

D. Learning Paradigm )

DAL learning paradigms range from vanilla supervised training to semi-supervised, contrastive, transfer, meta, and imitation learning, with each incorporating different sources of supervision or adaptation. These paradigms aim to reduce labeling effort, improve representations, adapt querying, or leverage expert decisions.

Traditional Learning Paradigm: Traditional learning iteratively queries and labels samples, then trains models in a vanilla supervised manner without advanced learning paradigms.
Semi-supervised Learning: Semi-supervised DAL jointly uses real-labeled and pseudo-labeled samples, reducing human labor while achieving performance comparable to supervised learning with larger labeled datasets.DBAL and CoreSet assess pseudo-label confidence, while LADA and BGADL generate augmented samples from labeled data.
Meta Learning: Meta learning supplies acquisition functions or favorable initialization by controlling knowledge transfer from multiple source tasks.Learning-to-Sample dynamically improves a boosting model and sampling model through iterative interaction.
Transfer Learning: Transfer learning adapts models across domains or tasks and can reduce annotation requirements by selecting target samples similar to the source distribution.
Imitation Learning: Imitation learning trains query policies by mimicking expert actions, assuming an expert can provide the optimal action for each queried state.The approach is used for structured prediction and can alleviate data needs while leveraging entity information.
Multi-task Learning: Multi-task adversarial DAL uses task and diversity discriminators to remove irregular task-specific features and satisfy sample-diversity constraints.

E. Training Process )

DAL training commonly alternates between model training, sample selection, annotation, and retraining until the sample budget or iteration limit is reached. The survey also relates these processes to applications spanning many domains and practical settings.

Traditional Training: Traditional training initializes a model, selects unlabeled samples using current predictions, adds annotations to the training set, and retrains iteratively.The process stops when the sample budget or number of DAL iterations is reached, with parameters randomly re-initialized before each retraining epoch.
Applications: DAL applications span agriculture, industry, artificial intelligence, biomedical fields, and other domains, motivating a broad application-oriented review.

A. Applications in Natural Language Processing

DAL is applied across NLP tasks to reduce annotation and resource demands while maintaining or improving task performance. The surveyed examples include text classification, summarization, question answering, information extraction, and broader data-centric applications.

Overview: DAL helps fine-tune large language models by searching for high-quality small samples, saving data, computation, and time.
Text Classification: Text classification methods select or generate informative examples, with some reducing annotation cost by presenting annotators only a few summary words.
Abstractive Text Summarization: In abstractive text summarization, DAL reduces annotation needs through uncertainty estimation or diversity-based sampling while targeting strong summarization performance.Monte Carlo dropout approximates Bayesian inference for uncertainty estimation, while in-domain diversity sampling selects dissimilar instances that resemble domain-core documents.
Question Answering: Question-answering methods such as DataMap and hierarchical dialog policies seek high performance with minimal labeling effort.DataMap detects and removes outlier examples, while interactive clarification during querying improves model performance.
Information Extraction: Information extraction uses alternative annotation strategies to address the time and expense of manually labeling large-scale sequences.
Computer Vision: Across computer-vision applications discussed alongside NLP, DAL reduces noise and time consumption as image datasets and model training become increasingly costly.

C. Applications in Graph Data Mining and Learning

Graph data mining applications use DAL to select informative entities, metadata, or cross-domain samples while reducing annotation costs. The surveyed scope also connects these methods to engineering and treatment-effect settings, alongside identified challenges and opportunities.

Graph Data Mining: Graph-based DAL represents network entities with neural embeddings and uses multiple query strategies to select informative user pairs for labeling and training.
Graph Data Mining: Multi-view graph DAL selectively queries informative metadata and maps visual information to text, using richer views than single-view methods.
Graph Data Mining: Transfer-based DAL for recommender-system link prediction iteratively selects source entities for target systems using uncertainty criteria.
Engineering Systems: Engineering applications use variance-based weighted and D-optimal weighted active learning algorithms for uncertain Gaussian-process models.Numerical studies report improved predictive modeling for automatic shape control.
Treatment Effects: Personalized treatment-effect DAL uses uncertainty-aware acquisition methods, including expected information gain over the S-type error rate.
Challenges and Opportunities: The survey organizes application challenges together with potential solutions and future opportunities in a consolidated table.

A. Pipeline-related Issues )

Pipeline-related issues constrain DAL through human annotation costs, stopping-strategy gaps, cold-start sensitivity, transfer difficulties, and unstable initialization. Proposed remedies include pseudo-labeling, stopping rules, pretrained knowledge, diverse initialization, and hybrid query strategies.

Annotation efficiency: Human annotation is time-consuming, expert-dependent, and can suspend DAL when annotators are unavailable.Pseudo-labeling and multiple pseudo-annotators reduce reliance on human experts.
Stopping strategies: Stopping strategies limit labeling effort and avoid noisy or redundant samples, using classifier confidence to determine when DAL should halt.One proposed strategy stops when mean confidence over remaining unlabeled documents stabilizes.
Cross-domain transfer: Cross-domain transfer is difficult because DAL is model-dependent and transfer learning can bias selections toward the source-domain distribution.Joint disagreement, multi-agent reinforcement learning, multi-task transfer, and corrective weighting are proposed responses.
Initialization and stability: Diverse or pretrained initialization methods aim to stabilize DAL performance, while TrustAL transfers distilled knowledge to data selection for more consistent results.TrustAL jointly optimizes knowledge distillation and DAL across three benchmarks.
Query strategies: TYROGUE combines diversity and uncertainty to select samples near decision boundaries, addressing the limitations of using either criterion alone.The method is designed to improve generalization while reducing labeling costs.

C. Dataset-related Issues )

Dataset-related issues arise from outliers, noisy oracles, scarce data, difficult annotations, class imbalance, and mismatch between labeled and unlabeled class distributions. Methods respond with filtering, augmentation, pseudo-labeling, and distribution-aware selection.

Outliers and noisy oracles: DAL can repeatedly select outliers because they receive high uncertainty scores, damaging learning and creating a selection feedback loop.Noisy or inaccurate annotations further threaten model robustness in real-world settings.
Outliers and noisy oracles: MQ-Net balances sample purity and informativeness to filter noisy open-set data, while other methods remove redundancy or outliers through diversity and knowledge distillation.Annotation quality is also addressed through relabeling support and annotator training.
Data scarcity and annotation difficulty: Scarce or difficult-to-label datasets motivate data augmentation, pretrained-language-model fine-tuning, and self-supervised pseudo-label estimators.These approaches expand training data, reduce required labels, or correct acquisition bias from distribution shift.
Class distribution mismatch: Class distribution mismatch occurs when labeled and unlabeled data do not share the same categories, causing DAL to select irrelevant uncertain samples and waste annotation budget.Energy discrepancy and teacher-student distillation are used to avoid selecting samples from unseen classes.

VII. CONCLUSION

The survey organizes DAL research through a broad synthesis of methods, applications, challenges, and resources. It concludes that DAL is useful across domains but still requires refinement in pipeline, task, and dataset settings.

Conclusion: The survey covers paper selection, DAL definitions, influential baselines, datasets, method taxonomies, applications, challenges, and future prospects.Its taxonomy analyzes query strategies, model architectures, learning paradigms, annotation strategies, and training processes.
Conclusion: DAL methods offer efficiency, effectiveness, and robustness, but unresolved issues include annotation cost, transfer, stability, scalability, imbalance, and class distribution mismatch.The authors provide guidelines and DAL-related resources while calling for further work on these problems.