Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers

• Active learning lowers labeling costs by querying an oracle to label instances selected as informative for the model’s next iteration.Source: §1 · p. 1
• Transformer parameters make neural-network and text-classification query strategies computationally expensive, causing prohibitive turnaround times between labeling steps.Source: §1 · p. 1
• Pre-trained transformers can be fine-tuned with far less labeled data than training models from scratch, making them relevant to low-data active-learning settings.Source: §1 · p. 1
• The paper evaluates uncertainty-based query strategies with BERT and DistilRoBERTa across five sentence-classification benchmarks and five query strategies.Source: §1 · p. 1

• Gradient-based strategies require per-instance computation and scale with a transformer’s many parameters, making them especially expensive.Source: §2 · p. 2
• Transformer active learning is constrained by ensemble costs, gradient strategies’ parameter scaling, and prior evidence that neural networks are overconfident.Source: §2 · p. 2
• Related work reduced query costs through unlabeled-pool subsampling, larger query batches, or binary classification, but these choices reduce comparability or increase labeling effort.Source: §2 · p. 2
• Prediction entropy became a common baseline, yet uncertainty strategies had not been systematically analyzed with transformers or compared against Zhang et al. (2017)’s benchmarks.Source: §2 · p. 2
• The paper reevaluates neglected uncertainty strategies alongside prediction entropy, challenging prediction entropy’s status as the default baseline.Source: §2 · p. 2

Transformer-based Active Learning

• Pool-based active learning repeatedly selects informative unlabeled instances, obtains oracle labels, and retrains or updates the model using all labeled data.Source: §3 · p. 2

Query Strategies

• The evaluated strategies are Prediction Entropy, Breaking Ties, Least Confidence, Contrastive Active Learning, and Random Sampling.Source: §3 · pp. 2–3
• Breaking Ties selects instances with the smallest margin between the two most likely class probabilities, while Least Confidence selects instances whose top prediction has the lowest confidence.Source: §3 · pp. 2–3
• Contrastive Active Learning selects instances whose predicted class distributions differ most, on average, from those of their m nearest neighbors.Source: §3 · p. 3

Oracle

• The study uses standard text-classification tasks and reports dataset information, including dataset types and whether predefined test sets were adopted.Source: §3 · p. 3, Table 1 · p. 3, Table 2 · p. 3

Classification

• The experiments fine-tune BERT and the smaller DistilRoBERTa, which has 82M parameters versus BERT’s 336M, and also train a passive full-data model.Source: §3 · p. 3
• The classifier adds a fully connected projection layer and softmax output layer on top of a transformer’s “[CLS]” sentence representation, with active learning querying 25 instances per iteration.Source: §3 · p. 3

Experimental setup

• The experiments use five established benchmarks spanning binary and multi-class classification, with 25 initial training instances and 20 iterations querying 25 instances each.The benchmark set comprises AG’s News, Customer Reviews, Movie Reviews, Subjectivity, and TREC-6; early stopping uses 10% of labeled data for validation.Source: §4 · pp. 3–4

Results

• BERT generally learns faster than DistilRoBERTa, but DistilRoBERTa later reaches nearly comparable scores on every dataset except Movie Reviews.The curves compare both models with Prediction Entropy, Breaking Ties, Least Confidence, Contrastive Active Learning, and Random Sampling; the horizontal reference is full-data passive BERT.Source: §4 · p. 4, Figure 1 · p. 4
• Random Sampling can compete during early iterations on balanced data, yet is usually overtaken later and is less effective on the imbalanced TREC-6 dataset.The early advantage is illustrated for Customer Reviews, while the later pattern reverses in most cases.Source: §4 · p. 4
• Breaking Ties achieves the best average AUC rankings and scores with transformers, often also leading accuracy, while Least Confidence surpasses Prediction Entropy for DistilRoBERTa.Prediction Entropy is commonly used as a strong baseline, but Breaking Ties is on average superior and Least Confidence can compete with it for DistilRoBERTa.Source: §4 · p. 5
• Across five datasets, active-learning models use only 0.4% to 14% of training data while approaching or surpassing state-of-the-art results, except for the largest gap on AGN.The largest discrepancy from passive classification occurs on AGN, where active learning uses less than 1% of the data.Source: Table 3 · p. 4, §4 · p. 5
• BERT obtains the best AUC score on every dataset and substantially improves over Zhang et al. (2017), with Breaking Ties highest in two cases and Contrastive Active Learning in the other two.Table 4 reports best AUC scores averaged over five runs and compares them with Zhang et al. (2017).Source: §4 · p. 5, Table 4 · p. 5

• Uncertainty-based strategies achieved results close to state-of-the-art text classification on four of five datasets using only a small fraction of training data.Source: §5 · p. 5
• Breaking ties outperformed prediction entropy, challenging its status as the default uncertainty baseline for transformer-based active learning.Source: §5 · p. 5
• The findings show that uncertainty-based strategies remain effective with BERT and DistilRoBERTa for active learning on sentence-classification datasets.Source: §5 · p. 5

• Active learning supports targeted labeling, potentially reducing the data, budget, or time needed to create otherwise infeasible machine-learning models.Source: Ethical Considerations · p. 6
• Pretrained models may contain one or more types of bias, so bias must be kept in mind and mitigated when using them.Source: Ethical Considerations · p. 6

Supplementary Material

• The experiments can be reproduced using code referenced on the paper’s first page, with supplementary details including results.Source: Supplementary Material · p. 8

A Technical Environment

• Fine-tuning transformers and training KimCNN were performed on an NVIDIA GeForce RTX 2080 Ti GPU within a Python 3.8 and CUDA 11.1 environment.Source: A Technical Environment · p. 8

• The experiments used PyTorch, huggingface transformers, scikit-learn, scipy, and numpy.Source: B Implementation Details · p. 8
• The small-text library supported the active-learning and text-classification experiments.Source: B Implementation Details · p. 8

C Experiments

• Each experiment configuration combines a model, dataset, and query strategy, and is run five times.A class-balanced initial set supports the first model on the imbalanced TREC-6 dataset.Source: C Experiments · p. 8

C.1 Pre-Trained Models

• The study fine-tunes DistilRoBERTa and BERT-large as its pretrained transformer models.Source: §C.1 · p. 8
• Maximum sequence length is configured per dataset using the minimum multiple of ten covering at least 95% of sentences.Source: Table 6 · p. 8

C.2 Datasets

• The experiments use well-known text-classification and active-learning benchmarks accessed through gluonnlp and Hugging Face datasets.CR and SUBJ are obtained with gluonnlp, while AGN, MR, and TREC use Hugging Face datasets.Source: §C.2 · p. 9

C.3 Hyperparameters

• Transformer training uses 50 epochs for AGN, 15 for other datasets, AdamW, a 2e−5 learning rate, and batch size 12.Source: §C.3 · p. 9

D Standard Deviations and Runtimes

• Final accuracy and AUC with standard deviations are measured after the last active-learning iteration, while query-step runtimes are reported separately.Source: D Standard Deviations and Runtimes · p. 9

D.1 Evaluation Metrics

• The evaluation uses accuracy and area under the learning curve as its standard active-learning metrics.Source: §D.1 · p. 9