PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Learning by Acquiring Contrastive Examples

Katerina Margatina, Giorgos Vernikos, Loïc Barrault, Nikolaos Aletras

arXiv:2109.03764v1cs.CLcs.AIcs.LG

TL;DR

Active learning must choose informative data from unlabeled pools while balancing uncertainty and diversity. CAL selects feature-space-similar examples with maximally different predictive likelihoods, and it performs consistently better or equal to baselines across in-domain and out-of-domain tasks. The analysis reports a better uncertainty–diversity trade-off, while noting that no acquisition function consistently wins by a large margin.

  • Problem

    Uncertainty and diversity acquisition can select redundant, uninformative, or easy examples, motivating a better balance between these criteria.

  • Method

    CAL acquires contrastive examples that are similar in model feature space but have maximally different predictive likelihoods.

  • Results

    CAL performs consistently better or equal to all baselines across tasks in both in-domain and out-of-domain settings.

  • Takeaways & Limitations

    CAL achieves a better trade-off between uncertainty and diversity than other acquisition strategies.

  • Takeaways & Limitations

    The authors note that no single acquisition function consistently outperforms all others by a large margin.

Abstract

from arXiv · show

Common acquisition functions for active learning use either uncertainty or diversity sampling, aiming to select difficult and diverse data points from the pool of unlabeled data, respectively. In this work, leveraging the best of both worlds, we propose an acquisition function that opts for selecting \textit{contrastive examples}, i.e. data points that are similar in the model feature space and yet the model outputs maximally different predictive likelihoods. We compare our approach, CAL (Contrastive Active Learning), with a diverse set of acquisition functions in four natural language understanding tasks and seven datasets. Our experiments show that CAL performs consistently better or equal than the best performing baseline across all tasks, on both in-domain and out-of-domain data. We also conduct an extensive ablation study of our method and we further analyze all actively acquired datasets showing that CAL achieves a better trade-off between uncertainty and diversity compared to other strategies.

1 Introduction

Active learning seeks informative unlabeled examples while reducing annotation cost, but uncertainty and diversity strategies can select redundant or easy data. CAL addresses this trade-off by acquiring contrastive examples and performs at least as well as baselines across evaluated tasks and settings.

  • Active learning concentrates annotation on informative points from a large unlabeled pool to reduce annotation cost.
  • Uncertainty sampling may select repetitive uncertain examples, whereas diversity sampling may select diverse but easy examples.
  • CAL selects unlabeled examples whose feature-space neighbors have maximally divergent predictive likelihoods.
  • CAL was evaluated on seven datasets spanning sentiment analysis, topic classification, natural language inference, and paraphrase detection, including in-domain and out-of-domain tests.
  • CAL performs consistently better or equal to all baselines across tasks in both in-domain and out-of-domain settings.
  • CAL achieves a better trade-off between diversity and uncertainty than the baseline acquisition functions.

2 Contrastive Active Learning

CAL defines contrastive examples through feature-space similarity and predictive disagreement, then ranks unlabeled candidates by divergence from nearby labeled examples. Its scoring procedure targets candidates suggestive of decision-boundary proximity, while the paper notes limitations around strict boundary guarantees and possible outlier selection.

  • 2.1 Contrastive Examples: Contrastive examples have similar model encodings but very different model predictions.
  • 2.1 Contrastive Examples: The similarity criterion uses an encoder, a distance metric, and a small threshold ϵ to identify nearby examples.
  • 2.1 Contrastive Examples: The second criterion measures divergence between predictive probability distributions using Kullback-Leibler divergence.
  • 2.1 Contrastive Examples: In a binary example, x1 with probabilities (0.8, 0.2) is more contrastive with x3, (0.6, 0.4), than with x2, (0.7, 0.3).
  • 2.2 Active Learning Loop: CAL does not guarantee that contrastive examples lie on the decision boundary, and adding neighbor distances is proposed to mitigate possible outlier selection.
  • 2.2 Active Learning Loop: CAL repeatedly trains on labeled data, scores pool candidates against nearby labeled examples, and acquires a batch for annotation.
  • 2.2 Active Learning Loop: For each pool candidate, CAL uses BERT [CLS] embeddings, Euclidean distance, and K-nearest neighbors to construct a labeled neighborhood.
  • 2.2 Active Learning Loop: CAL averages KL divergences against neighborhood members and selects the top b candidates by score.

3 Experimental Setup

The experiments evaluate CAL across four natural language understanding tasks and seven datasets, comparing it with uncertainty-, diversity-, and hybrid-based acquisition functions under in-domain and out-of-domain settings.

  • CAL is evaluated on sentiment analysis, topic classification, natural language inference, and paraphrase detection across seven datasets.
  • The study uses SST-2 as out-of-domain data for IMDB, IMDB as out-of-domain data for SST-2, and TWITTERPPDB as out-of-domain data for QQP.
  • CAL is compared with ENTROPY, BERTKM, BADGE, and ALPS, representing uncertainty-, diversity-, and hybrid-based acquisition strategies.
  • The model uses BERT-BASE with a task-specific classification layer and is evaluated on held-out test sets using five random seeds.
  • BERTKM and BADGE are omitted from large-pool datasets because high-dimensional clustering becomes computationally heavy and complexity grows exponentially with acquisition size.
  • Each experiment uses a 15% Dpool budget, a 1% initial training set, and an acquisition size of b = 2%.

4 Results

CAL is consistently competitive or superior in in-domain and out-of-domain evaluation, while its advantage varies across datasets and is accompanied by a stronger uncertainty–diversity trade-off.

  • CAL is consistently the top-performing method across datasets, especially on DBPEDIA, PUBMED, and AGNEWS.
  • CAL performs slightly better than ENTROPY on IMDB, QNLI, and QQP, while most methods yield similar results on SST-2.
  • ENTROPY is the second-best acquisition function overall and generally outperforms diversity-based and hybrid baselines.
  • CAL achieves a better diversity–uncertainty trade-off than the baselines, while BERTKM, BADGE, and ALPS show weaker acquisition performance in the reported comparisons.
  • A future direction is testing which pretrained-language-model representations best support similarity search and clustering.
  • Out-of-domain Performance: CAL achieves the highest SST-2-to-IMDB out-of-domain performance by a large margin, whereas ENTROPY is highest in the reverse direction.
  • Out-of-domain Performance: CAL obtains the highest QQP out-of-domain accuracy among RANDOM, ENTROPY, and ALPS, and overall matches or exceeds other approaches on OOD data.

5 Ablation Study

The ablation study tests CAL’s decision-boundary hypothesis, neighborhood construction, scoring function, and feature space, finding that the original contrastive design is generally strongest.

  • Decision Boundary: Reversing CAL’s ranking to select minimum rather than maximum divergence leaves performance unchanged or degraded after acquiring 15% of unlabeled data.
  • Decision Boundary: The opposite-ranking ablation indicates that CAL selects informative examples associated with its maximum-divergence criterion.
  • Neighborhood: Constructing a neighborhood for each labeled example is more computationally efficient but slightly underperforms the original approach.
  • Neighborhood: The labeled-example neighborhood variant may underperform because it considers only a small fraction of the pool and omits uncertainty from other unlabeled points.
  • Scoring function: The ablation study also compares alternative scoring functions, including cross entropy with gold labels instead of KL divergence between predictive probabilities.
  • Feature Space: TF-IDF and mean-embedding alternatives largely underperform the original approach using representations that acquire ambiguous data near the model decision boundary.

6 Analysis

The analysis compares acquisition strategies across uncertainty, diversity, and representativeness, finding that CAL selects highly uncertain yet representative examples with low feature-space diversity. These results support CAL’s contrastive strategy of focusing uncertainty sampling near the decision boundary rather than on redundant uncertain points.

  • Metrics: Table 3 averages uncertainty, input-space diversity, feature-space diversity, and representativeness across datasets and acquisition functions.The analysis uses batches acquired after active-learning iterations, with L denoting labeled data, U the unlabeled pool, and Q an acquired batch.
  • Diversity: ALPS acquires the most diverse data in input space, while BERTKM has the highest DIV.-F score and CAL and ENTROPY the lowest.Input-space diversity uses token overlap; feature-space diversity uses BERT [CLS] representations.
  • Uncertainty: ENTROPY and CAL acquire the most uncertain examples, with average entropy almost twice as high as all other methods.Uncertainty is estimated by averaging predictive entropy from a model trained on the full training dataset.
  • Representativeness: CAL obtains the highest representativeness score, followed by ENTROPY, while the remaining strategies acquire less representative data.Representativeness is based on KNN density using Euclidean distances among [CLS] representations with K=10.
  • Discussion: Low DIV.-F does not imply low DIV.-I, indicating that CAL can select feature-space-similar examples with sufficiently diverse inputs.The authors attribute CAL’s advantage over ENTROPY to guiding uncertainty sampling toward informative regions near the decision boundary and away from redundant uncertain data.

7 Related Work

Related active-learning work is organized around uncertainty sampling, diversity or representative sampling, and hybrids combining the two. Prior NLP studies also show that active learning with pretrained language models can improve over random sampling.

  • Uncertainty Sampling: Uncertainty-based methods select low-confidence examples using criteria such as least confidence, predictive entropy, or BALD.BALD selects examples maximizing mutual information between predictions and posterior probabilities, and has been adapted to deep models with Monte Carlo dropout.
  • Diversity Sampling: Diversity sampling selects batches representative of the unlabeled pool, including through core-set construction in deep learning.The approach treats representative labeled examples as a surrogate for the broader available data.
  • Hybrid: Hybrid methods combine uncertainty and diversity through learned policies or uncertainty-based representations followed by clustering.Examples include meta-learning, reinforcement learning, and clustering methods that combine both acquisition signals.
  • NLP Applications: Active learning with pretrained language models such as BERT has shown substantial improvements over random sampling across NLP tasks.This finding is attributed to recent empirical evaluations of active learning in NLP.

8 Conclusion & Future Work

The paper concludes that CAL acquires contrastive examples by combining feature-space neighborhoods with predictive-likelihood ranking, and performs better than other acquisition functions in most evaluated scenarios. It also identifies limits to universal superiority and proposes broader analyses and interpretability-focused future work.

  • Conclusion: CAL selects examples similar in model feature space but maximally different in predicted class probabilities.Neighborhoods are formed in feature space, while predictive likelihoods rank candidates.
  • Conclusion: CAL performs better than other acquisition functions in the majority of evaluated in-domain and out-of-domain scenarios.The conclusion is based on empirical experiments across the paper’s evaluated settings.
  • Limitations: No single acquisition function consistently outperforms all others by a large margin, leaving room for further improvement in active learning.The authors also note that complex acquisition functions can underperform random sampling in specific tasks such as VQA when selecting collective outliers.
  • Future Work: Future work could compare contrastive examples for the task with contrastive examples for the model in terms of interpretability and benefit.CAL currently acquires contrastive examples for the model.

A.1 Data & Hyperparameters

The experiments use dataset-specific validation splits and standard or GLUE-held-out test sets, with BERT-BASE trained under largely fixed optimization settings. IMDB receives a longer maximum sequence length because its inputs are longer.

  • Data Splits: QNLI, IMDB, and SST-2 use random 10% training validation splits, while AG-NEWS and QQP use 5%.DBPEDIA training and validation sets are undersampled to facilitate active-learning simulation.
  • Data Splits: The experiments use standard test sets except for SST-2, QNLI, and QQP, which use GLUE development sets as held-out tests.Additional development data is subsampled from the training data for those GLUE datasets.
  • Training: All datasets use BERT-BASE with batch size 16, learning rate 2e-5, no weight decay, AdamW, and maximum sequence length 128 except IMDB.IMDB uses maximum sequence length 256 because its input texts are longer.

A.2 Efficiency

The efficiency analysis separates inference from selection time and compares acquisition runtimes across datasets using several average-time measures. CAL is typically more efficient than clustering-based methods as pool and acquisition sizes grow, although it remains slower than ENTROPY.

  • Runtime definition: CAL requires m+n forward passes for logits and CLS representations, followed by one iteration over Dpool to compute scores.The m passes cover labeled data and the n passes cover unlabeled data.
  • Runtime definition: Total acquisition time combines inference time for model passes with selection time for ranking and choosing b unlabeled examples.Inference is explicitly required by uncertainty-based methods and CAL; selection covers the remaining scoring and ranking computations.
  • Empirical efficiency: CAL’s inference time is slightly higher than ENTROPY because it processes both Dpool and Dlab, while its selection step is often less costly than inference.The analysis identifies the CAL selection loop as a remaining opportunity for time-complexity improvement.
  • Empirical efficiency: ENTROPY is the most efficient overall, while CAL ranks second in AVG.-ALL and AVG.-6 comparisons.The reported averages compare methods across all datasets or across all datasets except QQP, respectively.
  • Scaling behavior: On the three smallest datasets, ENTROPY is fastest, followed by ALPS and CAL at almost three times ENTROPY’s computation time.BERTKM and BADGE require substantially more time in this comparison, at 13 and 100 times ENTROPY’s time, respectively.

A.3 Reproducibility

The paper supports reproducibility by making its preprocessing, model, and active-learning algorithm code publicly available. Implementation questions can be directed to the first author.

  • Code availability: Code for data preprocessing, model implementations, and active learning algorithms is available at the authors’ GitHub repository.The repository URL is https://github.com/mourga/contrastive-active-learning.
  • Code availability: Implementation questions can be sent to the first author.The paper provides this contact guidance alongside the code-availability statement.
Loading 2109.03764v1…