PaperlayerView exact revision on arXiv

Source-linked AI summary

Variational Adversarial Active Learning

Samarth Sinha, Sayna Ebrahimi, Trevor Darrell

arXiv:1904.00370v3cs.LGcs.CVstat.ML

TL;DR

Active learning seeks label-efficient sampling because annotated computer-vision data can be costly or unavailable. VAAL learns a task-agnostic latent representation and adversarial sampling mechanism, achieving state-of-the-art results across classification and segmentation benchmarks while sampling efficiently.

  • Problem

    Annotated computer-vision data can be prohibitively costly or impossible to obtain at large scale, motivating label-efficient sample acquisition.

  • Method

    VAAL jointly trains a VAE and adversarial discriminator to learn a low-dimensional latent space and select unlabeled samples independently of the downstream task.

  • Results

    VAAL achieves state-of-the-art accuracy and sampling time across CIFAR10, CIFAR100, Caltech-256, ImageNet, Cityscapes, and BDD100K.

  • Takeaways & Limitations

    VAAL provides an effective low-dimensional representation and computationally efficient sampling method for large-scale active learning.

  • Takeaways & Limitations

    The experiments assume an ideal oracle unless stated otherwise.

Abstract

from arXiv · show

Active learning aims to develop label-efficient algorithms by sampling the most representative queries to be labeled by an oracle. We describe a pool-based semi-supervised active learning algorithm that implicitly learns this sampling mechanism in an adversarial manner. Unlike conventional active learning algorithms, our approach is task agnostic, i.e., it does not depend on the performance of the task for which we are trying to acquire labeled data. Our method learns a latent space using a variational autoencoder (VAE) and an adversarial network trained to discriminate between unlabeled and labeled data. The mini-max game between the VAE and the adversarial network is played such that while the VAE tries to trick the adversarial network into predicting that all data points are from the labeled pool, the adversarial network learns how to discriminate between dissimilarities in the latent space. We extensively evaluate our method on various image classification and semantic segmentation benchmark datasets and establish a new state of the art on $\text{CIFAR10/100}$, $\text{Caltech-256}$, $\text{ImageNet}$, $\text{Cityscapes}$, and $\text{BDD100K}$. Our results demonstrate that our adversarial approach learns an effective low dimensional latent space in large-scale settings and provides for a computationally efficient sampling method. Our code is available at https://github.com/sinhasam/vaal.

1. Introduction

VAAL addresses costly or unavailable annotations by learning a task-agnostic sampling strategy from labeled and unlabeled data. It uses a VAE and adversarial discriminator to select informative unlabeled instances through latent-space differences.

  • Annotated computer-vision data can be prohibitively costly to label or impossible to obtain at large scale, motivating label-efficient active learning.
  • VAAL learns a low-dimensional latent space from labeled and unlabeled data using a variational autoencoder.
  • Sample selection is separate from the task being labeled, making VAAL task-agnostic.
  • An adversarial network classifies labeled versus unlabeled latent representations, while the VAE learns to fool it into treating all samples as labeled.
  • The discriminator’s predictions estimate sample representativeness, replacing explicit uncertainty measurement on the main task.

2. Related Work

Prior pool-based active learning methods select informative samples through uncertainty, representation, or combinations of both. The paper positions VAAL as jointly learning representation and uncertainty without depending on the main task.

  • Pool-based active learning methods are commonly categorized as uncertainty-based, representation-based, or hybrid approaches.
  • Classical uncertainty heuristics such as decision-boundary distance, entropy, and expected risk have been widely investigated.
  • Core-set methods select samples by minimizing feature-space distances to unsampled points, but performance deteriorates as class counts grow.
  • Hybrid methods typically use a two-step process combining uncertainty with representativeness, but may require more samples or additional modeling.
  • VAAL learns representation and uncertainty together in a task-independent adversarial framework, and the paper reports outperforming both ensemble and hybrid competitors.

3. Adversarial Learning of Variational Autoencoders for Active Learning

The active learner queries a fixed budget of informative unlabeled samples, while VAAL learns transductive representations and adversarially distinguishes labeled from unlabeled data. The task module is trained separately, and experiments consider oracle noise.

  • Active learning iteratively queries a fixed sampling budget of informative unlabeled samples for oracle annotation while minimizing expected loss.
  • Transductive representation learning: The β-VAE learns a low-dimensional representation with a Gaussian prior and reconstructs inputs, incorporating unlabeled data to capture features missing from labeled-only representations.
  • Adversarial representation learning: VAAL maps labeled and unlabeled samples into a shared latent space and trains a discriminator to distinguish their encoded features.
  • Adversarial representation learning: The VAE and discriminator are learned together adversarially, with the discriminator estimating whether latent samples come from the unlabeled pool.
  • The task module is trained separately from the active learner, so task learning and sample selection do not depend on each other.
  • The experiments distinguish ideal oracles, which provide correct labels, from noisy oracles, which provide erroneous labels for some samples.

4. Experiments

Experiments evaluate VAAL across image classification and semantic segmentation benchmarks under progressively larger labeled-data budgets. VAAL achieves strong performance against established active-learning baselines, including large-scale ImageNet and driving-scene segmentation settings.

  • Experimental setup: Experiments begin with 10% labeled data, add 5% of the training set per batch, and repeatedly retrain after oracle annotation.The unlabeled pool contains the remaining training data.
  • Datasets: VAAL is evaluated on CIFAR10, CIFAR100, Caltech-256, ImageNet, Cityscapes, and BDD100K, spanning classification and semantic segmentation.ImageNet contains more than 1.2M images across 1000 classes; Cityscapes and BDD100K each use 19 segmentation classes.
  • Evaluation: Classification performance is measured by accuracy and segmentation performance by mean IoU at labeled-data ratios from 10% to 40%.Results are averaged over five runs except ImageNet, which uses two repetitions and ratios through 30%.
  • Classification benchmarks: 80.9% mean accuracy on CIFAR10 and 47.95% on CIFAR100 are achieved with 40% of the data, while VAAL outperforms most baselines and remains competitive with Core-set and Ensembles w. VarR.On CIFAR100, VAAL uses about 2.5% fewer labels than Core-set or Ensembles w. VarR for equal accuracy above 20% labeled data.
  • Classification benchmarks: VAAL improves its margin over Core-set as classes increase from 10 to 100 to 256, and reaches 48.61% ImageNet accuracy with 256K images.Core-set and Ensembles w. VarR require almost 32K additional labeled images for the same ImageNet accuracy.
  • Segmentation benchmarks: VAAL achieves the highest mean IoU on Cityscapes and BDD100K, reaching 57.2% and 42.3% respectively with 40% labeled data.On Cityscapes, VAAL reaches 54.1% mean IoU with 743 annotations, versus nearly 800–1041 for the other listed methods.

5. Analyzing VAAL in Detail

VAAL’s components are examined through ablations, robustness tests, and sampling-time comparisons. The results indicate that its adversarial VAE–discriminator design improves selection under challenging conditions while remaining computationally efficient.

  • 5.1. Ablation study: VAAL’s ablation variants remove the VAE, freeze the VAE, or remove the discriminator to assess each module’s contribution.The study uses BDD100K segmentation because it is larger and more challenging than Cityscapes.
  • 5.1. Ablation study: The VAE-only heuristic improves over random sampling, while VAAL outperforms the ablations by implicitly learning uncertainty through the adversarial game.The heuristic uses 2-Wasserstein distance from the labeled-data cluster centroid in latent space.
  • 5.2. VAAL’s Robustness: VAAL outperforms Core-Set and random sampling when the initial CIFAR100 labeled pool omits 10 or 20 randomly chosen classes.With 20 missing classes, VAAL performs nearly identically to Core-Set and significantly better than random sampling.
  • 5.2. VAAL’s Robustness: VAAL outperforms Core-Set, Ensemble w. VarR, and random sampling at both CIFAR100 budget sizes of b = 5% and b = 10%.The experiments note slightly better performance at b = 5%, attributing the difference to redundancy in larger sampled batches.
  • 5.2. VAAL’s Robustness: The relative performance of Core-Set and VAAL is comparable to the ideal oracle because neither depends on the task learner.The architecture comparison also reports that using ResNet18 instead of VGG16 does not affect the performance gap between VAAL and Core-Set.
  • 5.3. Sampling time analysis: VAAL samples by discriminator inference and ranking, whereas Core-Set solves an optimization problem and Ensemble w. VarR uses five models.Random sampling takes only a few milliseconds, while VAAL is faster than the ensemble method.

6. Conclusion

The paper concludes that VAAL is a task-agnostic, batch-mode active learner built around an adversarial VAE–discriminator game. It reports state-of-the-art accuracy and sampling-time results across classification and segmentation benchmarks, along with robustness to noisy labels, biased initial pools, and varying budgets.

  • 6. Conclusion: VAAL learns a latent representation from labeled and unlabeled data through an adversarial game between a VAE and discriminator.It implicitly learns uncertainty for samples predicted to belong to the unlabeled pool.
  • 6. Conclusion: VAAL achieves state-of-the-art accuracy and sampling-time results on CIFAR10, CIFAR100, Caltech-256, ImageNet, Cityscapes, and BDD100K.These benchmarks cover image classification and semantic segmentation at small and large scales.
  • 6. Conclusion: VAAL is reported to be robust to noisy labels, biased initial labeled data, and different oracle budgets.The conclusion presents these properties alongside its benchmark results.

A. Datasets

The experiments use four image-classification datasets and two large-scale semantic-segmentation datasets. Dataset budgets specify the number of images sampled at each training iteration.

  • A. Datasets: CIFAR10, CIFAR100, Caltech-256, and ImageNet are used for image classification, while BDD100K and Cityscapes are used for semantic segmentation.Table 2 summarizes dataset sizes, class counts, and budget sizes.
  • A. Datasets: The budget for each dataset is the number of images sampled at each training iteration.The datasets differ in size and number of classes, as summarized in Table 2.

B. Hyperparameter Selection

VAAL hyperparameters were selected through grid search, and CIFAR100 results indicate performance remains consistently better than Core-set across ResNet18 and VGG16 architectures.

  • Grid search was used to identify the hyperparameters for the VAAL models.The selected settings are tabulated in Table 3.
  • Table 3 defines d as the VAE latent-space dimension and α1, α2, and α3 as learning rates for VAE, discriminator, and task module.It also identifies λ1 and λ2 as regularization parameters and β as the Lagrangian parameter.
  • CIFAR100 performance was consistently better than Core-set across ResNet18 and VGG16.This supports robustness to the choice of architecture.
Loading 1904.00370v3…