Source-linked AI summary
Bayesian Active Learning for Classification and Preference Learning
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, Máté Lengyel
TL;DR
Active learning must select useful labels efficiently, but information-theoretic objectives become difficult for complex nonparametric classification models. The paper rewrites information gain using predictive entropies and applies the resulting BALD method to Gaussian-process classification and preference learning. Experiments show favorable performance against popular active-learning methods, while the approach compares well with decision-theoretic methods that use more information and computation.
Problem
Information-theoretic active learning is difficult to compute for complex nonparametric classification models because posterior entropies and parameter spaces can be intractable.
Method
The paper expresses information gain as predictive entropies and applies this minimally approximated criterion to Gaussian Process Classification and preference learning.
Results
BALD has the best overall performance in the reported classification experiments, while decision-theoretic methods require more computation and access to test inputs.
Takeaways & Limitations
The reformulation provides an information-theoretic active learning method applicable to Gaussian-process classification and preference learning without depending on a particular posterior-approximation type.
Takeaways & Limitations
Without hyperparameter learning, GP overfitting with little data can lead BALD to select abnormal query locations, while maintaining hyperparameter uncertainty substantially increases computation.
Abstract
from arXiv · showhide
Information theoretic active learning has been widely studied for probabilistic models. For simple regression an optimal myopic policy is easily tractable. However, for other tasks and with more complex models, such as classification with nonparametric models, the optimal solution is harder to compute. Current approaches make approximations to achieve tractability. We propose an approach that expresses information gain in terms of predictive entropies, and apply this method to the Gaussian Process Classifier (GPC). Our approach makes minimal approximations to the full information theoretic objective. Our experimental performance compares favourably to many popular active learning algorithms, and has equal or lower computational complexity. We compare well to decision theoretic approaches also, which are privy to more information and require much more computational time. Secondly, by developing further a reformulation of binary preference learning to a classification problem, we extend our algorithm to Gaussian Process preference learning.
1 Introduction
Active learning selects informative measurements instead of passively collecting data, aiming to build the best model with fewer labeled examples. Information-theoretic methods avoid dependence on a particular decision loss or test distribution but are difficult to apply to complex nonparametric models.
- Active learning chooses the most useful measurements to produce the best model with the least possible data.This is motivated by the abundance of unlabeled data and the cost of obtaining labels.
- Decision-theoretic active learning minimizes expected losses or Bayes posterior risk after data collection.Evaluating this objective can be difficult when losses or test distributions are unknown.
- Information-theoretic approaches reduce uncertainty about feasible models without committing to a particular decision task or test dataset.They use heuristics or quantities such as Shannon entropy and KL-divergence.
- Applying information-theoretic criteria to complicated nonparametric models is difficult because their parameter spaces can be infinite-dimensional and posterior entropies may be intractable.This difficulty has led to approximate entropy calculations, sampling, and related non-probabilistic methods.
- The paper presents a minimally approximated information-theoretic active learning algorithm for Gaussian Process Classification and extends it to preference learning.The approach is evaluated on a wide variety of datasets and contrasted with other active learning methods.
2 Bayesian Information Theoretic Active Learning
The paper formulates active learning as reducing posterior uncertainty about latent parameters, then rewrites the objective as predictive-entropy calculations in output space. This yields BALD, which selects inputs where posterior parameter settings disagree most about the output while enabling application to Gaussian-process classification and preference learning.
- Information-theoretic active learning selects queries that minimize uncertainty about latent parameters governing the input-output relationship.The learner chooses inputs x_i and observes their responses y_i rather than passively receiving labeled pairs.
- The exact sequential objective is generally NP-hard, so the paper uses a myopic greedy policy that maximizes expected posterior-entropy reduction.The objective must average over the unseen output y.
- Direct parameter-space entropy calculations are difficult because posteriors may be high-dimensional or infinite-dimensional and require O(NxNy) posterior updates.Approximations or sampling can introduce additional computational or estimation difficulties.
- The objective equals conditional mutual information between the unknown output and parameters, allowing entropy calculations to move from parameter space into usually low-dimensional output space.For binary classification, the required quantities are Bernoulli entropies, while only O(1) posterior updates are needed.
- BALD selects inputs with high predictive entropy but low parameter-conditional entropy, representing disagreement among posterior parameter settings about the outcome.The method can be applied directly to Gaussian Process Classification and preference learning with minimal additional approximations.
3 Gaussian Processes for Classification and Preference Learning
The paper develops BALD for Gaussian process classification by expressing information gain through predictive entropies and approximating otherwise intractable quantities. It then reformulates pairwise preference learning as classification, allowing the same active-learning algorithm to be applied.
- Gaussian Process Classification: GPC is challenging for information-theoretic active learning because its parameter space is infinite-dimensional and posterior inference is analytically intractable.Approximate inference methods provide Gaussian posterior approximations used by the derivation.
- Gaussian Process Classification: BALD computes query informativeness from predictive entropies, using an analytic probit entropy term and an approximation for the expected conditional entropy.The intractable integral is approximated through a Taylor expansion and Gaussian convolution to obtain a closed-form objective.
- Gaussian Process Classification: 0.27% error is obtained in the integral when the posterior variance tends to zero, while the approximation is reported to have striking accuracy in Figure 1.The approximation is compared with extensive Monte Carlo simulation as a gold standard.
- Gaussian Process Classification: BALD first obtains approximate posterior predictive mean and variance, then selects the point maximizing the resulting objective.For practically relevant kernels, the objective is smooth and differentiable, enabling gradient-based query optimization.
- Preference Learning: Pairwise preference learning is represented as classification on input pairs, with labels indicating which item is preferred.The preference model uses a latent function whose pairwise difference induces a probit-classification likelihood.
- Preference Learning: The induced preference kernel is antisymmetric, ensuring P[u ≻v] = 1 −P[v ≻u], so the GPC active-learning algorithm applies directly to pairwise preferences.The kernel is constructed from covariance terms involving both items in each pair.
4 Related Methodologies
Related methods differ in what information they use, how they approximate posterior uncertainty, and the computational burden of selecting queries.
- Information-theoretic methods: The Informative Vector Machine and BALD share an information-theoretic motivation but differ fundamentally under approximate inference.IVM updates approximate posteriors after candidate outcomes, whereas BALD computes an entropy difference without recomputing each candidate posterior.
- Information-theoretic methods: BALD requires O(1) posterior updates, allowing more accurate iterative inference such as EP, whereas IVM requires O(NxNy) updates and uses ADF.The IVM therefore makes a further approximation relative to BALD.
- Information-theoretic methods: Maximum Entropy Sampling is unsuitable for classification because it cannot distinguish model uncertainty from observation uncertainty.Its ignored second term is constant only for regression with input-independent observation noise.
- Information-theoretic methods: Other mutual-information objectives target measured variables or predefined interest points and are not tractable for classification or preference learning with input-dependent observation noise.These approaches are related to BALD but are not applied to classification in the cited formulations.
- Approximation methods: Figure 2 compares approximation errors across inference methods and methods for evaluating Eqn. (4), reporting that the 2≈ approximation is very accurate while Laplace performs worse than EP.The experiments use EP.
- Decision-theoretic methods: Decision-theoretic methods minimize expected loss but require test-point locations and O(NxNy) posterior updates, making them transductive and computationally expensive.Designing an inductive decision-theoretic algorithm remains an open, hard problem because it requires integration over possible test-data distributions.
- Non-probabilistic methods: For SVMs, version-space volume approximates posterior entropy, while Query by Committee selects inputs with the most balanced committee vote.Its deterministic vote criterion discards predictive confidence and can exhibit pathologies associated with maximum-disagreement methods.
5 Experiments
Experiments evaluate BALD against eight alternative active-learning algorithms on artificial, classification, and preference-learning datasets. BALD performs consistently well overall, matching or outperforming competitors while decision-theoretic methods require more computation and information.
- Experimental setup: The experiments compare BALD with random sampling, MES, QBC, active SVM, IVM, decision-theoretic methods, and empirical-error minimization.The evaluation uses pool-based active learning for GPC and preference learning, including artificial and UCI datasets.
- Results: BALD makes significant gains over random sampling and is consistently among the best-performing algorithms across datasets.This pattern holds in both classification and preference learning domains.
- Results: BALD has the best overall classification performance; every other method requires more data on average to reach the same accuracy.Figure 5 measures additional data points needed to achieve at least 97.5% of full-pool predictive performance, relative to BALD.
- Results: The closest decision-theoretic competitor has a median increase of 1.4 data points, but requires much more computation and access to test-point locations.The decision-theoretic methods were also not completed for all datasets because they took a long time to run.
- Algorithm behavior: MES performs poorly on noisy data because it discards observation-noise information, while IVM can reduce entropy artificially by favoring one class.MES can match BALD when observation noise is zero; IVM also performs significantly worse than BALD on some noise-free datasets.
- Algorithm behavior: QBC usually suffers only a small performance decrement but performs poorly on noisy data, and SVM performance varies substantially with the approximation used.The reported SVM configuration was selected because it produced the most consistently good performance among the tested approximations.
6 Conclusions
The paper applies the full information-theoretic active-learning criterion to GP classification with minimal approximations and extends it to preference learning. Its performance compares favourably with other active-learning methods and decision-theoretic methods requiring more computation.
- 6 Conclusions: The method applies the full information-theoretic active-learning criterion to GP classification with, as far as the authors are aware, minimal approximations.It is reported to have good computational complexity.
- 6 Conclusions: The authors extend the GPC model with a preference-learning kernel, enabling direct application of the active-learning algorithm to preference learning.
- 6 Conclusions: The method naturally supports active learning of kernel hyperparameters, described as a hard, mostly unsolved problem in SVM active learning.
- 6 Conclusions: The approach is agnostic to approximate inference methods, allowing trade-offs between computational complexity and accuracy across several inference methods.The listed methods include EP, the Laplace approximation, ADF, and sparse online learning.
- 6 Conclusions: Experimental performance compares favourably with many classification active-learning methods and with decision-theoretic methods that use test data and require much more computation.
Taylor Expansion for Approximation
The paper approximates the relevant entropy expression using a Taylor expansion. Because the function is even, the x^3 term vanishes, and the approximation is truncated at O(x^4).
- Taylor Expansion for Approximation: The method performs a Taylor expansion of ln H[Φ(x)] to construct an approximation.
- Taylor Expansion for Approximation: The expansion includes a second-order term represented with a 2! denominator, followed by additional terms.
- Taylor Expansion for Approximation: Because the function is even, the x^3 term is zero, so exponentiation yields an approximation up to O(x^4).
Preference Kernel
The preference-learning extension derives the GP's preference mean and covariance from the mean and covariance of the underlying GP over f.
- Preference Kernel: The preference GP mean µ_pref and covariance function k_pref are computed from the mean and covariance of f ∼ GP(µ, k).