Source-linked AI summary
Weight Uncertainty in Neural Networks
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra
TL;DR
Neural networks can overfit and make overconfident decisions when they cannot represent uncertainty in their training data. Bayes by Backprop learns a probability distribution over network weights by variational Bayesian inference and backpropagation-compatible Monte Carlo gradients. It performs comparably to dropout on MNIST, supports more reasonable nonlinear-regression predictions, and drives exploration-exploitation in contextual bandits.
Problem
Feedforward neural networks are prone to overfitting and often cannot correctly assess training-data uncertainty, leading to overly confident predictions or actions.
Method
Bayes by Backprop learns a distribution over neural-network weights by optimising a variational Bayesian compression cost with unbiased Monte Carlo gradient estimates.
Results
Bayes by Backprop performs comparably to dropout on MNIST, improves uncertainty-aware predictions in nonlinear regression, and learns an exploration-exploitation trade-off in contextual bandits.
Takeaways & Limitations
Learned weight uncertainty provides regularisation, richer predictions through model averaging, and exploration driven by uncertainty in bandit decisions.
Takeaways & Limitations
Optimising prior parameters empirically was not useful and yielded worse results, because the prior could quickly fit the empirical weight distribution.
Abstract
from arXiv · showhide
We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.
1. Introduction
The paper introduces Bayes by Backprop to address overfitting and overconfident predictions by learning uncertainty over neural-network weights. This uncertainty supports regularisation, richer predictions through model averaging, and exploration in contextual bandits.
- 1. Introduction: Bayes by Backprop uses variational Bayesian learning to introduce uncertainty into neural-network weights.The algorithm is presented as an efficient, principled regularisation method with backpropagation-like learning.
- 1. Introduction: Weight uncertainty enables robust predictions in regions with little or no regression data and supports systematic exploration in contextual bandits.The paper connects uncertainty to improved generalisation and exploration beyond ε-greedy behaviour.
- 1. Introduction: Each weight is represented by a learned probability distribution, producing an effectively infinite ensemble with typically about twice as many parameters.Networks are sampled from a shared learned distribution, with unbiased Monte Carlo gradient estimates.
- 1. Introduction: The paper evaluates Bayes by Backprop on classification, regression, and contextual bandit problems.The stated empirical program covers MNIST classification, nonlinear regression, and a bandit problem.
2. Point Estimates of Neural Networks
Point-estimate neural networks map inputs to output distributions using fixed weights learned by likelihood-based optimisation. Regularisation adds a prior over weights, yielding MAP estimation and familiar L2 or L1 penalties.
- 2. Point Estimates of Neural Networks: A neural network represents P(y|x, w), mapping input x and weights w to a probability distribution over outputs y.Classification uses a categorical distribution, while regression uses a Gaussian distribution.
- 2. Point Estimates of Neural Networks: Inputs pass through successive linear transformations and element-wise nonlinearities parameterised by the weights.These layers map x to the parameters of the output distribution.
- 2. Point Estimates of Neural Networks: Maximum likelihood estimates weights from training examples by optimising log P(D|w), typically with gradient descent or backpropagation.The objective assumes the data likelihood is differentiable in w.
- 2. Point Estimates of Neural Networks: MAP estimation adds log P(w) to the likelihood objective, recovering L2 regularisation with Gaussian priors and L1 regularisation with Laplace priors.The prior imposes a simplicity preference on the weights.
3. Being Bayesian by Backpropagation
Bayes by Backprop approximates Bayesian inference over neural-network weights with Monte Carlo, backpropagation-compatible gradients while balancing data fit against prior complexity. The method supports flexible weight distributions, minibatch optimization, and learned uncertainty for robust predictions.
- Variational Bayesian learning: Bayesian inference represents neural-network weights with a distribution q(w|θ), so predictions average over possible weight configurations rather than using one fixed network.This is equivalent to using an ensemble whose weights are sampled from the learned distribution.
- Variational Bayesian learning: The variational objective combines a likelihood cost with a complexity cost, trading fit to training data against the simplicity prior P(w).The same objective is also interpreted as a minimum-description-length cost.
- Unbiased Monte Carlo gradients: Monte Carlo sampling and a reparameterized noise transform produce unbiased, backpropagation-like gradient estimates for learning the weight distribution.The transform maps parameter-free noise and variational parameters into sampled weights, while sharing the usual backpropagation gradients.
- Unbiased Monte Carlo gradients: Bayes by Backprop avoids requiring a closed-form complexity cost, allowing more combinations of prior and variational-posterior families and supporting weight-level uncertainty.The method operates over many weights rather than only stochastic hidden units and can use a scale-mixture prior with a diagonal Gaussian posterior.
- Scale mixture prior: The scale-mixture prior combines a broad component with a narrow component, providing heavier tails while concentrating many weights near zero.The components share prior parameters across all weights.
- Minibatches and KL re-weighting: Minibatch training reweights the KL complexity cost across batches, emphasizing the prior early and the data more strongly in later minibatches.The reported weighting scheme is π_i = 2M−i / 2M−1.
4. Contextual Bandits
In contextual bandits, Bayes by Backprop uses uncertainty over neural-network weights to support Thompson-sampling actions and balance exploration with exploitation. As learning reduces posterior uncertainty, action selection becomes more focused on discovered high-reward actions.
- Contextual bandits require choosing among actions with unknown rewards while observing no counterfactual reward for unchosen actions.This can cause suboptimal behavior when the model is trained online without exploratory actions.
- Thompson sampling trades off exploitation and exploration by sampling model parameters and selecting the action best under those parameters.The procedure repeats parameter sampling, action selection, and model updating.
- Bayes by Backprop adapts Thompson sampling to neural networks by sampling weights from the variational posterior before selecting an action.The sampled-weight network evaluates expected rewards for the available actions, after which the variational parameters are updated from the received reward.
- Initially, posterior uncertainty keeps action selection approximately uniform; as uncertainty decreases, choices become more deterministic and concentrate on high expected reward actions found so far.Using multiple Monte Carlo samples can reduce gradient-estimate variance but trades off against reduced exploration.
- Variational methods can underestimate uncertainty, potentially causing under-exploration and premature convergence, although the authors did not observe this in practice.
5. Experiments
The experiments evaluate Bayes by Backprop on MNIST classification, weight pruning, nonlinear regression, and contextual bandits. Across these tasks, learnt weight uncertainty supports competitive classification, uncertainty-aware extrapolation, and exploration.
- 5.1. Classification on MNIST: Bayes by Backprop achieved performance comparable to dropout on MNIST, eventually obtaining lower test error after 600 epochs.The comparison used networks with two layers of 1200 rectified linear units; each Bayes by Backprop iteration was around twice as slow as dropout.
- 5.1. Classification on MNIST: Bayes by Backprop weights used the greatest range among the compared regularised networks, with fewer weights centred at zero than SGD weights.Figure 3 compares weight histograms for dropout, plain SGD, and samples from Bayes by Backprop.
- 5.1. Classification on MNIST: Removing 95% of weights by lowest signal-to-noise ratio preserved good performance, while removing 98% caused a significant performance drop.The pruning experiment ordered weights by |µ_i|/σ_i and replaced low-ratio variational posteriors with zero.
- 5.1. Classification on MNIST: The two signal-to-noise modes aligned with a test-performance change from 1.24% to 1.29%, suggesting the pruning heuristic relates to test performance.The 75% cut-off separated the two modes in the density and cumulative distribution plots.
- 5.1. Classification on MNIST: 1.12% was the lowest observed MNIST test error, using a network taken from the end of training rather than one selected by validation cost.The reported protocol differs from the results in Table 1 because the network was not selected using the lowest validation cost.
- 5.2. Regression curves: In nonlinear regression, Bayes by Backprop predictive intervals diverged where data were absent, unlike the standard network’s zero-variance extrapolation.The training data were generated from a noisy sinusoidal curve, and the figure displays median predictions with interquartile ranges.
- 5.3. Bandits on Mushroom Task: On the mushroom bandit task, Bayes by Backprop explored from the beginning and quickly converged to near-perfect eating and non-eating rates, producing almost flat regret.Cumulative regret was averaged over five runs; the comparison included ε-greedy agents with ε values of 0%, 1%, and 5%.
6. Discussion
Bayes by Backprop learns neural-network weight uncertainty and performs well across classification, regression, and contextual bandits. Its gradient-based implementation also supports scalable optimisation and GPU use.
- Bayes by Backprop achieves MNIST classification performance comparable to dropout.
- In nonlinear regression, learnt weight uncertainty enables more reasonable predictions about unseen data.
- In contextual bandits, Bayes by Backprop automatically learns the exploration–exploitation trade-off.
- Because it uses gradient updates, Bayes by Backprop can scale with asynchronous SGD and GPU implementations.