Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning

• BNN+LV represents complex stochastic patterns through latent-variable distributions while modeling parameter uncertainty through weight distributions.Source: §1 · p. 1
• The paper derives uncertainty decompositions that separate aleatoric and epistemic components from the predictive distribution of BNN+LV.Source: §1 · p. 1
• The decomposition identifies informative regions for efficient active learning even with bimodal and heteroscedastic noise.Source: §1 · p. 1
• A risk-sensitive model-based RL criterion lets domain experts trade off model-bias risk against risk induced by stochasticity.Source: §1 · p. 1
• The work extends uncertainty-based risk management from discrete MDPs to continuous nonlinear functions with complex noise.Source: §1 · p. 1

Model definition

• BNN+LV models targets with a neural network of weights W, latent variables z, and additive observation noise ϵ.The latent variables capture unobserved stochastic features affecting network outputs, while the observation noise has diagonal covariance.Source: §2 · pp. 1–2
• Latent variables extend additive Gaussian noise, which alone can describe only limited stochastic patterns.Source: §2 · pp. 1–2

Posterior approximation

• The approximate posterior q over weights and latent variables is fitted by minimizing a divergence from the exact posterior p(W, z | D).The experiments use black-box α-divergence minimization with α = 1.0.Source: §2 · p. 2

Model capabilities

• Jointly learning q(z) and q(W) enables flexible conditional distributions with complex stochasticity, such as bimodal or heteroscedastic noise.q(z) describes latent-variable values associated with the training data, while q(W) represents parameter uncertainty.Source: §2 · p. 2

Predictive distribution

• For an unseen test input, the latent variable z⋆ is integrated using its prior p(z⋆), because no additional evidence about z⋆ is available.Source: §3 · p. 2

Predictive uncertainty

• BNN+LV prediction uncertainty arises jointly from sampled weights W, test-time latent variables z⋆, and additive noise ϵ.Source: §3 · p. 2

Uncertainty types

• Aleatoric uncertainty comes from z⋆ and ϵ and cannot be reduced by collecting more data, whereas epistemic uncertainty comes from W and can typically decrease with more data.Source: §3 · p. 2

Decomposition metrics

• The paper presents entropy-based and variance-based decompositions for separating aleatoric and epistemic uncertainty in predictions.Source: §3 · p. 2

Entropy decomposition

• In the entropy decomposition, E_q(W)[H(y⋆|W, x⋆)] measures aleatoric uncertainty, while total entropy minus this quantity gives epistemic uncertainty.The difference is identified as the mutual information between y⋆ and W.Source: §3 · p. 2

Variance decomposition

• Variance provides an alternative uncertainty measure, with total predictive variance decomposed using the law of total variance.Source: §3 · p. 2

4. Active Learning with Complex Noise

• The decomposition selects inputs with high epistemic uncertainty, which corresponds to expected posterior-entropy reduction and identifies the most informative points to label.The approach samples model weights and latent variables and estimates the relevant entropies with nearest-neighbor methods.Source: §4 · p. 3

4.1. Experiments

• In heteroscedastic data, epistemic uncertainty is inversely proportional to input density, whereas total predictive uncertainty fails to identify informative regions.This makes scarce regions between dense data clusters the preferred active-learning targets.Source: §4.1 · p. 3, Figure 1 · p. 4
• In bimodal data, BNN+LV learns the two-mode predictive structure, total uncertainty rises where data are scarce, and epistemic uncertainty increases with x.Aleatoric uncertainty is nearly symmetric around x = 0.75 and is lower there because the generating process becomes unimodal near that point.Source: §4.1 · pp. 3–4, Figure 2 · p. 4
• In wet-chicken dynamics, epistemic uncertainty highlights regions where boundary-dependent transitions are harder to learn, while total predictive uncertainty does not.Learning is more difficult near y = 5 than y = 0 because the transition behavior differs at those boundaries.Source: §4.1 · p. 4, Figure 3 · p. 5
• The experiments train BNN+LV models, select batches from uniformly sampled input pools for 150 iterations, and repeat each experiment five times against decomposed, total-uncertainty, and GP baselines.The GP baseline assumes Gaussian noise and cannot capture complex stochastic patterns.Source: §4.1 · pp. 3, 5
• Overall, BNN+LV outperforms Gaussian processes, while the decomposed criterion significantly improves over total predictive uncertainty on heteroscedastic and wet-chicken tasks but performs similarly on bimodal data.The bimodal similarity occurs because epistemic and total uncertainty behave similarly there.Source: Table 1 · p. 4, §4.1 · p. 5

5. Risk-sensitive Reinforcement Learning

• The proposed criterion extends risk-sensitive model-based RL by balancing expected cost with separate epistemic and aleatoric risks.Parameters β and γ control the trade-off between these two uncertainty components.Source: §5 · pp. 5–6
• The criterion decomposes risk over individual costs, using the sum of per-step risks because low risk for each c_t implies low risk for the total cost, but not conversely.The resulting terms arise from variance over latent-variable and additive-noise sampling versus variance over model weights.Source: §5 · p. 6

5.1. Model-bias and Noise aversion

• Epistemic risk estimates model-bias and guides policies toward regions where the learned dynamics are expected to differ less from the ground truth.Aleatoric risk instead guides policies toward regions with lower stochasticity in the cost.Source: §5.1 · p. 7

5.2. Experiments

• On the industrial benchmark, increasing β reduced model-bias but increased average cost, while adding aleatoric risk with γ > 0 decreased performance.The nearest-neighbor baseline showed a similar pattern with a worse model-bias/cost trade-off.Source: §5.2 · p. 7
• Roll-outs showed that ignoring risk produced increasing model-ground-truth mismatch and high test-time cost, whereas epistemic-risk penalties lowered ground-truth cost and noise aversion reduced stochasticity.The β = 4, γ = 0 policy had similar model and ground-truth costs; β = 0, γ = 4 produced slightly higher model-bias but lower stochasticity.Source: §5.2 · p. 7
• Policies with γ = 7.5 achieved the best expected-cost/model-bias trade-offs in the wind-turbine simulator by avoiding noisy state-space regions.The paper links these regions to strong wind-turbulence effects on average cost.Source: §5.2 · p. 7

• Figure 4 compares average cost against model-bias across industrial and wind-turbine benchmarks while varying epistemic and aleatoric risk weights.Source: Figure 4 · p. 8
• Figure 5 compares ground-truth and BNN+LV roll-outs for policies with no risk penalty, epistemic-risk penalties, or aleatoric-risk penalties.Source: Figure 5 · p. 8
• Related reinforcement-learning studies commonly use model uncertainty for online exploration, whereas this work focuses on batch learning with stochastic dynamics.Source: §6 · p. 8
• The work extends uncertainty-aware reinforcement learning from discrete or specialized settings to continuous state and action spaces using scalable probabilistic models.Source: §6 · p. 8
• The active-learning approach differs from deterministic information-theoretic methods because BNN+LV can model complex noise such as bimodality and heteroscedasticity.Source: §6 · p. 8

7. Conclusion

• The decomposition separates predictive uncertainty into model-bias and noise-aversion components for efficient and risk-sensitive learning.Source: §7 · p. 8

A. Hamiltonian Monte Carlo Solution to Toy Problems

• HMC is used to test whether the active-learning decomposition persists beyond black-box α-divergence minimization, using posterior samples after burn-in and thinning.Source: A. Hamiltonian Monte Carlo Solution to… · p. 11

A.1. Heteroscedastic Problem

• The heteroscedastic experiment samples 750 inputs from a three-Gaussian mixture, leaving sparse data between dense border and central regions.Source: §A.1 · p. 11

A.2. Bimodal Problem

• The HMC figures for heteroscedastic and bimodal data show raw inputs, predictive distributions, entropy estimates, conditional entropy, and estimated entropy reduction by x.Source: Figure 6 · p. 11, Figure 7 · p. 12

B. Solutions to Toy Problems for different values of α

• Lower α values place increasing emphasis on the latent variable z, while epistemic uncertainty vanishes as α-divergence minimization approaches variational Bayes.Source: B. Solutions to Toy Problems for… · p. 12
• The bb-α = 0.5 figures visualize predictive distributions, entropy, conditional entropy, and entropy reduction for heteroscedastic and bimodal toy data.Source: Figure 8 · p. 12, Figure 9 · p. 13

B.2. V B solutions to Toy Problems

• Variational Bayes is approximated with α = 10^-6 and evaluated using the same active-learning visualization components.Source: §B.2 · p. 13, Figure 10 · p. 13

C.1. Active Learning

• The active-learning implementation trains a two-hidden-layer BNN with 20 units per layer for 5000 epochs using Adam with learning rate 0.001.Entropy estimation uses k = 25 nearest neighbors, 500 samples of q(W), and 500 samples of p(z).Source: §C.1 · pp. 13–14
• Figure 11 compares predictive entropy, conditional entropy, and estimated entropy reduction across inputs in a BNN active-learning example with bimodal data.The figure reports raw data, input density, predictive distributions, entropy estimates, and entropy-reduction estimates.Source: Figure 11 · p. 14
• Active learning uses held-out evaluation, repeated pool sampling, five selected points per iteration, and retraining from scratch after each selection.The protocol runs for n = 150 iterations, with pool sets of size 50 and test sets of 500 or 2500 depending on the problem.Source: §C.1 · p. 14

C.2. Wet-chicken

• The wet-chicken benchmark models a two-dimensional canoeist navigating a river toward a waterfall while stochastic drift and turbulence depend on horizontal position.The river has width w = 5 and length l = 5, and crossing the waterfall sends the canoeist back to the origin.Source: §C.2 · p. 14
• The wet-chicken dynamics combine position-dependent drift and turbulence with boundary resets, producing bimodal, heteroscedastic, and truncated transition distributions.The drift is v_t = 3x_t w^-1, turbulence magnitude is s_t = 3.5 − v_t, and turbulence is sampled through τ_t ∼ Unif([−1, 1]).Source: §C.2 · pp. 14–15

C.3.1. INDUSTRIAL BENCHMARK

• Industrial policies control changes in velocity, gain, and shift as functions of the turbine state, with stochastic behavior-policy changes sampled from uniform variables.Velocity and gain are bounded in [0, 100], while the collection policy pushes them toward the medium range [40, 60].Source: §C.3.1 · p. 15
• The industrial benchmark trains a latent-variable BNN on 30,000 state transitions and then trains policies with a Monte Carlo approximation over a 100-step horizon.The behavior policy keeps velocity and gain mainly within [40, 60], leaving large parts of the state space unobserved.Source: §C.3.1 · p. 15

C.3.2. WIND TURBINE SIMULATOR

• The wind-turbine simulator seeks to maximize energy output over a T-step horizon using bounded actions that adjust turbine behavior from observed state features.The state includes features such as wind speed and produced power.Source: §C.3.2 · p. 15
• The wind-turbine data contain around 5,000 transitions generated by a behavior policy with limited exploration around the neutral action a(t) = 0.The paper expects action effects on reward to be difficult to learn from this limited dataset and high-dimensional state observation.Source: §C.3.2 · p. 15
• For the wind-turbine simulator, a two-hidden-layer BNN with 50 units per layer is trained by α-divergence minimization before Monte Carlo policy training.The reported training uses α = 1.0 and a policy network with 20 units per layer.Source: §C.3.2 · p. 15