PaperlayerView exact revision on arXiv

Source-linked AI summary

Modeling Latent Disturbances for Robust Decision-Making in World Models

Junwon Seo, Andrea Bajcsy

arXiv:2610.07599v1cs.ROcs.AIcs.LG

TL;DR

World models make robust optimization difficult because their learned latent spaces lack a clear way to represent physically plausible disturbances. LUCID constructs and calibrates plausible latent dynamics, then optimizes robust actions against worst-case disturbances; hardware experiments report reduced failures in safety filtering and improved visuomotor policy success.

  • Problem

    It is unclear how to define latent-space disturbances that faithfully represent physical uncertainty, since arbitrary latent perturbations may produce implausible system evolutions.

    Source: §I · p. 2

  • Method

    LUCID combines dynamics-aware similarity and out-of-distribution detection to construct a conformally calibrated set of plausible latent dynamics for game-theoretic robust optimization.

    Source: §I · p. 2, A. Uncertainty Set over Plausible… · p. 4

  • Results

    Hardware experiments reduced the safety-filter failure rate from 44% to 13% and increased visuomotor policy success from 35% to 70%.

    Source: §I · p. 2

  • Takeaways & Limitations

    The framework supports robust decision-making directly in learned latent spaces without requiring physically interpretable disturbances.

    Source: §XI · p. 13

    The constraints are plausibility proxies and soft penalties can still yield implausible imaginations, while conformal coverage is marginal rather than input-conditional.

    Source: §X · p. 12

Abstract

from arXiv · show

In this paper, we study robust decision-making in the latent space of world models (WMs). Robust optimization is a mathematical framework where, given explicitly specified dynamics and physically meaningful disturbances, a robot can select actions that remain effective even under worst-case disturbances. However, applying this principle to the learned latent space of WMs introduces a fundamental challenge: because WMs have fully learned state spaces and dynamics inferred from high-dimensional observations, it is unclear how to define latent-space disturbances that faithfully represent uncertainty in the underlying system. Our key idea is to model a latent-space disturbance as a perturbation to the learned latent dynamics that induces pessimistic but plausible transitions. Specifically, we construct a set of plausible latent dynamics by combining a dynamics-aware similarity metric that captures plausible transitions with out-of-distribution detection that excludes implausible latent states. We calibrate this uncertainty set over latent dynamics using conformal prediction, ensuring that WM imaginations induced by the latent disturbance remain plausible without becoming overly pessimistic. We then jointly optimize robust robot actions and the worst-case latent disturbances through game-theoretic optimization. We leverage this latent-space robust optimization to robustify policy steering, considering two paradigms: latent safety filtering and sample-and-verify steering of a generative control policy. Our controlled simulation experiments show that our latent disturbance enables robust decision-making directly in WM latent spaces, and hardware experiments with a Franka manipulator show that modeling latent disturbances enables robust policy steering, reducing failures by 70% in safety filtering and 54% in sampling-based policy steering. Project website: https://junwon.me/LatentDisturbance/.

I. INTRODUCTION

The paper asks how to define plausible disturbances in learned world-model latent spaces, then introduces LUCID to calibrate and optimize them for robust decisions. Hardware experiments report improvements in safety filtering and visuomotor policy success.

I. INTRODUCTION

  • Latent world models learn compact representations and dynamics from observation-action data, but disturbances in their learned spaces are difficult to define because arbitrary latent perturbations may be physically implausible.The motivating example is egg transfer, where hidden mass or spatula oil can change the outcome of the same motion.Source: §I · pp. 1–2
  • LUCID combines dynamics-aware similarity, out-of-distribution detection, conformal calibration, and game-theoretic optimization to find plausible worst-case latent disturbances and robust actions.It is applied to latent safety filtering and sampling-based policy steering.Source: §I · p. 2
  • Hardware safety filtering reduced the failure rate from 44% to 13%, while visuomotor policy success rose from 35% to 70%.These are the reported Franka-robot outcomes for safety filtering and policy steering, respectively.Source: §I · p. 2

II. RELATED WORK

  • Prior work uses world models for robot prediction and decision-making, while latent safety filters typically evaluate safety under nominal dynamics and may overestimate it.World-model imaginations may also miss rare, high-impact outcomes when decisions rely on finite samples.Source: §II · pp. 2–3

III. PRELIMINARIES: LATENT WORLD MODELS

  • A latent world model encodes observation histories, decodes latent states into observations, and predicts distributions over next latent states; probabilistic dynamics capture uncertainty from unobserved disturbances.The paper focuses on recurrent state-space models trained with reconstruction and KL-divergence objectives.Source: §III · p. 3

IV. SETUP: ROBUST OPTIMIZATION IN LATENT SPACE

The setup casts latent-space robust optimization as choosing actions against worst-case plausible transitions. LUCID constructs and conformally calibrates the transition set to avoid both implausible pessimism and overly conservative decisions.

IV. SETUP: ROBUST OPTIMIZATION IN LATENT SPACE

  • A robust action minimizes task cost against plausible latent transitions, but misleading latent distances and high-likelihood out-of-distribution states can make the uncertainty set overly pessimistic.The setup allows immediate, finite-horizon cumulative, or infinite-horizon value costs.Source: §IV · pp. 3–4

V. CONFORMALIZED LATENT-SPACE DISTURBANCE

  • LUCID models disturbances as alternative latent dynamics and jointly optimizes the worst-case disturbance and robust action through game-theoretic optimization.Its uncertainty set combines dynamics-aware similarity, out-of-distribution detection, and conformal calibration.Source: §V · p. 4

A. Uncertainty Set over Plausible Latent Dynamics

  • A left-KL ball around the learned predictive distribution defines plausible latent transitions, avoiding reliance on Euclidean proximity that can include dynamically infeasible states.The KL radius controls the set's size.Source: A. Uncertainty Set over Plausible… · p. 4
  • An in-distribution constraint excludes high-likelihood but implausible latent states that could otherwise let the disturbance produce overly pessimistic imaginations.The implementation uses a flow-matching density estimator trained on training latent states; smaller scores indicate in-distribution.Source: A. Uncertainty Set over Plausible… · p. 4

B. Conformal Calibration of the Uncertainty Set

  • Conformal prediction calibrates the KL radius and OOD threshold using held-out in-distribution trajectories, with trajectory-level calibration addressing temporal dependence between transitions.The stated guarantees rely on exchangeability of calibration trajectories and the test trajectory; the OOD threshold provides an in-distribution recall guarantee at the selected miscoverage level.Source: B. Conformal Calibration of the… · pp. 4–5

C. Solving Latent-Space Robust Optimization

The section models uncertainty as plausible perturbations to latent dynamics and uses them to find robust actions. It applies this approach to safety filtering and sample-and-verify steering.

C. Solving Latent-Space Robust Optimization

  • The latent disturbance is an alternative dynamics model constrained to induce plausible transitions, rather than an arbitrary latent-state perturbation.The method jointly optimizes the worst-case disturbance and robust action within the calibrated uncertainty set.Source: C. Solving Latent-Space Robust… · p. 5, A. Setup: Policy Steering with World… · p. 6
  • A disturbance network adds residuals to the nominal dynamics’ distribution parameters, producing perturbed Gaussian or categorical latent dynamics.For Gaussian dynamics it perturbs the mean and covariance; for categorical dynamics it perturbs the logits.Source: C. Solving Latent-Space Robust… · p. 5
  • Gradient ascent searches for adverse dynamics under soft KL and OOD constraint penalties, then the robust action is optimized against the resulting disturbance.The disturbance and action form a two-player minimax game, with the disturbance updated faster to promote convergence.Source: C. Solving Latent-Space Robust… · p. 5

A. Setup: Policy Steering with World Models

  • Latent safety filters evaluate task actions with a safety value and override them when unsafe, while sample-and-verify selects among candidate action sequences using imagined outcomes.Both paradigms rely on sequences of plausible future transitions, rather than a single worst-case transition.Source: A. Setup: Policy Steering with World… · pp. 5–6
  • Nominal world-model imaginations can overestimate safety when disturbances make the same action produce qualitatively different outcomes.This can lead policy steering to choose actions that work only under nominal system realizations.Source: A. Setup: Policy Steering with World… · p. 6

B. Robustifying Policy Steering

The paper robustifies policy steering by using latent-space Hamilton–Jacobi–Isaacs reachability to learn worst-case disturbances and safety values. At runtime, those pessimistic imaginations support both safety filtering and sample-and-verify.

B. Robustifying Policy Steering

  • Both steering paradigms require a latent disturbance that drives the system toward failure through a sequence of plausible transitions.The method uses latent-space Hamilton–Jacobi–Isaacs reachability to recursively evaluate safety across those transitions.Source: B. Robustifying Policy Steering · p. 6
  • Off-policy SAC learns a robust safety critic, a disturbance network that perturbs nominal latent dynamics, and a robust safety policy.The disturbance is updated to minimize the safety value while staying within the calibrated uncertainty set.Source: B. Robustifying Policy Steering · p. 6
  • At runtime, the robust safety filter overrides task actions with non-positive robust safety values, while robust sample-and-verify scores candidates under pessimistic imaginations.The latter evaluates imagined latent trajectories over each action chunk so selection accounts for adverse outcomes under calibrated uncertainty.Source: B. Robustifying Policy Steering · pp. 6–7

VII. CASE STUDY: DUBINS’ CAR WORLD MODEL WITH KNOWN DISTURBANCES

In a controlled 3D Dubins’ car study, the authors compare latent safety filters against analytically computable robust solutions under discrete and continuous disturbances. LUCID more accurately identifies unsafe states and improves closed-loop safety relative to the stated baselines.

A. Experimental Setup

  • The study tests a 3D Dubins’ car with random action-sign flips and continuous positional disturbances in [−0.3, 0.3].For both settings, the ground-truth worst-case disturbance and corresponding robust action can be computed analytically.Source: A. Experimental Setup · p. 7
  • Dreamer’s Gaussian latent dynamics are trained offline on 4,000 observation–action trajectories, and a latent failure-margin predictor is trained from ground-truth labels.Evaluation measures unsafe-state classification and closed-loop failure rate on 1,000 trajectories.Source: A. Experimental Setup · p. 7

B. Can LUCID Make Robust Decisions in World Models?

  • LUCID closely approximates robust unsafe-set boundaries under two disturbance types, whereas Nominal is overly optimistic and WoN/CVaR approximate them inaccurately.The figure compares red unsafe-set estimates with solid ground-truth boundaries.Source: Figure 3 · p. 7, B. Can LUCID Make Robust Decisions in… · p. 8
  • Finite nominal sampling can miss adverse but plausible outcomes, especially with continuous disturbances; CVaR performance also depends on its risk level.The authors motivate explicitly optimizing a worst-case latent disturbance within the calibrated uncertainty set.Source: B. Can LUCID Make Robust Decisions in… · p. 8

C. Does the Uncertainty Set Represent Plausible Dynamics?

The experiments examine whether LUCID’s uncertainty set preserves plausible dynamics and whether it robustly steers vision-based manipulation. Results support the role of dynamics-aware similarity, OOD constraints, and calibration in avoiding both missed adverse transitions and implausible pessimism.

C. Does the Uncertainty Set Represent Plausible Dynamics?

  • Dynamics-unaware Euclidean similarity can admit infeasible latent states or exclude plausible distant ones, while removing OOD detection allows implausible transitions and high FPR.The uncertainty-set ablation also tests conformal calibration and OOD constraints.Source: C. Does the Uncertainty Set Represent… · p. 8, Table II · p. 8
  • A narrow KL threshold yields high FNR, while a broad threshold raises FPR; conformal calibration balances coverage of plausible dynamics against excessive conservatism.The thresholds control the uncertainty-set radius around nominal latent dynamics.Source: C. Does the Uncertainty Set Represent… · p. 8

D. Ablation: Optimistic Imagination with Latent Disturbances

  • In optimistic Dubins’ car imagination, LUCID learns a safety value close to the disturbance-free reference, while removing the OOD constraint permits implausible trajectories that evade failure.The figure contrasts estimated unsafe sets with ground-truth unsafe boundaries and the failure set.Source: Figure 4 · p. 9, Table III · p. 9

A. Experimental Setup: Vision-Based Block Pouring

  • The block-pouring simulation randomizes the green block’s mass, friction, and restitution without revealing those parameters to the robot.The setup uses visual observations and proprioception, with DreamerV3 trained on about 3,000 trajectories.Source: A. Experimental Setup: Vision-Based… · p. 9

B. Can LUCID Robustly Prevent Failures Under Uncertainty?

  • LUCID achieves lower failure rates while maintaining higher success rates than the compared filters in block pouring under randomized physical parameters.Nominal-sampling baselines are less effective in this higher-dimensional setting, where diverse plausible transitions make worst-case outcomes harder to sample.Source: Table IV · p. 9, B. Can LUCID Robustly Prevent Failures… · p. 10
  • LUCID’s robust monitor intervenes more preemptively than the nominal filter and achieves lower conditional failure rates and larger positive realized safety gains.Its safety actions move the system toward safer realized next states, while the nominal filter can show smaller or negative gains.Source: B. Can LUCID Robustly Prevent Failures… · p. 10

C. Ablation: Robustness Under Controlled Uncertainty

Across controlled disturbances, partial observability, and hardware manipulation, LUCID improves robust decision-making by reasoning over plausible adverse latent dynamics. The experiments also identify limits tied to uncertainty calibration, world-model fidelity, and game-theoretic optimization.

C. Ablation: Robustness Under Controlled Uncertainty

  • 77/100 teleoperation trajectories remained failure-free across all 20 physics settings under LUCID.It had the lowest overall and conditional failure rates among methods with nonzero task success; UnConf. achieved lower failure rates only with near-zero success.Source: C. Ablation: Robustness Under… · p. 10

D. Ablation: Robustness Under Partial Observability

  • LUCID further reduces failures and increases success in deterministic block pouring, where transition uncertainty comes from partial observability and world-model approximation.The authors qualify this benefit: the realized test-time transition must lie within the calibrated uncertainty set.Source: D. Ablation: Robustness Under Partial… · p. 11

IX. HARDWARE: ROBUST STEERING DURING CONTACT-RICH VISUOMOTOR POLICY EXECUTION

  • The hardware task is serving an egg from a spatula without dropping or flipping it, with uncertainty from unobserved physical properties such as friction.The Franka Research 3 uses a visual manipulation setup.Source: §IX · p. 11

A. Experimental Setup: Serving Sunny-Side-Up Fried Eggs

  • Hardware tests vary spatula surface among no tape, Scotch tape, and heavy-duty tape to introduce unobserved friction disturbances.The setup uses two RGB camera views and joint-position proprioception; actions run at 15 Hz.Source: A. Experimental Setup: Serving… · p. 11

B. Robust Latent Safety Filtering

  • LUCID lowers failure rates and raises robust rates across spatula surfaces, improves policy success over the Nominal filter, and proactively intervenes with safer fallback actions.The surface evaluation replays trajectories across three conditions; the policy comparison uses Scotch tape, and the teleoperator results are qualitative.Source: B. Robust Latent Safety Filtering · p. 12

C. Robust Sample-and-Verify with Latent Disturbances

  • LUCID improves sample-and-verify policy success, while Nominal does not improve on the base policy and UnConf. imagines implausible failures that weaken candidate discrimination.LUCID selects actions favorable under plausible worst-case futures.Source: C. Robust Sample-and-Verify with… · p. 12, Figure 10 · p. 12

X. LIMITATIONS

  • Plausibility constraints are proxies, conformal coverage is marginal rather than input-conditional, and calibration may weaken under distribution shift.The method also depends on adequate learned dynamics and objectives, and its high-dimensional game optimization does not guarantee a global saddle point.Source: §X · pp. 12–13

XI. CONCLUSION

  • LUCID models disturbances as adverse but plausible perturbations to learned latent dynamics, using similarity, out-of-distribution detection, and conformal calibration.The framework is instantiated for latent safety filtering and sample-and-verify steering, with improved policy success in simulated and real-robot manipulation.Source: §XI · p. 13

APPENDIX A THEORETICAL ANALYSIS: ROBUST SAFETY FILTER

The analysis shows that robust safety values do not overestimate realized safety under plausible latent dynamics, while nominal values can. The exact robust policy has zero worst-case suboptimality, subject to coverage and optimization conditions.

Zero Safety Overestimation

  • For any policy and plausible latent dynamics, the robust safety value is a lower bound on the realized safety value.The nominal value may overestimate safety when realized transitions are more pessimistic than nominal ones.Source: Appendix A · p. 15

Guarantee scope

  • The guarantee assumes the true latent dynamics lie in the uncertainty set, whereas conformal calibration provides only marginal probabilistic coverage.The analysis also assumes no uncertainty beyond that in the learned latent dynamics; learned disturbances and policies only approximate the game, so guarantees depend on achieving ambiguity-set feasibility and optimization.Source: Appendix A · pp. 15–16

Sub-optimality of Safety Policy

  • The exact robust safety policy achieves zero worst-case suboptimality for every latent state.This result is stated for the global uncertainty set under the rectangular ambiguity condition.Source: Appendix A · p. 16
  • A nominal safety policy can be suboptimal when adverse plausible dynamics are realized.An adverse but plausible dynamic can also make nominal safety values overestimate realized safety.Source: Appendix A · p. 16

APPENDIX B THEORETICAL ANALYSIS: CONFORMAL PREDICTION

This appendix establishes conformal coverage guarantees for calibrated latent-dynamics and in-distribution thresholds. It also extends calibration to dependent transitions by calibrating trajectory-level maximum scores.

Conformal Prediction

  • Conformal prediction calibrates nonconformity thresholds on held-out data to provide finite-sample, distribution-free coverage under exchangeability.The calibrated thresholds are used for the KL-ball radius and the OOD threshold.Source: Appendix B · pp. 16–17

Coverage of the Dynamics-Aware Similarity

  • Under exchangeability, the calibrated KL ball contains the test-time encoded latent distribution with probability at least 1 − α_KL.The bound is marginal over calibration data and the test sample.Source: Appendix B · p. 17

Dataset-Conditional KL Coverage

  • With i.i.d. calibration and test samples, the calibrated KL ball has coverage at least b_η with probability at least 1 − η over calibration data.This dataset-conditional bound accounts for coverage varying across calibration datasets.Source: Appendix B · p. 17

Class-Conditional In-Distribution Detection

  • The calibrated OOD detector accepts in-distribution latent states with recall at least 1 − α_ood under the stated exchangeability condition.This limits incorrect rejection of an ID state to α_ood, but ID calibration alone does not guarantee the ability to reject OOD states.Source: Appendix B · pp. 17–18

Causal Reconstruction from Trajectory Calibration

  • Calibrating the maximum nonconformity score per trajectory gives simultaneous coverage at every transition despite temporal dependence within trajectories.The guarantee relies on exchangeability between calibration and deployment trajectories.Source: Appendix B · p. 18

APPENDIX C BACKGROUND: HJ REACHABILITY ANALYSIS

The appendix presents HJ reachability as a safety-value approach: the controller maximizes the minimum future safety margin against disturbances. Its value function characterizes the maximal robust safe set and supports safety filtering.

Maximal Robust Safe Set

  • The maximal robust safe set contains states from which some control policy avoids failure indefinitely under every admissible disturbance.Failure is defined by a negative safety margin.Source: Appendix C · p. 18

Dynamic Programming Isaacs Equation

  • HJ reachability evaluates the smallest safety margin over time, with the controller maximizing that margin against the worst-case disturbance.A positive safety value indicates safety can be maintained; a negative value indicates unavoidable failure under an adverse disturbance.Source: Appendix C · p. 18
  • The infinite-horizon safety value defines the maximal robust safe set.The value is computed through the discrete-time dynamic programming Isaacs equation.Source: Appendix C · p. 18

Safety filtering

  • A safety filter permits a task policy’s action when its worst-case next-state value is nonnegative, and otherwise intervenes with the robust policy.This criterion keeps states within the robust safe set.Source: Appendix C · p. 18

Discounted Safety Update and Game-Theoretic Adversarial RL

  • Discounting makes the safety backup a contraction, while high-dimensional direct solution is intractable and motivates adversarial actor–critic approximation.The disturbance space is prescribed to capture uncertainty within the operational design domain; LUCID instead uses calibrated uncertainty over learned latent dynamics.Source: Appendix C · p. 19

A. Adversarial Reinforcement Learning

The algorithm trains a robust latent safety filter through adversarial reinforcement learning over world-model imaginations. It reuses imagined transitions for replay and stops rollouts when states are classified as out of distribution.

Adversarial Reinforcement Learning

  • The replay buffer stores imagined transitions and nominal latent predictive distributions, allowing disturbance resampling without re-querying the world model.Parallel rollouts are aggregated for minibatch replay updates, and twin safety critics use their minimum to mitigate value overestimation.Source: A. Adversarial Reinforcement Learning · p. 19
  • Training truncates an imagination when the next latent state exceeds the OOD threshold, preventing rollout into regions unsupported by training data.The world model and uncertainty models remain fixed throughout safety-filter training.Source: A. Adversarial Reinforcement Learning · p. 19
  • The algorithm alternates critic, controller, and disturbance updates during replay-based training from newly encoded trajectories.Each iteration imagines up to H_imag steps, with replay-update intervals and learning rates specified for the respective components.Source: A. Adversarial Reinforcement Learning · p. 19

B. Latent Safety Filter Training

Safety-filter training discourages actions that exploit out-of-distribution world-model predictions. The disturbance perturbs stochastic latent dynamics, while the Dubins’ car analysis derives a Hamiltonian-optimal robust action.

B. Latent Safety Filter Training

  • OOD state–action pairs can induce hallucinated transitions that let the safety policy exploit model errors and overestimate safety.Adding epistemic uncertainty to the failure margin encourages actions with outcomes that are both safe and supported by offline data.Source: B. Latent Safety Filter Training · p. 19

C. Disturbance Parameterization

  • The disturbance adds a residual to nominal dynamics parameters, changing only the stochastic prior while conditioning on nominal next-state predictions.For Gaussian dynamics it perturbs standard deviation; for categorical dynamics it adds to prior logits, with KL and OOD penalties constraining the perturbation.Source: C. Disturbance Parameterization · pp. 19–20

D. 3D Dubins’ Car with Disturbances

  • a⋆ = 0 is Hamiltonian-optimal for the Dubins’ car, and any nonzero turn lets the disturbance decrease safety through the heading contribution.Driving straight eliminates that contribution; the worst-case disturbance response depends on the sign of gθa.Source: D. 3D Dubins’ Car with Disturbances · p. 20

E. Simulation: Block Pouring

The block-pouring simulation uses randomized, paired physics configurations and safety filtering that accounts for model uncertainty. The real-world egg-serving setup trains a latent world model and safety filter from camera and robot observations.

E. Simulation: Block Pouring

  • Block-pouring physics are randomized at reset and held fixed per episode; paired evaluations reuse 20 pre-sampled configurations across source trajectories and filtering conditions.The safety margin uses a five-member probabilistic ensemble’s log-JRD uncertainty score to discourage OOD action exploitation of world-model imaginations.Source: E. Simulation: Block Pouring · pp. 20–21
  • DreamerV3 and behavior-cloned Diffusion Policy are trained independently as task policies, with the deployed policy executing four actions before replanning.The diffusion policy predicts eight future 7D actions using 100 denoising steps.Source: E. Simulation: Block Pouring · p. 21

F. Real-World: Sunny-Side-Up Egg Serving

  • The egg-serving world model uses two camera views, robot proprioception, transformer-based latent dynamics, and a 544-dimensional latent state for safety filtering.It jointly trains pixel and frozen-feature reconstruction with continuation, falling, and flipping prediction; failures are manually labeled from falling and flipping conditions.Source: F. Real-World: Sunny-Side-Up Egg Serving · pp. 21–22
  • The real-world safety-filter setup updates its critics, safety policy, and disturbance every 1, 4, and 2 replay updates, respectively.Source: F. Real-World: Sunny-Side-Up Egg Serving · p. 22
Loading 2610.07599v1…