PaperlayerView exact revision on arXiv

Source-linked AI summary

ACID: Action Consistency via Inverse Dynamics for Planning with World Models

Gawon Seo, Dongwon Kim, Suha Kwak

arXiv:2607.02403v1cs.ROcs.AIcs.CV

TL;DR

ACID addresses the blind spot in decision-time planning costs that score only terminal-state proximity and ignore whether intermediate predicted transitions are realizable. It adds inverse-dynamics cycle action consistency to the planning cost, yielding consistent improvements across four action-conditioned world models and six diverse tasks with substantially less planning compute than the baseline.

  • Problem

    Decision-time planning costs typically score only terminal-state proximity, leaving whether intermediate predicted transitions are realizable unchecked.

  • Method

    ACID feeds predicted transitions to an inverse dynamics model and adds the discrepancy between inferred and conditioning actions to the planning cost with a scale-invariant adaptive weight.

  • Results

    ACID consistently improves planning across four action-conditioned world models and six tasks, with substantially less total planning compute than the baseline.

  • Takeaways & Limitations

    Decision-time action-consistency verification improves planning across rigid and deformable manipulation, articulated control, and visual navigation.

  • Takeaways & Limitations

    ACID’s action-identification assumption weakens under partial observability or exogenous interventions, and the method requires one-time inverse-dynamics verifier training.

Abstract

from arXiv · show

Decision-time planning with action-conditioned world models has become a popular paradigm for embodied control. However, the standard planning cost judges a candidate solely by how close its predicted terminal state lies to the goal, leaving the realizability of the intermediate transitions unchecked -- a predicted trajectory can look convincing while the environment rollout drifts away from it. In this paper, we propose ACID, a decision-time planning framework that introduces cycle action consistency: the action inferred backward from a predicted transition by an inverse dynamics model should recover the one that was conditioned on. We fold this per-step residual into the planning cost via a scale-invariant adaptive weight. Across four action-conditioned world models and six tasks spanning rigid and deformable manipulation, articulated control, and visual navigation, ACID consistently improves planning and matches the baseline's accuracy with substantially less planning compute.

1 Introduction

Decision-time planning with world models can reward visually plausible terminal predictions without checking whether intermediate transitions are realizable by their conditioning actions. ACID addresses this blind spot by adding cycle action consistency to the planning objective, yielding consistent gains across models and tasks with less planning compute.

  • Problem: Standard planning costs score only terminal-state proximity, so unrealizable intermediate transitions can produce deceptively strong candidates.Realizability concerns whether each transition could be produced by the conditioning action, which terminal-state proximity cannot assess.
  • Approach: ACID costs the whole trajectory rather than only its terminal state by augmenting decision-time planning with cycle action consistency.This directly targets the planning-cost blind spot identified in the introduction.
  • Results: Across 4 action-conditioned world models and 6 tasks, ACID delivers consistent planning gains spanning manipulation, articulated control, and visual navigation.The models include three JEPA-style latent predictors and one video generative model; tasks span rigid and deformable object manipulation, articulated control, and visual navigation.
  • Results: ACID’s gains remain robust to hyperparameter choices while requiring substantially less total planning compute than the baseline.The comparison is explicitly against the baseline’s total planning compute.

2 Related Work

Prior work on executable-action world models is organized by how actions are obtained, with action-conditioned models using test-time search or optimization. Inverse dynamics models have generally decoded or shaped trajectories after planning rather than influencing the planner’s selected action sequence.

  • Action-conditioned world models: Action-conditioned world models predict next states from current states and actions, then search or optimize at test time for goal-reaching action sequences.This family includes latent predictors such as JEPA-based models and large video backbones.
  • Inverse dynamics models: Inverse dynamics models map consecutive states to the action that produced their transition and are widely used in robot learning.Prior work uses inverse dynamics models in three roles, none of which determines the planner’s selected action sequence.
  • Inverse dynamics models: As action decoders, inverse dynamics models recover executable actions from trajectories generated by state-only or video-only planners after those trajectories have already been selected.UniPi generates a future video and LDP generates a latent state trajectory before inverse dynamics recovers the actions.
  • Inverse dynamics models: As auxiliary tasks, inverse-dynamics prediction adds a loss term during training rather than guiding action-sequence selection at planning time.The supplied passage describes this role as shaping representations during training.

3 Method

ACID augments CEM-based decision-time planning with cycle action consistency, using an inverse dynamics verifier to penalize predicted transitions that do not match their conditioning actions. The resulting adaptive, scale-invariant cost combines goal proximity with per-step realizability while reusing the existing predicted trajectory.

  • Method: ACID adds a per-step realizability measure to decision-time planning and integrates it into the planning cost with a scale-invariant adaptive weight.The method comprises world-model MPC with CEM, cycle action consistency verified by an inverse dynamics model, and adaptive cost weighting.
  • Cycle action consistency: The inverse dynamics model infers the action responsible for each predicted latent transition, enabling comparison with the action that conditioned the forward prediction.If a forward prediction reflects its conditioning action, backward inference should recover that action.
  • Cycle action consistency: The action consistency residual vanishes when a predicted transition matches the conditioning action and increases as the transition drifts from it.Residuals are aggregated across the planning horizon to form the candidate sequence’s action consistency cost.
  • Cycle action consistency: The consistency cost reuses the predicted trajectory already evaluated for the goal cost, requiring no additional world-model rollout.This preserves the existing trajectory computation while adding the verifier-based signal.
  • Augmented planning cost: CEM ranks candidates using an augmented cost that jointly accounts for goal proximity and action consistency, with adaptive weighting based on their relative spread.The weight equalizes the consistency and goal-cost spreads up to λ, which is tuned once per world model and transfers across tasks without per-task retuning.

4 Experiment

Experiments evaluate ACID across six tasks and four action-conditioned world models, testing generality, robustness, and planning efficiency. ACID improves planning across every evaluated world model and task while reducing compute needed to reach baseline quality.

  • Experimental setup: Experiments span six tasks covering robotic-arm, articulated, tabletop-pushing, deformable-object, and goal-conditioned visual-navigation settings.The tasks are Cube, Reacher, PushT, Rope, Granular, and visual navigation.
  • Experimental setup: Planning uses four action-conditioned world models: DINO-WM, PLDM, Le-WM, and the video-generative NWM trained with CompACT.The models include three JEPA-style latent predictors and one video generative model, with training and the CEM planner held fixed while only planning cost varies.
  • Planning performance: ACID improves planning across every world model and task, raising success rate for Le-WM and PLDM and reducing Chamfer distance for DINO-WM on deformable manipulation.Benefits also extend to visual navigation with NWM and CompACT; lower is better for Chamfer distance, ATE, and RPE.
  • Hyperparameter robustness: ACID remains effective across CEM budgets from 30 to 300 samples and consistency weights λ from 0.005 to 0.1.It consistently outperforms Original at every tested budget, and a broad λ range yields gains without task-specific tuning.
  • Planning efficiency: 11.5%, 8.9%, 25.0%, and 39.0% are the inverse-dynamics verifier’s per-iteration latency overheads for Le-WM, PLDM, NWM with CompACT, and DINO-WM, respectively.Despite this overhead, ACID reaches Original’s final Chamfer distance in less than half the planning steps, requiring approximately 0.7× the compute to match baseline quality.
  • Planning efficiency: At the smallest CEM budget, n=30, ACID already matches full-budget Original at n=300 on Le-WM and PLDM.This corresponds to an order-of-magnitude reduction in samples, while ACID reaches a given quality with strictly less total planning compute across both tested axes.

5 Conclusion

ACID is a decision-time planning framework for action-conditioned world models that augments the standard goal cost with cycle action consistency. It computes this consistency cost by comparing conditioning actions with actions inferred from the predicted trajectory by a separately trained inverse dynamics model.

  • ACID augments planning for action-conditioned world models with a cycle action consistency cost.
  • The framework feeds the predicted trajectory into a separately trained inverse dynamics model.
  • The added cost measures discrepancy between inferred actions and the actions used to condition the predicted trajectory.

6 Limitation · Appendix

ACID relies on consecutive observations identifying the intervening action, a property weakened by partial observability and exogenous transition perturbations. It also requires one-time inverse-dynamics verifier training using the same offline trajectories, with bounded overhead.

  • 6 Limitation: ACID’s action-identification property weakens under partial observability.The method assumes a pair of consecutive observations identifies the action between them.
  • 6 Limitation: Exogenous interventions can perturb transitions beyond what the conditioning action explains.Such perturbations weaken the property ACID exploits throughout.
  • 6 Limitation: ACID requires one-time inverse-dynamics verifier training, but the overhead is bounded by reusing the same offline trajectories.The passage states that verifier training adds no new trajectories.

A Implementation Details of IDM · B World model baselines · B.1 Model Descriptions

The appendix specifies a transformer-based inverse dynamics model trained on task-specific latent transitions and describes four action-conditioned world-model baselines. These baselines comprise three JEPA-style latent predictors and one video generative model with distinct representation-learning and collapse-prevention strategies.

  • A Implementation Details of IDM: The IDM is a 4-layer, 3-head transformer with width 192 that predicts actions between consecutive latents using a prefix–suffix attention structure.Latents z_t and z_t+1 form the prefix, while a suffix token carries the noisy action and is mapped to action-space velocity.
  • A Implementation Details of IDM: Each IDM is trained separately per task on offline transitions within episode boundaries, using frozen world-model encoder latents and standardized actions.Actions are normalized with per-dimension training-set statistics and de-normalized during inference.
  • A Implementation Details of IDM: IDM training runs for 200K steps with batch size 256 and AdamW, while inference uses one Euler step for most tasks and ten for visual navigation.The optimizer uses peak learning rate 10^-4, linear warmup, and cosine decay.
  • B World model baselines: Experiments compare four action-conditioned world models: three JEPA-style latent predictors and one video generative model.The JEPA-style models share encoder–predictor structure but differ in latent acquisition and representation-collapse prevention.
  • B.1 Model Descriptions: Le-WM jointly learns a ViT encoder and transformer predictor from pixels, using adaptive layer normalization for actions and SIGReg for collapse prevention.SIGReg encourages isotropic-Gaussian latent embeddings through normality tests on random one-dimensional projections.
  • B.1 Model Descriptions: PLDM jointly trains encoder and predictor from pixels, optionally ensembles predictors, and prevents collapse with variance, covariance, temporal, and IDM regularization.Its training objective therefore combines several regularizers.
  • B.1 Model Descriptions: DINO-WM learns only its predictor over frozen pretrained DINOv2 features, and experiments disable non-visual inputs so methods observe only pixels unless noted.The fixed representation sidesteps representation collapse.
  • B.1 Model Descriptions: NWM with CompACT is a video-generation world model using a Conditional Diffusion Transformer, with CompACT-16 replacing SD-VAE to encode observations into 16 discrete tokens.The tokenizer replacement keeps decision-time prediction tractable.

B.2 Training configuration · C Experimental settings · C.1 Environments and Tasks.

The experiments use established training configurations or released checkpoints across six continuous-control benchmarks. These tasks cover rigid and deformable manipulation, articulated control, and goal-conditioned visual navigation, testing contact, geometry, and collective dynamics.

  • B.2 Training configuration: PLDM and Le-WM use the official Le-WM codebase and training configuration, with PLDM adopting hyperparameters and code from the Le-WM paper.This preserves the reported implementation settings for both world models.
  • B.2 Training configuration: DINO-WM follows the original paper’s training configuration and official codebase without modification for Rope and Granular tasks.The configuration is applied specifically to these two tasks.
  • B.2 Training configuration: NWM uses an author-provided checkpoint trained with the 16-token CompACT tokenizer rather than reproducing training.The checkpoint is used to reproduce the reported results.
  • C.1 Environments and Tasks.: The evaluation spans six continuous-control benchmarks covering rigid and deformable manipulation, articulated control, and goal-conditioned visual navigation.The benchmarks present distinct world-model planning challenges, including contact dynamics and deformable-object coordination.
  • C.1 Environments and Tasks.: Push-T requires a blue circular agent to push a green T-shaped block into a prescribed gray target configuration.Success depends on accurately modeling contact dynamics between the agent and object.
  • C.1 Environments and Tasks.: Reacher controls a two-jointed arm to align its full configuration, rather than only its end-effector, with a target pose in 2D.Its training dataset contains 10,000 episodes of 200 steps collected with a Soft Actor-Critic policy.
  • C.1 Environments and Tasks.: OGBench-Cube requires a robotic arm to grasp and relocate a cube to a target in 3D space, using the single-cube variant.The benchmark combines full 3D geometry with coordinated grasping and placing and is described as the most challenging of the three benchmarks considered.
  • C.1 Environments and Tasks.: Rope Manipulation tests deformable-object control by having an XArm move a soft rope toward a test-time goal configuration in Nvidia Flex.The task probes prediction of deformable dynamics and robot-induced contact behavior; its dataset contains 1,000 trajectories of 20 random-action steps.

C.2 Details of evaluation protocol … D Comparison of CEM and ACID.

The evaluation follows prior protocols across four world-model settings, while planning uses CEM configurations tailored to each task. ACID augments CEM with inverse-dynamics action consistency, adaptive weighting, and an explicitly provided optimization procedure; visualization decoders are trained separately and used only for visualization.

  • C.2 Details of evaluation protocol: Le-WM and PLDM follow the original Le-WM goal-conditioned evaluation protocol, defined by evaluation budget and goal distance.The initial state is randomly sampled from an offline trajectory.
  • C.2 Details of evaluation protocol: DINO-WM follows its original protocol; deformable-object tasks use Chamfer Distance rather than binary success and evaluate 10 instances each.The rope and granular tasks push an object into a target configuration, with environment stepping cost motivating the 10-instance evaluation.
  • C.2 Details of evaluation protocol: NWM with CompACT follows the original configuration and reports Absolute Trajectory Error and Relative Pose Error for navigation trajectories.ATE measures global trajectory fidelity, whereas RPE measures local consistency.
  • D Comparison of CEM and ACID.: ACID’s procedure adds inverse-dynamics actions, an action-consistency cost, and an adaptive weight wa = λ · σg/σa to CEM planning.Algorithm 3 retains CEM sampling, elite selection, and optimized-sequence return while incorporating the consistency residual.
  • C.3 Planning solver: CEM uses 300 samples and 30 optimization steps for Le-WM and PLDM, 200 or 100 samples and 10 steps for DINO-WM, and 80 samples with one step for NWM.The elite counts are 30 for Le-WM, PLDM, and DINO-WM, and 5 for NWM.
  • C.4 Decoders of Le-WM and PLDM, for visualization only: The visualization decoder maps a frozen world-model CLS embedding to pixels using 196 patch queries and a 4-layer, 8-head Transformer decoder.It is trained for visualization only; the encoder and projector remain frozen, and the decoder cannot exploit encoder spatial features.
  • D Comparison of CEM and ACID.: Algorithms 2 and 3 provide pseudo-code for the two test-time optimization methods, with Algorithm 3 highlighting its differences from Algorithm 2.The ACID pseudo-code version matches the version presented in the main paper.

E Empirical justification for the scale-invariant adaptive weight

The empirical measurements show that the relative within-pool spread of goal and action-consistency costs varies across world models, tasks, and CEM iterations. Recomputing w_a = λ · σ_g/σ_a therefore keeps the weighted consistency spread aligned with the goal-cost spread at every iteration.

  • Motivation: The adaptive weight is motivated by variation in the relative spreads σ_g and σ_a across world models, tasks, and CEM iterations.The proposed setting is w_a = λ · σ_g/σ_a.
  • Measurement protocol: Each plotted value is the median σ_g, σ_a, or σ_g/σ_a across candidate pools at a given CEM iteration.A pool contains N CEM candidates evaluated for one environment, MPC step, and CEM iteration.
  • Why spread matters: Candidate reranking depends on within-pool variation because constant cost offsets do not change elite ordering.The augmented cost is c_g + w_a c_a, and each term influences selection through its differences across candidates, measured by standard deviation.
  • Observed variation: Figure 6 shows that σ_g/σ_a has different overall levels across world models and varies across tasks and CEM iterations.The figure plots σ_g, σ_a, and their ratio on a log-scaled y-axis; the dashed σ_g/σ_a = 1 line marks equal spread.
  • Effect of adaptation: Recomputing σ_g/σ_a each iteration makes w_aσ_a = λσ_g, preventing the consistency term from dominating or vanishing as σ_a changes.This leaves λ as the parameter controlling the strength of action consistency.

F Per-step overhead of inverse dynamics verifier

This section measures inverse-dynamics verifier overhead relative to world-model rollout cost during planning. Latencies are reported per CEM iteration on the same batched candidate set, excluding fixed goal-image encoding.

  • Latency comparison: Table 5 compares world-model forward latency tWM with inverse-dynamics verifier latency tIDM and reports tIDM/tWM as a percentage.tIDM is measured at one ODE step for all models except NWM, which uses ten; values are mean per-iteration wall time averaged across tasks when applicable.
  • Measurement protocol: Each latency measurement is one batched forward over all B × S candidate samples in a single CEM iteration, rather than per candidate.The measured primitives are the world-model rollout and inverse-dynamics verifier forward.
  • Measurement protocol: Verifier latency is normalized to world-model latency using the same candidate set and call, while fixed per-iteration goal-image encoding is excluded.Measurements use a single RTX 6000 Ada GPU, with S and CEM optimization steps varying by task.

G Qualitative Results

The qualitative evaluation compares imagined and actual rollouts to assess whether latent predictions follow planned actions. For navigation without environment rollouts, it instead compares imagined frames with ground-truth video and overlays action consistency scores.

  • Navigation comparison: Figure 8 evaluates goal-conditioned navigation with NWM and CompACT by comparing ground-truth videos against imagined rollouts driven by ground-truth actions.Because no environment rollout is available, the evaluation feeds the ground-truth action sequence into the world model from the initial observation.
  • Rollout comparison: Figure 7 compares baseline planning with ACID across Le-WM and PLDM on OGBench-Cube, Reacher, and Push-T.An auxiliary decoder maps latent states to pixels, enabling visual comparison between imagined and actual environment rollouts.
  • Navigation comparison: A drop in ACID’s action consistency cost identifies a step where the predicted next frame fails to follow the action condition.Figure 8 overlays the per-step action consistency score with navigation trajectories and highlights the failure step in red.
Loading 2607.02403v1…