Source-linked AI summary
CAER: Causal Action Effect Reweighting for World Model Training
Jianjie Fang, Xvyuan Liu, Ziyou Wang, Rongze Tang, Zhaolu Wang, Zhuohang Li, Xin Zhang, Haisheng Su, Chen Gao, Wei Wu, Xinlei Chen, Yong Li
TL;DR
Uniform MSE can let background tokens dominate training while sparse interaction dynamics remain under-optimized in action-conditioned world models. CAER reallocates fixed supervision mass toward tokens identified online through action-conditioned prediction contrasts, without external annotations. Across heterogeneous tasks, it converges to better solutions than uniform MSE, improving visual quality, physical consistency, and controllability.
Problem
Uniform space-time MSE allows abundant background tokens to dominate gradients while sparse action-dependent interactions receive diluted supervision, limiting learning of how actions change the world.
Method
CAER contrasts predictions under real and null-action conditions online, normalizes the resulting action-effect map, and reallocates fixed coefficient mass toward causally responsive tokens.
Results
Across four heterogeneous action-conditioned settings, CAER consistently outperforms uniform MSE and develops increasingly accurate interaction focus as training progresses.
Takeaways & Limitations
CAER provides an annotation-free, action-agnostic training signal that improves interaction modeling together with visual generation across distinct control modalities.
Takeaways & Limitations
CAER’s action-effect contrast is predictive rather than interventional in the do-calculus sense and depends on a bounded residual dependence through the noised future.
Abstract
from arXiv · showhide
World models are becoming core infrastructure for embodied intelligence, with action-conditioned video generation providing controllable predictions of how scenes evolve after agent interventions. Yet existing models are commonly trained with space-time-uniform mean squared error, allowing abundant background tokens to dominate the gradient while sparse interaction dynamics remain under-optimized; such uniform fitting rewards reconstructing appearance rather than learning how actions change the world. We introduce Causal Action Effect Reweighting (CAER), a general training paradigm that redistributes supervision toward the tokens whose predicted future is causally affected by the action. CAER contrasts the model's own predictions with and without action conditioning to localize these tokens online, then normalizes the resulting effect map into a weight that preserves the total coefficient mass and changes only where it is spent. This online signal requires no external annotations or offline preprocessing, avoids additional data-processing time, and scales naturally with model and dataset size. Experiments across heterogeneous action-conditioned world-model tasks show that CAER converges to better solutions than uniform MSE training, with consistent improvements in the physical consistency, controllability, and visual quality of generated videos.
1 Introduction
CAER addresses the mismatch between uniformly weighted reconstruction and sparse action-dependent interactions by reallocating supervision toward tokens whose predicted futures respond to actions. It computes this signal online, preserves total coefficient mass, and improves interaction modeling and visual generation across heterogeneous action-conditioned tasks.
- Motivation: Uniform space-time MSE lets abundant background tokens dominate gradients while sparse contact events, manipulated objects, and actor motion receive diluted supervision.This can produce visually realistic rollouts with mistimed contacts, incorrect object motion, or failures to respond to commanded actions.
- Motivation: External segmentation, tracking, optical flow, and contact annotations are costly to scale and can transfer the biases and failure modes of their source models.They also define relevance through a fixed annotation pipeline rather than the model’s action-conditioned response.
- Research questions: The experiments test whether CAER improves interaction-region prediction, whether action-dropout and fixed-noise choices affect stability, and whether its self-computed signal sharpens during training.The study also examines whether gains extend to task-level success rather than appearance metrics alone.
- CAER: CAER identifies action-causal regions by contrasting predictions under the observed action and a learned null action at a fixed intermediate noise level.Action dropout keeps the counterfactual query in distribution, and the resulting response map is normalized sample-wise.
- CAER: The normalized effect map preserves total coefficient mass, reallocating supervision without enlarging the loss scale or excluding tokens from optimization.This makes comparison with uniform averaging an allocation question while action dropout continues to provide stochastic supervision for every token.
- Findings: Across heterogeneous action-conditioned tasks, CAER converges to better solutions than matched uniform-MSE training, with stronger visual quality and physical consistency.As training proceeds, its focus contracts onto causal interaction regions and predicts those dynamics with progressively greater accuracy.
2 Methodology
CAER reallocates a fixed amount of supervision toward tokens whose predicted futures are sensitive to actions. It estimates this dependence online through matched action and null-action predictions, then uses normalized effect weights while retaining coverage and analyzing optimization benefits.
- Effect estimation: CAER compares action-conditioned and null-action predictions at a fixed noise level to identify tokens whose predicted velocity changes with the action.The null branch is trained through action dropout, keeping action-free queries in distribution; shared noise removes sampling variation.
- Budget-neutral weighting: The action-effect map is normalized independently for each sample, preserving the total coefficient mass while redistributing it across future tokens.The detached weights avoid increasing the loss scale or shifting coefficient mass between samples.
- Recall floor: Action dropout supplies a recall floor: dropped samples use unit weights through the null branch, so every token remains reachable by optimization.This addresses false negatives in the measured effect map without inflating coefficient mass through a hand-set lower bound.
- Optimization effect: At equal coefficient mass, focused weighting reduces interaction risk faster than uniform weighting when token weights have positive covariance with token utility.The analysis separates action-dependent tokens from background tokens and links positive weight–utility association to a faster first-order reduction.
- Optimization effect: Focused weighting can decrease interaction risk even when uniform training stalls because background gradients cancel sparse interaction gradients.The strict first-order decrease holds without assuming alignment between interaction and background gradients, under the stated condition.
- Population behavior: The reweighted objective can retain the uniform population optimum under an uncorrelated effective weight, while variance inflation grows as the map concentrates.The construction limits bias through shared noisy inputs, fixed effect-map noise, and sign-discarding channel norms; the variance factor is tied to the weight’s second moment.
3 Experiments
CAER is evaluated against matched uniform-MSE training across four heterogeneous action-conditioned settings, with analyses of benchmarks, training dynamics, and hyperparameters. It consistently improves overall world-model performance while concentrating supervision on action-relevant regions, although early checkpoints may favor uniform MSE and some appearance metrics remain unchanged or decrease.
- Main comparison: CAER is compared with uniform MSE across four heterogeneous action-conditioned tasks using matched initialization, data, interfaces, and optimization schedules.The evaluation covers camera control, LIBERO, RoboTwin, and PoseAnything, with official metrics spanning visual quality, motion quality, physical consistency, controllability, and semantics.
- Main comparison: CAER consistently outperforms the matched uniform-MSE baseline across all four action-conditioned world-model settings.The improvement holds for robot manipulation, camera control, and pose-conditioned generation under the same backbone, data, and schedule.
- Main comparison: CAER improves benchmark scores from 0.6412 to 0.6614 on camera control, 57.66 to 61.79 on LIBERO, 62.35 to 63.13 on RoboTwin, and 0.7422 to 0.7746 on PoseAnything.The largest gains occur in motion- and interaction-sensitive dimensions, while several appearance or trajectory metrics remain unchanged or slightly decrease.
- Qualitative analysis: CAER loss concentrates on action-relevant regions across four control modalities while downweighting already-modeled content.Figure 3 visualizes generated videos alongside CAER supervision maps, where darker red denotes larger loss values and darker purple denotes smaller loss values.
- Hyperparameter analysis: Approximately 10% action dropout balances null-branch training against action-irrelevant regions dominating the reweighting signal.Very small dropout rates insufficiently train the null-action branch, whereas excessively large rates make irrelevant regions dominate.
- Hyperparameter analysis: τ_S = 0.5 preserves action dependence and scene context, producing the most informative action-effect map.Extremely small or large fixed-noise values either suppress branch differences or weaken scene-specific localization.
- Training dynamics: Uniform MSE often leads at the earliest checkpoints, but CAER catches up and generally overtakes it later on both benchmarks.The late-stage advantage persists despite local fluctuations, while uniform MSE largely plateaus with interaction errors unresolved.
4 Related Work
Recent video world models support diverse action-conditioned interfaces, including camera, navigation, game, robot, hand, body, and pose control. Despite this diversity, many retain space–time-uniform denoising objectives, while external interaction-aware reweighting pipelines require preprocessing or simulation.
- Video-generative world models: Video world models use large diffusion and flow-matching backbones to provide visual and temporal priors for interactive scene exploration and control.Examples include Wan, HunyuanVideo, CogVideoX, Cosmos, Genie, Matrix-Game, HY-World, Cosmos-Predict, and Worldscape-MoE.
- Video-generative world models: Action-conditioned systems span camera trajectories, navigation commands, game controls, robot actions, and human motion.Related systems extend world models to robot, hand, body, and navigation control.
- Video-generative world models: Many action-conditioned world models retain space–time-uniform denoising objectives, allowing sparse action consequences to be overwhelmed by background tokens.CAER addresses this shared optimization problem by redistributing supervision after action injection.
- External spatial supervision: External interaction-aware reweighting methods use segmentation, optical flow, masks, rendered supervision, or simulator contact maps.These approaches include SAM, RAFT, MotiF, BridgeV2W, Mask World Model, Mask2Real-WM, and ChronoDreamer.
- External spatial supervision: External supervision pipelines require per-frame preprocessing, rendering, or simulation.Their use of external models or fixed supervision sources distinguishes them from CAER’s online model-derived signal.
5 Conclusion and Future Work
The paper proposes CAER, which identifies action-causal regions by contrasting predictions with and without actions, emphasizes unresolved interactions, and preserves overall coefficient mass. Across four heterogeneous action-conditioned settings, CAER consistently outperforms uniform MSE while developing increasingly accurate interaction focus.
- Conclusion: CAER identifies action-causal regions by contrasting predictions with and without actions, then emphasizes unresolved interactions while preserving the overall coefficient mass.The framework is presented as a general world-model training objective.
- Conclusion: Across four heterogeneous action-conditioned settings, CAER consistently outperforms uniform MSE and develops increasingly accurate interaction focus as training progresses.The paper positions this result as evidence for the generality of CAER as a training paradigm.
- Future Work: Future work will examine action causality in learned dynamics, generalization, controllability, longer horizons, larger backbones, evaluation, and data selection.These directions extend the contrast-based framework beyond its current training use.
A Additional Qualitative Comparisons
The appendix compares uniform MSE and CAER supervision maps across four action-conditioned settings. CAER concentrates supervision on regions associated with camera motion, manipulation, articulated pose, and relevant workspace interactions rather than distributing it broadly across scenes.
- Comparison Guide: Each figure presents representative conditioning or original frames with supervision maps from uniform MSE and CAER, using warmer colors for larger responses and cooler colors for smaller responses.The appendix applies the same color convention as Figure 3 throughout.
- Camera Control: In camera-control examples, CAER emphasizes regions associated with camera-induced motion and scene changes while suppressing already modeled background content.Uniform MSE produces spatially broader responses.
- RoboTwin: In RoboTwin manipulation examples, CAER concentrates responses around the robot end-effector, manipulated objects, and interaction or contact regions.Uniform MSE assigns substantial responses to broader scene regions.
- PoseAnything: In PoseAnything examples, CAER focuses supervision on articulated subjects and pose-bearing regions rather than surrounding visual content.Uniform MSE exhibits more diffuse responses.
- LIBERO: In LIBERO manipulation examples, CAER emphasizes the robot, manipulated object, and relevant workspace involved in the action.Uniform MSE spreads supervision more broadly across the scene.