PaperlayerView exact revision on arXiv

Source-linked AI summary

Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goals

Nate Gillman, Yinghua Zhou, Zitian Tang, Evan Luo, Arjan Chakravarthy, Daksh Aggarwal, Michael Freeman, Charles Herrmann, Chen Sun

arXiv:2601.05848v2cs.CVcs.AIcs.RO

TL;DR

Existing goal specifications for video-based planning can be too abstract or infeasible for dynamic physical tasks. Goal Force conditions video generation on desired force vectors and trains on synthetic causal primitives, enabling the model to infer antecedent actions and generalize to complex scenarios. The paper reports causal-chain planning, tool use, and diverse valid plans despite the simple training data.

  • Problem

    Text instructions can be too abstract for physical nuances, while target images can be burdensome or infeasible for dynamic tasks.

  • Method

    Goal Force trains a video model with explicit goal-force control and synthetic causal primitives to generate physically plausible antecedent causal chains.

  • Results

    The model generalizes from simple synthetic interactions to multi-object force propagation, tool use, and human-object planning in complex out-of-domain scenarios.

  • Takeaways & Limitations

    The results support using grounded video generation as an implicit neural physics planner for reasoning backward from desired physical outcomes.

Abstract

from arXiv · show

Recent advancements in video generation have enabled the development of ``world models'' capable of simulating potential futures for robotics and planning. However, specifying precise goals for these models remains a challenge; text instructions are often too abstract to capture physical nuances, while target images are frequently infeasible to specify for dynamic tasks. To address this, we introduce Goal Force, a novel framework that allows users to define goals via explicit force vectors and intermediate dynamics, mirroring how humans conceptualize physical tasks. We train a video generation model on a curated dataset of synthetic causal primitives-such as elastic collisions and falling dominos-teaching it to propagate forces through time and space. Despite being trained on simple physics data, our model exhibits remarkable zero-shot generalization to complex, real-world scenarios, including tool manipulation and multi-object causal chains. Our results suggest that by grounding video generation in fundamental physical interactions, models can emerge as implicit neural physics simulators, enabling precise, physics-aware planning without reliance on external engines. We release all datasets, code, model weights, and interactive video demos at our project page.

1. Introduction

Goal Force addresses the difficulty of specifying physical goals by conditioning video generation on desired forces and intermediate dynamics. Trained on simple synthetic causal interactions, it generalizes to multi-step physical reasoning and complex out-of-domain scenarios.

  • Motivation: Text instructions can be too abstract for complex physical tasks, while target images can be burdensome or infeasible to specify.The paper motivates force-based goals as an alternative for dynamic tasks requiring precise physical outcomes.
  • Motivation: Goal Force defines goals through desired force vectors and intermediate dynamics rather than static endpoints or direct low-level scene manipulation.This framing follows how people decompose physical tasks into concrete sub-goals involving location, motion, and dynamics.
  • Method: The method trains a video model with explicit goal-force conditioning and synthetic causal primitives such as elastic collisions and falling dominos.The training data pairs videos with goal forces, while the approach uses fundamental interactions to bootstrap more complex reasoning.
  • Results: The model learns to propagate forces through chains of events, where one object influences another and then a third.This capability supports causal planning rather than merely rendering an immediate physical reaction.
  • Results: Despite simple training data, the model performs zero-shot tool-use planning, including using a golf club to impart force and picking up a rose by its stem.These results are presented as evidence that the model acts as an implicit neural physics simulator.
  • Resources: The paper releases training and evaluation code, model weights, synthetic data, and benchmark datasets.The materials are available at the project page.

2. Related Works

Prior video-generation methods provide strong visual priors but generally offer limited physical control or specify direct motion and interventions. Goal Force instead conditions generation on desired effects and plans antecedent causal chains without requiring an external simulator at inference.

  • Video generation and control: Video generative models can render visually faithful sequences and complex dynamics but typically lack fine-grained control over physical actions and interactions.Existing controllable methods include camera, drag-based, trajectory, and optical-flow controls.
  • Force-conditioned generation: Trajectory-based controls specify motion rather than the underlying cause, while Force Prompting specifies direct physical interventions.Goal Force targets the causal structure behind a desired physical outcome.
  • Goal Force: Goal Force asks the model to generate an antecedent causal chain, such as ball A striking ball B to produce a desired force on ball B.The desired effect is specified while the low-level causal plan is inferred by the model.
  • Physics simulators and hybrids: Explicit physics simulators can be physically accurate but generally require 3D geometry, while hybrid models remain constrained by their internal simulator.Goal Force pursues physics-aware planning through learned video generation rather than an external simulator at inference.

3. Method: Prompting with Goal Force

Goal Force reframes force-conditioned video generation as planning: users specify a desired force on a target, and the model generates a physically plausible causal chain to achieve it. A multi-channel control signal and synthetic causal curriculum train the model as an implicit neural physics planner.

  • Prompting with Goal Force: Given a starting frame and text prompt, Goal Force generates an antecedent causal chain that achieves a specified goal force on a target object.The low-level causal plan, such as which ball strikes another, remains unspecified so the model must infer it.
  • Prompting with Goal Force: The method trains a video generative model to act as an implicit neural physics planner using a multi-channel control signal and synthetic-data curriculum.The approach requires no simulator at inference.
  • Multi-Channel Physics Control Signal: A 3-channel physics tensor encodes direct force, goal force, and optional mass across video space and time.Direct force represents the immediate cause; goal force represents the desired effect; mass provides privileged physical information when available.
  • Goal Reaching via Implicit Planning: Training data combines synthetic causal chains and complex dynamics, including 3k domino videos, 6k rolling-ball videos, and 3k PhysDreamer Carnation videos.The scenes are generated with Blender and PhysDreamer.
  • Goal Reaching via Implicit Planning: Figure 4 contrasts prior methods that directly apply forces or follow trajectories with Goal Force, which generates causes for a desired effect.PhysGen, PhysDreamer, and Force Prompting accept force conditioning, while Tora accepts trajectories.
  • Goal Reaching via Implicit Planning: Randomly masking direct, goal, and mass channels trains both goal-to-plan inference and action-to-outcome simulation.The model must infer antecedent direct-force events from goals and simulate collisions and secondary forces from actions.
  • Architecture and Training Details: The architecture uses a ControlNet physics-conditioning module on Wan2.2, fine-tuned for the high-noise expert responsible for global structure and low-frequency dynamics.The ControlNet clones the first 10 DiT layers from the pretrained model while the base model remains frozen.

4. Experimental Comparisons

Goal Force is evaluated against text-only baselines through human judgments and visual planning tests spanning increasingly generalized physical interactions. It achieves stronger goal-force adherence while preserving motion realism and visual quality, and prior direct-force methods are not designed for antecedent causal planning.

  • Comparison to Text-Only Baselines: The evaluation uses a 75-scene benchmark and a 2AFC human study with 40 Prolific participants comparing Goal Force against text-only baselines.The benchmark scenes are drawn from permissively licensed web sources and generative models.
  • Human Study for Generalization: The benchmark spans two-object collisions, multi-object collisions, human-object interactions, and other increasingly generalized categories.These categories test generalization beyond the simple synthetic training scenes.
  • Comparison to Text-Only Baselines: Goal Force consistently outperforms both text-only baselines on goal-force adherence, with minimal degradation in motion realism and visual quality.Participants evaluated goal-force adherence, realistic motion, and visual quality.
  • Comparison to Prior Methods: Prior force-conditioned methods simulate direct forces but are not designed to plan the antecedent action required to achieve a goal force.Motion-conditioned models can follow specified trajectories but do not ensure causal adherence.
  • Visual Planning Accuracy: Table 2 evaluates visual planning accuracy by measuring whether the correct unconstrained object initiates the goal force across selected scenes.The analysis filters stochastic visual degradation and reports accuracy over valid trials.

5. Goal Force Enables Visual Planning

Goal Force is evaluated as a visual planner for physically valid, diverse, and mass-aware causal plans. It selects feasible initiators, samples multiple valid domino-chain strategies, and adjusts projectile motion when object masses change.

  • Evaluation scope: The evaluation tests physical accuracy, plan diversity, and use of privileged mass information.These properties are assessed as core dimensions of visual planning.
  • Physical accuracy: In 22 natural-blocker scenes, successful plans select an unconstrained object rather than a distractor to initiate the goal force.The benchmark places target objects in contextual environments while physically constraining distractors.
  • Physical accuracy: 98% of valid pool trials selected the white ball as initiator when a stick blocked the orange ball.Most failures involved spontaneous target motion rather than choosing a constrained initiator.
  • Generalization: Physically grounded planning also generalizes to other natural scenarios beyond the evaluated benchmark scenes.The paper reports this trend in scenarios shown in Figure 2.
  • Plan diversity: The model achieved a diversity score of 0.6577 across 26 seeds, exceeding the deterministic baseline of 0.3900.The metric rewards sampling across the five targetable dominos rather than collapsing onto one solution.
  • Privileged physics: Goal Force adjusts projectile speed when object masses change, satisfying desired force relationships in both in-distribution and out-of-distribution scenarios.The mass-aware tests vary projectile and target masses while holding the goal force magnitude fixed.

6. Conclusion

Goal Force reframes video control around desired effects rather than direct causes, enabling models to generate antecedent causal chains. The resulting planning capability generalizes from simple synthetic training to complex out-of-domain interactions.

  • Conclusion: Goal Force specifies a desired goal force instead of a direct force and generates a plausible antecedent causal chain to achieve it.The model reasons backward from the requested physical outcome.
  • Conclusion: Training on simple synthetic causal primitives supports planning in complex out-of-domain scenarios involving tool use and human-object interactions.The paper presents this as evidence that the model functions as an implicit neural physics planner.
  • Conclusion: The authors position the approach as a step toward interactive world models that simulate reactions and plan actions for desired physical outcomes.

7.1. Comparison to Prior Works: Direct Force Prompting Quantitative Comparison

The study compares Goal Force’s direct-force capability with prior force-prompting methods using human pairwise preferences across PhysGen and PhysDreamer benchmarks.

  • Experimental setup: Goal Force encodes goal-force prompts in one control channel and direct-force prompts in another for comparison with prior methods.The comparison includes PhysGen, Force Prompting, and PhysDreamer on their respective benchmark scenes.
  • Human comparison: A 2AFC human study with N = 10 compares Goal Force against prior methods on PhysGen and PhysDreamer benchmarks.PhysGen models rigid-body mechanics, whereas PhysDreamer models oscillations.

7.2. Synthetic Data Generation

Synthetic training data combines randomized domino chains, rolling-ball collisions, and non-rigid flower dynamics, with 3D physical information projected into image-space control signals. The resulting benchmarks also support comparisons of direct-force quality and visual-planning success across complex scenes.

  • Control-signal generation: Synthetic control signals project 3D forces and object properties onto 2D image coordinates using camera parameters.This projection enables physical interactions to be represented within video frames.
  • Domino data: The domino dataset contains 3k Blender videos with randomized counts, geometry, causality, visuals, and force magnitude.Each video records initial and adjacent contact dominos plus their framewise 2D coordinates.
  • Rolling-ball data: The rolling-ball dataset contains 6k Blender videos split into 4.5k collision and 1.5k non-collision examples.Collision scenes include straight-on and indirect impacts with randomized scene and force parameters.
  • Evaluation data: The evaluation includes direct-force human comparisons and visual-planning results across diverse tool-use scenarios.The reported tables compare visual quality and force adherence, while tool-use success requires achieving the specified goal force amid distractors.
  • Non-rigid data: The PhysDreamer carnation data models non-rigid flower motion after direct pokes with randomized camera, object, contact, angle, and magnitude conditions.

7.3. Ablation studies

Ablations show that both mass and direct-force channels contribute to reliable physics-conditioned generation, while action wording can remain underspecified.

  • Mass channel: Masking the mass channel and replacing it with text yields worse performance than the full model.The mass channel provides optional privileged physical information about object properties.
  • Direct force channel: Masking the direct-force channel causes failures on some complex out-of-domain causal chains, including human-object interactions.The direct-force channel supplies information needed for modeling intermediate causes in these chains.
  • Action specification: The model performs similarly when prompts describe an action causally or explicitly, but unspecified action sources may be selected from its learned prior.Those selected causes can sometimes be invisible forces.
Loading 2601.05848v2…