Source-linked AI summary
IMPLY: Physically Anchored Consistency for World-Model Rollouts
Aman Mehta, Riya Baviskar
TL;DR
Existing consistency checks test whether a model’s futures agree with one another, not whether they agree about the object in the world. IMPLY infers rollout physics by inverting a simulator and anchors its score to two observed calibration pushes. Anchoring exposes object-blind rollouts in controlled tests and distinguishes correct from swapped calibration on an adapted V-JEPA 2-AC model.
Problem
Consistency checks for world-action models test agreement among outputs, but do not establish whether those outputs agree about the physical object being acted on.
Source: §1 · p. 1Method
IMPLY inverts a simulator to infer the physics implied by rollouts and scores their joint residual with two observed calibration pushes included as an anchor.
Source: §1 · p. 1Results
Takeaways & Limitations
A model can be self-consistent while ignoring the object, so consistency scoring must be anchored to evidence about that object.
Source: §1 · pp. 1–2Learned-model evidence is from one architecture, and use of calibration was learned during adaptation rather than present in the pretrained checkpoint.
Source: §6 · pp. 5–6
Abstract
from arXiv · showhide
A world model asked what happens if an object is pushed at several speeds produces several futures. If the model has the object in mind, those futures agree about it: each implies the same mass and friction. The consistency checks now used to vet world-action models ask whether a model's futures agree with each other, and none of them knows any physics. We show that this is not enough, and what to do instead. IMPLY reads the physics each rollout implies by inverting a simulator and scores a set of rollouts by how well one object explains all of them, anchored to two calibration pushes the model has observed. In a controlled setting, self-consistency gives a perfect score to a model that ignores the object and always predicts a typical push; anchoring exposes it (AUROC 0.70 versus 1.00). On a real model, V-JEPA 2-AC adapted to the scene, the same thing happens. Given its own calibration pushes the model tracks the object (per-object correlation with the truth 0.91); given another object's, it does not (0.05). Self-consistency cannot tell these apart, preferring the right evidence on 52% of objects, chance level, while anchored disagreement prefers it on 73% and correlates 0.92-0.99 with the rollouts' error. Used to choose among candidate rollout sets, it comes within 0.003 of an oracle that sees the truth. A model that has internalised the wrong object is exactly as self-consistent as one that has internalised the right one; consistency has to be anchored to evidence.
1 Introduction
Existing consistency checks ask whether a model’s outputs agree with one another, not whether they agree about the object in the world. IMPLY anchors physical consistency to observed pushes, exposing object-blind rollouts that self-consistency misses.
- A model can generate mutually consistent futures while imagining the same plausible object regardless of what is actually present.Source: §1 · p. 1
- IMPLY inverts a simulator to find the single mass and friction that best explain all rollouts, then scores their residual using two observed calibration pushes.The score uses no ground truth about the object.Source: §1 · pp. 1–2
- Anchoring raises controlled object-blind detection from AUROC 0.70 to 1.00; on adapted V-JEPA 2-AC, object tracking is r = 0.91 with own calibration and r = 0.05 with another’s.Across candidate rollout sets, the anchored score comes within 0.003 of an oracle that sees the truth.Source: §1 · p. 2
2 Related Work
Prior world-action consistency signals compare predicted actions, frames, views, or sampled futures, but do not refer to the physical object being acted on. Related work also connects object-blind rollouts to hallucination and calibration-based physics inference.
- Action-state consistency, SC3-Eval, geometry-guided self-consistency, MG-Select, and Pre-VLA assess rollout agreement or train verification, but none refers to the acted-on object.Action-state consistency passes the no-motion case called background collapse; IMPLY requires that the model observed the object interact.Source: §2 · p. 2
- The object-blind stand-in is action-marginalised hallucination in its purest form, a failure self-consistency cannot see [Hansen and Wang, 2026].Source: §2 · p. 2
- Classical work infers mass and friction from motion [Wu et al., 2015, 2016], while CALIPER [Mehta and Baviskar, 2026] uses two calibration pushes to infer latent physics.IMPLY instead uses that calibration evidence to anchor a consistency test for generative rollouts.Source: §2 · p. 2
3 Implied Physics and Anchored Disagreement
The method infers an object’s mass and sliding friction from rollout displacements at multiple commanded speeds. It anchors the joint fit to observed calibration slides, making the score test agreement with the particular object rather than merely among futures.
- The setting maps commanded push speed, object mass, and friction to log displacement, and gives the model two observed calibration pushes before K new-speed rollouts.Source: §3 · p. 2
- A single speed and displacement leave a one-dimensional family of mass–friction pairs, while several speeds can pin down the pair.Source: §3 · p. 2
- The joint-fit RMS residual measures disagreement in log displacement, without calibration for self-disagreement and with the two observed slides for anchored disagreement.The fit includes the calibration observations so the rollouts must describe the object that produced them.Source: §3 · p. 3, Table 1 · p. 3
- For latent models, displacement is read from a ridge fit on real clips; pixel models use frame differences, which read displacement within a few millimetres in a fixed-camera scene.Validation treats readout error as noise and tests how much the score tolerates.Source: §3 · p. 3
4 Controlled Validation
Controlled tests show that self-consistency misses a model that predicts typical pushes regardless of the object, while anchoring to observed pushes exposes the mismatch and supports candidate selection.
4 Controlled Validation
- The controlled test compares four stand-in models, each producing K = 5 rollouts for each of 200 CALIPER objects.The models represent true physics, per-rollout physics drift, object-blind typical displacement, and no movement.Source: §4 · p. 3
- 0.006 self-disagreement makes the object-blind model indistinguishable from the oracle at 0.005, despite 0.368 log-displacement error against truth.Self-disagreement separates jitter and collapse from the oracle, but not this model that repeatedly predicts a typical object.Source: §4 · p. 3
- 0.165 anchored disagreement separates the object-blind model from the oracle with AUROC 1.00, and correlates 0.96 with jitter error and 1.00 with object-blind error.The two calibration observations are included in the fit.Source: §4 · p. 3
- 98% of the time, anchored disagreement selects the oracle candidate, with a 1.5 mm median miss versus 7.1 mm for the first sample and 8.6 mm for consensus.The test chooses among four noisy candidate rollout sets of mixed quality.Source: Table 2 · p. 4, §4 · p. 4
- At readout-noise sd 0.2, anchored disagreement still separates the object-blind model from the oracle with AUROC 0.78.The noise is applied to every rollout’s displacement before scoring.Source: §4 · p. 4, Table 2 · p. 4
5 A Learned World Model: V-JEPA 2-AC
On adapted V-JEPA 2-AC, calibration evidence determines whether rollouts track the pushed object, while anchored disagreement distinguishes correct from incorrect evidence and tracks rollout error. The study also finds that this calibration use was learned during adaptation, not present in the pretrained checkpoint.
5 A Learned World Model: V-JEPA 2-AC
- The adapted V-JEPA 2-AC used a frozen ViT-g encoder and a fine-tuned 300M predictor trained on 1,500 episodes; a displacement readout reached held-out R2 = 0.984.The two variants differed in whether they also received three frames from each of the object's two calibration pushes.Source: §5 · p. 4
- Each object's five deterministic futures came from five speeds, with simulator runs at those speeds as truth.Source: §5 · p. 5
- Without calibration evidence, per-object predictions correlated 0.03 with truth; anchored disagreement correlated 0.99 with error, while self-disagreement was uncorrelated (r = 0.10).The model's low self-disagreement was 0.158.Source: §5 · p. 5
- Anchored disagreement preferred correct calibration on 73% of objects and tracked error at r = 0.92 and 0.99, while self-disagreement preferred it on 52%.With another object's calibration, object tracking fell to r = 0.05, compared with r = 0.91 with the object's own calibration.Source: §5 · p. 5
- Calibration use was learned during V-JEPA adaptation, not present in its pretrained checkpoint; Cosmos-Predict2 adaptation lacked action-dependent dynamics.Source: §5 · p. 5
6 Limitations
The inversion assumes a known simulator and a single contact type, and the score inherits forward-model error. Anchoring requires prior observation of the pushed object; the learned-model evidence is limited in scope, and the pixel-space attempt did not yield action-dependent rollouts.
6 Limitations
- Inversion assumes a known simulator and a single contact type; errors in a calibrated simulator or learned forward model carry into the score.Source: §6 · p. 5
- Anchoring requires that the model has observed the object being pushed.The authors note this is the same requirement as CALIPER and can be met by probing.Source: §6 · p. 5
- Learned-model evidence comes from one architecture, and calibration use was learned during adaptation rather than present in the pretrained checkpoint.Source: §6 · pp. 5–6
- The pixel-space Cosmos-Predict2 attempt did not produce action-dependent rollouts; deterministic candidate sets differ by context rather than sampling.The stand-ins are caricatures of failure modes, not a substitute for the learned-model study.Source: §6 · p. 6