MultiEcho: An Experimental Science of Learned Worlds

1 Introduction

• Three labeled observations provide reference and intervention-range endpoints for fitting complete response changes and estimating intervention parameters from observed responses.The experiments use seven frozen configurations spanning encoders, latent predictors, and a video generator; interfaces record native outputs as fixed-size arrays.Source: §1 · pp. 1–2
• Event-window, viewpoint, object-geometry, and scene conditions delimit where response laws apply, while halved log-parameter changes probe locality and calibration convergence.Finite differences separate sensitivity errors from effects shared by opposite interventions, and remainder bounds state when refined calibration converges.Source: §1 · p. 2
• Median reverse R2 ranges from 0.208 at VideoSAUR to 0.986 at the Cosmos tokenizer, while LeWM encoder forward R2 is 0.975 in heat and 0.231 in mechanics plus optics.C-JEPA heat distance association is 0.861 for context-averaged responses but 0.186 within a typical context.Source: §1 · p. 2
• LeWM heat responses can be described substantially better than their physical effects are matched, while Cosmos heat effect errors plateau as true effects shrink.C-JEPA and LeWM effect errors remain about as large as true effects across sampled magnitudes.Source: §1 · p. 2
• In the viscoelastic exact-reset experiment, C-JEPA and AC meet registered visible-response criteria but fail the material-future criterion under fixed readout despite matched positions and velocities.Forward R2 is 0.125 for C-JEPA and 0.031 for AC; the contrast supports that geometric regularities alone are insufficient evidence of physical-law emergence.Source: §1 · p. 2
• MultiEcho estimates learned-world response laws, investigates their applicability and locality, and compares them with physical-world references across time, space, and interventions.The framework treats the complete recorded intervention response as the object for quantitative analyses of magnitude, geometry, predictability, and intervention identification.Source: §1 · pp. 1–2

Interrogating learned representations

• MultiEcho varies physical conditions and describes complete responses before choosing a physical readout, unlike work probing representations or extracting properties from predictors.Related approaches use visual prompts and Jacobians, layer probes, decoded motion, or energy and force readouts [Bear et al., 2023; Joseph et al., 2026; Interno et al., 2026].Source: §2 · pp. 2–3

Counterfactual physical prediction

• Prior counterfactual prediction work edits initial conditions or evaluates gravity-conditioned generation, whereas MultiEcho fits the response of an already frozen subject.Physion++ provides a precedent for exact-reset designs with matched visible starts [Bear et al., 2021; Tung et al., 2023].Source: §2 · p. 3

Physical behavior and generalization

• Unlike physical-behavior benchmarks, MultiEcho measures learnability of a subject’s intervention-response map rather than continuation fidelity or prescribed physical targets.The cited targets include event plausibility, physical parameters, prescribed equations, and generalization over physical conditions.Source: §2 · p. 3

Law discovery and system identification

• MultiEcho targets the frozen learned system’s own intervention-response law, complementing methods that infer symbolic laws or identify physical properties and geometry.Prior work includes symbolic regression [Cranmer et al., 2020], SINDy, and visual system identification [Kaneko, 2024; Zhao et al., 2025].Source: §2 · p. 3

Experimental Method

• A matched context and branch family varying one condition define the experiment; the complete recorded output difference is the response, retaining coupled changes beyond predefined summary observables.Each response has a fixed number of coordinates set by the interface, and the paired construction can accommodate more complex interventions when matched contexts and comparable observations are available.Source: §3 · p. 3
• The experimental unit fixes non-target conditions and varies intervention coordinate u; a shared adapter, frozen model, and observation map process each branch.For the video-input handle, the compiler simulates and renders each branch history.Source: §3 · p. 3, Figure 1 · p. 4
• Three labeled reference measurements support a forward fit of the intervention response and a reverse readout estimating the intervention parameter from an observed response.The reverse readout uses the same local references to calibrate response geometry to parameter values.Source: §3 · p. 4
• Distance association, direction consistency, and squared response magnitude respectively describe response organization across parameter changes and contexts, common direction, and effect size.Distance association is computed within contexts and separately for responses averaged across contexts.Source: §3 · p. 4
• The program spans 24 physical axes in nine scenes, with discovery, validation, and confirmation contexts and moved-camera rerenders of discovery contexts.It includes bounce event-window and visibility studies, magnitude sweeps against true futures, an RGB-trained inverse, and a viscoelastic exact reset; 268,506 runs produced 105,600 response records and 84,183 eligible query records.Source: §3 · pp. 4–5

4.1 Response Organization across Contexts

• Context averaging can obscure response separation: C-JEPA’s thermal-diffusivity association falls from 0.861 across mean responses to 0.186 within contexts, unlike LeWM’s 0.995 and 0.996.LeWM’s mean responses retain 80.0% and 79.1% of within-context separation; AC’s nearly uniform distances can still turn by about 119° between levels.Source: §4.1 · p. 5

4.2 From Response Structure to Predictive Laws

• LeWM’s encoder and predictor alone predict complete responses better than first-future RGB on most axes, while the Cosmos tokenizer alone recovers interventions better on most axes.For LeWM, forward R2 is 0.619 and 0.742; the tokenizer’s reverse R2 is 0.986.Source: §4.2 · p. 6
• AC recovers intervention parameters well but predicts complete responses poorly, while VideoSAUR is lowest on all four response statistics and below first-future RGB on every axis.AC’s reverse R2 is 0.898 and 0.885, while its forward R2 is 0.359 and 0.350; VideoSAUR’s reverse R2 is 0.208.Source: §4.2 · p. 6
• Text prompting with stronger guidance lowers Cosmos reverse R2 versus null prompting on paired queries, with intervals above zero across discovery, validation, confirmation, and moved-camera contexts.Forward R2 differences are smaller, 0.026–0.064, and their intervals include zero.Source: §4.2 · p. 6

4.3 The Domain of a Response Description

• Across eight validation and eight confirmation contexts per scene, LeWM remains the only model whose forward R2 exceeds first-future RGB on most axes, while the tokenizer alone does so for reverse R2.Source: §4.3 · p. 6

4.4 Temporal and Visual Applicability

• Applicability depends on temporal and visual conditions: restitution readability varies by event window and visibility channel, while camera changes preserve or improve forward R2 on at least 13 of 24 axes per interface.Under moved cameras, reverse R2 falls on 12 to 15 axes per interface.Source: §4.4 · p. 7

5 Response Laws across Physical Scales

• For spring stiffness, thermal diffusivity, and bounce gravity, the experiments query symmetric log-parameter offsets ±δ0/j at fixed observation times and image dimensions.Here δ0 = log(1.5) and j ∈ {2, 4, 8, 16, 32}.Source: §5 · p. 7
• At each magnitude, Figure 2 compares true and predicted effect sizes and their absolute error above, then reports per-query relative effect error eW below.The physical comparison uses simulator RGB for Cosmos and native target coordinates for C-JEPA and LeWM.Source: §5 · pp. 7–8, Figure 2 · p. 8
• At the finest heat scale, Cosmos null/text effect errors plateau at 0.00146/0.00359 RGB RMS as the true effect falls to 0.00025, while C-JEPA and LeWM relative errors stay near one across scenes and magnitudes.Cosmos’s finest-scale eW values are 5.843/14.605; C-JEPA’s and LeWM’s eW spans are 1.025–1.121 and 0.972–1.056, respectively.Source: §5 · p. 8
• In heat, whole-future errors stay near 0.416–0.418 for C-JEPA and 0.426 for LeWM across magnitudes, while subtracting reference-branch error leaves absolute effect errors that fall with intervention size.After this correction, C-JEPA’s heat error is below last-input-latent prediction, but its spring and bounce errors remain above that baseline.Source: §5 · p. 9
• At the finest LeWM heat scale, self-description error is 0.556 [0.484, 0.594] while physical effect error eW is 1.013 [0.994, 1.026] in identical post-projection coordinates.PL interpolation gives a similar self-description error of 0.555 [0.481, 0.600], and the gap widens at coarser magnitudes.Source: §5 · p. 9, Figure 3 · p. 9

6 Transferring an RGB-Trained Inverse

• Only Cosmos null’s transferred RGB skill interval lies above zero, at 0.102 [0.018, 0.262]; four learned interfaces have positive median paired skills and five have negative skills.High raw scores at tokenizer or AC do not establish a transferred RGB increment.Source: §6 · p. 9
• In the 24-context exact-reset experiment, C-JEPA and AC meet registered visible-compression criteria with forward R2 = 0.125/0.031, but material-future scores are −0.066/0.339 versus a 0.817 visual-history benchmark.Both material-future scores are inferior to the benchmark under the frozen ridge readout (Holm-adjusted p = 0.0004).Source: §6 · p. 9

7 Interpreting Local Response Error

• For paired ±h responses, the exact identity resolves squared prediction error into slope-error and even-response-error contributions, each scaled by h^2.The odd and even finite differences are defined from responses at ±h relative to the reference response.Source: §7 · p. 10
• At the finest heat scale, the even share of paired error is 0.730/0.741 for Cosmos null/text and 0.164/0.053 for C-JEPA/LeWM.The identity separates error into slope-related odd mismatch and an even component; at that scale, true heat effects are mostly odd, while Cosmos mostly adds changes shared by opposite interventions and C-JEPA and LeWM mostly miss the odd part.Source: §7 · p. 10
• If the response has derivative J at the reference with a remainder bounded by |s|ω(|s|), where ω(t) tends to zero, a slope re-estimated at ±2h predicts responses at ±h with o(h) error.The condition also entails Dh → J and Kh → 0.Source: §7 · p. 10

8 Discussion and Conclusion

• MultiEcho treats the complete recorded response of a frozen world model to counterfactual interventions as the experimental object, with findings valid under declared subject, intervention, context, observation-time, and readout conditions.The paper describes the resulting conditional facts as part of the science of a learned world.Source: §8 · p. 10
• The framework links simulated and learned worlds through controlled intervention histories, model-specific response interfaces, and simulated futures used as references.The paper also identifies training-data augmentation and selected-neuron interventions as possible extensions (Geiger et al., 2021; Meng et al., 2022).Source: §8 · p. 10
• The study does not claim exhaustive subject characterization, optimal estimators or fits, model or architecture rankings, or evaluation of the strongest current systems.Broader scene coverage and joint interventions on multiple parameters are identified as extensions.Source: §8 · p. 10

• AI assistants supported methodology refinement, mathematical arguments, and proof writing.Source: AI Use Statement · p. 11
• They also helped implement synthetic experiments and analysis code, check sources, interpret results, and create scientific figures.Source: AI Use Statement · p. 11
• AI-assisted work was checked against saved artifacts, numerical identities, and executable code and figure validation.The authors are responsible for the final text, claims, and artifacts.Source: AI Use Statement · p. 11

• Methods and appendices specify the intervention families, model access, estimators, and evaluation stages.Source: Reproducibility Statement · p. 11
• Mathematical results are accompanied by assumptions and proofs, and figures are generated from completed measurements.Source: Reproducibility Statement · p. 11
• The reference library provides the experimental interfaces described in Appendix A.3.Source: Reproducibility Statement · p. 11

A.1 Scenes and Sample Units

• 268,506 saved runs span the nine main campaigns and additional repeatability, visibility, and viscoelastic studies; 24 axes yield 9,600 axis-level-context cases.The atlas records 105,600 response records, with discovery, validation, and confirmation contexts.Source: §A.1 · p. 16

A.2 Adapters and Saved Interfaces

• All subjects receive the same 16-frame source stimulus, while model adapters record native observations and use model-specific inference histories.The section also notes differing input resolutions and that Cosmos null/text configurations differ in both prompt and guidance.Source: §A.2 · p. 17

A.3 Reusable Experimental Operations

• The reusable WorldAdapter and PairRunner construct paired interventions, record observations, and evaluate response fits against independent queries and controls.Additional interfaces support extensions beyond the reported video experiments.Source: §A.3 · pp. 18–19

A.4 Related Experimental Questions

• Related physical-behavior and world-model evaluations target physical plausibility or declared world models, whereas MultiEcho measures a subject’s own responses and separately compares them with physical references.Source: §A.4 · pp. 19–20
• MultiEcho describes finite intervention contrasts rather than temporal differential equations, and distinguishes identifying a physical system from identifying a learned subject’s response.Source: §A.4 · p. 20

B.1 A Common Neural Tool across Observation Dimensions

• The forward neural tool adds a learned residual to linear interpolation between bracketing references, with a gate preserving the three reference observations.Fully flat calibration yields the unchanged forward output, an explicit information limitation of the tool.Source: §B.1 · p. 21

B.2 Training, Selection and Scores

• An inner context holdout selects the residual multiplier; test contexts do not select the multiplier, architecture, or training duration, and scores retain negative results.Source: §B.2 · p. 22

B.3 Subject-Resolved Atlas

• The scene-resolved atlas reports median forward, reverse, and fixed future-RGB reverse R2 by scene and observation.Main comparisons use a common neural procedure and equally weight the same 24 axes; no held-out best-family selection occurs.Source: Table 6 · p. 22, §B.3 · p. 22

B.4 Frozen Fits across Experimental Stages

• Frozen discovery fits are applied unchanged to validation and confirmation contexts; neither external-stage outcomes nor target-side regression select weights or multipliers (Table 7).External-stage queries use each context’s own baseline and parameter-range endpoints; intervals are conditional on saved fits and reused trajectories limit independent dynamical inference.Source: §B.4 · pp. 24–25, Table 7 · p. 25

B.5 Frozen Fits under a Moved Camera

• Moved-camera evaluations reuse frozen discovery fits and each moved view’s own three references, while simulator RGB is omitted because its moved-view first future was never rendered (Table 8).The camera changes vary by system, and heat changes the scene, lighting, supports, textures, and camera together.Source: §B.5 · p. 26, Table 8 · p. 26

B.6 Equal-Calibration Controls

• The PL controls use the same three measurements as neural predictors, and their inverse uses the same query and polyline projection but omits the residual (Table 10).Paired gains are computed per axis before cross-axis medians; marginal-median differences are not paired contrasts, and conditional intervals do not account for tool selection or reused trajectories.Source: §B.6 · pp. 26–27, Table 10 · p. 27, Table 11 · p. 27

B.8 Context-Resolved Distances and Local Geometry

• C-JEPA heat responses show mean-response distance association 0.861 but median within-context association 0.186; LeWM encoder/predictor medians are 0.995/0.996 (Table 12).For C-JEPA, mean squared-distance retention is 0.082; LeWM retains 0.800/0.791.Source: §B.8 · p. 29, Table 12 · p. 29
• Heat AC interfaces have similar context-pair distance discrepancies of 0.022/0.060 yet median local turns of 119.2°/119.1°; nearly equidistant triples can produce turns near 120°.The pattern therefore does not establish a physical discontinuity.Source: §B.8 · p. 29

C.2 Fitting Diagnostics and Repeatability

• All 24 same-input, same-seed repeats reproduce the first RGB frame exactly, while cross-seed response-direction agreement varies by scene and configuration, with median pairwise cosine −0.006 to 0.542.These preliminary observations do not establish seed-independent response laws; the main sweeps use fixed realizations.Source: §C.2 · p. 34

C.3 What the Scale Scores Compare

• The fine-scale supplementary test evaluates a fixed law after coarse local calibration, not a fine-scale refit or performance at all smaller steps.The score compares each query’s complete-response error with response magnitude; narrow symmetric ranges can make small reverse parameter errors large, and the test does not treat dependent points as independent replicates.Source: §C.3 · p. 35

C.4 Direct True-Future Comparisons

• At the finest heat scale, LeWM’s median future RMS is 0.4261 versus persistence RMS 0.0023, with median pointwise ratio 181.48 (Table 16).The ratios can reflect both poor prediction and small physical temporal change; whole-future error and intervention-effect error use different denominators.Source: Table 16 · p. 36, §C.4 · p. 37

C.5 Effect and Precision Diagnostics

• Subtracting the reference prediction error from each prediction yields exactly the intervention-effect error, without fitting on query truths, but requires physical reference truth and is not a deployable forecast.No matched training-domain positive-control run has been performed.Source: §C.5 · p. 39
• At the finest heat scale, after allowing a full 8-bit code step (q = 1/255) per contrast coordinate, the conservative residual was 0.00017/0.00175 RGB RMS for Cosmos null/text.Source: §C.5 · p. 40

C.6 Self-Description in the Physical Target Space

• The shared target-space comparison refits both subjects using references 0 and ±log(1.5)/2, then tests four finer magnitudes held out by context (Table 20).Self-description ratios normalize by each subject’s response norm, whereas physical ratios normalize by the encoded true-effect norm.Source: §C.6 · pp. 40–41, Table 20 · p. 41

D Mathematical Arguments

• Exact finite-scale identities apply to recorded observations, while derivative recovery requires a first-order remainder for the intervention-to-observation map; stronger rates require stronger smoothness.Native quantization and event transitions belong to that map and are distinct from additional observation or export error.Source: §C.5 · p. 37, D Mathematical Arguments · pp. 41–42

D.1 Absolute Error, Derivatives and Refinement

• Paired relative error separates odd- and even-response discrepancies, while the ratio alone cannot identify their source.The same ratio can arise from a zero-response predictor or from a predictor that doubles both signed effects, so amplitude and direction comparisons are needed.Source: §D.1 · p. 42
• Differentiability requires the central slope to converge to the derivative and the even component to vanish; first-order remainder bounds specify when updated calibration converges.Finite-scale identities also show that even-component alignment alone does not establish smooth curvature, and sampled convergence can miss between-scale variation.Source: §D.1 · p. 43

D.2 Finite Probes and Recovered Laws

• Finite directional probes recover a derivative under remainder and measurement-error bounds, but rank deficiency prevents full-Jacobian recovery without extra structure.The reported runs supply no uniform measurement-error or remainder bound.Source: §D.2 · pp. 43–44

D.3 Energy and Predictor Classes

• Whole-response R2 is a variation-weighted average of component scores, while projection alone gives neither a physical interpretation nor a general target-only error floor.That floor applies only when the predictor lies in the projected calibration span.Source: §D.3 · p. 44

D.4 Continuity, Derivatives, and Joint Changes

• Finite samples constrain errors between intervention values only with regularity bounds, and value accuracy alone does not guarantee derivative accuracy or constrain untested interactions.Single-axis and temporal accuracy constrain their respective slices; joint interventions provide observations of interactions.Source: §D.4 · p. 45

E.1 Temporal Windows and Event Opportunities

• Bounce-parameter readability depends on observation window and model interface; C-JEPA, both AC interfaces, and Cosmos tokenizer meet the registered pattern across discovery and independent stages.LeWM’s encoder and predictor have different gravity-readability windows, showing that applicability cannot be inferred from scene identity or one aggregate score.Source: §E.1 · p. 46

E.3 Material Futures after Exact Reset

• After an exact reset matching positions and velocities, C-JEPA and AC meet visible-compression criteria but fall below the registered material-future benchmark under the fixed readout.The 24-context extension establishes this contrast at both interfaces; fixed-readout failure does not rule out recovery by a nonlinear readout.Source: §E.3 · p. 47

F Single-Frame Reference Comparisons

• Cosmos null improves on its target PL inverse on 19/24 axes, but its paired relative-skill advantage over text is inconclusive; learned-interface transfer scores vary widely.Table 22 defines paired skill as the effect of adding the frozen RGB correction; the reported null-versus-text relative-skill interval includes zero.Source: Table 22 · p. 49, F Single-Frame Reference Comparisons · p. 49