InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling

• Existing multimodal systems rely on superficial correlations and remain unreliable under context shifts, while counterfactual reasoning requires tracing intervention effects in a specific observed situation.Source: §1 · p. 1
• Prior causality-guided approaches do not jointly provide an explicit spatial state where local intervention effects propagate across regions over multiple reasoning steps.Source: §1 · p. 1
• InfluenceField inserts a differentiable influence field between the visual encoder and language decoder, jointly trained with language, invariance, counterfactual rollout, and structural objectives.Source: §1 · p. 2
• Under sufficient conditions, target-aligned interventions identify the nonlinear finite-basis transition graph up to within-location reparameterization; the linear specialization gives finite-coverage and finite-loss guarantees.Source: §1 · p. 2
• InfluenceField outperforms strong baselines across causal video QA, multimodal causal discovery, OOD generalization, and counterfactual consistency, with largest gains on planning and hypothetical questions.Source: §1 · p. 2

• Counterfactual augmentation, confounder adjustment, and causal discovery encode causal information differently, but do not jointly integrate continuous spatial states with multi-step language-conditioned reasoning.Source: §2 · p. 2
• CausalVQA focuses on physically grounded videos where causal effects unfold over time, whereas CELLO, CausalVLBench, MuCR, and CausalSpatial primarily evaluate image-based causal reasoning.Source: §2 · p. 2
• Prior causal representation work uses interventions, temporal or auxiliary variables, and mechanism constraints; this paper tailors identification to deterministic finite-basis field transitions.Source: §2 · p. 2
• World models and continuous scene representations provide prediction or spatial coordinates, while InfluenceField adds directed influence and localized interventions in the visual-language pathway.Source: §2 · p. 2

3.1 Overview

• Four jointly trained components—field construction, directional influence, propagation, and field intervention—support factor localization, directed influence inference, and intervention-aware counterfactual prediction.Source: §3.1 · p. 2

3.2 Field Construction

• Gaussian interpolation lifts patch features into a continuous, differentiable, smooth field queryable at arbitrary positions; implementation evaluates it on a regular grid matching the patch grid.Source: §3.2 · p. 3

3.3 Directional Influence and Propagation

• A state-dependent directional influence matrix transports field information across locations, while an attention refinement branch preserves contextual semantics beyond the sparse graph.Source: §3.3 · p. 3

3.4 Field Intervention and Counterfactual Rollout

• Localized field-space edits use a Gaussian mask, and factual and counterfactual trajectories share the same transition operator and readout.Source: §3.4 · p. 4

3.5 Cross-Modal Fusion and Training

• The propagated field attends to language tokens for answer generation, and training combines language modeling with invariance, counterfactual rollout, and structural regularization.Invariance compares nuisance-varying environments over unaffected foreground regions; simulator-produced post-intervention targets prevent the degenerate solution of ignoring interventions.Source: §3.5 · p. 4

• Under finite-basis, product-domain, regularity, and population conditions including one-step separation, target-aligned interventions identify the nonlinear transition graph exactly up to within-location reparameterizations.The result concerns the complete transition map, not raw attention weights.Source: §4 · pp. 4–5
• One anchor intervention value per node suffices only when paired with every pre-intervention state in the product domain; finite training pairs alone do not establish global nonlinear identification.Source: §4 · p. 5
• A uniform inverse-Lipschitz condition converts finite factual and counterfactual residuals into a bound on off-target change, but this does not by itself recover nonlinear Jacobian support.Source: §4 · p. 5
• In the linear specialization, factual fitting alone leaves a similarity-orbit ambiguity; interventions at every node identify transition support and edge directions, with weights determined up to node-wise scaling.Source: §4 · p. 5
• With partial intervention coverage, covered-node edits remain exact, but generic full-graph identification requires full coverage and the covered subgraph is determined only under a stated edge condition.Source: §4 · p. 6
• For the sampled Gaussian field, a coefficient intervention changes the field along its interpolation-kernel column; paired channel offsets at one site identify a shared channel map that transfers across sites.The practical convex-blend edit has smoothness and finite-grid boundary residuals; kernel matching, not Gaussianity, is the principle.Source: §4 · p. 6
• The theorem identifies the complete transition dependency graph, not raw influence-matrix support; the nonlinear result is a finite-dimensional population statement, not identification of an unrestricted operator or network parameters.Source: §4 · p. 6

5.1 Setup

• Evaluation uses CLEVRER and Causal3DIdent/CITRIS for pretraining, a Qwen3-VL-8B QA backbone, and benchmarks spanning video QA, distribution shifts, coherence, and discovery.CausalVQA has 793 questions across five categories; downstream results generally summarize three runs, while synthetic identifiability results summarize five.Source: §5.1 · pp. 6–7

5.2 Q1: Causal Video Reasoning

• InfluenceField reaches 67.4 overall on CausalVQA, 13.1 points above its backbone, and outperforms automated baselines, with largest gains on Planning and Hypothetical.On zero-shot NExT-QA, gains are Causal +4.4, Temporal +1.4, and Descriptive +0.4.Source: Table 1 · p. 7, §5.2 · p. 7, Table 2 · p. 7

5.3 Q2: Capacity-Matched and Structure Controls

• Capacity-matched controls attribute +6.0 of the in-domain gain to causal objectives, which also account for robustness and coherence improvements beyond non-causal C2.C2 recovers only about a fifth of the Consistency and Flip improvements, and its OOD drop matches the backbone’s.Source: §5.3 · p. 7, Table 3 · p. 8
• Transposing learned directions selectively lowers Planning and Counterfactual accuracy, shuffled intervention–outcome pairs erase counterfactual-supervision gains, and theory-aligned restrictions cost 1.5 overall points.Transposition lowers Planning by 11.1 and Counterfactual by 8.3; accuracy declines across full, symmetrized, transposed, and random influence structures.Source: §5.3 · p. 7

5.4 Q3: Empirical Identifiability

• Synthetic recovery tracks intervention coverage, while the observed √ε trend lies outside the certified regime and therefore supports the functional form without verifying Proposition 2.With counterfactual supervision, cross-seed correlation of learned influence structures is 0.91 versus 0.43 without it; learned influence agrees with operational influence at Spearman ρ=0.74 and 82% direction agreement. Recovery also degrades as kernel conditioning worsens; σ=0.5 improves synthetic recovery marginally but reduces downstream QA by 0.9 points, so σ=1.0 is used.Source: §5.4 · p. 8, Figure 3 · p. 8

5.5 Q4: Robustness and Consistency

• InfluenceField has the highest mean OOD accuracy (61.5) and smallest average degradation (5.9 points), while Consistency rises 66.8→78.6 and Invalid Transitions fall 12.9→8.1.Intervention Shift has the largest degradation at 7.8 points; occluded or off-screen objects, long dependencies, and ambiguous targets account for 61% of remaining errors.Source: §5.5 · pp. 8–9, Table 4 · p. 9, Table 5 · p. 9

5.6 Q5: Multimodal Causal Discovery

• On Lung Cancer and MAG, InfluenceField improves reported node and edge metrics over MLLM-CD, but these small-scale benchmarks use factors that are not spatially indexed.On Lung Cancer, NF/AF are 0.94/0.73 versus 0.90/0.68 and ESHD is 4.30 versus 5.50; on MAG, NF/AF are 0.93/0.64 versus 0.89/0.57 and ESHD is 11.10 versus 13.42.Source: §5.6 · p. 9

5.7 Ablation and Efficiency

• DIF and dynamic propagation raise accuracy from 54.3 to 61.4, invariance and counterfactual supervision add +2.9 and +3.1, and latency overhead over the backbone is 7%.Planning improves from 66.4 at T=0 to 73.2 at T=3; accuracy dips slightly at T=4.Source: §5.7 · p. 9, Table 6 · p. 9

6 Conclusion

• InfluenceField propagates directed influence and supports field-level intervention; the nonlinear population result identifies the complete transition graph under stated conditions, while representations remain identifiable only up to within-location diffeomorphisms.Linear results provide partial-coverage and finite-loss refinements, and the experiments attribute improvements in causal reasoning, robustness, and factual–counterfactual coherence to causal objectives.Source: §6 · p. 9

Technical Appendix

• The technical appendix provides proofs for the identifiability results, including the nonlinear population result, linear full and partial coverage, finite-loss bounds, and field-structured mixing.It also documents protocols, implementation details, additional results, and further discussion.Source: Technical Appendix · p. 10

A Notation and Formal Setup

• The linear formal setup assumes linear latent dynamics and invertible linear encoding, spanning initial states, known-site nontrivial interventions at every node, nonsingular dynamics, and zero population loss for exact-recovery results.The finite-loss theorem’s ε is the maximum of one-step factual and per-site counterfactual errors, not the scalar training loss; no empirical-to-population generalization argument is supplied.Source: A Notation and Formal Setup · p. 10

B Nonlinear Transition Identifiability

• Target-aligned interventions and a one-step separation condition restrict nonlinear representations to within-location diffeomorphisms and preserve the transition graph exactly.The result assumes full support on product domains; rollout predictions also hold for supervised edits along trajectories remaining in the domain.Source: B Nonlinear Transition Identifiability · pp. 10–11

C Observational Non-Identifiability

• Factual trajectories alone do not generally identify the block transition graph, because observationally equivalent coordinate changes can introduce cross-edges.A shear transformation of diagonal dynamics provides an example of a spurious directed edge.Source: B Nonlinear Transition Identifiability · p. 10, C Observational Non-Identifiability · p. 11

B Nonlinear Transition Identifiability

• Calibrated multi-step rollouts match edited learned states to the corresponding latent interventions for every supervised anchor edit whose trajectory stays in the domain.For a new target, the result applies when its learned replacement is the within-location map ψ_i(a).Source: B Nonlinear Transition Identifiability · p. 11
• Finite nonlinear supervision does not imply global identification without a restricted function class, and small function-value error alone does not control derivatives.Recovering Jacobian support from approximate nonlinear fits would require derivative-level control and a beta-min condition.Source: B Nonlinear Transition Identifiability · p. 11

D Exact Identifiability from Full Coverage

• Under the linear specialization’s full-coverage assumptions, one-step counterfactual loss of zero implies zero k-step loss, while shared spatial indexing removes permutation ambiguity.The full-coverage argument forces a diagonal coordinate transformation; the rollout result holds for all k ≥ 1.Source: D Exact Identifiability from Full… · p. 11

E Partial Coverage

• Under partial coverage, zero-loss solutions constrain covered-coordinate columns while leaving the transformation on uncovered coordinates free.The solution set has diagonal invertible scaling on covered coordinates and an otherwise free invertible uncovered block.Source: E Partial Coverage · p. 12
• Rollouts are exact for interventions on covered nodes, but the result provides no such guarantee for interventions touching uncovered nodes.This prediction guarantee holds unconditionally for solutions in the characterized set.Source: E Partial Coverage · p. 12
• For generic transitions, full support identification requires full coverage; under the stated structural condition, covered-node pairs and structural zeros from covered to uncovered nodes are identified.Edges from uncovered nodes into covered nodes and pairs within the uncovered set remain unidentified.Source: E Partial Coverage · p. 12
• With coverage fraction c, the identified fraction of ordered pairs is c under the stated descendant-closed setting, but this is not edge F1.The covered-to-uncovered zeros rely on assumed structure; the conservative comparison is c^2 rather than c.Source: E Partial Coverage · p. 12

F Approximate Identifiability under Finite Loss

• The corollary gives exact support recovery by thresholding D−1ÂD at β/2 under its additional condition.The explicit finite-loss proposition assumes factual and one-step counterfactual losses at most ε, with ε no greater than ε0.Source: F Approximate Identifiability under… · pp. 12–13
• The certified finite-loss guarantee is highly restrictive in the stated simulator: even at κ = 10, ε0 is approximately 10^-14.For evaluated checkpoints, ε ranges from 3.7 × 10^-3 to 1.5 × 10^-1.Source: F Approximate Identifiability under… · p. 13
• The finite-loss experiment tests whether degradation follows a √ε functional form, not whether the certified guarantee holds.No evaluated checkpoint reaches the guarantee; the text notes that closing the gap would require a sharper ε0 or a simulator with κ(H) near 1.Source: F Approximate Identifiability under… · p. 13

G Field-Structured Mixing

• For σ > 0 and distinct grid points, the Gaussian interpolation matrix Wσ is invertible.The paper uses this property in its field-structured mixing analysis.Source: G Field-Structured Mixing · p. 13
• Under Assumption 1, known invertible Wσ and a spanning set imply ÂWσ = WσA*; the paper cautions this factorization is not guaranteed by the deployed nonlinear encoder.The shared-channel calibration and intervention-geometry results depend on this assumption.Source: G Field-Structured Mixing · p. 13
• Hard-row replacement has a positive loss floor, but its stated lower bound is about six orders of magnitude below the measured floor.The paper recommends reporting the exact minimum; the bound proves positivity but is not quantitatively explanatory.Source: G Field-Structured Mixing · pp. 13–14
• The convex blend’s intervention residual vanishes when F is constant on the mask support, while the centered additive variant has no residual.On the finite grid, boundary normalization makes even the centered edit only approximately match the kernel-column profile near boundaries.Source: G Field-Structured Mixing · p. 14
• For the periodic-grid Gaussian kernel, conditioning grows at least as e^Ω(σ^2/h^2); the result is a lower-growth statement, not a Θ bound.The numerical comparison uses the periodic idealization, not a finite-grid implementation proxy.Source: G Field-Structured Mixing · p. 14

H.1 Hyperparameters and Infrastructure

• The intervention-mask bandwidth is tied to interpolation bandwidth σ rather than tuned independently.The reported architecture uses a 14×14 field grid, C = 256, four heads, T = 3, and σ = 1.0.Source: §H.1 · p. 14, Table 7 · p. 15

H.2 Reporting Conventions

• CausalVQA category scores are per-category accuracies, while Reasoning and All pool 505 non-descriptive and 793 total questions, respectively.OOD is the unweighted mean across four shifts, and Drop is per-run in-domain All accuracy minus OOD.Source: §H.2 · p. 15

H.3 OOD Split Construction and Full Results

• The four OOD shifts vary scene and viewpoint, object–action combinations, question templates, or intervention-type–target combinations.Their sample sizes are 214, 189, 793, and 176, respectively.Source: §H.3 · p. 15, Table 8 · p. 16

H.4 Synthetic Identifiability Protocols

• Across 100 synthetic checkpoints, aligned recovery error is approximately linear in √ε (R2 = 0.97), with display-bin errors rising from 0.09 to 0.50.The four √ε bin means are 0.06, 0.12, 0.21, and 0.38; coverage and matched full-coverage comparisons are reported in Table 10.Source: §H.4 · p. 15, Table 10 · p. 16

H.5 Sensitivity Analyses

• Table 12’s sensitivity analyses indicate that σ = 1 and T = 3 balance accuracy, conditioning, and computational cost; measured latency rises from 0.98 to 1.05 s, a 7% overhead.The latency comparison is against Qwen3-VL-8B under the stated profiling configuration.Source: §H.5 · p. 16, Table 12 · p. 17

H.6 Multimodal Causal Discovery

• On MAG, InfluenceField’s backbone-averaged NF/AF/ESHD values are 0.93/0.64/11.10, versus 0.89/0.57/13.42 for MLLM-CD.The comparison averages four backbone MLLMs; the MLLM-CD MAG entries are transcribed from its published table.Source: §H.6 · pp. 16–17

H.7 Failure Analysis

• Among 164 annotated residual errors, 27.4% involved ambiguous intervention targets, 18.9% horizons exceeding T, 14.6% occlusion or off-screen interactions, and 39.0% other or mixed causes.The first three regimes account for 61.0% in aggregate.Source: §H.7 · p. 17

H.8 Numerical Consistency

• OOD and Drop are computed per run from unrounded values; finite-loss bins use 100 checkpoints, and failure percentages use integer annotation counts.Discovery aggregation follows each table caption, and displayed values are rounded only for presentation.Source: §H.8 · p. 17

I.1 Lightweight Projection Heads

• The discovery head mean-pools directed-influence features for candidate variable pairs and classifies ordered pairs as no edge, forward, or backward.It uses a two-layer MLP with dimensions 512 → 256 → 3 and GELU.Source: §I.1 · p. 17

I.2 Backbone Freezing Strategy

• The visual encoder is frozen, while downstream fine-tuning updates the final two decoder blocks, output projection, InfluenceField, and task head.All other backbone modules remain frozen.Source: §I.2 · p. 17

I.3 Three-Stage Progressive Training

• Training first grounds the field with language modeling and spatial losses, adds cross-environment invariance in stage 2, and adds counterfactual refinement before downstream task fine-tuning.Stage 1 lasts six epochs, stage 2 four, and stage 3 five; all field modules remain differentiable end-to-end.Source: §I.3 · pp. 17–18

I.4 Intervention Operations

• Object removal, attribute modification, and custom interventions share a kernel mask whose bandwidth is tied to interpolation bandwidth σ.Removal uses a learnable null state, attribute changes use a lightweight transformation, and custom interventions accept a caller-supplied target value.Source: §I.4 · p. 18

I.5 Training and Inference Procedures

• The inference procedure constructs a factual rollout, applies an intervention, and reads out the counterfactual using the same transition operator.The resulting output can optionally be contrasted with the factual answer for “what changed” queries.Source: §I.5 · p. 18, §I.6 · p. 19

I.6 Complexity

• Without an explicitly implemented sparse operator, the sigmoid gate does not reduce worst-case complexity; measured overhead remains small relative to the 8B backbone.The text connects this observation to the matched latency profile reported in Appendix H.5.Source: §I.6 · pp. 18–19

J.1 Hard-Subset Analysis

• On the hardest CausalVQA quartile, Planning improves by 15.5 points and Hypothetical by 10.2, while Counterfactual improves by 1.8 and aggregate Reasoning by 9.4.The hardest questions remain difficult for both models.Source: §J.1 · p. 19, Table 15 · p. 19

J.2 Published NExT-QA Baselines

• Published CaKE-LM NExT-QA scores use a different taxonomy and video-QA training pipeline, so they are not directly matched to Table 2.The reported historical scores are explicitly framed as context rather than a matched comparison.Source: §J.2 · p. 19

J.3 Relation to MLLM-CD

• InfluenceField’s influence operator constrains causal statements to be mutually consistent under propagation, while MLLM-CD aggregates pairwise judgments and lacks an intervention channel.The discussion says ESHD improves most on MAG, whose larger graph penalizes locally plausible but globally inconsistent edge sets.Source: §J.3 · p. 19

K Additional Discussion

• Factual objectives alone leave representations within a similarity orbit; the discussion presents interventional objectives as the source of causal content.The cited controls compare C2 with the full model to measure this gap.Source: K Additional Discussion · p. 19
• Continuous fields defer commitment to a node set until intervention supervision indicates which spatial regions behave as causal units.The trade-off is conditioning: kernel inversion becomes ill-conditioned as σ grows.Source: K Additional Discussion · p. 19