PaperlayerView exact revision on arXiv

Source-linked AI summary

Do World Models Learn Global Understanding?

Alexander Detkov, Matt Thomson

arXiv:2609.34058v1cs.LGcs.AI

TL;DR

The paper asks how to test whether models lift local observations into global understanding, framed as learning constraints and propagating their consequences. It constructs monoid-world tasks and compares training objectives, finding that compositional training improves propagation across constraints, architectures, vision, and language, while deeper propagation remains limited.

  • Problem

    The paper asks how to determine whether models learn constraints from local observations and propagate them to unseen facts, a formal test of global understanding.

    Source: §1 · p. 1

  • Method

    The authors construct monoid worlds with controlled constraints and entailed held-out transitions, then compare next-state training with compositional training that hides intermediate states.

    Source: §1 · pp. 1–2

  • Results

    Compositional training averages 96% accuracy on inverse, commutative, and composition constraints across architectures and improves propagation in visual and language tasks, while next-state training fails on non-trivial constraints.

    Source: §1 · p. 2, §3.2 · p. 4

  • Takeaways & Limitations

    The results suggest that training objectives that require combining local observations can promote propagation and integration of information across the tested architectures and domains.

    Source: §6 · p. 9

    Even compositional models generalize sharply less as proof depth increases, though longer composition lengths improve deeper recovery.

    Source: §5.3 · p. 8, §6 · p. 9

Abstract

from arXiv · show

AI systems often feel brittle and fragmented. A large language model (LLM) may correctly explain a concept but fail to apply it, or follow safety instructions in one context but not another. This behavior suggests a general failure to lift local information to a global understanding. To gain fundamental insight, we frame "understanding" as learning constraints and propagating their consequences. We construct learning tasks on monoid worlds, sets of states connected by action transitions, where observed training transitions and an unseen constraint jointly determine held-out transitions. Measuring generalization tests whether models can learn global constraints from local transitions and propagate their consequences. We consider inverse, commutativity, composition, and periodicity constraints relevant to spatial and semantic structure. Across attention, recurrent, and state-space architectures, next-state training fits the data but fails to propagate non-trivial constraints. Compositional training, which uses identical paths but hides intermediate states from the input, achieves 96% accuracy on inverse, commutativity, and composition constraints across architectures, yields corresponding improvements in geometric generalization of world models trained on embodied environments and relational generalization in Wikidata-finetuned LLMs. How far do models propagate constraints when inferring an unseen fact may depend on first inferring others? We define proof depth d of a held-out transition, measuring the minimum number of inference rounds to infer the transition, and find that model generalization decreases sharply with proof depth. Increasing compositional path length T improves generalization. These results provide a formal way to investigate global understanding in language and world models and demonstrate that compositional training promotes information propagation and integration.

1 INTRODUCTION

The paper frames understanding as learning constraints from local observations and propagating their consequences to unseen facts. It introduces controlled monoid worlds and evaluates how training objectives affect propagation, including across multiple inference steps.

1 INTRODUCTION

  • The paper treats understanding as learning constraints and propagating their consequences, connecting spatial inverses with semantic relations such as parent and child.A few observed facts plus a constraint can determine unseen facts, such as a parent relation implying its converse.Source: §1 · p. 1
  • Monoid worlds make constraints controllable and their consequences mathematically determinable, with constraints represented as equalities over action sequences.The paper gives spatial inverse relations and semantic composition, such as a parent of a parent being a grandparent, as examples.Source: §1 · pp. 1–2
  • Held-out transitions are selected when observed transitions and a constraint jointly entail them, making generalization a test of constraint learning and propagation.The paper also defines proof depth as the minimum inference rounds needed to derive an unseen transition.Source: §1 · p. 2
  • Compositional training improves propagation across architectures and domains, while generalization still falls sharply as held-out transitions require deeper inference.The paper reports 96% average accuracy on inverse, commutative, and composition constraints, and finds that longer compositional paths improve deeper propagation.Source: §1 · p. 2

2 A MONOID FRAMEWORK FOR GLOBAL CONSTRAINTS

The framework tests whether models trained on paths of observed transitions can recover held-out transitions entailed by those observations and a world constraint. It distinguishes visible-intermediate next-state training from compositional training and measures inference complexity with proof depth.

2 A MONOID FRAMEWORK FOR GLOBAL CONSTRAINTS

  • The framework trains on paths of observed transitions and evaluates on held-out transitions jointly implied by the observed data and the world's constraint.The global constraint is not provided directly to the model.Source: §2 · p. 2

2.1 MONOID WORLD AND GLOBAL CONSTRAINTS

  • A monoid world is a finite state set with actions that are functions on states; its global constraints equate action sequences that induce the same map across the state space.In the rotation example, inverse LR = RL = 1 and periodicity L4 = 1 are essential constraints.Source: §2.1 · p. 2

2.2 EVALUATION OF CONSTRAINT PROPAGATION BY ENTAILED HOLD-OUTS

  • A held-out transition is included when every assignment satisfying the constraint and matching seen transitions agrees on that transition.The paper samples transitions using semantic-entailment checks.Source: §2.2 · p. 3

2.3 NEXT-STATE AND COMPOSITIONAL TRAINING

  • Next-state training shows intermediate states as inputs, whereas compositional training hides them and requires predicting from the starting state and action sequence.Composition length T controls the number of actions composed; at T = 1, the objectives coincide.Source: §2.3 · p. 3, Figure 1 · p. 3

2.4 MEASURING INFERENCE COMPLEXITY OF A TRANSITION WITH PROOF DEPTH

  • Proof depth is the minimum number of inference rounds needed to derive a transition using rules based on the world's algebraic constraints.Each round can use transitions derived in earlier rounds.Source: §2.4 · p. 4

3 CONSTRAINT PROPAGATION IN CONTROLLED MONOID WORLDS

Across architectures, compositional training propagates several non-trivial constraints that next-state training does not, but neither objective propagates periodicity. Compositional generalization also exhibits grokking-like delay and depends on the training path length.

3.1 NEXT-STATE TRAINING FAILS TO PROPAGATE CONSTRAINTS

  • Across attention, recurrent, and state-space architectures, next-state training fails at chance on inverse, commutative, composition, and periodicity constraints, succeeding only on trivial copy worlds.Many incorrect predictions are consistent with a copy failure mode, despite seen transitions contradicting the copy constraint at almost every state.Source: §3.1 · p. 4

3.2 COMPOSITIONAL TRAINING IMPROVES CONSTRAINT PROPAGATION

  • Transformers achieve at least 93% accuracy on inverse constraints with compositional training, and generalization jumps 98% when path length increases from T = 1 to T = 2.At T = 1 the objectives coincide; T = 2 first includes two-action compositions.Source: §3.2 · p. 4
  • Compositional training fits observed transitions well before recovering held-out ones, with 50% inverse generalization reached at 16k steps versus 100% seen-transition accuracy at 1.2k steps.The authors describe this as a grokking-like delay.Source: §3.2 · p. 4
  • 96% average accuracy across inverse, commutative, and composition constraints, and 100% on copy, are achieved by compositional training across architectures; neither objective propagates periodicity.Compositional models also fail periodicity worlds for path lengths T from 1 to 8.Source: §3.2 · p. 4, Figure 2 · p. 5

4 CONSTRAINT PROPAGATION IN VISION AND LANGUAGE

Compositional objectives improve constraint propagation in visual world models and Wikidata-finetuned language models, while relational converse propagation remains difficult. The section evaluates geometric and relational constraints using held-out transitions entailed by observed data and approximate domain constraints.

4.1 GEOMETRIC CONSTRAINTS IN VISUAL WORLD MODELS

  • The visual evaluation uses approximate inverse and commutative constraints, which hold roughly 80% of the time because object collisions can violate them.Held-out transitions are chosen away from collisions, and a pixel-shift baseline tests whether visual similarity could recover them without geometric constraint learning.Source: §4.1 · p. 5, §4 · p. 5
  • In visual worlds, compositional training reaches 83% inverse and 93% commutative accuracy on observations, and 94% and 98%, respectively, when predicting states directly.Next-observation training reaches 50% and 51%; next-state generalization collapses to random guessing, while compositional state prediction achieves 94% and 98%.Source: §4.1 · pp. 5–6, Figure 3 · p. 6

4.2 RELATIONAL CONSTRAINTS IN LARGE LANGUAGE MODELS

  • In Wikidata-finetuned LLMs, compositional phrasing achieves 45% average inverse accuracy and 73% composition accuracy, while next-state phrasing averages 1% on inverse and composition; neither propagates converse.The evaluation uses held-out relational transitions jointly entailed by training transitions and constraints.Source: §4.2 · pp. 6–7
  • Compositional models recover composition without shortcut paths, achieving 50% without shortcuts versus 92% with them, while next-state models fail in both settings.The comparison uses T = 2 paths in a monoid composition world with N = 1024.Source: §4.2 · p. 7
  • In a branching world, compositional converse accuracy falls from 99% at β = 0 to 63% at β = 0.1 and at most 3% for β ≥ 0.75.The β = 0.1 setting adds only a 10% chance that a parent has another child.Source: §4.2 · p. 7

5 HOW FAR DO CONSTRAINTS PROPAGATE?

The section measures how far models propagate constraints by the proof depth of held-out transitions, then examines how training dynamics and composition length affect that propagation.

5.1 CONSTRAINT PROPAGATION DECREASES SHARPLY WITH PROOF DEPTH

  • With composition length T = 2, recovery falls from 95% at d = 1 to 49% at d = 2, 15% at d = 3, and 5% at d = 4; d = 5 onward is at chance.Next-state-trained models do not generalize at any proof depth; the compositionally trained models’ accuracy decays approximately geometrically, at a rate of 0.4 ± 0.1 over depths 1–4.Source: §5.1 · p. 7

5.2 PROPAGATION DYNAMICS SUGGEST INTERNAL CONSTRAINT CASCADES

  • For composition length T = 4, the steps to reach half of final accuracy rise with depth from 26k at d = 1 to 236k at d = 4, about 2.1× per depth.The ordering remains monotonic through d = 8, but spacing narrows beyond d = 4; d = 5–8 reach half their final accuracy between 364k and 376k steps.Source: §5.2 · pp. 7–8
  • Only 3% of correct inferences lacked correctly inferred derivation premises, compared with 11% under a timing-matched shuffled null.This pattern is consistent with models applying local rules sequentially in proof-depth order, though the authors describe the resemblance as qualitative.Source: §5.2 · p. 8

5.3 LONGER COMPOSITION LENGTH ENABLES DEEPER PROPAGATION

  • At the same number of training steps, increasing composition length improves higher-depth recovery: T = 3 raises d = 2 by 43% over T = 2, and T = 4 adds 14% for d = 3–8 over T = 3.The decay curve shifts from near-chance by d = 5 at T = 2 to plateaus near 10% at T = 3 and 20% at T = 4.Source: §5.3 · pp. 8–9

6 DISCUSSION

The discussion argues that training objective shapes whether models propagate constraints, while emphasizing that propagation remains shallow and improves with longer composition lengths.

6 DISCUSSION

  • Across attention, recurrent, and state-space architectures, compositional training propagates constraints where next-state training does not, with corresponding improvements in vision and language tasks.The objectives use identical path data; compositional training hides intermediate states, encouraging integration across actions rather than local prediction.Source: §6 · p. 9
  • The consistency of propagation patterns across architectures and domains suggests controlled monoid worlds capture learning biases that extend to richer tasks.The authors propose these worlds as simplified environments for studying and designing better-propagating algorithms.Source: §6 · p. 9
  • Generalization decreases sharply with proof depth, and deeper transitions are recovered later; increasing composition length improves generalization to deeper consequences.The training dynamics suggest a slow, inaccurate process resembling a constraint cascade.Source: §6 · p. 9

7 RELATED WORK

The related work situates constraint propagation among research on relational failures and training changes, and describes how this paper extends prior approaches across domains and constraint types.

7 RELATED WORK

  • Prior work reports that language models may fail to reverse or compose learned relations, while proposed training changes address particular failures.The paper frames these issues as instances of learning and propagating constraints across monoid worlds, visual environments, and knowledge-based language tasks.Source: §7 · p. 9
  • Earlier approaches improve consistency through reverse-prediction training or architectural changes that recover group structures in synthetic environments.This framework also covers semantic constraints, including the converse relation between parent and child.Source: §7 · p. 9
  • Training without intermediate states has been studied in knowledge-graph embeddings and planning, while this work examines propagation across architectures, visual environments, and language tasks.The paper reports that compositional training helps with many geometric and semantic constraints but not all, and that generalization still declines with proof depth.Source: §7 · p. 9

8 CONCLUSION

The conclusion frames global understanding as learning constraints from local observations and deeply propagating their consequences, while noting that deeper propagation remains limited and composition length helps.

8 CONCLUSION

  • Across architectures, next-state training fails to propagate non-trivial constraints, whereas compositional training substantially improves propagation.The paper reports corresponding improvements under compositional-style objectives in visual and language tasks.Source: §8 · p. 10
  • Deep propagation remains limited, though longer composition lengths help models reach higher proof depths.The authors conclude that global understanding requires training methods that encourage integration of local observations and deep propagation of consequences.Source: §8 · p. 10
  • The authors used generative AI tools to implement methods, improve readability, suggest edits, and proofread, and say they reviewed the assisted work.They report reviewing and testing AI-written code and taking responsibility for the final paper content and artifacts.Source: AI USE STATEMENT · p. 11

A SCALING MODEL, WORLD, AND COMPUTE UNDER NEXT-STATE TRAINING

Scaling model width, world size, or training time under next-state training does not enable recovery of held-out inverse transitions. A compositionally trained control generalizes on the same inverse world.

A SCALING MODEL, WORLD, AND COMPUTE UNDER NEXT-STATE TRAINING

  • Every next-state model fits observed transitions but remains at random-floor accuracy on held-out inverse transitions despite scaling width, world size, or training time.The tested scales range from 64 to 320 in width, 1024 to 8192 states, and 15k to 300k training steps.Source: A SCALING MODEL, WORLD, AND COMPUTE… · p. 13
  • 97% held-out inverse-transition accuracy is achieved by the compositionally trained control.It uses the same world, training paths, and model as the next-state comparison.Source: A SCALING MODEL, WORLD, AND COMPUTE… · p. 13, Table 1 · p. 13

B MODELS INCORRECTLY PROPAGATE COPY CONSTRAINTS IN CONFLICT

Models often substitute another action’s target when they fail to recover an unseen transition, while compositional generalization on composition transitions rises with the amount of constraint evidence.

B MODELS INCORRECTLY PROPAGATE COPY CONSTRAINTS IN CONFLICT

  • When an unseen transition is missed, models often predict another action’s target from the same state.A copy is counted when the prediction is B(s) rather than the target A(s) for the queried action A.Source: Figure 5 · p. 13, B MODELS INCORRECTLY PROPAGATE COPY… · p. 13
  • 64% of next-state errors are copies, compared with 36% of compositional errors across four constraint worlds and all four architectures.Among next-state-trained models, copy rates are 90% for LSTMs, 77% for GRUs, and 58% for transformers.Source: B MODELS INCORRECTLY PROPAGATE COPY… · p. 14

C COMPOSITIONALLY TRAINED MODELS PROPAGATE CONSTRAINTS IN PROPORTION TO OBSERVED EVIDENCE

  • Compositionally trained models’ accuracy rises monotonically from 2% at evidence 31 to 93% at evidence 922, while next-state models do not generalize.The comparison uses transformers trained at T = 2 on a composition world with N = 1024.Source: C COMPOSITIONALLY TRAINED MODELS… · p. 14, Figure 6 · p. 14

D COMPOSITION LENGTH HAS MINIMAL EFFECT ON RECOVERY OF IMMEDIATELY DERIVABLE (PROOF DEPTH d = 1) TRANSITIONS

For immediately derivable transitions, increasing compositional path length from T = 2 to T = 4 leaves accuracy unchanged on three constraints and does not rescue periodicity. Generalization also falls with branching, and composition can propagate without shortcut paths.

D COMPOSITION LENGTH HAS MINIMAL EFFECT ON RECOVERY OF IMMEDIATELY DERIVABLE (PROOF DEPTH d = 1) TRANSITIONS

  • At T = 4, compositional training reaches 96% average accuracy on inverse, commutative, and composition constraints, matching T = 2; copy accuracy is 100%.Neither T = 2 nor T = 4 propagates periodicity; the compared unseen transitions have proof depth d = 1.Source: D COMPOSITION LENGTH HAS MINIMAL… · p. 14, Figure 7 · p. 15

E CONVERSE CONSTRAINT PROPAGATION DECREASES SHARPLY WITH BRANCHING

F COMPOSITION PROPAGATION OCCURS WITHOUT PATH SHORTCUTS

G PERIODICITY IS NOT PROPAGATED ACROSS COMPOSITION LENGTHS

Loading 2609.34058v1…