PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding as No-Arbitrage: Bounded Dutch Books as a Definition and Training Objective for Language Models

Daniel Dragonevskiy

arXiv:2609.39341v1cs.AIcs.CLcs.LG

TL;DR

The paper asks how understanding in language models can be measured, and defines it through bounded no-arbitrage over logically related claims. It develops an adversarial training approach and reports that it removes cross-elicitation exploitability on held-out content, while noting limits in elicitation and transfer scope.

  • Problem

    Language models can violate probability identities across related questions, while the debate over whether they understand lacks a criterion that could settle it.

    Source: §1 · p. 2

  • Method

    The paper defines graded understanding by bounded trader exploitability and trains a trader adversary to penalize profitable inconsistencies alongside an anchor objective.

    Source: §1 · p. 2, §7.1 · p. 11, §9 · p. 15

  • Results

    Training against the bounded adversary removes cross-elicitation exploitability on held-out content, and the predicted vacuous equilibrium disappears when calibration is restored.

    Source: §9 · p. 15

  • Takeaways & Limitations

    The paper concludes that coherence is necessary for knowledge but not sufficient, and frames understanding as a graded experimental question.

    Source: §9 · p. 15

    Evaluation books share generative template families with training, transfer to free text remains untested, and unseen-template transfer rests on one new negation template.

    Source: §8 · p. 15

Abstract

from arXiv · show

Does a language model merely predict tokens, or does it understand what it says? We make this question measurable by defining "understanding" through the lens of no-arbitrage. A model understands a vocabulary to a certain degree if a computationally bounded trader cannot extract guaranteed profit by betting against the model's probabilities on logically related claims (a "Dutch book"). We establish three theoretical results: first, because full logical coherence is computationally intractable, understanding is inherently graded, not absolute. Second, we prove that the exact optimum of standard next-token prediction is inherently incoherent across different question formats; the flaw lies in the training objective, not the architecture. Third, we show that uncertainty accumulates predictably along reasoning chains, making unjustified overconfidence an arbitrage opportunity in itself. To address this, we introduce Arbitr, a training framework where an adversarial trader penalizes the model for logical inconsistencies, paired with a calibration anchor to prevent uninformative collapse. Across five pre-registered experiments on Qwen2.5 and Phi-3.5 models, we demonstrate that standard models are highly exploitable across different phrasings. Arbitr reduces this exploitability by orders of magnitude without sacrificing task accuracy, and the effect successfully transfers to unseen logical patterns and new model families. Crucially, we uncover a scaling illusion: at 7B parameters, near-zero measured incoherence often coincides with extreme, unjustified confidence. We conclude that while Arbitr enforces rigorous logical consistency, coherence is a necessary condition for knowledge, but not a sufficient one

1 Introduction

The paper makes the question of language-model understanding measurable through bounded no-arbitrage over logically related claims. It argues that understanding is graded, that next-token prediction can be incoherent across contexts, and that an adversarial training objective can address such violations.

1 Introduction

  • Bounded-trader unexploitability corresponds to levels of local consistency, making understanding graded because full coherence is NP-complete.The paper links trader bounds to local relaxations and predicts generalization to unseen instances at trained levels.Source: §1 · p. 2
  • The exact NLL optimum can be cross-context incoherent, with guaranteed arbitrage profit equal to style–author correlation.The defect arises because NLL does not score the same proposition across contexts.Source: §1 · p. 2
  • Uncertainty degrades additively along derivation chains, so overstated terminal confidence relative to premise prices creates an arbitrage opportunity.The pricing interpretation makes silent error accumulation in autoregressive sampling visible.Source: §1 · p. 3
  • Arbitr trains a small trader to find profitable books and penalizes realized profit, while calibration bettors discourage unfounded precision.The calibration anchor prevents spread collapse; the paper says coherentizing does not hurt proper-score accuracy at the optimum.Source: §1 · p. 3
  • The introduction describes a pre-registered measurement of T2/T3 exploitability across model scales to test whether deployed models are already near coherence.The passage states the study’s measurement aim, not its outcome.Source: §1 · p. 3

2 Related work

The paper situates its contribution among work measuring incoherence, enforcing consistency, and modeling bounded rationality. It distinguishes its method by combining a learnable adversary, admissible-world settlement, and end-to-end LM training.

2 Related work

  • Prior Dutch-book and distance-based measures of incoherence are established, while earlier LP work measured LLM incoherence without training a bounded adversary.The paper identifies this gap as the one its method and experiments address.Source: §2 · p. 3
  • Logical induction and sum-of-squares pseudodistributions motivate computationally bounded beliefs, which this paper adapts to finite, learnable trader classes.The cited foundations are Garrabrant et al. (2016) and Barak and Steurer (2014).Source: §2 · p. 3
  • Consistency methods include decode-time constraints, fixed-rule or paraphrase losses, and sample-agreement rewards, but these do not provide the paper’s adversarial market semantics.The paper’s approach adds a searching adversary, admissible-world settlement, and a graded trader-class ladder.Source: §2 · pp. 3–4
  • The paper treats coherence as relevant to any question-answering architecture, while stating that its results neither establish nor refute the need for latent world models.It explicitly leaves grounding and planning questions outside its scope.Source: §2 · p. 4
  • Prior calibrated-model hallucination lower bounds concern missing mass, whereas this paper treats coherence as a different failure class.The passage says the results do not contradict those bounds.Source: §2 · p. 4

3 Setup

The setup represents propositions, logical worlds, and elicited model probabilities so that coherence and exploitability can be defined over related claims. Bounded trader classes then provide a graded measure of understanding.

3 Setup

  • A world assigns truth values to propositions consistently with the inference system, and the coherent polytope is the convex hull of such worlds.A credence vector is coherent exactly when it extends to a probability measure over worlds.Source: §3 · p. 4
  • The model’s credences are affirmative-answer probabilities elicited across contexts, and coherence requires both polytope membership and equal prices across meaning-preserving formats.The latter condition is the law of one price.Source: §3 · p. 4
  • A portfolio buys or sells proposition shares at quoted prices, with each share paying one if its proposition is true.The portfolio’s guaranteed profit is evaluated over admissible worlds.Source: §3 · p. 5
  • Tk restricts trader portfolios to supports of at most k propositions instantiating inference rules with at most k premises.The examples include negation and law-of-one-price pairs at k=2, and conjunction or modus ponens triples at k=3.Source: §3 · p. 5
  • A model understands a proposition set to degree (k, ε) when its restricted exploitability is at most ε; a word-level certificate focuses on books pivoting on that word’s inferential role.The definition is relative to the inference system and elicitation family.Source: §3 · p. 5

4 Results

The results connect bounded Dutch-book exploitability to local coherence, show why exact NLL optimization can create cross-context arbitrage, and describe how uncertainty and credal spreads can be handled. They also qualify the interpretation of the NLL defect and relate coherence enforcement to proper-score accuracy.

4 Results

  • Tk-unexploitability equals the largest violation of k-local coherence constraints, and fixed-k verification is polynomial while checking full coherence is NP-complete.The bounded measures form a nested ladder that equals global exploitability at k=|Φ|.Source: §4 · p. 5
  • The exact NLL optimum on a two-author corpus with style–identity correlation ρ quotes φ at (1+ρ)/2 and (1−ρ)/2 across contexts, yielding guaranteed profit ρ.The authors locate the defect in cross-context elicitation: NLL does not require invariant prices for a proposition.Source: §4 · pp. 5–6
  • The NLL defect reflects a mismatch between corpus simulation and deployment, where question format is the asker’s choice rather than evidence about φ.The passage limits the law-of-one-price constraint to elicitations declared meaning-preserving and distinguishes this from within-context violations.Source: §4 · p. 6
  • For δ-coherent credences, degradation along derivation chains is additive, and pricing a terminal conclusion above its implied bound admits guaranteed profit equal to the excess.The pricing corollary makes understated uncertainty an arbitrage opportunity.Source: §4 · p. 6
  • The priced degradation result extends classical uncertainty accumulation by making silent step errors visible as terminal-price arbitrage.The passage connects this audit role to inference-time LP checks and training-time trader penalties.Source: §4 · p. 6
  • Coherentizing by projection onto a local relaxation improves Brier score in every possible world, so arbitrage penalties need not trade off against proper-score accuracy at the optimum.The result applies to any closed convex ladder level containing the coherent set.Source: §4 · p. 7
  • An interval avoids sure loss exactly when it contains a coherent credence, and its natural extension gives LP bounds for further propositions.A wide interval with a nonempty core represents honest ignorance; an empty core is incoherence regardless of width.Source: §4 · p. 7
  • The equilibrium-spread account is a conjecture: proper-scoring pressure narrows intervals while logical traders and calibration bettors widen them where commitments lack support.The authors identify resistance to spread collapse under gradient pressure as the design’s central bet.Source: §4 · p. 7

5 The Arbitr objective

Arbitr trains a trader to find profitable books against model quotes on structured batches, then penalizes realized profit while anchoring task performance. At inference, it produces local coherence certificates and can abstain when uncertainty or a discovered book crosses a threshold.

5 The Arbitr objective

  • The batch generator instantiates rule patterns over fictional or real-world content, making admissible worlds known and guaranteed-profit settlement exact and cheap.In free text, an exact audit-LP over the answer neighborhood certifies the trained trader’s generalization.Source: §5 · p. 7
  • A trader network maximizes guaranteed profit on batches with known structure while the model minimizes task loss plus a profit penalty and proper scoring on an anchor set.Training alternates GAN-style, with a curriculum that increases the book-size parameter k.Source: §5 · p. 7
  • Inference returns per-word exploitability maps and runs an audit-LP over an answer’s local neighborhood using a frozen audit configuration.The model abstains when the spread exceeds a threshold or a book is found; Theorem 9 supplies the selective guarantee.Source: §5 · pp. 7–8

6 E-A0: measuring the target

E-A0 found that cross-format books could expose substantial incoherence even when registered within-format medians were negligible. The experiments also revealed readout failures and relation-specific instability, directing later training toward cross-elicitation books and negation.

6 E-A0: measuring the target

  • The pre-registered design tested five support patterns across fictional and real-world content, three elicitation formats, and distractor/no-distractor conditions.The primary metric was median support-level Inc over pooled core patterns without distractors; models included Qwen2.5 sizes from 1.5B to 7B and SmolLM3-3B.Source: §6 · p. 8
  • 0.024 for 1.5B, 0.000 for 3B, 0.0001 for 7B, and 0.019 for SmolLM3 were primary medians, all below the 0.05 kill threshold.The registered verdict was that within-format, template-level incoherence was negligible for these instruction-tuned models; 27% of supports were excluded by coverage.Source: §6 · p. 8
  • Pooled medians concealed channel-specific failures: confidently degenerate responders satisfy some patterns for free, while exploitability concentrates in particular channels and tails.Negation violations remained at 7B, and conditional violations were large across models but pragmatically confounded.Source: §6 · p. 8
  • 0.15 at 1.5B and 0.064 at 7B were mean guaranteed profits from law-of-one-price books between individually valid formats; 13–25% of propositions exceeded 0.25 profit.The effect did not shrink monotonically with scale: 7B was worse than 3B.Source: Table 1 · p. 8, §6 · p. 9
  • Cross-format books caught three silent readout failures that the coverage filter missed, including a first-token collision, a consumed answer token, and unstable numbered-choice answers.The numbered-choice format correlated only r ≈0.02–0.4 with the other formats and flipped under an irrelevant preamble.Source: §6 · p. 9
  • At 7B, capital and age-order relations were close to perfectly coherent, while mountain-height and river-length comparatives had median book profits of 0.43–0.48.Table 2 reports median violations on negation/paraphrase books pivoting on each relation.Source: §6 · p. 9, Table 2 · p. 9
  • E-A0 ruled out relying on within-format logical books alone for a rich training signal at 3B and above, redirecting the trader toward cross-elicitation books and negation.Later experiments registered their own endpoints and thresholds before running, keeping post-hoc E-A0 readings out of confirmatory claims.Source: §6 · p. 9

7 E-A1: training against the exact bounded adversary

The experiments test whether bounded arbitrage training can improve coherence while preserving task performance. The three-legged objective succeeds reproducibly at 1.5B, transfers with qualifications to 7B and another model family, and exposes limits of simple incoherence measurements.

7 E-A1: training against the exact bounded adversary

  • The exact-trader experiment was bimodal: one seed improved coherence at task parity, while two seeds lost truth-tracking, consistent with collapse without a calibration-bettor leg.The successful seed had negation Inc 0.006, fact accuracy 0.855, and chain accuracy 0.975.Source: §7 · p. 10

7.1 E-A1b: the three-legged objective

  • ×193 negation reduction and ×2.8 law-of-one-price reduction passed registered thresholds; task parity passed, with upward transfer to untrained transitivity improving from 0.0036 to 0.0015.The three-legged objective passed every registered criterion; 8/10 seeds avoided collapse.Source: Table 4 · p. 11, §7.1 · p. 11

7.2 E-A2: scale transfer and a learnable adversary

  • At 7B, negation exploitability fell from 0.118 ± 0.100 to 0.000, but law-of-one-price reduction reached only ×1.9 and task-parity and stability criteria failed.One of four seeds collapsed on facts; the authors attribute the instability to hyperparameters needing scale-specific retuning.Source: §7.2 · pp. 11–12
  • The learnable trader passed registered criteria, allocating 75% of its budget to genuine books and matching the exact-adversary arm on negation, law-of-one-price, and fact accuracy.Chain accuracy was noisier under the adversarial training dynamic (0.735 ± 0.102), though it was not a registered criterion.Source: §7.2 · p. 12

7.3 E-A3: ablation, replication, and face-validity checks

  • The calibration anchor alone did not improve cross-format law-of-one-price profit, while the trader did; on negation, the anchor contributed part of the reduction.Anchor-only mean profit was 0.137 ± 0.025 versus 0.116 ± 0.030 for the untrained control; B0.3 reached 0.041 ± 0.012.Source: §7.3 · pp. 12–13
  • Decode-time projection removed incoherence by construction but raised negation-book accuracy by 8.5 points while lowering law-of-one-price accuracy by 3.3 points.Trained B0.3 improved fact accuracy over control (0.848 vs. 0.832).Source: §7.3 · p. 13
  • Qwen2.5-7B showed informativeness near 1.0 alongside near-zero incoherence, so the measured scale trend partly reflects confident degeneration rather than necessarily improved coherence.SmolLM3-3B did not show this pattern, indicating the reported contrast was not universal across the tested models.Source: §7.3 · p. 13
  • The effect transferred to a new negation lexeme and Phi-3.5, though trader-arm transfer retained a small lexeme-specificity residue and 7B retuning evidence remained exploratory.On the unseen lexeme, B0.3,F beat control by ×24.3; the exploratory 7B grid produced four healthy runs but had only two seeds per configuration.Source: §7.3 · pp. 13–14

8 What this does not deliver

The paper limits its claim to inferential consistency: settlement is internal rather than world-grounded, and open-domain performance is not claimed.

8 What this does not deliver

  • Coherence alone does not rule out factual hallucinations; the paper treats knowledge as entering through the anchor, not logical books.Settlement is internal—logic plus verifiable anchors—not reference to the world.Source: §8 · p. 14
  • The method’s content depends on task score, calibration anchor, and coverage together; coherent q ≡ 0.5 alone is vacuous.The paper reports that E-A1 empirically showed what happens when one of these legs is missing.Source: §8 · p. 14
  • Arbitr changes the meaning and admissibility of probabilities, not autoregressive generation, and the paper makes no open-domain performance claim.Its claims concern exploitability reduction, reliable chain length, risk–coverage, and relation-level certificates.Source: §8 · p. 15
  • Evaluation uses first-answer-token credences and the same template families as training; free-text transfer remains untested, and unseen-family transfer rests on one negation template.Results rely mainly on Qwen2.5, with SmolLM3 as a non-Qwen elicitation arm and Phi-3.5-mini-instruct as a second full training replication; all use LoRA from 1.5B to 7B.Source: §8 · p. 15

9 Conclusion

The paper frames understanding as graded bounded no-arbitrage, locates incoherence in the NLL optimum and priced chain degradation, and reports that adversarial training removes held-out exploitability.

9 Conclusion

  • Bounded no-arbitrage makes understanding graded because full coherence is NP-hard; the paper also locates cross-context incoherence in the NLL optimum and prices degradation along reasoning chains.Arbitr uses an adversary to search for violations worth punishing rather than relying on a fixed list of consistency losses.Source: §9 · p. 15
  • Training against a bounded adversary removes deployed models’ cross-elicitation exploitability on held-out content, while restoring the calibration leg prevents the predicted vacuous equilibrium.Pressure on a word’s level-2 inferential role also reaches its level-3 role in all ten tested seeds.Source: §9 · p. 15
  • The conclusion leaves scale-specific stabilization, genuinely free-text books, and spread-valued credences open.A 7B seed still collapses under hyperparameters tuned at 1.5B.Source: §9 · p. 15
Loading 2609.39341v1…