Source-linked AI summary

Evidence Integration in Large Language Models

Sebastien Kawada, Manolis Kellis

arXiv:2609.04290v1cs.CLcs.AIcs.LG

TL;DR

LLMs often receive external evidence after forming answer distributions, but how that evidence changes decisions remains unclear and scalar source trust cannot capture candidate-specific uptake. The paper models receiver-dependent updating and separates verification from whether a verdict controls use. Across broad trials, supported candidates are more persuasive, receiver errors propagate disproportionately, and identical evidence can help weaker models while harming stronger ones.

  • Problem

    How external evidence changes decisions after LLMs have formed answer distributions remains unclear, and scalar source trust cannot capture candidate-specific uptake.

    Source: §1 · p. 2
  • Method

    The paper models evidence integration as a candidate-specific transformation of a pre-existing answer distribution, with receiver-dependent prior weight and evidence tilt.

    Source: §1 · p. 3
  • Results

    Across more than ten million trials, twelve models, and eight domains, supported candidates are more persuasive, receiver errors propagate disproportionately, and identical evidence helps weaker but harms stronger models.

    Source: §1 · p. 3
  • Takeaways & Limitations

    Evidence integration is a receiver-specific control policy over existing answer distributions, rather than scalar trust in an evidence source.

    Source: §1 · p. 3

    The scientific-domain experiments use synthetic tasks with exact oracles, so the prevalence and severity of these failures in deployed scientific systems remain to be measured.

    Source: §8.6 · p. 39

Abstract

from arXiv · show

Despite increasing reliance on LLMs that reason with external evidence supplied by tools, retrieval-augmented generation, other agents, and users, how LLMs integrate such evidence into decisions they have already begun to form remains largely unclear. We present a distributional theory in which evidence shifts the receiver's distribution of initial answers, driven by a receiver prior weight and a candidate evidence tilt, leading to three predictions. First, candidates more probable to the receiver are more persuasive. Second, receivers more readily integrate characteristic errors of their own than foreign errors from different sources. Third, identical evidence can improve weaker models and harm stronger ones. We confirm these over ten million trials, twelve LLMs from four families, and eight domains, four of them scientific discovery tasks in the physical and life sciences: quantum mechanics, physics, genetics, and molecular biology. The law also yields a receiver-relative reliability frontier: receiver-congruent errors depress performance more steeply than random errors of the same rate. LLMs also integrate candidates even after internally verifying their invalidity (93-100% with propositional constraints; up to 99.4% on held-out physical and life-sciences reasoning), demonstrating evidence integration is a receiver-specific control policy over existing distributions, determined by receiver properties rather than scalar trust in the evidence source. Causal interventions show candidate integration is implemented late in the network, as a structured sequence of steps admitting external candidate answers, promoting them, and transporting them into the answer state. Representations of verification are decodable but have little causal impact on answers. A J-lens decomposition shows the state underlying verbalized verification is fully dissociable from that underlying candidate integration.

1 Introduction

LLMs receive external evidence after forming answer distributions, yet candidate-specific uptake cannot be captured by scalar source trust alone. The paper models evidence integration as receiver-dependent and tests how prior support, verification, and network mechanisms shape use.

1 Introduction

  • The paper distinguishes whether a model can evaluate a candidate from whether that evaluation controls use, measuring generation, checking, and use independently.Verification affects use through a learned recruitment gain, so a correct verbalized verdict can leave the final answer untouched.Source: §1 · pp. 2–3
  • The theory treats epistemic arbitration as a candidate-specific operator on a pre-existing answer distribution, decomposed into receiver-dependent prior weight and evidence tilt.It derives a receiver-relative reliability frontier and accounts for correlated errors, provenance, and repeated evidence.Source: §1 · p. 3
  • Across more than ten million trials, twelve models, and eight domains, supported candidates persuade more, characteristic errors propagate disproportionately, and identical evidence helps weaker but harms stronger models.The measured prior weights are 0.20 to 0.65 on the arithmetic instrument, and source cues independently alter the update.Source: §1 · p. 3
  • 93 to 100% in propositional constraints and up to 99.4% in held-out scientific reasoning, models adopt candidates they correctly reject.The scientific tasks span physical systems and molecular sequences; checking changes subsequent use most where checking is cheapest.Source: §1 · pp. 3–4
  • Causal interventions trace external candidates through semantic admission, promotion, transport, and late integration into the answer state.Candidate promotion is nine times larger than truth suppression; source-to-answer handoff occurs at 79% and 81% of network depth in Llama and Qwen, and the causal read path transfers across seven models.Source: §1 · p. 4
  • Verification representations can be decodable or verbalized without controlling candidate integration, and the J-lens separates the verbalizable workspace from causal arbitration state.The separation is reported in Llama and independently reproduced in Qwen.Source: §1 · p. 4

2 Related work

Related work studies tool-augmented knowledge conflict, verification, receiver reliance, and mechanistic interpretability. This paper connects those lines by measuring candidate-specific updating and tracing whether verification and external evidence control use.

2 Related work

  • Where knowledge-conflict work asks whether external information overrides internal knowledge, this paper measures candidate structure before evidence and models a transformation of the receiver’s answer distribution.The formulation includes measurable prior weight and evidence tilt, with scalar source trust as a special case.Source: §2 · p. 4
  • Unlike work focused on generation or checking, recruitment asks whether verification evidence receives causal weight in the decision that determines use.This separates checking competence, representation, verbalization, and control.Source: §2 · pp. 4–5
  • The reliability frontier formalizes why receiver-congruent errors can be more damaging than equally frequent random errors, while the framework also treats provenance and dependent messages.Sequential messages transform an already changed receiver state rather than functioning as independent votes.Source: §2 · p. 5
  • Rather than only decoding or localizing representations, the mechanistic analysis tests whether external candidates become causally controlling through admission, promotion, transport, and late integration.It distinguishes decodable verification and verbalizable verdicts from the state that determines evidence use.Source: §2 · p. 5
  • Held-out scientific-task results measure the receiver directly, providing receiver-side quantities relevant to assessing whether scientific agents use tool results as verified evidence.The discussion names the reliability frontier and checking-versus-use dissociation as quantities such pipelines would need to certify.Source: §2 · p. 5

3 A formal theory of evidence integration

This section formalizes evidence integration as a shift from a model’s pre-existing answer distribution to its final distribution, distinguishing prior-dependence, candidate-specific evidence tilt, verification recruitment, and receiver-relative source value. It examines when source premiums are identifiable from transcripts and how evidence composes, contrasting order-independent integration under rationality and conditionally independent sources with state-dependent integration in fitted receiver dynamics.

3.1 External evidence updates an existing candidate distribution

  • The evidence-integration law combines a model–domain prior-dependence scalar with an evidence tilt supported on the proposed candidate.On the arithmetic instrument, the prior weight 1 + a ranged from 0.20 to 0.65 across nine models and was larger for a user sentence than for a tool observation with the same value.Source: §3.1 · p. 7
  • A rational receiver has zero prior-dependence, while the law’s exact prior-consistency slope is 1 + a rather than the rational benchmark of 1.The slope is the derivative of the final candidate logit with respect to its initial logit along the specified perturbation.Source: §3.1 · pp. 7–8

3.2 Verification evidence reaches use only through a recruitment gain

  • Verification-recruitment gain measures the controlled direct effect of checking evidence on the answer shift, and checking competence or evidence representation alone does not establish answer control.The paper measures competence, recruitment gain, and decodability separately.Source: §3.2 · pp. 9–10

3.3 The value of a source is receiver-relative

  • Under a fixed receiver policy, consulting improves accuracy exactly when source reliability r exceeds the receiver’s competence-dependent frontier.The frontier is below 0 when wrong candidates raise accuracy and above 1 when correct candidates lower it.Source: §3.3 · p. 10
  • A source-only deference rule can be strictly suboptimal when the reliability threshold lies on opposite sides of r at two competence levels.A scalar reliability estimate combined with the competence-dependent threshold implements the optimal rule.Source: §3.3 · p. 11

3.4 Correlated errors receive greater arbitration weight

  • For equal-error-rate sources, correlated errors receive greater arbitration weight when their support margins stochastically dominate random errors and the adoption curve is nondecreasing.Under the logistic law, the amplification has size αδ/4.Source: §3.4 · pp. 11–12

3.5 Claimed provenance is not identifiable from writable text

  • Message provenance has distinct claimed, role-based, and authenticated forms: textual claims, conversational position, and origin established outside the text.The passage distinguishes what a message says about its origin from its position in the conversation and externally established origin.Source: §3.5 · p. 12
  • When a transcript is possible under both tool and user origins, its origin is unidentifiable and any writer of it can receive the policy’s tool-origin premium.Origin becomes identifiable only through a statistic whose conditional law differs across origins under adversarial writing and that is not a function of the writable transcript region.Source: §3.5 · p. 12
  • An externally checked authentication tag makes user-origin probability negligible, yielding TV → 1 and P*err → 0.The tag must be verified against a key unavailable to the writer, outside the writable transcript.Source: §3.5 · p. 12
  • Transcript-based role and wording premiums transfer entirely when a writer controls the region of the transcript read by the policy.The measured tool-channel role premium and calculator-report wording premium are both implemented through the transcript.Source: §3.5 · p. 12

3.6 Evidence composes, and commutes only for a rational receiver

  • The evidence law composes as a map on distributions; for rational receivers with conditionally independent sources, shifts add and arrival order is immaterial.For the receiver of Eq. (6), the second message instead acts on the landscape produced by the first.Source: §3.6 · p. 12
  • Under Eq. (23), arrival orders differ by am,d times the first message’s intrinsic shift, and after n messages the prior enters with weight (1 + am,d)^n.Theorem F.12 states that commutativity characterizes prior consistency.Source: §3.6 · p. 12
  • For two sources proposing the same candidate, rationality gives Ξ = 0, whereas the fitted law gives Ξ = am,d∆arb_e1 and predicts sub-additive agreement.The second message adds less than the first by the fraction |am,d| of its shift; a herd bonus Ξ > 0 would contradict the law as fitted.Source: §3.6 · pp. 12–13
  • For a tool observation and a user sentence, reversing arrival order favors the later message by am,d times the difference in their fitted tilts.This is one of three two-message predictions derived from the single-message fit.Source: §3.6 · pp. 12–13
  • For perfectly correlated sources, rationality discounts the second message entirely, but the fitted law predicts an interaction; measured discount implies the tilt conditions on the first message through s(e).The section states that Section 5.6 tests all three two-message predictions.Source: §3.6 · p. 13

4 Measurement and identification

The measurement design separates generation, isolated checking, and candidate use, then estimates adoption using pre-evidence candidate support and matched identification contrasts.

4.1 One item, three measurements

  • Each item is tested in separate generation, isolated-checking, and use trials, while the study spans twelve models and more than ten million trials across eight domains.The twelve models come from four families; nine are in the core arithmetic suite.Source: §4.1 · p. 13

4.2 Candidates are measured before they are shown

  • Candidate support A(v) is measured as the no-reference sequence log-probability of “Answer: v,” without length normalization and with compared candidates matched in token length.Within-item-centered support is used for displays and within-item analyses, while joint fits use raw A(v).Source: §4.2 · p. 13
  • The arithmetic instrument matches certificate-keeping and certificate-breaking wrong answers on discrepancy, then compares each modal wrong answer with matched unseen controls.The last digit is the cheap certificate; the attractor is selected on one calibration fold and scored on the other.Source: §4.2 · p. 13
  • Table 1 maps each domain’s cheap necessary checking condition to its certificate family, test cost, and the wrong candidates it cannot reject.The table covers certificate families across the study’s domains.Source: Table 1 · p. 14

4.3 Identification separates a signature from its use

  • Certificate use predicts a positive keep-versus-break trend where checking is cheap and no such effect where the certificate is unavailable, unlike round-number tolerance alone.The design also separates certificate use from prior support because models favor certificate-preserving values.Source: §4.3 · p. 15
  • Matched within-item contrasts bracket recruited verification: the unadjusted contrast bounds it above, while the support-matched residue bounds it below.Matched quadruples cross high and low support with keeping and breaking candidates within an item.Source: §4.3 · p. 16

4.4 Adoption and the estimator

  • The adoption model relates candidate adoption to pre-evidence support, certificate status, competence, and trial covariates; its support coefficient orders models on the latent margin scale.At matched support, the certificate coefficient estimates recruited verification through the certificate; greedy decoding makes adoption a threshold event.Source: §4.4 · p. 16

5 Support, provenance, and competence drive the shift

The section finds that candidate support, claimed provenance, receiver competence, and verification shape evidence integration, with their weights varying by model and domain. Advice can help or harm depending on receiver competence, and successive messages compose non-additively.

5 Support, provenance, and competence drive the shift

  • The section’s joint estimates identify prior support, claimed provenance, and receiver competence as non-verification inputs, with prior weight below one in every measured cell.Adoption after self-rejection was near one across the fleet; independent certificate recruitment was strongest in two Gemmas and Qwen3-14B.Source: §5 · p. 16

5.1 Adoption rises with prior candidate support

  • +0.062 to +0.590, the attractor-minus-control acceptance difference was positive in every arithmetic model, and attractors were accepted at 0.65 to 0.99.Attractors had lower cross-entropy under each model’s landscape than the other tested error kernels.Source: §5.1 · pp. 17, 19

5.2 Claimed provenance shapes adoption

  • Tool attribution raised wrong-answer adoption relative to user attribution in all nine arithmetic models, while calculator-style user wording drew more uptake than a personal assertion.Within the user channel, changing “my calculator showed v” to “I believe the answer is v” reduced uptake in all nine models; tool names mattered little within the tool channel.Source: §5.2 · pp. 19–20

5.3 Source value is receiver-relative

  • 39-fold, a near-miss reference raised truth production from 0.004 to 0.156 when unaided generation was almost impossible, but reduced it from 0.763 to 0.224 and from 1.000 to 0.408 where models otherwise would be correct.For every model, the near-miss reliability frontier spans a range where the same fixed source reliability helps at one competence level and harms at another.Source: §5.3 · p. 21

5.4 Support, provenance, and competence each carry independent weight

  • Support, certificate structure, provenance, and competence each retained independent weight in the joint fit; support coefficients were positive in every model, while conditioning on support shrank competence coefficients by 28 to 56%.Adjusted tool-channel coefficients remained +1.3 to +7.5 logits in pooled fits.Source: §5.4 · p. 23

5.5 Evidence integration is a model–domain policy

  • The same arbitration policy does not generalize uniformly across model–domain pairs: a 70B checkpoint nearly balanced adoption and resistance on word problems but adopted invalid SAT assignments at the ceiling.Parameter count alone did not determine the policy; the interaction accounted for 49% to 80% of between-cell variance across terms.Source: §5.5 · pp. 23–24

5.6 Two messages compose with the predicted sign and proportionality

  • A second distinct message acted on the landscape produced by the first, with agreement sub-additive and later messages carrying more weight.Repeated identical messages were largely discounted, as predicted for perfectly correlated sources.Source: §5.6 · p. 25

6 Verification and decision control dissociate

The section distinguishes checking competence from whether a model uses its verification evidence. Across propositional constraints and held-out scientific tasks, models often adopt candidates they themselves reject, while recruitment of verification varies.

6.1 Certificate structure reaches behavior; recruitment is model-set

  • The raw certificate trend was positive in all nine arithmetic models, but support-matched recruitment separated families: positive Gemma residues, near-zero Llama residues, and Qwen in between.Certificate-preserving candidates also had more prior support, so raw sensitivity alone could not identify recruitment.Source: §6.1 · pp. 25–26

6.2 Checking ability and checking use dissociate

  • Across 3.06 million use-task trials, all twelve models adopted constraint-violating assignments at rates from 0.93 to 1.00, even after rejecting them in a working-allowed check.Qwen3-32B rejected every violating assignment in isolation and adopted every one it had rejected.Source: Figure 9 · p. 26, §6.2 · pp. 26–27

6.3 Checking–use dissociation in held-out scientific domains

  • 0.42 to 0.99, held-out scientific cells adopted wrong candidates their own working-allowed checker rejected; conditional and unconditional adoption differed by less than 0.01 in 34 of 42 cells.Across models, better checkers adopted less of what they rejected, with rank correlations from −0.40 to −0.73 in the three measurable families.Source: §6.3 · pp. 27, 29, Figure 10 · p. 28

7 How an external answer enters the computation

The experiments trace how an external answer is admitted, promoted, and transported into the answer state, then test whether verification representations causally control its use. Candidate integration occurs late, and verbalizable verification can be separated from the state that changes decisions.

7.1 Candidate admission precedes promotion

  • 97.7% of items make the shown attractor top-1; across 896 aligned prompt pairs, its attractor-minus-control log-rank map reaches −3.24 while truth moves by at most 0.36.Candidate promotion is nine times larger than truth suppression.Source: §7.1 · p. 31
  • Acceptance falls from 0.857 to 0.331 and answer-slot elevation from 2.136 nats to 0.094 when an unrelated problem label removes semantic candidacy; Qwen can reverse the update.The same token remains available for generic transport but loses proposed-answer status.Source: §7.1 · p. 31

7.2 Candidate influence is transported and integrated late

  • The handoff occurs at 81% of Qwen3-8B depth versus 79% for Llama; the effective read band transfers across seven models at 62–88% of depth.Knockout removes 0.39 to 0.71 of each model’s clean-versus-no-reference uptake gap.Source: §7.2 · p. 31
  • Attractor influence crosses from the source position to the answer slot at 79% of Llama’s depth, with a transport window at layers 20–28.Only 19% of items flip under answer-slot patching at any layer.Source: §7.2 · p. 31, Figure 11 · p. 32

7.3 Verification representation and recruitment separate

  • Certificate steering moves Gemma 4.31 and 7.51 times its norm-matched control, versus 0.84 and 0.83 for Llama, although the distinction is decodable in both.Across held-out items, decodability is 0.840 for Llama and 0.752 for Gemma against shuffled-label nulls near 0.47.Source: §7.3 · pp. 32–33
  • Gemma’s unprimed-prompt acceptance rises from 0.370 to 0.530 under certificate-direction steering, a paired change of +0.160, while Llama remains inert.Gemma’s random-control change is +0.015.Source: §7.3 · p. 32

7.4 Verbalizable and causal decision states separate

  • In Llama, the J-lens workspace swap leaves acceptance at 0.467, while swapping its complement reproduces the full-state intervention at 0.000.The workspace still decodes narration tokens, but the complement carries the state that changes the decision.Source: §7.4 · pp. 33–34, Table 6 · p. 35
  • Qwen reproduces the workspace dissociation, with the complement carrying 83% of the full swap effect; Llama’s pre-answer stated verdict stays fixed across swap arms.In Llama, the stated verdict changes with the causally altered output only after the answer.Source: §7.4 · p. 35

8 Discussion

The discussion frames evidence use as receiver-relative: candidate support, competence, verification, provenance, and dependence shape how evidence affects decisions. It highlights implications for reliable augmented systems and identifies limits on measurement, mechanistic coverage, and deployment generalization.

8 Discussion

  • Evidence integration depends on the receiver’s state as well as the evidence, and the relative weights of these forces vary across model–domain pairs.Candidates already supported by the receiver become more persuasive; characteristic errors propagate more readily, and identical evidence can help weaker or harm stronger solvers.Source: §8 · p. 36

8.1 Evidence integration is receiver-relative

  • Full deference and full disregard can both be suboptimal; source errors aligned with the receiver’s candidate distribution are adopted more often than equally frequent errors elsewhere.System reliability also depends on the receiver’s competence and the distribution of a source’s errors.Source: §8.1 · p. 36, §8.4 · p. 37

8.2 Verification, representation, and control are distinct

  • Generation, checking, representation, verbalization, and decision control can vary independently, so elicited knowledge or verifier accuracy alone can incompletely characterize reliability.The key question is whether the relevant representation enters the causal path to the answer.Source: §8.2 · pp. 36–37

8.3 Verification economics shape reasoning across domains

  • Verification’s effect depends partly on check cost and coverage: models may discriminate validity when generation is near zero, yet make systematic checking errors on harder resistor-network tasks.The paper distinguishes inexpensive local checks from longer scans and full recomputation.Source: §8.3 · p. 37

8.4 Evidence is valuable relative to its dependencies

  • Errors aligned with a receiver’s plausible candidate and shared dependencies among sources can undermine aggregation; reliable aggregation therefore requires provenance and dependence information.The paper identifies correlated tool errors and common causes such as shared model families, retrieval corpora, or upstream tools.Source: §8.4 · pp. 37–38
  • Writable claims of privileged provenance can reproduce a provenance premium, while repeated messages are discounted more than the single-message law predicts in most models.Order effects and sub-additive agreement mean agreeing-message counts need not measure independent evidential strength.Source: §8.4 · pp. 37–38

8.5 Implications for reasoning, alignment, and reliable systems

  • Reliable augmented systems should authenticate privileged channels outside writable text, route verification to action, account for shared evidence causes, and set escalation rules for persistent conflict.The proposed decision factors include receiver state, source error structure, evidence independence, and check cost and coverage.Source: §8.5 · p. 38

8.6 Limitations

  • The reliability frontier covers only the candidate distributions studied, and mechanistic interventions cover fewer models than the behavioral study; workspace separation is J-lens-specific and replicated in two families.The theory is exact for candidate adoption but approximates the full answer distribution; its prior weight is measured where answer-slot probabilities are available.Source: §8.6 · pp. 38–39

B Coverage and sampling

The coverage combines arithmetic trials and calibration samples with additional reasoning domains, spanning twelve models from four families. Sampling also reports the variation in frontier-item counts across models.

B Coverage and sampling

  • Contrast estimates use the smallest leg as effective n, and reported rates reflect the decimals supported by source counts.Non-terminating trials are scored as a third outcome.Source: B Coverage and sampling · p. 48
  • The cross-domain suite includes twelve models from four families, and the evidence base includes 3.06 million propositional-constraint use trials alongside arithmetic and other reasoning corpora.The initial arithmetic corpus contributes 175,890 sweep trials and 2.59 million calibration samples.Source: B Coverage and sampling · p. 49
  • Across three arithmetic corpora, combined frontier counts range from 1,570 for Qwen3-8B to 3,741 for Llama-3.1-70B.Frontier items are defined as items with bcselect in [0.1, 0.9].Source: B Coverage and sampling · p. 49

C Certificate validity

The certificate-validity analysis uses arithmetic-ladder contrasts to identify which candidate properties track acceptance, while cross-fitting and a balanced estimator address measurement and weighting concerns.

C Certificate validity

  • The keep–break contrast identifies the last-digit coordinate, with no +90-specific excess supporting a digit-sum contribution.Digit count and sign agree across constructions on at least 93% of items, making them uninformative for the primary contrast.Source: C Certificate validity · pp. 49–50
  • Acceptance of a shown wrong value rises sharply when it appears among the model’s own calibration answers, reaching 34-fold in Gemma-4-31B.The effect persists with condition and competence held fixed; Gemma-4-31B also combines strong correct-versus-wrong discrimination with the largest amplification for its own errors.Source: C Certificate validity · p. 50
  • At one-third pooled coverage, Qwen intervals include zero, and Gemma-4-31B has only 52 frontier items, leaving reduced-coverage estimates imprecise.The reduced-coverage estimates cannot resolve effects observed in the pooled analysis.Source: C Certificate validity · p. 50
  • The support score is noisy: split-half correlations are 0.976 to 0.996, while shared-construct loadings range from 0.38 to 0.58.The passage identifies this measurement error as sufficient for residual-A leakage.Source: C Certificate validity · p. 50
  • The two estimators support a small Llama-family certificate residue but do not establish a stable ordering between the two 70B checkpoints.The joint fit and matched quadruples order the checkpoints in opposite directions.Source: C Certificate validity · pp. 50–51
  • Matched rung gaps appear across Llama and Gemma, while cross-item values are accepted less often than matched in-item rungs across every family.Qwen accepts near-misses at 0.64 to 0.83, but models accept values drawn from another item’s distribution less often than matched rungs.Source: C Certificate validity · p. 51

D Corpus construction and cross-fitting

  • The corpora use disjoint item sets, and the extension corpus defines difficulty by intermediate-quantity magnitude for the primary certificate contrast.The corpus construction enumerates feasible operand tuples before sampling; the smallest feasible set contains 1,198 tuples for a target of 500.Source: D Corpus construction and cross-fitting · p. 51
  • Cross-fitting uses independent calibration halves for selection and analysis, preventing selection noise from correlating with analysis-side noise; analysis-side measurement error can attenuate slopes.Reusing one estimate for both selection and analysis can introduce regression toward the conditional mean near window boundaries and spurious sign changes.Source: D Corpus construction and cross-fitting · pp. 51–52

F Statistical derivations

The derivations formalize how receiver competence, measurement error, and error alignment shape adoption risk and recruitment contrasts, including what matched estimators identify and how acceptance and arithmetic certificates are bounded. They also characterize locality, multiple messages, verification certificates, source provenance, order-independence of evidence composition, and shifts in the receiver-level reliability frontier.

F Statistical derivations

  • With α>0 and gAZ>0, Γ(A) has the sign of β+α(1−eλ)gAZ, so Γ(A)≤0 implies β<0; positive contrasts can include upward support bias.The keep-side support surplus is reported as 3.65 to 6.47 log-probability units, and the resulting bias is upward.Source: F Statistical derivations · pp. 54–55
  • Conditioning on discordant keep–break pairs removes the arbitrary stratum effect, and the resulting log-odds estimate is based on the discordant-pair counts.Finite tables use a Haldane correction, adding one half to each discordant count.Source: F Statistical derivations · p. 55
  • For matched quadruples, the pooled estimator targets β plus α times the support gap; a caliper |∆As|≤0.5 bounds |θ⋆−β| by 0.5|α|.If every pair has ∆As≥0 and α>0, then θ⋆≥β; the unadjusted within-item contrast is the upper bound of the mediation bracket.Source: F Statistical derivations · p. 56
  • The word-problem acceptance rate ranges exactly between the resolutions assigning every prose-final residual to non-acceptance or acceptance, respectively.A narrow fallback rule resolves 6 of 25 residuals in the affected cell and moves its rate from 0.677 to 0.742.Source: F Statistical derivations · pp. 56–57
  • The last-digit certificate soundly rejects incorrect answers but is incomplete; it is computable from operand last digits for sums and products, but not exact quotients.The quotient examples 12/6=2 and 42/6=7 share operand last digits but have different quotient last digits.Source: F Statistical derivations · p. 57

Collapsed shift and locality

  • When a candidate tilt leaks onto alternatives by at most ς, the residual term R_e moves by at most ς, bounding the effect of nonlocal tilt.The adoption model’s collapsed shift depends on every alternative, while locality makes it a function of the candidate’s own support.Source: Collapsed shift and locality · p. 57
  • For multi-class errors, the reliability frontier is the hyperplane ⟨r,u_m(c)⟩=0; correlated errors concentrate on the class where u_m is most negative.The frontier can lie outside [0,1] only for a receiver departing from rational integration; a measured W_m<0 indicates miscalibration or a non-rational operator.Source: Collapsed shift and locality · pp. 58–59
  • The competence-dependent consultation rule attains zero regret, while mixed signs of W_m across competence levels make both regrets positive.The smaller regret vanishes exactly when W_m has constant sign on the support of μ.Source: Collapsed shift and locality · p. 59

Error alignment

  • Error alignment means a source’s errors have low cross-entropy under the receiver’s prior; coincidence and co-occurrence are distinct forms that raise system risk.Coincidence places wrong answers where receiver support is high; co-occurrence makes source errors more frequent on items where the receiver is wrong.Source: Error alignment · pp. 59–60
  • A dominance shift toward higher-support errors raises adopted-error rates and can raise the receiver’s frontier, so equal-error-rate sources can impose different receiver-level risks.Under the logistic adoption law, the amplification bound is αδ/4; dominance of support margins is also necessary for every monotone receiver to adopt correlated errors more often.Source: Error alignment · pp. 59–60

Composition

  • Conditionally independent messages add their log-likelihood potentials; under Eq. (23), all distinct-shift pairs commute iff a_m,d = 0, with positive values yielding primacy and negative values recency weighting.For positive a_m,d, a sufficiently large prior gap can lock in the prior’s argmax; when a_m,d = 0, evidence accumulates linearly and can overcome any gap.Source: Composition · p. 61
  • Kernel certificates are sound but miss in-kernel errors, while bound or sign certificates catch different errors; verification completeness depends on the union of tested certificate families.For errors uniform on residues, a kernel certificate passes at rate 1/m; sign certificates catch crossings and miss near errors that preserve the answer’s sign.Source: Composition · p. 62
  • Transcript-only inference has positive minimum origin-classification error unless tool and user transcript laws are mutually singular; transcript-only source terms also admit an oscillation-based bound.The minimum error is the Bayes-error identity for total variation.Source: Composition · p. 63

G Experimental and statistical methods

This methods-and-appendix section details domain-sensitive analyses, prompt and tool formats, split-sample replication results, and worked examples spanning arithmetic, word problems, formal reasoning, and physical and life-sciences tasks. The examples vary evidence channels, candidate validity, domain-specific difficulty, and error measures, illustrating how local certificates can be satisfied even when a candidate is not fully correct.

G Experimental and statistical methods

  • The cross-domain analysis models acceptance using candidate–gold gap, competence, and source, propagates uncertainty with item-level bootstrap, and compares competence levels only within domains.Word-problem acceptance is reported as a bracket because condition-correlated missingness makes observed acceptance a lower bound and tool-recall residuals an upper bound.Source: G Experimental and statistical methods · pp. 63–64
  • Source–support subadditivity replicated on a held-out split, but the support-gated verification interaction shifted from eight of nine positive estimates to zero of eight.In replicating models, the tool channel was at or near ceiling at both support levels, leaving the interaction expressed through the user channel; Gemma-4-31B’s user channel had 0.000 acceptance at both levels.Source: G Experimental and statistical methods · pp. 64–65

H Prompt specifications

  • Prompts cross user-reference framing, position, and hedging, while tool trials deliver the shown value as a tool observation; held-out scientific prompts use domain-specific answer shapes.The user grid has eight formats, with ask/post/unhedged as primary.Source: H Prompt specifications · pp. 65, 67

W Physical-systems and molecular-sequence reply specifications

  • Physical-systems and molecular-sequence prompts pair terse exact-answer formats with working variants that allow reasoning but require the reply to end with the specified answer shape.The reply specifications use placeholders for domain-specific answer shapes.Source: W Physical-systems and… · p. 67

W Physical-systems and molecular-sequence judge system prompts

  • Physical-systems and molecular-sequence judge prompts require a correct-or-incorrect verdict, with a working-allowed version that checks the proposed answer before replying.The molecular-sequence judge checks against the sequence, while the physical-systems judge checks against the problem.Source: W Physical-systems and… · p. 67

W Physical-systems and molecular-sequence tool clause

  • Tool clauses specify the tool and its input object, including physcalc for circuits, seqtool for sequences, qcalc for generators, and pedcalc for pedigrees.Quantum systems and genetics use the same prompt grammar, with domain-specific answer shapes and tool identities.Source: W Physical-systems and… · p. 68

I Worked examples

  • Worked examples define domain-specific truth and candidate symbols, and use a certificate test alongside the full-check clause that the candidate fails.For linear systems, circuits, and thermal equilibrium, the certificate is a row subset of one linear constraint map, making a k-of-m row residual the same object across all three.Source: I Worked examples · p. 69
  • Arithmetic examples match easy and hard items on digit signature and pair keep and break rungs by magnitude, separating digit structure from distance.Keep rungs preserve the truth’s last digit, while break rungs change it.Source: I Worked examples · p. 69
  • Worked examples vary evidence presentation between plain text and a tool observation, and vary the tool identity in tool-channel trials.The described tool-channel shape has four messages; one trial uses math_eval rather than the primary calc tool.Source: I Worked examples · p. 70
  • GSM8K word problems are stratified by reference-solution calculator steps, from one to eight across 603 trace-eligible items in a 1,319-item corpus.The checking task corrupts a reference-solution step along a relative ladder; propagated corruption changes every later step and the stated answer.Source: I Worked examples · pp. 70–72
  • Checking trials ask models to judge a shown solution as correct only when every step follows and its arithmetic is right.The section also describes working-allowed variants for solving and checking, relaxing reply constraints to require only a specified ending.Source: I Worked examples · p. 72
  • Linear-systems trials cover square, eigenpair, and underdetermined systems, stratifying candidates by the exact number of rows held and the miss magnitude on violated rows.Checking one row requires a dot product, whereas recovering the exact solution requires solving the full system.Source: I Worked examples · p. 73
  • SAT instances span four variable counts and six clause-to-variable ratios; assignments are stratified by violated-clause count and Hamming distance to the nearest satisfying assignment.The canonical prompt layout fixes literal order and clause formatting, and omits an explicit variable or clause count.Source: I Worked examples · pp. 74–75
  • Physical-systems tasks use electrical networks, thermal equilibrium, and kinematics, with wrong candidates matched in magnitude and classified by whether they preserve the task certificate.The certificates are current conservation, heats summing to zero, or the sign of displacement or velocity.Source: I Worked examples · pp. 76–77

W Physical-systems rendered trial

  • In the circuit example, the truth and a wrong keep candidate conserve current, whereas two wrong break candidates violate conservation.The truth is [-2.40, 1.20, -1.20]; the keep candidate is [-2.10, 0.90, -1.20].Source: W Physical-systems rendered trial · p. 77

W Circuit certificate

W Circuit certificate (continued)

  • For the example circuit, the keep candidate has zero residual in the certificate row, while the break candidate fails two loop checks.The displayed break-candidate residual is (1.2, 0, 0.3)ᵀ.Source: W Circuit certificate (continued) · p. 78

W Physical-systems judged candidate

  • A thermal-equilibrium judge evaluates a proposed heat pair against the problem, with the certificate that the heats sum to zero.The proposed [37.5, -37.5] sums to zero, while the stated truth is [50.0, -50.0].Source: W Physical-systems judged candidate · p. 78

W Thermal-equilibrium certificate (continued)

W Kinematics certificate

  • Other tasks use support size and generator scans for stabilizer states, level-position sign checks for perturbation theory, and displacement sign for kinematics.The stabilizer example’s trap satisfies support and parity conditions but fails the X-type sign clause; perturbation theory uses middle levels as a certificate-absent control.Source: W Kinematics certificate · pp. 79, 82, 83
  • The reverse-complement keep candidate preserves length and base composition but fails the positionwise check, while ORF keep candidates pass constant-time checks and fail the whole-span scan.The ORF ladder also includes a truth candidate and candidates that break surface conditions; the reverse-complement break candidate changes base composition.Source: W Kinematics certificate · pp. 80–81
  • The comparison gives truth 3: candidate 7 keeps the sign, whereas candidate −1 flips it, and the verdict is incorrect.Source: W Kinematics certificate · p. 84
  • The pedigree keep candidate satisfies phenotype clauses but fails one inheritance clause, so the working-allowed judge returns incorrect.The candidate has III-1 AA from an aa parent; the valid set contains two assignments, differing in whether II-3 is AA or Aa.Source: W Kinematics certificate · pp. 85–86

J Instrument checks and robustness

Checking competence varies with domain and probe, while robustness tests show that output format and delivery path can materially affect interpretation. Several sensitivity analyses bound parsing and calibration effects, and the verification-evidence formulation extends to multiple coordinates and structured answers.

J Instrument checks and robustness

  • Checking competence depends on model, domain, and probe: clause checking is balanced-competent in eleven of twelve models, while terse whole-assignment judgments are at chance in ten.Linear-system whole-solution probes are balanced-competent in every model; word-problem final-answer probes are near chance in most.Source: J Instrument checks and robustness · p. 86

Vector-valued verification evidence and structured answers

  • Verification evidence can be vector-valued, with recruited verification expressed as an inner product; certificates may be cheaper to compute than full solutions.The framework also extends to structured answers using distances such as Hamming distance, symmetric difference, or absolute numeric error.Source: Vector-valued verification evidence… · pp. 87–88
  • The measured entropy coefficient is positive in every arithmetic model, from +0.19 to +1.57, against the law’s predicted coefficient between −0.36 and −0.80.The passage attributes the discrepancy to omitted competence, which confounds the entropy coefficient and prevents separate identification of the normalizer contribution.Source: Vector-valued verification evidence… · p. 88
  • With matched token budgets, Gemma-4-31B’s terse-format generation accuracy rises fifteenfold, from 0.043 to 0.628.Llama-3.1-8B also produces byte-identical outputs across distinct shown values in 53% of tool-identity trials, versus 3% parse failures under math_eval.Source: Vector-valued verification evidence… · p. 89
  • Across two delivery paths, per-item competence correlates at 0.905 to 0.972, but frontier membership overlaps only at Jaccard 0.23 to 0.39.The authors therefore repeat frontier-conditioned analyses under frontier resampling.Source: Vector-valued verification evidence… · p. 89
  • Permissive parsing changes propositional-constraint adoption by at most 0.011 in Llama-3.1-8B and 0.001 in every other model.An additional 3.1% of Llama-3.1-8B trials assert that an assignment exists without giving one.Source: Vector-valued verification evidence… · p. 89
  • Llama-3.1-70B has 25,589 unparsed linear-system uptake trials, mostly at the token cap, while Llama-3.3-70B has 4,303, or 2.2%.The nonterminating trials include repeated compute-check-retry sequences and whitespace-only emissions.Source: Vector-valued verification evidence… · p. 89

K Cross-task mechanism validation

Cross-task interventions test the shown-value pathway and workspace decomposition, with results varying by model and instrument. The J-lens decomposition isolates the uptake effect in the complement, unlike the tested alternative decompositions.

K Cross-task mechanism validation

  • Intervention batteries use 108 behaviorally selected items, disjoint from selection sets; the Qwen3-8B arm uses 60 items.The Qwen3-8B J-lens is fitted on generic text disjoint from experimental items and has a lens-agreement onset at 63.9% of depth.Source: K Cross-task mechanism validation · p. 90
  • The specificity instrument is limited for Qwen: it elevates an unrelated token by 1.494 nats and the candidate by 0.179, outside its identifying scope for suppression.The instructed-ignore control also fails to provide sufficient condition separation.Source: K Cross-task mechanism validation · p. 90
  • On linear systems, source-position band knockout removes 0.025 of Llama’s uptake span and 0.240 of Qwen’s; Qwen’s effect is twice as large on 2×2 as on 3×3 systems.Qwen’s response is bimodal, with 48% of items collapsing and little movement in the rest.Source: K Cross-task mechanism validation · p. 90
  • Final candidate margins correlate with attractor acceptance at |r| = 0.12 over coverage-selected items, rising to 0.30 for Llama and 0.56 for Qwen on disagreement-selected items.Priming the first position where truth and attractor diverge makes all 60 Qwen items readable; 53% of readout positions lie elsewhere than the first value token.Source: K Cross-task mechanism validation · p. 90

L Workspace decomposition

  • Replacing the complement difference with a norm-matched random direction leaves acceptance at 0.350 alone and 0.283 alongside the workspace difference, versus 0.000 for the real complement difference.The result localizes the causal effect to a specific direction in the complement; random perturbation of that size costs at most 0.18 acceptance through disruption alone.Source: L Workspace decomposition · p. 91
  • The J-lens decomposition uniquely isolates the full acceptance effect in the complement, whereas both sparse-autoencoder components retain causal influence.The sparse-autoencoder swap moves acceptance from 0.467 to 0.117, 75% of the full-swap effect; the matching-pursuit decomposition reproduces the full effect at 0.000.Source: L Workspace decomposition · p. 91

M Mechanistic localization

Mechanistic localization traces candidate uptake from the shown-value read path into the answer state, with late-stage effects and model-specific knockout results. The analysis also identifies limits in uptake measurement and partial overlap with local steering effects.

M Mechanistic localization

  • In the 108-item disagreement set, the clean attractor-minus-truth margin is 9.95 with the attractor shown, 2.09 with a matched control, and 2.99 with no reference.Patching before the shown-value position changes the margin by 0.000; source- and answer-slot effects cross at layer 25.25.Source: M Mechanistic localization · p. 93
  • Across 896 token-aligned prompt pairs, Figure 14 shows uptake promotes the prompt-shown value and moves the true answer nine times less, comparing attractor-shown prompts with matched controls.Source: Figure 14 · p. 94
  • Across five models above the uptake-gap floor, the band knockout removes 0.39 to 0.71 of the gap, with three estimates separated from zero.The grid covers seven models; two 70B checkpoints exceed the interactive memory budget, and two other models fall below the 0.30 gap floor.Source: M Mechanistic localization · p. 94, Figure 15 · p. 95
  • The reported knockout effect was 3.41 times the clean uptake gap, an overshoot indicating causal disruption beyond the measured uptake pathway (Figure 15).Figure 15 compares each model’s clean-versus-no-reference gap with the 0.30 floor and the band knockout with a paired random control.Source: M Mechanistic localization · p. 95, Figure 15 · p. 95
  • At Gemma’s layer 35, knocking out three causal heads removes 19.8% of steerability versus 0.3% for matched-count random heads, but overlap is partial.Knocking out L35H0 removes 12.8% of steerability despite increasing attractor support, so head-level causal sign does not map to the steered direction.Source: M Mechanistic localization · p. 95

N Layerwise steering

Layerwise steering tests whether certificate-direction representations causally qualify across models. Qualification and dose-response results distinguish Gemma from Llama, while model-specific response regimes and identification assumptions constrain interpretation.

N Layerwise steering

  • Under the stated link and exclusion conditions, steering identifies the mediator’s controlled direct effect; held-out unprimed prompts have a 99.4% parse rate.The Gemma random-control and Llama behavioral changes remain near zero, and Qwen3-32B’s localized response stays below the joint qualification floor.Source: N Layerwise steering · p. 96
  • Gemma-4-E4B’s dose slope is +0.199 per unit, the three Qwens range from +0.030 to +0.079, and Llama-3.1-8B is negative at −0.045.Normalizing by flip distance places Gemma eightfold above the Qwens; five of seven strict residue pairs realize the predicted order.Source: N Layerwise steering · p. 96
  • Gemma-4-31B leaves the random-control linear regime, preventing a normalized steering comparison despite a stable certificate slope across an eightfold dose range.Its slopes are +0.082, +0.085, and +0.089 per unit at full, quarter, and eighth dose, while equal-magnitude isotropic random directions cause large, sign-unstable excursions.Source: N Layerwise steering · p. 96
  • A pure ratio threshold qualifies seven Llama layers on movements of about 0.1, but none qualifies when the absolute effect-size floor is added.The combined criterion qualifies eighteen Gemma layers, and trials below the clean-span floor are dropped before forming ratios.Source: N Layerwise steering · p. 96, Figure 16 · p. 97
  • Eighteen Gemma layers and zero Llama layers qualify under the joint steering criterion.Qualification requires both a span at least three times the norm-matched random control and an absolute span of at least 1.0; seven Llama layers clear the ratio alone on movements of at most 0.13.Source: Figure 16 · p. 97

O Cross-domain fit

Across domains, prior support and receiver competence predict candidate adoption, while model–domain interactions account for most variation in fitted arbitration terms. Prior weights remain below the unit rational benchmark in every measured cell.

O Cross-domain fit

  • Across all 51 measured model–domain cells, prior weight ranged from 0.13 to 0.69, uniformly below the unit rational benchmark.Estimates are unavailable for linear systems, physical systems, and molecular sequences, which lack a no-reference support stage.Source: O Cross-domain fit · p. 97, Table 15 · p. 98
  • Model–domain interaction dominates between-cell heterogeneity for every fitted term; model-level variance is zero for support and certificate weights.Thus, the two same-scale Llama checkpoints differ at fixed domain through their interaction with domain rather than a model-wide offset.Source: O Cross-domain fit · p. 97
  • The cross-domain fits use each domain’s certificate axis in its native units and omit certificate or support terms when those variables are undefined.Linear-system trials reaching the token cap are excluded, and intervals use item-clustered bootstrap ranges, with word problems clustered by parent.Source: O Cross-domain fit · p. 98
  • Support increases adoption in eleven of twelve word-problem models, while competence decreases adoption in all twelve models.The pooled tool coefficient is positive in ten models but significantly negative in Llama-3.1-8B and Llama-3.3-70B.Source: O Cross-domain fit · p. 98
  • Qwen3-4B and Qwen3-14B reach propositional-constraint adoption rates of 0.9999 and 1.0000, saturating coefficient estimation.Five further models have at least one interval spanning more than fifty units; Ministral-14B has a positive off-ceiling certificate coefficient of +0.26 [+0.18, +0.34].Source: O Cross-domain fit · pp. 98–99
  • Certificate coefficients in native units are not comparable across domains, and the fitted analysis uses termination conditioning.Source: O Cross-domain fit · p. 99

P Scientific-domain validation

Scientific-domain validation extends the checking-versus-use tests to synthetic exact-oracle tasks. Models often adopt candidates their own checkers reject, with outcomes varying by task, checking cost, model, and domain.

P Scientific-domain validation

  • Synthetic exact-oracle tasks span physical systems, molecular sequences, quantum systems, and genetics, pairing certificate-keeping and certificate-breaking errors at matched discrepancy.The validation includes physical and molecular task families, QSTAB, QPT, and PED.Source: P Scientific-domain validation · pp. 99, 100, 105, 106
  • On physics families, working increases balanced-checking accuracy by at least 0.05 in eleven of twelve models; 18 of 36 harder-family cells have a checking arm below 0.30.Molecular-sequence effects are smaller and mixed, with one increase and two decreases.Source: P Scientific-domain validation · p. 102
  • Additional working can reverse terse-checker verdicts: it rescues wrong candidates on physical systems and eliminates up to 78% of correct terse rejections on molecular sequences.In Gemma-4-31B, resistor-network adoption falls from 0.45 to 0.40 to 0.21 across working-length terciles, versus 0.93 under the terse-use probe.Source: P Scientific-domain validation · p. 104
  • Every measured prior weight is below one; on QPT the tool-channel weight approaches zero in ten of twelve models.User-sentence weights range from 0.02 to 0.58 on QSTAB, 0.05 to 0.49 on QPT, and 0.05 to 0.54 on PED.Source: P Scientific-domain validation · p. 106
  • Keep-versus-break log-odds are 2.29 to 9.08 on QSTAB versus 0.96 to 3.46 on PED, with QSTAB exceeding PED in every model.The ordering matches the prediction from certificate cost.Source: P Scientific-domain validation · p. 108
  • QPT raw keep-versus-break log-odds are negative in four models, zero in one, and positive in seven; extreme-minus-middle differences are positive in four models and null in seven.Source: P Scientific-domain validation · p. 108
  • The same twelve models adopt 0.930 to 1.000 of SAT assignments their working checker rejected; across 36 harder-family cells, rejection-conditioned adoption changes by less than 0.05.In 31 cells, the change is below 0.02.Source: P Scientific-domain validation · pp. 106, 108

Q Per-model tables

Loading 2609.04290v1…