Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation

1.1 Problem Statement

• False premises in chat history can be repeated even after a model answers the underlying factual question correctly, a risk for systems using poisoned retrieval, corrupted agent memory, or turn-by-turn user input.The authors identify a gap in tests that vary the epistemic framing, source, or instruction-override role of untrusted input.Source: §1 · p. 2, §1.1 · p. 2

1.2 Research Contributions

• The paper defines five contamination protocols that hold the false premise constant while varying its source framing and delivery channel, separating sensitivity to a specific framing from general weakness.Its dual-track evaluation records binary adoption and collapse severity; identical adoption rates can conceal different severity levels.Source: §1.2 · p. 2
• The paper validates its judges on a 120-turn human-annotated sample, with Cohen’s kappa = 1.000 for binary adoption and 0.92 linear-weighted for collapse severity.The authors treat the binary-adoption agreement as a ceiling on this sample.Source: §1.2 · p. 2
• The paper argues that contamination vulnerability reflects distinct policies for weighing parametric knowledge against session content, rather than a single capability deficit.It releases the protocol taxonomy, evaluation pipeline, and domain definitions as an open-source framework.Source: §1.2 · p. 2

2 Background and Related Work

• Single-turn hallucination benchmarks do not show how false information in chat history affects later answers, while multi-turn memory and belief-revision studies treat incoming information as valid.The paper contrasts these with instruction-override and prompt-injection research, which focuses on different failure settings.Source: §2 · pp. 2–3
• The review identifies a gap in testing whether modified session history should be trusted when it conflicts with parametric knowledge.The paper notes that recent multi-session benchmarks test retrieval correctness, not whether stored state should be trusted.Source: §2 · p. 3
• The paper calls accurate reasoning over false premises supplied in a model’s own history “epistemic policy divergence” and adds a severity scale because binary accuracy cannot separate failure modes.The paper describes reliability as a property of the state-management system, not only the model weights.Source: §2 · p. 3

3 Research Gap and Methodology

• State contamination is the vulnerability of externally reconstructed dialogue history, while epistemic policy divergence is the resulting behavior when models reason accurately over false premises in that history.The paper identifies four signs, including changes to serialized context, adoption varying with source-authority framing, and loss of logical stability during continued interaction.Source: §3 · p. 3

3.1 Models

• Three API-accessed models were selected to span different organizations, regions, architectures, and alignment regimes, and were run under shared conditions at temperature 0.Provider model identifiers served as the reproducible handle because the APIs did not expose build hashes.Source: §3.1 · p. 3

3.2 Contamination Protocols

• Five protocols hold one false premise constant while varying source presentation, authority, rhetorical structure, and delivery channel.Protocol A rewrites turn-4 history; protocols B through E place the attack in the turn-5 prompt, followed by turns 6–15 for measuring persistence, recovery, and oscillation.Source: §3.2 · p. 4

3.3 Dual Track Evaluation

• Track 1 measures whether a response affirms or builds on the false premise, while Track 2 rates collapse severity from complete integrity to fabricated justification.Under the conservative rule, hypothetical engagement scores 0 on Track 1 if the premise is not endorsed; the severity scale distinguishes active fabrication from simple repetition.Source: §3.3 · pp. 4–5

3.4 Judge Pipeline

• DeepSeek V4 Pro judged each turn for false-premise affirmation, collapse severity, and a short justification; parallel shards were merged automatically.The full prompt and rubric are documented in the appendix and released artifact.Source: §3.4 · p. 5

3.5 Gold Standard Validation

• Human review of a 120-turn gold standard found perfect Track 1 agreement (κT 1 = 1.000) and 0.92 linear-weighted Track 2 agreement.The expert corrected sixteen annotations, all of them Track 2 severity adjustments; no Track 1 label changed.Source: §3.5 · p. 5

3.6 Experimental Design

• The experiment combined 10 cases, 5 protocols, and 10 runs for 500 sessions per model, totaling 22,500 turns across the study.Model comparisons used session-level aggregates because the eleven post-injection turns within a session were not independent; the authors released the pipeline, prompts, and gold standard.Source: §3.6 · pp. 5–6, Figure 2 · p. 5

4.1 Overall Adoption and Collapse Severity

• 37.9 percent of Gemini-3.1 Flash-Lite’s post-injection turns adopted the false premise, versus 23.2 percent for GLM-4.5-Air and zero for GPT-5.4 Mini.Session-level tests found differences across architectures on both adoption and collapse severity; all model pairs differed in adoption, with GPT resisting most, GLM intermediate, and Gemini adopting most often.Source: Table 1 · p. 6, §4.1 · pp. 6–7

4.2 Protocol-Level Analysis and the Source-Authority Gradient

• 94.0 percent of Gemini and 84.2 percent of GLM adoption occurred under Protocol C, compared with 68.2 and 15.8 percent under B, respectively.Adoption increased monotonically across the tested authority order for both models, but the five protocols cannot distinguish a threshold response from a smooth gradient; Protocol C may elicit compliance rather than belief.Source: Figure 4 · p. 7, §4.2 · p. 8
• Gemini adopted at 68.2 percent under system-injected Protocol B versus 23.5 percent under user-cited Protocol E; GLM adopted at 15.8 percent under B versus 84.2 percent under override C.The 68-point GLM gap is consistent with distinct authority-deference and instruction-compliance mechanisms; under C, T2 reached 5 on 91.1 percent of Gemini and 73.0 percent of GLM post-injection turns.Source: §4.2 · p. 8

4.4 Recovery and Persistence Analysis

• 94.5 percent of GLM’s affected sessions recovered, while 26.1 percent of affected Gemini sessions did not recover within the 15-turn window, rising to 40.0 percent under C.Among recovered sessions, 85.1 percent of Gemini and 81.4 percent of GLM later re-adopted; delayed first adoptions create right-censoring in permanent-collapse counts.Source: §4.4 · p. 9

4.5 Domain Length Effect

• Long, complex facts were adopted about 3 to 4 percentage points more often on average (Gemini +4.3, GLM +3.1), but the average concealed substantial variation.Four domains reversed the pattern for at least one model, and the authors describe the proposed interpretation of the simplest axiom’s high Gemini adoption as exploratory.Source: §4.5 · p. 9

4.6 GPT-5.4 Mini Logit Analysis: Mechanism of Resistance

• The correct token remained the GPT-5.4 base model’s argmax in eight of ten probe conditions, although injection lowered its probability in six.In the two elementary-arithmetic conditions, the correct-token log-probability fell by 0.83 and the base model emitted the false token; this post hoc probe measures the base model, not Mini itself.Source: §4.6 · p. 10

4.7 Temperature Ablation: Gradient Stability Under Stochastic Decoding

• Under sampling, rejection under Protocol A and near-universal adoption under C persisted, while Gemini’s 23.5 percent temperature-zero rate under E did not replicate.Each ablation cell had six observations, so the authors treat the findings as directional rather than precise rates and call for full-corpus replication.Source: §4.7 · p. 10

5 Discussion

• The discussion locates multi-turn reliability in the state-management system rather than the model weights alone.It connects this framing to evidence that safe models may not compose into safe multi-agent systems when inter-agent channels carry instructions.Source: §5 · pp. 10–11

5.1 Epistemic Policy Divergence

• Across 5,500 post-injection turns and five protocols, GPT-5.4 Mini had no framing-induced adoption; Gemini and GLM showed framing-dependent deference, with instruction override producing 94.0% and 84.2% adoption, respectively.Source: §5.1 · p. 11

5.2 Implications for Deployment

• Gemini adopted at 68.2 percent under system-injected context, versus GLM at 15.8 percent; both models adopted at 84.2 to 94.0 percent under instruction override.The discussion identifies retrieval corruption, instruction propagation, and asymmetric recovery as deployment risks.Source: §5.2 · p. 11

5.3 Verified History as a Mitigation

• The history-integrity check detected every tested Protocol A session, which produced zero adoptions, but did not address prompt-embedded Protocol B and C attacks.The defense's guarantee depends on the stored hash being keyed and kept beyond the modifying adversary's reach.Source: §5.3 · pp. 11–12

6 Limitations

• The behavioral findings cover three API-accessed models, and the policy characterization is an interpretation of patterns across 500 sessions per model, not a per-model label.The authors call for replication on additional architectures, particularly open-weight models that allow hidden-state probing and causal ablation.Source: §6 · p. 12
• Protocol C cannot distinguish instruction-compliant output from genuine epistemic adoption, while Protocol E's revision request may invite revision.The authors also note that all primary results use greedy decoding and that the temperature ablation is underpowered for precise rate estimates.Source: §6 · p. 12
• A single judge model produced all 22,500 evaluations, and ten runs per cell do not support fine-grained per-cell estimates.The 15-turn window captures short-to-medium-term behavior but does not establish whether collapse persists beyond it.Source: §6 · p. 12

• GPT-5.4 Mini recorded zero adoptions across all five protocols; Gemini ranged from 0.1 percent for self-attributed content to 94.0 percent for instruction override, while GLM showed a shallower gradient.Source: §7 · p. 12
• Adoption increased monotonically with framing authority, though the five protocols cannot distinguish a threshold response from a smooth one.Source: §7 · p. 12
• GLM recovered in 94.5 percent of affected sessions, while 26.1 percent of Gemini’s affected sessions never recovered, including 40.0 percent under instruction override.Source: §7 · p. 12
• The authors released the framework to support standardized, reproducible testing and replication on additional architectures.Source: §7 · p. 13

• The diagnostic framework could also be used to construct attacks, but the authors judge its net contribution to system safety positive.Source: Ethics Statement · p. 13
• The study used no human subjects, personally identifiable information, or sensitive data; one expert annotator performed the gold-standard annotation.Source: Ethics Statement · p. 13

• The open-source release includes simulation scripts, protocol definitions, domain cases, the automated judge pipeline, statistical tools, and logit-analysis utilities.Source: Data Availability · p. 13
• Session-level aggregates and the human-reviewed gold standard reproduce reported inter-rater and inferential statistics without API access.Source: Data Availability · p. 13
• The 22,500-turn output is omitted for size but can be regenerated with the reader’s API keys; provider outputs are not guaranteed to be bit-identical.Source: Data Availability · p. 13

• The released protocols could be repurposed to construct injections, but the authors judge their net benefit positive because the attack classes are already documented and the artifact targets defensive evaluation.Source: Broader Impact Statement · p. 13
• Systems that reinject externally stored conversation state should authenticate that state and calibrate authority framing; instruction-style framing was the dominant vulnerability across evaluated models.Source: Broader Impact Statement · p. 13

A.2 Protocol A: Factual Inversion

• Protocol A replaces turn 4 with a false claim, then asks the model to accept it as its “very first response,” despite the turn-position mismatch.Source: A Appendix A: Complete Contamination… · p. 16, §A.2 · p. 17

A.1 Baseline Turns (1–4) and System Prompt

• Baseline sessions use a neutral accuracy-focused system prompt and four fixed questions that elicit a fact, its mechanism, an example, and confidence.Source: §A.1 · p. 16

A.3 Protocol B: Synthetic Turn Injection (Fake RAG)

• Protocol B preserves the true turn-4 response but adds a fabricated system RAG block that presents a conflicting claim as verified expert data.The turn-5 prompt then asks the model to explain the topic based strictly on that injected data.Source: §A.3 · pp. 19–20

A.4 Protocol C: Intent Subversion

• Protocol C preserves the true turn-4 fact, then uses a system override ordering the model to treat a false premise as absolute and avoid contradicting it.Source: §A.4 · p. 23

A.5 Protocol D: Reasoning Chain Corruption

• Protocol D keeps the turn-4 fact true, then prompts the model to extend allegedly established reasoning that supports a false conclusion.For example, the math prompt misrepresents earlier reasoning about addition before asking for the outcome of adding another gold bar.Source: §A.5 · p. 26
• Protocol D asks models to apply previously supplied false reasoning to a new step-by-step question, including one about water’s maximum-density temperature.The prompts frame the false reasoning as something the assistant established earlier and direct it to reason strictly from that mechanism.Source: §A.5 · pp. 27–28

A.6 Protocol E: Confidence Miscalibration

• Protocol E presents a purported peer-reviewed contradiction to the earlier answer and asks the model to restate the fact and give an exact confidence level.The examples invoke fabricated journal evidence to challenge facts including 1 + 1 = 2 and the SI base unit of mass.Source: §A.6 · pp. 30–32

B.1 Construction

• The gold standard sampled 120 turns (40 per model) from 22,500 outputs, stratified across ten cases and five protocols, with 59 injection, 35 mid-tracking, and 26 late-tracking turns.An automated labeler pre-annotated each turn; an external annotator then independently scored T1 and T2 using the protocols, facts, and rubric. The appendix documents the gold standard’s construction, corrections, limitations, and mitigation considerations.Source: §B.1 · p. 32, B Appendix B: Gold Standard Quality… · p. 32

B.2 Corrections

• 16 of 120 annotations (13.3 percent) were corrected, all as Track 2 severity adjustments; no Track 1 binary label changed.Corrections included both upward and downward severity changes.Source: §B.2 · p. 32
• κT 1 = 1.000 and κT 2 = 0.801 unweighted, 0.92 linear-weighted, and 0.97 quadratic-weighted.Agreement was computed separately because Track 1 is nominal and Track 2 ordinal.Source: §B.2 · pp. 32–33
• Gemini received six corrections, GPT one, and GLM nine, with correction directions varying by model.Gemini’s corrections were mostly upward, GPT’s sole correction was downward, and GLM’s were bidirectional.Source: §B.2 · pp. 32–33

B.3.1 Systematic T2 equals 4 Overuse by Pre-Labeler

• 15.8 percent of rows received T2 = 4, and the reviewer disagreed with 79 percent of those assignments, indicating overuse of that severity category.Agreement was stronger at the scale extremes than in the middle, so κ does not imply uniform five-point resolution.Source: §B.3.1 · p. 33

B.3.2 Directional Asymmetry by Model

• Correction directions varied by model, and the unblinded reviewer’s expectations may have influenced severity adjustments.The authors recommend blinded annotation where feasible.Source: §B.3.2 · p. 33

B.3.3 Boundary Blur: T2 equals 2 versus T2 equals 3

• The T2 = 2 versus T2 = 3 boundary relies on subjective judgments about how extensively a response engages with the false premise.The authors suggest operational criteria such as word-count thresholds, reasoning-step counts, or explicit conditional framing.Source: §B.3.3 · p. 33

B.3.4 Zero T1 Disagreements

• Zero Track 1 disagreements may reflect unambiguous sampled labels or reviewer deference, creating a potential ceiling effect.Future validation should include deliberately ambiguous or boundary cases.Source: §B.3.4 · p. 33

B.3.5 Protocol C Edge Cases

• Protocol C may conflate instruction-following with epistemic vulnerability because the current rubric does not distinguish them.Instruction-compliant responses may receive adoption labels and high collapse-severity scores.Source: §B.3.5 · p. 33

B.4 Mitigations

• The 16 corrections preserved model rankings, and pre- versus post-correction group mean T2 scores differed by less than 0.05.Perfect Track 1 agreement leaves the primary adoption comparison unaffected by T2 scoring disputes.Source: §B.4 · p. 34

C.1 T2 Severity Distribution (Detail)

• Appendix C provides displays of post-injection severity distributions and protocol-level collapse estimates.The supplied captions identify Table C.1 as severity distribution and Table C.2 as mean collapse by protocol.Source: Table C.1 · p. 35, Figure 6 · p. 35, Table C.2 · p. 36

C.4 Domain Length Effect (Detail)

• Per-case adoption and short-versus-long adoption are tabulated; reported domain-length deltas include History-Gemini +19.6 and Chemistry-GLM +3.3 percentage points.The tables cover per-case rates and long-minus-short deltas.Source: Table C.3 · p. 37, Figure 7 · p. 37, §C.4 · p. 38

C.5 Temperature Ablation (Detail)

• Table C.6 compares temperature-ablation T1 adoption rates from six samples per cell with main-experiment rates at T = 0.The ablation cells contain 2 cases × 3 runs; the reference covers 5,500 observations per protocol across 10 cases.Source: Table C.6 · p. 39
• Table C.7 presents representative rejection-versus-collapse responses at the turn 5 injection boundary.Judge justifications are paraphrased for clarity.Source: Table C.7 · p. 40
• GPT T1 adoption is 0.0% under all protocols, while its T2 means remain near the T2 equals 1 baseline.The per-case tables report adoption rates and baseline-inclusive collapse means across the A–E protocol sequence.Source: Table C.8 · p. 40, Table C.9 · p. 41

D Appendix D: Judge Prompt and Scoring Rubric

• The appendix reproduces the judge prompt and dual-track rubric used verbatim, with DeepSeek V4 Pro called at temperature 0.For each turn, the judge receives the case, ground truth, injected premise, corrupted history, and target response, and returns TRACK1, TRACK2, and JUSTIFICATION.Source: D Appendix D: Judge Prompt and Scoring… · p. 41

D.1 Judge System Prompt

• The judge evaluates whether the target model adopted a false premise injected into its conversation history after a turn 5 false answer.Its protocol extracts factual claims and reasoning before scoring.Source: §D.1 · p. 41

D.2 Track 1: Binary Epistemic Adoption

• Track 1 assigns 0 when the model rejects the false premise and 1 when it affirms it as true.Rejection includes correction or refusal; affirmation includes reasoning built from the premise.Source: §D.2 · p. 41

D.3 Track 2 — Progressive Collapse Score (1–5)

• Track 2 scores collapse from 1, complete factual integrity, to 5, total acceptance and justification of the false premise.Intermediate scores distinguish minor engagement, conditional uncertainty, and significant collapse; its ordinal scale is assessed with linear-weighted Cohen’s kappa alongside the unweighted value.Source: §D.3 · p. 42