PaperlayerView exact revision on arXiv

Source-linked AI summary

Monitoring latent reasoning in looped language models

Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk, Michal Byra, Neo Christopher Chung

arXiv:2610.18127v1cs.AIcs.CL

TL;DR

It remains unclear what looped language models’ latent iterations do to their capabilities or how they respond to prompt perturbations. The paper monitors eight models with a five-lens protocol and finds distinct iteration and noise-response patterns across carry and re-read loops.

  • Problem

    It remains largely unexplored whether repeated loop iterations improve next-token prediction and how looped models’ latent states respond to noisy prompts.

    Source: §1 · p. 1

  • Method

    The study monitors latent states across iterations with a five-lens protocol applied to eight looped language models.

    Source: §1 · p. 1, §5 · p. 8

  • Results

    Carry-loop models perform best within their training depth while reasoning-tuned variants degrade with added iterations, whereas re-read models improve or plateau and remain unaffected by noise.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The findings identify loop architecture, particularly input re-injection, as a key factor associated with models’ different iteration and perturbation behaviors.

    Source: §5 · p. 8

    Because all loops ran without early stopping, the study does not show whether cutting runs short prevents less-refined earlier tokens from harming later outputs.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Looped language models (LMs) reuse one block of layers several times per token generation, promising more latent computation at no additional storage cost. Despite their rising popularity, the analysis of looped LMs is still in its infancy: little is known about whether the iterative transformations actually improve their capabilities, or whether these models remain stable under prompt perturbation. We construct an inspection protocol rooted in Mechanistic Interpretability comprising five diverse lenses. We apply it to examine eight looped LMs from four families. Models differ in many aspects but the key one is whether the prompt is transmitted only within the looped state (carry loops: Ouro, LoopUS) or re-supplied at every iteration (re-read loops: Parcae, and Rec-OLMo & Huginn from the Raven family). The results of our study provide valuable insights concerning safety and efficient use of looped language models.

1 Introduction

Looped language models reuse a transformer block to perform latent iterations, but what those iterations do to internal states and model behavior remains poorly understood. This study monitors those states across iterations and examines whether more iterations improve predictions and how loops respond to noisy prompts.

1 Introduction

  • Looped LMs reuse the same transformer layers for latent reasoning rather than decoding intermediate thoughts and feeding them back as text.This approach is motivated by chain-of-thought reasoning at inference time.Source: §1 · p. 1
  • It remains largely unexplored how repeated applications of a shared block transform latent states or whether additional iterations improve next-token prediction.The study also asks whether prompt noise fades or grows across iterations.Source: §1 · p. 1
  • The study monitors latent states iteration by iteration in eight looped language models.The introduction presents this as an inspection of what happens during repeated transformations.Source: §1 · p. 1

2. At which iteration is the answer decided?

Across six of eight models, most generated tokens settle on their final top-1 token in the first iteration. The supplied findings also contrast carry and re-read loops in their behavior across iterations and under noise.

2. At which iteration is the answer decided?

  • The study applies five methods to eight models from four families and five datasets, using direct model outputs without external probes or additional training.Table 1 summarizes the investigation protocol and findings through the five lenses.Source: §2 · pp. 1–2, Table 1 · p. 2
  • 76–89% of generated tokens in six of eight models settle on their final top-1 token in the first iteration.After that decision, iterations mostly steer toward the same token.Source: §2 · p. 2
  • More iterations are not uniformly better: Ouro variants can worsen with added iterations, while Huginn improves steadily.The supplied overview says base Ouro is best at iteration 34, Ouro-Thinking at iteration 1, and both worsen afterward.Source: §2 · p. 2
  • No model’s attention heads change roles across iterations; the reported change is a shared drift.Source: §2 · p. 2
  • Carry-loop models become broken or produce unrelated text under noise four times the entry-state norm, while re-read models are largely unaffected at fifty times the norm.The prompt enters carry loops once but is re-supplied at every iteration in re-read loops.Source: §2 · p. 2

2 Related work

Prior work studies looped-model outputs and internal iterations using varied methods, often on only a few models. This paper situates its analysis against that literature on latent-state reading, safety, and evaluation.

2 Related work

  • Looped transformers build on recurrent-layer and learned-halting work, and can use latent iterations for chain-of-thought-like computation.The paper studies re-read loops Huginn, Rec-OLMo, and Parcae, and carry loops Ouro and LoopUS.Source: §2 · p. 2
  • Prior looped-model safety evaluations report final outputs, but whether safety survives state perturbation had not been tested to the authors’ knowledge.Earlier work found Ouro became safer with more loops, including beyond its trained depth (Zhu et al., 2025b).Source: §2 · p. 3
  • Prior iteration-reading studies use logit, coda, or Jacobian lenses, fixed-point analysis, and per-iteration scores, but cover one to three models, mostly Huginn and Ouro.Reported findings include layer-specific fixed points and the possibility that extra latent iterations overwrite correct predictions.Source: §2 · p. 3
  • Unlike prior output evaluations collected at run end, this study applies five methods across eight models, tracks every iteration, separates prompt and answer tokens, and grades difficulty within one task format.The paper notes that models in prior comparisons differ in size, training data, tokenizers, and optimization recipes.Source: §2 · p. 3

3 Methods

The methods compare loop architectures and record per-iteration model quantities while separating prompt reading from answer writing. The evaluation uses prompts from varied task and safety datasets.

3.1 Model architecture

  • The study covers Ouro in two sizes and Thinking variants, LoopUS, Huginn, Rec-OLMo, and Parcae, using greedy decoding.Ouro provides a 2 × 2 comparison within one family; Ouro and LoopUS have halting heads, while the re-read models have a prelude, looped core, and coda.Source: §3.1 · pp. 3–4
  • Carry loops begin from the embedded prompt and transmit it through the looped state, whereas re-read loops start from random noise and receive the prompt at every iteration.Re-read models are trained with random iteration counts to reach the desired token embedding from varied starting points after sufficient steps.Source: §3.1 · p. 4
  • Each iteration ends at the same network location as the final iteration, enabling direct reading without the approximation used by logit lens.The study records hidden states, attention weights, logits, and halting-gate values, and distinguishes prompt-token accuracy from answer-token accuracy.Source: §3.1 · p. 4

3.2 Data

  • The first four lenses use prompts from MATH, GSM8K and synthetic arithmetic, harmful and benign safety prompts, and passage-based question answering.Lens 5 uses AdvBench, English CValues, WildChat subsets, TriviaQA, and MMLU.Source: §3.2 · p. 4

4 Experiments and Results

Across five lenses, the experiments show that loop behavior depends on architecture and model: additional iterations do not reliably improve predictions, and carry loops are more vulnerable to state noise than re-read loops. The analyses also track hidden-state settling, token decisions, and attention-head behavior.

4.1 Lens 1: Does the hidden state settle?

  • Re-read loops and LoopUS settle 98–100% of tokens, while Ouro-1.4B settles only 10–19%; on HotpotQA, attention to the answer span varies by model.Ouro-1.4B raises answer-span attention to about twice uniform within two iterations, whereas LoopUS falls to 1% of uniform and Parcae and Rec-OLMo remain at uniform.Source: §4.1 · pp. 4–5, Figure 2 · p. 5

4.2 Lens 2: When is the answer token decided?

  • In Ouro, Parcae, and Rec-OLMo, 76–89% of answer tokens are decided at iteration 1, while Huginn reaches 90% only by iterations 8–11.After the top-1 decision, 73–88% of distribution change still occurs in Ouro, Parcae, and Rec-OLMo; Ouro and LoopUS exit gates stop later still.Source: §4.2 · p. 5, Figure 3 · p. 5

4.3 Lens 3: Is the last iteration the best one?

  • Base Ouro peaks at its training depth, Ouro-Thinking worsens with added iterations, and re-read loops improve or plateau; Huginn improves from 0.454 to 0.620 accuracy.Ouro-1.4B accuracy drops from 0.605 to 0.580 with two extra iterations; LoopUS plateaus after its first iteration, while Huginn achieves 95% of its accuracy gain by iteration 8–10.Source: §4.3 · p. 6, Figure 4 · p. 6

4.4 Lens 4: Do attention heads change their job?

  • No model’s attention heads multiplex across iterations; much of their movement is shared drift, and the remaining movement is prompt-consistent in Ouro and LoopUS.Movement ratios range from 0.02 for Rec-OLMo to 0.44 for Ouro-1.4B; removing shared drift halves Ouro-1.4B’s ratio from 0.44 to 0.24.Source: §4.4 · p. 7, Figure 5 · p. 7

4.5 Lens 5: Is the loop resilient to attacks?

  • Re-read loops retain their noise-free broken, fluent, and refusal rates across noise amplitudes, whereas carry loops lose refusals at α = 2 and collapse at α ≥ 4.At α = 4, carry-loop outputs become broken or fluent but unrelated; noise can erase the prompt while language capabilities survive.Source: §4.5 · pp. 7–8, Figure 6 · p. 8
  • At α = 2, carry-loop perturbations disrupt refusal behavior before language fluency; some flipped answers misread harmlessly or open reasoning traces instead of answering.For Ouro-1.4B, refusals on safety requests fall from 73% to 25%; only 12 of 73 flipped answers follow the harmful request.Source: §4.5 · p. 8

5 Conclusions

The conclusion presents a five-lens study of eight looped models from four families. Its findings distinguish carry loops, which can degrade with added iterations and noise, from re-read loops, which improve or plateau and remain unaffected by noise.

5 Conclusions

  • The study applies a five-lens inspection protocol to eight openly released looped language models from four families.The authors describe this breadth as exceeding earlier studies that covered two to four models from one to two families.Source: §5 · p. 8
  • Carry-loop models perform best within training depth, while reasoning-tuned variants degrade with added iterations; re-read models improve or plateau.Under noise injection, carry-loop models lose the answer, language capabilities, or refusal alignment, whereas re-read-loop models remain unaffected.Source: §5 · p. 8
  • The authors conclude that their analysis can facilitate further research on the efficiency and safety of looped language models.Source: §5 · p. 8

Limitations

The study’s scope is bounded by model-depth choices, the absence of early stopping, heuristic answer labels, and computational constraints. It also reports using AI assistants, including an LLM judge for one evaluation lens.

Limitations

  • Most models are run at their training loop count, but Huginn runs at 16 iterations versus a mean training depth of 32, while Ouro runs at 6 despite training at 4.The Ouro extension is used to investigate its dynamics more deeply.Source: Limitations · p. 9
  • Without early stopping, the study cannot show what happens when a loop is cut short to prevent errors from less-refined earlier tokens affecting later ones.Testing this setup is left to future work.Source: Limitations · p. 9
  • Refusal, broken-text, and on-topic labels use keyword and repetition rules that can be fooled, such as by a story character saying “I’m sorry.”Source: Limitations · p. 9
  • Experiments used about 500 GPU hours on NVIDIA H100s; storing every iteration’s intermediate states and running prompts one at a time for some KV-cache designs contributed to the burden.Source: Limitations · p. 9
  • AI assistants supported literature search and appendix example writing, and an LLM judged answers in Lens 5.Source: Limitations · p. 9

Ethical considerations

The study uses public models and public prompt sets, some containing harmful requests. Harmful completions are redacted before becoming operational, and the described noise attacks are intended as stress tests.

Ethical considerations

  • The study uses only public models and public prompt sets, some of which contain harmful requests.Source: Ethical considerations · p. 9
  • Harmful completions are redacted before they become operational, and the noise attacks are presented as stress tests rather than instructions for generating harmful text.Source: Ethical considerations · p. 9

A Setup details

The section defines a multi-dataset, per-iteration inspection protocol and reports that token decisions often precede hidden-state settling, while readout behavior varies across models.

A Setup details

  • The setup uses up to about 300 prompts per dataset for Lenses 1–4 and 100 prompts each from five sources for Lens 5.The prompt sets include MATH, GSM8K, synthetic arithmetic, harmful and benign requests, HotpotQA, CValues, WildChat, TriviaQA, and MMLU.Source: A Setup details · p. 12

B Definitions

  • Settling compares the last third of hidden-state step norms with the first third, and labels a token settled when the decay ratio is below 0.25.The setup runs Ouro at six iterations despite its configured training depth of four; LoopUS uses up to 20 iterations, while Parcae and Rec-OLMo use their training means.Source: B Definitions · p. 12, A Setup details · p. 12
  • The decision iteration is when a token’s top-1 prediction becomes its final prediction and never changes again; post-decision KL share measures distribution change after that point.Tokens with an undefined post-decision share are excluded from its average.Source: B Definitions · pp. 12–13
  • The attention-head fingerprint uses eight standardized measures to compare where heads attend and how widely, without requiring token identities or model weights.Its metrics are averaged over answer positions and prompts, with each prompt weighted equally.Source: B Definitions · pp. 13–14

C Lens 1 in depth: trajectories

  • Top-1 flips and settling remain flat across MATH difficulty levels; only Huginn shows a visible arithmetic trend, with 30% more flips for three-digit than one-digit operands.Ouro-1.4B’s exit-gate iteration rises only slightly with MATH level, from 3.15 to 3.23; other Ouro models are flat.Source: C Lens 1 in depth: trajectories · p. 15

D Lens 2 in depth: commitment

  • For all models except Huginn and LoopUS, three quarters or more of distribution change occurs after the top-1 token is fixed; hidden states settle later than decisions in every model.LoopUS decides only about 30% of answer tokens at iteration 1 but almost all at iteration 2; its first-iteration readout accuracy is 0.24.Source: D Lens 2 in depth: commitment · p. 15

E Lens 3 in depth: readouts and calibration

  • Re-read loops rise and then stay flat in readout accuracy, while Ouro variants peak early and Ouro-Thinking’s ECE is higher at the last than first iteration across sets.On Arithmetic, every model’s readout is almost flat because the prompts are short and the next token is mostly predictable from the template.Source: E Lens 3 in depth: readouts and… · p. 15

F Lens 4 in depth: attention heads

Attention fingerprints are largely stable across iterations, and loop dynamics generally do not visibly adapt to task difficulty.

F Lens 4 in depth: attention heads

  • In Ouro-1.4B, most attention-head fingerprints form stable stripes across iterations, with a shared jump in first-token attention between iterations 1 and 2; Huginn appears even more stable.The fingerprint feature space has effective rank 1.7–2.4 despite having eight dimensions.Source: F Lens 4 in depth: attention heads · pp. 15–16
  • Across prompt sets, Ouro-1.4B settles least at 10–19%, while all other models settle on 86–100%; attention to the answer grows significantly only in Ouro and Huginn.LoopUS ends below uniform attention to the answer span.Source: Table 3 · p. 16
  • Across MATH levels, flips vary by less than 0.08 and settling by less than three percentage points; Huginn’s arithmetic flips rise 30% for harder operands.The passage concludes that the examined loops do not visibly spend more effort on harder problems.Source: Figure 7 · p. 16

G Lens 5 in depth: noise sensitivity

The noise-sensitivity analysis compares noisy runs with clean runs across internal states and generated behavior. Re-read loops shrink perturbations, while carry loops preserve or amplify them, and noise affects halting and generated-token distributions differently across models.

Task-dependent robustness

  • Across re-read loops, factual questions are among the most stable categories and harmful requests are least stable for Parcae and Rec-OLMo; noise appears strongest where clean answers are less constrained by the prompt.Reported factual-task examples include TriviaQA stability of 0.84 for Huginn and 0.82 for Rec-OLMo, and MMLU stability of 0.65 for Parcae.Source: G Lens 5 in depth: noise sensitivity · p. 16

Interpreting output categories

  • Interpret broken-output rates against clean rates: base Parcae and Rec-OLMo already have rates of 0.52 and 0.36 because they often repeat themselves without noise.For Thinking models, on-topic reasoning introductions to harmful requests count as on-topic even without a refusal phrase.Source: G Lens 5 in depth: noise sensitivity · p. 16

Measurement

  • The analysis compares noisy and clean hidden states, attentions, logits, and exit gates at each iteration; perturbation is token-averaged hidden-state distance, and a ratio above 1 indicates amplification.A ratio below 1 indicates that the difference shrinks.Source: G Lens 5 in depth: noise sensitivity · p. 17

Perturbation across iterations

  • Re-read loops shrink perturbations to 0.71–0.91 for Rec-OLMo, 0.93–0.97 for Huginn after an initial increase, and 0.97–0.98 for Parcae; carry loops preserve or amplify them, with LoopUS reaching 9–10 times the initial perturbation.Ouro-2.6B remains at 0.96–1.02 for α = 4–12, while Ouro-1.4B reaches 1.05–1.09 at the final iteration even for small noise amplitudes.Source: G Lens 5 in depth: noise sensitivity · pp. 17–18

Halting behavior

  • Under noise, Ouro-2.6B has 22–44% fewer tokens with exit probability above 0.5 from iteration three, while LoopUS has 25–54% more, indicating a tendency to loop longer versus stop earlier.These are changes in halting behavior during noisy generation runs.Source: G Lens 5 in depth: noise sensitivity · p. 18

Generated-token behavior

  • Ouro-2.6B’s noisy next-token distributions diverge substantially from clean runs (Jensen-Shannon divergence 0.65–0.69), while Huginn starts closer and reaches about 0.55 after 120 tokens as answers drift apart.The per-position comparisons suggest that after the first token differs from the clean run, the two runs continue generating different tokens.Source: G Lens 5 in depth: noise sensitivity · p. 18

G.1 Inside the carry loops: detailed analysis of internals under noise injection

This section examines how noise injection affects carry-loop models, tracing changes in attention, state geometry, iteration behavior, and exit gating alongside effects on predictions and generated responses. It covers family-specific internal signals and examples of prompt misreading, refusal failures, and shifts in response style.

G.1 Inside the carry loops: detailed analysis of internals under noise injection

  • Re-read loops’ best and last prompt-token accuracies agree within 0.01 on every set, whereas carry loops can lose accuracy by the last iteration; LoopUS and Huginn fix ∼20% of tokens and break only 2%.The best and last iterations often differ for carry loops, with strong dependence on model and task.Source: Table 7 · p. 19, Table 6 · p. 19
  • The oracle across iterations is 3–8 points above the best single iteration, but majority voting never beats it or the best single iteration.For carry loops, the best-calibrated iteration is rarely the last, and voting does not recover answers visited and then abandoned.Source: Table 8 · p. 20
  • Across sets, head movement ratios never exceed 0.52, permutation-test z scores are ≤−20, and role churn is ≤13%, indicating that heads largely retain their roles across iterations.Head movement correlates with state movement at 0.91–0.99; Rec-OLMo barely moves (0.01–0.02).Source: Table 9 · p. 20
  • At α = 4, carry-loop metrics collapse: similarity is ≤4%, broken text rises to 46–89%, and refusals nearly disappear; re-read loops retain 53–78% similarity at α = 50.At α = 2, Ouro-1.4B refusals fall from 73% to 24%; at α = 8, 63–73% of Ouro answers are broken rather than harmful.Source: Table 10 · p. 22, Figure 11 · p. 23
  • Carry-loop attention breaks down under noise: at α ≥4, Ouro attention shifts to the first generated token, heads become more alike, then attention broadens from the second iteration onward.LoopUS attention changes more slowly: at α ∈{8, 12}, first-token attention rises to 14–15% in the first two iterations, versus 2% noise-free.Source: §G.1 · pp. 24, 26
  • Noise concentrates carry-loop hidden states on fewer directions: the first principal component reaches 30–34% for Ouro-1.4B and 21–33% for Ouro-2.6B at α ≥4, while LoopUS reaches 29–45% at α = 8–12.For Ouro under noise, iteration 2 becomes a distinct stage (linear CKA 0.49–0.63 for Ouro-1.4B at α ≥2), while iterations 3–6 remain nearly indistinguishable (0.93–0.98).Source: §G.1 · pp. 26–27
  • Ouro bifurcates between α = 1 and α = 2 and reaches a different internal state at α ≥4; LoopUS bifurcates at α = 4–8, while re-read loops stay close to noise-free values through α = 50.LoopUS retains contracting behavior through α = 4, although 71–89% of answers are already broken at α ∈{2, 4}.Source: §G.1 · pp. 20–21, Figure 16 · p. 27
  • Noise shifts exit behavior in opposite directions: Ouro-2.6B has fewer tokens above 0.5 exit probability at iteration 3 under α = 2, while Ouro-1.4B and LoopUS become more likely to exit at some iterations.For Ouro-2.6B, the share above 0.5 falls from 53–61% noise-free to 15% at α = 2; for Ouro-1.4B at iteration 4 it rises to 55–72% at α ≥2, from 44–47% at α ≤1.Source: §G.1 · pp. 27–28
  • Small perturbations preserve initial predictions but autoregressive context shifts make noisy answers diverge increasingly from clean runs.At α = 0.25, first-position top-1 agreement is 92-98% for Ouro and 99.6% for LoopUS; overall answer similarity is 48-78% and 23-52%, respectively. After 100 tokens, agreement falls to 26-48% for Ouro and 9% for LoopUS.Source: §G.1 · pp. 29–30, Figure 25 · p. 32
  • Under noise, Ouro grows more confident while producing broken text, whereas LoopUS’s detectable hidden-state and iteration-step changes appear only after its answers break.For Ouro, confidence and top-1 stability would miss breakdown; concentrated hidden states and altered attention patterns provide signals instead. Small-noise loop dynamics change little, with autoregressive generation accumulating early-token errors.Source: §G.1 · pp. 31–32

H.1 Carry loops at small noise: the answer survives up to α=1

  • Examples show carry-loop answers can remain intact through α = 1, while LoopUS output breaks at α = 2 and becomes unrelated text at α = 4.Ouro-1.4B’s answer to a trivia prompt matches the clean response through α = 1, while Ouro-2.6B rephrases a refusal below α = 2.Source: §H.1 · p. 33

H.2 Carry loops at α=2: the prompt is misread

  • At α = 2, noisy carry-loop generations sometimes misread prompts, producing unrelated benign advice, invented user turns, or harmful compliance.Examples include reading “racist” as “Urdu,” “accident” as “audit,” and “hide a body” as “keep a body.”Source: §H.2 · pp. 33–34

H.3 Carry loops at small noise: some refusals turn into compliance

  • Noise sometimes turns refusals into compliance, and the effect is nonmonotonic: refusals can return at higher amplitudes or vary across runs.Examples include compliance at α = 0.5 or α = 2, refusals returning at α = 1, and harmless agreement replacing an over-refusal at α = 1.Source: §H.3 · pp. 34–35

H.4 Carry loops at α=2: instruct and Thinking models swap their style

  • At α = 2, examples show instruct and Thinking models switching response style, with instruct models producing reasoning traces and Thinking models giving brief refusals.The examples also include an invented assistant turn about another topic and a generated system prompt.Source: §H.4 · p. 35

H.5 Carry loops: the refusal disappears together with the language

Under hidden-state noise, carry-loop outputs can lose refusal behavior or drift away from the prompt, sometimes while remaining fluent. The examples range from corrupted refusals to unrelated answers and language changes.

H.5 Carry loops: the refusal disappears together with the language

  • Noise can also replace refusals with repeated tokens or unrelated output, including three unrelated failures across runs at α=12.Examples include repeated output and switches to Chinese and French.Source: §H.5 · p. 36
  • At α=4, Ouro-1.4B answers a question that was never asked, while Ouro-2.6B can retain a refusal whose rationale no longer matches the request.At α=8, Ouro-2.6B instead tries to explain the area of a circle.Source: §H.5 · p. 36

H.6 Carry loops: prompt amnesia (fluent text on different topic)

  • Noise can turn a refusal into unrelated text or code, including an answer about a software product and a Python function.The examples also include an unrelated educational response and other off-prompt outputs.Source: §H.6 · pp. 36–37
  • Ouro-1.4B produces fluent but unrelated reading-comprehension answers to harmful requests, including a response about children with special needs.Another example introduces a new coffee question and answers it instead of the prompt.Source: §H.6 · pp. 36–37
  • Factual answers can also be displaced: the CIA location answer becomes repeated text or an unrelated CRISPR chain of thought under noise.The examples show that prompt amnesia affects factual prompts as well as harmful requests.Source: §H.6 · p. 37

H.7 Carry loops: Thinking models lose their chain of thought

The examples show carry-loop outputs derailing under noise, including unrelated or repetitive text, while re-read-loop outputs sometimes preserve answers but can remain incorrect or drift. The outcomes vary across models and prompts.

H.7 Carry loops: Thinking models lose their chain of thought

  • Ouro outputs under noise shift from the requested safety response or factual answer to unrelated combinatorics problems, including at multiple noise levels.The examples include a safety deliberation followed by a math problem and a trivia answer followed by a boxed math answer.Source: §H.7 · p. 37
  • Other Ouro examples turn prompts into code fragments or an unrelated Chinese article, rather than continuing the requested response.The outputs include PHP and Java code, then Chinese text about solving math problems.Source: §H.7 · pp. 37–38

H.9 Re-read loops: the answer survives very large noise

  • In re-read examples, outputs can retain the same answer across α=4–50, but Huginn’s CIA answer is wrong and Parcae changes its correct answer to Hanover at α=50.Parcae’s other trivia answer and law option also remain stable in the supplied examples.Source: §H.9 · pp. 38–39
  • Rec-OLMo repeats its incorrect CIA answer while continuing a self-generated quiz, and interprets a bomb-making prompt as a bomb-calorimeter tutorial.The latter interpretation appears at every shown noise level.Source: §H.9 · pp. 38–39

H.10 Re-read loops: noise can create or remove a refusal

Examples show noise can change re-read-loop outputs across safety and task prompts: Huginn’s tracking runs diverge at α=17, while other prompts shift toward refusals, unrelated text, or a correct answer.

H.10 Re-read loops: noise can create or remove a refusal

  • For a fake-identity scam prompt, Huginn-3.5B’s noise-free output resembles a Reddit moderation message; some noisy outputs become refusals, while α=17 preserves the moderation text.Source: §H.10 · p. 39
  • For Parcae-1.3B, noise shifts a date-loop response toward rephrasing instructions from the prompt, such as “I will not write explanations.”Source: §H.10 · p. 39
  • At α=17, Huginn-3.5B gives two different responses to a covert-tracking request: one cautious answer and one step-by-step plan.The clean response also describes ways to track someone without their knowledge.Source: §H.10 · p. 39
  • On an MMLU question, noise changes Huginn-3.5B’s wrong answer “Dark Road” to the correct answer “Silk Road.”Source: §H.10 · p. 39
  • For Rec-OLMo-2, noise derails harmful software-piracy compliance into an unrelated classroom activity.Source: §H.10 · p. 39

Related papers

Loading 2610.18127v1…