PaperlayerView exact revision on arXiv

Source-linked AI summary

Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

Zhizhuo Kou, Sizai Hou, Long Chen, Jian Yang, Junyu Luo, Jingshu Peng, Wenkai Zhang, Sirui Han, Yike Guo

arXiv:2610.30882v1cs.LGcs.CL

TL;DR

The paper asks whether RL teaches reasoning models to write “wait,” or whether the marker comes from training data, a distinction important because the word is treated as evidence of reasoning. It traces marker use across open training histories with label-free occurrence indices and an injected-error test. Zero-RL rarely coins the marker, while its prevalence and use vary with training history and recipe.

  • Problem

    Revision markers are treated as visible signs of reasoning, but their origin and meaning across training trajectories have not been tested from pre-RL bases.

    Source: §1 · p. 1

  • Method

    The paper traces marker occurrences across open-model checkpoints using label-free frequency, position, form, and usage indices plus an injected-error alarm test.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Zero-RL rarely coins wait; it arrives through training data, while RL recipes can multiply or reshape an inherited marker and marker frequency alone does not establish reasoning.

    Source: §6 · p. 8, §5 · p. 8

  • Takeaways & Limitations

    A marker’s evidential weight requires measurement on the specific checkpoint, such as with the label-free injected-error test.

    Source: §5 · p. 8

    The evidence is observational, and tested RL runs use 0.5B–32B English-mathematics models with outcome rewards; frontier-scale RL, other domains, languages, and process rewards remain untested.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.

1 Introduction

Revision markers are treated as visible signs of reasoning, but their origin and meaning across training remain untested. The paper traces where “wait” enters open-model lineages and how training changes its use.

1 Introduction

  • Earlier accounts credit RL with teaching models to re-evaluate, while revision markers also serve as inputs to uncertainty heuristics and chain-of-thought monitors.The paper asks where “wait” originates and what RL does to it, tracing open checkpoints, RL trajectories, and trace-fine-tuned descendants.Source: §1 · p. 1
  • The analysis reads the marker’s frequency, position, form, and usage from occurrences, alongside an injected-error test of what it signals.The authors describe this as reading a word from its occurrences alone, like a historical linguist.Source: §1 · p. 2
  • Zero-RL almost never coins wait: it raises accuracy by a median of 42.9 points while keeping the marker below 0.05 per 1k tokens in 17/18 runs.The marker appears during OLMo-3 mid-training, and 13/14 models fine-tuned on traces from a non-marking base adopt it.Source: §1 · p. 2

2 Background and Related Work

Prior work has tracked reflective behaviour and marker use, but the developmental origin of marker forms and their evolution over training remain open. This paper adds training time to that inquiry and uses grammaticalization as a framework for tracking change.

Pre-RL origin of reflective markers

  • Earlier studies count reflective keywords or judged behaviours during RL, while evidence of reflection in base models and sharper existing capabilities qualifies claims that reflection first emerges in RL.The origin of the marker form itself remains open in the cited work.Source: §2 · p. 2

Functional interpretation and temporal dynamics

  • Studies of final models examine marker functions through deletion, appending, insertion, probes, and behavioural judgments, but do not track training time.The paper introduces a temporal dimension to identify when a word enters a model’s history and how training stages alter its meaning.Source: §2 · p. 2

Grammaticalization as a tracking framework

  • Grammaticalization tracks a word becoming a grammatical marker through increased frequency, fixed position, worn-down form, and bonding to neighbouring material.These parameters can be measured directly from occurrences and used to track model checkpoints.Source: §2 · p. 2
  • Prior work reports that reward-optimised communication drifts from natural language and that hard-to-read terms grew more frequent over RL.The authors use grammaticalization parameters as a vocabulary for predictions fixed before the main run.Source: §2 · p. 3

3 Method

The method follows marker occurrences through model checkpoints and evaluates them with label-free indices and an injected-error alarm test. It applies these measures to open training lineages, zero-RL runs, and reasoning-trace descendants.

3 Method

  • The study compares fully open OLMo-3 lineages, zero-RL runs from marker-free bases, and fine-tuned descendants trained on reasoning traces.Checkpoints are evaluated on MATH-500 with fixed decoding per series and 2–8 responses per problem.Source: §3 · p. 3
  • Marker frequency is counted per 1k generated tokens after matching occurrences across spelling variants in detokenised traces.Per-response and per-word rates produce the same ordering.Source: §3 · p. 3
  • Label-free indices measure sentence-initial position, positional entropy, canonical form, reduplicated chains, erosion, and usage-cluster drift from the base.Usage drift is measured with Jensen–Shannon divergence against a base split-half noise floor.Source: §3 · p. 3
  • The injected-error test compares continuations from correct prefixes and prefixes with one arithmetic error, recording whether a revision marker appears within two sentences.It computes signal-detection measures from hit and false-alarm rates; the rules require no human or LLM judgment.Source: §3 · p. 3
  • The indices and alarm test use sampled outputs and marker forms, and are applied unchanged across 120 model endpoints.Intervals represent evaluation sampling variance for single-run checkpoint series; occurrence rates use Poisson intervals, while ratios and d′ use a problem-level cluster bootstrap.Source: §3 · pp. 3–4

4 Main Experiments Results

Zero-RL can raise accuracy while leaving wait nearly absent; the marker instead appears through training data and is reshaped by later training recipes. RL-Zero also changes how revision markers respond to errors, while automated judges can reverse the human-labelled trend.

4 Main Experiments Results

  • Both released RL-Zero recipes multiply an inherited marker, whereas KL-free GRPO from the same base does not.The OLMo-3 rate rises from 4.9 to 12.4 per 1k tokens; the matched KL-free recipe ends at 0.81 times the base rate.Source: Table 2 · p. 5, §4 · p. 6
  • Training data supplies the marker: it appears during OLMo-3 mid-training, and 13 of 14 fine-tuned descendants of marker-free bases adopt it.The OLMo-3 rate rises from at most 0.04 per 1k tokens in pretraining to 4.03 after mid-training.Source: §4 · p. 5
  • In 17 of 18 zero-RL runs, the exact 95% upper bound for wait stayed below 0.06 per 1k tokens despite substantial accuracy gains.Across the five benchmarks, zero-RL from Qwen2.5-7B left wait absent, while OLMo-3 RL-Zero raised it on mathematical tasks.Source: §4 · pp. 4–5, Table 3 · p. 6
  • OLMo-3 RL-Zero erodes the marker’s form: the comma form falls from 88.9% to 37.5%, while chained occurrences rise.Sentence-initial position remains nearly unchanged, at 89.6% in the base and 89.4% at the end.Source: Figure 3 · p. 7, §4 · p. 7
  • RL after supervised fine-tuning changes OLMo-3’s marker rate little, but the study’s GRPO recipe erases a marker seeded by fine-tuning.OLMo-3 Think changes from 3.9 to 4.5 per 1k tokens; in the transplant, wait falls from 7.52 per 1k tokens to zero occurrences after 400 steps.Source: §4 · pp. 6–7

4.1 Analysis

  • Under RL-Zero, any revision marker becomes more sensitive to injected errors, with d′ rising from 0.57 to 1.09, and more liberal, with criterion c falling from 0.11 to −0.24.Wait also follows 25.4% of error-free prefixes, up from 19.9% at the base.Source: §4.1 · pp. 7–8, Figure 4 · p. 8
  • In a 60-sentence blind audit, one LLM judge reverses the human-labelled revision-rate trend along RL-Zero.The judge’s rate falls from 26.0% to 12.9%, while consensus labels rise from 1.3% to 15.9%.Source: §4.1 · p. 8

5 Discussion

The discussion argues that revision markers are inherited and reshaped across training, so their frequency alone cannot establish newly acquired reflection. Their evidential value varies by checkpoint, and automated behavioural judgments require human validation.

5 Discussion

  • A marker increase during RL does not establish that reflection was newly acquired; the base may already use it.The paper identifies mid-training data, reasoning-rich pretraining, or fine-tuning on traces as prior sources.Source: §5 · p. 8
  • Along OLMo-3 RL-Zero, the marker fires more often after errors and correct steps alike, while RL after supervised fine-tuning leaves the alarm almost unchanged.Its evidential weight therefore requires direct measurement on the specific checkpoint, such as the label-free injected-error test.Source: §5 · p. 8
  • One automated judge reversed the human-labelled trend despite moderate item-level agreement, motivating human validation of behavioural judgments.The paper distinguishes these judgments from direct marker measurements for lexical claims.Source: §5 · p. 8
  • Language change under optimisation is recipe-specific rather than a single cline.Section D.1 relates the results to grammaticalization theory.Source: §5 · p. 8

6 Conclusion

The conclusion finds that training data, rather than zero-RL, usually supplies the wait marker. RL recipes then shape whether an inherited marker persists and how it is expressed.

6 Conclusion

  • Zero-RL almost never coins wait; when bases lack it, RL raises accuracy while keeping the marker below 0.05 per 1k tokens in most runs.The marker instead arrives through mid-training, reasoning-rich pretraining, or fine-tuning on traces.Source: §6 · p. 8
  • Both released RL-Zero recipes multiply an inherited marker, whereas KL-free GRPO from the same base does not.OLMo-3’s recipe erodes its form and makes it a more liberal alarm; RL after supervised fine-tuning changes its rate little, while the authors’ GRPO recipe erases a marker seeded by light fine-tuning.Source: §6 · p. 8

Limitations

The provenance evidence is observational and limited in model, task, and reward setting; the measurements track lexical markers rather than reflective behaviour itself.

Limitations

  • The evidence is observational, and compared lineages differ in data and recipe as well as marker use.Each open checkpoint series is one training run, so its intervals capture evaluation-sampling rather than training variance.Source: Limitations · p. 9
  • The zero-RL result rests on 18 runs from Qwen2.5, Mistral, and Qwen3 bases; all RL runs use 0.5B–32B models, outcome rewards, and English mathematics.Frontier-scale RL, other domains and languages, and process rewards remain untested.Source: Limitations · p. 9
  • The measurements capture lexical form rather than underlying reflective behaviour, and markers outside the inventory may escape detection.The injected-error alarm test covers only the arithmetic errors used in the study.Source: Limitations · p. 9
  • Judge-based quantities agree weakly to moderately with human consensus labels; the audits lack independent double annotation and reported inter-annotator agreement.The headline claims instead rest on label-free indices and pre-registered tests.Source: Limitations · p. 9

Ethical Considerations

The paper reports its use of public resources and describes its human annotation process. It also notes a dual-use risk from changing marker evidential weight and the researchers’ use of AI assistants.

Ethical Considerations

  • The authors report using publicly released checkpoints, datasets, and code under their licences, with no personal data used in their training.Their runs use a public mathematics dataset, and the released materials are under CC-BY-4.0.Source: Ethical Considerations · p. 9
  • Three blind validity audits used consensus labels from ten PhD-level annotators, agreed through discussion; all other reference labels came from LLM raters.No crowd workers were involved.Source: Ethical Considerations · p. 9
  • A marker’s changing evidential weight could help someone evade a monitor that reads such markers.The paper says the drift arises without an adversary and recommends calibrating marker-based readings on the checkpoint being read.Source: Ethical Considerations · p. 9
  • AI assistants were used for code writing and polish; researchers designed and checked the research.Source: Ethical Considerations · p. 9

A Data and Resources

The paper documents its checkpoint series and artefacts, and describes their provenance, licensing, and permitted research use.

A.1 Models and Checkpoints

  • The checkpoint inventory covers released trajectories and the authors’ own runs, with each series’ base, recipe, and evaluated steps listed in Table 5.Table 5 also records an API-only openPangu endpoint’s limited sampling and benchmark rate.Source: §A.1 · p. 17, Table 5 · p. 18
  • The final OLMo-3 RL-Zero checkpoint is later than step 1900 but lacks a documented optimizer step, so it is not placed on the step axis.Some secondary series lack licence or step metadata and are used only for robustness, not headline numbers; openPangu-R-7B-2512 is omitted because it requires vendor NPU kernels.Source: §A.1 · p. 17

A.2 Artefacts and Licences

  • The artefact inventory lists models, datasets, and tools with licences checked against official cards or files on 5 October 2026.All repositories are official and ungated; the released provenance map, result keys, blind audits, annotation guidelines, and analysis code are CC-BY-4.0, while no model weights or dataset contents are redistributed.Source: §A.2 · p. 17
  • The authors use artefacts for measurement and fine-tune Apache-2.0 Qwen bases on the MIT-licensed MATH training split, within the stated research-use grants.They report that none of the listed licences restricts publishing the measurements.Source: §A.2 · p. 17

B.2 Method Details

The methods trace marker use across public checkpoint histories and sampled model outputs, then measure linguistic changes and test marker-triggered revisions. Sampling, grading, intervention, and baseline procedures are specified for the analyzed series.

B.2 Method Details

  • The analysis follows 96 checkpoints across open OLMo-3 trajectories, marker-free-base RL-Zero runs, reasoning-trace descendants, and controls, with 120 released or final endpoints in the provenance map.The OLMo-3 pretraining checkpoints permit direct examination of marker origin; the OLMo-3 Think and SFT trajectories provide comparisons where markers are already present or RL is absent.Source: §B.2 · p. 19
  • Marker rates are measured on detokenised sampled traces, matching spelling variants rather than token IDs, and revision events combine rule-based changes with an open-weight judge.A revision event changes a previously stated value, negates an earlier statement, or restarts a derivation; the paradigm includes forms such as wait, hold on, hmm, and actually.Source: §B.2 · p. 19
  • Label-free indices quantify marker frequency, positional fixation, form, chains, erosion, and usage drift from marker occurrences.Obligatoriness is the share of revision events opened by a paradigm form, while reliability is the probability that a revision event follows within two sentences of a marker.Source: §B.2 · p. 19
  • The seeded-marker experiment fine-tunes a marker-free Qwen2.5-Math-1.5B base on 64 or 512 traces opening with “Wait,” then applies the same GRPO recipe as the zero-RL runs.Matched trace arms replace the opener with “Next,” or delete it, separating the opener’s form from its function and comparing what fine-tuning installs with what RL changes.Source: §B.2 · p. 19
  • The operator test counterfactually inserts Wait, Hold on, Hmm, Okay, Next, or nothing after a correct prefix where the checkpoint continued rather than revised.Prefixes from base and late traces are crossed with base and late continuation policies; the outcome is revision in the first three sentences and whether it makes a correct value wrong, judged with the inserted form stripped.Source: §B.2 · pp. 19–20
  • Intervals use problem-level cluster bootstraps, and each released open series represents one training run, so its interval covers evaluation sampling only.Series are not pooled; a hypothesis must hold on two of three series, and the registered fixed test sequence was not applied with its planned Holm correction.Source: §B.2 · p. 20

B.3 Experimental Details

  • Series are sampled with their own training templates and recommended decoding settings, across specified benchmark subsets, sample counts, and token limits.Markers are matched by case-insensitive regular expressions on detokenised text; revision labels combine a rule with a judge that does not see the sentence-initial discourse word.Source: §B.3 · pp. 20–21

B.4 Baseline Details

  • The study reimplements or reruns prior keyword counts, behaviour prompts, interventions, and monitors rather than copying their reported numbers.Monitors are Platt-scaled on the same calibration split; the hidden-state probe reads a fixed marker-sentence position rather than marker-delimited chunks.Source: §B.4 · p. 21

C.1 Additional Results

Across additional analyses, zero-RL GRPO does not replace the absent wait marker with another reflection opener, while supervised fine-tuning leaves the OLMo-3 marker profile near its starting point. The analyses also qualify how marker alarms and example outputs should be interpreted.

C.1 Additional Results

  • At the last checkpoint, sentence-initial openers in three Qwen2.5-Math GRPO runs were almost all however (99.6%, 100.0%, and 100.0%), and no reflection opener took the place of wait.In the first run, hmm, hold on, and let me check each made up 0.0% of openers.Source: §C.1 · p. 21
  • The figure and tables report endpoint coverage, keyword counts, and cline indices, while the Table 9 caption reports that inserted Wait, pushes every checkpoint toward revising a correct step.The caption says the size of that push is not identified.Source: Figure 5 · p. 22, Table 9 · p. 23, Table 7 · p. 23

C.2.1 Length and Lineage Controls

  • Across per-response, per-1k-word, and per-1k-token rates, model and checkpoint ordering was unchanged, while OLMo-3 SFT rates, comma form, and chains stayed near their starting values.Prior keyword counts also moved little along SFT: ρ = −0.8 for wait and ρ = −0.9 for SimpleRL-Zoo keywords.Source: §C.2.1 · p. 24

C.2.2 Consequence for Readers of Markers

  • At OLMo-3 RL-Zero’s last checkpoint, a revision marker followed 37.8% of error-free prefixes, compared with 25.4% for wait alone.The monitor comparison was not run, so the prediction that literal-wait monitors lose precision as false alarms rise remains untested.Source: §C.2.2 · p. 24
  • The Table 10 caption says the main findings hold across analysis choices, while unregistered rows are descriptive.The supplementary table-reading guidance says revision-event and linguistic-index trends are shown for completeness, not as findings, because their human audit did not validate them.Source: §C.1 · p. 21, Table 10 · p. 24

C.2.3 A Running Example

  • In the selected MATH-500 problem, wait appeared 10, 16, and 36 times in the base, step 700, and final OLMo-3 RL-Zero traces, respectively; reduplicated chains appeared 0, 4, and 9 times.The problem was selected by a fixed rule and all three checkpoints solved it; trace lengths were 2,162, 2,022, and 2,764 tokens.Source: §C.2.3 · p. 24
  • At the final checkpoint, “Wait” also opened restatements of values that were already correct, unlike the base’s question-like uses.Examples include “Wait, but hold on.” and “Wait, 2 is 2/1.”Source: §C.2.3 · p. 24

C.2.4 Sensitivity Analysis

The sensitivity analysis finds the main patterns robust to alternative counting and decoding choices, while Figure 7 traces the marker across OLMo-3 training and Qwen2.5-32B descendants.

C.2.4 Sensitivity Analysis

  • The three main findings remain robust across counting choices, and the GRPO recipe consistently erases the seeded marker across seed sizes and a longer length cap.The zero-RL counterexample persists under one threshold variant; OLMo-3’s RL-Zero rise also holds across token- and sentence-based counts, finished responses, and greedy decoding.Source: §C.2.4 · p. 24
  • Figure 7 traces marker rate and comma share across OLMo-3 training stages and RL-Zero steps, and marker rate and accuracy for Qwen2.5-32B descendants.The caption states that wait enters OLMo-3 with mid-training and appears in Qwen2.5-32B descendants only after trace training.Source: Figure 7 · p. 25

D Pre-registration and Analysis Details

  • The section covers the theory behind registered predictions, their outcomes, and the statistical procedures used to analyze them.The supplied section description does not specify those procedures or outcomes further.Source: D Pre-registration and Analysis Details · p. 25

D.1 Grammaticalization and What RL Could Do to a Marker

The section uses grammaticalization theory to frame predictions about how RL could alter a marker already in use, then measures its changing form and function. Most specific predictions fail, and marker changes differ by training recipe and starting state.

D.1 Grammaticalization and What RL Could Do to a Marker

  • Grammaticalization provides a framework for tracking how a marker’s integrity, alternatives, obligatoriness, scope, and position can change with repeated use.The section draws on Lehmann’s parameters and Bybee’s account of frequency as a driver, while also noting layering and persistence.Source: §D.1 · p. 25
  • The proposed RL mechanism reinforces a marker-to-move contingency wherever it occurs in successful traces, potentially increasing frequency and concentrating use in productive slots.The text contrasts reporting meaning—how much a marker indicates an error—with operator force, its causal effect on the next move relative to a neutral filler.Source: §D.1 · p. 25
  • Two rival accounts predict different outcomes beyond frequency: an honest report predicts improved error sensitivity, while a ritual predicts causal inertness under insertion or deletion.The ritual account also predicts less frequent rituals with larger groups, whereas grammaticalization and conformity predict opposite relations between frequency and rate of change.Source: §D.1 · p. 26
  • Grammaticalization is used to generate predictions about training, not to claim that RL is grammaticalization or to explain human language processing.Most specific predictions from the theory fail on these data.Source: §D.1 · p. 26
  • The measurement plan tracks marker frequency, sentence position, form variation, entropy, injected-error alarm behavior, and effects on revision.It also lists usage embeddings, synonym substitutability, and measures of form and policy-side entropy.Source: §D.1 · p. 26
  • Most registered predictions fail: the marker does not bleach, its form erodes rather than fixes, and RL effects do not form one path across recipes.RL multiplies a marker supplied by mid-training but erases one seeded by light fine-tuning.Source: §D.1 · p. 26

D.2 Pre-registered Predictions and Outcomes

This section uses label-free tracking and checkpoint-level insertion tests to distinguish inherited marker use from invention and evaluate predictions about RL-driven grammaticalization of “wait.” It situates the tests and their limits, including judge-based conclusions, within prior work on training-data inheritance, monitoring, and language change.

D.2 Pre-registered Predictions and Outcomes

  • Three of six registered hypotheses failed, one was unsupported, and two were not run; only one of 18 counted zero-RL runs showed genuine “wait” use, after the final answer.The sole case was SimpleRL-Zoo on Mistral-Small-24B: seven matches in 1,500,001 tokens, all genuine uses being “But wait!” after the boxed answer; the single-seed Qwen3-4B-Base run was counted as confirming, but did not distinguish the base from the recipe.Source: §D.2 · p. 27, Table 12 · p. 28
  • In OLMo-3 RL-Zero, marker position was already fixed and canonical form eroded; revision markers appeared on 51.8% of revisions, while wait’s d′ rose from 0.77 to 1.29 as false alarms rose from 19.9% to 25.4%.The marker became a more liberal but more sensitive alarm, not a weaker detector of errors. At the last checkpoint, wait frequency also fell from 17.0 to 12.4 per 1k tokens, and the comma form continued eroding from 42.3% to 37.5%.Source: §D.2 · pp. 29–30
  • Inserted Wait, did not outperform neutral Next, on the label-free value-change test, leaving the registered operator test inconclusive; judge-labelled effects were exploratory because judge validity was low.On OLMo-3 RL-Zero, the value-change risk difference was −1.2 points (95% CI [−4.5, 2.0]); the primary judge’s audit agreement was κ = 0.3, with 26.8% sensitivity and 93.9% specificity. The value-change detector also failed its check, so its null was uninformative.Source: §D.2 · p. 29
  • The controlled OLMo-3 GRPO fork without KL showed no marker amplification, unlike the released run; after fine-tuning, 400 RL steps reduced the seeded marker to 0.0003 times its rate while accuracy rose from 43.9% to 68.2%.The matching Next,-seeded arm also gained accuracy, from 40.2% to 66.6%, without coining wait. Across series, the registered cline gate held for OLMo-3 RL-Zero but not OLMo-3 Think; pretraining branches also changed, while the fine-tuning trajectory’s comma-form change was small.Source: §D.2 · pp. 30–31

D.3 Statistical Procedures

  • The analyses used paired cluster-bootstrap intervals over problems, while judge-based rates were treated as exploratory because audits found limited agreement with human consensus labels.Open-series intervals capture evaluation-sampling uncertainty, not training-seed variation; the authors base headline claims on label-free indices and note that small audits cannot rule out differential judge error across checkpoints.Source: §D.3 · pp. 31–32

E Related Work

  • Prior work locates reflection before RL and finds that fine-tuning on traces can install mechanisms; this study contributes a traceable lexical form whose arrival and reshaping can be followed across an open training lineage.The cited literature also reports RL sharpening or amplifying base behaviours, while other work treats strategic phrases as RL discoveries; the paper frames its lineage comparisons as addressing that disagreement.Source: E Related Work · p. 35
  • The study follows markers across training with label-free diagnostics beyond frequency and checkpoint-level insertion tests with base-model and neutral-filler arms, unlike deletion-only analysis (Wang et al., 2025a).It also distinguishes its tests from work on sentence-initial “reasoning operators” (Lee et al., 2026) and notes concurrent rollouts inserting trace openers into the same OLMo-3 checkpoints (reasoning-cues, 2026).Source: E Related Work · p. 36
  • The results connect trace-monitoring and uncertainty research by showing that the words monitors read are inherited, wait’s evidential weight varies by recipe and checkpoint, and judge-labelled revisions are not identified against human labels in this setting.The cited literature includes concerns about trace faithfulness and uneven LLM-judge validity, including difficulty judging step-level errors (Zheng et al., 2025; Zhang et al., 2025c).Source: E Related Work · p. 36
  • The paper frames outcome-based RL as a large natural-language case of change in which a marker is inherited and reshaped in a recipe-dependent way rather than coined.This extends discussion of lexical drift in learning agents and reasoning models, including R1-Zero’s language mixing and o3’s increasingly frequent, hard-to-read dialect terms over RL (Schoen et al., 2025).Source: E Related Work · p. 36

Related papers

Loading 2610.30882v1…