When Should an In-Context Learner Expand Its Hypothesis Space?

1. Introduction

• The motivating decision distinguishes ignoring a deviation, recording an exception, and searching a richer family, whose extra cost can expose missing structure.A deviation alone does not reveal which response is warranted.Source: §1 · pp. 1–2
• A single surprising observation may be weak evidence of missing structure, while small recurring deviations across related inputs can accumulate stronger evidence.Even strong evidence does not settle the action: its value depends on computation cost and the remaining horizon.Source: §1 · p. 1
• The Structural Revision Environment (SRE) matches failures from noise, local exceptions, within-family change, and outside-family structure, while varying expansion costs and future value independently.It supports exact Bayesian calculations and an exact normative solution of the one-shot revision decision.Source: §1 · p. 2
• The normative solution makes revision a value boundary rather than an evidence threshold: one history can call for different actions as prices, horizons, or announced queries change.The boundary between local repair and expansion depends on inputs a rule predicts that a repair cannot cover; richer-family belief can cross well before action changes.Source: §1 · p. 2, Figure 1 · p. 2
• Transformers trained in the environment reproduce the revision boundary from utility alone, without action labels or the reference posterior.The paper also reports that language models’ predictions carry a failure-sensitive signal that their revision decisions do not reflect, and that controlled post-training shifts priors and prediction sharpness without shifting the criterion.Source: §1 · p. 2

The revision problem

• Misspecification is a latent property, whereas surprise is observed, structural evidence is extracted from history, and expansion requires estimating the value of acting.The paper distinguishes surprise, cumulative evidence against the current family, and the advantage of expansion after accounting for alternatives.Source: §2 · p. 3

Actions

• The available structural actions are stay, patch, expand, and defer; a change-point prior lets persistent failures spread across inputs while remaining within the current family.Expansion is absorbing, making its timing an optimal-stopping problem; defer is chosen when waiting is better than committing now or never revising.Source: §2 · p. 3

The normative problem

• Expansion is optimal exactly when its advantage over the best non-expansion action is positive, with the information state including history, prices, remaining horizon, and announced queries.A one-step comparison also depends on the posterior need for the richer family and the per-step losses of staying or expanding incorrectly.Source: §2 · p. 3

The revision problem

• The amortized price is cE/H, so the natural decision coordinates are structural evidence and amortized price.When expansion has no cost and both families are evaluated at every step, the Bayesian model average removes the revision decision.Source: §2 · p. 3

Active set

• The active set is defined by influence on predictions: κ_F compares predictions on a probe across histories that differ in their evidence for family F.Its response curve can indicate whether F is already being averaged over, absent from computation, or gated out; separate probes distinguish the latter possibilities.Source: §2 · p. 3

Generative process

• The SRE’s four classes isolate noise, local exceptions, within-family drift, and interval rules outside the threshold family.A rule episode appears from the threshold family as exceptions sharing one constant-length description.Source: §3 · p. 4

Matched failures

• Queries repeat earlier inputs with probability r = 0.35, and the four classes share the same query schedule and noise process so failure counts and placement are matched.An announced query makes waiting valuable when competing explanations disagree and worthless when they agree.Source: §3 · p. 4

Predictors and prices

• The exact predictors are E0, a change-point predictor for the current family; EP, which adds per-input exception slots; and E1, which uses the full generative model.Patching costs cp per slot, expansion costs cE once, and prices, horizon, and announced query vary independently of history.Source: §3 · p. 4

Protocols and reference policies

• The one-shot protocol fixes a history prefix and asks for one choice among stay, patch, expand, and defer, with defer buying the announced query’s label before commitment.The reference posterior enumerates 7,352 latent states; the one-shot reference is exact, while the sequential reference uses two-step lookahead.Source: §3 · p. 4

The normative solution

• The reference policy expands when predicted inputs that a repair cannot cover would lose more over the remaining horizon than the price difference.In the evidence coordinate of the phase diagram, the reference’s posterior belief in the richer family crosses well before the decision boundary.Source: §3 · pp. 4–5
• At cE = 4 and H = 30, the same history calls for waiting when the announced query is input 16 but expansion when it is input 2; changing price or horizon also changes the optimal action.The posterior assigns 0.71 to the rule class and 0.21 to the exception class; waiting at input 16 is worth 0.33 nats over the best immediate commitment.Source: §3 · p. 4
• With two anomalous inputs, the optimal amortized price difference (cE − cp)/H reaches 0.20 for expansion; with five, it reaches 0.56.For anomalies at opposite ends of a region, the boundary rises from 0.14 for a two-input region to 0.68 for a five-input region, because uncovered inputs impose continuing loss on patching.Source: §3 · p. 5

Transformers trained from scratch

• Explicit Transformers read history, announced query, prices, and remaining horizon, and are trained either to imitate the reference or maximize utility directly.A diagnostic variant also receives the reference posterior, isolating the remaining decision from the cost of forming evidence.Source: §4 · p. 5

Language models

• The language-model protocols test history, failure, and gain prompts, with the gain condition stating expected per-round expansion benefit.The models come from OLMo 3 and Tülu 3 checkpoint lineages spanning post-training stages; an implicit protocol reads their label predictions.Source: §4 · p. 5

Measurements

• Regret is measured against the reference on shared episodes, while decision regression tests expansion choices against benefit, horizon, effective price, and last-observation surprise.Reference coefficients on benefit, horizon, and price are equal, with zero surprise coefficient; price and horizon sensitivity are expressed relative to benefit.Source: §4 · p. 5

Readouts without actions

• Prediction readouts use κ to measure response to structural evidence and compare its shape with Bayesian response curves.Additional readouts measure prediction calibration and surprise leakage in prediction updates.Source: §4 · p. 5

5.1. Learners trained in the environment reproduce the value boundary

• Explicit learners lose 0.04 to 0.09 nats per episode to the reference, and utility-only training yields price and horizon sensitivities of 0.91 and 0.95.The bounded reference loses 0.24 nats; the e-process trigger tuned per cell loses 0.42, and the fixed-threshold version loses 1.19.Source: §5.1 · pp. 5–6

5.2. Language models decide without the evidence

• Language-model decisions at the history level barely respond to benefit, even where predictions move toward rule labels at 0.53 to 0.65 of reference-expansion points.Reasoning makes three models weigh a stated gain against price and remaining rounds in roughly the reference’s proportions, but does not make their history-level decisions price-sensitive.Source: §5.2 · pp. 6–7, Figure 2 · p. 6

5.3. What responds to a failure

• Across 14 language models, κ with three anomalous inputs is 0.09 to 0.43, versus above 0.80 for every positive-prior Bayes predictor.Their predictions show no step at the rule boundary and shift outside the rule region, resembling a learner without the richer family; post-training stages do not change this response shape.Source: §5.3 · p. 7

5.4. Post-training moves the prior and the sharpness, and neither moves the criterion

• Removing rule episodes lowers the implied richer-family base rate from 0.25 to 10^-7.4, while the evidence-response curve tracks the resulting prior.For the learner trained without rules, κ lifts seven steps later than the generative-prior Bayes predictor and stays within 0.03 of the learner-prior predictor at one, three, and five anomalous inputs.Source: §5.4 · p. 8
• The reward objective multiplies log-odds scale by eight to ten without changing base rate, and neither data nor objective produces a criterion shift.The rule log-odds remain decodable after both changes, and 2,000 steps of original training bring prior, κ curve, and latency back within 200 to 600 steps.Source: §5.4 · p. 8

The value of computation

• Self-refinement has been framed as optimal stopping with a threshold independent of remaining rounds, while this paper’s costly action changes the available family and yields returns over the remaining horizon (Hammar et al., 2026).The related work also notes that reasoning models respond weakly to point values under a shared token budget (Fan et al., 2026).Source: §6 · p. 8
• A selector can be hard to evaluate when evidence has the same law across deployments with different best actions; the paper’s reference policies instead hold learner information fixed (Jhawar & Wang, 2026).Source: §6 · p. 8

Creating and expanding hypotheses

• Hypothesis-pool expansion often uses fixed fit thresholds, whereas the paper measures what such triggers lose as price and horizon vary.Examples include per-form likelihood, median residual (Murphy, 2026), and PROBE’s fixed satisficing threshold.Source: §6 · p. 8

• The reference expands when losses on rule-predicted inputs that repair cannot cover exceed the price over the remaining horizon.Source: §7 · p. 8
• Transformers reproduce the reference’s decision from utility alone, without labels or a posterior, so exact evidence formation is not required for that decision.Source: §7 · p. 8
• Language models’ predictions respond to failures, but their history-level decisions do not follow that signal; reasoning values stated gains without estimating what expansion buys from history.Source: §7 · p. 8

Impact Statement

• The authors characterize the work as measurement in a small synthetic environment, with no direct societal risk beyond risks of the models studied.Source: Impact Statement · p. 9

A. The Structural Revision Environment

• The environment draws noise, exception, drift, or rule episodes, with labels flipped independently at probability 0.08 and queries generated from a separate stream.At each step, the next query is announced; after the first uniform query, the stream repeats a previously queried input with probability 0.35 or otherwise selects an unqueried one.Source: A. The Structural Revision Environment · p. 11
• E0 predicts with Bayesian noise-and-drift inference, while E1 uses the full four-class posterior and, once switched on, replaces E0 and drops patch slots.The exception predictor patches at most two inputs, predicting each from observations at that input alone.Source: A. The Structural Revision Environment · p. 11
• Expansion prices span 0.5 to 16, patch prices 0.1 to 2, and sequential horizons use episode lengths 16, 24, 32, or 40.The price grid contains 120 price-and-horizon cells.Source: A. The Structural Revision Environment · p. 11
• Matched histories share query schedules and label flips while differing in whether anomalies fit an interval rule; designed states vary anomaly placement, query order, and announcements.The matched-pair construction retains histories whose E0 and E1 predictives differ by specified logit margins at an unqueried probe.Source: A. The Structural Revision Environment · pp. 11–12
• Two additional worlds test feature-based Boolean rules and cyclic shifts with localized deviations, extending the environment beyond threshold and interval rules.Source: A. The Structural Revision Environment · p. 12

Exact reference

• The exact reference enumerates the posterior over latent states and values future engine scores, while its sequential protocol uses a two-step lookahead with four continuation choices.The continuation choices include never revising, expanding at the leaf, delaying expansion, or patching a repeatedly anomalous input.Source: B. Exact and bounded reference policies · pp. 12–13

Calibration

• At T = 12, the two-step lookahead with exact leaves is within 0.002 to 0.015 nats per episode of the optimum and matches its action in 98.5% to 99.9% of visited decision states.With Monte Carlo leaves and a fitted stopping rule, it loses at most 0.008 nats per decision and matches the optimal action in 96% to 99% of visited states; at T = 40, control is approximate.Source: B. Exact and bounded reference policies · p. 13

Bounded reference

• The bounded reference solves expansion by backward induction from simulated laws of a count-based state, and selects among three fixed patching rules by cross-fitting.Its state tracks queried inputs, anomaly categories, refitting, slots, and the current query, but excludes the announced query.Source: B. Exact and bounded reference policies · p. 13
• The bounded reference lacks evidence distinguishing a small rule-generated anomalous region from one or two exceptions, and its gap from the richer reference averages 0.24 nats per episode across 120 price cells.At the default cell, it expands one step later at the median and loses 1.48 nats per rule episode.Source: B. Exact and bounded reference policies · p. 13

Utility training

• Utility-only learners regress on realized expansion returns that are unbiased targets for the action value at each reached state.With a sufficient state and exact fits, a fixed point of the update scheme satisfies the stopping problem’s Bellman equation.Source: B. Exact and bounded reference policies · p. 14

Fixed thresholds

• The e-process trigger tuned at the default cell loses 1.19 nats per episode on average over 120 cells, versus 0.42 when retuned in each cell, mostly because it over-expands at high amortized prices.A fixed evidence threshold acts like a fixed price and cannot adapt to changing expansion prices.Source: B. Exact and bounded reference policies · p. 14

Collapse to model averaging

• When every family is instantiated at every step and structural actions have no price, the utility-maximizing predictor is the Bayesian model average, and free expansion weakly dominates staying with the current-family predictor.The engines may still predict differently; the decision disappears, leaving a control that compares learner predictions with the model average.Source: C. Normative fragments · pp. 14–15

Decision value

• At zero prices the model average is optimal, while at positive prices the exact one-shot decision compares expected future scores minus action prices.The normative one-step comparison is not a fixed evidence threshold: the same posterior can imply different actions as price or horizon changes.Source: C. Normative fragments · p. 15

One-step threshold

• In the one-step case, expansion rather than staying is optimal when p > (cE/H + δ)/(∆+ δ), so the threshold rises with price and falls with horizon.The price and horizon enter through the amortized price cE/H, while the base rate affects the posterior probability p.Source: C. Normative fragments · p. 15

Price and horizon

• The expansion region grows with horizon and shrinks as expansion price rises, holding beliefs and the transition kernel fixed for the horizon comparison.A change in SRE episode length also shifts the change-point window.Source: C. Normative fragments · p. 15

Patch or expand

• Expansion competes with a patch at the inputs a rule predicts but the patch cannot cover; the decision boundary depends on the expected loss there and the amortized price difference.Diversity can raise rule belief and enlarge the region the rule is inferred to cover, while posterior support saturates after two or three distinct anomalies.Source: C. Normative fragments · p. 16

Belief and decision

• Belief and decision crossings differ: posterior odds favor the rule over the exception list after one anomaly in the cited history, while the expansion decision also depends on price and uncovered inputs.The rule-versus-exception prior log-odds crossing is 0.5014, and the history’s posterior odds are log(0.710/0.213) = 1.20.Source: C. Normative fragments · p. 16

Decision boundary

• The solved decision crossing can lie below the belief crossing: at the lowest grid price, mD ranges from 0 to 0.4 in the feature and shift worlds, below belief crossings of 0.50 and 0.91.In SRE, mD ranges from 0 to 0.65.Source: C. Normative fragments · p. 17

Structural evidence

• The structural-evidence process controls false expansion under the static threshold null, but its trigger reads none of the expansion price, patch price, or horizon.A single extreme label can raise surprise while leaving structural evidence almost unchanged; recurring related deviations raise evidence at each recurrence.Source: C. Normative fragments · p. 18

D. Same history, different optimal action

• At default prices, the reference expands in 75% of rule episodes after a median five-step wait, but expands in only 22% of episodes observationally equivalent to exceptions.It patches most of the latter episodes.Source: D. Same history, different optimal… · p. 18
• At an amortized price of 0.0125, the reference expands in 98% of noise episodes, but in 0% at 0.4; expansion errors are defined by ∆Q∗, not episode class.At intermediate prices, expansion occurs in 31% of noise episodes at 0.025 and 7% at 0.1.Source: D. Same history, different optimal… · p. 18
• The bounded count-statistic policy loses 0.49 nats per episode at default prices, while single-statistic triggers lose 0.68 to 2.0 nats and never revising loses 2.2.The bounded policy expands one step later than the full reference at the median; a fixed threshold is optimal for one amortized price.Source: D. Same history, different optimal… · p. 18

E. The boundary across description languages

• In feature and shift worlds, informative announced queries make waiting optimal, uninformative queries have zero value, and higher prices move the choice from expansion through deferral to patching.A third observation of an anomalous input leaves the decision crossing unchanged.Source: E. The boundary across description… · p. 19
• In the two new worlds, the boundary rises with anomalous inputs before reaching a ceiling, and the predicted comparison matches the solved boundary within a median ratio of 0.98.In the shift world, equally large deviations lead to different actions: an unrepeated single deviation is left alone, a repeated one is patched, and predicate-sharing deviations are expanded.Source: E. The boundary across description… · p. 19

F. Identifiability, delay and the value of waiting

• When ϱ = 0, a rule and matching exception cannot be distinguished by any history, so no policy can meet α + β < 1; prices and horizon determine whether distinguishing them is necessary.A rule applying only to input 16 is observationally equivalent to an exception there.Source: F. Identifiability, delay and the… · p. 19
• In the one-shot protocol, deferral is optimal exactly when the value of information exceeds its cost; an informative announced input can make waiting worth 0.33 net of cost, while an uninformative one has zero value.For the example with input 16 announced, the value of information is 0.43; with input 2 announced, it is zero.Source: F. Identifiability, delay and the… · pp. 19–20
• When discriminating observations arrive at rate ϱ, the bound orders expected delay across regimes, but observed expansion is later because the reference also optimizes utility under prices and horizon.As repeat probability rises from 0.1 to 0.6, the bound rises from 4.0 to 5.0 steps, while mean expansion time rises from 15.1 to 16.8 steps.Source: F. Identifiability, delay and the… · pp. 19–20

G. Active-space transitions

• The active-set readout compares log-odds shifts at unqueried inputs across positions relative to the rule boundary, while calibration slope and surprise leakage separate prediction flattening from structural evidence and surprise effects.Latency is the first step at which predictions beyond the boundary move toward the rule beyond the corresponding E1 movement.Source: G. Active-space transitions · p. 21
• Meta-trained learners’ κ curves match Bayesian predictors under their training priors: rising κ with evidence can reflect a small prior, not late entry of a richer family.The learner trained at rule base rate 0.02 rises from κ = 0.33 with one anomalous input to 0.95 with four; surprise leakage is within 0.007 of zero across the three learners.Source: G. Active-space transitions · p. 21
• Language-model shifts are local smoothing with demonstrations but mostly label-frequency shifts without them, and predictions move toward the rule in 22% to 40% of matched threshold episodes versus 2% for the Bayes predictor.Demonstrations sharing test label symbols suppress responses at frequently demonstrated positions; moving demonstrations or changing their symbols removes most or all of that suppression.Source: G. Active-space transitions · p. 21
• Across OLMo 3 stages and contexts, κ with three anomalous inputs is 0.12 to 0.40, whereas every positive-prior Bayes predictor exceeds 0.80.No OLMo 3 rendering exceeds κ = 0.51.Source: G. Active-space transitions · p. 21
• None of the Qwen 3 8B Base or Qwen 3 8B κ curves follows a prior-indexed family; with three anomalous inputs, κ is 0.17 to 0.26 with demonstrations and 0.41 or 0.49 without them.The final model reaches κ = 0.68 with five anomalous inputs, still off every prior-indexed curve; Figure 5 compares the pair across contexts and evidence levels.Source: G. Active-space transitions · p. 21, Figure 5 · p. 22

H. Learned policies: regret at the default cell and results beyond the main text

• Latency is measured as expansion time in excess of the reference on the same history, with episodes that never expand treated as censored.An expansion error is defined by ∆Q∗(st) < 0, regardless of episode class.Source: H. Learned policies: regret at the… · p. 22
• Beyond the trained price range, price sensitivity falls to 0.43 to 0.58 under every training regime, and longer-than-trained episodes preserve the reported ratios.Below the training range, learners expand less than at the lowest trained price, while the reference expands more.Source: H. Learned policies: regret at the… · p. 22
• With labels for states beyond the reference’s expansion, the imitation learner’s boundary distance falls from 0.22 to 0.03 nats and its expansion rate beyond that boundary rises to 0.34, versus 0.41 for the reference.Without those labels, the learner expands in such states at half the reference’s rate.Source: H. Learned policies: regret at the… · p. 22
• Learner regret comes from patching too little in exception episodes, while on matched failures learners expand more when anomalies are spread across inputs than when concentrated on one.Imitation learners expand in 0.70 to 0.77 of rule episodes and 0.03 to 0.05 of threshold episodes; utility-trained learners expand in 0.03 to 0.09 of threshold episodes.Source: H. Learned policies: regret at the… · p. 22
• Learners stay more often when an informative announced query repeats an anomaly, by 0.10 to 0.22 in stay probability versus 1 for the reference; their price and horizon responses resemble the reference, but the value of the next observation mostly does not.The pattern holds whether the reference’s alternative action is patching or expansion.Source: H. Learned policies: regret at the… · pp. 22–23

Stages

• Preference tuning raises deferral and response to stated gain in both Instruct lineages, whereas verifiable-reward training lowers deferral in OLMo 3 and barely changes it in T¨ulu 3.Action order shifts history-level constants more than rendering does.Source: §I · p. 24
• Preference tuning worsens threshold-episode fit, far-from-boundary accuracy and lift timing in OLMo 3 Instruct and T¨ulu 3, but these shifts occur during supervised fine-tuning in OLMo 3 Think.Preference tuning changes nothing in the Think chain; Llama 3.1 Instruct differs from its base in five small readouts, while T¨ulu 3 differs from Llama 3.1 Instruct in 29.Source: §I · p. 24

Recipes

• From base to final checkpoints, OLMo 3 Instruct, T¨ulu 3, OLMo 3 Think and Qwen 3 show 10, 28, 9 and 7 qualifying readout changes, respectively.A change must exceed cross-rendering spread and have a 95% interval excluding zero in each rendering.Source: §I · pp. 24–25

Adapter branches

• Removing rule episodes halves the implied base rate on both adapter bases, while reward training multiplies sharpness by about ten and twelve, respectively.The separation of prior and sharpness holds under the pre-set criterion for Llama 3.1 and narrowly fails for OLMo 3.Source: §I · p. 25

Controlled post-training

• Rule-free data reproduce none of the preference-tuning markers in OLMo 3 Instruct and T¨ulu 3, while reward training reproduces accuracy drops but not their later lifts.The reward objective raises in-family log-loss 26 to 58 times more, attributed to sharpening rather than later lifting.Source: §I · p. 26

Invariance of the search decision

• Search is invariant to library enlargement when count-statistic laws, expected expansion gains, class prior, region distribution and prices remain unchanged; commitment still depends directly on library size.With a uniform prior, commitment requires the posterior of the true rule to exceed 1 − e only after competing rules are sufficiently excluded.Source: J. Detection and commitment · pp. 26–27

Experiment J1

• At |R| = 16, 40, 80 and 120, m*_B is −0.231, 0.102, 0.354 and 0.501, while m*_D shifts only 7% to 13% of the belief-crossing shift.The decision boundary depends on inputs a rule predicts but a local repair cannot cover, not library size.Source: J. Detection and commitment · p. 27

Experiment J2

• Search delay shows no trend with log |R|, but commitment delay grows with slope +1.42 ± 0.19; when both events occur, expansion precedes commitment by a median 13 to 17 steps.Commitment growth occurs between 16 and 40 rules (+3.2 ± 0.4 per unit), not from 40 to 120 (+0.02 ± 0.01).Source: J. Detection and commitment · p. 27

K. Architectures

• Two fixed-state alternatives, a GRU and Mamba-2-form selective state-space model, match the Transformer at about 4.8 million parameters under the same data, optimizer and training steps.Recovery is measured on rule episodes aligned at each learner’s own lift.Source: K. Architectures · p. 28
• The Transformer, GRU and selective state-space learner agree within seed spread, and each stays within about 2 × 10^-3 nats of the Bayes predictor after lift.Their boundary steps are 1.105 to 1.114 versus 1.122 for the Bayes predictor, with matched-pair κ between 0.977 and 0.995 and surprise leakage about 0.001.Source: K. Architectures · p. 28, Figure 9 · p. 28

L. Mechanism: probes and steering

• The steering intervention probes rule log-odds and catch-all evidence, then adds scaled decoding, orthogonal, encoding or random directions to the residual stream at prediction positions.The dose k is the probe readout shift in nats.Source: L. Mechanism: probes and steering · p. 29
• Encoding-direction pushes move lift monotonically, while probe directions move nothing and random directions do not reproduce the effect; no direction meets the pre-set one-step criterion at k = ±1.On rule-free branches, catch-all encoding pushes partly restore delayed lift, but also produce false lifts on in-family controls.Source: L. Mechanism: probes and steering · p. 29, Figure 10 · p. 29

M. Prompts and read-outs

• Implicit prompts present an episode as labeled observations and elicit probabilities at 16 inputs, with contexts varying between the test episode alone and demonstrations from threshold rules or a generative mixture.Renderings vary symbol and separator conventions, while predictions are read as the renormalized probability of the label-1 symbol.Source: M. Prompts and read-outs · pp. 29–30
• Prediction and action read-outs use token probabilities, and the path read-out is valid in nearly all chat-format model combinations, with specified exceptions.OLMo 3 Think RLVR excludes five order-B combinations; OLMo 3 Think DPO is invalid in every format and was not run. Raw-text validity is reported for OLMo 3 Base, its stage-1 checkpoint, Llama 3.1 8B, and T¨ulu 3.1.Source: M. Prompts and read-outs · p. 32
• At the history level, language models generally do not respond to expansion benefit, and 78 of 92 robustness cells are null in each of 20 bootstrap redraws.Small or mixed responses occur in selected post-trained model cells; the passage reports these as exceptions rather than a general pattern.Source: M. Prompts and read-outs · p. 32
• In 10 of 13 models, decisions almost never follow predictions that moved toward the rule; the other three expand often, but no more when predictions moved.The comparison is made at the models’ decision points in Section 5.2.Source: M. Prompts and read-outs · p. 32
• With thinking off, OLMo 3 Think’s expansion probability falls only from 0.79 to 0.73 as price rises from 0.5 to 16, versus 0.77 to 0.07 for the reference.With room to reason, the models’ gain, price, and horizon slopes grow tenfold and their ratios settle near one.Source: M. Prompts and read-outs · p. 32
• The Qwen 3 final model patches with probability 0.92 and responds to stated gain at 0.76, but its expansion probability falls only from 0.65 to 0.58 across prices.Its price sensitivity is 0.37 and horizon sensitivity 0.14; the reference falls from 1 to 0 over that price range.Source: M. Prompts and read-outs · pp. 32–33

M.1. Controls on task specification and read-out context

• The M.1 control adds a prompt description of the four machine types and their base rates, then compares that rendering with a length-matched restatement on paired decision points.The supplied passage describes the control design and sample points, but does not report its outcomes.Source: §M.1 · p. 33
• Stating the generative process changes expansion probability but meets no evidence-use criterion; Qwen 3 8B rises from 0.015 to 0.712 while matched-history gaps remain −0.064 to 0.000.The length control also shifts expansion probability without producing a gap above 0.05, and identified price and horizon sensitivities remain far from the reference’s value of 1.Source: §M.1 · p. 34, Table 4 · p. 34
• On decision-context prefixes, predictions register matched-pair structure, yet expansion decisions do not track it; at Figure 2(c) points, no model expands where its prediction has moved toward the rule.The prediction moved toward the rule at 0.48 to 0.65 of points, while in twelve models expand was never the most probable action at such points.Source: §M.1 · p. 35, Table 5 · p. 35

N. Training and evaluation details

• Post-training crossed MIX versus F0ONLY episode streams with log-loss versus expected-reward objectives, reading branches at six checkpoints through 20,000 steps.Branches sharing data used the same episode stream; the first learner also had additional data-seed and reward-gradient recovery runs.Source: N. Training and evaluation details · p. 36

O. Statistical procedures

• The decision regression formalizes expansion as optimal when its horizon-wide gain exceeds expansion cost plus the best alternative, yielding reference price and horizon sensitivities of 1.00 and 1.03.The bounded reference’s surprise leakage is 0.05; intervals use episode-cluster bootstrap resampling, and regret is paired on reference episodes.Source: O. Statistical procedures · p. 38

P. On the framing

• In SRE, inference updates beliefs within the current family, repair adds a costly input-specific slot, and expansion pays once for a rule that predicts unqueried anomalies.The repair–revision boundary depends on price: repair is optimal with a small region, high expansion price, or short horizon.Source: P. On the framing · p. 38

Q.2. Test-time reasoning

• The reasoning study tests whether six 32B OLMo models or three thinking models use evidence in decisions and weigh stated gains more like the reference.Evidence use requires an identified benefit response and at least a 0.10 rule-versus-threshold expansion gap; reference-proportion weighing requires both sensitivities within [0.7, 1.3].Source: Q. Scale and test-time reasoning · p. 38, §Q.2 · pp. 39–40

Q.1. The 32B end points

• At 32B, implicit responses retain the 7B models’ shape, while calibration changes: test-only slopes are lower by 0.27 to 0.39 and demonstrations bring the lift 3.1 to 4.3 steps earlier.No model’s matched-pair κ curve follows the prior-indexed family, and each model’s boundary-step interval includes zero.Source: §Q.1 · p. 39, Table 8 · p. 39

Q.2.1. QWEN 3 8B UNDER A THINKING BUDGET

• Under Qwen 3 8B thinking budgets, expansion stays at 0.016 to 0.038 with rule–threshold gaps below 0.01, while price and horizon weighing reaches the reference’s range at 2,048 tokens.No chain closed naturally within 2,048 tokens, so each budget-arm read-out followed a forced close; the weighing criterion was met in both action orders at 2,048 tokens.Source: §Q.2.1 · p. 40, §Q.2 · p. 40

Q.2.2. QWEN 3 8B AT ITS OWN CLOSE

• At Qwen 3 8B’s own close, the history-level expansion gap is 0.060 [0.008, 0.115] in order A and 0.028 [−0.036, 0.090] in order B, below the 0.10 criterion.On rule histories, expansion probability changes only from 0.10 to 0.09 to 0.05 as price rises, while the reference falls from 0.93 to 0.03.Source: §Q.2.2 · pp. 40–41, Table 12 · p. 41, Figure 11 · p. 43
• At forced prefixes, Qwen’s gain-level sensitivities enter the reference’s proportion from 1,024 tokens, but the history-level gap stays between −0.015 and +0.019 through 8,192 tokens.The local history-level response appears only when Qwen closes on its own.Source: §Q.2.2 · p. 41, Table 13 · p. 42, Figure 11 · p. 43
• At Qwen’s own close, rule-history chains name expand near the end more often than counterfactual chains, 0.74 versus 0.57, but rarely conclude it, 0.09 versus 0.04.Mentions of “between positions” follow “boundary” within 80 characters 93% of the time; “interval” and “block” occur in under 1% of chains.Source: §Q.2.2 · p. 41

Q.2.3. OLMO 3 THINK UNDER A LONG BUDGET

• At 8,192 tokens, OLMo 3 Think’s price and horizon sensitivities are 0.99 [0.83, 1.19] and 0.97 [0.75, 1.18]; OLMo 3 Think SFT’s are 0.85 [0.70, 1.06] and 0.74 [0.51, 1.05].These are the only order-A gain-level cells reported for the two models.Source: §Q.2.3 · p. 43

Q.3. Conclusions

• Scale leaves the history-level decision failure unchanged, while reasoning brings stated-gain valuation closer to the reference without estimating what expansion would buy.The 32B models retain the 7B models’ predictive-response shape and do not separate rule histories from matched threshold histories.Source: §Q.3 · p. 43