EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering

• Parameter optimization incorporates task signals into weights, while context optimization uses feedback to revise textual instructions for an individual question (Hu et al., 2021; Shao et al., 2024).Source: §1 · p. 1
• Ordinary conditional generation offers no separate control over the strength or token support of distribution changes caused by a revised context.The paper asks how feedback-driven context updates can provide a controllable intervention on the generation they guide.Source: §1 · pp. 1–2
• EMO alternates context revision by an external judge without gold answers with a confidence-constrained logit correction based on revised and preceding contexts at a shared prefix.Feedback on the steered response informs the next context update while model weights remain fixed.Source: §1 · p. 2
• Parameter and context updates admit additive logit representations, and EMO derives its steering correction from context-induced logit displacement.The one-round analysis identifies sufficient conditions for improvement toward a target distribution.Source: §1 · p. 2
• Across eight models and six benchmarks, EMO beats Base in 39 of 40 aggregated settings, averaging +9.10 percentage points.Source: §1 · p. 2

• Parameter adaptation incorporates training signals into model weights, whereas EMO keeps a frozen checkpoint and adapts context and decoding from feedback on the current question (Hu et al., 2021; Schulman et al., 2017; Shao et al., 2024; Guo et al., 2025; Yu et al., 2025; Snell et al., 2022).Source: §2 · p. 2
• Textual-gradient and prompt-optimization methods revise textual inputs with frozen weights; EMO adds distribution contrasts between successive contexts to control decoding at each prefix.The steered response then supplies the basis for further textual feedback.Source: §2 · pp. 2–3
• Inference-time methods modify activations or contrast next-token distributions (Turner et al., 2023); EMO links such intervention to per-question context updates, and compares with SC/BoN at matched target-model generation budgets.Related test-time compute approaches include self-consistency (Wang et al., 2022), best-of-N (Brown et al., 2024), and sequential refinement (Snell et al., 2024).Source: §2 · p. 3

3.1 A Unified View: Optimization as Logits Steering

• Parameter and context updates share an NLL–KL objective and can each be represented as additive logit changes, but their feasible distributions and update directions differ.The computable context-induced contrast is used because the target distribution is unavailable.Source: §3.1 · pp. 3–4

3.2 EMO: Alternating Context Optimization and Logits Steering

• EMO alternates question-specific context estimation and distribution refinement; the EM analogy is structural, not a shared-likelihood optimization.The method uses feedback on generated responses to couple context updates with decoding interventions.Source: §3 · p. 3, §3.2 · p. 4

3.2.1 E-Step: Context Optimization via Textual Gradients

• The E-step uses judge feedback without gold-answer access to revise the context, and later rounds feed it the preceding M-step’s steered response.Answer-bearing rewrites are rejected, and target-KL improvement is an analysis assumption rather than a guarantee of the textual update.Source: §3.2.1 · p. 4

3.2.2 M-Step: Distribution Contrast as Logits Steering

• At each shared generated prefix, the M-step contrasts distributions under the revised and preceding contexts to construct a steering correction.Tokens pass the confidence mask when Popt(v) ≥ α maxv′ Popt(v′); an unmasked positive step can increase divergence from the preceding-context distribution without ensuring target alignment.Source: §3.2.2 · pp. 4–5
• The algorithm runs K optimization rounds, stores each response’s judge score, and returns the earliest highest-scoring candidate.For K = 4 it yields five candidates and nine target-model generation equivalents, excluding judge/rewrite costs and response-length differences; equal latency is not implied.Source: §3.2.2 · p. 5

3.3 Theoretical Analysis

• Under context-improvement and faithful-steering assumptions, sufficiently small μ gives a one-round decrease in target KL divergence.The guarantee does not certify the default μ = 2.4, monotonic accuracy, or improvement between steered rounds.Source: §3.3 · pp. 5–6

4.1 Experimental Setup

• The evaluation covers eight models, six benchmarks, a shared external judge, and four optimization rounds for both EMO and TextGrad.Main results use greedy decoding; budget comparisons use top-p sampling with three seeds.Source: §4.1 · pp. 6–7

4.2 Main Results

• +9.10 percentage points on average over Base across 40 model–task-group pairs, with EMO improving in 39; it adds +1.53 points over TextGrad on average.All eight models improve on average; Table 1 reports Base, TextGrad, and EMO accuracy under greedy decoding.Source: §4.2 · p. 7, Table 1 · p. 7
• Qwen3-14B EMO exceeds Qwen3-32B Base on all five task groups, while mean Qwen3 gains are non-monotonic across model sizes.Across Qwen2.5-7B, Qwen3-8B, and Qwen3.5-9B, mean gains over Base are 9.64, 9.42, and 6.40 points; increments over TextGrad are 2.74, 1.76, and 4.74 points.Source: §4.2 · pp. 7–8
• EMO narrows Qwen3-8B–Llama-3.1-8B accuracy gaps on MMLU-Pro, MATH-500, and GSM8K, while gains occur in both thinking and non-thinking models.On MMLU-Pro, the gap shrinks from 18.2 to 3.6 points; across non-thinking models, mean gains range from 9.64 to 12.96 points.Source: §4.2 · p. 8
• All eight models gain on MMLU-Pro from steering over TextGrad, averaging 3.76 points; MMLU-Pro oracle coverage also rises for all eight.Mean MMLU-Pro oracle-coverage increment is 9.6 points; mean increments on MATH-500 and GSM8K are at most 1.4 points.Source: §4.2 · p. 8

4.3 Component Ablations and Robustness

• Steering’s contribution varies with judge and target model, while EMO improves over Base in all 18 tested single-seed decoding configurations.On Qwen3-8B/GPQA-D, steering adds 2.1 points with the external judge but loses 3.5 with self-judging; on Qwen3-32B/GPQA-D, it loses 2.5 under either judge. Gains also persist under tested greedy, top-p, and top-k decoding strategies.Source: §4.3 · pp. 8–9, Table 3 · p. 9

4.4 Comparison at Matched Generation Budgets

• At N = 9, EMO exceeds BoN on all six Llama tasks, while its Qwen3-8B accuracy is close to BoN on GPQA-D and MATH-500.At matched target-model generation budgets, EMO’s five-candidate oracle coverage exceeds that of nine independent samples in seven of nine settings.Source: §4.4 · pp. 9–10, Table 4 · p. 10, §4.5 · p. 11

4.5 Iteration Dynamics and Selection

• The first two rounds capture roughly 70–89% of final gain in five of six Qwen3-8B and Llama-3.1-8B pairs, but only about 46% on Qwen3-8B/GPQA-D.On Llama-3.2-3B/MATH-500, K = 2 reaches 55.5 accuracy, or 76% of the eventual gain; later rounds add 4.8 points. At N = 9, EMO’s coverage exceeds independent sampling in seven of nine settings, but selected accuracy can lag behind coverage.Source: §4.5 · pp. 10–11

5 Conclusion

• The conclusion frames generation and verification as coupled test-time computation, with context revision, decoding strength, and verification effort adjusted to the reasoning bottleneck.The empirical scope is two model families and reasoning benchmarks; steering benefits vary by model and task.Source: §5 · p. 11
• The one-round analysis identifies sufficient conditions for improvement toward a target distribution, not a guarantee for arbitrary context edits or steering directions.The conclusion associates this analysis with EMO’s connection between context optimization and logits steering.Source: §5 · p. 11

A.1 Proof of the Unified Objective

• The shared target-KL objective has the same minimizers as cross-entropy for either parameter or context optimization within each chosen feasible set.The feasible sets need not have the same minimizer, and the gold response remains unavailable to the judge and textual optimizer.Source: §A.1 · p. 15

A.2 Proof of the Unified Steering Form

• Both parameter and context updates admit additive logit representations, but their actual update directions are not generally the direct-logit target-gradient direction.Context edits induce finite logit differences; parameter updates also change responses to subsequent contexts and prefixes.Source: §A.2 · pp. 15–16

A.3 From the Context Contrast to the Steering Vector

• A positive correction requires a token’s log-ratio to exceed its optimized-distribution average; merely having Popt(v) > Pbase(v) is insufficient.The expression is evaluated in linear vocabulary time using log-softmax values, without a dense Jacobian or model-parameter backpropagation.Source: §A.3 · p. 16
• The context-induced displacement supplies EMO’s steering correction, whose target-KL benefit requires favorable target alignment rather than following from the observable contrast alone.The ordered relationship is complementary: context optimization supplies task information, steering executes an intervention, and the steered response informs the next update; the correspondence alone does not establish classical EM monotonicity.Source: §A.3 · p. 16

A.4 Proof of Conditional One-Round Improvement

• Under the stated conditions, every 0 < µ < 2Γ/L improves over the unsteered updated-context distribution; µ = Γ/L guarantees a decrease of at least Γ^2/(2L).The sufficient interval depends on the unknown target and is not an implementable tuning prescription or a certificate for the empirical default.Source: §A.4 · p. 18
• For fixed contexts and threshold, positive aggregate alignment is necessary and sufficient for some positive steering strength to improve target KL over the updated-context distribution.If alignment is nonpositive, convexity implies that every nonnegative strength is no better along this path.Source: §A.4 · p. 18
• The guarantee conditions on fixed contexts and threshold, uses full softmax distributions, and compares steering with the preceding unsteered context distribution.It does not establish unconditional iterative convergence or an accuracy guarantee, and context improvement alone does not imply faithful steering.Source: §A.4 · pp. 17–18
• A context update can improve target KL while producing a steering direction that worsens it, so the proof treats context improvement and faithful steering as distinct assumptions.The constructed three-token example illustrates why one assumption does not entail the other; it is not an experimental result.Source: §A.4 · p. 18

A.5 Non-Regression under Judge Selection

• Judge selection preserves the highest stored judge score as candidates are added, but does not guarantee non-regression in rule-based correctness.The selected candidate is the earliest response attaining the maximum stored score.Source: §A.5 · p. 18

A.6 Implementation Conventions

• With K = 4 rounds, EMO produces five candidates and uses nine target-model generation equivalents, counting context streams rather than exact FLOPs or latency.The budget excludes judge, backward, and rewrite costs, as well as differences in context and response lengths.Source: §A.6 · p. 19

B.1 Datasets

• The six-benchmark suite spans free-form mathematical reasoning and multiple-choice scientific knowledge, with datasets converted to a unified schema for prompting and rule-based scoring.Task type determines the prompt template and scoring rule.Source: §B.1 · p. 19
• AIME 2025 and 2026 each contain 30 competition-math problems, and the main table merges the editions for noise reduction.Answers are integers from 0 to 999; each edition is also used as a separate test set.Source: §B.1 · p. 19
• GPQA-Diamond contains 198 expert-written and validated multiple-choice science questions across physics, chemistry, and biology.Each question has four options.Source: §B.1 · p. 19
• GSM8K has 1,319 grade-school word problems in its test split and is the suite’s most saturated benchmark, with three models above 89% baseline accuracy.The stated saturation bounds observable headroom.Source: §B.1 · p. 19
• MMLU-Pro evaluation uses a stratified subset of 560 questions: 40 sampled uniformly from each of 14 categories with seed 42.The full test set has 12,032 questions across 14 categories.Source: Table 5 · p. 19, §B.1 · p. 20

B.2 Model Selection

• The evaluated models span 3B–32B, three Qwen generations, two families, and thinking and non-thinking modes.The models are organized around four analysis axes, with Qwen3-8B as the anchor.Source: Table 6 · p. 20, §B.2 · p. 20
• Qwen3 mean gains are non-monotonic across 8B, 14B, and 32B models: +9.42, +7.44, and +8.68 points, respectively.Larger gains on AIME than on near-saturated GSM8K also indicate task dependence within each scale.Source: §B.2 · p. 20
• Every generation in the Qwen2.5-7B-Instruct → Qwen3-8B → Qwen3.5-9B line gains +6 to +10 points on average after optimization.The passage concludes that newer base models do not obsolete per-question adaptation.Source: §B.2 · p. 20
• Llama-3.1-8B gains 22.4 points on MATH-500, demonstrating applicability beyond the Qwen family.The family comparison does not isolate an individual training choice.Source: §B.2 · pp. 20–21
• Non-thinking models also benefit: on MATH-500, they gain more over Base than the thinking Qwen3-8B, though checkpoint differences prevent attributing this to thinking mode.For Llama-3.1-8B, TextGrad already accounts for most of the MATH-500 improvement.Source: §B.2 · p. 21
• EMO improves the distilled DeepSeek-R1-0528-Qwen3-8B by 6.02 points on average, but its mean M-step increment over TextGrad is −0.64.MMLU-Pro is an exception, improving by 1.2 points; the comparison is not a controlled pre-/post-distillation pair.Source: §B.2 · p. 21

B.3 Experimental Settings

• Final accuracy is determined by task-specific extraction and gold-answer rules, while the judge score is used only as an internal optimization signal.The judge never sees gold answers; a corrected boxed-answer extraction cascade was used to rescore affected MATH-500 responses.Source: §B.3 · pp. 21–22
• Greedy experiments use four optimization rounds, total steering strength µ=2.4, and confidence threshold α=0.1; gains on Qwen3-8B/GPQA remain +5.6 to +7.6 across µ ∈[1.6, 3.2].The Llama information-saturation detector hits 97% of degenerate chains with 0% false positives on normal chains and speeds up MATH-500 by ∼12×; early termination has negligible accuracy effect.Source: §B.3 · pp. 21–23, Table 7 · p. 22
• Each EMO round generates one response using two context-conditioned model streams; EMO@k uses 2k + 1 target-model generation equivalents for k + 1 candidates.The study matches this target-model budget to independent SC/BoN sampling, excluding judge and rewrite calls, response-length differences, and latency.Source: §B.3 · pp. 21, 23

B.4.1 AIME-2025 and AIME-2026 Reported Separately

• EMO more than doubles Qwen3-8B Base accuracy on both AIME editions: 16.7→43.3 on 2025 and 23.3→50.0 on 2026.Each edition has only 30 problems, so one problem corresponds to 3.3 points; Llama models remain near zero, and Llama-3.2-3B pass@5 reaches only 3.3 and 6.7 on the respective editions.Source: §B.4.1 · p. 23, Table 8 · p. 24

B.4.2 Accuracy vs. Iteration for All Models

• Iteration 2 realizes about half of EMO’s final gain across 44 non-saturated model–task pairs; gains on AIME and weak-model GPQA-D can accrue through iteration 4.On saturated Qwen3-family GSM8K tasks, curves flatten after one iteration; MMLU-Pro coverage and selected accuracy favor EMO across all eight models, while free-form math coverage gaps are at most 1.4 points on average.Source: §B.4.2 · pp. 23–24, Figure 3 · p. 24, Figure 5 · p. 25

B.4.3 Component Ablation on Qwen3-32B

• On Qwen3-32B, adding the M-step ties context-only optimization on AIME-2025 at 43.3 but adds 10.0 points on AIME-2026, reaching 53.3 versus 43.3.With steering retained, external judging adds 13.3 and 6.6 points over self-judging on the two editions, respectively.Source: §B.4.3 · p. 25
• Steering on GPQA-D loses 2.5 points under either judge choice; external judging does not universally resolve an unfavorable steering effect.The steering increment changes from −3.5 to +2.1 when Qwen3-8B switches from self-judging to external judging, illustrating a possible interaction between feedback quality and steering utility.Source: §B.4.3 · p. 25
• With steering retained, the external-judge increment on MMLU-Pro reverses from +2.3 at 8B to −1.1 at 32B.The comparison changes both feedback and candidate selection and does not estimate uncertainty across runs.Source: §B.4.3 · p. 25
• The GPQA-Diamond and AIME-2025 figures plot greedy accuracy against EMO iteration for all models.The captions specify the plotted dimensions but do not state a comparative outcome.Source: Figure 6 · p. 26, Figure 7 · p. 26

B.4.4 Full Compute-Matched Results

• Sampling results average three seeds at temperature 0.6 and top-p 0.95; EMO uses N = 2k + 1 generation equivalents and k + 1 candidates.SC and BoN are reported at budgets N = 1,…,10.Source: §B.4.4 · p. 26, Table 10 · p. 28, Table 11 · p. 28
• At k = 4 (N = 9), EMO exceeds BoN in mean selected accuracy on eight of nine pairs; its clearest advantages are on the six Llama tasks.The Llama gains over BoN are +5.2, +4.8, and +5.7 points for Llama-3.1-8B and +6.5, +7.6, and +12.7 for Llama-3.2-3B across GPQA-D, MATH-500, and MMLU-Pro, respectively.Source: §B.4.4 · p. 26
• With five candidates versus nine independent samples, EMO’s oracle coverage is higher in seven of nine pairs; the largest gap is 73.7 vs. 60.3 on Llama-3.1-8B/GPQA-D.The exceptions are Qwen3-8B/GPQA-D (79.1 vs. 80.0) and Qwen3-8B/MATH-500 (90.4 vs. 90.9).Source: §B.4.4 · pp. 26–27
• On Llama-3.1-8B/GPQA-D, EMO’s selected-to-covered ratio is 37.4/73.7 ≈50.7%, below BoN’s 32.2/60.3 ≈53.4%, despite EMO’s greater coverage.These descriptive ratios do not isolate causal effects of ranking because candidate pools and covered questions differ.Source: §B.4.4 · p. 27
• On Llama-3.1-8B/MMLU-Pro, SC and BoN share 76.4 oracle coverage, but judge-selected BoN accuracy is 62.5 versus SC’s 51.4; EMO reaches 83.2 coverage and 68.2 accuracy with five candidates.On Qwen3-8B/MATH-500, SC is strongest at 86.1 and EMO has no coverage advantage over independent sampling.Source: §B.4.4 · pp. 27–28

B.4.5 Statistical Significance of Sampling Results

• Paired tests match each question under the same seed, pool three seeds, and use Holm–Bonferroni correction over 18 comparisons per test.BoN significance tests use ten independent samples against EMO’s nine target-model generation equivalents; matched-budget comparisons use BoN@9.Source: §B.4.5 · p. 28
• All nine model–task pairs show significant gains over Base (paired t-test, pHolm ≤2.2×10−3); five of six Llama cells show significant advantages over BoN after correction.The Llama exception is Llama-3.1-8B/GPQA-D (+4.4, pHolm = 1.3×10−1); none of the three Qwen cells reaches significance against BoN.Source: §B.4.5 · pp. 28–29
• The Wilcoxon signed-rank test agrees with the paired t-test in every reported comparison.Source: §B.4.5 · p. 29

B.5 Hyperparameter Analysis

• Across µ=1.6–3.2, the gain stays within 5.6–7.6 points; α=0.05 yields +1.0, versus +7.6 to +9.6 at α=0.1–0.4.The Qwen3-8B/GPQA-Diamond sweep reports 56.6 (+7.6) for the default µ=2.4, α=0.1.Source: §B.5 · pp. 29–30, Table 15 · p. 30

B.6 Prompt Templates

• The four E-step prompt templates are task-agnostic across six benchmarks, with only the question stem and option block varying by task.Source: §B.6 · p. 30

B.6.1 Judge Scoring Prompt

• The judge grades responses on a 0–10 scale without seeing the gold answer and gives feedback that identifies errors without revealing the answer.The score supports judge-select, while feedback feeds the backward pass.Source: §B.6.1 · p. 31

B.6.2 Backward (Text Gradient) Prompt

• The backward engine turns the current context, response, and judge feedback into criticism about how to change the context, not a rewritten context.The full response is embedded in this prompt and drives the judge-context length budget, reaching about 14.5k tokens at peak for the longest responses.Source: §B.6.2 · p. 31

B.6.3 Optimizer (Context Rewrite) Prompt

• The optimizer uses that feedback to rewrite the context concisely, with its role description steering rewrites toward domain-specific guidance rather than generic encouragement.Optional momentum, constraints, and in-context examples are disabled in the reported experiments.Source: §B.6.3 · p. 31

B.6.4 Answer-Leakage Filter

• The judge’s answer-blind feedback and a regex filter rejecting answer-like rewrites guard against gold-answer leakage into generation.Rejected rewrites retain the previous context.Source: §B.6.4 · pp. 31–32