PaperlayerView exact revision on arXiv

Source-linked AI summary

A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement

Yu Liu, Youxing Li, Yuedong Tan, Zhiwei Yang, Zihan Su, Hanqi Feng, Shaojie Zhang, Libin Liao, Zihao Wan, Chenxu Wu, Cong Cao, Fangfang Yuan, Yanbing Liu, Ziheng Chen, Pei Fu, Radu Timofte, Zongwei Wu, Nicu Sebe, Barnabas Poczos, Jian Luan

arXiv:2610.38568v1cs.AI

TL;DR

Recursive self-improvement must decide when a current training pool no longer provides useful learning signal, while generated-task answers need a source of correctness independent of model predictions. A-RSI uses training logs to time pool refreshes and program-authored tasks with computed answers; on the primary eleven-benchmark suite, Qwen’s average rises from 57.77 to 61.05 with 800 fresh tasks.

  • Problem

    Self-improvement research has emphasized what experience to generate more than when to replace a pool whose learning signal has declined, while model-voted training targets can make supervision depend on uncertain predictions.

    Source: §1 · p. 4

  • Method

    A-RSI monitors reward variation and accuracy in training logs to time pool refreshes, while fixed model-written programs generate multimodal tasks and compute their reference answers.

    Source: §1 · p. 4

  • Results

    61.05 versus 57.77 for Qwen on the primary eleven-benchmark suite after eight continuation windows and 800 fresh tasks.

    Source: §1 · p. 4

  • Takeaways & Limitations

    A-RSI supports training without external examples or human-written answer labels by combining training-log refresh decisions with program-computed answers.

    Source: §1 · p. 4

    The refresh signals do not establish that the model has mastered the pool or stopped improving on external tasks.

    Source: §3.3 · p. 8

Abstract

from arXiv · show

Recursive self-improvement requires models not only to acquire useful training experience, but also to decide when their current experience has ceased to be useful. Existing methods often focus on what experience to generate next, while paying less attention to when fresh experience should be introduced. Many also construct training targets through self-consistency or majority voting, making supervision depend on the model's own uncertain predictions. We address these two questions separately: training dynamics determine when to acquire new experience, while the answers to new tasks remain independently checkable. A-RSI monitors reward variation and accuracy in training logs to refresh a task pool when its learning signal declines, without consulting validation or test scores. A static library of model-authored programs produces multimodal questions and derives their reference answers; we specify the task-generation contract, automatically verify chosen properties of the reward and refresh rules, and separately audit the generated answers. The training loop uses no external training examples or human-written answer labels. On the primary eleven-benchmark suite, A-RSI raises the performance average from 57.77 to 61.05 with 800 fresh tasks.

1 Introduction

Recursive self-improvement must decide both what experience to generate and when to replace experience whose learning signal has declined; answer correctness is a separate concern when models train on their own generated tasks. A-RSI uses training logs to time pool refreshes and program-authored tasks with computed answers to support independently checkable supervision, improving the primary benchmark average with 800 fresh tasks.

1 Introduction

  • A-RSI separates the source of refresh decisions from answer correctness: training dynamics guide when to draw tasks, while new-task answers remain independently checkable.This addresses the risk that self-consistency or majority-vote errors become training labels.Source: §1 · p. 4
  • A-RSI refreshes the task pool when training-log reward variation and accuracy indicate a declining learning signal, without consulting validation or test scores.The rule is an operational indicator, not proof that the pool has been mastered.Source: §1 · p. 4
  • 61.05 versus 57.77 for Qwen on the primary eleven-benchmark suite after eight continuation windows and 800 fresh tasks.The comparison starts at the post-initialization checkpoint St.Source: §1 · p. 4
  • Program-written programs generate multimodal tasks and compute reference answers, allowing training without external examples or human-written answer labels.The task-generation contract and answer audits support program-computed supervision without claiming every rendered question is formally proved correct.Source: §1 · p. 4
  • Matched-progress comparisons and response-level analysis distinguish gains associated with answer completion from corrections among already completed answers.The Qwen results identify the implemented replacement policy’s benefit while qualifying the claim about refresh timing.Source: §1 · p. 4

2 Related Work

Prior self-improvement work has explored task generation, adaptation, and supervision, while curriculum methods select examples by difficulty or estimated learning value. A-RSI focuses instead on when to replace a training pool, and evaluates both replacement strategies and whether measured gains persist under changes to evaluation budgets and scoring.

Recursive self-improvement and control question

  • Earlier recursive self-improvement work changes systems or model weights through generated rationales, filtered samples, self-play, or model-based judgments.A-RSI focuses on a control question within weight-based improvement: detecting from training logs when a pool’s useful signal has declined, without evaluation feedback.Source: §2 · p. 5

Zero-data self-improvement and verifiable rewards

  • A-RSI fixes its task-generating program library and studies when to draw a fresh pool, unlike methods centered on proposing, solving, or adapting tasks.Its program-computed answers do not depend on model votes; its pool-level signal differs from batch-level prompt filtering.Source: §2 · p. 5

Procedural tasks and task selection

  • Curriculum and adaptive-sampling methods select examples by difficulty or learning value, but do not settle how to choose a replacement pool after a plateau.A-RSI tests frontier, ladder, and rollout-based selection against random draws; these tested strategies do not outperform random replacement within the stated generator, model, and rule scope.Source: §2 · p. 5

Training duration and evaluation budgets

  • A-RSI’s evaluation audit checks whether apparent gains persist with a longer generation budget and format-corrected scoring, including for released MM-Zero weights.The audit distinguishes task performance from the chance of completing within the evaluation limit; it is not a reward or refresh rule.Source: §2 · p. 5

3 Method

A-RSI trains a Solver on programmatically generated tasks and uses training-side signals to decide whether to retain or replace the active pool. The templates, sampling quotas, and controller remain fixed; replacement carries forward accepted model weights.

3 Method

  • A-RSI trains the Solver on an active pool, reads training-side accuracy and group-relative advantage signals after accepted windows, and retains or replaces the pool while carrying forward model weights.Training experience changes replacement timing, not the generator or task-sampling distribution.Source: §3 · p. 6, §3.4 · p. 9

3.1 Programmatic task construction

  • Frozen templates sample parameters to produce an image, question, and reference answer from the same underlying configuration.Programs construct the underlying data first; the learner receives only the image and question, and replacement tasks need no further model call for question or answer generation.Source: §3.1 · p. 6

3.2 Rewards and policy optimization

  • GRPO updates only the Solver policy using answer, format, and stopping rewards; the templates and controller remain fixed.Reaching the response limit incurs a stopping penalty even if an earlier boxed expression is present; normally completed responses with a final boxed answer receive the same stopping reward regardless of length.Source: §3.2 · pp. 7–8

3.3 Detecting pool saturation

  • The saturation detector tracks distinct prompts with nonzero advantages and answer accuracy, refreshing after a minimum observation period when either signal meets its specified trigger.The signals are a training-side heuristic, not evidence that the model mastered the pool or stopped improving on external tasks; reward differences in format or stopping can also contribute to the advantage signal.Source: §3.3 · p. 8

3.4 Fresh replacement and continued training

  • A refresh replaces the whole pool with unused tasks drawn without replacement under fixed family quotas, while accepted model weights carry forward.Replacement pools are prepared in advance; the detector decides when to activate the next one, and exhausted pools are not silently reused.Source: §3.4 · p. 9

4 Experimental Setup

The experiments evaluate A-RSI’s training-log-driven task-pool refresh using model-authored programs with computed answers. They compare refresh policies and selection variants, examine decisions across stages and models, audit reward scoring and generated-task validity, and report Macro11 performance after introducing 800 new tasks.

4 Experimental Setup

  • A-RSI and four control arms start from the initialized model; refreshes replace the 500-task pool with 400 unused tasks, while No-Refresh retains the old pool.The arms vary refresh timing and selection: A-RSI uses detector-triggered random sampling; Frontier uses detector-triggered frontier selection; Periodic refreshes every two windows; Continuous refreshes every window.Source: §4 · p. 10

5.2 Continued gains with fewer fresh tasks

  • Macro11 reaches 61.05 with 800 new tasks; Continuous scores 60.64 with 3,200, Periodic 59.89 with 1,200, and Frontier 59.59 with 800.At continuation windows 4, 6, and 8, A-RSI scores 59.84, 60.40, and 61.05, respectively, from St at 57.77.Source: §5.2 · pp. 11–12, Figure 4 · p. 12

6.3 Adaptive decisions across stages and models

  • MiMo’s feedback triggers replacement on its initial and first replacement pools but keeps its second replacement pool; at the same stage, replay requests replacement for Qwen but reuse for MiMo.MiMo retains 62.45% and 71.86% of initial advantage counts on the first two pools, below the 85% threshold; on the second, it retains 98.68%. Qwen retains 83.98% on its second replacement pool, while accuracy rises by 6.59 points. The final-stage decisions are offline replay decisions.Source: Figure 4 · p. 12, §6.3 · p. 14

5.3 Training signals identify repeated pool saturation

  • The detector triggers after three windows as nonzero-advantage tasks decline, with ratios .816 on the old pool and .832 for A-RSI and .826 for Frontier on new pools.The accuracy condition never fires alone; No-Refresh later records 35 zero-signal warnings, completes only 94 of 100 normal gradient steps, fails its health gate twice, and halts.Source: §5.3 · p. 12

6.1 Fresh data and refresh timing

  • At matched S23 progress, A-RSI scores 60.40 versus 58.35 for original-pool continuation, a +2.05-point difference; the no-refresh arm has no S32 result after halting.At S32, A-RSI exceeds Periodic by +1.15 points and Continuous by +0.40, but those comparisons change both selection and timing; with frontier selection fixed, detector-triggered refresh trails Periodic by 0.30 points. The first two random-refresh decisions coincide with a three-window schedule, so the run does not establish better timing than a fixed schedule.Source: Table 4 · p. 13, §6.1 · p. 13

6.2 Replacement selection

  • Failure-prioritized, learnable-band, and always-wrong selection trail every-window random control, as do the tested ladder, always-increase, truncation-aware, and all-tier controllers by point estimate.The variants share control runs and training histories, so they are not independent replications.Source: Table 5 · p. 13, §6.2 · p. 13

7.1 Reward hacking and non-truncated corrections

  • Auxiliary rewards can increase for incorrect answers without reversing the correctness ranking, while the production checker fails 220 of 16,000 specified checks.An isolated candidate checker passes all 16,000 checks, but changes one score among 16,000 logged responses and was not substituted into the reported runs.Source: §7.1 · pp. 14–15

7.2 Pipeline validity: verifiable data and experience-driven refresh

  • Independent solvers agree with all 1,000 audited generated-task answers; task-specific checks pass for all 1,000, and frozen replay reproduces 800 images byte for byte.The remaining 200 referring charts match in plot pixels and category and numeric-axis labels, though whole-image reproduction remains unresolved because of font differences. SMT checks cover specified properties, not the full implementation or renderer, and the audit does not certify every visual or linguistic property.Source: §7.2 · pp. 15–16

9 Discussion

  • Observable training feedback lets a common fixed rule respond to changing model decisions across pools, unlike elapsed training alone, without consulting evaluation performance.Measured difficulty also diagnoses tier saturation and mismatches; self-assessed difficulty with AUC near 0.50 could not support those diagnoses.Source: §9 · p. 17

10 Conclusion

  • A-RSI uses program-based generators with computed answers and a training-log detector that refreshes a spent 400-task pool with 400 fresh random tasks.The conclusion says the work also audits length-budget sensitivity, answer formatting, and completed responses.Source: §10 · pp. 17, 20
  • 57.77 to 61.05 Macro11 with 800 new tasks after eight continuation windows on the main model.Each arm was trained once.Source: §10 · p. 20
  • The tested difficulty-based, controller-based, and rollout-based selectors do not outperform random selection in matched comparisons; small single-run MiMo margins do not establish a timing advantage.MiMo’s random-refresh endpoint is 60.71 versus 60.01 for continued reuse, and exceeds every-window and periodic refresh by 0.02 and 0.18 points, respectively.Source: §10 · p. 20

A Supplementary Eleven-Benchmark Policy Comparisons

  • The supplementary policy, endpoint, stable-format, and per-benchmark comparisons use Qwen3-VL-8B-Instruct under the same eleven-benchmark protocol as Section 6, with LJ as an auxiliary scoring view.Visual-type and yes/no subsets retain only annotated items; the default output limit is 8192 tokens, each arm is one run, and policies are compared after matched update counts.Source: A Supplementary Eleven-Benchmark… · p. 24

A.1 Endpoint, data use, and visual types

The endpoint comparisons favor A-RSI over the reported refresh policies, while the analyses distinguish fresh-pool effects from refresh timing and replacement selection. Results also show that the observed trigger timing does not establish an advantage over a fixed schedule.

A.1 Endpoint, data use, and visual types

  • At S32, A-RSI is 0.40 points above Continuous, 1.15 above Periodic, and 1.45 above Frontier on the eleven-benchmark suite.Table 7 reports the S32 policy results at 8192 tokens; visual-type results show the largest gains over St for geometry and counting, alongside large truncation drops.Source: §A.1 · p. 24, Table 7 · p. 24, Table 8 · p. 24

A.2 Fresh data versus continued reuse

  • At S23, detector-triggered random refresh reaches 60.40 versus 58.35 for retaining the original pool, a +2.05-point difference.This comparison includes replacing the original pool with a newly constructed pool, so it does not isolate freshness from pool-composition differences.Source: §A.2 · p. 24, Table 9 · p. 25

A.3 Refresh timing

  • The tested detector timing does not outperform every-two-window refreshing: scores are 59.59 and 59.89, respectively.In the recorded Qwen run, the first two random-pool replacements followed three windows, so the trajectory does not establish an advantage over that fixed schedule.Source: §A.3 · p. 25

A.4 Replacement selection

  • Frontier selection scores below A-RSI at S32, including a −1.45 difference at 8192 tokens; its second refresh was followed by 8.14% truncation and a score decline.The detector gives Frontier and A-RSI the same trigger point because they share the old pool; their replacement pools differ by frontier versus random selection.Source: §A.4 · p. 25
  • Pass-rate and ladder selectors do not improve over random replacement in the tested configurations, whose point estimates are not independent replications.The probe-selection and ladder-control comparisons share a random control, and multiple variants share training histories.Source: §A.4 · p. 25, Table 10 · p. 25
  • All recorded Qwen refresh decisions are supported by declining total-reward signal; the accuracy-increase condition does not fire alone.Removing the accuracy condition would leave those observed decisions unchanged, but this log replay is not an independently trained component ablation.Source: §A.4 · pp. 25–26

B Supplementary Reward and Length-Budget Audits

The length-budget audits find that larger generation limits can recover part of reported gains, while longer-limit comparisons preserve gains for some trained weights. The paper also identifies potential reward-label vulnerabilities and qualifies comparisons against MM-Zero.

Length-budget audit

  • At 8192 tokens, MM-Zero v5 remains +2.45 above the base model, retaining 59% of its 4096-token gain.At 4096 tokens, MM-Zero v5 is +4.18 over the base model; raising the untrained base model to 8192 tokens yields +2.00.Source: B Supplementary Reward and… · p. 26, Table 11 · p. 26
  • On four datasets, the untrained base model at 16384 tokens scores 51.12, comparable to MM-Zero after training at 4096 tokens (51.08).This is a cross-limit subset comparison and supports a budget-related interpretation, not a precise ranking.Source: B Supplementary Reward and… · p. 26

Reward and output audit

  • On 7,257 fixed items with an independent selector, raising only the base model’s limit to 8192 yielded +1.92 (p < 0.0001); paired testing detected no training contribution beyond termination.The two earlier MM-Zero versions tested gave +0.69 (p = 0.138) and +0.37 (p = 0.434); these were not v5 weights, and non-detection does not establish absence.Source: B Supplementary Reward and… · p. 26
  • The source audit identifies possible proxy-reward vulnerabilities, including majority-vote labels that agree with generator answers only 5–12% of the time.Other identified pathways include truncated-response exclusion from the self-consistency denominator and images that may be unnecessary or nearly black; the paper does not show these cause the measured effect.Source: B Supplementary Reward and… · p. 26

Length-budget audit

  • A-RSI retains about 80% of its gain over the base model when the limit grows from 4096 to 8192, but earlier refresh arms still show truncation at 8192.At window 4, reported 8192-token truncation is 5.4% for every-window random refresh, 9.6% for ladder control, and 11.9% for failure-prioritized sampling.Source: B Supplementary Reward and… · p. 27

Comparison scope

  • A-RSI’s +0.79 over MM-Zero is within estimated training noise and establishes neither superiority nor non-inferiority.The fixed-pool continuation comparison is between fixed weights trained once and belongs to an earlier lineage, not A-RSI.Source: B Supplementary Reward and… · p. 27

C Supplementary Fresh-Pool Training Comparison

At matched update progress, the observed Qwen comparison favors the fresh-pool policy over continued reuse. Format-corrected scoring preserves most of the measured gain, but the decomposition does not establish how much reflects reasoning improvement.

Fresh-pool comparison

  • At 8192 tokens, fresh-pool replacement scores 60.40 versus 58.35 for continued training on the original pool, a +2.05-point Macro11 difference.Both arms share the trajectory through S13; replacement uses a fresh 400-task pool while continuation retains the original 500-task pool.Source: C Supplementary Fresh-Pool Training… · p. 27, Table 12 · p. 27

Format-corrected scoring

  • Format-corrected scoring preserves a +3.27 gain (57.77 to 61.05), close to the +3.33 gain under original answer matching.Checkpoints and responses are held fixed; fewer truncated answers therefore do not explain every improvement, but the result does not determine the overall reasoning contribution.Source: C Supplementary Fresh-Pool Training… · p. 27

D Independent Training-Pool Checks and Controller Replay

Independent checks confirm consistency for audited answers and data records, while identifying limits in answer coverage and validity. Controller replay verifies the realized rule path without proving optimal timing; output decomposition provides a model-specific observation beyond truncation recovery.

D Independent Training-Pool Checks and Controller Replay

  • The independent answer audit found five “third highest” ties in hr, so the pipeline does not guarantee zero label ambiguity.Audited groups mv, ref, and tab/sc/ca had no observed disagreements, while the plane-geometry result covered only a sample.Source: D Independent Training-Pool Checks and… · p. 28, Table 13 · p. 28
  • Construction hop counts describe program structure, not model error rates; at five hops, accuracy was 21.2% and truncation reached 75%.The earlier ladder supports model- and budget-dependent difficulty, but does not establish better transferable training data; the historical twelve-benchmark comparison found no detected transfer advantage (−0.37 points, interval [−1.50, +0.74]).Source: D Independent Training-Pool Checks and… · p. 28
  • The two consumed Qwen replacement pools contain 800 tasks with matching training records and no repeated content keys or diagnostic-holdout overlap.Independent solvers agree on 508/508 tasks across mv, ref, ca, tab, and sc; 199 ga and 93 hr tasks were not independently solved in this check.Source: D Independent Training-Pool Checks and… · p. 28
  • The controller’s pool bookkeeping prevents normal refresh before nmin accepted windows and keeps lagged comparisons within a pool, but does not show that the detector finds a plateau or optimal timing.In the recorded Qwen run, refreshes occur at S13 and S23, supported by declining Uw; the accuracy condition never fires alone.Source: D Independent Training-Pool Checks and… · p. 28, Table 14 · p. 29
  • Replay of 38 receipts from 28 logs shows the second Qwen refresh depends on auxiliary-reward variation when observed rollouts are held fixed.Total-reward signal counts fell from 196 to 163; correctness-only counts fell from 166 to 155, below the corresponding trigger threshold.Source: D Independent Training-Pool Checks and… · pp. 28–29
  • Removing the accuracy condition leaves recorded Qwen decisions unchanged, while MiMo’s first two refreshes satisfy both conditions and neither holds in its final-stage replay.Full final-pool signal sequences are 181, 168, 152 for Qwen and 152, 154, 150 for MiMo.Source: D Independent Training-Pool Checks and… · p. 29, Table 14 · p. 29

E Full-Suite Output Decomposition

  • MiMo improves from 59.08 to 60.71 Macro11, with net correction among responses finishing at both checkpoints, repeated under another decoding.This is a model-specific observation beyond truncation recovery; completion status is affected by training, making the stratum contribution descriptive rather than a causal fraction of reasoning improvement.Source: E Full-Suite Output Decomposition · p. 29, Table 15 · p. 29
  • Qwen’s aggregate both-finished contribution is negative in both decodings, while MiMo’s is positive, so the paper limits its full-suite non-truncation observation to MiMo.Qwen’s complete ChartQA both-finished responses nevertheless show +3.04 points in the primary decoding and +3.00 in the repeat, local observations that do not override the full-suite result.Source: E Full-Suite Output Decomposition · p. 30, Table 16 · p. 30

F Generated-Task and Reward-Check Protocols

The audit fixes a 1,000-task cohort before checking it, then tests answer recomputation and reward behavior with independent procedures. The probes support specific claims about these tasks and scoring rules, not exhaustive correctness.

F Generated-Task and Reward-Check Protocols

  • Table 17 summarizes checks on a fixed cohort whose membership was set before checking, without replacing failed cases.The table notes that solvers use recorded textual state and image hashes check file identity.Source: Table 17 · p. 30
  • All 1,000 sampled tasks have independently recomputed answers that agree with their references, and their source labels and image hashes also agree.The cohort contains 200 tasks each from mv, ref, ca, tab, and sc; ga and hr are outside the finite audit.Source: F Generated-Task and Reward-Check… · p. 30
  • Across 1,000 task references, every correct committed answer outranks every wrong committed answer; formatting alone adds 0.1 and normal commitment alone adds 0.3.These are constructed scoring probes, not sampled model behavior or a reward-removal training experiment.Source: F Generated-Task and Reward-Check… · p. 30

G Adversarial Checks and Machine-Checked Specification Properties

Finite adversarial, implementation, and data-binding checks test specified properties without establishing exhaustive correctness. A fixed, outcome-independent output sample separately compares checkpoint responses and shows that answer matching and format-corrected scoring can differ.

G Adversarial Checks and Machine-Checked Specification Properties

  • The production checker passes 15,780 of 16,000 specified adversarial checks; a candidate requiring a complete terminal option in its choice branch passes all 16,000.The 220 production-checker failures are two variants on each of 110 choice tasks; the finite adversarial set is not exhaustive.Source: G Adversarial Checks and… · p. 31
  • Z3 finds the negation of the specified reward-ranking, lagged-comparison, difficulty-pointer, and pool-disjointness properties unsatisfiable under their stated bounds and construction assumptions.Renderer correctness and all Python execution paths are outside the model.Source: G Adversarial Checks and… · p. 31
  • Finite tests match the production detector in 321,605 cases, while task-level difficulty trends vary across four axes and referring charts do not pass the adjusted permutation check.The detector identifies its specified refresh condition, not mastery; the difficulty diagnostics do not establish a training advantage from difficulty-aware selection.Source: G Adversarial Checks and… · p. 31
  • Image checks reproduce 800 original images byte-for-byte and verify plot regions and key text on the remaining 200 referring-chart tasks, without certifying every rendered property.The applicable renderer reproduces the first groups; whole-image reproduction remains 0/200 for referring charts because generic-font differences are unresolved.Source: G Adversarial Checks and… · p. 32
  • All 4,000 injected file, reference, source-question, and PNG defects are detected, while all 1,000 tasks pass applicable construction checks.The specified checks do not prove that every production admission path enforces them or that all rendered questions are unambiguous.Source: G Adversarial Checks and… · p. 32

H Outcome-Independent 1,000-Question Output Audit

  • The 1,000-question sample uses nearly equal quotas across eleven benchmarks, selects questions without model outcomes, and compares the same S0 and S32 questions at 8192 tokens.Its pooled accuracy is not the full-suite micro accuracy.Source: H Outcome-Independent 1,000-Question… · p. 32, Table 18 · p. 33
  • In a second decoding of the same checkpoints, total gains are +3.70 for Qwen and +1.00 for MiMo; both-finished-stratum changes are −0.80 and +0.20, respectively.These are decoding repeats, not independent training runs; completion status is affected by training, so the partition is descriptive rather than causal.Source: H Outcome-Independent 1,000-Question… · p. 33
  • Original answer matching gives sample gains of +5.10 for Qwen and +13.90 for MiMo, versus +4.70 and +1.30 after format correction.The sample diagnoses output changes; attributing them to a reward component requires a matched training ablation.Source: H Outcome-Independent 1,000-Question… · p. 33

I Audit: Is Our Own Gain Real?

The audit finds gains across all eleven benchmarks and retained visual-type subsets, including after format correction. It also reports that random refresh retains more truncation-prone tasks and that selected pools differ in task mix and difficulty signals.

I Audit: Is Our Own Gain Real?

  • All eleven benchmark point estimates improve from St to S32, with gains ranging from +1.19 on SCEMQA-MC to +8.33 on PhyX-MC.Retained visual-type subsets also improve: geometry 56.19 to 59.83, chart 79.63 to 81.66, table 70.96 to 72.99, and counting 54.81 to 61.74.Source: I Audit: Is Our Own Gain Real? · p. 33, Table 19 · p. 33
  • Format-corrected scoring rises from 57.77 at St to 61.05 at S32, a gain of +3.27 on the eleven-benchmark outputs.Original answer matching gives 57.24 to 60.57, a gain of +3.33; rescoring holds checkpoints and responses fixed.Source: I Audit: Is Our Own Gain Real? · p. 33
  • On ChartQA, accuracy rises from 82.84 at St to 84.96 at S32 while truncation changes from 0.60% to 0.56%.This demonstrates improvement on a benchmark with little truncation, but does not quantify the aggregate gain from improved reasoning.Source: I Audit: Is Our Own Gain Real? · p. 34

J Why Random Wins: Mechanisms

  • At window 8, truncation is 4.16% for A-RSI and 8.14% for Frontier; at window 4, it is 6.56% for every-window random refresh, 12.70% for learnable-band sampling, and 14.30% for failure-prioritized sampling.The authors state that the proposed mechanisms are supported by logs but not isolated by ablations.Source: J Why Random Wins: Mechanisms · p. 34
  • Failure-prioritized sampling contains no multiple-choice tasks, compared with 222 in every-window random refresh and 227 in learnable-band sampling, mixing format effects into the comparison.Failure-prioritized sampling scores 72.81% on the 114 retained yes/no items under the recomputed sentinel.Source: J Why Random Wins: Mechanisms · p. 34
  • Every-window random refresh retains 209, 180, 189, and 183 tasks with training signal per window, so the data do not support longer training letting a controller overtake random.Source: J Why Random Wins: Mechanisms · p. 34
  • In the first 6,000 probed candidates, 39% of geometry answers truncate at 4096 tokens; format errors also make some apparently difficult tasks score incorrect.A lenient grader identified 317 tasks among the first 12,000 candidates that a strict grader marked wrong at least three times more often; 311 were multiple-choice multi-hop geometry.Source: J Why Random Wins: Mechanisms · p. 34

K Evidence from Earlier Stages

Earlier stages tested model-written task generation, difficulty escalation, and selection strategies under distinct evaluation scopes. Their results include held-out synthetic-task differences, length-sensitive gains, and mixed evidence about difficulty learning and partial refresh.

K Evidence from Earlier Stages

  • Across three design versions, the reported system moved from model-written next tasks and tier control to programmatic tasks with every-window refresh, then to a log-based refresh detector.The chains use eleven-benchmark scores at 4096 tokens, one run per arm; held-out synthetic tasks and length-budget audits retain separate evaluation scopes.Source: K Evidence from Earlier Stages · p. 34, Table 20 · p. 35
  • On held-out synthetic tasks, keeping the initial state was best and re-selecting by state while climbing to H3 was worst; the fixed-writer arm also outperformed the state-following writer.On Macro11, saturation-based escalation scores 56.69, while six benchmark point estimates span 55.88–56.69; single-run results do not establish a stronger-writer benefit or isolate escalation.Source: K Evidence from Earlier Stages · p. 35, Table 21 · p. 35
  • Regenerating answers truncated at 4096 tokens with a 16k limit leaves the seven-chain mean gain over St nearly unchanged, +1.18 to +1.17, with retention 0.99.Differences between selection rules mostly vanish; the reported interval is 0.44–1.43 and flagged as unstable.Source: K Evidence from Earlier Stages · p. 35, Table 22 · p. 35
  • On one multi-hop ladder, accuracy falls from .997 at one hop to .212 at five, while providing the same information as text yields .84–1.0 accuracy and truncation rises at three to five hops.Long-chain training improves held-out in-domain tasks by +6.58 but does not transfer to the listed external benchmarks; the historical diagnostic retains a twelve-benchmark scope.Source: K Evidence from Earlier Stages · p. 36
  • Held-out pass rates rise on two multi-hop ladders, including tier 2 from .58 to .93 on one ladder, but the controller moves on only two ladders.The tested comparison is mainly a frontier-weighted mix against random, not frequent promotion and demotion.Source: K Evidence from Earlier Stages · p. 36
  • On the eleven-benchmark 8192-token readout, every-window random refresh at window 4 is +1.82 above original-pool continuation and +2.52 above St; partial refresh trails fixed-pool continuation at shared windows 7–11.The policies differ in task exposure, and partial refresh leads only on a held-out chart set, which does not establish broader transfer.Source: K Evidence from Earlier Stages · p. 36, Table 23 · p. 36

L Supplementary MiMo Results

On MiMo, the refresh detector fires on both the old and first new pools, and the refreshed-pool endpoint exceeds continued reuse in the observed run. The small margins over other refresh policies and one run per arm do not establish an advantage for detector-triggered timing.

L Supplementary MiMo Results

  • On the old pool, the detector fires at window 3 as on Qwen; on MiMo’s first new pool, it fires again at window 6 as U falls from 199 to 143 (ratio .72).On the old pool, U falls 245, 194, 153 (ratio .62) while accuracy rises from .490 to .633.Source: L Supplementary MiMo Results · p. 36
  • At window 8, MiMo reaches 60.71 Macro11 with A-RSI versus 60.01 with original-pool continuation, a +0.70-point difference; every-window and periodic refresh score 60.69 and 60.53.Table 25 reports one run per arm; A-RSI’s margins are +0.02 over every-window refresh and +0.18 over periodic refresh.Source: L Supplementary MiMo Results · p. 37, Table 25 · p. 37

M What the Evidence Does and Does Not Support

  • The observed +0.70-point fresh-versus-retained endpoint difference does not establish detector-timing superiority: margins over other refresh policies are small, and each policy has one run.The comparison pools for Continuous and Periodic were selected at Qwen’s difficulty frontier, not MiMo’s.Source: L Supplementary MiMo Results · pp. 36–37

N Scores by Model State

  • Table 27 reports matched-progress Macro11 scores, including St at 57.77 at 8192 tokens; shared prefixes reuse checkpoints, and two 4096-token St evaluations score 54.55 / 54.72.Table 28 gives eleven-benchmark pairwise accuracy differences at 8192 tokens, with stable-format differences in parentheses; these are one-run point estimates, and No-Refresh lacks S32.Source: Table 27 · p. 39, Table 28 · p. 39, N Scores by Model State · p. 39

O Per-Benchmark Results at S32

  • Table 29 reports per-benchmark S32 results at 8192 tokens, with each row’s best accuracy bold and truncation shown per benchmark.Its final row uses equal-benchmark accuracy and pooled-item truncation.Source: Table 29 · p. 39

P Evaluation Protocol and Noise Analysis

The evaluation protocol defines benchmark scope, scoring, and measured variability. The methods describe fixed program-generated task templates and pools, the training setup and refresh controller, and how difficulty-based comparison arms select tasks.

P Evaluation Protocol and Noise Analysis

  • The eleven-benchmark primary suite contains 22,042 questions, uses equal-benchmark Macro11, and evaluates complete test splits with format-corrected scoring at a primary 8192-token limit.Format-corrected scoring retains accepted answers and additionally accepts non-truncated unboxed final answers and normalized option values; ChartQA numeric answers use a 5% tolerance except year answers.Source: P Evaluation Protocol and Noise Analysis · p. 40
  • Each policy arm was trained once; observed score variation includes a 1.25 range across four ladder-control runs, while available repeats do not estimate variation across endpoint policies.The same St weights differ by 0.17 across two 4096-token evaluations on the nine common benchmarks; endpoint differences alone do not establish significance, equivalence, or superiority.Source: P Evaluation Protocol and Noise Analysis · p. 40

Q.1 Template synthesis and task instantiation

  • Fixed templates turn sampled parameters into an image, question, and answer, with generator-specific checks for consistency, validity, and ambiguity.The base model writes reusable structures and executable implementations before GRPO training; retained candidates also pass freshness and contamination screens.Source: §Q.1 · p. 41

Q.2 Candidate reservoir and pool construction

  • Independent recomputation matched mv 3,881/3,881, ref 1,568/1,568, tab/sc/ca 4,580/4,580, and sampled ga 150/150; hr matched 1,922/1,927.The reservoir contained 16,000 candidates; after exclusions, 14,482 were eligible, and the two pre-prepared random replacement pools share no tasks.Source: §Q.2 · p. 41, Table 30 · p. 42

Q.3 Initial pool and optimization

  • Continuation starts from St after 100 GRPO updates on the old pool; replacement pools contain 400 tasks, and refresh settings were fixed before training.The initial pool has 500 tasks; controller settings require at least three accepted windows and trigger on a 15% signal decline or a ten-point logged-accuracy rise relative to two windows earlier.Source: §Q.3 · pp. 42–43

Q.4 Diagnostic difficulty tiers

  • Frontier and Periodic select the easiest ladder tier with held-out pass rate at most 70%, allocating half its quota there and a quarter to each adjacent tier.Continuous moves its selected tier up at pass rates of at least 80%, down at at most 20%, and otherwise leaves it unchanged.Source: §Q.4 · p. 43

R Training Health Gate

The training health gate accepts or retries windows using logged execution and gradient checks, while documented results and limitations qualify what the reproduction and case studies establish.

R Training Health Gate

  • Accepted windows require a complete checkpoint, at least 50 batches, all 100 updates at the specified learning rate, recorded finish reasons, changed weights, and at least 97 finite nonzero-gradient updates.One failed window is retried from the preceding checkpoint; a second failure halts the run. Six consecutive batches with a zero-advantage group abort an attempt, but do not show every pool task is uninformative.Source: R Training Health Gate · p. 43
  • Under the revised 97-of-100 threshold, No-Refresh failed twice at window 7 with 94 normal steps, 35 zero-signal warnings, and training accuracy .728.Earlier 99-of-100 requirements led to stopped runs after repeated retries failed.Source: R Training Health Gate · pp. 43–44

T Further Documented Issues in the MM-Zero Reproduction

  • The reproduction reports base and v5 scores of 50.27 and 54.25 against published 50.03 and 54.18, and local rounds v1–v3 gain +2.67, +4.05, and +3.30 over base.The authors characterize these as observations in their reproduction setting, not claims about the released system’s correctness.Source: T Further Documented Issues in the… · p. 44
  • The pre-registered 8192-token comparison retains 59% of the gain; paired analysis and strict reproduction attribute much of the reproduced gain to finishing responses.The paper also reports an untrained base at 16384 matching MM-Zero at 4096, 51.12 against 51.08.Source: T Further Documented Issues in the… · p. 44
  • The reproduction review lists conflicting equations and labels, missing image-necessity checks, and inconsistencies between reported text and tables; a 16k fair-budget comparison and mechanistic ablations were not run.The authors say the review concerns their reproduction setting, not the correctness of the released system.Source: T Further Documented Issues in the… · p. 44

U Limitations •

  • Each arm was trained once, the main comparison spans eight continuation windows without a 16k evaluation, and small score differences lack significance testing.MiMo’s cross-model comparison is one run with pools selected at Qwen’s difficulty frontier; the fixed generators omit function plots, numerical estimation, and spatial tasks.Source: U Limitations • · pp. 44–45

V Additional Qualitative Case Studies

  • Ten favorable examples across eight benchmarks show wrong-to-right corrections in two independent decodings, but do not establish population frequency, overall capability gain, or a component’s causal effect.Several corrected responses are longer than their base responses, so shorter output alone cannot explain these cases.Source: V Additional Qualitative Case Studies · p. 45

Related papers

Loading 2610.38568v1…