PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoEnvScaling: Automating the Data Flywheel with Terminal Agents

Simon Yu, Baolin Peng, Bo Liu, Zhengyan Shi, Zhaoyang Wang, Hao Cheng, Qianhui Wu, Wenlin Yao, Natasha Jaques, Weiyan Shi, Jianfeng Gao

arXiv:2610.14163v1cs.SEcs.CLcs.LG

TL;DR

Useful RL environments are scarce and their supply grows only as fast as researchers can work. AutoEnvScaling has a proposer build and validate environments from solver rollouts, with new solver rollouts guiding later rounds. The paper reports higher environment-generation efficiency and benchmark gains, including a same-model proposer–solver result.

  • Problem

    Training terminal agents with RL requires diverse, high-quality environments, but useful environments are scarce and their supply grows only as fast as researchers can work.

    Source: §1 · pp. 1–2

  • Method

    AutoEnvScaling makes environment design a terminal task: a sandboxed proposer builds and validates environments from solver rollouts, and later rollouts guide the next round.

    Source: §1 · pp. 2–3

  • Results

    AutoEnvScaling improves over fixed environment pools, including a 12.2-point Terminal-Bench 2.1 gain over the Cold-Start checkpoint and a Terminal-Bench 4.0 increase from 1.6 to 3.2 with Qwen3.6-35B-A3B in both roles.

    Source: §1 · p. 2, §5 · p. 11

  • Takeaways & Limitations

    The results show that a single model can co-evolve as proposer and solver while training on environments it designs itself.

    Source: §5 · p. 11

    Validation does not guarantee environment correctness, and cross-domain results do not yet demonstrate downstream training gains in those domains.

    Source: §A.1 · p. 18

Abstract

from arXiv · show

Terminal agents solve tasks in software engineering and scientific research by executing commands in a terminal. Training them via reinforcement learning (RL) requires diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work. We introduce AutoEnvScaling, which treats environment design as a terminal task and uses agents to automate the data flywheel. A proposer agent builds new RL environments from the rollouts of the solver being trained, and the solver's new rollouts guide the next round. By turning environment generation into a terminal task, AutoEnvScaling increases the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment. We show that training Qwen3.5-35B-A3B on AutoEnvScaling environments from GPT-5.6-sol improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline. The broader goal of AutoEnvScaling is to enable recursive self-improvement (RSI): by making both environment generation and solving terminal tasks, it lets one model co-evolve in both roles. We show that Qwen3.6-35B-A3B, acting in both roles, gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2. The pipeline also generalizes across agentic domains, generating environments for software engineering, computer use, professional work, and GPU kernel programming. Overall, these results show a path toward agents building their own training environments, reducing human reliance and taking a core step toward RSI.

1 Introduction

Useful RL environments are scarce because their supply depends on researcher effort and must change as agents improve. AutoEnvScaling makes environment design a terminal task, using solver feedback to guide generation and reporting gains in environment efficiency and benchmark performance.

1 Introduction

  • Environment supply is researcher-limited, and useful environments lose value once an agent has learned what they teach.As the agent improves, researchers must identify remaining skill gaps and design further environments.Source: §1 · pp. 1–2
  • Earlier LLM-based approaches leave environment generation fixed in advance, whereas AutoEnvScaling delegates the researcher’s end-to-end workflow to an agent.The proposer decides what to build, then builds and tests environments; the setup also extends to software engineering, computer use, professional work, and GPU kernel tasks.Source: §1 · p. 2
  • AutoEnvScaling treats environment design as a terminal task: a proposer builds and tests complete Harbor environments in a sandbox using solver rollouts, search, and a coding harness.Environments include instructions, an execution environment, a reference solution, and tests.Source: §1 · p. 2
  • New environments enter the pool only after validation and calibration against the current solver, whose subsequent rollouts guide the next generation round.This creates a solver-guided data flywheel as the training pool changes with solver progress.Source: §1 · pp. 2–3
  • 12.2 points on Terminal-Bench 2.1 over the Cold-Start checkpoint, and Terminal-Bench 4.0 rose from 1.6 to 3.2 with Qwen3.6-35B-A3B as proposer and solver.The same-policy setup uses the model in both roles.Source: §1 · pp. 2–3
  • Up to 25× higher valid-environment rate and up to 75% lower cost per valid environment were reported against pipeline synthesis.Training on generated environments also improved Terminal-Bench 2.1 scores by up to 7.4 and 4.3 points over static Tmax pools; gains shrank without solver feedback.Source: §1 · p. 2

2 Related Work

Prior environment-generation work includes fixed synthesis pipelines, feedback-driven repair, agentic customization, and self-play, but these approaches leave generation choices constrained or do not build new environments. AutoEnvScaling targets training data itself, placing a proposer in the training loop to build and calibrate environments against the current solver.

2 Related Work

  • Existing environment-generation work spans fixed synthesized pools [8] [13] [67], verified-task expansion [7] [20] [54], and environment reconstruction from repository changes or execution histories [4] [49].Other methods calibrate tasks against solver pools [25] or rewrite seeds toward a target pass rate [50], without the generator learning from its results.Source: §2 · p. 3
  • AutoEnvScaling places a proposer in the training loop to build and calibrate environments against the current solver, letting the pool evolve as the solver improves.This contrasts with methods where researchers fix generation in advance.Source: §2 · p. 3
  • Prior adaptive and self-play approaches include prompt revision [17], wrapper editing [12], proposer–solver training [22] [51] [65], and shared-model Gym environment design [23].AutoEnvScaling additionally improves the proposer’s harness, retaining tools, skills, and memory across rounds so closed-weight proposers can improve.Source: §2 · p. 3
  • In the shared-policy setup, the model earns a proposer reward based on validation and calibration and a solver reward equal to the fraction of task tests passed.The figure also depicts the proposer converting a failed PyTorch seed into a new environment through read, build, write, and test actions.Source: Figure 3 · p. 4
  • AutoEnvScaling targets training data for recursive self-improvement: what the model learns as a solver can improve the environments it generates next.The related-work discussion frames RSI as systems helping build more capable successors.Source: §2 · pp. 3–4

3 AutoEnvScaling: Automating the Data Flywheel with Terminal Agents

AutoEnvScaling makes environment design a terminal task: a proposer builds and tests tasks using solver rollouts, while validation and calibration govern admission to the training pool. The solver’s updated rollouts guide new environments as both roles train and the proposer can improve its harness.

3.1 Turning Environment Design into a Terminal Task

  • The proposer reads solver trajectories and verifier outputs, then builds simpler tasks for recurring failures or harder variants when the solver succeeds.It decides what to build, writes task files, runs builds, reference solutions, and tests in child containers, and revises tasks until tests behave as intended.Source: §3.1 · p. 5

3.2 Environment Validation and Calibration

  • The host validates tasks by testing the proposer’s reference solution and an empty solution, then calibrates accepted tasks against the current solver.Calibration keeps tasks with mean reward between 0.25 and 0.75 and variation across attempts; failed checks return tasks for revision.Source: §3.2 · p. 5

3.3 Training the Proposer and Solver

  • At each refresh, the host reruns the solver on the pool, removes tasks outside the calibration band, and requests replacements based on recent rollouts.The proposer receives full reward only when an environment passes validation and calibration; its reward distinguishes broken tasks from working tasks that do not suit the solver.Source: §3.3 · p. 5
  • Across every model, valid-environment rates rank Prompt < Prompt + Env Feedback < AutoEnvScaling.Within GPT-5.6, prompt-based rates fall as model strength decreases, while AutoEnvScaling stays between 84% and 92%.Source: Figure 4 · p. 6
  • Between rounds, the proposer can update its memory, skills, and tools from past rewards and logs, while validation and calibration checks remain fixed.One example is merging validation steps into a faster script.Source: §3.3 · p. 6
  • When one open-weight policy plays both roles, proposer and solver rewards are trained jointly with separate advantage baselines and reward groups.The solver trains at every step; every k solver steps, the proposer generates environments and trains on the resulting trajectories.Source: §3.3 · p. 6

4 Experiments

The experiments test terminal-based environment generation, its value for solver training, and whether a model can co-evolve as proposer and solver. They also examine generated-task difficulty, proposer behavior, and learning to ask for missing information.

4.1 Does Environment Design Benefit from a Terminal?

  • 47.2 versus 33.8 average solver turns and 51.0% versus 61.0% solver pass rate for AutoEnvScaling versus Prompt + Env Feedback.AutoEnvScaling also produces tasks with more data files.Source: Figure 5 · p. 7, §4.1 · p. 7
  • 89.0% valid environments for DeepSeek-V4-Flash with AutoEnvScaling versus 3.5% with Prompt; across eight models it has the highest valid rate and cuts cost by up to 75% versus Prompt.AutoEnvScaling lowers cost per valid environment for six of the eight models.Source: Table 2 · p. 7, §4.1 · p. 7

4.3 AutoEnvScaling Enables Recursive Self-Improvement

  • 12.2 points on Terminal-Bench 2.1 over its starting score and 6.1 above Tmax; on Terminal-Bench 4.0, the score rises from 1.6 to 3.2 versus Tmax’s 1.4.The Qwen3.6-35B-A3B proposer-and-solver run without proposer-side training later declines as validation errors grow and destabilize training.Source: Figure 6 · p. 8, §4.3 · p. 9

4.2 AutoEnvScaling’s Model-Designed Curricula Outperform Static Environment Pools

  • AutoEnvScaling outperforms static Tmax pools with both frontier proposers on held-out Terminal-Bench 2.1 under matched training steps.Each comparison regenerates the static Tmax pool using the same proposer as AutoEnvScaling.Source: Table 3 · p. 8, §4.2 · p. 8

4.4 Qualitative Study for AutoEnvScaling

  • From the same timed-out PyTorch rebuild, GPT-5.6-sol tested the reference solution and an empty workspace; Claude Opus 5 also checked rewards for four partial solutions, while Qwen checked compilation only.GPT-5.6-sol and Qwen3.6-35B-A3B followed shorter action sequences than Claude Opus 5.Source: §4.4 · p. 9, Figure 8 · p. 10

4.5 Beyond Task Performance: Learning Behavior

  • 55.3% Ask-F1 and 33.9% task success on generated-task training, versus 29.6% and 6.8% on official-task training.The generated-task model’s ask rate grows only from 84.2% to 89.8%, and most of the gain comes from asking better questions.Source: Table 5 · p. 11, §4.5 · p. 11

5 Conclusion

AutoEnvScaling makes environment design a terminal task, where a sandboxed proposer builds and validates environments from solver failures. Its evolving curriculum outperforms fixed pools and lets proposer and solver co-evolve toward recursive self-improvement.

5 Conclusion

  • The proposer builds and validates training environments from the solver’s past failures inside a sandbox with a coding harness.The conclusion frames environment design as a terminal task.Source: §5 · p. 11
  • A single model can construct its own curriculum and learn from it by acting as both proposer and solver.The proposer improves its harness or weights across rounds while the solver learns from generated environments.Source: §5 · p. 11
  • The evolving curriculum outperforms fixed environment pools, taking a step toward recursive self-improvement and automating the data flywheel for post-training.The paper presents agent-led environment design as a capability agents could learn and improve themselves.Source: §5 · p. 11

A.1 Limitations

The experiments show recursive self-improvement over the evaluated runs but do not establish whether gains continue with longer training. Cross-domain training and stronger verification remain open directions.

A.1 Limitations

  • The experiments show recursive self-improvement over evaluated runs, but do not establish whether gains continue with longer training.Terminal-Bench 4.0 performance also remains low, leaving room to improve on long-horizon tasks.Source: §A.1 · p. 18
  • Passing validation does not guarantee correctness because reference-solution and do-nothing checks cover only a small part of possible agent behavior.Generated tests can miss incorrect solutions or reject valid ones.Source: §A.1 · p. 18
  • The cross-domain results establish environment generation and validation, but not downstream training gains in those domains.Source: §A.1 · p. 18

A.2 Future Work

  • Future work proposes rewarding the proposer directly for solver learning progress to support automatic curriculum adaptation.This proposal follows work using self-play rewards to guide environment generation.Source: §A.2 · p. 18
  • Training jointly across software engineering, computer use, and GPU programming remains unevaluated, including whether capabilities transfer between environment design and solving.Source: §A.2 · p. 18
  • Future verification work should test partial, incorrect, and alternative valid solutions and isolate evaluation services against infrastructure exploits.The passage notes that such exploits can extract ground-truth answers instead of solving tasks.Source: §A.2 · p. 18

B Example Generated HiL-Bench Task

The example HiL-Bench task withholds five details that affect expected reports and makes those answers available through ask_human. The cross-domain study generates tasks across seven domains and evaluates their technical admission.

B Example Generated HiL-Bench Task

  • The generated task asks the solver to reconcile billing and ledger exports and produce three reports, while withholding five details that change expected outputs.The blocker registry provides the withheld answers through ask_human; the task packages both exports and tests in the Harbor layout.Source: B Example Generated HiL-Bench Task · p. 18, Figure 9 · p. 19, Table 6 · p. 19

C Cross-Domain Environment Generation

C.1 Settings and Admission Protocol

  • Generation agents write instructions, initialization code, a reference solution, and a verifier, then build and test each environment with up to two revision rounds.Source: §C.1 · p. 18
  • Tasks run in Linux environments, with local web applications for browser tasks, a graphical desktop for computer use, and an NVIDIA T4 for GPU tasks.Source: §C.1 · p. 20
  • Admission requires successful initialization, full reward for the reference solution, zero reward for a do-nothing agent, reproducible resets, and a passing reference execution.Reported results exclude runs terminated by external execution failures.Source: §C.1 · p. 20

C.2 Results

Across seven domains, 1,061 of 1,489 evaluable generation runs were accepted, with more revision rounds needed for computer-use and professional tasks. The examples span terminal, browser, desktop, professional, and GPU workflows; the appendix describes the proposer workspace and runtime.

C.2 Results

  • 71.3% of 1,489 evaluable runs were accepted; computer-use and professional tasks averaged 2.32–2.80 rounds, versus 1.30–2.35 for terminal and software-engineering tasks.Acceptance confirms executable-check feasibility, not independent task quality or training usefulness; proposer harnesses, task choices, and budgets also limit model comparisons.Source: §C.2 · p. 20

C.3.1 Terminal, Repository, and Tool Workflows

  • The terminal and tool examples test precedence-sensitive data handling and service-state changes, with verifiers checking output rules, decisions, reasons, closure, and preserved state.Examples include warehouse transaction reconciliation, integer time-window repair, and cold-chain consignment disposition.Source: §C.3.1 · pp. 21–22

C.3.2 Browser Use: Linking an Incident to a Runbook

  • The browser task requires selecting a runbook from the incident symptom, completing and resolving the case, and preserving unrelated cases.The verifier checks saved fields and the draft-to-resolve transition.Source: §C.3.2 · p. 22, Figure 10 · p. 22, Figure 11 · p. 23

C.3.3–C.3.4 Computer Use

  • The computer-use tasks require saved document formatting and spreadsheet formulas while preserving existing text or stock data; the spreadsheet workflow finds five discrepant lines and a shortfall of 12.Source: §C.3.4 · p. 23, §C.3.3 · p. 23, Figure 13 · p. 25

C.3.5 Professional Workflow: A Treasury Decision Pack

  • The treasury task combines three 13-week forecasts with a CFO memo; its stress case falls to $18,796.86 against a $30,000 cash floor and reaches the $70,000 facility limit.The memo recommends deferral while retaining the $65,000 week-14 obligation.Source: §C.3.5 · p. 23, Figure 12 · p. 24, Figure 15 · p. 27

C.3.6 Professional Workflow: An Engineering Review Pack

  • The engineering review evaluates three beam sections against stress and deflection limits and selects UB-B at GBP 108/m, with stress 76.52 MPa and deflection 5.64 mm.The task requires a workbook, machine-readable results table, and PDF memo.Source: §C.3.6 · p. 24, Figure 16 · p. 28, §C.3.7 · p. 28

C.3.7 GPU Kernels: Workloads, Generated Code, and Execution Evidence

  • The ragged RMSNorm reference passes 96 correctness trials and runs within 70% of baseline time on each measured shape; the gated reference meets an aggregate limit of 60%.The two tasks use different timing methods: CUDA-event timing for ragged RMSNorm and synchronized wall-clock timing for gated RMSNorm.Source: §C.3.7 · pp. 24, 29, Table 9 · p. 29, Figure 17 · p. 29

D.1 Workspace Structure; D.3 Image and Runtime Configuration; E Proposer Skills

  • The proposer workspace refreshes rollouts and past environments each round while retaining editable memory and tools; instructions and admission checks remain fixed outside the sandbox.The coding harness and skills are packaged in the image, and launch configuration supplies service endpoints and retained memory.Source: §D.1 · p. 29, §D.3 · p. 31, E Proposer Skills · p. 31

E.1 Harbor create-task Skill

This section describes Harbor’s create-task skill for designing, building, and verifying tasks, including its mechanics and verifier-design checks. It also covers an AutoEnvScaling curriculum extension that targets environments with reward spread across solver attempts and checks that they are solvable but not gameable.

E.1 Harbor create-task Skill

  • Task creation proceeds from scaffolding and clear instructions to an environment Dockerfile, interactive environment testing, and a reference solution for Oracle sanity checks.Instructions should specify concrete goals, expected outputs, and constraints without revealing tests; the environment should install task dependencies, not the solution.Source: §E.1 · pp. 31–32
  • The skill treats verifier design as the key task-authoring decision, guiding users to choose shared or separate environments and Reward Kit, pytest, or custom shell checks.Reward Kit suits multi-criterion, partial-credit, or judge-based grading; pytest suits deterministic assertions, while custom shell checks suit simple commands.Source: §E.1 · pp. 32–33
  • Task configuration covers metadata and resource settings, while network access can use public, no-network, or allowlist baselines with phase overrides subject to provider support.The skill recommends populating keywords and notes that differing phase overrides require dynamic_network_policy support; a separate environment is an alternative for distinct baselines.Source: §E.1 · pp. 34–35
  • Oracle validation runs the reference solution and verifier, with debugging focused on solution correctness, verifier detection, paths, and installed dependencies.An optional real-agent run can reveal tasks that are too easy or impossible; README.md is the final step and should document the task, environment, verifier, layout, and run commands.Source: §E.1 · pp. 35–36
  • Multi-step tasks use ordered phases in a shared container with persistent files, per-step instructions and verification, and optional early stopping or separate scoring.The skill describes a steps/ directory structure and task-level reward aggregation strategies, including mean and final.Source: §E.1 · p. 36
  • Harbor supports mean aggregation for signal across steps and final-step aggregation for end-to-end checks, but min_reward can make final use an aborted step’s result.Step artifacts are collected after verification alongside task- and trial-level artifacts, and oracle runs execute each step’s solution before its verifier.Source: §E.1 · p. 37

E.2 AutoEnvScaling Skill

  • The AutoEnvScaling skill targets high reward_std across completed solver attempts, using avg_reward only as a secondary difficulty check and calibrating task difficulty in both directions.It favors variants where genuine solver decisions produce different outcomes, rather than identical partial rewards.Source: §E.2 · p. 38
  • Proposers must inspect solver trajectories and their corresponding past environments, then adapt exemplars toward a different, sometimes-solvable weakness rather than inventing randomly or duplicating tasks.The skill directs proposers to examine reward statistics, raw rewards, trajectories, verifier output, and the environment’s instructions, tests, build setup, and taxonomy.Source: §E.2 · p. 38
  • Useful environments are deterministic, self-contained terminal tasks whose reference solution scores 1.0 and whose no-op solution scores below full credit.When a managed fraction-scoring verifier is mandated, meaningful gated milestones should create across-attempt spread; an additive checklist that gives every attempt the same partial credit is discouraged.Source: §E.2 · p. 38
  • Before submission, proposers must verify that the oracle passes with reward 1.0 and that replacing it with a no-op fails; otherwise they must fix the solution-verifier mismatch or check for real work.The skill also requires using Harbor’s scaffold and placing the finished canonical task at the specified environment path.Source: §E.2 · pp. 38–39

F.1 Reference Execution Ablation

Reference-solution self-checks improve valid-environment yield across all five tested models. The decontamination check flags generated tasks with matching 13-gram text windows, but does not rule out paraphrased copies.

F.1 Reference Execution Ablation

  • Enabling reference-solution execution improves valid-environment yield for all five tested models.The proposer can observe failures and repair an environment before submission; the authors attribute much of the coding-agent interface’s benefit to this capability.Source: §F.1 · p. 39

F.2 Decontamination

  • The decontamination check flags a generated task if any sliding 13-token window matches a benchmark task’s 13-gram.The check uses a stride of one and covers Terminal-Bench 2.1 and 4.0 task descriptions; it does not rule out paraphrased copies or other contamination.Source: §F.2 · p. 39

G.1 Shared-Model Training

In shared-model recursive training, the full system outperforms Tmax and the proposer-training ablation on Terminal-Bench 2.1, while proposer training maintains environment validity. Results use matched solver-update counts, and the experiments specify calibration and resource constraints.

G.4 Solver-Guided Calibration

  • The active pool is reviewed every 16 solver updates, and tasks no longer meeting learning criteria are selectively removed.Proposer trajectories are generated and trained every 16 solver updates while the solver trains at every step; proposer rewards are mean-centered without standard-deviation scaling. Admission and calibration checks target mean reward in [0.25, 0.75] with reward standard deviation at least 0.1, using four or eight solver rollouts and up to two revisions.Source: §G.1 · p. 39, §G.4 · pp. 40–41

G.1 Shared-Model Training

  • Each solver update uses 16 environment groups with 16 trajectories per group, and groups without reward variation are oversampled and discarded.Unfinished terminal trajectories resume with the updated policy while retaining stored sampling log-probabilities for previously generated tokens.Source: §G.1 · p. 40
  • 48.9, 44.9, and 40.1 are the Terminal-Bench 2.1 scores for the full system, Tmax, and the proposer-training ablation, respectively, at the common 160-update endpoint.The full AutoEnvScaling run and Tmax are evaluated through 288 updates; the ablation peaks at 44.1 at update 96. The full system reaches 3.2 on Terminal-Bench 4.0 at update 192 and retains that score through update 288.Source: §G.1 · p. 40

G.2 Proposer Validity During Shared-Policy Training

  • With proposer training, shared-policy environment validity ends at 94.7%, whereas without it validity falls to 22.4% by step 160, when training collapses.Both runs start from the Cold-Start checkpoint at 86.8%; the trained proposer’s validity rises above 88% from step 80 onward.Source: §G.2 · p. 40, Figure 20 · p. 41

G.3 Evaluation Harness and Resources

  • All Terminal-Bench results use the Pi coding agent in sandboxes with 4 CPUs and 8 GiB of memory.The authors report that harness differences account for most of the gap from model-card scores, with smaller sandboxes accounting for part of the rest; comparisons within the paper use the same harness and resources.Source: §G.3 · p. 40

H Cold-Start Controls

Cold-Start fine-tunes Qwen3.6-35B-A3B on proposer trajectories and improves environment-generation acceptance, while paired solving results do not show a detected performance change. A separate pool-generation experiment finds enough trainable tasks from Cold-Start for one rollout batch, but not from the base model.

H Cold-Start Controls

  • Cold-Start fine-tunes Qwen3.6-35B-A3B on 600 high-quality proposer trajectories from Claude Opus 5, DeepSeek-V4-Flash, and Kimi-K3.Figure 21 compares base and Cold-Start Qwen3.6-35B-A3B with GPT-5.6-sol across the same 136 environment-generation assignments; its caption specifies 95% Wilson intervals and notes assignment-suite variation across models.Source: H Cold-Start Controls · pp. 41–42, Figure 21 · p. 41
  • Figure 22 compares proposer episode turns for frontier, Cold-Start, and base models using medians, interquartile ranges, and whiskers extending to observations within 1.5 times the interquartile range.Opus uses an earlier assignment suite; the other models use the shared 136-assignment suite.Source: Figure 22 · p. 41
  • Of 38 disagreements in task acceptance, 32 favor Cold-Start and six favor the base model; paired solving performance changes by 1.1 percentage points, with a 95% bootstrap interval of [−4.9, 7.4] and sign-test p = 0.88.The pass@5 difference is also not statistically significant (p = 0.15); Table 13 reports environment acceptance on the same 60 assignments and solving over five attempts on 89 Terminal-Bench 2.1 tasks.Source: H Cold-Start Controls · p. 42, Table 13 · p. 42
  • Cold-Start supplies 16 tasks with non-degenerate solver rewards, enough for one rollout batch; the base supplies four trainable tasks among 13 evaluated admissions.One base-model admission was unevaluated, but even if trainable, the pool would remain below the 16-task minimum.Source: H Cold-Start Controls · p. 42

I Environment Generation on Two Further Seed Cases

  • The two further seed cases use the same template and the same three proposers as the Figure 8 seed case.The passage identifies these as additional cases in the environment-generation evaluation without reporting their outcomes.Source: I Environment Generation on Two… · p. 42

I.1 Labelling protocol

The protocol relies on hand-labelled solver paths, with classifier densities shown only as background. Trace selection and visibility impose caveats on comparisons of solver rationale.

I.1 Labelling protocol

  • The three seed cases where all proposers shipped without hitting the wall clock had distinct failures: no output after budget exhaustion, a repair-loop timeout, and a nearly completed hard task.Source: §I.1 · p. 42
  • Cold-Start Qwen3.6-35B-A3B traces omit native thinking, making its public record thinner than its reasoning and limiting stated-rationale comparisons.Source: §I.1 · p. 42
  • All nine drawn paths were hand-labelled turn by turn across 475 turns; the keyword classifier supplies background density, not the path labels.The classifier agreed with hand labels on 83–90% of turns per episode; an earlier version misclassified Claude Opus 5’s difficulty sweeps as build, inverting the reading.Source: §I.1 · p. 42
  • Episodes use the first sample except Claude Opus 5’s fastText case and Cold-Start Qwen3.6-35B-A3B’s primer-design case, where other samples did not ship.Claude’s first fastText sample timed out waiting on its reference run; the other primer-design samples did not ship.Source: §I.1 · p. 42

Related papers

Loading 2610.14163v1…