Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards

• Latent-world-first synthesis builds tasks from a complete, correct hidden specification, preserving intermediate states for controllable complexity, exact verification, and dense progress rewards.Source: §1 · p. 2
• PPRS scores workspaces after every turn against known-correct intermediate states, providing within-group reward variation even when no rollout finishes.The progress potential rises as the workflow is completed and falls on regressions.Source: §1 · pp. 2–3
• TEV-GYM contains 1,170 verified long-horizon environments across 7 domains, with median rollouts 12.4 minutes long and about 6 times longer than the most time-intensive prior corpus.The collection can also be extended with coding-agent-written generators across seeds and scale tiers without further model calls.Source: §1 · p. 3
• 65%: TEV-27B raises ALE-CM Pass@5 over Qwen3.8-27B to 0.28, matching GPT-6 Astra and Claude Opus 5.Source: §1 · p. 3
• TEV-27B generalizes from ALE-related training data to Terminal-Bench 3.0 and 4.0 and reaches Claude Opus 4.8-level performance on middle-horizon TB 2.1.Source: §1 · p. 3

2.1 LATENT-WORLD-FIRST TASK SYNTHESIS

• Each seeded generator constructs and validates a complete latent world, then derives a workflow DAG whose critical path defines the task horizon.The DAG encodes dependencies, permitted parallelism, and per-stage semantic-work estimates.Source: §2.1 · p. 4
• Rendering projects the latent world into target-format files and verifies exact round-tripping; declared, invertible corruptions then produce the agent’s task while preserving recoverable ground truth.Source: §2.1 · p. 4
• The prompt, starting workspace, strict verifier, and progress potential all derive from the same latent world, preventing task artifacts and checks from describing different problems.Source: §2.1 · p. 4
• The public contract specifies inputs, outputs, schemas, requirements, precedence rules, and tools; a hidden suite checks semantics-preserving transformations and rejects plausible-but-wrong states.Source: §2.1 · p. 4

2.2 COVERAGE, CALIBRATION, AND QUALIFICATION

• Coverage is labeled across 19 axes, while scale tier and workflow dependencies control operational horizon and dependency depth.Source: §2.2 · p. 5
• Environments must pass eight benchmark-independent gates, including exploit resistance, valid monotonic progress scores, and separation of scorer, verifier, and solver code.Source: §2.2 · p. 5

3.1 TERMINAL REWARD STARVES GROUP-RELATIVE RL

• Under terminal-only rewards, all-failure GRPO groups have zero advantages and waste all G rollouts; a single success still cannot distinguish near-misses from no progress.When all G rollouts fail, the task contributes nothing to the gradient; with one success, the G −1 failures receive the same negative advantage.Source: §3.1 · p. 6

3.2 A PROGRESS POTENTIAL FROM THE LATENT WORLD

• The progress potential is a weighted sum of prerequisite-gated workspace scores, with anchored, monotone, and independent properties enforced by qualification gates.Scoring correct intermediate states against the latent world provides deterministic progress feedback for generated seeds and scales without human step labels, a trained reward model, or an LM judge.Source: §3.2 · p. 6

3.3 PROGRESS-POTENTIAL REWARD SHAPING (PPRS)

• PPRS combines terminal pass reward with per-step changes in workspace progress potential and retains a terminal potential term.Even when every rollout fails, differing progress can yield nonzero advantages that rank rollouts; any passing rollout still scores above any failing one.Source: §3.3 · p. 6

4.1 SETUP

• Training scales with model size: the 9B model uses 264 unit-tier tasks, while the 27B model trains on full TEV-GYM; TB evaluation uses 22- and 18-task CM subsets.No TEV-GYM environment shares a 13-gram or 8-gram with the evaluation tasks; the subsets correlate with full-set P@1 at r = 0.96 for TB 3.0 and r = 0.97 for TB 4.0.Source: §4.1 · p. 7

4.2 MAIN RESULTS

• TEV-9B and TEV-27B outperform their bases on ALE-CM, while TEV-27B improves TB 3.0/4.0 P@1 but remains substantially behind frontier agents.The TB evaluation uses the CM subsets; Claude Opus 5 reaches P@1 scores of 0.53 on TB 3.0 and 0.52 on TB 4.0.Source: Table 3 · p. 8, §4.2 · p. 8
• P@1 0.794 and P@5 0.884: TEV-27B achieves these TB 2.1 scores without data targeting that benchmark, on par with Claude Opus 4.8.TEV-9B also improves over its base, raising P@1 by 28% and P@5 by 19%.Source: Table 4 · p. 8, §4.2 · pp. 8–9
• TEV-320B raises P@1 from 0.20 to 0.27 on TB 3.0 and from 0.29 to 0.37 on TB 4.0, becoming the strongest open-source model on both subsets.P@5 gains 0.05 on TB 3.0 and is unchanged on TB 4.0; the largest category gain is TB 3.0 security, from 0.22 to 0.45.Source: §4.2 · p. 8

4.3 ABLATION STUDY

• 0.05–0.06 S@1 and 0.05 P@5: the full latent-world method leads public-view and prompt-first data-curation ablations; public-view discards 88% of sampled groups.Prompt-first replaces the construction pipeline, leaving no latent world or workflow DAG.Source: Table 5 · p. 9, §4.3 · p. 9
• 0.04 S@1: PPRS improves over sparse reward; turn-index normalization preserves later-turn credit, while γ = 1 propagates progress to earlier actions.At 27B, shaped reward trains to completion, whereas terminal-only reward fails to complete a single optimizer step.Source: Table 5 · p. 9, §4.3 · p. 9

4.2 MAIN RESULTS

• TEV-27B and Claude Fable 5 ground the liveness-probe cause in the explanatory measurement and judge rollback safe, unlike the base model and GPT-6 Astra.All four agents find the same root causes, but the base model and GPT-6 Astra cite symptoms and decline rollback, forfeiting the decisive grader component.Source: §4.2 · p. 9

Synthetic corpora for coding and terminal agents

• Existing coding and terminal-agent corpora commonly mine repositories or generate tasks, while Tev uses seeded generator code for long, multi-stage workflows across several domains.The cited corpora include repository issues and tests, bug-injected repositories, and reconstructed terminal sessions (Yang et al., 2025a; Ivison et al., 2026; Lin et al., 2026).Source: §5 · p. 9

Backward construction

• Prior backward-construction methods corrupt or invert a single artifact; Tev corrupts a rendering of an entire latent world, tying the task, verifier, and progress signal to that world (Zeng et al., 2026a; Ivanov & Rana, 2026).The cited established approaches include denoising, controlled error injection, code mutation, environment rewinding, instruction back-translation, and procedural puzzle generation (Lewis et al., 2020; Arocena et al., 2015; Tran et al., 2013; Yang et al., 2025a; Hariharan et al., 2025; Lin et al., 2026; Li et al., 2023; Wu et al., 2024; Stojanovski et al., 2025; Chen et al., 2025a; Liu et al., 2025; Li et al., 2025).Source: §5 · p. 10

Sparse reward and dense signal in agentic RL

• Other dense-feedback approaches use learned judges or proxies such as test-pass rates; Tev’s potential measures exact progress and regression at each workflow stage without a critic or language-model judge.The cited alternatives include filtering identical-reward groups, selecting tasks near current ability, expert prefixes, learned judges, and proxy signals (Yu et al., 2025; Florensa et al., 2018; Zhang et al., 2025c; Cui et al., 2025; Choudhury, 2025; Du et al., 2025; Shen et al., 2026; Yang et al., 2026).Source: §5 · p. 10

6 CONCLUSION

• Each task is built backward from a known-correct latent world rendered into interrelated files, then modified through controlled corruptions for exact verification and dense progress rewards.The approach is instantiated in TEV-GYM, with 1,170 verified environments across seven domains.Source: §6 · p. 10
• Training improves 9B and 27B open models on ALE-CM, with gains transferring to Terminal-Bench tasks of different horizons and to a 320B base model.The conclusion describes the evidence as gains across these settings, without specifying individual benchmark scores here.Source: §6 · p. 10
• The study is limited to synthetic terminal environments; the hardest benchmarks retain a gap from leading frontier agents, and supporting new workflows requires authoring and validating more environment components.Source: §6 · p. 10
• Future work could simplify environment development, expand domain coverage, and combine the data with stronger open models for reproducible RL without human step labels or language-model judges.Source: §6 · p. 10

A DATA CONSTRUCTION DETAILS

• Appendix A expands the condensed description of data construction.Source: A DATA CONSTRUCTION DETAILS · p. 20

A.1 TASK SOURCES AND DOMAIN TAXONOMY

• TEV-GYM’s seven domains follow ALE-CM subdomains spanning software, data, AI/CS research, math and operations research, security, infrastructure, and quantum computing.The taxonomy uses ALE identifiers 9.1–9.7; each domain must have at least two generators.Source: §A.1 · p. 20
• Generators are newly designed around randomly sampled specifications from internet sources, MLE-bench, or public quantum-computing references, without reproducing source prompts, answers, or graders.Source: §A.1 · p. 20
• Candidates must have knowable ground truth and run on batch/CPU-only resources; similarity screening and adversarial review reject near-duplicates, shortcuts, short horizons, or disguised sources.Source: §A.1 · p. 20

A.2 DETERMINISM, REPRODUCIBILITY, AND GENERATION COST

• 30 s of CPU time is the median marginal cost to generate an environment, including reference solvers, hidden-suite sealing, and progress-potential preview, with no model calls.Generators are authored once, then seeded environments are reproducible; two seeds yield distinct worlds.Source: §A.2 · p. 20

A.3 COVERAGE AXES AND COVERING ARRAY

• A strength-2 covering array over 19 axes requires every valid pair of axis values to co-occur, making pairwise corpus diversity checkable rather than nominal.Reseeding varies world identities and corruption placements; environments include heterogeneous file artifacts and sometimes images.Source: §A.3 · pp. 20–21

A.4 SCALE LADDER

• Scale tiers enlarge worlds and workflow horizons: warehouse examples range from 30 customers and 48 monthly transactions to 480 customers and 960 transactions, while DAGs typically have five to six dependent stages.Across three-tier generators, the median input tree grows from 17 files and 51 KB at unit to 28 files and 586 KB at full, with the workflow DAG unchanged.Source: §A.4 · pp. 21–22

A.5 MEASURING HORIZON AND DIFFICULTY

• Horizon is measured by the workflow DAG’s critical path rather than rollout length, while difficulty is learner-relative and reported as verified success under a declared harness.When success saturates at zero, the progress potential still resolves difficulty.Source: §A.5 · p. 22

A.6 QUALIFICATION GATES

• Every environment must pass benchmark-independent gates, including causal reward checks, resistance to 14 gaming probes, independent solver agreement, valid progress behavior, seed diversity, and public-contract hygiene.Generation-time gating is used because dense progress is useful only when causal and resistant to gaming.Source: §A.6 · p. 22

A.7 DATASET STATISTICS

• 0.89 across capability categories and 0.91 across domains are TEV-GYM’s balance scores; environments have a median of 6 workflow stages, 8 progress nodes, and 6 reward components.The median prompt is 583 tokens, and the median input comprises 12 files across 23 wire formats.Source: §A.7 · p. 22
• 3.2% of environments are fully solved in untrained Qwen3.8-27B single-rollout probes, with mean graded score 0.06; Gemini 3.6 Flash reaches pass@1 29% and pass@8 40% on a fixed 92-environment subsample.52% of untrained Qwen3.8-27B rollouts exhaust the 65,536-token context before submission.Source: §A.7 · p. 23

A.8 COMPARISON PROTOCOL

• 12.4 min is TEV-GYM’s median agent time, versus 0.5–2.0 min for prior corpora, measured with the same evaluator; no TEV-GYM rollout reached its 300-call or 2-hour budget.The measure includes model calls, retries, and command execution, but is not an estimate of human effort.Source: §A.8 · p. 23
• Difficulty comparisons use Gemini 3.6 Flash with a shared 60-tool-call budget and 8 rollouts per task, though prior corpora use Harbor while TEV-GYM uses single-bash; excluded hard CLI-Gym tasks make its score a mild upper bound.Terminal-Universe could not be measured because its environments are not publicly released; SETA’s end-of-episode test fraction does not count as a progressive reward.Source: §A.8 · p. 24

A.9 DECONTAMINATION

• Fresh procedural seeding yields distinct public trees for every environment and evaluation instance, so TEV-GYM contains no verbatim copy of an evaluation task.Source: §A.9 · p. 24
• 0 of 1,170 TEV-GYM environments share a 13-gram with any evaluation set, and none shares an 8-gram; nearest-neighbor Jaccard maxima are 0.18 on TB 3.0/4.0, 0.16 on TB 2.1, and 0.38 on ALE-CM.Source: §A.9 · pp. 24–25, Table 10 · p. 25

EXAMPLE 1: MULTIMODAL CLOUD COST OPTIMIZATION (CLOUD OPERATIONS)

• The cloud-optimization task combines 430-resource inventory and pricing tables with dashboard-only findings, then derives reference recommendations and savings from the latent fleet record.The grader scores coverage, action correctness, savings accuracy, and report completeness; the screenshot findings appear in no text file.Source: EXAMPLE 1: MULTIMODAL CLOUD COST… · pp. 25, 27

EXAMPLE 2: CLOSED-LOOP BUILDING MPC (ENERGY, POWER & NUCLEAR ENGINEERING)

• The building-MPC environment grounds nine deliverables in a pinned EnergyPlus simulator and declared procedure, with references generated by executing the procedure in the sealed image.The grader compares artifacts field by field; closed-loop control is required, so an open-loop schedule, first-order model, or single policy cannot reproduce them.Source: EXAMPLE 2: CLOSED-LOOP BUILDING MPC… · p. 29

EXAMPLE 3: FINANCIAL-STATEMENT RECONSTRUCTION (ACCOUNTING & FINANCE)

• The financial-statement task requires three deliverables with source-resolving evidence and exception records, applying explicit precedence across structured facts and filings.The policy selects the latest accepted structured fact, lets amendments override only explicitly restated rows, and prefers structured values when they conflict with filing rows.Source: EXAMPLE 3: FINANCIAL-STATEMENT… · pp. 29–30

A.10 THREE WORKED TRAINING ENVIRONMENTS

• The latent world defines 16 integer line items across FY2025 and FY2024, while the task renders them as structured company facts and inline-XBRL filing rows.The ground truth satisfies three accounting identities and ties connecting balance-sheet and cash-flow values.Source: EXAMPLE 3: FINANCIAL-STATEMENT… · p. 31
• The defect plan places incorrect values in rejected sources, so policy selection recovers the latent-world values and yields exactly four recorded conflicts.The FY2025 filing misstates cash, accounts receivable, and other current assets; the amendment changes cash from 239 to 243, while the structured value is 246.Source: EXAMPLE 3: FINANCIAL-STATEMENT… · p. 31
• A round-trip self-check reselects all values, verifies identities and statement ties, and confirms the expected exceptions before publication; the grader compares three files leaf by leaf and cell by cell at zero tolerance.The reference deliverables are financial_statements.json, exceptions.json, and evidence.csv, with 32 evidence rows covering each line item and period.Source: EXAMPLE 3: FINANCIAL-STATEMENT… · p. 31

B DENSE REWARD DETAILS

• 44% of groups still have no reward variance at p=0.05 with G=16, illustrating why terminal-only rewards can starve long-horizon training.The appendix also reports that only about half of synthesized terminal tasks yield any reward variance under binary reward.Source: B DENSE REWARD DETAILS · p. 32
• PPRS retains terminal potential, making the objective terminal success plus λ Φ(sT); this grades partial progress but can change stochastic-policy rankings in expectation.A passing trajectory earns 1 + λ and a failing one at most λ, so deterministic policies maximizing either objective agree.Source: B DENSE REWARD DETAILS · p. 32
• Across 192 warehouse rollouts, terminal reward was zero for all, leaving all 24 groups with zero advantage, while Φ had non-zero variance in every group.Rollouts reached prerequisite-gated plateaus from 0.05 through 1.Source: B DENSE REWARD DETAILS · p. 32
• Within an all-failing group, PPRS orders rollouts by workflow completion and assigns negative increments to regressions such as corrupting a correct table.The fraction of groups yielding gradient rises from the all-fail fraction to groups whose rollouts are not all stalled at the same stage.Source: B DENSE REWARD DETAILS · pp. 32–33
• PPRS keeps the task acceptance set unchanged by anchoring Φ = 1 exactly at verifier passage, while its task-aligned scorer tracks intermediate artifacts and regressions.The scorer has code closure disjoint from the verifier and passed the paper’s causality and exploit-resistance gates.Source: B DENSE REWARD DETAILS · p. 33

C.1 TRAINING RECIPE AND HYPERPARAMETERS

• Every training step sees 8 informative groups after zero-variance groups are discarded and refilled by active sampling.The policy-ratio total-variation divergence bound is 0.1, with no KL anchor.Source: §C.1 · p. 33
• Turn t receives λrt, the final turn additionally receives the terminal term, and each turn’s advantage is its return-to-go minus the same-index group mean.The reward uses Eq. (4) with λ = 0.25 in the shared recipe.Source: §C.1 · p. 33
• Table 11 lists the hyperparameters shared by every RL run; runs differ in corpus subset, reward mechanism, model size, and hardware.Source: §C.1 · p. 33, Table 11 · p. 33

C.2 EVALUATION PROTOCOL

• ALE-CM reports mean graded score and full-solve fraction at Pass@1 and best-of-five, while TB CM evaluations use 22 tasks for TB 3.0 and 18 for TB 4.0.Qwen models use Vanillux2; frontier models use their native agent harnesses.Source: §C.2 · p. 34
• ALE-CM uses 18 of 23 computing-and-math tasks that run in the Linux Docker sandbox; five tasks requiring virtual machines are excluded for runtime reasons.TB 3.0’s 22-task CM subset was selected from 74 tasks, and 18 of those remain in TB 4.0.Source: §C.2 · p. 34
• TB 3.0 and 4.0 CM subsets score somewhat above their full benchmarks, but model comparisons remain closely aligned.Across models, Pearson r is 0.96 (P@1) and 0.89 (P@5) on TB 3.0, and 0.97 and 0.92 on TB 4.0.Source: §C.2 · p. 34, Table 14 · p. 35, Table 15 · p. 35
• Across five models, ALE-CM pass-rate rankings agree more closely than score rankings with the full leaderboard: Spearman ρ is 0.95 for pass rate and 0.63 for score.The comparisons differ in harness configuration and number of rollouts; the full benchmark also spans domains beyond computing and math.Source: §C.2 · pp. 34, 36, Table 16 · p. 36

C.3 QUALITATIVE EXAMPLES ON ALE-CM

• Figure 5 traces four single rollouts on a task where Tev-27B beats the base rollout; all traced tasks are also solved by at least one frontier agent.The figure marks credited decisions green, forfeited decisions red, routine steps grey, and trajectory step as t; Table 17 lists graded scores.Source: §C.3 · p. 36, Table 17 · p. 36

Synthetic corpora for coding and terminal agents

• Prior coding and terminal-agent corpora mine repositories or synthesize tasks, but typically use pull-request-sized instances or single sessions with test-suite verifiers.The cited corpus-generation approaches include model-written tasks and environments, trajectory reconstruction, and repository-based task synthesis.Source: D EXTENDED RELATED WORK · p. 36

Qualitative examples

• Qualitative examples show that task completion can hinge on independent checks and deliverable writing, not merely reaching an intermediate solution.One base model reaches a working bound but never writes the required file; another enumerates correctly but does not write its deliverable, while the Ising rollouts independently cross-check the ground state.Source: Figure 8 · p. 37, Figure 7 · p. 37, Figure 9 · p. 38

Synthetic corpora for coding and terminal agents

• Programmatic environment-scaling work includes short API episodes, while TeV’s environments span long, multi-stage workflows and ship a progress potential alongside success conditions.The passage connects difficulty control and dense signal to the same generator that supplies task coverage.Source: D EXTENDED RELATED WORK · p. 38

Backward construction

• Earlier backward-construction methods generally corrupt one artifact or create one-step puzzles; TeV instead corrupts a rendering of an entire world across mutually constraining files.The latent world serves as the oracle, preserving exactness and intermediate-stage ground truth across a multi-hour horizon; CLI-Gym rewinds one environment and uses a pass/fail check.Source: D EXTENDED RELATED WORK · pp. 38–39

World-state verification, environment scaling, and long-horizon benchmarks

• World-state verification has been used in hand-authored evaluation tasks, while generated-world systems and long-horizon benchmarks have distinct dependencies or scale limits.The passage notes that AgentMercury and Zhang et al. depend on a model to write worlds, while cited long-horizon benchmarks have task checkpoints written by hand or model-graded and contain tens of instances.Source: D EXTENDED RELATED WORK · p. 39

Sparse reward and dense signal in agentic RL

• Existing dense-reward approaches include learned or judged rewards and exact proxies, but test-fraction rewards probe terminal tests and do not track monotone workflow progress or regression.TeV’s potential is described as exact progress at every workflow stage, anchored so Φ = 1 exactly when the verifier passes, and available for every seed.Source: D EXTENDED RELATED WORK · p. 39