PaperlayerView exact revision on arXiv

Source-linked AI summary

Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement

Yukai Wu, Yuanjing Yang, Le Zhou, Shaokun Han, Haoyu Wang, Zirui Tang, Weihuang Zheng, Maxm Pan, Xuanhe Zhou, Fan Wu

arXiv:2609.29773v1cs.AI

TL;DR

Persistent environments can leave agents to resolve scattered evidence, conflicting versions, and implicit cross-file relationships. Env-Rethink organizes and prepares those environments, and evolves them into new verifiable challenges. Across nine downstream configurations on 30 tasks, its mean evaluation-check pass rate rises from 59.4% with Qwen3.8-27B to 72.7%.

  • Problem

    Agents must determine which files are authoritative, current, and corroborated, but relevant relationships across files may be absent from retrieved text or left implicit.

    Source: §1 · p. 2

  • Method

    Env-Rethink organizes files and evidence-linked context, learns environment preparation from tool-using trajectories, and evolves environments with corresponding reference outcomes and evaluation checks.

    Source: §1 · pp. 2–3, §2.2 · p. 4

  • Results

    Across nine downstream configurations on 30 tasks, Env-Rethink raises mean evaluation-check pass rate from 59.4% with Qwen3.8-27B to 72.7%.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Evolving environmental states under an unchanged request while updating references and checks yields new, verifiable challenges for testing preparation under new conditions.

    Source: §2.2 · p. 4

    The motivating analysis identifies incomplete evidence, incorrect version choices, and outputs that miss user intent as recurring noisy-environment failures.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

Many real-world tasks (e.g., office workflows, scientific experimentation) require LLM agents to interact repeatedly with their environments for context-dependent operations. However, such environments are often not agent-ready. First, information is often scattered and fragmented across the environment. Second, relevant evidence in the environment is often mixed with misleading information and conflicting versions. Third, environments evolve over time, introducing new noise and more challenging tasks. These challenges can substantially degrade performance for state-of-the-art AI agents (e.g., from 83.9% to 57.6%). To address these challenges, we propose Env-Rethink (a system with 27B post-trained model) that supports three main capabilities: (1) It adaptively builds Collection Maps (for organizing related files) and Event Logs (for contextualizing cross-data relationships) to supplement necessary context; (2) It further leverages the post-trained model (through offline trajectory learning) to identify underlying noise issues in the environment; (3) It ultimately evolves environments through virtual event histories that alter environmental states and evidence relationships, producing more tricky ones for further agent improvement. Experiments show that Env-Rethink can effectively improve downstream task performance (with over 15.1% rubric pass rate improvement across nine models on 30 tasks).

1 Introduction

Persistent environments scatter relevant evidence across files, where conflicting versions and implicit relationships make it difficult to determine what to trust. Env-Rethink organizes and prepares existing environments, then evolves them into harder, verifiable instances.

Motivation

  • Agents can select misleading sources, trust withdrawn versions, or mistake repeated records from one source for independent confirmation.These failures arise when relationships among files remain unresolved, despite agents being able to read the files and produce the requested artifact.Source: §1 · p. 2

Contributions

  • Collection Maps group semantically related files, while Event Logs reconstruct evidence-reviewed workflows and expose revision and provenance relationships.Both accompany the original files during task execution.Source: §1 · pp. 2–3
  • A trajectory-supervised preparation model learns to read files and verify source authority and version relationships, then selects files and prepares an evidence report for a separate task agent.Preparation may use either environment-only or task-guided information.Source: §1 · pp. 2–3
  • Event-driven evolution introduces plausible incorrect choices together with resolving evidence, while updating reference outcomes and evaluation checks under an unchanged request.Generated instances undergo structural and answer validation, followed by paired seed–variant runs to assess difficulty.Source: §1 · pp. 2–3

Evaluation

  • Across nine downstream configurations on 30 tasks, Env-Rethink raises mean evaluation-check pass rate from 59.4% with Qwen3.8-27B to 72.7%.On Terminal-Bench 2.1, evolution lowers task success in at least three models on 32 of 55 retained tasks with paired results.Source: §1 · p. 3

2 Preliminaries and Motivating Analysis

The section distinguishes the task model, harness, and persistent environment, then defines preparation and evolution as environment-side operations. Its motivating analysis shows that noisy environments reduce performance and elicit recurring evidence and version errors.

2.1 Model, Harness, and Environment

  • The task model and harness form the downstream agent; the environment comprises persistent resources and state, with interventions distinct from task-model parameter updates.The environment is represented as E = (F, S), where F comprises files and service implementations and S denotes their current state.Source: §2.1 · p. 3

2.2 Environment Preparation and Evolution

  • Preparation organizes evidence in an existing instance, whereas evolution changes environmental state while retaining the request and consistently updating reference outcomes and evaluation checks.Figure 2 contrasts these two roles; generated instances can provide new learning opportunities.Source: §2.2 · pp. 3–4, Figure 2 · p. 4

2.3 Limitations of Existing AI Agents

  • 83.9% in Clean falls to 57.6% in Noise, a 26.3-percentage-point decrease averaged across nine model–harness configurations.The comparison uses 30 difficult tasks and holds the model, harness, and reasoning effort fixed.Source: §2.3 · p. 4
  • Noisy-environment failures include incomplete reading, trusting filenames as version indicators, and reusing similar files without checking task requirements.These behaviors can produce incomplete evidence, incorrect version choices, and outputs that miss user intent.Source: §2.3 · p. 4

3 System Overview

Env-Rethink combines environment organization, learned file preparation, and environment evolution to support both task execution and challenge generation.

3 System Overview

  • Env-Rethink has three modules: evidence-linked environment organization, learned file verification and preparation, and evolution that produces changed states with updated outcomes and checks.The system supports requests to prepare an existing environment for execution or generate a challenge from a task, seed files, and generation direction.Source: §3 · pp. 4–5
  • In the budget example, the verifier selects evidence supporting a total of 35; an evolved version adds an amendment to 50 and an unapproved draft of 55.The task remains unchanged, but the evolved case requires resolving version and approval relationships.Source: §3 · p. 5

3.1 Modules and Interfaces

  • The organization module builds Collection Map and Event Log, while the evolution module produces changed states with corresponding reference outcomes and checks.The evolution history is a generation record, unlike Event Log, which supplies context to the task agent.Source: §3.1 · p. 5

3.2 Offline Learning and Task Execution

  • Offline learning fine-tunes a verifier on tool-using teacher trajectories; at deployment it prepares selected files and an evidence report for a separate task agent.The organization module instead exposes Collection Map, Event Log, and source files through the file-access interface.Source: §3.2 · p. 5

3.3 Environment Evolution Workflow

  • The evolution workflow assembles and validates runnable environments, executes reference solutions, and compares seed and variant difficulty across task agents.The Terminal-Bench 2.1 study evaluates seed and evolved environments across 59 seeds; failure trajectories can guide subsequent changes.Source: §3.3 · p. 5

4 Evidence-Linked Environment Organization

Collection Map and Event Log supplement source files with cross-file groupings and evidence-linked histories, helping task agents interpret records in context without changing the originals.

4 Evidence-Linked Environment Organization

  • Collection Map identifies files to read together, while Event Log describes their use and revision as ordered, source-linked operations reviewed against evidence.Both representations are synthesized before execution without modifying source files, and can be retrieved alongside the originals.Source: §4 · pp. 5–6

4.1 Collection Map

  • Each Collection Map card summarizes a collection’s theme, contents, and representative evidence, then semantic regrouping refines directory-based groups while preserving file coverage.Collections may overlap, and affected cards are reread and updated after changes.Source: §4.1 · p. 6

4.2 Event Log

  • Event Log histories record sessions and file operations with provenance, while keeping quoted source evidence distinct from synthesizer inference.Deterministic checks and an independent reviewer assess evidence and event consistency; accepted histories are integrated into a single log with their causal order retained.Source: §4.2 · pp. 6–7

4.3 Preparation Inputs

  • Environment-Grounded Synthesis uses environment files and metadata; Task-Guided Synthesis additionally uses task supervision and labels to guide grouping and history synthesis.Task guidance aims to bring clean files together and exclude noise from that collection, while public logs must not copy or restate rubric entries.Source: §4.3 · p. 7

4.4 Task Execution

  • The Task Executor uses collection cards to locate materials, retrieves log sessions by file path, and follows source references to evidence while producing the requested deliverable.Tool-use hooks surface the prepared context during execution in both preparation settings.Source: §4.4 · p. 7

5 Learning to Prepare Agent-Ready Environments

Env-Rethink learns file verification from qualified tool-using trajectories, then uses verifier judgments to select files and prepare an evidence report for a separate task agent.

5 Learning to Prepare Agent-Ready Environments

  • The preparation model combines a file-verification target, related-record training sub-environments, and teacher trajectories that supervise both file-reading actions and final judgments.At deployment, verifier judgments drive file selection and an evidence report for the downstream agent.Source: §5 · p. 7

5.1 Verification Task

  • Verification judges file status relative to other records in a supplied collection, distinguishing a file’s expressed stage from its interpreted category and family relationships.Outputs include a standard/noise partition, category, observed stage, family assignment, timeline information, and supporting evidence; the downstream agent determines relevance to the request.Source: §5.1 · p. 8

5.2 Data and Teacher Trajectories

  • Training sub-environments retain related records and evidence relationships; teacher trajectories are admitted only after checks of file-reading fidelity and classification judgments.A separate revision agent can use failed trajectories and annotations to revise hints, with up to two further teacher attempts; still-failing sub-environments are excluded.Source: §5.2 · p. 8

5.3 Model Training

  • Training examples pair verification instructions with file-reading interactions and final judgments, applying the loss to assistant tokens rather than file contents returned by tools.LoRA fine-tuning of Qwen3.8-27B produces Env-Rethink.Source: §5.3 · pp. 8–9

5.4 Environment Preparation and Deployment

  • At deployment, the verifier’s judgments determine selected files and an evidence report that the task agent reads before executing the original request.Figure 4 depicts qualified teacher trajectories and the separate task agent receiving the prepared materials.Source: §5.4 · p. 9, Figure 4 · p. 9

6 Event-Driven Environment Evolution

Event-driven evolution creates harder instances of the same request by coordinating environmental changes with updated reference outcomes and evaluation checks.

6 Event-Driven Environment Evolution

  • Environment evolution changes a seed environment while keeping its user request, updating the reference solution and checks when changed records alter the applicable answer.The generated answer must remain determinable from the files.Source: §6 · p. 9

6.1 Decision Points and Reference Outcomes

  • Each counter-default decision point pairs a plausible incorrect choice with uniquely resolving evidence and a check that rejects the incorrect choice.Variants require at least three such points and evidence chains no shorter than the seed’s.Source: §6.1 · p. 10

6.2 Event-Driven Environment Generation

  • The Evolver produces file changes, event-history fragments, and revised reference outcomes and evaluator checks, while later generations can build on earlier changes.Each update preserves the task objective and existing scoring criteria; the generation history is not supplied as task-execution context.Source: §6.2 · p. 10

6.3 Instance Validation

  • Validation checks structure and answer correctness, requiring the new reference solution to pass while the previous expected output fails.Shortcut audits check whether generated materials reveal conclusions the task requires the agent to infer.Source: §6.3 · p. 11

6.4 Difficulty Evaluation and Feedback

  • Matched seed–variant runs compare task success and failed-check sets, and available failure summaries can guide the Evolver’s next generation.Records retain seed identifiers, artifacts, checks, and acceptance results, including repair attempts.Source: §6.4 · p. 11

7 Experiments

The experiments evaluate organized synthesis, learned file preparation, and event-driven environment evolution as complementary components. Results report downstream gains and lower success on many evolved Terminal-Bench tasks.

7.1 Evaluation Overview

  • 26.3 percentage points separates mean pass rates in Clean (75.1%–95.1%) and Noise (45.1%–68.5%) across configurations.The evaluation compares Clean and Noise conditions across the same 30 tasks.Source: §7.1 · p. 11

7.2.2 Synthesis Results

  • 87.3% with TGS, 82.9% with EGS, and 64.4% with Noise are reported for DeepSeek-V4.1-Flash; TGS beats EGS and both beat Noise for all five models.TGS adds task-specific supervision to the Collection Map, Event Log, and tool-use hooks shared by both synthesis modes.Source: §7.2.2 · pp. 11–12, §7.2.1 · p. 11, Table 1 · p. 12

7.2.3 From Context Synthesis to Environment-Understanding Training

  • Task-guided synthesis outperforms environment-grounded synthesis in matched comparisons, motivating learned judgments about valid evidence and file relationships.The proposed preparation targets include acquiring evidence, identifying source authority, resolving versions, and preparing structured reports.Source: §7.2.3 · p. 12

7.3.1 Module Performance

  • On held-out environments, partition accuracy increases from 56.2% to 76.5%, while joint partition, category, and stage accuracy rises from 19.4% to 32.4%; the split has 15 tasks and 617 files.Source: §7.3.1 · p. 12

7.3.2 Downstream Utility

  • 72.7% with Env-Rethink versus 59.4% with Qwen3.8-27B and 57.6% with full environments across nine models; gains occur across all nine.The comparison covers the same 30 tasks.Source: Figure 6 · p. 13, §7.3.2 · p. 13

7.3.3 Transfer and Reuse

  • The same Env-Rethink model prepares environments for all nine downstream models, with file judgments determining the evidence available to each task agent.Environment evolution extends this preparation work by using virtual event histories to create harder instances under the same request.Source: §7.3.3 · p. 13

7.4 Environment Evolution

  • 32 of 55 retained tasks (58.2%) show lower success in at least three models after evolution, including 21 tasks with decreases in all four.The effects span ten categories; 10 of 13 software-engineering tasks, four of five debugging tasks, and all three model-training tasks meet the three-model criterion.Source: §7.4 · p. 14, Figure 7 · p. 14
  • Evolved states can lower success despite unchanged task requests; examples include all four models falling from 3/3 to 0/3 on modernize-scientific-stack.The reported task-level changes require agents to recover which records, versions, and operational assumptions govern the current outcome.Source: §7.4 · p. 14

8 Related Work

Related work covers agent interfaces, environment construction, learning from interaction, and feedback-guided challenge generation. These approaches address information access, verifiability, reuse of interaction records, or adaptation of challenge difficulty.

Agent tools and context management

  • Agent interfaces support information access and continuity through file search, selective retrieval, persistent notes, structured handoffs, and repository tools.OpenAI, Anthropic, and SWE-agent are cited as examples.Source: §8 · p. 14

Environment adaptation and generation

  • Environment approaches adapt existing environments or generate training tasks, simulations, and stateful or executable settings [8] [9] [11] [12].The cited work treats environment structure and verifiability as explicit design choices.Source: §8 · pp. 14–15

Learning from interaction

  • Interaction-based methods reuse tool observations, textual feedback, practitioner corrections, or recurring failures to inform later decisions.The cited examples include ReAct, Reflexion, Tax AI, and Signals.Source: §8 · p. 15

Feedback-guided environments

  • POET and PAIRED link challenge generation to learner behavior, using solution optimization or agent returns to shape difficulty.POET also transfers solutions across environments.Source: §8 · p. 15

9 Conclusion

The conclusion reports gains from environment organization and learned preparation, alongside harder challenges generated through environment evolution. The appendix documents the data, annotation, teacher-trajectory, and qualification procedures supporting file verification.

9 Conclusion

  • 32 of 55 retained Terminal-Bench tasks show lower success in at least three models under an unchanged request, alongside reported gains from organization and learned preparation.The paper identifies independently collected environments, joint module use, and training on validated evolved instances across repeated rounds as future work.Source: §9 · p. 15

A.1 Label Definitions

  • Files are labeled by partition, category, observed stage, family, and evidence; six categories describe authority and relationships within a collection.The appendix frames these procedures as support for the file-verification module.Source: §A.1 · p. 17, A Data · p. 17

A.2 Data Manifest; A.3 Selection

  • The selection pool contains 15 tasks and 1,579 files; selection favors strong clean performance, substantial noise sensitivity, and file-level evidence of failure.The manifest permits identical file sets at most twice and checks file, family, and semantic overlap across sub-environments.Source: §A.3 · p. 17, §A.2 · p. 17

A.4 Annotation

  • Annotation initializes stage and category from filename and path rules, uses task-analysis reports for strong decoys, and applies human review to decoys and disagreements.The construction record reports consistency checks passed for all 1,597 labeled files.Source: §A.4 · p. 17

A.5 Sub-Environments

  • Sub-environments combine attack scenes, version families, redirects, and fillers while varying authoritative records and checking whether visible evidence supports each label.The specified batch mixture is 45%/35%/20% across small, medium, and large sizes.Source: §A.5 · p. 18

A.6 Teacher Reading

  • Teacher trajectories come from DeepSeek, GLM, and Gemini families; hints range from task-only instructions to checking sequences, with revised hints used for retries.Complete messages retain tool calls, observations, and final judgments, and read-fidelity checks connect judgments to accessed content.Source: §A.6 · p. 18

A.7 Qualification and Outputs

  • Qualification requires partition F1 ≥0.9, no false rejection of authoritative records, correct strong-decoy classifications, and correct ordering within version families.Up to two additional retries are permitted; remaining failures are retained for analysis.Source: §A.7 · p. 18
  • The record reports 2,116 qualified trajectories linked to file, family, scene, annotation, and sub-environment manifests.Source: §A.7 · p. 18

A.8 Unseen-Task Extension

The appendices describe an unseen-task extension, model training and trajectory preparation, and evaluation of file verification, environment preparation, and downstream performance.

A.8 Unseen-Task Extension

  • The unseen-task extension adds nine tasks with annotations for 304 files, and those tasks enter the unified 30-task downstream evaluation.Six existing-noise tasks contribute 224 files, while three other tasks receive 68 generated noise files.Source: §A.8 · p. 18

B.1 Training Configuration

  • Env-Rethink adapts Qwen3.8-27B with LoRA, using model-native tool-call serialization to preserve rendered training messages.The run completed three epochs in 168 optimization steps.Source: §B.1 · p. 18, Table 4 · p. 19

B.2 Optimization Diagnostics

B.3 Trajectory Preparation and Deployment

  • Trajectory preparation truncates excessively long tool outputs while preserving invocation-response pairs, discards remaining overlength samples, and rotates retained trajectories among teachers.The assembled corpus contains 1,780 training and 201 development examples; deployment merges the LoRA adapter after step 168.Source: §B.3 · p. 20

B.4 Module Evaluation Details

  • On held-out environments, Qwen3.8-27B and Env-Rethink produce 120 and 200 correct joint file judgments, respectively.The evaluation pools predictions over 617 files from 15 held-out tasks; joint accuracy requires partition, category, and stage judgments to match reference labels.Source: §B.4 · p. 20

B.5 Environment Preparation Details

  • Across 30 tasks and 2,196 files, Env-Rethink retains 160 of 192 standard files and admits 271 noise files, compared with 136 and 606 for Qwen3.8-27B.Source: §B.5 · p. 20, Table 6 · p. 20

B.6 Thirty-Task Downstream Evaluation

  • Downstream evaluation compares Full, Qwen3.8-27B, and Env-Rethink conditions across nine models on the same 30 tasks, pooling passed rubric checks.Each condition covers 11,520 checks across the nine models; missing outputs count as zero passed checks while retaining the task denominator.Source: §B.6 · p. 20, Table 7 · p. 21

B.7 TGS Component Ablations

Removing either TGS environment component lowers DeepSeek-V4.1-Flash’s rubric pass rate. The event-driven protocol keeps each changed environment aligned with its reference outcome and evaluator.

B.7 TGS Component Ablations

  • Removing Collection Map lowers rubric pass rate from 87.3% to 75.1% (−12.2 percentage points); removing Event Log lowers it to 84.8% (−2.4 percentage points).Both ablations retain tool-use hooks and the other environment component.Source: Table 8 · p. 21, §B.7 · p. 21

C Event-Driven Generation Protocol

  • The protocol represents the materialized environment separately from its generation record, which retains history, reference outcome, evaluator, and variant information.The task agent receives the environment and unchanged request; generation and verification retain the record.Source: C Event-Driven Generation Protocol · p. 21
  • Generation uses a seed or prior variant, history, and execution feedback to produce environmental changes, causally linked events, and revised reference outcomes.Assembly applies the changes to create a runnable environment.Source: C Event-Driven Generation Protocol · p. 21
  • Variants require at least three counter-default decision points, each linking a plausible wrong choice to resolving evidence and an executable check.Validation also checks event links, file references, evaluation assets, and request preservation; reference execution checks that the new answer passes and the prior answer fails, while shortcut audits inspect for bypasses.Source: C Event-Driven Generation Protocol · pp. 21–22

C.1 Evaluation Settings

Evaluation covers retained Terminal-Bench seeds and paired seed–variant outcomes, with task success, trial-based screening, and category means defined by specific inclusion rules.

C.1 Evaluation Settings

  • 32 of 55 retained paired tasks showed a success-rate decrease of at least 20 percentage points in three or more models; four did so in exactly two, and 19 in at most one.The reported 58.2% uses the 55 retained paired tasks as its denominator; sanitize-git-repo was excluded after its variant reference run failed.Source: §C.1 · p. 22
  • Evaluation matched model configurations and execution budgets across seed and variant, targeting three independent trials per arm.Four Flash models were evaluated: DeepSeek-V4.1, Gemini-3.8, Qwen3.8, and GLM-5.3.Source: §C.1 · p. 22
  • Task success requires passing all native checks; check-level pass rates are used only for diagnosis.The main result screens for tasks with h_i ≥ 3; one changed outcome across three trials corresponds to 33.3 percentage points.Source: §C.1 · p. 22
  • Category means average task-level contrasts after excluding missing pairs and pairs with zero seed success.Table 9 reports sample counts; “Other” combines games, personal assistant, mathematics, and optimization.Source: Table 9 · p. 22, §C.1 · p. 22

Related papers

Loading 2609.29773v1…