Source-linked AI summary
EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent
Yirong Zeng, Shen You, Jinhang Feng, Yufei Liu, Xiao Ding, Yutai Hou, Hao Cong, Yuxian Wang, Wu Ning, Wang Xu, Bibo Cai
TL;DR
Agentic RL for Claw-like agents lacks scalable environments that support stateful, long-horizon execution beyond isolated tool endpoints. EnvCraft automates executable sandbox and task synthesis, producing verifiable trajectories and demonstrating gains across Claw-style and general tool-use benchmarks. The authors report up to +11.9 points on Claw-style benchmarks and +8.0 on general tool-use benchmarks, alongside reduced inference token costs.
Problem
Scalable Agentic RL is limited by scarce executable environments, while existing synthetic environments focus on isolated tool endpoints rather than stateful Claw-like workflows.
Method
EnvCraft automatically synthesizes sandbox-isolated executable environments and verifiable RL tasks from natural-language descriptions using environment and data generation engines.
Results
EnvCraft improves both Claw-style and general tool-use benchmark performance, with reported gains up to +11.9 points and +8.0 points, while reducing inference token costs.
Takeaways & Limitations
The results support scalable automated executable-environment synthesis as a viable approach for training claw-like agents.
Takeaways & Limitations
Synthesized environments remain simplified abstractions that may not capture external-service failures, nondeterminism, evolving interfaces, or complex deployment security constraints.
Abstract
from arXiv · showhide
The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic Reinforcement Learning (Agentic RL) provides a promising path to optimize these agents, its scaling is heavily bottlenecked by the severe scarcity of interactive training environments. Existing synthetic environments are strictly limited to tool-calling endpoints, rendering them insufficient for accommodating the end-to-end real-world demands of claw-like agents. To bridge this gap, we introduce EnvCraft, an automated framework for synthesizing executable environments and scalable training data. Specifically, EnvCraft employs an environment synthesis engine to build sandbox-isolated workspaces, alongside a topology-aware data generation engine to produce coherent task trajectories. Overall, we synthesize 139 interactive environments comprising approximately 20K complex tasks for Agentic RL training. Experiments on Qwen3/3.5 models (8B-32B) show that our method yields gains of up to +11.9% on Claw-style benchmarks and +8.0% on general tool-use benchmarks, with concurrent reductions in inference token cost. The results confirm that synthesized executable environments provide robust and generalizable learning signals for training.
1 Introduction
Claw-like agents execute long-horizon tasks in stateful systems, but scalable Agentic RL is bottlenecked by scarce executable environments. EnvCraft addresses this gap and improves performance across Claw-style and general tool-use benchmarks.
- Claw-like agents operate across operating systems, file systems, databases, and terminal shells, directly mutating persistent environments.
- Executable environments must support tool execution, state maintenance, result feedback, low latency, high fidelity, and cost-effective training.
- Claw-specific training improves PinchBench by up to +11.9 points and Claw-Eval by up to +11.4 points while reducing per-task token cost by up to 35%.
- EnvCraft gains transfer to disjoint general-purpose tool-use benchmarks, including BFCL-v3 and τ 2-bench.
2 Preliminaries
EnvCraft formalizes an executable environment as documentation, tools, persistent state, and transition dynamics, then represents each training task with a query, initial state, and verifier.
- An executable environment is modeled as E = ⟨Ddoc, Itool, S, T⟩, connecting RL abstractions to executable system components.
- Ddoc specifies system rules and safety boundaries, Itool defines exposed interfaces, and S captures persistent observable state.
- Environment state separates underlying data states from temporal runtime sessions.
- Each training datum is P = ⟨Q, S0, Vscript⟩, combining a user query, concrete initial workspace state, and execution verifier.
- The verifier audits state transitions from S0 to Sn to produce deterministic, programmatic reward signals.
3 Environmental Synthesis Engines
EnvCraft synthesizes executable environments from structured scenario specifications and validates them through static and dynamic auditing. Its prototype-driven process produced 139 accepted environments from 324 raw candidates.
- Scenario collection: The pipeline collects general tool-use and Claw-specific scenarios before expanding them into executable environment specifications and tool contracts.
- Prototype-driven synthesis: Prototype-driven synthesis uses interaction archetypes, reference implementations, and architectural blueprints to generate standardized environments.
- Environment generation: The synthesis engine combines interaction prototypes, environment specifications, and tool specifications in a structured prompt for executable code generation.
- Claw-specific enhancements: Claw-specific enhancements add system-level Bash access and concurrently generate reusable domain-tailored Agent Skills.
- Quality auditing: Static AST checks and dynamic runtime tests verify code structure, tool schemas, state transitions, and edge-case handling.
- Quality auditing: 139 environments passed auditing from 324 raw candidates, a 42.9% pass rate comprising 70 general-tool and 69 Claw-specific environments.
4 Data Synthesis Engines
EnvCraft synthesizes executable tool-use tasks by modeling dependencies across tools and environments, then converting sampled chains into interactive, verifiable training records. Auditing filters these records for consistency, difficulty, and execution quality.
- Tool Chain Sampling: EnvCraft represents tool and environment dependencies as a bi-level directed graph for coherent single- and cross-environment workflows.The lower graph captures intra-environment tool dependencies, while the upper graph links environments through semantic domains and data flows.
- Tool Chain Sampling: A hierarchical weighted random walk samples tool chains, traversing one tool graph or sequencing typically 2–5 connected environments before intra-environment walks.These chains become execution skeletons for downstream intent reverse-engineering and scenario construction.
- Task Construction: The synthesis process hides tool-derived values, decomposes composite intents into incremental user instructions, and preserves explicit user-specified values.This design makes agents resolve intermediate results through sequential tool execution rather than receiving hidden ground-truth values directly.
- Task Construction: EnvCraft instantiates initial workspaces with required entities and distractors, then generates deterministic verification scripts packaged with the query and state.The resulting task record contains the task identifier, user task, initial environment state, and executable verifier.
- Quality Auditing: 19,777 high-quality samples were retained from 35k raw candidates, a 55% pass rate, after rubric-based auditing and difficulty calibration.The final set contains 14,215 general tool-use and 5,562 Claw-specific instances, excluding tasks with 0% or 100% pass rates under eight Qwen3.5-27B trials.
5 Experiments
Experiments train Qwen3 and Qwen3.5 models with EnvCraft data and evaluate them on Claw-style and general tool-use benchmarks. EnvCraft improves performance across model configurations, with Claw-specific training producing large task gains and lower inference token costs.
- Setup and Main Results: EnvCraft training improves Claw-style benchmark scores across Qwen3-8B, Qwen3-32B, and Qwen3.5-9B configurations.The evaluation uses PinchBench and Claw-Eval for end-to-end long-horizon tasks, with average scores reported over three runs.
- Claw-style Benchmarks: Qwen3-8B gains +11.91% on PinchBench and +11.36% on Claw-Eval after Claw-specific training.PinchBench rises from 12.94% to 24.85%, while Claw-Eval rises from 44.06% to 55.42%.
- Claw-style Benchmarks: Claw-specific trajectories contribute most of the improvement, while general tool-use data yields only +1.57% on PinchBench and +1.74% on Claw-Eval.Sequential Tool-to-Claw training achieves comparable gains of +11.25% and +11.57%.
- Training Dynamics: Qwen3-8B rises from 12.9% to 24.85% on PinchBench during training, while Qwen3.5-9B rises from 47.3% to 52.2%.The reported checkpoint trends show steady improvement across both backbones during reinforcement learning.
- Tool-use Benchmarks: EnvCraft-Claw training transfers to general tool-use benchmarks, including a +8.00 gain for Qwen3.5-9B on BFCL-v3.On Qwen3-8B, EnvCraft-Claw reaches 45.88% on BFCL-v3, close to EnvCraft-Tool at 46.25%.
- Inference Efficiency: EnvCraft-Claw reduces aggregate per-task token consumption from 42.5K to 34.0K, an average reduction of 20%, while improving task scores.The reductions are 35% for Qwen3-8B, 16% for Qwen3-32B, and 8% for Qwen3.5-9B.
6 Conclusion
EnvCraft automates the synthesis of executable environments and verifiable RL training tasks, producing 139 environments and approximately 20K training instances. Experiments show consistent gains on Claw-style and general tool-use benchmarks alongside reduced inference token costs.
- EnvCraft automates executable-environment and verifiable-task generation from natural-language descriptions, removing the need for manual environment engineering.
- 139 RL environments and approximately 20K training instances are synthesized at scale.
- EnvCraft training consistently improves performance on both Claw-style and general tool-use benchmarks.
- The improvements are accompanied by reduced inference token costs.
Limitations
The synthesized environments and training pipeline remain bounded by abstraction, generator dependence, verifier scope, and limited evaluation coverage. These constraints affect realism, workflow coverage, task evaluability, and generality beyond the tested settings.
- Synthesized environments may not capture external-service failures, nondeterministic behavior, evolving interfaces, or complex deployment security constraints.
- LLM-generated specifications, dependency graphs, intents, and verification scripts can propagate generator biases or errors across synthesis stages.
- Predefined interaction archetypes and topology-based sampling may underrepresent workflows outside the covered design patterns.
- State-based deterministic verifiers are less suitable for subjective-quality tasks, multiple acceptable outcomes, or purely read-oriented objectives.
- Experiments focus on Qwen3 and Qwen3.5 backbones and a limited set of agent benchmarks, leaving broader models, production systems, and longer workflows for future evaluation.
Ethical Statement
EnvCraft is intended for controlled research environments because its generated environments and tasks involve potential risks. Before release or deployment, they should undergo security auditing, access control, and human review, especially when sensitive or consequential tools are involved.
- EnvCraft is intended for controlled research environments because the proposed capabilities involve potential risks.
- Generated environments and tasks should receive security auditing, access control, and human review before release or deployment.
- These safeguards are especially important for sensitive data, external services, or tools capable of consequential state changes.
Generative AI Statement
The paper situates EnvCraft within Agentic RL and executable-environment synthesis, while the supplied statement says generative AI was used only to refine writing. The surrounding research distinguishes production, simulated, and executable synthetic environments and describes GRPO-based trajectory optimization.
- Agentic Reinforcement Learning: Agentic RL optimizes complex tool-use and multi-turn policies through autonomous exploration and rule-based or trajectory-level feedback.
- Related applications: General tool-use, search-oriented, and mathematical-reasoning agents apply reinforcement learning to tool selection, evidence retrieval, and interactive problem solving.
- Executable Environment Synthesis: Executable-environment synthesis is presented as a category alongside production and simulated environments for scalable agent RL training.
- Environment categories: Production environments offer authentic execution but are expensive to scale and can suffer network latency and irreproducible states.
- Environment categories: Simulated environments enable rapid prototyping but are prone to hallucination, additional inference cost, and instability.
- Environment categories: Executable synthetic environments reconstruct tools and stateful backends through sandbox code generation, balancing scalability, reproducibility, and execution fidelity.
- Optimization: GRPO samples groups of candidate responses and optimizes agent trajectories using outcome rewards from deterministic task verification.
- Scenario construction: The work organizes environment domains into 83 fine-grained scenarios spanning 41 general tool-use and 42 Claw-specific scenarios.
B.3 Interaction Archetype Specifications
EnvCraft models diverse Claw-like workflows through nine interaction archetypes and synthesizes executable, session-isolated environments with deterministic tools, state transitions, evaluation logic, and tests.
- Interaction archetypes: EnvCraft defines nine interaction archetypes covering multi-tool orchestration, asynchronous execution, and dynamic context shifts.
- Interaction archetypes: The archetypes include transactional workspaces, human-agent interaction, event triggering, state transitions, temporal scheduling, bilateral matching, and stream processing.
- Interaction archetypes: Fault-tolerant stream processing models typed event streams, filtered subscriptions, and multiple delivery modes for asynchronous publish-subscribe systems.
- Environment synthesis: The synthesis protocol generates standalone Python packages with persistent sessions, domain tools, deterministic transitions, executable evaluation, documentation, and concurrency tests.
- Environment synthesis: Explicit contracts and unified schemas make generated samples executable and deterministically verifiable while enabling automated audits of interfaces, state isolation, paths, and reward logic.
D.4 Evaluation Configuration
Evaluation covers Claw-style end-to-end tasks and general multi-turn tool-use benchmarks under standardized decoding, execution, and repeated-run settings.
- Decoding and execution: Evaluation uses temperature 1.0, top-p 0.6, maximum generation length 32768 tokens, at most 32 agent turns, and a 300-second timeout.
- Reporting: Scores are averaged over eight independent runs, with reported error terms given as standard deviations across runs.
- Benchmark configuration: Models are evaluated on PinchBench and Claw-Eval for long-horizon Claw tasks, plus BFCL-v3 and τ 2-bench for multi-turn tool use.
D.5 Training Dynamics and Data Composition
EnvCraft training improves both training and held-out performance, with Claw-specific trajectories producing much larger Claw-task gains than general tool-use data while retaining general tool-use capability.
- Training dynamics: All three backbones improve on training and 50 held-out EnvCraft-Tool tasks despite checkpoint-level fluctuations.
- Data composition: Claw-specific training raises Qwen3-8B by +11.91% on PinchBench and +11.36% on Claw-Eval, versus +1.57% and +1.74% from general tool-use training.
- Data composition: Sequential Tool-to-Claw training achieves comparable Qwen3-8B results of 24.19 on PinchBench and 55.63 on Claw-Eval.
- Interpretation: Stateful workspaces, long-horizon execution, and cross-tool dependencies contribute the majority of system-level gains, while general tool-use data provides complementary supervision.
- Capability retention: After switching to Claw-specific environments, BFCL remains 60.0 compared with 59.5 before the transition.
D.6 Data Quality Analysis
EnvCraft combines LLM-based auditing, strict rubric gates, rollout-based difficulty calibration, and environment ablations to assess and improve the quality and usefulness of synthesized tasks.
- Quality auditing: The inspector evaluates task solvability, validation-protocol integrity, and cross-component data consistency from code, state, intent, and validation inputs.
- Quality auditing: Candidates are retained only when ssol = 3, sval = 3, scon ≥2, and ssol + sval + scon ≥8.
- Quality auditing: The strict gate removes tasks with contradictory logic, inconsistent states, or unreliable reward functions before rollout-based difficulty calibration.
- Audit effectiveness: Retained candidates have a mean rubric score of 2.97 versus 1.38 for filtered candidates, with the largest separation in task solvability.
- Difficulty calibration: Difficulty calibration uses eight independent rollouts, treating verifier rejection, action-turn exhaustion, and 300-second timeouts as unsuccessful outcomes.
- Ablation studies: Built-in system-level tools raise PinchBench from 14.96 to 24.02, while parallel skill co-synthesis further increases it to 24.33.