PaperlayerView exact revision on arXiv

Source-linked AI summary

Computer Environments Elicit General Agentic Intelligence in LLMs

Daixuan Cheng, Shaohan Huang, Yuxian Gu, Huatong Song, Guoxin Chen, Li Dong, Wayne Xin Zhao, Ji-Rong Wen, Furu Wei

arXiv:2601.16206v3cs.CLcs.AI

TL;DR

The paper asks whether computer environments themselves can elicit general capabilities from LLMs, an important question because existing systems combine environmental interaction with other optimizations. It introduces a minimal code-sandbox paradigm and a reinforcement-learning extension trained on non-agentic data, finding broad gains and efficiency improvements that support computer environments as a foundation for generalist agents.

  • Problem

    The intrinsic value of computer environments for eliciting general capabilities from LLMs has not been systematically investigated.

  • Method

    LLM-in-Sandbox gives LLMs a minimal virtual computer for terminal access, file management, and code execution, while LLM-in-Sandbox-RL trains environmental interaction using non-agentic data.

  • Results

    Strong models gain across mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following, with improvements reaching +15.5% in mathematics, while weaker models can initially perform worse.

  • Takeaways & Limitations

    Computer environments provide a systematic mechanism for eliciting broader LLM capabilities, improving capability and efficiency together and offering a foundation for generalist agents.

  • Takeaways & Limitations

    The reported creative outputs remain limited: videos are simple 11-second animations, music lacks human-level expressiveness and creativity, and posters may not match professional quality.

Abstract

from arXiv · show

Agentic intelligence in large language models (LLMs) requires not only model intrinsic capabilities but also interactions with external environments. Equipping LLMs with computers now represents a prevailing trend. However, the computer environment's intrinsic value has not been systematically investigated, particularly its potential to elicit general capabilities. Here we introduce LLM-in-Sandbox, which virtualizes the computer as a code sandbox with only basic functionalities, and demonstrate that this minimal setting elicits computer-based meta-capabilities for general task solving: external resource access, file management, and code execution. Without additional training, strong models achieve substantial gains (up to 15.5%) across mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following, while reducing token consumption by up to 8 times. Furthermore, we develop LLM-in-Sandbox-RL to train models exclusively on non-agentic data within the sandbox, empowering weaker models to harness the environment and internalize these interactions. Our results demonstrate that computer environments elicit general intelligence, yield efficiency gains, and can be harnessed through training, serving as a promising foundation for generalist agents.

1 Introduction

The paper investigates whether a minimal computer environment can elicit general agentic capabilities from LLMs beyond intrinsic model ability. LLM-in-Sandbox and its reinforcement-learning extension show cross-domain gains while isolating environmental interaction and addressing weaker models’ difficulties.

  • Motivation: Computer environments may determine whether LLMs’ intrinsic capabilities translate into effective task solving across diverse domains.The paper frames environmental interaction as a defining architectural component for retrieving information, taking actions, and adapting to feedback.
  • Research gap: Existing computer-using systems combine environment interaction with high-level interfaces and system optimizations, making the environment’s intrinsic contribution difficult to isolate.The paper identifies this confounding problem as the motivation for a minimal experimental setting.
  • Approach: LLM-in-Sandbox virtualizes a computer as a minimal, exploratory code sandbox supporting terminal access, file management, and code execution.The design excludes domain-specific additions and tests whether benefits extend beyond coding into non-code domains.
  • Results: +15.5% in mathematics and +14.4% in instruction following are among the gains strong models achieve without additional training across six evaluated domains.The evaluation covers mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following.
  • Results: External resource access, file management, and code execution emerge as environment-elicited meta-capabilities that support general task solving.Models use these capabilities to acquire knowledge, process documents exceeding context windows, and verify answers computationally.
  • Training extension: LLM-in-Sandbox-RL trains models with non-agentic tasks inside the sandbox to help weaker models learn to leverage environmental interaction.The setup stores background materials as files and uses outcome-based rewards without coding- or benchmark-specific training.
  • Overall findings: Computer-environment gains appear in both training-free and post-trained settings, alongside substantial efficiency advantages.The paper also reports that post-training gains extend to settings without environmental access and supports release of an open-source library.

2 Computer Environments Elicit General Intelligence

LLM-in-Sandbox virtualizes a computer as a minimal, general-purpose code sandbox that supports external resource access, file management, and code execution through multi-turn interaction. Across diverse tasks, strong models benefit from these capabilities, while usage patterns vary by domain and weaker models may use the environment inefficiently.

  • 2.1 Virtualizing the Computer as a Code Sandbox: LLM-in-Sandbox provides a minimal, general-purpose sandbox whose core computer capabilities are external resource access, file management, and code execution.The environment uses basic tools and lets models acquire or create additional domain-specific functionality when needed.
  • 2.2 LLM-in-Sandbox Workflow: The workflow uses multi-turn ReAct-style interaction: models issue tool calls, receive sandbox observations, and continue until completion or a turn limit.The workflow includes bash, file_editor, and finish, with outputs extracted from the sandbox.
  • 2.3 General Task Performance in Computer Environments: Strong models improve across computation-intensive, knowledge-intensive, and general-capability domains when using LLM-in-Sandbox.The largest reported gain reaches +15.5% for Qwen3-Coder on Mathematics, whereas weaker models can perform worse.
  • 2.4.2 Quantitative Analysis: Mathematics has the highest computation frequency at 43.4%, while Chemistry has the highest external resource frequency at 18.4%.Mathematical usage reflects numerical verification, whereas chemistry models install domain-specific packages.
  • 2.4.2 Quantitative Analysis: Long-context tasks emphasize file operations, and storing documents in the sandbox generally improves performance, although Qwen performs worse with environment-based context.Models use shell tools and scripts to locate and extract information from reports exceeding 100K tokens.
  • 2.4.2 Quantitative Analysis: Strong models use all three capabilities at 6–21% rates, whereas Qwen3-4B-Instruct uses them below 3% while taking 23.7 versus 12.6 turns.The comparison characterizes the weaker model as wandering without effective tool utilization.

3 Computer Environments Enable Efficient Deployment

The sandbox can improve deployment efficiency despite multi-turn exploration, especially for long-context tasks, while maintaining competitive throughput and lightweight infrastructure overhead.

  • Cost: Up to 8× fewer tokens are consumed for long-context tasks when content is stored in local files rather than prompts.For Qwen, consumption falls from 100K to 13K tokens.
  • Speed: 37%–51% of trajectory tokens come from the environment, while environment execution accounts for less than 4% of total time.Environment tokens are processed through fast prefill rather than slow autoregressive decoding.
  • Speed: MiniMax achieves a 2.2× throughput speedup, while other models range from 0.6× to 1.1×.Throughput is measured in queries per minute from submission to final answer.
  • Infrastructure: Each sandbox uses approximately 50 MB idle memory and 200 MB peak memory, with 5% system-RAM overhead at K = 512.A shared Docker image of approximately 1.1 GB reduces storage requirements by orders of magnitude.

4 Reinforcement Learning within Computers Enhances Generalization

LLM-in-Sandbox-RL trains models to explore computer environments using general context-based tasks. It generalizes across domains, model capability levels, and inference modes, with sandbox-placed context producing the strongest overall performance.

  • Method: LLM-in-Sandbox-RL trains models in a computer environment using general context-based data rather than task-specific agentic data.Background materials are stored as files, requiring active environment exploration.
  • Method: Contexts are split across files or supplemented with distractors, encouraging navigation and filtering of relevant information.The final answer is written to /testbed/answer.txt after the model uses prior related tasks as in-context examples.
  • Generalization across Domains: LLM-in-Sandbox-RL improves performance across domains without overlap with evaluated benchmark training or test sets.Improvements extend from Long-Context, Math, and Physics to Instruction-Following and SWE.
  • Generalization across Inference Modes: Training exclusively in LLM-in-Sandbox mode also improves LLM mode and outperforms LLM-RL on most tasks.This result suggests that agentic skills transfer to non-agentic generation.
  • Data and Context Placement: Gen. in Sandbox achieves the best overall performance among the four training-data and context-placement variants.Placing context in the sandbox yields stronger generalization than placing the same context in the prompt.
  • Learned Interaction: After training, computer-capability usage increases across external resources, file management, and computation, while Qwen3-4B average turns decrease from 23.7 to 7.0.The weaker model replaces ineffective wandering with fewer, more purposeful interactions.
  • Learned Interaction: Reasoning patterns learned through multi-turn environmental interaction transfer to LLM mode without environment access.Each action receives explicit feedback during environmental interaction.

5 LLMs within Computer Environments Go Beyond Text Generation

Computer access extends LLMs beyond text-in-text-out generation by enabling tool acquisition, file-level operations, and multimodal digital outputs. Case studies illustrate these capabilities, while current creative outputs remain limited in quality and complexity.

  • Capabilities: LLM-in-Sandbox enables processing and generating images, videos, audio, and interactive applications through specialized software.The environment supports outputs such as .html, .png, .mp4, and .wav files.
  • Capabilities: Models can autonomously discover, install, and use arbitrary software libraries rather than relying only on predefined APIs.This provides on-demand tool access within the sandbox.
  • Capabilities: LLM-in-Sandbox directly produces usable files instead of merely describing their contents.Outputs include actual images, videos, audio files, and interactive webpages with execution-based feedback.
  • Case Studies: A travel-planning case produces a functional map.html with clickable markers and day-by-day route visualization.The agent installs Leaflet.js and generates JavaScript for 12 locations.
  • Limitations: Current generated videos are limited to simple 11-second animations, while composed music lacks human-level expressiveness and creativity.Posters follow basic design principles but may not match professional graphic-design quality.

6 Conclusion and Future Work

The paper presents computer environments as both inference infrastructure and a benchmark for agentic capabilities, and proposes training computer interaction as a transferable skill using general data.

  • Computer Environments as Default Inference Infrastructure: Computer environments can serve as default inference infrastructure for verifiable computation, file-based long-context management, and actual creative outputs.The paper envisions transforming LLMs from text generators into general-purpose digital workers.
  • Computer Environments as an Agentic Capability Benchmark: LLM-in-Sandbox provides a unified testbed for exploration, tool use, and self-verification beyond downstream-specific benchmarks.The metric ∆ = LLM-in-Sandbox − LLM measures how effectively a model leverages computational environments.
  • Computer-Native Model Training: LLM-in-Sandbox-RL trains computer interaction as a transferable skill using only general, non-agentic data.The paper proposes making computer interaction a first-class training objective.

A Sandbox Implementation

The sandbox implementation builds on R2E-Gym and adapts its framework for general-purpose exploration across diverse non-code domains.

  • The sandbox framework is built upon R2E-Gym.
  • The framework is adapted for general-purpose exploration.
  • The adaptation targets diverse non-code domains.

bash

The sandbox provides persistent terminal execution, file editing, and an explicit completion function as its core interaction tools.

  • bash executes commands in a persistent shell session.Environment variables, virtual environments, and the working directory persist across commands.
  • file_editor supports viewing, creating, and editing files.Its operations include view, create, str_replace, and insert.
  • finish ends the interaction when the task is complete or further progress is impossible.

B Model Configurations

Inference configurations specify turn and token limits for vanilla LLM and sandbox modes, with a special API-constrained limit for Claude-Sonnet-4.5-Think.

  • The maximum turn is 100.
  • The maximum generation length per turn is 65,536 tokens for most models.Claude-Sonnet-4.5-Think is limited to 64,000 tokens due to API constraints.
  • Vanilla LLM mode applies the limit to one response, while LLM-in-Sandbox applies it to each turn.

C Evaluation Details

The evaluation covers six non-code domains and one code domain using benchmark-specific datasets, repetition schemes, answer formats, and evaluators.

  • The benchmark suite contains six non-code domains and one code domain.The benchmark summary is provided in Table 14.
  • Mathematics: AIME25 uses all 30 problems, repeats each 16 times, and reports average accuracy.Math-Verify evaluates the answers.
  • Physics: UGPhysics samples 50 problems from each of 13 subjects, yielding 650 problems.Responses are evaluated by an LLM-based judge.
  • Chemistry: ChemBench samples 50 problems from each of nine sub-domains, totaling 450 problems.Evaluation uses exact match.
  • Biomedicine: MedXpertQA uses 500 text-based questions evaluated with exact match.The questions assess expert-level medical knowledge and reasoning.
  • Long-Context Understanding: AA-LCR contains 100 multi-document reasoning questions with document sets averaging approximately 100K tokens.In sandbox mode, related documents are stored as text files in /testbed/documents/; each problem is repeated four times.
  • Instruction Following: IFBench uses 300 single-turn questions covering 58 verifiable constraints and evaluates them with the official code in loose mode.
  • Software Engineering: SWE-bench Verified uses 500 problems and the official rule-based evaluation script.The setup uses the R2E-Gym sandbox and the toolset described in Section 2.2.

D Computer Capability Classification

The paper classifies model actions into external resource access, file management, and code execution using pattern matching over bash commands and Python code. Capability usage is measured per interaction turn.

  • Three categories capture computer capability usage: external resource access, file management, and code execution.The categories are detected through patterns applied to model-generated bash commands and Python code.
  • Capability usage rate equals turns containing at least one matched pattern divided by total interaction turns.

E LLM-in-Sandbox-RL Training Details

LLM-in-Sandbox-RL trains models in the sandbox with rule-based task rewards and terminates excessively long unsuccessful trajectories. The training setup uses the same minimal sandbox and a baseline interaction prompt that can be adapted across use cases.

  • E LLM-in-Sandbox-RL Training Details: LLM-in-Sandbox-RL penalizes excessively long trajectories by terminating episodes with zero reward when models exceed turn or token limits without answering.
  • E LLM-in-Sandbox-RL Training Details: The sandbox configuration matches the paper’s described environment, while the training table specifies GRPO++ and on-policy batch-update details.The update mini batch size equals the train batch size, so one policy update is performed per batch.
  • E LLM-in-Sandbox-RL Training Details: Rule-based rewards assign correctness-based scores for multiple-choice and binary tasks, F1 for multiple correct options, and ROUGE-L for free-form generation.
  • E LLM-in-Sandbox-RL Training Details: The baseline prompt establishes the core interaction protocol and can be adapted by changing formats, adding domain instructions, or incorporating tools.The prompt examples include an instance-specific problem statement and instructions to solve tasks using code.
Loading 2601.16206v3…