Source-linked AI summary
Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity
Leon Engländer, Sophia Althammer, Ahmet Üstün, Matthias Gallé, Tom Sherborne
TL;DR
Current LLM-based agents often discover unexpected, relevant information without using it to revise their behavior. The paper injects complete solutions across three benchmarks and measures discovery separately from interaction, finding a persistent discovery–exploitation gap shaped by scaffold, compute, prompting, and training factors. Even optimized agents ignore discovered solutions in most trials, while the authors identify richer curiosity measurement and training or scaffold designs as future needs.
Problem
The paper asks whether LLM-based agents can recognize and investigate unexpected but relevant environmental observations rather than merely following learned patterns.
Method
The paper injects complete, discoverable task solutions into three agent benchmarks and separately measures whether agents discover and interact with those solutions.
Results
Across three benchmarks, agents consistently discover injected solutions but rarely interact with them; test-time factors modulate the gap, while narrow in-distribution fine-tuning suppresses curiosity further.
Takeaways & Limitations
Environmental curiosity is a necessary complement to outcome metrics because current agents use environments to fetch expected information but often fail to ground strategy revision in observations.
Takeaways & Limitations
The paper identifies the need for diverse benchmarks and metrics beyond solution injection, plus new training paradigms and scaffold designs for reflective behavior.
Abstract
from arXiv · showhide
LLM-based agents are assumed to integrate environmental observations into their reasoning: discovering highly relevant but unexpected information should naturally lead to a model exploiting its own discoveries. We show that this assumption is false for current LLM-based agents, which struggle to reflect or react to unexpected information. Across three benchmarks (Terminal-Bench, SWE-Bench, AppWorld), we inject complete task solutions into the agent environments to deliberately expose a task's solution to a model. While agents discover these solutions on Terminal-Bench in 79-81% of runs, they interact, or exploit, them in only 37-50% of cases. This gap is starkest in AppWorld: agents see documentation stating that a command "returns the complete solution to this task" in over 90% of attempts but exploit this in fewer than 7% of trials. We show that agents lack what we call environmental curiosity: the capability to recognize and investigate unexpected but relevant observations in response to environmental stimuli. We identify three main factors influencing environmental curiosity: available tools in the agent scaffold, test-time compute, and training data distribution. Our findings identify configurations that maximize curiosity also achieve the best performance on the unmodified benchmarks. Yet even jointly optimized agents still ignore discovered solutions in the majority of trials: current agents use the environment to fetch expected information, but not to revise their strategy or maximally exploit useful stimuli.
1 Introduction
Current LLM-based agents often discover unexpected, relevant solutions but fail to integrate them into subsequent behavior. The paper frames this discovery–interaction gap as a lack of environmental curiosity and shows that tools, reasoning budget, prompting, and training distribution influence it.
- Motivation: Agents must explore unfamiliar environments and integrate unexpected but relevant observations into their reasoning to solve tasks.The paper defines environmental curiosity as recognizing and investigating such observations in response to environmental stimuli.
- Approach: Solution injection exposes complete task solutions inside agent environments and measures whether agents discover and interact with them.This distinguishes adaptive responses to observations from fixed patterns learned during training.
- Findings: 97.54% of runs discover AppWorld documentation with gpt-oss-120b, but only 0.53% call the documented solution API.The trajectory shows the agent observing that the API would display the current task’s solution and proceeding without calling it.
- Findings: Across Terminal-Bench, SWE-Bench Verified, and AppWorld, agents frequently discover injected solutions but rarely interact with them.The benchmarks span terminal, software-engineering, and everyday digital tasks, and the gap appears across two agent scaffolds.
- Findings: Tool availability, reasoning budget, exploration-oriented prompting, and training distribution influence environmental curiosity, but jointly optimized agents still ignore discovered solutions in most trials.Restricting tools to bash-only roughly doubles interaction rates, while narrow in-distribution fine-tuning reduces curiosity and explored-path diversity.
- Implications: Prompts that improve interaction rates also achieve the best performance on the original, unmodified benchmarks.The paper reports that optimizing environmental curiosity consistently improves performance on those benchmarks.
2 Method
The method injects complete, discoverable task solutions into agent environments and separately measures solution discovery and interaction. These metrics distinguish finding relevant information from investigating and exploiting it.
- Solution injection: Solution injection places a complete task solution inside the environment in a form discoverable through normal agent actions.Following the injected solution guarantees task success, making it unexpected but highly relevant.
- Evaluation: The evaluation measures whether agents discover the injected solution and whether they interact with it after discovery.Interaction includes reading or executing a solution file or querying a solution API.
- Metrics: pass@k is the probability that at least one of k attempts successfully completes the task.It uses cpass for the number of attempts that pass the task.
- Metrics: discovery@k is the probability that at least one of k attempts executes a command surfacing the injected solution in the agent’s context.The metric checks whether the solution is discoverable through normal agent actions.
- Metrics: interaction@k is the probability that an agent interacts with the injected solution at least once across k attempts.A high interaction rate indicates investigation of unexpected relevant information, whereas a low rate indicates that it was ignored.
3 Experiments
The experiments measure whether agents discover and act on injected solutions across three benchmarks, then test how scaffolds, tools, reasoning effort, prompting, and training distribution affect this behavior. Agents often discover solutions but interact with them less frequently, while several configurations improve both curiosity-related measures and benchmark performance.
- Experimental setup: Agents are evaluated on Terminal-Bench, SWE-Bench Verified, and AppWorld using two agent scaffolds and separate discovery@k, interaction@k, and pass@k metrics.The setup includes Terminus and SWE-agent, with bash-only or bash plus str_replace_editor tool suites.
- 3.1 Agents lack environmental curiosity: On Terminal-Bench, discovery@1 reaches 78.6–81.2% but interaction@1 only 37.1–50.3%; AppWorld discovery@1 exceeds 90% while interaction@1 never exceeds 6.3%.SWE-Bench discovery@1 ranges from 53.4–98.2%, whereas interaction@1 is only 5.9–17.4%.
- 3.1 Agents lack environmental curiosity: Injecting solutions improves gpt-oss-120b performance most on Terminal-Bench, with a +11.4 gain, but only +2.6 on AppWorld where solution-API calls are rare.The reported original-benchmark pass@1 range is 40–46%.
- 3.2 Test-time factors: Increasing reasoning raises Terminal-Bench interaction@1 from 11% to 37% and SWE-Bench interaction@1 from 0.78% to 17.42%, but AppWorld interaction remains near zero.On Terminal-Bench, conditional interaction given discovery rises from 17.65% at low reasoning to 45.69% at high reasoning.
- 3.3 Effect of training distribution: Narrow in-distribution fine-tuning compresses explored solution paths, while broader training improves higher-k scaling and curiosity-related performance across benchmark settings.On AppWorld, AppWorld-SFT leads at pass@1 but T-Bench-SFT reaches 69.0 versus 65.8 at pass@10 and 41.5 versus 29.8 at interaction@10.
4 Discussion
The discussion argues that environmental curiosity requires agents to reflect on unexpected observations, yet current training and inference configurations leave this capability incomplete. Agents remain brittle in novel environments, motivating richer metrics, training methods, and scaffold designs.
- Agents use environments to fetch expected information rather than revise strategies in response to unexpected observations.
- Environmental curiosity requires agents to assess whether observations conflict with their current environmental model.
- Narrow in-distribution training may reinforce plans whose tool outputs align with expectations, reducing attention to information agents did not seek.
- Three attempted fine-tuning setups did not improve interaction rates, showing that training environmental curiosity is not straightforward.
- Outcome metrics such as pass@k cannot distinguish adaptive reasoning from rigid plan execution, so process metrics like interaction@k are needed.
- Future work should develop broader curiosity benchmarks, reflective training paradigms, and scaffolds that trigger reflection on observations.
5 Related work
Related work situates the paper among agent scaffolds, broad task benchmarks, and exploration research. Existing benchmarks emphasize end-to-end success, while prior exploration methods either use tools or constrain exploration through predefined pipelines.
- LLM-Based Agents: LLM agents interleave reasoning with actions, while terminal scaffolds differ in whether they provide bash alone, curated editing tools, or additional capabilities.
- Benchmarks: Agent benchmarks span software engineering, everyday digital tasks, scientific discovery, and assistant coordination.
- Benchmarks: Existing benchmarks measure task completion but do not reveal whether agents adapt to observations or execute fixed learned patterns.
- Agentic Exploration: Prior terminal exploration uses standard shell commands, supplementary search tools, or predefined localization pipelines rather than fully open-ended exploration.
6 Conclusion
The conclusion finds a persistent disconnect between what agents observe and how they act: they discover relevant solutions but usually ignore them. Test-time settings can improve curiosity and benchmark performance, but do not close the gap.
- Agents consistently discover unexpected relevant information across diverse domains yet systematically fail to interact with it.
- Tool availability, reasoning budget, and prompting modulate the discovery-interaction gap, while jointly optimized configurations still leave it open.
- The paper characterizes current agents as open-loop sequence generators that retrieve expected information without revising their plans.
- LLM-as-a-judge analysis finds that ignored solutions are generally absent from subsequent reasoning rather than deliberately rejected as traps.
A.1 LLM-as-a-Judge analysis: agents ignore rather than reject discovered solutions
The LLM-as-a-judge analysis tests whether agents ignore discovered solutions because they reject them or fail to register their relevance. The results overwhelmingly support passive non-interaction, not suspicion or inability to use the information.
- Method: The analysis focuses on each discovery event and the agent’s reasoning and actions over the following three turns.
- Method: The judge classifies post-discovery behavior into no acknowledgment, acknowledgment without investigation, deliberate rejection, preference for an own approach, and unexecuted planned interaction.
- Results: Categories 1–2 dominate non-interactions, indicating that agents either omit the solution from reasoning or acknowledge it without investigating.
- Results: 0 cases show deliberate rejection due to suspicion across models and benchmarks.
- Results: On SWE-Bench, no-acknowledgment rates exceed 96% for both evaluated models.
- Oracle Interventions: Oracle interventions test whether supplying the missing trigger at different trajectory stages enables agents to use the injected information.
- Oracle Interventions: Reflection at discovery raises interaction@1 from 37.12 to 53.33 and pass@1 from 55.88 to 60.00.
A.3 Additional factors: reasoning history and temperature
Reasoning-history retention and sampling temperature do not materially improve agents’ environmental curiosity, despite a small history-related interaction difference.
- Oracle interventions show that agents can use injected solutions when the solution is made explicit, but lack a spontaneous trigger to investigate them.
- 56.25% pass@1 with reasoning history versus 55.42% without shows no meaningful task-performance effect.
- 40.41% interaction@1 without history versus 35.83% with history indicates a slight interaction increase when reasoning history is discarded.
- Sampling temperature has negligible effect on interaction@k across temperatures from 0 to 1.0.Interaction rates remain stable across the full tested range, so low sampling diversity does not explain the curiosity deficit.
B.1 Expanded results
Expanded evaluations test model coverage, injection formats, difficulty levels, and prompts, showing that solution neglect persists while explicit exploration instructions improve performance and interaction.
- Expanded evaluation coverage: The evaluation spans multiple architectures, scales, reasoning budgets, benchmarks, and command-a-reasoning fine-tuned variants.
- Injection robustness: Stable interaction rates across injection file names show that the discovery-interaction gap is not an artifact of the injection format.The comparison includes explicit names such as SOLUTION_TO_THIS_SPECIFIC_TASK_READ_THIS_FIRST.md and familiar names such as README.md.
- Solution-injection difficulty: 37–38% interaction across difficulty levels 1–3 falls to ∼20–25% at levels 4–5, where decryption or checksum verification adds multi-step barriers.Because agents already ignore solutions at level 1, the main paper focuses on that trivial setup.
- Prompting: +2.57 average pass@1 on original benchmarks and +2.96 on solution-injected variants result from adding an instruction to explore the environment.
- Prompting: 44.50 original and 55.88 solution-injected Terminal-Bench pass@1 are achieved by the prompt instructing agents to investigate all discovered files.This prompt also maximizes interaction rates and achieves the highest pass@1 on the original benchmark.
- Prompting: The prompt comparisons use complete variants sharing an identical base prompt across conditions.
C SFT Training Data Details
The SFT experiments generate concise successful trajectories and train separate models on terminal, AppWorld, and software-engineering task distributions with auxiliary general-purpose tool-use data.
- Trajectory generation: Five trajectories per training instance are generated with gpt-oss-120b, retaining only the shortest successful trajectory.Each trajectory is a complete multi-turn sequence of reasoning, action, and observation turns.
- Task distributions: Three models are trained on distinct Terminal-Bench-like, AppWorld, and SWE-Bench-SFT task distributions.
- Training configuration: The training runs target approximately 30k effective task-specific turns and mix auxiliary general-purpose tool-use SFT data at a 1:1 ratio.
- Training configuration: Training only on the general-purpose SFT mixture yields 5.12 pass@1 on Terminal-Bench and 0.18 pass@1 on AppWorld.The paper uses this near-zero baseline to assess task-specific agentic capability and environmental curiosity.
D AppWorld is a subset of Terminal-Bench
AppWorld’s API-discovery interaction pattern appears in only a small subset of Terminal-Bench, while the benchmarks differ substantially in task complexity.
- AppWorld scope: AppWorld spans nine simulated everyday applications and 457 API endpoints, requiring multi-turn API discovery, calls, and reasoning.
- Overlap with Terminal-Bench: Four of Terminal-Bench v1’s 80 tasks share AppWorld’s core API-discovery and interaction loop.These include spreadsheet, web-scraping, S3-bucket, and MinIO-service tasks.
- Overlap with Terminal-Bench: 5% of T-Bench v1 tasks share AppWorld’s defining pattern, representing only a narrow slice of Terminal-Bench’s broader distribution.
- Task complexity: 26.2 turns on Terminal-Bench versus 29.6 on AppWorld average on successful gpt-oss (120b) high-reasoning trajectories.SWE-Bench averages 62.8 turns, reflecting substantially different per-task complexity.
E Agent Implementations
The evaluation compares Terminus 1 and SWE-agent, whose implementations differ in prompting, output truncation, command execution, and terminal-history formatting. The section also identifies the trainable turn as the unit of an assistant trajectory.
- Terminus 1 and SWE-agent differ in prompts, output truncation, command execution, and terminal-history formatting.SWE-agent uses blocking commands, while Terminus allows specified timeouts and interruption of long-running processes.
- 5A trainable turn is one assistant message containing reasoning and action within a trajectory.
- Figures 10–12 present the Terminal-Bench, AppWorld, and SWE-Bench system prompts provided to Terminus during evaluation.
- SWE-Bench uses a prompt closely following SWE-agent’s prompt, adapted to the scaffold’s terminal_use interface.