PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments

Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu, Bingxuan Li, Shujin Wu, Qing Zong, Tianshi Zheng, Chunyang Li, Qihan Lin, Chen Li, Dadi Guo, Xiusi Chen, Yangqiu Song

arXiv:2610.25950v1cs.CLcs.AI

TL;DR

Existing benchmarks rarely test whether agents can proactively discover environmental rules and adapt as those rules change. AdaExploreBench evaluates this through hidden rules and unannounced drift; results show that proactive and adaptive exploration remain challenging, with performance improving when evidence is supplied or drift is removed.

  • Problem

    Existing benchmarks rarely test agents’ ability to proactively discover environmental dynamics and adapt to unannounced changes.

    Source: §1 · p. 1

  • Method

    AdaExploreBench instantiates environmental dynamics as latent rules and evaluates agents as they explore for evidence and respond to rule drift.

    Source: §1 · p. 1

  • Results

    All models score below 70% calibrated accuracy in the default setting; providing extra evidence improves average performance by 9.4%, and removing rule changes improves it by 6.2%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Current agents remain limited in proactively acquiring evidence and keeping their understanding of evolving environments up to date.

    Source: §7 · pp. 8–9

    Exploration metrics primarily measure effort rather than quality; direct spatial-diversity and decision-region-coverage metrics are omitted.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.

1 Introduction

AdaExploreBench targets whether LLM agents can gather evidence to learn hidden environmental rules and adapt when those rules change without warning. Results show that proactive exploration and test-time adaptation remain difficult for current models.

1 Introduction

  • Existing benchmarks often provide or leak evidence and use stationary environments, motivating tests of self-directed evidence gathering and unannounced environmental change.The benchmark is intended to support understanding limitations and training improvements in exploration and adaptation.Source: §1 · pp. 1–2
  • AdaExploreBench models environmental dynamics as latent rules and requires agents to decide when, what, and how much to explore while recognizing rule drift.Its evaluation cases are presented in sequential small batches, allowing drift detection through further exploration or degraded performance.Source: §1 · pp. 1–2
  • All evaluated models score below 70% calibrated accuracy by default; extra evidence improves average performance by 9.4%, while removing rule changes improves it by 6.2%.The results also indicate that exploration quality matters more than quantity, and agents often report partially correct but incomplete rules and rule changes.Source: §1 · p. 2
  • The benchmark uses a scalable LLM-based generation pipeline to synthesize diverse evaluation instances without extra human labor.It provides an evolving-environment testbed and evaluates eight LLMs.Source: §1 · p. 2

2 Related Work

Prior work studies proactive environment learning and test-time adaptation, but typically addresses these capabilities separately from unannounced rule drift. AdaExploreBench combines proactive re-exploration with adaptation to such drift.

2 Related Work

  • Prior proactive-exploration work studies scientific discovery, interactive learning, hypothesis testing, long-horizon induction, and alien-world exploration, but typically uses static environments and may leak unqueried information.AdaExploreBench prevents information leakage and introduces unannounced rule drift that requires re-exploration.Source: §2 · p. 2
  • Related adaptation work addresses changing conditions through concept drift, changing constraints, unreliable tools, and changing tool costs, but emphasizes statistical adaptation or task-oriented replanning.AdaExploreBench instead requires recognizing unannounced rule drift, revising hypotheses, and proactively re-exploring.Source: §2 · pp. 2–3

3 Benchmark Construction

AdaExploreBench constructs episodes with hidden decision rules and evolving dynamics, then evaluates agents through exploration windows and sequential evaluation batches. Its protocol preserves probe evidence while masking diagnostic evaluation history and separately assesses rule understanding.

3.1 Data Generation

  • Each episode defines an agent’s decision setting with observable fields, four ordered actions, factors that determine intervention, and potentially changing dynamics.A latent rule maps each case to an intervention score, and a fixed utility function rewards actions whose intervention levels better match that score.Source: §3.1 · p. 3
  • Tasks use operational scenarios and action blueprints across finance, healthcare, cybersecurity, and manufacturing, with LLM-assisted field catalogs and sampled relevant factors and distractors.Cross-field and dynamics constraints specify feasible field combinations, factor interactions, and environmental changes.Source: §3.1 · p. 3
  • Four rule families represent additive, conjunctive, disjunctive, and lexicographic decision strategies, with dynamic settings introducing unannounced factor replacement and direction reversal.Changes remain within the same family and are constrained to be semantically plausible and to cause moderate shifts in optimal actions.Source: §3.1 · p. 3
  • Candidate episodes undergo structural, numerical, and semantic validation, followed by LLM-based review of realism, change plausibility, and distractor quality.Only qualified candidates are retained for the benchmark.Source: §3.1 · pp. 3–4

3.2 Runtime Protocol

  • Each episode alternates exploration windows with evaluation batches; agents may probe self-designed cases for utilities or proceed directly to the next batch.Task descriptions, field ranges, actions, rule-family definitions, and tools are visible, but the instantiated family, relevant factors, parameters, and change timing remain hidden.Source: §3.2 · p. 4
  • Completed evaluation cases and selected actions are removed from later context, while realized utilities remain; self-collected probe observations are retained.This separates proactive exploration from environment-provided evidence while preserving feedback that may signal environmental changes.Source: §3.2 · p. 4
  • Auxiliary assessments check whether agents identify relevant factors, their directional effects, and changes in dynamics, without feeding assessment results back into interaction.In dynamic settings, assessments occur immediately before rule drift and at episode completion; static settings are assessed only at completion.Source: §3.2 · p. 4

4 Experiments

The experiments compare eight language models across settings that vary rule drift, evidence visibility, and probing requirements, and assess both task performance and exploration and adaptation behavior. All models remain below 70% calibrated accuracy, while recognition ability aligns with final performance more clearly than exploration effort does.

4.1 Settings

  • The four settings vary whether rule drift occurs, evaluation batches are masked, and probing is mandatory; Default combines drift, masking, and agent-chosen probing.Stationary removes drift, Evidence Given unmasks batches, and Forced-Probe requires probing in each window.Source: §4.1 · p. 4

4.2 Metrics

  • The metrics assess decision quality, exploration volume, post-change verification, and rule recognition; CA gives partial credit for near-optimal choices, while recognition uses F1 scores.Mean Re-probes after Change counts distinct pre-change probe points queried again; Factor Recognition averages pre- and post-change F1, and Change Recognition scores inferred changes by set F1.Source: §4.2 · pp. 4–5

4.3 Results

  • Every model stays below 70% CA; Gemini-3.8-Flash leads, and the EA–CA gap grows from 2–5 points for the top three models to 8–22 for the others.Table 2 reports the main evaluation in the default setting.Source: §4.3 · p. 5, Table 2 · p. 5
  • The four strongest models by CA have the same Change Recognition ranking; remaining models reach at most 13.80% Change Recognition, while probe and re-probe volume tracks outcomes less consistently.DeepSeek-V4.1-Flash probes approximately 24 times as often as Llama-3.1-8B, yet has lower CA than Gemini-3.8-Flash and GPT-6-Sol.Source: §4.3 · pp. 5–6

5 Analysis

The ablations indicate that rule drift and self-directed evidence acquisition constrain performance, while simply increasing exploration effort does not resolve the gap. Models benefit from sufficient evidence, and probe informativeness appears to be a key remaining bottleneck.

Rule Drift Adaptation

  • 6.2% average calibrated-accuracy gain follows removal of rule drift, with gains up to 14.7% for DeepSeek-V4.1-Flash.The weakest models improve by less than 1%, while GPT-6-Sol barely changes because its default performance is near its stationary ceiling.Source: §5 · p. 6

Evidence Sufficiency

  • Six of seven models improve with sufficient evidence, and four gain more than 15 CA points.Qwen3.8-27B is the exception; it reduces exploration most and may be misled by stale evidence when it does not re-probe.Source: §5 · p. 6

Exploration Quantity vs. Quality

  • Mean calibrated accuracy changes by −2.0% under Forced-Probe despite mean unique probes rising from 33.9 to 53.8.Re-probing after rule change also rises substantially, so increasing exploration quantity alone does not improve performance.Source: §5 · p. 6

Synthesis

  • The ablations suggest probe informativeness is a key remaining bottleneck, though probe quality is not directly measured.More effort fails to improve performance, whereas providing sufficient evidence improves it substantially.Source: §5 · pp. 6–7
  • Agents learn from given evidence but do not acquire comparably useful evidence themselves or reliably detect and revise around rule drift.The authors identify exploration quality and test-time adaptation, rather than exploration quantity or basic rule learning, as binding constraints.Source: §5 · p. 7

6 Error Analysis

Rule drift causes a sharp performance drop that models do not fully recover from. Error analyses point to incomplete environmental models and difficulty revising learned directional effects as obstacles to adaptation.

Performance after rule drift

  • 26.9 points is the average first-post-change loss across six above-floor models, and all remain 6.9 to 19.2 points below their pre-change plateaus.Drops are significant and recovery is partial; weak models stay near the random baseline, while Llama-3.1-8B falls to 1.3% by the final batch.Source: §6 · p. 7

Re-probing and change recognition

  • Re-probing correlates positively with Change Recognition within models, but normalized re-probe counts do not explain cross-model differences.DeepSeek-V4.1-Flash re-probes more than Gemini-3.8-Flash yet has lower Change Recognition, suggesting evidence use also matters.Source: §6 · pp. 7–8

Factor and change recognition

  • Precision generally exceeds recall for Factor and Change Recognition, consistent with agents recovering mostly correct but incomplete environmental models.In 36.3% of eligible episodes, agents report identical pre- and post-change assessments despite environmental changes.Source: §6 · p. 8
  • Agents identify factor replacements more reliably than direction reversals, missing the reversal in 17.7% of episodes versus 4.1% for the converse.Reversals require inferring directional effects both before and after drift, and agents may retain outdated interpretations.Source: §6 · p. 8

Synthesis

  • The findings suggest persistent unchallenged assumptions can leave agents with incomplete models and blind spots as environments evolve.Insufficient exploration and hypothesis persistence may reinforce one another.Source: §6 · p. 8

7 Conclusion

AdaExploreBench evaluates proactive exploration and test-time adaptation by representing environmental dynamics as latent rules that can drift. The results identify evidence acquisition and adaptation as continuing challenges for LLM agents.

Conclusion

  • AdaExploreBench benchmarks LLM agents’ proactive exploration and test-time adaptation in evolving environments.Environmental dynamics are modeled as latent rules; agents decide when, what, and how much to explore, then recognize rule drift and revise hypotheses.Source: §7 · p. 8
  • Current LLM agents remain limited in proactive exploration and test-time adaptation.The findings highlight the broader challenge of autonomously acquiring evidence while keeping understanding of evolving environments up to date.Source: §7 · p. 9
  • The authors envision AdaExploreBench as a foundation for developing agents capable of reliable exploration and adaptation in dynamic environments.Source: §7 · p. 9

Limitations

The benchmark uses a simplified abstraction of environmental dynamics and covers a limited range of rule changes. Exploration quality is assessed indirectly rather than with direct quality metrics.

Simplified Environmental Dynamics

  • Modeling dynamics as latent rules and rule drift is a controlled but simplified abstraction that cannot fully capture real-world complexity and uncertainty.Agents nevertheless struggle with proactive exploration and adaptation even in these simplified settings.Source: Limitations · p. 9

Indirect Assessment of Exploration Quality

  • Exploration metrics primarily measure effort; direct spatial-diversity and decision-region-coverage metrics are omitted because they favor extensive probing and are incomparable across rule families.Ablations are used instead to examine indirectly whether ineffective evidence acquisition limits performance.Source: Limitations · p. 9

Limited Coverage of Environmental Changes

  • Evaluation covers within-family factor replacement and direction reversal, leaving cross-family changes, other change types, and systematically varied change magnitudes for future work.Source: Limitations · p. 9

Ethics Statement

AdaExploreBench combines hidden rule learning with proactive interaction and implicit rule drift, while its construction uses generated, validated episodes and explicit operational definitions.

Ethics Statement

  • The benchmark uses synthetically generated scenarios without sensitive user data or offensive content, and plans MIT and Creative Commons licensing.Open-source models ran locally with vLLM; closed-source models were accessed through official APIs, and PhD-level researchers performed annotations.Source: Ethics Statement · p. 9

A Terms Explanation in Positioning Table

  • Proactive exploration lets agents choose when, what, and how much evidence to query; adaptive exploration requires detecting unannounced change and exploring again.The benchmark also targets environment learning, in-process evaluation, heterogeneous rules, and scalable generation.Source: A Terms Explanation in Positioning Table · p. 11

B More on Related Work

  • Prior rule-learning benchmarks often provide predetermined evidence, whereas AdaExploreBench lets agents construct probes before decisions.The related-work discussion contrasts passive rule learning with interactive evidence gathering.Source: B More on Related Work · p. 11

C Environment Formalization

  • The four rule families model weighted compensation, all-condition requirements, any-condition sufficiency, and fixed-priority decisions.Additive, conjunctive, disjunctive, and lexicographic rules represent structurally distinct attribute-combination strategies.Source: C Environment Formalization · p. 12
  • Pre-change evidence quartiles calibrate raw evidence to action boundaries, while action utilities favor anchors nearest the intervention score.The calibration is fitted once and held fixed after environmental changes; utilities sum to zero for every score.Source: C Environment Formalization · p. 13

D.1 Semantic Field Catalog Construction

  • A predefined schema and task blueprints guide LLM-assisted field catalog synthesis, followed by programmatic validation and semantic auditing.The catalog includes field semantics and parameters, priority orders, feasibility constraints, and admissible cross-field dynamics.Source: §D.1 · p. 13

D.2 Episode Generation

  • Episode generation enumerates valid field-rule configurations and numerically certifies candidates, including dynamic cases under shared pre- and post-change calibration.Semantic review rejects invalid field configurations after numerical certification.Source: §D.2 · p. 13, §D.2.1 · p. 14

D.2.2 Evaluation Case Sampling

  • Dynamic evaluation cases share a pre-transition prefix across branches, after which IID and targeted branches use different post-transition distributions.Selected episodes must pass review for field realism, plausible rule transitions, and distractor quality.Source: §D.2.2 · p. 14, §D.3 · p. 14

D.4 Dataset Configuration

The dataset configuration specifies dynamic episodes, runtime and model settings, and evaluation measures; a human review and model-level correlation analysis provide additional characterization.

D.4 Dataset Configuration

  • The evaluation set contains 256 dynamic episodes balanced across domain–rule-family combinations, with 16 episodes per combination.Each episode has 12 batches of 8 cases, with rule drift after batch 5, 6, or 7; the training set contains 1,000 episodes.Source: §D.4 · pp. 14–15

D.5 LLM Prompt Specifications

  • The prompt specifications cover semantic field construction and episode and dataset review.They include prompts for field discovery, semantics, parameter calibration, priority orders, constraints, replacements, and review.Source: §D.5 · p. 15

E Human Study

  • Human annotators rated task plausibility favorably, with mean scores above 4.0 across the three assessed dimensions.The study sampled 40 episodes from the 256 dynamic episodes and used three annotators; positive ratings were scores of 4 or 5.Source: E Human Study · pp. 15–16, Table 5 · p. 16

F Detailed Runtime Settings

  • Runtime limits and failure handling are fixed across models and episodes, with repair opportunities for protocol errors and retries for provider failures.Probing is optional, limited to one probe per window, and capped in size; the text describes fallback handling when limits are exhausted.Source: F Detailed Runtime Settings · pp. 16–17

G Model Configs

  • Reported runs use a frozen configuration snapshot; open-source small models run on four Nvidia L20 GPUs, while larger and closed-source models use official APIs.All models answer in JSON, and temperature is 0.0 wherever the endpoint accepts it.Source: G Model Configs · p. 17, Table 7 · p. 17

H Formal Definitions of Evaluation Metrics

  • Calibrated Accuracy normalizes selected-action utility between the minimum and maximum available utilities, with random and oracle policies set to expected scores of 0 and 1.This metric retains information about relative suboptimality that binary accuracy does not.Source: H Formal Definitions of Evaluation… · p. 17
  • Change Recognition scores predicted transition atoms against a three-atom ground truth using strict set F1, while Factor Recognition evaluates relevant-field predictions.Change Recognition is averaged over episodes with completed paired assessments, and its completion rate is reported alongside it.Source: H Formal Definitions of Evaluation… · p. 18

I.1 Correlation Analysis

  • Table 8 correlates exploration volume and recognition measures with end-to-end outcomes across eight model means.Its average columns summarize correlations for exploration ability and calibrated accuracy.Source: §I.1 · p. 18, Table 8 · p. 18

I.2 Full Results for Analysis

The appendix reports full per-model results for diagnostic settings and paired significance tests, but the supplied passages do not give the underlying table values.

I.2 Full Results for Analysis

  • Tables 11, 14, and 15 provide per-model results for Stationary, Evidence Given, and Forced-Probe settings alongside default results.Stationary omits re-probe and adaptation metrics; Evidence Given and Forced-Probe retain the six main-table metrics.Source: §I.2 · pp. 18–19
  • On the default setting, Table 10 reports a higher replacement rate than reversal rate for change recognition by atom type, significant at p < 10−70 under a paired Wilcoxon signed-rank test.Source: Table 10 · p. 19

I.3 Significance Analysis

  • The Stationary and Evidence Given gains in per-episode Calibrated Accuracy are significant at the model level; Forced-Probe effects are mixed and not significant on average.Evidence Given gains are positive and significant for nearly every model.Source: §I.3 · p. 19

J Ablation Studies

The ablation finds little effect from IID case sampling in the stationary setting; accompanying decomposition passages define how change-recognition outcomes are categorized.

J Ablation Studies

  • Stationary-IID differs from Stationary by at most 2.1 Calibrated Accuracy points per model and 1.0 point on average, with no significant sampling effect.The authors attribute default-setting difficulty to unannounced rule drift and self-collected evidence rather than evaluation-case distribution.Source: J Ablation Studies · p. 19

K Change Recognition Decomposition Results

  • Change Recognition is decomposed by removal, addition, and reversal atoms, and by episode-level combinations of recovered replacement and reversal changes.Every transition has two replacement atoms and one reversal atom; episode-level categories require completed paired factor assessments.Source: K Change Recognition Decomposition… · p. 19

L.1 Data Case Study

  • The production-scheduling case study presents a task, input case, and reference output alongside the hidden rules governing decisions and changing field effects.The hidden rules permit inspection of which fields determine the decision and how their effects change.Source: §L.1 · p. 20

L.2 Error Case Study

This section examines an illustrated Default episode in which GPT-6 Sol identifies a replacement factor after rule drift but retains the old throughput-trend direction despite controlled probes. It situates the error in a setting where agents infer hidden, potentially changing rules from probes and action feedback, with declining batch utility indicating a mistaken hypothesis or rule drift.

L.2 Error Case Study

  • GPT-6 Sol chooses action_3 for case_059, then reports the correct replacement factor but the old throughput direction after the rule changes.The paired probes differ only in throughput trend, while the agent continues to use the old direction.Source: §L.2 · p. 20, Figure 7 · p. 24

Error Case Study

  • Agents must infer both the relevant fields and one of four hidden rule families from probe results and action feedback.Irrelevant fields are distractors; the families are additive aggregation, conjunctive gating, disjunctive sufficiency, and lexicographic priority.Source: §L.2 · p. 37
  • Probes reveal the utilities of all four actions on example cases but earn no utility themselves; submitted-action feedback reveals only the chosen actions’ per-case utility.Probe cases should avoid redundancy, such as repeating a case or using cases with nearly identical field values.Source: §L.2 · pp. 37–38
  • A batch scoring markedly worse than earlier batches signals that the inferred rule may be wrong or may have changed, prompting consideration of further probing.The environment can change the rule without notice after an action batch; low or negative utilities flag mismatches with the agent’s hypothesis.Source: §L.2 · pp. 37, 39
Loading 2610.25950v1…