Source-linked AI summary
TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agents
Hang Yan, Xinyu Che, Fangzhi Xu, Qiushi Sun, Zichen Ding, Kanzhi Cheng, Jian Zhang, Tao Qin, Jun Liu, Qika Lin
TL;DR
LLM agents can improve through iterative interaction, but existing metrics provide limited insight into how efficiently they improve, adapt after errors, or use working memory. TIDE introduces three trajectory-based diagnostics and finds that improving agents requires explicitly optimizing interaction dynamics, not only scaling internal reasoning.
Problem
Existing evaluations do not adequately capture TTI's optimization efficiency, behavior adaptation after errors, or working-memory utility.
Method
TIDE is an agent-agnostic and environment-agnostic framework that diagnoses TTI through the complementary metrics AUV, LR, and MI.
Results
Experiments across diverse models and environments show that agent performance depends on how efficiently, adaptively, and effectively agents improve through interaction, beyond final success.
Takeaways & Limitations
The findings support shifting from static proficiency evaluation toward diagnosing and explicitly optimizing agent–environment interaction dynamics.
Takeaways & Limitations
In one reported case, SR and AUV reveal similar evaluation results, although the authors state that AUV captures more detail than SR.
Abstract
from arXiv · showhide
Recent advances in autonomous LLM agents demonstrate their ability to improve performance through iterative interaction with the environment. We define this paradigm as Test-Time Improvement (TTI). However, the mechanisms under how and why TTI succeed or fail remain poorly understood, and existing evaluation metrics fail to capture their task optimization efficiency, behavior adaptation after erroneous actions, and the specific utility of working memory for task completion. To address these gaps, we propose Test-time Improvement Diagnostic Evaluation (TIDE), an agent-agnostic and environment-agnostic framework that decomposes TTI into three comprehensive and interconnected dimensions. The framework measures (1) the overall temporal dynamics of task completion and (2) identifies whether performance is primarily constrained by recursive looping behaviors or (3) by burdensome accumulated memory. Through extensive experiments across diverse agents and environments, TIDE highlights that improving agent performance requires more than scaling internal reasoning, calling for explicitly optimizing the interaction dynamics between the agent and the environment.
1 Introduction
Test-Time Improvement (TTI) is an interaction-driven process in which agents iteratively improve through environmental feedback, but existing evaluations miss its temporal efficiency, behavioral adaptation, and memory utility. TIDE addresses these gaps with three complementary diagnostics and experiments that motivate optimizing agent–environment interaction beyond internal reasoning alone.
- TTI describes agents accumulating experience through continuous interaction to iteratively rectify actions.
- Existing static metrics equate immediate success with delayed success and therefore miss optimization efficiency over interaction.
- Interaction-turn counts conflate corrective adaptation with repetitive failure actions, obscuring recursive failure.
- Longer working memory may contain useful experience and misleading noise, while existing analyses confound memory effects with model scale or interaction length.
- TIDE decomposes TTI into optimization efficiency, behavior adaptation, and memory utility using AUV, LR, and MI.AUV captures temporal efficiency, LR identifies loop-induced stagnation, and MI isolates working-memory contribution.
- Experiments reveal failure modes and bottlenecks invisible to existing metrics, supporting explicit optimization of agent–environment interaction beyond scaling internal reasoning.
2 Preliminaries
The paper models multi-turn agent–environment interaction as a POMDP and distinguishes reasoning-bound MDP tasks from information-bound POMDP tasks according to their dependence on external feedback.
- Multi-turn Interaction Agent: Multi-turn interaction is formalized as a POMDP with latent states, actions, observations, transitions, rewards, and a goal.The resulting rollout alternates observations and actions, with reward indicating whether the trajectory reaches the goal.
- Multi-turn Interaction Agent: MDP is treated as a special case of POMDP in which the unobservable state space is empty.
- Table 1 reports overall success rate and area under variation across five widely adopted benchmarks.Higher values are better, and bold values indicate the best results.
- Multi-turn Interaction Agent: Reasoning-bound tasks can be solved independently of immediate environmental responses, whereas information-bound tasks require interaction to acquire information.
3 Breakdown Test-Time Improvement
TIDE breaks Test-Time Improvement into temporal optimization efficiency, behavior adaptation, and memory utility, revealing distinct constraints on agent performance. Across environments, AUV captures progress dynamics, LR diagnoses recursive loops, and MI exposes whether accumulated memory helps or burdens task completion.
- Optimization Efficiency: AUV measures optimization efficiency by integrating the performance variation curve over the evaluation window.It captures how quickly and steadily agents convert interaction turns into task success, rather than relying only on terminal outcomes.
- Optimization Efficiency: AUV 0.499 rewards GLM-4-32B-0414 for later-stage gains despite comparable or lower early-stage SR than Qwen3-4B-Instruct.The result illustrates that sustained convergence can matter more than early success.
- Behavior Adaptation: High Loop Ratio is associated with low AUV, making loop reduction necessary but insufficient for high TTI capacity.Most evaluated agents exhibit recursive failure; Qwen3-4B-Instruct reaches 32.0% LR in FrozenLake.
- Behavior Adaptation: Scaling reduces stagnation within the Qwen3 family, lowering average LR from 15.8% to 1.0% in BlocksWorld and from 36.7% to 5.7% in WebShop.The reported pattern is associated with greater strategic diversity and behavior adaptation as model size increases.
- Memory Utility: MI quantifies the performance gap between complete working memory and access restricted to the task description plus immediate observation.The measure isolates working-memory contribution rather than judging memory-management competence.
- Memory Utility: Working-memory benefits show diminishing returns: gains are concentrated in the first 5 window sizes before the curve rapidly plateaus.Window size denotes the number of most recent interaction turns retained in memory.
- Memory Utility: Memory helps in partially observable WebShop but can negatively influence reasoning-bound FrozenLake, where feedback makes historical context redundant.The results challenge the assumption that expanding context universally improves performance and motivate active memory management.
4 Application of TIDE Framework
TIDE’s cross-environment analysis diagnoses how agents improve through interaction, revealing distinct roles for reasoning, working memory, and loop avoidance. Its trajectory-based evaluation also generalizes to GUI logs without re-executing experiments.
- TIDE decomposes test-time improvement into optimization efficiency, behavior adaptation, and memory utility across diverse agents and environments.The framework combines AUV, LR, and MI to analyze interaction dynamics.
- Gemini and DeepSeek consistently achieve superior AUV and LR across environments, while showing relatively low MI in reasoning-bound tasks.The authors attribute this pattern to robust reasoning reducing reliance on interaction memory.
- GLM-32B-0414 exhibits high memory utility, a small AUV–SR gap, and low LR, indicating effective integration of interaction history into adaptive behavior.The paper reports that agents with minimal prior knowledge can attain SOTA performance when they ground actions in environmental feedback.
- Reasoning-enhanced models such as gpt-oss-120b often fail to convert internal chain-of-thought into effective external actions in information-bound tasks.The authors therefore distinguish internal cognitive capacity from external interactive efficacy.
- TIDE can analyze recorded GUI interaction trajectories post hoc, including loop-separated OSWorld trajectories, without re-executing the original experiments.The framework is designed to be agent-agnostic and environment-agnostic.
- In OSWorld, UI-TARS-1.5-72B-DPO’s AUV falls from 26.3 without loops to 5.3 with loops, whereas Claude3.7-Sonnet maintains AUV 6.9 with loops.The analysis associates approximately 50% Click actions among loop actions with performance degradation and identifies grounding as a GUI bottleneck.
5 Related Works
Related work frames multi-turn agents as needing environmental feedback and self-refinement, while existing evaluation increasingly seeks finer-grained process analysis beyond final success rates.
- Purely internal reasoning lacks environmental grounding, leaving agents vulnerable to hallucination and suboptimal performance in partially observable or non-static environments.Active self-refinement adapts actions from environmental feedback.
- Outcome-oriented agent evaluations primarily use final Success Rate, which obscures interaction costs and efficiency by treating the task process as a black box.Subsequent work has shifted toward finer-grained analysis.
6 Conclusion
The paper formalizes test-time improvement as dynamic and evaluates it through complementary diagnostics of efficiency, looping, and memory. Across models and environments, performance depends on how effectively agents improve through interaction, not only on final success.
- TIDE characterizes test-time improvement through three complementary metrics: AUV, LR, and MI.The framework is agent-agnostic and environment-agnostic.
- Experiments show that agent performance hinges on how efficiently, adaptively, and effectively agents improve through interaction, beyond final success.The authors advocate diagnosing and optimizing agent dynamics rather than evaluating only static proficiency.
A Theoretical Analysis of AUV Properties
Theoretical analysis establishes AUV as a path-dependent measure of convergence dynamics and marginal interaction gains, distinct from final success rate. It also proves that the empirical AUV estimator is unbiased, statistically consistent, and has variance decaying as O(1/N).
- AUV captures convergence rate and marginal interaction gain rather than merely duplicating final success rate.
- AUV is a time-weighted linear combination of marginal gains δk, with coefficients determined by their interaction times.
- AUV distinguishes efficient and inefficient trajectories that share the same final success rate because final success rate is path-agnostic while AUV is path-dependent.
- Moving an identical gain from a late step to an earlier step leaves SR unchanged but increases AUV.
- The empirical AUV estimator converges in probability to the population value, is unbiased, and has variance decaying at O(1/N).
B.1 Metrics Necessity Analysis
The metric triad is presented as necessary for diagnosing distinct dimensions of test-time improvement: temporal efficiency, behavior adaptation, and memory utility. AUV and LR use interaction logs, while MRI requires a control experiment and AUV extends beyond binary success rate.
- AUV captures dynamic efficiency that static assessments obscure, while LR and MI diagnose behavior adaptation and memory utility, respectively.
- AUV and LR derive directly from standard interaction logs, whereas MRI requires a straightforward control experiment.
- AUV applies to progress rate and other metrics in addition to binary success rate.
- A higher success rate does not necessarily produce a higher AUV, demonstrating that AUV captures information beyond SR.
B.4 Loop Phenomenon Analysis
Looping is widespread among evaluated LLM agents, reflecting repeated execution of erroneous actions instead of adaptation to negative environmental feedback. Entropy analysis associates loop behavior with overconfidence, including in high-dimensional GUI environments.
- Low action entropy during stagnation indicates that agents are confidently fixated on erroneous paths.
- Loop actions are significantly associated with overconfidence, even in GUI environments with high-dimensional, open-ended action spaces.
B.5 More Results of MI
The MI analysis examines how working memory affects agent performance and finds negative influence in reasoning-bound tasks, long-range recall failures, and diminishing benefits from larger memory windows. Summarization does not remove the cognitive burden of accumulated working memory.
- Negative MI is widely observed in reasoning-bound tasks.
- Successful AlfWorld rollouts cluster at low memory recall lags, whereas failures show a long tail of high lags.
- High recall lag indicates that agents retrieve critical information but fail to reason across long-range information.
- Summarizing interaction trajectories and reasoning does not eliminate the cognitive burden imposed by accumulated working memory.
B.9 More Evaluation Results of Existing GUI Agent Trajectories
The GUI trajectory analysis compares standard success with temporal efficiency and shows that looping behavior can sharply reduce performance, with Claude3.7-Sonnet as a notable exception.
- Complementarity of SR and AUV: Table 4 identifies model pairs where Success Rate is at least as high but AUV is lower, demonstrating complementary evaluation perspectives.The listed cases include equal Success Rate with divergent AUV, equal AUV with divergent Success Rate, and opposing Success Rate–AUV rankings.
C.1 Experiment Environment
The evaluation spans reasoning-bound and information-bound environments plus GUI-agent trajectories, using TIDE metrics to characterize optimization efficiency, looping, and memory recall. Experimental procedures set environment-specific horizons and state-matching rules, while radar profiles normalize metrics for comparison.
- Environment coverage: Five environments are divided into reasoning-bound MDPs—BlocksWorld, FrozenLake, and Sudoku—and information-bound POMDPs—AlfWorld and WebShop.The framework is additionally applied to AndroidWorld, OSWorld, and WindowsAgentArena trajectories.
- Environment coverage: The benchmark suite includes 100 instances each for BlocksWorld, FrozenLake, and Sudoku, 140 AlfWorld entries, and 500 WebShop entries.BlocksWorld, FrozenLake, and Sudoku data were generated for these experiments; AlfWorld uses in-distribution evaluation and WebShop uses its full dataset.
- Evaluation protocol: For GUI environments, CLIP image embeddings with a 0.999 threshold identify repeated states, while exact action matching identifies repeated actions.Exact text matching represents states and actions in BlocksWorld, FrozenLake, Sudoku, AlfWorld, and WebShop.
- Evaluation protocol: AUV uses empirical performance-saturation points to set tmax, with horizons 20, 30, 20, 60, and 15 for BlocksWorld, FrozenLake, Sudoku, AlfWorld, and WebShop.GUI tasks use tmax = 50; for multi-step tasks, AUV is bounded below 1 and can be compared with an empirical upper bound from extensive SOTA rollouts.
- Memory evaluation: Memory Recall Distance measures the temporal gap between observing a task-relevant object and later interacting with it.The procedure extracts task-relevant objects, tracks their latest observation timestep, computes t − LastSeen(obj, t), and aggregates valid distances across the trajectory.
- Cross-model comparison: Radar profiles min-max normalize metrics to [0,1] and invert LR as 1 − LR so larger values consistently indicate better TTI.The reported radar results use distinct profiles for each task environment and apply a minimum baseline without changing model diagnosis.