Source-linked AI summary
AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?
Dong Yan, Jian Liang, Dapeng Hu, Ran He, Nicholas Jing Yuan, Qi Zhang, Tieniu Tan
TL;DR
Existing studies mainly evaluate self-evolving agents on isolated tasks, leaving their behavior in realistic, diverse task streams unclear. AgentStream evaluates five methods across three models and streaming scenarios, finding that reliability varies by scenario, capability, and method, with no universal winner.
Problem
Existing studies predominantly evaluate self-evolving agents independently, leaving their reliability across diverse, boundary-free task streams and evolution components unclear.
Method
AgentStream organizes multiple benchmarks into configurable Isolated, Sequential, and Interleaved streams and combinatorially evaluates self-evolving methods across models and scenarios.
Results
Self-evolution reliability varies by streaming scenario, its benefit is capability-gated and non-monotonic in model strength, and no method dominates across models and scenarios.
Takeaways & Limitations
Method selection should depend on model and stream structure, and self-evolving agents should be evaluated under realistic task streams rather than isolated tasks.
Takeaways & Limitations
The capability-strength conclusions are specific to the experimental setup, and the three scenarios and six benchmarks do not exhaust possible streams and domains.
Abstract
from arXiv · showhide
Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominantly adopt independent evaluation. Consequently, the behavior of self-evolving agents in realistic streaming settings, where agents adapt to diverse and complex task streams, remains poorly understood. To address this gap, we introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components by organizing agentic benchmarks into a configurable task stream and instantiating the \texttt{Isolated}, \texttt{Sequential}, and \texttt{Interleaved} streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.
1 Introduction
AgentStream addresses the limits of independent evaluation by jointly studying model capability, self-evolving method, and streaming structure in configurable task streams. Its findings show that self-evolution reliability depends on the streaming scenario and model capability rather than improving uniformly.
- Motivation: Self-evolving agents update prompts, structured memory, reusable skills, or integrated harnesses from accumulated deployment experience.These updates define the main evolution components studied in recent methods.
- Problem: Independent evaluation solves tasks in isolation without cross-task state, leaving improvements under realistic, diverse, boundary-unclear streams uncertain.Streaming evaluation must account for foundation model, self-evolving method, and task-stream structure jointly.
- Approach: AgentStream organizes multiple benchmarks into configurable within-domain and cross-domain streams and evaluates context, memory, skill, and integrated-harness evolution.The framework combinatorially evaluates models, self-evolving methods, and streaming scenarios.
- Findings: Self-evolution is not uniformly beneficial: Isolated is most reliable, while Interleaved generally outperforms Sequential despite its more heavily mixed stream.Reliability therefore varies across streaming scenarios.
- Findings: Self-evolution gains are gated by model capability and non-monotonic in model strength, with the weakest model showing negative evolution gains.The study systematically analyzes how streaming scenario, model capability, and method architecture shape performance changes.
2 Related Work
Prior work studies adaptation to streaming data through test-time and continual learning, while LLM-agent self-evolution varies by the system component being improved. Agentic benchmarks evaluate agents across interactive, tool-use, software-engineering, and skill-oriented tasks, motivating evaluation across diverse task streams.
- Streaming and continual learning: Streaming-data learning primarily comprises test-time learning for incoming instances or distribution shifts and continual learning for sequential task streams with resistance to catastrophic forgetting.For LLMs and agents, test-time learning operates at both parameter-adaptation and other levels.
- Streaming and continual learning: Continual learning for LLM agents includes gradient-free inference-time updates, parameter-level forgetting mitigation, and experience-centric reuse of accumulated knowledge across tasks.
- Self-evolving agents: Self-evolving agents differ mainly in which agent-system component evolves, with memory-based methods accumulating, retrieving, reorganizing, and pruning historical experience across tasks.
- Agentic benchmarks: Agentic benchmarks cover diverse environments, including interactive web and application tasks, structured tool use, repository-level software engineering, and modular skill acquisition.These benchmarks assess long-horizon interaction, API selection and invocation, executable code resolution, and skill reuse across heterogeneous tasks.
3 The AgentStream Framework
AgentStream evaluates self-evolving agents on realistic task streams by accumulating experience after each interaction and varying how task domains and evolution states are composed. Its three scenarios isolate within-domain learning, sequential transfer, and cross-domain interference under interleaved task streams.
- Framework and problem setup: AgentStream models test-time learning as sequential task processing, where an agent uses an evolving state and distills each interaction trajectory into persistent experience.Experience may include refined context, memory entries, reusable skills, or a revised harness.
- Framework and problem setup: The agent updates its evolution state using self-generated interaction feedback without access to ground-truth labels at test time.Feedback can include execution outcomes and reflective self-evaluation.
- Framework and problem setup: Self-evolution is compared with the same model solving every task with an empty evolution state; positive ∆ indicates improvement, whereas negative ∆ indicates interference.The baseline is Perf(M, Q, ∅), with St = ∅ for all t.
- Streaming scenarios: Isolated assigns each benchmark an independent agent and state, measuring useful experience accumulation within a single task domain without cross-benchmark transfer.Each agent processes only its benchmark’s task subset Q(k).
- Streaming scenarios: Sequential retains one evolution state across benchmarks in a fixed order, testing whether earlier-domain experience enables or interferes with performance on later domains.The resulting stream is Q = Q(1) ⊕ Q(2) ⊕· · · ⊕ Q(K).
- Streaming scenarios: Interleaved shuffles tasks from all benchmarks into one stream with a shared state, testing domain-relevant retrieval while suppressing cross-domain interference under maximal task diversity.The unified stream is Q = shuffle(Q(1) ∪· · · ∪Q(K)).
4 Experimental Settings
The experiments compare five self-evolving methods across three frontier foundation models and six diverse agentic benchmarks. All methods are adapted to a unified test-time infrastructure built on Exgentic, with standardized embeddings and benchmark-specific task sampling.
- Models: The evaluation spans GPT-5.4-medium, Gemini 3.1 Pro-medium, and Claude Opus 4.7-high to analyze how model capacity influences test-time evolution.The models represent different families and scales.
- Tasks: Six diverse benchmarks cover agentic capabilities including interactive coding, multi-step function calling, and deep research.AppWorld tests multi-app workflows, BFCL tests context-dependent tool use, and BrowseComp-Plus tests iterative web retrieval.
- Self-Evolving Methods: Five representative methods span context, memory, skill, and integrated harness evolution, including ACE, A-Mem, ReasoningBank, and Harness.The methods collectively represent the principal evolution components.
- Implementation Details: The evaluation infrastructure uses Exgentic to standardize communication between heterogeneous agent interfaces and benchmarks, adapting all methods to test-time operation.Text embeddings use all-MiniLM-L6-v2 across experiments.
- Implementation Details: N = 50 tasks are sampled from each benchmark, with benchmark-specific splits including AppWorld test-challenge and BFCL multi-turn base.The supplied implementation description also identifies benchmark-specific sampling choices for Tau2.
5 Results
Self-evolution reliability varies substantially by streaming scenario, model capability, and method architecture. Isolated is most reliable overall, while benefits are gated and non-monotonic across models, with method choice remaining scenario- and model-dependent.
- Streaming scenarios: Isolated is most reliable, achieving a 75.7% positive rate, +1.37% average evolution gain, and 38% Top-1 rate.It benefits from distributional coherence within a single benchmark, where solved tasks provide directly relevant knowledge without cross-domain interference.
- Streaming scenarios: Interleaved generally outperforms Sequential despite comparable 62.3% positive rates, with +0.90% versus +0.75% average evolution gain.Across 15 configurations, Interleaved leads in 10 cases, compared with 5 for Sequential, including 4 of 5 Gemini 3.1 Pro configurations and 3 of 5 for both Claude Opus 4.7 and GPT-5.4.
- Model capability: GPT-5.4 has negative evolution gains from −0.35% to −0.78% and exceeds its vanilla baseline in only 4 of 15 configurations, whereas Gemini 3.1 Pro and Claude Opus 4.7 do so in 14 and 13.The benefiting models show gains of +1.98% to +2.71% for Gemini 3.1 Pro and +0.90% to +1.75% for Claude Opus 4.7 across all streaming scenarios.
- Model capability: Evolution gain is non-monotonic in model strength: Gemini 3.1 Pro averages +2.37%, exceeding Claude Opus 4.7 at +1.23% despite Claude’s stronger vanilla baseline.This pattern holds across Isolated, Sequential, and Interleaved scenarios, with the difference varying as streaming complexity changes.
- Method effects: Method sensitivity decreases with capability, while the best method varies by model; the best–worst spread is 5.3% on GPT-5.4, 2.0% on Gemini 3.1 Pro, and 0.9% on Claude Opus 4.7.Only 1 of 5 methods beats baseline on GPT-5.4, versus all 5 on the other two models; averaged across methods, the GPT-5.4–Claude Opus 4.7 gap widens from 18.1% vanilla to 20.0% after evolution.
- Method effects: Context-integrated methods perform best under Isolated, with ACE at +2.28%, whereas retrieval-based methods peak under Interleaved, with A-Mem at +2.22%.Under Interleaved, ACE drops from +2.28% to −1.26%, illustrating the interaction between method architecture and streaming scenario.
6 Conclusion
AgentStream is a unified framework for evaluating self-evolving agents on configurable task streams across three streaming scenarios, multiple models, and methods. Its findings show scenario-dependent reliability, capability-gated and non-monotonic benefits, and no universally dominant method, motivating realistic streaming evaluation.
- Framework: AgentStream organizes agentic benchmarks into configurable task streams for evaluating self-evolving agents across three streaming scenarios, multiple models, and methods.The framework supports systematic analysis across varied streaming settings.
- Findings: Self-evolution reliability varies across streaming scenarios, so performance depends on the composition and organization of the task stream.The conclusion emphasizes streaming scenarios as a determinant of reliability.
- Findings: Self-evolution benefits are gated by model capability and non-monotonic in model strength, while no single method dominates across models and streaming scenarios.These findings provide practical guidance for deploying self-evolving agents and support evaluation under realistic streaming settings.
7 Limitations
The paper’s model-capability rankings are specific to its Exgentic-based experimental setup rather than universally valid. Different frameworks, prompting strategies, or benchmark selections may change relative model ordering and capability-gating conclusions.
- 7 Limitations: Model capability strength is grounded in empirical performance under the specific Exgentic evaluation setup, not a universal ranking.A given model performs unevenly across benchmarks.
- 7 Limitations: Different agent frameworks, prompting strategies, or benchmark selections may alter the relative ordering of models.
- 7 Limitations: Conclusions about capability gating may therefore depend on the experimental setup.
A Cost Analysis
Self-evolution’s cost-performance tradeoff depends strongly on the foundation model: it can reduce inference cost on Gemini 3.1 Pro but substantially increase cost on GPT-5.4. ReasoningBank is the most cost-efficient method on Gemini, whereas only A-Mem improves performance on GPT-5.4, at much higher cost.
- Cost measurement: Tables 7–8 report per-task cost, relative cost overhead, and average agent steps for GPT-5.4 and Gemini 3.1 Pro using LiteLLM pricing data.Figure 4 aggregates these results by method across three streaming scenarios.
- Model-dependent cost: On Gemini 3.1 Pro, ReasoningBank, Harness, AutoSkill, and A-Mem reduce cost to 64%, 71%, 82%, and 84% of the vanilla baseline, respectively.These methods also reduce average agent steps, suggesting accumulated experience enables more efficient solution finding.
- Cost-performance tradeoff: On Gemini 3.1 Pro, ACE and AutoSkill achieve evolution gains above +3% with moderate or reduced cost overhead.The passage identifies both methods as combining meaningful evolution gains with manageable cost.
- Cost-performance tradeoff: ReasoningBank delivers a +2.3% evolution gain at 64% of baseline cost on Gemini 3.1 Pro, making it the most cost-efficient method there.On GPT-5.4, only A-Mem achieves a positive evolution gain, reaching +3.2% at 577% of baseline cost.
B Model Evolution Behavior Analysis
This section examines how the three models self-evolve under the Interleaved stream by tracking the evolution states they accumulate and update during task processing.
- Interleaved stream analysis: The analysis compares self-evolution behavior across three models in the Interleaved stream.It goes beyond aggregate performance to characterize model-specific evolution behavior.
- Evolution state: It focuses on the evolution state each model accumulates as tasks are processed.The accumulated state is treated as a central object of analysis.
- Evolution state: It also examines how each model updates that state throughout task processing.The analysis follows state updates over the course of the stream.
B.1 Evolution State Accumulation
Evolution-state accumulation differs substantially across models in both scale and structure. Claude Opus 4.7 builds the largest states, Gemini 3.1 Pro the most compact, and GPT-5.4 occupies an intermediate position with concise items.
- Evolution state size: Claude Opus 4.7 accumulates the largest evolution states, averaging 1251 ACE playbook bullets and 463 Harness skills across methods.Its states are largest across all evaluated methods.
- Evolution state size: Gemini 3.1 Pro produces the most compact states, with only 209 ACE bullets and 117 AutoSkill skills.This contrasts sharply with Claude Opus 4.7’s substantially larger accumulated states.
- Evolution state size: GPT-5.4 generates a moderate number of evolution-state items with relatively concise per-item content.Its accumulated state balances breadth of coverage with manageable state complexity.
- Architectural preference: Under Harness, Claude Opus 4.7 allocates most of its evolution budget to the skill library while retaining only 9K characters of system prompt and memory.This indicates an architectural preference for expanding skills rather than prompt and memory content.
B.2 Update Dynamics
The models adopt distinct strategies for updating their Harness state, which jointly maintains the system prompt, memory, and skill library. GPT-5.4 is largely append-only, while Gemini 3.1 Pro distributes updates more evenly across components.
- Update Dynamics: GPT-5.4 is largely append-only, adding skills while rarely editing them and never modifying the system prompt.It averages only 5 edits over 300 tasks.
- Update Dynamics: Gemini 3.1 Pro spreads updates more evenly, revising memory and the system prompt 267 times while refining skills.
- Update Dynamics: Table 10 summarizes evolution behavior on the Harness state over the Interleaved stream.The Harness state jointly maintains the system prompt, memory, and skill library.
C Detailed Results across Random Seeds · D Cumulative Accuracy Dynamics · E Method Prompts
The appendix reports per-seed results, cumulative-accuracy dynamics, and the full prompts used to implement five self-evolving methods in the test-time streaming setting. Together, these materials document result variability, performance trajectories, and reproducible method configurations.
- E Method Prompts: The appendix provides the full prompts used by self-evolving methods, adapted for test-time operation without ground-truth labels and Exgentic’s multi-step tool-use tasks.This establishes the reproducibility setting for the method implementations.
- C Detailed Results across Random Seeds: Tables 11–13 report method performance across three streaming scenarios for seeds 42, 44, and 46, marking below-baseline and best-within-method scores.Red indicates scores below the model’s vanilla baseline, while blue identifies the best scenario within each self-evolving method.
- D Cumulative Accuracy Dynamics: The remaining dynamics figures track AutoSkill and Harness across GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.7.Figures 14–19 cover these two methods over the three streaming scenarios.
- E Method Prompts: ACE uses a playbook of accumulated strategies and insights, applies relevant guidance during tool use, and reflects on reasoning traces to identify helpful, harmful, or neutral bullets.Its prompts also instruct the curator to add only new, actionable insights rather than regenerate the entire playbook.
- E Method Prompts: A-Mem evolves memories through keyword and context extraction, neighbor-based decisions, connection selection, tagging, and optional neighbor updates.Its decision options include NO_EVOLUTION, STRENGTHEN, UPDATE_NEIGHBOR, and STRENGTHEN_AND_UPDATE.