PaperlayerView exact revision on arXiv

Source-linked AI summary

LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners

Junhao Zheng, Xidi Cai, Qiuke Li, Duzhen Zhang, ZhongZhi Li, Yingying Zhang, Le Song, Qianli Ma

arXiv:2505.11942v3cs.AI

TL;DR

LLM agents remain stateless and current benchmarks do not evaluate lifelong learning across tasks. LifelongAgentBench tests skill-grounded, interdependent tasks in three interactive environments and experiments find that group self-consistency improves experience replay, while memory and context overhead remain challenges.

  • Problem

    LLM agents lack memory to accumulate and transfer knowledge over time, and existing benchmarks do not systematically evaluate lifelong learning across dependent tasks.

    Source: §1 · p. 1

  • Method

    LifelongAgentBench evaluates agents on skill-grounded, interdependent tasks across Database, Operating System, and Knowledge Graph environments, with automatic verification and reproducible, modular infrastructure.

    Source: §1 · p. 2

  • Results

    Group self-consistency significantly improves experience replay across multiple model backbones, while replay’s impact varies and more experience can hurt due to irrelevant information and context limits.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The benchmark measures agents’ ability to accumulate, retain, and transfer knowledge, and experiments show the potential of experience replay and group self-consistency to improve performance.

    Source: §7 · p. 10

    Experience replay adds substantial memory and context-length overhead, especially on long-horizon tasks, and smaller or reasoning-optimized models benefit less from replay.

    Source: §7 · p. 10

Abstract

from arXiv · show

Lifelong learning is essential for intelligent agents operating in dynamic environments. Current large language model (LLM)-based agents, however, remain stateless and unable to accumulate or transfer knowledge over time. Existing benchmarks treat agents as static systems and fail to evaluate lifelong learning capabilities. We present LifelongAgentBench, the first unified benchmark designed to systematically assess the lifelong learning ability of LLM agents. It provides skill-grounded, interdependent tasks across three interactive environments, Database, Operating System, and Knowledge Graph, with automatic label verification, reproducibility, and modular extensibility. Extensive experiments reveal that conventional experience replay has limited effectiveness for LLM agents due to irrelevant information and context length constraints. We further introduce a group self-consistency mechanism that significantly improves lifelong learning performance. We hope LifelongAgentBench will advance the development of adaptive, memory-capable LLM agents.

1 Introduction

LifelongAgentBench addresses the lack of standardized evaluation for LLM agents that learn across dependent tasks by testing skill acquisition, transfer, and retention in three interactive environments. Experiments identify limits of conventional experience replay and introduce group self-consistency to improve it.

1 Introduction

  • Existing benchmarks focus on isolated tasks and lack standardized evaluation of lifelong learning, inter-task dependencies, skill reuse, retention, and forgetting.Prior benchmark adoption is also hindered by label inaccuracies, weak verifiability, and poor reproducibility.Source: §1 · pp. 1–2
  • LifelongAgentBench evaluates agents’ acquisition, transfer, and retention of skills across dependent tasks in Database, Operating System, and Knowledge Graph environments.It is presented as the first unified benchmark for lifelong learning in LLM agents.Source: §1 · p. 2
  • The benchmark combines skill-grounded task dependencies, automatic label verification, containerized reproducibility, and modular agent interfaces.Verification examples include SQL validation, OS state hashing, and SPARQL output verification.Source: §1 · p. 2
  • Group self-consistency partitions historical experience and uses voting, significantly enhancing experience replay across multiple model backbones.The experiments also report that adding more past experience can hurt when it brings irrelevant information or exceeds context limits.Source: §1 · p. 2
  • Experience replay’s impact varies with model size, architecture, and task complexity, and more stored experience can degrade performance through irrelevant information and context limits.These findings motivate the proposed grouped voting mechanism.Source: §1 · p. 2

2 Related Work

Prior work largely evaluates static or non-interactive learning, while LLM-agent benchmarks emphasize single-episode performance rather than cumulative learning. LifelongAgentBench is presented as a standardized framework for evaluating agents under lifelong-learning constraints.

Lifelong Learning

  • Lifelong learning for LLM agents in complex interactive environments over long horizons remains largely unexplored.Earlier lifelong-learning research mainly studies static settings such as image classification and continual instruction tuning, where inputs and outputs are fixed.Source: §2 · pp. 2–3

LLM Agent Benchmarks

  • WebArena [22], AgentBench, and VisualWebArena assess single-episode performance in static environments, without sequential decision-making, cumulative learning, or skill transfer.Game-playing [3] and tool-use work explores interaction but does not provide standardized lifelong-learning protocols.Source: §2 · p. 3
  • LifelongAgentBench provides persistent environment states, explicit task dependencies, and scalable experience replay for reproducible lifelong evaluation.The framework is intended to support systematic study of generalization, skill transfer, and long-term retention.Source: §2 · p. 3

3 Problem Formulation of Lifelong Learning for LLM Agents

The paper formulates lifelong learning for LLM agents as sequential decision-making over tasks modeled with goal-conditioned POMDPs. The benchmark assesses whether agents can use past experience to improve current-task performance.

3 Problem Formulation of Lifelong Learning for LLM Agents

  • Each task is modeled as a goal-conditioned partially observable Markov decision process, with an agent policy mapping observations to natural-language actions.The environment is specified through states, actions, goals, transitions, rewards, observations, and an observation function.Source: §3 · p. 3
  • The agent receives a single binary reward after submitting its final answer, and the objective is to maximize cumulative expected reward across a sequence of tasks.The task sequence is represented as a set of tasks.Source: §3 · p. 3
  • LifelongAgentBench evaluates whether agents leverage past experience to improve performance on current tasks.Source: §3 · p. 3

4 Data Construction

LifelongAgentBench builds skill-grounded tasks across Database, Operating System, and Knowledge Graph environments, with task complexity and skill overlap designed to support lifelong-learning evaluation. Its construction balances skill coverage and validates labels using automated checks and manual review.

4 Data Construction

  • The dataset combines three environments with skill-grounded tasks, controlled skill distributions, varied task complexity, and noise management.Tasks are associated with atomic skills, and task relatedness is quantified using the harmonic mean of shared-skill proportions. Infrequent skills receive higher sampling probabilities; task complexity and rare-skill prevalence support robustness analysis, with extensive skill connections shown in Figure 2.Source: §4 · p. 4, §4.1 · p. 4, Figure 2 · p. 4

4.2 Environment Implementations

  • The environments operationalize 22 SQL skills, 29 Bash skills, and SPARQL queries over GrailQA-derived tasks in isolated database, Ubuntu, and knowledge-graph settings.Database tasks are generated and filtered, while OS command sequences are generated for logical consistency; KG tasks are mapped from S-expressions to logical action sequences.Source: §4.2 · pp. 4–5

4.3 Quality Control

  • Automated SQL, Bash, and SPARQL checks are supplemented by manual review of 10% of tasks from each environment.Validation includes result comparison, exit-code checking, or output verification, depending on the environment.Source: §4.3 · p. 5
  • 500 Database tasks were selected from 1,306 generated; OS retains 500 complex 9–12-step tasks, while KG contributes 396 tasks spanning 2–9 steps.OS tasks of 1–8 steps were excluded after preliminary tests showed minimal replay benefits; replay effects diminished for KG sequences exceeding six steps.Source: §4.3 · p. 5

5 Evaluation Framework

LifelongAgentBench evaluates agents through sequential task execution and experience accumulation. Its modular framework supports reproducible experiments, while strict sequencing distinguishes it from benchmarks designed for parallel task execution.

5 Evaluation Framework

  • The framework evaluates lifelong learning through sequential tasks and accumulated experience rather than static or single-task evaluation.Its six loosely coupled components are the model pool, agent, environment, chat history factory, controller, and callbacks.Source: §5 · p. 5, §5.1 · p. 5

5.1 System Architecture

  • The six components communicate through an RPC toolkit and can be deployed locally or across servers.The agent turns observations and dialogue history into actions; the environment executes them, while the controller manages task scheduling and interaction.Source: §5.1 · pp. 5–6

5.2 Reproducibility and Modularity

  • Fixed random seeds and containerized environment snapshots ensure deterministic behavior and identical task conditions across runs.Modular APIs allow new environments, task generators, agent architectures, and evaluation metrics to be integrated with minimal engineering overhead.Source: §5.2 · p. 6

5.3 Differences from Prior Benchmarks

  • Strict sequential execution preserves experience accumulation and transfer assessment, unlike prior frameworks that parallelize tasks or sequences.The design also supports single-process debugging with optional distributed scalability, lowering the barrier to lifelong-learning research.Source: §5.3 · p. 6

6 Evaluation of LifelongAgentBench

LifelongAgentBench evaluates LLM agents across three interactive environments, testing how experience replay performs across models and task difficulties. Group self-consistency is presented as a way to improve replay scalability, while failure analysis identifies recurring agent and context-management problems.

6.1 Experimental Setup

  • The evaluation compares four agents across DB, OS, and KG using task success rate, with baseline, replay-buffer, and group self-consistency settings.Replay uses recent successful trajectories; success means completing a task with the correct action sequence.Source: §6 · p. 6, §6.1 · p. 6

6.2 Main Results

  • Replay raises DB accuracy from 19% to 78% at 64 examples, OS accuracy from 43% to 50% at 4–16 examples, and KG accuracy from 28% to 35% with 1 example.The comparison uses the Llama-3.1-8B-Instruct backbone.Source: Table 2 · p. 7, §6.2 · p. 7
  • Replay beyond the useful range can yield diminishing or negative returns through longer inputs, greater reasoning complexity, and out-of-memory failures.Shorter DB trajectories tolerate larger buffers, while long-form OS and KG tasks peak at smaller replay sizes; memory-efficient replay remains an open challenge.Source: §6.2 · p. 7

6.3 Effect of Model Backbone and Task Difficulty

  • Llama-3.1-70B-Instruct reaches 0.90 accuracy with 64 examples, while Qwen2.5-32B-Instruct achieves 0.82 without replay and shows no clear replay improvement.Reasoning-optimized DeepSeek variants perform worse and are prone to OOM at large replay sizes; larger models generally outperform smaller counterparts, though Llama-3.1-8B-Instruct reaches 0.78 at 64 examples.Source: §6.3 · pp. 7–8, Table 3 · p. 7

6.4 Effect of Task Difficulty

  • In DB, replay gains are larger on Hard tasks, rising from 49% to 62%, than on Easy tasks, rising from 70% to 76%; in KG, short tasks benefit more than long ones.KG tasks of action-sequence length 2 improve from 48% to 84%, while lengths 7–9 see minimal or no improvement; longer trajectories can dilute signal and risk context overflow.Source: Table 4 · p. 8, §6.4 · p. 8, Table 5 · p. 8

6.5 Scaling Experience with Group Self-Consistency

  • Group self-consistency splits retrieved experiences into smaller groups and aggregates predictions with self-consistency voting, stabilizing performance and alleviating memory bottlenecks across environments and models.Qwen2.5-7B-Instruct DB accuracy rises from 0.72 to 0.77; smaller DeepSeek-R1-Distill variants show limited gains.Source: §6.5 · pp. 8–9
  • Group self-consistency raises DB accuracy from 0.61 to 0.75 for Llama-3.1-8B-Instruct and cuts KG input tokens from 56,409 to 11,002 while maintaining stable accuracy.For KG, Qwen2.5-7B-Instruct token use falls from 59,950 to 11,339 with minimal accuracy loss.Source: §6.5 · pp. 8–9, Table 6 · p. 9

6.6 Failure Mode Analysis

  • Four recurring failures are wrong final submissions, failure to commit, format violations, and context overflow, reflecting unstable reasoning, poor instruction adherence, and context-management issues.Source: §6.6 · p. 9

7 Conclusion, Limitation, and Future Works

LifelongAgentBench offers a unified platform for evaluating lifelong learning in agents. The conclusion highlights replay’s overhead and model-dependent benefits, and points to more efficient memory strategies and broader task settings as future directions.

7 Conclusion, Limitation, and Future Works

  • The experiments demonstrate the potential of experience replay and group self-consistency to improve agent performance, while also revealing remaining challenges.The conclusion presents LifelongAgentBench as a unified benchmark for evaluating agents’ ability to accumulate, retain, and transfer knowledge across interactive environments.Source: §7 · p. 10
  • Experience replay adds substantial memory and context-length overhead on long-horizon tasks, and smaller or reasoning-optimized models benefit less.Replay performance varies across model architectures.Source: §7 · p. 10
  • LifelongAgentBench provides a standardized platform, baselines, and diagnostic tools for studying continual adaptation in agents.The authors identify efficient memory retrieval, dynamic experience selection, and multimodal and real-world tasks as future directions.Source: §7 · p. 10

A Lifelong Learning Environment Design

A.1.1 Environment Introduction

  • The database environment reuses one Docker container per evaluation, creating and deleting task-specific tables to isolate tasks and avoid repeated initialization.The container is initialized from a predefined image; frequent restarts would increase evaluation time.Source: §A.1.1 · pp. 12–13
  • Database tasks use 22 defined skills, and task success is checked against submitted query results or a ground-truth database-state hash.Source: §A.1.1 · p. 13

A.1.2 Data Construction Process

The appendix describes skill-grounded construction and environment-specific task execution and evaluation. Construction balances skill coverage, while the knowledge-graph tasks derive from an existing question-answering dataset.

A.1.2 Data Construction Process

  • Database task construction uses an LLM to generate benchmark data and samples less frequent skills more often, enforcing a minimum occurrence threshold.The construction process is illustrated in Figure 5 and follows the cited methodology to prevent data leakage [16].Source: §A.1.2 · pp. 13–14, Figure 5 · p. 14

A.2.1 Environment Introduction

  • Operating-system tasks run Bash commands in Ubuntu containers, with success determined by task-specific evaluation scripts and a restricted set of 29 commands.The environment uses new containers to isolate tasks because system state is difficult to fully roll back.Source: §A.2.1 · p. 14

A.2.2 Data Construction Process

  • Operating-system task generation uses an LLM, with fewer construction calls than in the database environment to reduce token costs.The process is illustrated in Figure 6.Source: §A.2.2 · p. 14, Figure 6 · p. 15

A.3.1 Environment Introduction

  • Knowledge-graph agents use constrained actions to explore relevant graph information progressively, with retrieved entity sets represented by reusable variables for multi-hop reasoning.The action sequence for each task defines its skills.Source: §A.3.1 · p. 15

A.3.2 Data Construction Process

  • Knowledge-graph tasks are based on a prior question-answering dataset [6], converting its graph-query S-expressions into ground-truth action sequences.Source: §A.3.2 · p. 16

B Evaluation Framework Design

The evaluation framework is designed to support modular, comparable evaluation while addressing slow execution and task parallelism that can disrupt lifelong-learning order. It supports both single-machine and distributed deployments, using persistent orchestration, synchronized interaction-history access, and callback-controlled task flow.

B Evaluation Framework Design

  • The framework addresses prolonged evaluation, parallel execution that breaks lifelong task order, and missing standardized interfaces for fair comparison of agent-enhancement methods.The paper notes that WebArena tasks can take up to ten minutes and that parallel task scheduling can prevent transfer-relevant experience from following the intended sequence.Source: B Evaluation Framework Design · p. 16

B.1 Framework Architecture

  • Its six components are a language model pool, agent, environment, interaction-history item factory, controller, and callbacks.The pool maps string identifiers to model objects and can integrate different model classes; the agent turns prior actions and observations into input for generating the next action.Source: §B.1 · p. 17, §B.1.1 · p. 17, §B.1.2 · p. 17

B.1.3 Environment

  • A shared abstract environment interface defines task reset, interaction, completion, metric calculation, and resource release, supporting extensions to new environments and broadly applicable enhancement methods.The framework requires each environment to implement the parent class’s abstract methods and expose only its specified public methods to other components.Source: §B.1.3 · p. 18

B.1.4 Interaction History Item Factory

  • The interaction-history item factory separates history construction from environment-specific logic and initializes task history using the goal and initial observation.This separation is presented as improving modularity, code clarity, and extensibility.Source: §B.1.4 · p. 18

B.1.5 Controller

  • The controller creates task sessions, records execution history, and periodically snapshots experiment state so interrupted runs can resume while skipping completed tasks.After each task, its session is saved; an interruption therefore loses at most one task’s interaction data.Source: §B.1.5 · p. 18

B.1.6 Callback

  • Callbacks let researchers modularly add capability-enhancement methods at task events, using session history and a shared model pool to support comparable experiments with lower memory requirements.Callbacks can modify the current task’s interaction history using prior sessions; sharing model parameters means identical models used by the agent and callbacks are loaded into GPU memory only once.Source: §B.1.6 · pp. 18–19

B.2 Differences Between Distributed Deployment and Single-Machine Deployment

  • The framework supports single-machine, single-process development and distributed evaluation, with RPC intended to make switching modes require little additional reusable code.Local deployment is intended to ease prototyping and debugging, while distributed deployment allows components to run across servers and processes for large-scale models.Source: §B.2 · p. 20

B.2.1 Remote Procedure Call Toolkit

  • The RPC toolkit wraps remotely callable objects with client and server counterparts, handling serialization and HTTP calls so components can communicate across processes.The distributed setup typically places the client-side controller on an HPC node and the controller, environment, and history-factory servers on Docker-capable servers.Source: §B.2.1 · pp. 20–21, §B.2.2 · p. 22

B.2.2 Distributed Deployment Framework Structure

  • After each experiment, the client-side controller, factory server, and environment server terminate, while the server-side controller persists to reinitialize servers from the next configuration.The client-side controller wraps the single-machine controller with functions to start and shut down the required servers.Source: §B.2.2 · pp. 23–24
  • Running the interaction-history factory in a separate process lets environment-side and client-side callers modify it through remote procedure calls, synchronizing changes across processes.Without this separation, client-side changes affect only a local mutable copy and are not synchronized with the environment server.Source: §B.2.2 · p. 24

B.3 The Execution Process of a Task

  • Task execution uses callback-controlled reset, inference, interaction, and completion flags that default to true, and component interactions remain the same in distributed and single-machine deployments.Callbacks receive parameters by reference and can change the flags to control agent-environment interaction; in single-machine deployment, components interact directly without RPC servers or clients.Source: §B.3 · p. 24

C.1 Database Samples

The appendix illustrates database and operating-system task samples through instructions, ground-truth actions, and skill lists. Examples range from SQL edits and queries to shell configuration and administration tasks.

C.1 Database Samples

  • Database samples pair natural-language goals with SQL actions and skill labels, including updates and deletions.Source: §C.1 · pp. 24–25
  • Other database examples cover filtering, ordering, aggregation, aliases, and nested subqueries, with skills listed for combined operations.Source: §C.1 · pp. 26–27

C.2 Operating System Samples

  • Operating-system samples include multi-step configuration edits, such as changing Nginx settings with shell commands.Source: §C.2 · p. 27
  • Additional OS tasks illustrate user and group administration, permissions, file operations, and backup-related commands.Source: §C.2 · pp. 27–28

D Error Case Analysis

The analysis separates successful and incorrect outcomes by agent behavior and system status, including cases where the database state is correct despite no explicit submission. It highlights reasoning and SQL translation errors alongside interaction-limit and infrastructure failures.

D Error Case Analysis

  • The taxonomy distinguishes valid completion from correct database state without submission, and classifies incorrect submissions, non-convergence, format violations, context limits, and rare system failures.It is intended to diagnose bottlenecks in agent behavior and system status.Source: D Error Case Analysis · pp. 30–31
  • The benchmark also records rare environment, unknown-task, GPU-memory, and agent-internal failures, separating system-level errors from the agent's common failure modes.The taxonomy is presented as a diagnostic aid for developing more robust lifelong-learning agents.Source: D Error Case Analysis · p. 31

D.1.1 Error Case 1

  • In Error Case 1, the agent used LIMIT 4 instead of LIMIT 10, then corrected the query but had already incurred an extra interaction and an incorrect task outcome.The case attributes the failure to unstable execution of a correctly understood natural-language request involving LIMIT and OFFSET.Source: §D.1.1 · pp. 31–32
  • The case analysis describes the initial query as an execution error despite correct reasoning, with the agent recognizing the mistake after seeing only four returned entries.It recommends checking SQL components against the instruction or adding intermediate verification.Source: §D.1.1 · pp. 32–33

D.1.2 Error Case 2

Error Case 2 illustrates repeated confusion about SQL LIMIT and offset syntax in a grouped database query. The agent's attempted correction still returned a shifted result set.

D.1.2 Error Case 2

  • The agent first used LIMIT 2, 5 and then LIMIT 3, 5 instead of OFFSET 2 LIMIT 5, producing incorrect or shifted results.The case identifies persistent misunderstanding of LIMIT and OFFSET syntax, rather than a one-off oversight.Source: §D.1.2 · pp. 33–34
  • The analysis recommends reinforcing SQL syntax knowledge or adding explicit syntactic checks during intermediate reasoning and execution.It presents these as potential ways to improve accuracy.Source: §D.1.2 · p. 34
  • For the task's grouped region query, the initial result included five entries, while the subsequent offset adjustment returned four entries.The requested results were grouped by region and subject to usage and call-duration conditions.Source: §D.1.2 · p. 35

D.2.1 Error Case 3

Error Case 3 shows an agent performing the requested database update but reaching the interaction limit before explicitly committing task completion. The analysis points to task-closure and interaction planning as improvement targets.

D.2.1 Error Case 3

  • Although the operations were logically sound and correct, the task was marked unsuccessful because the agent did not explicitly conclude it before the limit.The task required a 15% increase for Security roles with over 30 hours and rates below the Security-role average.Source: §D.2.1 · p. 36
  • The analysis suggests intermediate commitment strategies and better interaction planning to manage strict task limits in multi-step tasks.It frames these as potential ways to improve task performance and success rates.Source: §D.2.1 · p. 36
  • The agent calculated the Security-role average, selected eligible staff, and performed the 15% rate increase, but reached the interaction limit while trying to verify the update.It failed to explicitly commit a final answer within the allowed steps.Source: §D.2.1 · pp. 36–37

D.3 Error Case 4

The examples show agents failing despite correct work: one repeats queries after obtaining the requested result, another violates the one-statement rule, and a third ignores required SQL formatting.

D.3 Error Case 4

  • The agent obtained the correct tax totals but pursued unnecessary sorting changes and exhausted the interaction limit without submitting a final answer.The totals were unpaid 6800 and pending 6100; no equal totals required extra tie-breaking logic.Source: §D.3 · p. 37

D.4.1 Error Case 5

  • A valid workaround for MySQL’s unsupported LIMIT in an IN subquery failed because the agent executed two SQL statements in one operation.The instructions required one statement per interaction.Source: §D.4.1 · p. 40

D.4.2 Error Case 6

  • The agent’s INSERT was logically correct but failed validation because it omitted the required Markdown code block.The task’s formatting and execution structure were explicit requirements.Source: §D.4.2 · p. 42

D.5.1 Error Case 7

In the Ubuntu task, the agent’s unscoped directory listing produced over four million characters, exceeded the context window, and prevented task completion.

D.5.1 Error Case 7

  • The task required appending text, directory contents, hostname and file data to a log, then setting its ownership and permissions within five rounds.The requested directory contents included / and /home.Source: §D.5.1 · pp. 43–44
  • Over 4 million characters of output from find / exceeded the LLM context window and caused the task to fail with agent_context_limit.The agent could not continue interacting or process the remainder of the task.Source: §D.5.1 · p. 43
  • The case identifies unbounded command output as a context-management risk and suggests limiting output or chunking operations across interactions.Examples include find / -maxdepth 1 and head.Source: §D.5.1 · p. 43
Loading 2505.11942v3…