PaperlayerView exact revision on arXiv

Source-linked AI summary

LifeMem: Enabling Lifelong Experience Reuse for LLM Agents

Yuli Qiu, Yutong Li, Wei Su, Zeming Liu, Wanxiang Che, Heyan Huang, Haifeng Wang, Yuang Guo

arXiv:2609.12655v1cs.CL

TL;DR

LLM agents are expected to adapt across environments by reusing experience, but existing memory-based approaches struggle with cross-environment transfer and accumulated experience can impair retrieval and forgetting. LifeMem clusters trajectories by workflow, distills cluster-specific skills, and recalls skills alongside aligned trajectories; its experiments report reduced forgetting and stronger cross-task transfer. The authors also identify real-world deployment at very large memory scales as an opportunity for further engineering-level co-optimization.

  • Problem

    LLM agents need to reuse experience across evolving environments, but existing memory-based agents struggle with cross-environment transfer and accumulated experience can impair retrieval and contribute to forgetting.

    Source: §1 · pp. 1–2

  • Method

    LifeMem clusters accumulated trajectories by workflow, distills skills within clusters, and recalls skills together with environment-aligned trajectories to guide inference.

    Source: §1 · p. 2, §4.3 · p. 5

  • Results

    LifeMem reports reduced forgetting on learned tasks and stronger cross-task transfer to unseen domains across experiments covering 10 environments and more than 13k tasks.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Environment-aligned trajectories guide action formats, while abstracted skills generalize better to unfamiliar scenarios; consolidating structurally similar trajectories benefits performance.

    Source: §1 · p. 2, §7 · p. 9

    Real-world deployment with millions of stored experiences remains an engineering co-optimization opportunity; lightweight on-device models or approximate retrieval are suggested as possible accelerations.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language model agents are expected to continuously adapt to new tasks and environments over their lifetime by reusing past experience. However, existing memory-based agents struggle to transfer reusable experience across environments and suffer from catastrophic forgetting as experience accumulated. To address these challenges, we propose LifeMem, a lifelong learning framework that enables agents to transfer knowledge across multiple environments. During learning, LifeMem clusters accumulated interaction trajectories based on underlying workflows to extract reusable skills. When solving a new task at inference time, the agent recalls relevant skills and trajectories to guide actions. To validate our method, we conduct experiments across 10 environments and over 13k tasks with 2k newly annotated interaction trajectories. Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer. Further analysis reveals that task streaming impacts learning, while consolidating structurally similar trajectories within memory boosts performance.

1 Introduction

LifeMem targets the challenge of reusing experience across heterogeneous environments without forgetting earlier experience. It clusters trajectories by workflow, distills skills within clusters, and uses both skills and aligned trajectories at inference; experiments report reduced forgetting and stronger cross-task transfer.

1 Introduction

  • Existing memory-based agents struggle to transfer experience across environments, while accumulated experience can impair retrieval and contribute to forgetting.Trajectory-only retrieval can fail across mismatched action spaces, and global skill abstraction can introduce cross-environment noise.Source: §1 · pp. 1–2
  • LifeMem clusters trajectories by underlying workflows, distills skills within each cluster, and recalls both skills and environment-aligned trajectories to guide new tasks.This design aims to avoid cross-environment retrieval interference and noisy skills from global abstraction.Source: §1 · p. 2
  • Across 10 environments and more than 13k tasks, LifeMem shows reduced forgetting on learned tasks and more effective cross-task transfer to unseen domains.The evaluation includes over 2k newly annotated training trajectories for four environments lacking interaction trajectories in existing benchmarks.Source: §1 · p. 2
  • The study contributes over 2k newly annotated interaction trajectories as training data for four environments lacking such trajectories in existing benchmarks.The experiments span embodied action, tool utilization, web browsing, web search, and data analysis.Source: §1 · p. 2
  • Experiments find that task-streaming order impacts learning outcomes, while consolidating structurally similar trajectories benefits agent performance.The paper also reports that environment-aligned trajectories guide action formats and abstracted skills generalize better to unfamiliar scenarios.Source: §1 · p. 2

2 Related Work

Prior lifelong-learning research largely studies static datasets or experience accumulation within one environment, while memory research explores trajectory retrieval and abstract representations. LifeMem's related-work discussion distinguishes these approaches from cross-environment experience reuse based on structured workflows.

2.1 Lifelong Learning in LLMs

  • Prior LLM lifelong-learning work primarily studies continual pretraining, fine-tuning, and alignment on predefined static datasets, while agent studies mainly examine accumulation within a single environment.The latter settings keep state and action spaces fixed.Source: §2.1 · p. 2
  • Cross-environment lifelong learning instead integrates trajectories from heterogeneous environments and transfers experience across domains while preserving previously encountered ones.This is the setting the paper focuses on.Source: §2.1 · p. 2

2.2 Memory Mechanism of LLM Agents

  • Trajectory retrieval by semantic similarity may fail on different-environment or out-of-distribution tasks, while global skill extraction from unorganized experience can yield unrepresentative skills through cross-environment noise.The paper contrasts workflow clustering before distillation with dialogue summaries and knowledge graphs aimed primarily at conversation compression.Source: §2.2 · pp. 2–3

3 Preliminaries

The preliminaries model an LLM agent interacting with an observable environment and define lifelong learning as accumulating trajectories across a sequence of tasks. The objective is to improve future-task performance while mitigating forgetting.

3 Preliminaries

  • The agent's environment is represented as E = (S, A, Ω, T), with states, actions, observations, and transition dynamics.At each time step, the agent receives an observation and selects an action.Source: §3 · p. 3
  • A task is represented by an environment, an initial observation, and a goal; solving it produces a trajectory of observation-action pairs.The supplied formalization denotes the trajectory as ξ_i = {(o_0, a_0), (o_1, a_1), . . . , (o_T, a_T)}.Source: §3 · p. 3
  • In lifelong learning, trajectories from a sequence of tasks, including tasks in distinct environments, enter memory to guide future task solving.The objective is to improve performance on new tasks while mitigating forgetting of previously learned behaviors.Source: §3 · p. 3

4 Method

LifeMem stores experience in workflow-level trajectory clusters that evolve as new experience arrives, then combines environment-aligned examples with transferable skills at inference.

4.1 Memory Initialization

  • LifeMem initializes memory by grouping trajectories into clusters that represent distinct behavioral workflows and describing each cluster.Source: §4.1 · p. 3

4.2 Memory Evolution

  • Incoming trajectories join an existing cluster when functionally aligned or initialize a new cluster when misaligned; diverse, sufficiently large clusters can be partitioned.Partitioning requires both n > L and R < τmrl.Source: §4.2 · pp. 3–4
  • Each structured workflow cluster is distilled into an environment-invariant textual skill capturing shared execution principles beyond low-level environment details.Source: §4.2 · p. 4

4.3 Inference with Memory

  • At inference, LifeMem retrieves same-environment trajectories and activates their associated cluster skills to guide actions, combining concrete exemplars with cross-environment transfer.Source: §4.3 · pp. 4–5

5 Experiment

The experiments evaluate LifeMem across diverse agent tasks, memory baselines, and lifelong-learning metrics. LifeMem outperforms the baselines overall and its workflow-clustered memory mitigates negative transfer as experience accumulates.

5.1 Dataset Construction

  • The evaluation covers five widely used agent scenarios—embodied action, tool utilization, web search, data analysis, and web browsing—and ten tasks across diverse environments, comprising 12,489 training instances and 1,230 test instances.gAgentBench and ToolQA lack training splits; the authors collected 2,310 trajectories, including 1,541 successful trajectories. Human evaluation judged over 90% of the model’s actions reasonable.Source: §5.1 · p. 5

5.2 Baselines

  • The baselines include memory-free ReAct, retrieval-based Synapse, and skill-extraction methods ExpeL and AutoSkill, evaluated with three language-model backbones.The backbones are GPT-4o-mini, Deepseek-v3.2-exp, and Qwen3-32b.Source: §5.2 · p. 5

5.3 Evaluation Metrics

  • Lifelong-learning evaluation measures average performance after all tasks and backward transfer, which captures how subsequent tasks affect earlier-task performance.Positive backward transfer may indicate that later tasks facilitate earlier ones; negative values reflect forgetting caused by subsequent tasks.Source: §5.3 · pp. 5–6

5.5 Main Results

  • LifeMem consistently outperforms all baselines across metrics and five agent scenarios, while workflow clustering suppresses negative transfer and supports positive backward transfer.Figure 3 categorizes transfer scores from -5 to 5 as unchanged; the results attribute baseline interference to cluttered memory and irrelevant retrieval.Source: Table 2 · p. 6, §5.5 · p. 7, Figure 3 · p. 7

5.4 Implementation Details

  • The implementation stores workflow clusters and retrieves trajectories from memory, while the main task stream groups homogeneous environments and orders tasks roughly from easy to hard.The authors also examine randomized and buffer-based streams.Source: §5.4 · pp. 6–7

6 Analysis

The analysis tests LifeMem's components, zero-shot transfer, and sensitivity to task-stream strategies. It finds complementary contributions from trajectories and skills, strong cross-task transfer, and improved stability from grouped or buffered streams.

6.1 RQ1: Ablation Study

  • Environment-aligned trajectories and abstracted skills each contribute positively, and their joint configuration yields the optimal success rate.Trajectories provide demonstrations of successful execution formats, while skills offer macro-guidance; a more powerful skill generator has only a marginal advantage, and LifeMem with GPT-4o-mini still substantially outperforms vanilla GPT-4.1.Source: §6.1 · pp. 7–8, Table 3 · p. 7

6.2 RQ2: Zero-Shot Cross-Task Transfer

  • LifeMem transfers to unseen task formulations and environments by using environment-invariant skills, while baselines show limited gains or performance degradation against ReAct.The tests cover FEVER in the Wikipedia environment and an airline task transferred from τ-bench's retail task; raw retrieval can misguide actions or fail to bridge domain differences.Source: §6.2 · p. 8, Table 4 · p. 8

6.3 RQ3: Impact of Task Stream Strategies

  • Grouping related tasks improves memory stability and reduces retrieval interference, while a buffer-based strategy stabilizes experience reuse under uncurated streams.The analysis compares similarity-ordered, randomized, and buffer-based task streams; Table 5 reports recovery relative to degradation under the random stream.Source: §6.3 · p. 8, Table 5 · p. 8

7 Conclusion

LifeMem structures interaction trajectories into abstracted skills to reuse experience across heterogeneous environments. Across 10 environments and over 13k tasks, it mitigates forgetting and supports zero-shot cross-task transfer.

7 Conclusion

  • LifeMem structures interaction trajectories into abstracted skills to enable experience reuse across heterogeneous environments.Source: §7 · p. 8
  • Across 10 environments and over 13k tasks, LifeMem significantly improves lifelong performance, mitigating forgetting on learned tasks and achieving robust zero-shot cross-task transfer.Source: §7 · pp. 8–9
  • Environment-aligned trajectories guide action formats, while abstracted skills generalize better to unfamiliar scenarios.Source: §7 · p. 9

Limitations

The authors identify engineering-level co-optimization as an opportunity for fully realizing LifeMem’s potential in large-scale industrial deployments.

Limitations

  • Fully realizing LifeMem’s potential in large-scale industrial deployments remains an engineering co-optimization opportunity.Source: Limitations · p. 9
  • For applications with millions of stored experiences, the authors suggest lightweight on-device language models or approximate retrieval as possible efficiency improvements.Source: Limitations · p. 9
  • The proposed deployment optimizations are examples of system-level enhancements, not evaluated results in the supplied passage.Source: Limitations · p. 9

Ethical Consideration

The paper reports data-use and annotation safeguards, then describes benchmark settings spanning embodied action, tools, web search, databases, browsing, and transfer to new tasks and environments.

Ethical Consideration

  • All datasets are publicly available for research under their original licenses; collected trajectories contain no PII, sensitive content, or offensive material.Source: Ethical Consideration · p. 9

A.1 Embodied Action

  • ALFWorld and ScienceWorld use discrete, text-based embodied actions; ScienceWorld adds varied science tasks and a stricter success criterion requiring a score of 100.Source: §A.1 · p. 12

A.2 Tool Utilization

  • APIBank tests multi-step API use, while τ-bench Retail covers multi-turn service tasks involving database-backed tool calls.Source: §A.2 · p. 12

A.3 Web Search

  • HotpotQA and Webshop assess web information seeking and decisions, using Exact Match and mean reward/task success rate, respectively.Source: §A.3 · p. 12

A.4 Data Analysis

  • Database tasks span SQL-related skills, while ToolQA training includes custom-function and SQLInterpreter trajectories and tests SQL-only generalization.Source: §A.4 · p. 13

A.5 Web Browser

  • MiniWoB++ uses simplified web interfaces, whereas Mind2Web uses realistic websites; their contrast provides a curriculum for studying cross-environment transfer.Source: §A.5 · p. 13

A.6 Out-of-Distribution Datasets

  • FEVER tests fact verification in a Wikipedia-based search setting, while Airline tests service tasks in a novel domain with more stringent policy constraints.Source: §A.6 · p. 13

B Human Annotation in Dataset Construction

The appendix reports human assessment of generated trajectories, explains lifelong-learning metrics, and tests stronger retrieval components against Synapse.

B Human Annotation in Dataset Construction

C An Intuitive Illustration of Evaluation Metrics

C.1 Overall Performance

C.2 Backward Transfer

  • Backward Transfer measures the effect of subsequent learning on an earlier task; the example reports that learning MIND2WEB negatively affects MINIWOB++.Source: §C.2 · p. 14

D Investigation on Advanced Retrieval Components

E Hyperparameter Sensitivity Analysis

Across the evaluated settings, LifeMem remains the best-performing memory approach on random task streams, while its success rate is stable across retrieval volumes. Its cluster boundaries involve a trade-off between merging distinct workflows and fragmenting reusable patterns; reported inference costs are modest.

E Hyperparameter Sensitivity Analysis

  • Success rate remains stable across retrieval volumes, suggesting performance depends more on high-quality workflow abstractions than on the number of retrieved trajectories.The evaluated retrieval sizes were top−k values of 2, 4, and 8.Source: E Hyperparameter Sensitivity Analysis · pp. 15–16
  • Under-clustering merges distinct trajectories into bloated clusters, while over-clustering fragments them and can limit transferability by fitting rollout-specific noise.Under-clustering is associated with L being too large or τmrl too low; over-clustering can occur when L is too small.Source: E Hyperparameter Sensitivity Analysis · p. 16

F Ablation Study

  • Table 9 compares three backbone models and defines ablations that remove abstracted skills or environment-aligned trajectories during inference.Source: Table 9 · p. 16

G Costs in Learning and Inference Time

H Performance in Random Settings

I Analysis of Memory Clusters

Human annotations indicate that LifeMem clusters capture consistent execution workflows. Cluster sizes and types vary with workflow diversity, and a failure case illustrates how environment-aligned retrieval can avoid misleading cross-environment trajectories.

I Analysis of Memory Clusters

  • 91% average workflow purity across sampled clusters provides human evidence that clusters capture consistent execution workflows rather than merely embedding similarity.Three annotators identify each cluster’s dominant macro-workflow and mark trajectories whose execution logic deviates from it.Source: I Analysis of Memory Clusters · pp. 16–17
  • WebShop clusters average 29 trajectories versus 12 for τ-bench, while APIBank has more specialized cluster types than AlfWorld and ToolQA.Most clusters consolidate multiple trajectories; the reported patterns suggest granularity adapts to trajectory diversity without manual specification.Source: Figure 5 · p. 17, Figure 6 · p. 17, I Analysis of Memory Clusters · p. 17

N Failure Case Study

  • In the FEVER failure case, Synapse retrieves irrelevant APIBank trajectories, whereas LifeMem restricts retrieval to the environment-aligned HotpotQA repository and provides reusable insights.The passage says the irrelevant trajectories mislead action generation and cause task failure.Source: N Failure Case Study · p. 17
Loading 2609.12655v1…