PaperlayerView exact revision on arXiv

Source-linked AI summary

Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents

Xiangyi Zeng, Baihang Liu, Xutong Wang, Ze Jin, Yunpeng Li, Qixu Liu

arXiv:2610.12124v1cs.AI

TL;DR

Long-term LLM-agent operation raises the challenge of turning continuous experience into reusable knowledge. Hippocam organizes work as nested intents and consolidates history while preserving details for recall. It leads or matches the best compared method on seven of eight measures across memory, knowledge use, and learning.

  • Problem

    Long-term, open-ended LLM-agent interactions make it challenging to continually learn from accumulating experience through memory.

    Source: §1 · p. 1

  • Method

    Hippocam organizes interaction history around nested intents, consolidates completed work and older prefixes, and preserves originals for selective recall and subsequent consolidation.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    Hippocam leads or matches the best compared method on seven of eight measures across memory, knowledge use, and learning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest a path toward agents that continually learn from their own work through consolidation and selective recall of interaction history.

    Source: §6 · p. 9

Abstract

from arXiv · show

The evolution of Large Language Model agents from single-task execution to long-term autonomous operation highlights the critical challenge of transforming continuous experiences into reusable knowledge. To address this, we propose Hippocam, a hierarchical memory and continual learning architecture. Hippocam draws inspiration from two characteristics of human memory: cognitive processes selectively maintain information relevant to current goals, while long-term memories form gradually through repeated consolidation. Accordingly, Hippocam structures an agent's ongoing work as nested intents. The active context remains centered on the current intent, while completed intents are consolidated into the task-relevant outcomes and state needed for subsequent work, rather than carrying forward their full working details. Concurrently, a recursive prefix consolidation mechanism repeatedly consolidates earlier history, causing long-unused experiences to become increasingly abstract. Original interactions are preserved, allowing the agent to progressively recover finer-grained details through the hierarchy and stop once sufficient information is available. Crucially, when past experiences are recalled and reintegrated into active work, they undergo subsequent consolidation alongside new experiences, thereby being reinforced, supplemented, and updated. Through this memory dynamic of use and disuse, Hippocam connects working context, long-term memory, knowledge accumulation, and skill learning within a single continuously evolving experiential process. This enables agents to learn and evolve capabilities through their own experiences without parameter updates.

1 INTRODUCTION

Hippocam addresses the challenge of learning from accumulated experience during long-term interaction by organizing and consolidating interaction history. Its reported evaluations span memory, knowledge use, and learning.

1 INTRODUCTION

  • Long-term, open-ended interaction makes continual learning from accumulating experience through memory a central challenge for sustained autonomous operation.Source: §1 · p. 1
  • The approach draws on a view of memory in which short-term memory stays focused on current intent while long-term memory develops through repeated practice and consolidation.Source: §1 · p. 1
  • Hippocam leads or matches the best compared method on seven of eight measures across memory, knowledge use, and learning.Figure 1 reports performance (%) using DeepSeek-V4.1-Flash as the task backbone.Source: Figure 1 · p. 2, §1 · p. 2
  • Hippocam organizes ongoing work as nested intents and consolidates completed work and earlier history, preserving original interactions for selective recovery and later updating.Source: §1 · p. 2

2 HIPPOCAM

Hippocam manages work through nested intents, consolidates completed work and older prefixes, and preserves originals for progressive recall. Recalled knowledge and new experience are consolidated together, while unused details become more abstract.

2.1 OVERVIEW

  • Hippocam combines a main agent with JANUS for intent-stack guidance, PRECIP for intent consolidation, and PALIM for recursive prefix consolidation and knowledge updating.Source: §2.1 · p. 2

2.2 INTENT-STRUCTURED WORKING CONTEXT

  • Nested intents encode bounded purposes, while JANUS maintains the stack by continuing, deepening, closing, or shifting intents as work changes.Source: §2.2 · p. 3

2.3 INTENT CONSOLIDATION

  • When intents close, PRECIP replaces their messages with a self-contained account retaining relevant state, evidence, provenance, and informative failures while pruning irrelevant process details.Source: §2.3 · pp. 3–4

2.4 PREFIX CONSOLIDATION

  • PALIM consolidates earlier context prefixes to reduce interference, recursively abstracting narratives and merging, supplementing, or updating retained facts and lessons.Source: §2.4 · p. 4

2.5 MESSAGE TREE AND PROGRESSIVE RECALL

  • Consolidation preserves originals as ordered children in a message tree, letting the agent recall missing details and descend further only while needed for ongoing work.Source: §2.5 · p. 4

2.6 LEARNING THROUGH USE AND DISUSE

  • Use and disuse form a continuous learning cycle: successes and failures inform later consolidation, reinforcing, supplementing, or updating used knowledge while unused details lose specificity.Source: §2.6 · pp. 4–5

3 RELATED WORK

Hippocam relates to work on context management, agent memory, and continual learning from experience. It distinguishes itself by managing earlier prefixes and developing knowledge through work and feedback without a separate memory bank or skill library.

Context Management

  • Unlike context-management approaches focused on suffixes or subtask execution, Hippocam uses explicit intent requirements to guide retention and also manages earlier prefixes.Source: §3 · p. 5

Agent Memory

  • Unlike memory systems with separate stores, scoring, or decay rules, Hippocam treats context and messages as memory; its tree emerges from ongoing work consolidation rather than semantic similarity.Source: §3 · p. 5

Learning from Experience

  • Hippocam develops knowledge through work and feedback, unifying memory and learning through use and disuse rather than consolidation alone.Source: §3 · p. 5

4 EXPERIMENTS

Across interactive and memory-intensive evaluations, Hippocam is tested for accumulated learning, cross-domain transfer, long-interaction reliability, and mechanism generality. It leads or ties on seven of eight measures, though results vary across tasks.

4.1 RQ1: LEARNING FROM ACCUMULATED EXPERIENCE

  • Across ALFWorld, ScienceWorld, and StreamBench, Hippocam improves on all three, tying ACE on ALFWorld and leading StreamBench at 81.7%.It records 97.8% ALFWorld success and 81.9% mean ScienceWorld progress; on StreamBench it narrowly exceeds Letta’s 81.3%. At task 300, it maintains its DDXPlus lead and moves ahead of Letta on ToolBench.Source: Figure 4 · p. 6, §4.1 · p. 6, Table 1 · p. 6

4.2 RQ2: GENERALIZATION ACROSS TASK DOMAINS

  • On ScienceWorld, Hippocam reaches 78.9% mean progress, 30.1 pp above Plain and 16.2 pp above ACE; FEVER scores show more limited transfer.For sequential integration, Hippocam gains 19.2 pp on ScienceWorld and 6.0 pp on FEVER, exceeding ACE’s respective gains of 14.0 pp and 2.0 pp.Source: Table 2 · p. 7, §4.2 · p. 7

4.3 RQ3: MEMORY RELIABILITY OVER LONG INTERACTIONS

  • Across interleaving, conversation recall, and updated-fact resolution, Hippocam remains effective, reaching 77.6% LoCoMo accuracy, 10.1 pp above A-Mem.On GoodAI LTM, it scores 85.1% in isolation and 88.7% under interleaving without degradation. On MemoryAgentBench, it averages 71.7% versus Letta’s 44.3%, retaining 77.0% accuracy at 32k inputs and 41.0% at 64k, where no baseline exceeds 37.0% and 19.0%, respectively.Source: Figure 5 · p. 7, Table 3 · p. 8, §4.3 · p. 8

4.4 RQ4: GENERALITY OF THE UNIFIED MEMORY–LEARNING MECHANISM

  • Hippocam leads or ties on seven of eight measures, while other methods show uneven capability profiles across memory, knowledge use, and learning.The paper attributes this breadth to a shared use-and-disuse cycle over interaction history rather than separate capability-specific mechanisms.Source: §4.4 · p. 8

5 ANALYSIS

Ablations test intent-conditioned consolidation, prefix consolidation, and recall; additional analyses examine strategy induction, adaptive retrieval depth, and how use and disuse change retained knowledge.

5.1 ABLATION STUDIES

  • Removing intent conditioning raises filler retention from 14% to 63% and worsens interleaved GoodAI performance, supporting intent-aware selection of content and detail.Under interleaving, scores fall by 23.1 pp, compared with 7.3 pp in isolation; a case also shows individual menu options retained for ordering.Source: Table 4 · p. 9, §5.1 · p. 9
  • Without prefix consolidation, ToolBench accuracy falls from 78.7% to 65.7%, while ALFWorld actions and inspected episode-start context increase.Mean ALFWorld actions rise from 9.62 to 11.74, and episode-start context grows from 19.7k to 50.7k tokens.Source: Table 4 · p. 9, §5.1 · p. 9
  • Disabling recall reduces MemoryAgentBench 64k accuracy from 41.0% to 27.0%, showing that access to underlying evidence adds support beyond consolidated context.Source: §5.1 · p. 9

5.2 LEARNING THROUGH EVOLVING EXPERIENCE

  • In ALFWorld, replacing Prefix Consolidation with ACE-style playbook generation yields near-perfect success but slightly more actions, with no clear advantage.The playbook variant uses 9.00 actions on average versus 8.12 for the original configuration.Source: §5.2 · p. 9
  • MemoryAgentBench cases show retrieval depth adapting to the question: one needs two recall levels, while another is resolved after one.The cases illustrate recovery of sufficient evidence without expanding the entire history.Source: §5.2 · p. 9
  • Knowledge linked to intervening work is more often supplemented or updated, while recursive consolidation lets unused episode details recede.ALFWorld traces show repeated successful desklamp use supporting a lesson across objects and instructions, and later searches revising earlier failure accounts.Source: §5.2 · p. 9

6 CONCLUSION

Hippocam unifies memory, knowledge, and learning through continual consolidation and selective recall of interaction history. The authors also report using AI tools during development and manuscript preparation.

6 CONCLUSION

  • Hippocam connects memory, knowledge, and learning through continual consolidation and selective recall of interaction history.Source: §6 · p. 9

AI USE STATEMENT

  • AI tools supported code development, literature searches, manuscript polishing, and figure preparation.Source: AI USE STATEMENT · p. 10

A EXPERIMENTAL PROTOCOLS AND ADDITIONAL RESULTS

The appendix documents evaluation scope, implementation choices, and protocols for experience-accumulation benchmarks. It specifies how task scores and extended StreamBench horizons are reported.

A EXPERIMENTAL PROTOCOLS AND ADDITIONAL RESULTS

  • Evaluations report task-level scores from single runs, and the comparison is not matched by total inference compute or auxiliary output budget.Source: A EXPERIMENTAL PROTOCOLS AND… · p. 14
  • Baseline implementations differ in reasoning and extraction settings, while ACE uses its upstream components in online mode with benchmark-specific feedback protocols.HiAgent is evaluated using an implementation that includes a subsequently added summarization module.Source: A EXPERIMENTAL PROTOCOLS AND… · p. 14
  • Consolidation thresholds control when context is consolidated, and Table 5 reports the default implementation budgets, including configured context capacity W.The working-tail budget uses the 95th percentile of observed main-agent action-round lengths.Source: A EXPERIMENTAL PROTOCOLS AND… · p. 14, Table 5 · p. 14
  • Table 6 reports StreamBench accuracy at two horizons, with the mean available only when both streams reach the stated horizon.Source: Table 6 · p. 15, A EXPERIMENTAL PROTOCOLS AND… · p. 15

A.1 EXPERIENCE ACCUMULATION

  • ALFWorld measures target success after source experience, while ScienceWorld reports normalized mean target progress and StreamBench equally weights its two stream accuracies.Source and target episodes remain separate in reported scores; task streams preserve memory across tasks.Source: §A.1 · p. 15

A.2 SOURCE INITIALIZATION AND CROSS-DOMAIN EVALUATION

The section specifies two source-initialization evaluation protocols and reports sensitivity of the FEVER result to answer extraction.

A.2 SOURCE INITIALIZATION AND CROSS-DOMAIN EVALUATION

  • Sequential ScienceWorld comparisons pair target identifiers between source-initialized and empty-memory conditions, while both conditions accumulate target-domain experience.The comparison therefore contrasts two learning streams rather than measuring source-only knowledge on independent tasks.Source: §A.2 · p. 15
  • Frozen-source evaluation restores the same source snapshot before each target task and clears the workspace, preventing within-task changes from carrying to the next task.Source: §A.2 · p. 15
  • Under alternative answer extraction, Hippocam’s sequential FEVER improvement falls from six to two percentage points; the main table uses the original scoring.Source: §A.2 · p. 15

A.3 RETENTION, INTERFERENCE, AND CONFLICTING INFORMATION

The appendix describes protocols for retention, interference, and conflicting-information evaluations, with boundaries on how some comparisons should be interpreted.

A.3 RETENTION, INTERFERENCE, AND CONFLICTING INFORMATION

  • GoodAI’s isolated and interleaved modes differ in reset behavior, wait-action handling, and answer brevity, so their scores are interpreted as performance under separate protocols.Source: §A.3 · pp. 15–16
  • Table 7 reports frozen-source performance and target-evaluation costs, excluding source experience acquisition from the cost total.Source: Table 7 · p. 16
  • LoCoMo covers ten conversations and reports binary correctness from DeepSeek-V4-Pro, not the benchmark’s lexical F1; the judge does not receive the conversation itself.For adversarial questions, correctness requires identifying that the requested information is absent.Source: §A.3 · p. 16
  • MemoryAgentBench conflict-resolution results use all 100 questions at each multi-hop input length and substring matching, comparing complete systems with different retained representations and memory budgets.Source: §A.3 · p. 16, Table 8 · p. 16
  • DetectiveQA evaluates ten novel-length memories containing 71 questions, with scores counted by question rather than by ingested document.Source: §A.3 · p. 16
  • Separate 32k-window experiments terminate after capacity failures and are not pooled with the main results.Source: §A.3 · p. 16
  • Inspected MemoryAgentBench failures include lost updates during consolidation and obsolete-value selection despite retained evidence; complete-system comparisons do not isolate individual mechanisms.Source: §A.3 · pp. 16–17
  • Table 9 compares prefix consolidation and alternative knowledge configurations, including an ACE variant that replaces prefix consolidation with playbook generation.Source: Table 9 · p. 17

A.4 ABLATIONS AND CONSOLIDATION COMPARISONS

The ablation removes explicit intent conditioning while retaining the intent controller and consolidation mechanism; a separate case illustrates how intent guidance affects retained detail for later actions.

A.4 ABLATIONS AND CONSOLIDATION COMPARISONS

  • Without explicit intent conditioning, the interleaved score falls from 88.7% to 65.6%, while the isolation score falls from 85.1% to 77.8%.Each condition is a single run, and the paired two-sided sign test for interleaved task scores gives p = 0.11.Source: §A.4 · p. 17
  • Removing intent-specific retention instructions and the explicit intent chain leaves the intent controller and consolidation mechanism intact.Janus’s focus and completion messages remain in the conversation.Source: §A.4 · p. 17
  • Without intent conditioning, filler question–answer retention rises from 14% to 63%, while a SallyAnne case shows a task-associated story beginning retained before later events remain active.The trace does not establish that event order was lost, and other score differences involve answer behavior or grading, so the aggregate gap cannot be attributed entirely to consolidation.Source: §A.4 · p. 17
  • Excluding Jokes, whose grader can credit a list containing the requested joke, preserves the interleaved advantage at 91.1% versus 72.8%.The reported comparison is the interleaved score without the Jokes category.Source: §A.4 · p. 17
  • Other comparisons include removing prefix consolidation while truncating older prefixes, removing recall with ingestion held fixed, replacing prefix consolidation with ACE playbook generation, and using empty-archive or flat-summary baselines.The flat summary changes both representation and length; the prefix variant keeps intent consolidation and recall available, while the recall variant uses the same post-ingestion snapshot.Source: §A.4 · p. 17

A.5 INTENT AND THE GRANULARITY OF RETAINED INFORMATION

  • In the Restaurant example, intent-guided consolidation retains individual dishes and leads to a listed choice, whereas unguided consolidation retains category names and number ranges and leads to an unlisted dish.Figure 6 illustrates a retention-granularity contrast, not an isolated cause of the response.Source: §A.5 · p. 18, Figure 6 · p. 18

A.6 KNOWLEDGE CHANGES ACROSS CONSECUTIVE CONSOLIDATIONS

The analysis tracks knowledge-item changes across successive consolidations and groups transitions by proxies for use. Changes are associated with detected use in ToolBench and, more clearly in ALFWorld than ScienceWorld, textual reuse.

A.6 KNOWLEDGE CHANGES ACROSS CONSECUTIVE CONSOLIDATIONS

  • Transitions are classified as unchanged, supplemented, updated, rewritten, or dropped; dropping from the next representation does not mean the underlying interactions were deleted.A transition can contribute multiple times across successive consolidations, so counts are not distinct items or independent samples.Source: §A.6 · p. 18
  • ToolBench use is an API-linked proxy, while interactive-environment reuse is detected through contiguous four-word overlap that may not reflect substantive use.ToolBench items mentioning no API are unclassified; in ALFWorld and ScienceWorld, only unchanged proportions are available, so the remaining outcomes are not divided among other transition types.Source: §A.6 · pp. 18–19
  • Supplementation occurs in 28.4% of matched-call transitions and 11.7% of differing-call transitions, versus 0.9% without detected use; concrete-value updates occur in 3.4%, 0.3%, and 0.1%, respectively.The comparisons group ToolBench transitions by API-linked use and whether the call matches the reference.Source: §A.6 · pp. 18–19, Table 10 · p. 19
  • Most ToolBench items without detected use remain unchanged (95.8%), so adjacent-consolidation statistics do not establish general fading of unused knowledge.The observed associations do not establish that use alone causes knowledge changes.Source: §A.6 · p. 19
  • In ALFWorld, referenced items remain unchanged less often than items without detected textual reuse (41.6% versus 63.2%), while the ScienceWorld difference is small (57.7% versus 59.1%).Under this textual proxy, the association is clearer in ALFWorld than in ScienceWorld.Source: §A.6 · p. 19, Table 11 · p. 19

A.7 CASE EVIDENCE FOR EVOLVING MEMORY

Case traces show reusable knowledge persisting as episode-specific details recede or as later outcomes revise earlier interpretations. Progressive recall opens nested detail until the current question is supported, leaving unrelated children unopened.

A.7 CASE EVIDENCE FOR EVOLVING MEMORY

  • Repeated prefix consolidation turns a concrete desklamp operation into generalized knowledge, while a remotecontrol episode’s destination eventually recedes from its narrative.In the same run, reusable knowledge persists as episode-specific details recede.Source: §A.7 · pp. 19–20, Figure 7 · p. 19
  • Later successful searches revise an earlier account of failed apple and potato searches to attribute the failures to search cost, with further outcomes then consolidated alongside it.The trace connects knowledge updates to continuing experience.Source: §A.7 · p. 20

A.8 PROGRESSIVE RECALL PROCEDURES

  • In two MemoryAgentBench cases, recall opens nested references as needed, then consolidates recalled evidence into the answering context and completes evidence-gathering intents before responding.One case stops after a single round; the other opens a second-level reference, while unrelated children remain unopened.Source: §A.8 · p. 20, Figure 8 · p. 20

A.9 THE COST OF MAINTAINING AND USING EXPERIENCE

This section examines how Hippocam organizes, retains, consolidates, recalls, and acts on experience through nested intents, purposeful memory formation, and edits using retained line anchors. It also describes StreamBench accuracy and cost comparisons, including workload-dependent variation in cumulative cost per correct answer.

A.9 THE COST OF MAINTAINING AND USING EXPERIENCE

  • On ToolBench’s first 150 tasks, Hippocam costs 0.0173 USD per correct answer versus A-Mem’s 0.0271, alongside 30.4k versus 52.7k uncached input tokens per task.Stable prefixes permit cache reuse, while prepending retrieved memories can invalidate reusable prefixes; shorter, variable DDXPlus prompts leave less shared input to reuse.Source: §A.9 · pp. 21–22
  • On ToolBench, Hippocam’s cumulative cost per correct answer is 0.0159 USD versus A-Mem’s 0.0260, while on DDXPlus it is 0.0097 versus 0.0091 USD.Across the first 150 tasks, Hippocam combines higher accuracy with lower cost than A-Mem and mem0 on ToolBench, but those methods remain cheaper on DDXPlus.Source: §A.9 · pp. 21–22, Figure 9 · p. 21, Table 13 · p. 22
  • Across both intervals, prefix-consolidation calls account for 34.0% of Hippocam’s ToolBench cost and 26.8% on DDXPlus; these shares are not estimates of net overhead.Savings depend on substantial cache discounts and recorded implementations: Hippocam’s lower ToolBench cost per correct answer than A-Mem holds when a cache hit costs less than approximately 15% of a miss, versus 2% in the reporting schedule.Source: §A.9 · p. 22

B CORE ALGORITHMS AND PROMPT SPECIFICATIONS

  • The algorithms describe context and archive operations, while prompts determine intent boundaries, retained content, and how agents use experience.Default thresholds are specified in Table 5.Source: B CORE ALGORITHMS AND PROMPT… · p. 23

B.1 EXECUTION AND MEMORY OPERATIONS

  • Consolidation replaces a contiguous context span with an archive-referenced message, preserving the replaced messages and their existing subtrees for later recall.Recall returns one archived layer as a tool result, after which the main agent may follow deeper references; prefix consolidation protects recent work and unfinished rounds.Source: §B.1 · p. 23

B.2 STRUCTURED PROMPTS AND INTERFACES

  • Auxiliary components observe the main agent’s conversation without participating in the task, and the prompts specify their roles rather than benchmark-specific instructions.The implementation calls PRECIP “hippocam” and PALIM “rippler.”Source: §B.2 · p. 24

B.2.1 JANUS: MAINTAINING THE INTENT STACK

  • JANUS maintains a nested LIFO hierarchy of purposes, distinguishing changes in purpose from changes in the actions used to pursue it.Each intention has a label, reason, and concrete completion condition; the stack can deepen when unexpected subproblems emerge.Source: §B.2.1 · p. 25

B.2.2 PRECIP: INTENT-CONDITIONED CONSOLIDATION

  • PRECIP forms a self-contained memory conditioned on the closing focus in a nested telos chain, retaining relevant impressions, state changes, conclusions, and supporting evidence.Unspent evidence that outer intents may need is preserved as-is.Source: §B.2.2 · p. 28
  • Intent-conditioned consolidation retains established state changes, explicit conclusions, and informative failures while excluding irrelevant material and unprocessed evidence.It can merge shared lessons across phases, and each conclusion in the replacement memory must carry its own evidence.Source: §B.2.2 · p. 29

B.2.3 PALIM: RECURSIVE PREFIX CONSOLIDATION

  • PALIM recursively coarsens undertaking-level narratives while preserving extracted knowledge with identifying details, relations, temporal status, and later revisions.Independently meaningful user-supplied records retain their values, conditions, relations, and identifiers entry by entry.Source: §B.2.3 · pp. 30–31

B.2.4 THE MAIN AGENT: LEARNING AND PROGRESSIVE RECALL

  • The agent recalls archived messages only when a consequential gap remains, follows deeper references as needed, and stops once it has enough information to proceed.If recall does not resolve the gap, it seeks other evidence or states uncertainty.Source: §B.2.4 · pp. 32–33
  • During work, the agent connects relevant prior knowledge to current conditions, treats unsupported deductions as hypotheses, and revises its understanding when observations contradict memory.When an approach changes, it states what will change and why, then uses the outcome to update understanding.Source: §B.2.4 · pp. 32–33
  • The agent can edit directly from retained line anchors; if a file change makes an anchor stale, the edit is rejected and the agent reads the stale line.Because a rejected edit changes nothing, confirming anchors before editing provides no information beyond the rejection itself.Source: §B.2.4 · p. 33
  • The main agent treats consolidated messages as its own prior work, linking retained knowledge to action, feedback, and further inspection.Source: §B.2.4 · p. 33
Loading 2610.12124v1…