PaperlayerView exact revision on arXiv

Source-linked AI summary

FlexA-Mem: Budget-Adaptive Agentic Memory for LLM Agents

Wujiang Xu, Jiaojiao Han, Xi Zhu, Qizheng Zhang, Kaiqu Liang, Yutong Yin, Kai Mei, Dimitris N. Metaxas

arXiv:2610.33274v1cs.CL

TL;DR

LLM agents need memory that preserves exact facts for conversation questions and task traces for multi-step work, but existing memories favor one need or fix their budget split in advance. FlexA-Mem stores history once and renders a timeline with budget-fitted detail at each call. It improves AppWorld results, matches keyword retrieval on fact recall, and its limitations include a narrow frozen-design test and histories that fit the model context.

  • Problem

    Existing agent memories are designed for either exact-fact recall or multi-step task traces, although the needed balance varies by scenario, task step, and call budget.

    Source: §1 · pp. 1–2

  • Method

    FlexA-Mem stores raw history units once and renders a timeline plus detail recovered by a fixed, budget-aware rule at each call.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 4

  • Results

    FlexA-Mem exceeds the second-best method by 11.5% on AppWorld challenge tasks at 4k and Summary+A-Mem by 12.2% on 147 unused tasks, while matching keyword retrieval on fact recall.

    Source: §1 · p. 2, §5.2 · p. 6, §5.3 · p. 6

  • Takeaways & Limitations

    The timeline carries AppWorld completion, while recovered text decides accuracy on exact facts, so one write-once store can serve both kinds of scenario.

    Source: §7 · p. 8

    Only the 147 unused AppWorld tasks test the final design with predictions fixed in advance; histories fit the model context, and all four benchmarks are in English.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language model (LLM) agents work in scenarios with very different memory needs. A question about past conversations needs the exact facts it names, while a multi-step task needs a trace of the steps already taken together with details that the next step cannot name. Existing memories are each designed and tested for one kind of scenario. Retrieval memories such as A-Mem miss steps that the current query does not express, summaries drop exact strings, and pairing the two fixes the split of the budget in advance. We propose FlexA-Mem, a general memory that writes each unit of history once into an immutable store and renders a fresh memory for every call. The rendered memory is a compact timeline of records plus detail that a fixed rule fits into the remaining room without an LLM call. The detail is either relevant units as raw text or excerpts or the verbatim pieces that best match the query. With one store, FlexA-Mem serves both kinds of scenario. On AppWorld challenge tasks at 4k, it outperforms the second-best method by 11.5%, and on 147 unused tasks with predictions fixed in advance it outperforms A-Mem with a summary by 12.2%. On LongMemEval and LoCoMo, it matches a dedicated keyword retriever that writes no memory, while memories built on text written by an LLM fall far behind at tight budgets. Ablations show that the timeline carries AppWorld completion, whereas recovery mainly saves steps there and decides accuracy on exact facts. Code is available at https://github.com/WujiangXu/Flex-Amem

1 Introduction

LLM agents need memory that balances exact facts with a trace of multi-step work, but existing designs favor one need and may commit to that balance before the call. FlexA-Mem keeps history in one store and renders it anew, with reported gains across agentic tasks and fact recall.

1 Introduction

  • A conversation question needs a few exact facts, while a multi-step task needs an overview of prior actions plus details that the next step may not name.Without the earlier login trace, an AppWorld agent may repeat steps; some calls need whole past units, while others need details across many units.Source: §1 · p. 1
  • Retrieval favors query-matched facts but can omit earlier task steps, summaries preserve trajectory overviews but lose exact strings, and a fixed budget split combines both trade-offs.Tool-managed hidden details are also rarely retrieved by agents.Source: §1 · p. 1
  • The needed balance changes by scenario, task step, and call budget, so a write-time commitment cannot fit every call; keyword matching alone also finds needed AppWorld details no better than chance.The decision must be made at each call while remaining cheap.Source: §1 · pp. 1–2
  • FlexA-Mem stores each history unit once and renders a fresh memory per call, preserving raw text while showing a query-independent timeline and filling remaining room with recovered detail by a fixed rule.The renderer requires no LLM call.Source: §1 · p. 2
  • FlexA-Mem beats the second-best method by 11.5% on AppWorld challenge tasks at 4k and by 12.2% on 147 unused tasks, while matching a dedicated keyword retriever on fact recall.Its ablations attribute AppWorld completion to the timeline, with recovery mainly saving steps there and deciding accuracy on exact facts.Source: §1 · p. 2

2 Related Work

Prior memory systems retrieve notes, maintain histories at multiple detail levels, or let agents manage context through tools or learned policies. FlexA-Mem combines known resolution and replacement ideas but makes the choices at read time with a fixed rule.

2 Related Work

  • Related memory work retrieves interaction notes, preserves histories at several detail levels, or delegates context management to tools or reinforcement learning, though agents seldom act on their own.The cited approaches include A-Mem, MemGPT, RAPTOR, MemTree, MemGAS, SCM, ReadAgent, AFM, and tool-based systems.Source: §2 · p. 2
  • FlexA-Mem combines budget-aware degradation from AFM with ReadAgent’s in-place gist replacement, but makes these choices at read time using a fixed rule without an LLM call or agent action.Unlike A-Mem it never rewrites notes; unlike AFM it retains a record of every unit and visits units by relevance rather than age.Source: §2 · p. 2

3 FlexA-Mem

FlexA-Mem stores raw history once and renders a timeline plus recovered detail on each call. Its recovery modes allocate the remaining budget at different granularities, while the mode is selected by task type rather than at run time.

3 FlexA-Mem

  • Each call renders a fresh memory from an immutable store, combining a timeline of past records with one of two recovery modes.The recent window contains newest units unchanged; agent units pack consecutive steps up to 1,000 tokens, while reader units split sessions at user turns into spans of about 2,000 tokens.Source: §3 · pp. 2–3, Figure 2 · p. 3

3.1 Writing Memory

  • When a unit leaves the recent window, FlexA-Mem stores its raw text unchanged and writes a record of up to 80 words with retrieval fields and an embedding.It also cuts raw text at line breaks into pieces while keeping each piece within a conversation turn or agent-step block.Source: §3.1 · p. 3

3.2 The Timeline of Records

  • The timeline keeps past records in time order; when it grows too long, up to four consecutive records are merged, oldest first, while the underlying raw units remain recoverable.On agentic tasks, the oldest records can also be hidden for one call when the newest unit exceeds a quarter of the budget.Source: §3.2 · p. 3

3.3 Unit Recovery

  • Unit recovery ranks query-related seeds and linked or adjacent units, then gives candidates the highest resolution that fits: raw text, a verbatim excerpt, or the record.Excerpt pieces are BM25-ranked against the query; more relevant units are not degraded to make room for less relevant ones.Source: §3.3 · p. 4

3.4 Piece Recovery

  • Piece recovery ranks pieces from the whole history by BM25, fits them into the room left after the timeline, and presents selected pieces grouped by unit in time order.A unit may contribute several pieces, while many units may contribute one each.Source: §3.4 · p. 4

3.5 Choosing a Recovery Mode

  • FlexA-Mem uses unit recovery for agents, whose next steps need context around details, and piece recovery for questions that name evidence in past conversations.The recovery mode is not selected at run time; only merged-record writing uses an LLM at read time.Source: §3.5 · p. 4

4 Experimental Setup

The evaluation covers agentic task completion, software issue resolution, and question answering over long conversational histories under token budgets. Baselines share key model and unit settings, with FlexA-Mem tuned on LongMemEval development questions and default values retained for agentic tasks.

4.1 Benchmarks

  • AppWorld evaluates API-based everyday tasks at 4k and 6k budgets, SWE-bench Verified evaluates Python issue resolution at 8k, and LongMemEval and LoCoMo test answers over long histories.AppWorld includes 168 Test-Normal and 417 Test-Challenge tasks; SWE-bench reports 480 gradable issues, and LongMemEval reports 440 held-out questions.Source: §4.1 · pp. 4–5

4.2 Baselines and Implementation

  • Methods share the units, embedding model, budget, token counter, and GPT-5.4 mini; baselines include retrieved notes, summary-plus-retrieval, LLM-controlled granularity, tool management, and retrieval without written memory.FlexA-Mem was tuned on LongMemEval development questions and used default values on agentic tasks; Claude Sonnet 4.6 and GPT-5.4 judge answers.Source: §4.2 · p. 5

5 Results

FlexA-Mem improves AppWorld task completion, including on unused tasks, while serving fact-recall workloads at performance comparable to keyword retrieval. It uses substantially fewer tokens than competing memory methods, though methods that write no memory can be cheaper.

5.1 Agentic Tasks

  • 11.5% at 4k and 7.7% at 6k: FlexA-Mem outperforms second-best ReadAgent on AppWorld Test-Challenge.At 6k, it matches full history at 62.4% versus 62.1%.Source: §5.1 · p. 5

5.2 Confirmation on Unused Tasks

  • 12.2% at 4k: FlexA-Mem outperforms Summary+A-Mem on 147 unused AppWorld tasks with predictions fixed in advance.The 6k lead misses after correction; three of five predictions hold.Source: §5.2 · p. 6, Table 3 · p. 6

5.3 Fact Recall

  • On LongMemEval, piece recovery stays within 0.7% of keyword retrieval at every budget; on LoCoMo, it leads piece retrieval by 1.0% to 2.6%.At 4k on LongMemEval, piece recovery outperforms A-Mem by 25.9%; at 32k, A-Mem leads 88.6% to 85.7%.Source: §5.3 · p. 6, Figure 3 · p. 6

5.4 Cost and Other Models

  • FlexA-Mem uses one seventh to one third of A-Mem and Summary+A-Mem’s tokens per task, while LongMemEval piece retrieval matches its accuracy using 8.0k rather than 176.6k tokens.On AppWorld, observation masking uses fewer tokens but completes 28.6% fewer tasks; piece retrieval writes no memory.Source: §5.4 · pp. 6–7

6 Analysis

AppWorld baselines often repeat setup, while the timeline preserves task progress and recovery supplies exact details. The two recovery modes lead in different settings, and tested runtime signals do not reliably select the better mode for an individual query.

6.1 How the Baselines Fail on AppWorld

  • At 4k on Test-Normal, 23.8% of FlexA-Mem episodes hit the step limit, versus 55.4% to 85.7% for all baselines except Summary+A-Mem.Of baseline failures, 90% are login and documentation-reading loops; increasing the limit to 150 steps does not improve completion for observation masking, truncation, or piece retrieval.Source: §6.1 · p. 7, Figure 5 · p. 8

6.2 The Timeline Keeps the Trace

  • Removing the timeline lowers AppWorld completion by 15.5% at 4k and 10.1% at 6k; on SWE-bench, resolve rate barely changes but steps rise from 43.8 to 56.9 per issue.On unused tasks, timeline alone stays within 2.0% of the full method; removing it changes LongMemEval accuracy by at most 3.6%.Source: §6.2 · p. 7

6.3 Recovery Brings Back the Detail

  • On LongMemEval, removing recovery lowers accuracy by 28.2% at 4k, while random candidates lower it by 47.5%.On unused AppWorld tasks, recovery reduces steps per episode from 32.2 to 26.9 at 4k; excerpts improve SWE-bench file-display balanced accuracy to 0.93 from 0.83 without excerpts, while observation masking reaches 0.96.Source: §6.3 · pp. 7–8, Table 7 · p. 7, Figure 6 · p. 8

6.4 When Each Recovery Mode Wins

  • Unit recovery leads piece recovery by 4.8% to 10.1% across five agentic settings, while piece recovery leads on LongMemEval up to 16k and on LoCoMo at every budget.BM25 evidence rank is associated with the mode gap, but tested runtime signals do not reliably predict the better mode for a single query.Source: §6.4 · p. 8, Figure 4 · p. 8

7 Conclusion

FlexA-Mem uses one write-once store and renders a timeline plus recovered detail at each call, serving both agentic tasks and fact recall.

7 Conclusion

  • A write-once store rendered anew as a timeline plus recovered detail serves both agentic tasks and fact recall.The conclusion describes a fixed-rule renderer for each call.Source: §7 · p. 8
  • On agentic tasks, the timeline carries completion and comes within 2.0% of the full method on tasks unused in design.Source: §7 · p. 8
  • On fact recall, recovered text decides accuracy, while keyword retrieval over the kept raw text is already strong.Source: §7 · p. 8
  • Choosing the recovery mode for each query without labels remains a next step.Source: §7 · p. 8

Limitations

The final FlexA-Mem was designed after the main experiments, and its frozen predictions were tested in advance only on 147 unused AppWorld tasks.

Limitations

  • Only 147 unused AppWorld tasks test the final method with predictions fixed in advance.Source: Limitations · p. 9
  • Most configurations ran once, with up to 4.8% between-run variation on AppWorld.Source: Limitations · p. 9
  • Recovery mode is set per task type rather than per query.Source: Limitations · p. 9
  • Every evaluated history fits in the model context, where the full history costs fewer tokens, and all four benchmarks are in English.Source: Limitations · p. 9

A Implementation Details

Implementation details specify the prompt context, recent-window and query construction, merged-record handling, and piece-recovery ranking and rendering.

A Implementation Details

  • Implementation settings count tokens with o200k; agents and readers use medium reasoning, memory calls use low effort, and embeddings use all-MiniLM-L6-v2.Source: Table 8 · p. 14, A Implementation Details · p. 14
  • On agentic tasks, the context combines the system prompt, task, memory, and recent original messages; the recent window is the longest newest-unit run within a quarter of budget and includes the newest unit.Source: A Implementation Details · p. 14
  • The agentic query embedding sums the task and newest-unit embeddings, while excerpt pieces are ranked against the newest unit alone.Source: A Implementation Details · p. 14
  • Merged agentic records persist for an episode, use the larger of current and reference allowances, and hide oldest records for a call when the newest unit makes the current allowance smaller.Merged-record prompts request at most 120 words, and outputs longer than 150 are cut.Source: A Implementation Details · p. 14
  • BM25 uses lower-cased content words without stop words or markup; ties use cosine similarity and then earlier-piece order, and pieces stay within a turn or output block.Source: A Implementation Details · p. 14
  • Piece recovery places the timeline first, lends unused timeline allowance to pieces, groups pieces by unit in time order, and rerenders with fewer pieces if over budget.Source: A Implementation Details · p. 14

B Experiment Details

The experiments compare memory methods across agentic and conversational benchmarks, then probe budget sensitivity, recovery choices, model variation, and token costs. Results favor FlexA-Mem on harder AppWorld tasks and show distinct strengths for its recovery modes.

B Experiment Details

  • The evaluation spans AppWorld, SWE-bench Verified, LongMemEval, and LoCoMo, with benchmark-specific budgets and controls.Baseline coverage varies by benchmark; some methods were evaluated only on SWE-bench’s 95-issue subset, and AFM was not run there.Source: B Experiment Details · p. 15

D Additional Related Work

  • Earlier memory designs either commit to a granularity, rely on agent-issued recovery, or remove raw text; FlexA-Mem retains every unit and recovers it by rule.Prior work also motivates retaining raw episodes as evidence rather than repeatedly rewriting them.Source: D Additional Related Work · p. 17

E.1 Confirmation Fixed in Advance

  • On 147 unused AppWorld tasks, the code, six arms, two budgets, and five predictions were frozen before evaluation; adjusted p-values were 0.006, 0.061, 0.345, 0.0005, and 0.024.The predictions tested leads over Summary+A-Mem and timeline-only memory, plus non-inferiority to piece recovery.Source: §E.1 · pp. 17–18

E.2 Results by Budget

  • 11.5%: FlexA-Mem beats the second-best method on AppWorld Test-Challenge at 4k, with its lead shrinking to 2.4% on Test-Normal.Figure 7 plots LongMemEval accuracy and AppWorld completion by budget.Source: §E.2 · p. 18, Figure 7 · p. 19

E.3 AFM and LoCoMo

  • FlexA-Mem leads both tested AFM threshold settings on AppWorld and LoCoMo, including by 7.7%, 4.5%, and 11.6% on LoCoMo without the calibration conversation.AFM calibration raises its LoCoMo scores but lowers them on AppWorld; the selected thresholds differ by benchmark.Source: §E.3 · p. 18

E.4 Recovery Modes in Detail

  • At 4k, piece recovery leads on most LongMemEval question types, while unit recovery leads on questions about assistant statements, preferences, and abstention.The modes cost within 0.2k tokens per question; piece recovery alone exposes part of the evidence on 106 questions and wins 54 of 56 decided there.Source: §E.4 · pp. 18–19, Table 12 · p. 19

E.5 Other Agents and Readers

  • FlexA-Mem’s AppWorld lead persists with Gemini 3.1 Pro and Claude Haiku 4.5, and it is the most accurate memory-writing method on LongMemEval at 8k across three readers and both judges.On AppWorld, it beats observation masking by 6.5% with Gemini and 8.9% with Haiku.Source: §E.5 · p. 19

E.6 Token Costs

  • Token-cost reporting includes memory construction and covers AppWorld at 4k plus SWE-bench and LongMemEval at 8k, while methods without cost measurements are omitted from the figure.LongMemEval is plotted using unit-recovery cost, within 0.2k tokens of piece-recovery cost.Source: §E.6 · p. 20, Table 14 · p. 20

E.7 Robustness

Additional runs test whether FlexA-Mem’s performance persists across budgets, settings, and model choices, and inspect what its timeline and recovery expose. The results also illustrate where budget limits and benchmark conditions constrain performance.

E.7 Robustness

  • 7.7%: At 8k on AppWorld, FlexA-Mem beats A-Mem, while on 385 SWE-bench issues it beats observation masking by 1.8% and Summary+A-Mem by 3.1%.A second AppWorld run changed 15% to 17% of FlexA-Mem task outcomes but left every gap to rerun baselines significant.Source: §E.7 · p. 20

E.8 What the Timeline Needs

  • At 4k on AppWorld Test-Normal, a fixed-rule trace completes 61.3% versus 67.3% for GPT-5.4 mini-written records, a nonsignificant gap; at 6k, the 13.7% gap is significant.Recovery closes most of the weaker-writer gap: Haiku-written memory reaches 62.5%, 1.8% below GPT-5.4 mini-written memory.Source: §E.8 · p. 20

E.9 What the Memory Recalls on SWE-bench

  • On SWE-bench file-display questions, FlexA-Mem reaches 0.93 balanced accuracy versus 0.83 without excerpts, 0.73 for Summary+A-Mem, and 0.61 for A-Mem.In 39.2% of re-runs where the relevant raw unit was a candidate, that unit exceeded available room; in about 70% of those cases, even an excerpt would not fit.Source: §E.9 · p. 20

E.10 How the Memory Uses the Budget

  • At 8k, unit recovery shows 77.4% of held-out LongMemEval evidence turns verbatim versus 71.8% without excerpts; excerpts use 7.4% of tokens at 8k and 2.4% at 32k.As budgets grow, raw text rises from 68.2% to 73.5% of memory tokens while records remain near a quarter.Source: §E.10 · p. 21

E.11 Excerpt Variants and Plug-ins

  • Fit-or-degrade improves A-Mem and memory evolution at 4k by 13.4% and 13.9%, respectively, and at 8k by 5.7% and 3.7%.Shorter excerpts help at 4k and 8k, while giving raw text to every candidate first loses most of the gain at 4k.Source: §E.11 · p. 21, Table 15 · p. 21

E.12 A Case Study

  • In the AppWorld case study, five records and two excerpts preserve login-token details at step 16, and the agent submits at step 22 without logging in again.On this task, truncation, A-Mem, and piece retrieval reached the 60-step limit after reading API documentation 74 to 109 times.Source: §E.12 · p. 21

E.13 Numbers Reported by Other Papers

  • Reported results from other papers use different backbones, judges, splits, harnesses, and budgets than this study’s results.The cited no-budget reference agents and reader score 88.4% on LongMemEval, 67.9% on SWE-bench, and 78.6% and 62.1% on the two AppWorld test sets.Source: §E.13 · p. 21

F.1.1 Agentic Tasks

FlexA-Mem performs strongly on both AppWorld test sets and SWE-bench Verified, with its largest AppWorld advantage on the harder challenge tasks. Its SWE-bench advantage is smaller, and full-history performance remains a useful reference.

F.1.1 Agentic Tasks

  • 40.3%: FlexA-Mem completes AppWorld Test-Challenge tasks at 4k, beating ReadAgent by 11.5%; at 6k, it reaches 62.4%, near the full-history agent’s 62.1%.The 4k and 6k Test-Challenge leads are significant; the 6k difference from full history is not significant.Source: §F.1.1 · p. 21
  • On AppWorld Test-Normal, FlexA-Mem leads ReadAgent by 2.4% at 4k and 6.0% at 6k, neither significant; its 8.9% lead over AFM at 4k is significant.At 6k it comes within 0.6% of full history.Source: §F.1.1 · p. 21
  • 66.5%: On all 480 SWE-bench Verified issues, FlexA-Mem resolves more than any other method evaluated on the full set, beating observation masking by 2.3% and Summary+A-Mem by 4.0%.On 385 issues not used to choose settings, the leads are 1.8% over observation masking and 3.1% over Summary+A-Mem.Source: §F.1.1 · p. 21

F.1.2 Confirmation on Unused Tasks

FlexA-Mem’s results persist on unused AppWorld tasks, while its timeline and recovery modes contribute differently across agentic tasks and exact-fact recall. The comparisons also show cost and agent-dependent boundaries, and that the best recovery mode varies with the scenario and how explicitly a question names its evidence.

F.1.2 Confirmation on Unused Tasks

  • 76.9% of unused AppWorld tasks are completed at 4k, beating Summary+A-Mem by 12.2% (Holm-adjusted p = 0.006).The code was frozen before the first episode; differences from full history were not significant.Source: §F.1.2 · p. 22

F.1.3 Fact Recall

  • At 4k, piece recovery answers 78.0% of held-out LongMemEval questions and beats A-Mem by 25.9% and Summary+A-Mem by 27.3%.On LoCoMo, unit recovery beats every LLM-built baseline memory by 6.2%–9.4% over A-Mem and Summary+A-Mem and by 5.2%–13.1% over AFM with calibrated thresholds.Source: §F.1.3 · pp. 22–23

F.1.4 Cost and Other Models

  • FlexA-Mem uses about one seventh to one third of A-Mem and Summary+A-Mem’s tokens per task, but full history costs less.With Gemini 3.1 Pro, it completes 94.0% on Test-Normal at 4k versus 94.6% for full history; on SWE-bench Verified with Haiku, it resolves 61.1% versus 63.2% for observation masking.Source: §F.1.4 · p. 23

F.2.1 How the Baselines Fail on AppWorld

  • At 4k on Test-Normal, 90% of baseline failures are loops repeating setup, while FlexA-Mem reaches the step limit in 23.8% of episodes.Increasing the limit from 60 to 150 steps raises FlexA-Mem completion from 64.3% to 80.4%, but does not improve completion for observation masking, truncation, or piece retrieval.Source: §F.2.1 · p. 23

F.2.2 The Timeline Keeps the Trace

  • Removing the timeline cuts AppWorld completion by 15.5% at 4k and 10.1% at 6k, but changes LongMemEval accuracy by at most 3.6%.On unused tasks, the timeline alone completes 76.2% and 79.6%, within 2.0% of the full method.Source: §F.2.2 · pp. 23–24

F.2.3 Recovery Brings Back the Detail

  • On LongMemEval, removing recovery lowers accuracy by 28.2% at 4k, whereas on AppWorld recovery changes effort more than completion.On unused tasks, unit recovery takes 26.9 versus 32.2 steps per episode at 4k; fit-or-degrade rendering of A-Mem notes raises Test-Normal completion by 10.7% at 4k.Source: §F.2.3 · p. 24

F.2.4 When Each Recovery Mode Wins

  • Unit recovery leads in all five agentic settings by 4.8%–10.1%, while piece recovery leads on LongMemEval by 8.4%, 6.6%, and 3.6% from 4k to 16k.The mode gap is associated with evidence rank: piece recovery leads by 17.6% when the evidence ranks first, while unit recovery leads by 26.5% when it ranks below twentieth; runtime signals did not reliably predict the better mode for a query.Source: §F.2.4 · p. 24

Related papers

Loading 2610.33274v1…