PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Efficient Agents: Memory, Tool learning, and Planning

Xiaofang Yang, Lijun Li, Heng Zhou, Tong Zhu, Xiaoye Qu, Yuchen Fan, Qianshan Wei, Rui Ye, Li Kang, Yiran Qin, Daizong Liu, Qi Li, Ning Ding, Siheng Chen, Jing Shao

arXiv:2601.14192v2cs.AIcs.CL

TL;DR

LLM-based agents improve capabilities but introduce substantial costs through long contexts, multi-step interactions, tool calls, and retries. This survey synthesizes efficiency methods for memory, tool use, and planning, along with evaluation practices and challenges. It finds convergence around compression, reusable structures, controlled interaction, and shifting computation from online search toward offline learning, while emphasizing cost–performance trade-offs and standardized reporting.

  • Problem

    Agent effectiveness has improved, but efficiency remains underexplored despite the latency, token, step, and compute costs required for real-world deployment.

  • Method

    The survey organizes research on memory, tool use, and planning, synthesizes methods and benchmarks, and examines efficiency through cost–performance trade-offs.

  • Results

    The reviewed methods converge on bounding context, reducing unnecessary tool calls, reusing plans and memory, and controlling end-to-end reasoning and interaction costs.

  • Takeaways & Limitations

    Efficient agents should be evaluated by total trajectory cost and reported with standardized, transparent metrics to support fair comparison and reproducibility.

  • Takeaways & Limitations

    Efficiency in multimodal LLM-based agents remains relatively under-explored, particularly under realistic latency and compute constraints.

Abstract

from arXiv · show

Recent years have witnessed increasing interest in extending large language models into agentic systems. While the effectiveness of agents has continued to improve, efficiency, which is crucial for real-world deployment, has often been overlooked. This paper therefore investigates efficiency from three core components of agents: memory, tool learning, and planning, considering costs such as latency, tokens, steps, etc. Aimed at conducting comprehensive research addressing the efficiency of the agentic system itself, we review a broad range of recent approaches that differ in implementation yet frequently converge on shared high-level principles including but not limited to bounding context via compression and management, designing reinforcement learning rewards to minimize tool invocation, and employing controlled search mechanisms to enhance efficiency, which we discuss in detail. Accordingly, we characterize efficiency in two complementary ways: comparing effectiveness under a fixed cost budget, and comparing cost at a comparable level of effectiveness. This trade-off can also be viewed through the Pareto frontier between effectiveness and cost. From this perspective, we also examine efficiency oriented benchmarks by summarizing evaluation protocols for these components and consolidating commonly reported efficiency metrics from both benchmark and methodological studies. Moreover, we discuss the key challenges and future directions, with the goal of providing promising insights.

1. Introduction

Agentic systems extend LLMs into multi-step workflows that require memory, planning, and tool use, creating substantial efficiency costs. This survey frames efficiency as a system-level objective and organizes research around these components.

  • Motivation: Multi-step agent execution causes compounding token use, context saturation, latency, and computational cost through repeated memory, planning, tool-use, and observation cycles.The output of one step becomes part of the next step’s input cost.
  • Definition: An efficient agent maximizes task success while minimizing token usage, inference latency, and computational cost across memory, tool usage, and planning.
  • Research gap: Existing component-specific surveys do not directly answer the system-level question of how to optimize agent efficiency across the workflow.
  • Scope: The survey categorizes efficiency research into memory compression and retrieval, tool-use reduction and decision-making, and planning strategies that shorten trajectories and API calls.
  • Scope: Memory, tool use, and planning are analytically distinct but operationally coupled because each component can store, trigger, or reshape the others.
  • Organization: The survey proceeds from preliminaries through component-level optimizations and efficiency quantification before discussing open challenges and future directions.

2. Preliminaries

The paper models agents as partially observable systems augmented with external tools and explicit memory, then defines efficiency through the cost–performance trade-off. Agent costs extend beyond generation to tool, memory, and retry overhead.

  • Agent formulation: The agent formulation augments a partially observable Markov decision process with an external tool interface and an explicit memory component.
  • Agent formulation: The model includes latent states, observations, actions, transition dynamics, rewards, discounting, tools, tool execution, memory updates, and initial memory.
  • Efficiency definition: Efficiency means achieving comparable performance at lower cost or higher performance under a similar cost budget.
  • Agent-specific efficiency: Agent efficiency differs from standalone LLM efficiency because agents plan and act over multiple steps, invoke external tools, and condition decisions on memory.
  • Cost model: Agent cost combines generated-token cost with tool, memory, and retry costs, making selective invocation and reduced retries essential efficiency targets.The formulation uses indicator variables for whether tools, memory, or retries occur.

3. Efficient Memory

Efficient memory addresses long-context and long-horizon overhead by converting experience into reusable representations while controlling storage, retrieval, and update costs. The survey organizes memory efficiency around construction, management, and access, highlighting a trade-off between compactness, interpretability, capacity, and write cost.

  • Motivation: Memory reduces repeated processing by storing and reusing past successes, failures, and interaction traces.This can reduce redundant computation and costly retries during long-horizon agent execution.
  • Organization: The taxonomy organizes memory efficiency through construction, management, and access rather than mutually exclusive individual operations.The lifecycle perspective reflects that updating and forgetting may be jointly decided within one pipeline.
  • Memory Construction: Textual memory stores readable records, whereas latent and parametric memory stores experience in implicit representations or model parameters.Latent memory includes hidden states and KV caches; parametric memory uses dedicated memory-module parameters.
  • Memory Construction: Compact representations reduce token growth and repeated context processing, but excessive compression can omit details needed for correct decisions.Latent and parametric approaches avoid serializing full histories, while textual memory remains easier to inspect, edit, retrieve, and cite.
  • Overview: The survey’s memory overview tables consolidate efficiency-oriented mechanisms across latent, parametric, textual, procedural, and multi-agent memory.Table 1 covers latent and parametric memory and textual memory, while Table 2 covers procedural reuse via skills and multi-agent memory.
  • Memory Construction: Selective memory writes avoid redundantly re-encoding stable or low-information content across latent and parametric memory.Latent updates are fast and gradient-free but capacity-bounded; parametric updates can encode broader associations at the cost of a forward-backward or equivalent weight-update step.

Latent and Parametric Memory

Memory representations trade off efficiency against transparency and capacity. Compact prompt, item, graph, and hierarchical designs reduce context or retrieval burdens, while structured textual forms preserve interpretability but can incur token and maintenance costs.

  • Latent and Parametric Memory: Latent and parametric memory avoid serializing and rereading full histories, but latent capacity is bounded and parametric writes require update computation.Latent memory stores experience in activation space, whereas parametric memory stores it in dedicated module parameters.
  • Textual Memory: Textual memory remains interpretable but becomes inefficient when stored text is too long, redundant, or poorly indexed.MemoryBank and Expel illustrate token-cost limitations from accumulating or summarizing experiences in natural language.
  • Prompt-resident Memory: Prompt-resident memory rewrites a compact textual state to keep effective context length below the full interaction history.MemAgent and MEM1 sequentially update compact memory states as long inputs are processed.
  • Prompt-resident and Item-based Memory: Compact prompt memory reduces attention cost and improves long-context performance, while item-based memory can retain discrete experiences for retrieval and reuse.Item-based systems may store events, insights, profiles, or plan templates, but full trajectories can create long contexts and inefficient retrieval.
  • Item-based Memory: Memory extraction, compression, summarization, and indexing shorten entries and make stored experiences easier to retrieve and reuse.Structured indices attach attributes or abstractions that reduce retrieval ambiguity and support faster lookup.
  • Graph-based Memory: Graph-based memory merges repeated entity content into nodes and retains relevant relations as edges, producing compact structures for later retrieval.This organization avoids unbounded prompt growth and supports fast access to multi-hop evidence.
  • Hierarchical Memory: Hierarchical memory enables coarse-to-fine, on-demand access through storage tiers or multi-granularity summaries.System-oriented designs separate working context from external recall and archival memory, while content-oriented designs index pages, subgoals, observations, or summaries.

Textual Memory

Textual memory keeps agent experience in readable records but requires lifecycle management to prevent redundancy, staleness, retrieval latency, and wasted prompt capacity. Management strategies trade inexpensive rule-based control against semantic adaptivity and model-call overhead.

  • Textual Memory: Textual memory can be human-readable, editable, and inspectable while keeping historical content outside the active prompt until retrieval.Retrieval and reinsertion still incur costs, and poorly structured stores may return noisy, redundant, or overly long text.
  • Memory Management: Appending every new observation makes memory redundant, stale, expensive to retrieve, and potentially harmful to downstream decisions.Memory management therefore includes updating, merging, deleting, consolidating, and archiving.
  • Management Strategies: Rule-based management uses recency, frequency, capacity, or feedback signals to maintain memory without extra LLM calls.These policies bound memory size and retrieval cost but may discard semantically important infrequent information.

Rule-based Management

Rule-based management minimizes memory-maintenance overhead through predefined policies, while LLM-based and hybrid approaches add semantic judgment when simple signals are insufficient. The resulting design space balances predictable cost against task-aware memory quality.

  • Rule-based Management: Rule-based management is fast, predictable, and inexpensive because updates, pruning, and merging follow predefined rules.It works when importance is approximated by recency, frequency, capacity, or explicit feedback.
  • Rule-based Management: Static rules may remove rare critical memories or retain frequent low-value ones because they use task-agnostic proxies for usefulness.This is the central semantic limitation of minimizing management overhead with fixed policies.
  • LLM-based Management: LLM-based management makes semantic decisions about memory importance, consistency, redundancy, and operations such as ADD, DELETE, UPDATE, or NOOP.Memory-R1 learns operation choices with reinforcement learning, while Mem0 selects operations after vector-based retrieval.
  • LLM-based Management: Generative management lets the model produce memory updates directly instead of selecting from a fixed action set.A-MEM retrieves similar notes and has an LLM create links and rewrite related notes.
  • Hybrid Management: Hybrid management reserves costly LLM judgment for semantic cases while using rules for frequent operations such as overflow, deduplication, and tier transfer.MemoryOS and LightMem apply trigger-driven updates in hierarchical memory systems.
  • Hybrid Management: Hybrid systems also combine heuristic pruning with semantic keep-or-drop decisions and lifecycle policies for consolidation, deduplication, and archiving.Agent KB and ACE exemplify item-level selection and pruning, while MemOS tracks lifecycle and versions.
  • Hybrid Management: Graph-memory maintenance uses rule-based graph updates alongside LLM checks for semantic conflicts or stale relations.Zep, Mem0g, and AriGraph follow this pattern for relation invalidation, removal, and insertion.

Hybrid Management

Efficient memory management selects and integrates only task-relevant information while balancing retrieval cost, context length, recall, and generation quality. Approaches range from rule-enhanced, graph-based, hierarchical, and learned retrieval to textual and latent integration.

  • Memory Access: Memory access must balance missing critical information against retrieving excessive context that recreates context bloat.The trade-off directly affects retrieval latency, inserted-token cost, and downstream generation quality.
  • Retrieval Strategies: Rule-enhanced, graph-based, hierarchical, and learned retrieval methods reduce search or improve utility through task-sensitive selection.Examples include recency and importance scoring, local graph expansion, coarse-to-fine search, and reinforcement-learned reranking.
  • Retrieval Strategies: LLM- or tool-based retrieval can improve difficult-query recall and reasoning accuracy, but usually costs more than direct vector or graph retrieval.MemGPT exposes memory tiers as tools and enforces token budgets during selection.
  • Textual Integration: Textual integration compresses retrieved histories into focused snippets, compact memory blocks, working sets, strategy bullets, or adapted plan templates.These methods retain actionable signals while limiting prompt expansion and irrelevant context.
  • Latent Integration: Latent integration stores long-term information in hidden states or key–value representations, avoiding repeated text encoding and slowing cost growth for long-horizon experience.MemoryLLM uses trainable memory tokens, while related systems maintain compact latent memory with roughly constant GPU KV-cache requirements.

3.4. Procedural Reuse via Skills

Procedural skills shift computation from repeated online reasoning into reusable memory for recurring tasks. Their efficiency is amortized: construction and management costs must be justified by reuse across future executions.

  • Skill Lifecycle: Skills encode how agents should act in recurring situations rather than recording only what happened.They are constructed from experience, selectively accessed, maintained, and verified before reuse.
  • Benefits of Reuse: Reusable skills can reduce repeated planning, redundant tool invocation, and trial-and-error interactions across similar tasks.The savings arise when procedural knowledge is reused rather than recomputed online.
  • Skill Management: Skill libraries require construction, retrieval, verification, curation, updating, and pruning, creating operational overhead beyond initial creation.Recent systems study both validation of generated skills and their evolution across interactions.
  • Skill Access: Skill access must avoid loading an entire growing library by selecting or adapting only relevant procedural components.SkillLens uses adaptive multi-granularity reuse to limit token cost, retrieval latency, and distraction.
  • Amortized Efficiency: Skill-based efficiency is beneficial when reuse across many tasks, users, or domains amortizes construction and maintenance costs.The central design goal is a compact, reliable, selectively accessible skill layer.

3.5. Multi-Agent Memory

Multi-agent memory must balance shared knowledge reuse with role-specific context, communication cost, consistency, and synchronization. Shared, local, and mixed designs make different trade-offs between coordination and specialization.

  • Design Trade-offs: Multi-agent memory changes the design question from individual remembering to deciding where reusable information should live and what should be shared.This introduces a team-level memory lifecycle alongside individual construction, maintenance, and access.
  • Design Trade-offs: Duplicating full traces across agents raises context and communication costs, whereas overly centralized memory can create synchronization overhead, stale state, and noisy context.Efficient designs must balance coordination, specialization, consistency, and knowledge reuse.
  • Shared Memory: Shared memory reduces duplicated exploration and context replay by making reusable observations, decisions, and intermediate results available across agents.Structured graph memories retrieve generalizable insights alongside condensed agent-specific trajectories.
  • Shared Memory: Latent shared memory exchanges compact internal states or KV caches, reducing token-level replay and information overload between agents.Systems use shared latent working memories, neural fusion, gating, or role-aware latent composition.
  • Local Memory: Local memory lowers retrieval noise and synchronization cost but can isolate discoveries, causing other agents to repeat work or lack relevant evidence.It best fits role-specialized agents, private user state, and subtasks where local context is more valuable than global synchronization.
  • Mixed Memory: Mixed memory combines private and shared stores, requiring routing, tier selection, redundancy control, and conflict resolution.This flexible organization supports both agent-specific efficiency and team-level coordination but adds synchronization complexity.

3.6. Discussion

The discussion frames efficient memory as lifecycle-level budget allocation rather than storage expansion alone. Compression, updating, retrieval, and evaluation must jointly preserve decision-relevant information while measuring effectiveness against end-to-end cost.

  • Core Principle: Efficient memory depends on deciding when experience should be compressed, maintained, retrieved, or reused, not simply adding storage.The same cost-versus-future-utility question recurs across memory operations.
  • Lifecycle Budgets: Lifecycle optimization must account for write, between-interaction, and inference-time costs because improving one stage can shift burden elsewhere.Over-compression may require repair, weak maintenance can leave noisy storage, and high-recall retrieval can recreate long-context cost.
  • Compression Trade-offs: Excessive compression can degrade accuracy by removing critical information, while milder compression preserves performance at higher cost.Task-aware objectives should retain causal facts, constraints, failures, and reusable procedures.
  • Update Policies: Online memory updates adapt immediately but increase latency and cost, whereas offline updates reduce inference overhead but adapt more slowly.Selective updating can reserve expensive semantic consolidation for memory changes likely to affect future decisions.
  • Evaluation: Memory evaluation should report marginal task success per token, retrieval, update, or latency alongside lifecycle costs.Relevant measures include write cost, update frequency, retrieval latency, inserted-token budget, and failure-recovery savings.
  • Cross-Component Interaction: Memory can improve tool use and planning by caching successful plans, constraints, outputs, subgoals, search outcomes, and failed branches.This can reduce redundant selection, parameter filling, calls, and deliberation when stored experience changes future behavior.

4. Efficient Tool Use

Efficient tool use addresses the costs of selecting, invoking, and integrating tools across complex agent trajectories. The survey organizes methods around reducing unnecessary interactions, execution overhead, and reasoning cost under latency, token, and monetary constraints.

  • Motivation: Complex tasks can require hundreds of tool calls, creating long trajectories that challenge context comprehension and impose substantial costs.One example reports an LLM calling a search API 600 times for a deep-research problem.
  • Motivation: Tool-use efficiency includes both solving complex problems with shorter trajectories and reducing the number of tool calls themselves.The survey distinguishes efficient reasoning through tools from optimizing tool use to call fewer tools.
  • Tool Selection: Tool selection methods narrow large tool pools through external retrieval, multi-label classification, or vocabulary-based retrieval before generation.These approaches differ in whether selection uses an independent retriever, direct classification, or special vocabulary tokens.
  • Tool Selection: External retrievers improve selection through redesigned retrieval, stronger retrievers or rerankers, and refined tool documentation, while hierarchical and progressive designs narrow search spaces.Examples include ProTIP’s progressive selection, AnyTool’s divide-and-conquer retrieval, and tool-side document refinement in DRAFT.
  • Tool Selection: Tool selection trades efficiency against coverage: aggressive filtering lowers prompt and planning costs but can remove necessary tools, whereas broader selection preserves coverage at higher cost.External retrievers suit dynamic or large collections, while classification and vocabulary-based methods suit relatively fixed inventories requiring low-latency selection.
  • Tool-Integrated Reasoning: Tool-integrated optimization teaches agents when and how to invoke tools and avoid redundant trajectories, but shifts costs to data, feedback, reward design, and rollout optimization.Poor objectives may produce tool overuse, underuse, format overfitting, or shorter but less reliable trajectories.
  • Discussion: Efficient tool use optimizes which external actions are worth their cost rather than minimizing tool count universally.Selection reduces the action space, calling optimizes execution trajectories, and integrated reasoning controls when external evidence or computation enters reasoning.
  • Discussion: Tool efficiency depends on memory reuse and planning depth because stored templates and failures can reduce future costs while planning determines whether tools or direct reasoning are appropriate.The survey argues for jointly optimizing tool choice, memory reuse, and planning depth instead of treating calls as isolated actions.

5. Efficient Planning

Efficient planning treats reasoning as budget-aware allocation across inference depth, search breadth, memory reuse, tool use, and coordination. The surveyed methods divide into single-agent strategies and learning-based evolution, alongside multi-agent approaches that control collaboration overhead.

  • Efficient Planning: Planning allocates limited reasoning resources across tokens, latency, tool calls, environment interactions, and communication overhead.Its objective balances the benefit of further deliberation against computational and interaction costs.
  • Single-Agent Planning Efficiency: Single-agent methods use adaptive control, structured search, decomposition, or learning-based evolution to reduce the cost of reaching valid solutions.Inference-time strategies optimize planning on the fly, while learning-based evolution improves intrinsic capabilities.
  • Learning-Based Evolution: Reusable skills and structured memory amortize planning costs by allowing agents to avoid re-planning and retrieve information from long contexts efficiently.VOYAGER builds a reusable skill library, while graph-based methods support structured retrieval and GAP identifies parallelizable actions.
  • Discussion: Efficient planning reframes reasoning as budget-aware control: online inference allocates current computation, while learning, reusable memory, and structured retrieval shift cost offline.In multi-agent systems, efficiency additionally depends on controlling communication topology and protocol overhead.
  • Discussion: Three recurring axes are depth control, breadth control, and amortization, which complement one another but compete for the same budget.Stronger memory or skill reuse can reduce both search depth and communication breadth, while deeper search may reduce the need for multi-agent debate.
  • Discussion: Efficient planners need progress signals to estimate the marginal value of continued reasoning instead of relying on fixed steps, agent counts, or debate rounds.Additional deliberation can have diminishing returns, while extra agents may create redundant messages and longer consensus processes.
  • Discussion: Planning controls cross-component cost allocation by deciding when to retrieve or update memory and when to call tools.An efficient planner may spend more tokens upfront when this avoids repeated retrieval, unnecessary tool calls, or failed environment interactions.

6. Benchmarks

The survey organizes agent evaluation around effectiveness benchmarks and cost measurements, emphasizing that efficiency claims require preserved task quality. It frames efficiency as achieving comparable performance at lower cost or higher performance under a similar budget.

  • Evaluation framework: Efficiency evaluation first verifies task effectiveness, then compares cost at comparable effectiveness or effectiveness under a fixed cost budget.This cost–performance trade-off can also be represented as a Pareto frontier.
  • Effectiveness benchmarks: Holistic benchmarks measure end-to-end agent trajectories, while downstream benchmarks measure final-output quality in task settings.Examples include GAIA, SWE-Bench, WebArena, WebShop, HotpotQA, and Natural Questions.
  • Effectiveness benchmarks: Component-specific benchmarks diagnose memory, tool-use, and planning abilities rather than only measuring overall task completion.Memory benchmarks test retention and memory-grounded question answering; tool benchmarks test selection, parameters, and workflows; planning benchmarks test valid action sequences and long-horizon recovery.
  • Effectiveness benchmarks: Protocol-level benchmarks evaluate standardized tool interactions, including tool selection, execution, parallelism, and avoidance of redundant calls.MCP-RADAR and MCP-Bench evaluate these dimensions under the Model Context Protocol.
  • Efficiency measurements: Efficiency measurements group reported costs into token and monetary cost, time and runtime overhead, resource cost, and interaction cost.Benchmarks and methods report signals such as environment steps, token use, dollar cost, latency, retrieval time, and search latency.
  • Joint evaluation: Effectiveness and efficiency should be reported jointly because holistic, downstream, and component-specific benchmarks explain capability while cost metrics show the resources required.Reported resources include tokens, time, hardware cost, and interactions.

7. Challenges and Future Directions

The survey identifies missing unified efficiency evaluation and underexplored deployment settings, then proposes common reporting foundations and several directions for agent-specific reasoning and deployment-aware design.

  • Unified evaluation: Unified efficiency evaluation remains missing because studies use inconsistent metrics and unclear boundaries for runtime, latency, memory, tool-use, and planning overhead.The survey calls for common stage definitions, metric granularity, and reporting protocols.
  • Unified evaluation: Common stage definitions, metric granularity, and reporting protocols would improve comparability and clarify cost–performance trade-offs across agent designs.This direction directly targets inconsistent efficiency reporting.
  • Agentic latent reasoning: Agentic latent reasoning is underexplored despite agent-specific requirements involving tool use, long-horizon planning, memory management, and action verification.The survey suggests new training objectives, interfaces, and evaluation protocols tailored to agents.
  • Deployment-aware design: Deployment-aware research should compare true multi-model deployments with single-model role-play pipelines because they differ in orchestration overhead, latency, and reliability.The proposal is motivated by chunked, sequential approaches to long-context reasoning.
  • MLLM-based agents: Efficiency in multimodal large-language-model agents remains relatively under-explored across multimodal memory, planning, decision-making, and multi-agent systems.The survey identifies this as an efficiency challenge for realistic deployments.

8. Conclusion

The survey reviews efficiency in memory, tool use, and planning, synthesizes benchmarks and metrics, and identifies shared principles across diverse methods. It concludes by outlining challenges and future directions for efficient agent systems.

  • Scope and synthesis: The survey examines memory, tool use, and planning with emphasis on efficiency across diverse agent methods.It highlights convergence on shared high-level ideas despite implementation differences.
  • Evaluation: The survey consolidates efficiency-oriented benchmarks and commonly used metrics from benchmark and methodological studies.These resources support evaluation of agent capabilities and costs.
  • Future directions: The survey outlines key challenges and future directions for improving the efficiency of agent systems.The conclusion positions these directions as opportunities for further research.
Loading 2601.14192v2…