PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills

Pengcheng Jiang, Jiacheng Lin, Zhiyi Shi, Zifeng Wang, Luxi He, Yichen Wu, Ming Zhong, Peiyang Song, Qizheng Zhang, Heng Wang, Xueqiang Xu, Hanwen Xu, Pengrui Han, Dylan Zhang, Jiashuo Sun, Chaoqi Yang, Kun Qian, Tian Wang, Changran Hu, Manling Li, Quanzheng Li, Hao Peng, Sheng Wang, Jingbo Shang, Chao Zhang, Jiaxuan You, Liyuan Liu, Pan Lu, Yu Zhang, Heng Ji, Yejin Choi, Dawn Song, Jimeng Sun, Jiawei Han

arXiv:2512.16301v3cs.AIcs.CL

TL;DR

Agentic AI research is fragmented across post-training, retrieval, memory, and skill systems, while capable models still face adaptation needs and limitations. This survey organizes adaptation through four agent- and tool-focused paradigms and synthesizes their methods and trade-offs. It concludes that paradigm effectiveness varies with feedback and that evaluation needs multiple dimensions.

  • Problem

    Agentic systems remain limited by unreliable tool use, shallow long-horizon planning, domain-specific reasoning gaps, and poor generalization to unfamiliar environments.

    Source: §1 · p. 2

  • Method

    The survey organizes agentic adaptation into four paradigms based on what is optimized and how its adaptation signal is obtained.

    Source: §1 · p. 3

  • Results

    A1 tends to suit domains with dense, verifiable execution feedback, while A2 is needed when only sparse outcome signals are available; evidence for a universal claim remains incomplete.

    Source: §10 · p. 67

  • Takeaways & Limitations

    Paradigm choice and evaluation should account for feedback type, task requirements, and system constraints, with no single metric sufficient across paradigms.

    Source: §1 · p. 3, §10 · p. 67

    Cross-paradigm efficiency comparisons are not controlled experiments because systems differ in optimization target, backbone composition, and architecture.

    Source: §10 · p. 67

Abstract

from arXiv · show

Large language model (LLM) agents are moving beyond prompting alone. ChatGPT marked the rise of general-purpose LLM assistants, DeepSeek showed that on-policy reinforcement learning with verifiable rewards can improve reasoning and tool use, and OpenClaw highlights a newer direction in which agents accumulate persistent memory and reusable skills. Yet the research landscape remains fragmented across post-training, retrieval, memory, and skill systems. This survey studies these developments under a single notion of \emph{adaptation}: improving an agent, its tools, or their interaction after pretraining. We organize the field with a four-paradigm framework spanning agent adaptation and tool adaptation. On the agent side, A1 (tool-execution-signaled) and A2 (agent-output-signaled) improve the agent itself through supervised fine-tuning, preference optimization, and reinforcement learning with verifiable rewards. On the tool side, T1 (agent-agnostic) provides reusable pre-trained modules any agent can call, while T2 (agent-supervised) uses the agent's outputs to train memory systems, skill libraries, or lightweight subagents. Using this framework, we review post-training methods, adaptive memory architectures, and agent skills; compare their trade-offs in cost, flexibility, and generalization; and summarize evaluation practices across deep research, software development, computer use, and drug discovery. We conclude by outlining open problems in agent-tool co-adaptation, continual learning, safety, and efficient deployment.

1 Introduction

The survey focuses on adaptation as a way to address agentic systems’ task and deployment limitations, organizing methods by whether they optimize the agent or its tools and by their adaptation signals. It synthesizes methods, trade-offs, evaluation, applications, and open challenges across this framework.

Adaptation paradigms

  • A1 and A2 optimize the agent, while T1 and T2 adapt tools independently of or using signals from a frozen agent.A1 uses verifiable tool outcomes; A2 uses evaluations of agent outputs; T2 adapts tools using signals derived from those outputs.Source: §1 · p. 3

Trade-offs

  • Agent adaptation requires more training resources but offers flexibility, while tool adaptation costs less but remains constrained by the frozen agent’s capabilities.T1 tools can generalize across agents and tasks; T2 enables independent tool upgrades, whereas A1/A2 adaptation risks catastrophic forgetting.Source: §1 · p. 3

Framework and scope

  • The unified 2 × 2 framework distinguishes agent versus tool adaptation and execution-grounded versus output-evaluated signals, yielding four paradigms with different cost, flexibility, and generalization profiles.The survey assigns boundary cases by their dominant locus of optimization and notes secondary components.Source: §1 · pp. 3–4

Cross-paradigm synthesis

  • In retrieval settings, T2 tool adaptation can match A2 agent-training accuracy with far fewer examples, and A1-trained agents can be frozen and redeployed as T1 tools.The survey cautions that cross-paradigm comparisons draw on case studies that may differ in confounding factors.Source: §1 · p. 4

Memory and skills

  • Adaptive memory systems instantiate T2, while skill libraries connect agent post-training with reusable tool ecosystems.The surveyed memory types include episodic buffers, reflective databases, and knowledge graphs.Source: §1 · p. 4

Evaluation and applications

  • The evaluation framework distinguishes execution metrics from holistic utility and identifies gaps particularly in T2 and cross-paradigm comparison.The survey maps applications across deep research, software development, computer use, and drug discovery, where dominant paradigms vary with feedback availability and retraining cost.Source: §1 · p. 4

Challenges and methodology

  • The survey identifies open challenges in agent-tool co-adaptation, continual adaptation, safe exploration during on-policy RL, and efficient deployment.Its reviewed literature extends through early 2026 and includes top-tier venues and high-impact arXiv preprints.Source: §1 · p. 4

2 Background

The background introduces agentic systems and the main ways they are adapted, from prompt changes that leave model parameters untouched to fine-tuning that updates them. It also locates adaptive memory within the agent-tool distinction.

2 Background

  • The survey first establishes foundational concepts and adaptation forms as context for the rest of its discussion.It focuses on how systems specialize for particular tasks or deployment scenarios.Source: §2 · p. 4

2.1 Agentic AI Systems

  • An agentic system combines a foundation model with planning, tool use, and memory to execute multi-step tasks using environmental feedback.The survey focuses on single-agent systems as a controlled setting for studying adaptation.Source: §2.1 · pp. 4–5

2.2 Adaptation

  • Adaptation changes an agent or its tools to improve performance for particular domains, tasks, or deployment conditions.The survey distinguishes prompt-based from fine-tuning-based adaptation.Source: §2.2 · p. 5, §2.2.2 · p. 5

2.2.1 Prompt Engineering

  • Prompt engineering shapes agent behavior with instructions and context without changing model parameters or requiring additional training.Prompts can be modified or composed to adapt an agent to new goals or environments.Source: §2.2.1 · p. 5

2.2.2 Fine-Tuning

  • Fine-tuning updates core-model parameters, using full or parameter-efficient training and methods including SFT, preference optimization, and reinforcement learning.LoRA typically matches full fine-tuning quality while updating less than 1% of parameters; most adaptive external memory updated from frozen-agent outputs is classified as T2.Source: §2.2.2 · pp. 5–6

3 Overview of Adaptation Paradigms of Agentic AI

The survey organizes agentic AI adaptation by what is optimized and how its adaptation signal is obtained, yielding four canonical paradigms.

3 Overview of Adaptation Paradigms of Agentic AI

  • The framework categorizes studies by whether they adapt the agent or tool and by the source of the adaptation signal, producing four paradigms.These paradigms form the paper’s analytical basis.Source: §3 · p. 6

3.1 Mathematical Notations

The section establishes notation for adaptation targets, data sources, and objectives in agentic systems.

3.1 Mathematical Notations

  • The agent is the core reasoning and decision-making model, which can be adapted through parameter updates, prompt refinement, or other policy modifications.External tools include retrievers, planners, executors, simulators, and dynamic memory modules that learn from agent outputs.Source: §3.1 · p. 7
  • Adaptation signals can come from offline supervision data, including demonstrations, synthetic trajectories, and prior interaction logs, or from environment feedback during interaction.Environment feedback reflects task performance or execution quality.Source: §3.1 · p. 7
  • The objective function evaluates agent-tool system performance, using losses such as SFT for offline data or outcome metrics such as task success for interactive learning.The notation section groups its definitions around adaptation targets, data sources, and objectives.Source: §3.1 · pp. 6–7

3.2 Four Adaptation Paradigms of Agentic AI

The framework classifies adaptation by its optimization target and signal source. It distinguishes agent adaptation from tool adaptation and specifies how memory systems fit, including boundary cases.

3.2 Four Adaptation Paradigms of Agentic AI

  • Adaptation targets either the agent or its tools; agent adaptation is signaled by tool-execution feedback (A1) or final-output evaluation (A2), while tool adaptation is independent (T1) or fixed-agent-supervised (T2).The section describes a single-turn interaction as the basic formulation, with multi-turn use extending naturally.Source: §3.2 · p. 7, §3.2.1 · p. 7

3.2.1 A1: Tool Execution Signaled Agent Adaptation

  • A1 adjusts the agent using measurable tool-execution outcomes, such as execution success or retrieval scores.It can imitate successful recorded tool-call trajectories through SFT or optimize actions interactively using tool feedback through RL.Source: §3.2.1 · pp. 7–8

3.2.2 A2: Agent Output Signaled Agent Adaptation

  • A2 adjusts the agent using evaluations of its final output, whether or not tools contributed to that output.Its SFT combines final-output supervision with tool-call imitation, while RL can reward final-response quality irrespective of intermediate tool calls.Source: §3.2.2 · pp. 8–9

3.2.3 T1: Agent-Agnostic Tool Adaptation

  • T1 keeps the agent fixed and trains external tools independently, making them usable as plug-in components.Tool training may use supervised, contrastive, or reinforcement learning and evaluate retrieval, ranking, simulation, or downstream task performance.Source: §3.2.3 · p. 9

3.2.4 T2: Agent-Supervised Tool Adaptation

  • T2 keeps the agent fixed and adapts tools using signals from its outputs so the tools better support that agent’s downstream reasoning.Supervised approaches can weight tool-training trajectories by the quality of the agent’s output; RL can reward the final output.Source: §3.2.4 · pp. 9–10
  • External memory updated from a frozen agent’s outputs is predominantly T2, while independently trained memory is predominantly T1; parametric and hybrid forms can blur the boundary.When memory updates change the agent’s own parameters, the survey classifies them under A1 or A2; auxiliary-parameter updates with a frozen core lie at the T1/T2 boundary.Source: §3.2.4 · p. 10

3.3 Illustrative Examples

The examples contrast adaptation signals across retrieval and code-execution tasks, then illustrate agent-agnostic and agent-supervised tool adaptation in RAG.

3.3.1 Agent Adaptation Examples Across Two Applications

  • DeepRetrieval [22] optimizes retrieval quality directly, whereas Search-R1 [50] optimizes final-answer correctness and must learn retrieval strategies as a side effect, making credit assignment harder.The examples use the same retrieval action to contrast tool-feedback-based A1 with output-based A2.Source: §3.3.1 · p. 11
  • DeepSeek-R1 uses code execution outcomes such as test-case pass rates for A1, while ReTool [51] uses final-answer correctness for A2, a richer but noisier learning signal.In the ReTool example, the sandbox result is returned to the agent before it generates its final answer.Source: §3.3.1 · p. 11

3.3.2 Tool Adaptation Examples in the RAG Setting

  • Because closed-source API agents may not be fine-tunable, adapting auxiliary tools such as RAG retrievers is presented as a more feasible strategy.The passage notes that matching closed-source model performance with an open-source model is challenging due to data curation, scaling, and infrastructure demands.Source: §3.3.2 · pp. 11–12
  • T1 includes dense retrievers and learned query-rewriting subagents trained independently of the fixed agent and reusable for its retrieval and answer synthesis.A DeepRetrieval-style model can serve as a subagent that reformulates queries or curates documents for a fixed closed-source agent.Source: §3.3.2 · p. 12
  • In T2 examples, S3 [28] updates a search subagent using the fixed agent’s answer-quality reward, while AgentFlow [52] applies that supervision to a planning-oriented subagent.The adapted tools aim to produce retrieval behavior or planning policies aligned with the fixed agent’s downstream needs.Source: §3.3.2 · p. 12

4 Agent Adaptation

This section surveys agent adaptation from tool-execution feedback and evaluations of agent outputs, tracing A1 from off-policy training to on-policy RLVR and contrasting it with A2’s end-to-end feedback. It also covers how these methods vary in feedback granularity, weight updates, and tool-use strategy learning.

4.1 A1: Tool Execution Result as Signal

  • A1 uses verifiable tool or environment outcomes for dense, action-level learning, but those signals may miss failures that emerge when actions compose.Execution feedback is determined externally, supporting supervised and reinforcement learning; its focus on individual actions can leave system-level failures invisible.Source: §4.1 · p. 13

4.1.1 Earlier Works: SFT & Off-Policy Methods

  • Earlier A1 methods move from answer- and format-level correctness toward direct execution feedback, but off-policy training on pre-collected trajectories cannot explore unseen strategies.Using failed trajectories as contrastive supervision can improve data efficiency, while format-valid calls still need not be functionally correct.Source: §4.1.1 · pp. 15–16

4.1.2 RLVR-Based Methods

  • On-policy RLVR learns through online action, execution, and policy updates; its efficiency depends on reward density, quality, and composition, while stabilization and interactive compute remain costly.Dense per-step feedback eases credit assignment; across domains, clean verified rewards, task-plus-format rewards, and mechanisms such as KL regularization or curriculum scheduling recur.Source: §4.1.2 · pp. 16, 18
  • DeepRetrieval’s RLVR framework yielded roughly threefold recall improvement (65.1% vs. 24.7%) on real-world search engines.It casts query reformulation as an MDP and optimizes retrieval metrics with KL-regularized PPO, combining retrieval and format rewards.Source: §4.1.2 · p. 17

4.2 A2: Agent Output as Signal

  • A2 evaluates reasoning chains, answers, or artifacts, enabling strategic behaviors such as tool choice and self-correction, but episode-level rewards complicate credit assignment across decisions.Source: §4.2 · p. 18

4.2.1 Agent Adaptation w/o Tools

  • RLVR with binary final-answer correctness established a scalable reasoning pathway beyond SFT in DeepSeek-R1, and Kimi-1.5 extended the approach to multimodal agents [96].Later variants target clinical reasoning, self-knowledge calibration, human empowerment, contrastive-learning signals, and recommendation metrics [97] [98] [99] [100] [92].Source: §4.2.1 · p. 21
  • Self-Refine’s generate-critique-revise loop became trainable through SCoRe’s multi-turn on-policy RL, replacing inference-time heuristics with learned policies that improve with scale [101].Self-Refine uses the same LLM to generate, critique, and revise responses without supervised data or auxiliary models.Source: §4.2.1 · pp. 21–22
  • 26% to 36% on LEETCODE-HARD and 91.2% to 95.1% on MMLU-Physics: TextGrad uses natural-language critiques to improve outputs without parameter access.metaTextGrad [103] recursively uses validation feedback to refine the optimizer’s prompts; these methods avoid reward-function design and propagate structured credit through multi-component systems.Source: §4.2.1 · p. 22
  • A2 without tools spans scalar correctness rewards, structured critiques, weight updates, and inference-time refinement, with objectives ranging from task correctness to calibration and human empowerment.SFT warm-up can ground domain knowledge before RL; scalar rewards are broadly applicable but sparse, while linguistic feedback is richer but requires a capable critic, and weight updates amortize cost across queries.Source: §4.2.1 · pp. 21–22

4.2.2 Agent Adaptation w/ Tools

  • R1-Searcher reports up to 24% over RAG baselines, while ReSearch integrates queries and results into reasoning and exhibits emergent reflection and self-correction [108] [109].Retrieval-based methods range from expert-demonstration SFT to RL discovery of multi-turn search strategies; Search-R1 jointly rewards retrieved evidence and final correctness [50].Source: §4.2.2 · p. 22
  • Code execution bridges A1 and A2: CodePRM scores intermediate steps from execution results, while ReTool trains agents to decide when interpreter use helps the overall answer [51].Because ReTool’s reward depends on the agent’s answer rather than only the tool call, A2 can discover tool-use strategies that A1 cannot.Source: §4.2.2 · p. 22
  • Self-Challenging Agents’ self-generated curriculum achieves over twofold improvement on multi-turn benchmarks, and Agent-R’s critique and MCTS rollouts yield +5.6% across interactive environments [118] [120].Other approaches use environment feedback to refine trajectories [119], inference-time fine-tuning on uncertain cases [121], or infrastructure for complex workflows, cost-regularized reasoning and acting, and asynchronous rollouts [122] [124].Source: §4.2.2 · pp. 22–23
  • A2 with tools optimizes when to invoke tools, which to choose, and how to integrate results, but sparse episode-level rewards make credit assignment and data efficiency central challenges.This strategic training can elicit unsupervised self-correction, adaptive search depth, and query refinement, while requiring large datasets or designed curricula; A1 instead targets per-action precision.Source: §4.2.2 · pp. 22–23

5 Tool Adaptation

This section surveys tool adaptation that improves an agent’s ecosystem around a fixed or frozen LLM, spanning reusable tools and trainable modules such as adaptive memory, skills, and lightweight subagents. It covers approaches shaped by agent outputs or task outcomes, including tool creation, retrieval, skill reuse, and embodied action.

5.1 T1: Agent-Agnostic Tool Adaptation

  • T1 tools are independently pretrained components invoked by frozen agents, forming a compositional substrate for A1/A2 agents and T2 subagents.The section describes orchestration through prompting, code, knowledge graphs, multimodal text bridges, or execution infrastructure such as MCP.Source: §5.1 · p. 23, §5.1.1 · p. 24

5.1.1 Foundational Systems and Architectures

  • HuggingGPT [126] orchestrates models through prompts, ViperGPT [127] composes them with generated code, and SciToolAgent retrieves tools from a knowledge graph.Visual ChatGPT [129] exemplifies multimodal bridging by serializing vision operations as text API calls; clear tool interfaces support these integration patterns.Source: §5.1.1 · p. 24

5.1.2 Categories and Training Methodologies

  • Structured-output tools such as code generators and retrievers are easier to optimize with agent feedback than perceptual vision or speech tools, which mainly remain T1 plug-ins.The distinction follows whether the frozen agent can automatically score a tool’s outputs.Source: §5.1.2 · p. 25

5.2 T2: Agent-Supervised Tool Adaptation

  • T2 treats a fixed foundation model as a source of supervision, training smaller peripheral tools instead of modifying the costly agent.Agent outputs provide signals for reshaping information into forms the agent can use; the section describes a shift from proxy signals toward verifiable outcomes.Source: §5.2 · pp. 25–26

5.2.1 Earlier Methods: From Proxy Signals to Structured Preferences

  • Early T2 retrieval work used frozen-LM perplexity or preferences to train tools, progressing from REPLUG [11] to preference-based methods including AAR [29] and RA-DIT [155].LLM-R [24] adds an intermediate cross-encoder reward model that captures frozen-LM preferences before distilling them into a fast bi-encoder retriever.Source: §5.2.1 · pp. 28–29
  • UPRISE [159] transfers prompt-selection skill across tasks, improving reading comprehension by +8.5% and paraphrase detection by +14.6%.It trains a prompt retriever using the frozen LLM’s task performance across diverse tasks.Source: §5.2.1 · p. 29
  • 35.6% exact-match accuracy on HotpotQA versus 25.8% with the best prior retriever: BGM uses a bridge model to align retrieved context with a frozen generator.BGM first trains on synthetic preference data, then uses the frozen LLM’s task success as a reinforcement-learning reward; its bridge model handles preference alignment while the retriever handles broad recall.Source: §5.2.1 · pp. 29–30
  • Cascaded T2 systems combine specialized tools such as query reformulators, retrievers, and selectors, each trained on different aspects of a frozen LLM’s behavior.This decomposition lets tools specialize in subproblems such as recall versus precision or speed versus quality.Source: §5.2.1 · p. 30
  • Empirical results suggest that 2–3 stages of tool adaptation, such as query reformulation, retrieval, and reranking, balance benefits against compounding errors.How deep a tool hierarchy can go before errors overwhelm its benefits remains an open question.Source: §5.2.1 · p. 30

5.2.2 Subagent-as-Tool

  • 58.9% average generation accuracy with 2.4k samples: s3 uses 70× less data and trains 33× faster than Search-R1, while reaching 76.6% versus 71.8% on specialized medical QA.The frozen generator supervises a lightweight searcher, which learns iterative query generation, retrieval, and evidence selection; the tool focuses on search rather than relearning domain knowledge and reasoning.Source: §5.2.2 · p. 30
  • ≈3–7% gains across reasoning, planning, and personalization benchmarks: Matryoshka Pilot transfers its controller across black-box backends, while AgentFlow reaches 57.3% on search tasks, 51.5% on math, and 33.1% on GAIA.Matryoshka Pilot uses trajectory-level success signals and Iterative DPO; AgentFlow trains a planner to orchestrate mostly frozen specialists using shared trajectory-level rewards.Source: §5.2.2 · pp. 31–32
  • T2 subagents expand from agentic search to memory construction, planning and orchestration, and self-evolving systems that shape tasks, rewards, and curricula for frozen reasoning cores.Mem-α trains a memory-writing controller and generalizes from approximately 30k-token training sequences to contexts exceeding 400k tokens; R-Zero and Multi-Agent Evolve train peripheral subagents rather than the main solver.Source: §5.2.2 · pp. 30–32

5.2.3 Agentic Memory and Skills

  • Agentic memory spans storage modality, duration, content, abstraction level, and update policy, with designs ranging from tiered stores and reflective memories to graphs and trainable or hybrid modules.Examples include MemGPT’s RAM/disk-style tiers, A-MEM’s dynamic linking and restructuring, HippoRAG’s associative graph retrieval, and parametric or hybrid memory architectures.Source: §5.2.3 · pp. 33–35
  • 91% pass@1 on HumanEval: Reflexion stores verbal self-critiques, while Agent Workflow Memory reduces web-navigation planning errors by 24% versus agents without workflow memory.Other approaches include retrieval followed by post-thinking, test-time curation of concise strategies, and extracting lessons from failed experiences for reuse.Source: §5.2.3 · pp. 34–36
  • 87.88% on GAIA validation, 79.40% on GAIA test, and 95.0% on SimpleQA: Memento trains case retrieval for a frozen GPT-4.1 planner.A soft Q-learning policy learns which past trajectories to retrieve from binary task-success rewards; case memory adds 4.7–9.6% absolute improvement on out-of-distribution tasks.Source: §5.2.3 · p. 35
  • MemSkill learns memory operations under frozen-agent supervision, while skill libraries turn successful experience into reusable, composable capabilities that agents retrieve and execute on later tasks.Skills can be represented as code, APIs, MCP protocols, or textual procedures; their lifecycle includes acquisition, representation, invocation, and refinement, and MemSkill adapts memory granularity to the task.Source: §5.2.3 · pp. 33, 35, 36
  • Voyager discovers 63 unique Minecraft items (3.3× more than baselines) without catastrophic forgetting; JARVIS-1 improves over Voyager and other baselines on 200+ tasks.JARVIS-1 retrieves multimodal successful-task trajectories as planner examples, with particular gains on long-horizon tasks involving 10+ sequential sub-goals.Source: §5.2.3 · p. 37
  • EXPEL [214] improves task success by 18% on ALFWorld and 12% on WebShop over non-adaptive baselines, while OS-Copilot [216] outperforms skill-storage-free baselines on GAIA.EXPEL extracts reusable rules from successful and failed trajectories; OS-Copilot creates and retains Python-function skills when execution succeeds, using outcomes rather than human annotation to curate its library.Source: §5.2.3 · p. 37
  • Computer-use systems reuse or grow skills through curation and retrieval, from CRADLE [217] and AppAgent to Agent S [219] and AgentStore [220].CRADLE retrieves or generates skills for GUI sub-goals and refines successful ones; AppAgent discovers mobile-app interactions by trial and error; Agent S matches experience to sub-goals, while AgentStore registers and composes specialist agents without retraining its meta-controller.Source: §5.2.3 · pp. 37–38
  • Skill discovery ranges from ExACT [221]’s exploration-built skill tree and SkillWeaver [223]’s shared API skills to PAE and SAGE [225]’s reinforcement-learning approaches.Zhao et al. [222] formalize autonomous skill discovery; ASI [209] induces structured programs for compositional generalization, and CASCADE [226] validates and evolves computational-chemistry skills for scientific workflows.Source: §5.2.3 · p. 38
  • T2 skill systems keep the agent fixed while agent-derived supervision evolves the library, trading fine-grained composability against broader transfer; LEGOMem [229] and ADAS [230] extend reuse to teams and architectures.Stored skills range from executable code and heuristics to trajectories, specialist agents, and callable functions; acquisition includes demonstrations, exploration, reflection, and reinforcement learning. LEGOMem shares procedural memory across agents, while ADAS iteratively refines reusable architectural patterns.Source: §5.2.3 · p. 39
  • Tool adaptation also spans embodied skills, interactive document retrieval, and lightweight modules that bridge modalities or adapt retrieval behavior.SayCan [231] grounds plans in robot affordances, ProgPrompt [232] composes primitive actions, and Eureka [233] matches or exceeds human-designed rewards in 83% of 29 IsaacGym tasks. ReadAgent [236] uses 3–5× fewer tokens than full-context systems on QuALITY and NarrativeQA; CoRAG [237] learns multi-step retrieval and Adaptive-RAG routes by query complexity. Other approaches include multimodal embeddings, prompt adaptation, and image-to-token mapping; common patterns are small trained modules with frozen LLMs and generalization to unseen settings.Source: §5.2.3 · pp. 39–40
  • ToolkenGPT reaches 73% one-hop accuracy on FuncQA vs. 57% for ReAct, 75% supervised accuracy on 234-relation KAMEL, and 68% success on VirtualHome with 58 tools.It trains only a small tool-embedding matrix while keeping the LLaMA model frozen; adding 234 tools requires about 1M parameters, and new tools can be added without full model retraining.Source: §5.2.3 · p. 40

6 Comparison of Adaptation Paradigms

The survey compares four adaptation paradigms by optimization target and signal source, then weighs their costs, flexibility, data efficiency, generalization, and modularity. Its synthesis contrasts adapting a whole agent with adapting tools around a fixed agent, while emphasizing the potential and limits of modular systems.

6.1 A Framework for Comparison

  • A1/A2 modify the agent for parametric flexibility but incur retraining costs and possible side effects; T1/T2 enable modular tool updates but remain bounded by the frozen agent.The survey says tool-centric adaptation tends to favor data efficiency and generalization, while modularity can matter more than raw performance in practice.Source: §6.1 · p. 41

6.2.1 A1: Optimizing Tool Mechanics via Causal Feedback

  • A1 uses tool-execution feedback to train tool mechanics; DeepRetrieval reports 65.1% vs. 24.7% recall, and R1-Code-Interpreter 72.4% accuracy on 37 test tasks.Trial-and-error training requires careful reward design and stabilization methods such as KL-regularized PPO or GRPO, curriculum learning, and dynamic sampling.Source: §6.2.1 · pp. 41–42

6.2.2 A2: Optimizing Tool Strategy via Holistic Rewards

  • A2 uses whole-output rewards to learn tool-use strategy; ReSearch gains 9–22% absolute over iterative RAG baselines, while R1-Searcher reports up to 24% over strong RAG baselines.Unlike A1’s feedback on tool calls, A2 rewards the result of the full thinking, searching, and reasoning process.Source: §6.2.2 · p. 42

6.3.1 T1: The “Graduated Agent” as Subagent-as-Tool

  • T1 tools may be pretrained components or A1/A2-trained experts frozen for reuse; DeepRetrieval and SWE-Grep illustrate graduated subagents callable by different agent systems.This agent-agnostic reuse supports tool swapping, but host agents may need to adapt their prompts or orchestration to each tool’s interface.Source: §6.3.1 · pp. 42–43

6.3.2 T2: Inverting the Optimization Target

  • T2 keeps the host agent fixed and trains tools under its supervision, letting lightweight subagents learn procedural skills; AgentFlow scores 33.1% on GAIA, surpassing GPT-4.T2 subagents can be added incrementally, but they trade some agent-agnosticity for tighter compatibility with the frozen host.Source: §6.3.2 · p. 43

6.4 Synthesis: Data Efficiency and Modularity (A2 vs. T2)

  • 2.4k training samples yielded comparable performance (58.9% average accuracy) for s3 versus Search-R1’s roughly 170k examples, but this is not a controlled comparison.The systems also differ in backbone composition and architecture, so the efficiency gap cannot be attributed to paradigm choice alone.Source: §6.4 · p. 44
  • On specialized medical QA, s3 reaches 76.6% accuracy versus Search-R1’s 71.8%, consistent with—but not proving—the hypothesis that narrower optimization targets generalize more robustly.The survey notes that training-data differences are an alternative explanation; T2 tools can also be hot-swapped without retraining the host agent.Source: §6.4 · p. 44

6.5 Strategic Recommendations

  • The survey recommends choosing among A1/A2 for changing core reasoning and T1/T2 for modular specialization, while describing hybrid systems with stable foundation agents and evolving subagents.T2 capability remains bounded by its supervising agent, and multi-subagent pipelines can add orchestration complexity and compound errors.Source: §6.5 · p. 45

7 Evaluation

The evaluation framework treats adapted agent-tool systems as ongoing processes rather than static predictors. It organizes evaluation around benchmarks, adaptation signals, system dynamics, and systemic requirements.

7 Evaluation

  • Adapted agent-tool systems allocate computation, coordinate tool calls, and stabilize under delayed or noisy feedback, so evaluation should move beyond static single-score benchmarking.The framework presents evaluation as examining an ongoing process rather than only a static prediction.Source: §7 · p. 46
  • The framework maps benchmark suites to A1/A2/T1/T2 and distinguishes verifiable execution metrics from holistic utility metrics.These are two of the four evaluation lenses identified by the survey.Source: §7 · p. 46
  • Its other lenses examine sample efficiency, generalization, and continual stability, alongside cost, safety, alignment, benchmark evolution, and open challenges.The framework groups these topics under adaptation dynamics, systemic requirements, and future benchmark work.Source: §7 · p. 46

7.1 Benchmark Landscape Mapped to A1/A2/T1/T2

The survey maps benchmarks to the adaptation signals and targets they assess, finding uneven coverage across paradigms. It highlights the lack of standardized, controlled cross-paradigm evaluation and describes protocol-based assessment as a possible route toward it.

7.1 Benchmark Landscape Mapped to A1/A2/T1/T2

  • A1 measures verifiable tool outcomes while A2 measures end-to-end output quality; their complementary metrics can reveal synthesis bottlenecks, shortcut reasoning, or test-gaming.For example, high A1 scores with low A2 scores can signal a synthesis bottleneck, whereas the reverse can signal memorization or shortcut reasoning.Source: §7.1 · p. 46
  • T2 evaluation should hold the agent fixed while varying the tool; current gaps include standardized tests of marginal tool value and skill acquisition, reuse, or drift.LongMemEval tests five long-term memory capabilities, while skill-security evaluations assess attack exposure rather than adapted skills’ marginal utility or long-term quality.Source: §7.1 · p. 47
  • Integrated long-horizon benchmarks stress-test agent-tool interaction but typically report endpoint metrics that cannot isolate which adaptation paradigm drives observed gains.Examples span web and desktop environments, multi-tool tasks, and large-scale tool-use scenarios.Source: §7.1 · p. 47
  • A1/A2 benchmarks and standalone T1 tool benchmarks are relatively mature, but T2 evaluation remains ad hoc and no suite supports controlled comparison across all four paradigms on matched tasks.The benchmark mapping is intended to expose which aspect of adaptation a suite evaluates and where gaps remain.Source: §7.1 · pp. 46, 49
  • Protocol-based AAA proposes assessor agents communicating with assessee agents through open protocols, but whether it enables cross-paradigm evaluation at scale remains empirically unvalidated.The proposed decoupling is intended to reduce integration overhead from evaluation harnesses tied to specific agent interfaces.Source: §7.1 · p. 49

7.2 Evaluating the Adaptation Signal

The survey contrasts verifiable execution metrics, which provide grounded process feedback, with holistic metrics that assess final outputs. Each offers a different view of system performance and distinct diagnostic limitations.

7.2.1 Verifiable Execution Metrics (A1 & T1)

  • Execution-based code assessment uses pass@k or test-suite pass rate, but cannot assess code quality, efficiency, or maintainability beyond functional correctness.The survey also notes that clean verified test suites and task heterogeneity affect the usefulness and stability of code-RL feedback.Source: §7.2.1 · p. 49
  • Retrieval metrics such as Recall@K, nDCG, and MAP support per-query diagnosis, yet high retrieval recall does not guarantee downstream utility because of a retriever–LLM preference gap.The gap is between surface-level relevance optimized by retrievers and the contextual coherence or inferential support useful to LLM reasoning.Source: §7.2.1 · p. 49
  • Formal verification can provide noise-free, step-wise feedback that eases long-horizon credit assignment, but applies only where tasks have formal specifications.It excludes specification-free tasks whose success criteria are implicit, subjective, or context-dependent.Source: §7.2.1 · p. 50
  • Verifiable execution metrics give dense, reproducible feedback for tool-use adaptation, but local success such as a passing test does not establish sound overall reasoning.Agents may game execution metrics without improving task-solving capability.Source: §7.2.1 · pp. 49–50

7.2.2 Holistic Utility Metrics (A2 & T2)

  • Holistic metrics assess final outcomes but obscure whether failures arise from reasoning, tool choice, execution, or synthesis; sparse episode-level feedback also makes intermediate diagnosis harder.LLM-as-judge adds verbosity, position, self-enhancement, and sycophancy biases, while calibration against human judgments remains essential.Source: §7.2.2 · pp. 50–51

7.3 Evaluating the Adaptation Dynamics

The section argues that endpoint scores miss adaptation dynamics and that efficiency, generalization, and stability must be evaluated over learning. Evidence shows that efficiency and transfer depend on task structure and signals, while co-adaptation stability remains an open measurement problem.

7.3.1 Sample and Interaction Efficiency

  • S3 (T2) reaches comparable retrieval accuracy to Search-R1 (A2) with roughly 70× fewer training examples, while code and multi-tool results show efficiency depends on signal and task structure.RLEF is more data-efficient than A2 in code because of dense execution rewards; Self-Challenging Agents [118] improve 2× by generating their own curriculum.Source: §7.3.1 · p. 51
  • Efficiency evaluation should report interaction-count distributions and jointly compare accuracy with training and inference compute, because tool latency and model size change the trade-offs.Smaller models may succeed more under tight time budgets with frequent cheaper calls, whereas larger models benefit from fewer, higher-quality interactions [290].Source: §7.3.1 · pp. 51–52

7.3.2 Generalization and Robustness

  • Generalization varies by paradigm: S3 (T2) reaches 76.6% on specialized medical QA versus 71.8% for Search-R1 (A2), while Agent-R [120] improves performance by 5.6% across interactive environments.The section recommends held-out task categories and cross-domain reporting; S3 also shows preliminary transfer across frozen generators, but systematic cross-agent testing remains rare.Source: §7.3.2 · p. 52
  • Robustness tests should include out-of-distribution inputs, degraded tools, and changing environments; multimodal and multi-agent robustness remains inadequately covered by current suites.Visual inputs add modality-specific shifts, while multi-agent systems face inter-agent shifts, fragile communication, and coordination failures; current suites largely lack systematic variation of modality and interaction structure.Source: §7.3.2 · pp. 52–53

7.3.3 Continual and Co-Adaptation Stability

  • Forgetting evaluation should track retained-task performance alongside new-task performance; RL may forget less than SFT under some conditions, and T2 structurally preserves the frozen core agent.The section recommends a held-out retention set and reporting backward transfer.Source: §7.3.3 · p. 53
  • No work has systematically measured agent-tool co-adaptation stability; proposed trajectory metrics remain theoretical and require empirical validation.Suggested measures include sliding-window variance, performance-gradient sign changes, and convergence rate, with cycling and collapse as pathologies to detect.Source: §7.3.3 · p. 53
  • Entropy and per-update entropy change can diagnose adaptation health: rapid entropy depletion signals premature convergence, while large fluctuations in entropy change signal instability.The section also recommends monitoring reward curves, tool-call frequency and diversity, and KL divergence from the reference policy.Source: §7.3.3 · p. 53

7.4 Systemic Evaluation

Systemic evaluation must consider cost, safety, and alignment alongside accuracy. The section recommends measuring training and inference costs, monitoring safety throughout adaptation, and evaluating alignment between users, agents, and tools.

7.4 Systemic Evaluation

  • Cost, safety, and alignment can dominate deployment decisions despite being orthogonal to accuracy.Source: §7.4 · p. 53

7.4.1 Cost and Inference-Time Compute

  • Agentic cost evaluation should decompose prompt and completion tokens, tool overhead, and retry costs, and report accuracy against compute or time budgets.AgencyBench [260] reports realistic tasks averaging one million tokens and 90 tool calls; training costs also vary, with A2 requiring large-scale RL and T1 typically cheapest.Source: §7.4.1 · p. 54

7.4.2 Safety

  • On-policy RL exploration can trigger dangerous tool actions, so evaluation should use sandboxes and test whether agents identify and refuse unsafe requests.ToolEmu emulates a sandbox, while R-Judge benchmarks safety-risk awareness.Source: §7.4.2 · p. 54
  • Dense deterministic execution rewards are harder to game than sparse holistic rewards, while T2 tools may inflate training scores without improving genuine utility.Reward-hacking checks should include held-out reward distributions, human review of high-reward trajectories, and detection of known gaming patterns.Source: §7.4.2 · pp. 54–55
  • Aggressive RL can erode supervised-fine-tuning safety guardrails, so safety should be evaluated at regular checkpoints as a trajectory alongside task performance.DeepSeek-R1 demonstrates reasoning around refusal mechanisms.Source: §7.4.2 · p. 55

7.4.3 Alignment

  • Alignment evaluation should jointly assess fidelity to user intent and whether adapted tools provide information in forms agents can use.BGM [160] reports a 38% relative improvement in downstream QA accuracy from a bridge model that reformats retrieval output for a frozen LLM; intermediate interaction quality should also be measured.Source: §7.4.3 · p. 55

7.5 Discussion

The discussion argues that endpoint scores alone obscure efficiency, stability, and safety differences, and that evaluation protocols shape which adaptation strategies are rewarded. It calls for counterfactual, multidimensional, and longitudinal benchmarks that better capture these differences.

7.5.1 Concrete Illustrations of the Dynamics Gap

  • Similar retrieval accuracy can conceal a 70× difference in training-example requirements, while identical code pass@k can mask divergent entropy trajectories.These examples illustrate why endpoint-equivalent methods may differ in efficiency and retained exploration capacity.Source: §7.5.1 · pp. 55–56
  • A1 metrics emphasize tool mechanics and A2 metrics strategic reasoning; aggressive RL can also temporarily erode safety before partial restoration.Such metric dependence and non-monotonic safety changes can remain invisible to endpoint-only checks.Source: §7.5.1 · p. 56

7.5.2 Tool-Centric vs. Agent-Centric Evaluation

  • Tool-centric evaluation supports independent tool comparison but misses interaction effects, whereas agent-centric evaluation captures user-facing outcomes but makes component attribution difficult.Source: §7.5.2 · p. 56
  • Counterfactual evaluation isolates a component’s marginal contribution by varying it while holding others fixed, but tool swaps can confound results when agents adapt their behavior to tool quality.Progressive tool degradation is proposed as a complementary sensitivity measure.Source: §7.5.2 · pp. 56–57

7.5.3 How Evaluation Reshapes Adaptation Design

  • Execution-based metrics favor A1 and holistic metrics favor A2, risking benchmark co-adaptation toward measured scores rather than unmeasured qualities.For example, pass@k-trained code may pass tests while remaining unreadable, unmaintainable, or inefficient.Source: §7.5.3 · p. 57
  • The proposed reporting standard includes multi-seed learning curves, cost-conditioned performance, retention-set results, and safety trajectories when RL is used.Joint accuracy, cost, and safety evaluation and Pareto frontiers are proposed to incentivize and display trade-offs.Source: §7.5.3 · p. 57

7.5.4 What Is Missing: Toward Next-Generation Benchmark Suites

  • Current benchmarks often use static tasks and single-paradigm evaluation, limiting assessment of continual adaptation and matched comparison across A1, A2, T1, and T2.The section calls for persistent evolving environments and matched task distributions with signals and protocols for all four paradigms.Source: §7.5.4 · pp. 57–58
  • Multimodal and multi-agent benchmarks do not yet measure adaptation trajectories, and current suites lack unified infrastructure for performance dynamics, forgetting, cost, safety, and alignment.Source: §7.5.4 · p. 58

8 Applications

The survey maps adaptation across deep research, software development, computer use, and drug discovery, showing how each domain combines agent and tool changes according to available feedback and task demands.

8.1 Deep Research

  • Deep research combines A2 reasoning-agent adaptation with T2 search subagents or T1 retrievers, while sparse deterministic execution signals make pure A1 less natural.Systems may still use retrieval metrics such as Recall@K for A1-style feedback; current systems built on generic corpora may struggle with expert-level inquiries.Source: §8.1 · p. 59

8.2 Software Development

  • Software development supports A1 through test and compilation feedback, while holistic issue completion calls for A2; tools such as SWE-Grep adapt code retrieval under agent supervision.Tool adaptation also includes evolving compilers, debuggers, and linters.Source: §8.2 · p. 60

8.3 Computer Use

  • Computer-use agents can adapt from human demonstrations or synthesized tutorial trajectories, with automatically evaluated trajectories supporting low-cost data generation.OpenCUA uses human demonstrations, whereas AgentTrek synthesizes trajectories from tutorials and retains correct executions.Source: §8.3 · p. 61
  • Reusable skill bases and evolving contextual playbooks shift some computer-use adaptation into tools, supporting skill retrieval, failure recovery, and persistent context.CUA-Skill uses parameterized execution and composition graphs; ACE curates tool-use strategies through execution feedback.Source: §8.3 · p. 61

8.4 Drug Discovery and Development

  • Drug-discovery tools range from curated T1 repositories to agent-supervised predictors and autonomously created tools, with scientific correctness requiring domain-expert validation.ToolUniverse and STELLA illustrate increasingly autonomous tool creation.Source: §8.4 · p. 62
  • Drug discovery uses all four paradigms: execution signals suit computational outputs, holistic research quality suits A2, and wet-lab validation creates sparse, delayed rewards.Clinical evidence retrieval can use A1-style citation recall, patient-to-trial matching favors A2, and trial design may combine both signals.Source: §8.4 · pp. 61–62

9 Opportunities

The opportunities section argues that adaptation must extend from optimizing agents or tools separately toward joint, continual, safe, and efficient systems. It outlines specific challenges and candidate mechanisms for each direction.

9.1 Co-Adaptation

  • Agent-tool co-adaptation is a bi-level optimization problem complicated by ambiguous credit assignment and unstable mutual learning; pacemaker mechanisms are one proposed direction.Current paradigms generally freeze one component to stabilize the other's learning target.Source: §9.1 · pp. 63–64

9.2 Continual Adaptation

  • Continual adaptation addresses shifting tasks and catastrophic forgetting through parameter-update methods and evolving external memories; on-policy reverse-KL RL can forget less than SFT in reported results.External tools can preserve new capabilities outside a shared parameter space, while parameter updates mitigate rather than structurally remove interference.Source: §9.2 · pp. 64–65

9.3 Safe Adaptation

  • Adaptation creates safety risks including unsafe exploration and exploitative optimization; proposed mitigations include safety shields, verifiable rewards, and pre-tool safety checks.The section also identifies specification gaming, adversarial tooling, and sycophancy loops as failure modes.Source: §9.3 · pp. 65–66
  • On-policy RL can erode SFT-established safety guardrails, with analysis of DeepSeek-R1 reporting greater jailbreak susceptibility than non-adapted baselines.The passage links this risk to aggressive RL optimization for reasoning and complex chain-of-thought justifications.Source: §9.3 · p. 65

9.4 Efficient Adaptation

  • Efficient adaptation targets parameter count, numerical precision, and deployment location through low-rank updates, quantized rollouts, and on-device personalization.FlashRL uses truncated importance sampling to stabilize gradients from quantized rollouts while preserving downstream performance.Source: §9.4 · pp. 66–67
  • The three efficiency directions are composable: a LoRA adapter quantized to INT8 and updated on-device combines all three.Source: §9.4 · p. 67

10 Conclusion

The conclusion presents A1/A2 and T1/T2 as a framework for organizing adaptation by optimization target and signal source. It highlights trade-offs and open needs spanning evaluation, memory, skills, and joint adaptation.

10 Conclusion

  • Agent-centric A1/A2 offer parametric flexibility but incur high computation, and adaptation—especially A2—can degrade previously learned capabilities.Reverse-KL on-policy RL shows less forgetting than SFT in some settings, while T2 avoids shared-parameter interference by keeping the core agent frozen.Source: §10 · p. 67
  • Tool-centric T1/T2 provide reusable modules and can achieve competitive retrieval-augmented QA accuracy with fewer examples in case studies, but comparisons are uncontrolled.Differences in optimization targets, backbones, and system architectures prevent attributing the efficiency gap solely to paradigm choice.Source: §10 · p. 67
  • Adaptation effectiveness varies with feedback density, and evaluation should report tool-execution quality, end-to-end success, cost, safety, and adaptation dynamics.The evidence for a universal signal-density pattern remains incomplete, and no single metric suffices.Source: §10 · p. 67
  • Memory and skills span the taxonomy, while future progress calls for integrating paradigms through co-adaptation, continual learning, safety, and efficient deployment.A1/A2-trained agents can become T1 tools, and external T2 memory can preserve experience without parameter updates.Source: §10 · pp. 67–68
Loading 2512.16301v3…