PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Reasoning for Large Language Models

Tianxin Wei, Ting-Wei Li, Zhining Liu, Xuying Ning, Ze Yang, Jiaru Zou, Zhichen Zeng, Ruizhong Qiu, Xiao Lin, Dongqi Fu, Zihao Li, Mengting Ai, Duo Zhou, Wenxuan Bao, Yunzhe Li, Gaotang Li, Cheng Qian, Yu Wang, Xiangru Tang, Yin Xiao, Liri Fang, Hui Liu, Xianfeng Tang, Yuji Zhang, Chi Wang, Jiaxuan You, Heng Ji, Hanghang Tong, Jingrui He

arXiv:2601.12538v1cs.AIcs.CL

TL;DR

LLMs perform strongly in closed-world reasoning but struggle in open-ended, dynamic environments requiring action and adaptation. This survey organizes agentic reasoning into foundational, self-evolving, and collective layers, distinguishing in-context orchestration from post-training optimization. It synthesizes applications and benchmarks into a roadmap connecting reasoning with action while identifying open challenges for deployment.

  • Problem

    LLMs struggle in open-ended environments because conventional reasoning approaches assume static contexts and short-horizon inference rather than continual action, adaptation, and improvement.

  • Method

    The survey systematizes agentic reasoning across three layers and two optimization modes, covering formal models, systems, applications, and evaluation benchmarks.

  • Results

    The survey synthesizes agentic reasoning methods into a unified roadmap bridging reasoning and acting across realistic domains and task settings.

  • Takeaways & Limitations

    Future progress depends on addressing personalization, long-horizon interaction, world modeling, scalable multi-agent training, and governance for real-world deployment.

  • Takeaways & Limitations

    Current world-model designs often use ad hoc representations and short-horizon or environment-specific training data, raising concerns about calibration and generalization.

Abstract

from arXiv · show

Reasoning is a fundamental cognitive process underlying inference, problem-solving, and decision-making. While large language models (LLMs) demonstrate strong reasoning capabilities in closed-world settings, they struggle in open-ended and dynamic environments. Agentic reasoning marks a paradigm shift by reframing LLMs as autonomous agents that plan, act, and learn through continual interaction. In this survey, we organize agentic reasoning along three complementary dimensions. First, we characterize environmental dynamics through three layers: foundational agentic reasoning, which establishes core single-agent capabilities including planning, tool use, and search in stable environments; self-evolving agentic reasoning, which studies how agents refine these capabilities through feedback, memory, and adaptation; and collective multi-agent reasoning, which extends intelligence to collaborative settings involving coordination, knowledge sharing, and shared goals. Across these layers, we distinguish in-context reasoning, which scales test-time interaction through structured orchestration, from post-training reasoning, which optimizes behaviors via reinforcement learning and supervised fine-tuning. We further review representative agentic reasoning frameworks across real-world applications and benchmarks, including science, robotics, healthcare, autonomous research, and mathematics. This survey synthesizes agentic reasoning methods into a unified roadmap bridging thought and action, and outlines open challenges and future directions, including personalization, long-horizon interaction, world modeling, scalable multi-agent training, and governance for real-world deployment.

1. Introduction

The survey reframes LLMs as autonomous agents that plan, act, and learn through continual environmental interaction. It organizes agentic reasoning across foundational, self-evolving, and collective layers, and distinguishes in-context from post-training optimization.

  • Motivation: Conventional LLM reasoning remains limited by static contexts and short horizons, whereas open-ended environments require agents to act, adapt, and improve.Existing methods such as decomposition and program-aided solving improve inference performance but generally assume closed-world settings.
  • Conceptual framing: Agentic reasoning reframes LLMs as autonomous agents that plan, act, and learn through continual interaction, unifying reasoning with perception, planning, decision, and verification.The framework includes tool use, workflow orchestration, long-horizon planning, partial observability, and feedback-driven improvement.
  • Applications and agenda: The survey reviews applications and benchmarks spanning mathematics, coding, science, robotics, healthcare, autonomous research, and web exploration.It also identifies future priorities including robustness, trustworthiness, efficiency, personalization, long-horizon interaction, world modeling, multi-agent training, and governance.
  • Optimization modes: Across these layers, in-context reasoning scales inference-time interaction without parameter updates, while post-training reasoning internalizes strategies through reinforcement learning and fine-tuning.The survey examines how both modes support planning, memory, and self-improvement in single-agent and multi-agent settings.
  • Taxonomy: The survey formalizes three layers: foundational capabilities, self-evolving adaptation, and collective multi-agent coordination.These layers cover planning, tool use, search, feedback, memory, continual adaptation, role assignment, communication, and shared memory.

2. From LLM Reasoning to Agentic Reasoning

Agentic reasoning reframes LLMs from passive predictors into systems that think, act, adapt, and coordinate through interaction. The survey formalizes this shift across latent reasoning, inference-time orchestration, post-training optimization, self-evolution, and multi-agent communication.

  • Traditional LLM reasoning improves inference over static inputs but lacks mechanisms to interact, remember, or adapt to changing goals.
  • Agentic systems scale reasoning through iterative interaction, invoking tools, exploring alternatives, updating memory, and integrating feedback.
  • The survey positions agentic reasoning at the intersection of model-centric reasoning and system-level intelligence, treating reasoning as interaction.
  • A latent-space formulation separates observations, external actions, internal reasoning traces, and memory within a partially observable environment.
  • In-context reasoning searches over reasoning trajectories with frozen parameters, whereas post-training reasoning optimizes policy parameters for long-horizon rewards.
  • Self-evolving and multi-agent formulations extend agentic reasoning through updates to memories or artifacts and communication among decentralized policies.

3. Foundational Agentic Reasoning

Foundational agentic reasoning turns an LLM into an autonomous problem solver by organizing planning, search, and tool use into interactive action loops. The survey categorizes planning methods by workflow structure, external aids, search, formalization, decomposition, and reward optimization.

  • Foundational agents translate reasoning into structured action through planning, search, and tool use rather than passive text generation.
  • A canonical workflow interleaves goal decomposition, external tool invocation, retrieval or exploration, and feedback-informed decision-making.
  • Planning research is organized into six methodological styles that support complex agentic reasoning.
  • Workflow Design: Workflow designs structure solving into stages such as perception, reasoning, execution, and verification, often through plan-and-act pipelines.
  • Tree Search / Algorithm Simulation: Tree-search methods use BFS, DFS, A*, MCTS, or beam search to explore alternatives, while hierarchical systems connect node expansion with grounded execution and feedback.
  • Process Formalization / Decoupling / Reward Design: Other approaches formalize plans symbolically, decompose planning into modular components, or optimize behavior through rewards and control-theoretic tools.
  • External Aid / Tool Use: External-aid methods use retrieval, knowledge graphs, world models, and general-purpose tools to provide structured information or planning support.

3.2. Tool-Use Optimization

Tool-use optimization augments LLM capabilities by intelligently selecting, invoking, and orchestrating external modules. The survey distinguishes in-context, post-training, and orchestration-based approaches, while noting that in-context methods remain bounded by frozen-model capabilities and context length.

  • Tool-use optimization enables agents to augment intrinsic capabilities by deciding when, which, and how to invoke external modules.External tools can address outdated knowledge, imprecise computation, and missing private information.
  • In-context tool-integration: In-context tool integration equips frozen models through prompts, demonstrations, and contextual instructions without modifying model parameters.The approach uses carefully crafted instructions and examples to guide complex task performance at inference time.
  • In-context tool-integration: Interleaved thought-tool-observation loops let agents reflect on intermediate outputs and dynamically plan subsequent tool queries.ChatCoT structures reasoning as alternating natural-language thoughts, tool calls, and observations.
  • In-context tool-integration: In-context methods can lose performance with large toolsets, motivating optimization of tool documentation and contextual information.Research explores clearer tool descriptions and computationally efficient context-management approaches.
  • In-context tool-integration: In-context performance is bounded by the frozen LLM’s capabilities and context-window length, motivating post-training methods.Post-training addresses these constraints by changing model capabilities rather than relying only on inference-time context.
  • Post-training tool-integration: Post-training integrates tools through supervised fine-tuning and reinforcement learning, enabling models to access current information, compute precisely, and ground multistep reasoning.SFT bootstraps tool invocation from curated demonstrations, while RL refines when, how, and which tools to use through outcome-driven rewards.
  • Orchestration-based tool-integration: Tool orchestration plans, sequences, and manages dependencies across multiple tools, including through centralized agents, API matching, and structured planning.Frameworks such as HuggingGPT and TaskMatrix.AI coordinate specialized tools or models for complex tasks.
  • Orchestration-based tool-integration: Tool representations can encode semantic similarity and dependency relations to improve tool selection, composition, and coordination.Network-based representations help agents distinguish similar tools and account for interdependencies during selection.

3.3. Agentic Search

Agentic search turns retrieval from a fixed preliminary step into an adaptive reasoning process. The survey organizes these systems by retrieval architecture and training mode, spanning in-context prompting, structure-enhanced reasoning, supervised fine-tuning, and reinforcement learning.

  • Agentic search agents dynamically decide when, what, and how to retrieve, unlike traditional RAG systems that perform fixed one-shot retrieval.This enables retrieval refinement and evidence integration during reasoning.
  • In-context Search: In-context agentic RAG interleaves reasoning and search through prompting, few-shot examples, or special tokens without additional training.ReAct alternates reasoning with search commands, while recursive and reflective methods retrieve sub-evidence or assess whether more information is needed.
  • Structure-Enhanced Search: Structure-enhanced agentic RAG enables agents to query symbolic sources such as knowledge graphs, invoke tools, and monitor whether evidence is sufficient.These systems move beyond static graph retrieval toward active, feedback-driven symbolic reasoning.
  • Table 4 categorizes representative agentic search systems by reasoning structure, format, and tool use.NL denotes natural-language reasoning traces, Ops denotes symbolic or graph operations, and KG denotes knowledge graph; tools include search APIs, browser actions, and KG retrieval.
  • Post-Training Search: Post-training agentic RAG fine-tunes models to decide when retrieval is needed, formulate queries, and incorporate retrieved evidence across multistep reasoning.Both supervised fine-tuning and reinforcement learning are used to instill retrieval-aware capabilities.
  • SFT-Based Agentic Search: SFT-based agentic search learns retrieval behavior from curated or synthetic datasets that interleave retrieval operations with natural-language reasoning.Toolformer and multi-task instruction-tuning approaches reduce reliance on human annotation through scalable data generation.
  • RL-Based Agentic Search: RL-based agentic search uses rewards tied to answer quality, factuality, or preferences to train dynamic evidence access.Reported emergent behaviors include iterative decomposition, re-verification, evidence planning, and more interpretable reasoning trajectories.

4. Self-evolving Agentic Reasoning

Self-evolving agentic reasoning turns reasoning into a continual improvement loop driven by feedback and memory. The survey distinguishes reflective, parametric, and validator-driven feedback, alongside increasingly structured and learning-controlled memory.

  • Core mechanisms: Feedback and memory transform reasoning from a static process into an adaptive loop that supports self-correction, knowledge reuse, and continual improvement.Feedback supplies evaluative signals, while memory stores and organizes prior interactions for accumulation and reuse across tasks.
  • Feedback mechanisms: The survey organizes agentic feedback into reflective feedback, parametric adaptation, and validator-driven feedback.These regimes span inference-time revision, parameter updates, and retry based on external success or failure signals.
  • Feedback mechanisms: Reflective feedback revises reasoning during inference through generate–critique–revise cycles while leaving model parameters unchanged.Critiques may be repeated or augmented with external evidence such as retrieval.
  • Feedback mechanisms: Parametric adaptation consolidates feedback into model weights, producing durable cross-task improvements but requiring additional training and reducing flexibility.Feedback-enriched reasoning trajectories can be used as supervised training data to internalize improved patterns.
  • Feedback mechanisms: Validator-driven feedback repeatedly samples outputs until external validators accept them, offering efficient correction when reliable validators are available.The approach is especially suited to domains such as program synthesis and software engineering, where validation is reliable and inexpensive.
  • Agentic memory: Agentic memory progresses from contextual recall toward structured representations and learning-based control for coherent, adaptive, long-horizon behavior.Structured formats align task semantics, temporal dependencies, and multimodal signals, while reinforcement or imitation learning optimizes memory use.
  • Evolving capabilities: Self-evolving planning and search generate tasks, refine strategies, evolve memory bases, and synthesize retrieval into actionable knowledge over extended horizons.These methods support continuous improvement through iterative environmental interaction and adaptive retrieval.

5. Collective Multi-agent Reasoning

Collective multi-agent reasoning extends single-agent perception–action loops into collaborative systems with specialized roles, communication, and shared memory. The survey frames this expansion around coordination challenges and a unified analysis of how distributed agents evolve collectively.

  • Collective reasoning: Multi-agent reasoning assigns complementary roles such as Manager, Worker, and Verifier to jointly solve complex tasks through specialization.Role differentiation distributes decomposition, execution, and evaluation across agents.
  • Collective reasoning: Collaboration exchanges reasoning traces, resolves conflicts, and maintains shared memory, turning reasoning into a distributed iterative loop.Agents refine one another’s outputs and collectively converge toward better solutions.
  • Open challenges: Multi-agent systems introduce system-level challenges in role differentiation, communication, labor division, collective memory, and continual adaptation.These challenges arise because coordination must be designed across interacting agents rather than within one solver.
  • Survey structure: The survey analyzes multi-agent reasoning through role taxonomies, collaboration and division of labor, and memory-enabled collective evolution.Together, these perspectives describe how reasoning scales from individual agents to adaptive collaborative intelligence.

5.1. Role Taxonomy of Multi-Agent Systems (MAS)

The role taxonomy organizes multi-agent systems around generic coordination, execution, evaluation, memory, and communication functions, then maps them onto domain-specific responsibilities. This specialization mirrors professional workflows in software engineering, finance, legal practice, and education.

  • Domain adaptations: Domain adaptations specialize these generic functions for professional practices across software engineering, finance, legal activities, education, healthcare, biomedicine, and music.The survey presents domain-specific roles as mappings of industry responsibilities onto MAS architectures.
  • Generic roles: Generic MAS roles include Leader/Coordinator, Worker/Executor, Critic/Evaluator, Memory Keeper, and Communication Facilitator.These roles respectively manage objectives, execute actions, verify outputs, maintain long-term state, and govern information exchange.
  • Software engineering: Software-engineering MAS roles mirror the development lifecycle through architects, developers, reviewers or testers, CI orchestrators, and release managers.The division balances design, implementation, verification, automation, and deployment governance.
  • Finance: Financial MAS commonly divide work among analysts, risk managers, traders or execution agents, and compliance officers.This structure separates signal extraction, exposure control, market interaction, and regulatory oversight.
  • Design rationale: Role specialization and checks-and-balances are presented as practical design principles for modular, scalable multi-agent systems.The survey connects these principles to systems such as R&D-Agent-Quant, FinRobot, MetaGPT, and ChatDev.
  • Legal activities: Legal MAS decompose consultation, statutory reasoning, and courtroom simulation among intake, lawyer, documentation, evaluation, knowledge, and adversarial roles.These arrangements support consultation workflows, reusable legal ontologies, formal case analysis, and iterative argument refinement.
  • Education: Educational MAS distribute tutoring and learner-memory functions to provide personalized, adaptive instruction.A tutor conducts Socratic dialogue while a memory dispatcher tracks progress and misconceptions to adjust lessons.

5.2. Collaboration and Division of Labor

Collaboration and division of labor coordinate specialized agents through inference-time orchestration or post-training optimization. The survey reviews manual and LLM-driven pipelines, routing, theory-of-mind collaboration, and graph- or policy-based topology generation.

  • Coordination modes: Multi-agent collaboration decomposes responsibilities across specialized agents, with in-context coordination specified at inference time and post-training collaboration optimized through learning or search.Agentic routing explicitly offloads cognition and computation according to task demands.
  • In-context collaboration: In-context systems range from manually crafted pipelines with fixed roles and workflows to LLM-driven orchestrators that dynamically decompose, route, and refine tasks.Manual systems predefine execution order and communication rules, whereas LLM-driven pipelines adapt using intermediate feedback.
  • In-context collaboration: Agent routing selects role-specialized agents for subtasks, with knowledge-graph-guided dispatch enabling collaborative question answering without modifying individual agents.Routing is treated as a decision layer within broader orchestration.
  • In-context collaboration: Theory-of-mind collaboration represents other agents’ beliefs, intentions, and mental states to improve cooperative reasoning and joint action.Approaches include belief-state representations, hypothesis refinement, counterfactual reflection, and nested-belief simulation.
  • Post-training collaboration: Post-training collaboration optimizes prompts, personas, interaction structures, and routing policies to improve coordination and task performance.Prompt optimization refines roles and workflows, while topology optimization learns communication structures or agent-selection policies.
  • Topology optimization: Graph-based topology methods model agents as nodes and communication as edges, then generate, prune, or search task-appropriate coordination graphs.These methods target effective pipelines under communication, token, or compute constraints.
  • Topology optimization: Policy-based topology generation learns which agents should act next using supervised fine-tuning or reinforcement learning.Auxiliary training signals are used to strengthen reasoning and coordination reliability.

5.3. Multi-Agent Evolution

Multi-agent evolution extends self-improvement from individual agents to coordinated systems that adapt across agents and time. The survey highlights multi-agent memory as a central design problem spanning organization, storage, content, and management, while identifying joint post-training as an open gap.

  • Multi-agent evolution extends adaptation from individual agents to distributed teams, targeting collective performance on complex, long-horizon tasks.
  • Temporal evolution distinguishes intra-test-time adaptation during task execution from inter-test-time learning that transfers improvements across tasks.Intra-test-time methods use reflection, replanning, memory rewriting, or localized updates to correct failures and refine strategies within an episode.
  • Multi-agent evolution combines reinforcement learning, game-theoretic co-optimization, and shared memory to support self-organization, self-correction, and social learning.
  • Multi-agent memory design must jointly address architecture, topology, content, and management to preserve shared and private knowledge across collaborating agents.These dimensions concern memory structure, storage and sharing, stored knowledge, and maintenance or updating.
  • Memory-sharing topologies trade knowledge diversity against verification rigor, with shared pools producing more diverse collective memory and improved open-ended-task performance.
  • Current multi-agent memory research focuses mainly on organization and retrieval for pretrained models, leaving joint post-training optimization of memory structures and policies open.

6. Applications

The survey examines agentic reasoning across mathematics and coding, scientific discovery, robotics, healthcare, and autonomous web exploration and research. It organizes these applications using the previously established foundational, self-evolving, and collective reasoning layers.

  • Agentic reasoning applications span math exploration, vibe coding, scientific discovery, robotics, healthcare, and autonomous web exploration and research.

6.1. Math Exploration & Vibe Coding Agents

Math exploration and vibe coding shift evaluation and practice from static problem solving toward dynamic processes involving exploration, adaptation, collaboration, and interactive software development. The surveyed systems operationalize this shift through planning, tools, search, feedback, reflection, and memory.

  • Traditional math benchmarks are increasingly saturated, motivating evaluation through dynamic exploration, adaptation, and collaboration.
  • Agentic coding treats the model as an interactive collaborator that maintains context, adapts to changing requirements, and self-corrects through dialogue.
  • Planning: Mathematical agents use planning structures including staged construction, lemma generation, verification, and iterative refinement of candidate programs or mathematical objects.
  • Planning: Code agents use explicit planning and action spaces, external knowledge, and increasingly structured organizations to support modular multi-step code construction.
  • Tool-Use: Tool integration extends single-agent reasoning through theorem provers, APIs, search, documentation, static analysis, and execution feedback.
  • Search and Retrieval: Search and retrieval structure mathematical discovery through propose–test–refine exploration and enrich code generation with repository-level context adjusted using execution feedback.
  • Agentic Feedback and Reflection: Feedback and reflection create closed-loop refinement by exposing discrepancies, diagnosing failures, repairing code, and reusing earlier drafts, analyses, and execution traces.
  • Memory: Memory preserves earlier reasoning states and information, helping agents maintain consistency and improve intermediate states over extended problem-solving horizons.

6.2. Scientific Discovery Agents

Scientific discovery agents combine planning, tool use, retrieval, execution, self-evolution, and multi-agent coordination to support research workflows from hypothesis generation through experimentation and analysis.

  • Scope: Scientific discovery agents couple LLMs with simulators, laboratory automation, and current literature to pursue verifiable, heterogeneous, safety-constrained, long-horizon research goals.Their workflows span hypothesis generation through experimental execution.
  • Foundational capabilities: The foundational layer decomposes research goals, selects tools or simulators, retrieves evidence, and revises plans as new information arrives.Planning, retrieval, and tool use can be integrated into executable scientific workflows.
  • Foundational capabilities: Execution layers connect high-level plans to code, simulations, laboratory hardware, and automated platforms for multi-step scientific workflows.Examples include robotic experiments, atomistic simulations, and wet-lab control scripts.
  • Self-evolving reasoning: Self-evolving agents learn from experience by retaining hypotheses, revising internal representations, and adapting future plans using feedback and reflection.Memory supports reusable scientific knowledge, while measurable feedback can steer proposals and critiques.
  • Collective reasoning: Multi-agent scientific frameworks distribute retrieval, analysis, simulation, and evaluation across specialized roles that coordinate and adjudicate conflicting evidence.ProtAgents illustrates role-based coordination across protein-design subtasks.

6.3. Embodied Agents

Embodied agents ground language reasoning in perception and action, then extend autonomy through memory, feedback, reflection, and multi-robot coordination. The surveyed systems address planning, tool-assisted execution, retrieval, continual adaptation, and collaborative control.

  • Scope: Embodied agents embed LLMs in robotic or simulated bodies to support real-world generalization, continual adaptation, and multimodal grounding.The survey organizes them into foundational, self-evolving, and collective reasoning layers.
  • Foundational capabilities: Embodied planning maps language and multimodal observations to structured subgoals, spatial representations, reasoning traces, or physically grounded transitions.Approaches include skill-affordance estimates, 3D scene graphs, video-derived chains of thought, and reinforcement learning.
  • Foundational capabilities: Tool use and execution translate high-level instructions into perception operations, continuous motor commands, simulations, and physical actions.Systems use skill APIs, retrieval, code, and execution modules for manipulation, navigation, drones, and laboratory-like environments.
  • Foundational capabilities: Search and retrieval ground embodied decisions in semantic maps, 3D scene graphs, prior trajectories, and task-relevant spatial subgraphs.These mechanisms connect language instructions to navigational goals and grounded waypoints.
  • Self-evolving reasoning: Self-evolving embodied agents store experiences, monitor feedback, and reflect on failures to refine plans, policies, skills, and task execution over time.Memory may combine episodic observations, structured long-term semantics, and reusable skills.
  • Collective reasoning: Multi-agent embodied systems divide labor through language-based negotiation, role allocation, cooperative planning, and feedback loops across heterogeneous robots.Theory-of-mind representations and flexible protocols support collaborative coordination.

6.4. Healthcare & Medicine Agents

Healthcare agents extend LLM reasoning across clinical planning, tool use, retrieval, longitudinal adaptation, and multi-agent collaboration. Their workflows must integrate structured evidence while operating under safety, multimodal, and legal constraints.

  • Scope: Healthcare agents support the clinical decision pipeline from symptom triage through treatment planning using patient records, medical ontologies, and expert guidelines.These systems operate under strict safety constraints and require multimodal evidence and legal justification.
  • Foundational capabilities: Clinical planning decomposes long-horizon care into diagnostic and treatment phases while coordinating tools, teams, guidelines, and changing patient conditions.Hierarchical and multi-agent workflows dispatch specialized modules for diagnostic execution.
  • Foundational capabilities: Healthcare tool use expands action spaces through calculators, biomedical databases, medical imaging, code execution, and robotic procedures.Function signatures and nested calls support structured, safer execution of multi-step medical calculations.
  • Foundational capabilities: Search and retrieval connect clinical reasoning to external biomedical sources, knowledge graphs, molecular graphs, assay results, and executable EHR operations.RAG systems can issue targeted knowledge-graph queries to anchor individual reasoning steps.
  • Self-evolving reasoning: Self-evolving healthcare agents preserve patient history and intermediate computations, then revise strategies when new evidence or execution outcomes conflict with prior plans.Memory supports longitudinal coherence, while feedback and reflection provide learning signals.
  • Collective reasoning: Multi-agent healthcare systems coordinate specialists, patients, trial designers, and simulated clinical roles to support consensus, evidence gathering, and diagnostic dialogue.Some frameworks adapt collaboration structures to task complexity, while simulation-based systems generate experience for training.

6.5. Autonomous Web Exploration & Research Agents

Web, GUI, and autonomous research agents form a progression from dynamic evidence retrieval to interface manipulation and end-to-end scientific workflow orchestration. The survey covers planning, tool use, search, retrieval, and reinforcement-learning-based adaptation across these settings.

  • Scope: Web agents retrieve dynamic online evidence, GUI agents manipulate software interfaces, and autonomous research agents orchestrate scientific workflows end to end.The progression differs in the environment acted upon, interaction mode, and goal.
  • Web agents: Web-agent planning decomposes long-horizon tasks, adapts to dynamic pages, and coordinates browser or web-tool invocation strategies.Methods include text-based browsing and multimodal visual-HTML grounding.
  • Web agents: Reinforcement learning improves web-agent planning through online curricula, outcome-supervised rewards, and end-to-end optimization from multi-turn rollouts.WebRL generates tasks from unsuccessful attempts, while WebAgent-R1 learns directly from online interactions.
  • GUI agents: GUI agents plan across applications by decomposing tasks, retrieving experience, and optimizing long action sequences with replay buffers or online interaction.These approaches replace hand-crafted scripts with learned policies for desktop control.
  • Autonomous research agents: Autonomous research agents organize literature review, experimentation, evidence gathering, analysis, and report writing into structured tool-supported itineraries.Representative workflows use staged pipelines or plan → research → write cycles.
  • Tool use: Web and GUI tool-use systems execute browser actions, discover reusable tools from websites, synthesize datasets, and reason over structured application capabilities.Examples span real-site interaction, tool reverse engineering, and UI abstraction.
  • Autonomous research agents: Specialized tools and execution components connect research reasoning to web search, structured memory, code, simulations, laboratory protocols, and runnable implementations.These components feed tool outputs back into the reasoning process and support verifiable scientific outcomes.
  • Search and retrieval: Search systems interleave browsing, multi-query retrieval, uncertainty reduction, and exemplar-trajectory retrieval to support complex web and GUI tasks.Retrieval is used both to ground information seeking and to guide unseen interface actions.

7. Benchmarks

The survey organizes agentic-reasoning benchmarks around mechanism-level capabilities and interactive settings, covering tool use, search, memory, planning, and multi-agent coordination. These benchmarks range from controlled primitive evaluations to more realistic end-to-end interactions.

  • Benchmark organization: Mechanism-centric benchmarks decompose agentic behavior into tool use, search, memory, planning, and multi-agent coordination to support more interpretable evaluation.They complement end-to-end benchmarks that may conflate multiple sources of error.
  • Tool Use: Tool-use benchmarks span single-turn invocation, multi-turn sequencing, realistic user queries, multimodal inputs, and tool retrieval across diverse APIs and tasks.Examples include ToolQA, APIBench, GTA, ToolRet, ToolAlpaca, and SambaNova-ToolBench.
  • Search: Search benchmarks evaluate sequential information acquisition through planning, interaction, evidence synthesis, and multimodal grounding in unimodal and multimodal settings.WebWalker models structured website traversal, while MMSearch and MMSearch-Plus evaluate interactive multimodal retrieval and reasoning.
  • Memory: Memory benchmarks assess both memory management and memory utilization, distinguishing long-horizon episodic memory from multi-session recall.They measure retention, retrieval, reasoning, planning, and feedback over extended or discontinuous interactions.
  • Planning and Feedback: Planning-and-feedback benchmarks test whether agents use historical information to maintain coherent internal states and execute multi-step plans under environmental feedback.ALFWorld evaluates consistency across accumulated observations and actions.
  • Multi-Agent Coordination: Multi-agent benchmarks place systems in interactive environments to evaluate coordination, competition, and decision-making across games, real-world simulations, and language-driven social reasoning.These settings extend evaluation beyond isolated reasoning.

8. Open Problems

The survey identifies open problems spanning personalization, long-horizon planning, world models, multi-agent collaboration, latent reasoning, and governance. These challenges concern adaptation, reliability, interpretability, scalable coordination, and auditing in dynamic environments.

  • User-Centric Agentic Reasoning: User-centric agentic reasoning must adapt to individual users by modeling evolving characteristics, preferences, interaction history, and intent over extended conversations.The user is treated as part of the environment rather than as a fixed task objective.
  • Long-Horizon Interaction and Credit Assignment: Robust long-horizon planning and credit assignment remain difficult because errors compound across extended interactions and existing rewards are heavily engineered and domain-specific.The survey also notes that many reinforcement-learning agents treat episodes independently.
  • World-Model-Based Reasoning: World-model agents support internal simulation and lookahead, but current designs rely on ad hoc representations and short-horizon or environment-specific training data.The survey calls for joint training, updating, and evaluation of world models and agents in non-stationary environments.
  • Multi-Agent Collaboration and Training: Multi-agent collaboration can scale reasoning through specialization and division of labor, yet manually designed structures, poorly understood group-level credit assignment, and population-scale coordination remain challenges.Larger populations introduce topology adaptation, coordination overhead, and safety concerns.
  • Latent Agentic Reasoning: Latent agentic reasoning may improve efficiency and scalability, but reduced interpretability and controllability make alignment, failure diagnosis, and auditing more difficult.Open problems include learning objectives, probing methods, and benchmarks for effective and auditable latent reasoning.
  • Governance: Governance must address risks from long-horizon planning, persistent memory, real-world action, and interactions across components and time.Existing benchmarks and guardrails mainly target short-horizon behavior, leaving planning-time failures and multi-agent dynamics underexplored.
Loading 2601.12538v1…