PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search

Yiqun Chen, Lingyong Yan, Zixuan Yang, Erhan Zhang, Jiashu Zhao, Shuaiqiang Wang, Dawei Yin, Jiaxin Mao

arXiv:2601.04703v1cs.AI

TL;DR

Agentic search is hindered by monolithic architectures that produce long trajectories, rely on sparse feedback, and absorb noisy search context, complicating stable learning. M-ASK separates search behavior from knowledge management and uses turn-level rewards, achieving higher benchmark performance and more stable training across multi-hop QA settings.

  • Problem

    Monolithic agentic-search systems combine planning and information processing, exposing training to unconstrained outputs, sparse rewards, and search noise.

  • Method

    M-ASK assigns search planning and execution to Search Behavior Agents while Knowledge Management Agents filter observations and maintain concise context using turn-specific dense rewards.

  • Results

    M-ASK achieves the highest average F1 score of 50.09 across seven datasets, outperforming monolithic and multi-agent frameworks.

  • Takeaways & Limitations

    Across seven benchmarks, M-ASK consistently outperforms state-of-the-art baselines and improves training stability, particularly for complex multi-hop scenarios.

  • Takeaways & Limitations

    The evaluation is confined to textual QA, while the framework’s generalizability to code generation and multimodal reasoning remains unexplored.

Abstract

from arXiv · show

Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use. However, prevailing systems rely on monolithic agents that suffer from structural bottlenecks, including unconstrained reasoning outputs that inflate trajectories, sparse outcome-level rewards that complicate credit assignment, and stochastic search noise that destabilizes learning. To address these challenges, we propose \textbf{M-ASK} (Multi-Agent Search and Knowledge), a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context. This decomposition allows each agent to focus on a well-defined subtask and reduces interference between search and context construction. Furthermore, to enable stable coordination, M-ASK employs turn-level rewards to provide granular supervision for both search decisions and knowledge updates. Experiments on multi-hop QA benchmarks demonstrate that M-ASK outperforms strong baselines, achieving not only superior answer accuracy but also significantly more stable training dynamics.\footnote{The source code for M-ASK is available at https://github.com/chenyiqun/M-ASK.}

1 Introduction

Agentic search extends LLM-based retrieval through iterative planning, tool use, and synthesis, but monolithic agents face compounding problems from verbose trajectories, sparse rewards, and search noise. M-ASK addresses these issues by separating search behavior from knowledge management and applying turn-specific dense rewards.

  • Agentic search enables LLMs to plan iteratively, query external tools, and synthesize information for complex multi-hop needs.
  • Monolithic agents combine trajectory planning and information processing in one continuous response, creating unconstrained output length, sparse rewards, and search noise.
  • These obstacles compound into a long-horizon credit-assignment problem in which final rewards are difficult to attribute to distant actions or tokens.
  • M-ASK separates Search Behavior Agents, which plan and execute search, from Knowledge Management Agents, which filter observations and maintain concise internal knowledge.
  • Turn-specific dense rewards jointly supervise search and knowledge updates, helping stabilize state management and improve planning under noisy, long-horizon search.

2 Related Work

Related work moves from static retrieval toward dynamic agentic search, evolving context, and multi-agent role specialization. M-ASK combines these directions while targeting limitations in joint optimization and credit assignment.

  • Retrieval-augmented generation has progressed from static retrieve-then-read pipelines to iterative systems in which LLMs decide when to retrieve.
  • Agentic search extends iterative retrieval into multi-step trajectories that use external tools to solve open-ended problems.
  • Dynamic context methods include uncertainty-based filtering, explicit memory agents, and evolving-state approaches, but prior systems lack either joint end-to-end optimization or fine-grained credit assignment.
  • Multi-agent frameworks decompose complex tasks into specialized roles, with related information-retrieval systems applying planners and web-search agents.

3 Method

M-ASK decomposes agentic search into specialized search-behavior and knowledge-management roles that coordinate through a shared, evolving knowledge state. Its workflow alternates search, evidence filtering, state updates, and answer evaluation, while turn-level rewards supervise incremental progress.

  • Framework design: M-ASK decouples search planning and execution from information integration through specialized Search Behavior and Knowledge Management Agents.Search Behavior Agents handle planning, search, and answering; Knowledge Management Agents filter evidence and maintain the state.
  • Problem formulation: Agents communicate across turns by reading and writing a shared structured knowledge state containing an evolving reasoning trajectory.Each trajectory step records a sub-query and associated evidence, allowing the answer to be updated as multi-hop search progresses.
  • Agent roles: The Planning Agent initializes the trajectory and preliminary answer, after which the Search Agent either issues a sub-query or terminates the search.Termination occurs when the Search Agent outputs <end> or the maximum step limit is reached.
  • Knowledge management: For each retrieved sub-query, the Summary Agent extracts pertinent evidence while the Update Agent refines an existing step or appends a necessary new reasoning step.The Update Agent's two operations balance precision against information growth by replacing vague or hallucinated content or adding a distinct logical hop.
  • Turn-level training: The cooperative loop is optimized with Independent PPO, and termination receives zero reward so the team stops only when further search yields no positive gain.The shared incremental signal jointly incentivizes the Search, Summary, and Update Agents to maximize each turn's marginal information gain.
  • Turn-level training: During training, the Answer Agent evaluates every intermediate state, while search and knowledge agents share incremental rewards based on marginal answer-quality gains.Absolute F1 rewards supervise state quality for planning and answering, whereas shared iteration rewards coordinate search, summarization, and updating.

4 Experiments

M-ASK is evaluated across seven QA benchmarks and shows higher average accuracy, stronger multi-hop performance, and more stable training than competing frameworks. Ablations indicate that knowledge management and turn-specific rewards are especially important for complex reasoning.

  • Main Results: M-ASK achieves the highest average F1 score, 50.09, across seven datasets, outperforming both monolithic and multi-agent frameworks.The evaluation covers single-hop and multi-hop QA benchmarks.
  • Main Results: On HotpotQA, M-ASK surpasses the best baseline by +5.82 and generalizes to unseen datasets including 2Wiki and Musique.The paper reports consistent gains over other Agentic baselines on these out-of-domain datasets.
  • Main Results: M-ASK exceeds DeepNote by +3.30 through end-to-end joint optimization of Search and Knowledge Management agents.DeepNote achieves 46.79, while M-ASK further improves the modular framework by jointly updating both roles.
  • Training Stability: Search-r1’s collapse rate rises from 10% at 200 steps to 90% at 1000 steps, whereas M-ASK maintains a 0% collapse rate.The comparison uses 10 independent training runs and defines collapse as performance dropping near zero without recovery.
  • Ablation Studies: Removing knowledge management reduces performance by avg. ∆−2.86% on multi-hop benchmarks, including −6.81% on Musique and −3.25% on HotpotQA.The authors attribute these drops to retrieval noise overwhelming the search agent and conclude that explicit knowledge management stabilizes reasoning.
  • Ablation Studies: Replacing turn-specific rewards with global outcome rewards causes avg. ∆−15.23% degradation on multi-hop datasets, compared with avg. ∆−3.91% on single-hop tasks.The paper links the larger multi-hop decline to ambiguous credit assignment across long trajectories.

5 Conclusion

M-ASK addresses instability in monolithic agentic search by decoupling search planning from knowledge management and using turn-specific dense rewards. Across seven benchmarks, it consistently outperforms state-of-the-art baselines, especially in complex multi-hop scenarios, while improving training stability.

  • M-ASK decouples search planning from knowledge management through collaborative multi-agent roles and turn-specific dense rewards.The framework separates search behavior from information integration to provide intermediate supervision.
  • M-ASK consistently outperforms state-of-the-art baselines across seven benchmarks, particularly in complex multi-hop scenarios.
  • Decoupling search and knowledge-management roles significantly enhances training stability compared with end-to-end reinforcement-learning approaches.
  • Future work will extend M-ASK to heterogeneous model architectures and broader tool-use scenarios beyond information retrieval.

Limitations

M-ASK’s multi-agent workflow increases inference frequency and computational cost, while its current evaluation is limited to textual question answering and its parameter-sharing strategy may depend on backbone-model capacity.

  • The collaborative workflow requires discrete inference calls for search, summarization, and updating, increasing computational costs and potentially challenging latency-sensitive applications.
  • Evaluation is confined to textual QA, leaving generalizability to code generation and multimodal reasoning unexplored.
  • Parameter sharing relies on the backbone LLM’s capacity to follow diverse role instructions, warranting further study on smaller architectures.

Ethics Statement

The work applies LLM-based multi-agent collaboration to textual question answering using publicly available datasets without personally identifiable information. Its parameter-sharing design reduces resource requirements, but LLM outputs may still contain bias or incorrect information.

  • The framework uses LLM-based multi-agent collaboration for textual question answering, with publicly available datasets containing no personally identifiable information.
  • LLM outputs may reflect pretraining-data biases or produce incorrect information, requiring caution in real-world applications where harm is possible.
  • A shared parameter set supports the framework’s functional agents, while parameter sharing can reduce storage requirements from O(N) to O(1).

3. Inherent Multi-Task Capability of LLMs

M-ASK uses LLMs’ multi-task capabilities to specialize agents through role-specific prompts while sharing the same underlying model parameters. This provides role specialization without duplicating the model architecture.

  • LLMs can perform distinct tasks through contextual instructions without modifying their internal weights.
  • M-ASK uses role-specific prompts to specialize a shared model across search and summarization functions.
  • Shared representations allow capabilities learned in summarization to support answer generation without architectural redundancy.

B Implementation Details of Baselines

The paper evaluates M-ASK against closed-book, RAG, RL-based, and adaptive agentic-search baselines under a unified Qwen2.5-7B-Instruct setting. Baseline implementations include specialized training procedures, workflow configurations, and strict output formats for query decomposition, search, evidence extraction, updates, and answer generation.

  • Baseline Categories: The comparison covers standard baselines, static modular RL systems, and adaptive agentic-search methods, including Search-r1, DeepNote, and MAO-ARAG.The baselines span closed-book and Vanilla RAG systems, query reformulation, document selection, multi-agent RL, workflow selection, planner-executor, knowledge-management, and monolithic agent designs.
  • Unified Experimental Setting: All baselines and M-ASK use Qwen2.5-7B-Instruct, with pretrained components retained and trainable components fine-tuned from the same initialization.This unified setting is intended to make the comparison consistent across methods with different training requirements.
  • RL-Based Baselines: RRR trains a query rewriter with PPO and fine-tunes answer generation, while BGM trains document selection with PPO and likewise fine-tunes answer generation.Both reproductions adjust generator training to avoid performance bottlenecks and align the comparison settings.
  • RL-Based Baselines: MMOA-RAG trains query-rewriting, document-selection, and answer-generation agents with MAPPO under a shared final reward.Its reproduction follows the original logic while using Qwen2.5-7B-Instruct as the backbone.
  • Adaptive and Knowledge-Management Baselines: Adaptive RAG trains a classifier to select Directly Answer, Simple RAG, or Iterative RAG, whereas MAO-ARAG trains a planner over pretrained executors.DeepNote instead optimizes knowledge management with DPO and uses upgraded gpt-4o-generated data for reproduction.
  • Adaptive and Knowledge-Management Baselines: Search-r1 uses a monolithic multi-turn search-and-reasoning response optimized with outcome-based RL rewards.The Search-r1 results were obtained from the authors’ official open-source repository.
  • Prompt and Workflow Specifications: The implementation prompts enforce structured outputs for decomposition, search, evidence extraction, updates, and final answer generation.Search queries must avoid duplicates or semantically similar prior queries, while update detection either replaces an existing trajectory item or adds a new one.

D Detailed Agent Specifications

This section specifies M-ASK’s multi-agent architecture and its structured knowledge state. The framework represents multi-hop reasoning through decomposed sub-questions, retrieved evidence, and an evolving trajectory supporting the final answer.

  • Agent Architecture: M-ASK defines detailed roles, input-output interfaces, and action spaces for Search Behavior Agents and Knowledge Management Agents.The specifications clarify how each agent participates in the iterative retrieval process.
  • Structured Knowledge State: The structured knowledge state K_t includes a trajectory of sub-query and sub-answer pairs that logically support the final answer.The trajectory is dynamically modified as multi-round search proceeds, updating the answer from the evolving evidence chain.

F Joint Training Algorithm Details

M-ASK’s joint training uses one shared policy across functional roles and separates data collection from unified optimization. Intermediate answer evaluation supplies dense, turn-specific supervision during interaction.

  • Parameter Sharing: A unified policy πθ is shared across Planning, Search, Summary, Update, and Answer roles through role-specific instructions.The shared-parameter strategy is executed across all functional agents rather than using separate model parameters for each role.
  • Training Procedure: Training proceeds in two phases: sequential data collection followed by joint PPO optimization over experiences stored in a mixed replay buffer.The shared model parameters θ and critic ϕ are updated using heterogeneous interaction trajectories.
  • Turn-Level Supervision: At each step t, the Answer Agent computes marginal performance gain ΔF1 as an intermediate evaluator, providing dense turn-specific supervision.This evaluator role occurs during training, even though the Answer Agent generates the final output only after search terminates during inference.

G Detailed Case Study: Structured Knowledge State Evolution

The case study shows how M-ASK evolves a structured knowledge state through correction, evidence-chain expansion, and in-place refinement. These operations replace hallucinated information, verify temporal context, and prevent redundant context growth.

  • Structured Knowledge State Evolution: M-ASK overwrites the initial hallucinated “Guadalajara” with retrieved evidence identifying “Puebla” during the first update.The Update Agent treats the initial plan as a search target rather than ground truth, preventing the error from propagating.
  • Structured Knowledge State Evolution: The second turn adds historical evidence about the league’s professionalization in 1943, forming an evidence chain from entity existence to historical context.The new item is appended as t2 because it supplies distinct background information supporting the answer.
  • Structured Knowledge State Evolution: The third turn refines t1 with more precise evidence linking Puebla to the Athletic Club instead of creating a redundant t3.In-place refinement keeps K3 concise and produces a denser, less noisy reasoning state.
  • Case-Study Representation: Table 6 records the execution log and indexes trajectory steps so the Update Agent can target specific entries such as τ1.The state rows expose the three components of K_t for the case-study query.
Loading 2601.04703v1…