PaperlayerView exact revision on arXiv

Source-linked AI summary

WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning

Zelai Xu, Zhexuan Xu, Ruize Zhang, Chunyang Zhu, Shi Yu, Weilin Liu, Quanlu Zhang, Wenbo Ding, Chao Yu, Yu Wang

arXiv:2602.04634v3cs.AIcs.LGcs.MA

TL;DR

Broad information-seeking tasks expose limits of depth-focused, single-agent systems and existing multi-agent workflows. WideSeek-R1 addresses this with a MARL-trained lead-agent–subagent framework using shared-model agents, isolated contexts, and parallel execution. WideSeek-R1-4B achieves 40.0% item F1 on WideSearch, comparable to DeepSeek-R1-671B, while performance rises consistently with more parallel subagents.

  • Problem

    Broad information-seeking tasks require organizational capability, but existing systems rely on sequential single-agent execution or hand-crafted, non-scalable multi-agent workflows.

  • Method

    WideSeek-R1 jointly trains a lead agent and parallel subagents with MARL using a shared LLM, isolated contexts, specialized tools, and a 20k-task dataset.

  • Results

    40.0% item F1 on WideSearch is achieved by WideSeek-R1-4B, comparable to single-agent DeepSeek-R1-671B, with continuous gains as parallel subagents increase.

  • Takeaways & Limitations

    Width scaling provides a complementary route to broad information seeking, with parallel-agent performance improving as the number of subagents increases.

  • Takeaways & Limitations

    Shared final-outcome rewards create coarse-grained credit assignment, making it difficult to distinguish lead-agent orchestration failures from subagent execution errors.

Abstract

from arXiv · show

Recent advancements in Large Language Models (LLMs) have largely focused on depth scaling, where a single agent solves long-horizon problems with multi-turn reasoning and tool use. However, as tasks grow broader, the key bottleneck shifts from individual competence to organizational capability. In this work, we explore a complementary dimension of width scaling with multi-agent systems to address broad information seeking. Existing multi-agent systems often rely on hand-crafted workflows and turn-taking interactions that fail to parallelize work effectively. To bridge this gap, we propose WideSeek-R1, a lead-agent-subagent framework trained via multi-agent reinforcement learning (MARL) to synergize scalable orchestration and parallel execution. By utilizing a shared LLM with isolated contexts and specialized tools, WideSeek-R1 jointly optimizes the lead agent and parallel subagents on a curated dataset of 20k broad information-seeking tasks. Extensive experiments show that WideSeek-R1-4B achieves an item F1 score of 40.0% on the WideSearch benchmark, which is comparable to the performance of single-agent DeepSeek-R1-671B. Furthermore, WideSeek-R1-4B exhibits consistent performance gains as the number of parallel subagents increases, highlighting the effectiveness of width scaling.

1. Introduction

Broad information-seeking tasks shift the bottleneck from individual competence to organizational capability, motivating width scaling through parallel multi-agent execution. WideSeek-R1 uses MARL to learn scalable orchestration and parallel information seeking, with experiments showing strong benchmark performance and gains from more subagents.

  • Motivation: Broad information seeking requires synthesizing attributes of multiple entities into structured tables, unlike deep research focused on one complex query.Single-agent systems face context pollution and sequential execution in these broad tasks.
  • Motivation: Existing multi-agent systems often use hand-crafted workflows and turn-taking interactions that limit scalable coordination and parallel execution.The paper identifies shortcomings at both orchestration and execution levels.
  • Approach: WideSeek-R1 is a lead-agent–subagent system trained via MARL to jointly learn scalable orchestration and parallel information seeking.A shared LLM uses isolated contexts and specialized tools for the lead agent and subagents.
  • Results: 40.0% item F1 on WideSearch is achieved by WideSeek-R1-4B, comparable to single-agent DeepSeek-R1-671B.The model also significantly outperforms multi-agent 8B baselines.
  • Results: WideSeek-R1-4B exhibits continuous performance gains as the number of parallel subagents increases.The experiments also evaluate depth scaling, standard QA benchmarks, and ablations of agents and training data.
  • Contributions: The work releases a 20k-task broad information-seeking dataset as a training resource complementary to existing multi-hop datasets.The dataset supports MARL training for broad information-seeking behavior.

2. Related Work

Prior LLM scaling emphasizes depth through more sequential reasoning and tool use, while multi-agent reinforcement learning and scalable width remain less explored. WideSeek-R1 differs by jointly optimizing shared-model lead and subagents rather than relying on separate or partially trained agents.

  • Scaling Dimensions in LLMs: Training-time scaling increases parameters, data, and compute, whereas test-time scaling allocates additional inference-time computation.Depth scaling is a prominent test-time approach for reasoning and agentic models.
  • Search Agents and Systems: Single-agent search systems optimize multi-turn tool use effectively for deep multihop queries but struggle with context pollution and sequential execution in broad tasks.Multi-agent frameworks are introduced to address these bottlenecks.
  • Agentic RL for LLMs: Multi-agent reinforcement learning for LLMs remains relatively underexplored across distributed self-play and hierarchical role-based systems.Existing work includes both fully distributed and hierarchical approaches.
  • Agentic RL for LLMs: WideSeek-R1 jointly optimizes the lead agent and subagents with a shared model, unlike prior systems that train only some agents or use separate models.This design targets learned coordination across the hierarchy.

3. WIDESEEK-R1

WideSeek-R1 combines a trained lead agent with parallel, isolated-context subagents and jointly optimizes them through end-to-end MARL. Its training extends GRPO with shared multi-agent advantages and token- and agent-level reweighting.

  • Rollout Architecture: The lead agent decomposes broad tasks into parallelizable subtasks and delegates them to subagents through learned orchestration.Unlike hand-crafted workflows, the orchestration is trained to remain scalable as the number of subagents changes.
  • Rollout Architecture: Subagents execute assigned subtasks in parallel within isolated contexts using search and access tools, then return findings to the lead agent.This design addresses context pollution and sequential execution bottlenecks.
  • Multi-Agent Reinforcement Learning: End-to-end MARL jointly trains orchestration and information-seeking behaviors with a shared model.The method builds on GRPO and introduces multi-agent advantage assignment and dual-level advantage reweighting.
  • Multi-Agent Reinforcement Learning: A group-normalized, verifiable outcome advantage is shared across agents and tokens within each multi-agent rollout.This provides a common learning signal without complex credit assignment.
  • Multi-Agent Reinforcement Learning: Token-level reweighting averages advantages across an agent’s tokens and turns, while agent-level reweighting averages across agents in each rollout.Agent-level averaging prevents rollouts with many subagents from dominating the gradient and rewards added agents only when final reward improves.

4. Training Data Construction

WideSeek-R1 constructs broad information-seeking training data through automated query generation, answer generation, and quality filtering. The resulting schema-constrained samples are designed to provide reliable, sufficiently difficult data for training.

  • Motivation: Existing QA resources are largely designed for depth scaling, while broad-information benchmarks are smaller and often manually annotated.These constraints limit their suitability for data-intensive MARL training.
  • Pipeline Overview: The automated pipeline synthesizes schema-constrained queries and standardized tabular outputs for broad information seeking.It is introduced to address scale and annotation limitations in existing resources.
  • Query Generation: Query generation extracts user intent from HybridQA, samples target row counts, and refines queries with broad-coverage and formatting constraints.The target row count is stochastically varied between 10 and 50 to diversify training data.
  • Answer Generation: Answer generation produces two independent tabular responses and identifies the minimal unique column set needed to distinguish rows.The duplicated responses support later consistency verification.
  • QA Pair Filtering: QA filtering removes samples below a factual-consistency threshold of 0.9 and tables with fewer than 3 rows.Only samples passing both consistency and complexity filters are retained.
  • Evaluation Context: WideSeek-R1-4B outperforms all 4B and 8B baselines on five of six WideSearch metrics and is comparable to DeepSeek-R1-671B with nearly 170× fewer parameters.The table reports Avg@4 and Max@4 for Item and Row F1, plus Avg@4 and Pass@4 for Success Rate.

5. Experiments

Experiments evaluate WideSeek-R1-4B on broad information seeking, scaling behavior, role and data ablations, and standard QA. The results show strong benchmark performance, gains from MARL and width scaling, and benefits from jointly optimizing agents with hybrid training data.

  • 5.1. Main Results: WideSeek-R1-4B achieves the best results on five of six metrics among 4B and 8B baselines, including an 11.9% absolute item F1 improvement over the single-agent variant.Against the base Qwen3-4B in the same multi-agent setting, it improves item F1 by 8.8%.
  • 5.1. Main Results: WideSeek-R1-4B performs comparably to single-agent DeepSeek-R1-671B despite using nearly 170× fewer parameters.The evaluation uses the WideSearch benchmark for broad information-seeking tasks.
  • 5.2. Exploring Width Scaling: Width scaling with WideSeek-R1-4B produces continuous gains as parallel subagents increase, whereas depth scaling quickly plateaus.The base model’s width-scaling performance initially improves but declines when the number of subagents reaches ten.
  • 5.3. Ablation Studies: Assigning WideSeek-R1-4B to both the lead agent and subagents yields the best performance, with further gains from combining both roles.Upgrading either role provides comparable item F1 gains, while subagent assignment produces higher row F1 and Success Rate than lead-only assignment.
  • 5.3. Ablation Studies: Hybrid training data consistently outperforms wide-only and deep-only data across item F1, row F1, and Success Rate.The compared settings use equal total sample counts and otherwise constant training parameters; the hybrid dataset combines both data types in equal proportions.
  • 5.4. Standard QA Benchmarks: WideSeek-R1-4B achieves a 59.0% average score across seven single-hop and multi-hop QA benchmarks and surpasses larger OWL-8B and MiroFlow-8B systems.It also gains 7.7% over multi-agent Qwen3-4B and 2.0% over SingleSeek-R1-4B.

6. Conclusion

The paper presents width scaling through a MARL-trained lead-agent–subagent framework for broad information seeking. WideSeek-R1-4B matches DeepSeek-R1-671B comparably on WideSearch, gains with more parallel subagents, and benefits from joint agent optimization and curated training data.

  • 6. Conclusion: WideSeek-R1 explores width scaling as a complement to depth scaling by combining scalable orchestration with parallel execution.The framework targets broad information-seeking tasks through a MARL-trained multi-agent system.
  • 6. Conclusion: WideSeek-R1-4B achieves performance comparable to single-agent DeepSeek-R1-671B on the WideSearch benchmark.The comparison is presented as evidence of width scaling for broad information seeking.
  • 6. Conclusion: Depth scaling faces diminishing returns, whereas width scaling shows consistent performance gains as the number of parallel subagents increases.The paper also releases a curated 20k dataset for future research on scalable multi-agent systems.

Impact Statement

WIDESEEK-R1 uses width scaling through multi-agent collaboration, enabling smaller models to approach the performance of much larger single models while raising deployment risks.

  • Impact Statement: Parallelized small agents can rival giant single models, motivating exploration of collaborative architectures beyond simple parameter scaling.
  • Impact Statement: 4B parameters enable performance comparable to models exceeding 600B parameters, lowering computational barriers to advanced reasoning capabilities.The paper frames this as democratizing access for researchers and organizations with limited compute resources.
  • Impact Statement: Autonomous agent swarms could be misused for scalable automated data gathering or misinformation generation.The paper identifies robust safety guardrails and usage policies as necessary for responsible deployment.

A. Limitation

The paper identifies computational, credit-assignment, hierarchy, and rollout-efficiency constraints, while describing design choices used to stabilize training and control agent behavior.

  • Model Size: Approximately 3,000 GPU hours on H100 GPUs are required to train the 4B-parameter model, creating substantial computational cost.
  • Credit Assignment: Shared final-outcome rewards make it difficult to distinguish lead-agent orchestration failures from subagent execution errors.The authors suggest more granular, role-specific reward modeling as future work.
  • Nested Hierarchy: The system restricts training to a static two-layer hierarchy because recursive delegation severely destabilizes MARL training.This limits autonomous organizational deepening for unexpectedly complex subtasks.
  • Training Efficiency: Nearly 90% of training step time is dominated by rollout, increasing latency under collocated reinforcement-learning training.The paper proposes asynchronous rollout or decoupled generation and training as possible efficiency improvements.
  • Training Procedure: Filtering excludes trajectories that exceed context or turn limits and penalizes invalid formatting, repetition, and other rollout failure modes.

C.1. Dataset Statistics

The constructed dataset contains 20,000 structured information-seeking instances spanning varied table sizes, with JSON fields and metrics designed to evaluate increasingly complete retrieval accuracy.

  • Dataset Statistics: 20,000 instances form the constructed dataset, whose row counts span a broad spectrum with a median of 30.Column counts cluster around a median of 6, providing variation in retrieval complexity.
  • Dataset Construction: 73.28% of synthesized instances were retained after Stage 3 filtering, at an approximate generation cost of $0.10 per instance.Gemini-3-pro-preview was used for synthesis after GPT-4 and Claude-4-sonnet showed suboptimal performance for this task.
  • Data Format: Each JSON instance contains a question, answer, and unique_columns field.The question specifies table constraints, the answer is a Markdown table, and unique columns align predicted rows with ground truth despite row permutations.
  • Evaluation Metrics: Item F1, Row F1, and Success Rate measure cell-level, row-level, and complete-table correctness, respectively.

D. Training Detail

Training used Qwen3-4B with specified hardware, optimization, rollout, parallelism, and hybrid-batching settings for WIDESEEK-R1 and its single-agent baseline.

  • Training Setup: WIDESEEK-R1-4B was trained on 32 H100 GPUs, while SingleSeek-R1-4B used 16 H100 GPUs.Both experiments used Qwen3-4B with thinking mode enabled, batch size 128, 32K maximum context, and 150 training steps.
  • Infrastructure and Optimization: RLinf, SGLang, and Megatron supported rollout and optimization, using a learning rate of 1 × 10^-6 and specified sampling and clipping parameters.
  • Agent Configuration: WIDESEEK-R1-4B allowed up to 10 parallel subagents per turn, with 10 lead-agent turns and 20 turns per subagent.Parallel search and access tool calls were capped at 5 per turn.
  • Data Mixing: Hybrid-dataset training used batches containing 64 deep samples and 64 wide samples to improve training stability.
  • Baselines: The benchmark compares WIDESEEK-R1 with representative single-agent systems and state-of-the-art multi-agent frameworks.

E.1. Benchmark

The benchmark section evaluates broad information seeking on WideSearch and compares WIDESEEK-R1 with single-agent and multi-agent baselines under specified configurations and tool settings.

  • WideSearch: WideSearch contains 200 questions, balanced between 100 English and 100 Chinese queries, and evaluates structured multi-entity information seeking.The benchmark reports Item F1, Row F1, and Success Rate, with Avg@4 and Max@4 or Pass@4 variants.
  • Evaluation setup: The evaluation uses gpt-4.1-2025-04-14 as the LLM judge and the official WideSearch evaluation code.WideSearch evaluation uses the standard online Serper2 search and Jina access tools.
  • Standard QA: Standard QA evaluation covers seven datasets divided into single-hop and multi-hop reasoning groups.The single-hop group includes NQ, TriviaQA, and PopQA; the multi-hop group includes 2WikiMultiHopQA, HotpotQA, Bamboogle, and MuSiQue.
  • Single-agent baselines: SingleSeek-R1-4B is trained on Qwen3-4B with broad information-seeking and standard QA data combined in a 1:1 ratio.It uses search and access tools, permits 50 turns, and allows at most one tool call per turn.

F. Pattern Analysis

The pattern analysis describes a trained lead agent that decomposes broad tasks and delegates independent work in parallel, alongside behavioral changes associated with MARL training.

  • Behavioral metrics: WIDESEEK-R1-4B produces nearly 4× more total turns than the base model, including about 1.6× more lead-agent turns and 2.1× more subagent turns.The analysis associates these increases with deeper interaction and more tool use after MARL training.
  • Behavioral metrics: WIDESEEK-R1-4B spawns roughly 2× more subagents than the base model and raises access calls from 33.8% to 46.9%.The paper characterizes increased access use as a way to gather richer evidence beyond brief search snippets.
  • Subagent execution: Subagents are instructed to focus on assigned subtasks and can perform parallel search and access calls without sequential dependencies.Search discovers webpages and access retrieves more detailed content from URLs returned by search.
  • Example workflow: The demonstrated workflow first delegates entity-list discovery, then launches parallel subtasks to research attributes for the identified entities.The example processes batches of individuals across multiple turns, reflecting staged decomposition with a maximum of 10 parallel subtasks at a time.
  • Example workflow: For the Ivy League example, the lead agent first identifies the universities and then delegates city and founding-year research for each school.The example illustrates that later subtasks depend on information returned from an earlier delegation step.
Loading 2602.04634v3…