PaperlayerView exact revision on arXiv

Source-linked AI summary

Web2BigTable: A Bi-Level Multi-Agent LLM System for Internet-Scale Information Search and Extraction

Yuxuan Huang, Yihang Chen, Zhiyuan He, Yuxiang Chen, Ka Yiu Lee, Huichi Zhou, Weilin Luo, Meng Fang, Jun Wang

arXiv:2604.27221v1cs.AI

TL;DR

Agentic web search must handle both deep, coherent reasoning over one target and broad, verified aggregation across many entities. Web2BigTable addresses both with a bi-level, memory-mediated multi-agent framework and achieves state-of-the-art WideSearch performance while reaching 73.0 accuracy on XBench-DeepSearch.

  • Problem

    Existing systems remain limited in both deep search requiring long coherent reasoning and wide search requiring broad, verified, mutually consistent coverage.

  • Method

    Web2BigTable uses an upper-level orchestrator and parallel workers coordinated through persistent human-readable memory and a shared workboard.

  • Results

    Web2BigTable achieves state-of-the-art WideSearch performance and generalises to depth-oriented XBench-DeepSearch with 73.0 accuracy.

  • Takeaways & Limitations

    The framework supports both breadth-oriented and depth-oriented web-to-table search through memory-mediated coordination and adaptation.

Abstract

from arXiv · show

Agentic web search increasingly faces two distinct demands: deep reasoning over a single target, and structured aggregation across many entities and heterogeneous sources. Current systems struggle on both fronts. Breadth-oriented tasks demand schema-aligned outputs with wide coverage and cross-entity consistency, while depth-oriented tasks require coherent reasoning over long, branching search trajectories. We introduce \textbf{Web2BigTable}, a multi-agent framework for web-to-table search that supports both regimes. Web2BigTable adopts a bi-level architecture in which an upper-level orchestrator decomposes the task into sub-problems and lower-level worker agents solve them in parallel. Through a closed-loop run--verify--reflect process, the framework jointly improves decomposition and execution over time via persistent, human-readable external memory, with self-evolving updates to each single-agent. During execution, workers coordinate through a shared workspace that makes partial findings visible, allowing them to reduce redundant exploration, reconcile conflicting evidence, and adapt to emerging coverage gaps. Web2BigTable sets a new state of the art on WideSearch, reaching an Avg@4 Success Rate of \textbf{38.50} ($7.5\times$ the second best at 5.10), Row F1 of \textbf{63.53} (+25.03 over the second best), and Item F1 of \textbf{80.12} (+14.42 over the second best). It also generalises to depth-oriented search on XBench-DeepSearch, achieving 73.0 accuracy. Code is available at https://github.com/web2bigtable/web2bigtable.

1 Introduction

Web2BigTable is a bi-level multi-agent framework for web-to-table search that addresses both breadth-oriented aggregation and depth-oriented reasoning. It combines adaptive decomposition, parallel workers, persistent external memory, and asynchronous coordination, achieving state-of-the-art WideSearch results and 73.0 accuracy on XBench-DeepSearch.

  • Framework: Web2BigTable supports both breadth-oriented and depth-oriented web-to-table search through an upper-level orchestrator and parallel lower-level worker agents.The task produces schema-aligned tables from open-web sources, with distinct entities as rows, requested attributes as columns, and independently verified cells.
  • Adaptation: Persistent, human-readable external memory enables a closed-loop run-verify-reflect process that jointly improves task decomposition and worker execution over time.The orchestrator accumulates decomposition skills, while workers acquire reusable skills for retrieval, evidence verification, and intermediate synthesis.
  • Results: 38.50 Avg@4 Success Rate on WideSearch is 7.5× the second best at 5.10, with Row F1 of 63.53 (+25.03) and Item F1 of 80.12 (+14.42).These results establish state-of-the-art performance on the broad-coverage benchmark.
  • Results: 73.0 accuracy on XBench-DeepSearch demonstrates generalisation to depth-oriented search.The benchmark evaluates the structurally distinct deep-search regime, which requires long, coherent reasoning over indirect clues.
  • Coordination: Asynchronous coordination lets parallel workers share progress, reduce redundant exploration, reconcile conflicting evidence, and respond to emerging coverage gaps.Workers coordinate through a shared workspace during search, making partial findings visible across agents.

2 Web2BigTable: A Bi-Level Memory-Based Framework

Web2BigTable addresses the breadth and structured-evidence demands of web-to-table search with a bi-level multi-agent architecture. It separates persistent skill memory from transient workboard coordination, enabling orchestration, parallel execution, and shared-state collaboration while keeping the underlying LLMs frozen.

  • Self-evolution: Persistent skill banks evolve through automated Run–Verify–Reflect updates, while the underlying language models remain frozen.The banks store human-readable, generalised procedures that accumulate monotonically across runs.
  • Multi-agent coordination: The shared workboard gives concurrent workers globally visible progress and partial evidence, helping them avoid redundant exploration and reconcile conflicting retrievals.It is a structured Markdown document containing a task checklist, worker slots, and globally readable execution state.
  • Task formulation: Web-to-table search requires structured tables covering many entities, with every cell anchored in live web evidence, making breadth of coverage central to success.A single query may span hundreds of rows, unlike deep question answering, which typically converges on one free-form response.
  • Bi-level architecture: The framework factorises a monolithic policy into an upper-level orchestrator and lower-level workers that independently produce partial outputs under a shared workboard.The final table is assembled from worker outputs, while each worker reads the workboard slice relevant to its subtask.
  • Memory design: Web2BigTable uses long-term semantic memory for orchestrator and worker skills, and short-term working memory for transient within-episode coordination.The skill banks So and Sw persist across training episodes, whereas the workboard me is discarded when an episode terminates.

3 Experiments

Web2BigTable is evaluated on complementary breadth-oriented structured extraction and depth-oriented multi-hop reasoning benchmarks, using benchmark-specific training and held-out test protocols. Ablations show that learned orchestration, shared workboard coordination, and worker skill evolution each contribute to performance, while the full framework achieves strong results on both tasks.

  • Benchmarks: WideSearch tests large-scale structured extraction across 200 tasks, spanning 100 English and 100 Chinese tasks in 15 domains.Each task requires multi-dimensional atomic data to be organised into a structured table.
  • Benchmarks: XBench-DeepSearch evaluates depth through multi-hop reasoning, cross-source verification, and precise extraction from dynamic web content.The benchmark complements WideSearch’s emphasis on breadth with professionally annotated Chinese tasks.
  • Ablation analysis: 38.50 to 7.00: ablating learned decomposition skills drops WideSearch Success Rate, while XBench-DeepSearch accuracy falls from 73.0 to 41.0.The ablation also reduces WideSearch Row F1 from 63.53 to 45.23 and Item F1 from 80.12 to 62.87.
  • Ablation analysis: 63.53 to 54.81: disabling the shared workboard reduces WideSearch Row F1, while XBench-DeepSearch accuracy declines from 73.0 to 60.0.Without shared memory, workers cannot share sources, inspect peers’ findings, or support orchestrator-triggered follow-up iterations.
  • Ablation analysis: 59.67 Row F1 and 64.0 accuracy: removing worker skill evolution causes more modest declines on WideSearch and XBench-DeepSearch, respectively.Workers lose the ability to discover task-specific cloud skills or autonomously repair execution failures.
  • Main results: 73.0 accuracy: Web2BigTable surpasses all reported XBench-DeepSearch baselines, including Minimax-M2 and MiroFlow at 72.0.The result supports generalisation of the bi-level framework to depth-oriented multi-hop reasoning.

1. User Query (ws_en_006)

The query requests a complete chronological table of every concert on Taylor Swift’s official tours from January 1, 2010, through May 1, 2025. Each show must occupy its own row with specified date, concert, country, city, and venue fields.

  • The task covers every concert on Taylor Swift’s official tours from Jan 1, 2010 to May 1, 2025.
  • Each concert must appear on its own row in chronological order.
  • The required columns are Date, Concert Name, Host Country, Host City, and Host Venue.

2. Baseline Decomposition (without learned orchestrator skills)

Without learned orchestrator skills, splitting the search by time period mixes multiple tours within each worker’s scope and overloads broad windows, producing noisy retrieval and incomplete coverage. The baseline retrieves 234 of 534 rows.

  • 2. Baseline Decomposition (without learned orchestrator skills): 234 / 534 rows were retrieved using a default LLM strategy that split the task into time-period windows.Workers covered 2010–2011, 2012–2013, 2014–2015, 2016–2018, 2019–2023, and 2024–2025.
  • 2. Baseline Decomposition (without learned orchestrator skills): Workers mixed multiple tours within each time window, while the 2019–2023 worker was overwhelmed by its five-year scope.The Eras Tour alone had 130+ dates, and broad queries returned noisy, unfocused results.

3. Learned Orchestrator Skill (self-evolved in Phase 1)

The learned orchestrator skill classifies entity-list queries as split-by-entity and decomposes them using rules derived from training-time error analysis. It assigns workers by entity, adds coverage checks, and triggers refinement when gaps are substantial.

  • Learned orchestrator skill: The task-router classifies this query as split-by-entity, using an automatically generated decomposition skill derived from training-time error analysis.The skill is stored as decompose-split-by-entity/SKILL.md.
  • Split-by-entity decomposition: For lists of named entities, the orchestrator splits by entity name rather than time period, assigning each worker one entity as its search keyword.The rule applies to queries targeting entities such as tours, brands, or athletes.
  • Split-by-entity decomposition: Entities with more than 80 expected items are split further by region, while every decomposition includes a gap-detection worker.If more than 10% of items remain missing after gap detection, the orchestrator triggers Round 2.

4. Web2BigTable Decomposition (applying learned skills)

Web2BigTable decomposes the search across tour-specific workers, adds a dedicated gap-detection worker, and dispatches additional verification when incomplete data is detected. Across six tours, this process produced 556 unique rows after deduplication and aggregation, achieving a Row F1 of 93.8%.

  • Worker decomposition: Eight workers divide the task across six tours, with one worker handling gap detection and completeness checking.The system assigns workers to individual tours and designates Worker 7 for coverage verification.
  • Worker decomposition: Two additional verification workers were dispatched after the system detected incomplete Red Tour data.This illustrates a second-round correction based on an identified coverage gap.
  • Results: 653 raw rows were retrieved across 6 tours, yielding 556 unique rows after deduplication and aggregation.The submitted dataset reflects post-processing rather than the full raw-row count.
  • Results: Row F1 = 93.8%.This is the reported row-level extraction result for the decomposed search.

5. Per-Tour Retrieval Distribution (after deduplication)

After deduplication, retrieval covered six Taylor Swift tours with 556 unique rows from 653 raw rows. Entity-based decomposition with adaptive region splitting achieved 93.8% Row F1, far above single-agent and skill-less baselines.

  • Tour distribution: The Red Tour yielded 142 retrieved rows, followed by The Eras Tour with 118 and the Speak Now World Tour with 93.The six-tour case study covers Taylor Swift concerts.
  • Deduplication: 653 raw rows were collectively retrieved and reduced to 556 unique rows for evaluation.The task contains 534 ground-truth rows across six tours.
  • Orchestration: 93.8% Row F1 was achieved by entity-based decomposition with adaptive region splitting, versus 12.8%–26.8% for single-agent and skill-less baselines.The auto-generated orchestrator skill selected this decomposition strategy.

4 Related Work

Web2BigTable extends autonomous web search, self-evolving agents, and agentic memory with a training-free hierarchical multi-agent architecture. Its orchestrator and workers jointly refine decomposition strategies and executable skills through closed-loop reflective learning without parameter updates.

  • Autonomous Web Search and Deep Research Agents: Autonomous web search progressed from single-turn retrieval to multi-step navigation and deep research, while broad web-extraction systems often remain heuristic or computationally expensive.
  • Autonomous Web Search and Deep Research Agents: Web2BigTable evolves web-search and task-decomposition strategies through closed-loop verify-reflect cycles without parameter updates.It also discovers, synthesizes, and refines executable search skills at inference time.
  • Self-Evolving Agents: Self-evolving agent frameworks use experiential learning, reflection, closed-loop self-distillation, and reinforcement learning to develop transferable insights or structured skill libraries.
  • Self-Evolving Agents: Web2BigTable distinguishes itself through concurrent evolution of macro-level orchestrator decomposition strategies and micro-level worker execution skills within one gradient-free learning loop.
  • Agentic Memory Systems: Extending agentic memory systems, Web2BigTable maintains macro-level decomposition memories and micro-level executable skill memories, continuously refining both through a unified Read-Write Reflective Learning loop.

5 Conclusion · A Theoretical Extension: Memento-Team

Web2BigTable is presented as a bi-level multi-agent framework addressing the tension between breadth and reliability through memory-mediated self-evolution. The paper also introduces Memento-Team as a theoretical generalisation that formalises orchestrator–worker interaction as a Stackelberg game.

  • 5 Conclusion: Web2BigTable is a bi-level multi-agent framework for large-scale web-to-table construction.
  • 5 Conclusion: The system addresses the tension between breadth and reliability in agentic web search through a memory-mediated self-evolving architecture.
  • 5 Conclusion: The upper-level orchestrator automatically learns reusable decomposition strategies from a small training split.
  • 5 Conclusion: A lower-level pool of asynchronous workers coordinates through a shared Markdown workboard.
  • 5 Conclusion: The architecture evolves worker execution skills, with adaptation mediated through persistent, human-readable memory.
  • A Theoretical Extension: Memento-Team: Memento-Team develops a rigorous theoretical generalisation of Web2BigTable’s bi-level game formulation.
  • A Theoretical Extension: Memento-Team: Memento-Team formalises orchestrator–worker interaction as a full Stackelberg game with frozen LLM agents.
  • A Theoretical Extension: Memento-Team: The theoretical extension treats strategy memory and execution memory as the sole de...

B Detailed Results … 4. Web2BigTable Decomposition (applying learned skills)

The case study asks Web2BigTable to compile a 331-row, 12-column table of AMD Zen processors released from 2014 through 2024. The comparison contrasts a time-based baseline that produced incomplete, inconsistent records with a learned category-based decomposition designed to improve coverage and specification accuracy.

  • C Case Study: 331 ground-truth rows and 12 required columns make this AMD processor task especially demanding for comprehensive, cell-level extraction.The columns span product identity, architecture, manufacturing, compute, cache, frequency, and graphics specifications.
  • 2. Baseline Decomposition (without learned orchestrator skills): The time-based baseline returned incomplete and inconsistent results across the 12 required columns.Broad temporal queries did not reliably cover the heterogeneous product lines and specifications.
  • 2. Baseline Decomposition (without learned orchestrator skills): The 12-column requirement amplified retrieval errors, leaving partially retrieved rows with many NA or incorrect cache, graphics, and manufacturing-process cells.The failure particularly affected cache sizes, graphics models, and manufacturing process fields.
  • 3. Learned Orchestrator Skill (auto-generated in Phase 1): The learned task-router classified the query as split-by-category and partitioned work by product line rather than by time.This decomposition assigns workers to product categories across the full date range.
  • 3. Learned Orchestrator Skill (auto-generated in Phase 1): The learned skill assigns workers to major product lines, further splits product lines with >50 SKUs by generation, and dedicates a worker to cache and GPU lookup.The listed categories include Desktop, Server, Mobile, Workstation, Embedded, and PRO.

Strategy: split-by-category (product line) + split-by-generation for large lines … 3. Learned Orchestrator Skill (auto-generated in Phase 1)

Web2BigTable uses learned decomposition strategies matched to task structure: product-line and generation splits improve AMD processor aggregation, while source-based partitioning with temporal verification addresses heterogeneous paper catalogs. The learned strategy substantially outperforms single-agent and time-split baselines on coverage, deduplication, and date consistency.

  • Strategy: split-by-category (product line) + split-by-generation for large lines: ∼350 raw rows across eight product lines became ∼334 unique submissions after deduplication, with Row F1 = 89% and Item F1 = 96%.Workers covered Ryzen Desktop, EPYC Server, Ryzen Mobile, Threadripper, Ryzen PRO, Athlon/Embedded, and specification verification.
  • Strategy: split-by-category (product line) + split-by-generation for large lines: Web2BigTable’s learned product-line decomposition with dedicated specification-verification workers achieves 89% Row F1 and 96% Item F1, versus fewer than 50 rows and Item F1 below 26% for single agents.The strategy splits large product lines further by generation to support broad, structured retrieval.
  • 1. User Query (ws_zh_069): The ByteDance Seed–DeepSeek query requires compiling papers from two source organisations over 30 months, with cross-source temporal verification against arXiv.The query requests organisation, publication date, title, and primary authors in one Markdown table, using the arXiv first-submit timestamp for duplicate records.
  • 1. User Query (ws_zh_069): The task’s core difficulty is that the organisations use separate web platforms and formats, while mirrored papers require reconciling publisher dates with arXiv submission records.This makes both source coverage and canonical-date resolution necessary for a consistent output table.
  • 2. Baseline Decomposition (without learned orchestrator skills): The baseline split the 30-month window into five six-month chunks, mixing organisations and producing fragmented third-party results instead of canonical source pages.Generic time-window queries diluted the smaller DeepSeek catalog across workers and left arXiv verification incomplete.
  • 2. Baseline Decomposition (without learned orchestrator skills): The time-based baseline sparsely sampled Seed’s ∼120 papers, lost DeepSeek coverage in empty early windows, and left date inconsistencies unresolved in the Publication Date column.These failures arose because workers treated Seed as one of many possible sources and did not reconcile homepage listings with arXiv records.
  • 3. Learned Orchestrator Skill (auto-generated in Phase 1): The learned task-router classified the query as split-by-source, assigning one worker per named organisation and a dedicated worker for cross-source temporal verification.The learned rule explicitly avoids mixing multiple named sources within one worker and incorporates lessons from prior coverage and date-resolution failures.
  • 3. Learned Orchestrator Skill (auto-generated in Phase 1): 91% Row F1 and 94% Item F1 were achieved on the ByteDance Seed–DeepSeek case, compared with fewer than 30 rows and Item F1 below 40% for single agents.The learned source-based decomposition included a dedicated arXiv verification worker for the ∼130-row task.
Loading 2604.27221v1…