DocDancer: Towards Agentic Document-Grounded Information Seeking

• DocQA enables models to access, reason over, and synthesize information from complex, heterogeneous documents, making it central to document-centric intelligence.The introduction situates this capability in understanding and answering questions over long, multimodal documents (Liu et al., 2025b).Source: §1 · p. 1
• Existing DocQA agents are typically prompt-based pipelines with limited learning of autonomous behavior, despite agentic methods supporting iterative exploration, tool use, and multi-step reasoning.The introduction contrasts these agents with OCR-based methods and retrieval-augmented generation (Saad-Falcon et al., 2024).Source: §1 · p. 1
• DocDancer frames DocQA as information seeking, combining global search tools with fine-grained read tools to explore long documents and adapt to observations.The authors report that the framework achieves state-of-the-art performance and exceeds reported human-level performance when instantiated with a proprietary LLM.Source: §1 · pp. 1–2
• Experiments on MMLongBench-Doc (Ma et al., 2024) and DocBench (Zou et al., 2025) demonstrate DocDancer’s effectiveness; analyses provide insights into parsing strategies, tool design, and synthetic data in agent learning.Source: §1 · p. 2

Document Question Answering Methods

• OCR-based and vision–language DocQA struggle with long documents, single-shot retrieval is brittle for complex queries, and recent agents rely on closed-source LLMs.This work aims to train an open-source document agent with learnable behaviors for robust and scalable DocQA (Ding et al., 2022; Zhang et al., 2025).Source: §2 · p. 2

Synthetic Data for Agent Training

• Existing DocQA datasets require substantial human involvement or lack question depth, motivating synthetic training data for DocQA agents.The work frames DocQA as agentic information seeking and targets high-quality training data tailored to DocQA agents.Source: §2 · p. 2

3.1 Agent Setup

• DocDancer uses a single-agent ReAct policy that interleaves thoughts and actions, supported by an enhanced document outline and the Search and Read tools (Yao et al., 2022).The outline enriches extracted elements with layout and semantic attributes, segments sections using visually clustered titles, and adds image and chart captions; Search locates text globally, while Read retrieves section text and visual context for a goal.Source: §3.1 · p. 3

3.2 Data Synthesis

• The Exploration-then-Synthesis pipeline iteratively explores source documents for structured evidence, then reasons over observations to produce grounded, coherent QA pairs.Exploration uses intent-action steps with document tools rather than explicitly constructing a document-level knowledge graph, trading engineering overhead for a lightweight generation process.Source: §3.2 · pp. 4–5
• A strong open-source model applies rejection sampling to generated QA pairs to obtain high-quality training trajectories.The grounding corpus draws on LongDocURL (Deng et al., 2025), MMDocRAG (Dong et al., 2025b), CUAD (Hendrycks et al., 2021), and DUDE (Van Landeghem et al., 2023).Source: §3.2 · pp. 3–5

3.3 Agent Training

• Training masks loss on observation tokens so that the objective covers the agent’s decision steps rather than external feedback.The paper motivates this choice as mitigating interference from external feedback and cites prior findings of improved performance and robustness (Chen et al., 2023).Source: §3.3 · p. 5

4.1 Experimental Setup

• DocDancer is produced by fine-tuning Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507 on only 5,000 agent trajectories and is evaluated on MMLongBench-Doc and DocBench.MMLongBench-Doc contains 135 documents averaging 47.5 pages and 1,091 questions; DocBench contains 229 real-world documents and 1,082 questions.Source: §4.1 · p. 5

4.2 Overall Performance (RQ1)

• 56.8 F1 / 67.6 LasJ on MMLongBench-Doc and 85.5 on DocBench (4 points above human) are reported for DocDancer with GPT-5.2, which outperforms prior methods.Agent-based approaches outperform VLM-, OCR-, and RAG-based baselines; under the same backbone, the single-agent framework matches or surpasses multi-agent systems. The trained 30B-A3B and 4B agents are competitive with closed-source models.Source: Table 1 · p. 6, §4.2 · p. 6

4.3 Effectiveness of Synthetic Data (RQ2)

• With equal-sized 5,000-instance training sets, DocDancer trained on synthesized QA data outperforms the OS-QA model across all metrics and benchmarks.On MMLongBench-Doc, DocDancer also outperforms the QA baseline in Academic, Financial, Industry, and Report domains, with larger gains in structurally complex domains requiring iterative information seeking and fine-grained reasoning.Source: §4.3 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7

4.4 Influence of Agentic Tools (RQ3)

• MinerU2.5 outlines and a two-tool Search-and-Read design together achieve the best results, while the outline and reduced tool set each improve performance over the DocAgent baseline.The outline method consistently outperforms the baseline when paired with the same tools; the two-tool design also performs better with either the new outline or DocAgent’s outline.Source: §4.4 · p. 7, Figure 4 · p. 7
• Replacing the Read tool’s default Qwen3-VL-235B-A22B-Instruct with Gemini-3-Pro improves DocBench accuracy by 0.2 points overall, with gains in Government, Law, and News.The results indicate the tool design does not depend on an exceptionally strong external model.Source: §4.4 · p. 8, Figure 8 · p. 8

4.5 Qualitative Analysis (RQ4)

• In a 73-page financial-document case, DocDancer gathers advertising expense and revenue evidence from separate sections and computes the correct ratio, while OS-QA returns an erroneous ratio.DocDancer grounds $714.3M in FY 2015 advertising expense and $6,779.5M in total revenue to compute 714.3/6,779.5 ≈0.105; OS-QA substitutes marketing expense and gives 0.122.Source: §4.5 · p. 8, Figure 7 · p. 8

• DocDancer frames document question answering as information seeking and uses complementary search and read operations to explore and comprehend long, structured documents.Source: §5 · p. 8
• DocDancer achieves strong and competitive performance on MMLongBench-Doc and DocBench, validating agentic information seeking for document understanding.Source: §5 · p. 9
• The Exploration-then-Synthesis pipeline generates compact yet effective training data to address the lack of high-quality supervision for learning agentic behaviors.Source: §5 · pp. 8–9

Limitations

• Experiments cover only Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507, not larger-scale models or models from other families.Source: Limitations · p. 9
• The study uses supervised fine-tuning (SFT) exclusively and does not explore agentic reinforcement learning (RL).Source: Limitations · p. 9
• The training data is not further scaled, so performance with larger or more diverse data remains unexamined.Source: Limitations · p. 9

• The pipeline uses publicly available research documents, creates source-grounded question–answer pairs, and neither introduces proprietary data nor reproduces large copyrighted passages verbatim.Source: Ethical Considerations · p. 9
• Although the method collects no personal information, downstream use on sensitive or private documents requires authorization and privacy safeguards, and synthesized data and models may inherit source biases.Source: Ethical Considerations · p. 9
• Responsible deployment matters because stronger document exploration could be misused; grounding in retrieved evidence and releasing code and data support transparency, reproducibility, and responsible research use.Source: Ethical Considerations · p. 9

A Case Study of Synthetic Data

• The framework navigates a 73-page document, aggregating text, chart, and table evidence scattered across pages 40, 41, and 49 to synthesize a high-quality question.The example draws on text in Sec. 2.43, Figure 1, and Table 1.Source: A Case Study of Synthetic Data · p. 13
• Exploration iteratively generates a trajectory ξ as a “random walk” over the document’s implicit knowledge graph, bridging chart trends with precise text and table values.The chart is on p. 40, the text on p. 49, and the table on p. 41.Source: A Case Study of Synthetic Data · p. 13
• The synthesis model constructs a complex multi-hop numerical question (Wu et al., 2025b) requiring 29.92% − 15% = 14.92% arithmetic rather than simple retrieval.This design ensures deep document grounding and prevents shortcut learning.Source: A Case Study of Synthetic Data · p. 13

B.1 Details on Prompts

• The DocDancer prompts are shown in Figure 10.Source: §B.1 · p. 13

B.2 Tool Schema

• The agent has two primary tools: keyword-based search and read for extracting content from specific document sections; their JSON schemas are shown in Figure 11.Source: §B.2 · p. 13

B.3 Training Details

• Both models use 128k context and Megatron-LM (Zhao et al., 2024; Shoeybi et al., 2019), with AdamW and a precision-aware configuration.Training uses a cosine-decay learning-rate scheduler.Source: §B.3 · p. 13
• The cosine scheduler uses a 1.0 × 10^-5 peak learning rate, a 1.0 × 10^-6 minimum, and 5% warmup.Source: §B.3 · p. 13
• Training uses batch sizes of 16 for Qwen3-30B-A3B-Think and 40 for Qwen3-4B-Think; the former uses a 10^-3 auxiliary loss coefficient, and both models train 10 epochs with best-performance checkpoint selection.Source: §B.3 · p. 13

B.4 Inference Details

• Inference uses vLLM (Kwon et al., 2023), temperature 0.6, top-p 0.95, and presence penalty 1.1.Source: §B.4 · p. 13

B.5 Hyperparameter

• The method substantially outperforms the base model without fine-tuning and the model trained with reject sampling.The passage attributes the comparisons to Table 2 and says they demonstrate the training strategy’s effectiveness and the quality of the synthesized question–answer data.Source: §B.5 · p. 13
• Mm defaults to Qwen3-VL-235B-A22B-Instruct, while gpt-oss-120b performs rejection sampling as Mt and Exploration-then-Synthesis data generation as Ms.Source: §B.5 · p. 13

B.6 Details on Prompts for Data Synthesis

• The appendix presents Exploration and Synthetic prompts and sets maximum sampling depth to 20 for LongDocURL and MMdocRAG, and 15 for DUDE and CUAD.The prompt figures are Figures 12 and 13, respectively.Source: §B.6 · p. 13

Naive VLM Baselines

• Naive VLM baselines receive PDF pages as 144-DPI images without external parsing or retrieval, following MMLongBench-Doc settings (Ma et al., 2024).The reported models include GPT-4o and Gemini-2.5-Pro.Source: C Baselines · pp. 13–14

OCR-based Baselines

• OCR-based baselines extract text with Tesseract or PyMuPDF before QA with GPT-4, GPT-4o, or Gemini-2.0-Flash.Source: C Baselines · p. 14

RAG-based Baselines

• Visual retrieval uses VisRAG (Yu et al., 2024) and ColPali (Faysse et al., 2024), while hybrid approaches include M3DocRAG and RAGAnything (Guo et al., 2025b).Visual methods retrieve page- or patch-level evidence for GPT-4o generation; hybrid methods combine multimodal retrieval or cross-modal knowledge-entity retrieval with language-model backbones.Source: C Baselines · pp. 14–15

Prompt-based Agentic Baselines

• Prompt-based agentic baselines include Doc-React, which balances information gain and uncertainty, and MDocAgent (Han et al., 2025), which uses five specialized retrieval roles.MACT (Yu et al., 2025) adds adaptive test-time scaling, while SimpleDoc (Jain et al., 2025) retrieves pages with ColQwen2.5 and applies LLM evidence selection.Source: C Baselines · p. 15
• DocLens (Zhu et al., 2025) is a tool-augmented multi-agent framework for focused reading, while DocAgent (Sun et al., 2025a) combines a tree-structured outline with retrieval tools.The listed backbones are Claude-4-Sonnet and Gemini-2.5-Pro for DocLens, and GPT-4o and Claude-3.5-Sonnet for DocAgent.Source: C Baselines · p. 15