Source-linked AI summary
DocDancer: Towards Agentic Document-Grounded Information Seeking
Qintong Zhang, Xinjie Lv, Jialong Wu, Baixuan Li, Zhengwei Tao, Guochen Yan, Huanyao Zhang, Bin Wang, Jiahao Xu, Haitao Mi, Wentao Zhang
TL;DR
Existing DocQA agents have limited tool use and often rely on closed-source models, motivating an open-source, end-to-end approach. DocDancer formulates DocQA as information seeking, uses complementary search and read tools, and trains on Exploration-then-Synthesis data; experiments on MMLongBench-Doc and DocBench show strong and competitive performance.
Problem
Existing DocQA agents typically use prompt-based pipelines with limited autonomous agentic behavior, while high-quality DocQA training pairs are scarce.
Method
DocDancer is an end-to-end agentic DocQA model using complementary search and read tools plus an Exploration-then-Synthesis pipeline for grounded training data.
Results
DocDancer achieves strong and competitive performance on MMLongBench-Doc and DocBench, while the framework reaches state-of-the-art and exceeds reported human-level performance with a proprietary LLM.
Takeaways & Limitations
The results validate agentic information-seeking as an effective approach for understanding long, structured documents.
Takeaways & Limitations
The study evaluates only Qwen3 models, uses supervised fine-tuning without agentic reinforcement learning, and does not scale training data or test larger or other model families.
Abstract
from arXiv · showhide
Document Question Answering (DocQA) focuses on answering questions grounded in given documents, yet existing DocQA agents lack effective tool utilization and largely rely on closed-source models. In this work, we introduce DocDancer, an end-to-end trained open-source Doc agent. We formulate DocQA as an information-seeking problem and propose a tool-driven agent framework that explicitly models document exploration and comprehension. To enable end-to-end training of such agents, we introduce an Exploration-then-Synthesis data synthesis pipeline that addresses the scarcity of high-quality training data for DocQA. Training on the synthesized data, the trained models on two long-context document understanding benchmarks, MMLongBench-Doc and DocBench, show their effectiveness. Further analysis provides valuable insights for the agentic tool design and synthetic data.
1 Introduction
DocDancer addresses limitations in existing DocQA agents by combining end-to-end agent training with tool-driven document exploration and an Exploration-then-Synthesis data pipeline. The framework uses search and read capabilities, and achieves strong results across long-context document benchmarks.
- Motivation: Existing DocQA agents are typically prompt-based pipelines with limited learning of autonomous agentic behaviors.
- Approach: DocDancer formulates DocQA as an agentic information-seeking problem and trains an end-to-end agent model beyond prompt-based designs.
- Approach: The framework combines efficient search tools for global information acquisition with fine-grained read tools for localized comprehension.These tools support active exploration, hypothesis refinement, and strategy adaptation based on intermediate observations.
- Results: The framework achieves state-of-the-art performance with a proprietary LLM and exceeds reported human-level performance.
- Training Data: The Exploration-then-Synthesis pipeline collects grounded evidence through tool-augmented exploration and generates document-grounded QA pairs through multi-observation reasoning.
- Results: Experiments on MMLongBench-Doc and DocBench demonstrate DocDancer’s effectiveness, while further analyses examine parsing, tool design, and synthetic data.
2 Related Work
Prior DocQA methods use OCR, vision–language processing, or retrieval, but long documents and complex multi-step queries remain challenging. Synthetic data is increasingly used to provide scalable training supervision for agent learning.
- Document Question Answering Methods: OCR-based and end-to-end vision–language DocQA methods are constrained by limited input length and struggle with long documents.
- Document Question Answering Methods: Retrieval-augmented generation improves scalability but often decouples retrieval and reasoning, making systems brittle to retrieval errors on complex multi-step queries.
- Synthetic Data for Agent Training: Synthetic data offers scalable, rapidly iterable, and trainable supervision as an alternative to manually annotated datasets for agent learning.
- Synthetic Data for Agent Training: Prior work has generated synthetic training data for search, code, and GUI agents.
3 Methods
DocDancer formulates document question answering as agentic information seeking, using a ReAct-style agent with complementary document tools and a two-stage pipeline that synthesizes grounded training data. The resulting trajectories support QA generation and agent training while balancing document coverage, fine-grained comprehension, and toolkit simplicity.
- 3.1 Agent Setup: The agent uses ReAct trajectories that interleave reasoning traces, actions, and observations, with each action conditioned on the full interaction history.The trajectory ends in the task answer, while intermediate steps support iterative document exploration.
- 3.1 Agent Setup: DocDancer frames DocQA as information seeking over the given documents and uses a single agent with a deliberately small, effective toolset.The design prioritizes efficient information access while keeping toolkit complexity low.
- 3.1 Agent Setup: Search provides global textual signals through keyword matches and section context, while Read extracts goal-relevant information from selected sections.Together, the tools support localization followed by fine-grained comprehension and integrate textual and visual document signals.
- 3.2 Data Synthesis: The Exploration-then-Synthesis pipeline first explores diverse source documents through intent-guided tool interactions, then reasons over collected observations to generate grounded QA pairs.The exploration trajectory serves as structured evidence, and synthesis emphasizes multi-observation reasoning, document grounding, and coherent outputs.
- 3.2 Data Synthesis: Exploration acts as a lightweight random walk over the document’s implicit knowledge graph, avoiding explicit graph construction while retaining a trade-off between efficiency and challenging QA generation.The method can bridge disjoint evidence across a document through repeated exploration.
- 3.2 Data Synthesis: The synthesized QA pairs are rejection-sampled by a strong open-source model to obtain high-quality training trajectories for the agent.These trajectories provide the supervision needed to train agentic document-question-answering behavior.
- 3.3 Agent Training: Training masks loss contributions from observation tokens so optimization focuses on the agent’s decision steps rather than external feedback.The masking is reported to improve performance and robustness by mitigating interference from observations.
4 Experiments
Experiments evaluate DocDancer on two long-context document QA benchmarks, comparing agent designs, synthetic versus open-source training data, domains, tools, and qualitative information gathering. Results show strong benchmark performance, broad domain robustness, benefits from the proposed parsing and tool design, and more accurate multi-round evidence aggregation.
- Experimental Setup: The experiments evaluate DocDancer on MMLongBench-Doc and DocBench using multimodal, long-context document question answering benchmarks.The setup includes 135 documents averaging 47.5 pages and 1,091 questions for MMLongBench-Doc.
- Overall Performance: Agent-based approaches substantially outperform VLM-, OCR-, and RAG-based baselines across the evaluated long-document benchmarks.The comparison attributes this pattern to explicit tool use and iterative reasoning for long-context document understanding.
- Overall Performance: 56.8 F1 / 67.6 LasJ on MMLongBench-Doc and 85.5 on DocBench are reported for DocDancer with GPT-5.2, with DocBench exceeding the human baseline by 4 points.Under the same backbone, the single-agent framework matches or surpasses multi-agent systems.
- Effectiveness of Synthetic Data: Models trained on 5,000 synthesized instances outperform equally sized OS-QA-trained models across all reported metrics and benchmarks.The comparison uses the same PDF sources and Qwen3-30B-A3B-Thinking-2507 for both training sets.
- Detailed Results on Domains: DocDancer consistently outperforms the QA baseline across Academic, Financial, Industry, and Report domains on MMLongBench-Doc.The gains are more pronounced in structurally complex domains requiring iterative information seeking and fine-grained reasoning.
- Ablation Studies: MinerU2.5-based outline construction and the two-tool Search-and-Read design jointly produce the best ablation results.The enhanced processing approach outperforms the baseline with the same tools, while the reduced tool set performs better with either outline.
- Ablation Studies: Replacing Qwen3-VL-235B-A22B-Instruct with Gemini-3-Pro improves DocBench accuracy by 0.2 points, with gains in Government, Law, and News.The result indicates robustness of the tool design to the external Read-tool model.
- Qualitative Analysis: DocDancer reaches the correct financial ratio, 714.3/6,779.5 ≈0.105, whereas OS-QA produces 0.122 from insufficiently grounded retrieval.DocDancer gathers advertising expense and revenue from separate sections and verifies their semantic relevance before computing.
5 Conclusion
DocDancer formulates document question answering as agentic information seeking and combines complementary search and read operations for long, structured documents. Its Exploration-then-Synthesis pipeline supplies training data, while experiments on two benchmarks show strong and competitive performance.
- DocDancer is an end-to-end trained agentic model that formulates document question answering as an information-seeking process.
- Its tool-centric framework combines complementary search and read operations to support exploration and comprehension of long, structured documents.
- The Exploration-then-Synthesis pipeline generates compact training data for learning agentic behaviors despite limited high-quality supervision.
- Experiments on MMLongBench-Doc and DocBench demonstrate strong and competitive performance, supporting agentic information-seeking for document understanding.
Limitations
The study identifies scope and scaling limitations in its model, training-method, and data evaluations. It evaluates only selected Qwen models, uses supervised fine-tuning, and does not scale the training data.
- Experiments use only Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507, excluding larger-scale models and other model families.
- The study focuses exclusively on supervised fine-tuning and does not explore agentic reinforcement learning.
- The training data is not further scaled, leaving performance under larger or more diverse datasets unexamined.
Ethical Considerations
The work uses public research documents and synthetic document-grounded data, but downstream deployment on sensitive documents requires authorization and privacy safeguards. The data and models may also inherit source biases, and improved exploration could be misused.
- The study uses publicly available benchmarks and research-released documents, without introducing new proprietary data or reproducing large copyrighted passages verbatim.
- Applications to sensitive or private documents require appropriate authorization and privacy safeguards.
- Synthesized data and trained models may inherit domain and content imbalances from their underlying document sources.
- Improved document exploration capabilities could be misused if deployed irresponsibly, although the work emphasizes grounding, transparency, reproducibility, and responsible research use.
A Case Study of Synthetic Data
The case study shows Exploration-then-Synthesis navigating a long document to gather heterogeneous evidence scattered across pages. The accumulated trajectory then supports synthesis of a complex multi-hop numerical question.
- The framework navigates a 73-page document while aggregating text, charts, and tables distributed across disjoint pages.The example links evidence from Section 2.43, Figure 1, and Table 1 across pages 40, 41, and 49.
- During exploration, iterative interactions generate a trajectory that bridges visual trends with precise values across charts, text, and tables.
- During synthesis, the model reasons over the accumulated trajectory to construct a complex multi-hop numerical question.
B Implementation Details
DocDancer uses a compact tool interface centered on keyword search and section-level reading. The prompts and JSON schemas define how the agent accesses document content.
- Prompts: The system prompts used by DocDancer are presented in Figure 10.
- Tool Design: DocDancer provides two primary tools: search for keyword-based retrieval and read for extracting content from specified document sections.Their JSON schemas are shown in Figure 11.
- Tool Design: Search and read form the documented tool schema for the agent’s document-access workflow.
B.3 Training Details
Training uses long-context Qwen3 models with fixed optimization settings, followed by checkpoint selection based on performance. The exploration configuration varies its maximum depth by document source.
- Model Training: Qwen3-30B-A3B-Think2 and Qwen3-4B-Think3 are fine-tuned with 128k context lengths for long-document processing.
- Optimization: Both models use AdamW, precision-aware training, cosine learning-rate decay, peak learning rate 1.0 × 10−5, minimum 1.0 × 10−6, and 5% warmup.
- Optimization: Global batch sizes are 16 for Qwen3-30B-A3B-Think and 40 for Qwen3-4B-Think, with a 10−3 auxiliary loss coefficient for balanced routing in the larger model.
- Model Training: The models are trained for 10 epochs, and the checkpoint with the best performance is selected.
- Inference: Inference uses vLLM with temperature 0.6, topp 0.95, and presence penalty 1.1.
- Exploration Configuration: Exploration depth is capped at 20 for LongDocURL and MMdocRAG, and 15 for DUDE and CUAD.
C Baselines
The paper compares DocDancer with naive VLM, OCR-based, retrieval-based, and prompt-based agentic baselines. Its exploration framework targets grounded, multi-page, multi-element, multi-hop document QA data.
- Naive VLM Baselines: Naive VLM baselines directly process PDF pages converted to 144-DPI images without external parsing or retrieval.
- OCR-based Baselines: OCR-based baselines extract document content with Tesseract or PyMuPDF before using language models for text-only question answering.
- RAG-based Baselines: Retrieval baselines include visual retrieval with VisRAG and ColPali, plus hybrid retrieval with M3DocRAG and RAGAnything.
- Prompt-based Agentic Baselines: Prompt-based agentic baselines include iterative decision-making, multi-agent retrieval, adaptive test-time scaling, evidence selection, focused reading, and tree-structured outline retrieval.
- Exploration-then-Refine Framework: The exploration stage collects grounded evidence bundles for hard document QA synthesis from parsed PDFs containing outlines, paragraphs, images, tables, and page screenshots.
- Exploration-then-Refine Framework: Generated QA pairs must combine at least three pages or sections, two evidence source types, and at least two reasoning points.