Source-linked AI summary
DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering
Zihan Huang, Shihang Wu, Junle Liu, Peirong Zhang, Yongxin Shi, Xuhan Zheng, Lianwen Jin
TL;DR
Real-world degradations impair document VQA, while existing restoration strategies are labor-intensive or optimize generic image quality instead of question needs. DocIntent selectively restores task-relevant degradations through answerability assessment and rollback, consistently improving MLLM performance and stability on WildDoc. Its effectiveness remains bounded by the underlying restoration tools and MLLM judgment capabilities.
Problem
Real-world document degradations impair MLLM document VQA, while existing restoration approaches require manual strategy design or overlook downstream question-specific needs.
Method
DocIntent assesses question answerability, selectively invokes restoration tools for relevant degradations, and uses Comparison-Based Rollback to reject harmful results without additional pretrained models.
Results
DocIntent consistently improves average scores and answer consistency across open- and closed-source MLLMs on WildDoc.
Takeaways & Limitations
Answerability-guided selective restoration with quality safeguards improves degraded-document VQA through efficient tool utilization.
Takeaways & Limitations
Performance is fundamentally limited by the capabilities of the underlying restoration tools and the intrinsic judgment abilities of the MLLM.
Abstract
from arXiv · showhide
Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimodal Large Language Models (MLLMs). Applying restoration tools before Visual Question Answering (VQA) is an intuitive solution. However, existing restoration approaches remain limited, as manually designing and executing restoration strategies is labor-intensive and requires domain expertise. Agentic restoration offers new possibilities for automation, yet existing frameworks primarily target natural images and pursue perceptual quality, overlooking that restoration should serve downstream tasks rather than optimize generic image quality metrics. To this end, we explore the value of agentic restoration for real-world degraded document VQA and propose DocIntent, a training-free Answerability-Guided Agentic Restoration framework. DocIntent first assesses question answerability, then identifies task-relevant degradations and selectively invokes restoration tools. A Comparison-Based Rollback mechanism validates each restoration step and reverts it when question-relevant evidence becomes less decipherable. The entire process requires no additional pretrained degradation classifier or image quality assessment model. Extensive experiments on the WildDoc benchmark show that DocIntent consistently improves the average score and consistency of different open- and closed-source MLLMs. The code and experimental data will be publicly available.
1 Introduction
Real-world document degradations severely impair MLLM-based document VQA, while manual restoration is labor-intensive and existing agentic approaches overlook question-specific needs. DocIntent addresses this with answerability-guided selective restoration and rollback, improving performance on WildDoc.
- Real-world blur, shadow, and distortion substantially hinder MLLMs’ ability to extract information from document images.
- Manual degradation diagnosis and tool selection make restoration pipelines impractical for non-experts.
- DocIntent assesses answerability, identifies the degradation most affecting the question, and invokes a corresponding restoration tool.
- Its two-phase mechanism selectively invokes tools and rolls back harmful results without additional training, a pretrained degradation classifier, or an image quality assessment model.
- DocIntent consistently improves degraded-document question answering across capability levels and approaches human expert preprocessing on WildDoc.
2 Related work
Document VQA has progressed from OCR-based pipelines to end-to-end and MLLM approaches, but real-world degradations still impair performance. DocIntent selectively restores task-relevant degradations rather than pursuing generic image quality.
- Document VQA methods evolved from OCR-based pipelines to end-to-end approaches that directly learn document representations.
- Specialized document-understanding MLLMs achieve strong standard-benchmark results, while real-world degradations remain a major robustness challenge.
- DocIntent uses MLLMs’ degradation-recognition capability to selectively restore degradations that affect downstream tasks.
- Existing agentic restoration frameworks primarily target natural images and optimize perceptual quality rather than downstream document-question-answering needs.
- Its comparison-based rollback accepts restoration results selectively, avoiding unnecessary processing and risk of text-quality degradation.
3 Method
DocIntent cycles between answerability-guided restoration and comparison-based review. It restores only degradations that hinder answering and validates each result before continuing, using a modular tool set.
- The method is organized around Answerability-Guided Analysis, Tool Execution, and Comparison-Based Rollback.
- Answerability-Guided Analysis: If the image already supports an answer, the system answers directly; otherwise, it selects the most severe answerability-hindering degradation with an available tool.
- Tool Execution: The initial restoration tool list includes deblurring, dewarping, deshadowing, appearance enhancement, demoiréing, and rotations.
- Comparison-Based Rollback: Comparison-Based Rollback presents before-and-after images with the question to the MLLM instead of relying on a separate degradation classifier or image-quality assessor.
- Workflow: The cycle continues until the image is answerable or all tools addressing relevant degradations are disabled.
- Workflow: The design enforces minimality by targeting only answerability-impairing degradations and safety by validating every restoration before commitment.
4 Experiments
DocIntent is evaluated on WildDoc across degraded document VQA tasks, comparing backbone models, restoration strategies, and component ablations. It improves average performance and consistency while approaching question-aware human preprocessing.
- 4.1 Experimental Settings: WildDoc covers WildDocVQA, WildChartQA, and WildTableVQA across blur, shadow, geometric distortion, moiré, uneven illumination, and rotation conditions.Each question has images captured under four real-world conditions, enabling evaluation of performance and consistency across degradations.
- 4.2 Main Results: Baseline models show a substantial score-consistency gap across degradation conditions, including 27.0 percentage points for Claude Sonnet 4.6 and 7.5 points for Gemini-3.1-Pro.The reported gaps are 57.5% versus 30.5% for Claude Sonnet 4.6 and 83.0% versus 75.5% for Gemini-3.1-Pro.
- 4.2 Main Results: DocIntent improves Gemini-3.1-Pro from 83.0% to 84.3% average score and consistency from 75.5% to 80.0%.The consistency gain is 4.5 points, narrowing the gap between average score and consistency.
- 4.2 Main Results: DocIntent improves Qwen3.5-9B from 68.5% to 71.6% average score and consistency from 49.7% to 55.6%.The consistency gain is 5.9 points, and the complete system achieves these values after incorporating restoration and rollback.
- 4.3 Comparisons with Other Strategies: Blindly applying all tools causes severe information corruption, while MLLM-Judge remains unstable and question-aware DocIntent approaches question-aware human preprocessing.The comparison attributes remaining restoration errors to inappropriate tool use and the lack of rollback, whereas DocIntent selectively restores relevant evidence.
- 4.4 Ablation Study: Ablations show that selective restoration and rollback, rather than answerability-guided prompting alone, drive DocIntent’s gains across models.For Gemini3.1-Pro, AGRes raises average score and consistency to 83.7% and 78.4%, while CBRb further raises them to 84.3% and 80.0%; the complete system reaches 71.6% and 55.6% on Qwen3.5-9B.
5 Conclusion
DocIntent argues that restoration orchestration—deciding when to restore, which tool to use, and whether to accept results—is central to degraded-document VQA. Its two-phase cyclic mechanism improves question-answering performance through efficient tool utilization.
- DocIntent makes restoration orchestration central by determining when to restore, which tool to use, and whether to accept the result.
- The framework uses selective restoration with a quality safeguard through a two-phase cyclic mechanism.
- Extensive experiments show improved question-answering performance on degraded documents through efficient tool utilization.
Limitations
DocIntent remains bounded by the restoration tools and the underlying MLLM. These dependencies constrain both restoration quality and the system’s judgment capability.
- DocIntent’s performance is fundamentally limited by the capabilities of the underlying restoration tools.
- Because the framework is training-free and agentic, its judgment capability depends on the intrinsic abilities of the underlying MLLM.
Ethical Considerations
The paper reports no significant ethical risks and describes experiments using public datasets without personally identifiable or sensitive content. It also states that the system restores degraded documents without content tampering or forgery.
- The research reports no significant ethical risks because experiments use publicly available open-source datasets without personally identifiable or sensitive content.
- The system restores degraded documents and does not involve content tampering or forgery.
- Evaluation metrics: Accuracy measures the proportion of questions whose predictions precisely match at least one designated target answer.
- Evaluation metrics: ANLS uses normalized Levenshtein similarity, assigning zero when normalized distance reaches the threshold; the paper sets τ = 0.5.
- Evaluation metrics: Consistency evaluates robustness across four scenarios by requiring a correct answer in all scenarios for document-level accuracy.
- Algorithm workflow: DocIntent’s workflow analyzes answerability, selects a tool for the most severe hindering degradation, and invokes restoration when needed.
D Analysis of Tool Invocation Decisions
Tool-invocation analysis shows that answerability judgments separate easier from harder direct-answering cases, while DocIntent improves results on samples judged to require restoration. Context management further trades off performance against token consumption, favoring text-only context.
- Tool invocation decisions: Confident samples usually score higher than the overall baseline, whereas Forced samples score lower, indicating differentiated direct-answering difficulty.
- Tool invocation decisions: DocIntent obtains higher scores for all models and subtasks on the Forced subset, showing consistent gains on harder-to-answer samples.
- Context management: Text-only context retains tool decisions and quality assessments without intermediate restored images, balancing performance and token consumption.
- Context management: No Context reduces token consumption but can substantially degrade performance; Qwen3.5-9B reaches only 40.7% accuracy on WildChartQA.
- Context management: Full Context achieves high accuracy but increases token use; Qwen3.5-9B consumes 29439 average tokens, 1.55 times Text-only.
G Cost Analysis
DocIntent adds inference overhead through answerability assessment and image comparison, trading cost for document VQA performance and consistency. Costs vary with model-specific tool invocation behavior, so deployment can select a backbone based on performance–cost needs.
- DocIntent’s additional token overhead mainly comes from answerability assessment and before–after image comparison supporting selective restoration and rollback.
- 5,033 input tokens and 1,073 output tokens cost approximately $0.0394 per Gemini-3.1-Pro sample.
- Doubao-2.0-Pro invokes tools 0.69 times per sample, while GPT-4o and GPT-5.2 invoke them approximately twice.These invocation differences indicate that model-specific decision preferences affect final token consumption.
- DocIntent trades additional inference cost for improved document VQA performance and consistency, allowing backbone selection according to deployment priorities.
I Impact of Tool Selection
DocIntent maintains similar performance across two restoration-tool backends, suggesting its gains arise primarily from answerability-guided orchestration rather than improvements to individual restoration tools. The comparison spans the WildDoc benchmark’s three subtasks and uses matched answer-format rules for fair evaluation.
- Tool configurations: Group 1 uses DocRes and UniDemoiré, whereas Group 2 replaces selected restoration tools with NAF-DPM and BGShadowNet.
- Results: For Qwen3.5-9B, Group 1 reaches 71.6% average score and 55.6% consistency, compared with 70.9% and 54.4% for Group 2.
- Results: For Gemini-3.1-Pro, Group 1 reaches 84.3% average score and 80.0% consistency, compared with 84.1% and 78.9% for Group 2.
- Interpretation: The similar results across configurations indicate that DocIntent improves performance regardless of tool backend choice, emphasizing orchestration over restoration-tool enhancement.
K Repeated-Run Evaluation
Repeated WildDoc evaluations show stable DocIntent improvements in both average score and consistency, while clean-document testing indicates that the method largely preserves existing performance. The supplied prompts define question-focused restoration, tool use, comparison, rollback, and task-specific answer formats.
- Repeated-run results: Across three runs, DocIntent consistently improves average score and consistency for Qwen3.5-9B and Gemini-3.1-Pro, with small standard deviations.
- Clean-document evaluation: On the clean DocVQA test set, DocIntent largely preserves Qwen3.5-9B and Gemini-3.1-Pro performance without notable degradation.
- Evaluation controls: Baseline and DocIntent prompts share identical answer-format rules, so their main difference is restoration strategy and tool guidance.
- Prompt design: DocIntent prompts combine task instructions, answerability-guided restoration, tool descriptions, and mandatory comparison-based review after each tool execution.
- Question-focused workflow: The workflow answers directly when relevant information is readable, otherwise selects a relevant restoration tool and rolls back results judged worse after comparison.