Source-linked AI summary
Less Detail, Better Answers: Degradation-Driven Prompting for VQA
Haoxuan Han, Weijie Wang, Zeyu Zhang, Yefei He, Bohan Zhuang
TL;DR
VLMs can be misled when high-resolution details act as distracting noise, motivating a method that emphasizes structural information instead. DDP combines degradation, targeted prompting, and agentic verification, and the paper reports improved performance across challenging visual benchmarks.
Problem
VLMs remain vulnerable to deceptive images because local textures and passive single-shot observation can override global visual structure.
Method
DDP uses multi-scale downsampling, fine-grained visual prompts, task classification, external visual tools, and critic-based verification to support active perception.
Results
DDP consistently outperforms traditional end-to-end VLM inference and reaches 65.8% overall accuracy on V*Bench, surpassing GPT-4V by 10.8%.
Takeaways & Limitations
The paper presents DDP as a scalable, robust, and interpretable pathway toward more reliable and human-like machine vision.
Takeaways & Limitations
The comparative analysis excludes models without publicly available and validated metrics, limiting the evaluation set.
Abstract
from arXiv · showhide
Recent advancements in Vision-Language Models (VLMs) have significantly pushed the boundaries of Visual Question Answering (VQA).However,high-resolution details can sometimes become noise that leads to hallucinations or reasoning errors. In this paper,we propose Degradation-Driven Prompting (DDP), a novel framework that improves VQA performance by strategically reducing image fidelity to force models to focus on essential structural information. We evaluate DDP across two distinct tasks. Physical attributes targets images prone to human misjudgment, where DDP employs a combination of 80p downsampling, structural visual aids (white background masks and orthometric lines), and In-Context Learning (ICL) to calibrate the model's focus. Perceptual phenomena addresses various machine-susceptible visual anomalies and illusions, including Visual Anomaly (VA), Color (CI), Motion(MI),Gestalt (GI), Geometric (GSI), and Visual Illusions (VI).For this task, DDP integrates a task-classification stage with specialized tools such as blur masks and contrast enhancement alongside downsampling. Our experimental results demonstrate that less is more: by intentionally degrading visual inputs and providing targeted structural prompts, DDP enables VLMs to bypass distracting textures and achieve superior reasoning accuracy on challenging visual benchmarks.
1. Introduction
The paper argues that VLMs remain vulnerable to deceptive visual signals because they over-rely on local textures and passive observation. DDP addresses this vulnerability by degrading inputs, guiding attention structurally, and adding agentic verification tools.
- VLMs can perform strongly on standard benchmarks yet remain vulnerable to optical illusions and other visually deceptive images.
- Local textures and statistical patterns can override global structure, producing confident but incorrect hallucinations.
- The framework shifts inference from passive observation toward active perception through tool-use and iterative verification.
- DDP combines downsampling, prompt engineering, and agentic tool-use to improve VLM performance on challenging visual tasks.
- Downsampling suppresses deceptive high-frequency details so models prioritize global topological structures, while targeted prompts focus attention on contradictory regions.
- DDP augments VLMs with blur masks, grid auxiliary lines, and localized magnifiers for testing visual hypotheses against geometric evidence.
2. Related Works
Related work develops visual reasoning through modular tool use, active analysis, and increasingly precise visual prompting. These approaches connect image regions with language and support interaction with complex visual scenes.
- Visual programming systems use code generation and modular tool use to support complex visual reasoning.
- Tool-augmented systems orchestrate external tools or task-specific models to expand multimodal reasoning capabilities.
- Active-perception research emphasizes fine-grained grounding and visual prompts for analyzing or editing image content.
- Semantic overlays use alphanumeric markers and spatial coordinates to associate image regions with linguistic instructions.
- Visual anchors guide cross-attention toward relevant image areas for answering questions or performing tasks.
- Visual learning methods provide demonstrations that clarify desired output formats or reasoning logic.
3. Methodology
DDP is a multi-stage inference pipeline that classifies visual tasks, invokes specialized tools, and progressively degrades images to emphasize structural evidence. A critic then synthesizes purified visual inputs with targeted prompts through logical consistency checks to produce the answer.
- Pipeline overview: DDP combines hierarchical task classification, agentic tool invocation, and low-resolution refinement with targeted prompting.The pipeline routes images and questions into task categories before selecting tools and prompts.
- Task classification: The classifier smooths the input image and uses the image-question pair to produce a task configuration for downstream routing.Tasks are grouped into Physical Attributes and Perceptual Phenomena, with further subcategories guiding specialized visual primitives.
- Degradation-driven prompting: DDP compresses tool-processed images to approximately 150 pixels initially and then to a maximum dimension of 80 pixels to suppress high-frequency noise.The later reduction acts as a structural bottleneck that discards deceptive textures while retaining global structural cues.
- Tool manager: The Tool Manager autonomously selects external tools that convert ambiguous inputs into evidence-enhanced representations for objective reasoning.Physical-attribute tools include auxiliary lines and spatial isolation, while perceptual-phenomenon tools include masks, blur, orthogonal lines, and localized crops.
- Target prompting: The critic receives the low-resolution purified image and alignment prompts, performs consistency and physical-dimension checks, and generates the final answer.This reframes visual recognition as structured logical criticism rather than direct interpretation of the original image.
4. Experiments
DDP is evaluated across public benchmarks, fine-grained visual grounding, illusion-sensitive tasks, and ablations of its pipeline components. Results report consistent gains over tested baselines, while ablations attribute performance to degradation, visual tools, and prompt engineering.
- Performance on General Benchmarks: DDP consistently outperforms existing methods across common public benchmark metrics.The evaluation reports accuracy comparisons on MME, SEED-Bench, ScienceQA, and VQAv2, with GPT-4o results taken from its official technical report.
- Performance on High-Resolution Visual Grounding: 65.8% overall accuracy on V*Bench surpasses GPT-4V by 10.8% and LLaVA-1.5 by 17.1%.DDP also reaches 62.2% on Attribute Recognition and 71.2% on Spatial Relationship, including a 10.7% improvement over GPT-4V on the latter.
- Performance on Perceptual Phenomena: Existing models including OpenAI o1, Gemini-2.5-Pro, and Qwen2.5-VL-72B achieve zero Pass@1 on the TeT benchmark.The comparison emphasizes the benchmark’s difficulty before reporting DDP’s breakthrough.
- Performance on Perceptual Phenomena: 28.89% Pass@1 accuracy exceeds the 15.50% baseline and the tested general-purpose VLMs.The baseline omits blur and visual enhancement, whereas the full pipeline adds fine-grained prompting and enhancement.
- Workshop Evaluation: DDP achieves 82.26% accuracy on Track 2, outperforming the baseline by 10.89%, while maintaining superiority across both tracks.Track 1 is described as the first-place solution, and Track 2 evaluation did not use permitted measurement tools.
- Ablation Study: Removing image degradation lowers overall performance by 8.7%, visual tools by 5.5%, prompt engineering by 3.4%, and direct inference by 30.7%.The ablation study evaluates tool components, prompt engineering, and image degradation on V*Bench.
5. Conclusion
The paper presents DDP as an agentic strategy for reducing VLM reliance on deceptive local textures and improving structural visual reasoning. Its conclusion emphasizes deliberate degradation and active tool use as a pathway toward more reliable machine vision.
- 5. Conclusion: DDP uses multi-scale downsampling, task classification, tool management, and critic-based reasoning to transform passive observation into active visual verification.The pipeline targets structural over-sensitivity and local-texture biases through a structural bottleneck and objective geometric and photometric analysis.
- 5. Conclusion: Across rigorous benchmarks, DDP is reported to establish a new state-of-the-art by degrading distractive inputs and adding active tool use.The conclusion frames this result as evidence that less visual detail can improve visual perception.
- 5. Conclusion: DDP provides a scalable, robust, and interpretable pathway toward more reliable and human-like machine vision.