Source-linked AI summary
From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models
Hongrui Jia, Chaoya Jiang, Yongrui Heng, Shikun Zhang, Wei Ye
TL;DR
Large multimodal model training remains constrained by scarce annotated data and static recipes that do not diagnose capability blind spots for targeted reinforcement. DPE closes this loop with diagnostic-guided data generation, adaptive mixtures, and multi-agent quality-controlled multimodal evolution, producing stable gains across eleven benchmarks with limited data.
Problem
Annotated multimodal reasoning data remains scarce, while static training makes it difficult to target model capability gaps.
Method
DPE uses a diagnosis, generation, and reinforcement loop with adaptive data mixtures and multi-agent tool-assisted construction of weakness-focused samples.
Results
DPE delivers comprehensive reasoning enhancement with small amounts of training data and sustained improvements across open-source LMMs and long-tail reasoning abilities.
Takeaways & Limitations
DPE supports controllable evolution directions and stable training dynamics while addressing long-tail multimodal reasoning through closed-loop data evolution.
Takeaways & Limitations
The framework assumes reference answers are verifiable to support stable reinforcement learning with verifiable rewards.
Abstract
from arXiv · showhide
As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.
1. Introduction
Existing LMM training and self-evolution methods face scarce multimodal reasoning data and weak failure diagnosis, limiting targeted improvement on long-tail capabilities. DPE addresses these gaps through diagnostic-guided data generation, adaptive mixture optimization, and multi-agent tool-assisted evolution.
- Annotated multimodal reasoning data remains scarce, making large-scale LMM training difficult.
- Existing self-evolution methods rely on heuristic signals rather than explicit failure attribution, limiting principled capability improvement.
- DPE uses diagnosis and targeted correction to identify capability gaps, steer data generation, and optimize the training mixture.
- DPE uses multi-agent image search and editing to construct diverse, high-quality, weakness-focused training samples with controlled distributions.
- DPE improves capabilities broadly across 11 multimodal reasoning benchmarks using small amounts of training data.
- With only 1000 training examples, DPE achieves broad improvements in multimodal reasoning and quantitatively analyzes diagnosis-related training stability.
2. Related Work
Related work addresses multimodal reasoning through verifiable rewards and self-evolving data generation. These approaches include filtering noisy samples, generating and verifying queries, and incorporating tool use.
- Recent multimodal RL work introduces rule-based or rendering-based feedback loops to ground visual reasoning.
- Filtering-based self-evolution selects high-quality samples from noisy generations using uncertainty metrics or process reward models.
- Generative self-evolution uses proposer-solver loops for query creation and consistency-based verification, while newer approaches incorporate tool use.
3. Methods
DPE iteratively diagnoses capability blind spots, converts failures into targeted data-generation constraints, and reinforces the model with verifiable rewards. Its multi-agent pipeline controls category proportions, image and question requirements, and sample quality before RL training.
- DPE repeats diagnosis, targeted generation, and reinforcement-based updating to align training resources with current capability blind spots.Each iteration re-enters diagnosis after the model update.
- Diagnostic mechanism: The diagnostic mechanism evaluates 200 sampled instances across 12 capability dimensions and attributes recurring errors beyond category accuracy.It analyzes failures such as OCR omissions, chart-unit errors, dropped mathematical steps, and multi-image entity misalignment.
- Diagnostic mechanism: Category accuracy is converted into a normalized proportion vector that determines next-round generation quotas.The resulting quotas impose hard category-count constraints on the generated training set.
- Multiple Agents Questioner System: Four specialized agents coordinate planning, image retrieval or editing, question construction, and validation through shared diagnostic information and quota state.The Planner translates diagnostic outputs into per-sample image and question requirements, while the other agents instantiate and check samples.
- LMM training: Accepted datasets contain single- or multi-image inputs with verifiable reference answers for GRPO optimization.GRPO generates grouped trajectories, assigns scalar rewards, and uses a clipped surrogate objective with KL regularization.
- Multiple Agents Questioner System: Validation accepts samples only when category consistency, solvability, answer verifiability, and format compliance checks pass.Rejected candidates are discarded and regenerated, reducing training noise and distribution drift.
4. Experiments
DPE is evaluated under low-data conditions against self-evolving and static-training baselines, across multiple models, iterations, and multimodal reasoning benchmarks. Results show broad, stable gains attributed to diagnosis-guided data allocation, visual diversity, and high-quality generated questions.
- Comparison with Self-evolving Methods: DPE consistently improves capabilities over VisPlay, including CharXivRQ by 4.11 points and HallusionBench from 68.35% to 69.19% on Qwen2.5-VL-7B-Instruct.DPE also maintains smoother iteration-wise training dynamics than VisPlay.
- Comparison with Self-evolving Methods: On Qwen3-VL-8B-Instruct, DPE improves MMMU by +3.67 and MMStar by +10.86, demonstrating transfer across model scales.The reported gains support transferability beyond the smaller evaluated backbone.
- Comparison with State-of-the-Art Models: An 8B DPE model reaches an average score of 64.39, exceeding Qwen2.5-VL-72B at 61.9 and GPT-4o at 56.1.DPE also reports MathVista 76.2, MathVision 53.88, and HallusionBench 74.13.
- Impact of Static Data: With approximately 3,000 iteratively generated samples, about 1/15 of Vision-SR1-47K, DPE improves MMMU from 54.8 to 56.44 and HallusionBench from 67.6 to 69.0.Further gains are reported on MathVista, 68.8 →69.5, and RealWorldQA, 69.9 →70.5.
- Impact of the Diagnostic Module: Removing diagnostics leaves CharXiv near baseline, while full DPE rises from 36.8 to 40.91 across iterations; on MathVision, full DPE reaches 26.51 as the ablation declines.The diagnostic module is reported to reduce performance oscillation and distribution drift.
- Diagnosis-guided Data Distribution: Diagnosis increases sampling of underperforming categories, coinciding with CharXiv gains from 36.8 to 37.7 and MathVision progression through 26.28, 26.41, and 26.51.The reported redistribution targets text-dense, chart-related, mathematical-formula, and symbolic-reasoning samples.
- Impact of Image Retrieval and Editing: Removing image tools limits later gains: CharXiv reaches 38.1 instead of 40.91, a 2.81-point drop, while MathVision reaches 26.18 instead of 26.51.The paper links this difference to expanded visual diversity and long-tail coverage from retrieval and editing.
5. Conclusion
DPE introduces a diagnostic-generation-reinforcement framework for LMMs that identifies blind spots, constructs targeted data, and uses unlabeled multimodal resources. Experiments report stable training dynamics and sustained long-tail reasoning improvements.
- DPE integrates diagnosis, targeted data construction, and reinforcement to address model blind spots in LMM training.The framework uses cooperative multi-agent annotation of large-scale unlabeled multimodal resources.
- DPE provides controllable evolution directions and stable training dynamics for multimodal reasoning improvement.
- DPE sustains improvements on long-tail reasoning abilities that traditional self-expansion approaches fail to address.Experiments on several open-source LMMs report comprehensive reasoning enhancement using only a small amount of training data.
- Detailed analyses validate the diagnostic mechanism’s role in improving stability and mitigating marginal utility saturation.
Impact Statement
The approach is presented primarily as a methodological contribution to improving multimodal reasoning in large models. Its stated impacts concern data efficiency, training stability, and transparency in self-evolution pipelines.
- The approach aims to improve multimodal reasoning in large models through a diagnostic-driven machine learning framework.
- Its primary methodological impacts are more data-efficient capability enhancement, reduced training instability, and improved self-evolution transparency.
A. Case Study
The case study qualitatively compares VisPlay and DPE-generated questions. VisPlay examples can be unanswerable or structurally incomplete, whereas DPE produces complete, sufficiently informative, clear questions.
- VisPlay generates a question lacking necessary information, making it unanswerable from the image content.
- VisPlay also produces a multiple-choice question without options, resulting in an incomplete structure.
- DPE generates questions with complete structure, sufficient information, and clear semantics in the qualitative comparison.