Source-linked AI summary
DeepVision-103K: A Visually Diverse, Broad-Coverage, and Verifiable Mathematical Dataset for Multimodal Reasoning
Haoxiang Sun, Lizhen Xu, Bing Zhao, Wotao Yin, Wei Wang, Boyu Yang, Rui Wang, Hu Wei
TL;DR
Existing multimodal RLVR datasets have limited diversity and coverage because they rely heavily on small-scale manual construction or recombination of prior resources. DeepVision-103K addresses this gap with a curated, verifiable dataset spanning K12 mathematics and rich visual content. Models trained on it achieve top performance across mathematical and general multimodal tasks, while analyses report improved visual perception, reflection, and reasoning.
Problem
Existing multimodal RLVR training sets have limited data diversity and coverage because they predominantly use manual construction or recombination of prior resources.
Method
DeepVision-103K is curated from multimodal mathematical sources through validity filtering, pass-rate-based difficulty calibration, and query correctness verification.
Results
DeepVision-trained models achieve top performance on mathematical and general multimodal reasoning, including 85.11% on WeMath and 65.62% on LogicVista.
Takeaways & Limitations
Analyses report enhanced visual perception, reflection, and reasoning capabilities in models trained on DeepVision-103K.
Takeaways & Limitations
The dataset focuses on K12 problems with unique final answers and therefore does not fully cover open-ended mathematical tasks such as proof writing or multi-solution problems.
Abstract
from arXiv · showhide
Reinforcement Learning with Verifiable Rewards (RLVR) has been shown effective in enhancing the visual reflection and reasoning capabilities of Large Multimodal Models (LMMs). However, existing datasets are predominantly derived from either small-scale manual construction or recombination of prior resources, which limits data diversity and coverage, thereby constraining further gains in model performance. To this end, we introduce \textbf{DeepVision-103K}, a comprehensive dataset for RLVR training that covers diverse K12 mathematical topics, extensive knowledge points, and rich visual elements. Models trained on DeepVision achieve strong performance on multimodal mathematical benchmarks, and generalize effectively to general multimodal reasoning tasks. Further analysis reveals enhanced visual perception, reflection and reasoning capabilities in trained models, validating DeepVision's effectiveness for advancing multimodal reasoning. Data: \href{https://huggingface.co/datasets/skylenage/DeepVision-103K}{this url}.
1 Introduction
DeepVision-103K addresses limitations in existing multimodal RLVR training data by combining broad mathematical coverage with richer visual diversity and automatic curation. Models trained on it achieve top performance on mathematical and general multimodal reasoning benchmarks.
- Motivation: Existing multimodal RLVR datasets rely on synthetic construction, expert annotation, or recombination, limiting real-world coverage, scalability, or novelty.Synthetic datasets lack real-world mathematical scenarios, human annotation limits scalability, and recombination creates overlap without novel problems.
- Contribution: DeepVision-103K is proposed as a large-scale multimodal mathematical dataset designed for reinforcement learning with verifiable rewards.Its stated purpose is to address the limitations of existing training resources.
- Visual Diversity: DeepVision covers geometry, analytic plots, charts, and real-world mathematical items with richer visual element types than existing open-source datasets.Figure 1 compares the number of different visual element types across training datasets.
- Broad Coverage: The dataset combines broad multimodal mathematics with visual-logic problems such as mazes, chess, and tetris.This design is intended to jointly enhance mathematical and visual logic reasoning.
- Data Curation: Its automatic curation pipeline includes validity filtering, pass-rate stratification, and query correctness verification.The paper describes these stages as part of constructing the dataset for RLVR training.
- Results: Models trained on DeepVision-103K achieve top performance on mathematical and general multimodal reasoning benchmarks.The comparison includes models trained on other open-source datasets, the official thinking variant on the same base model, and strong closed-source baselines.
2 Overview of DeepVision-103K
DeepVision-103K is a richly annotated multimodal mathematical dataset spanning diverse visual elements, broad K12 topics, and a multi-stage curation pipeline for verifiable RL training.
- Dataset format: Each sample uses a rich annotation schema for multimodal reasoning tasks.
- Visual diversity: DeepVision includes diverse visual elements across six categories, including cross-category combinations and real-world items in mathematical contexts.
- Broad coverage: The dataset covers geometry, algebra, probability and statistics, and fundamental mathematical skills.
- Broad coverage: DeepVision contains over 200 fine-grained topics and nearly 400 distinct knowledge points.
- Data construction: The curation pipeline applies validity filtering, difficulty filtering through rollout pass rates, and query correctness verification.
- Data construction: Difficulty filtering removes zero-pass samples as too hard or unverifiable and full-pass samples as overly easy for exploration during RL training.
- Data construction: Query verification checks question completeness, image–text consistency, and answer correctness to remove corrupted samples.
4 Experiments
The experiments train multimodal models with correctness-based RL on DeepVision and evaluate them across mathematical and general multimodal benchmarks. DeepVision training yields consistent gains over foundation, thinking, and other dataset-trained baselines.
- Training setup: Models are trained with GSPO using rule-based rewards of +1 for correct answers and 0 otherwise.
- Evaluation: Evaluation covers four multimodal mathematics benchmarks and three general multimodal benchmarks.
- Results: DeepVision models improve over their respective Instruct or SFT baselines on every evaluated benchmark, with gains from 2.91% to 8.56%.
- Results: Qwen3-VL-8B-DeepVision reaches 85.11% on WeMath, while MiMo-VL-7B-DeepVision reaches 65.62% on LogicVista.
- Results: DeepVision models outperform models trained on other open-source datasets, demonstrating advantages as an RL training resource.
- Generalization: On all three general multimodal benchmarks, DeepVision models improve over foundation models and surpass official thinking variants.
5 Analyses
The analyses examine capability improvements after RL on DeepVision, the contribution of visual-logic data, and whether query correctness verification is necessary.
- Research questions: The analysis asks which capabilities improve after RL on DeepVision-103K and what role visual-logic tasks play.
- Capability analysis: Training on DeepVision produces increasing response length, rising rewards, and stable entropy.
- Capability analysis: The study compares Qwen3-VL-8B-Instruct with Qwen3-VL-8B-DeepVision across multiple benchmarks.
- Capability analysis: Human annotators analyze cases where DeepVision succeeds but Instruct fails using a defined protocol and verbatim response evidence.
4 else
The annotation protocol distinguishes visual misperception from reasoning errors and checks whether correct responses arise immediately or through reflection.
- Improvement mechanisms: The protocol identifies visual misperception when descriptions contradict the image.
- Improvement mechanisms: It distinguishes responses correct at first observation from those corrected through reflection.
- Improvement mechanisms: For incorrect reasoning, the protocol checks whether the DeepVision response contains a valid reasoning chain.
20 else
DeepVision training improves visual perception, visual reflection, and mathematical reasoning, while mixed multimodal-math and visual-logic data outperforms math-only training. Reliable query verification also materially improves results over unverified data.
- Enhanced Capabilities: DeepVision exhibits three enhancement types: stronger visual perception, visual reflection, and mathematical reasoning.The analysis identifies these categories through comparisons with the base model.
- Enhanced Capabilities: DeepVision correctly identifies geometric shapes, numerical values, and spatial relationships on the initial observation.This one-shot perception reduces the need for iterative visual re-examination.
- Enhanced Capabilities: DeepVision re-examines visual content after perceptual errors, whereas the base model tends to rephrase conclusions without revisiting the image.The re-examination includes recounting elements, remeasuring angles, and inspecting spatial relationships.
- Enhanced Capabilities: DeepVision applies more rigorous mathematical reasoning when both models extract identical visual information.One example systematically enumerates possible angle combinations instead of assuming symmetry without justification.
- Data Composition: Under equal total exposure, mixed training reaches 70.10% math average and 65.92% LogicVista, versus 69.02% and 63.53% for Math-77K200.The mixed setting combines multimodal math and visual-logic data, while Math-77K200 uses math alone.
- Data Quality: Unverified200 reaches 67.93% math average and 69.61% general average, below DeepVision200 at 70.10% and 70.92%, respectively.The comparison supports query correctness verification as an important component of the training pipeline.
6 Conclusion
DeepVision-103K combines broad multimodal coverage with verifiable RLVR-oriented curation and produces strong mathematical and general multimodal performance. Its main limitations are imbalanced visual coverage, reliance on external verification models, and focus on uniquely answerable K12 tasks.
- Conclusion: DeepVision-103K is curated from diverse real-world K12 sources through validity filtering, difficulty calibration, and query correctness verification.The dataset includes multimodal mathematics and visual logic across geometry, analytic plots, charts, and real-world mathematical items.
- Conclusion: Training on DeepVision-103K yields top performance on both mathematical and general multimodal tasks, alongside enhanced visual perception, reflection, and reasoning.The paper also reports that multimodal math and visual logic data contribute to each other in multimodal reasoning.
- Limitations: Visual diversity remains imbalanced, with planar geometry dominating and some rare element types underrepresented.This limits uniform representation across the visual taxonomy.
- Limitations: The curation pipeline relies on external models such as Gemini, introducing potential bias, additional cost, and possible removal of valid hard samples.These constraints affect the verification process rather than the dataset’s stated task coverage.
- Limitations: The dataset focuses on K12 problems with unique final answers and does not fully cover open-ended mathematical tasks requiring richer evaluation signals.Examples include proof writing and multi-solution problems.
A Visual Examples
The dataset presents cross-category visual examples and a taxonomy-based analysis of visual elements, while selecting data to broaden knowledge coverage and maintain manageable size. Retrieval-guided training is reported to outperform training without retrieval.
- Visual Examples: The visual examples span solid geometry, analytic plots, planar geometry, and real-world items.The paper explicitly frames these as cross-category combinations in DeepVision-103K.
- Visual Annotation: GPT-5 mini annotates visual elements according to a proposed taxonomy to characterize DeepVision-103K and existing datasets.The annotation uses temperature 0.1 and a low thinking budget.
- Data Selection: Math data selection retains examples from a target pass-rate interval and selectively samples easier examples to prioritize underrepresented knowledge points.This improves coverage while keeping dataset size manageable.
- Retrieval: Training with knowledge-guided retrieved data outperforms training without retrieval under the same training steps.The top 10 retrieval knowledge points are reported in Figure 13 and Table 6.
- Data Selection: Visual-logic data includes tetris, maze, chess, and game problems selected at a specified pass-rate range to broaden the training distribution.The selection also keeps the dataset size manageable.
C.2 Correctness Verification
The correctness-verification procedure jointly checks each multimodal instance and applies a strict hierarchy for input integrity, answer correctness, and acceptance. Instances requiring answer correction are discarded rather than automatically edited.
- Verification Procedure: Gemini 3 Flash jointly inspects the image, question, reference answer, and produces a label with a judge trace.The verifier follows a deterministic decision rule with a strict precedence hierarchy.
- Input Correctness: The verifier first rejects instances with missing images, missing text, or image–text mismatches.These conditions receive the labels ERR_IMG_MISSING, ERR_TEXT_MISSING, or ERR_MISMATCH.
- Answer Correctness: For well-formed inputs, the verifier evaluates the reference answer and outputs CORRECTION with a revised solution when the answer is incorrect.Answer evaluation occurs after input-level checks.
- Acceptance: An instance is accepted only when no input-level or answer-level errors are detected.Acceptance therefore requires both valid multimodal inputs and a correct reference answer.
- Acceptance: Instances flagged as CORRECTION are discarded instead of having their answers automatically replaced.This choice avoids introducing noise from automatic edits.
E Evaluation Details
The evaluation section covers three benchmark categories, model-family-specific inference settings, and data-source and training configuration details. It also documents verification labels and benchmark materials.
- The section provides information on evaluation benchmarks, inference hyperparameters, training configurations, and data-source licenses.
- GEMINI 3 FLASH returns exactly one verification label per instance, including error, correction, and perfect-match categories.
- Evaluation spans three benchmark categories summarized in Table 10.
- Inference hyperparameters differ across model families, with DeepVision variants matching their corresponding instruct models.
E.3 Evaluation Method
The evaluation method combines automated accuracy checking with secondary review, while the accompanying analysis tracks training dynamics through response length, rewards, and entropy. The paper also reports no anticipated significant risks for the dataset’s intended use.
- E.3 Evaluation Method: Final benchmark labels combine MathVerify accuracy with GPT-5-mini re-judgment of cases initially marked incorrect.The second review addresses parsing errors, equivalent expressions, and formatting variations.
- Training Dynamics: Training dynamics are analyzed through response length, trainset rewards, and entropy.These quantities correspond to Figures 14, 15, and 16, respectively.
- Risk Considerations: DeepVision-103K uses publicly available K12 educational content for verifiable-answer multimodal reasoning rather than sensitive decision-making.
- Risk Considerations: The dataset contains no personal identifiers, and curation filters out corrupted or unsafe samples.