PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepVision-103K: A Visually Diverse, Broad-Coverage, and Verifiable Mathematical Dataset for Multimodal Reasoning

Haoxiang Sun, Lizhen Xu, Bing Zhao, Wotao Yin, Wei Wang, Boyu Yang, Rui Wang, Hu Wei

arXiv:2602.16742v1cs.LGcs.AI

TL;DR

Existing multimodal RLVR datasets have limitations in diversity and coverage, motivating broader training data. DeepVision-103K is a large-scale, visually diverse, verifiable K12 dataset; models trained on it achieve top performance on mathematical and general multimodal tasks.

  • Problem

    Existing multimodal RLVR training sets have limitations, while high-quality training data is central to progress in visual reflection and reasoning.

    Source: §1 · p. 1

  • Method

    DeepVision-103K is a large-scale, verifiable RLVR dataset curated through filtering, difficulty calibration, and correctness verification, spanning K12 mathematical and visual-logic problems.

    Source: §6 · p. 8

  • Results

    DeepVision models achieve top performance on mathematical and general multimodal tasks, outperforming models trained on other open datasets, same-base official thinking variants, and strong closed-source baselines.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper concludes that multimodal math data and visual-logic data contribute to each other.

    Source: §6 · p. 8

    The dataset focuses on K12 problems with unique final answers and does not fully cover open-ended tasks such as proof writing or multi-solution problems.

    Source: §7 · p. 9

Abstract

from arXiv · show

Reinforcement Learning with Verifiable Rewards (RLVR) has been shown effective in enhancing the visual reflection and reasoning capabilities of Large Multimodal Models (LMMs). However, existing datasets are predominantly derived from either small-scale manual construction or recombination of prior resources, which limits data diversity and coverage, thereby constraining further gains in model performance. To this end, we introduce \textbf{DeepVision-103K}, a comprehensive dataset for RLVR training that covers diverse K12 mathematical topics, extensive knowledge points, and rich visual elements. Models trained on DeepVision achieve strong performance on multimodal mathematical benchmarks, and generalize effectively to general multimodal reasoning tasks. Further analysis reveals enhanced visual perception, reflection and reasoning capabilities in trained models, validating DeepVision's effectiveness for advancing multimodal reasoning. Data: \href{https://huggingface.co/datasets/skylenage/DeepVision-103K}{this url}.

1 Introduction

The introduction argues that high-quality data is central to extending RLVR’s reasoning benefits to multimodal models, and presents DeepVision-103K as a diverse, broad-coverage dataset with an automatic curation pipeline. It reports that models trained on the dataset achieve top performance on mathematical and general multimodal reasoning benchmarks.

  • RLVR encourages problem decomposition and step-by-step self-correction, and recent work extends these reasoning benefits to large multimodal models (Wang et al., 2025a; Xia et al., 2025; Yang et al., 2025a).Examples of RLVR-trained LLMs include DeepSeek-R1 (DeepSeek-AI et al., 2025) and OpenAI o-series (OpenAI et al., 2024).Source: §1 · p. 1
  • Existing multimodal RLVR data is limited by synthetic datasets’ narrow real-world coverage (Lu et al., 2021; Qiao et al., 2025), expert-annotation scalability (Meng et al., 2025; Liu et al., 2024), and recombination’s overlap and lack of novel problems (Wang et al., 2025d; Zha et al., 2025; Peng et al., 2025; Yang et al., 2025b; Zhang et al., 2025).Source: §1 · p. 1
  • DeepVision-103K covers geometry, analytic plots, charts, and real-world mathematical items, with richer visual element types within each category than existing open-source datasets.The introduction points to Figure 1 for the comparison of visual element types.Source: §1 · p. 1, Figure 1 · p. 2
  • DeepVision-103K spans multimodal mathematical problems and visual logic tasks such as mazes, chess, and tetris to support mathematical and visual logic reasoning.Source: §1 · p. 1
  • An automatic curation pipeline uses validity filtering, pass-rate stratification, and correctness verification to transform diverse, noisy real-world K12 problems into structured, verifiable QA pairs.Source: §1 · pp. 1–2
  • DeepVision-trained models achieve top performance on mathematical and general multimodal reasoning, outperforming models trained on other open-source datasets, the same-base-model thinking variant, and strong closed-source baselines.The introduction points to Figure 2 for benchmark performance but provides no numerical results here.Source: Figure 2 · p. 2, §1 · p. 2

2 Overview of DeepVision-103K · 3 Construction of DeepVision-103K

DeepVision-103K combines rich annotations, diverse visual and mathematical coverage, and a three-stage curation process to produce verifiable multimodal math data for RL training.

2 Overview of DeepVision-103K

  • DeepVision-103K uses a rich annotation schema intended to support varied downstream multimodal reasoning tasks.Figure 3 presents a sample, and Table 1 is captioned as listing annotation fields and definitions; the supplied text does not specify those fields.Source: §2 · pp. 2–3, Table 1 · p. 3

2.1 Visual Diversity

  • A taxonomy based on (Mo et al., 2018) guided GPT-5 mini annotation of visual elements into categories and fine-grained types across six visual categories.The categories present distinct perceptual challenges.Source: §2.1 · p. 3
  • DeepVision captures cross-category visual combinations and real-world items in math contexts, requiring reasoning across multiple visual representations.The category coverage is summarized in Table 2.Source: §2.1 · p. 3, Table 2 · p. 3

2.2 Broad Coverage

  • DeepVision spans four major math disciplines, with geometry the largest, and includes over 200 fine-grained topics and nearly 400 knowledge points.It also includes visual logic problems such as mazes, chess, and Tetris from Zebra-CoT (Li et al., 2025) and GameQA (Tong et al., 2025).Source: Figure 5 · p. 3, §2.2 · p. 4

3 Construction of DeepVision-103K

  • The dataset starts from 3.3M K12 problems in MM-MathInstruct-3M (Wang et al., 2025c) and MultiMath-300K (Peng et al., 2024), then applies three curation stages.The stages filter validity and difficulty, then verify question-and-answer correctness.Source: §3 · p. 4
  • Validity filtering removes proof, explanation, and other unsuitable tasks, retaining only unique-answer questions that genuinely require visual information; this stage yields 880K questions.Rule-based keyword filtering is followed by Qwen3-VL-32B-Instruct (Bai et al., 2025) analysis of answer count and visual necessity.Source: §3 · p. 4
  • Difficulty filtering uses eight MiMo-VL-7B-SFT (Team et al., 2025) rollouts per question and MathVerify, discarding zero-pass and full-pass samples.The method retains intermediate-pass questions; the same rollout-and-filtering pipeline yields 26K clean, verifiable visual-logic examples from sources including Zebra-CoT (Li et al., 2025) and GameQA (Tong et al., 2025).Source: §3 · p. 4
  • Gemini-3-Flash checks question completeness, image–text consistency, and answer validity, leaving 77K correct and verifiable QA pairs for RL training.Only samples passing all three checks are retained, addressing corrupted text, image–text mismatches, and incorrect answers.Source: §3 · p. 4

4 Experiments

Experiments evaluate DeepVision-trained models on multimodal mathematics and general multimodal reasoning, reporting gains across mathematical benchmarks and general-purpose tasks.

4.1 Setup

  • GSPO training used answer-correctness rewards without format rewards, while baseline comparisons included closed-source, official-thinking, and open-source-dataset models.The trained models were MiMo-VL-7B-SFT-2508 (Team et al., 2025) and Qwen3-VL-8B-Instruct (Bai et al., 2025); evaluation covered multimodal math and general multimodal benchmarks, with a 32K maximum token length.Source: §4.1 · p. 5

4.2 Multimodal Mathematics Reasoning Results

  • Across mathematical benchmarks, DeepVision models improved over their respective Instruct/SFT baselines by 2.91% to 8.56%; Qwen3-VL-8B-DeepVision scored 85.11% on WeMath and MiMo-VL-7B-DeepVision scored 65.62% on LogicVista.The paper reports these as state-of-the-art results on WeMath and LogicVista, respectively.Source: §4.2 · pp. 5–6
  • DeepVision models surpassed official thinking variants and closed-source models on WeMath and LogicVista, and exceeded or substantially narrowed the gap on MathVision and MathVerse.MiMo-VL-7B-DeepVision also showed clear advantages over models trained on other open-source datasets.Source: §4.2 · pp. 5–6

4.3 Generalization Beyond Mathematics

  • DeepVision models improved over foundation models and surpassed official thinking variants across all three general multimodal benchmarks; other open-source-dataset models showed limited general-domain gains.The authors attribute this generalization to DeepVision’s diverse visual elements and broad domain coverage.Source: §4.1 · p. 5, §4.3 · p. 6

5 Analyses

Section 5 frames analyses around capabilities enhanced by RL on DeepVision-103K, the role of visual logic data, and the need for query-correctness verification. For enhanced capabilities, it compares Qwen3-VL-8B-Instruct with Qwen3-VL-8B-DeepVision and uses human annotation to analyze cases where DeepVision succeeds and Instruct fails.

5 Analyses

  • The section asks which capabilities are enhanced after RL on DeepVision-103K.Source: §5 · p. 6
  • It asks what role visual logic tasks, including mazes, tangrams, and games, play in DeepVision-103K.Source: §5 · p. 6
  • It asks whether query-correctness verification is necessary, given studies suggesting RLVR can work with random rewards (Wu et al., 2025; Shao et al., 2025).Source: §5 · p. 6

5.1 Enhanced Capabilities

  • The analysis compares Qwen3-VL-8B-Instruct and Qwen3-VL-8B-DeepVision, then has annotators examine cases where DeepVision succeeds but Instruct fails.The annotation seeks to identify the underlying improvement mechanism.Source: §5.1 · p. 6

1 Analyze visual descriptions in RI · 4 else · 6 end

The procedure checks for contradictions between visual descriptions and the image, then distinguishes visual misperception from incorrect reasoning. It also checks whether RD is correct at first observation, corrected via reflection, or shows a valid reasoning chain.

1 Analyze visual descriptions in RI

  • The procedure checks whether visual descriptions contradict the image.Source: §1 · p. 6

6 end

  • If the root cause is visual misperception, the procedure follows that branch.Source: §6 · p. 6
  • The procedure checks whether RD is correct at first observation.Source: §6 · p. 6
  • The procedure checks whether RD was corrected via reflection.Source: §6 · p. 6
  • If the root cause is incorrect reasoning, the procedure follows that branch.Source: §6 · p. 6
  • The procedure checks whether RD shows a valid reasoning chain.Source: §6 · p. 6

23 end

The analyses find that DeepVision improves visual perception, visual re-examination, and mathematical reasoning; ablations also show benefits from visual-logic data and query-correctness verification.

23 end

  • DeepVision strengthens one-shot visual perception, genuine visual re-examination after errors, and mathematical rigor when visual extraction is already correct.Examples include identifying a shaded region on the first attempt, revisiting visual content to correct errors, and systematically checking angle combinations rather than assuming symmetry.Source: §23 · pp. 6–7, Figure 8 · p. 7, Figure 9 · p. 7

5.2 The Value of Visual Logic Data

  • Under the same total exposure, Math-77K200 underperforms the mixed setting on math average (69.02% vs. 70.10%), with a clear LogicVista gap (63.53% vs. 65.92%).Source: §5.2 · p. 8

5.3 Necessity of query correctness verification.

  • DeepVision200 exceeds Unverified200 on math average (70.10% vs. 67.93%) and general average (70.92% vs. 69.61%), supporting query-correctness verification.The analysis attributes the gap to corrupted inputs or incorrect answers hindering progress and emphasizes reliable reward signals for multimodal RLVR.Source: Table 4 · p. 8, §5.3 · p. 8

5.2 The Value of Visual Logic Data

  • Visual-logic-only training improves over the foundation model across benchmarks, indicating positive transfer to mathematical and general evaluations.The authors attribute the gains to spatial reasoning and pattern recognition as shared primitives that visual-logic training strengthens more directly than multimodal math alone.Source: §5.2 · p. 8

6 Conclusion · 7 Limitations

DeepVision-103K is a diverse, verifiable K12 multimodal dataset for RLVR whose training supports strong multimodal performance and improved perception and reasoning. Its coverage and verification pipeline have limitations, including distribution imbalance, reliance on external models, and exclusion of open-ended tasks.

6 Conclusion

  • Training on DeepVision-103K yields top performance on mathematical and general multimodal tasks, with enhanced visual perception, reflection, and reasoning.Source: §6 · p. 8
  • DeepVision-103K is curated from diverse real-world K12 sources through validity filtering, pass-rate-based difficulty calibration, and query correctness verification.Source: §6 · p. 8
  • The dataset covers multimodal math and visual logic problems, including geometry, analytic plots, charts, and real-world items in mathematical contexts.Source: §6 · p. 8
  • Multimodal math and visual logic data contribute to each other in multimodal reasoning, and query correctness matters for multimodal RLVR training.Source: §6 · p. 8

7 Limitations

  • Despite increased visual diversity, the distribution is imbalanced, planar geometry dominates, and some rare element types remain underrepresented.Verification relies on strong external models such as Gemini, which can introduce bias and cost and may filter out some valid but hard samples.Source: §7 · p. 8
  • The dataset focuses on K12 problems with unique final answers for verifiable rewards, so it does not fully cover open-ended tasks such as proof writing or multi-solution problems.Those tasks require richer evaluation signals.Source: §7 · p. 9

A Visual Examples … C.1 Difficulty Filtering

The appendix illustrates cross-category visuals and describes visual-element annotation and difficulty-based data selection. It also reports that knowledge-guided retrieval outperforms training without retrieval under the same training steps.

A Visual Examples

  • The appendix presents cross-category visual-combination examples in DeepVision-103K, including solid geometry and analytic plots.Figure 11 is captioned “Solid Geometry & Analytic Plots.”Source: A Visual Examples · p. 11, Figure 11 · p. 11

B Visual Elements Annotation

C.1 Difficulty Filtering

  • For math data, examples with pass rates in [1/8, 4/8] are retained, while easier [5/8, 7/8] examples are selectively sampled to improve coverage and manage dataset size.The easier-range sampling prioritizes knowledge points under-represented in the [1/8, 4/8] portion.Source: §C.1 · p. 11
  • Figure 12 presents examples spanning planar geometry, solid geometry, and real-world items.The supplied caption associates these examples with the difficulty-filtering subsection.Source: Figure 12 · p. 11
  • Knowledge-guided retrieved data outperforms training without retrieval under the same training steps; the paper presents the top 10 retrieval knowledge points in Figure 13 and Table 6.The supplied passages do not provide the comparison’s numerical results.Source: §C.1 · p. 11, Figure 13 · p. 11, Table 6 · p. 12
  • Visual-logic training data uses Tetris, maze, and chess examples from Zebra-CoT (Li et al., 2025) and game data from GameQA-140K (Tong et al., 2025), filtered to pass rates in [3/8, 4/8].The selection broadens the training-data distribution while keeping dataset size manageable.Source: §C.1 · p. 11

C.2 Correctness Verification · C.3 Data Licenses · D Training Details

The sections describe a deterministic verification process that filters unreliable data, identify where data-collection protocols are listed, and summarize the training setup and prompt format.

C.2 Correctness Verification

  • GEMINI 3 FLASH jointly checks each image, question, and reference answer, producing a label and judge trace under a deterministic precedence hierarchy.The decision rule is summarized in Table 7.Source: §C.2 · p. 12
  • The verifier rejects malformed inputs, flags incorrect reference answers for correction, accepts only error-free instances, and discards flagged cases rather than applying automatic edits.Input errors include missing images or text and image-text mismatches; correction outputs include a revised solution.Source: §C.2 · p. 12

C.3 Data Licenses

  • The data-collection protocol sources are listed in Table 8.Source: §C.3 · p. 12

D Training Details

  • The authors used verl as the training framework, with configurations for DeepVision-series models listed in Table 9.Source: D Training Details · p. 12
  • Each training run used 32 H20 GPUs and took 0.5 hours per step.Source: D Training Details · p. 12
  • The prompt asks the model to reason step by step, re-check the image, and give a concise final answer in a boxed format.For multiple choice, it specifies only the answer letter in the box; for fill-in-the-blank and problem-solving, only the final answer.Source: D Training Details · p. 12

E Evaluation Details · E.1 Benchmarks · E.2 Inference Hyperparameters

The appendices describe evaluation benchmarks and model-family-specific inference settings, with matching settings to the corresponding instruct or base models for two DeepVision families.

E Evaluation Details

  • The evaluation section provides benchmark information and inference hyperparameters for each model.The overview identifies benchmark and inference-configuration material but does not provide the benchmark names or hyperparameter values in the supplied passages.Source: E Evaluation Details · p. 12
  • GEMINI 3 FLASH returns exactly one verification label per instance.Table 7 is described as listing the verification labels, but the labels themselves are not included in the supplied passage.Source: Table 7 · p. 13
  • The data-source documentation covers licenses and usage permissions.Table 8 is identified as the overview of licenses and permissions; the individual terms are not supplied.Source: Table 8 · p. 13
  • Training configurations for the DeepVision series models are documented in Table 9.The supplied caption identifies the table's scope but does not give configuration values.Source: Table 9 · p. 13

E.1 Benchmarks

  • The authors evaluate their models across three benchmark categories, summarized in Table 10.The supplied passages do not name the categories or benchmarks.Source: §E.1 · p. 13, Table 10 · p. 13

E.2 Inference Hyperparameters

  • Inference hyperparameters differ across model families to support performance, with detailed configurations listed in Table 11.The supplied passages do not provide the specific settings.Source: §E.2 · p. 13, Table 11 · p. 13
  • Qwen3-VL-DeepVision uses Qwen3-VL-Instruct hyperparameters, while MiMo-VL-DeepVision uses MiMo-VL hyperparameters.Source: §E.2 · p. 13

E.3 Evaluation Method · F Training Curves · G Potential Risks

Evaluation uses MathVerify followed by GPT-5-mini review of cases marked incorrect. Training curves indicate increasing response length and rewards alongside stable entropy, while the authors describe the dataset’s educational scope and safety measures as limiting potential risks.

E.3 Evaluation Method

  • MathVerify accuracy judgments were revised when GPT-5-mini re-judged cases marked incorrect, with revised judgments used as final labels.The re-judging was intended to reduce false negatives from parsing errors, equivalent expressions, or formatting variations.Source: §E.3 · p. 14

F Training Curves

G Potential Risks

  • The authors anticipate no significant risks, noting that DeepVision-103K uses public K12 educational content for verifiable-answer reasoning and contains no personal identifiers.It is designed for multimodal reasoning rather than sensitive decision-making.Source: G Potential Risks · p. 14

F Training Curves

G Potential Risks

Related papers

Loading 2602.16742v1…