PaperlayerView exact revision on arXiv

Source-linked AI summary

ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch

Zheng Liu, Honglin Lin, Chonghan Qin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Bin Cui, Conghui He, Wentao Zhang, Lijun Wu

arXiv:2601.13606v2cs.CV

TL;DR

Chart reasoning lacks scalable training data that is simultaneously diverse, complex, and reliable, limiting open-source VLM development. ChartVerse addresses this gap with RPE-guided chart synthesis and truth-anchored inverse QA, and its ChartVerse-8B model surpasses its 30B teacher while approaching Qwen3-VL-32B-Thinking. Its QA scope remains focused primarily on precise numerical and logical reasoning.

  • Problem

    Existing chart-reasoning datasets lack sufficient scale, diversity, complexity, and reliable reasoning supervision for open-source VLM development.

  • Method

    ChartVerse uses RPE to guide executable chart synthesis and generates QA from code-derived answers, filtering difficult samples with VLM fail-rate.

  • Results

    ChartVerse-8B surpasses Qwen3-VL-30B-A3B-Thinking and approaches Qwen3-VL-32B-Thinking on chart-reasoning benchmarks.

  • Takeaways & Limitations

    ChartVerse demonstrates that its synthesized datasets can support smaller chart-reasoning models with state-of-the-art benchmark performance.

  • Takeaways & Limitations

    The truth-anchored QA pipeline focuses primarily on precise numerical and logical reasoning rather than qualitative visual interpretations.

Abstract

from arXiv · show

Chart reasoning is a critical capability for Vision Language Models (VLMs). However, the development of open-source models is severely hindered by the lack of high-quality training data. Existing datasets suffer from a dual challenge: synthetic charts are often simplistic and repetitive, while the associated QA pairs are prone to hallucinations and lack the reasoning depth required for complex tasks. To bridge this gap, we propose ChartVerse, a scalable framework designed to synthesize complex charts and reliable reasoning data from scratch. (1) To address the bottleneck of simple patterns, we first introduce Rollout Posterior Entropy (RPE), a novel metric that quantifies chart complexity. Guided by RPE, we develop complexity-aware chart coder to autonomously synthesize diverse, high-complexity charts via executable programs. (2) To guarantee reasoning rigor, we develop truth-anchored inverse QA synthesis. Diverging from standard generation, we adopt an answer-first paradigm: we extract deterministic answers directly from the source code, generate questions conditional on these anchors, and enforce strict consistency verification. To further elevate difficulty and reasoning depth, we filter samples based on model fail-rate and distill high-quality Chain-of-Thought (CoT) reasoning. We curate ChartVerse-SFT-600K and ChartVerse-RL-40K using Qwen3-VL-30B-A3B-Thinking as the teacher. Experimental results demonstrate that ChartVerse-8B achieves state-of-the-art performance, notably surpassing its teacher and rivaling the stronger Qwen3-VL-32B-Thinking. We release our code, model weights, and datasets in https://chartverse.github.io.

1 Introduction

ChartVerse targets the lack of scalable, diverse, complex, and reliable chart-reasoning data by combining complexity-aware chart synthesis with truth-anchored QA generation. Its resulting datasets support smaller models that achieve strong chart-reasoning performance, including ChartVerse-8B surpassing its 30B teacher.

  • Motivation: Existing chart-reasoning data lacks the scale, diversity, and complexity needed for open-source VLM development.Manual datasets are costly to annotate, while synthetic charts and QA pipelines remain limited in visual complexity or reliability.
  • ChartVerse: ChartVerse is a scalable code-driven framework designed to synthesize complex, diverse, and reliable chart QA data without fixed templates or manual verification.The framework jointly targets visual complexity, distributional diversity, and reasoning reliability.
  • Chart Complexity: Rollout Posterior Entropy estimates intrinsic chart difficulty from the consistency of executable chart-code reconstructions across repeated VLM rollouts.Diverse and inconsistent reconstructions indicate higher posterior entropy and greater chart ambiguity.
  • Chart Synthesis: A complexity-aware chart coder uses RPE-filtered data and iterative generation, difficulty filtering, and retraining to produce diverse charts from scratch.High-temperature sampling enables exploration beyond fixed templates and seed-conditioned approaches.
  • QA Synthesis: Truth-anchored inverse QA computes answers from chart code before reverse-synthesizing questions and retaining difficult samples based on VLM fail-rate.This pipeline prioritizes correctness while emphasizing challenging reasoning cases.
  • Results: ChartVerse-8B surpasses Qwen3-VL-30B-A3B-Thinking, while ChartVerse-4B surpasses Qwen3-VL-8B-Thinking and ChartVerse-8B approaches Qwen3-VL-32B-Thinking.The models are trained using ChartVerse-SFT-600K and ChartVerse-RL-40K with Qwen3-VL-30B-A3B-Thinking as teacher.

2 Related Work

Prior chart systems use manual annotation, rule-based rendering, templates, or LLM-generated code, but existing approaches remain constrained in scalability, diversity, QA reliability, or reasoning depth.

  • Chart VLMs: Early chart VLMs primarily targeted structural extraction tasks such as OCR and element counting through visual instruction tuning.Later methods shifted toward reasoning-centric designs.
  • Chart Image Synthesis: Conventional chart synthesis relies on rule-based rendering engines, often producing rigid layouts with limited diversity.These methods visualize data tables using predefined rendering procedures.
  • Chart Image Synthesis: Recent LLM-based synthesis methods improve flexibility by generating executable code or augmenting fixed visual seeds, but remain constrained by finite design spaces.Their flexibility does not fully remove the limitations of fixed or bounded visual constructions.
  • Chart QA Construction: Human annotation provides accurate chart QA but does not scale, motivating automated generation methods.Automated pipelines address scalability but face verification and quality challenges.
  • Chart QA Construction: Without reliable ground truth, automated QA methods tend to produce uncalibrated, simplistic questions lacking multi-step Chain-of-Thought and accurate answers.These limitations constrain the reasoning depth and fidelity of generated chart QA.

3 Methodology

ChartVerse combines RPE-guided complexity measurement and generation with truth-anchored inverse QA synthesis to create diverse, challenging, and reliable chart-reasoning data.

  • 3.1 Quantifying Intrinsic Chart Complexity: RPE estimates intrinsic chart complexity from variability across executable-code reconstructions, with higher RPE indicating greater reproduction difficulty.The method repeatedly generates chart code, renders reconstructions, computes feature variation, and normalizes for execution failures.
  • 3.2 Complexity-Aware Chart Coder: RPE-guided filtering identifies difficult charts and supports a coder that generates non-standard, high-complexity charts from scratch.The coder iteratively samples, filters by complexity and similarity, retrains, and repeats this process to improve structural diversity.
  • 3.2 Complexity-Aware Chart Coder: The complexity-aware coder produces a large synthetic corpus of rendered charts and executable codes for downstream QA generation.The final synthetic dataset comprises 700k image-code pairs after RPE filtering.
  • 3.3 Truth-Anchored Inverse QA Synthesis: Truth-anchored inverse synthesis computes answers directly from chart code, generates questions conditioned on those answers, and retains only consistency-verified pairs.A deterministic Python execution environment produces the ground-truth answer before question generation and answer matching.
  • 3.3 Truth-Anchored Inverse QA Synthesis: The inverse pipeline operates in the textual code domain and is designed to ensure logically sound, factually accurate QA samples while avoiding visual encoding errors.The supplied passage states that this code-text construction enables efficient synthesis and eliminates potential visual encoding errors.

4 Experiments

ChartVerse models perform strongly across chart and STEM benchmarks, with gains accumulating through supervised fine-tuning and reinforcement learning. The results also show strong performance at smaller scales and student models surpassing their teacher.

  • ChartVerse models are evaluated on six complex chart-understanding benchmarks, using Qwen3-VL-Instruct initialization, SFT, and RL training.
  • 64.1 average score: ChartVerse-8B surpasses its 30B teacher at 62.9 and approaches Qwen3-VL-32B-Thinking at 67.0.
  • 54.3 average score: ChartVerse-2B exceeds chart-specific baselines, while ChartVerse-4B reaches 61.9 and outperforms Qwen3-VL-8B-Thinking at 60.0.
  • Training stages: ChartVerse-SFT and ChartVerse-RL consistently improve chart reasoning across model sizes, with RL adding gains after SFT.ChartVerse-2B rises from 42.5 to 49.8 after SFT and 54.3 after RL; ChartVerse-8B rises from 56.9 to 62.5 and 64.1.
  • Generalization: ChartVerse-trained models outperform the Instruct baseline across all evaluated STEM scales, indicating transfer to out-of-domain STEM reasoning.ChartVerse-8B improves from 56.7 to 61.2 after SFT and 62.2 with RL.

5 Ablations and Analysis

Ablations attribute ChartVerse’s gains to data quality, broad feature coverage, RPE-based hard-example selection, and truth-anchored QA synthesis with failure-rate filtering.

  • At a fixed 100K-sample budget, ChartVerse-SFT reaches 57.8 average score, exceeding competing datasets that provide marginal gains or degrade performance.CoSyn decreases performance from 53.9 to 51.0, while START and ECD reach 53.8 and 54.2.
  • RPE selection: RPE selects harder samples, with a 27.6% Qwen3-VL-4B-Instruct failure rate versus 21.1% for VLM-as-Judge and 23.5% for Python Code Complexity.
  • RPE selection: RPE-selected data achieves the highest downstream average score of 55.4 among the compared selection strategies.
  • Dataset diversity: ChartVerse-SFT spans a broader feature space than prior datasets and largely subsumes their distributions in t-SNE visualizations.
  • Dataset diversity: ChartVerse-SFT records the highest Color Distribution Entropy and Semantic Embedding Spread among existing datasets.
  • QA synthesis: Truth-anchored QA synthesis raises the average score to 57.4, and failure-rate selection further increases it to 57.8.Direct image-space and code-space generation reach 56.5 and 56.8, respectively.

6 Conclusion

ChartVerse addresses scarce complex chart-reasoning data with complexity-aware chart synthesis and truth-anchored QA generation. Its resulting datasets enable ChartVerse-8B to surpass its teacher and approach Qwen3-VL-32B-Thinking.

  • ChartVerse uses Rollout Posterior Entropy and a Complexity-Aware Chart Coder to synthesize diverse, non-trivial charts.
  • Truth-Anchored Inverse QA Synthesis uses code-derived ground truths, reverse-synthesized questions, and consistency checks to ensure accuracy.
  • ChartVerse-8B, trained on ChartVerse-SFT-600K and ChartVerse-RL-40K, surpasses Qwen3-VL-30B-A3B-Thinking and approaches Qwen3-VL-32B-Thinking.

Limitations

ChartVerse’s limitations concern computational cost, answer-type coverage, and teacher-model capacity. The authors identify lightweight RPE approximations, qualitative visual reasoning, and larger teachers as future directions.

  • Computational Overhead of RPE: RPE requires large-scale sampling, creating computational overhead despite mitigation through vLLM.The authors identify lightweight approximation methods as a future optimization direction.
  • Scope of Answer Types: Truth-Anchored Inverse QA primarily produces datasets focused on precise numerical and logical reasoning.Future work aims to add qualitative visual interpretations while preserving verification standards.
  • Teacher Model Capacity: The synthesis pipeline uses Qwen3-VL-30B-A3B-Thinking, while larger models may further improve generated-data reasoning depth and complexity.The authors specifically identify a capability gap relative to frontier models.

A More Training Details

The training pipeline fine-tunes Qwen3-VL-Instruct models with long visual and textual contexts, then applies GRPO to further align reasoning capabilities. Training uses distributed infrastructure and memory-efficiency techniques.

  • Infrastructure: The pipeline is implemented with LLaMA-Factory and Verl on 32 A100 GPUs across four nodes.The cluster contains 4 nodes with 8 A100 GPUs each.
  • Supervised Fine-Tuning: Supervised fine-tuning performs full-parameter updates on Qwen3-VL-Instruct models with dynamic image resolutions and a 22,000-token context cutoff.The visual encoder handles resolutions from 802,816 to 3,211,264 pixels.
  • Supervised Fine-Tuning: DeepSpeed ZeRO-3, BF16, and sequence packing improve memory and training efficiency during supervised fine-tuning.The 8B model’s full-parameter fine-tuning takes approximately 1.5 days.
  • Reinforcement Learning: Following supervised fine-tuning, GRPO further aligns the model’s reasoning capabilities using vLLM rollouts.The rollout samples 16 distinct responses per prompt at temperature 1.0 without a top-k restriction.

B Evaluation Details

Evaluation uses VLMEvalKit with domain-specific chart benchmarks converted into its interface. Response correctness is assessed with compassverifier rather than traditional exact string matching, alongside specified sampling settings.

  • Benchmark Preparation: ChartX, EvoChart, and ChartBench are preprocessed into the standard VLMEvalKit interface for evaluation.These domain-specialized datasets are not natively integrated into VLMEvalKit.
  • Evaluation Metric: The evaluation replaces exact string matching with compassverifier, an LLM-as-a-Judge used to assess response correctness.The verifier is intended to evaluate correctness more accurately than traditional exact matching.
  • Rollout Settings: Evaluation rollouts use temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, and a bounded response length.These settings follow OpenDataArena guidelines and official Qwen documentation.

C Data Synthesis Details

ChartVerse synthesizes training data through iterative chart generation, complexity filtering, decontamination, and truth-anchored QA construction. The resulting training phases and data-selection strategies are evaluated across chart and STEM benchmarks.

  • Data Initialization: The cold-start dataset Ccold initializes the chart coders, while Claude-4-Sonnet generates code representations for complex charts.The generation prompt is provided in Table 11.
  • Chart Coder Training: The complexity-aware chart coder uses Qwen2.5-Coder-7B-Instruct and applies consistent training settings across iterative stages.The coder is trained with full-parameter fine-tuning, sequence packing, and a 4,096-token context cutoff.
  • RPE Sampling: Sampling uses temperature 1.0, top-p 0.95, and top-k 20; RPE processes approximately 4 million samples on 64 A100 GPUs in 3 days.A second account specifies approximately 32 million inference calls to Qwen3-VL-2B-Thinking.
  • Data Decontamination: Training data undergoes rigorous decontamination, including removal of images with CLIP cosine similarity above 0.65 to any benchmark image.Because charts are synthesized from scratch, the number of filtered images is negligibly small.
  • Inverse Logic Construction: Inverse QA construction uses prompts for Python generation, reverse question synthesis, and consistency checking to enforce logical and factual consistency.The prompts are presented in Tables 13–15.
  • Inverse Logic Construction: Generated Python scripts must execute to a precise numerical value or definitive categorical label, and synthesized questions must match the execution result exactly.This answer-first procedure makes the QA answers deterministic.
  • Inverse Logic Construction: Each chart receives two Python scripts with separate reverse-question synthesis and consistency checks, requiring approximately 4 million teacher-model calls.This process takes approximately 4 days on 128 A100 GPUs.
  • CoT Distillation & Difficulty Filtration: CoT outputs are filtered for required structure, minimum length, and repeated 50-gram patterns before retention.Traces shorter than 100 words or containing a 50-gram repeated at least three times are discarded.

D.2 Comparison with Existing Datasets

ChartVerse-SFT improves benchmark performance under a fixed sample budget and uses RPE to select harder charts, while its synthesis supports both Instruct and Thinking models. RPE’s 2B Thinking configuration offers broad discrimination with reliable execution, and stronger teachers further improve data quality.

  • RPE ablation: 78.6 on CharXiv (DQ) is achieved by RPE-trained models, exceeding VLM-as-Judge 75.7 and Python Code Complexity 77.7.The corresponding model failure rate is 27.6, compared with 21.1 for VLM-as-Judge.
  • Model compatibility: The full ChartVerse-SFT-600K + ChartVerse-RL-40K recipe benefits both Instruct and Thinking bases, with Thinking models gaining additional improvements.The performance gap between base types remains relatively narrow, which the authors attribute to SFT reshaping Instruct-model reasoning behavior.
  • Teacher-student synthesis: ChartVerse’s truth-anchored synthesis and failure-rate filtering enable the student model to surpass its Qwen3-VL-30B-A3B-Thinking teacher.The teacher is selected to balance data quality and computational feasibility.
  • Teacher scaling: 64.1 is achieved by an 8B Instruct model trained on 100K samples synthesized by Qwen3-VL-235B-A30B-Thinking, outperforming the full 600K dataset from the 30B teacher.The ablation reports that a stronger teacher directly enhances synthesized data quality.
  • RPE configuration: RPE uses Qwen3-VL-2B-Thinking with K = 8 because it combines practical large-scale estimation, sharp complexity discrimination, and 88.6% code execution success.The 2B model provides the widest RPE range and highest standard deviation, while the Thinking variant exceeds the Instruct variant’s 63.2% execution success rate.

F ChartVerse-SFT Analysis

ChartVerse-SFT combines broad visualization coverage with structurally complex, multi-subplot charts and QA tasks requiring multi-step reasoning. Its examples include cross-subplot verification, derived statistical calculations, and aggregation across spatially distinct plots.

  • Visual diversity and complexity: ChartVerse-SFT spans statistical, 3D, hierarchical, and mixed-type visualizations, including intricate multi-subplot layouts and dashboards.Examples include violin plots, radar charts, treemaps, and high-dimensional 3D representations.
  • QA difficulty: ChartVerse QA pairs require multi-step reasoning and cross-subplot integration rather than simple data retrieval.The synthesized charts’ high information density supports challenging queries across multiple visual regions.
  • QA case studies: Figure 7 presents multi-condition verification across three metrics and distinct experiment sub-panels.The task sequentially validates average scores and total efficiency.
  • QA case studies: Figure 8 requires extracting quartiles and medians to determine the region with the greatest proportional variation.The task uses derived calculations rather than direct visual lookup.
Loading 2601.13606v2…