Source-linked AI summary
MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning
Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, Linjiang Huang, Si Liu, Rui Liu, Hongsheng Li
TL;DR
MathCanvas addresses the limitations of textual and tool-dependent visual reasoning for mathematics by training unified LMMs to generate, edit, and strategically interleave visual aids with textual deduction. Its BAGEL-Canvas model achieves an 86% relative improvement over strong baselines on MathCanvas-Bench.
Problem
Textual chain-of-thought and rigid tool-based VCoT approaches limit mathematical reasoning in domains requiring flexible, high-fidelity visual aids.
Method
MathCanvas uses two-stage training with large datasets for diagram generation and editing, followed by strategic visual-aided reasoning, and evaluates models on a 3K-problem benchmark.
Results
86% relative improvement over strong baselines on MathCanvas-Bench, with BAGEL-Canvas achieving a 34.4% weighted score and surpassing all open-source competitors.
Takeaways & Limitations
The training paradigm teaches models when and how to draw while enhancing textual reasoning, providing a foundation for broader multimodal reasoning.
Takeaways & Limitations
Calculus & Vector shows only a +0.8 gain, indicating that this domain may require specialized reasoning capabilities beyond the current visual augmentation techniques.
Abstract
from arXiv · showhide
While Large Language Models (LLMs) have excelled in textual reasoning, they struggle with mathematical domains like geometry that intrinsically rely on visual aids. Existing approaches to Visual Chain-of-Thought (VCoT) are often limited by rigid external tools or fail to generate the high-fidelity, strategically-timed diagrams necessary for complex problem-solving. To bridge this gap, we introduce MathCanvas, a comprehensive framework designed to endow unified Large Multimodal Models (LMMs) with intrinsic VCoT capabilities for mathematics. Our approach consists of two phases. First, a Visual Manipulation stage pre-trains the model on a novel 15.2M-pair corpus, comprising 10M caption-to-diagram pairs (MathCanvas-Imagen) and 5.2M step-by-step editing trajectories (MathCanvas-Edit), to master diagram generation and editing. Second, a Strategic Visual-Aided Reasoning stage fine-tunes the model on MathCanvas-Instruct, a new 219K-example dataset of interleaved visual-textual reasoning paths, teaching it when and how to leverage visual aids. To facilitate rigorous evaluation, we introduce MathCanvas-Bench, a challenging benchmark with 3K problems that require models to produce interleaved visual-textual solutions. Our model, BAGEL-Canvas, trained under this framework, achieves an 86% relative improvement over strong LMM baselines on MathCanvas-Bench, demonstrating excellent generalization to other public math benchmarks. Our work provides a complete toolkit-framework, datasets, and benchmark-to unlock complex, human-like visual-aided reasoning in LMMs. Project Page: https://mathcanvas.github.io/
1 Introduction
MathCanvas addresses the limitations of textual and tool-dependent visual reasoning by training unified LMMs to generate, edit, and strategically interleave diagrams with mathematical deduction. It contributes large-scale visual-manipulation and reasoning datasets plus a benchmark for interleaved visual-textual solutions.
- MathCanvas targets intrinsic VCoT for mathematics by interleaving textual deduction with the creation and modification of visual aids.
- The framework uses two complementary phases: Visual Manipulation builds diagram generation and editing, while Strategic Visual-Aided Reasoning teaches when to interleave visual actions with textual reasoning.
- MathCanvas-Instruct contains 219K interleaved visual-textual reasoning examples designed to coordinate diagrammatic actions with reasoning trajectories.
- MathCanvas-Bench contains 3K problems requiring coherent interleaved reasoning and visual outputs, and benchmarks 20 leading LMMs.
- The 15.2M-pair pretraining corpus combines 5.2M step-by-step editing trajectories with 10M caption-to-diagram pairs.
2 Related Work
Prior multimodal mathematical reasoning advances either improve vision-text alignment or integrate visual aids through external tools or intrinsic generation. MathCanvas addresses the remaining lack of dynamic, step-by-step visual reasoning resources.
- Visual Chain-of-Thought: Existing VCoT methods mainly use external vision models or code interpreters, but their rigidity and input-format dependence limit broader applicability.
- Visual Chain-of-Thought: Intrinsic VCoT lets unified LMMs natively generate visual thoughts, yet prior attempts remained confined to simple domains and had not succeeded in mathematics.
- Datasets and Benchmarks for Multimodal Mathematical Reasoning: Existing multimodal mathematics benchmarks use static problem-solution pairs and lack step-by-step visual demonstrations for dynamic, process-oriented reasoning.
- Datasets and Benchmarks for Multimodal Mathematical Reasoning: MathCanvas addresses this gap with MathCanvas-Instruct and MathCanvas-Bench for interleaved visual-textual reasoning.
3 Method
MathCanvas builds datasets for diagram manipulation, interleaved visual-textual reasoning, and evaluation, then trains a unified LMM in two stages. The recipe first establishes visual generation and editing, then jointly teaches strategic visual reasoning.
- 3.1.1 Million-scale Pretraining Corpus: MathCanvas-Edit provides 5.2M step-by-step edit trajectories from competition-level and systematically generated geometric structures.
- 3.1.1 Million-scale Pretraining Corpus: MathCanvas-Imagen contains 10M caption-to-diagram pairs, supporting textual generation of mathematical diagrams alongside editing pretraining.
- 3.1.2 MathCanvas-Instruct: MathCanvas-Instruct is a large-scale dataset for interleaved visual-textual reasoning, constructed through filtering, deduplication, image enhancement, and taxonomy classification.
- 3.2 The MathCanvas-Bench Evaluation Benchmark: MathCanvas-Bench samples 3K non-multiple-choice problems with weighted category sampling and evaluates complete accuracy alongside weighted scoring of intermediate sub-questions.
- 3.3 Two-stage Training Recipe: The two-stage recipe first trains visual manipulation on MathCanvas-Edit and MathCanvas-Imagen, then fine-tunes the entire model on MathCanvas-Instruct for strategic visual-aided reasoning.
- 3.3 Two-stage Training Recipe: Stage I freezes the understanding pathway and trains the Generation Expert, whereas Stage II unfreezes all components to jointly learn visual-textual reasoning.
4 Experiments
BAGEL-Canvas leads open-source models on MathCanvas-Bench and generalizes across public multimodal math benchmarks. Gains are largest in geometry-heavy subjects, while ablations associate performance drops with removing visual pretraining or visual reasoning.
- 4.1 Benchmark Results: 34.4% weighted score makes BAGEL-Canvas the top-performing open-source model, exceeding Qwen-2.5-VL-72B (32.8) and InternVL3.5-30B-A3B (22.2).It also surpasses Gemini-2.0-Flash (32.6) and GPT-4.1 (30.0).
- 4.1 Benchmark Results: Trigonometry (+27.1), Plane Geometry (+19.2), and Solid Geometry (+12.3) show BAGEL-Canvas’s largest domain gains.Analytic Geometry (+14.1) and Algebra (+11.8) also improve, whereas Calculus & Vector gains only +0.8.
- 4.2 Generalization: BAGEL-Canvas improves over BAGEL across MathVista, MathVision, and MathVerse, with particularly strong gains of +17.9 on MathVerse and +10.5 on MathVista.Table 2 reports generalization performance across these three public benchmarks.
- 4.3 Ablation Studies: Removing MathCanvas-Edit lowers weighted score by 2.4 points, while removing the entire pretraining stage causes an additional 1.2-point decrease.The ablation attributes importance to step-by-step diagram editing and the broader visual pretraining corpus.
- 4.3 Ablation Studies: Training without visual data lowers weighted score by 3.5 points, while constraining the full model to skip image generation also produces a significant drop.These comparisons test the contribution of visual data during training and visual generation during inference.
- 4.3 Ablation Studies: The – Skip Image variant scores 1.0 point higher than BAGEL-Canvas-Text despite both producing text-only solutions.The authors interpret this as evidence that interleaved training also improves underlying textual reasoning.
5 Conclusion
MathCanvas equips LMMs with intrinsic visual chain-of-thought for mathematical reasoning through two-stage training on diagram and interleaved reasoning data. BAGEL-Canvas achieves an 86% relative improvement over strong baselines on MathCanvas-Bench while also enhancing textual reasoning.
- 5 Conclusion: MathCanvas combines MathCanvas-Edit, MathCanvas-Imagen, and MathCanvas-Instruct in a two-stage recipe for diagram manipulation and interleaved visual-textual deduction.The framework trains BAGEL-Canvas to learn when and how to draw during mathematical reasoning.
- 5 Conclusion: BAGEL-Canvas achieves an 86% relative improvement over strong baselines on MathCanvas-Bench.The benchmark evaluates intrinsic visual chain-of-thought for mathematical reasoning.
- 5 Conclusion: The training paradigm is reported to enhance core textual reasoning in addition to teaching strategic visual-aid use.The conclusion frames MathCanvas as a foundation for broader and more complex multimodal reasoning.
A Training Details
Training uses two stages with different optimization objectives. Stage I trains visual generation, while Stage II jointly optimizes textual token prediction and diagram generation.
- Implementation: The framework is implemented on BAGEL-7B-MoT and trained with AdamW on 16 NVIDIA H800 GPUs.Detailed hyperparameters for both stages are provided in Table 5.
- Stage I: Stage I freezes the understanding expert, trains only the generation expert, and uses the Rectified-Flow objective for diagram generation.A ViT condition dropout rate of 0.3 is used for regularization.
- Stage II: Stage II unfreezes all components and combines Cross-Entropy token prediction weighted by 0.25 with Rectified-Flow diagram generation weighted by 1.0.The learning rate is halved and training steps are reduced for fine-tuning.
B.1 MathCanvas-Edit and MathCanvas-Imagen
MathCanvas-Edit and MathCanvas-Imagen provide complementary data for diagram editing and generation, supported by automated geometric construction and diverse training examples. MathCanvas-Instruct contains 219K problems with both multimodal and text-only instances.
- Foundational Structure Generation: The foundational generation pipeline starts from 18 geometric primitives and incrementally applies 41 predefined geometric relations.Relations are grouped into 37 one-point and 4 two-point constructions to build progressively complex diagrams.
- Geometric Primitive Set: The primitive inventory includes triangles, quadrangles, and polygons such as isosceles triangles, rectangles, trapezoids, and equilateral pentagons.The listed objects define the basic geometric structures sampled by the automated process.
- MathCanvas-Instruct: MathCanvas-Instruct contains 219K problems, with 65% multimodal and 35% text-only examples.The dataset statistics also cover problem sources, question and solution lengths, and image counts.
C.1 Weighted Scoring Weights
Weighted Scoring assigns greater importance to later sub-questions using exponentially increasing weights, with normalized weights provided for problems containing two to four sub-questions.
- C.1 Weighted Scoring Weights: Later sub-questions receive greater weight through an exponential growth factor of 1.3.The weighting formula assigns the i-th sub-question a normalized weight based on 1.3^(i−1).
- C.1 Weighted Scoring Weights: The final problem score is the sum of the weights of correctly answered sub-questions.
- C.1 Weighted Scoring Weights: For two sub-questions, the normalized weights are [0.4348, 0.5652].
- C.1 Weighted Scoring Weights: For three sub-questions, the normalized weights are [0.2506, 0.3258, 0.4236].
- C.1 Weighted Scoring Weights: For four sub-questions, the normalized weights are [0.1616, 0.2101, 0.2732, 0.3551].
D Additional Qualitative Results
Additional qualitative examples illustrate how visual manipulation supports mathematical reasoning and contrast text-only or alternative visual approaches with MathCanvas outputs.
- D Additional Qualitative Results: Text-only solutions from Gemini-2.5-Pro and GPT-5 use complex or sometimes flawed algebraic approaches on a problem benefiting from visual manipulation.
- D Additional Qualitative Results: The qualitative comparisons include BAGEL-Zebra-CoT, Nano-Banana, and MathCanvas.
- D Additional Qualitative Results: One example asks for the relationship between CD and CE in a right trapezoid with parallel and perpendicular constraints.
- D Additional Qualitative Results: MathCanvas-Instruct examples present mathematical reasoning together with corresponding visual inputs or actions.
- D Additional Qualitative Results: Another example asks for the value of m in a constrained linear objective problem, whose stated solution gives m = 3 and option A.