Source-linked AI summary
MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data
Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, Lantao Yu, Yicheng He, Jing Xie, Xiaomin Wu, Zhichao Liu, Jiarui Zhang, Fuxiao Liu
TL;DR
VLM self-evolution typically needs visual seed data, unlike recent data-free progress for LLMs. MM-Zero removes that dependency with a tri-role Proposer–Coder–Solver reinforcement-learning framework that generates, renders, and solves visual tasks. It reports consistent reasoning improvements across diverse multimodal benchmarks and multiple base models, while larger-scale validation remains unresolved.
Problem
VLM self-evolution requires visual inputs, and existing approaches remain dependent on costly, static image datasets or human-curated supervision.
Method
MM-Zero initializes Proposer, Coder, and Solver roles from one base model and trains them with reinforcement learning to generate concepts, render executable visual code, and solve the resulting images.
Results
MM-Zero achieves consistent VLM reasoning improvements across diverse multimodal benchmarks and multiple base models without external training data.
Takeaways & Limitations
MM-Zero demonstrates zero-data multimodal self-evolution through interactions among more than two roles, extending self-evolving training beyond the conventional dual-role setup.
Takeaways & Limitations
The scaling trend could not be validated on larger VLMs such as 38B models because of prohibitive computational cost.
Abstract
from arXiv · showhide
Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention. While recent approaches have demonstrated that LLM agents can self-evolve from scratch with little to no data, VLMs introduce an additional visual modality that typically requires at least some seed data, such as images, to bootstrap the self-evolution process. In this work, we present Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning. Moving beyond prior dual-role (Proposer and Solver) setups, MM-Zero introduces a multi-role self-evolving training framework comprising three specialized roles: a Proposer that generates abstract visual concepts and formulates questions; a Coder that translates these concepts into executable code (e.g., Python, SVG) to render visual images; and a Solver that performs multimodal reasoning over the generated visual content. All three roles are initialized from the same base model and trained using Group Relative Policy Optimization (GRPO), with carefully designed reward mechanisms that integrate execution feedback, visual verification, and difficulty balancing. Our experiments show that MM-Zero improves VLM reasoning performance across a wide range of multimodal benchmarks. MM-Zero establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.
1 Introduction
MM-Zero addresses the challenge of zero-data self-evolution for VLMs by generating visual content rather than relying on collected images or human labels. Its Proposer–Coder–Solver pipeline uses executable code and reinforcement learning to create, render, and reason over progressively harder visual tasks.
- Motivation: VLM self-evolution is difficult because visual reasoning requires images, while existing post-training depends on costly expert-curated tasks, labels, or static image datasets.Static images constrain evolution by their distribution, quality, and diversity, while sourcing and filtering them remains time-consuming and costly.
- MM-Zero: MM-Zero is a zero-data reinforcement learning framework that trains VLMs without external images, questions, or human-annotated data.The framework generates visual content internally, beginning with simple scenes and progressively creating more complex ones.
- Tri-role pipeline: The Proposer formulates visual concepts and questions, the Coder converts them into executable Python or SVG code, and the Solver reasons over the rendered images.All three roles are initialized from the same base model and trained with GRPO.
- Tri-role pipeline: The tri-role pipeline forms a closed loop in which abstract language, executable code, and visual reasoning mutually reinforce one another.The Proposer creates challenging yet solvable tasks, while the Coder receives execution and visual-quality feedback from the Solver.
- Results: MM-Zero produces consistent visual-reasoning gains across diverse VLM benchmarks and multiple model sizes, demonstrating self-play-based emergence without real-world training data.The framework was applied to Qwen3-VL-Instruct 4B and 8B and Mimo-VL-7B-Instruct.
2 Methodology
MM-Zero trains three interacting VLM roles with verifiable rewards, using generated code and images to create and solve multimodal tasks without external data. GRPO updates the roles through execution feedback, semantic verification, and difficulty balancing.
- Optimization: RLVR assigns binary rewards when generated outputs are verifiably correct, and GRPO normalizes rewards across response groups while constraining policy drift with KL regularization.GRPO avoids a learned value function by computing relative rewards among multiple samples from the same prompt.
- Framework: MM-Zero evolves Proposer, Coder, and Solver agents from the same base model, optimizing each sequentially with GRPO while freezing the other two roles.Role-specific rewards form a closed training loop without additional data.
- Data construction: Training data are filtered by role-specific criteria, including coder rendering success rates between 0.25 and 0.75 and retention of successfully rendered images for Solver training.The pipeline uses proposer outputs to train the Coder and proposer–coder outputs to construct Solver data.
- Framework: The Proposer generates a scene description with easy and hard questions, while the Coder renders the description into images and the Solver answers both questions.Easy-question answers verify rendering semantics; hard-question answers support Solver training and Proposer difficulty rewards.
- Reward design: The Proposer reward combines successful execution, capped solvability, difficulty, formatting, and diversity criteria to favor renderable tasks that are challenging but solvable.Rendering success is indicated by executable code, while solvability measures agreement with the intended easy answer and difficulty peaks at Solver uncertainty c_i = 0.5.
3 Experiment
MM-Zero is evaluated across diverse visual reasoning benchmarks, model sizes, and continued training iterations. Performance improves consistently, with especially strong gains in visual mathematics and increasingly capable generated visual tasks.
- Main Results: 54.1% average visual math reasoning accuracy is achieved by Qwen3-VL-8B-Instruct at iteration 3, versus 50.7% for the base model.The reported improvement is 4 percentage points without any data.
- Main Results: Average scores improve across model sizes, including 50.2 to 53.4 for the 4B model and 50.9 to 56.0 for Mimo-VL-7B-SFT.The 4B model shows the smallest gain, alongside lower initial rendering success than the 7B and 8B models.
- Generated Data Quality: Coder rendering success and image solvability increase during training, indicating stronger code generation and more faithful, answerable rendered images.Solvability measures the fraction of successfully rendered images enabling a ground-truth answer on easy questions.
- Continued Training: 56.6% average accuracy is reached at iteration 5 for Qwen3-VL-8B-Instruct, with performance continuing to improve beyond three iterations.The paper reports that solver performance does not saturate through the additional iterations.
- Generated Task Evolution: Across iterations, generated visuals become cleaner and questions progress toward multi-step compositional reasoning while retaining foundational questions.By iteration 3, examples require combining chart values with percentage annotations to derive answers.
4 Ablation Study
The ablation study shows that MM-Zero’s reward design is important for maintaining difficult, diverse, and generalizable generated tasks. Removing solvability balancing or content diversity causes reward hacking or overfitting.
- Reward Design: MM-Zero’s three-role framework requires reward components beyond the difficulty and diversity rewards used in prior two-role self-evolving systems.The coder’s image-rendering responsibility motivates additional code and visual-quality reward design.
- Solvability and Difficulty Balance: Capping solvability at 0.5 balances solvability and difficulty rewards, discouraging answer text from being embedded directly in rendered images.The uncapped condition increasingly produces explicitly solvable images as training progresses.
- Solvability and Difficulty Balance: Removing solvability capping reduces improvement to 2.3% over baseline, compared with 3.9% for MM-Zero from 50.2% to 54.1%.Without the cap, solvability receives disproportionate reward weight and encourages easy rendered questions.
- Content Diversity: Removing content diversity yields 51.7% accuracy at iteration 1 versus 50.3% baseline, followed by steadily declining accuracy.The condition overfits to narrow visual problem types that are easy to render and reward.
- Content Diversity: Without diversity reward, accuracy drops to 49.4% by iteration 3 as generated content converges toward easy visual types such as histograms.This shortcut emerges early and improves reward while degrading performance on diverse benchmarks.
5 Related Work
Self-evolution has expanded from LLMs to VLMs, but multimodal systems must also generate or source visual data. Existing approaches adapt proposer–solver pipelines while remaining dependent on collected image datasets.
- Self-Evolution: Self-evolution uses independently generated and refined experiences, with recent success in LLM reasoning and broader tasks with programmatically evaluated correctness.The paradigm includes self-play, synthetic data generation, and reinforcement learning.
- Vision-Language Models: VLM self-evolution is more challenging than single-modality self-evolution because training requires generating or sourcing additional visual data.This distinguishes multimodal self-evolution from approaches operating over text alone.
- Existing Approaches: Existing VLM approaches such as VisPlay adapt challenger–solver pipelines but require a collection of static images.MM-Zero addresses this dependence by generating visual data within its self-evolving loop.
6 Limitation
MM-Zero demonstrates self-evolution across multiple model sizes, but the paper does not validate whether the observed scaling trend extends to larger VLMs.
- Scope Boundary: The scaling trend remains unvalidated on larger VLMs such as 38B-parameter models because of prohibitive computational cost.The authors identify full scaling behavior as an important direction for future work.
7 Conclusion
MM-Zero trains VLMs from zero data through three specialized roles initialized from one base model. Sequential role training and tailored rewards improve reasoning, while future work targets richer tools, larger models, and additional roles.
- MM-Zero trains VLMs starting from zero data with Proposer, Coder, and Solver roles initialized from the same base model.
- Tailored reward functions and sequential training across roles progressively improve the agents’ reasoning capabilities.
- Future work includes broader tool use for diagrams, plots, and 3D renderings to create richer visual training data.
- The framework could be tested with larger base models and extended with additional agent roles.
A Prompt Templates
The appendix specifies prompts for visual content design, SVG generation, Solver reasoning, and LLM-based answer judging. These templates constrain outputs, question difficulty, rendering content, reasoning format, and correctness evaluation.
- A.1 Proposer: The Proposer outputs six XML blocks covering content type, caption, easy and hard questions, and their answers.Allowed content types include data_chart, diagram, geometry, timeline, map, table, and other.
- A.1 Proposer: Proposer captions must include at least three complexity features such as multiple series, annotations, colors, derived values, or non-trivial patterns.
- A.1 Proposer: Hard questions must require multi-step reasoning and extraction of at least one value from the visualization.
- A.1 Proposer: Answers are restricted to a single number, word, or short phrase.
- A.2 CodeGen: The CodeGen prompt generates raw SVG markup from a caption and questions, with the rendered image containing the data needed for the easy question.
- A.3 Solver: The Solver receives an image, reasons step-by-step inside <think> tags, and returns a short final answer inside \boxed{}.
- A.4 LLM-as-a-Judge: Evaluation extracts each model response’s final answer and uses Qwen2.5-14B-Instruct as a judge across benchmarks.
- A.4 LLM-as-a-Judge: The judge accepts equivalent numeric answers, option variants, and semantic paraphrases, returning exactly Yes or No.
B Training Configuration
The training configuration tables document main script parameters and per-role settings. The models use full fine-tuning with a trainable vision tower.
- Tables 4 and 5 list the main script parameters and per-model training configuration.
- All three models use full fine-tuning with LoRA rank 0, and the vision tower is trainable.
- ∼8.77B total trainable parameters are reported for Qwen3-VL-8B-Instruct.
C Rendering Pipeline
The SVG-only rendering pipeline converts generated SVG strings into validated PNG images for Solver input, using timeouts and parallel workers.
- Generated SVG markup is converted to PNG with cairosvg before being provided to the Solver.
- Each snippet has a 30 s timeout, and conversion uses parallel ProcessPoolExecutor workers.
- Images exceeding aspect-ratio or dimension limits are discarded, while valid outputs become base64-encoded PNGs for vLLM input.