PaperlayerView exact revision on arXiv

Source-linked AI summary

MiMo-VL Technical Report

Xiaomi LLM-Core Team, :, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, Bingquan Xia

arXiv:2506.03569v1cs.CL

TL;DR

The paper addresses how to build compact vision-language models with strong general understanding and multimodal reasoning. It combines four-stage, high-quality multimodal pre-training with Mixed On-policy Reinforcement Learning using verifiable and human-preference rewards. MiMo-VL-7B-RL achieves broad state-of-the-art performance, including 35 wins over Qwen2.5-VL-7B across 40 tasks and 59.4 on OlympiadBench.

  • Problem

    The paper seeks to build compact vision-language models that perform strongly across general visual understanding, multimodal reasoning, grounding, and preference alignment.

  • Method

    The authors combine four-stage pre-training on curated multimodal data with long-CoT reasoning examples and MORL integrating verifiable rewards with human feedback.

  • Results

    MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 of 40 evaluated tasks and scores 59.4 on OlympiadBench, surpassing models up to 72B parameters.

  • Takeaways & Limitations

    The released models and 50-plus-task evaluation suite establish a reproducible open-source basis for multimodal perception, reasoning, grounding, and preference evaluation.

  • Takeaways & Limitations

    Interference among reasoning, perception, grounding, and counting tasks makes stable simultaneous improvement across capabilities difficult.

Abstract

from arXiv · show

We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing models with up to 78B parameters. For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens) with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward signals. We identify the importance of incorporating high-quality reasoning data with long Chain-of-Thought into pre-training stages, and the benefits of mixed RL despite challenges in simultaneous multi-domain optimization. We also contribute a comprehensive evaluation suite covering 50+ tasks to promote reproducibility and advance the field. The model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.

1 Introduction

MiMo-VL-7B combines a native-resolution vision encoder, cross-modal projector, and reasoning-focused language model with four-stage pre-training and Mixed On-policy Reinforcement Learning. The resulting models achieve strong open-source performance across perception, reasoning, grounding, and preference evaluation, while simultaneous gains across capabilities remain challenging.

  • 1 Introduction: MiMo-VL-7B combines a native-resolution Vision Transformer, MLP projector, and reasoning-optimized MiMo-7B language model.The architecture is designed to preserve fine-grained visual details while supporting cross-modal alignment and complex reasoning.
  • 1 Introduction: The training pipeline uses four-stage pre-training on 2.4 trillion tokens followed by MORL integrating verifiable rewards and human feedback.Pre-training includes projector warmup, vision-language alignment, multimodal pre-training, and long-context SFT before post-training.
  • 1 Introduction: Long-chain-of-thought reasoning data in pre-training provides richer supervision for complex logical relationships and generalizable reasoning patterns.The report describes synthesized long-CoT data as improving performance and training efficiency.
  • 1 Introduction: MORL improves diverse capabilities but interference across domains makes stable simultaneous improvement difficult.The reported challenge involves differing response-length and task-difficulty trends across reasoning, perception, grounding, and preference tasks.
  • 1 Introduction: MiMo-VL-7B-RL scores 66.7 on MMMU and surpasses Qwen2.5-VL-7B on 35 of 40 evaluated tasks.The model also scores 59.4 on OlympiadBench, achieves 54.7 on OSWorld-G, and attains the highest Elo rating among open-source VLMs in the in-house preference evaluation.
  • 1 Introduction: The authors release model checkpoints and an evaluation suite covering more than 50 tasks to support transparency and reproducibility.The suite includes complete prompts and protocols for community use.

2 Pre-Training

MiMo-VL-7B pre-training uses a three-part vision-language architecture and a 2.4-trillion-token multimodal dataset curated across images, videos, text, OCR, grounding, GUI interactions, and reasoning. Dedicated curation and staged modality balancing produce a diverse dataset that supports visual perception, GUI understanding, and multimodal reasoning.

  • 2 Pre-Training: MiMo-VL-7B uses a Vision Transformer encoder, an MLP projector aligning visual features with the language model, and an LLM for textual reasoning.Qwen2.5-ViT supports native-resolution inputs, while MiMo-7B-Base supplies the language backbone.
  • 2 Pre-Training: 2.4 trillion tokens of multimodal data cover images, videos, text, captions, OCR, grounding, GUI interactions, reasoning examples, and text-only sequences.The dataset spans diverse modalities and task types rather than relying on a single visual data source.
  • 2 Pre-Training: Dedicated curation pipelines adjust modality proportions across training stages, while perceptual-hash deduplication reduces overlap with evaluation benchmarks.The procedures target data quality, training stability, efficiency, and reduced contamination.
  • 2 Pre-Training: The image-caption dataset is made balanced and diverse through deduplication, re-captioning, quality filtering, and bilingual metadata construction.The report states that this dataset improves generalization and qualitative performance, although those gains are not always reflected on specialized benchmarks.
  • 2 Pre-Training: OCR and grounding data include difficult text conditions, bounding-box annotations, multiple-object scenes, and complex referential expressions.These data target recognition robustness and spatial localization using absolute coordinates.
  • 2 Pre-Training: GUI pre-training covers mobile, web, and desktop platforms, with element and instruction grounding plus standardized action trajectories.Synthetic Chinese GUI data supplements open-source data, while unified actions reduce cross-platform conflicts.
  • 2 Pre-Training: The resulting high-fidelity data transfers MiMo-7B-Base reasoning abilities into multimodal contexts and supports strong performance across downstream benchmarks.Stage 4 long-context SFT is emphasized as enabling intricate STEM reasoning that generalizes to common perception tasks.

3 Post-Training

Post-training applies Mixed On-policy Reinforcement Learning to combine rule-based verifiable rewards with human-preference rewards across reasoning, perception, grounding, and multimodal tasks. The framework uses on-policy GRPO updates and a unified reward-service layer to coordinate heterogeneous reward functions and models.

  • 3 Post-Training: MORL jointly optimizes RLVR and RLHF to improve challenging reasoning tasks and alignment with human preferences.RLVR supplies rule-based rewards, while RLHF complements them for undesirable-behavior mitigation and preference alignment.
  • 3 Post-Training: RLVR covers visual reasoning, text reasoning, image grounding, visual counting, and temporal video grounding with task-specific verifiable rewards.Rewards use Math-Verify for mathematical correctness, GIoU or point accuracy for grounding, count accuracy, and temporal IoU for video segments.
  • 3 Post-Training: RLHF collects diverse multimodal and text-only queries, screens and clusters them, and uses pairwise-ranked responses to train reward models.Separate text-only and multimodal reward models are trained with the Bradley-Terry objective.
  • 3 Post-Training: The on-policy GRPO variant samples response groups and performs single-step policy updates using within-group reward-derived advantages.Compared with vanilla GRPO, it removes the need for a clipped surrogate objective and incorporates dynamic sampling, easy-data filtering, and resampling.
  • 3 Post-Training: Reward-as-a-Service provides a unified, low-latency interface that routes each query to the appropriate reward function or model.Rewards are deployed as standalone services and normalized to [0, 1].

4 Evaluation

The evaluation spans 50 tasks and finds MiMo-VL-7B models consistently leading across diverse vision-language, text, multimodal reasoning, GUI, and preference benchmarks. MiMo-VL-7B-RL particularly strengthens reasoning, GUI grounding, and user-preference performance relative to competing models.

  • General Capabilities: MiMo-VL-7B models demonstrate consistently leading performance across diverse vision-language and text benchmarks, establishing state-of-the-art results among open-source models and surpassing proprietary counterparts.Table 2 covers diverse visual-language and text benchmarks, with best open-source results identified in the comparison.
  • Reasoning Tasks: MiMo-VL-7B-RL improves multimodal reasoning on most benchmarks, including a MathVision increase from 57.9% to 60.4%.Both SFT and RL models outperform compared open-source baselines, while the RL model further improves most reasoning results.
  • GUI Tasks: MiMo-VL-7B-RL achieves comparable or superior performance to GUI-specialized models, particularly on the challenging Screenspot-Pro and OSWorld-G benchmarks.Figure 4 compares the general-purpose model with GUI-specialized systems across GUI understanding and grounding tasks.
  • Elo Rating: The evaluation combines benchmark testing with pairwise preference comparisons on a balanced bilingual dataset of real user prompts.Elo ratings use GPT-4o judgments and cover multimodal reasoning, image understanding, and GUI interaction scenarios.
  • Elo Rating: MiMo-VL-7B-RL achieves the highest Elo rating among evaluated open-source VLMs, ranking first across models from 7B to 72B parameters.Its performance closely approaches proprietary models such as Claude 3.7 Sonnet, while MORL boosts MiMo-VL-7B-SFT by more than 22 points.

5 Discussion

Later-stage reasoning data produces substantial gains, while mixed on-policy RL improves performance broadly but faces interference across task domains.

  • 5 Discussion: +9 on MMMU, +14 on OSWorld-G, and +16 on OlympiadBench follow the addition of synthetic long-form reasoning data in Stage 4.Performance continues improving without saturation during this stage.
  • 5 Discussion: On-policy RL improves consistently with more training data, whereas vanilla GRPO has higher early sample efficiency but only transiently.The on-policy learning curve shows no saturation within the observed training window.
  • 5 Discussion: MORL improves nearly all evaluated tasks, but stable simultaneous gains across reasoning, perception, and grounding remain difficult.Task interference makes matching standalone RL performance on individual tasks challenging.
  • 5 Discussion: Reasoning tasks lengthen Chain-of-Thought responses during RL, while grounding and counting tasks shorten them, contributing to interference.The report also identifies task-difficulty disparities and reward hacking as possible contributors.

6 Case Study

The case studies illustrate MiMo-VL-7B’s ability to interpret complex plots, solve STEM problems with explicit reasoning, and complete multi-step website interactions.

  • 6 Case Study: MiMo-VL-7B converts an intricate plot into a well-structured Markdown table, demonstrating plot-understanding capability.The report presents this as the top example in Figure 8.
  • 6 Case Study: MiMo-VL-7B-RL solves a beam-tension problem by balancing torques and estimates the wire tension at approximately 5900 N.The reasoning identifies the beam and block torques and uses the tension’s vertical component.
  • 6 Case Study: MiMo-VL-7B successfully navigates a website to add the Xiaomi SU7 to a wishlist while customizing paint and interior options.The demonstration uses screenshots with resolution 1886*1544.

7 Conclusions

The report presents MiMo-VL-7B models built with curated pre-training data and MORL, reports strong benchmark performance, and releases checkpoints and evaluation resources for reproducibility.

  • 7 Conclusions: MiMo-VL-7B-SFT and MiMo-VL-7B-RL demonstrate state-of-the-art performance across evaluated benchmarks.The report attributes this development to curated high-quality pre-training datasets and the MORL framework.
  • 7 Conclusions: The report identifies reasoning-data gains, on-policy RL advantages over vanilla GRPO, and task interference across diverse MORL capabilities.These are presented as key observations from model development.
  • 7 Conclusions: The authors open-source model checkpoints and a comprehensive evaluation suite to promote transparency and reproducibility in multimodal research.The report frames these resources as contributions for the research community.

B Model Configuration of MiMo-VL-7B

The model configuration combines a Qwen2.5-ViT visual encoder with a MiMo-7B-Base language backbone, using a larger language-model configuration than Qwen2.5-VL-7B in several dimensions.

  • B Model Configuration of MiMo-VL-7B: MiMo-VL-7B adopts Qwen2.5-ViT for native-resolution inputs and MiMo-7B-Base as its language backbone for reasoning capability.The configuration table identifies both components and their intended roles.
  • B Model Configuration of MiMo-VL-7B: Compared with Qwen2.5-VL-7B, the language backbone uses 36 versus 28 layers and a 4096 versus 3584 hidden size.The cited configuration passage also reports intermediate sizes of 11008 versus 18944.

C Evaluation Benchmarks

The evaluation spans 50 diverse tasks covering visual understanding, grounding, documents, video, GUI interaction, text-only benchmarks, and multimodal and text reasoning.

  • Benchmark coverage: 50 diverse tasks cover visual understanding, grounding and counting, document and chart understanding, video, GUI interaction, text-only benchmarks, and multimodal and text reasoning.The suite includes benchmarks such as MMMU, RefCOCO, ChartQA, Video-MME, OSWorld-G, GPQA, OlympiadBench, and AIME.
  • General Visual Understanding: General visual understanding is evaluated with image, multimodal, and real-world visual benchmarks including AI2D, BLINK, MMMU, MMBench, and VLMs are Blind.
  • GUI Understanding and Grounding: GUI understanding and grounding are evaluated on WebSrc, VisualWebBench, ScreenSpot, ScreenSpot-V2, ScreenSpot-Pro, and OSWorld-G.
  • Text Reasoning and Text-only Benchmarks: Text-only reasoning and general language performance are covered by MATH500, AIME 2024, AIME 2025, GPQA, SuperGPQA, DROP, MMLU-Pro, and IFEval.
  • Multimodal Reasoning: Multimodal reasoning is assessed with OlympiadBench, MathVision, MathVerse, DynaMath, WeMath, LogicVista, and MathVista.

D GUI Action Space

The GUI action space is documented through a syntax-and-definition reference that specifies the available actions and marks optional elements.

  • GUI Action Space: Table 5 summarizes the syntax and definition for each action in the GUI action space.The table caption specifies that “opt.” denotes optional.

E More Qualitative Examples

The qualitative examples illustrate MiMo-VL-7B-RL solving visual tasks involving whole-test-paper answering, OCR, perception, counting, and geometry.

  • Qualitative examples: MiMo-VL-7B-RL examples include answering all questions within a whole test paper and solving multiple geometry problems.
  • Qualitative examples: The model demonstrates reasoning on complex OCR and detailed perception problems.
  • Counting examples: 6 muffins are counted by summing two muffins in each of three grid rows.
  • Counting examples: 13 dogs are counted by combining 12 printed dogs on a blanket with one real dog in the foreground.
  • Geometry examples: A geometry example computes a right triangle’s height as 22 from hypotenuse 44 and a 30° angle.
Loading 2506.03569v1…