Source-linked AI summary
Ovis2.5 Technical Report
Shiyin Lu, Yang Li, Yu Xia, Yuwei Hu, Shanshan Zhao, Yanqing Ma, Zhichao Wei, Yinglun Li, Lunhao Duan, Jianshan Zhao, Yuxuan Han, Haijun Li, Wanying Chen, Junke Tang, Chengkun Hou, Zhixing Du, Tianli Zhou, Wenjie Zhang, Huping Ding, Jiahe Li, Wen Li, Gui Hu, Yiliang Gu, Siran Yang, Jiamang Wang, Hailong Sun, Yibo Wang, Hui Sun, Jinlong Huang, Yuping He, Shengze Shi, Weihong Zhang, Guodong Zheng, Junpeng Jiang, Sensen Gao, Yi-Feng Wu, Sijia Chen, Yuhui Chen, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang
TL;DR
Ovis2.5 targets Ovis2’s limitations in native-resolution visual perception and deep multimodal reasoning, which matter for dense visuals and complex tasks. It combines NaViT, reflection-oriented training with optional thinking mode, and a five-phase curriculum, achieving leading open-source results including 78.3 on OpenCompass for Ovis2.5-9B.
Problem
Ovis2 struggled with visually dense content and deep reasoning because fixed-resolution tiling compromised visual structure and training lacked reflective self-correction.
Method
Ovis2.5 combines native-resolution NaViT perception, reflection-oriented reasoning with optional thinking mode, five-phase training, and efficient multimodal infrastructure.
Results
Ovis2.5 achieves leading open-source performance across overall multimodal, STEM, grounding, video, and complex chart-analysis benchmarks, with Ovis2.5-9B scoring 78.3 on OpenCompass.
Takeaways & Limitations
The released 9B and 2B models extend strong multimodal performance across model scales, including resource-constrained scenarios for the smaller model.
Takeaways & Limitations
Future work remains on 4K-level perception, long-input video with richer temporal reasoning, and tighter tool-use integration.
Abstract
from arXiv · showhide
We present Ovis2.5, a successor to Ovis2 designed for native-resolution visual perception and strong multimodal reasoning. Ovis2.5 integrates a native-resolution vision transformer that processes images at their native, variable resolutions, avoiding the degradation from fixed-resolution tiling and preserving both fine detail and global layout -- crucial for visually dense content like complex charts. To strengthen reasoning, we train the model to move beyond linear chain-of-thought and perform reflection -- including self-checking and revision. This advanced capability is exposed as an optional "thinking mode" at inference time, allowing users to trade latency for enhanced accuracy on difficult inputs. The model is trained via a comprehensive five-phase curriculum that progressively builds its skills. The process begins with foundational visual and multimodal pretraining, advances through large-scale instruction tuning, and culminates in alignment and reasoning enhancement using DPO and GRPO. To scale these upgrades efficiently, we employ multimodal data packing and hybrid parallelism, yielding a significant end-to-end speedup. We release two open-source models: Ovis2.5-9B and Ovis2.5-2B. The latter continues the "small model, big performance" philosophy of Ovis2, making it ideal for resource-constrained, on-device scenarios. On the OpenCompass multimodal leaderboard, Ovis2.5-9B averages 78.3, marking a substantial improvement over its predecessor, Ovis2-8B, and achieving state-of-the-art results among open-source MLLMs in the sub-40B parameter range; Ovis2.5-2B scores 73.9, establishing SOTA for its size. Beyond aggregate scores, Ovis2.5 achieves leading results on STEM benchmarks, exhibits strong capabilities on grounding and video tasks, and achieves open-source SOTA at its scale for complex chart analysis.
1 Introduction
Ovis2.5 addresses Ovis2’s difficulties with visually dense content and deep reasoning by combining native-resolution perception with reflection-based reasoning. It also introduces efficient training infrastructure and achieves leading multimodal benchmark performance.
- Core improvements: Ovis2.5 replaces fixed-resolution tiling with a native-resolution ViT that preserves fine details and global layout in dense visuals.The design is intended for charts, diagrams, and other visually dense content.
- Core improvements: Reflective training teaches self-checking and revision beyond linear chain-of-thought, with an optional thinking mode for difficult inputs.Users can leave thinking mode off for efficiency or enable it to trade latency for higher accuracy.
- Training and efficiency: A five-phase curriculum progresses from visual and multimodal pretraining through instruction tuning, DPO, and GRPO.The curriculum is designed to build skills from foundational perception to advanced reasoning.
- Evaluation: 78.3 is Ovis2.5-9B’s average OpenCompass score, establishing open-source SOTA in the sub-40B parameter range.Ovis2.5-2B scores 73.9, a state-of-the-art result among open-source models of comparable size.
- Evaluation: Ovis2.5 leads among similarly scaled open-source models across STEM, grounding, video, and complex chart-analysis benchmarks.The report also highlights strong performance beyond aggregate leaderboard scores.
- Training and efficiency: 3–4× end-to-end speedup comes from multimodal data packing and hybrid parallelism in the training infrastructure.These optimizations scale the training process more efficiently.
2 Architecture
Ovis2.5 retains the Ovis architecture’s visual tokenizer, visual embedding table, and language model while adding native-resolution processing and an upgraded language backbone. These changes target better spatial perception and multimodal reasoning.
- Base architecture: The architecture retains three modules: a visual tokenizer, visual embedding table, and pretrained open-source LLM.The tokenizer extracts patch features, the table represents visual words, and the LLM generates textual output from visual and textual embeddings.
- Base architecture: The visual tokenizer projects image-patch features into a probability distribution over a discrete vocabulary of visual words.The visual embedding table uses those probabilities to form the final visual embedding.
- Base architecture: The visual embedding table alleviates structural mismatches between visual and textual modalities by assigning embeddings to visual words.The final embedding is the probability-weighted sum of the table’s visual-word embeddings.
- Architectural enhancements: Native-resolution processing replaces fixed-size image splitting, preserving global structure and fine-grained details.NaViT processes images at native resolutions and incorporates RoPE into every ViT block to reinforce spatial awareness.
- Architectural enhancements: The language backbone changes from Qwen2.5 to Qwen3 to improve deep reasoning on complex tasks and overall multimodal proficiency.The report attributes the upgrade’s motivation to Qwen3’s superior deep-reasoning capabilities.
3 Model Training
Ovis2.5 is trained through a three-phase pretraining process followed by DPO and GRPO post-training, using curated multimodal data for perception, instruction following, grounding, OCR, and reasoning. Data packing and parallelism address multimodal training inefficiencies and deliver faster end-to-end training.
- Training pipeline: The five-phase pipeline comprises visual pretraining, multimodal pretraining, instruction tuning, multimodal DPO, and multimodal reinforcement learning.The first three phases are pretraining; the final two are post-training.
- Data composition: The training dataset combines public and in-house multimodal sources and is processed with data-type-specific quality pipelines.Its components support the distinct phases of the training pipeline.
- Data composition: OCR data covers documents, charts, posters, and screenshots, with MLLM-generated annotations filtered by resolution, language, and scene diversity.The curation process is designed to produce a high-quality and diverse OCR corpus.
- Data composition: Grounding data combines public bounding-box datasets with an automated detection-and-question-answer pipeline for scalable object localization supervision.Detection models identify entities, and MLLMs generate corresponding question-answer pairs.
- Data composition: Reasoning data combines vanilla CoT with thinking-style samples that explicitly teach reflection and self-correction.MLLMs synthesize reasoning paths and thinking-style data, with cross-verification used to improve synthesized-data quality.
- Post-training: GRPO updates only the LLM while freezing vision modules, optimizing verifiable reasoning tasks without degrading overall multimodal abilities.This phase builds on alignment established during DPO.
- Infrastructure: 3–4× faster end-to-end training results from optimizations targeting multimodal load imbalance and memory limitations.The infrastructure work addresses varying image, video, and text sizes alongside constraints in the previous training framework.
4 Experiments
Ovis2.5 is evaluated across overall multimodal performance, reasoning, OCR and chart analysis, visual grounding, and multi-image/video understanding. Results show state-of-the-art or leading performance across these capability areas among comparable open-source models.
- Overall Performance: 78.3 and 73.9 average OpenCompass scores establish state-of-the-art performance for Ovis2.5-9B and Ovis2.5-2B at their respective scales.OpenCompass averages results across eight multimodal benchmarks.
- Multimodal Reasoning: 71.2 on MMMU and 54.4 on MMMU-Pro demonstrate strong general academic reasoning, while Ovis2.5-9B leads MathVista and WeMath.The model also ranks among the top two on MathVerse, MathVision, LogicVista, and DynaMath.
- OCR & Chart: Ovis2.5 surpasses leading open-source competitors and GPT-4o on OCRBench v2, while Ovis2.5-9B achieves the top average across OCR and chart benchmarks.The evaluated benchmarks include ChartQA Pro, ChartQA, DocVQA, and TextVQA.
- Visual Grounding: 90.1 average score on RefCOCO, RefCOCO+, and RefCOCOg establishes state-of-the-art visual grounding among compared open-source models.Ovis2.5 performs especially strongly on RefCOCOg, outperforming all competitors on its validation and test sets.
- Multi-image & Video: 67.3 on BLINK and 69.3 on MMT-Bench show strong multi-image reasoning, complemented by consistent performance across video benchmarks.The results cover VideoMME, MVBench, MLVU, and TempCompass.
5 Conclusion
Ovis2.5 advances visual perception and multimodal reasoning through native-resolution processing and reflection-oriented thinking mode. Future work targets higher-resolution perception, longer-video reasoning, and tighter tool integration.
- Ovis2.5 processes images at native, variable resolutions and uses reflection-oriented thinking mode for self-correction and revision.Thinking mode offers a controllable latency–accuracy trade-off at inference.
- Ovis2.5 achieves leading results across STEM, OCR and chart, visual grounding, and multi-image/video comprehension benchmarks.
- Future work includes 4K-level perception, richer temporal reasoning for long-input video, and tighter tool-use integration.
- The authors identify these directions as areas for future work.
6 Core Contributors
This section lists Shiyin Lu, Yang Li, Yu Xia, Yuwei Hu, Shanshan Zhao, Yanqing Ma, Zhichao Wei, Yinglun Li, and Lunhao Duan.
- Contributors listed are Shiyin Lu, Yang Li, Yu Xia, Yuwei Hu, and Shanshan Zhao.
- The list continues with Yanqing Ma, Zhichao Wei, Yinglun Li, and Lunhao Duan.
7 Contributors
This section lists the report’s additional contributors, including Jianshan Zhao, Yuxuan Han, Haijun Li, Wanying Chen, Junke Tang, and others.
- Contributors listed include Jianshan Zhao, Yuxuan Han, Haijun Li, Wanying Chen, Junke Tang, Chengkun Hou, Zhixing Du, and Tianli Zhou.
- The list also includes Wenjie Zhang, Huping Ding, Jiahe Li, Wen Li, Gui Hu, Yiliang Gu, Siran Yang, and Jiamang Wang.
- Additional listed contributors are Hailong Sun, Yibo Wang, Hui Sun, Jinlong Huang, Yuping He, Shengze Shi, Weihong Zhang, Guodong Zheng, Junpeng Jiang, Sensen Gao, Yi-Feng Wu, Sijia Chen, and Yuhui Chen.
8 Project Leaders
This section lists Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang.
- The listed project contributors are Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang.
A Qualitative Examples
This section presents qualitative examples intended to showcase Ovis2.5’s capabilities and strengths across diverse prompts.
- Qualitative examples demonstrate Ovis2.5’s capabilities and strengths through responses to diverse prompts.
A.1 World Knowledge
The examples show Ovis2.5 identifying a plant species and inferring a geographic location from visual and contextual clues. Its Istanbul answer combines waterways, terrain, architecture, and traffic, while acknowledging that other coastal regions could share some traits.
- Plant species recognition: Ovis2.5 identifies the pictured plant as a baobab tree, specifically Adansonia digitata.
- Plant species recognition: The model explains baobabs through their water-storing trunks, rounded canopies, ecological roles, and cultural significance.
- Geographic recognition: Ovis2.5 infers that the pictured coastal area is likely Istanbul, Turkey, near the Bosphorus Strait.
- Geographic recognition: Its location reasoning uses water traffic, hilly shoreline development, architecture, and Istanbul’s waterways connecting the Black Sea and Sea of Marmara.
- Geographic recognition: Other coastal regions could share some traits, but the model considers Istanbul the most plausible location because of the combined visual clues.
A.2 Math Problem Solving
The math example addresses toothpick growth in square-grid patterns by analyzing counts across successive arrangements. The response derives a general counting rule for an n × n grid and checks it against the first two patterns.
- The task asks how many toothpicks Belinda adds when moving from the 30th pattern to the 31st.
- The response models each pattern as an n × n square grid and examines how toothpick counts change between patterns.
- For an n × n grid, it counts horizontal and vertical segments separately, each as (n+1)*n, giving 2*(n+1)*n total toothpicks.
Step 1: Identify the formula for the n-th pattern
The supplied material combines worked visual-reasoning examples with OCR, chart, and multimodal benchmark demonstrations. It shows formula-based pattern solving, geometric and scientific reasoning, biology inference, structured extraction, localization, and chart search.
- Step 1: Identify the formula for the n-th pattern: For square-grid patterns, horizontal and vertical toothpicks each total n(n + 1), giving the basis for the total-count formula.The grid has n + 1 lines in each orientation, with n segments per line.
- Step 2: Find the difference between consecutive patterns: The increase from pattern n to pattern n + 1 simplifies to 4(n + 1).The difference is obtained by subtracting consecutive total-count expressions.
- Step 3: Apply the formula to the 30th to 31st pattern: 124 toothpicks are added when transitioning from the 30th pattern to the 31st.The worked example substitutes n = 30 into the consecutive-pattern difference.
- Step 1: Define Variables and Key Properties: In the concentric-circle problem, the small radius is r, the large radius is 3r, and tangency makes OT perpendicular to chord BC.These relationships define the right-triangle construction used in the solution.
- Step 2: Use the Right Angle in Semicircle; Step 5: Find the Radius of the Big Circle: The circle solution uses right-triangle geometry and obtains r = 6, so the big circle’s radius is 18.The right angle follows from the diameter, while tangency supplies the perpendicular radius.
- A.3 Scientific Question Answering: Physics; Final Directions: For water-to-glass refraction, the reflected ray is 60° from the normal and the refracted ray is approximately 49.3° from the normal, bending toward the normal.The directions are determined using the law of reflection and Snell’s Law.
- A.4 Scientific Question Answering: Biology: The pedigree example identifies X-linked recessive inheritance and assigns Individual 9 the carrier genotype XAXa.Individual 9 inherits Xa from the affected father and must inherit XA from the carrier mother because she is unaffected.
- A.5 OCR; A.6 Chart: The demonstrations include JSON extraction, word-level text boxes, conditional chart search, and visual examples of OCR, localization, and complex reasoning.The OCR extraction pairs requested keys with values, while localization returns normalized bounding-box coordinates.