Source-linked AI summary
Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, Ji-Rong Wen
TL;DR
Multimodal slow-thinking systems remain difficult to develop because MLLMs must integrate perception and reasoning. The paper fine-tunes a capable MLLM with textual and multimodal long-thought data, finding that text-only reasoning transfers effectively and is generally more effective than visual reasoning data, while performance remains limited in some settings.
Problem
Existing slow-thinking research focuses mainly on textual tasks, leaving multimodal reasoning comparatively underexplored despite the added need to coordinate perception and reasoning.
Method
Virgo fine-tunes Qwen2-VL-72B-Instruct with textual long-thought data and compares it with visual long-thought data distilled from slow-thinking MLLMs.
Results
Text-only long-thought fine-tuning transfers slow-thinking ability to MLLMs, producing competitive results across four benchmarks and generally outperforming multimodal reasoning data for elicitation.
Takeaways & Limitations
Slow-thinking capacity can be transferred across modalities through the language-model component using natural-language reasoning data.
Takeaways & Limitations
Virgo can fail when it does not reflect on erroneous visual perception, and longer thought processes can degrade performance on problems that do not require complex reasoning.
Abstract
from arXiv · showhide
Recently, slow-thinking reasoning systems, built upon large language models (LLMs), have garnered widespread attention by scaling the thinking time during inference. There is also growing interest in adapting this capability to multimodal large language models (MLLMs). Given that MLLMs handle more complex data semantics across different modalities, it is intuitively more challenging to implement multimodal slow-thinking systems. To address this issue, in this paper, we explore a straightforward approach by fine-tuning a capable MLLM with a small amount of textual long-form thought data, resulting in a multimodal slow-thinking system, Virgo (Visual reasoning with long thought). We find that these long-form reasoning processes, expressed in natural language, can be effectively transferred to MLLMs. Moreover, it seems that such textual reasoning data can be even more effective than visual reasoning data in eliciting the slow-thinking capacities of MLLMs. While this work is preliminary, it demonstrates that slow-thinking capacities are fundamentally associated with the language model component, which can be transferred across modalities or domains. This finding can be leveraged to guide the development of more powerful slow-thinking reasoning systems. We release our resources at https://github.com/RUCAIBox/Virgo.
1 Introduction
Multimodal slow-thinking remains less developed than text-based reasoning because MLLMs must coordinate perception and reasoning. Virgo investigates whether text-based long-thought fine-tuning can transfer slow-thinking ability across modalities.
- Research gap: Multimodal reasoning systems lag behind text-based slow-thinking systems, partly because MLLMs must combine visual perception with problem-solving.The paper identifies this multimodal setting as an underexplored challenge relative to commercial systems such as o1 and QVQ.
- Research questions: The paper asks whether text-based long-thought fine-tuning can elicit slow-thinking ability in MLLMs.The motivation rests on the view that reasoning is primarily handled by the language-model component and that text-only instructions can enhance MLLMs.
- Approach: Virgo adapts a capable MLLM by fine-tuning it with textual and visual reasoning instructions to study cross-modal transfer.The backbone is Qwen2-VL-72B-Instruct, and the study compares textual instructions with visual instructions distilled from multimodal slow-thinking systems.
- Findings: Experiments on MathVerse, MathVision, OlympiadBench, and MMMU find that text-only long-thought data can produce competitive multimodal reasoning performance.The reported results are comparable to or sometimes better than commercial reasoning systems.
- Findings: Text-based reasoning instructions are generally more effective than multimodal reasoning data for eliciting MLLM slow-thinking capacities.This is presented as a major empirical finding of the study rather than as a universal claim about every benchmark or model.
2 Method
The method compares textual long-thought fine-tuning with multimodal instruction distillation for adapting MLLMs. It uses curated textual and visual reasoning data, fine-tunes a strong MLLM, and evaluates individual datasets and their combinations.
- Method overview: The study compares text-based long-thought transfer with multimodal long-thought distillation as two MLLM adaptation strategies.The goal is to determine how slow-thinking capacities are elicited and which instruction type is more effective.
- Textual data: Approximately 5K textual long-thought instructions are collected from DeepSeek-R1-Lite-Preview and QwQ across math, science, code, and puzzle domains.The data contain separate thought-process and final-solution segments marked by special symbols.
- Model training: Qwen2-VL-72B-Instruct is fine-tuned to emulate slow-thinking behavior while updating the language-model and cross-modal connector parameters.The visual encoder remains frozen, and the study intentionally uses straightforward imitation fine-tuning rather than DPO or RLHF.
- Visual data: Visual long-thought data are built from geometry, table, figure, and object VQA sources containing questions, images, and answers.The selected sources include Geos, GeoQA+, Geometry3K, UniGeo, TabMWP, FigureQA, ChartQA, and CLEVR.
- Visual distillation: Visual reasoning trajectories are generated either by distilling QVQ or through self-distillation from a text-fine-tuned model.The self-distillation pipeline first produces a text-fine-tuned model, then uses it to generate visual long-thought instructions for further fine-tuning.
- Evaluation: The experiments test DT, DSD, and DQVQ individually and in combination to measure their effects on slow-thinking performance.Table 2 compares these variants with general-purpose and o1-like MLLMs across four representative benchmarks.
3 Experiments
Experiments evaluate Virgo across four multimodal benchmarks and compare textual and visual long-thought instruction tuning, including smaller and larger MLLMs. Text-only tuning produces strong results on difficult benchmarks, while performance depends on benchmark difficulty, instruction amount, thought length, model size, and visual-perception reliability.
- Experimental setup: Virgo is evaluated on MathVerse, MathVision, OlympiadBench, and MMMU, using Qwen2-VL-72B-Instruct as its main backbone and comparisons with reasoning and general-purpose models.The study also trains Virgo-7B to examine model-size effects.
- Main results: 5K textual long-thought instructions transfer slow-thinking ability effectively, yielding 38.4% accuracy on MathVision and 29.3% on OlympiadBench.The model approaches or surpasses industry reasoning systems on these evaluations, but shows no significant overall improvement on MMMU.
- Main results: Textual reasoning instructions generally outperform synthesized visual instructions for eliciting slow-thinking capacity in the larger MLLM.Human review found many synthesized visual questions insufficiently complex and more dependent on perception than reasoning.
- Model-size effects: For the 7B model, visual long-thinking instructions outperform textual instructions particularly on MathVerse and MMMU, while long-thought tuning substantially decreases MMMU performance.The authors speculate that smaller models may struggle to manage complex long-thought processes on simpler tasks.
- Task difficulty: 18.1% and 12.4% absolute gains occur on OlympiadBench and MathVision, respectively, whereas the easier MMMU benchmark with shorter responses shows limited gains.The analysis associates longer responses with more challenging tasks and larger improvements from the approach.
- Instruction scaling: Increasing textual instructions from 1K to 5K produces a 7.7% average gain on MathVision for both model sizes and a 1.8% gain on OlympiadBench.Increasing reasoning length from 2K to 4K tokens improves all benchmarks, but extending it to 8K degrades most benchmarks; on MMMU, performance initially falls and later recovers as instruction count grows.
- Case studies: A visual case study shows textual long-thought tuning improving detailed image description, self-reflection, and solution verification, while failures occur when Virgo does not reflect on incorrect perception.The successful example concerns an integral involving an image of three semicircles; the failure case shows that a mistaken perception can collapse the subsequent reasoning process.
4 Conclusion
The paper presents textual long-form thought fine-tuning as a simple approach for giving MLLMs enhanced slow-thinking capacities, with competitive performance across challenging benchmarks. The authors characterize the current multimodal slow-thinking effort as preliminary and propose broader problems and more principled methods for future work.
- Conclusion: Fine-tuning MLLMs with textual long-form thought data substantially enhances their slow-thinking capacities.The approach is presented as simple yet effective.
- Conclusion: Experiments on four challenging benchmarks show highly competitive performance compared with industry reasoning systems.
- Future work: The current multimodal slow-thinking attempt is preliminary and requires broader challenging multimodal problems and more principled enhancement methods.