Source-linked AI summary
Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
Dongjie Cheng, Yongqi Li, Zhixin Ma, Hongru Cai, Yupeng Hu, Wenjie Wang, Liqiang Nie, Wenjie Li
TL;DR
Existing interleaved-modal reasoning methods often use task-specific patterns, while diverse multimodal tasks require a broader unified approach. The paper introduces Omni-R1 with perception-supervised SFT and RL, plus Omni-R1-Zero, which bootstraps visualizations without costly annotations; both support unified generative reasoning, with Omni-R1-Zero sometimes surpassing Omni-R1.
Problem
Existing interleaved-modal methods rely on task-specific reasoning patterns, while functional image generation is difficult and interleaved reasoning annotations are costly and scarce.
Method
Omni-R1 uses two-stage SFT and RL with perception alignment loss and perception-calibrated reward, while Omni-R1-Zero bootstraps step-wise visualizations from text-only reasoning data.
Results
Omni-R1 enables unified generative reasoning across multimodal tasks, while Omni-R1-Zero can match or surpass Omni-R1 on average.
Takeaways & Limitations
The results support unified generative multimodal reasoning and motivate scalable supervision signals that reduce reliance on multimodal annotations.
Takeaways & Limitations
Synthetic visualization traces are not intended as perfect supervision; they teach the interleaved format and expose intermediate multimodal states.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) are making significant progress in multimodal reasoning. Early approaches focus on pure text-based reasoning. More recent studies have incorporated multimodal information into the reasoning steps; however, they often follow a single task-specific reasoning pattern, which limits their generalizability across various multimodal tasks. In fact, there are numerous multimodal tasks requiring diverse reasoning skills, such as zooming in on a specific region or marking an object within an image. To address this, we propose unified generative multimodal reasoning, which unifies diverse multimodal reasoning skills by generating intermediate images during the reasoning process. We instantiate this paradigm with Omni-R1, a two-stage SFT+RL framework featuring perception alignment loss and perception reward, thereby enabling functional image generation. Additionally, we introduce Omni-R1-Zero, which eliminates the need for multimodal annotations by bootstrapping step-wise visualizations from text-only reasoning data. Empirical results show that Omni-R1 achieves unified generative reasoning across a wide range of multimodal tasks, and Omni-R1-Zero can match or even surpass Omni-R1 on average, suggesting a promising direction for generative multimodal reasoning.
1 Introduction
Existing multimodal reasoning methods increasingly incorporate visual information, but typically use task-specific patterns. Omni-R1 proposes a unified generative paradigm that produces intermediate images to support diverse reasoning skills while addressing functional image generation and annotation costs.
- Intermediate visual information can simplify multimodal reasoning, such as zooming into a critical region for spatial-relation questions.
- Current interleaved-modal methods generally rely on one reasoning pattern tailored to a particular multimodal task.
- Diverse multimodal tasks require skills including zooming, grounding, and visual prediction, motivating a unified paradigm within one MLLM.
- Functional image generation is difficult because reasoning images are often unnatural, while interleaved-modal annotations are costly and scarce at scale.
- Omni-R1 combines SFT and RL with perception alignment loss and perception-calibrated reward to supervise visual perception during optimization.
- Omni-R1-Zero bootstraps step-wise visualizations to remove the need for costly interleaved-modal reasoning annotations.
- Omni-R1 supports unified generative reasoning across multimodal tasks, while Omni-R1-Zero can surpass Omni-R1 to some extent.
2 Related Work
Related work progresses from text-centered multimodal reasoning toward interleaved multimodal trajectories, tool use, visualized intermediate states, and supervised image-text reasoning data.
- Earlier methods generate textual rationales and final answers while using images primarily as context, with some approaches adding structured external knowledge.
- Interleaved-modal reasoning augments reasoning trajectories with additional multimodal evidence, including tool-acquired observations.
- MVoT generates image-text interleaved rationales for spatial reasoning, while Zebra-CoT provides 182K interleaved text-image reasoning examples.
3 Omni-Bench
Omni-Bench evaluates multimodal reasoning across four task families designed to require varied visual skills, using a fixed answer-matching evaluation procedure.
- Omni-Bench contains diverse multimodal tasks that require different multimodal reasoning skills.
- Its four Uni-Tasks are Natural-Scene Perception, Diagrammatic Math, Structured-Image, and Vision-Operational Scenes.
- Uni-Tasks: Natural-Scene Perception requires evidence localization, while Diagrammatic Math combines visual grounding with arithmetic and geometric reasoning.
- Uni-Tasks: Structured-Image covers figures and charts combining text and graphics, whereas Vision-Operational Scenes involve visual operations or scene-state prediction.
- Evaluation: Evaluation extracts the final answer span and uses an LLM judge as a deterministic binary correctness checker against the gold reference.
- The benchmark comprises 800 samples spanning four Uni-Tasks.
4 Method
Omni-R1 and Omni-R1-Zero use a two-stage pipeline to learn unified generative multimodal reasoning, combining supervised format learning with perception-aware optimization. Omni-R1-Zero bootstraps interleaved visual trajectories from text-only reasoning, while PeRPO refines trajectories using composite rewards for accuracy, format, and perception.
- Training Framework: Both frameworks use two stages: PeSFT learns interleaved reasoning and Uni-Skills prediction, while PeRPO refines the policy on unified tasks.PeSFT uses cross-entropy and perception losses; PeRPO extends training beyond multimodal annotations with a perception-calibrated reward.
- PeSFT: PeSFT applies cross-entropy across the generative trajectory and uses image-token supervision to reproduce the interleaved reasoning format and functional images.The perception loss additionally aligns hidden states with a frozen visual codebook to stabilize autoregressive image-token generation.
- PeRPO: PeRPO integrates a perception-calibrated reward with group-relative PPO optimization for long, interleaved multimodal sequences.The reward combines answer accuracy, format compliance, and perceptual consistency, with group filtering retaining only non-degenerate mixed-outcome samples.
- PeRPO: The perception reward verifies answer correctness and format, then measures coherence of intermediate image generations using 2D Total Variation on codebook embeddings.Image-token segments are reshaped from a 1-D sequence into the quantized image grid before spatial variation is computed.
- Omni-R1-Zero: Omni-R1-Zero constructs synthetic interleaved trajectories by generating one image for each reasoning step from text-only Chain-of-Thought seeds.It reuses PeSFT and PeRPO; the synthetic traces teach the interleaved format and expose intermediate multimodal states without human-annotated rationales.
5 Experiment
Omni-R1 and Omni-R1-Zero are evaluated across Omni-Bench, general multimodal benchmarks, ablations, and qualitative analyses. Results indicate competitive cross-task performance, improved general reasoning and visual perception, and distinct generative reasoning behaviors.
- Experimental Setup: Omni-Bench covers four Uni-Tasks, while evaluation also includes standard multimodal benchmarks using accuracy or POPE F1-score.The Uni-Tasks are Natural-Scene Perception, Diagrammatic Math, Structured-Image, and Vision-Operational.
- Main Results: Omni-R1 and Omni-R1-Zero perform competitively across all four Omni-Bench slices and consistently improve over their corresponding base models.Omni-R1-M (Bagel) achieves the strongest average performance, while Omni-R1-Zero variants outperform the base model across different RL budgets.
- Main Results: On standard benchmarks, both methods outperform baselines on the Anole backbone, with Omni-R1 generally stronger on perception-oriented evaluations and Omni-R1-Zero competitive on reasoning-oriented metrics.The reported benchmarks include MME, MM-Vet, V∗-Bench, POPE, MMVP, and BLINK.
- In-depth Analysis: Removing PeRPO causes the largest ablation drop, especially on Vision-Operational and Diagrammatic tasks, while removing the perception reward causes a smaller but consistent degradation.The authors associate PeRPO with multi-step and complex multimodal reasoning and the perception-calibrated reward with visual evidence utilization and policy optimization stability.
- In-depth Analysis: t-SNE projections show compact generated-image modes for Omni-R1 and more dispersed multimodal structure for Omni-R1-Zero, with correct instances clustering more tightly than incorrect ones.The analysis links trace supervision with canonical, stable generations and reward guidance with broader exploration that can still support correct decisions.
- In-depth Analysis: Qualitative cases demonstrate task-conditioned intermediate visual evidence for grounding, marking, visual prediction, and commonsense multimodal reasoning.Omni-R1-Zero generates intermediate visual evidence despite having no supervised multimodal traces.
6 Conclusion and Future Work
The paper formulates generative multimodal reasoning as interleaved textual rationales and executable visual actions that update intermediate images before final answering. Its atomic action protocol defines how visual states are transformed and represented during reasoning.
- Conclusion and Future Work: Generative multimodal reasoning interleaves textual rationales with executable visual actions and post-action images before producing the final answer.The trajectory includes textual rationales, atomic visual actions, post-action visual rationales, and a final textual answer.
- Trajectory and Policy: At each reasoning step, the policy selects an action, an executor deterministically updates the visual state, and the answer uses the full interaction history.The renderer produces exactly one post-action image for each selected action.
- Atomic Action Space: The action vocabulary includes operations for highlighting regions, drawing lines, applying state deltas, zooming, boxing, and marking image content.Actions are applied to the current visual state through the executor, with task-dependent transitions supported by serialized deltas.
- Coordinate Convention: Spatial arguments use normalized image coordinates, with (0, 0) at the top-left and (1, 1) at the bottom-right.Boxes use top-left coordinates plus width and height in [0, 1], while out-of-range or degenerate arguments are format errors.
B Rule-based verifier for RAcc
The RAcc verifier deterministically extracts and normalizes a final answer, then applies reliability-first matching checks. Accuracy and formatting are scored independently.
- Answer Extraction and Normalization: The verifier extracts a dedicated final-answer segment and lightly normalizes whitespace, punctuation, and common answer-introduction phrases.This normalization improves evaluation stability without relaxing the core matching criteria.
- Prioritized Matching: Matching prioritizes numeric equivalence, symbolic or mathematical equivalence, conservative textual matching, and limited domain-specific handling.The ordered checks are designed to emphasize reliability and avoid false positives.
- Scoring: RAcc ranges from 0 to 1, while RFmt independently indicates whether the generated trajectory is well-formed and parsable.Numeric and symbolic checks are typically binary, whereas conservative textual matching may provide partial credit.
C Dataset Details
The dataset setup uses different supervised-data sources for Omni-R1 and Omni-R1-Zero, with additional training data for reinforcement learning. General benchmark descriptions cover task formulations, sizes, and metrics.
- Training Data: Omni-R1 uses the 182K-sample Zebra-CoT image-text interleaved multimodal reasoning dataset for PeSFT supervision.The dataset is sampled as supervised data for the PeSFT stage.
- Training Data: Omni-R1-Zero bootstraps PeSFT data from a small text-only M3CoT chain-of-thought subset, yielding 791 interleaved samples.Its PeRPO data additionally includes approximately 10% of Zebra-CoT, 1K ArxivQA samples, and 250 Geometry3K training samples.
- General Benchmarks: The general benchmarks document task formulations, dataset sizes, and evaluation metrics in Table 5.These benchmark details are presented as part of the general benchmark specification.
D Training Details
The implementation relies on established open-source libraries, while the experimental configuration is organized into four hyperparameter groups. These groups cover objectives, batching, optimization, and rollout.
- Implementation Details: The implementation uses verl, Transformers, Flow-GRPO, and trl under their respective open-source licenses.These libraries provide the implementation foundation for the experiments.
- Hyperparameter Settings: Hyperparameters are organized into Objective, Batching, Optimization, and Rollout groups.The settings are summarized in Table 6.
E Additional Results
Figures 7–9 provide additional full interleaved trajectories illustrating Omni-R1’s generative multimodal reasoning across diverse tasks and capability requirements.
- Figures 7–9 present additional full interleaved trajectories for Omni-R1.
- The examples span diverse tasks and capability requirements.
- The trajectories show Omni-R1 generating intermediate visual evidence and composing Uni-Skills across scenarios.
F Prompt Templates
This section provides evaluation and training references alongside multimodal question examples and reasoning traces, including visual evidence used to produce answers.
- Omni-Bench evaluation uses a fixed LLM-judge prompt template with binary decisions based on ground-truth answers and model outputs.The template standardizes numeric formatting, unit conversion, and multiple-choice edge cases.
- Tables 5–8 document benchmark details, training hyperparameters, and the Omni-Bench GPT-judge template.The supplied table captions identify Tables 5, 6, and 7; Table 8 is not represented in the provided passages.
- Another trace zooms toward the object beneath a coffee cup before answering “plate.”
- One reasoning trace applies 4-connectivity, uses an intermediate image, and identifies 4 connected components marked with different colors.
- The examples include questions about puzzle completion, graph connectivity, connected hole components, and objects shown in images.