Source-linked AI summary
OneThinker: All-in-one Reasoning Model for Image and Video
Kaituo Feng, Manyuan Zhang, Hongyu Li, Kaixuan Fan, Shuang Chen, Yilei Jiang, Dian Zheng, Peiwen Sun, Yiyuan Zhang, Haoze Sun, Yan Feng, Peng Pei, Xunliang Cai, Xiangyu Yue
TL;DR
Existing multimodal reasoning systems commonly separate tasks and image from video, limiting the development of a unified reasoning generalist. OneThinker addresses this gap with joint multimodal training, CoT-supervised data, and EMA-GRPO for heterogeneous-task RL. It reports strong performance across diverse benchmarks, with knowledge transfer and preliminary zero-shot generalization.
Problem
Existing approaches typically train separate models for different tasks and treat image and video reasoning as disjoint domains, limiting scalability toward a multimodal reasoning generalist.
Method
OneThinker jointly trains a unified image-and-video model using OneThinker-600k, its CoT-annotated OneThinker-SFT-340k subset, and EMA-GRPO with task-wise adaptive reward normalization.
Results
OneThinker achieves strong performance across 31 benchmarks and 10 visual understanding tasks, including 70.6% on MMMU and 64.3% on MathVerse for image QA.
Takeaways & Limitations
Unified training supports knowledge sharing between related tasks and preliminary zero-shot generalization across unseen scenarios.
Abstract
from arXiv · showhide
Reinforcement learning (RL) has recently achieved remarkable success in eliciting visual reasoning within Multimodal Large Language Models (MLLMs). However, existing approaches typically train separate models for different tasks and treat image and video reasoning as disjoint domains. This results in limited scalability toward a multimodal reasoning generalist, which restricts practical versatility and hinders potential knowledge sharing across tasks and modalities. To this end, we propose OneThinker, an all-in-one reasoning model that unifies image and video understanding across diverse fundamental visual tasks, including question answering, captioning, spatial and temporal grounding, tracking, and segmentation. To achieve this, we construct the OneThinker-600k training corpus covering all these tasks and employ commercial models for CoT annotation, resulting in OneThinker-SFT-340k for SFT cold start. Furthermore, we propose EMA-GRPO to handle reward heterogeneity in multi-task RL by tracking task-wise moving averages of reward standard deviations for balanced optimization. Extensive experiments on diverse visual benchmarks show that OneThinker delivers strong performance on 31 benchmarks, across 10 fundamental visual understanding tasks. Moreover, it exhibits effective knowledge transfer between certain tasks and preliminary zero-shot generalization ability, marking a step toward a unified multimodal reasoning generalist. All code, model, and data are released.
1 Introduction
OneThinker addresses the fragmentation of multimodal reasoning by unifying image and video understanding across diverse visual tasks. It combines large-scale CoT-supervised data with EMA-GRPO and achieves strong cross-benchmark performance with knowledge transfer and preliminary zero-shot generalization.
- Existing reasoning models typically isolate tasks and modalities, limiting practical versatility and potentially hindering cross-task and cross-modal knowledge transfer.
- OneThinker unifies image and video reasoning across question answering, captioning, grounding, tracking, and segmentation in a single multimodal generalist.
- OneThinker-600k provides approximately 600k multimodal samples, while its Seed1.5-VL-annotated OneThinker-SFT-340k subset supplies a supervised cold start.
- EMA-GRPO balances heterogeneous multi-task RL by maintaining task-wise exponential moving averages of reward standard deviations for adaptive normalization.
- OneThinker achieves strong results across 31 benchmarks and 10 visual understanding tasks, while enabling knowledge sharing and preliminary zero-shot generalization.
2 Related Works
Related work applies RL-based reasoning to multimodal models across image and video tasks. However, most existing systems remain limited in task coverage or modality, motivating unified approaches.
- DeepSeek-R1-inspired work uses GRPO-based rule-driven RL to optimize outcome-level rewards without explicit intermediate supervision.
- RL-based multimodal reasoning has been applied to image QA, video QA, object detection, and other visual tasks.
- Figure 3 depicts a curated dataset spanning image and video modalities for diverse visual understanding tasks.
- Most prior models remain restricted to limited task sets and support either image or video reasoning alone.
3 Method
OneThinker builds a unified image-and-video reasoning pipeline spanning diverse tasks, with structured outputs and task-specific rewards. EMA-GRPO addresses both within-task and across-task reward imbalance during multi-task reinforcement learning.
- Data Collection and Curation: OneThinker-600k covers image and video modalities across QA, captioning, grounding, tracking, and segmentation tasks.The corpus is curated across domains and difficulty levels to support logical, spatial, temporal, and causal reasoning.
- CoT Annotation: The model is initialized from OneThinker-SFT-340k, a filtered and validated subset with task-specific CoT annotations generated by Seed1.5-VL.The annotations are filtered by task-specific thresholds and checked with rules and quality validation.
- Task Types and Rewards: All tasks use a unified <think>/<answer> interface, with structured outputs receiving format checks alongside task-specific accuracy rewards.Rewards include deterministic rule-based feedback, external similarity scoring for open-ended QA and captioning, and overlap-based metrics for localization and segmentation.
- Task Types and Rewards: Spatial, temporal, spatio-temporal, tracking, and segmentation tasks are evaluated with overlap, distance, or temporal-kernel rewards tailored to their predicted structures.Examples include temporal IoU for event spans, spatial IoU for boxes, mean IoU over tracking trajectories, and point- or keyframe-aware segmentation rewards.
- EMA-GRPO: EMA-GRPO addresses intra-task bias from standard GRPO and inter-task imbalance caused by differing reward scales and densities.Standard GRPO can under-optimize medium-difficulty samples, while removing standard-deviation normalization can let sparse-reward tasks dominate dense-reward tasks.
- EMA-GRPO: EMA-GRPO uses task-wise exponential moving averages of reward moments to estimate adaptive standard deviations for advantage normalization.The moving statistic tracks each task’s intrinsic reward scale while adapting to changing reward distributions; the decay factor is set to 0.99.
4 Experiments
OneThinker performs strongly across image and video reasoning benchmarks spanning question answering, captioning, grounding, tracking, and segmentation. Ablations and transfer studies indicate that unified RL training and cross-task learning contribute to its broad capabilities.
- OneThinker achieves strong results across diverse image and video benchmarks, covering question answering, captioning, grounding, tracking, and segmentation.The evaluation spans multiple tasks and modalities, with results reported across tables 1–8.
- Question Answering: 70.6% on MMMU and 79.2% on LongVideo-Reason demonstrate strong image and video question-answering performance against open-source competitors.The model reaches 70.6% on MMMU and 79.2% on LongVideo-Reason, compared with 67.2% for Video-R1-7B and 71.5% for Qwen3-VL-Instruct-8B on LongVideo-Reason.
- Grounding: OneThinker leads spatial and temporal localization benchmarks, including 93.7 on RefCOCO testA and 65.0 R@0.3 on ActivityNet.It also reports strong results on RefCOCO+, RefCOCOg, ANet-RTL, and STVG.
- Tracking and Segmentation: 84.4 R@0.5 on GOT-10k and 54.9 J&F on ReasonVOS show strong tracking and video-segmentation performance.The tracking evaluation uses 32 frames, while the segmentation results are reported across both image and video benchmarks.
- Ablations: RL improves over the SFT baseline, while standard GRPO and DrGRPO underperform EMA-GRPO across tasks.The ablation attributes these degradations to unresolved intra-task and inter-task reward imbalances.
- Knowledge Transfer: Removing temporal grounding reduces video QA and tracking performance, removing spatial grounding harms image QA and segmentation, and excluding ImageQA severely hurts video QA.These ablations support transfer between related tasks and from static-image reasoning to video understanding.
- Zero-Shot Generalization: OneThinker-8B outperforms Qwen3-VL-Instruct-8B on multiple unseen MMT-Bench tasks, including point tracking, image quality assessment, GUI tasks, and rotated object detection.The evaluated unseen tasks come from MMT-Bench, which contains 162 diverse visual tasks.
5 Conclusion
The paper presents OneThinker as an all-in-one model for unified image and video reasoning across diverse visual foundation tasks. It combines broad training data with EMA-GRPO to support strong performance and scalable multimodal reasoning.
- OneThinker unifies diverse image and video foundation tasks in one multimodal reasoning model.The paper positions this unified model as a step toward a scalable multimodal reasoning generalist.
- OneThinker-600k supports RL training, OneThinker-SFT-340k provides a CoT-annotated SFT cold start, and EMA-GRPO balances optimization across heterogeneous tasks.EMA-GRPO uses task-wise adaptive reward normalization.
- Extensive experiments show strong performance across visual tasks, supporting the paper’s aim of scalable and unified multimodal reasoning.
A Reasoning Examples
The image question-answering example presents a geometry problem involving a circle, diameter, and angle choices.
- The example asks for ∠BCD given that AB is a diameter and ∠ABD equals 50.0°.
- The question uses a four-choice multiple-choice format, with answers ranging from 30° to 45°.
- The image QA task is illustrated as a visual reasoning example.
Video QA
The examples cover video question answering, captioning, temporal and spatial grounding, tracking, and image and video segmentation.
- Video QA examples include narrative interpretation and locating described events in time.
- Captioning is represented through separate image-caption and video-caption reasoning examples.
- Grounding examples request temporal intervals, spatial bounding boxes, or both for described video content.
- Tracking examples require outputting an object's bounding box across frames, once per second.
- Segmentation examples provide bounding boxes and positive or negative points as inputs for image or video mask generation.
B Prompt Template
The prompt template uses a shared system prompt alongside task-specific formats for QA, grounding, tracking, and segmentation.
- A common system prompt is provided for all tasks.
- QA tasks have a dedicated prompt template.
- Temporal grounding requires returning a time span in seconds as JSON within answer tags.
- Grounding and tracking share a task-specific prompt category.
- Segmentation prompts specify one bounding box together with positive and negative points for specialized mask-generation models.