Source-linked AI summary
Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
Jingqi Tong, Yurong Mou, Hangcheng Li, Mingzhe Li, Yongzhuo Yang, Ming Zhang, Qiguang Chen, Tianyi Liang, Xiaomeng Hu, Yining Zheng, Xinchi Chen, Jun Zhao, Xuanjing Huang, Xipeng Qiu
TL;DR
Existing text- and image-based reasoning paradigms are limited by static visual representations and separated modalities. The paper proposes Thinking with Video, evaluates Sora-2 with VideoThinkBench across vision- and text-centric tasks, and finds strong reasoning performance with potential for unified multimodal understanding and generation.
Problem
Text- and image-based reasoning are limited because images represent single moments and textual and visual reasoning remain separated.
Method
The paper proposes Thinking with Video and evaluates Sora-2 on VideoThinkBench, which combines vision-centric and text-centric reasoning tasks.
Results
Sora-2 is generally comparable to SOTA VLMs on vision-centric tasks, surpasses GPT-5 by 10% on eyeballing puzzles, and achieves 92.0% on MATH and 69.2% on MMMU.
Takeaways & Limitations
The findings position video generation models as potential unified multimodal understanding and generation models and Thinking with Video as a potential unified multimodal reasoning paradigm.
Takeaways & Limitations
The evaluation primarily covers Sora-2, whose closed-source status limits analysis of its internal mechanisms.
Abstract
from arXiv · showhide
The "Thinking with Text" and "Thinking with Images" paradigms significantly improve the reasoning abilities of large language models (LLMs) and Vision-Language Models (VLMs). However, these paradigms have inherent limitations. (1) Images capture only single moments and fail to represent dynamic processes or continuous changes, and (2) The separation of text and vision as distinct modalities, which hinders unified multimodal understanding and generation. Therefore, we propose "Thinking with Video", a new paradigm that leverages video generation models such as Sora-2 to use video frames as a unified medium for multimodal reasoning. To support this exploration, we developed the Video Thinking Benchmark (VideoThinkBench), which covers both vision-centric tasks (e.g., Eyeballing Puzzles) and text-centric tasks (e.g., GSM8K and MMMU). Our evaluation on VideoThinkBench establishes Sora-2 as a capable reasoner. On vision-centric tasks, Sora-2 is comparable to state-of-the-art (SOTA) VLMs, and even surpasses GPT-5 by 10% on eyeballing puzzles. On text-centric tasks, Sora-2 achieves 92% accuracy on MATH, and 69.2% accuracy on MMMU. Furthermore, we systematically analyze the source of these abilities. We also find that self-consistency and in-context learning can improve Sora-2's performance. In summary, our findings show that the video generation model is the potential unified multimodal understanding and generation model, positioning "Thinking with Video" as a potential unified multimodal reasoning paradigm.
1 Introduction
The paper proposes Thinking with Video to address the static and modality-separated limits of text- and image-based reasoning. Using VideoThinkBench, it evaluates Sora-2 as a potentially unified multimodal reasoner across visual and textual tasks.
- 1 Introduction: Video reasoning addresses images’ inability to represent dynamic processes and the separation of textual and visual reasoning modalities.The proposed paradigm uses a coherent temporal structure to integrate textual and visual reasoning.
- 1 Introduction: Thinking with Video uses generated video frames for dynamic reasoning, multimodal fusion, and human-like drawing and imagination.Video can visualize continuous transformations and embed text within frames.
- 1 Introduction: VideoThinkBench spans vision-centric tasks from geometric intuition through visual induction, abstract rule induction, and spatial planning, alongside text-centric benchmarks.The benchmark includes eyeballing puzzles, visual puzzles, ARC-AGI-2, mazes, and adapted language and multimodal reasoning tasks.
- 1 Introduction: Sora-2 is generally comparable to state-of-the-art VLMs on vision-centric tasks and surpasses GPT-5 by 10% on eyeballing puzzles.Its visual reasoning uses drawing and imagination, including painting lines for spatial problems.
- 1 Introduction: On text-centric tasks, Sora-2 achieves 98.9% accuracy on GSM8K, 92.0% on MATH, 67.3% on MMLU, 75.7% on MathVista, and 69.2% on MMMU.These results cover both text reasoning and multimodal reasoning benchmarks.
- 1 Introduction: Sora-2 shows few-shot learning and benefits from self-consistency, while analysis suggests its text-centric abilities may originate from the prompt rewriter model.The model often makes reasonable ARC-AGI-2 predictions but cannot strictly match dataset annotations.
2 VideoThinkBench and Evaluation
VideoThinkBench evaluates video-generation reasoning across vision-centric and text-centric tasks, spanning geometric intuition, pattern induction, abstract rules, spatial planning, and language reasoning. Sora-2 performs competitively with leading VLMs, with especially strong results on eyeballing puzzles and text-centric benchmarks.
- Benchmark Design: VideoThinkBench covers vision-centric and text-centric tasks organized around five reasoning abilities, from geometric intuition to language conceptual understanding.Its vision-centric tasks include eyeballing puzzles, visual puzzles, ARC-AGI-2, and mazes; text-centric tasks include mathematical, logical, and language reasoning.
- 2.1.1 Geometric Reasoning: Eyeballing Puzzles: Sora-2’s Major Frame evaluation reaches 40.2% on eyeballing puzzles, exceeding Last Frame at 33.4%, Audio at 28.0%, and all listed VLM competitors.The listed VLM accuracies are Claude 4.5 Sonnet 35.1%, GPT-5 high 29.7%, and Gemini 2.5 Pro 26.5%.
- 2.1.2 Inductive Reasoning: Visual Puzzles: Sora-2 demonstrates pattern induction across symmetry, gradient, and compositionality visual puzzles, with symmetry performance comparable to Claude Sonnet 4.5.The benchmark evaluates the model on adapted PuzzleVQA tasks using manually selected best frames.
- 2.1.3 Few-Shot Learning: ARC-AGI-2: On ARC-AGI-2, Sora-2 achieves 1.3% accuracy, while manual analysis finds 3% fully correct and 14% mostly correct cases.These results indicate reasonable predictions that often fail to strictly match the ground-truth grids.
3 Analysis Experiment
The analysis experiments examine how examples, self-consistency, data leakage, written reasoning quality, and prompt rewriting relate to Sora-2’s reasoning performance. More examples and aggregation improve performance, while generated written processes are often incoherent and prompt rewriting may contribute to text reasoning.
- Few-Shot Learning: Few-shot inputs produce fewer low-accuracy samples and more high-accuracy samples than 1-shot inputs on ARC-AGI-2.The result supports Sora-2’s in-context learning ability and its improvement when more examples are provided.
- Self-Consistency: Major Frame accuracy rises from 56% with one Last Frame analysis to 68%, then to 90% after majority voting over five retries.Sampling across the video reduces failures from corrupted endings and aggregates consistent results across attempts.
- Data Leakage Analysis: Sora-2 maintains comparable performance on adapted math problems with different numerical values, reducing evidence for test-set leakage.The adapted-problem experiment preserves solution structure while changing numerical values.
- Text Generation Process: Only 13.91% of sampled correct text-centric solutions contain fully correct written processes, while 43.48% are unreadable or logically incorrect.The analysis categorized 115 cases answered correctly in both video and audio.
- Prompt Rewriting: Wan 2.5 has nearly zero accuracy without prompt rewriting and improves dramatically with it, suggesting Sora-2’s text reasoning may also originate from an internal rewriter.The comparison is indirect because Sora-2’s internal rewriter cannot be controlled.
4 Related Work
Prior work established text- and image-based reasoning paradigms and began exploring video-generation reasoning. This work extends those directions with broader multimodal evaluation, systematic benchmarking, and comparison against VLMs.
- Earlier paradigms use Chain-of-Thought for text reasoning and interleaved text-image reasoning for visual reasoning, motivating broader multimodal approaches.
- Video-generation reasoning has recently been explored through tasks such as maze solving and visual symmetry, demonstrating perception, modeling, and manipulation of the visual world.
- Existing video-reasoning studies mainly target vision-centric tasks, use case-based evaluations, and lack systematic comparison with state-of-the-art VLMs across diverse categories.
- This work evaluates video models on vision-centric, text-centric, and multimodal tasks, constructs a programmatically verifiable benchmark, and systematically compares them with VLMs.
- The study further reports that video-model reasoning can improve through few-shot learning and test-time self-consistency.
5 Conclusion
The paper introduces Thinking with Video and evaluates Sora-2 on VideoThinkBench. Its analysis suggests that video frames support drawing- and imagination-based reasoning, including textual reasoning toward unified multimodal understanding and generation.
- Thinking with Video evaluates Sora-2 on VideoThinkBench to study video generation as a multimodal reasoning medium.
- Sora-2 demonstrates potential for human-like reasoning through drawing and imagination, while video frames may support more unified multimodal understanding and generation.
6 Limitations and Future Work
The study primarily evaluates the closed-source Sora-2, limiting analysis of internal mechanisms. Future work targets broader model coverage, verifiable-reward training, and unified multimodal training for video models.
- The evaluation focuses primarily on Sora-2, whose closed-source status limits analysis of its internal mechanisms.
- Future evaluations plan to include more video-generation models, especially open-source models, and explore additional video-model capabilities.
- Scaling verifiable tasks in VideoThinkBench through Reinforcement Learning with Verifiable Rewards is proposed to enhance video-model reasoning.
- The authors propose converting textual corpora into video-form training data to investigate unified multimodal understanding and generation.
A.1 Detailed Sample Distribution
VideoThinkBench contains full and mini evaluation sets spanning vision-centric and text-centric reasoning. Its vision tasks range from eyeballing and visual puzzles to ARC-AGI-2 and mazes, while text tasks cover mathematical, general-knowledge, and multimodal benchmarks.
- VideoThinkBench contains 4,149 samples: 2,696 vision-centric and 1,453 text-centric examples, with task distributions summarized in Figures 8a and 8b.
- Vision-Centric Tasks: The vision-centric set includes Eyeballing Puzzles, Visual Puzzles, ARC-AGI-2, and Mazes, with 1,050, 496, 1,000, and 150 samples respectively.
- Text-Centric Tasks: Text-centric evaluation covers text-only math reasoning, text-only general knowledge reasoning, and multimodal reasoning across GSM8K, MATH-500, MMLU, GPQA-diamond, MathVista, and MMMU.
- Mini Test Set: A 750-sample mini test set reduces evaluation cost while covering all benchmark task categories, including 500 vision-centric and 250 text-centric samples.
- ARC-AGI-2: ARC-AGI-2 evaluates few-shot inductive reasoning by requiring Sora-2 to infer an abstract transformation rule and generate the corresponding test-case grid.
B More Evaluation Results
The mini-test evaluation broadens comparison across video and image generation models and Qwen3-VL models, with results reported in Tables 7 and 8.
- Tables 7 and 8 report mini-test results for additional video generation, image generation, and Qwen3-VL models.The evaluated video models include Seedance 1.0 Pro, MiniMax Hailuo 2.3, Wan2.2-TI2V-5B, and MOVA; image models include Nano Banana Pro, Seedream 4.5, GPT Image 1.5, BAGEL, and Qwen-Image-Edit-2511.
- The comparison covers both vision-centric and text-centric tasks in VideoThinkBench’s mini test set.
- The passage identifies the expanded evaluation set but does not report individual model accuracies.
C Detailed Evaluation Protocols
The evaluation protocols define datasets, model inputs, output formats, and task-specific scoring procedures across maze, eyeballing, and visual-puzzle tasks.
- Maze Tasks: 150 mazes are split evenly among square, hexagon, and circle geometries, requiring paths between red endpoints without crossing black walls.
- Maze Tasks: Maze success requires a continuous red path connecting both endpoints and zero overlap with black wall pixels in the final frame.
- Maze Tasks: Sora-2 solves 20 of 50 square mazes for 40% accuracy but solves none of the hexagon or circle mazes.
- Eyeballing Puzzles: Eyeballing puzzles use audio, last-frame, or major-frame evaluation to map generated outputs to answer options.
- Eyeballing Puzzles: Major-frame evaluation samples one frame every five frames and selects the majority option, excluding “None” from voting.
- Visual Puzzles: Visual-puzzle evaluation compares generated and ground-truth images using task-specific pixel differences, while VLMs are rule-evaluated against ground-truth answers.
C.6 Text-Centric Tasks
Text-centric evaluation includes multimodal reasoning questions and validates automated scoring against human judgments. The human alignment check finds 89.6% video and 97.7% audio agreement.
- 89.6% video and 97.7% audio evaluation accuracy were observed in a human alignment check of 173 text-centric responses.
- Sora-2 receives an input image containing the original multimodal problem and question, alongside a text prompt containing the question text.
- The supplementary analysis presents additional text-centric cases and experimental results, including output-modality and manual-evaluation analyses.
E.1 Data Leakage Analysis
The data-leakage analysis constructs new math problems from GSM8K and MATH-500 while preserving underlying structure and difficulty but changing numerical or contextual details.
- The analysis uses LLMs to derive analogous evaluation problems from sampled GSM8K and MATH-500 instances to investigate possible data leakage.
- New GSM8K and MATH-500 problems preserve the original problem-solving structure while changing numbers and, when possible, context or expression.
- The adapted problems are designed to maintain the original difficulty and appropriate answer complexity, including integer-answer constraints where applicable.
E.2 Output Modality Analysis
The section examines how output modality and prompt rewriting affect video-based reasoning evaluation. It defines multiple answer-extraction methods, analyzes reasoning quality, and shows that Wan 2.5’s apparent reasoning depends heavily on prompt rewriting.
- Arc Connect Evaluation: Arc Connect evaluation compares audio answers, the last frame, and majority votes across sampled video frames.The last-frame method checks the final connection, while the major-frame method aggregates evaluations from every fifth frame.
- Prompt Rewriting: Prompt rewriting converts reasoning tasks into explicit step-by-step visual instructions, effectively solving the problem before video generation.Disabling the prompt rewriter leads to nearly zero accuracy.
- Reasoning Process Analysis: 43.48% of correct-answer reasoning processes fall into the “Unreadable or Incorrect Logic” category, indicating difficulty generating coherent reasoning.
- Visual Puzzle Prompts: The visual-puzzle evaluation uses different prompts for Sora-2 and VLMs, with Sora-2 instructed to manipulate or mark visual answers and speak them phonetically.VLM prompts generally ask for a multiple-choice answer directly.