PaperlayerView exact revision on arXiv

Source-linked AI summary

V-Thinker: Interactive Thinking with Images

Runqi Qiao, Qiuna Tan, Minghan Yang, Guanting Dong, Peiqing Yang, Shiqiang Lang, Enhui Wan, Xiaowan Wang, Yida Xu, Lan Yang, Chong Sun, Chen Li, Jing Lyu, Honggang Zhang

arXiv:2511.04460v2cs.CV

TL;DR

LMMs often reason coherently without remaining grounded in visual evidence, while existing interactive approaches have limited visual tools and task-specific workflows. V-Thinker combines evolving interactive-data synthesis, progressive perception-to-reasoning training, and expert-verified evaluation; it consistently outperforms LMM baselines across interactive and general reasoning scenarios.

  • Problem

    LMMs can produce lengthy reasoning while detaching from visual grounding, and existing interactive approaches remain limited in visual actions and workflow generality.

  • Method

    V-Thinker combines a Data Evolution Flywheel with a Visual Progressive Training Curriculum for end-to-end interactive, vision-centric reasoning.

  • Results

    V-Thinker achieves a 12.5% average accuracy improvement across three interactive reasoning domains and over 22% improvement in Instruction-Guided Interaction.

  • Takeaways & Limitations

    The framework supports interactive thinking with images across both interactive and general reasoning scenarios and provides VTBench for expert-verified evaluation.

  • Takeaways & Limitations

    VTBench judgments use fixed prompts because model perception limitations and prompt-sensitive evaluation criteria affect consistency and fairness.

Abstract

from arXiv · show

Empowering Large Multimodal Models (LMMs) to deeply integrate image interaction with long-horizon reasoning capabilities remains a long-standing challenge in this field. Recent advances in vision-centric reasoning explore a promising "Thinking with Images" paradigm for LMMs, marking a shift from image-assisted reasoning to image-interactive thinking. While this milestone enables models to focus on fine-grained image regions, progress remains constrained by limited visual tool spaces and task-specific workflow designs. To bridge this gap, we present V-Thinker, a general-purpose multimodal reasoning assistant that enables interactive, vision-centric thinking through end-to-end reinforcement learning. V-Thinker comprises two key components: (1) a Data Evolution Flywheel that automatically synthesizes, evolves, and verifies interactive reasoning datasets across three dimensions-diversity, quality, and difficulty; and (2) a Visual Progressive Training Curriculum that first aligns perception via point-level supervision, then integrates interactive reasoning through a two-stage reinforcement learning framework. Furthermore, we introduce VTBench, an expert-verified benchmark targeting vision-centric interactive reasoning tasks. Extensive experiments demonstrate that V-Thinker consistently outperforms strong LMM-based baselines in both general and interactive reasoning scenarios, providing valuable insights for advancing image-interactive reasoning applications.

1. Introduction

V-Thinker addresses the gap between coherent multimodal reasoning and grounded visual interaction by combining interactive image reasoning, evolving data synthesis, progressive training, and expert-verified evaluation.

  • 1. Introduction: Current LMMs can produce coherent chain-of-thought yet often detach from visual grounding, causing hallucinations and reliance on linguistic priors.Recent visual-tool approaches shift toward interactive image reasoning, but their available actions remain limited.
  • 1. Introduction: V-Thinker enables vision-centric thinking through autonomous image interaction and end-to-end reinforcement training.Its framework spans perception and reasoning through code-driven visual tools.
  • 1. Introduction: The Data Evolution Flywheel synthesizes, verifies, and progressively expands interactive reasoning data across diversity, quality, and difficulty.It produces the V-Interaction-400K visual interactive reasoning dataset.
  • 1. Introduction: The Visual Progressive Training Curriculum first aligns visual localization and referencing, then uses supervised fine-tuning and reinforcement learning for interactive reasoning.Perception alignment uses V-Perception-40K, followed by reasoning alignment in a sandboxed executor.
  • 1. Introduction: VTBench evaluates visual-interactive reasoning with expert-verified instances sourced from diverse public datasets.The benchmark targets tasks that inherently demand visual interaction.

2. Related Work

Related work has established interactive visual reasoning and reinforcement-learning-based visual tools, while V-Thinker extends the paradigm toward autonomous visual-code interaction.

  • 2. Related Work: Interactive visual reasoning evolved from early visual operations to reinforcement-learning-guided executable tools such as cropping.The related work frames thinking with images as a developing direction for multimodal reasoning and agentic RL.
  • 2. Related Work: DeepSketcher explores implicit visual reasoning through abstract visual cues rather than explicit pixel-level manipulation.This contrasts with approaches that directly operate on image pixels.
  • 2. Related Work: V-Thinker advances prior work by autonomously generating, executing, and iteratively refining visual code during reasoning.The approach connects visual interaction with multimodal reasoning through executable operations.

3. Preliminary

V-Thinker formulates reasoning as iterative code-driven interaction with an image and revisits data synthesis so models can create diverse multimodal problems rather than only solve predefined ones.

  • 3.1. Problem Formulation: At each reasoning step, V-Thinker generates a textual thought and optionally code that operates on the current image, whose updated state guides later reasoning.The trajectory maps a query and initial image to reasoning steps and a final answer.
  • 3.1. Problem Formulation: The formal process represents the task query, reasoning trajectory, final answer, textual thoughts, image-action code, and environment-produced image updates.The environment executes each code segment to produce the next visual state.
  • 3.2. Rethinking on Data Synthesis Paradigm: Traditional manually defined datasets limit diversity and scalability, while interactive reasoning requires precise spatial and logical alignment.This motivates reconsidering models as data creators rather than only solvers.
  • 3.2. Rethinking on Data Synthesis Paradigm: Strong LMMs can generate Python code, original images, auxiliary diagrams, and reasoning trajectories, enabling creation of complex multimodal problems.The paper reports that GPT-5 can directly render high-quality images and annotations.
  • 3.2. Rethinking on Data Synthesis Paradigm: Knowledge concepts act as condensed semantic anchors specifying what to reason about, while models determine how interaction unfolds.This knowledge-driven representation is intended to cover diverse real-world scenarios.

4. Methodology

V-Thinker combines an automated Data Evolution Flywheel with a visual progressive training curriculum to support vision-centric interactive reasoning. The method synthesizes and verifies increasingly difficult interactive data, then aligns perception before interactive reasoning.

  • Overview: V-Thinker uses a Data Evolution Flywheel and Visual Progressive Training Curriculum to move from perception understanding to vision-centric interactive reasoning.The flywheel synthesizes, evolves, and verifies data across diversity, quality, and difficulty; the curriculum progressively aligns perception and reasoning.
  • 4.1.1. Diversity: Knowledge-driven Evolution: The flywheel co-evolves knowledge concepts and visual tools, generating interactive reasoning data from combinations of both.A strong generator creates question-image-code trajectories from knowledge combinations and complementary data from tool combinations, while newly predicted elements expand the knowledge and tool sets.
  • 4.1.2. Coordinated Calibration: Coordinated calibration retains samples only when answers, rendered images, and intermediate visual states are mutually consistent.A checker validates all three aspects, while a repairer reconstructs inconsistent questions and repeatedly re-evaluates them.
  • 4.1.3. Progressive Expansion: Progressive expansion increases reasoning difficulty through parallel and sequential extensions, with verified data revalidated until convergence and limited to three added steps.Parallel extensions add independent observations, whereas sequential extensions depend on prior results or geometric entities; the merged outputs form the final dataset.
  • 4.2. Visual Progressive Training Curriculum: The resulting V-Interaction-400K dataset supports visual interactive reasoning, while point-level supervision addresses models’ difficulty localizing fine-grained visual anchors.The training curriculum begins with perceptual grounding and advances to interactive reasoning; the perception stage is intended to improve point-level localization.

5. VTBench

VTBench evaluates vision-centric interactive reasoning through expert-verified tasks spanning perception, instruction-guided interaction, and interactive reasoning. Its construction combines broad benchmark sampling, expert selection, and interaction annotation converted into validated QA pairs.

  • Benchmark Construction: Samples enter VTBench when at least three of five experts judge visual interaction necessary for solving them.The benchmark uses expert evaluation both during sample selection and in defining its interaction-focused scope.
  • Benchmark Construction: Interaction annotations are transformed into QA pairs and then expert-validated to ensure consistency and accuracy.This annotation pipeline follows expert-based sample selection and supports the benchmark’s interaction-centered evaluation design.
  • Task Dimensions: VTBench evaluates perception, instruction-guided interaction, and interactive reasoning as three stages from visual understanding to adaptive interaction.The tasks range from identifying visual coordinates to following drawing or labeling instructions and solving problems involving visual modifications.
  • Benchmark Composition: VTBench combines nine open-source benchmarks and additional public samples into 1,500 question-answer pairs across logical reasoning, geometry, algebra, and statistics.The benchmark allocates 500 samples to each of its three task types.

6. Experiments

Experiments evaluate V-Thinker on VTBench and general visual-reasoning benchmarks, combining quantitative, qualitative, ablation, and flywheel analyses. V-Thinker improves interactive reasoning and generalizes across reasoning domains.

  • Main Results: 8.8% for Qwen2.5-VL on VTBench illustrates that existing LMMs struggle with fine-grained spatial relationships and precise point localization.The results distinguish general visual reasoning from the perceptual grounding required for interactive reasoning.
  • Main Results: 12.5% average accuracy improvement across three interactive reasoning domains, with over 6% gains in each domain and over 22% improvement in Instruction-Guided Interaction.Under identical setups, V-Thinker consistently outperforms baseline LMMs on VTBench.
  • Qualitative Analysis: V-Thinker-7B autonomously edits images, annotates intermediate steps, explores broader solution spaces, and forms a think–edit loop through rendered code outputs.Qualitative analyses cover interaction editing, rollout sampling, and complete end-to-end cases.
  • Generalized Reasoning: +6.3% on MathVision demonstrates substantial improvement without specific in-domain data introduction, alongside consistent gains across three general reasoning benchmarks.The reported gains include vision-centric mathematics and logical reasoning.
  • Ablation Study: Removing any curriculum stage reduces performance, while removing perception alignment hurts grounding and removing RL causes over 6% drops on MathVision and We-Math.The ablation identifies perception alignment as foundational and RL as important for interactive-pattern exploration and generalization.
  • Data Evolution Flywheel: After five evolution iterations, the knowledge system reaches approximately 50× the initial seed size, while the evolved structure spans 25 domains, 7 layers, and 24,767 nodes.These analyses report non-linear, non-saturating growth in knowledge concepts and visual tools.

7. Conclusion

The conclusion presents V-Thinker as an end-to-end framework for interactive, vision-centric reasoning, supported by data evolution, progressive training, and expert-verified evaluation. It also summarizes prompt-based procedures for synthesizing, calibrating, expanding, and aligning visual reasoning data.

  • 7. Conclusion: V-Thinker bridges visual grounding and code-driven interactive reasoning through an end-to-end framework for Interactive Thinking with Images.The framework treats image interaction as part of the reasoning process.
  • 7. Conclusion: The Data Evolution Flywheel synthesizes, evolves, and verifies datasets across diversity, quality, and difficulty, producing V-Interaction-400K.The paper describes knowledge-driven and tool-driven generation, calibration, repair, and progressive expansion.
  • 7. Conclusion: The Visual Progressive Training Curriculum aligns perception and interactive reasoning through a two-stage training framework.Perception-oriented prompts construct localization, semantic, and integrated reasoning tasks from visual structures.
  • 7. Conclusion: VTBench provides expert-verified evaluation for vision-centric interactive reasoning, while experiments report consistent gains over mainstream vision-language baselines.The benchmark and framework are presented as practical contributions to multimodal system development.

B.1. Extended Quantitative Analysis

The extended comparison confirms that fine-grained perceptual grounding remains weak across model scales, while V-Thinker improves perception, instruction-guided interaction, and visual reasoning.

  • Extended Quantitative Analysis: Large models including GPT-4o, Qwen2.5-VL-72B, and InternVL3-78B remain limited on tasks requiring local spatial grounding.The difficult cases include identifying specific points, intersections, and geometric primitives.
  • Extended Quantitative Analysis: +22.8% on instruction-guided interaction, +6.0% on perception-oriented questions, and +8.6% on visual reasoning relative to 7B-scale open-source models.These improvements remain consistent across the expanded Qwen2.5-VL, InternVL3, and LLaVA-OneVision-1.5 baseline set.

B.2. Additional Analysis of the Data Evolution Flywheel

Additional flywheel analyses show that iterative evolution produces broad, deep knowledge structures while visual tools require consolidation to distinguish meaningful operations from parameter-level variants.

  • Additional Analysis of the Data Evolution Flywheel: The evolved hierarchy spans 25 domains, reaches 7 layers, and contains 24,767 nodes, illustrating expanded conceptual coverage.The extended visualization provides a comprehensive view of the structure produced through iterative evolution.
  • Additional Analysis of the Data Evolution Flywheel: Visual tools undergo consolidation because executable drawing routines can differ only in minor parameters while representing functionally identical operations.Without consolidation, such variants can artificially inflate apparent tool diversity.

C.1. Evaluation Dimensions

VTBench evaluates vision-centric interactive reasoning as a progression from fine-grained perception to instructed visual actions and interaction-dependent reasoning. It covers three task types across diverse mathematical and reasoning benchmarks, using task-specific visual or answer-based evaluation.

  • VTBench organizes evaluation into perception, instruction-guided interaction, and interactive reasoning dimensions.The dimensions model progression from identifying visual details to executing visual instructions and solving tasks that require interaction.
  • Table 3 reports overall VTBench performance across perception, instruction-guided interaction, and interactive reasoning.
  • The benchmark contains 1,500 QA pairs across nine open-source benchmarks and four domains, including geometry, algebra, statistics, and logical reasoning.
  • Perception and instruction-guided interaction generate Python code whose rendered outputs are compared with annotations, whereas interactive reasoning answers are judged for correctness.

D.1. Implementation Details (Evaluation)

The evaluation compares V-Thinker with multimodal systems spanning proprietary and open-source architectures and parameter scales. The broader motivation is to assess interactive, structured visual reasoning for applications requiring traceable visual inference.

  • Evaluation covers MathVision, VisuLogic, We-Math, and VTBench under official protocols, with accuracy as the primary metric.
  • The baseline suite spans GPT-4o, Qwen2.5-VL, InternVL3, and LLaVA-OneVision-1.5 families across diverse architectures and parameter scales.
  • V-Thinker’s interactive capabilities target structured visual reasoning applications such as diagram understanding, scientific figure analysis, embodied perception, and collaborative interfaces.
  • The paper frames creator-oriented visual data generation and explicit reasoning traces as bridges from multimodal foundation models to practical tool-oriented systems.

F. Limitation

V-Thinker establishes a generalized paradigm for reasoning through visual interaction but remains constrained by model scale and domain-specific capability. Future work targets stronger robustness and generalization across reasoning domains.

  • Computational constraints limit the current model’s scale and reduce capability on knowledge-intensive or domain-specific tasks.
  • The paper’s primary scope is establishing a unified framework for interactive reasoning, with future work focused on optimization, robustness, and broader generalization.
  • The authors argue that advancing model capabilities may require re-examining data-construction paradigms and the upper bound of model reasoning.
Loading 2511.04460v2…