Source-linked AI summary
Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search
Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, Hengshuang Zhao
TL;DR
Existing open-source multimodal models often rely on rigid reasoning and few interaction turns, limiting performance on visual search tasks requiring exploration. Mini-o3 addresses this with a challenging dataset, diverse cold-start trajectories, and over-turn masking, achieving state-of-the-art visual-search performance while scaling from six training turns to tens of inference turns. Accuracy continues to improve as interaction depth increases.
Problem
Existing open-source VLMs show monotonous reasoning patterns and limited interaction turns, weakening them on challenging visual search tasks that require trial-and-error exploration.
Method
Mini-o3 combines the Visual Probe Dataset, iterative cold-start trajectory synthesis, and reinforcement-learning over-turn masking for multi-turn image-tool reasoning.
Results
Mini-o3 achieves state-of-the-art performance across VisualProbe, V* Bench, and HR-Bench, with trajectories scaling to tens of turns and accuracy improving as interaction depth increases.
Takeaways & Limitations
Training with a six-turn upper bound can yield inference trajectories that scale to substantially more turns for difficult visual search problems.
Abstract
from arXiv · showhide
Recent advances in large multimodal models have leveraged image-based tools with reinforcement learning to tackle visual problems. However, existing open-source approaches often exhibit monotonous reasoning patterns and allow only a limited number of interaction turns, making them inadequate for difficult tasks that require trial-and-error exploration. In this work, we address this limitation by scaling up tool-based interactions and introduce Mini-o3, a system that executes deep, multi-turn reasoning -- spanning tens of steps -- and achieves state-of-the-art performance on challenging visual search tasks. Our recipe for reproducing OpenAI o3-style behaviors comprises three key components. First, we construct the Visual Probe Dataset, a collection of thousands of challenging visual search problems designed for exploratory reasoning. Second, we develop an iterative data collection pipeline to obtain cold-start trajectories that exhibit diverse reasoning patterns, including depth-first search, trial-and-error, and goal maintenance. Third, we propose an over-turn masking strategy that prevents penalization of over-turn responses (those that hit the maximum number of turns) during reinforcement learning, thereby balancing training-time efficiency with test-time scalability. Despite training with an upper bound of only six interaction turns, our model generates trajectories that naturally scale to tens of turns at inference time, with accuracy improving as the number of turns increases. Extensive experiments demonstrate that Mini-o3 produces rich reasoning patterns and deep thinking paths, effectively solving challenging visual search problems.
1 Introduction
Mini-o3 targets difficult visual search by combining diverse, exploratory multi-turn reasoning with a training recipe designed to support deeper test-time interaction. Despite training with only six turns, its accuracy improves as the inference-time turn limit increases from 4 to 32.
- Motivation: Existing open-source VLMs struggle with challenging visual search because their reasoning patterns are monotonous and their tool interactions are shallow.DeepEyes reaches 35.1% accuracy on VisualProbe-Hard and averages only one image-tool turn per HR-Bench-4K example.
- Results: Mini-o3 generates complex reasoning patterns and scales interaction depth at inference, with accuracy improving as the maximum number of turns rises from 4 to 32 after six-turn training.The model produces trajectories spanning many turns, including an 11-turn example, while the training budget remains capped at six turns.
- Approach: Mini-o3 combines a Visual Probe Dataset, iterative cold-start trajectory synthesis, and over-turn masking to reproduce diverse, deep image-tool reasoning.The dataset emphasizes trial-and-error problems; synthesized trajectories include depth-first search, self-reflection, and goal maintenance; masking avoids penalizing trajectories that hit interaction or context limits.
- Approach: The framework iteratively generates a thought and action from the prior observation, then obtains a new observation from the action’s image-tool parameters.This thought–action–observation loop continues until a final answer or a predefined context-length or turn limit is reached.
2 Related Work
Prior work established vision-language modeling and reinforcement-learning approaches for visual and textual reasoning, while newer tool-augmented agents demonstrated long-horizon interaction. Mini-o3 extends this direction toward multimodal agents with deeper and more diverse image-tool reasoning.
- Vision–Language Models: Foundational VLMs couple pretrained vision encoders with language models through a projector and benefit from larger models and diverse image–text data.BLIP-2, Flamingo, and the LLaVA series established this paradigm.
- Reinforcement Learning: Reinforcement-learning methods have improved structured reasoning in VLMs, especially for textual tasks such as counting, logical inference, and mathematics.The cited approaches include REINFORCE, PPO, RLOO, ReMax, GRPO, REINFORCE++, Dr.GRPO, and GSPO.
- Tool-Integrated Agents: Tool-augmented agents have shown strong abilities on challenging compositional problems, but current models often use limited interaction depth and rigid reasoning patterns.These constraints motivate methods that support longer, more adaptable multimodal interaction.
- Tool-Integrated Agents: Mini-o3 advances this line with a training recipe for multi-turn image-tool use that improves reasoning diversity and adaptability in visually grounded tasks.The paper positions its contribution as a multimodal agentic approach rather than a purely textual reasoning method.
3 Our Approach
Mini-o3 combines an iterative visual-tool pipeline with challenging training data, cold-start trajectories, and over-turn masking. The approach supports diverse multi-turn reasoning and test-time scaling beyond the training turn limit.
- Overall Agentic Pipeline: The agent iteratively generates thoughts and actions, invokes image tools, appends observations to its history, and stops at a final answer or predefined limits.Grounding actions can zoom into the original image or any prior observation.
- Two-phase Training: The two-phase training procedure uses supervised fine-tuning on multi-turn tool-use trajectories followed by GRPO with verifiable, semantics-aware rewards.Training imposes upper bounds of 6 interaction turns and a 32K context length.
- Visual Probe Dataset: VisualProbe contains 4,000 training and 500 testing visual question–answer pairs featuring small targets, distractors, and high-resolution images that require iterative trial-and-error exploration.The dataset spans easy, medium, and hard difficulty levels.
- Diverse Cold-start Data: Cold-start collection prompts an existing VLM to imitate manually crafted exemplars by producing thoughts and actions across turns, retaining only correct trajectories.Approximately 6,000 trajectories are collected from 6 exemplars.
- Lower Down Max Pixels: Reducing the maximum image budget to 2M pixels or lower allows more interaction turns within the 32K-token context budget.The adjustment is intended to improve solve rates on long-horizon problems.
- Over-turn Masking: Vanilla GRPO penalizes over-turn responses with negative advantages despite their correctness being unknown, encouraging premature answers and limiting difficult-task exploration.Over-turn responses can exceed the turn or context limits and occur frequently early in training.
- Over-turn Masking: Over-turn masking applies a completion mask so trajectories hitting interaction or context limits do not contribute negative learning signals or gradients.The masked objective is normalized by the number of completed generations.
- Over-turn Masking: Despite a small training turn bound, test-time trajectories can extend to dozens of rounds, with accuracy improving monotonically as interaction turns increase.The paper identifies over-turn masking as essential for realizing this test-time scaling.
4 Experiment
Experiments evaluate Mini-o3 across visual search benchmarks and isolate the contributions of hard data, cold-start SFT, over-turn masking, pixel budgets, and training turn limits. Mini-o3 achieves state-of-the-art performance, while ablations reveal trade-offs between convergence speed, reasoning depth, perceptual accuracy, and interaction depth.
- Main Results: Mini-o3 achieves state-of-the-art performance across VisualProbe, V* Bench, and HR-Bench, substantially outperforming open-source baselines.VisualProbe and V* Bench use Avg@32, HR-Bench uses Avg@8, and MME-Realworld uses Avg@1.
- Ablation Study: Removing hard RL data decreases VisualProbe-Hard performance by approximately 8.6 points.The result indicates that challenging reinforcement-learning samples support complex reasoning trajectories.
- Ablation Study: Without cold-start SFT, performance collapses, supporting its necessity for multi-turn tool use.The authors hypothesize that the base model lacks exposure to multi-turn agentic trajectories before this initialization.
- Ablation Study: Over-turn masking benefits multi-turn reinforcement learning by avoiding penalties for truncated responses and enabling test-time turn scaling.Trajectories reaching the interaction or context limit are excluded from policy updates, so their losses contribute no gradients.
- Ablation Study: Both overly large and overly small pixel budgets are suboptimal because they respectively reduce interaction depth through early stopping or increase perceptual hallucinations.Overall performance is best when the maximum pixel budget is tuned to balance perceptual accuracy and interaction depth.
- Ablation Study: A 6-turn training budget converges faster initially, whereas a 12-turn budget reaches a higher performance ceiling but converges more slowly.With the lower budget, performance plateaus after approximately 150 steps on VisualProbe-Hard.
5 Conclusion
Mini-o3 targets challenging visual search problems with diverse, deep multi-turn reasoning. Its trajectories scale to tens of turns at inference, while the three-part recipe combines VisualProbe, cold-start data collection, and over-turn masking.
- Conclusion: Mini-o3 produces diverse reasoning patterns and deep chains of thought for visual search problems requiring iterative exploration and trial-and-error.Its trajectories scale to tens of turns, with accuracy continuing to improve during those interactions.
- Conclusion: With training turn limits of 6 and 12, Mini-o3's VisualProbe-Hard accuracy can be compared during training to assess convergence and performance ceilings.The cited figure contrasts the two upper limits on interaction turns.
- Conclusion: The recipe combines the VisualProbe dataset, an in-context pipeline for cold-start data, and over-turn masking added to GRPO.The masking strategy prevents undue penalties on responses exceeding the training turn budget and facilitates test-time turn scaling.
A More illustrations of multi-turn trajectories
Illustrations show Mini-o3 using multi-turn visual search to inspect cluttered scenes, revise hypotheses, and verify localized details. The examples span directional signs, container markings, and road-sign recognition.
- Urban Intersection: In an urban intersection, Mini-o3 progressively zooms, revises hypotheses, and backtracks to identify the direction of an arrow.The associated question asks for the arrow below the text “PRAKING”.
- Container Yard: In a container yard, targeted zoom-ins, cross-checking, and corrective backtracking locate “67.200” and retrieve the value “22G1” beneath it.The example demonstrates precise text localization and step-by-step verification in a cluttered scene.
- Lakeside Village: In a lakeside village, coarse-to-fine zooming and verification locate the circular sign above a triangular warning sign and recognize “30” after 18 reasoning turns.The example illustrates extended multi-turn search for a small road-sign target.