Source-linked AI summary
Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators
Chenming Zhu, Jingli Lin, Yilin Long, Peizhou Cao, Tai Wang, Jiangmiao Pang, Xihui Liu
TL;DR
VLMs struggle to infer unobserved spatial layouts and useful alternative viewpoints from limited egocentric observations. Astra combines an RL-trained agentic VLM with an action-conditioned world simulator, and Astra-WM raises simulator-augmented Gemini-3-Flash on MMSI-Bench from 45.1 to 49.5.
Problem
VLMs often struggle to infer unobserved layouts, maintain cross-view consistency, and reason from alternative viewpoints when observations are limited.
Method
Astra couples an agentic VLM policy with an action-conditioned world simulator that generates imagined novel views during reasoning.
Results
45.1 to 49.5: Astra-WM improves simulator-augmented Gemini-3-Flash on MMSI-Bench, showing that reliable simulation and selective tool-use policy learning are both necessary.
Takeaways & Limitations
Effective visual imagination requires a learned interaction process for acquiring, grounding, and using spatial evidence, rather than simulator access alone.
Takeaways & Limitations
Forced tool use can underperform direct answering because models may fail to choose useful actions or integrate imagined observations effectively.
Abstract
from arXiv · showhide
While Vision-Language Models (VLMs) have shown strong visual reasoning capabilities, their spatial reasoning abilities remain largely constrained to the observed images and text-oriented chain-of-thought. They often struggle to infer unobserved layouts, maintain cross-view consistency, and reason from alternative viewpoints when only limited egocentric observations are available. In this work, we study this problem as thinking with imagination, where a VLM actively acquires imagined visual evidence by interacting with a world simulator during reasoning. We propose Astra, an agentic spatial reasoning framework that empowers VLMs with action-conditioned visual imagination. Specifically, Astra couples Astra-VL, an RL-trained VLM policy, with Astra-WM, a Bagel-based world simulator that generates novel-view observations from context images and natural-language camera motions. To provide reliable imagined evidence, Astra-WM is trained with view consistency tuning to improve pose and content consistency across views. In the RL stage, we propose a world-simulator-in-the-loop two-phase RL curriculum to stabilize tool-use exploration and advance the model's ability to invoke the simulator only when imagined observations improve over direct answering. Experiments demonstrate that both the world simulator and the agentic policy are necessary: Astra-WM improves simulator-augmented Gemini-3-Flash on MMSI-Bench from 45.1 to 49.5, while Astra-VL improves the Qwen3-VL backbone from 29.8 to 38.8 on MMSI-Bench and from 36.8 to 42.7 on MindCube. These results show that imagined observations can provide useful spatial evidence, but effective world-model-augmented reasoning requires learning when, where, and how to imagine.
1 Introduction
Astra frames visual spatial reasoning as interactive evidence acquisition, enabling a VLM to request action-conditioned imagined views when limited egocentric observations leave layouts or spatial relations ambiguous. Its reliable world simulator and learned tool-use policy are jointly necessary, with Astra-WM improving simulator-augmented Gemini-3.0-Flash on MMSI-Bench from 45.1 to 49.5.
- Motivation: Limited egocentric views force agents to infer unobserved layouts, maintain cross-view consistency, and reason from alternative perspectives.Small viewpoint changes can resolve ambiguities involving camera-object relations or viewpoint alignment.
- Motivation: Existing reasoning chains, intermediate views, and cognitive maps generally use fixed visual contexts or predefined representations rather than acquiring missing viewpoints actively.This motivates treating spatial reasoning as an interactive evidence-acquisition problem.
- Framework: Astra integrates Astra-VL, a Qwen3-VL-based agentic reasoning model, with Astra-WM, a Bagel-based simulator that generates action-conditioned novel views.The simulator must produce visually plausible views that remain consistent with requested camera motion, scene content, and spatial layout.
- Training: A two-phase world-simulator-in-the-loop RL curriculum teaches valid simulator invocation before encouraging selective imagination over direct no-tool answering.The first phase maintains on-policy tool-use trajectories, while the second compares tool-augmented reasoning with direct answering.
- Results: 45.1 to 49.5: Astra-WM improves simulator-augmented Gemini-3.0-Flash on MMSI-Bench, while off-the-shelf Bagel provides limited benefit.The result supports training the simulator for stronger pose and content consistency rather than relying on generic image generation.
2 Related Work
Prior work improves VLM spatial intelligence through specialized training data and geometric architectural priors, while image-based reasoning systems interleave textual reasoning with iterative visual operations. These studies also show that reinforcement learning and staged training can support effective multi-turn visual tool use.
- VLMs for Spatial Intelligence: VLM spatial reasoning remains challenging, with existing methods relying on specialized spatial datasets or geometric priors such as 3D representations, depth cues, and structure-aware features.These approaches target understanding and manipulating 3D relationships from visual observations.
- Thinking with Images: Recent image-reasoning systems interleave textual reasoning with iterative visual operations, using reinforcement learning and staged training to develop effective multi-turn tool use.DeepEyes demonstrates reinforcement-learning-induced image-text reasoning, while Pixel Reasoner and Mini-o3 highlight staged training for overcoming learning traps and discovering deeper trajectories.
3 Method
Astra formulates spatial reasoning as an interactive process in which an agentic VLM can answer directly or invoke a spatially consistent world simulator for imagined novel views. The method combines action-conditioned view consistency tuning for Astra-WM with a two-phase RL curriculum that first teaches valid tool use and then selective imagination.
- Task and Objective: The agent maintains a trajectory containing the question, context images, reasoning, simulator actions, and outputs, then chooses either Invoke or Answer.Invoke queries an imagined observation; Answer terminates the trajectory with the final response.
- Action Space: Invoke queries specify a reference image, camera-motion type, and magnitude, translated into natural-language instructions covering lateral, forward/backward, yaw, vertical, and pitch movements.The policy must track motion provenance to distinguish original images from generated views and identify each view’s reference and camera motion.
- Astra-WM: Astra-WM fine-tunes Bagel with view consistency tuning so generated novel views preserve scene identity, follow requested motion, and maintain relative object layouts across viewpoints.The simulator is conditioned on context images, a selected reference image, and a natural-language camera-motion instruction.
- Astra-VL Training: The two-phase RL curriculum first teaches valid simulator interaction with a capped use bonus, then rewards simulator calls only when they improve over direct answering.Phase two compares each tool-using rollout with a no-tool baseline, rewarding positive gains and penalizing harmful simulator use.
- World Simulator SFT Data: 544k quality-verified SFT samples from posed multi-view indoor scenes train Astra-WM on context images, camera-motion queries, and target observations.The data covers IsaacSim, ScanNet++, ScanNet, Matterport3D, DL3DV, and ARKitScenes.
4 Experiments
Experiments evaluate Astra on MMSI-Bench and MindCube using exact-match accuracy, alongside tool-use behavior and simulator quality. Results show that spatially consistent imagined observations and learned agentic control are both necessary for effective spatial reasoning.
- Evaluation setup: Evaluation uses MMSI-Bench and MindCube, reporting exact-match accuracy plus tool-call rate, average calls, and world-simulator quality measures.MMSI-Bench contains 1,000 multi-view examples, while MindCube uses structured 3D environments.
- Effectiveness of Astra-WM: Forced tool use does not teach whether evidence is needed, which viewpoint is informative, or how to ground imagined observations, and can underperform direct answering.Qwen3-VL and Bagel are cited as open-source examples that may perform worse under forced tool use than direct answering.
- Effectiveness of Astra-VL: Agentic Tool-Use conditions simulator interactions on the current reasoning state, requiring decisions about evidence need, camera motion, and observation grounding.Unlike forced tool use, Astra autonomously decides when and how to interact with the simulator.
- Ablations: View consistency tuning improves Astra-WM’s motion following and scene-content preservation, making imagined observations more spatially reliable than off-the-shelf Bagel’s.The ablations identify simulator spatial consistency, reward design, and inference-time interaction control as determinants of Astra’s effectiveness.
- Ablations: 4.9% tool-call rate: a single-stage tool-gain reward causes rapid collapse toward direct answering, with 0.049 calls per rollout.The result indicates sparse relative-gain rewards are too weak early in training to maintain useful simulator exploration.
5 Conclusion
Astra couples a spatially consistent world simulator with an RL-trained VLM policy to help models acquire missing spatial evidence through imagined alternative viewpoints. The framework relies on both reliable simulation and selective tool-use policy learning.
- Framework: Astra couples a spatially consistent world simulator with an RL-trained VLM policy for agentic spatial reasoning.The framework is designed around coordinated simulation and policy learning.
- Imagined evidence: By querying imagined observations from alternative viewpoints, Astra enables VLMs to acquire missing spatial evidence beyond the given images.This extends reasoning with visual evidence generated during interaction with the simulator.
- Key requirements: Both reliable simulation and selective tool-use policy learning are necessary for Astra’s spatial reasoning framework.View consistency tuning improves the usefulness of generated views, while policy learning supports selective simulator use.
A Additional Details on Training Data · A.1 World Simulator SFT Data
The world simulator SFT data trains Bagel to synthesize target views from context images and natural-language camera motions. Samples are collected from 11,292 scanned scenes and filtered using coverage, viewpoint-diversity, and single-motion constraints.
- A.1 World Simulator SFT Data: The dataset consists of tuples (Ictx, p, Itgt) that pair context images and a camera-motion instruction with the target image from the transformed viewpoint.The model must infer scene geometry and synthesize the target observation under the specified camera pose.
- A.1 World Simulator SFT Data: 11,292 scenes from five datasets cover indoor and outdoor environments through scanned RGB-D videos.The sources are ScanNet, ScanNet++, Matterport3D, ARKitScenes, and DL3DV training splits.
- A.1 World Simulator SFT Data: Each scene is represented by per-camera RGB images, depth images, and poses Ti ∈SE(3).The scene representation is an RGB-D video with tuples {(Ii, Di, Ti)}M.
- A.1 World Simulator SFT Data: Each training sample uses 2–3 context cameras and one target camera selected as a camera pair.The pair is denoted (Cctx, Ctgt).
- A.1 World Simulator SFT Data: The view-coverage constraint requires most of the target camera’s visible scene to be observable from the context cameras.Visible point clouds are obtained by back-projection, and the coverage ratio is explicitly defined for sample filtering.
- A.1 World Simulator SFT Data: Camera pairs must provide sufficient viewpoint diversity, with yaw and pitch differences used to quantify angular variation.For context-camera pairs Ci and Cj, at least one specified diversity condition must hold; ∆θij and ∆ϕij denote yaw and pitch differences.
- A.1 World Simulator SFT Data: A source context camera is transformed relative to the target, and the resulting dx, dy, dz, dθ, and dϕ components become natural-language motion prompts.The components represent horizontal translation, vertical motion, yaw, and pitch.
- A.1 World Simulator SFT Data: Samples retain only one-step cases where a single motion component changes significantly more than the others.This filtering simplifies the task so one of the five motion components dominates.
B World Simulator Evaluation
The world simulator is evaluated on 1,000 samples from five datasets using complementary pose- and content-consistency metrics. Together, these metrics assess camera-motion fidelity, object-level preservation, and spatial structure in generated views.
- Evaluation Set: The evaluation samples 200 examples each from DL3DV, ScanNet, ScanNet++, Matterport3D, and ARKitScenes, yielding 1,000 samples.Generated views are assessed from pose-consistency and content-consistency perspectives.
- Pose Consistency: Pose consistency compares predicted and ground-truth camera motions across translation, yaw, and pitch components.The generated RGB image is depth-estimated with Depth Anything, then aligned with the source RGB-D observation to recover its camera pose.
- Pose Consistency: A sample is pose-consistent when unchanged dimensions are preserved and the dominant changed dimension has the correct direction and sufficiently close magnitude.The final pose score averages success rates across all evaluation samples.
- Content Consistency: Content consistency measures preservation of object categories, counts, locations, and pairwise spatial relations between generated and target images.A VLM extracts shared key object categories, and GroundingDINO detects corresponding objects for matching.
- Content Consistency: Object-level recall and precision quantify target-object recovery and consistency of generated objects with the target image.Objects are matched using category consistency and bounding-box overlap.
- Content Consistency: Topology consistency averages whether matched object pairs preserve the target image’s directions between bounding-box centers.Relations are treated as missing when either object cannot be matched in the generated image.
C Case Study and Error Analysis · D Prompt Templates and Tool Schemas
The case studies trace successes and failures across the full visual-CoT reasoning chain, showing that effective simulator-augmented reasoning depends on identifying uncertainty, choosing informative actions, trusting consistent observations, and using them correctly. The paper also provides the complete Agentic Workflow prompt used for training and inference.
- C Case Study and Error Analysis: Case studies evaluate the initial spatial hypothesis, tool-use decision, generated action, simulator observation, and final answer rather than assigning failures to isolated categories.These cases show why effective world-model-augmented reasoning requires more than tool access.
- C Case Study and Error Analysis: The error analysis demonstrates that simulator-augmented reasoning requires coordinated control of uncertainty, action selection, observation reliability, and evidence integration.Failures can occur at multiple linked stages of the reasoning chain, not only during tool invocation.
- C Case Study and Error Analysis: Successful imagination resolves ambiguous spatial relations by querying a viewpoint that supplies missing visual evidence, enabling the model to revise or confirm its hypothesis.The imagined observation contains evidence unavailable in the original context images.
- C Case Study and Error Analysis: Correct tool access can still fail when the chosen camera action does not reduce relevant uncertainty or leaves the target unrelated to the queried relation.Examples include unhelpful rotations, moves from poor reference images, and irrelevant viewpoint changes.
- C Case Study and Error Analysis: Simulator failures arise when generated observations violate the requested camera motion or fail to preserve scene layout across views.Typical errors include incorrect viewpoint changes, missing or hallucinated objects, identity drift, and layout inconsistency.
- C Case Study and Error Analysis: A potentially useful observation may be ignored or misused when the model relies on its initial hypothesis, confuses image indices, or misinterprets camera motion.The model can also over-trust generated views and discard reliable evidence from the original context.
- D Prompt Templates and Tool Schemas: The full Agentic Workflow prompt used during training and inference is illustrated in Fig. 7.The paper presents the prompt template as part of its implementation materials.
E Limitations and Future Work
Astra still lacks robust selective imagination: exploration can cause direct-answer collapse or simulator overuse, while generated views and tool-state handling may be unreliable. Future work targets imagination governance through better routing, information-seeking actions, verification, and preference data for helpful versus harmful tool calls.
- Current limitations: The policy can collapse to direct answering without exploration or overuse the simulator when given a simple usage bonus.These failures show that tool-use exploration remains difficult to regulate.
- Current limitations: Generated views may be visually plausible yet irrelevant to the queried relation.The limitation concerns the usefulness of imagined evidence, not only its visual plausibility.
- Current limitations: The policy may confuse original and generated image indices, over-trust generated observations, or stop exploring after the first generated observation.These issues can undermine reliable use of simulator outputs during reasoning.
- Future work: Future work should strengthen imagination governance with routing, information-gain action policies, post-observation verification, and preference data contrasting helpful and harmful tool calls.The proposed improvements address when to imagine, how to act, how to assess observations, and how to distinguish valuable from harmful tool use.
F License Information for the Public Datasets Used
The paper identifies the public datasets used for World Simulator SFT and RL training and records their licensing conditions. The SFT data draw from five scene datasets, while RL samples come from two additional datasets with distinct usage terms.
- World Simulator SFT Data: World Simulator SFT data use raw scene data from ScanNet, Matterport3D, ARKitScenes, DL3DV, and ScanNet++.These five datasets provide the scene data used to construct the World Simulator SFT data.
- RL Training Data: RL training samples are sourced from SenseNova-SI-800K and VST-500K.The passage distinguishes these RL sources from the datasets used for World Simulator SFT.
- Licensing Conditions: ARKitScenes uses the Apple license, SenseNova-SI-800K uses Apache-2.0, VST-500K is research-only, and ScanNet, ScanNet++, Matterport3D, and DL3DV use custom licenses.The custom licenses are identified as the respective licenses governing those four datasets.