Learning to Think and Explore in 3D World

1 INTRODUCTION

• Existing 3D VLM work often provides observations specified in advance, while partial observability requires models to acquire missing task-relevant evidence.The paper asks whether a VLM can learn active exploration as part of 3D reasoning.Source: §1 · p. 1
• Act3D keeps the 3D scene external and lets the model access it through sequential action-observation interactions from a fixed initial egocentric view.The model navigates and controls its viewpoint before committing to an answer.Source: §1 · p. 1, Figure 1 · p. 2
• Trajectory-supervised fine-tuning establishes the interaction protocol, then RL optimizes the full policy mainly through final-answer correctness without direct supervision of intermediate actions or viewpoints.Exploration must be learned through credit assignment from task performance.Source: §1 · p. 2
• Across six 3D benchmarks, Act3D achieves 53.2% average accuracy, improving its base Qwen model by 22.0% and surpassing Gemini 3.6 Flash by 1.0%.It also outperforms open-source 3D methods while using 50–70% fewer input frames.Source: §1 · p. 2

2 RELATED WORK

• Prior 3D VLM approaches largely assume the evidence needed to answer is already provided, including scene representations, fixed views or videos, and reconstruction-enhanced inputs.Act3D instead receives egocentric 2D observations and learns to acquire task-relevant evidence through interaction.Source: §2 · p. 3
• Spatial-reasoning RL and multimodal agent methods primarily manipulate or revisit evidence contained in the input, whereas Act3D actively acquires new evidence in a 3D environment.The model jointly optimizes reasoning and interaction in a single rollout.Source: §2 · p. 3
• EmbodiedQA and active-viewpoint methods study navigation or view selection, while Act3D abstracts embodiment to camera control over scanned scenes and jointly learns exploration and question answering.Related approaches include training-free view selection, RL-optimized reconstruction tools, and distilled view graphs.Source: §2 · p. 3

3 METHOD

• Two-stage training initializes a reason–act–observe policy from curated demonstrations, then optimizes exploration through final-performance rewards without intermediate viewpoint or action supervision.The demonstrations establish an interaction protocol and behavioral prior rather than prescribing canonical exploration targets.Source: §3 · p. 3, §3.2 · p. 6

3.1 REASONING IN 3D UNDER PARTIAL OBSERVABILITY

• The policy observes rendered egocentric views and camera configurations while the reconstructed scene remains hidden and fixed.The task is formalized as a POMDP with a question, scene, and ground-truth answer; the camera viewpoint changes through the policy’s actions.Source: §3.1 · pp. 3–4
• Relative navigation actions pair a motion primitive with a step multiplier, while absolute actions specify a camera position and look-at point.Relative actions are used throughout training and evaluation because they are camera-centric and map to interpretable motions; the scene-bounding-box projection clamps camera position without collision checking.Source: §3.1 · p. 4

3.2 TRAJECTORY-SUPERVISED POLICY INITIALIZATION

• Trajectory-supervised fine-tuning uses curated demonstrations to establish interaction competence, after which reinforcement learning optimizes the full interactive policy.The curated corpus contains 5,712 trajectories, 20,603 rendered observations, and 830 scenes; demonstrations initialize behavior rather than prescribe a unique exploration strategy.Source: §3.2 · pp. 5–6

3.3 OUTCOME-DRIVEN REINFORCEMENT LEARNING

• RL training uses question–answer pairs and valid initial views, drawing 51,543 questions from ScanQA, MP3D, and generated questions over ScanNet and ARKitScenes.Rollouts from the same initial condition can acquire different evidence, enabling comparison of information-acquisition strategies under the same task objective.Source: §3.3 · p. 6
• Reinforcement learning assigns rollout-level credit across reasoning and interaction, allowing the policy to learn which evidence to acquire as well as how to reason over it.The interaction reward is credited only for correct-answer rollouts and rewards verified object visibility or, otherwise, interactions up to a cap; it does not prescribe intermediate viewpoints or actions.Source: §3.3 · pp. 6–7

4.1 SETUP

• Evaluation spans six benchmarks and compares passive perception methods, active VLMs, and base and SFT checkpoints under the stated protocols.Training uses ScanQA-only or mixed-data configurations, with trajectory-supervised fine-tuning followed by GRPO; the evaluation initializes models from one render and allows up to twelve interactions.Source: §4.1 · p. 7

4.2 MAIN RESULTS

• 53.2% average accuracy across six benchmarks makes mixed-data Act3D the highest-performing evaluated method, surpassing the Qwen3.5-9B base by 22.0% and Gemini 3.6 Flash by 1.0%.It also outperforms evaluated open-source 3D methods, using a single initial view and acquiring additional evidence through interaction.Source: §4.2 · p. 7, Table 1 · p. 8
• RL raises mixed-data average accuracy from 48.6% to 53.2% (+4.6%) over SFT initialization, with gains on five of six benchmarks.The largest reported gains are ARKit (+14.1%) and SPBench-MV (+8.1%).Source: §4.2 · p. 8
• ScanQA-only RL raises average accuracy from 47.8% to 50.6% (+2.8%) over SFT and improves average accuracy on the other five benchmarks from 49.5% to 51.6% (+2.1%).Gains occur on four of those five benchmarks, including MP3D (+6.0%) and SQA3D (+3.8%).Source: §4.2 · p. 8

4.3 LEARNING DYNAMICS AND EXPLORATION BEHAVIOR

• ScanQA-only validation accuracy rises from 40.2% to 45.5%, while mixed-data training increases mean interactions per episode from approximately 3.3 to 8.3.Mixed-data validation improves across all three benchmarks, with the largest gains on ARKitScenes; Figure 4 distinguishes validation accuracy in panels (a) and (b) from in-training rollout interactions in panel (c).Source: §4.3 · p. 9
• 16.3% combined SFT-and-RL improvement in the highest exploration-demand group contrasts with 0.6% in the lowest; RL adds 10.9% over SFT in the highest group.Demand is estimated geometrically from the answer object's bearing and distance from the initial viewpoint, independently of the model's actual interactions.Source: §4.3 · p. 9, Figure 5 · p. 9

4.4 QUALITATIVE ANALYSIS.

• Act3D changes viewpoint to find occluded or off-screen targets, uses brief verification when a target is visible, and gathers complementary views for spatial comparisons.Reasoning is refined as observations arrive, helping resolve ambiguities through successive interactions.Source: §4.4 · p. 9

4.5.1 INITIAL OBSERVATION

• Despite using only one initial image instead of five, initialization with a bird’s-eye view (BEV) and four rendered views achieves comparable accuracy; combining BEV with one rendered view reduces accuracy to 36.7%.Source: §4.5.1 · p. 10

4.5.2 OBSERVATION INTERFACE: RENDERING VS. RETRIEVAL

• 10.6% higher average accuracy across all 13 evaluation checkpoints is reported for rendering over nearest-view real-image retrieval under matched training conditions.Retrieved images can deviate from requested camera poses, whereas renders can be generated at arbitrary poses; Figure 7 illustrates pose discrepancies.Source: §4.5.2 · p. 10, Figure 7 · p. 10

4.6 ACTION SPACE

• 41.4% ScanQA accuracy with relative camera movements exceeds 36.7% with absolute camera-pose prediction.Relative actions use the camera’s local coordinate system and intuitive translations and rotations, aligning with sequential, observation-conditioned exploration.Source: §4.6 · p. 11

4.7 SUPERVISED INITIALIZATION

• With SFT initialization, ScanQA accuracy rose from 39.5% to 42.7% as interactions increased from 2.7 to 8.7 per episode; without SFT, 470-step RL reached 41.4% with 2.0 interactions.The settings differ in backbone and training budget, so the comparison suggests SFT facilitates adaptation but is not necessary for learning to interact.Source: §4.7 · p. 11

4.8 INTERACTION REWARD

• Adding a 0.05 per-interaction reward raised interactions to 9.4–10.1 per episode and improved all three held-out benchmarks, including SPBench-MV by 6.4%, while reducing training-domain accuracy by 0.3–2.3%.Figure 10 compares ScanQA accuracy with and without the interaction reward across held-out and training benchmarks.Source: §4.8 · p. 11, Figure 10 · p. 12
• An explicit interaction reward mainly encourages evidence acquisition, while outcome-based feedback guides the policy toward effective reasoning.Source: §4.8 · p. 12

• Across six 3D benchmarks, Act3D achieved the highest average accuracy of 53.2%, improving on its SFT initialization by 4.6% and outperforming active and passive 3D reasoning methods.Source: §5 · p. 12
• Act3D treats 3D representations as interactive environments rather than complete inputs, iteratively reasoning, acting, and acquiring visual evidence from a single observation.Source: §5 · p. 12
• The gains extend beyond the training benchmark and are particularly pronounced on questions requiring more extensive exploration, while qualitative results show question-conditioned strategies adapting to new observations.Source: §5 · p. 12

• The consistency set contains all scenes indistinguishable from the true scene under the interaction history, and a question is answerable when those scenes all induce the same answer.Source: A WHY IS INTERACTION NECESSARY? · p. 18
• In a deterministic environment, each added observation can only shrink the set of compatible scenes, and expected answer uncertainty cannot increase, though a particular observation may leave uncertainty unchanged or increase it.Source: A WHY IS INTERACTION NECESSARY? · p. 18
• When the initial question-view pair is consistent with multiple scenes inducing different answers, no model capacity can resolve that ambiguity from the initial view alone.Source: A WHY IS INTERACTION NECESSARY? · p. 18
• Successful interaction therefore requires selecting viewpoints informative for the current question, because task-dependent evidence unavailable initially can make the question answerable.Source: A WHY IS INTERACTION NECESSARY? · p. 18

• The reproduction used the released LLaVA-Video-Qwen2-7B checkpoint, official dataloader, frame sampler, scorer, and bird’s-eye-view input.Source: B REPRODUCING ROSS3D · p. 18
• The only implementation change was directly calling the evaluation function because the original eight-GPU Ray launcher was incompatible with the scheduler.Source: B REPRODUCING ROSS3D · p. 18
• On 4,675 ScanQA validation questions, the reproduced Ross3D scores closely matched published results across every metric, validating the evaluation pipeline and comparability with other reported ScanQA results.Source: B REPRODUCING ROSS3D · p. 18, Table 3 · p. 18

C.1 INTERACTIVE POLICY AND EVALUATION

• The policy prompt is unchanged between reinforcement learning and evaluation, except that room dimensions are omitted when scene bounds are unavailable.Default steps are about 1 metre forward or backward, 0.6 metres for other translations, and 30 degrees for turns; distances can be adjusted.Source: §C.1 · p. 19
• The policy starts from one egocentric image and a visual question, then uses render_view_tool actions to obtain views before answering.The initial user turn may include room dimensions; each valid action returns a rendered view and camera-position information.Source: §C.1 · p. 19
• Policy outputs take either a tool-call format for movement or a final-answer format, with the answer restricted to a short phrase of 1–5 words.The prompt asks the model to reason inside <think> before either action or answer.Source: §C.1 · p. 19

C.2 TRAJECTORY-SUPERVISED TEACHER

• The trajectory teacher reuses the policy prompts but receives more verbose observations, including scene coverage and bounds, to audit coverage while creating trajectories.The teacher observation also reports camera and look-at positions.Source: §C.2 · pp. 19–20
• The exact-match training variant deterministically replaces the teacher’s final answer with the closest normalized ScanQA ground-truth string, prioritizing exact matches and then token-F1.First-listed order breaks ties.Source: §C.2 · p. 20

C.3 ACCURACY-REWARD JUDGE

• The training judge awards a binary reward only for CORRECT without INCORRECT, requiring a coherent answer semantically equivalent to the ground truth.The prompt explicitly rejects gibberish, word salad, self-contradiction, and broken or non-English tokens.Source: §C.3 · p. 20
• The offline evaluation judge uses a shorter prompt and permits terse answer fragments, unlike the training judge’s coherence requirement.It scores stored predictions, including proprietary baselines.Source: §C.3 · p. 21