Reasoning in Space via Grounding in the World

• Existing 3D LLMs face difficulty aligning geometric and semantic cues efficiently, while available VQA datasets lack grounding annotations and reasoning steps.The introduction identifies grounding as a prerequisite for reasoning about spatial relationships.Source: §1 · pp. 1–2
• GS-Reasoner uses a unified image patch-based hybrid representation that aligns geometric features with semantic features and 3D position.The representation is designed to support object localization without external modules.Source: §1 · p. 2
• The GCoT dataset pairs 3D bounding-box annotations for referenced objects with step-by-step reasoning paths that include grounding.Its task structure encourages models to identify relevant objects before reasoning about their spatial relationships.Source: §1 · pp. 2–3
• Extensive experiments report impressive GS-Reasoner results on both 3D visual grounding and spatial reasoning tasks.Source: §1 · p. 3

3D Large Language Models for 3D Understanding

• 3D LLM approaches use point-cloud, voxel-based, object-centric, or vision-foundation-model features, but joint semantic and geometric understanding remains difficult.Source: §2 · p. 3
• Object-centric methods are described as lacking global context, whereas positional-feature approaches retain visual-LLM generalization but still leave joint semantic-geometric capture unresolved.Source: §2 · p. 3

Video-Language Models for Spatial Reasoning

• Spatial-reasoning models add geometric features and large-scale QA pairs, while constrained answer formats may limit use of encoded 3D information.Source: §2 · p. 3

3.1 OVERVIEW

• The framework extracts semantic features from RGB images and geometric and positional features from depth-derived point maps, then fuses these signals into one scene representation.Depth maps are assumed available or obtainable with visual geometry methods.Source: §3.1 · p. 3
• The unified semantic-geometric representation and text query feed a video LLM that autoregressively grounds relevant objects and performs spatial reasoning to produce an answer.The model’s output can list relevant objects’ 3D bounding boxes before reasoning and giving a concise final answer.Source: §3.1 · pp. 3–4

3.2 SEMANTIC-GEOMETRIC HYBRID 3D SCENE REPRESENTATION

• The geometric encoder processes the point cloud as a whole after uniformly sampling K points per image-aligned patch, producing spatially aligned geometric feature maps.The encoder uses PTv3, with progressive unpooling mapping features back to the original point-map resolution.Source: §3.2 · pp. 4–5
• Patch-level dual-path fusion addresses semantic-geometric and position-geometric misalignment that can make naive pooling harm 3D bounding-box accuracy.The method uses semantic-aligned geometric features and geometric features aligned to a sampled 3D point.Source: §3.2 · p. 5

4 GCOT: GROUNDED CHAIN-OF-THOUGHT DATASET

• Earlier spatial QA datasets often use single-choice or short numerical answers, whereas GCoT adds grounding and geometry reasoning as intermediate steps.The paper motivates this supervision as richer and more interpretable.Source: §4 · pp. 5–6
• GCoT contains 156k spatial QA pairs, 79% with CoT annotations; CoT is omitted for Appearance Order, Object Counting, and Room Size Estimation.Those tasks are described as not requiring complex spatial reasoning.Source: §4 · p. 6
• GPT-4o generates coherent CoT paths from QA pairs, object bounding boxes, and bird’s-eye scene views, with the paths leading to final answers.The QA pairs are first generated without CoT while preserving object bounding-box information.Source: §4 · p. 6, Figure 3 · p. 6

5.1 IMPLEMENTATION DETAILS

• GS-Reasoner uses SigLIP for semantic encoding and Sonata for geometric encoding, and is pretrained on grounding data before finetuning on GCoT, remaining grounding data, and other 3D tasks.The model is built on LLaVA-Video 7B.Source: §5.1 · p. 6

5.2 EVALUATION ON 3D VISUAL GROUNDING

• GS-Reasoner achieves state-of-the-art F1@25 on Multi3DRef among 3D LLM-based methods, matching external-module methods on ScanRefer Acc@50 and Multi3DRef F1@50.It still trails methods using mesh-point-cloud proposals on those latter metrics; the authors note that these use more complete, less noisy inputs.Source: Table 1 · p. 7, §5.2 · p. 8

5.3 EVALUATION ON SPATIAL REASONING

• With more accurate depth, VSI-Bench average score exceeds 70, nearly 10 points above the previous state of the art.GS-Reasoner also achieves state-of-the-art results on Appearance Order and performs particularly well on Relative Direction and Absolute Distance.Source: Table 2 · p. 7, §5.3 · p. 8

5.4 EVALUATION ON GENERAL 3D TASKS

• GS-Reasoner sets a new state of the art on Scan2Cap across all reported metrics and achieves comparable performance on ScanQA and SQA3D.The authors attribute the Scan2Cap gains to explicit coordinate prediction for 3D grounding.Source: Table 3 · p. 8, §5.4 · p. 8

5.5 ANALYSIS AND ABLATION STUDIES

• Trained only on ScanNet and evaluated without finetuning, GS-Reasoner performs comparably to state-of-the-art expert models on ARKitScenes and shows strong novel-scene spatial reasoning.The zero-shot grounding evaluation uses ScanNet++ and ARKitScenes validation splits.Source: Table 4 · p. 9, §5.5 · p. 9
• Adding geometric features and Dual-Path Pooling yields substantial Acc@25 and Acc@50 gains after data augmentation alone remains insufficient for precise localization.The ablation evaluates autoregressive bounding-box prediction on ScanRefer.Source: §5.5 · p. 9, Table 5 · p. 9
• Including grounding in GCoT reasoning substantially improves performance across its five evaluated tasks, especially absolute distance, relative direction, and route planning.The ablation compares answers with grounding-integrated chain-of-thought against direct answer prediction.Source: §5.5 · p. 9, Table 6 · p. 9

6 CONCLUSION

• GS-Reasoner performs grounding without external modules as an intermediate step in spatial reasoning, and the GCoT dataset further strengthens its ability to handle both tasks.The framework is built on a hybrid semantic–geometric 3D scene representation.Source: §6 · p. 10

ETHICS STATEMENT

• The LLM-generated dataset contains no personally identifiable information or sensitive human data, but may inherit LLM biases and could be misused.The authors intend it solely for academic research and encourage consideration of downstream ethical implications.Source: ETHICS STATEMENT · p. 10

A.1 INSTRUCTION QA GENERATION

• QA pairs draw on ScanNet, ScanNet++, and ARKitScenes, with per-scene preprocessing and metadata used to generate spatial and temporal questions.Scene metadata includes object AABBs and global statistics; frame metadata records each object's first appearance above a mask-area threshold.Source: §A.1 · p. 15
• Questions cover object and room size, distances, relative directions, appearance order, and route planning based on simulated trajectories and nearby objects.The construction also includes object-count questions and computes absolute distances for selected object pairs.Source: §A.1 · pp. 15–16

A.2 CHAIN-OF-THOUGHT GENERATION

• GPT-4o generates CoT paths using each QA pair and a bird’s-eye-view scene map annotated with relevant object boxes; simple tasks are excluded from CoT generation.Room Size, Object Count, and Appearance Order are excluded because they can be answered without step-by-step grounding-based reasoning.Source: §A.2 · p. 16

A.3 DATA FORMAT

• A question-analysis preprompt teaches the model to decide whether grounding is needed before reasoning, and the final format supports autonomous step-by-step answers.The examples distinguish grounding-based reasoning from a room-size answer that relies on overall spatial information without identifying object boxes.Source: §A.3 · pp. 16–17

B.1 TRAINING DETAILS

• GS-Reasoner is pretrained on 3D grounding datasets, then fine-tuned on GCoT, remaining grounding data, and other 3D tasks.Training uses next-token cross-entropy; geometric and positional cues are decoupled through rotations, scaling, and translations to encourage their use in grounding.Source: §B.1 · p. 17

B.2 INFERENCE DETAILS

• At inference, VGGT-SLAM reconstructs depth maps and relative camera parameters, while MoGe-2 estimates absolute-scale depth; a global scale aligns their point maps across views.A per-scene axis-alignment transform is then computed from estimated camera poses and reconstructed point clouds.Source: §B.2 · p. 17

C ADDITIONAL RELATED WORK

• Point-cloud representation methods commonly group points by spatial distribution, extract features per group, and aggregate them into a global representation.These sparse features can be upsampled to the original resolution for tasks such as semantic segmentation.Source: C ADDITIONAL RELATED WORK · p. 18

D.1 ANALYSIS ON GENERAL 3D TASKS

• GS-Reasoner does not achieve leading results on ScanQA and SQA3D.The authors attribute this possibly to ambiguous target references and answer-distribution bias, which may encourage reliance on textual patterns over 3D tokens.Source: §D.1 · p. 18
• Ambiguous target references and answer-distribution bias may encourage overfitting to textual patterns instead of use of 3D tokens.The authors identify these as possible reasons for the ScanQA and SQA3D results.Source: §D.1 · p. 18
• The authors leave reconstruction constraints in model outputs, as used by ROSS3D, as a possible way to encourage 3D-token use.They note prior studies found that LLM finetuning without 3D input can yield results comparable to state of the art on ScanQA and SQA3D.Source: §D.1 · p. 18

D.2 MORE QUALITATIVE RESULTS

• Figures 4–6 present qualitative GS-Reasoner results on VSI-Bench.The text introduces the qualitative results without specifying their outcomes.Source: §D.2 · p. 18, Figure 4 · p. 18, Figure 5 · p. 19

E FUTURE WORK

• The authors propose jointly fine-tuning with GCoT and action data, and using GS-Reasoner as an embodied brain for planning and task decomposition.They frame these as future directions for leveraging spatial reasoning in robotic tasks.Source: E FUTURE WORK · pp. 19–20

F THE USE OF LARGE LANGUAGE MODELS (LLMS)

• LLMs generate GCoT spatial-reasoning paths, capturing intermediate steps beyond question-answer pairs.Source: F THE USE OF LARGE LANGUAGE MODELS… · p. 20