SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

1 Introduction

• Existing spatial MLLMs depend on large-scale data synthesis, architecture changes, or explicit 3D inputs, while sparse accuracy-only rewards provide limited grounding guidance.Prior scene-graph pipelines commonly use graphs for offline data curation rather than integrating them into reasoning.Source: §1 · p. 2
• SpatialThinker integrates task-relevant scene-graph construction into the reasoning chain and uses lexicographically ordered format, count, accuracy, and spatial rewards.The model reasons over objects, spatial relations, and localized coordinates toward an answer.Source: §1 · pp. 2–3
• Trained on 7K samples, SpatialThinker-7B outperforms SFT by +5.5% and conventional RL by +3.2% across fourteen benchmarks, while matching GPT-5 (−0.9% avg.).Dense rewards improve the base model by +7.7%, versus +4.4% for vanilla sparse RL; on OOD real-world VQA, SpatialThinker reaches +5.2% versus +2.7% for sparse RL and +2.9% for SFT.Source: §1 · p. 3
• SpatialThinker-30B achieves the best overall average, surpassing GPT-5 by +3.0% and Claude 4 Sonnet by +5.8%.It also reaches 93.6% on CV-Bench 3D, +3.3% over GPT-5.Source: §1 · p. 3
• STVQA-7K is a scene-graph-grounded spatial VQA dataset produced by a scalable synthesis pipeline with dual-LLM verification.The pipeline supports generation of up to 108K samples.Source: §1 · p. 3
• The dense, lexicographically gated reward guides regionally focused reasoning and is reported to improve generalization across spatial, generic VQA, and real-world benchmarks.The paper also reports gains over conventional RL and SFT baselines, open-source models, and proprietary models.Source: §1 · p. 4

2 Preliminaries

• A scene graph represents an image as a directed graph whose object nodes contain category labels and 2D boxes, and whose edges encode relationships.The relations can describe spatial or interactive connections such as left of, on, or under.Source: §2 · p. 4
• A question-focused scene subgraph Gq retains only the objects and relations relevant to query q.Source: §2 · p. 4
• An MLLM is modeled as an autoregressive policy that produces a trajectory of reasoning steps followed by a final answer.The task input comprises an image and a natural-language query, and the answer is evaluated against ground truth.Source: §2 · p. 4
• Supervised fine-tuning imitates reference reasoning traces but often struggles with out-of-distribution generalization, whereas RL optimizes trajectories using explicit rewards.The trajectory reward evaluates format adherence, object counting, answer correctness, and spatial localization.Source: §2 · p. 4

3 SpatialThinker: Scene Graph-Grounded Spatial Reasoning MLLMs

• The task requires constructing question-focused scene subgraphs and reasoning over objects, bounding boxes, and relations, with training verified against dataset ground-truth scene graphs.The formulation targets structural validity, count fidelity, answer accuracy, and spatial grounding.Source: §3 · p. 4

3.1 Multi-Objective Reward Design

• Rewards combine format, count, answer accuracy, and spatial localization, with spatial reward gated on a correct answer and format prioritized over count and accuracy.The accuracy reward is a binary exact-string match; the count reward penalizes under- and over-generation of relevant objects and relations.Source: §3.1 · pp. 4–5

3.2 Online RL Policy Optimization

• GRPO samples multiple policy trajectories, scores them with the dense spatial reward, and updates the policy with a PPO-style clipped loss and KL regularization.The KL term constrains policy divergence to support stability and generalization.Source: §3.2 · pp. 5–6

3.3 STVQA-7K: Dataset Construction

• STVQA-7K contains 7,587 spatially grounded multiple-choice pairs across nine 2D and 3D reasoning types, generated from Visual Genome scene graphs and filtered through dual-LLM validation.The pipeline retains top-rated, difficult, verified questions from an initial pool of 56,224 and is described as scalable to approximately 108K samples.Source: §3.3 · p. 6

3.4 Training Details

• Training uses Qwen-based 3B, 7B, and 30B backbones with RL on STVQA-7K and no prior SFT; inputs range from 512 × 512 to 2048 × 2048 pixels.The reported setup uses 75 training steps for the smaller models and updates all model parameters, including the vision encoder.Source: §3.4 · pp. 6–7

4 Experiments

• The evaluation spans eight spatial and six real-world VQA benchmarks, uses zero-shot greedy decoding with accuracy as the primary metric, and compares proprietary, generalist, and spatially tuned models.The spatial suite includes 2D and 3D reasoning; the real-world suite includes embodied and general VQA tasks.Source: §4 · pp. 8–9

4.1 Results

• SpatialThinker-30B tops MMVP (79.7%), SpatialReasonerEval (92.6%), and SpatialBench (69.5%), surpassing GPT-5 and Claude 4 Sonnet on each.SpatialThinker-7B is the best small model on MMVP at 78.0%.Source: Table 2 · p. 9, §4.1 · p. 10
• On VStarBench, SpatialThinker-7B scores 81.7% (+15.7% over GPT-4o), while 30B leads at 85.9%; 30B improves +25.0% over base on RoboSpatial-Home.SpatialThinker-30B also leads on MM-Star and HallusionBench.Source: Table 3 · p. 10, §4.1 · p. 10
• 70.5% average accuracy across 14 benchmarks for SpatialThinker-7B exceeds SFT by +5.5% and sparse GRPO by +3.2%; SpatialThinker-30B reaches 74.5%, surpassing GPT-5 by +3.0%.The 7B model essentially matches GPT-5 (−0.9% avg.); 30B surpasses Claude 4 Sonnet by +5.8%.Source: Table 4 · p. 10, §4.1 · p. 11
• 87.9% accuracy follows pass@2 filtering; localized RoI rewards and correctness-gated spatial feedback recover and exceed the 76.3% baseline after naive spatial rewards trigger box overgeneration (23.7%).Adding count reward mitigates overgeneration, with a +38% relative gain.Source: Table 5 · p. 11, §4.1 · p. 11
• On held-out benchmarks, SpatialThinker-7B gains +5.3% on real-world tasks versus sparse GRPO’s +1.9%, while dense-reward models gain +7.6% at 3B and +5.3% at 7B.Sparse GRPO’s spatial gains are +4.4% for 3B and +5.9% for 7B, with smaller real-world gains.Source: Table 6 · p. 11, §4.1 · p. 11

3D Spatial Reasoning in MLLMs

• Spatial MLLMs use large-scale 3D-augmented data, specialized inputs, or architectural changes, but may remain data-intensive or fall short on structured relational modeling.Examples include SpatialVLM, SpatialPIN, SpatialBot, SpatialRGPT, MM-Spatial, SpatialLLM, and SpaRE.Source: §5 · p. 12

Structured Visual Grounding in MLLMs

• Scene graphs structure objects and relations, while region-aware MLLMs align text with image regions; SpatialThinker grounds reasoning in question-focused scene subgraphs.Prior scene-graph methods include LLM4SGG and GPT4SGG (Kim et al., 2024; Chen et al., 2023).Source: §5 · p. 12

Multimodal Reinforcement Learning

• Spatial RL methods often use sparse accuracy or coarse-location rewards; SpatialThinker instead rewards subgraph construction, localization, relational grounding, counting, and final correctness.The related work characterizes sparse rewards as offering limited support for fine-grained spatial reasoning.Source: §5 · p. 12

6 Conclusion

• With 7K samples, SpatialThinker-7B matches GPT-5 and beats GPT-4o on average, while SpatialThinker-30B surpasses GPT-5 and Claude 4 Sonnet.The models also surpass proprietary and open-source MLLMs on spatial, real-world, and generic VQA benchmarks.Source: §6 · p. 12
• Dense spatial rewards nearly double standard GRPO gains, and the authors conclude that 2D grounding with 3D-linked relational supervision is sufficient to induce robust 3D spatial priors.The conclusion frames this result as evidence for structured spatial grounding with reward-aligned reasoning.Source: §6 · p. 12
• The method relies on explicit scene graphs; future work may explore latent spatial reasoning, spatio-temporal and real-world tasks, and unified multi-objective policies.The conclusion names web navigation as one possible real-world task.Source: §6 · p. 12

A STVQA-7K: Dataset Construction

• STVQA-7K is a spatial VQA dataset of 7,587 samples synthetically constructed using Visual Genome’s human-annotated scene graphs; the pipeline can scale to 108K samples, Visual Genome’s maximum.Source: A STVQA-7K: Dataset Construction · p. 20
• The dataset extends VG150’s 50 predicates with 34 more and retains absolute pixel coordinates to support broader spatial coverage and CIoU-based reward supervision.The added relations cover positional, distance, comparative-size, orientation, and containment categories.Source: A STVQA-7K: Dataset Construction · p. 20
• Claude Sonnet 4 generates multiple-choice questions across nine spatial categories, including depth, distance, viewpoint orientation, reach, counting, and existence.Questions can involve partially visible or occluded objects.Source: A STVQA-7K: Dataset Construction · p. 20, Figure 6 · p. 21
• Dual-LLM validation checks Claude Sonnet 4 labels with GPT-4o pass@2 agreement; filtering retains 6,895 training and 692 validation samples from 56,224 generated questions.The final set allocates half its samples to relations and half across the other eight categories.Source: A STVQA-7K: Dataset Construction · p. 22
• Question-aligned scene subgraphs retain objects named in the question and relations connecting retained objects when their predicates match the question context.This targets the relevant region of interest so training encourages focus on question-relevant entities and relations.Source: A STVQA-7K: Dataset Construction · p. 22

B Experimental Setup Details

• The experiments assess dense rewards and data-efficient training across multiple spatial reasoning benchmarks.The study also evaluates real-world and generic VQA performance.Source: B Experimental Setup Details · p. 22

B.1 Implementation Details

• The 3B and 7B models use Qwen2.5-VL backbones and train with GRPO on STVQA-7K without prior SFT; the 30B model uses Qwen3-VL-30B with LoRA adapters.The 30B setup uses the same dense-reward GRPO objective and dataset.Source: §B.1 · pp. 22–23

B.2 Experimental Setup

• The evaluation spans eight spatial benchmarks and six general-purpose VQA and real-world datasets, comparing proprietary, generalist open-source, and spatially specialized models.Accuracy is measured zero-shot with greedy decoding; SpatialRGPT receives depth inputs while other models use RGB images alone.Source: §B.2 · pp. 23–24

B.3 SpatialThinker Prompt Format

• The prompt separates observation, scene-graph construction, reasoning, and answering into tagged stages, with format rewards enforcing tag order and valid structured output.The scene section contains a JSON subgraph, and image dimensions help constrain predicted box coordinates evaluated using IoU-based rewards.Source: §B.3 · p. 24

B.4 Details on SFT Training

• SFT baselines use the same STVQA-7K data and 3B and 7B base models, training directly on question-answer pairs without reasoning traces.The 30B SFT baseline was not trained because of compute limits; the authors cite the additional processing, annotation, and API costs of ground-truth traces.Source: §B.4 · pp. 24–25

B.5 Details on RL Training

• RL is applied directly to base models using GRPO and dense spatial rewards; the 30B model uses the same objective, while sparse-reward GRPO baselines use only accuracy and format rewards.The 3B and 7B models train for 75 steps; the 30B sparse-reward baseline is omitted because of compute bottlenecks.Source: §B.5 · pp. 25–26

B.5.1 SpatialThinker RL Training Curves

• Format, accuracy, count, and spatial rewards improve during training, while response length first declines and then rises as reasoning traces become more deliberate and structured.The authors interpret the length pattern as emergent reasoning strategies; reward trends indicate learning of structured output, answer correctness, question-relevant focus, and localization.Source: §B.5.1 · p. 26, Figure 7 · p. 26

C Reward Design Process

• The reward-design process responds to reward hacking observed when spatial localization rewards lacked constraints on generation quantity.The section examines successive reward components, count-reward weights, and the rationale for individual design choices.Source: C Reward Design Process · p. 27

C.1 Reward Design Ablation

• 87.9% validation accuracy followed dataset filtering with pass@2 verification, after question-relevant RoIs and lexicographic gating restored performance to 76.3% from a 23.7% collapse under unconstrained spatial rewards.Naive spatial rewards reduced accuracy from 74.9% to 23.7%; adding the count reward raised it to 61.7%.Source: §C.1 · p. 27

C.2 Count Reward Weight Ablation

• The count reward penalizes both under- and over-generation of question-relevant objects and relations, and the 0.7/0.3 split gives more weight to the cleaner object-count signal.The authors describe object counts as the first constraint on over-generation, with relation counts refining the remaining relation set.Source: §C.2 · p. 27
• 76.02% is the best STVQA-7Kval accuracy among the tested count-reward splits, achieved by the 0.7/0.3 object/relation weighting.The 0.8/0.2 split scores 75.24%, while 1.0/0.0 and 0.5/0.5 each score 74.85%.Source: §C.2 · p. 27, Table 7 · p. 28

C.3 Reward Design Rationale

• Unconstrained CIoU rewards can be gamed with many boxes, so count penalties limit generation and question-filtered scene graphs avoid exhaustive descriptions of irrelevant objects.CIoU supplies a denser localization signal than IoU for non-overlapping boxes; the count and accuracy terms gate spatial rewards to correct answers.Source: §C.3 · pp. 28–29
• Overly prescriptive rewards can produce template-style answers and mid-training accuracy drops, so the final design preserves exploration and applies spatial rewards only to correct answers.The hierarchy is format, then count and accuracy, then spatial grounding.Source: §C.3 · p. 28

D Scene-Graph Error Propagation Impact

• Object identification has mean Jaccard 0.957 and is uninformative as an answer-accuracy predictor, while only 232/692 samples meet the triple-overlap correctness threshold.The authors therefore use Jtrip≥0.5, rather than object overlap, to define correct scene-graph generation.Source: D Scene-Graph Error Propagation Impact · p. 29
• When the predicted scene graph is correct, answer accuracy is 95.26%, versus 79.35% when it is incorrect, a +15.91 pp gap on STVQA-7Kval.Triple-level correctness is defined as Jtrip≥0.5; the relationship also appears across stricter and looser cuts.Source: D Scene-Graph Error Propagation Impact · p. 29
• The authors characterize scene graphs as useful but non-load-bearing scaffolding: answers remain correct about 80% of the time when the intermediate graph is incorrect.The analysis uses 692 held-out STVQA-7K examples and compares normalized predicted object and relation sets with ground truth.Source: D Scene-Graph Error Propagation Impact · p. 29

E Ablation on Divergence Constraints

• 73.7% CV-Bench average is the best result for the KL-regularized variant, which also achieves the strongest 3D reasoning performance.Removing KL lowers performance, particularly on 3D tasks; chi-squared regularization underperforms both alternatives on several subtasks, especially depth and distance reasoning.Source: E Ablation on Divergence Constraints · p. 30
• The authors interpret the results as evidence that a modest KL penalty helps stabilize policy updates and prevent reward overoptimization in this multimodal spatial-reasoning setup.The ablation compares a no-penalty variant, chi-squared divergence with coefficient 0.01, and KL divergence with coefficient 0.01.Source: E Ablation on Divergence Constraints · p. 30

F Additional Results: Abstract Reasoning

• SpatialThinker-7B achieves the highest open-source performance, scoring 37.7% on Lego Puzzles and 52.6% on BLINK Multi-View, closely approaching GPT-4o and surpassing Claude 3.5 Sonnet on BLINK.Both benchmarks test reasoning beyond the training distribution.Source: F Additional Results: Abstract Reasoning · p. 31
• Vanilla GRPO performs competitively on BLINK Multi-View but underperforms on Lego Puzzles, while dense spatial rewards provide complementary signals for compositional reasoning.Source: F Additional Results: Abstract Reasoning · p. 31

G Detailed Results: CV-Bench

• Table 11 breaks down CV-Bench results across Count, Relation, Depth, and Distance subtasks.Source: Table 11 · p. 32

H Detailed Results: 3DSRBench

• Table 12 breaks down 3DSRBench results across Height, Location, Orientation, and Multi-Object tasks.Source: Table 12 · p. 33

I Detailed Results: STVQA-7Kval

• SpatialThinker models lead across nearly all held-out STVQA-7Kval categories, with the largest gains in relational, size, and existence reasoning.Source: Table 13 · p. 34

• SpatialThinker more accurately identifies 3D relations that GPT-5-0807 often confuses, while distinguishing objects more finely.Source: Figure 8 · p. 35