PaperlayerView exact revision on arXiv

Source-linked AI summary

SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Models

Rafi Ibn Sultan, Xiangyu Zhou, Md. Sajid Alam Chowdhury, Chengyin Li, Prashant Khanduri, Marco Brocanelli, Dongxiao Zhu

arXiv:2609.38716v1cs.CVcs.AI

TL;DR

LVLMs can identify objects yet remain unreliable at reasoning about spatial relationships, and generated-grounding methods do not explicitly account for grounding confidence. SpatialCORE uses confidence-weighted grounding rewards, gated by answer correctness, during post-training. It reports state-of-the-art results among open-source and specialized models and effective zero-shot generalization.

  • Problem

    LVLMs often struggle to reason about spatial arrangements despite accurate object identification, and generated-grounding objectives do not explicitly capture confidence in that grounding.

    Source: §1 · pp. 1–2

  • Method

    SpatialCORE post-trains LVLMs with a self-regulating spatial reward that weights box matching quality by coordinate uncertainty and scales the reward by final-answer correctness.

    Source: §1 · p. 2

  • Results

    SpatialCORE reports state-of-the-art results among open-source and specialized spatial reasoning models, effective zero-shot generalization, and improved alignment between confidence and localization quality.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The findings advance confidence-aware spatial reasoning by encouraging models to reason from grounding they produce with confidence.

    Source: §5 · p. 10

    SpatialCORE changes the BBox-based post-training objective but leaves the backbone and input representation unchanged, leaving 3D and non-boxable concepts for future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Large Vision-Language Models (LVLMs) have made remarkable progress across visual perception tasks, yet spatial reasoning remains a persistent weakness, especially for questions that require reasoning over visual space. Recent spatial-reasoning methods incorporate generated grounding, where models predict bounding boxes, masks, or other localization outputs for task-relevant objects as part of their reasoning trace. However, these approaches typically optimize final-answer correctness alone, allowing correct answers to be rewarded even when the model does not reason from confidently localized task-relevant objects. We introduce SpatialCORE (Spatially COnfident REasoning), a post-training framework that turns the model's own confidence in generated grounding into a learning signal for spatial reasoning. Its central idea is to reinforce grounding that is both accurate and confident, encouraging the model to reason from confidently localized task-relevant objects. SpatialCORE realizes this through a self-regulating spatial reward that weights each predicted bounding box's matching quality by its coordinate-token confidence. An answer gate further ties grounding optimization to final-answer correctness. SpatialCORE achieves state-of-the-art results among open-source and specialized spatial reasoning models across diverse benchmarks, and transfers effectively in zero-shot settings to unseen data distributions. The source code is available at https://github.com/rafiibnsultan/SpatialCORE.

1 INTRODUCTION

LVLM spatial reasoning remains unreliable even when objects are recognized, while existing grounding approaches do not account for confidence in generated localization. SpatialCORE makes grounding confidence part of post-training and reports strong benchmark results and zero-shot generalization.

1 INTRODUCTION

  • Spatial reasoning requires understanding relationships among objects, but LVLMs can identify objects without reliably understanding their spatial arrangements, limiting real-world applications.Source: §1 · p. 1
  • Existing approaches use external spatial guidance or supervised grounding, while generated-grounding methods still lack an explicit measure of grounding confidence.External guidance depends on supplied anchors, and standalone localization supervision can remain separate from the reasoning trace.Source: §1 · pp. 1–2
  • Answer-only rewards can reinforce a correct trajectory even when its predicted boxes poorly localize task-relevant objects, motivating confidence-aware spatial reasoning.Source: §1 · p. 2
  • SpatialCORE weights geometric overlap with reference boxes by coordinate-token confidence and uses an answer gate to scale the spatial reward by final-answer correctness.Source: §1 · p. 2
  • SpatialCORE reports state-of-the-art results among open-source and specialized models, effective zero-shot generalization, and ablations supporting confidence weighting and improved confidence-localization alignment.Source: §1 · p. 2

2 RELATED WORKS

Prior spatial-reasoning work supplies anchors, changes visual representations, structures inference, or trains grounding. SpatialCORE distinguishes its approach by incorporating grounding confidence into training.

Externally Guided Spatial Grounding

  • Externally guided methods provide spatial anchors, but their dependence on inference-time guidance limits open-ended use when anchors are absent or ambiguous.Source: §2 · p. 3

Geometry-Enhanced Visual Understanding

  • Geometry-enhanced methods add depth, point-cloud, mask, or multi-view cues, but do not specify how grounding confidence should affect post-training rewards.Source: §2 · p. 3

Inference-Time Reasoning Scaffolds

  • Inference-time scaffolds structure responses without post-training, but their gains can depend on task formats, prompts, scaffolds, or decoding procedures.Source: §2 · p. 3

Training-Based Spatial Reasoning

  • Training-based methods reward generated grounding or correct localization, whereas SpatialCORE adds the model’s confidence in producing grounding as a learning signal.Source: §2 · p. 3

3 METHOD

SpatialCORE post-trains LVLMs to reason from confidently localized task-relevant objects, using confidence-weighted grounding rewards within a GRPO framework. It combines spatial, format, and answer rewards, with an answer gate tying spatial reward to answer correctness.

3 METHOD

  • SpatialCORE uses GRPO to optimize sampled reasoning trajectories with a self-regulating spatial reward, format reward, and answer reward.Group-relative advantages from trajectory rewards are used for the policy update.Source: §3 · p. 3, §3.1 · p. 4

3.1 PROBLEM FORMULATION

  • Given an image and spatial query, the policy generates a reasoning trajectory with BBoxes for task-relevant objects, followed by a final answer.A rollout samples a group of trajectories and assigns each a reward and group-relative advantage.Source: §3.1 · pp. 3–4

3.2 SELF-REGULATING SPATIAL REWARD

  • The spatial reward evaluates predicted BBoxes against REC-generated pseudo-GT boxes, using geometric overlap, label similarity, and pseudo-GT validity.Hungarian matching produces a one-to-one assignment; spatial reward also incorporates validity-weighted coverage of pseudo-GT boxes.Source: §3.2 · p. 4, §3.2.1 · pp. 5–6

3.2.1 CONFIDENCE-AWARE SPATIAL REWARD

  • Coordinate-token uncertainty estimates each predicted BBox’s confidence, with lower uncertainty indicating higher confidence.The uncertainty is based on normalized entropy averaged over coordinate digits and then over coordinates; full-vocabulary entropy also captures probability assigned to non-digit tokens.Source: §3.2.1 · p. 5
  • A confidence weight increases the contribution of high-certainty matched BBoxes while remaining positive even when uncertainty is high.The confidence-weighted matching scores form a matching-quality measure, and one-to-one assignment prevents ambiguous or duplicate matches from inflating it.Source: §3.2.1 · p. 5

3.2.2 FORMAT AND ANSWER REWARDS

  • The format reward assesses valid reasoning structure, a single final answer token, and valid BBox formatting, penalizing malformed or misplaced outputs.The answer reward is one for a correct final answer and zero otherwise.Source: §3.2.2 · p. 6

3.3 ADAPTIVE REWARD COMPOSITION

  • An answer gate applies the full spatial reward to correct-answer trajectories and reduces positive spatial rewards by γ for incorrect-answer trajectories.Incorrect-answer trajectories can therefore retain partial credit for useful generated grounding.Source: §3.3 · p. 6

3.4 POLICY OPTIMIZATION

  • The policy update uses a GRPO-style clipped objective with KL regularization against a reference policy and group-relative advantages.The objective uses the trajectory likelihood ratio between the current and rollout policies, along with a clipping threshold and KL penalty coefficient.Source: §3.4 · p. 6

4 EXPERIMENTS

Experiments evaluate SpatialCORE on OmniSpatial and zero-shot SpatiaLab against backbone, open-source, specialized, and proprietary model baselines. The results show strong benchmark performance and a closer alignment between grounding confidence and localization quality.

4.1 IMPLEMENTATION DETAILS

  • The study uses Qwen3-VL-8B-Thinking and Qwen3-VL-4B-Thinking backbones, post-trains both with LoRA for three epochs on OmniSpatial training data, and uses manually audited offline pseudo-GT boxes.Training uses two H100 GPUs, effective batch size 32, and four rollout generations.Source: §4.1 · pp. 6–7

4.3 RESULTS

  • SpatialCORE-8B achieves the highest weighted-average accuracy among open-source and specialized models on OmniSpatial and leads them on zero-shot SpatiaLab, gaining 4.56% over its OmniSpatial backbone and 3.2% on SpatiaLab.The tables compare category-weighted accuracy; proprietary models are included as reference points.Source: Table 1 · p. 7, Table 2 · p. 7, §4.3 · pp. 8–9
  • Qualitative examples show SpatialCORE producing more accurate BBoxes and correct answers where the baseline mislocalizes task-relevant objects and gives incorrect or poorly grounded answers.The examples cover OmniSpatial and SpatiaLab; overlapping boxes can obscure relative-position reasoning, while mislocalized objects can corrupt reachability judgments.Source: Figure 3 · p. 8, §4.3 · p. 9

4.2 BASELINES

  • Evaluation compares SpatialCORE with its backbone, a standard GRPO-trained variant, open-source and specialized models, and proprietary references on held-out OmniSpatial and zero-shot SpatiaLab.OmniSpatial spans four reasoning dimensions and 50 subcategories; SpatiaLab spans six categories and 30 task types in naturalistic scenes.Source: §4.2 · p. 8

4.3 RESULTS

  • Gains are strongest in traffic analysis and localization on OmniSpatial and Relational Positioning on SpatiaLab, while allocentric, hypothetical, and Size and Scale reasoning show smaller gains.The paper attributes smaller allocentric and hypothetical gains to multi-view reasoning that single egocentric-view boxes cannot address; it identifies scene-level scale estimation as a distinct challenge.Source: §4.3 · pp. 8–9
  • On analyzed OmniSpatial samples, mean matched IoU falls from 77% in the lowest-uncertainty quartile to 36% in the highest, while entropy–IoU correlation shifts from ρ = +0.38 to ρ = −0.42.The figure compares mean matched BBox IoU across model-specific coordinate-token entropy quartiles for the baseline and SpatialCORE-8B.Source: Figure 4 · p. 8, §4.3 · p. 9
  • On those samples, answer accuracy rises from 44% to 48% as BBox coordinate uncertainty falls from 92% to 74% relative to Qwen3-VL-8B-Thinking.Figure 5 compares answer accuracy and predicted BBox coordinate-token uncertainty for the baseline and SpatialCORE-8B.Source: §4.3 · p. 9, Figure 5 · p. 9

5 CONCLUSION

SpatialCORE uses confidence in generated grounding as a learning signal, and its reported findings support confidence-aware spatial reasoning. The framework modifies the BBox-based post-training objective without changing the backbone or input representation.

5 CONCLUSION

  • SpatialCORE advances confidence-aware spatial reasoning by learning not only to produce grounding but also to reason from it with confidence.The conclusion reports benchmark gains and controlled ablations, alongside stronger alignment between confidence and localization quality.Source: §5 · p. 10
  • The framework modifies the BBox-based post-training objective while leaving the backbone and input representation unchanged.Future work could add depth, multi-view context, or geometry-enhanced encoders for 3D and non-boxable spatial concepts.Source: §5 · p. 10

A.1 HYPERPARAMETER SETTINGS

Table 4 reports the main hyperparameters used for SpatialCORE-8B post-training.

A.1 HYPERPARAMETER SETTINGS

  • Table 4 covers the GRPO training setup, optimization settings, and reward configuration for SpatialCORE-8B post-training.Source: §A.1 · p. 16

A.2 TRAINING ALGORITHM

Algorithm 1 summarizes a SpatialCORE training iteration, from sampling and trajectory generation through reward computation and policy updates.

A.2 TRAINING ALGORITHM

  • Each training step samples a batch and rolls out multiple reasoning trajectories from the LVLM policy for each image-question pair.The algorithm retrieves precomputed pseudo-GT BBoxes and parses generated grounding from each trajectory.Source: §A.2 · p. 17
  • The algorithm computes BBox matching and coordinate-token uncertainty, then uses confidence-weighted spatial, format, and answer rewards to form an adaptive reward.Source: §A.2 · p. 17
  • GRPO updates the policy with KL regularization to the reference policy after advantages are computed for the sampled trajectories.Source: §A.2 · p. 17

A.3 PSEUDO-GT SUPERVISION AND RELIABILITY

The appendix describes how pseudo-GT BBoxes are constructed and weighted, audits their reliability, and tests coverage and robustness to corrupted references.

A.3 PSEUDO-GT SUPERVISION AND RELIABILITY

  • Pseudo-GT annotations are built offline by extracting object phrases, querying Grounding DINO, and retaining the highest-scoring label-specific boxes above 0.3 confidence.Each retained box stores its label and detector score as validity, with coordinates scaled to the LVLM's [0, 1000] convention.Source: §A.3 · p. 17
  • Of 262 audited BBoxes, 251 were correct (95.8%); every audited box at Grounding DINO confidence of at least 0.70 was correct.All 11 observed errors were in the two lower confidence ranges; removing pseudo-GT validity weighting reduced accuracy from 56.33% to 53.33%.Source: Figure 6 · p. 17, §A.3 · p. 18
  • Pseudo-GT covered 71.0% of BBox-localizable samples and 98.6% of samples with phrase queries; uncovered examples often involved paths, sequences, or spatial relationships.Source: §A.3 · p. 18
  • When no valid reference exists, the spatial reward is zero; when a reference exists but the model emits no BBox, it receives a negative spatial reward.Source: §A.3 · p. 18
  • 46.70% accuracy under 40% pseudo-GT BBox corruption remained 2.80 points above the unadapted backbone, while falling 1.76 points versus original pseudo-GT training.The corruption test randomly removed, relabeled, or displaced selected boxes; 60% remained intact.Source: Table 5 · p. 18, §A.3 · p. 18

A.4 REWARD AND IMPLEMENTATION DETAILS

The reward matches generated and pseudo-ground-truth boxes using geometry and label similarity, then weights matches by grounding confidence. Implementation details cover reward shaping, initialization, prompt format, and training compute.

Reward matching

  • Pairwise BBox matching combines geometric overlap and label similarity, while pseudo-GT validity gives more influence to more reliable reference boxes.Hungarian matching selects a maximum-reward one-to-one assignment; lexical similarity distinguishes compatible phrases such as “black sedan ahead” and “black sedan.”Source: §A.4 · p. 19

Confidence-weighted spatial reward

  • The spatial reward weights BBox matching quality by coordinate-token confidence, making confidence a learning signal even among trajectories with the same correct answer.The confidence weight is based on predicted BBox coordinate uncertainty.Source: §3.2.1 · p. 5

Additional reward shaping

  • Two shaping terms stabilize training: a 0.05 BBox-attempt bonus for relevant valid predictions and a 0.3 penalty for unmatched predicted boxes.The bonus applies when pseudo-GT boxes are available; the penalty limits extra grounding beyond task-relevant objects.Source: §A.4 · p. 19

SFT cold start

  • The SFT cold start uses 1,000 OmniSpatial samples to teach the grounded output format before GRPO optimizes the spatial reward.It trains a LoRA adapter with completion-only cross-entropy, freezing the visual encoder and updating language-side parameters.Source: §A.4 · p. 19
  • Without the SFT cold start, grounding reward ended at 0.000 and the model stopped generating BBoxes; with it, BBoxes appeared in 95–100% of training responses.The cold-start model maintained a positive grounding reward of 0.15–0.33; general reasoning and final-answer format remained stable without SFT.Source: §A.4 · p. 19

System prompt and compute

  • The fixed system prompt standardizes trajectories by requesting relevant BBoxes before reasoning and one final answer letter after the reasoning segment.The prompt closes the prefilled <think> segment with </think>. The final run used two H100 GPUs for approximately 31 hours, or 62 GPU-hours.Source: §A.4 · p. 20

A.5 BENCHMARK DETAILS

The appendix describes the benchmark splits and accuracy protocol, then explains how answer-only rewards can be indifferent to grounding confidence. SpatialCORE’s confidence-weighted reward and answer gate address that indifference, including for incorrect-answer trajectories.

A.5 BENCHMARK DETAILS

  • OmniSpatial has over 8.4K question–answer pairs across four dimensions and 50 subcategories; evaluation uses its 6,902-sample training split for post-training and 1,533-sample held-out test split.SpatiaLab contains 1,400 pairs from realistic, unconstrained scenes and is used only for zero-shot multiple-choice evaluation.Source: §A.5 · p. 20
  • Both benchmarks report accuracy by category and overall, with overall accuracy computed over the full set or equivalently as a sample-weighted category average.A prediction is correct when its selected option matches the ground-truth answer.Source: §A.5 · p. 20

B APPENDIX: RATIONALE FOR CONFIDENCE-GUIDED GROUNDING

  • Answer-only rewards give trajectories with the same final answer the same signal, regardless of whether their generated BBoxes are confidently localized.The confidence-weighted spatial reward incorporates coordinate-token uncertainty to reduce this indifference.Source: B APPENDIX: RATIONALE FOR… · p. 20

B.1 GRADIENT INDIFFERENCE UNDER ANSWER-ONLY SUPERVISION

  • Under answer-only supervision, equally correct trajectories with low- and high-uncertainty BBoxes receive the same advantage, so uncertain grounding may still be reinforced.The resulting gradient signal does not distinguish the two trajectories’ grounding uncertainty.Source: §B.1 · p. 21

B.2 CONFIDENCE-WEIGHTED REWARDS BREAK THIS INDIFFERENCE

  • When matched BBox quality is comparable, lower coordinate-token uncertainty yields a larger confidence weight and spatial reward, favoring that trajectory despite equal answers.This produces an advantage gap between the confident and uncertain trajectories.Source: §B.2 · p. 21

B.3 ANSWER GATE PRESERVES GENERATED GROUNDING SIGNAL ON INCORRECT TRAJECTORIES

  • The answer gate preserves reduced spatial reward for incorrect-answer trajectories, allowing useful generated grounding to receive partial credit before answers become consistently correct.The retained portion is governed by γ, where 0 < γ < 1.Source: §B.3 · p. 21

C APPENDIX: ADDITIONAL QUALITATIVE RESULTS

The qualitative examples show how SpatialCORE structures grounded reasoning and uses localized task-relevant objects as evidence for a spatial decision.

System prompt

  • Figure 7 presents the system prompt used to standardize generated grounding trajectories during training.The prompt figure continues across two pages.Source: Figure 7 · pp. 22–23

Qualitative trajectory

  • Figure 8’s example grounds the thermostat, scissors, lamp, and cup before reasoning about their positions relative to a chair, selecting the thermostat as hardest to reach.The example identifies the correct final answer; boxes are overlaid for visualization.Source: C APPENDIX: ADDITIONAL QUALITATIVE… · p. 21, Figure 8 · p. 24
Loading 2609.38716v1…