Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning

1 Introduction

• High 3D-annotation costs restrict training coverage, leaving spatial knowledge implicit and predictions physically inconsistent on novel scenes.Existing training approaches commonly fine-tune on synthesized spatial VQA data, while test-time predictions do not jointly constrain inter-dependent geometric attributes.Source: §1 · p. 1
• Confidence-based test-time objectives can over-collapse representations and disrupt fine-grained geometric distinctions required for quantitative spatial reasoning.The paper contrasts classification’s benefit from compact representations with continuous spatial prediction’s need to preserve geometric distinctions.Source: §1 · p. 2
• TTL-SR augments queries to probe inter-dependent attributes, aligns token-level predictions with geometric constraints, and uses the resulting objective for test-time parameter updates.The paper describes this geometry-consistent objective as enabling reliable updates and correcting physically implausible predictions.Source: §1 · p. 2
• TTL-SR reports consistent accuracy, robustness, and generalization improvements across spatial reasoning benchmarks for specialized and general-purpose VLMs.The framework is model-agnostic and plug-and-play, and its test-time objective aligns discrete language-model predictions with continuous geometric relations.Source: §1 · p. 2

2.1 Spatial Reasoning in VLMs

• Spatial reasoning methods add training supervision, external geometric priors, or region-aware grounding, while benchmarks span diverse tasks and evaluation protocols.The cited approaches include curated spatial VQA fine-tuning, preference optimization, 3D foundation models, and localization-oriented architectures.Source: §2.1 · p. 2
• High 3D-supervision costs narrow training scenarios, leaving spatial knowledge implicit in fixed parameters and unavailable for explicit activation at test time.Source: §2.1 · p. 2

2.2 Test-Time Learning for Reasoning

• Test-time learning for reasoning includes self-feedback, reinforcement learning, perplexity minimization, unsupervised incentivization, and multimodal post-training.Source: §2.2 · p. 2
• Existing test-time reasoning methods rely mainly on self-certainty objectives and lack mechanisms for structured numerical or physical constraints.The passage says they can produce logically consistent but physically impossible results, motivating a paradigm tailored to quantitative spatial reasoning.Source: §2.2 · p. 2

3.1.1 Test-time Learning for Spatial Reasoning.

• A VLM maps an image and spatial query to a predicted continuous spatial attribute.Source: §3.1.1 · p. 2
• Figure 2 organizes TTL-SR into query augmentation, adaptive geometric consistency modeling, and test-time optimization.These stages generate coupled queries, filter estimates into token-level pseudo-labels, and iteratively update model parameters with a multi-objective loss.Source: Figure 2 · p. 3
• The adaptation setting uses an unlabeled test dataset drawn from a target domain.The stated aim is unsupervised optimization for that target distribution.Source: §3.1.1 · p. 3
• Only a subset of model parameters is updated at test time, and the objective must activate spatial knowledge without ground-truth labels.Source: §3.1.1 · p. 3

3.1.2 Motivation for Geometry Consistency Optimization.

• Naive entropy minimization gives only marginal gains or can degrade quantitative spatial reasoning performance.The paper reports this preliminary finding in Figure 3.Source: §3.1.2 · p. 3
• Entropy minimization rapidly increases feature similarity, but continuous prediction relies on preserving fine-grained geometric relationships between samples.This motivates a geometry-consistent objective that constrains relative feature structure and prevents representation degeneration.Source: §3.1.2 · p. 3

3.2 Method Overview

• Spatial Query Augmentation expands an input query into geometrically coupled auxiliary queries evaluated independently with the same image.The resulting raw estimates include direct, vertical, and horizontal distances.Source: §3.2 · p. 3
• Adaptive Geometric Consistency Modeling derives numerical pseudo-targets from auxiliary predictions and uses relative error to assess their reliability.A target is treated as reliable when its relative error is below a threshold, then converted into structured token-level pseudo-labels.Source: §3.2 · pp. 3–4
• Test-Time Optimization updates the model by minimizing the total loss using the generated pseudo-labels.The geometric term uses pseudo-labels, KL regularization mitigates distributional drift, and entropy regularizes predictive certainty.Source: §3.2 · p. 4

3.3 Spatial Query Augmentation

• The augmentation operator expands a target query into coupled auxiliary queries that probe complementary components of the same scene.This exposes geometric dependencies that independent VLM inference leaves under-constrained.Source: §3.3 · p. 4
• Direct-distance queries are paired with vertical- and horizontal-distance queries, while intrinsic queries probe attributes such as object height and width.The final query set combines relation queries with attribute queries, generated by template-based transformations without external supervision.Source: §3.3 · p. 4
• Predictions across the coupled query set provide an internal-consistency signal for subsequent test-time adaptation.The queried estimates retain metric scale from the base VLM; geometry relates coupled quantities rather than inferring absolute scale on its own.Source: §3.3 · p. 4

3.4 Adaptive Geometric Consistency Modeling

• Geometric relations serve as a reliability filter rather than direct constraints, so optimization is selectively enabled to avoid updates driven by erroneous estimates.The model first produces raw estimates that retain metric scale; geometry relates coupled quantities but does not infer absolute scale alone.Source: §3.4 · p. 4

3.4.1 Adaptive Geometric Triggering.

• A lightweight trigger checks Pythagorean consistency, aspect-ratio stability, and semantic agreement across differently phrased queries before using predictions for self-supervision.These conditions assess reliability rather than acting as direct supervision.Source: §3.4.1 · p. 4
• For each target attribute, optimization activates only when the reconciled reference estimate's relative error from the model prediction meets a predefined tolerance.The reference value is derived from coupled predictions.Source: §3.4.1 · p. 4, §3.4.2 · p. 4

3.4.2 Structured Token-level Pseudo-labeling.

• Reconciled numerical estimates are converted into token-level targets because continuous geometric constraints cannot be directly optimized in autoregressive VLMs.Answers are aligned as integer, decimal-point, fractional, and unit tokens; formatting tokens use deterministic anchors, including the model's predicted unit.Source: §3.4.2 · p. 4
• For numeric tokens, a uniform pseudo-label distributes probability across valid tokens in a plausible interval, yielding gradients more robust to noisy geometry-derived estimates.Candidate numeric tokens are collected within a range around the reconciled estimate.Source: §3.4.2 · pp. 4–5

3.5 Test-Time Optimization

• Optimization is restricted to integer, decimal-point, fractional, and unit positions, so gradients affect tokens that determine numerical value and physical scale.The model obtains predictive distributions for the selected positions via softmax.Source: §3.5 · p. 5

3.5.1 Geometric Consistency Loss.

• Multi-label cross-entropy encourages probability mass on all geometrically valid candidate tokens, while unlikelihood loss suppresses confident predictions outside that set.The unlikelihood term samples a negative token from the top-k at numeric positions.Source: §3.5.1 · p. 5

3.5.2 KL Loss.

• A global KL-divergence loss anchors the updated model's full-sequence output distribution to that of the frozen pretrained model.The comparison uses logits from the frozen and current models.Source: §3.5.2 · p. 5

3.5.3 Entropy Minimization Loss.

• Entropy minimization is restricted to geometry-relevant positions and sharpens belief within an already validated geometric interval alongside geometric-consistency and KL losses.Source: §3.5.3 · p. 5

3.5.4 Parameter Optimization.

• Only low-rank adapter weights are updated; their updates accumulate across valid test instances instead of resetting per sample.The backbone remains frozen, and later predictions use the adapted adapters.Source: §3.5.4 · p. 5

4.1 Experimental Settings

• Benchmarks cover distance, object size, orientation, and object-pair distances; Accuracy uses a 0.25 relative-error threshold (2-hour direction tolerance), while MARE measures average absolute relative error.Source: §4.1 · pp. 5–6

4.2 Comparison Experiments

• TTL-SR improves original backbones across benchmarks, with a 6.47% absolute accuracy gain for Qwen3-VL-4B-Instruct and a 9.41% overall accuracy gain for SpatialRGPT-VILA-1.5-8B.Table 1 covers Q-Spatial-ScanNet, Q-Spatial++, and SPAR-Bench; Table 2 covers SpatialRGPT-Bench.Source: Table 1 · p. 6, §4.2.1 · p. 6, Table 2 · p. 7

4.1 Experimental Settings

• Two open-source backbones are tested against Tent, COME, TLM, and closed-source VLMs across Q-Spatial-ScanNet, Q-Spatial++, SPAR-Bench, and SpatialRGPT-Bench.Source: §4.1 · p. 6, §4.2 · p. 6

4.2.1 Comparison with Original VLMs.

• On Q-Spatial-ScanNet, Tent and TLM lower Qwen3 average accuracy from 40.00% to 38.24% and 39.41%; TTL-SR reaches 46.47% and lowers vertical-distance MARE from 1.33 to 1.23.On SpatialRGPT-Bench, all evaluated methods regress on horizontal distance.Source: §4.2.2 · p. 6

4.2.2 Comparison with TTA / TTL Methods.

• Frontier closed-source VLMs excel at width and height estimation but struggle with complex distance queries.Source: §4.2.3 · p. 6

4.2.3 Comparison with Closed-source VLMs.

• On Q-Spatial-ScanNet, TTL-SR Qwen3 reaches 63.9% direct-distance accuracy, surpassing Gemini-3-Flash’s 58.3%.Source: §4.2.3 · pp. 6–7

4.3.1 Effectiveness of Loss Components.

• Entropy minimization raises average accuracy from 40.00% to 40.18% but degrades height estimation, while KL regularization raises it to 40.59%.These constrained gains indicate that confidence-based objectives alone are insufficient for quantitative spatial reasoning.Source: §4.3.1 · p. 7, Table 3 · p. 7
• 46.47% average accuracy with all components jointly optimized is a 6.47% absolute improvement over baseline, with geometric consistency providing the primary performance boost.Combining geometric consistency with KL regularization reaches 42.94%; the all-component result is summarized in Table 3.Source: §4.3.1 · p. 7, Table 3 · p. 7

4.3.2 Analysis of Geometric Constraints.

• Activating only the direct-distance constraint raises direct accuracy from 55.6% to 58.3% but lowers vertical-distance accuracy from 37.9% to 31.0%.The authors report that unified geometric regularization avoids such skewed updates across coupled spatial attributes.Source: §4.3.2 · p. 7, Table 4 · p. 7

4.3.3 Effectiveness of Framework Components.

• Removing any framework component lowers accuracy and raises MARE; omitting Adaptive Triggering costs 2.94% accuracy, while excluding Unlikelihood Loss raises vertical-distance MARE from 1.23 to 1.46.Replacing soft pseudo-labeling with hard one-hot encoding also degrades precision.Source: §4.3.3 · pp. 7–8, Table 5 · p. 8
• A trigger threshold of 0.20 is most effective; loosening it to 0.40 admits noisy predictions and drops performance by 3.53%.Raising sampling top-k from 1 to 10 increases average accuracy from 44.12% to 46.47%, with diminishing gains beyond 10.Source: §4.3.3 · p. 8, Table 6 · p. 8

4.4 Qualitative Comparison

• TTL-SR’s qualitative predictions are more numerically precise and geometrically coherent than the Qwen3-VL-4B-Instruct baseline across diverse scenes.The authors also report supplementary failure-case analysis.Source: §4.4 · p. 8, Figure 4 · p. 8

• TTL-SR is a geometry-aware test-time learning framework that uses geometric consistency as an intrinsic supervisory signal to adapt VLMs without supervision at test time.The conclusion describes it as a lightweight alternative to retraining.Source: §5 · p. 8
• Experiments across multiple benchmarks report consistent improvements for specialized and general-purpose VLMs, alongside more physically plausible and logically coherent predictions.The authors suggest explicit geometric structure in test-time learning can bridge data-driven prediction and physically grounded reasoning.Source: §5 · p. 8