GeoLatent: Geometry-Guided Latent Structuring with Routed Optimization for 3D Reasoning

1 INTRODUCTION

• Text-based intermediate reasoning represents continuous geometry with discrete language or numerical tokens, which may lose fine-grained spatial detail and increase susceptibility to language priors.A single latent type also has difficulty accommodating spatial cues with different functions and levels of granularity.Source: §1 · p. 1
• CR-GEO models shared teacher geometry at the set level and encourages geometry states to capture complementary residuals, while routed optimization temporarily directs answer-relevant visual information through the latents.Training jointly optimizes answer generation and geometric supervision, then restores full attention while retaining geometric supervision.Source: §1 · p. 2
• 73.0% on SPAR-Bench and 72.1% on SPBench improve over reported GeoAnchor results by 4.6 and 2.4 points, respectively.Under identical initialization and batch order, replacing GeoAnchor’s geometry alignment objective with CR-GEO raises effective rank from 1.00 to 3.87; blocking latent readout at the bottleneck lowers direction accuracy on 128 fixed questions from 89.1% to 25.8%.Source: §1 · p. 2
• After full visual access is restored, differentiated geometry representations and a latent-mediated visual route remain available alongside direct image access.The final predictions remain strongly dependent on visual evidence.Source: §1 · p. 2

2.1 SPATIAL REASONING IN VISION-LANGUAGE MODELS

• Spatial-reasoning methods use instruction tuning, progressive training, or explicit 2.5D/3D cues, while GeoAnchor decomposes 3D information into position, direction, and geometry latents.Intermediate reasoning in these approaches is still often expressed in discrete language, which can be insufficient for continuous spatial relations.Source: §2.1 · p. 3

2.2 LATENT REASONING AND VISUAL INTERMEDIATES

• Latent-reasoning methods include continuous hidden states, latent visual intermediates, and visual bottlenecks; GeoLatent evaluates both geometry organization and latent use during answer learning.Prior work also identifies latent-trajectory collapse and answer bypass, while mechanistic evidence cautions that gains can reflect boundary markers, formatting, or attention patterns rather than slot content alone.Source: §2.2 · p. 3

2.3 REPRESENTATION COLLAPSE

• Under coverage-based alignment, balanced utilization of explicitly supervised geometry states does not ensure that different states encode complementary information.CR-GEO represents shared geometry at the set level and encourages individual states to capture residual variation; effective rank quantifies representation diversity.Source: §2.3 · p. 3

3 METHOD

• GeoLatent adds CR-GEO for global geometry and routed optimization to promote the use of spatial latents during answer learning.It builds on GeoAnchor’s interleaved text–latent framework.Source: §3 · pp. 3–4

3.1 PRELIMINARIES: DECOMPOSED SPATIAL LATENTS

• POS and DIR represent local position and direction cues, while multiple GEO states represent global scene geometry under corresponding geometric supervision.The framework interleaves language tokens and continuous spatial latents in one reasoning trajectory.Source: §3.1 · p. 4

3.2 REDUNDANCY UNDER COVERAGE-BASED GEO ALIGNMENT

• Balanced utilization can still permit identical GEO states, so coverage-based alignment does not ensure differentiated geometry representations.When all GEO features are identical, the balance term vanishes; with a strong shared teacher component, the collapsed family can be favored.Source: §3.2 · p. 5

3.3 COMMON–RESIDUAL GEOMETRY ALIGNMENT

• CR-GEO separates shared teacher geometry from residual variation, aligning the common geometry once and encouraging GEO states to encode complementary residuals.Its objective combines common alignment with residual matching and specificity terms.Source: §3.3 · pp. 5–6

3.4 ROUTED OPTIMIZATION

• Routed optimization jointly trains language and geometry, temporarily makes answer learning depend on latents for visual information, then restores full visual attention while retaining geometry supervision.During the bottleneck stage, latent queries retain visual access while question, intermediate-text, and answer queries cannot directly attend to image keys.Source: §3.4 · p. 6

4 EXPERIMENTS

• The evaluation uses official prompts and scorers on SPAR-Bench, SPBench, and ViewSpatial; analyses separately test representation structure, latent use, task performance, and visual dependence.The overlap audit finds no exact input-image or image/frame–question–answer duplicates under available metadata.Source: §4 · p. 6, §4.1 · p. 7

4.2 MAIN BENCHMARK RESULTS

• 73.0% on SPAR-Bench and 72.1% on SPBench improve over reported GeoAnchor results by 4.6 and 2.4 points, respectively.All five SPAR categories improve; the largest gain is View (+7.5 points).Source: §4.2 · p. 7, Table 1 · p. 7

4.3 STRUCTURING THE LEARNED SPATIAL REPRESENTATION

• Under matched initialization and batch order, CR-GEO raises GEO effective rank from 1.00 to 3.87 and improves the three-benchmark mean by 1.29 points.At recovered endpoints, GeoLatent retains effective rank 3.62, versus rank one for the original objective.Source: §4.3 · p. 8
• On 504 held-out records from 248 training-disjoint scenes, POS error is 0.338 m and independent-DIR cosine is 0.899.Replacing the RGB input with a different-scene image raises POS error from 0.389 to 1.474 m.Source: §4.3 · p. 8
• Table 2’s 2 × 2 comparison finds that geometric supervision improves the three-benchmark mean by 1.2 points without bottleneck training and by 2.6 points after it.The reported interaction is 1.4 points.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 OPERATIONAL USE AND VISUAL DEPENDENCE

• At the bottleneck, blocking latent readout lowers matched K=8 direction accuracy from 89.1% to 25.8% on 128 fixed questions, while public GeoAnchor remains at 82.0%.The intervention changes accuracy by 63.28 points at bottleneck but only 0.78 points at joint and recovery checkpoints.Source: §4.4 · p. 8, Figure 4 · p. 9
• After recovery, the latent-only route retains 52.2/51.3/33.5% on SPAR-Bench/SPBench/ViewSpatial; removing latent readout lowers scores by 5.3/3.7/1.8 points.This route remains usable without being the dominant path under full attention.Source: §4.4 · pp. 8–9, Table 3 · p. 9
• On ViewSpatial, blanking or mismatching the image lowers GeoLatent accuracy from 43.2% to 27.1% or 32.7%, and blanking removes 88.3% of its above-chance margin.The matched no-GEO control loses 60.1% of its above-chance margin when blanked.Source: §4.4 · p. 9, Figure 4 · p. 9

5 CONCLUSION

• CR-GEO separates shared teacher geometry from residual variation, while routed optimization directs visual answer learning through latents before restoring full attention.The restored route retains geometry supervision.Source: §5 · p. 10
• Experiments report differentiated GEO representations, decodable image-specific POS/DIR latents, and strong latent use at the bottleneck stage.A usable latent-mediated route remains after recovery.Source: §5 · p. 10
• Image interventions show recovered predictions remain tied to corresponding visual evidence rather than the ViewSpatial Person-template prior.The reported results improve spatial reasoning while distinguishing representation structure, latent use, and visual dependence.Source: §5 · p. 10

A PROOF OF PROPOSITION 1

• When all normalized geometry latents collapse to the same vector, teacher assignments are uniform across latents and the balance term vanishes.The passages also state that cosine similarities are equal across latent assignments.Source: A PROOF OF PROPOSITION 1 · p. 14
• On the frozen representation-diagnostic frame, analytical and computed collapsed-configuration losses differ by 1.4 × 10^-6 on average.The collapsed configuration has lower loss than learned latents on every tested sample and checkpoint.Source: A PROOF OF PROPOSITION 1 · p. 14

B.1 DATA AND BENCHMARK CONSTRUCTION

• The main GeoLatent runs use a 105,928-sample single-image mixture of SPAR-derived records and SpatialLadder samples.Structured3D spatial-imagination records retain their released sentence-format answers.Source: §B.1 · p. 14
• Geometry-enabled records supervise local POS/DIR spans when objects are present and align the GEO span to VGGT teacher features pooled at resolutions {1, 2, 4}.CR-GEO replaces GeoAnchor’s alignment term while retaining its teacher features and pooled resolutions.Source: §B.1 · p. 14

B.2 BENCHMARK DEFINITIONS

• SPAR-Bench and SPBench are in-domain, while ViewSpatial is an external cross-dataset transfer benchmark with distinct examples and partially overlapping task definitions.ViewSpatial tests camera- and person-centered reference frames.Source: §B.2 · p. 14, Table 5 · p. 15
• SPAR-Bench uses exact-option correctness for multiple-choice questions and threshold-averaged accuracy for numerical questions; SPBench reports relative and absolute spatial categories.ViewSpatial reports camera- and person-perspective subgroups.Source: §B.2 · p. 15
• The overlap audit finds no exact input-image or image/frame–question–answer composite shared with training, but SPAR-Bench exposes no scene/frame metadata for a scene-disjoint claim.Removing 37 records from two training-overlapping ScanNet scenes changes the three-benchmark mean from 62.76 to 62.59.Source: §B.2 · pp. 15–16, Table 6 · p. 16

B.3 FULL BLANK- AND MISMATCHED-IMAGE INTERVENTIONS

• GeoLatent’s real-image ViewSpatial Camera score is 53.20%, 8.1 points above GeoAnchor, and falls 24.53 points when images are removed; its blank Person score is 24.81%.Across SPAR-Bench and SPBench, GeoLatent loses 27.0 and 28.3 points under blanking, respectively.Source: Table 7 · p. 16, §B.3 · pp. 16–17
• On SPAR-Bench and SPBench, GeoLatent loses 27.0 and 28.3 points under blanking, and 34.9 and 27.6 under mismatched images.The reported sampled sign-flip test found no result as extreme as observed across 100,000 draws.Source: §B.3 · p. 16, Table 7 · p. 16

B.4 TEMPLATE AND ANSWER PRIORS IN VIEWSPATIAL

• SpatialLadder changes only from 44.76% real-image accuracy to 43.37% blank and 41.09% mismatched on ViewSpatial.Its subgroup results show a strong Person–Object-Orientation prior: 76.10% with blank images, with “front” the most frequent gold answer.Source: §B.4 · p. 17, Table 8 · p. 17
• Despite training on all 5,928 SpatialLadder single-image examples, GeoLatent falls from 43.20% to 27.13% when ViewSpatial images are blanked.The sources share template vocabulary and reference-frame construction, but no complete question strings overlap after normalization.Source: §B.4 · p. 17

• Training proceeds through joint, bottleneck, and recovery phases, with stage-specific local and GEO supervision weights.All phases use Qwen3-VL-2B-Instruct, a frozen vision encoder, trainable merger/LLM/latent modules, global batch 32, and bf16 computation.Source: Table 9 · p. 18, §B.5 · p. 18
• GeoLatent uses local/GEO weights 1.0/0.2, 0.1/0.1, and 0.1/0.2 across the three phases, versus 1.0/0.1, 0.1/0.01, and 0.01/0.01 for the reference.The reference shares the data, initialization, phase lengths, and learning rates.Source: §B.5 · p. 18
• The loss-only joint test changes only LCR-GEO versus Lcov, using bit-identical initialization, data order, optimizer, learning rate, steps, attention policy, and supervision weights.Recovered endpoints use the same curriculum settings but are trained separately under different GEO objectives.Source: §B.5 · p. 18
• During bottleneck training, non-latent queries cannot attend directly to image keys, while decomposed latent queries retain image access; recovery restores full attention with local and GEO supervision.Source: §B.5 · p. 18

B.6 FULL OBJECTIVE AND CURRICULUM ABLATION

• The full bottleneck-stage comparison improves the three-benchmark mean by approximately 3.6 points over the corresponding model using GeoAnchor’s GEO loss.Source: §B.6 · p. 18, Table 10 · p. 19
• Local-only recovery and recovery with local/GEO weights 0.01/0.05 beat NTP-only recovery by 1.5–1.7 points; standard GeoLatent weights yield the largest gain.Recovery restores task performance under the original objective, but only CR-GEO builds and retains differentiated GEO structure in this comparison.Source: §B.6 · p. 18
• CR-GEO yields an effective rank of 3.87 versus approximately 1.00 and a 1.29-point equal-weight mean gain in the matched-initialization joint test.It improves SPAR-Bench, SPBench, and ViewSpatial by 0.9, 1.1, and 1.85 points, respectively.Source: §B.6 · p. 18
• K=4 approaches the teacher’s residual capacity and gives the best recovery mean; larger GEO spans do not improve task performance and lower recovery throughput.Throughput falls from 2.86 samples/s at K=4 to 1.77 at K=16; K=4 is a compact operating point, not a monotonic benefit from adding vectors.Source: Table 11 · p. 19, §B.6 · p. 19

B.7 CROSS-BACKBONE REPRODUCTION

• With Qwen2.5-VL-3B, CR-GEO again retains nearly four effective GEO dimensions while the GeoAnchor-loss model remains rank one; in-domain benchmarks and their mean improve, while ViewSpatial decreases by 1.70 points.Source: §B.7 · p. 19, Table 12 · p. 20
• On the cross-backbone held-out set, replacing RGB with a different-scene image raises POS error from 0.444 to 1.477 m and lowers DIR cosine from 0.922 to 0.219.Local POS/DIR decodability transfers, with CR-GEO obtaining 0.396 m POS error and 0.888 DIR cosine.Source: §B.7 · pp. 19–20

B.8 ADDITIONAL DIAGNOSTICS AND QUALITATIVE VISUALIZATIONS

• Figure 5’s t-SNE projection shows reduced GEO-vector concentration under CR-GEO, while local-token maps show object-specific visual attention during latent generation.Source: Figure 5 · p. 20, §B.8 · p. 20

B.8.1 LOCAL-TOKEN DECODING ACROSS THE CURRICULUM

• For CR-GEO, recovery changes POS error by −0.011 m and DIR cosine by −0.024 relative to joint training; the POS change’s scene-bootstrap 95% CI is [−0.020, −0.0001].The evaluation uses 504 records from 248 scenes disjoint from training; the final model’s POS/DIR decodability exceeds train-derived constant priors.Source: §B.8.1 · p. 20, Table 13 · p. 21

B.8.2 DECOMPOSED-LATENT ROUTE INTERVENTIONS

• At the bottleneck checkpoint, jointly cutting POS and DIR removes nearly all above-chance accuracy on the tested direction subset.The public GeoAnchor checkpoint remains at 82.03% before and after the latent-readout cut on the same frozen pairs.Source: §B.8.2 · pp. 21–22
• Blocking latent readout changes accuracy by 0.78 points at joint, 63.28 at bottleneck, and 0.78 after recovery, indicating stage-specific latent dependence during bottleneck training.The intervention preserves latent generation and latent-to-latent relay while hiding latent keys from subsequent non-latent queries.Source: §B.8.2 · pp. 21–22
• After recovery, GeoLatent’s latent-only route retains 71.3%, 71.1%, and 77.4% of full-attention scores on SPAR-Bench, SPBench, and ViewSpatial, outperforming the neither condition by 5.31, 3.72, and 1.76 points.The direct-recovery control gains approximately zero; ordinary full attention also restores direct visual access.Source: §B.8.2 · p. 22

B.9 CONTROLLED NO-GEO CAPABILITY PROFILE

• GeoLatent reaches a 62.8% three-benchmark mean with a differentiated GEO span; its Person-centered direction task requires observer orientation Rperson, absent from the current camera-frame vocabulary.The same-data no-GEO control scores 62.4%; GeoLatent has stronger ViewSpatial visual reliance and a 5.9-point camera-relative-direction gain.Source: §B.9 · p. 23