Source-linked AI summary
Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models
Huang Fei, Boyi Chen, Hao Xue, Yang Yue, Yehong Xu, Xiaofang Zhou
TL;DR
At geographic scales, visual observations alone can make precise spatial relations difficult to recover, while common geometry interfaces may weaken spatial organization. GeoVec-VLM preserves vector structure and brings it into multimodal computation with vision and language. The paper reports improved accuracy and better alignment with spatial constraints, alongside stronger geometric consistency.
Problem
At geographic scales, precise spatial relations are difficult to recover from visual observations alone, and existing vector interfaces can weaken spatial organization for multimodal reasoning.
Source: §1 · pp. 1–2Method
GeoVec-VLM represents vector objects with structure-preserving point and path tokens and uses spatial states and geometric biases to associate geometry with vision and language.
Source: §1 · p. 2Results
GeoVec-VLM improves accuracy and better aligns model reasoning with underlying spatial constraints, while language-level accuracy can mask geometric inconsistency.
Source: §1 · p. 2Takeaways & Limitations
The authors conclude that organizing multimodal reasoning around vector geometry provides an effective spatial substrate, though geometric information may remain recoverable without being used effectively in language predictions.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Geospatial reasoning links VLMs to real-world action and decision making. Reliable reasoning requires accurate spatial information and effective spatial representations. However, current VLMs obtain this information mainly from language and vision. At larger geographic scales, limited visual observations make it difficult to establish accurate spatial references and recover precise spatial relations. This leads to a bottleneck for VLM spatial perception and reasoning. This paper proposes that geospatial vector geometry can become a native spatial modality within a VLM and introduces GeoVec-VLM. First, we develop a structured geometry representation that preserves the spatial structure and arrangement of vector objects, building an explicit spatial basis. Second, we introduce a geometry interaction strategy that brings this basis into multimodal computation, where it interacts with visual content and language references according to the queried spatial relations. Together, these designs allow perception and reasoning to operate on shared geometric references. Across five spatial reasoning tasks, GeoVec-VLM improves mean accuracy by about 11\% relative to the strongest baseline, maintains its gains under geographic transfer, and shows greater geometric consistency. Our results show that geospatial vector geometry offers a promising internal spatial basis for VLM spatial reasoning. https://github.com/TXFR/GeoVector-VLM
1 INTRODUCTION
Geographic-scale reasoning demands precise relations across large extents, but visual observations and common geometry interfaces may not preserve enough spatial structure. GeoVec-VLM represents vector geometry as a native modality and integrates it with vision and language; the paper reports improved accuracy and geometric consistency.
1 INTRODUCTION
- At geographic scales spanning hundreds of meters to tens of kilometers, precise relations are difficult to recover from visual observations alone under fixed input and encoding budgets.The introduction contrasts this challenge with camera-centered settings that provide rich local geometric cues.Source: §1 · p. 1
- The overview contrasts raster-and-language inference, where geometry and relations are implicit, with vector geometry integrated alongside vision and language.The proposed modality preserves vector structure during representation.Source: Figure 1 · p. 2, §1 · p. 2
- Coordinate serialization and fixed-size object embeddings can weaken the spatial organization available for multimodal reasoning.These are identified as common interfaces for vector geometry.Source: §1 · p. 2
- GeoVec-VLM encodes vector objects as point and path tokens to preserve global arrangement and local geometry, then associates geometry with visual content and language references.Learnable spatial states and geometric biases support query-specific relations, with structured objectives supervising the states.Source: §1 · p. 2
- GeoVec-VLM improves spatial accuracy and better aligns reasoning with underlying spatial constraints, while findings show language-level accuracy can mask geometric inconsistency.The introduction presents vector integration as improving both accuracy and consistency.Source: §1 · p. 2
2 RELATED WORK
Prior work makes spatial references accessible in language and vision models, but often uses points, regions, or object-level geometry summaries. GeoVec-VLM keeps vector structure available during multimodal interaction.
2.1 SPATIAL REASONING AND SPATIAL PRIMITIVES
- Spatial primitives such as points, regions, and boxes improve grounding, but polygon geometry also carries ordered coordinates, path progression, local direction, and object extent.The related-work discussion frames retaining this structure for multimodal reasoning as a distinct representation problem.Source: §2.1 · p. 3
2.2 VECTOR GEOMETRY FOR MULTIMODAL SPATIAL REASONING
- GeoVec-VLM keeps metric spatial references and vector structure accessible during vector–vision–language interaction, unlike approaches that typically summarize complex objects before interaction.Prior work includes serialized geometries, fixed-size geometry representations, and systems combining imagery with vector maps.Source: §2.2 · p. 3
3 METHOD
GeoVec-VLM converts vector geometry into structured tokens and integrates them with vision and language through object- and query-conditioned spatial states. These states organize geometric evidence for spatial reasoning and language generation.
3.1 PROBLEM FORMULATION AND OVERVIEW
- GeoVec-VLM encodes vector geometry into tokens, then uses object, role, and relation states to organize multimodal spatial reasoning and language generation.Numerical-query tokens are included only for tasks with explicit numerical spatial inputs.Source: §3.1 · p. 4
3.2 STRUCTURE-PRESERVING VECTOR GEOMETRY ENCODING
- The encoder preserves ordered within-object geometry and cross-object metric relations while retaining element-level geometry tokens rather than pooling each parcel.Each geometric element retains its source object, boundary position, element type, and metric coordinate.Source: §3.2 · pp. 4–5
3.3 SPATIAL PERCEPTION AND OBJECT STATES
- Bidirectional attention lets geometry, visual content, and optional numerical queries exchange information before parcel-specific object states aggregate geometry with overlapping visual states.Each object state attends only to itself, its parcel’s geometry states, and visual states whose image footprints overlap that parcel.Source: §3.3 · p. 5
3.4 QUERY-CONDITIONED SPATIAL STATES AND LANGUAGE GENERATION
- Query-conditioned role states bind language roles to objects, relation states compose their geometry, and gated cross-attention exposes spatial memory to language.Structured decoders provide auxiliary training supervision, but their predictions are not used by the language path.Source: §3.4 · pp. 5–6
4 EXPERIMENTAL SETUP
The experiments use geographically split street-block scenes and evaluate multiple spatial-reasoning tasks in China and transfer regions. Comparisons include general-purpose and spatial VLMs as well as controlled geometry-representation variants.
Dataset
- The dataset pairs 512×512 raster images with aligned vector geometry across 63,973 China scenes, with six U.S. metropolitan regions and Singapore reserved for zero-shot geographic transfer.Training uses 359,340 question–answer instances from 221 city-level regions, with geographically disjoint training, validation, and test regions.Source: §4 · p. 6
Tasks and Evaluation
- Evaluation covers five spatial reasoning categories and six task families, using accuracy, reciprocal consistency, and applicable component-wise RMSE.The protocol excludes ambiguous cases and includes hard samples where common geometric approximations disagree with exact relations; reversible queries are evaluated in pairs.Source: §4 · p. 6
Baselines and Input Interfaces
- Baselines include general-purpose and spatial VLMs, while controlled same-backbone comparisons isolate vector-input benefits from preserving fine-grained geometric structure.Controlled variants use the Qwen3-VL-8B backbone, training data, and task supervision, comparing serialized coordinates, Geo2Vec, Poly2Vec, pooled GeoVec, and element-level GeoVec.Source: §4 · p. 6
5 RESULTS AND ANALYSIS
GeoVec-VLM outperforms the strongest image/GeoJSON baseline on supervised tasks and maintains strong performance under geographic transfer. Its analyses also show more geometrically consistent reasoning and identify the roles of preserved geometry, cross-modal grounding, and learned spatial states.
5.1 SPATIAL REASONING ACCURACY AND GEOMETRIC COHERENCE.
- GeoVec-VLM exceeds the strongest image/GeoJSON baseline by 4.4–12.1 percentage points on three supervised tasks, remains competitive on Area, and achieves the best Distance accuracy.Table 1 evaluates geographically disjoint test regions from China; Area and Distance use language generation without corresponding structured decoders or downstream task supervision.Source: §5.1 · p. 6, Table 1 · p. 7
- Under zero-shot geographic transfer, GeoVec-VLM maintains performance, with 77.5% mean accuracy in the U.S. and 78.3% in Singapore, compared with 77.2% on the China test set (Appendix C.1).Source: §5.1 · p. 7
- GeoVec-VLM reaches 81% agreement between categorical direction and displacement and 0.88 reciprocal consistency, with more stable proximity and topology results.GPT-5.1 reaches 53% direction–displacement agreement; baseline proximity decisions can flip under candidate-order reversal, and topology outputs show inside/outside bias.Source: §5.1 · p. 7
5.2 ABLATION AND MECHANISM ANALYSIS
- Retaining point- and path-level states through multimodal interaction improves all five tasks by a further 5.1–17.5 points over pooled geometry tokens.The pooled and full variants share the geometry encoder and training setup, isolating the benefit of preserving element-level structure during reasoning.Source: §5.2 · p. 8, Table 2 · p. 8
- Cross-modal grounding reaches 87.9% accuracy, while disabling vector–visual interaction reduces it to 28.3%; shuffling geometry also lowers Object Top-1 and Visual Support F1.With aligned inputs, Object Top-1 is 0.85 and Visual Support F1 is 0.75; shuffling geometry lowers them to 0.24 and 0.17, respectively.Source: §5.2 · p. 9, Figure 4 · p. 9
- Removing O/A/Z reduces answer accuracy by 19.9–40.6 points, structured supervision adds 3.4–27.9 points, and role–object binding gains about 5 points when language must identify the object.Structured readouts remain 70.3–80.2% without O/A/Z; Area and Distance improve despite lacking corresponding structured decoders.Source: Table 3 · p. 9, §5.2 · pp. 9–10
6 CONCLUSION
GeoVec-VLM uses geospatial vector geometry as a shared internal basis for multimodal spatial reasoning, improving accuracy and geometric consistency. Its appendices describe a structure-preserving polygon representation and a two-stage geometry encoder pretraining process.
6 CONCLUSION
- The authors report that organizing multimodal reasoning around vector geometry improves spatial accuracy and geometric consistency, while limiting the study to accurate vector geometry and raster imagery in static 2D polygonal scenes.Noisy, incomplete, outdated, or misaligned vector data, other geometry types, and broader visual settings remain for future work.Source: §6 · p. 10
A.1 VECTOR GEOMETRY REPRESENTATION
- Polygon boundaries are represented by ordered reference points and path elements, preserving ring structure and local metric information while reducing explicit boundary elements.The representation retains part and ring membership, boundary order, connectivity, and geometry omitted by simplification; all features are computed in a region-specific metric coordinate system.Source: §A.1 · pp. 14–15, Figure 6 · p. 14
A.2 VECTOR GEOMETRY ENCODER PRETRAINING
- The vector encoder is pretrained first on polygon boundary structure and then on spatial relations across neighboring polygons.Object-level objectives target cyclic invariance, boundary-order sensitivity, and discrimination among embedding-similar parcels; scene-level objectives predict point and boundary relations plus six scene statistics.Source: §A.2 · pp. 15–16, Figure 7 · p. 15
A.3 SPATIAL STATES, STRUCTURED SUPERVISION, AND GEOVEC-VLM TRAINING
GeoVec-VLM organizes geometry, visual content, and queried spatial relations in object, role, and relation states. Training first warms up the structured spatial pathway, then enables language states to use the spatial states for answer generation.
Object and role states
- Each parcel’s object state attends only to itself, its geometry, and visual representations overlapping its footprint, while query roles read geometry from the selected parcel.The same object selection is reused across reasoning layers as geometry representations evolve.Source: §A.3 · p. 17
Relation states
- Relation states organize geometry according to the query, preserving argument order for direction and parcel topology through role-conditioned geometry and signed differences.For comparative proximity, element-level boundary pairs are scored before object-level aggregation, and a differentiable soft minimum summarizes the closest interactions.Source: §A.3 · pp. 17–18
Structured supervision
- Structured relation states support direction, continuous displacement, parcel topology, and comparative proximity predictions, with task-specific readouts and supervision.Topology uses order-sensitive inside/contains logits and a symmetric outside logit; displacement predictions are antisymmetrized across object orders and rescaled for RMSE evaluation in meters.Source: §A.3 · p. 18
GeoVec-VLM Training
- Training first warms up the structured spatial pathway on direction, topology, and proximity instances, then enables language–spatial fusion while optimizing language and structured objectives.Area and distance QA instances are excluded from the language-generation objective throughout training.Source: §A.3 · p. 19
B.1 DATASET CONSTRUCTION AND GEOGRAPHIC SPLITS
The dataset pairs spatially aligned imagery with vector geometry and uses geographically disjoint splits for evaluation. Separate U.S. and Singapore sets test zero-shot geographic transfer, where polygon geometry distributions differ from China training data.
Dataset construction
- China data use L3 and L4 polygons from 221 city regions; filtering removes polygons requiring excessive simplification, leaving 343,983 retained polygons and 230,566 candidate local contexts.Contexts retain 3–16 neighbors, a maximum inter-parcel gap of 120 m, and at least 0.30 facing-boundary coverage within a 60 m boundary buffer.Source: §B.1 · p. 20
Image–vector scene construction
- The China dataset contains 66,159 aligned image–vector scenes, with geographically disjoint training, validation, and test regions and no held-out-region polygons used for geometry-encoder gradients.The standard scene split is 47,980 training, 7,998 validation, and 7,995 test scenes; another 2,186 scenes are reserved for robustness evaluation.Source: Figure 8 · p. 19, §B.1 · p. 20, Table 7 · p. 20
Geographic transfer
- Zero-shot transfer uses six evaluated U.S. metropolitan regions and Singapore as a single-region set; neither geography contributes to training or model selection.U.S. polygons are moderately larger than China training polygons but similar in boundary complexity, while Singapore has more retained reference points and longer geometry-token sequences despite comparable polygon-area range.Source: §B.1 · p. 21, Figure 9 · p. 21
B.2 TASK CONSTRUCTION AND METRICS
The evaluation covers directional, topological, and metric reasoning using geometries and labels constructed from projected vector data. It also examines consistency across paired queries, qualitative reasoning cases, and the implementation choices used for baselines.
B.2 TASK CONSTRUCTION AND METRICS
- The benchmark evaluates six spatial task families spanning direction, proximity, topology, area, and boundary distance, with continuous displacement paired with direction queries.The task families cover directional, topological, and metric reasoning; the China test set has 1,800 questions per task, while transfer sets use the same protocol.Source: §B.2 · p. 21, Table 8 · p. 21
- Task labels are generated deterministically from exact projected geometries, and hard cases exclude ambiguous samples where common geometric proxies disagree with the target relation.Matched reversible or opposite queries are used to assess geometric consistency; evaluation prompts use forms that do not overlap with the 32 compositional training forms.Source: §B.2 · pp. 22–23
B.3 QUALITATIVE SPATIAL REASONING CASES
- Post-hoc examples show general-purpose VLMs giving plausible visual explanations yet missing proximity or ordered containment relations that GeoVec-VLM resolves using aligned vector geometry.The explanations are collected after constrained task answers and do not affect quantitative predictions or accuracy.Source: §B.3 · p. 24, Figure 11 · p. 24
B.4 BASELINE IMPLEMENTATION
- Baseline implementations vary in how spatial references and geometry are supplied, including visual parcel labels, serialized coordinate sequences, and fixed-size geometry embeddings.SpatialRGPT and SR-3D use released checkpoints adapted on 3,840 China-training instances; the controlled Geo2Vec and Poly2Vec comparison shares scene objects, query assignments, backbone, and split.Source: §B.4 · pp. 24–25
C.1 GEOGRAPHIC TRANSFER EVALUATION
Across the U.S. and Singapore transfer sets, GeoVec-VLM ranks first on all three supervised tasks, with stable performance across regions. Its margins are smaller on unsupervised Area and Distance, where it remains competitive.
C.1 GEOGRAPHIC TRANSFER EVALUATION
- GeoVec-VLM ranks first on all three supervised tasks in both the U.S. and Singapore, with Topology gains of +16.0 and +17.3 points over the strongest image+GeoJSON baseline.These are geographic-transfer results; Table 9 also marks Area and Distance as receiving no corresponding downstream task supervision.Source: Table 9 · p. 25, §C.1 · p. 26
- Performance on the three supervised tasks varies by only 1.7–2.6 percentage points between the U.S. and Singapore transfer sets.The regional split supports geographic robustness and transferability of the learned spatial representations.Source: §C.1 · p. 26
- Area and Distance receive no corresponding structured supervision, yet remain competitive in both transfer regions with smaller margins.Source: Table 9 · p. 25, §C.1 · p. 26
C.2 TARGETED GEOMETRY-TOKEN INTERVENTIONS
Targeted interventions show that GeoVec-VLM draws on different geometric cues for different tasks, with cue effects varying across regions. After language–spatial fusion, language states preserve continuous geometric structure that tracks spatial relations.
C.2 TARGETED GEOMETRY-TOKEN INTERVENTIONS
- The Parcel Topology drop from removing global placement rises from 20.1 points in China to 33.7 in the U.S. and 37.8 in Singapore; structure-ID effects also grow on several transfer tasks.These changes are consistent with regional shifts in polygon scale and boundary complexity, and suggest that task performance can rely on different mixtures of cues across parcel distributions.Source: Table 11 · p. 26, §C.2 · p. 27
- Removing global placement, local shape, or hierarchical IDs reveals distinct task roles: placement affects Direction and Proximity most, while local shape matters more for Parcel Topology and Area.Removing structure IDs causes broader but generally smaller degradation; the interventions preserve the other geometry cues.Source: §C.2 · pp. 26–27, Table 10 · p. 26
C.3 HOW VECTOR GEOMETRY INFLUENCES ANSWER GENERATION
- After language–spatial fusion, representational distance tracks direction’s angular separation (ρ = 0.895) and proximity’s physical distance advantage (ρ = 0.418).Figure 12 shows the sharp answer-accuracy transition after fusion and its suppression when fusion is removed; the fused language representation retains geometric structure beyond discrete task labels.Source: Figure 12 · p. 27, §C.3 · p. 27
D.1 INPUT TOKEN CONSUMPTION.
GeoVec’s element-level geometry tokens use more input tokens than fixed-size vector embeddings but yield higher mean accuracy, while serialized GeoJSON consumes substantially more tokens and remains less accurate. After GeoVec adaptation, the evaluated general capabilities show little change relative to the original backbone.
D.1 INPUT TOKEN CONSUMPTION.
- On China, serialized GeoJSON consumes 37.27M input tokens while reaching 45.6% mean accuracy; transfer-set API models use 143–325M tokens in the U.S. and 58–132M in Singapore for GeoJSON-based inputs, substantially more than GeoVec while remaining less accurate.The results suggest useful geometry-interface bandwidth depends on both accessible geometric information and its structural organization for multimodal computation.Source: §D.1 · p. 28, Figure 13 · p. 28
- On the controlled China benchmark, element-level GeoVec raises mean accuracy to 77.2%, from 41.1–41.5% for Geo2Vec and Poly2Vec, which use 4.90M input tokens.Figure 13 token counts include complete model inputs, including visual inputs, language instructions, and geometry when present.Source: §D.1 · pp. 27–28, Figure 13 · p. 28
D.2 GENERAL CAPABILITY PRESERVATION
- After GeoVec adaptation, IFEval strict prompt accuracy changes from 84.47% to 85.03%, while MMLU accuracy changes from 77.54% to 77.28%; both paired 95% confidence intervals include zero.The diagnostics provide no evidence of a measurable loss in instruction following or multidisciplinary knowledge; evaluation retains trained language parameters while disabling GeoVec-specific spatial modules.Source: Table 12 · p. 28, §D.2 · p. 28