UniGeo: A Multi-modal Large Language Model for Text-Guided Cross-View Geo-Localization

• Natural-language descriptions make drone geo-localization practically relevant by expressing appearance, spatial layout, and relations to surrounding structures.Source: §I · p. 1
• Existing methods mainly directly match open-ended queries to candidates, although queries may lack stable and discriminative geo-localization evidence.Source: §I · p. 1
• UniGeo unifies region grounding, captioning, region-grounded description, image-text matching, cross-view generation, and candidate verification in one MLLM.Source: Figure 1 · p. 1, §I · p. 2
• The framework targets geo-semantic understanding, cross-view semantic generation, and fine-grained verification to refine queries and distinguish confusable candidates.Source: §I · p. 2
• UniGeo consistently improves Text Query mAP over several retrieval backbones while adding only marginal parameters.Source: §I · p. 2, Figure 2 · p. 2

• Cross-view geo-localization addresses correspondence between observations from different platforms despite viewpoint changes, layout distortion, and imaging differences.Source: §II · p. 2
• GeoText-1652 introduced natural language for drone-to-satellite retrieval, while later methods improved alignment and transfer mechanisms.Source: §II · p. 3
• Vision-language pretraining established shared image-text embedding spaces that support transferable recognition and retrieval.Source: §II · p. 3
• Unified vision-language models increasingly combine retrieval, generation, and reasoning within a single framework.Source: §II · p. 3

• UniGeo uses a unified geospatial representation space for local semantics, language-to-region correspondence, and structured spatial relations.Source: §III · p. 3
• Its Progressive Learning Policy optimizes geospatial grounding, cross-view semantic augmentation, and candidate-level verification in sequence.Source: §III · p. 3

UniGeo Framework

• UniGeo uses a shared vision-language backbone with region-level semantic and structured discriminative supervision.Source: A. UniGeo Framework · p. 3

Region-level Semantic Supervision

• Region-level supervision generates descriptions for specified local regions using bounding-box conditioning, image overlays, and optional ROI crops.Source: A. UniGeo Framework · p. 3
• The conditional language objective establishes stable correspondences between local visual regions and language descriptions.Source: A. UniGeo Framework · p. 3

UniGeo Framework

• The framework combines unified tokens, task-specific heads, progressive three-stage learning, and plug-and-play candidate re-ranking.Source: Figure 3 · p. 4

Structured Discriminative Supervision

• A language-to-region head predicts referred bounding boxes from regional text and full-image tokens, explicitly enforcing grounding.Source: A. UniGeo Framework · p. 4
• Spatial supervision classifies horizontal relations as left, center, or right and vertical relations as upper, middle, or lower.Source: A. UniGeo Framework · p. 4

Stage-I: Geo-Semantic Understanding Learning

• Stage I jointly learns region semantics, image-text alignment, language-to-region grounding, and inter-region spatial structure.The objective combines region-level semantic, global alignment, fine-grained matching, spatial, and bbox-grounding supervision.Source: B. Progressive Learning Policy · p. 5

Stage-II: Pose-Aware Cross-View Generation

• Stage II adds pose-aware latent-diffusion generation to model semantic and geometric consistency across satellite and drone viewpoints.The shared vision-language backbone is inherited and frozen, while a pose teacher supplies heading and range supervision.Source: B. Progressive Learning Policy · p. 5
• The Stage-II objective combines diffusion denoising, pose consistency, and auxiliary heading regularization.Pose consistency decomposes into heading and range terms, with the heading term comparing predicted vectors against sine-cosine angle encodings.Source: B. Progressive Learning Policy · p. 5

Stage-III: Hard-Negative Verification Head Learning

• Stage III freezes the backbone and generation branch while training a lightweight verification head on real and synthetic hard negatives.The head scores query-candidate fused representations, using binary matching losses for positive pairs and hard negatives.Source: B. Progressive Learning Policy · p. 5
• Stage III integrates grounding and cross-view generation to improve candidate-level discrimination among highly confusable candidates.Synthetic negatives from the generation branch provide harder supervision than random negatives.Source: B. Progressive Learning Policy · p. 5

Plug-and-Play Inference

• UniGeo is plug-and-play at inference and compatible with arbitrary external retrieval backbones.It does not alter their original training protocol or initial retrieval pipeline.Source: C. Plug-and-Play Inference · p. 6

Initial Candidate Retrieval

• An external retrieval backbone first returns a top-K0 candidate pool without changing the original candidate-generation mechanism.UniGeo performs subsequent fine-grained reasoning on the retrieved candidates.Source: C. Plug-and-Play Inference · p. 6

Candidate-Conditioned Query Refinement

• Candidate-conditioned refinement supplements localization-relevant cues using the original query and initial candidate context.The refined-query candidates are unioned with the initial pool to form a shared candidate set.Source: C. Plug-and-Play Inference · p. 6

Shared-Pool Dual-Query Scoring

• UniGeo estimates refinement reliability from query consistency, candidate overlap, and newly introduced semantic content, then adaptively combines original and refined-query scores.The resulting score is the gated dual-query candidate score.Source: C. Plug-and-Play Inference · p. 6

Local Ambiguity Correction and Geo-Localization

• For a local shortlist, the verification head score is fused with the primary score before reranking; candidates outside the shortlist remain unchanged.The highest-scoring corrected candidate is selected as the geo-localization result.Source: C. Plug-and-Play Inference · p. 6

MLLMs

• UniGeo uses UniLIP as its underlying MLLM backbone and pairs it with the InternVL3-2B processor.The framework can be integrated with arbitrary external candidate-retrieval backbones.Source: A. Implementation Details · p. 6

Training

• Training follows a three-stage progressive strategy, beginning with region-level VQA fine-tuning and subsequent multi-task foundation training.The reported VQA setup uses one epoch, learning rate 5×10^-6, maximum sequence length 1024, batch size 1, and eight gradient accumulation steps.Source: A. Implementation Details · p. 6

Evaluation Metrics

• Evaluation covers geo-localization, text generation, and cross-view image generation.Geo-localization uses Recall@1, Recall@5, Recall@10, and mAP; generation metrics assess text quality, image quality, and pose control.Source: A. Implementation Details · p. 6

Reproducibility

• The authors state that code and model weights will be publicly released.Source: A. Implementation Details · p. 6

Geo-Localization Performance

• UniGeo consistently improves retrieval performance across GeoText-1652 and UAVReason with pre-trained, fine-tuned, and task-specific backbones.It is applied without modifying the underlying retrieval architectures.Source: B. Main Results · p. 7
• UniGeo mainly improves ranking and discrimination within the leading candidate set, with larger gains at R@5 and R@10 and consistent mAP improvements.On UAVReason, UniGeo raises HCCM R@10 from 63.90 to 64.71 and another baseline from 77.33 to 77.86.Source: B. Main Results · p. 8

Image Query

• Under image-query evaluation, UniGeo improves retrieval metrics on GeoText-1652 and UAVReason, indicating transfer of its candidate-level calibration across query modalities.For example, UAVReason XVLM-16M R@5/R@10 rises from 16.60/27.28 to 20.95/32.90, while mAP increases from 2.10 to 2.38.Source: B. Main Results · pp. 8–9

Query Refinement and Grounded Description

• UniGeo achieves the best reported region-grounded description quality, including Region-Match Acc@1 of 0.71 and BLEU-1/BLEU-4/F1 scores of 0.27/0.06/0.34.The results indicate stronger association between generated descriptions and target regions.Source: B. Main Results · p. 9, Table III · p. 9

Pose-Aware Cross-View Generation

• UniGeo produces stronger pose-aware cross-view generation, with FID/PSNR/SSIM of 41.34/13.12/0.16 in the reported reverse direction.Compared with UniLIP, FID decreases from 56.72 to 41.34 while PSNR and SSIM also improve; qualitative results show better topology, footprints, perspective, and spatial layout preservation.Source: B. Main Results · p. 10

Main mechanism ablation

• 14.56/32.32/42.44 R@1/R@5/R@10 results show hard-negative training substantially improves text-query localization over verification without hard negatives.Hard negatives expose the verifier to highly confusable candidates and strengthen fine-grained cross-view discrimination.Source: C. Ablation Studies and Further… · p. 10
• 29.32/56.80/69.57 R@1/R@5/R@10 results improve the image-query baseline through hard-negative verification learning, with more moderate gains than in text-query localization.The hard-negative variant and full model are identical for image queries because query refinement is not used.Source: C. Ablation Studies and Further… · p. 10

Hyperparameter sensitivity

• 14.72/33.80/45.37 R@1/R@5/R@10 is achieved at K = 16, while larger shortlists produce negligible changes and α = 0.1 performs best.Most competitive candidates are already covered by a compact local shortlist, and nearby α values remain comparable.Source: C. Ablation Studies and Further… · p. 10

Hard-negative supervision

• The Real+Syn 2:2 hard-negative mixture achieves the highest Real Acc. and lowest Gap while retaining competitive Syn Acc.The balanced mixture helps the verification head avoid overfitting to real retrieval errors or synthetic artifacts.Source: C. Ablation Studies and Further… · pp. 11–12

Pose-aware generation

• The full pose-aware generation objective achieves the lowest MeanIdxErr while maintaining competitive FID, PSNR, and SSIM.Diffusion reconstruction preserves image quality, pose consistency controls viewpoints, and heading supervision improves directional consistency.Source: C. Ablation Studies and Further… · p. 12

Efficiency and backbone compatibility

• UniGeo improves retrieval backbones without changing candidate generation, but requires approximately 2.6T FLOPs per query versus 106.42G for XVLM.The added overhead comes from dynamic refinement and candidate-level verification, while preserving the external retriever's high-recall process.Source: C. Ablation Studies and Further… · pp. 12–13

Qualitative analysis

• Heading-controlled generation preserves major geo-semantic elements across viewpoints, and generated samples provide harder supervision than random negatives for verification learning.The generated images are geographically plausible and semantically confusable rather than pixel-level reconstructions.Source: C. Ablation Studies and Further… · p. 13
• Region-level descriptions capture object categories, attributes, context, and spatial relations, supplying structured cues for subsequent geo-localization.UniGeo associates local region semantics with global aerial-scene context rather than recognizing isolated objects alone.Source: C. Ablation Studies and Further… · p. 13

Discussion

• UniGeo's larger gains on R@5 and R@10 than R@1 reflect region-level textual cues that rarely identify a unique target.Similar roads, buildings, and spatial layouts make exact top-1 localization difficult.Source: D. Discussion · pp. 13–14
• The framework preserves the original retrieval backbone and refines rankings among plausible candidates, promoting lower-ranked correct targets into the leading shortlist.This reduces the risk of missing correct locations during map inspection, human verification, or fine-grained localization.Source: D. Discussion · p. 14

Conclusion

• UniGeo integrates geo-semantic understanding, pose-aware cross-view generation, and candidate-level verification to mine fine-grained cues within the initial candidate pool.The pipeline addresses incomplete natural-language descriptions and visually similar geographic regions.Source: §V · p. 14
• Experimental results consistently improve multiple representative retrieval backbones on text-query geo-localization.Additional analyses support region-level understanding, controllable generation, and hard-negative verification as parts of the framework's geospatial reasoning capability.Source: §V · p. 14