PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics

Modi Jin, Yiming Zhang, Boyuan Sun, Dingwen Zhang, MingMing Cheng, Qibin Hou

arXiv:2602.12617v1cs.AI

TL;DR

Existing RL-based geolocation methods rely on AI-generated CoT data and reward strategies that may not align with geographic characteristics or human reasoning. GeoAgent addresses this with human-annotated GeoSeek data, geo-similarity and consistency rewards, and a two-stage training approach. It achieves strong performance across multiple geographic grains while producing more human-aligned reasoning, though its street-view focus limits indoor and low-cue scenarios.

  • Problem

    Existing methods rely on AI-generated CoTs and text-based geographic annotations or rewards that may not fully align with human reasoning and the non-unique relationship between language and locations.

  • Method

    GeoAgent trains on expert- and player-annotated GeoSeek data using SFT followed by GRPO with spatial-semantic geo-similarity and consistency rewards.

  • Results

    GeoAgent achieves outstanding performance across multiple benchmarks and exceeds other models on GeoSeek-Val, including country accuracy improving from 56.13 to 60.37.

  • Takeaways & Limitations

    Human geographic annotations and geographically aligned rewards support finer-grained geolocation outputs and reasoning intended to more closely align with human cognition.

  • Takeaways & Limitations

    Because its data comes from player communities, GeoAgent primarily targets street-view geolocation and performs poorly indoors or outdoors with minimal geographic cues.

Abstract

from arXiv · show

This paper presents GeoAgent, a model capable of reasoning closely with humans and deriving fine-grained address conclusions. Previous RL-based methods have achieved breakthroughs in performance and interpretability but still remain concerns because of their reliance on AI-generated chain-of-thought (CoT) data and training strategies, which conflict with geographic characteristics. To address these issues, we first introduce GeoSeek, a new geolocation dataset comprising CoT data annotated by geographic experts and professional players. We further thoroughly explore the inherent characteristics of geographic tasks and propose a geo-similarity reward and a consistency reward assessed by a consistency agent to assist training. This encourages the model to converge towards correct answers from a geographic perspective while ensuring the integrity and consistency of its reasoning process. Experimental results show that GeoAgent outperforms existing methods and a series of general VLLMs across multiple grains, while generating reasoning that closely aligns with humans.

1 Introduction

GeoAgent addresses limitations in AI-generated reasoning and text-equality rewards for geolocation by combining human-annotated GeoSeek data with geographically aligned rewards. The resulting method targets finer-grained, more interpretable, and more consistent geographic reasoning.

  • Motivation: Existing geolocation methods often rely on AI-generated CoTs and coarse annotations that may diverge from human reasoning.Traditional datasets commonly provide GPS coordinates or insufficiently fine-grained annotations, while AI-generated CoTs may amplify VLLM biases or superficial reasoning patterns.
  • GeoSeek Dataset: GeoSeek provides expert- and player-annotated reasoning at country, city, and precise-location granularity.The dataset also includes a benchmark using a more rational sampling strategy and supports human-understandable reasoning.
  • Training Rewards: Geo-similarity combines spatial and semantic similarity because different natural-language descriptions can identify the same geographic location.Spatial similarity uses the distance between actual and predicted locations, while semantic similarity compares predictions with ground truth step by step textually.
  • Training Rewards: The consistency reward uses a dedicated consistency agent to preserve the integrity and consistency of GeoAgent’s CoT.The agent derives answers from the reasoning process without task-specific prior knowledge, incentivizing reasoning that better supports the model’s conclusions.
  • Contributions: GeoAgent is designed to improve geolocation performance, geographic output granularity, and CoT quality.Its training pipeline uses GeoSeek-CoT and GeoSeek-Loc within a GRPO-based training process.

2 Related Work

Image geolocation has evolved from classification and retrieval toward VLLM-based reasoning, but existing datasets and methods still provide incomplete support for human-like, fine-grained reasoning. GeoAgent’s context builds on this shift while addressing missing reasoning annotations and geographic detail.

  • Image Geolocation: Traditional image geolocation frames the task as classification or retrieval, whereas newer methods use VLLMs to produce locations and reasoning.Classification assigns images to geographic grid cells, while retrieval uses images as queries against databases.
  • VLLM-Based Methods: Recent GRPO-based VLLM methods improve performance and alignment with human thinking but still face challenges from AI-generated reasoning.The related work positions these methods as extensions of VLLM-based geolocation rather than replacements for the underlying task.
  • Geolocation Datasets: Existing datasets exhibit regional bias, low locatability, or insufficient reasoning and fine-grained location annotations for RL-based approaches.AI-annotated CoT datasets offer performance gains, but their missing fine-grained annotations remain a limitation.

3 GeoSeek Dataset

GeoSeek addresses coarse, AI-generated reasoning and geographic sampling bias through human-labeled fine-grained data and stratified sampling. It includes expert reasoning data, globally distributed street-view samples, and a dedicated validation benchmark.

  • Dataset motivation and design: GeoSeek combines human-labeled CoT, fine-grained locations, and stratified sampling to improve reasoning quality, annotation granularity, and sampling balance.Its sampling strategy considers population, land area, and road mileage.
  • GeoSeek-CoT: 10k CoT examples were constructed jointly by geographic experts and experienced geolocation players, using images, GPS coordinates, and three-level location reasoning.Additional verified reasoning data came from GeoGuessr and TuXun communities.
  • GeoSeek-Loc: 20,000 high-resolution street-view samples form GeoSeek-Loc through hierarchical country and grid-cell sampling weighted by population, land area, and highway mileage.Logarithmic population weighting reduces excessive concentration, producing a globally distributed dataset.
  • GeoSeek-Val: GeoSeek-Val adds 3k OSV5M samples using the same stratified sampling strategy to evaluate street-view localization.Samples include GPT-4o locatability scores from 0 to 10 and scene categories based on geographic elements.
  • Geographic reasoning consistency: Figure 3 contrasts incomplete, inconsistent CoT with consistent CoT after consistency-agent training, while showing that different answers can denote the same location.The examples motivate handling geographic equivalence beyond exact text matching.

4 Methodology

GeoAgent combines human-annotated reasoning data with geographic rewards that measure spatial and semantic closeness, then uses a consistency agent to preserve coherent chain-of-thought reasoning. Its reward design progressively guides predictions from broad geographic areas toward precise locations while discouraging incomplete reasoning.

  • Pipeline: GeoAgent uses a two-stage pipeline: GeoSeek-CoT cold-start fine-tuning followed by GRPO reinforcement learning with geo-similarity rewards.The geo-similarity reward combines spatial and semantic similarity, while the consistency reward supports reasoning quality.
  • GRPO Training: GRPO samples G candidate replies, scores them with verifiable rewards, computes normalized within-group advantages, and applies a clipped PPO-like update.The same advantage is broadcast across each reply’s tokens, and ε limits update size for stability.
  • Geo-Similarity Reward: The geo-similarity reward combines spatial distance and semantic similarity because different address descriptions can refer to the same geographic location.Spatial similarity uses inverse geocoding and a nonlinear distance-based reward, while semantic similarity handles aliases, abbreviations, and translation variations through hierarchical address matching.
  • Geo-Similarity Reward: Semantic similarity trends closely with spatial similarity, unlike directly-judged text equality, whose reward is inconsistent with geographic distance.This supports semantic similarity as a more appropriate signal for geolocation training than exact textual matching.
  • Consistency Reward: The consistency agent evaluates GeoAgent’s reasoning without access to its conclusions, encouraging geographic clues and conclusions to remain connected across reasoning levels.Its reward includes granularity weights and a length-related penalty that discourages overly simplistic reasoning such as outputting only a final conclusion.
  • Consistency Reward: Consistency reward converges first; after convergence, spatial and semantic rewards increase and surpass the model without consistency reward.Before convergence, both similarity rewards remain lower than in the comparison model.

5 Experiments

GeoAgent is evaluated on public and newly constructed benchmarks, where it improves geolocation accuracy, especially at coarse geographic levels, and produces more coherent reasoning. Ablations attribute gains to geo-similarity, semantic, consistency, and cold-start training components.

  • Evaluation settings: GeoAgent uses LoRA fine-tuning on Qwen2.5-VL-7B, while a GPTQ-INT4 Qwen3-32B model serves as the consistency agent.The model uses LoRA with rank 64 and alpha 128; evaluations use IM2GPS3K and GeoSeek-Val.
  • Performance on Public Benchmarks: 76.21 versus 72.40 country accuracy on IM2GPS3K demonstrates GeoAgent’s improvement over the compared baseline with only 1.91% trainable parameters.The paper reports larger gains at macro geographic levels than at fine levels, attributing this to the distance-dependent geo-similarity reward.
  • Performance on GeoSeek-Val: 60.37 versus 56.13 country accuracy on GeoSeek-Val shows GeoAgent outperforming other models across locatability and geographic-element splits.The authors associate larger gains on highly locatable images with reasoning patterns closer to human thinking.
  • Dataset Quality: GeoAgent with only SFT surpasses other SFT-only methods despite using 10K data points instead of 20K, 133K, or 5M.The paper uses this comparison to demonstrate the quality of GeoSeek-CoT annotations from geographic experts and geolocation players.
  • Ablation study of components: Rspa, Rsem, and Rcon each improve performance, while Rcon alone slightly decreases performance but helps at regional and city levels when combined with the other rewards.Rspa contributes more than Rsem because it provides a direct distance-dependent signal; Rcon addresses consistency rather than geolocation capability directly.
  • Cold Start: 47.12 versus 11.13 country accuracy shows that GeoSeek-CoT cold-start SFT substantially improves the base model’s performance.The cold start establishes a reasoning framework aligned with human thinking patterns; models without it experience performance declines.
  • Discussion of Geo-Similarity and Directly-Judge: 60.37 versus 50.81 country accuracy shows that geo-similarity rewards improve more than directly judging text identity.Geo-similarity accounts for geographic distance and semantic similarity, rewarding progress toward correct answers despite aliases or cross-language descriptions.
  • Reasoning comparison: GeoAgent produces clearer hierarchical reasoning than the compared general VLLMs and its SFT-only variant.After GRPO training, it better identifies geographic features and reduces incomplete or inconsistent reasoning.

6 Conclusions

GeoAgent combines human-annotated GeoSeek data with two-stage SFT and GRPO training, using rewards tailored to geographic similarity and reasoning consistency. The paper concludes that this approach delivers strong performance across multiple benchmarks while supporting human-like geographic reasoning.

  • 6 Conclusions: GeoSeek provides chain-of-thought annotations from geographic experts and geolocation game players.The dataset supports the paper’s focus on human-aligned, fine-grained geolocation reasoning.
  • 6 Conclusions: GeoAgent uses two-stage SFT and GRPO fine-tuning together with geo-similarity and consistency rewards.The rewards address non-unique location descriptions and consistency between reasoning and conclusions.
  • 6 Conclusions: GeoAgent achieves outstanding performance across multiple benchmarks while providing geographic locations through human-like reasoning.The conclusion summarizes the reported performance and reasoning objectives without narrowing them to a single benchmark.

A More Implementation Details

The implementation uses standard optimization and hardware settings, evaluates established geolocation benchmarks, and constructs GeoSeek through hierarchical sampling based on geographic and population-related factors.

  • Implementation details: Training uses AdamW with an initial learning rate of 1e-5, DeepSpeed ZeRO-3, and eight NVIDIA A40 GPUs; evaluation uses two A40 GPUs.Training and testing are performed on CentOS 7.
  • Implementation details: The appendix documents prompts for the SFT, GRPO, locatability-scoring, and category-annotation stages.These prompts are presented in Figures 7 and 8.
  • Evaluation resources: IM2GPS3K contains 3,000 geotagged images sampled from a six-million-image Flickr collection and is used across geographic thresholds.It is a subset of IM2GPS and a common benchmark for image geolocation.
  • Sampling strategy: GeoSeek sampling weights countries using population, land area, and highway mileage before applying grid-based sampling.The reported coefficients are λ1 = 0.5, λ2 = 0.2, and λ3 = 0.3.
  • Experimental documentation: Tables 5 and 6 report hyperparameter values and SFT results for different base models on GeoSeek-Val.The supplied passages identify the table contents but do not provide their individual entries.
  • Sampling strategy: Each country is divided into a 360 × 180 longitude–latitude grid with 1° × 1° cells, and cell probabilities use logarithmic population adjustment.The adjustment increases coverage of populated regions without oversampling urban clusters.

B.2 Data Sources

GeoSeek combines human reasoning annotations, globally distributed street-view samples, and a stratified validation set, with reverse geocoding and GPT-4o-based image analysis supporting dataset construction.

  • Data sources: GeoSeek consists of GeoSeek-CoT, GeoSeek-Loc, and GeoSeek-Val.The three components support reasoning supervision, location data, and evaluation.
  • GeoSeek-CoT: GeoSeek-CoT contains 10,000 chain-of-thought examples, including about 6,000 human annotations and 4,000 examples from public geolocation guides.The human annotations come from geographic experts and volunteer geolocation players.
  • GeoSeek-Loc: GeoSeek-Loc contains 20,000 globally distributed high-resolution street-view samples, combining 12,500 OSV5M samples with 7,500 GeoComp samples.Both sources are sampled using the multi-level hierarchical strategy.
  • GeoSeek-Val: GeoSeek-Val selects 3,000 OSV5M samples using the stratified sampling algorithm.The validation set also obtains structured addresses and analyzes locatability and geographic elements.
  • GeoSeek-CoT: GeoSeek-CoT annotations provide country, region, and precise-location resolutions, with country-level labeling mandatory and finer reasoning left to volunteer judgment.The annotation process is based on images’ locatability.
  • Address construction: OpenCage reverse geocoding converts GeoSeek-Loc images into structured three-level addresses.The same address format is used when configuring GeoSeek-Val.

B.4 Comparison with AI-annotated Data

The comparison presents human-annotated and standardized GeoSeek reasoning alongside directly AI-annotated CoTs. Human-annotated data yields more accurate and sound reasoning, while GeoAgent produces geographically relevant, stepwise, fine-grained predictions.

  • Human-cleaned CoT data significantly outperforms directly AI-annotated data in both accuracy and reasoning soundness.Figure 9 compares volunteer reasoning, standardized CoTs, and direct AI annotations.
  • GeoSeek-CoT contains human-annotated high-quality reasoning, while GeoSeek-Loc provides locatability and geographically annotated clues.
  • Different base models show significant performance improvements after GeoSeek-CoT’s supervised fine-tuning cold start.The result supports the effectiveness of GeoSeek-CoT’s human-annotated CoT data.
  • GeoAgent captures geographically relevant features, reasons stepwise, and provides fine-grained locations on IM2GPS3K and GeoSeek-Val examples.
  • GeoAgent failures primarily arise when images contain too few locatable features or features that could occur in multiple locations.

D.3 Robustness of the Model

Robustness tests mask different geographic cues to assess whether GeoAgent can continue inferring locations from the remaining visual evidence.

  • GeoAgent continues inferring locations after utility poles, text, shops, or foreground elements are masked.The model also discovers new cues to support inference under these masking conditions.

E Discussion

GeoAgent uses hierarchical geographic reasoning to derive fine-grained locations and has practical relevance for geolocation applications. Its scope remains constrained by street-view-focused data, weak visual cues, and privacy concerns.

  • E Discussion: GeoAgent performs hierarchical reasoning from geographic clues to fine-grained geographic locations.
  • E Discussion: The model is intended to provide guidance for geolocation players and has practical application value.
  • E Discussion: Image geolocation can support criminal tracking, emergency response, social media, and cultural dissemination when image metadata is unavailable.
  • E Discussion: Geolocation can serve as an alternative to GNSS when satellite signals are unavailable.
  • E Discussion: The dataset’s player-community sourcing emphasizes street-view geolocation, limiting performance in indoor scenes and outdoor settings with minimal geographic cues.
  • E Discussion: Geolocation models require careful use because they may infringe personal privacy or assist criminal activities.
  • E Discussion: The examples illustrate country, regional, and precise-location reasoning linked to architectural, infrastructural, cultural, and environmental clues.Examples include Spain–Catalonia–Barcelona, Canada–Quebec–Montreal, and Indonesia–Bali–Bandung predictions.
Loading 2602.12617v1…