GeoVista: Web-Augmented Agentic Visual Reasoning for Geolocalization

Motivation and approach

• GeoVista addresses the gap in image-centric agentic reasoning by integrating web search with visual tools for geolocalization.Prior systems emphasize image manipulation and lack suitable external information retrieval; geolocalization requires both visual clues and web search to validate or refine hypotheses.Source: §1 · pp. 1–2
• GeoBench provides globally distributed high-resolution photos and panoramas, excluding non-localizable images and easily recognized landmarks while supporting level-wise and nuanced evaluation.The benchmark is designed to assess geolocalization beyond trivial landmark recognition and non-localizable scenes.Source: §1 · pp. 2–3
• GeoVista actively chooses when and how to invoke image zoom and web search, using external information to validate hypotheses and correct earlier wrong ones.This combines visual clue extraction with information retrieval in a dynamic reasoning loop.Source: §1 · pp. 2–3
• Cold-start SFT uses tool proposals, rationales, and resulting observations serialized as multi-turn trajectories to teach tool-use and reasoning priors.The number of interaction turns is controlled by limiting tool invocation proposals.Source: §1 · p. 3
• RL with GRPO uses hierarchical rewards based on multi-level geographical labels to encourage learning hierarchical geographic context and more accurate judgments.This follows cold-start SFT in the proposed training pipeline.Source: §1 · p. 3
• The paper reports extensive GeoBench experiments and analysis to assess GeoVista’s effectiveness.The supplied introduction passage does not provide specific experimental outcomes.Source: §1 · p. 3

2.1 Thinking with Images

• Thinking with images progressed from localized visual reasoning steps and editable canvases to tool-mediated operations inside the reasoning chain.OpenAI o3 (OpenAI, 2025b) marked a watershed for this approach, followed by open replications.Source: §2.1 · p. 3
• After OpenAI o3 (OpenAI, 2025b), Thyme’s code-executed image operators, mini-o3’s iterative think–act region search, OpenThinkIMG’s detector/OCR/drawing controller, and DeepEyes’ RL-induced zoom extend visual reasoning beyond perception toward interactive, auditable tool use.Source: §2.1 · p. 3

2.2 Real-World Geolocalization

• Geolocalization research spans global photo retrieval and metric learning, landmark recognition, cross-view matching, large-scale street-scene datasets, human reasoning traces, and agentic tool use.Examples include Im2GPS, VIGOR (Zhu et al., 2020), OpenStreetView-5M (Astruc et al., 2024), and GeoComp (Song et al., 2025).Source: §2.2 · p. 4

3.1 Agentic Pipeline

• The policy iterates through thoughts, tool actions, and observations, using crop-and-zoom for image regions and web search for relevant information.The loop ends when the model gives a final answer or reaches its interaction-turn limit.Source: §3.1 · p. 4

3.2 GeoBench Benchmark

• Evaluation checks country, province or state, and city predictions, then geocodes predicted locations to calculate haversine distance from ground truth.The multi-level checks combine rule-based term matching with a model-based verifier.Source: §3.2 · pp. 5–6
• GeoBench includes 512 photos, 512 panoramas, and 108 satellite images across 6 continents, 66 countries, and 108 cities; it filters non-localizable images and iconic landmarks.The images are high-resolution, and each receives multi-level geographic annotation.Source: §3.2 · pp. 5, 7

3.3 Cold Start and Thinking Trajectory curation

• Because RL-only training led to overly concise answers and fewer tool calls, cold-start data teaches tool-use patterns through VLM-proposed regions, search queries, and reasoning trajectories.The curated set contains 2,000 trajectories and is not filtered by answer correctness.Source: §3.3 · p. 7

3.4 Reinforcement Learning

• GRPO uses correctness rewards, but city-only credit underuses hierarchical labels; the adopted tiers reward city, province or state, and country correctness, with β = 2.The paper reports that city-only rewards underperform on GeoBench and produce fewer tool calls.Source: §3.4 · pp. 7–8

4 Training Recipe

• SFT runs one epoch on approximately 2,000 cold-start examples; RL uses 12k examples, global batch size 64, and at most 6 turns.SFT uses a maximum context length of 32,768 and learning rate 1 × 10−5; RL uses a maximum context length of 32K tokens and learning rate 1 × 10−6.Source: §4 · p. 8

5.1 Settings

• The evaluation compares closed-source systems, open-source reasoning models including 7B peers, and a base model, using common tools for open-source models and multi-level and distance-based metrics.GeoBench reports country-, province-, and city-level accuracy, city accuracy by image type, the share of predictions within 3 km, and median haversine distance.Source: §5.1 · p. 9

5.2 Main Results

• GeoVista achieves state-of-the-art performance among open-source models across GeoBench metrics and performs on par with closed-source models on most metrics despite having far fewer parameters.Gemini-2.5-pro achieves the best overall performance among closed-source counterparts.Source: §5.2 · pp. 9–10, Table 2 · p. 9
• On the nuanced distance metrics, GeoVista substantially outperforms open-source image-reasoning models with the same tool access, while remaining slightly behind closed-source models.The metrics are the rate of predictions within 3 km and median haversine distance.Source: §5.2 · p. 10, Table 3 · p. 10

5.3.1 RQ1: The Ablation Study

• Removing cold-start SFT causes GeoBench performance to collapse, SFT alone is insufficient, and disabling the hierarchical reward in favor of city-level reward reduces performance.Table 4 summarizes ablations of cold-start SFT, RL, and hierarchical rewards.Source: §5.3.1 · p. 10, Table 4 · p. 10

5.3.2 RQ2: The Scaling Effect in RL Stage

• Across RL datasets of 1,500, 3 k, 6 k, and 12 k samples, performance on 512 validation panoramas improves consistently with an almost log-linear scaling effect.Figure 7 also shows a declining tool-failure rate during RL.Source: §5.3.2 · p. 11, Figure 7 · p. 11

5.3.3 RQ3: Failure Tool Calls during RL

• During RL, erroneous tool-call rates decline even without direct tool-call optimization; invalid crop boxes and incomplete JSON are common errors that can hinder correct answers within limited turns.Figure 7 plots the declining tool-failure rate.Source: §5.3.3 · p. 11, Figure 7 · p. 11

6 Conclusion

• GeoBench contains 1,142 high-resolution images from diverse global locations across three data types for evaluating real-world geolocalization.The task requires fine-grained visual clues and external knowledge.Source: §6 · p. 11
• GeoVista combines crop–zoom-in and web-search tools with curated trajectories, cold-start SFT, reinforcement learning, and a hierarchical reward.Source: §6 · p. 11
• GeoVista outperforms open-source baselines and performs comparably to GPT-5 and Gemini-2.5-flash on most metrics.The authors describe the work as a foundation for future research on agentic visual reasoning and real-world geolocalization.Source: §6 · p. 11

A Raw Data Collection

• GeoBench raw-data curation uses multiple image types to improve generalizability rather than fitting to a single type.Source: A Raw Data Collection · p. 12
• Internet-collected normal photos cover varied settings and typically have a resolution of at least 1600 × 1200; panoramas are stitched from global 360° street-view tiles into 4096 × 2048 images.Source: A Raw Data Collection · p. 12
• Satellite images are typically 2000 × 2000; recent Sentinel–2 Level–2A imagery for cities worldwide is mosaicked from several low-cloud scenes within each city’s bounding box.Multiple images are saved with metadata from different viewport variants.Source: A Raw Data Collection · p. 12

B Case Study

• Figure 9 includes one satellite-image example and one photo example from GeoBench.Source: Figure 9 · p. 13

• The assistant is instructed to answer the user’s question based on the provided image and may call one or more functions for assistance.Source: C System Prompt · p. 14
• Tool calls must use a JSON object naming the function and arguments inside <tool_call> tags, with internal reasoning placed first inside <think> tags.Source: C System Prompt · p. 14
• The image_zoom_in_tool crops a specified region using a bounding box [x1, y1, x2, y2] on the original image.Source: C System Prompt · p. 14
• The search_web tool executes a web search, returning normalized results with titles and snippets, and accepts a query string.Source: C System Prompt · p. 14
• The prompt illustrates zoom and search calls, then requires reasoning before the final answer, which is enclosed in <answer> tags.Source: C System Prompt · p. 14