PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoCloak: An Agentic Framework for Semantics-Preserving Geolocation Privacy via Peripheral Clue Manipulation

Yixin Wu, Mingjie Li, Yun Shen, Michael Backes, Yang Zhang

arXiv:2610.20055v1cs.CVcs.AIcs.CR

TL;DR

LVLMs make image geolocation inference accessible, while existing image-level defenses can introduce visual artifacts and utility loss. GeoCloak uses image-specific semantic editing and utility-aware perturbation to protect peripheral clues, reducing city-level accuracy across its evaluation.

  • Problem

    LVLMs can infer locations from image clues, while existing image-level defenses rely mainly on perturbations that can introduce visual artifacts and utility degradation.

    Source: §1 · p. 1

  • Method

    GeoCloak identifies image-specific geolocation clues, edits peripheral clues, and iteratively configures adversarial perturbation under utility-aware feedback.

    Source: §1 · p. 2

  • Results

    City-level accuracy falls from 41.2% to 9.9%, compared with 24.5% for M-Attack and 19.7% for GeoShield.

    Source: §1 · p. 2

  • Takeaways & Limitations

    An interactive release lets users control protection according to their privacy preferences.

    Source: §1 · p. 2

    GeoCloak’s effectiveness depends on the underlying LVLM’s visual reasoning capability.

    Source: §6 · p. 12

Abstract

from arXiv · show

The rapid advancement of large vision language models (LVLMs) has made geolocation inference from images highly accurate and accessible. This poses serious privacy risks: a passive adversary with black-box LVLM access can infer a user's location from a single photo with no metadata. Existing protection methods rely almost exclusively on adversarial perturbation, which introduces perceptible visual artifacts and utility degradation. We identify the visual-geolocation asymmetry: geolocation-decisive clues, such as street signs, tend to be visually peripheral but dominate location inference. Exploiting this asymmetry, we propose GeoCloak, an agentic framework for semantics-preserving geolocation privacy protection. It acts as a clue-aware planner that analyzes each image to identify geolocation-relevant clues, dispatches targeted semantic edits to directly suppress peripheral clues, and autonomously configures adversarial perturbation to further suppress residual geolocation features, all under iterative utility-aware feedback. We evaluate GeoCloak on four diverse datasets spanning 108 countries and 576 cities against five LVLMs and three inference methods. GeoCloak reduces overall city-level accuracy from 41.2% to 9.9%, approximately half of the best-performing baseline, while achieving substantially superior utility preservation both qualitatively and quantitatively. We hope this work takes a meaningful step toward practical, semantics-preserving geolocation privacy protection, giving users greater control over the location information inadvertently embedded in their shared images.

1 Introduction

LVLMs make image geolocation inference accessible, while existing image-level defenses can degrade visual utility. GeoCloak exploits the visual-geolocation asymmetry through image-specific semantic edits and perturbations, reducing geolocation accuracy while preserving image quality.

1 Introduction

  • Existing image-level defenses use fixed perturbation pipelines and post-hoc utility evaluation, which can introduce perceptible artifacts and degrade image utility.Model-level moderation and refusal depend on provider enforcement and can be bypassed by using an unguarded model.Source: §1 · p. 1
  • Geolocation-decisive clues, such as street signs, can be visually inconspicuous even when they reveal locations users did not intend to share.The paper calls this visual-geolocation asymmetry and motivates protection focused on peripheral clues.Source: §1 · p. 2
  • GeoCloak combines targeted semantic editing of peripheral clues with utility-aware iterative adversarial perturbation for residual scene-level features.A clue-aware planner selects image-specific tools and iteratively evaluates the privacy-utility tradeoff, using metrics such as CLIP similarity.Source: §1 · p. 2
  • City-level accuracy falls from 41.2% to 9.9%, compared with 24.5% for M-Attack and 19.7% for GeoShield.Across four datasets spanning 108 countries and 576 cities, GeoCloak also reduced accuracy from 51.3% to 13.8% against the strongest evaluated attacker, Gemini-3.1-Pro under zero-shot inference.Source: §1 · p. 2
  • GeoCloak achieves a CLIP similarity of 0.9197, compared with 0.7719 for M-Attack and 0.4209 for GeoShield.In a human perception study, it received an overall quality rating of 4.43/5, versus 2.63 for M-Attack and 2.53 for GeoShield.Source: §1 · p. 2
  • GeoCloak also releases an interactive version that lets users control protection according to their privacy preferences.The evaluation targets images whose locations can be inferred from subtle clues users did not intend to reveal.Source: §1 · p. 2

2 Related Work

Prior geolocation work uses LVLM prompting, retrieval, tools, and geographic post-training to improve inference. Protection methods operate at model or image level, with image-level defenses dominated by adversarial perturbation.

2 Related Work

  • LVLM geolocation methods include prompting, retrieval augmentation, tool-using agents, and post-training with geographic reward signals [21] [25].Prompting ranges from zero-shot queries [33] [35] to structured chain-of-thought [26] [45], while agentic systems may gather evidence using tools such as web search or street-view navigation [28] [56].Source: §2 · p. 2
  • Prior retrieval-based [5] [48] and classification-based methods [16] [51] have been shown to underperform LVLMs despite requiring millions of labeled training images [33] [35] [45].The related work also describes end-to-end post-training pipelines that combine tool calling and geographic reasoning [19] [49].Source: §2 · pp. 2–3
  • Image-level defenses mainly use targeted perturbations to steer predictions [55] or untargeted perturbations to suppress geolocation [32] [54].Model-level defenses include output moderation [35] and prompt-based refusal, but depend on service-side deployment; image-level methods use post-hoc rather than active utility constraints, risking visible artifacts and utility degradation.Source: §2 · p. 3

3 Threat Model

The threat model considers a user protecting an image before public sharing against an attacker who has access only to the protected image. GeoCloak requires the image’s ground-truth location and scopes evaluation to images with subtle clues rather than visually dominant, globally recognizable landmarks.

3 Threat Model

  • The attacker seeks city-, state-, or country-level inference from the protected image alone, while the defender is a user who wants to share an image without revealing its location.The user provides the image and its ground-truth location to GeoCloak; when external APIs process these inputs, their providers are within the trust assumption.Source: §3 · p. 3
  • GeoCloak’s privacy goal is to prevent correct geolocation inference while its utility goal is to preserve semantic content and visual quality.The ground-truth location serves as the protection loop’s termination condition; protection is declared successful when GeoCloak can no longer infer it correctly.Source: §3 · p. 3
  • Evaluation excludes images with globally recognizable, visually dominant landmarks because they may reflect an intention to share location information.The scope instead targets images whose locations are inferred from subtle, inconspicuous clues users did not intend to reveal.Source: §3 · p. 3

4 GeoCloak

GeoCloak iteratively protects location privacy while preserving visual utility, using clue analysis, targeted editing, and adversarial perturbation. Its agents adapt edits and stop when geolocation is disrupted at the target granularity or the iteration limit is reached.

4.1 System Overview

  • GeoCloak alternates utility and protection checks, rolling back edits that fall below the utility threshold and stopping when the target location can no longer be inferred or the iteration limit is reached.If the limit is reached, the strongest achieved protection level is reported; if no protection is achieved, a fallback adversarial perturbation is applied.Source: §4.1 · pp. 3–4

4.2 ClueAgent

  • ClueAgent identifies and ranks location-revealing clues, predicts a location from them, and uses its prediction as the editing-loop stopping condition.If it cannot geolocate the original image, the system fixes GT-guided mode for the run, supplying the ground truth to help identify residual clues.Source: §4.2 · pp. 4–5

4.3 EditAgent

  • EditAgent combines semantic editing for discrete, localizable clues with gradient-based perturbation for large or structural clues that resist editing.Its five editing strategies are ReplaceText, AddDecoy, ReplaceObject, EraseObject, and StyleTransfer.Source: §4.3 · p. 5
  • Semantic edits prioritize clues with high localization scores relative to their area, while aiming for natural-looking changes, cultural plausibility, and preservation of the main subject.After a failed edit, EditAgent retries with a revised prompt before switching strategies.Source: §4.3 · p. 5
  • For perturbation, EditAgent selects a clue-region or full-image reference and a loss configuration, then adapts hyperparameters to the observed privacy-utility tradeoff.When utility is too low it can tighten the perturbation; when protection is insufficient and utility passes, it can increase its strength.Source: §4.3 · pp. 5–6

5 Evaluation

Across four datasets spanning 108 countries and 576 cities, GeoCloak is evaluated against diverse geolocation attackers and protection methods. It reduces overall city-level accuracy to 9.9% while preserving image utility better than the compared defenses.

5.1 Experimental Setup

  • Evaluation covers 1,779 images across four datasets, 108 countries, 576 cities, and 11 attacker configurations, using city-level accuracy as the primary metric.The benchmark includes zero-shot prompting, GeoMiner, and GeoVista; the primary GeoCloak configuration uses Gemini-2.5-Pro and Gemini-2.5-Flash-Image.Source: §5.1 · pp. 6–7

5.2 Evaluation Scoping

  • Exempt images with intentional global landmarks are excluded; across datasets, city-level accuracy drops from 73.0% to 48.3% on the non-exempt subset.The automated landmark-intention pipeline has 86.7% precision, and the authors note that user intent remains subjective.Source: §5.2 · p. 7, Table 2 · p. 7

5.3 Effectiveness

  • 9.9% overall city-level accuracy with GeoCloak, versus 19.7% for GeoShield and 24.5% for M-Attack, is the best protection across datasets and attackers.Against Gemini-3.1-Pro under zero-shot inference, accuracy falls from 51.3% to 13.8%.Source: §5.3 · p. 8, Table 3 · p. 8

5.4 Utility

  • GeoCloak receives a 4.43/5 overall human quality rating and achieves 0.9197 CLIP similarity, exceeding M-Attack and GeoShield on these measures.It also leads on BERT score (0.9158), BLEU (0.1136), and ROUGE-L (0.4054); users can specify regions that must remain unmodified in the interactive system.Source: §5.4 · p. 9, Table 5 · p. 9, Table 6 · p. 9

5.5 Skill Usage

  • Erase Object leads skill invocations (24.9%) and utility preservation (85.6%); successful strategies shift from removal at city level to recontextualizing edits at country level.The figure reports invocation frequency and utility preservation, and separates skills used at each exact successful protection level.Source: §5.5 · pp. 9–10, Figure 5 · p. 10

5.6 Design Choices of GeoCloak

  • On Im2GPS3k, semantic editing alone yields 5.1% city accuracy, the full system 4.0%, and the system without semantic editing 15.4%.Adaptive guidance leaves 2.9% of attackers successful, versus 11.8% for multi-agent debate. Other backbones also work: o3 plus Seedream 4.5 reaches 6.3%, and the fully open-source configuration reaches 7.4%.Source: §5.6 · pp. 10–11, Table 7 · p. 10

5.7 Cost and Efficiency Analysis

  • Median cost is $0.32 per image and median elapsed time is 8.3 minutes; country-level protection takes a median of 5 turns, versus 2 for city and state.The estimated median cost falls to approximately $0.16 per image with the Gemini Batch API.Source: §5.7 · p. 11, Figure 6 · p. 11

5.8 Adaptive Attacks

  • Conflict-aware adaptive prompting raises success on protected images to 15.9% from 11.1%, but GeoCloak still reduces its performance from 41.6%.Fine-tuning only partially recovers accuracy: GeoCLIP rises from 7.4% to 9.6%, and Qwen3.5-9B from 6.4% to 11.3%; JPEG compression provides little benefit, while DiffPure degrades image utility.Source: §5.8 · pp. 11–12

6 Discussion, Limitation, and Future Work

GeoCloak’s discussion argues that image-specific clues require adaptive reasoning and feedback, while noting limits in prompt optimization and dependence on LVLM capability. It also describes user controls and manual checks of sampled outputs.

The Necessity of Our Agent Architecture

  • GeoCloak’s image-specific clues make static strategies insufficient; adaptive reasoning, dynamic strategy selection, and iterative feedback support privacy and utility together.In Figure 7, two failed shop-sign edits are rolled back before a sofa edit shifts the prediction from Morocco to Algeria.Source: §6 · p. 12

Prompt Sensitivity

  • Systematic prompt optimization remains future work because prompts interact dynamically across agents and feedback loops in GeoCloak’s multi-turn setting.Paraphrasing prompts without changing the agents had negligible impact on protection performance.Source: §6 · p. 12

User-Centered Agentic System

  • The interactive version lets image owners specify objects to preserve and accept, roll back, or stop edits, moving beyond proxy-based intent and utility checks.Because ground-truth intent is unavailable in existing datasets, evaluation approximates it with visual landmark analysis.Source: §6 · p. 12

Manual Inspection of GeoCloak’s Outputs

  • Authors manually inspected random samples of intermediate trajectories for successful edit execution and decisions consistent with privacy and utility feedback.By the final iteration, sampled trajectories generally showed successful edits and coherent multi-turn refinement.Source: §6 · p. 12

LVLM Capability

  • GeoCloak’s effectiveness depends on LVLM visual reasoning; the open-source Qwen configuration has a 7.4% attack success rate versus 4.0% for the primary Gemini configuration.A final GT-guided audit could check for residual clues, but its added protection and processing overhead remain unevaluated.Source: §6 · p. 12

7 Conclusion

GeoCloak combines targeted editing and adversarial perturbation in an adaptive, utility-aware agentic framework. The conclusion reports stronger effectiveness and utility than baselines and describes an interactive version for user participation.

7 Conclusion

  • GeoCloak adaptively combines targeted editing and adversarial perturbation under utility-aware feedback to protect geolocation privacy.The approach is motivated by visual-geolocation asymmetry: decisive clues tend to be visually peripheral.Source: §7 · p. 13
  • Extensive evaluation demonstrates that GeoCloak outperforms baselines in both protection effectiveness and utility.Source: §7 · p. 13
  • An interactive version of GeoCloak allows users to participate directly in the protection process.The conclusion frames this as a step toward practical, user-intent-aware geolocation privacy protection.Source: §7 · p. 13

Open Science

This material describes the released interactive artifact, ethical considerations, and supplementary examples and prompts. The examples show multi-turn edits and semantic-editing strategies intended to redirect location predictions with subtle changes.

Open Science

  • The interactive GeoCloak code is publicly accessible, and the evaluation uses publicly available Im2GPS3k, GeoBench, Dataset-GSS, and Dataset-PCW datasets.Evaluated LVLMs are accessed through their official APIs.Source: Open Science · p. 15

Ethical Considerations

  • The authors identify possible deceptive misuse of image editing, while noting GeoCloak is optimized for subtle, semantics-preserving edits.Human annotation tasks involved no sensitive data collection, deception, or personally identifiable content.Source: Ethical Considerations · p. 15

B Prompt Templates

  • The appendix provides prompt templates for ClueAgent, EditAgent, semantic editing, adversarial perturbation, and evaluation scoping.Source: B Prompt Templates · p. 15

C Examples of Semantic Editing Skill

A Examples of Multi-Turn GeoCloak Protection Example

  • In the Figure 7 example, a failed shop-sign edit is rolled back before a sofa replacement redirects the prediction from Rabat to Oran with minimal visual change.An earlier license-plate replacement shifts the prediction from Rabat to Casablanca.Source: A Examples of Multi-Turn GeoCloak… · p. 15, Figure 7 · p. 16

D.1 Evaluation Datasets

The evaluation uses four geographically broad datasets and describes three inference paradigms alongside reproduced defense baselines. This setup spans varied geographic contexts and includes both prompt-based and tool-augmented attacks.

D.1 Evaluation Datasets

  • The four-dataset benchmark contains 1,779 images spanning 108 countries and 576 cities, with Dataset-PCW drawn from no publicly available sources to test unseen images.Im2GPS3k and Dataset-GSS are randomly sampled at 500 images each; all available GeoBench and Dataset-PCW data are used.Source: §D.1 · p. 16

D.2 Geolocation Inference Methods

  • The evaluation describes Zero-Shot prompting, GeoMiner’s two-stage Detector–Analyzer clue pipeline, and GeoVista’s iterative web-augmented agent with image-zoom and web-search tools.GeoMiner and GeoVista follow their official GitHub implementations with default settings.Source: §D.2 · pp. 16–17

D.3 SOTA Methods for Comparison

  • GeoShield and M-Attack are reproduced from released code with original hyperparameters; GeoShield uses GroundingDINO for revealing regions, while M-Attack samples target images from other locations in the same dataset.The listed perturbation setup includes ε = 8/255, α = 1/255, and 100 PGD steps.Source: §D.3 · p. 17

E Fine-Grained Complexity and Failure-Mode Analysis

Raw clue count does not track geolocation difficulty monotonically, so the analysis instead stratifies images by initial geo-identifiability. GeoCloak leaves the fewest successful attackers in every group, though its advantage is less pronounced for low-identifiability images.

Clue-count analysis

  • Clue count has no monotonic relationship with initial attack success: images with five clues average 6.76 successful attackers, while those with seven average 5.12.The authors conclude clue specificity and discriminative strength matter more than raw clue count, making clue count an unreliable proxy for geolocation difficulty.Source: Table 11 · p. 17, E Fine-Grained Complexity and… · p. 17

Initial geo-identifiability

  • Initial geo-identifiability is the number of attacker configurations that geolocate an image at city level without defense, ranging from 1 to 11.The configurations span zero-shot and GeoMiner inference with five LVLMs plus GeoVista; images are grouped into low, medium, and high identifiability.Source: E Fine-Grained Complexity and… · p. 18
  • For highly geo-identifiable images, GeoCloak reduces mean successful attackers from 9.84 to 2.75, versus 5.69 for GeoShield and 6.91 for M-Attack.Across low-, medium-, and high-identifiability groups, GeoCloak leaves the fewest successful attackers and yields the largest attacker reduction; its advantage is less pronounced for low-identifiability images.Source: Table 12 · p. 18, E Fine-Grained Complexity and… · p. 18

Failure-mode analysis

  • Weak or ambiguous geographic evidence in low-identifiability images makes actionable clues harder for ClueAgent to identify and leaves less evidence for suppression.Despite this constraint, GeoCloak still leaves the lowest mean number of successful attackers in that group.Source: E Fine-Grained Complexity and… · p. 18

F Evaluation Results Measured by Multiple Metrics

Across the reported level-wise and distance-based accuracy metrics, GeoCloak achieves better protection performance than the two baselines.

Evaluation results

  • GeoCloak achieves better protection performance than both baselines on level-wise and distance-based accuracy metrics.Table 13 aggregates geolocation accuracy under no defense and each defense across all datasets; lower is better for every metric.Source: F Evaluation Results Measured by… · p. 18

G Visual Quality Comparison

The visual comparisons report negligible utility degradation for GeoCloak relative to GeoShield and M-Attack, whose outputs show prominent artifacts. Additional cases where adversarial perturbation is key to successful protection show the same visual-fidelity contrast.

Figure 10

  • In additional cases where all three defenses successfully use adversarial perturbation, GeoCloak maintains high visual fidelity while GeoShield and M-Attack show perceptible artifacts.Figure 10 presents one such case per row, drawn from different datasets.Source: G Visual Quality Comparison · p. 18

Figures 8–9

H Human Perception Analysis

The human evaluation compares blinded protected-image outputs with their originals using two annotators and three five-point ratings. Scores are averaged by source-image–defense pair, with image-level bootstrap confidence intervals reported in Table 14.

Evaluation setup

  • The study samples 100 source images equally across four datasets and creates 300 blinded rating tasks, with two independent annotators evaluating each trial.Each trial presents the original image alongside one protected image from GeoCloak, GeoShield, or M-Attack.Source: H Human Perception Analysis · p. 18

Rating criteria

  • Annotators rate naturalness, semantic preservation, and overall quality on a five-point scale, with overall quality designated as the primary metric.The scale runs from 1 for severe and unacceptable problems to 5 for a natural, complete image with almost no visible problems.Source: H Human Perception Analysis · p. 18

Analysis

  • The analysis averages annotators’ scores for each source-image–defense pair and treats the source image, not repeated ratings, as the sampling unit.The study estimates 95% confidence intervals by bootstrapping 100 source images with 10,000 resamples.Source: H Human Perception Analysis · pp. 18–19

Results

  • Table 14 reports mean ratings and image-level bootstrap 95% confidence intervals for the human-perception measures; higher is better.The supplied table caption does not state the rating values or comparisons.Source: Table 14 · p. 19

I Invalid Responses in Evaluation

This section describes how invalid evaluation responses are handled and presents an interactive GeoCloak version with turn-by-turn control over privacy edits, including clue-editing and adversarial-perturbation choices. It also outlines a tiered analysis for determining whether a boxed object is an intentional landmark or incidental.

I Invalid Responses in Evaluation

  • Under GeoMiner, o3 has a 28.1% safety-refusal rate with no defense; Qwen3.5-Plus also has API-level failures, while other models have a 0.0% invalid-response rate.Initial invalid responses are excluded from post-defense evaluation and counted as localization failures across all metrics; invalid responses count as failures for every defense method.Source: I Invalid Responses in Evaluation · p. 19, Table 15 · p. 19

J Interactive GeoCloak

  • The interactive version lets users specify objects to preserve and a city-, state-, or country-level protection target, then accept, roll back, or stop after each proposed edit.It replaces automated utility checks with direct human judgment and transparent, turn-by-turn control.Source: J Interactive GeoCloak · p. 20, Figure 11 · p. 21
  • The agent chooses semantic editing for discrete clues and adversarial perturbation for large landmarks or scene-level features, retrying before switching strategies or skills.Editing options include replacing text or objects, erasing clues, adding decoys, and applying style transfer to peripheral elements.Source: J Interactive GeoCloak · pp. 22–23
  • Edits should blend naturally, prioritize small high-importance clues, and preserve the main subject; the agent checks utility and attack results and escalates after failed edits.The interactive workflow permits up to two retries within a strategy before switching strategies, then changes skills only after exhausting multiple strategies.Source: J Interactive GeoCloak · p. 23
  • GeoCloak’s interactive edits can replace or erase location-specific objects, add physically plausible decoys that recontextualize nearby clues, or shift scene style.The prompt selects among these strategies according to the clue and scene; decoys must rest on or attach to a real surface.Source: J Interactive GeoCloak · pp. 23–24
  • For adversarial perturbation, region_focus samples a clue region each step while retaining the full image in the reference pool; full_image instead uses the whole image as reference.The choice depends on whether clues have bounding boxes; the region-focused pool also disrupts global features.Source: J Interactive GeoCloak · p. 25

Evaluating Scope: Landmark Intention Analysis

  • Landmark intention analysis classifies boxed objects as widely known landmarks, locally recognizable spots, or generic structures, then checks prominence and subject-versus-background context.Tier A includes globally or nationally iconic sites; Tier B covers regionally recognizable landmarks; Tier C covers structures without identifying features.Source: Evaluating Scope: Landmark Intention… · pp. 26–27

Related papers

Loading 2610.20055v1…