PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaptVPR: Route-Aware Hard Positive Generation for Robust Visual Place Recognition

Shunpeng Chen, Jingyi Zhang, Changwei Wang, Shengpeng Xu, Yukun Song, Xingtian Pei, Jinzhou Lin, Li Guo, Shibiao Xu

arXiv:2609.04369v1cs.CV

TL;DR

VPR robustness suffers from limited same-place appearance diversity and domain shifts that challenge retrieval under changed conditions. AdaptVPR generates verified same-place hard positives through route-aware editing and selective refinement, producing consistent benchmark gains and stronger improvements under challenging shifts.

  • Problem

    Limited same-place appearance diversity and shifts in weather, illumination, seasons, and occlusions make robust VPR retrieval difficult.

  • Method

    AdaptVPR uses VLM scene understanding, rule-based route scheduling, three perturbation routes, geometry and diversity verification, and selective feedback refinement.

  • Results

    AdaptVPR consistently improves retrieval across VPR baselines and vision foundation backbones, with larger gains under challenging domain shifts.

  • Takeaways & Limitations

    AdaptVPR expands verified same-place training variations while controlling structural drift through route-specific generation and verification.

Abstract

from arXiv · show

Visual Place Recognition (VPR) localizes a query image by retrieving database images of the same or nearby place, yet its robustness is often degraded by domain shifts arising from illumination, weather, seasonal changes, and dynamic occlusions. One contributing factor is the limited appearance diversity of the same place in existing training data. To address this issue, we propose AdaptVPR, a route-aware generative augmentation framework that constructs same-place hard positives for robust VPR training. AdaptVPR first uses a vision language model to parse scene attributes and estimate editing feasibility, while a rule-based scheduler determines the generation route according to editability scores and risk constraints. The generation process is decomposed into three complementary routes: the Global Appearance Route introduces global scene changes in weather, illumination, and time of day; the Local Occlusion Route inserts plausible dynamic occluders; and the Dual Route combines both types of perturbations to produce more challenging appearance shifts. Each generated candidate is evaluated using a VPR-oriented verification scheme based on geometric consistency and appearance diversity, reducing the risk of structural drift while ensuring sufficient appearance variation. Global candidates are generated once and rejected if verification fails, while Local Occlusion and Dual candidates use verification feedback for limited prompt refinement and regeneration. Using this framework, we construct AdaptCities, containing 160K verified synthetic same-place hard positives. Experiments across multiple VPR baselines and vision foundation backbones show consistent gains on standard benchmarks and substantial improvements under challenging domain shifts, with R@1 gains of up to 9.2%. The source code and data resources are publicly available at https://github.com/chenshunpeng/AdaptVPR.

I. INTRODUCTION

VPR robustness is limited by scarce diverse same-place observations and the difficulty of generating appearance changes without altering place identity. AdaptVPR addresses this with route-aware generation, route-specific verification, and the 160K-image AdaptCities dataset.

  • Domain shifts from weather, illumination, seasons, traffic, and occlusions can alter appearance more than place identity, challenging robust retrieval.
  • Real revisits are costly to scale, while conventional augmentations cannot faithfully reproduce complex shifts such as rainy nights, reflections, traffic, or structured occlusions.
  • AdaptVPR uses VLM scene analysis, rule-based scheduling, and three routes for global appearance changes, local occlusions, and compound shifts.
  • Route-specific geometry and diversity verification retains informative same-place positives while controlling structural and identity drift.
  • 160K verified synthetic hard positives in AdaptCities support consistent gains across VPR methods and stronger improvements under challenging domain shifts.

II. RELATED WORK

Related VPR work has advanced representation learning, aggregation, and generative synthesis, but large appearance discrepancies remain difficult. AdaptVPR instead enriches the same-place variations observed during training through route scheduling, verification, and feedback refinement.

  • VPR progressed from handcrafted descriptors and aggregation to deep representations, classification-based localization, and vision foundation models.
  • Despite representation and retrieval advances, seasonal changes, adverse illumination, nighttime conditions, and dynamic occlusions remain challenging.
  • GAN-based translation expands selected appearance conditions but is constrained by predefined source and target domains.
  • Diffusion models enable more flexible controllable synthesis, motivating methods that jointly consider transformation difficulty and place consistency.
  • AdaptVPR combines image-dependent route scheduling, geometry and diversity verification, and selective feedback refinement for verified hard positives.

C. Controllable Generation and Agentic Image Editing

AdaptVPR frames same-place hard-positive construction as constrained, route-aware generation: visual changes should be challenging while place-defining structure remains sufficiently consistent. Scene understanding and deterministic routing select feasible edits before route-specific synthesis and verification.

  • Framework: The framework combines scene understanding, rule-based routing, route-specific generation, geometry-diversity verification, and selective prompt reflection.
  • Constrained hard-positive generation: AdaptVPR defines a same-place hard positive as a generated image preserving place-defining structure while introducing challenging appearance or local-visibility changes.
  • Verification objective: The generation objective requires sufficient geometric consistency and meaningful appearance diversity rather than unconstrained image synthesis.
  • Scene understanding and planning: Scene planning estimates weather and occlusion editability, flags unsuitable images, and supplies target conditions, occluder specifications, and prompt constraints.
  • Route selection: Global appearance, local occlusion, and dual routes are selected according to scene state and routing rules, with a skip route for unsuitable or ineligible samples.

2) Rule-Based Routing:

The scheduler selects among executable routes using editability-based capability scores and quota-aware deficits. Eligibility depends on scene suitability, while quota control limits concentration in any single route.

  • Rule-based routing: The scheduler scores eligible routes with capability and quota-deficit terms, selecting the remaining route with the highest S_r.S_r = w_rΔ_r + λc_r, with Δ_r measuring route quota deficit.
  • Route capability: Capability is defined by weather editability for Global Appearance, occlusion editability for Local Occlusion, and their minimum for Dual.
  • Quota control: Quota control increases deficit weight below a minimum ratio and removes routes exceeding a maximum ratio.
  • Decision and dispatch: The final decision is Global Appearance, Local Occlusion, Dual, or Skip, after which the selected route receives a structured generation command.
  • Agent record: Planning and routing are recorded in a structured agent record, while verification feedback is appended for later refinement of Local Occlusion and Dual candidates.

3) Route-Aware Generation:

AdaptVPR dispatches images to route-specific generators and verifies candidates using geometric consistency and appearance diversity. Global candidates are fast-rejected after one generation, whereas Local Occlusion and Dual routes can use feedback for refinement.

  • Route-specific generation: IC-Light performs global weather, illumination, and time-of-day changes, while Qwen-LightX2V generates local occlusions and dual perturbations.
  • Global Appearance Route: Global Appearance prompts constrain viewpoint, road layout, and architectural structure while changing image-wide appearance.
  • Selective refinement: Global candidates are generated once and rejected if verification fails; Local Occlusion and Dual candidates can use verification feedback to refine prompts.
  • Local and Dual Routes: Local Occlusion introduces vehicles or pedestrians in plausible ground-plane regions, and the Dual Route combines localized and compound visual changes.
  • Candidate verification: Each candidate is evaluated as a proxy for place preservation and challenge using complementary geometric and appearance criteria.
  • Verification metrics: RANSAC-based geometric consistency uses the final inlier ratio to detect structural drift, while CLIP feature distance measures appearance diversity.

D. Closed-Loop Self-Reflective Controller

AdaptVPR uses a closed-loop controller that verifies generated candidates and selectively refines prompts for Local Occlusion and Dual routes. Global Appearance candidates instead use one-shot generation with fast rejection.

  • Controller design: The controller coordinates generation branches, geometry-diversity verification, and prompt rewriting in an asymmetric feedback loop.Only failed Local Occlusion and Dual candidates re-enter generation through updated prompts.
  • Controller design: The generator produces an initial candidate from a planned prompt, which the verifier evaluates to provide route-specific feedback.The route and generator remain fixed during reflection, while the prompt is updated.
  • Verification: Verification checks geometric consistency and appearance diversity against route-specific thresholds.Geometry assesses preservation of layout, structure, and viewpoint; diversity assesses whether the edit is sufficiently informative.
  • Prompt refinement: Failed Local Occlusion and Dual candidates receive prompt-level repair instructions based on diagnosed failure modes.Refined prompts can strengthen the requested visual change or adjust occluder constraints.
  • Stopping policy: Local Occlusion and Dual candidates allow at most K = 3 reflection attempts after initial generation, stopping when verification passes or the budget is exhausted.The initial generation is indexed as t = 0, so each sample can be generated at most K + 1 times.
  • Stopping policy: Global Appearance candidates undergo one-shot generation and fast rejection, while accepted reflective candidates are retained and failed ones excluded from training.The system records prompts, scores, diagnostics, outputs, and success flags for traceability.

E. AdaptCities Dataset Construction

AdaptCities is constructed through automated sampling, semantic parsing, route scheduling, iterative synthesis, and route-specific verification. The resulting dataset contains 160K verified synthetic hard positives generated from 88,989 GSV-Cities reference images.

  • Pipeline overview: AdaptCities is built from GSV-Cities through four automated stages for scalable and reproducible processing.The pipeline produces verified same-place positives covering global appearance changes, local occlusions, and their combinations.
  • Sampling and parsing: Fixed-seed quota sampling across 23 metropolitan areas supports balanced geographic coverage before VLM-based semantic parsing.The parser generates localized physical-attribute dictionaries used for downstream routing and generation.
  • Route scheduling: A rule-based scheduler uses weather, occlusion, and bad-editability scores to select eligible routes or assign samples to Skip.When several routes are eligible, a quota-aware score determines the final route.
  • Route scheduling: Planning-time review reassigns high-risk local edits to Global Appearance when global editing remains reliable, while preserving the selected route and generator during synthesis.Local and Dual prompts encode occluder type, position, scale, and visibility before synthesis.
  • Synthesis and refinement: Global Appearance uses IC-Light with one-shot rejection, whereas Local Occlusion and Dual use Qwen-LightX2V with up to K = 3 feedback-guided reflection attempts.The refinement template combines current prompts and diagnostic feedback with route-specific constraints.
  • Verification: Every candidate is filtered using geometric consistency and appearance diversity with fixed thresholds tailored to the executable route.Global editing uses strict geometric preservation requirements and a moderate diversity requirement.
  • Dataset outcome: 160K verified synthetic hard positives are generated from 88,989 unique GSV-Cities reference images and inherit their parent geographical labels.The final training pool combines the original GSV-Cities training set with AdaptCities, with metadata logs supporting auditing.

F. VPR Training Protocol

AdaptVPR integrates verified synthetic positives into existing VPR training without changing model backbones, aggregators, or loss functions. The augmented samples expand each batch’s same-place positive set with constructed hard variations.

  • Training integration: AdaptVPR is model-agnostic and requires no modifications to existing VPR backbones, feature aggregators, or loss functions.It can be integrated into standard metric-learning objectives by expanding the local positive sample manifold.
  • Batch construction: For each query anchor, positives share its geographical label while negatives have different geographical labels.The positive and negative sets are defined within a training batch.
  • Batch construction: Verified synthetic instances expand the positive-set cardinality and introduce explicitly constructed hard-positive variations into each mini-batch.This incorporation relies on proxy-verified place consistency from AdaptCities.
  • Optimization: Each VPR baseline retains its original metric-learning objective during optimization.For Multi-Similarity Loss baselines, pairwise similarities are weighted using α, β, and λ.

IV. EXPERIMENTS

AdaptVPR is evaluated across ten benchmark settings spanning urban retrieval, viewpoint, seasonal, illumination, and occlusion variations. Recall@N is the primary metric under dataset-specific correctness protocols.

  • Benchmark suite: The evaluation covers ten benchmark settings with large-scale urban retrieval, viewpoint variation, seasonal changes, adverse illumination, and partial occlusion.The benchmark statistics and dominant visual variations are summarized in Table II.
  • Evaluation metric: Recall@N measures the proportion of queries with at least one correct match among the top-N retrieved database images.Correctness thresholds vary by benchmark, including spatial, heading, and temporal-alignment requirements.

B. Implementation Details

AdaptVPR is evaluated by adding generated same-place hard positives while preserving each baseline’s original model and training configuration. It improves retrieval broadly, with larger gains under seasonal, occlusion, and nighttime shifts, while increasing training difficulty without destabilizing optimization.

  • Evaluation setup: Each VPR baseline retains its original backbone, aggregation module, optimizer, loss, and checkpoint-selection protocol when AdaptVPR positives are added.Evaluation uses Recall@N, selecting the highest Pitts30k-val R@1 checkpoint.
  • Standard benchmarks: AdaptVPR improves R@1 across all 20 baseline–benchmark combinations, with gains spanning Pitts30k, MSLS-val, Tokyo24/7, Nordland, and SVOX.The reported gains are 0.1%–0.7% on Pitts30k, 0.5%–1.3% on MSLS-val, and 0.3%–0.9% on Tokyo24/7.
  • Domain-shift robustness: Under challenging domain conditions, R@1 gains reach 9.2% on SF-XL-Occlusion and 8.3% on Nordland⋆ for BoQ.Nighttime gains range from 1.7% to 3.2%, while Pitts250k-test changes range from −0.5% to +0.4%.
  • Cross-model consistency: The gains remain distributed across heterogeneous VPR architectures, supporting transfer beyond a single aggregation or retrieval design.Examples include SALAD’s 6.0% Nordland⋆ gain, BoQ’s 9.2% SF-XL-Occlusion gain, and EDTformer’s 8.1% Nordland⋆ gain.
  • Training dynamics: AdaptVPR raises BoQ’s training loss because generated positives are harder, yet the loss decreases steadily and validation R@1 remains higher during most training.The training dynamics indicate increased difficulty without optimization instability.

D. Ablation Studies

The ablations show that AdaptVPR benefits from combining complementary generation routes with geometry-and-diversity filtering and verifier-guided refinement. The complete procedure accepts more valid candidates but requires additional attempts and generation time.

  • Synthetic quality filtering: Joint geometry and appearance-diversity filtering outperforms either filter alone and improves R@1 over using all unfiltered synthetic samples.The gains are 0.4% on Pitts30k, 1.2% on MSLS-val, 0.6% on SF-XL-Night, and 2.6% on SF-XL-Occlusion.
  • Generation routes: The Global Appearance Route changes weather, illumination, and time of day, while the Local Occlusion Route adds localized dynamic occlusions without altering the background structure.The Dual Route combines both perturbation types for more challenging hard positives.
  • Complete generation procedure: Adaptive routing and verifier-guided refinement increase candidate acceptance from 50.8% to 65.6%.The full procedure combines route selection, structured prompts, verifier feedback, and repeated generation within a reflection budget.
  • Complete generation procedure: The acceptance gain increases average attempts from 1.00 to 2.24 and generation time from 21.0 s to 46.5 s.The ablation therefore exposes a practical trade-off between candidate quality and construction cost.

4) Generalization across Vision Foundation Backbones:

AdaptVPR generalizes across vision foundation backbones and training settings, while moderate synthetic exposure provides the strongest sampling balance. Verification examples illustrate appearance and visibility changes that retain structural correspondence, although place consistency is not fully guaranteed.

  • Backbone generalization: AdaptVPR improves retrieval across both DINOv2-B and DINOv3-B backbones on Pitts30k, MSLS-val, and Tokyo24/7.With DINOv2-B, R@1 gains are 0.7%, 1.3%, and 0.3%; with DINOv3-B, they are 0.5%, 1.3%, and 0.3%, respectively.
  • Synthetic sampling ratio: The 8:1 real-to-synthetic ratio achieves the highest R@1 across the evaluated benchmarks.Compared with real-only training, Pitts30k rises from 93.0% to 93.8% and MSLS-val from 93.2% to 94.3%.
  • Synthetic sampling ratio: Increasing synthetic exposure does not monotonically improve performance; moderate exposure better balances appearance enrichment with preservation of the real-data distribution.The discussion attributes this balance to informative, well-controlled hard positives rather than synthetic volume alone.
  • Limitations: Geometric consistency remains only a proxy for place identity and cannot fully guarantee that every generated image preserves the original location.The framework also depends on predefined routes and thresholds, which may limit flexibility in unseen environments.
  • Verification analysis: Verification examples combine geometric correspondences, sgeo, sdiv, and inlier/outlier counts to assess challenging appearance or visibility changes while retaining structural correspondence.The examples cover global appearance, local occlusion, and compound dual-route variations.
Loading 2609.04369v1…