PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Pre-Training and Augmentation for Zero-Shot Cross-City Object Detection

Long Hoang Pham, Quoc Pham-Nam Ho, Huy-Hung Nguyen, Duong Nguyen-Ngoc Tran, Ngoc Doan-Minh Huynh, Cu Quoc Le, Hoang-Khang Nguyen, Hyung-Min Jeon, Chi Dai Tran, Son Hong Phan, Duong Khac Vu, Trinh Le Ba Khanh, Jae Wook Jeon

arXiv:2608.24154v1cs.CVcs.AI

TL;DR

Traffic surveillance detectors can lose accuracy in unseen cities, while privacy constraints leave robust cross-city generalization underexplored. The paper proposes a modular pipeline combining class-agnostic multi-dataset pre-training and Grayworld augmentation. Its RF-DETR-HR and RF-DETR-Grayworld variants achieved 1st place with 47.53 mAP on the hidden evaluation benchmark.

  • Problem

    Cross-city detector accuracy drops on unseen locations, while privacy-constrained surveillance data collection leaves robust cross-city generalization underexplored.

    Source: §1 · pp. 1–2

  • Method

    The modular pipeline combines class-agnostic multi-dataset pre-training to learn vehicle geometry with Grayworld augmentation to remove volatile chromatic shortcuts.

    Source: §1 · p. 2

  • Results

    47.53 mAP earned RF-DETR-HR and RF-DETR-Grayworld 1st place on the hidden evaluation benchmark.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The optimized models operate within a 16GB VRAM limit for zero-shot cross-city detection under strict hardware and privacy constraints.

    Source: §6 · p. 14

    The study is scoped to blind deployment, where target-domain images remain hidden during training.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Real-world deployment of traffic surveillance systems is bottlenecked by geographic domain shift, in which models trained in one city underperform when applied to an unseen target city. Conventional domain adaptation relies on hyperparameter-sensitive architectures or direct profiling of target data. Both are fundamentally precluded in privacy-conscious ecosystems that require completely blind training and evaluation loops. In this setting, we explore the effects of pre-training and augmentation in addressing the domain shift problem. Specifically, we propose a new modular training pipeline for object detection structured around two core orthogonal pillars: (1) a multi-dataset pre-training strategy featuring a class-agnostic objectness distillation to decouple structural vehicle geometry from semantic taxonomies, and (2) a domain-resilient augmentation stream featuring a novel Grayworld transformation that forces global attention heads to strip volatile chromatic shortcuts in favor of robust shape priors. When evaluated with the real-time transformer-based detector RF-DETR, our framework bridges cross-city distribution gaps while using limited GPU memory (16GB). Our optimized variants, RF-DETR-HR and RF-DETR-Grayworld, deliver a substantial empirical gain of +24.29 over the baseline, achieving 1st place (47.53 mAP) on the AI City Challenge Track 6 leaderboard. Code and data are available at: \href{https://github.com/SKKUAutoLab/aic26_cross_city}{SKKUAutoLab/aic26\_cross\_city}.

1 Introduction

Geographic domain shift sharply reduces traffic-object detection accuracy in unseen cities, while privacy constraints limit data access and direct target-city profiling. The paper proposes a modular, blind-training pipeline combining class-agnostic multi-dataset pre-training, Grayworld augmentation, and memory-aware RF-DETR optimization.

1 Introduction

  • Object detectors that perform well in their training city can lose accuracy in unseen cities with different layouts, viewpoints, vehicle distributions, and environmental conditions.Privacy concerns constrain large-scale surveillance-data collection, leaving robust cross-city generalization underexplored.Source: §1 · pp. 1–2
  • The AIC2026 Track 6 benchmark uses blind training and inference because neither source-city nor target-city data are accessible.This setting favors practical methods over architecture scaling or direct target-data profiling.Source: §1 · p. 2
  • RF-DETR-HR achieved mAP 47.53 and RF-DETR-Grayworld achieved mAP 46.63; the methods secured SOTA, first-place performance on AIC2026 Track 6.The reported RF-DETR-HR score was 47.53 mAP on the hidden evaluation benchmark.Source: §1 · p. 2
  • Class-agnostic objectness distillation collapses heterogeneous labels into binary foreground to learn vehicle geometry while avoiding cross-domain class-alignment conflicts.The approach is used in a multi-dataset pre-training strategy.Source: §1 · p. 2
  • Grayworld augmentation directs global self-attention away from volatile, sensor-dependent chromatic shortcuts toward robust shape-centric representations.The paper also calibrates RF-DETR for training within a 16GB GPU memory limit.Source: §1 · p. 2

2 Related Work

Prior cross-domain detection methods address geographic shift through target-data adaptation or prompting, but their assumptions fail in completely blind, resource-constrained deployments. The section motivates source-side generalization and describes traffic datasets spanning varied environments and camera perspectives.

2.1 Cross-Domain Object Detection

  • CNN detectors overfit to local source-domain features, while transformers degrade under geographic shifts associated with style-sensitive queries and attention maps.The section distinguishes CNN-based methods such as Faster R-CNN and YOLOs from transformer-based methods such as DETR, Deformable-DETR, Co-DETR, and RF-DETR.Source: §2.1 · pp. 2–3
  • Existing cross-domain approaches include UDA feature alignment, F-TTA parameter adaptation, and VLM prompting to steer visual representations toward a conceptual target domain.UDA uses adversarial learning, feature-statistic alignment, or student-teacher pseudo-labeling; F-TTA uses entropy minimization or batch-normalization updates.Source: §2.1 · p. 3
  • UDA needs unlabeled target images, F-TTA requires runtime gradient updates, and VLM prompting assumes target-domain knowledge, limiting use in blind, constrained deployments.The discussion motivates source-side generalization that encourages color-invariant, domain-agnostic representations without target data, test-time updates, or prior domain assumptions.Source: §2.1 · p. 3

2.2 Traffic Surveillance Datasets

  • Hafnia comprises 27.8K traffic images and 10 fine-grained classes across Vehicle and Person categories, with testing spanning a training city and a new target domain.Its viewpoints, roadway types, camera perspectives, and imaging conditions vary.Source: §2.2 · p. 3
  • The study also describes TSBOW, TrafficCAM, FishEye8K, VisDrone, and MOT20 datasets spanning varied weather, lighting, camera perspectives, object scales, and crowd densities.TSBOW has 48K labeled images; TrafficCAM has over 4.3K images from more than 2K videos; FishEye8K has 8K images and 157K annotations; VisDrone has around 8.6K images; MOT20 has 8 sequences and over 13K images.Source: §2.2 · p. 4

3 Preliminary: Hafnia Training-aaS Platform

The Hafnia Training-aaS platform requires a blind, privacy-preserving training and evaluation workflow, with hidden datasets and tightly constrained compute. These conditions motivate memory-efficient training and inference rather than compute-intensive model expansion or adaptation.

3 Preliminary: Hafnia Training-aaS Platform

  • AIC2026-Track 6 uses Hafnia TaaS, where hidden data must be accessed through anonymized Docker jobs without debugging and evaluation uses hidden ground truth.The platform generates test predictions before submitting them to the AIC2026 evaluation server for scoring.Source: §3 · p. 5
  • Jobs run under deterministic resource limits, typically x1–x4 NVIDIA Tesla T4 GPUs with 16GB VRAM and limited training credits.Source: §3 · p. 5
  • These limits make VLM distillation [4] a bottleneck, favoring localized feature calibration and memory-efficient pipelines over backbone expansion or complex F-TTA.The proposed pipeline treats the constraint as a design feature to maximize mAP while optimizing training and inference.Source: §3 · p. 5

4 Methodology

The methodology optimizes detection mAP on an unseen target domain under a 16GB GPU-memory constraint, without access to target visual features or label spaces. It combines multi-dataset pre-training, domain-resilient augmentation, and grid-searched training and inference settings.

4.1 Problem Formulation

  • The optimization targets maximum mAP on an unseen domain while keeping model memory within the 16GB GPU limit.The searched factors include pre-training data, augmentation, epochs, batch size, training and inference resolutions, and confidence threshold.Source: §4.1 · pp. 5–6

4.2 Multi-Dataset Pre-Training (DS)

  • 40K auxiliary traffic images were selected for pre-training, with class-agnostic objectness distillation used to transfer geometry across inconsistent source taxonomies.All source labels are collapsed into a single object class for a 3–5-epoch warm-up, while pseudo-labeling with Qwen3-VL can harmonize labels but requires extensive manual refinement.Source: §4.2 · pp. 6–7

4.3 Data Augmentation (A)

  • Grayworld maps RGB images to grayscale to reduce reliance on city-specific color cues, while spatial-geometric augmentations address viewpoint and layout variation.CLAHE and edge sharpening compensate for lost color cues; grayscale is applied consistently during pre-training, fine-tuning, and inference.Source: §4.3 · pp. 7–9

4.4 Hyperparameter Calibration

  • E = 5 epochs and τc = 0.02 improved cross-city mAP, while the best inference scale depended on augmentation: 2400×2400 for AL and 2200×2200 for AG.With 40K traffic pre-training, AL and AG performed best; higher training resolution mainly benefited AL, while AG generalized without it. Performance declined beyond 5 epochs, and increasing inference resolution eventually saturated.Source: §4.4 · p. 10

4.5 Improved Methods

  • RF-DETR-HR combines 40K pre-training and light augmentation with 1080×1080 training and 2400×2400 inference; RF-DETR-Grayworld pairs 40K pre-training with Grayworld and 2200×2200 inference.HR emphasizes spatial feature density for small and distant targets, while Grayworld prioritizes structural geometry over location-specific color statistics.Source: §4.5 · p. 11

5 Experimental Results

On the hidden target-city benchmark, the final RF-DETR variants placed first and second, while experiments showed gains from larger pre-training data, calibrated resolutions, and Grayworld augmentation. The evaluation used COCO-Eval metrics, with mAP50:95 determining the final ranking.

5.1 Implementation Details

  • Experiments used brief local pre-training followed by hidden-split fine-tuning and official hidden-test evaluation.Local pre-training ran for 3–5 epochs, selecting top EMA weights; online fine-tuning used a Tesla T4 with 16GB VRAM and differential learning rates (decoder/classification heads 10−4; encoder 1.5 × 10−4). Final ranking used COCO-Eval mAP50:95.Source: §5.1 · pp. 11–13

5.2 Benchmark Results

  • 34.31 mAP made RF-DETR-2XLarge the strongest initial baseline, while limiting fine-tuning to E ≤5 epochs yielded 26.72 vs. 23.24 mAP for longer schedules.The short schedule was reported to prevent attention heads from memorizing source-city clutter and overfitting domain-specific artifacts.Source: §5.2 · p. 13
  • 42.04 cross-city mAP from 40K-instance pre-training exceeded 40.37 mAP from aggressive synthetic augmentation on smaller initializations.The authors attribute this to broader real-world environmental, lighting, and viewpoint diversity while preserving feature geometry.Source: §5.2 · p. 13
  • 46.29 vs. 42.17 mAP followed an inference-resolution increase to 2400 × 2400, while training at 1080×1080 added +1.24 mAP for AL augmentation.The higher training resolution required batch size B = 1 to fit within 16GB VRAM; lowering the confidence threshold to 0.02 added +0.13 mAP.Source: §5.2 · p. 13
  • 46.63 vs. 45.71 mAP at training resolution 880 showed Grayworld augmentation outperforming light-augmentation baselines.Grayworld removes volatile RGB distributions and neutralizes camera white-balance variance, discouraging reliance on photometric shortcuts.Source: §5.2 · p. 13

5.3 Comparison & Final Ranking

  • 47.53 mAP placed RF-DETR-HR 1st, while RF-DETR-Grayworld ranked 2nd nominally at 46.63 mAP; the leaderboard margin over second place was +4.72.The reported methods exceeded the subsequent competitor entry by +4.7 and +3.8 mAP, respectively.Source: Table 3 · p. 14, §5.3 · p. 14

6 Conclusion

The paper presents a modular approach to zero-shot cross-city detection under strict privacy and hardware constraints, combining class-agnostic pre-training with Grayworld augmentation. Its optimized models achieved substantial mAP gains over standard baselines and placed first on the official AIC2026 Track 6 leaderboard.

6 Conclusion

  • RF-DETR-HR and RF-DETR-Grayworld yielded substantial mAP gains over standard baselines and secured 1st place on the official AIC2026 Track 6 leaderboard.The models operated within a 16GB VRAM limit.Source: §6 · p. 14
  • The modular framework combines class-agnostic 40K pre-training to separate vehicle geometry from changing taxonomies with Grayworld transformation to remove sensor-dependent color shortcuts.The approach addresses geographic domain shifts without changing complex Vision Transformer architectures.Source: §6 · p. 14
  • Differential learning rates, gradient accumulation, and JIT tracing help the optimized models operate efficiently within a 16GB VRAM limit on the Milestone System’ Hafnia platform.Source: §6 · p. 14

Related papers

Loading 2608.24154v1…