Rethinking Pre-Training and Augmentation for Zero-Shot Cross-City Object Detection

Long Hoang Pham; Quoc Pham-Nam Ho; Huy-Hung Nguyen; Duong Nguyen-Ngoc Tran; Ngoc Doan-Minh Huynh; Cu Quoc Le; Hoang-Khang Nguyen; Hyung-Min Jeon; Chi Dai Tran; Son Hong Phan; Duong Khac Vu; Trinh Le Ba Khanh; Jae Wook Jeon

Source-linked AI summary

Rethinking Pre-Training and Augmentation for Zero-Shot Cross-City Object Detection

Long Hoang Pham, Quoc Pham-Nam Ho, Huy-Hung Nguyen, Duong Nguyen-Ngoc Tran, Ngoc Doan-Minh Huynh, Cu Quoc Le, Hoang-Khang Nguyen, Hyung-Min Jeon, Chi Dai Tran, Son Hong Phan, Duong Khac Vu, Trinh Le Ba Khanh, Jae Wook Jeon

arXiv:2608.24154v1Aug 25, 2026cs.CVcs.AI

TL;DR

Traffic detectors lose accuracy across unseen cities, while privacy-preserving deployment prevents access to source or target data during blind training and evaluation. The paper proposes class-agnostic multi-dataset pre-training and Grayworld augmentation for RF-DETR, with optimized variants achieving 47.53 mAP and first place on AI City Challenge Track 6. The approach is designed for the platform’s 16GB VRAM constraint.

Problem
Geographic domain shift causes traffic detectors trained in one city to underperform in unseen cities, while blind privacy-preserving settings preclude target-data access and profiling.
Method
The framework combines class-agnostic objectness distillation during multi-dataset pre-training with Grayworld augmentation that suppresses chromatic shortcuts and favors shape priors.
Results
47.53 mAP delivered 1st-place performance for RF-DETR-HR and RF-DETR-Grayworld on the hidden evaluation benchmark.
Takeaways & Limitations
The optimized RF-DETR pipeline bridges cross-city distribution gaps while operating within a 16GB GPU memory limit.
Takeaways & Limitations
The setting requires completely hidden data, blind evaluation, containerized execution, and hardware-constrained training and inference.

Abstract

from arXiv · show

Real-world deployment of traffic surveillance systems is bottlenecked by geographic domain shift, in which models trained in one city underperform when applied to an unseen target city. Conventional domain adaptation relies on hyperparameter-sensitive architectures or direct profiling of target data. Both are fundamentally precluded in privacy-conscious ecosystems that require completely blind training and evaluation loops. In this setting, we explore the effects of pre-training and augmentation in addressing the domain shift problem. Specifically, we propose a new modular training pipeline for object detection structured around two core orthogonal pillars: (1) a multi-dataset pre-training strategy featuring a class-agnostic objectness distillation to decouple structural vehicle geometry from semantic taxonomies, and (2) a domain-resilient augmentation stream featuring a novel Grayworld transformation that forces global attention heads to strip volatile chromatic shortcuts in favor of robust shape priors. When evaluated with the real-time transformer-based detector RF-DETR, our framework bridges cross-city distribution gaps while using limited GPU memory (16GB). Our optimized variants, RF-DETR-HR and RF-DETR-Grayworld, deliver a substantial empirical gain of +24.29 over the baseline, achieving 1st place (47.53 mAP) on the AI City Challenge Track 6 leaderboard. Code and data are available at: \href{https://github.com/SKKUAutoLab/aic26_cross_city}{SKKUAutoLab/aic26\_cross\_city}.

1 Introduction

The paper addresses severe geographic domain shift in traffic surveillance, where detectors trained in one city underperform in unseen cities. It proposes a modular, privacy-compatible pipeline combining class-agnostic pre-training, Grayworld augmentation, and optimized RF-DETR variants.

Motivation: Geographic domain shift sharply reduces detector accuracy when models trained in one city are applied to unseen cities.Differences include road layouts, camera viewpoints, vehicle distributions, and environmental conditions.
Benchmark setting: The privacy-preserving benchmark hides both source and target data, requiring fully blind training and inference without direct target profiling.The setting also discourages architecture scaling as the primary solution.
Core approach: Class-agnostic objectness distillation collapses heterogeneous taxonomies into binary foreground representations to learn vehicle geometry and avoid cross-domain class conflicts.This is the first pillar of the proposed multi-dataset pre-training strategy.
Core approach: Grayworld augmentation removes volatile sensor-dependent chromatic shortcuts and encourages global attention heads to favor shape-centric representations.It forms the pipeline’s second domain-resilient augmentation pillar.

2 Related Work

The related work contrasts conventional detector families and adaptation methods with the paper’s source-side strategy. Existing approaches often depend on target data, runtime updates, or target-domain assumptions that are unavailable in blind, hardware-constrained deployment.

Detection and adaptation: CNN detectors can overfit local source-domain features, while transformers can degrade under geographic shift through style-sensitive queries and attention maps.The cited examples include lighting, asphalt style, and source-specific visual patterns.
Detection and adaptation: Unsupervised domain adaptation aligns feature distributions using adversarial learning, feature statistics, or student-teacher pseudo-labeling.These methods are presented as one of three major approaches to cross-domain detection.
Detection and adaptation: Fully test-time adaptation updates models during runtime, whereas vision-language methods depend on prior knowledge of target conditions.Both assumptions conflict with completely blind deployment.
Research gap: True zero-shot blind deployment requires source-side generalization without target data, test-time gradient updates, or prior domain assumptions.The paper motivates color-invariant, domain-agnostic representations as the required direction.
Datasets: The datasets span diverse traffic scenes, viewpoints, weather, lighting conditions, and camera types used to provide broad detection priors.Examples include TSBOW, TrafficCAM, FishEye8K, and VisDrone.

3 Preliminary: Hafnia Training-aaS Platform

The Hafnia Training-as-a-Service platform replaces local open-loop experimentation with a blind, containerized training-and-inference workflow. Deterministic 16GB GPU limits therefore favor memory-efficient calibration over larger backbones or complex runtime adaptation.

Platform constraints: AIC2026 Track 6 hides the training and benchmarking datasets and accesses them only through anonymized jobs.Algorithms must be packaged as standalone Docker containers without debugging access.
Platform constraints: Blind evaluation generates predictions on hidden test images before submission to the evaluation server for comparison with hidden ground truth.The workflow prevents direct inspection of evaluation data.
Platform constraints: Training and inference run on deterministic resources typically limited to NVIDIA Tesla T4 GPUs with 16GB VRAM.The hardware limit is an explicit platform constraint.
Design implications: The platform makes VLM distillation computationally costly, motivating localized feature calibration and memory-efficient training rather than backbone expansion or complex test-time adaptation.The proposed pipeline treats the resource constraint as a design feature.

4 Methodology

The framework formulates zero-shot cross-city detection as memory-constrained optimization and combines heterogeneous pre-training, objectness distillation, augmentation, and calibrated training and inference settings. Grid-search findings motivate domain-aligned RF-DETR variants that improve robustness under blind deployment constraints.

Problem Formulation: The pipeline maximizes unseen-domain mAP without accessing target visual features or label spaces, subject to a GPU-memory constraint.The formulation treats dataset, augmentation, epochs, batch size, resolutions, and confidence threshold as optimization variables.
Pre-training: Class-agnostic objectness distillation collapses heterogeneous labels into binary foreground localization to learn vehicle geometry while limiting early source-specific classification overfitting.A warm-up schedule uses joint L1 and GIoU minimization to prioritize object boundaries and structural features.
Pre-training: The pre-training scale is evaluated at 25K, 30K, and 40K images, with DS = 40K selected for subsequent experiments.The datasets aggregate heterogeneous traffic-surveillance distributions, and their compositions and comparisons are shown in Fig. 3.
Augmentation: Grayworld augmentation targets style memorization by reducing reliance on volatile chromatic statistics, while geometric transformations improve robustness to cross-city layout variation.Augmentation pipelines are organized as sequential operations, including rotation, horizontal flipping, and random cropping.
Hyperparameter Calibration: Five training epochs provide optimal cross-city mAP, whereas extending fine-tuning to 10 or 15 epochs reduces target-domain accuracy.The study systematically evaluates model size, pre-training data, augmentation, epochs, resolutions, and confidence threshold under platform constraints.
Hyperparameter Calibration: Inference mAP saturates at 2400×2400 with light augmentation and 2200 × 2200 with Grayworld, while higher training resolution mainly benefits light augmentation.Beyond these inference scales, over-smoothing artifacts and logit noise reduce precision; Grayworld retains strong generalization without high training resolution.
Improved Methods: RF-DETR-HR combines 40K pre-training, light augmentation, 1080 × 1080 training, and 2400×2400 inference, while RF-DETR-Grayworld uses Grayworld and 2200 × 2200 inference.Both variants use domain-aligned pre-training and target either spatial feature density or domain-invariant structural geometry.

5 Experimental Results

The experiments evaluate training, fine-tuning, inference, and benchmarking choices for zero-shot cross-city detection under hidden-data constraints. Results identify short fine-tuning and expanded pre-training as important factors, while RF-DETR-HR ranks first officially.

Implementation Details: Pre-training uses 3–5 epochs on auxiliary datasets, selecting the strongest EMA weights as downstream initialization checkpoints.Dedicated checkpoints are required when training resolution changes because positional embeddings are scale-dependent.
Implementation Details: Online fine-tuning adapts localization-anchored weights to the 10-class target using a differential learning-rate strategy on a 16GB Tesla T4.The hidden training split is accessed within the Hafnia Training-as-a-Service environment.
Implementation Details: The benchmark runs concealed target-city inference first, then submits standardized predictions to the official evaluation server for hidden-ground-truth ranking.Primary evaluation uses COCO-Eval, with final ranking determined by mAP50:95.
Benchmark Results: 42.04 cross-city mAP is achieved by scaling domain-aligned pre-training to 40K instances, while fine-tuning for E ≤5 epochs outperforms longer schedules.The reported fine-tuning comparison is 26.72 versus 23.24 mAP.
Comparison & Final Ranking: 47.53 mAP places RF-DETR-HR first on the official AIC2026 Track 6 leaderboard, while RF-DETR-Grayworld ranks second nominally at 46.63 mAP.The methods exceed the subsequent competitor by +4.7 and +3.8 mAP, respectively; Table 3 separately reports a +4.72 margin over second place.

6 Conclusion

The paper presents a modular two-pillar framework for zero-shot cross-city detection under strict privacy and hardware constraints. Its optimized RF-DETR variants combine class-agnostic pre-training and Grayworld augmentation to achieve substantial mAP gains and first place on AIC2026 Track 6.

Conclusion: The framework combines class-agnostic 40K pre-training, which separates vehicle geometry from taxonomies, with Grayworld transformation to remove sensor-dependent color shortcuts.These are the framework’s two core pillars for addressing geographic domain shifts without changing complex Vision Transformer architectures.
Conclusion: RF-DETR-HR and RF-DETR-Grayworld operate within a 16GB VRAM limit using platform-optimized training and inference techniques.The cited techniques include differential learning rates, gradient accumulation, and JIT tracing.
Conclusion: The optimized models secure first place on the official AI City Challenge AIC2026 Track 6 evaluation leaderboard while delivering substantial mAP gains over standard baselines.