Source-linked AI summary
Domain Adaptation through Synthesis for Unsupervised Person Re-identification
Slawomir Bak, Peter Carr, Jean-Francois Lalonde
TL;DR
Person re-identification remains difficult because appearance changes across cameras and existing datasets lack diverse illumination conditions. The paper introduces synthetic, HDR-lit identities and an unsupervised domain-adaptation technique, achieving 43.0% rank1-accuracy on PRID and surpassing current state-of-the-art results by 8.2%.
Problem
Appearance varies drastically across cameras because of pose, illumination, camera configuration, and viewing angle, while synthetic data does not cover all illumination conditions and differs from real image distributions.
Method
The paper introduces SyRI, using 100 virtual humans rendered with 140 HDR environment maps, and a three-step unsupervised domain-adaptation technique with identity-preserving cycle-consistency translation.
Results
43.0% rank1-accuracy was achieved on PRID, surpassing current state-of-the-art results by 8.2%.
Takeaways & Limitations
Synthetic data can increase trained re-identification models’ generalization capabilities in unseen illumination conditions without fine-tuning.
Takeaways & Limitations
The domain-translation loss alone leaves many subjects with drastic appearance shifts.
Abstract
from arXiv · showhide
Drastic variations in illumination across surveillance cameras make the person re-identification problem extremely challenging. Current large scale re-identification datasets have a significant number of training subjects, but lack diversity in lighting conditions. As a result, a trained model requires fine-tuning to become effective under an unseen illumination condition. To alleviate this problem, we introduce a new synthetic dataset that contains hundreds of illumination conditions. Specifically, we use 100 virtual humans illuminated with multiple HDR environment maps which accurately model realistic indoor and outdoor lighting. To achieve better accuracy in unseen illumination conditions we propose a novel domain adaptation technique that takes advantage of our synthetic data and performs fine-tuning in a completely unsupervised way. Our approach yields significantly higher accuracy than semi-supervised and unsupervised state-of-the-art methods, and is very competitive with supervised techniques.
1 Introduction
Person re-identification must handle drastic appearance changes across cameras, but existing datasets provide limited lighting diversity. The paper introduces SyRI and an unsupervised domain-adaptation approach to improve recognition under unseen illumination.
- Person re-identification seeks the same subject across non-overlapping cameras despite changes in pose, illumination, camera configuration, and viewing angle.
- Existing datasets contain many identities but relatively few lighting conditions, biasing models toward illumination seen during training.
- SyRI uses realistic indoor and outdoor lighting generated from HDR environment maps, which serve as virtual light sources and background plates.
- A three-step adaptation method translates synthetic images toward target conditions, regularizes translation to preserve identity, masks foregrounds, and uses translated images for fine-tuning.
- 100 virtual humans are rendered with 140 HDR environment maps to increase illumination diversity and generalization in unseen conditions.
- The method improves re-identification accuracy without labeled target data by adapting synthetic images to target-domain conditions.
2 Related work
Prior person re-identification systems rely heavily on supervised learning and fine-tuning, while unsupervised approaches often trade discriminativity for feature invariance. The paper addresses this setting with synthetic data that provides realistic lighting variation.
- Person re-identification: Successful person re-identification approaches commonly use supervised learning, deep architectures, and increasingly large labeled datasets.
- Person re-identification: Existing approaches generally require fine-tuning for unseen target illumination conditions, which is infeasible across large camera networks.
- Person re-identification: Semi-supervised and unsupervised methods include transfer learning and dictionary learning, but typically seek feature invariance without labels and reduce discriminativity.
- Synthetic data: HDR environment maps capture realistic indoor and outdoor lighting and can be used to relight virtual humans.
3 SyRI Dataset
SyRI is built by rendering customizable virtual humans in realistic scenes under diverse HDR lighting. The dataset combines synthetic identities, animations, viewpoints, backgrounds, and environment-map illumination.
- 3.1 HDR environment maps: The environment maps are 360° panoramas containing lighting information for relighting virtual objects and supplying realistic backgrounds.
- 3.1 HDR environment maps: SyRI uses 140 HDR environment maps to model realistic lighting for synthetic person re-identification scenes.
- 3.2 3D virtual humans and animations: The dataset contains 100 character prototypes with customized body shapes, clothing, material textures, and colors.
- 3.2 3D virtual humans and animations: Virtual humans are animated with male or female walking rigs and rotation animations to acquire multiple viewpoints.
- 3.3 Rendering: Unreal Engine renders each character at the center of a surrounding sphere whose texture-mapped HDR environment map acts as the sole light source.
- 3.3 Rendering: The camera position is matched with existing person re-identification datasets to support comparable viewpoints.
4 Method
The method casts person re-identification as unsupervised adaptation across illumination-defined domains. It selects a similar synthetic domain, translates its images toward the unlabeled target, and fine-tunes a jointly trained network while preserving identity-relevant appearance.
- Problem formulation: The task is framed as unsupervised domain adaptation, treating each camera-specific lighting condition as a separate domain.The target domain contains no identity labels, while real and synthetic source domains provide labeled images.
- Joint learning: The generic feature extractor is trained jointly from scratch on all real and synthetic domains, using more than 3K identity classes and a 256-dimensional feature layer.Joint training is intended to learn discriminative and generalizable person representations.
- Three-step adaptation: The pipeline selects the closest synthetic illumination domain, translates it toward the unlabeled target domain, and fine-tunes the re-identification network using known synthetic identities.These are the three stages: illumination inference, domain translation, and fine-tuning.
- Illumination inference: Illumination inference identifies the synthetic domain most similar to the target by counting classifier predictions, minimizing the required domain shift before translation.The selected domain is then translated while preserving each 3D character’s identity.
- Domain translation: Cycle-consistent adversarial translation learns mappings between synthetic and real domains, with adversarial and cycle-consistency losses discouraging mode collapse.The two mappings are trained as approximate inverses, while discriminators distinguish real from translated images.
- Semantic shift regularization: A foreground-focused semantic-shift regularizer limits changes between synthetic images and their translations because unconstrained translation can alter clothing colors and identity cues.The mask preserves appearance near the image center while allowing the background to differ across domains; an identity loss alone still left drastic appearance shifts for many subjects.
5 Experiments
Experiments evaluate SyRI and the proposed domain adaptation across five re-identification datasets, testing generalization, illumination selection, image statistics, regularization, and state-of-the-art comparisons.
- Experimental setup: Experiments use VIPeR, iLIDS, CUHK01, PRID2011, and Market-1501, with generic feature extractors trained on CUHK03, DukeMTMC4ReID, and SyRI.Performance is reported using CMC rank-1 accuracy.
- Generalization properties: SyRI adds 100 identities under 140 illumination conditions, producing 56,000 sampled synthetic images for comparison with a real-only model.The real-only model contains 3,279 identities, while the combined model contains 3,379.
- Generalization properties: Adding synthetic data significantly improves direct target performance, while domain adaptation yields further gains but remains below the supervised fine-tuning upper bound.Table 1 compares real-only, real-plus-synthetic, adapted, and target-fine-tuned models.
- Illumination inference: Illumination inference improves recognition compared with random selection of the target illumination condition for domain translation.The comparison reports minimum and average random performance alongside the average obtained with illumination inference.
- Domain adaptation: Domain translation moves translated-image statistics closer to real-image statistics for corresponding VIPeR and SyRI camera conditions.The comparison uses a target illumination selected by illumination inference.
- Comparison with state-of-the-art methods: The method outperforms state-of-the-art unsupervised techniques on four of five datasets, with margins of 5.1%, 7.2%, 8.2%, and 7.5%, and is competitive on CUHK01.The reported margins correspond to VIPeR, iLIDS, PRID, and Market, respectively.