Source-linked AI summary
Domain Adaptation through Synthesis for Unsupervised Person Re-identification
Slawomir Bak, Peter Carr, Jean-Francois Lalonde
TL;DR
Person re-identification is difficult because appearance can vary sharply across cameras, including with illumination. The paper introduces the SyRI synthetic dataset and an unsupervised domain-adaptation method for unseen lighting conditions. On PRID, its method achieves 43.0% rank1-accuracy and surpasses current state-of-the-art results by 8.2%.
Problem
Person re-identification is challenging because a person's appearance can vary drastically across cameras due to illumination, pose, camera configuration, and viewing angle.
Source: §1 · p. 1Method
The paper introduces SyRI, rendering 100 virtual humans under realistic indoor and outdoor HDR lighting, and an unsupervised domain-adaptation method that translates synthetic images for target-domain fine-tuning.
Source: §1 · p. 2Results
43.0% rank1-accuracy is achieved on PRID, and the method surpasses current state-of-the-art results by 8.2%.
Source: §5.4 · p. 13Takeaways & Limitations
The authors report that SyRI increases trained models' generalization to unseen illumination conditions without fine-tuning.
Source: §1 · p. 2The reported loss helps only to some degree, as many subjects still exhibit drastic appearance shifts.
Source: §4.2 · p. 9
Abstract
from arXiv · showhide
Drastic variations in illumination across surveillance cameras make the person re-identification problem extremely challenging. Current large scale re-identification datasets have a significant number of training subjects, but lack diversity in lighting conditions. As a result, a trained model requires fine-tuning to become effective under an unseen illumination condition. To alleviate this problem, we introduce a new synthetic dataset that contains hundreds of illumination conditions. Specifically, we use 100 virtual humans illuminated with multiple HDR environment maps which accurately model realistic indoor and outdoor lighting. To achieve better accuracy in unseen illumination conditions we propose a novel domain adaptation technique that takes advantage of our synthetic data and performs fine-tuning in a completely unsupervised way. Our approach yields significantly higher accuracy than semi-supervised and unsupervised state-of-the-art methods, and is very competitive with supervised techniques.
1 Introduction
Person re-identification is difficult because appearance varies across cameras, while current datasets offer limited lighting diversity and models may need costly fine-tuning. The paper introduces SyRI and a three-step unsupervised adaptation technique for improving performance under unseen illumination.
1 Introduction
- Appearance changes across cameras due to pose, illumination, camera configuration, and viewing angle, making it difficult to identify the same person across non-overlapping cameras.Most effective state-of-the-art approaches use supervised learning and require thousands of labeled training images.Source: §1 · p. 1
- Limited lighting diversity in re-identification datasets biases models toward training conditions, while adapting to new cameras requires tedious annotation of hundreds of subjects.Merging datasets provides only tens of lighting conditions and still requires fine-tuning for generalization.Source: §1 · pp. 1–2
- SyRI provides 100 virtual humans rendered in 140 realistic HDR illumination conditions, improving generalization to unseen lighting without fine-tuning.The humans are based on 3D scans, and the HDR maps provide indoor and outdoor lighting and backgrounds.Source: §1 · p. 2, Figure 1 · p. 2
- The proposed three-step domain adaptation translates synthetic images to target conditions and uses them to fine-tune a re-identification model unsupervised.Cycle-consistent adversarial translation is regularized to limit semantic shifts and paired with masking to focus on the foreground object.Source: §1 · p. 2
2 Related work
Prior work addresses person re-identification through supervised learning, unsupervised adaptation, and synthetic data, but unseen illumination and synthetic-to-real domain shift remain challenges. This work frames adaptation as image translation to create labeled training data under target illumination.
Person re-identification
- Supervised re-identification methods require fine-tuning for unseen illumination, while unlabeled approaches often sacrifice discriminativity and trail supervised techniques.Unsupervised approaches include transfer learning and dictionary learning.Source: §2 · p. 3
Synthetic data
- Synthetic data has supported pose estimation, pedestrian detection, and segmentation, but synthetic-real statistical differences can undermine generalization.Renderer-based realism is one response, but non-differentiable renderers limit direct classifier-loss backpropagation to them, leaving simple sampling strategies.Source: §2 · p. 3
- Rather than optimize renderer parameters, this work treats synthesis as domain adaptation, jointly making synthetic images more realistic and reducing shifts between source and target illumination.Source: §2 · p. 3
Domain adaptation
- Inspired by unsupervised image translation, the method uses domain translation to hallucinate labeled target-domain images for fine-tuning.Domain adaptation addresses dataset bias; prior methods align source and target subspaces or learn feature transformations.Source: §2 · p. 3
3 SyRI Dataset
SyRI addresses the shortage of real re-identification data spanning illumination conditions with a synthetic dataset that combines HDR environment maps, virtual people, and rendered walking scenes.
3 SyRI Dataset
- Because exhaustive annotated real illumination data is prohibitive, SyRI uses HDR environment maps to relight virtual people against realistic backgrounds.The maps are 360° real-world panoramas containing lighting information.Source: §3 · p. 4
3.1 Environment maps
- SyRI’s lighting database contains 140 HDR environment maps—40 gathered online and 100 captured as 42-photo panoramas stitched into 22-f-stop maps spanning indoor and outdoor settings.The captured panoramas used seven bracketed exposures at 60° increments.Source: §3.1 · p. 4
3.2 3D virtual humans and animations
- The dataset includes 100 virtual character prototypes with customized body shapes, clothing, textures, and colors, animated in realistic walking poses.The characters were designed using Adobe Fuse CC and animated with rigs.Source: §3.2 · pp. 4–5
3.3 Rendering
- Unreal Engine 4 renders each of 100 subjects under 140 maps at two random sphere rotations, producing 1,680,000 frames across two-second, 30-fps videos.The HDR map is texture-mapped onto a surrounding sphere used as the sole light source; rendered data and the project for generating more data will be publicly available.Source: §3.3 · p. 5
4 Method
The method adapts person re-identification to an unlabeled target illumination by selecting a similar synthetic lighting domain, translating its images toward the target, and fine-tuning a jointly trained feature network. Foreground-focused regularization helps preserve people’s appearance during translation.
4 Method
- Person re-identification is framed as adaptation across camera-specific lighting, using labeled real and synthetic source domains to address an unlabeled target illumination.The setup assumes many more synthetic than real source domains because the synthetic data spans hundreds of lighting conditions.Source: §4 · p. 5
4.2 Domain adaptation
- For each unlabeled target camera, the method selects the closest synthetic illumination, translates its images toward the target, and fine-tunes the re-identification network on the translated images.The translated synthetic images retain their known identities for fine-tuning.Source: Figure 4 · p. 6, §4.2 · pp. 6, 9
4.1 Joint learning of re-identification network
- The feature extractor is trained jointly from scratch on merged real and synthetic domains, using 3K+ identities and a 256-dimensional feature layer.The CNN predicts person identity and adopts the model from.Source: §4.1 · p. 6
- The authors attribute the pre-trained model’s remaining target-domain performance gap to incomplete illumination coverage and differences between synthetic and real image distributions.These limitations motivate making synthetic images more realistic while reducing shifts between source and target illumination.Source: §4.1 · p. 6
4.2 Domain adaptation
- A ResNet-18 classifier fine-tuned for 140 illumination classes identifies the synthetic lighting condition most similar to the target camera.The classifier predicts illumination for target images, and the selected synthetic domain is used for translation.Source: §4.2 · p. 7
- Cycle-consistent adversarial translation maps between synthetic and real images, while a soft Gaussian mask regularizes appearance near the person and permits background changes.This foreground constraint addresses semantic shifts such as a person’s shirt changing color, which can corrupt identity labels.Source: §4.2 · pp. 8–9
5 Experiments
Across five re-identification datasets, the experiments show that synthetic illumination diversity and unsupervised domain adaptation improve recognition, with the final method outperforming unsupervised competitors on four datasets and remaining competitive with supervised methods.
5 Experiments
- Experiments evaluate rank-1 CMC accuracy on VIPeR, iLIDS, CUHK01, PRID2011, and Market-1501 using established probe/gallery protocols.The study uses CUHK03 and DukeMTMC4ReID alongside SyRI to train generic feature extractors.Source: §5 · pp. 9–10
5.1 Generalization properties
- Adding 56,000 SyRI images significantly improves fully unsupervised performance over real-only training, though supervised fine-tuning remains better, especially under PRID2011’s extreme illumination.The real-only model uses CUHK03 and DukeMTMC4ReID; SyRI adds 100 identities under 140 illumination conditions. On PRID2011, adding synthetic data doubles performance, but a significant gap to supervised performance remains.Source: §5.1 · pp. 10–11, Table 1 · p. 11
5.2 Illumination inference
- Reasoning about the target illumination condition improves recognition compared with randomly selecting a source condition for domain translation.The comparison reports the minimum and 10-run average performance for random selection alongside performance using illumination inference.Source: §5.2 · pp. 11–12, Table 2 · p. 12
5.3 Image statistics of SyRI
- After domain translation, synthetic-image statistics become much closer to real-image statistics for the corresponding VIPeR and SyRI camera views.The comparison uses one VIPeR camera and its corresponding SyRI camera selected by illumination inference.Source: §5.3 · p. 12, Figure 7 · p. 12
5.4 Domain adaptation
- On PRID, synthetic data raises performance from 7% to 15%, and domain adaptation reaches 43.0% rank-1 accuracy, 8.2% above state of the art.The ablations indicate that identity mapping regularization improves performance, reference-image regularization can reduce it and produce artifacts, and a foreground mask especially helps on less tightly cropped images such as PRID.Source: Table 1 · p. 11, §5.4 · pp. 12–13
5.5 Comparison with state-of-the-art methods
- The three-step domain adaptation technique outperforms state-of-the-art unsupervised results by 5.1%, 7.2%, 8.2%, and 7.5% on VIPeR, iLIDS, PRID, and Market, respectively; on CUHK01 it achieves competitive performance to CAMEL with a 2.4% performance gap.Against unsupervised hand-crafted approaches, rank-1 margins are 11.5% on VIPeR, 13.9% on CUHK01, 7.2% on iLIDS, and 18% on PRID; the approach is also very competitive with the best supervised techniques, regardless of dataset.Source: §5.5 · p. 13
- The three-step method selects synthetic images using unlabelled test-camera examples, translates them toward the test domain with cycle-consistent adversarial learning and regularization, and uses a soft matte to preserve foreground translation.The method requires no labelled data for each target domain, supporting large-scale deployment across camera networks.Source: §5.5 · p. 14