Source-linked AI summary
Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification
Yang Zou, Xiaodong Yang, Zhiding Yu, B. V. K. Vijaya Kumar, Jan Kautz
TL;DR
Cross-domain person re-id is difficult because large domain gaps and mixed identity-related and unrelated factors limit adaptation. DG-Net++ disentangles these factors into shared appearance and domain-specific structure spaces, then applies adversarial alignment and self-training only to shared appearance features. Across six benchmark pairs, it achieves superior results with clear margins over state-of-the-art methods.
Problem
Large domain gaps hinder generalization to new re-id domains, while existing adaptation operates on representations mixing identity-related and identity-unrelated factors.
Method
DG-Net++ jointly disentangles cross-domain images into shared appearance and separate structure spaces, applying adversarial alignment and self-training exclusively to shared appearance features.
Results
DG-Net++ achieves state-of-the-art results across six cross-domain benchmark pairs, with reported margins over second-best methods in both mAP and Rank@1.
Takeaways & Limitations
Separating identity-unrelated factors provides a foundation for more targeted cross-domain adaptation, and the disentangling module can also be applied to other adaptation methods.
Abstract
from arXiv · showhide
Although a significant progress has been witnessed in supervised person re-identification (re-id), it remains challenging to generalize re-id models to new domains due to the huge domain gaps. Recently, there has been a growing interest in using unsupervised domain adaptation to address this scalability issue. Existing methods typically conduct adaptation on the representation space that contains both id-related and id-unrelated factors, thus inevitably undermining the adaptation efficacy of id-related features. In this paper, we seek to improve adaptation by purifying the representation space to be adapted. To this end, we propose a joint learning framework that disentangles id-related/unrelated features and enforces adaptation to work on the id-related feature space exclusively. Our model involves a disentangling module that encodes cross-domain images into a shared appearance space and two separate structure spaces, and an adaptation module that performs adversarial alignment and self-training on the shared appearance space. The two modules are co-designed to be mutually beneficial. Extensive experiments demonstrate that the proposed joint learning framework outperforms the state-of-the-art methods by clear margins.
1 Introduction
Cross-domain person re-identification suffers from domain gaps, while existing adaptation methods entangle identity-related and unrelated factors. DG-Net++ jointly disentangles these factors and adapts only identity-related features, with mutually beneficial modules and strong benchmark results.
- Motivation: Closed-world re-id models degrade substantially on new domains because camera, season, background, viewpoint, and illumination create large domain gaps.Relabeling identities for every new domain is prohibitively costly.
- Problem: Existing UDA methods commonly adapt representations containing both identity-related and identity-unrelated factors, allowing unrelated factors to interfere with adaptation.The paper identifies joint disentangling and adaptation as an open problem in unsupervised cross-domain re-id.
- Approach: DG-Net++ uses cross-domain cycle-consistent generation to separate a shared identity-related appearance space from source and target identity-unrelated structure spaces.The structure spaces capture pose, position, viewpoint, background, and related variations.
- Approach: Adaptation operates exclusively on the shared appearance space, while disentangling and adaptation are co-designed so each module improves the other.The framework combines targeted adaptation with mutually beneficial joint learning.
- Results: The approach achieves superior results on six benchmark pairs and can substantially advance cross-domain person re-id toward real-world deployment.The contribution passage reports the six-pair evaluation as a major outcome.
2 Related Work
Related work covers disentanglement and unsupervised domain adaptation, including feature alignment, self-training, GAN-based image translation, and prior supervised person re-id disentangling. DG-Net++ extends these ideas to unsupervised cross-domain re-id with cross-domain cycle consistency.
- Disentangling: Prior disentangling methods learn factorized or interpretable representations, decompose content and style, or extract label-relevant information for downstream tasks.These approaches span adversarial auto-encoders, InfoGAN, β-VAE, and image-to-image translation.
- Unsupervised domain adaptation: UDA methods reduce domain gaps through feature-level distribution alignment, self-training, or input-level GAN-based transfer of source-image styles.Some methods combine input-level and feature-level adaptation.
- Person re-id disentangling: DG-Net++ extends DG-Net from fully supervised within-domain re-id to unsupervised cross-domain re-id without target supervision.Its new cross-domain cycle-consistency scheme supports disentangling across domains.
3 Method
DG-Net++ jointly disentangles identity-related appearance from identity-unrelated structure and adapts only the shared appearance space. Its modules use cross-domain generation, adversarial alignment, and self-training in a mutually beneficial training process.
- 3.1 Disentangling Module: DG-Net++ encodes images with a shared appearance encoder, separate source and target structure encoders, decoders, and image and domain discriminators.Adaptation and re-identification operate on appearance codes, while structure codes represent the domain-specific factors.
- 3.1 Disentangling Module: Cross-domain cycle-consistency generation swaps appearance or structure codes between source and target images and reconstructs the original images for self-supervision.The generated images are required to preserve the corresponding appearance and structure codes, enabling reconstruction without synthetic-image labels.
- 3.1 Disentangling Module: Source identity losses train the shared appearance encoder to capture identity information and preserve identity consistency in synthesized images.The losses are applied to real source images and synthetic images retaining source appearance codes.
- 3.2 Adaptation Module: Adversarial alignment trains the shared appearance encoder to produce appearance features whose domain membership cannot be distinguished across source and target domains.Self-training complements global alignment by clustering target appearance features into pseudo-identities and refining the network with pseudo-labeled images.
- 3.4 Optimization: The modules are jointly optimized so disentanglement excludes id-unrelated noise from adaptation while adaptation improves the shared appearance encoder.Training uses a weighted total objective combining cycle-consistency, identification, and adaptation-related losses, with staged warm-up and self-training.
4 Experiments
Experiments on six cross-domain benchmark pairs show that DG-Net++ consistently outperforms competing unsupervised adaptation methods, while ablations support the value of joint disentangling and adaptation. Qualitative analyses further examine training stability, hyper-parameter sensitivity, and cross-domain image synthesis.
- Quantitative Results: DG-Net++ achieves state-of-the-art results with margins over the second-best methods across six cross-domain benchmark pairs.The reported margins range from 3.4% to 24.5% mAP and from 2.1% to 16.8% Rank@1.
- Ablation Study: 4.9%, 11.8% mAP and 7.1%, 10.4% Rank@1 improvements over baseline are obtained by disentangling alone on two cross-domain pairs.These gains are achieved without adaptation, while combining adversarial alignment and self-training gives the best performance.
- Ablation Study: 8.6% and 6.4% mAP gains show that adversarial alignment works more effectively in the disentangled id-related space on Market →Duke and Duke →Market.Disentangling also improves self-training by 4.0% and 5.7% mAP on the same pairs.
- Training Analysis: The full model consistently outperforms self-training alone during training, with stable curves and slight fluctuations after 13 epochs.The authors identify this stability as useful for model selection when target supervision is unavailable.
- Hyper-parameter Analysis: DG-Net++ is not sensitive to λcyc values from 1 to 4 or λid values from 0.25 to 1 on Market →Duke.The experiments use λcyc = 2 and λid = 0.5 in all experiments.
- Qualitative Results: Compared with CycleGAN, SPGAN, PNA-Net, and CSGLP, DG-Net++ generates more realistic images in both foreground and background across cross-domain pairs.Across six benchmark pairs, the generated images cover diverse clothing styles, seasons, poses, viewpoints, backgrounds, and illuminations, with a reported failure case.
5 Conclusion
The paper jointly disentangles identity-related and identity-unrelated factors, then adapts exclusively in the identity-related feature space. Cross-domain image generation, adversarial alignment, and self-training are co-designed to mutually promote disentangling and adaptation.
- DG-Net++ performs adaptation exclusively on identity-related features after disentangling identity-related and identity-unrelated factors.This segregates identity-unrelated noise from the adaptation process.
- Cross-domain cycle-consistent image generation, adversarial alignment, and self-training are co-designed so the disentangling and adaptation modules mutually promote each other.
A Additional Implementation Details
DG-Net++ combines appearance and domain-specific structure encoders and decoders with image and domain discriminators. Training uses iterative pseudo-labeling, source and target self-training, DBSCAN clustering, and specified optimizer configurations.
- DG-Net++ includes an appearance encoder, separate source and target structure encoders, source and target decoders, an image discriminator, and a domain discriminator.The appearance encoder is modified from ResNet50, while the structure encoders and decoders follow DG-Net designs.
- The domain discriminator is a four-layer fully connected network with 2048-dimensional input, one-dimensional output, and hidden dimensions 1024, 512, and 256.Batch normalization and LReLU follow each fully connected layer.
- Self-training generates pseudo-labels every two epochs and uses labeled source data with pseudo-labeled target data under softmax loss.DBSCAN with k-reciprocal encoding computes pairwise distances for clustering.
- The full model uses 15.8 GPU memory and runs for 460K iterations on one NVIDIA TITAN V100 GPU.
B Feature Distribution Visualization
Target-domain feature visualization shows greater identity separability and within-identity clustering for DG-Net++ than the baseline. On Market →Duke, DG-Net++ also achieves a substantially higher purity score.
- DG-Net++ produces target-domain features in which different identities are more separable and images of the same identity are more clustered than with the baseline.The comparison uses t-SNE visualization on Market →Duke.
- 76.3% purity for DG-Net++ versus 51.9% for the baseline indicates stronger intra-class similarity and inter-class difference.Purity is the proportion of correctly assigned images after assigning each cluster its most frequent identity.
C Additional Ablation Study
Additional analyses examine DBSCAN sensitivity, feature visualization, and comparisons between DG-Net++ and DG-Net across six cross-domain benchmark pairs. The reported results support coupling cross-domain disentangling with adaptation.
- Feature distribution visualization: t-SNE visualizes target-domain feature distributions for the baseline and DG-Net++ on Market →Duke.
- Sensitivity analysis: DBSCAN sensitivity is evaluated by varying ε and MinPts, with Table 3 reporting results on Market →Duke.ε controls the neighborhood distance, while MinPts controls the minimum neighbors required for a core point.
- Sensitivity analysis: DG-Net++ is overall not sensitive to the DBSCAN hyper-parameters ε and MinPts.
- DG-Net++ vs. DG-Net: DG-Net++ substantially and consistently outperforms DG-Net across all six cross-domain benchmark pairs.Table 4 presents the comparison for unsupervised cross-domain person re-identification.