Source-linked AI summary
Joint Visual and Temporal Consistency for Unsupervised Domain Adaptive Person Re-Identification
Jianing Li, Shiliang Zhang
TL;DR
Unsupervised domain-adaptive ReID must learn across source-target domain gaps without labeled target data. The paper combines SAC for local one-hot classification with MTC for global multi-class prediction using visual and temporal consistency, achieving strong results across three datasets, including 79.5% rank1 on Market-1501 unsupervised and 86.8% after domain transfer.
Problem
Unsupervised domain-adaptive ReID lacks labeled target data and faces substantial source-target domain gaps, while supervised methods require expensive annotations.
Method
The framework combines SAC for local batch-level one-hot classification with MTC for global multi-class label prediction using visual similarity and temporal consistency.
Results
79.5% rank1 accuracy is reported on Market-1501 with unsupervised training and 86.8% after unsupervised domain transfer, with superiority reported across three datasets.
Takeaways & Limitations
Temporal consistency supports label prediction and model training without extra annotations or manual alignments, while the unified framework leverages unlabeled data for discriminative feature learning.
Takeaways & Limitations
Label accuracy depends on using a sufficiently small target-domain batch size and careful sampling for local labels, while global labels require accurate visual-temporal prediction.
Abstract
from arXiv · showhide
Unsupervised domain adaptive person Re-IDentification (ReID) is challenging because of the large domain gap between source and target domains, as well as the lackage of labeled data on the target domain. This paper tackles this challenge through jointly enforcing visual and temporal consistency in the combination of a local one-hot classification and a global multi-class classification. The local one-hot classification assigns images in a training batch with different person IDs, then adopts a Self-Adaptive Classification (SAC) model to classify them. The global multi-class classification is achieved by predicting labels on the entire unlabeled training set with the Memory-based Temporal-guided Cluster (MTC). MTC predicts multi-class labels by considering both visual similarity and temporal consistency to ensure the quality of label prediction. The two classification models are combined in a unified framework, which effectively leverages the unlabeled data for discriminative feature learning. Experimental results on three large-scale ReID datasets demonstrate the superiority of proposed method in both unsupervised and unsupervised domain adaptive ReID tasks. For example, under unsupervised setting, our method outperforms recent unsupervised domain adaptive methods, which leverage more labels for training.
1 Introduction
The paper addresses unlabeled target-domain ReID by jointly using local one-hot classification and global multi-class label prediction based on visual and temporal consistency. Its unified framework combines SAC and MTC to learn discriminative features and reports strong results in unsupervised and domain-adaptive settings.
- Supervised ReID requires expensive labeled data, while source-trained models can suffer substantial performance drops on different target domains.
- The method generates more reliable labels by combining local batch-level separation with global label prediction using visual similarity and temporal consistency.Temporal consistency uses video frame numbers without extra annotations or manual alignments.
- SAC assigns distinct person IDs within each training batch and uses a self-adaptive classifier for local one-hot classification.The classifier is generated from image features, while MTC performs global multi-class classification.
- MTC predicts global multi-class labels using visual similarity, temporal consistency, and memory-bank features to improve clustering reliability.Temporal cues can filter hard negatives with similar appearance and reduce the ReID search space.
- 79.5% rank1 accuracy is achieved on Market-1501 with unsupervised training, while 86.8% is achieved after unsupervised domain transfer.Experiments on three large-scale person ReID datasets report superiority over existing unsupervised and domain-adaptive methods.
- The framework’s novel components are SAC, MTC, and temporal consistency that requires no extra annotations or manual alignments.The temporal cue is used for both model training and ReID similarity computation.
2 Related Work
Prior work in unsupervised domain-adaptive ReID transfers models or pseudo-labeling strategies to unlabeled target data, but commonly relies only on visual similarity. The paper instead combines visual similarity with temporal consistency to improve robustness to visual variation and domain bias.
- Unsupervised domain adaptation generally seeks to align source and target domain distributions, often by minimizing their domain discrepancy.
- Earlier unsupervised domain-adaptive ReID methods included handcrafted features, source-domain transfer, soft-label learning, and clustering-based pseudo-label estimation.
- Most existing methods rely only on visual similarity for unlabeled feature learning, making them vulnerable to visual variation and domain bias.
- This work jointly uses visual similarity and temporal consistency for feature learning and reports stronger robustness and performance under unsupervised and domain-adaptive settings.It is distinguished from another temporal-cue ReID work that uses temporal information only for supervised-training re-ranking.
3 Proposed Method
The framework learns discriminative target-domain features by combining local one-hot classification with global multi-class classification, using source supervision when available. SAC handles batch-level separation, while MTC clusters the full target set using visual and temporal consistency.
- 3.1 Formulation: The goal is to learn a discriminative feature extractor for an unlabeled target domain using labeled source images and target images.
- 3.2 Self-Adaptive Classification: SAC assigns distinct one-hot labels within each target-domain training batch and separates the sampled images through classification.The batch size is assumed much smaller than the full target set, making distinct identities likely among sampled images.
- 3.3 Memory-based Temporal-guided Cluster: MTC assigns each target image a multi-class label by clustering images with similar visual features and temporally consistent camera-frame relationships.Images in the same cluster receive the same label, representing images regarded as containing the same person.
- 3.1 Formulation: The global target loss uses memory-bank features to classify images against multi-class labels, while the overall loss combines source, local, and global objectives.The loss weights w1 and w2 control the local and global target-loss contributions.
- 3.2 Self-Adaptive Classification: SAC builds a self-adaptive classifier from original and augmented image features, with averaged features improving classifier robustness to visual variations.Camera-style transfer generates augmented images, and the classifier averages each image with its augmentations.
- 3.3 Memory-based Temporal-guided Cluster: Joint visual-temporal similarity produces more discriminative retrieval results than visual similarity alone in the illustrated DukeMTMC-reID examples.The figure compares top-5 retrievals from visual similarity with retrievals using the joint similarity in Eq. (12).
4 Experiment
Experiments evaluate JVTC’s components, hyperparameters, and comparisons across Market1501, DukeMTMC-reID, and MSMT17. Results show gains from SAC, MTC, their combination, and joint similarity, with strong performance against existing methods.
- Experimental Setup: Experiments use Market1501, DukeMTMC-ReID, and MSMT17, with performance evaluated by CMC and mAP under standard unsupervised and domain-adaptive settings.JVTC uses a ResNet50 backbone with a 512-dimensional embedding layer; training follows the reported implementation settings.
- Ablation Study: 44.2% and 48.4% rank1 accuracy follow direct transfer on Market1501 and DukeMTMC-reID, respectively, illustrating substantial cross-domain degradation.The paper attributes this performance drop mainly to domain bias between datasets.
- Ablation Study: SAC improves rank1 accuracy from 42.1% to 64.5% on Market-1501 and from 31.8% to 59.4% on DukeMTMC-reID.These comparisons show the effect of applying local one-hot classification with SAC relative to direct transfer.
- Ablation Study: MTC improves mAP over the baseline by 9.8% on Market1501 and 5.2% on DukeMTMC-reID, while JVTC reaches 61.1% mAP on Market1501 versus 46.6% for the baseline.MTC uses visual similarity and temporal consistency, and JVTC combines MTC with SAC.
- Hyper-parameter Analysis: Smaller temperature factors generally improve ReID performance but can hinder convergence when too small; the selected values are β1 = 0.1 and β2 = 0.05.The experiments vary one hyperparameter at a time while keeping the others fixed on both datasets.
- Hyper-parameter Analysis: The best loss-weight setting on both datasets is w1 = 1 and w2 = 0.2; further increases substantially reduce performance.The paper explains that increasing these weights decreases the weight of Lsrc.
- Comparison with State-of-the-art Methods: 72.9% rank1 accuracy on Market1501 exceeds BUC’s 66.2% and DBC’s 69.2% under unsupervised learning.The paper relates the gain to using visual similarity and temporal consistency for label prediction together with SAC.
- Comparison with State-of-the-art Methods: 83.8% and 75.0% rank1 accuracy on Market1501 and DukeMTMC-reID, respectively, are the best reported results among compared unsupervised domain-adaptive methods.On MSMT17 with DukeMTMC-reID as source, JVTC reaches 45.4% rank1 accuracy and 52.9% after applying joint similarity.
5 Conclusion
The paper combines SAC and MTC in a unified framework for unsupervised and domain adaptive person ReID. Experiments on three datasets show superiority over state-of-the-art methods.
- SAC assigns distinct person IDs within each training batch and uses a self-adaptive classifier to separate images.
- MTC predicts multi-class labels using visual similarity and temporal consistency to improve label prediction quality.
- The unified SAC-MTC framework learns discriminative target-domain features and outperforms state-of-the-art unsupervised and domain adaptive ReID methods on three datasets.