Source-linked AI summary
Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification
Yuyang Zhao, Zhun Zhong, Fengxiang Yang, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe
TL;DR
Person ReID methods often require access to new-domain data, which may be unavailable, so the paper studies generalization to unseen domains from several labeled source domains. It proposes M3L, combining simulated meta-train/meta-test optimization with memory-based identification loss and MetaBN. Experiments report state-of-the-art generalization results on four large-scale ReID benchmarks.
Problem
Existing ReID methods often require new-domain data, motivating multi-source domain generalization that learns from several labeled source domains without target-domain access.
Method
M3L simulates the domain-generalization train-test process during optimization and combines meta-learning with memory-based identification loss and MetaBN.
Results
M3L achieves state-of-the-art generalization results on four large-scale ReID benchmarks.
Takeaways & Limitations
The framework trains a generalizable ReID model for evaluating directly on unseen domains.
Takeaways & Limitations
M3L and QAConv both produce poor results on CUHK03 and MSMT17, indicating substantial room remains for generalizable DG models.
Abstract
from arXiv · showhide
Recent advances in person re-identification (ReID) obtain impressive accuracy in the supervised and unsupervised learning settings. However, most of the existing methods need to train a new model for a new domain by accessing data. Due to public privacy, the new domain data are not always accessible, leading to a limited applicability of these methods. In this paper, we study the problem of multi-source domain generalization in ReID, which aims to learn a model that can perform well on unseen domains with only several labeled source domains. To address this problem, we propose the Memory-based Multi-Source Meta-Learning (M$^3$L) framework to train a generalizable model for unseen domains. Specifically, a meta-learning strategy is introduced to simulate the train-test process of domain generalization for learning more generalizable models. To overcome the unstable meta-optimization caused by the parametric classifier, we propose a memory-based identification loss that is non-parametric and harmonizes with meta-learning. We also present a meta batch normalization layer (MetaBN) to diversify meta-test features, further establishing the advantage of meta-learning. Experiments demonstrate that our M$^3$L can effectively enhance the generalization ability of the model for unseen domains and can outperform the state-of-the-art methods on four large-scale ReID datasets.
1. Introduction
Person ReID models often lose performance on unseen domains, motivating multi-source domain generalization that learns from several labeled sources without target-domain access. M3L simulates this train-test process during optimization, using memory-based identification loss and MetaBN to improve generalization.
- Motivation: Most ReID methods perform well on seen domains but suffer drastic performance decline on unseen domains.New scenes are inevitable in real-world applications, making generalization to unseen domains necessary.
- Domain Generalization: Domain generalization learns models for unseen domains without accessing target-domain data.Multi-source DG uses multiple labeled datasets with different distributions to provide more training data and scene information than single-source DG.
- Proposed Framework: M3L dynamically divides source domains into meta-train and meta-test sets to simulate the domain-generalization train-test process during optimization.Meta-train data represent source data, while meta-test data represent unseen data.
- Proposed Framework: The framework uses a non-parametric memory-based identification loss to prevent unstable meta-optimization caused by traditional parametric classifiers.This memory module is designed to harmonize identification learning with meta-learning.
- Proposed Framework: MetaBN generates diverse meta-test features and further improves the meta-learning framework.The layer is introduced to diversify features at the feature level.
2. Related Work
Related work spans supervised, unsupervised, and unsupervised domain-adaptive ReID, while domain generalization avoids target-data access. The paper positions its memory-based meta-learning approach as tailored to multi-source DG in ReID.
- Person Re-identification: Supervised ReID relies on labeled training data, whereas unsupervised domain adaptation uses labeled source data and unlabeled target data.These approaches address person ReID through different forms of annotation and target-domain access.
- Domain Generalization: Domain generalization improves performance on unseen domains without access to any target-domain data.Existing DG methods often focus on closed-set tasks, while recent work also studies generalizable models for person ReID.
- Meta Learning: M3L uses meta-learning to simulate the train-test process of domain generalization during model optimization.Its framework dynamically splits source domains into meta-train and meta-test domains and combines their losses to update the original model.
- Meta Learning: Meta-learning has been applied to model optimization, few-shot learning, and domain generalization, including closed-set and open-set DG.MAML and Reptile learn initializations for fast adaptation to new tasks.
3. Methodology
M3L simulates domain-generalization train-test behavior by splitting source domains into meta-train and meta-test sets, while memory-based identification loss and MetaBN address optimization instability and feature diversity.
- Meta-Learning for Multi-Source DG: M3L dynamically splits source domains into meta-train and meta-test sets, treating the latter as unseen data during optimization.The model is updated using losses from both stages to promote generalization across domains.
- Meta-Learning for Multi-Source DG: The meta-test loss steers optimization toward domain-invariant representations that avoid overfitting to domain bias and generalize to unseen domains.The meta-test domain acts as a proxy for an unseen target domain.
- Memory-based Identification Loss: A memory-based identification loss replaces parametric classifiers with per-domain identity centroids, reducing unstable meta-optimization and mitigating asynchronous encoder-classifier updates.Memory centroids are updated smoothly using momentum rather than an optimizer.
- MetaBN: MetaBN constructs Gaussian distributions from meta-train feature statistics, samples domain-level features, and mixes them with meta-test features to diversify meta-test inputs.The saved means and variances reflect high-level domain information aggregated over multiple identities.
- Training procedure of M3L: The training procedure combines memory-based identification and triplet losses for meta-train domains, then evaluates the optimized model on the meta-test domain.Training repeatedly selects one source domain as meta-test and the remaining domains as meta-train.
4. Experiments
Experiments evaluate M3L for multi-source domain generalization across four person ReID benchmarks, comparing its components, classifiers, source-domain settings, and state-of-the-art methods. M3L improves unseen-domain performance, with meta-learning, MetaBN, and memory-based identification loss each contributing to generalization.
- Experimental setup: Experiments use three datasets as source domains and the remaining dataset as an unseen target across Market-1501, DukeMTMC-reID, CUHK03, and MSMT17.Performance is evaluated using mAP and CMC Rank-1.
- Comparison with state-of-the-art methods: 6.7% mAP and 4.4% Rank-1 accuracy separate M3L from SNR on Market-1501 with the same backbone.M3L also exceeds QAConv50 by 12.5% mAP on Market-1501 and 3.4% mAP on DukeMTMC-reID.
- Comparison with state-of-the-art methods: M3L achieves higher results than QAConv on both CUHK03 and MSMT17, regardless of QAConv’s number of source domains.Both methods nevertheless perform poorly on CUHK03 and MSMT17, leaving substantial room for improvement.
- Effectiveness of Meta-Learning: 5.3% and 3.7% Rank-1 gains result from meta-learning with ResNet-50 on Market-1501 and CUHK03, respectively.With IBN-Net50, meta-learning improves mAP by 5.4% on Market-1501 and 2.8% on CUHK03.
- Effectiveness of MetaBN: MetaBN adds 1.3% and 1.6% Rank-1 accuracy to the ResNet-50 meta-optimized model on Market-1501 and CUHK03.The authors attribute the improvement to diversifying meta-test features.
- Losses and classifiers: The memory-based identification loss is the predominant supervision, while adding triplet loss slightly improves performance.The non-parametric classifier also gains more from meta-learning than the compared parametric classifiers.
- Effectiveness of Multi-Source: Training with more source domains consistently produces higher results on unseen domains despite introducing more domain bias.This comparison supports studying multi-source domain generalization.
5. Conclusion
The paper proposes M3L for multi-source domain generalization in person ReID, using meta-learning, a memory-based module, and MetaBN. Experiments on four large-scale benchmarks demonstrate state-of-the-art generalization results.
- Conclusion: M3L simulates the domain-generalization train-test process during training to improve generalization on unseen domains.The framework also incorporates a memory-based module and MetaBN for further improvement.
- Conclusion: M3L achieves state-of-the-art generalization results on four large-scale person ReID benchmarks.The conclusion reports extensive experiments demonstrating the framework’s effectiveness.
A. Visualization of the target domain
The appendix visualizes target-domain inference features for baseline and M3L models using t-SNE, with colors representing seven identities. The comparison is intended to clarify the meta-learning strategy’s advantage.
- Visualization setup: Figure A compares baseline and M3L t-SNE features for seven Market-1501 testing identities.Both models use ResNet-50 and are trained on DukeMTMC-reID, CUHK03, and MSMT17.
- Visualization comparison: Colors denote identity, so compare how the baseline and M3L organize the same target-domain identities in two-dimensional feature space.The passage introduces the visualization to understand the advantage of meta-learning.
B. Detailed comparison of different classifiers
The classifier comparison examines how classifier type and the number of training identities affect meta-learning improvements. The memory-based classifier becomes more advantageous as the number of identities increases.
- Classifier comparison: With fewer training identities, fully connected classifiers achieve higher improvement, but the memory-based classifier gains higher improvement as identities increase.The authors conclude that fully connected classifiers are less suitable for meta-learning when ReID has many classes.
- Classifier comparison: The large number of ReID identities leads fully connected classifiers to produce inferior meta-learning improvements compared with the memory-based classifier.The comparison concerns three classifier types evaluated across different numbers of training identities.