Source-linked AI summary
Generalizable Person Re-identification with Relevance-aware Mixture of Experts
Yongxing Dai, Xiaotong Li, Jun Liu, Zekun Tong, Ling-Yu Duan
TL;DR
DG ReID must generalize from labeled source domains to unseen targets without target data, yet conventional pipelines can overlook source-specific characteristics and target–source relevance. RaMoE preserves diverse source experts, adaptively votes among their features, and learns that voting through episodic relation alignment; the paper reports improved performance over state-of-the-art DG ReID methods.
Problem
DG ReID lacks unseen target-domain training data, while conventional hybrid-dataset single models may neglect source domains’ discriminative characteristics and target-domain relevance.
Method
RaMoE trains decorrelated source-domain experts and a relevance-aware voting network, updated with relation alignment through a learning-to-learn episodic procedure.
Results
RaMoE outperforms state-of-the-art DG ReID approaches by a large margin, with decorrelation improving Rank-1 by 1.3% on PRID over the no-decorrelation variant.
Takeaways & Limitations
Adaptive mixtures can exploit complementary source-domain characteristics and domain relevance to produce more generalizable features for unseen target domains.
Abstract
from arXiv · showhide
Domain generalizable (DG) person re-identification (ReID) is a challenging problem because we cannot access any unseen target domain data during training. Almost all the existing DG ReID methods follow the same pipeline where they use a hybrid dataset from multiple source domains for training, and then directly apply the trained model to the unseen target domains for testing. These methods often neglect individual source domains' discriminative characteristics and their relevances w.r.t. the unseen target domains, though both of which can be leveraged to help the model's generalization. To handle the above two issues, we propose a novel method called the relevance-aware mixture of experts (RaMoE), using an effective voting-based mixture mechanism to dynamically leverage source domains' diverse characteristics to improve the model's generalization. Specifically, we propose a decorrelation loss to make the source domain networks (experts) keep the diversity and discriminability of individual domains' characteristics. Besides, we design a voting network to adaptively integrate all the experts' features into the more generalizable aggregated features with domain relevance. Considering the target domains' invisibility during training, we propose a novel learning-to-learn algorithm combined with our relation alignment loss to update the voting network. Extensive experiments demonstrate that our proposed RaMoE outperforms the state-of-the-art methods.
1. Introduction
DG ReID trains on labeled multi-source domains without target-domain data, but conventional single-model pipelines can neglect domain-specific characteristics and target–source relevance. RaMoE addresses both issues by combining diverse source experts through relevance-aware voting, with decorrelation and learning-to-learn relation alignment.
- DG ReID learns from labeled multiple source domains without using target-domain training data, then evaluates on unseen target domains.
- Conventional DG ReID trains one model on a hybrid source dataset, potentially overlooking individual domains’ discriminative characteristics and target-domain relevance.
- RaMoE trains source-specific domain experts whose complementary characteristics are adaptively integrated according to unseen target-domain relevance.
- A decorrelation loss reduces correlation among source experts to preserve diverse, complementary, and discriminative domain information.
- A learning-to-learn algorithm with relation alignment trains the voting network by simulating unseen targets through episodic meta-train and meta-test domain splits.
- RaMoE is presented as the first DG ReID approach to formulate the task as a mixture-of-experts paradigm with voting-based feature integration.
2. Related Work
Prior work covers supervised and unsupervised ReID adaptation, domain-generalization strategies, and mixture-of-experts systems. RaMoE differs by using a learnable voting network updated through a meta-learning algorithm for DG ReID.
- Person Re-Identification: Supervised ReID methods address cross-camera identification, while UDA methods use unlabeled target data to adapt source-trained models against domain bias.
- Domain Generalization: DG ReID learns generalizable models from multiple source domains without target training data, extending domain-generalization research beyond standard classification settings.
- Domain Generalization: General DG methods pursue domain-invariant features, source-data augmentation, or episodic meta-learning to improve generalization to unseen domains.
- Mixture of Experts: Mixture-of-experts systems combine separated networks that specialize in subsets of data, while RaMoE adds a learnable voting network updated through meta-learning.
3. Methodology
RaMoE trains domain-specific experts to preserve complementary source-domain characteristics, then uses a voting network to weight and aggregate their features according to domain relevance. Meta-learning with relation alignment trains this voting network without access to unseen target-domain data.
- Overview: RaMoE trains separate source-domain experts so each learns discriminative features from its individual domain.The shared backbone is paired with branch networks for the source domains.
- Meta Optimizing: Meta-learning randomly separates source domains into meta-train and meta-test sets to update the voting network without unseen target-domain samples.The procedure computes domain and relation-alignment losses at meta-training and meta-testing stages before meta-optimizing the voting network.
- Optimizing Domain-specific Experts: The metric losses improve expert discriminability, while decorrelation reduces correlation among experts to preserve diverse and complementary information.The domain loss combines metric and decorrelation objectives.
- Optimizing the Voting Network: A voting network computes source-domain relevance and integrates expert features into a weighted aggregated feature for unseen target samples.Relevance scores are normalized before weighting and combining the expert features.
- Optimizing the Voting Network: Relation alignment pushes the aggregated feature to match the discriminative relationship of the domain-specific expert feature.The loss uses metric relationships for the aggregated and expert features and trains the voting network to produce reliable relevance scores.
4. Experiments
Experiments evaluate RaMoE across multiple DG ReID protocols, datasets, ablations, source-domain tests, and relevance visualizations. Results show gains from domain decorrelation, relevance-aware voting, and adaptive integration of source experts.
- Datasets and Evaluation Settings: Experiments use public ReID datasets, with mAP and CMC as evaluation metrics across multiple domain-generalization protocols.The protocols define different source-target splits, including leave-one-out evaluation and tests on small unseen datasets.
- Comparison with the State-of-the-Arts: RaMoE outperforms state-of-the-art DG ReID methods by a large margin under Protocol-1 and Protocol-2.The comparison evaluates models trained on source domains and applied to unseen target domains.
- Comparison with the State-of-the-Arts: RaMoE’s superiority under Protocol-3 demonstrates adaptive integration of source-domain characteristics for domain generalization.The protocol-3 comparison is conducted against the baseline on large-scale ReID datasets.
- Ablation Study: 1.3% higher Rank-1 accuracy on PRID shows that decorrelation improves the complementarity of source-domain experts.Without the decorrelation loss, experts provide less complementary information and aggregated features generalize less effectively.
- Ablation Study: 1.7%, 1.6%, 1.6%, and 0.9% mAP gains on PRID, GRID, VIPeR, and iLIDs show the benefit of relevance-aware voting over direct expert concatenation.Voting emphasizes more relevant experts and reduces the influence of less relevant ones; on iLIDs, Duke is the strongest and Market the weakest expert.
- Ablation Study: Aggregating individual experts produces more generalizable features than using individual experts alone, while concatenation preserves more information than summation.The experiments select softmax weighting with concatenation for feature integration.
5. Conclusion
RaMoE addresses DG ReID by preserving source domains’ diverse characteristics and estimating unseen-target relevance to form more generalizable features. Its decorrelation loss, voting network, and meta-learning-based relation alignment together support adaptive expert aggregation.
- RaMoE preserves source domains’ diverse and discriminative characteristics through a decorrelation loss, enabling experts to provide complementary information.
- A voting network adaptively integrates source experts’ features according to the unseen target domain’s relevance to source domains.
- Meta-learning combined with relation alignment loss updates the voting network to estimate more accurate relevance for unseen target domains.
- Extensive experiments demonstrate the effectiveness of RaMoE for domain generalizable person ReID.