PaperlayerView exact revision on arXiv

Source-linked AI summary

Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification

Yixiao Ge, Dapeng Chen, Hongsheng Li

arXiv:2001.01526v2cs.CV

TL;DR

Clustering-based UDA for person re-ID is hindered by noisy target pseudo labels, limiting feature learning. MMT refines hard and soft pseudo labels and introduces a soft softmax-triplet loss, achieving reported mAP improvements across four adaptation tasks.

  • Problem

    Clustering-generated pseudo labels in unsupervised person re-ID domain adaptation contain inevitable noise that hinders target-domain feature learning.

  • Method

    MMT jointly uses off-line refined hard pseudo labels, on-line refined soft pseudo labels, and a soft softmax-triplet loss in collaborative training.

  • Results

    MMT reports 14.4%, 18.2%, 13.4%, and 16.4% mAP improvements on Market-to-Duke, Duke-to-Market, Market-to-MSMT, and Duke-to-MSMT, respectively.

  • Takeaways & Limitations

    Refining pseudo labels and learning with soft triplet labels supports more discriminative target-domain person representations and strong UDA performance.

Abstract

from arXiv · show

Person re-identification (re-ID) aims at identifying the same persons' images across different cameras. However, domain diversities between different datasets pose an evident challenge for adapting the re-ID model trained on one dataset to another one. State-of-the-art unsupervised domain adaptation methods for person re-ID transferred the learned knowledge from the source domain by optimizing with pseudo labels created by clustering algorithms on the target domain. Although they achieved state-of-the-art performances, the inevitable label noise caused by the clustering procedure was ignored. Such noisy pseudo labels substantially hinders the model's capability on further improving feature representations on the target domain. In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner. In addition, the common practice is to adopt both the classification loss and the triplet loss jointly for achieving optimal performances in person re-ID models. However, conventional triplet loss cannot work with softly refined labels. To solve this problem, a novel soft softmax-triplet loss is proposed to support learning with soft pseudo triplet labels for achieving the optimal domain adaptation performance. The proposed MMT framework achieves considerable improvements of 14.4%, 18.2%, 13.1% and 16.4% mAP on Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT unsupervised domain adaptation tasks. Code is available at https://github.com/yxgeee/MMT.

1 INTRODUCTION

Domain gaps cause person re-ID models trained on one camera dataset to drop in performance on another, motivating UDA without target identity annotations. MMT addresses noisy clustering pseudo labels through refined hard and soft supervision, alongside a soft softmax-triplet loss.

  • Motivation: Domain gaps cause evident performance drops when person re-ID models trained on existing datasets encounter images from new camera systems.UDA adapts a source-domain model with identity labels to an unlabeled target image domain.
  • Motivation: Clustering-based UDA methods train on target pseudo labels, but inevitable label noise substantially hinders feature learning.The noise arises partly from limited source-feature transferability and uncertainty about target identities.
  • Method: MMT refines pseudo labels using joint supervision from off-line refined hard labels and on-line refined soft labels.Its peer networks use temporally averaged models to produce soft labels for one another and reduce training error amplification.
  • Method: A soft softmax-triplet loss enables triplet learning with softly refined pseudo labels.The collaborative two-network strategy is used only during training; inference retains one network without extra computational or memory cost.
  • Results: 14.4%, 18.2%, 13.4%, and 16.4% mAP improvements are reported on Market-to-Duke, Duke-to-Market, Market-to-MSMT, and Duke-to-MSMT tasks, respectively.These are reported comparisons with state-of-the-art methods.

2 RELATED WORK

Related work spans clustering-based UDA, style-transferred source images, generic domain adaptation, teacher-student learning, and noisy-label handling. The paper positions clustering-based methods as strong performers while emphasizing their unresolved pseudo-label noise.

  • UDA for person re-ID: Clustering-based UDA methods alternate pseudo-label assignment for unlabeled samples with network optimization and remain state-of-the-art for person re-ID.Existing approaches include bottom-up clustering with a repelled loss and hard pseudo labels for global and local features.
  • UDA for person re-ID: Style-transfer methods transform source-domain images toward target-domain styles while attempting to preserve relevant identity information.SPGAN and PTGAN are cited as examples of this category.
  • Generic domain adaptation: Generic domain adaptation methods reduce source-target distribution differences using domain classifiers that suppress discriminative domain information in learned features.The cited methods use adversarial learning for close-set recognition.
  • Teacher-student models: Teacher-student models create consistent supervision for labeled or unlabeled data from different models’ predictions.Temporal ensembling averages sample predictions, while mean-teacher models average model parameters.
  • Noisy-label learning: Noisy-label methods include loss correction and direct label correction, but estimating a noise-transition matrix is difficult for clustering-generated pseudo labels in unsupervised person re-ID.This motivates specialized treatment of pseudo-label noise in the target domain.

3 PROPOSED APPROACH

MMT addresses noisy clustering-based pseudo labels in unsupervised person re-ID adaptation through collaborative refinement of hard and soft target-domain supervision. It also introduces a soft softmax-triplet loss to train with refined soft labels.

  • Mutual Mean-Teaching framework: MMT jointly optimizes two collaborative networks with off-line refined hard pseudo labels and on-line refined soft pseudo labels.The target pseudo classes remain cluster-based, while network predictions provide additional soft supervision for cross-network training.
  • Training and inference: MMT jointly uses soft classification and soft softmax-triplet losses, and selects a better-validated temporal average model for inference.Average models are used because they perform better than models with current parameters.
  • Clustering-based UDA revisit: Clustering-based UDA alternates target-domain feature clustering with network optimization using generated pseudo labels, but clustering errors hinder subsequent feature learning.The errors arise from imperfect features and clustering algorithms.
  • Mutual Mean-Teaching framework: Temporal average models generate peer supervisions, helping keep the networks’ predictions less related and avoid error amplification during training.The temporal averages use an ensembling momentum α within [0, 1).
  • Soft softmax-triplet loss: The conventional triplet loss cannot use refined soft labels, motivating a soft softmax-triplet loss for learning from soft triplet supervision.The soft triplet labels are generated by the other network’s past temporal average model and fixed as training supervisions.

4 EXPERIMENTS

Experiments evaluate MMT across three datasets and four adaptation tasks, showing strong performance against existing methods and validating its pseudo-label refinement components through ablations.

  • Experimental Settings: MMT is evaluated on Market-1501, DukeMTMC-reID, and MSMT17 across Duke-to-Market, Market-to-Duke, Duke-to-MSMT, and Market-to-MSMT tasks.Mean average precision and CMC accuracies are used for domain-adaptation evaluation.
  • Comparison with State-of-the-Art Methods: MMT significantly outperforms existing approaches with both ResNet-50 and IBN-ResNet-50 backbones, nearly approaching fully supervised performance without target annotations.No re-ranking or multi-query fusion is used.
  • Comparison with State-of-the-Art Methods: 11.7% and 12.9% mAP gains over SSG are achieved on Market-to-Duke and Duke-to-Market with ResNet-50.The comparison uses simpler network architectures and lower output feature dimensions.
  • Comparison with State-of-the-Art Methods: MMT achieves state-of-the-art performance with pseudo-class counts both below and above the target domains’ actual identity numbers.The result supports pseudo-label refinement under imperfect cluster-count specification.
  • Ablation Studies: Removing online refined soft pseudo labels decreases mAP by 17.7% and 14.9% with ResNet-50 on Duke-to-Market and Market-to-Duke.IBN-ResNet-50 also shows decreases of 13.8% and 10.7%.
  • Ablation Studies: Removing the soft softmax-triplet loss causes mAP drops of 5.3% and 4.8% on Market-1501 with ResNet-50 and IBN-ResNet-50.Market-to-Duke drops are 3.6% and 4.0% for the two backbones.
  • Ablation Studies: Ablations show that hard pseudo-label classification remains foundational, while soft triplet supervision further improves feature learning.Removing hard-label supervision produces much lower performance, and removing soft triplet supervision causes distinct drops.

5 CONCLUSION

MMT addresses noisy pseudo labels in clustering-based unsupervised domain adaptation for person re-ID through collaborative hard- and soft-label refinement, achieving improvements of up to 18.2%.

  • Conclusion: MMT refines target-domain pseudo labels using offline refined hard labels and online refined soft labels in collaborative training.The framework is designed to better model inter-sample relations in the target domain.
  • Conclusion: The soft softmax-triplet loss enables learning with softly refined triplet labels.It is introduced to support optimal domain-adaptation performance.
  • Conclusion: 18.2% is the maximum reported improvement achieved by MMT on the domain-adaptation task.The conclusion states that MMT significantly outperforms existing person re-ID methods.

A.1 FUNCTIONS OF TEMPORAL AVERAGE MODELS IN MMT

Temporal average models provide complementary and robust soft pseudo labels, with their predictions compared against ordinary networks during training.

  • Functions of Temporal Average Models in MMT: Temporal average models produce more complementary and robust soft pseudo labels than ordinary networks.The comparison is between Proposed MMT-500 and Proposed MMT-500 (w/o E[θ]).
  • Functions of Temporal Average Models in MMT: The two temporal average models are introduced to provide complementary soft labels and avoid training error amplification.They ensemble past parameters to produce more independent predictions.

A.2 PARAMETER ANALYSIS

The parameter analysis evaluates how weighting hard and soft classification and triplet losses affects MMT-500 across adaptation tasks and backbones. Performance is generally robust, but eliminating hard classification loss causes substantial declines.

  • Triplet-loss weighting: IBN-ResNet-50 achieves its best Market-to-Duke results at λt_tri = 0.5, while all tested settings outperform state-of-the-art methods.The optimal triplet-loss weight depends on the backbone and task.
  • Classification-loss weighting: The classification-loss analysis varies λt_id, the weighting factor for hard and soft classification losses, while holding other parameters fixed.The parameter was tuned at λt_id = 0.5 in the reported experiments.
  • Classification-loss weighting: Removing hard classification loss at λt_id = 1.0 causes considerable performance declines, showing that hard classification loss is essential.Only soft labels are used when the weighting factor is 1.0, whereas λt_id < 1 retains hard-label supervision; the latter settings outperform state-of-the-art methods.
Loading 2001.01526v2…