PaperlayerView exact revision on arXiv

Source-linked AI summary

Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification

Zhun Zhong, Liang Zheng, Zhiming Luo, Shaozi Li, Yi Yang

arXiv:1904.01990v1cs.CVcs.LG

TL;DR

Unsupervised person re-ID must adapt from labeled source identities to an unlabeled target domain whose intra-domain variations are often neglected. The paper uses exemplar-, camera-, and neighborhood-invariance enforced through an exemplar memory, and reports state-of-the-art UDA accuracy on three large-scale domains.

  • Problem

    UDA person re-ID seeks discriminative target representations from labeled source and unlabeled target data, but existing methods largely overlook target-domain intra-domain variations.

  • Method

    An exemplar memory stores up-to-date target features and enforces exemplar-, camera-, and neighborhood-invariance over the global target training set alongside source classification.

  • Results

    The method achieves a new state of the art in UDA accuracy across Market-1501, DukeMTMC-reID, and MSMT17, outperforming prior approaches by a large margin.

  • Takeaways & Limitations

    The experiments identify the three invariance properties and exemplar memory as effective components for improving transferable deep re-ID models.

  • Takeaways & Limitations

    Neighborhood-invariance assumes each target image belongs to classes represented by its top-k candidate neighbors, which may include different identities.

Abstract

from arXiv · show

This paper considers the domain adaptive person re-identification (re-ID) problem: learning a re-ID model from a labeled source domain and an unlabeled target domain. Conventional methods are mainly to reduce feature distribution gap between the source and target domains. However, these studies largely neglect the intra-domain variations in the target domain, which contain critical factors influencing the testing performance on the target domain. In this work, we comprehensively investigate into the intra-domain variations of the target domain and propose to generalize the re-ID model w.r.t three types of the underlying invariance, i.e., exemplar-invariance, camera-invariance and neighborhood-invariance. To achieve this goal, an exemplar memory is introduced to store features of the target domain and accommodate the three invariance properties. The memory allows us to enforce the invariance constraints over global training batch without significantly increasing computation cost. Experiment demonstrates that the three invariance properties and the proposed memory are indispensable towards an effective domain adaptation system. Results on three re-ID domains show that our domain adaptation accuracy outperforms the state of the art by a large margin. Code is available at: https://github.com/zhunzhong07/ECN

1. Introduction

The paper frames person re-ID adaptation as an open-set problem where target-domain intra-domain variations are overlooked. It proposes three target invariances and an exemplar memory to improve transfer performance.

  • 1. Introduction: UDA person re-ID learns target representations from labeled source and unlabeled target data, but source and target identities are completely different.This makes person re-ID an open-set adaptation problem rather than the conventional closed-set setting.
  • 1. Introduction: Existing approaches mainly reduce inter-domain gaps while overlooking target-domain intra-domain variations that critically influence re-ID testing.The paper focuses on exemplar-, camera-, and neighborhood-invariance as underlying target-domain factors.
  • 1. Introduction: Exemplar-invariance separates each target exemplar from others, camera-invariance aligns an exemplar with its CamStyle images, and neighborhood-invariance pulls exemplars toward neighbors.The three constraints address different forms of target-image variation.
  • 1. Introduction: An exemplar memory stores up-to-date target representations and enforces invariance constraints over the global target training set instead of only a mini-batch.This uses sample similarity across the whole target set during optimization.
  • 1. Introduction: Experiments show the three invariance properties improve transferable re-ID ability, while the memory improves accuracy with very limited additional computation and GPU memory.The method outperforms state-of-the-art UDA approaches by a large margin on Market-1501, DukeMTMC-reID, and MSMT17.

2. Related Work

The related work discusses domain-distribution alignment and unsupervised person re-ID methods, while distinguishing this paper through joint modeling of three target invariances with memory.

  • Unsupervised domain adaptation: Closed-set UDA commonly aligns source and target feature distributions using MMD or adversarial domain classifiers, but unknown target classes should not be aligned with the source.Open-set methods instead discard or reject unknown-class target samples during alignment.
  • Unsupervised person re-identification: Unsupervised person re-ID methods adapt source-trained models through target metric learning, clustering, or joint use of labeled source and unlabeled target data.Supervised re-ID performance can drop significantly on unseen datasets.
  • Difference from previous works: The proposed framework combines a source classification module with a target exemplar-memory module that computes invariance-learning loss.The memory stores target representations while source data provide cross-entropy supervision.
  • Difference from previous works: Unlike prior works that separately present related properties, this method jointly considers exemplar-, camera-, and neighborhood-invariance for target-domain adaptation.Its neighborhood loss uses soft classification over top-k neighbors rather than a triplet loss over mutual top-1 neighbors.

3. The Proposed Method

The method combines supervised source-domain classification with target-domain invariance learning through an exemplar memory. It models exemplar, camera, and neighborhood invariance while balancing source and target losses.

  • 3.1. Overview of Framework: The framework uses a supervised classification module for labeled source images and an exemplar memory module for unlabeled target images.The memory stores up-to-date FC-4096 features for every target image and estimates similarities against mini-batch and memory samples.
  • 3.3. Exemplar Memory: The exemplar memory treats each target image as an individual category and stores its normalized feature and index label in a key-value structure.The memory contains one slot per target image, with fixed index labels and features updated during training.
  • 3.4. Invariance Learning for Target Domain: Exemplar-invariance separates individual target exemplars by encouraging each image to remain close to itself and distant from others.The method regards the Nt target images as Nt different classes and optimizes a negative log-likelihood objective.
  • 3.4. Invariance Learning for Target Domain: Camera-invariance treats each real target image and its camera-style-transferred counterparts as sharing an identity.CamStyle augments each image from camera c with C −1 images in other camera styles, and the loss forces different styles of the same sample closer.
  • 3.4. Invariance Learning for Target Domain: Neighborhood-invariance identifies k-nearest target neighbors in memory and encourages the query to belong to their candidate classes.The soft-label objective excludes the query’s own class when distinguishing neighborhood-invariance from exemplar-invariance.
  • 3.5. Final Loss for Network: The final objective combines source and target losses, while exemplar- and neighborhood-invariance create a trade-off between separating different identities and grouping same-identity exemplars.The parameter λ controls the relative importance of source and target losses.

4. Experiment

Experiments evaluate parameter sensitivity, invariance components, exemplar-memory cost, and comparisons with supervised, source-only, and state-of-the-art methods across three re-ID datasets.

  • Experiment Setting: The experiments use CMC and mAP on Market-1501, DukeMTMC-reID, and MSMT17, with ResNet-50 as the backbone.The first two residual layers are fixed to save GPU memory.
  • Parameter Analysis: β around 0.05 produces the best results, whereas β = 0.01 prevents network convergence.Lower β reduces entropy but becomes harmful when too low.
  • Parameter Analysis: Considering target-domain invariance learning significantly improves the source-only baseline for every λ > 0, including λ = 1 using only unlabeled target data.Results are best and change marginally when λ ranges from 0.3 to 0.8.
  • Parameter Analysis: Neighborhood-invariance improves performance when k > 1, with rank-1 accuracy and mAP best at k between 6 and 8 before declining for larger k.The decline is attributed to including excess false-positive samples.
  • Evaluation: Adding exemplar-, camera-, and neighborhood-invariance successively improves target-domain performance, including rank-1 gains from 43.1% to 48.7% and from 28.9% to 34.2% with exemplar-invariance.Camera-invariance raises one Market-1501 rank-1 result by 14.4%, while neighborhood-invariance raises another by 12%.
  • Comparison with State-of-the-art Methods: The method outperforms state-of-the-art UDA approaches by a large margin on Market-1501, DukeMTMC-reID, and MSMT17.Using DukeMTMC-reID as source for MSMT17, it achieves rank-1 accuracy = 30.2% and mAP = 10.2%, exceeding PTGAN by 18.4% and 6.9%, respectively.

5. Conclusion

The paper proposes an exemplar-memory-based UDA method for person re-ID that enforces target-domain invariance constraints during training. Experiments demonstrate improved transferable ability and state-of-the-art UDA accuracy across three large-scale domains.

  • The method uses exemplar memory to evaluate relationships between target samples during UDA training.This enables direct enforcement of target-domain invariance constraints.
  • Experiments demonstrate that invariance learning improves the transferable ability of deep re-ID models.
  • The approach achieves new state-of-the-art UDA accuracy on three large-scale re-ID domains.
Loading 1904.01990v1…