Source-linked AI summary
Joint Generative and Contrastive Learning for Unsupervised Person Re-identification
Hao Chen, Yaohui Wang, Benoit Lagadec, Antitza Dantcheva, Francois Bremond
TL;DR
Unsupervised ReID needs view-invariant representations without relying on labeled source data. The paper jointly trains a GAN-based mesh view generator with contrastive learning and reports strong performance in both fully unsupervised and UDA settings.
Problem
Existing contrastive methods rely on traditional augmentation, while GAN-based unsupervised ReID methods commonly require a labeled source dataset.
Method
The method jointly trains a 3D mesh-guided view generator and a contrastive module using a view-invariant loss on original and generated views.
Results
The method significantly outperforms state-of-the-art methods under both fully unsupervised and unsupervised domain adaptive settings.
Takeaways & Limitations
Generative and contrastive modules mutually promote view-invariant representation learning and generation quality without requiring a source dataset.
Abstract
from arXiv · showhide
Recent self-supervised contrastive learning provides an effective approach for unsupervised person re-identification (ReID) by learning invariance from different views (transformed versions) of an input. In this paper, we incorporate a Generative Adversarial Network (GAN) and a contrastive learning module into one joint training framework. While the GAN provides online data augmentation for contrastive learning, the contrastive module learns view-invariant features for generation. In this context, we propose a mesh-based view generator. Specifically, mesh projections serve as references towards generating novel views of a person. In addition, we propose a view-invariant loss to facilitate contrastive learning between original and generated views. Deviating from previous GAN-based unsupervised ReID methods involving domain adaptation, we do not rely on a labeled source dataset, which makes our method more flexible. Extensive experimental results show that our method significantly outperforms state-of-the-art methods under both, fully unsupervised and unsupervised domain adaptive settings on several large scale ReID datsets.
1. Introduction
Unsupervised ReID needs view-invariant representations without costly identity labels or dependence on a labeled source dataset. The paper jointly trains generative and contrastive modules, using mesh-based novel views and view-invariant learning to improve unsupervised ReID.
- Motivation: Self-supervised contrastive learning improves unsupervised ReID by matching representations of augmented views while preserving identity-related information.GAN-based augmentation can modify identity-unrelated features substantially while preserving identity-related features.
- Motivation: Previous GAN-based methods commonly formulate unsupervised ReID as UDA, requiring a large-scale labeled source dataset alongside unlabeled target images.Source-dataset scale and quality strongly affect UDA performance.
- Framework: The proposed framework jointly trains generative and contrastive modules that mutually promote one another through a shared identity feature encoder.Generated views improve contrastive representation learning, while improved representations enhance generation quality.
- Framework: A 3D mesh-based view generator rotates estimated meshes to create novel structures while preserving body shape more effectively than skeleton-guided generators.The method estimates meshes from unlabeled images and uses rotated mesh projections to simulate new viewpoints.
- Framework: A view-invariant loss reduces intra-class variation between original and generated images, helping learn view-invariant representations in fully unsupervised ReID.The contrastive module maximizes similarity for views of the same person and minimizes similarity for other persons.
- Outcome: The method does not require a labeled source dataset and reportedly surpasses state-of-the-art methods in both fully unsupervised and UDA settings.This removes the source-dataset dependence of earlier GAN-based approaches while retaining compatibility with UDA.
2. Related Work
Prior work develops unsupervised representations through contrastive instance discrimination and varied augmentation, while unsupervised ReID has largely relied on UDA or pseudo-label strategies. This paper differs by generating novel views through unsupervised 3D mesh rotation.
- Unsupervised representation learning: Contrastive instance discrimination treats each image as a class and learns representations by matching augmented views against unrelated negatives.Memory banks or large mini-batches support retrieval-style contrastive learning.
- Data augmentation: Traditional augmentation methods include random cropping, color jittering, horizontal flipping, grayscale conversion, Gaussian blur, and random erasing.Color jittering and grayscale can be unsuitable for fine-grained ReID because they change original color distributions.
- Data augmentation: Earlier GAN-based ReID methods disentangled or transferred appearance features, whereas this method generates novel views by rotating 3D meshes without supervision.The proposed strategy is distinguished from supervised GAN-based approaches that manipulate identity-related features or colors.
- Unsupervised person ReID: Unsupervised ReID research has predominantly used UDA, semantic attributes, pseudo labels, or source-to-target style transfer.These approaches generally adapt from labeled source data to an unlabeled target domain.
- Unsupervised person ReID: Fully unsupervised methods avoid identity labels through class merging, tracklet-based classes, soft labels, multilabel classification, or temporal similarity refinement.These methods address unlabeled ReID through alternative clustering, labeling, or temporal strategies.
3. Proposed Method
GCL jointly trains a mesh-based view generator and a contrastive module through a shared identity encoder. The generator creates novel viewpoints, while contrastive learning uses original and synthesized views to improve view-invariant representations and generation.
- Framework: GCL couples a View Generator and View Contrast Module through a shared identity encoder in joint training.The modules mutually promote representation quality and generation quality.
- View Generator: The View Generator uses identity and structure encoders, a decoder, and a discriminator to synthesize person images from novel structures.It estimates 3D meshes, rotates them to simulate viewpoints, and uses mesh projections as structure references.
- View Generator: Cycle consistency reconstructs the original image after structure swapping, while image, feature, and adversarial losses train the generative module.The GAN objective combines image and feature reconstruction losses with adversarial loss using weighting coefficients.
- View Contrast: The view-invariant loss treats original and generated views as positives and pushes synthesized views away from K negative identity instances.The contrastive module uses pseudo labels, a memory bank, and cosine similarity to construct positive and negative pairs.
- View Contrast: Pushing synthesized views away from negative instances improves generation of views that differ from those identities.The authors report this alternative performs better than pushing the original view away from negatives.
- Joint Training: Joint training applies the generative module before the contrastive module and uses an overall loss combining the GAN and contrastive objectives.The identity encoder is warmed up first, then the other generator components are warmed up with GAN loss before contrastive training begins.
4. Experiments
Experiments evaluate GCL on three fully unsupervised ReID datasets and four UDA protocols, examining recognition performance, ablations, and generated-view quality. GCL improves baseline performance, achieves state-of-the-art UDA performance, and produces diverse views while preserving identity-related structure.
- Datasets and protocols: Experiments cover Market-1501, DukeMTMC-reID, and MSMT17, plus four UDA protocols: Market→Duke, Duke→Market, Market→MSMT, and Duke→MSMT.The study reports quantitative and qualitative results for both unsupervised ReID and view generation.
- Unsupervised ReID evaluation: GCL improves MMCL, JVTC, and JVTC+ baselines by large margins under fully unsupervised ReID.The results indicate that GANs can support unsupervised ReID beyond cross-domain style transfer.
- Unsupervised ReID evaluation: Under UDA, GCL achieves state-of-the-art performance when added to the strong JVTC+ baseline.The labeled source dataset warms up Eid but is not used during joint generative and contrastive training.
- Ablation study: Ablations show that joint generation and contrastive learning use view-invariant losses to align original, synthesized, and memory-stored positive views while separating negatives.Without inter-view constraints, Eid tends to extract view-specific identity features; the qualitative study contrasts black-bag and red-shirt examples.
- Generation quality evaluation: GCL preserves better body-shape information and avoids incoherent objects than compared GAN-based generators, while outperforming DG-Net on SSIM but not FID.FID measures similarity between generated and real image distributions, whereas SSIM measures intra-class structural diversity.
- Generation quality evaluation: Changing structure features alters structures without affecting appearances, supporting separate roles for identity and structure features in generated views.The experiments also interpolate identity features while fixing structure features.
5. Conclusions
The paper presents a joint generative and contrastive framework for unsupervised person ReID that uses mesh-guided GAN views and memory-based contrastive learning without a source dataset.
- The framework generates diversified views with a 3D mesh-guided GAN and combines them with original images for memory-based contrastive learning.
- The generative and contrastive modules mutually promote each other by improving view-invariant representations and generation quality.
- The method does not rely on a source dataset, unlike style-transfer-based methods.
- Experiments on three datasets validate the framework in unsupervised person ReID and multi-view person image generation.
Appendix A. Cycle consistency
GCL uses two forms of cycle consistency to improve pseudo-label reliability and supervise generation when paired data are unavailable.
- K-reciprocal Jaccard distance supports DBSCAN clustering by making pseudo labels more reliable for contrastive learning.
- A CycleGAN structure supervises the generative module because paired data are unavailable.
Appendix B. View-invariant losses
View-invariant losses improve generation by encouraging the identity encoder to extract identity features shared across views and robust to image noise.
- Without view-invariant losses, GCL degrades to a traditional CycleGAN that is prone to noise in the original image.
- View-invariant losses help the identity encoder extract identity features shared between different views and robust to original-image noise.
Appendix C. Effects on pseudo labels
Contrasting diversified generated images reduces intra-class variance, increases inter-class separation, and improves pseudo-label clustering relative to traditional augmentation.
- Contrasting generated images minimizes intra-class variance and produces larger inter-class distance in latent space.
- View-invariant representations from diversified generated data help clustering algorithms produce more accurate pseudo labels.
- For Market-1501, which has 751 real identities, GCL produces a DBSCAN cluster count closer to the real identity number than traditional augmentation.
Appendix D. Generated views
The appendix evaluates generated novel views across three ReID datasets and examines their quality, view dependence, and usefulness for learning view-invariant representations. It also documents remaining visual artifacts and identity-detail losses.
- Qualitative examples: Generated views are shown for Market-1501, DukeMTMC-reID, and MSMT17 training and test sets.These examples use a JVTC fully unsupervised baseline and are presented to assess the GCL generation process.
- Qualitative examples: The generated views are generally good enough to help GCL learn view-invariant representations.
- Limitations: Some generated images still contain visual blurs and lose detailed identity information, such as a red logo disappearing from shorts.The authors suggest newer GAN architectures and stronger unsupervised baselines as future directions for addressing these issues.
- View dependence: Novel views at 45°, 180°, and 315° are easier to generate because they preserve more identity information or approximate horizontal flipping.The appendix states that 45° and 315° are small rotations, while 180° can be roughly regarded as horizontal flipping.
- View dependence: The appendix reports FID scores for different generated views in Table 5.