Source-linked AI summary
Camera-aware Proxies for Unsupervised Person Re-Identification
Menglin Wang, Baisheng Lai, Jianqiang Huang, Xiaojin Gong, Xian-Sheng Hua
TL;DR
Purely unsupervised person Re-ID avoids annotations but suffers from camera-driven intra-ID variance that camera-agnostic clustering overlooks. The paper splits clusters into camera-aware proxies and trains with intra- and inter-camera contrastive learning. Across three large-scale datasets, it outperforms purely unsupervised and UDA-based methods, including substantial gains on MSMT17.
Problem
Purely unsupervised Re-ID requires no labeled data, yet clustering-based methods neglect large intra-ID variance caused mainly by camera changes.
Method
The method splits each cluster into camera-aware proxies and uses intra- and inter-camera contrastive learning with proxy-balanced sampling.
Results
14.3% Rank-1 and 10.2% mAP improvements over the second place are reported on MSMT17, with gains across three large-scale datasets.
Takeaways & Limitations
Camera-aware proxies improve identity discrimination within and across cameras for purely unsupervised Re-ID.
Abstract
from arXiv · showhide
This paper tackles the purely unsupervised person re-identification (Re-ID) problem that requires no annotations. Some previous methods adopt clustering techniques to generate pseudo labels and use the produced labels to train Re-ID models progressively. These methods are relatively simple but effective. However, most clustering-based methods take each cluster as a pseudo identity class, neglecting the large intra-ID variance caused mainly by the change of camera views. To address this issue, we propose to split each single cluster into multiple proxies and each proxy represents the instances coming from the same camera. These camera-aware proxies enable us to deal with large intra-ID variance and generate more reliable pseudo labels for learning. Based on the camera-aware proxies, we design both intra- and inter-camera contrastive learning components for our Re-ID model to effectively learn the ID discrimination ability within and across cameras. Meanwhile, a proxy-balanced sampling strategy is also designed, which facilitates our learning further. Extensive experiments on three large-scale Re-ID datasets show that our proposed approach outperforms most unsupervised methods by a significant margin. Especially, on the challenging MSMT17 dataset, we gain $14.3\%$ Rank-1 and $10.2\%$ mAP improvements when compared to the second place. Code is available at: \texttt{https://github.com/Terminator8758/CAP-master}.
1 Introduction
Purely unsupervised person Re-ID avoids costly annotations but remains challenging because clustering-based methods neglect camera-driven intra-ID variance. The paper introduces camera-aware proxies with intra- and inter-camera learning and reports strong benchmark improvements.
- Motivation: Purely unsupervised Re-ID requires no labeled data, but its performance still trails supervised methods.Existing unsupervised approaches generate pseudo labels through clustering, k-nearest neighbors, or graph-based association.
- Motivation: Previous clustering methods treat each cluster as one pseudo identity, overlooking intra-ID variation from pose, illumination, and camera changes.Images of the same identity within one camera tend to cluster more closely than images across cameras.
- Proposed Approach: Camera-aware proxies split each cluster by camera, capturing local identity structure while reducing within-class variance and noise in pseudo labels.Each proxy represents instances from the same camera.
- Proposed Approach: Intra- and inter-camera contrastive components learn identity discrimination within cameras and across cameras using proxy-based labels.The inter-camera component uses positive and hard negative proxies across cameras.
- Results: 14.3% Rank-1 and 10.2% mAP improvements over the second place are reported on MSMT17.Experiments cover Market-1501, DukeMTMC-reID, and MSMT17 and compare against purely unsupervised and UDA-based methods.
2 Related Work
Related work spans purely unsupervised Re-ID, UDA-based Re-ID, intra-camera supervision, and proxy-based metric learning. The paper differs by generating camera-aware proxies according to camera views and emphasizing reliable proxy generation and learning.
- Purely Unsupervised Re-ID: Purely unsupervised Re-ID uses no annotations and commonly generates pseudo labels through clustering, k-nearest neighbors, or graph-based association.Camera-agnostic clustering can preserve identity similarity while neglecting camera-induced intra-ID variance.
- Unsupervised Domain Adaptation: UDA-based Re-ID uses fully annotated source datasets while leaving target data unlabeled, focusing on style transfer or cross-domain distribution alignment.These methods can leave the unlabeled target data underexploited.
- Intra-Camera Supervision: Intra-camera supervision labels identities independently within each camera but leaves inter-camera identity association unannotated.Its key problems are supervised intra-camera learning and unsupervised inter-camera learning.
- Proposed Framework: Figure 2 depicts alternating global clustering, camera-aware proxy generation, and proxy-level contrastive model updating.The framework includes intra- and inter-camera losses based on a proxy-level memory bank.
- Metric Learning with Proxies: Proxy-based metric-learning studies use multiple proxies to represent one cluster and capture local structures, while this work determines proxy counts from camera views.The paper avoids fixed proxy counts and complex adaptive strategies.
3 A Clustering-based Re-ID Baseline
The baseline iteratively clusters unlabeled image features and updates a Re-ID model using pseudo labels. A memory-bank non-parametric classifier performs contrastive-style classification against class centroids.
- Iterative Training: At each iteration, the baseline alternates between clustering and model updating.This follows the common procedure in clustering-based unsupervised Re-ID methods.
- Clustering Step: DBSCAN clusters features from all images and discards isolated points before assigning one pseudo identity label to each selected cluster.The Re-ID network is initialized from an ImageNet-pretrained model and extracts the features being clustered.
- Clustering Step: The selected clusters form a labeled dataset with generated pseudo labels, whose count corresponds to the number of clusters.N′ denotes images retained in selected clusters, while Y denotes the cluster number.
- Model Updating: Model updating uses an external memory bank and a non-parametric Softmax loss rather than a parametric classifier.The memory stores feature centroids for the pseudo classes and is updated during back-propagation.
- Model Updating: The non-parametric loss pulls each image toward its class centroid and pushes it away from all other class centroids.A temperature factor appears in the loss formulation.
4 The Camera-aware Proxy Assisted Method
The method alternates camera-aware proxy clustering with model updating, using per-camera pseudo labels and contrastive learning to address intra-ID camera variance. Its learning objective combines intra- and inter-camera losses with proxy-balanced sampling.
- Camera-aware proxy clustering: Camera-agnostic clusters are split into camera-aware proxies, each representing instances from one camera and receiving per-camera pseudo labels.Proxies split from the same global cluster may receive different labels in different camera views.
- Camera-aware proxy clustering: A proxy-level memory bank stores one entry for each proxy across all cameras and updates entries using features from that proxy.The total number of memory entries equals the number of proxies across cameras.
- Intra-camera contrastive learning: Intra-camera contrastive learning pulls each image toward its assigned proxy and pushes it away from other proxies in the same camera.Per-camera non-parametric classifiers jointly learn discrimination within cameras, while the shared feature extractor is optimized across cameras.
- Inter-camera contrastive learning: Inter-camera contrastive learning pulls images toward positive proxies sharing their global pseudo label and pushes them away from K-nearest hard negative proxies across cameras.The hard negatives address similarity across identity classes.
- Algorithm summary: The algorithm iteratively alternates proxy clustering and learning, using proxy-balanced batches and the combined loss L = LIntra + λLInter.The coefficient λ balances the intra- and inter-camera terms.
5 Experiments
Experiments evaluate CAP on three large-scale Re-ID datasets through ablations, feature visualization, and comparisons with unsupervised, UDA-based, and supervised methods. The results show consistent gains from camera-aware proxy learning, its components, and proxy-balanced sampling.
- Ablation studies: CAP6 significantly improves over the baseline on all three datasets, including 33.4% Rank-1 and 23.2% mAP gains on MSMT17.Market-1501 gains are 11.7% Rank-1 and 16.3% mAP, while DukeMTMC-reID gains are 6.8% Rank-1 and 9.8% mAP.
- Ablation studies: Intra-camera learning benefits from smaller identity sets, reduced intra-ID variance, and more reliable per-camera pseudo labels.CAP2 outperforms the baseline on all datasets when proxy-balanced sampling is used, while removing intra-camera loss reduces performance.
- Ablation studies: Inter-camera learning exploits cross-camera correlations by pulling positive proxies together and pushing hard negative proxies away.CAP5 and CAP6 outperform CAP1 and CAP2 respectively, while adding intra-camera learning or sampling further improves CAP3.
- Ablation studies: Proxy-balanced sampling outperforms class-balanced sampling in CAP2, CAP4, and CAP6.The strategy balances the different numbers of images contained in camera-aware proxies.
- Ablation studies: T-SNE visualizations show CAP2 separates some confusing identities better than the baseline, while CAP6 improves intra-ID compactness and inter-ID separability but retains difficult cases.The visualization compares Baseline, CAP2, and CAP6 features, with examples of IDs #4-7 shown below.
- Comparison with State-of-the-Arts: Compared with the second-place method, CAP gains 3.3% Rank-1 and 6.1% mAP on Market, 3.2% Rank-1 and 7.2% mAP on Duke, and 17.3% Rank-1 and 10.2% mAP on MSMT17.CAP outperforms the reported state-of-the-art purely unsupervised and UDA-based counterparts by substantial margins.
6 Conclusion
The paper presents camera-aware proxy assisted learning for purely unsupervised person Re-ID. Its intra- and inter-camera learning components improve ID discrimination within and across cameras, with experiments on three large-scale datasets demonstrating superiority over purely unsupervised and UDA-based methods.
- Camera-aware proxies address large intra-ID variance caused by changes in camera views in purely unsupervised person Re-ID.
- Intra- and inter-camera learning components improve ID discrimination within and across cameras using camera-aware proxies.
- Experiments on three large-scale datasets demonstrate superiority over both purely unsupervised and UDA-based methods.