Source-linked AI summary
Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
Xiao Zhang, Yixiao Ge, Yu Qiao, Hongsheng Li
TL;DR
Clustering-based unsupervised re-ID depends on pseudo labels, but noise and changing class assignments complicate refinement across training generations. RLCC estimates consensus between consecutive generations to propagate and refine labels, achieving an 8.8% mAP improvement over SpCL on MSMT17.
Problem
Clustering-based unsupervised re-ID suffers from noisy pseudo labels, while changing class assignments across generations make fixed-class temporal ensembling methods unsuitable for direct use.
Source: §1 · pp. 1–2Method
RLCC estimates cross-generation pseudo-label similarity and uses it to propagate prior hard or soft labels to refine current-generation labels.
Source: §3 · p. 3, §3.2 · p. 4Results
RLCC achieves an 8.8% mAP improvement over SpCL [10] on MSMT17.
Source: §4.4 · p. 8Takeaways & Limitations
The authors report that temporal ensembling with pseudo-label confidence propagation can effectively mitigate pseudo-label noise and achieve higher performance.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations. Clustering-based methods conduct training with the generated pseudo labels and currently dominate this research direction. However, they still suffer from the issue of pseudo label noise. To tackle the challenge, we propose to properly estimate pseudo label similarities between consecutive training generations with clustering consensus and refine pseudo labels with temporally propagated and ensembled pseudo labels. To the best of our knowledge, this is the first attempt to leverage the spirit of temporal ensembling to improve classification with dynamically changing classes over generations. The proposed pseudo label refinery strategy is simple yet effective and can be seamlessly integrated into existing clustering-based unsupervised re-identification methods. With our proposed approach, state-of-the-art method can be further boosted with up to 8.8% mAP improvements on the challenging MSMT17 dataset.
1. Introduction
Clustering-based unsupervised re-ID relies on pseudo labels, but noise remains a barrier to further improvement. RLCC uses cross-generation clustering consensus to propagate and refine labels, and reports gains over state-of-the-art methods.
1. Introduction
- Inevitable pseudo-label noise hinders training and leaves noticeable performance gaps from oracle experiments using ground-truth identities [10].The introduction identifies pseudo-label refinement as central to further improving unsupervised re-ID algorithms.Source: §1 · p. 1
- Off-the-shelf temporal ensembling assumes fixed class definitions, whereas unsupervised re-ID pseudo labels change between generations.This changing label space makes direct application of those methods non-trivial.Source: §1 · p. 2
- RLCC estimates cross-generation pseudo-class similarity with IoU, then propagates hard or soft labels by random walk and refines them with momentum averaging.Past-generation labels provide supervision that can smooth pseudo-label variation.Source: §1 · p. 2
- RLCC improves state-of-the-art unsupervised re-ID results across multiple benchmarks, with up to 8.8% mAP improvement over SpCL [10].The strategy is compatible with existing pseudo-label-based methods [46] [10].Source: §1 · p. 2
2. Related Works
Related work covers clustering-based pseudo-label methods, unsupervised domain adaptation, unsupervised representation learning, and temporal ensembling. These approaches differ in their label-generation strategies and assumptions about whether classes remain fixed.
Unsupervised person re-identification
Unsupervised domain adaptation
- For object re-ID domain adaptation, pseudo-label-based methods [8] [9] [10] [37] [45] [47] [49] [53] are described as more effective at capturing target-domain distributions than domain translation methods [6] [39] [44] [43].Examples include SSG [8] for multi-scale labels, MMT [9] for mutual-learning refinement, and AD-Cluster [47] for cluster refinement with augmented and generated images.Source: §2 · p. 2
Unsupervised representation learning
- Self-supervised contrastive methods learn instance-discriminative representations for pretraining, but require downstream fine-tuning with ground-truth labels.That fine-tuning requirement makes them inapplicable to the paper's unsupervised re-ID setting.Source: §2 · p. 2
Temporal ensembling
- Temporal ensembling aggregates prior models or predictions with moving averages, but existing label-ensembling methods assume fixed class definitions across generations.Unsupervised re-ID updates pseudo labels each generation, so those methods cannot be directly applied.Source: §2 · pp. 2–3
3. Method
Clustering-based re-identification generates noisy pseudo labels that vary between training generations, while changing cluster definitions make standard temporal ensembling unsuitable. RLCC estimates cross-generation cluster consensus and propagates prior labels or confidences to refine current supervision.
3. Method
- RLCC addresses noisy, temporally varying pseudo labels, which hinder effective optimization when cluster assignments change between generations.Source: §3 · p. 3
3.1. Revisit of Clustering-based Unsupervised Re-identification
- Each training generation alternates between clustering unlabeled features to create pseudo labels and training the network with those labels.Source: §3.1 · p. 3
3.2. Refined Pseudo Labels with Clustering Consensus over Generations
- The method matches clusters across generations using sample-set IoU, forming a row-normalized consensus matrix despite changing label sets.Source: §3.2 · p. 3
- It propagates either hard labels or prior-generation soft confidences through the consensus matrix, then momentum-averages them with current labels to smooth variation.Source: §3.2 · pp. 3–5
3.3. Generalization to State-of-the-art SpCL [10] Framework
- In SpCL, class prototypes estimate prior-generation confidences, which are ensembled into the labels used by its unified contrastive loss.Source: §3.3 · p. 5
4. Experiments
Experiments test RLCC on three person and one vehicle re-identification dataset, then ablate propagation choices and compare the method with unsupervised re-ID baselines. RLCC improves over SpCL across the reported person and vehicle benchmarks.
4.1. Datasets and Evaluation Metrics
- On Market-1501, DukeMTMC-reID, MSMT17, and VeRi-776, evaluation uses mAP and CMC without training identity labels or inference postprocessing.Source: §4.1 · p. 5
4.2. Implementation Details
- RLCC replaces SpCL's noisy hard pseudo labels with refined soft labels in its unified contrastive loss, using DBSCAN with self-paced clustering and α=0.9, τ=30.Source: §4.2 · pp. 5–6
4.3. Ablation Studies
- Ablation experiments vary one setting at a time; unless specified otherwise, they use τ=30 and α=0.9.Source: §4.3 · p. 6
4.3.1 Hard v.s. Soft Pseudo Label Propagation
- Soft pseudo-label confidence propagation outperforms hard-label propagation across tested momentum settings, and β=0 (soft-only propagation) performs best in the hard/soft mixture.Source: §4.3.1 · p. 6, Table 1 · p. 6
4.3.2 Class Prototypes for Confidence Estimation
- Caching class prototypes at the beginning of generation t−1 yields better confidence estimates for generation t than caching them at its end.Source: §4.3.2 · pp. 6–7, Table 2 · p. 7
4.3.3 Hyper-parameter Analysis
- With τ=30, the best mAP is 77.7% at α=0.9; with α=0.9, the best mAP is 77.7% at τ=30, and performance remains robust across the reported ranges.Source: Figure 4 · p. 7, Figure 5 · p. 7, §4.3.3 · pp. 7–8
4.4. Comparison with State-of-the-arts
- RLCC achieves 27.9% mAP on MSMT17, an 8.8% mAP improvement over SpCL; it also records 77.7% mAP on Market-1501 (+4.6% over SpCL) and a 3.9% mAP improvement on DukeMTMC-reID.Source: Table 5 · p. 8, Table 3 · p. 8, Table 4 · p. 8
- RLCC achieves a 2.7% mAP improvement over SpCL on the VeRi-776 vehicle re-ID benchmark.Source: Table 6 · p. 8, §4.4 · p. 8
5. Conclusions
The conclusion identifies pseudo-label noise as a major obstacle for clustering-based unsupervised object re-ID and presents temporal label propagation with clustering consensus as a way to address it.
5. Conclusions
- Pseudo-label noise is a major factor hindering further progress in clustering-based unsupervised object re-ID methods [46] [10].Source: §5 · p. 9
- The proposed strategy refines noisy labels with temporally propagated and aggregated soft labels, estimates clustering consensus, and can be integrated into existing methods with marginal modifications.Source: §5 · p. 9
- The authors conclude that confidence propagation can effectively mitigate pseudo-label noise, while calling for further study of temporal knowledge across more generations.Source: §5 · p. 9