Prototype Matters: Modality-unified Prototype Self-distillation for Unsupervised Visible-infrared Person Re-identification

Motivation

• Unsupervised visible-infrared re-ID estimates associations between daytime visible and nighttime or low-light infrared person images without costly annotation.Source: §1 · p. 1

Challenges

• Noisy intra-modality clusters make hard optimal-transport matching suboptimal, while evolving prototypes can drift from offline associations.The paper motivates soft association based on semantic consistency and online prototype similarity dynamics.Source: §1 · p. 1
• Cross-modality contrast alone ignores the lower cross-modality similarity relative to intra-modality similarity, potentially weakening generalization under severe modality shift.Source: §1 · p. 1

Method

• Modality-unified contrastive optimization jointly compares instances with intra- and cross-modality prototypes to reduce the similarity gap while preserving identity discrimination.This optimization uses intra-modality similarity as a reference for cross-modality semantic relations.Source: §1 · pp. 1–2
• Prototype-guided self-distillation uses each instance’s centroid prototype as a steady teacher, refining online instance-to-prototype similarity and complementing offline association.Centroid prototypes incorporate intra-cluster variation through online updates, providing an instance-adaptive soft distillation target.Source: §1 · p. 2

Contributions

• The unified framework combines offline matching with online distillation to improve cross-modality association estimation and learn modality-invariant, identity-discriminative representations.Source: §1 · p. 2

Unsupervised visible-infrared re-ID

• Unsupervised visible-infrared re-ID methods commonly combine iterative intra-modality clustering with cross-modality association or representation-learning strategies.Source: §2.1 · p. 2

Cross-modality learning

• Cross-modality association methods include optimal transport, label smoothing and propagation, matched-pair ranking, and related cluster-association strategies.Source: §2.1 · p. 2

Self-distillation

• Self-distillation typically uses predictions from one augmented view or network as supervision for another view of the same unlabeled instance.Source: §2.2 · p. 2

Prototype learning

• Prototype-based unsupervised learning uses cluster representations to guide discriminative representation learning through prototypical contrast.Prior designs capture camera-view variation, diversity, or improved prototype updating mechanisms.Source: §2.3 · p. 3

3.2 The UnVI-reID Baseline

• The framework first clusters visible and infrared images separately, initializes dual prototype memories, and trains with intra-modality prototypical contrast.Centroid prototypes capture cluster means, while hard prototypes use the farthest positive instance and are updated online.Source: §3.1 · p. 3, §3.2.1 · p. 3

3.2.2 Cross-modality learning

• Optimal transport estimates visible–infrared cluster correspondence from cosine similarities between modality-specific cluster prototypes.The resulting correspondence supports cross-modality contrastive learning against matched prototypes.Source: §3.2.2 · p. 3

3.3 Modality-unified Prototypical Contrast

• Modality-unified prototypical contrast assigns shared pseudo-labels and jointly optimizes similarity to intra- and cross-modality prototypes.Unlike cross-modality-only contrast, the unified loss explicitly addresses the modality-induced similarity gap.Source: §3.3 · p. 4

3.4 Prototype-guided Self-distillation

• Prototype-guided self-distillation uses online prototype similarity as a soft teacher to refine noisy and drifting instance–prototype relations.The teacher combines instance-to-prototype and prototype-to-prototype similarity, while separate modality normalization avoids intra-modality dominance.Source: §3.4 · pp. 4–5
• The method repurposes online-updated centroid prototypes as steady, informative teachers for distilling instance-to-prototype similarity distributions.This prototype-based distillation perspective extends the use of prototypes beyond ordinary assistance in unsupervised learning.Source: §3.4 · p. 5

Comparison with State-of-the-Art Methods

• 0.9% Rank-1 improvement on SYSU-MM01 and 4.4% Rank-1 plus 4.9% mAP improvements on RegDB Visible-to-Thermal search over prior unsupervised methods.The method consistently outperforms state-of-the-art unsupervised methods on SYSU-MM01 and RegDB.Source: §4.3 · p. 6, Table 1 · p. 6
• 7.3% and 5.9% Rank-1 gains over PGM on LLCM for Visible-to-Infrared and Infrared-to-Visible evaluation, respectively.The method also surpasses other unsupervised VI-ReID methods on LLCM.Source: §4.3 · p. 6, Table 2 · p. 6

Ablation Study

• Prototype-guided online self-distillation improves Rank-1 (All Search) accuracy by 6.4% over M2, while adding offline modality-unified contrast further increases Rank-1 by 0.6% on both SYSU-MM01 and RegDB.The ablation identifies complementary contributions from online distillation and offline association under the same modality-unified framework.Source: §4.4 · p. 7, Table 3 · p. 7

Further Analysis on Prototype-guided Self-distillation

• Teacher temperature achieves its best performance at 𝛾=0.05, while association recall rises during training and union association consistently exceeds offline-association recall.After around 30 epochs, association recall becomes stable; offline and online signals become more consistent as training converges.Source: §4.5 · p. 7, §4.6 · p. 8, Figure 3 · p. 8
• Separate intra- and cross-modality distillation decreases All Search performance, whereas replacing instance-adaptive fusion with prototype-to-prototype similarity causes a noticeable drop.These comparisons support unified optimization and instance-adaptive similarity fusion for handling modality discrepancy and intra-cluster noise.Source: §4.6 · pp. 7–8, Table 4 · p. 7

Visualization

• T-SNE visualizations show tighter intra-class feature distributions for the proposed method than for the baseline across visible and infrared images.The comparison describes baseline identity features as dispersed into sub-clusters, while the proposed method associates intra-class features more compactly.Source: §4.7 · p. 8, Figure 4 · p. 8

• The unified framework jointly optimizes contrastive similarity across modalities and uses prototype-guided self-distillation to address modality discrepancy.Centroid prototypes serve as steady online teachers, complementing offline association with stronger supervision for modality-unified contrastive learning.Source: §5 · p. 8