Source-linked AI summary
Clothes-Changing Person Re-identification with RGB Modality Only
Xinqian Gu, Hong Chang, Bingpeng Ma, Shutao Bai, Shiguang Shan, Xilin Chen
TL;DR
Clothes-changing person re-identification needs clothes-irrelevant features, yet existing methods underuse such information in RGB images and lack suitable video benchmarks. The paper proposes CAL to penalize clothes predictability, demonstrates strong benchmark performance using RGB images only, and introduces CCVID to study video re-identification and spatiotemporal modeling.
Problem
Clothes-changing re-identification must identify people across clothing changes, but existing methods underuse clothes-irrelevant RGB information and publicly available datasets lack clothes-changing video sequences.
Method
CAL adds a clothes classifier using fine-grained identity-based clothes classes and a multi-positive-class adversarial loss that penalizes clothes predictability while preserving identity discrimination.
Results
CAL consistently improves over the baseline by a large margin on CCVID and related datasets and, using RGB images only, outperforms all state-of-the-art methods.
Takeaways & Limitations
CCVID enables clothes-changing video re-identification evaluation, while its results indicate substantial room for improving spatiotemporal modeling.
Takeaways & Limitations
CAL requires clothes labels for adversarial learning, although collection dates can serve as pseudo clothes labels when such annotations are unavailable.
Abstract
from arXiv · showhide
The key to address clothes-changing person re-identification (re-id) is to extract clothes-irrelevant features, e.g., face, hairstyle, body shape, and gait. Most current works mainly focus on modeling body shape from multi-modality information (e.g., silhouettes and sketches), but do not make full use of the clothes-irrelevant information in the original RGB images. In this paper, we propose a Clothes-based Adversarial Loss (CAL) to mine clothes-irrelevant features from the original RGB images by penalizing the predictive power of re-id model w.r.t. clothes. Extensive experiments demonstrate that using RGB images only, CAL outperforms all state-of-the-art methods on widely-used clothes-changing person re-id benchmarks. Besides, compared with images, videos contain richer appearance and additional temporal information, which can be used to model proper spatiotemporal patterns to assist clothes-changing re-id. Since there is no publicly available clothes-changing video re-id dataset, we contribute a new dataset named CCVID and show that there exists much room for improvement in modeling spatiotemporal information. The code and new dataset are available at: https://github.com/guxinqian/Simple-CCReID.
1. Introduction
Clothes-changing person re-identification requires clothes-irrelevant features, but existing approaches underuse such information in RGB images. The paper proposes CAL to mine these features and introduces CCVID for video-based evaluation.
- Clothes-changing re-identification is necessary for long-term surveillance and scenarios where suspects change clothes to evade identification.
- Human recognition can use clothes-irrelevant cues such as face, hairstyle, body shape, and gait.
- Existing methods often use additional modalities or discard RGB frames, while RGB images contain underused clothes-irrelevant information.
- CAL adds a clothes classifier and multi-positive-class loss, penalizing clothes predictability while preserving identity-related information.
- Using RGB images only, CAL outperforms state-of-the-art methods on widely used clothes-changing re-identification benchmarks.
- The CCVID dataset supports video-based clothes-changing re-identification by providing video data and fine-grained clothes labels.
2. Related Work
Prior clothes-changing re-identification research largely relies on image settings or auxiliary modalities, while video-based methods commonly assume consistent clothing. This paper instead mines clothes-irrelevant information directly from RGB data using adversarial learning and contributes a clothes-changing video dataset.
- Clothes-changing person re-identification: Clothes-changing re-identification methods extract clothes-irrelevant features through disentangled RGB representations or auxiliary modalities such as silhouettes, skeletons, radio signals, sketches, and 3D shape.
- Clothes-changing person re-identification: The paper proposes a simple adversarial loss to decouple clothes-irrelevant features from the RGB modality.
- Clothes-changing person re-identification: Most clothes-changing re-identification datasets and methods are image-based, with only a few works addressing video-based settings.
- Video person re-identification: Video re-identification methods usually target clothes-consistent settings, where models can overfit clothes-relevant features and have limited application scenarios.
- Gait recognition: Gait recognition commonly discards RGB frames and models gait using skeletons, silhouettes, or disentangled representations, whereas this paper seeks additional clothes-irrelevant cues from RGB.
- Adversarial learning: The paper applies adversarial learning to penalize clothes-related predictive power while extracting clothes-irrelevant information from RGB.
3. Method
The method alternates between training a clothes classifier and optimizing identity and adversarial losses so the backbone learns clothes-irrelevant features while preserving identity information.
- 3.1. Framework and Notation: Each training iteration first minimizes clothes classification loss LC, then fixes the clothes classifier and minimizes LID and LCA.This two-step optimization trains the classifier and backbone in alternating stages.
- 3.1. Framework and Notation: Clothes labels form fine-grained identity classes: samples sharing an identity receive different clothes classes according to their outfits.Different identities never share a clothes label, even when wearing similar clothes.
- 3.2. Clothes-based Adversarial Loss: CAL is a multi-positive-class loss that treats all clothes classes belonging to the same identity as positive classes.This avoids penalizing identity-discriminative information while discouraging distinctions between outfits of the same person.
- 3.2. Clothes-based Adversarial Loss: The same-clothes positive class and different-clothes positive classes can receive different weights through ϵ, with ϵ = 1 recovering the equal-weight formulation.For ϵ < 1, the same-clothes positive class receives greater weight than positive classes representing different clothes.
- 3.2. Clothes-based Adversarial Loss: LID and LCA both pull same-identity features together, while LID additionally follows a curriculum-like progression from easy same-clothes samples to harder clothing-change cases.The authors retain LID alongside LCA because the two objectives have related but complementary roles.
- 3.3. Discussion: CAL differs from label smoothing because it assigns zero weight to negative clothes classes, whereas label smoothing gives them small non-zero weights.The distinction targets a trade-off between clothes-changing and clothes-consistent re-identification accuracy.
4. CCVID Dataset
Existing video re-id datasets lack clothes changes, while clothes-changing re-id datasets contain only still images. CCVID addresses this gap by providing a clothes-changing video benchmark constructed from gait-recognition data.
- Existing video person re-id datasets do not involve clothes changes, whereas existing clothes-changing person re-id datasets contain only still images.
- Video samples provide richer appearance and additional temporal information for clothes-changing re-id.
- CCVID is constructed from the raw data of the FVG gait-recognition dataset as a publicly available clothes-changing video person re-id benchmark.
- FVG contains 2,856 sequences from 226 identities, with each identity wearing 2–5 suits of clothes.
- The paper compares image-based and video-based settings on CCVID and evaluates existing video person re-id and gait-recognition methods.
5. Experiments
Experiments evaluate CAL on image- and video-based clothes-changing re-id benchmarks, compare it with baselines and alternative formulations, and analyze parameter sensitivity and feature behavior. Results support CAL's effectiveness, while video experiments show gains from temporal information and remaining room for spatiotemporal improvements.
- Datasets and Evaluation Protocol: The evaluation covers CCVID, LTCC, and PRCC, using top-1 accuracy and mAP across general, clothes-changing, and same-clothes settings.CCVID and LTCC report general and clothes-changing results; PRCC reports same-clothes and clothes-changing results.
- Comparison with State-of-the-art Methods: Using RGB images only and no additional data, CAL consistently outperforms compared methods on LTCC and PRCC, including methods using multiple modalities.The comparison notes that 3DSL and FSAM use at least three modalities and have at least four times CAL's computational cost.
- Ablation Studies: CAL surpasses the baseline by a large margin in general and clothes-changing settings, whereas directly minimizing clothes classification loss lowers clothes-changing performance.The ablation attributes this pattern to CAL encouraging clothes-irrelevant features while clothes classification loss encourages clothes-relevant features.
- Ablation Studies: CAL outperforms the opposite-loss formulation CAL (−LC) in general, clothes-changing, and same-clothes settings, whose accuracy is lower than the baseline in all three settings.The passage explains that penalizing predictive power for all fine-grained clothes classes can also reduce predictive power for identity.
- Ablation Studies: Triplet loss improves over the baseline, but CAL outperforms it significantly, especially in clothes-changing re-id.The proposed explanation is that CAL mines clothes-irrelevant features using proxies for all clothes classes globally, whereas Triplet loss mines hard cases only within a mini-batch.
6. Conclusion
CAL mines clothes-irrelevant information from RGB inputs and improves clothes-changing person re-identification without additional data or multimodal inputs. The paper also notes privacy risks from surveillance data and re-identification datasets.
- CAL penalizes clothes predictiveness to learn clothes-irrelevant features from original RGB images, improving robustness to clothes changes.The method is reported to consistently improve over the baseline by a large margin on CCVID and related datasets.
- Using RGB images only, CAL outperforms all state-of-the-art methods on the evaluated clothes-changing re-identification datasets.
- The method can be used without additional data or multimodality inputs, supporting more practicable long-term person re-identification in intelligent monitoring systems.
- Surveillance data and person re-identification datasets may create privacy-breach risks, identified as a potential negative impact.
A. Convergence Analysis
The convergence analysis distinguishes true-positive, pseudo-positive, and negative clothes classes and tests whether their learned probabilities follow the expected ordering. The observed ranges match that analysis.
- The analysis defines pP os, pP P os, and pNeg as probabilities for true-positive, same-identity different-clothes, and different-identity clothes classes.
- The expected convergence ordering is pP os ≫ pP P os ≫ pNeg after the two optimization steps.Minimizing LC increases pP os while decreasing pP P os and pNeg; minimizing LCA increases both positive probabilities while decreasing pNeg.
- pP os converges to 0.6∼1 for most LTCC samples at the last training epoch.
- pP P os converges to 1e-4∼0.1 and pNeg to 1e-5∼1e-2, consistent with the convergence analysis.
B. Experiments on VC-Clothes
On VC-Clothes, the paper compares CAL with RGB-only and multimodal re-identification methods across general, same-clothes, and clothes-changing settings. CAL outperforms the baseline and reported state-of-the-art methods, especially in clothes-changing evaluation.
- Dataset and settings: VC-Clothes contains 19,060 images from 512 identities and four cameras, with each identity wearing 1∼3 clothes suits.Camera 2&3 samples support same-clothes evaluation, while camera 3&4 samples support clothes-changing evaluation.
- Comparison: CAL is compared with two RGB-only methods and three multimodal methods on VC-Clothes.
- Results: CAL outperforms the baseline and all compared state-of-the-art methods in general, same-clothes, and clothes-changing settings using RGB images only.
- Results: CAL outperforms the baseline especially in the clothes-changing setting when evaluating data from all cameras.
C. Experiments on LaST and DeepChange
The LaST and DeepChange experiments compare CAL with baseline and state-of-the-art methods under long-term re-identification settings. CAL performs significantly better and remains effective when DeepChange collection dates provide pseudo clothes labels.
- Dataset handling: LaST provides training-set clothes labels, whereas DeepChange lacks clothes labels and uses collection dates as pseudo clothes labels.
- Evaluation setting: The DeepChange evaluation allows true matches from the same camera but different tracklets as queries, following prior work.
- Results: CAL significantly outperforms the baseline and state-of-the-art methods on both LaST and DeepChange.
- Results: CAL still works well on DeepChange with collection dates used as pseudo clothes labels, showing that it does not rely on accurate clothes annotations.