Source-linked AI summary
Long-Term Cloth-Changing Person Re-identification
Xuelin Qian, Wenxuan Wang, Li Zhang, Fangrui Zhu, Yanwei Fu, Tao Xiang, Yu-Gang Jiang, Xiangyang Xue
TL;DR
The paper addresses person Re-ID across long periods, where clothing changes make existing clothing-dominated representations unreliable and large-scale datasets are lacking. It introduces the LTCC dataset and a method combining shape embedding with cloth-elimination shape distillation. Experiments report superior performance on LTCC and BIWI, while direct use of shape embeddings performs worse because estimated keypoints can be unreliable under occlusion.
Problem
Long-term Re-ID across days or months is understudied because existing datasets and models assume short-term clothing consistency, despite frequent outfit changes.
Method
The method extracts body-shape information from human keypoints and uses cloth-elimination shape distillation to learn identity-sensitive, cloth-insensitive representations.
Results
The method outperforms competing methods on LTCC under both evaluation settings and achieves the highest Rank-1 on BIWI’s Still and Walking settings.
Takeaways & Limitations
The dataset and method support evaluation and modeling of Re-ID when clothing changes accompany long-term surveillance appearance variation.
Takeaways & Limitations
Directly using shape embeddings as Re-ID features performs worse, likely because 2D keypoints may be unreliable under occlusion; the embeddings are therefore used as ancillary features.
Abstract
from arXiv · showhide
Person re-identification (Re-ID) aims to match a target person across camera views at different locations and times. Existing Re-ID studies focus on the short-term cloth-consistent setting, under which a person re-appears in different camera views with the same outfit. A discriminative feature representation learned by existing deep Re-ID models is thus dominated by the visual appearance of clothing. In this work, we focus on a much more difficult yet practical setting where person matching is conducted over long-duration, e.g., over days and months and therefore inevitably under the new challenge of changing clothes. This problem, termed Long-Term Cloth-Changing (LTCC) Re-ID is much understudied due to the lack of large scale datasets. The first contribution of this work is a new LTCC dataset containing people captured over a long period of time with frequent clothing changes. As a second contribution, we propose a novel Re-ID method specifically designed to address the cloth-changing challenge. Specifically, we consider that under cloth-changes, soft-biometrics such as body shape would be more reliable. We, therefore, introduce a shape embedding module as well as a cloth-elimination shape-distillation module aiming to eliminate the now unreliable clothing appearance features and focus on the body shape information. Extensive experiments show that superior performance is achieved by the proposed model on the new LTCC dataset. The code and dataset will be available at https://naiq.github.io/LTCC_Perosn_ReID.html.
1 Introduction
The paper identifies a neglected long-term Re-ID setting in which people change clothes across days or months, undermining clothing-dominated representations. It introduces the LTCC dataset and a shape-focused method that removes cloth-related information.
- Motivation: Existing Re-ID datasets capture people across cameras within one day, so models mainly learn clothing-dominated representations.Pose, illumination, occlusion, and viewpoint changes are represented, but outfits remain consistent.
- Problem Setting: LTCC Re-ID matches people across days or months, when clothing changes are commonplace and the setting is more realistic but underrepresented.The lack of large-scale clothing-change datasets is identified as a primary reason for limited study.
- Contributions: The LTCC dataset contains 17,138 images of 152 identities wearing 478 outfits across 12 camera views over two months.It includes clothing, carrying, hairstyle, illumination, viewing-angle, and pose changes, as well as visually similar clothing.
- Method: The proposed method targets identity-sensitive, cloth-insensitive representations by extracting body shape from keypoints and eliminating cloth-related information.It introduces Shape Embedding and Cloth-Elimination Shape-Distillation modules.
2 Related Work
Prior Re-ID work mainly addresses short-term cloth-consistent data, where models handle pose, illumination, and viewpoint changes but remain vulnerable to clothing changes. Existing cloth-changing datasets are limited or unrepresentative for large-scale real-world Re-ID.
- Short-Term Cloth-Consistent Re-ID: Most established Re-ID datasets were collected over short periods, leaving each person’s clothing appearance largely consistent.This short-term setting supports learning robustness to pose, illumination, and viewpoint changes.
- Short-Term Cloth-Consistent Re-ID: Deep Re-ID models are vulnerable to clothing changes because they rely heavily on clothing appearance consistency.Their robustness to other appearance changes does not address outfit variation.
- Long-Term Cloth-Changing Re-ID Datasets: Existing cloth-changing datasets are constrained by small scale, non-purposeful video extraction, missing illumination and occlusion challenges, or professionally captured celebrity imagery.These properties limit their suitability for deep learning or real-world surveillance scenarios.
3 Long-Term Cloth-Changing (LTCC) Dataset
LTCC is a long-term surveillance dataset designed to add clothing changes to person Re-ID. It combines multi-camera collection over two months with diverse appearance and environmental variations.
- Dataset Characteristics: The dataset includes same-person examples with identical and different clothes alongside illumination, occlusion, viewpoint, carrying, and pose changes.These variations are illustrated in the LTCC examples.
- Data Collection: LTCC uses twelve cameras on three office-building floors and 24-hour videos collected over two months.Person images are obtained with Mask-RCNN detection.
- Statistics: The released LTCC version contains 17,138 person images from 152 identities, with each identity captured by at least two cameras.Images also receive clothing labels, while hairstyle and carrying-item changes do not alter cloth labels.
- Framework: The framework uses Shape Embedding to extract structural keypoint features and CESD modules to learn identity-sensitive, cloth-insensitive representations.Two CESD modules share the same Shape Embedding module.
4 Methodology
The method extracts body-shape information from human keypoints and uses cloth-elimination shape distillation to emphasize identity-relevant, cloth-insensitive features. The framework combines shape embedding, residual disentanglement, attention, and supervised identity/clothing objectives.
- Shape Embedding (SE): Shape Embedding encodes each human keypoint’s position and semantic identity, then refines these representations before modeling relationships between keypoints.The module uses learnable embeddings for the two keypoint attributes and a refinement network.
- Shape Embedding (SE): A relation network concatenates features for every pair of keypoints, applies convolutional relation reasoning, and uses global max pooling to produce the shape feature.Max pooling is selected because it is more robust to dramatic pose changes than average pooling.
- Cloth-Elimination Shape-Distillation (CESD): The CESD module instance-normalizes image features and re-scales them using shape-derived parameters to transfer shape information into the image representation.The image and shape features are processed jointly, with learned scale and bias parameters derived from the shape embedding.
- Cloth-Elimination Shape-Distillation (CESD): Self-attention disentangles the residual image feature into cloth-irrelevant and cloth-relevant components, supporting identity-sensitive representation learning while separating clothing information.The residual contains both discriminative features such as contour and features sensitive to clothing changes.
- Cloth-Elimination Shape-Distillation (CESD): Directly using shape embeddings as Re-ID features performs worse empirically, so the model treats them as intermediate ancillary features, likely because detected 2D keypoints can be unreliable under occlusion.The shape embedding therefore guides image-feature distillation rather than serving alone as the final Re-ID representation.
- Cloth-Elimination Shape-Distillation (CESD): The framework combines the cloth-irrelevant residual with the transferred shape feature for identity representation and uses separate identity and clothing losses for the resulting feature branches.CESD modules are inserted after ResNet-50 res3 and res4 blocks, with classification losses applied to identity-relevant and cloth-relevant features.
5 Experiment
Experiments evaluate the proposed method on LTCC and BIWI, including comparative, ablation, and feature-visualization studies. The method outperforms competitors and benefits from shape embedding and cloth-elimination design.
- Experimental Setup: The evaluation uses ResNet-50 initialization, detected human joints, 384×192 inputs, and concatenated CESD features for testing.Competitors include handcrafted, deep-learning, parsing-based, and advanced Re-ID methods.
- Comparative Results: The model beats all competitors under both standard and cloth-changing LTCC evaluation settings.Under cloth-changing, it surpasses ResNet-50 (Image) by 4.47%/3.29% in Rank-1/mAP and PCB (Image) by 3.22%/2.86%.
- Comparative Results: 49.51% and 39.80% Rank-1 are achieved on BIWI under Still and Walking settings, respectively.The method also exceeds SPT+ASE by over 28% and 21% Rank-1 in those settings.
- Ablation Study: The shape-embedding variants without SE or the relation network are clearly inferior to the full model.Body shape improves Rank-1 by about 3%, while its contribution is about 1% under standard evaluation and 3 points under cloth-changing evaluation.
- Ablation Study: CESD visualization clusters same-identity samples in identity-relevant feature space despite differing outfits.Cloth-relevant features instead organize samples by appearance, including warm versus cold color distributions.
6 Conclusion
The paper addresses realistic long-term person re-identification with clothing changes by introducing the LTCC dataset and a method for identity-sensitive, cloth-insensitive representations.
- Conclusion: LTCC contains 152 identities, 478 clothes, 17,128 images, and 12 cameras, with 91 people showing clothing changes.The dataset has no cloth-consistency constraint.
- Conclusion: The method extracts biological structural features from human-keypoint relations and disentangles identity-relevant information from clothing-related information.Its effectiveness is validated through extensive experiments.
A More details on BIWI dataset
BIWI is a smaller cloth-changing Re-ID dataset collected from two cameras, with participants performing prescribed motions.
- Dataset: BIWI contains 50 identities, including 28 that appear in two outfits.The dataset was collected from two cameras.
- Dataset: Participants perform a routine of motions in front of the cameras, creating a controlled cloth-changing setting.The supplied passage identifies BIWI as an existing smaller dataset for cloth-changing Re-ID.
B Results on Celeb-reID dataset
Celeb-reID provides a clothes-variation Re-ID setting based on Internet street snapshots of celebrities, where the same person usually wears different clothing across images.
- Dataset: Celeb-reID contains 1,052 identities and 34,186 images collected from celebrity street snapshots on the Internet.Its image source supports studying clothes-variation person re-identification.
- Results: The paper compares its method with MGN, ReIDCaps, and HACNN on Celeb-reID.The supplied table caption identifies the results as Celeb-reID evaluations.
C Evaluations on model generalization
The LTCC-trained model generalizes across datasets without fine-tuning, outperforming competing approaches on Market-1501 and DukeMTMC-reID.
- Body shape information is expected to be more robust to domain gaps than appearance information such as clothing.The model trained on LTCC is applied directly to Market-1501 and DukeMTMC-reID without fine-tuning.
- More than 6% Rank-1 accuracy improvement is achieved over ResNet-50 and PCB in both cross-domain settings.Table 5 evaluates models trained on LTCC and directly tested on Market-1501 and DukeMTMC-reID.
- The proposed model significantly outperforms all other methods, including MuDeep and OSNet, in the cross-domain task.The comparison includes strong methods that already perform well under cross-domain evaluation.
D More visualizations and discussions
Visual analyses indicate that CESD progressively organizes features by identity while separating clothing-related information, improving cloth-changing retrieval through shape-focused representations.
- CESD transforms chaotic input features into increasingly ordered representations, with final features clearly aggregated by identity.The visualization compares features associated with two CESD modules.
- CESD disentangles identity-relevant and cloth-relevant features: same-identity samples group in f+ space, while similarly dressed samples group in f− space.Colors denote identities and symbols denote different clothes.
- Using two CESD modules better removes cloth-sensitive features and distills discriminative features for person re-identification.The paper attributes this to both deeper task-relevant features and CESD’s cloth-elimination and shape-distillation roles.
- The proposed model correctly retrieves differently dressed matches, whereas ResNet-50 attends more to similar clothing such as yellow shorts.In one example, the proposed model’s top-two correct results wear different clothes.
E More details on LTCC dataset
LTCC is a long-term, multi-camera dataset designed to capture clothing changes alongside substantial environmental and viewpoint variation, unlike several existing benchmarks.
- LTCC contains 17,138 person images from 152 identities wearing 478 outfits, collected through a 12-camera CCTV system over two months.The system recorded continuously across three floors of an office building, followed by careful annotation.
- LTCC samples pair query images with same-clothing gallery images and five different outfits under other variations.The samples visualize the dataset’s cloth-changing retrieval structure.
- Compared with Market-1501 and DukeMTMC-reID, LTCC includes greater clothing, carrying, illumination, and other per-person variation.The two general STCC datasets are described as limited in these variations.
- BIWI contains only 28 people with two clothes under strict constraints, while LTCC uses more cameras and longer collection to support broader variation.The comparison notes BIWI’s limited view-angle, occlusion, carrying, and illumination variation.
- PRCC samples in the comparison were taken directly from a cited reference because the dataset was unavailable at submission time.This is an availability boundary affecting the visual comparison in Fig. 10.