Source-linked AI summary
Learning Disentangled Representation Implicitly via Transformer for Occluded Person Re-Identification
Mengxi Jia, Xinhua Cheng, Shijian Lu, Jian Zhang
TL;DR
Occluded person re-identification is hindered by misalignment and occlusion interference, while existing alignment-based solutions are complicated and noise-sensitive. DRL-Net uses transformer-guided, alignment-free disentanglement with contrast feature learning, and reports superior performance across occluded and holistic benchmarks, including state-of-the-art results under occlusion.
Problem
Occluded re-ID is challenged by incomplete body information, diverse occlusions, and interference from obstacle appearance, while existing alignment approaches are complicated and sensitive to noise.
Method
DRL-Net uses transformer object queries to disentangle ID-relevant and ID-irrelevant semantic components without additional supervision, combined with contrast feature learning and occluded-sample augmentation.
Results
DRL-Net achieves state-of-the-art re-ID performance under various occlusions without sacrificing performance on normal re-ID data with little occlusion.
Takeaways & Limitations
The paper demonstrates that alignment-free transformer reasoning and disentangled feature learning can support consistently superior re-ID performance across occluded and holistic benchmarks.
Abstract
from arXiv · showhide
Person re-identification (re-ID) under various occlusions has been a long-standing challenge as person images with different types of occlusions often suffer from misalignment in image matching and ranking. Most existing methods tackle this challenge by aligning spatial features of body parts according to external semantic cues or feature similarities but this alignment approach is complicated and sensitive to noises. We design DRL-Net, a disentangled representation learning network that handles occluded re-ID without requiring strict person image alignment or any additional supervision. Leveraging transformer architectures, DRL-Net achieves alignment-free re-ID via global reasoning of local features of occluded person images. It measures image similarity by automatically disentangling the representation of undefined semantic components, e.g., human body parts or obstacles, under the guidance of semantic preference object queries in the transformer. In addition, we design a decorrelation constraint in the transformer decoder and impose it over object queries for better focus on different semantic components. To better eliminate interference from occlusions, we design a contrast feature learning technique (CFL) for better separation of occlusion features and discriminative ID features. Extensive experiments over occluded and holistic re-ID benchmarks (Occluded-DukeMTMC, Market1501 and DukeMTMC) show that the DRL-Net achieves superior re-ID performance consistently and outperforms the state-of-the-art by large margins for Occluded-DukeMTMC.
I. INTRODUCTION
Occluded person re-identification is difficult because missing body parts and visually similar occlusions complicate alignment, matching, and representation learning. DRL-Net addresses these issues with alignment-free, unsupervised disentanglement and contrastive feature separation, achieving strong performance under occlusion and on normal re-ID data.
- Motivation: Occluded re-ID suffers from incomplete body information, diverse occlusions, increased intra-class variation, and interference from obstacle appearance.These factors complicate image matching and degrade learned person representations.
- Motivation: Existing methods align visible body parts using external cues or local-feature similarity, but these approaches are noise-sensitive, computationally complicated, and prone to mismatches.External cues can fail under severe occlusions, while local similarity may confuse human bodies with obstacles.
- Method: DRL-Net performs alignment-free occluded re-ID by using a transformer to reason globally over local features and disentangle undefined semantic components without additional supervision.The method separates ID-relevant features for matching from ID-irrelevant features for reducing occlusion interference.
- Method: Contrast feature learning and data augmentation are designed to mitigate occlusion interference and increase the diversity of occluded training samples.The augmentation strategy preserves identities while synthesizing images containing multiple obstacles.
- Results: DRL-Net achieves state-of-the-art performance under various occlusions without sacrificing performance on normal re-ID data with little occlusion.The reported result covers both occluded and relatively holistic re-identification settings.
II. RELATED WORK
Related occluded re-ID methods primarily address incomplete body information and spatial misalignment through external semantic cues or part-to-part matching. These approaches align body parts using parsing, pose, segmentation, local features, or clustering-based relations.
- Overview: Occluded re-ID methods mainly address body-information incompleteness and spatial misalignment through external-cue or part-to-part matching approaches.The two streams differ in whether alignment is guided by additional semantic information or local-feature relations.
- External cues: External-cue methods use human parsing, pose estimation, or foreground segmentation to align visible body parts.Examples include pose-guided feature alignment and pose-guided visible-part matching.
- Part-to-part matching: Part-to-part methods derive alignment relations from local-feature similarity across query and gallery images.Representative strategies divide feature maps into horizontal pieces, find shortest alignment paths, or cluster body parts and belongings at pixel level.
B. Visual Transformer
The visual-transformer component adapts transformer-based global reasoning to extract semantic-component representations from person images. Its CNN features are converted into sequences, while learnable object queries separate ID-relevant and occlusion-related representations for re-ID learning.
- Visual Transformer: Transformers use self-attention and have been extended from language processing to computer-vision tasks including recognition, detection, segmentation, and pose estimation.The section situates the proposed visual-transformer design within this broader development.
- Visual Transformer: DRL-Net extends DETR by using object queries to extract semantic-component features rather than performing extra prediction or relying on predefined components.This supports semantic representation extraction for occluded re-ID.
- Feature Extractor: The feature extractor combines a CNN backbone with transformer encoder-decoder layers to obtain compact representations and semantic-component features.Learnable positional encodings provide spatial information to encoder attention layers.
- Feature Extractor: CNN feature maps are activated, projected from C channels to a smaller d-dimensional space, and flattened across H×W spatial positions into a transformer sequence.The transformation reduces transformer computation while preserving spatially arranged feature tokens.
- Feature Extractor: The decoder uses Nq−1 human semantic queries and one occlusion query to generate ID-relevant and ID-irrelevant features.Human-query features are concatenated for person matching, while the occlusion-query feature is used to reduce occlusion and noise interference.
- Training: Identity learning uses cross-entropy loss with label smoothing to supervise feature-extractor training and reduce overfitting to training identities.The smoothing constant ϵ is set to 0.1.
2) Object Query Decorrelation Constraint:
The object query decorrelation constraint encourages different transformer queries to represent distinct semantic components with minimal overlap. This improves separation and localization without external supervision.
- 2) Object Query Decorrelation Constraint:: The constraint encourages object queries to be orthogonal to each other through a penalty based on their inner products.The loss uses absolute inner products between query vectors, scaled by a penalty factor α.
- 2) Object Query Decorrelation Constraint:: Object query decorrelation makes different object queries focus on respective semantic components with few overlaps.The queries are expected to represent different semantic components without external supervising.
1) Occluded Sample Augmentation (OSA):
Occluded Sample Augmentation generates identity-preserving images with varied obstacles, while contrastive triplets separate identity-relevant features from occlusion-related interference.
- 1) Occluded Sample Augmentation (OSA):: OSA randomly overlays selected training-set obstacles onto person images while preserving their identity labels.For each image, multiple augmented versions are created using randomly selected obstacles from the obstacle set.
- 1) Occluded Sample Augmentation (OSA):: Contrast triplets use the original image as anchor, a same-identity image with different obstacles as positive, and a different-identity image with the same obstacle as negative.This construction encourages identity-relevant features to distinguish people rather than obstacle appearance.
- 1) Occluded Sample Augmentation (OSA):: Reverse triplet loss swaps positive and negative roles to guide ID-irrelevant queries toward occlusion semantic components.Together with decorrelation and contrastive learning, it keeps occlusion components in ID-irrelevant features and frees ID-relevant features from occlusions.
C. Training and Inference
DRL-Net trains its convolutional and transformer feature extractor jointly with classification, decorrelation, and contrastive losses, then performs inference using only identity-relevant features.
- C. Training and Inference: Training constructs an obstacle set, augments each mini-batch, extracts ID-relevant and ID-irrelevant features, and optimizes the overall loss jointly.The overall loss combines Lce, Lo, Ltri, and λLrtri.
- C. Training and Inference: The total objective combines classification, object-query decorrelation, triplet, and reverse-triplet losses.The reverse-triplet term is weighted by λ, while the other scale factors are set to 1.
- C. Training and Inference: During inference, query and gallery images pass through DRL-Net without augmentation, and distances are computed from ID-relevant features while ID-irrelevant features are ignored.The algorithm computes the query-gallery distance matrix using cosine distance.
IV. EXPERIMENTS
Experiments evaluate DRL-Net on one occluded re-ID benchmark and two holistic re-ID benchmarks using standard dataset splits and image preprocessing.
- IV. EXPERIMENTS: Occluded-DukeMTMC contains 15,618 training images, 17,661 gallery images, and 2,210 occluded query images.Its standard setting uses 9%, 100%, and 10% occluded images in the training, query, and gallery sets, respectively.
- IV. EXPERIMENTS: Training and inference resize person images to 256 × 128, with training augmentation using horizontal flips, random crops, and random erasing.The obstacle set is built from training images using obstacles that rarely appear in test images.
C. Comparison with the State-of-the-Art
DRL-Net achieves strong occluded re-ID performance against holistic, external-cue, and part-matching methods, while also generalizing effectively to holistic benchmarks.
- DRL-Net is superior to holistic, external-cue, and part-to-part matching methods on Occluded-DukeMTMC.
- 79.3% Rank-5 accuracy, 83.6% Rank-10 accuracy, and 50.8% mAP are reported for DRL-Net on Occluded-DukeMTMC.
- DRL-Net performs comparably with methods across global-feature, part-feature, and external-cue categories on Market-1501 and DukeMTMC.
- DRL-Net achieves outstanding performance across all evaluation metrics on the MSMT17 holistic re-ID dataset.
D. Ablation Study
Ablation studies show that the transformer architecture, occluded sample augmentation, and contrast feature learning jointly improve occluded re-ID, with performance depending on architectural and parameter choices.
- The ablation study evaluates the transformer architecture, occluded sample augmentation, and contrast feature learning on Occluded-DukeMTMC.
- Occluded sample augmentation improves CMC Rank-1/5/10 but slightly decreases mAP when synthesized and real occlusions differ.
- Contrast feature learning further improves performance by separating occlusion features from discriminative ID-relevant features.
- 14.0% higher Rank-1 accuracy and 7.0% higher mAP than the Baseline demonstrate gains from the combined components.
- Two transformer layers achieve the best re-ID performance, while deeper models provide diminishing improvement.
- Increasing the number of object queries continuously improves re-ID performance but also increases inference cost, motivating Nq = 9.
- The best reported settings are λ = 1.0, which improves Rank-1/mAP by 4.5%/2.6%, and α = 1.0.
F. Visualization
DRL-Net improves occluded person re-identification by producing more accurate rankings and by automatically localizing occlusion and noise regions without explicit supervision.
- DRL-Net correctly identifies images of the same pedestrian in occluded queries, while the baseline produces many false matches.The comparison uses top-10 ranking results for each occluded query image.
- The ID-irrelevant object query automatically localizes occlusion and noise areas despite variation in obstacles across person images.Decoder cross-attention heat maps visualize these regions, with redder areas indicating higher attention scores.
- Extensive benchmark evaluations show that DRL-Net achieves consistently superior re-identification performance.