Source-linked AI summary
Dual Attention Matching Network for Context-Aware Feature Sequence based Person Re-Identification
Jianlou Si, Honggang Zhang, Chun-Guang Li, Jason Kuen, Xiangfei Kong, Alex C. Kot, Gang Wang
TL;DR
Single feature-vector representations are insufficient for reliable person ReID under visual ambiguity and sequence misalignment. DuATM learns context-aware feature sequences and compares them with dual attention, combining refinement and alignment, and reports effectiveness across image and video benchmarks.
Problem
Single feature-vector representations are insufficient for reliable person ReID when appearance changes, similar clothing, occlusion, or sequence misalignment create visual ambiguity.
Method
DuATM jointly learns context-aware feature sequences and uses intra-sequence refinement with inter-sequence feature-pair alignment in an end-to-end siamese network.
Results
DuATM demonstrates effectiveness and significant advantages on image- and video-based ReID benchmark datasets, achieving rank-1 accuracies of 91.24% on Market-1501 and 81.37% on DukeMTMC-reID.
Takeaways & Limitations
Dual attentive context-aware feature sequences provide an effective alternative to average-pooling-based single feature vectors for person ReID.
Takeaways & Limitations
Comparisons use different backbones across methods, so comprehensive evaluation with consistent backbones remains future work.
Abstract
from arXiv · showhide
Typical person re-identification (ReID) methods usually describe each pedestrian with a single feature vector and match them in a task-specific metric space. However, the methods based on a single feature vector are not sufficient enough to overcome visual ambiguity, which frequently occurs in real scenario. In this paper, we propose a novel end-to-end trainable framework, called Dual ATtention Matching network (DuATM), to learn context-aware feature sequences and perform attentive sequence comparison simultaneously. The core component of our DuATM framework is a dual attention mechanism, in which both intra-sequence and inter-sequence attention strategies are used for feature refinement and feature-pair alignment, respectively. Thus, detailed visual cues contained in the intermediate feature sequences can be automatically exploited and properly compared. We train the proposed DuATM network as a siamese network via a triplet loss assisted with a de-correlation loss and a cross-entropy loss. We conduct extensive experiments on both image and video based ReID benchmark datasets. Experimental results demonstrate the significant advantages of our approach compared to the state-of-the-art methods.
1. Introduction
Person ReID based on single feature vectors struggles with visual ambiguity and local or temporal misalignment. DuATM addresses these challenges by learning context-aware feature sequences and comparing them with dual attention.
- Single feature-vector representations struggle when pedestrians have drastic appearance changes or similar clothing, because they emphasize overall appearance over local discriminative parts.
- Feature-sequence and feature-set methods preserve detailed visual cues but can fail under spatial displacement, interference, temporal misalignment, or body-part occlusion.
- DuATM jointly learns context-aware feature sequences and performs attentive sequence comparison through cascaded extraction and matching modules.The extraction module handles image and video inputs, while the matching module compares paired sequences.
- Dual attention performs intra-sequence feature refinement and inter-sequence feature-pair alignment simultaneously.
- DuATM is trained as a siamese network with triplet, de-correlation, and cross-entropy losses, and experiments evaluate it on image- and video-based benchmark datasets.
2. Related Works
Person ReID research uses either single feature vectors or feature sets and sequences to represent pedestrians. DuATM applies dual attention to refine and align feature sequences instead of relying on heuristic correspondence structures.
- Feature-vector methods represent each image or video with one vector and learn a metric based on those representations.
- Feature sets and sequences preserve complementary visual cues or spatial information through multiple local, semantic, or temporal feature vectors.
- Feature-sequence methods can fail when heavy sequence misalignments or interferences occur.
- Attention mechanisms have been used for sequence alignment, progressive image comparison, and attentive aggregation in related recognition tasks.
- DuATM compares paired feature sequences through inter-sequence alignment and simultaneous intra-sequence refinement rather than heuristic correspondence structures.
3. Our Proposal: Dual Attention Matching Network (DuATM)
DuATM jointly extracts context-aware spatial or temporal feature sequences and compares possibly misaligned sequence pairs through bidirectional dual attention. It is trained end-to-end with triplet, de-correlation, and cross-entropy losses.
- Overall framework: DuATM uses separate cascaded modules to extract feature sequences and match them with dual attention.The extraction module handles image or video inputs, while the matching module refines and aligns sequence pairs.
- Feature sequence extraction: Image inputs become spatial feature sequences, whereas video inputs use bidirectional recurrence to encode temporal-spatial appearance and motion cues.DenseNet-121 produces image features; video frame features are passed through a bidirectional recurrent layer before hidden states form the sequence.
- Sequence matching: The matching module refines and aligns possibly unaligned sequences before aggregating distances between multiple feature pairs.This avoids relying on average pooling that would ignore sequence misalignment or corrupted feature vectors.
- Dual attention block: Intra-sequence attention refines feature vectors, while inter-sequence attention aligns feature pairs using contextual information.The dual attention block combines a transform layer for feature-aware filters with an attention layer for weights.
- Distance computation and aggregation: Bidirectional dual attention computes Euclidean local distances in both comparison directions and average-pools them into a holistic sequence distance.The resulting distance is defined by the sequence matching module after refined and aligned feature pairs are compared.
- Loss functions for training DuATM: Training uses a siamese architecture with triplet loss plus de-correlation and cross-entropy auxiliary losses.Random convex combinations during pooling are used with cross-entropy loss to learn more informative and robust feature sequences.
4. Experiments
DuATM is evaluated on image- and video-based ReID benchmarks through dataset comparisons, ablations, parameter analyses, and comparisons with attention, sequence/set, and state-of-the-art methods. Results support dual attention, auxiliary losses, context-aware sequences, and attentive matching, while backbone differences limit comprehensive comparisons.
- Datasets, Evaluation, and Implementations: Experiments use Market-1501, DukeMTMC-reID, and MARS, with CMC and mAP as evaluation metrics.Market-1501 and DukeMTMC-reID provide image-based benchmarks, while MARS provides video tracklets.
- Datasets, Evaluation, and Implementations: DuATM uses DenseNet-121 initialization, staged SGD training, hard triplet mining, 256-dimensional sequence features, and 64-frame video subsequences.The implementation freezes DenseNet parameters initially, then trains the entire network; image mini-batches use P = 10 and V = 4, while video settings begin with P = 7.
- Ablation Studies: Adding dual attention improves performance over single-attention variants, while the combined system outperforms the average-pooling baseline across the three datasets.The de-correlation loss yields a minor gain, whereas cross-entropy produces a significant improvement and combining all three losses improves accuracy further.
- Parameter Analysis: Moderate λ1 and λ2 values help, excessive values degrade performance or risk over-fitting, and performance is insensitive to corruption ratio p.Best settings are (0.3, 0.9, 0.4) on Market-1501, (0.1, 0.5, 0.6) on DukeMTMC-reID, and (0.5, 0.5, 0.4) on MARS.
- Feature Dimension and Sequence Length: For image ReID, low-dimensional sequences retain discrimination, while longer video sequences improve accuracy as they capture cues across more time points.Rank-1 remains 78.50% for D = 16 and 87.71% for D = 32; MARS mAP rises from 21.87% at T = 1 to 59.42% at T = 96.
- Comparison with Other Methods: DuATM achieves rank-1 accuracy of 91.24% on Market-1501 and 81.37% on DukeMTMC-reID, outperforming the compared state-of-the-art methods across the reported benchmarks.On Market-1501, it also surpasses body-part-based and densely matching sequence/set methods by adapting correspondence and removing local corruptions.
- Limitations: Comparisons are not fully comprehensive because evaluated methods use different backbones, including DenseNet, ResNet, and combined networks.The paper identifies evaluation across different backbones as future work.
5. Conclusions
DuATM is an end-to-end framework for person ReID that learns context-aware feature sequences and compares them with dual attention. Its experiments on large-scale image and video datasets confirmed significant advantages.
- DuATM jointly learns context-aware feature sequences and performs dually attentive comparison for person ReID.
- Its dual attention block simultaneously performs feature refinement and feature-pair alignment.
- DuATM is trained with a triplet loss assisted by de-correlation and cross-entropy losses.
- Experiments on large-scale image and video datasets confirmed the significant advantages of the proposed framework.