Source-linked AI summary
Person Re-identification with Deep Similarity-Guided Graph Neural Network
Yantao Shen, Hongsheng Li, Shuai Yi, Dapeng Chen, Xiaogang Wang
TL;DR
Person re-identification models often estimate probe-gallery similarities independently, leaving relations among image pairs underused for hard samples. SGGNN builds a graph of probe-gallery pairs and uses supervised similarity-guided message passing to refine their relation features end to end. The method is reported effective across three public datasets, including 82.8% mAP and 92.3% top-1 accuracy on Market-1501.
Problem
Existing models often ignore relations among different probe-gallery pairs, limiting the relationship information used to estimate similarities for hard samples.
Method
SGGNN represents probe-gallery pairs as graph nodes and updates their relation features through message passing whose fusion weights are supervised by gallery-pair similarity labels.
Results
82.8% mAP and 92.3% top-1 accuracy are reported for SGGNN on Market-1501, with effectiveness validated on three public person re-identification datasets.
Takeaways & Limitations
The paper's results support incorporating gallery-gallery relations into training rather than relying on post-processing alone for similarity refinement.
Abstract
from arXiv · showhide
The person re-identification task requires to robustly estimate visual similarities between person images. However, existing person re-identification models mostly estimate the similarities of different image pairs of probe and gallery images independently while ignores the relationship information between different probe-gallery pairs. As a result, the similarity estimation of some hard samples might not be accurate. In this paper, we propose a novel deep learning framework, named Similarity-Guided Graph Neural Network (SGGNN) to overcome such limitations. Given a probe image and several gallery images, SGGNN creates a graph to represent the pairwise relationships between probe-gallery pairs (nodes) and utilizes such relationships to update the probe-gallery relation features in an end-to-end manner. Accurate similarity estimation can be achieved by using such updated probe-gallery relation features for prediction. The input features for nodes on the graph are the relation features of different probe-gallery image pairs. The probe-gallery relation feature updating is then performed by the messages passing in SGGNN, which takes other nodes' information into account for similarity estimation. Different from conventional GNN approaches, SGGNN learns the edge weights with rich labels of gallery instance pairs directly, which provides relation fusion more precise information. The effectiveness of our proposed method is validated on three public person re-identification datasets.
1 Introduction
Person re-identification models commonly estimate probe-gallery similarities independently, overlooking relations among image pairs that could help resolve hard samples. SGGNN addresses this gap by integrating gallery relations into end-to-end graph-based feature learning with similarity-guided message passing.
- Research gap: Existing approaches often estimate each probe-gallery similarity independently, ignoring internal relations among the full image set.This limited relationship information makes some hard positive and hard negative pairs difficult to score properly.
- Research gap: Manifold learning and re-ranking incorporate image-set relations but are often unsupervised and cannot fully exploit training labels.The paper motivates a supervised alternative that uses available labels during learning.
- Proposed framework: SGGNN treats each probe-gallery image pair as a graph node and propagates learned messages among nodes to refine pairwise relation features.The framework incorporates graph computation during both training and testing to obtain similarity estimations and feature representations.
- Contributions: The paper proposes an end-to-end SGGNN that incorporates inter-gallery-image relations during training rather than only in post-processing.The stated contribution is to enhance feature learning and produce more discriminative and accurate person-image representations.
- Proposed framework: Unlike conventional GNN designs, SGGNN learns feature-fusion weights from supervised similarity scores between gallery-image pairs.This similarity-guided fusion is intended to use label information for more precise relation updates.
2 Related Work
Related work covers feature and metric learning for person re-identification, manifold and re-ranking methods for image-set relations, and graph neural networks for graph-structured data. SGGNN differs by using training-label supervision to generate feature-fusion weights during graph message passing.
- Person Re-identification: Person re-identification research mainly learns robust image features or designs metric distances to measure similarities between person images.Examples include Siamese, cross-input, pose-aware, and feature-alignment CNN approaches.
- Person Re-identification: Manifold learning and re-ranking methods enhance re-identification by modeling relations among multiple person images, especially gallery images.These approaches estimate or integrate contextual similarities beyond a single image pair.
- Graph Neural Network: Graph neural networks extend recursive neural networks and random-walk models to graph-structured data and can be trained end-to-end.Prior graph applications include language, pose, visual relationships, and image classification.
- Graph Neural Network: SGGNN uses training-data label supervision to generate more accurate feature-fusion weights for graph message passing.This distinguishes its graph design from most existing GNN approaches described in the paper.
3 Method
SGGNN formulates person re-identification as graph-based node classification, where probe-gallery relation features are refined using information from other pairs. Gallery-gallery similarity scores guide message passing and feature fusion, with the updated features used for similarity prediction.
- Graph Formulation: A probe image and N gallery images form a fully connected graph whose nodes represent probe-gallery pairs classified by similarity.Each node encodes the relation between its corresponding probe-gallery images.
- Node Features: The base Siamese-CNN encodes each probe-gallery relation by subtracting ResNet-50 pooled features, squaring the result element-wise, and applying batch normalization.
- Similarity-Guided Message Passing: Gallery-gallery similarity scores define edge weights that determine how information from other nodes contributes to probe-gallery feature updates.The graph is fully connected, and each edge weight represents the relation importance between two nodes.
- Similarity-Guided Message Passing: SGGNN fuses each node’s original relation feature with weighted messages from connected nodes, optionally applying the update iteratively.The refined relation features replace the initial features for similarity prediction and training.
- Deep Message Encoding: Each node first passes its relation feature through a two-layer fully connected message network with batch normalization and ReLU before propagation.The resulting deep message is used for node relation-feature updating.
- Relation to Conventional GNN: Unlike conventional GNNs, SGGNN directly supervises gallery-gallery similarity scores and uses them as fusion weights for more precise feature updating.Conventional GNN fusion weights are learned indirectly through backpropagation.
4 Experiments
Experiments on three public person re-identification datasets evaluate SGGNN against prior methods and alternative relation-integration strategies. SGGNN achieves strong benchmark performance, while ablations support end-to-end message passing and gallery-gallery-guided feature fusion.
- Experimental Setup: SGGNN is evaluated using mAP and CMC top-1, top-5, and top-10 accuracy under each dataset’s original single-query protocol.Experiments cover CUHK03, Market-1501, and DukeMTMC.
- CUHK03: 94.3% mAP and 95.3% top-1 accuracy are reported for SGGNN on CUHK03, outperforming all compared methods.The comparison is presented in Table 1.
- Comparisons: Relative gains include 21.1% top-1 accuracy over MSCAN, 18.7% over SSM, and 26.7% mAP plus 33.7% top-1 accuracy over k-reciprocal re-ranking.SSM uses post-processing manifold estimation, whereas SGGNN uses instance relations during training and testing; k-reciprocal uses gallery-gallery information during testing.
- Market-1501: 82.8% mAP and 92.3% top-1 accuracy are reported for SGGNN on Market-1501, where it significantly outperforms state-of-the-art methods.The comparison is presented in Table 2.
- DukeMTMC: SGGNN outperforms compared approaches on DukeMTMC, including methods using GAN-generated data or person attributes.The results are reported in Table 3.
- Ablation Study: Compared with k-reciprocal re-ranking and random walk, SGGNN retains higher mAP and top-1 accuracy across the reported datasets, supporting end-to-end learned message passing.The reported mAP drops for k-reciprocal are 4.3%, 4.4%, and 3.5%; random-walk mAP drops are 3.6%, 4.1%, and 2.2%.
- Feature Evaluation: SGGNN-learned visual features outperform the base model and conventional GNN setting when probe-gallery similarity is estimated by l2 feature distance.These comparisons are summarized in Table 5.
- Sensitivity Analysis: Higher top-K slightly increases accuracy but also increases computational cost.This sensitivity result is reported for different K and top-K choices.
5 Conclusion
The paper concludes that SGGNN incorporates gallery-gallery similarity information into training, unlike prior post-processing approaches, and reports effectiveness through overall results and ablation study.
- SGGNN incorporates rich gallery-gallery similarity information into the training process of person re-identification.The method is designed to use these labels for weighted deep message fusion.
- Unlike most previous methods, SGGNN updates probe-gallery similarity during training rather than only during post-processing.The paper states that post-processing updates cannot benefit visual-feature learning.
- Conventional GNN settings ignore rich gallery-gallery similarity labels, whereas SGGNN uses them to make weighted deep message fusion more effective.
- The reported overall performance and ablation study illustrate the effectiveness of the proposed method.Table 6 is identified as reporting performances for different K and top-K choices.