Source-linked AI summary
High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification
Guan'an Wang, Shuo Yang, Huanyu Liu, Zhicheng Wang, Yang Yang, Shuliang Wang, Gang Yu, Erjin Zhou, Jian Sun
TL;DR
Occluded person ReID must match incomplete images to holistic images across disjoint cameras despite occlusion, outliers, and alignment difficulties. The paper learns high-order relation and topology information with ADGC and CGEA, and reports favorable results across occluded, partial, and holistic ReID tasks, including a 6.5% mAP improvement on Occluded-Duke.
Problem
Occluded person ReID is challenging because missing regions reduce discriminative information while occlusions and outliers disrupt local-feature alignment.
Method
The framework extracts semantic key-point features, uses ADGC to learn directed relations among local-feature graph nodes, and uses CGEA for graph-based robust soft alignment and similarity prediction.
Results
6.5% mAP improvement on Occluded-Duke is reported, alongside favorable performance on occluded, partial, and holistic ReID tasks.
Takeaways & Limitations
High-order relation and topology information support discriminative features and robust alignment for occluded person ReID.
Abstract
from arXiv · showhide
Occluded person re-identification (ReID) aims to match occluded person images to holistic ones across dis-joint cameras. In this paper, we propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. At first, we use a CNN backbone and a key-points estimation model to extract semantic local features. Even so, occluded images still suffer from occlusion and outliers. Then, we view the local features of an image as nodes of a graph and propose an adaptive direction graph convolutional (ADGC)layer to pass relation information between nodes. The proposed ADGC layer can automatically suppress the message-passing of meaningless features by dynamically learning di-rection and degree of linkage. When aligning two groups of local features from two images, we view it as a graph matching problem and propose a cross-graph embedded-alignment (CGEA) layer to jointly learn and embed topology information to local features, and straightly predict similarity score. The proposed CGEA layer not only take full use of alignment learned by graph matching but also re-place sensitive one-to-one matching with a robust soft one. Finally, extensive experiments on occluded, partial, and holistic ReID tasks show the effectiveness of our proposed method. Specifically, our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
1. Introduction
Occluded person ReID must match incomplete pedestrian observations across disjoint cameras despite missing information and misalignment. The proposed framework addresses these problems by modeling high-order relations during feature learning and topology during alignment.
- Motivation: Occluded ReID is necessary because obstacles or limited camera views produce incomplete pedestrian observations.The task supports applications including video surveillance, security, and smart cities.
- Motivation: Occluded images contain less discriminative information and are more likely to match incorrect people.Occluded or outlier key-points can produce meaningless local features and misalignment.
- Motivation: Existing occluded and partial ReID methods mainly use one-order information for feature learning and alignment.Examples include predefined regions, poses, and human parsing.
- Proposed framework: The framework combines one-order semantic features with high-order relation and human-topology modules.A CNN backbone and key-points estimator provide semantic local features before graph-based processing.
- Proposed framework: ADGC models relations among local-feature nodes, while CGEA treats alignment as graph matching and learns node-to-node and edge-to-edge correspondence.ADGC suppresses messages from meaningless or noisy features; CGEA supports robust soft alignment instead of sensitive hard one-to-one matching.
2. Related Works
Prior work addresses holistic, occluded, and partial ReID, but commonly relies on one-order information. This paper instead uses high-order relation and human-topology information for feature learning and alignment.
- Person Re-Identification: Person ReID matches pedestrian images across disjoint cameras despite variation from views, poses, illumination, and occlusions.Existing approaches include handcrafted descriptors, metric learning, and deep learning methods.
- Occluded Person Re-identification: Occluded ReID matches occluded probe images to full-body gallery images but faces incomplete information and spatial misalignment.Prior methods use occlusion classification, saliency prediction, teacher-student learning, or pose-guided feature alignment.
- Partial Person Re-Identification: Partial ReID matches partial probe images to holistic gallery images, often under imperfect detection and camera-view outliers.Prior work includes global-to-local matching and feature-map reconstruction.
- High-order information: Unlike existing occluded and partial ReID methods, the proposed approach uses high-order relation and human-topology information for feature learning and alignment.The paper reports better performance from this distinction.
3. The Proposed Method
The proposed method jointly models semantic features, high-order relations, and human topology to learn discriminative features and robust alignment for occluded person ReID. It uses ADGC for adaptive relation propagation and CGEA for topology-guided soft alignment and similarity prediction.
- Framework: The framework jointly trains semantic, high-order relation, and high-order human-topology modules for feature learning and robust alignment.The semantic module extracts key-point-region features; the relation and topology modules model within-image relations and cross-image alignment.
- One-Order Semantic Module: CNN and key-points models extract semantic local features from key-point regions, while softmax-normalized heat maps help prevent noise and outliers.The module also extracts a global feature and applies classification and triplet losses with key-point confidence weighting.
- High-Order Relation Module: ADGC dynamically updates directed edge weights from differences between local and global features before graph convolution and residual fusion.This promotes message passing among semantic features and suppresses meaningless or noisy features caused by occlusion and outliers.
- High-Order Relation Module: A cascade of ADGC layers produces relation features whose pairwise similarity is calculated with cosine distance.The relation module is trained with classification and triplet losses, using key-point confidence in the similarity formulation.
- High-Order Human-Topology Module: Graph matching incorporates node-to-node and edge-to-edge scores, but CGEA embeds its learned correspondence into both graphs rather than relying solely on hard matching.The matching matrix represents correspondence between nodes across the two image graphs.
- High-Order Human-Topology Module: CGEA performs graph matching between hidden features, embeds the learned affinity matrix, and outputs topology-guided features for robust cross-image alignment.It uses correspondence-based message passing while avoiding sensitive hard one-to-one alignment.
4. Train and Inference
The framework is trained end-to-end with a weighted overall objective and combines relation- and topology-based similarities. During inference, relation similarity selects nearest gallery neighbors before the final similarity refines that shortlist.
- Training: The overall objective combines weighted training terms, and the framework is trained end-to-end by minimizing the resulting loss.The weights λ* control the corresponding terms.
- Similarity: For each image pair, the method obtains separate relation-based and topology-based similarities before combining them into a final similarity.The relation similarity comes from Eq.(6), while the topology similarity comes from Eq.(10).
- Inference: At inference, relation similarity ranks all gallery images and selects the top n nearest neighbors for refinement with the final similarity.The final similarity is then computed for the shortlisted neighbors.
5. Experiments
Experiments evaluate the framework on holistic, occluded, and partial ReID datasets, with ablations testing its modules and layers. The method achieves strong results on occluded and partial tasks, comparable holistic performance, and robustness to parameter choices.
- Benchmark Evaluation: The framework is evaluated on six public datasets: two holistic, two occluded, and two partial.
- Occluded ReID: 55.1% and 80.4% Rank-1 scores are achieved on Occluded-Duke and Occluded-ReID, respectively, outperforming competing methods.
- Partial ReID: The method outperforms other methods by at least 2.6% and 4.4% Rank-1 on Partial-REID and Partial-iLIDS, respectively.
- Holistic ReID: On holistic datasets, simply adding human-parsing or key-points information may not improve performance, while the proposed method achieves comparable results.
- Module Analysis: Ablations show that adding semantic, relation, and topology modules raises Rank-1 from 49.9% to 55.1%.Semantic information improves performance to 52.4%, relation information to 53.9%, and the full framework to 55.1%.
- Layer Analysis: Removing NORM, ADGC, or CGEA decreases Rank-1 by 2.6%, 1.4%, or 0.7%, respectively, supporting the effectiveness of these components.
- Parameter Analysis: The optimal parameter setting is γ = 0.5 and n = 8, and performance remains stable across different parameter values.The other parameter is fixed at its optimal value when analyzing one parameter.
6. Conclusion
The paper proposes a framework that learns high-order relation information for discriminative features and topology information for robust alignment in occluded person ReID.
- The framework models local features as graph nodes and uses ADGC to promote semantic message passing while suppressing meaningless and noisy features.
- The CGEA layer uses graph matching and verification loss to learn topology information, robustly align features, and avoid sensitive hard one-to-one alignment.
- Experiments on occluded, partial, and holistic datasets demonstrate the effectiveness of the proposed framework.