PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Graph Representation Learning for Video Person Re-identification

Yiming Wu, Omar El Farouk Bourahla, Xi Li, Fei Wu, Qi Tian, Xue Zhou

arXiv:1909.02240v2cs.CV

TL;DR

Video person Re-ID remains challenging because existing methods may not capture intrinsic relations between body parts across frames. This paper combines pose-aligned and feature-affinity graph connections with temporal-resolution regularization. It reports competitive, effective performance across four standard benchmarks.

  • Problem

    Part-attention methods may fail to use intrinsic relations between parts across frames, which matter when occlusion and pose variation make cues from other frames complementary.

    Source: §I · pp. 1–2

  • Method

    The method builds an adaptive graph from pose-alignment and feature-affinity connections, propagates neighboring regional features, and regularizes consistency across temporal resolutions.

    Source: §I · p. 2

  • Results

    The method is reported effective on iLIDS-VID, PRID2011, MARS, and DukeMTMC-VideoReID.

    Source: §I · p. 2, §V · p. 8

  • Takeaways & Limitations

    The authors conclude that graph-based contextual aggregation and temporal-resolution consistency support compact, discriminative video representations.

    Source: §V · p. 8

Abstract

from arXiv · show

Recent years have witnessed the remarkable progress of applying deep learning models in video person re-identification (Re-ID). A key factor for video person Re-ID is to effectively construct discriminative and robust video feature representations for many complicated situations. Part-based approaches employ spatial and temporal attention to extract representative local features. While correlations between parts are ignored in the previous methods, to leverage the relations of different parts, we propose an innovative adaptive graph representation learning scheme for video person Re-ID, which enables the contextual interactions between relevant regional features. Specifically, we exploit the pose alignment connection and the feature affinity connection to construct an adaptive structure-aware adjacency graph, which models the intrinsic relations between graph nodes. We perform feature propagation on the adjacency graph to refine regional features iteratively, and the neighbor nodes' information is taken into account for part feature representation. To learn compact and discriminative representations, we further propose a novel temporal resolution-aware regularization, which enforces the consistency among different temporal resolutions for the same identities. We conduct extensive evaluations on four benchmarks, i.e. iLIDS-VID, PRID2011, MARS, and DukeMTMC-VideoReID, experimental results achieve the competitive performance which demonstrates the effectiveness of our proposed method. The code is available at https://github.com/weleen/AGRL.pytorch.

I. INTRODUCTION

Video person Re-ID remains challenging under occlusion, viewpoint, illumination, and pose variation because part-attention methods may not use intrinsic relations between parts across frames. The paper proposes adaptive graph connections and temporal-resolution regularization, and evaluates the method on four benchmarks.

I. INTRODUCTION

  • Part-attention approaches often underuse intrinsic cross-frame relations between body parts, despite those relations helping provide complementary cues when parts are occluded.The introduction identifies occlusion, viewpoint, illumination, and pose variation as challenges for video Re-ID.Source: §I · pp. 1–2
  • Pose alignment connects regions containing the same part, while feature affinity connects regions according to visual correlations; their combination forms an adaptive graph for contextual propagation.A GNN aggregates neighboring messages to refine regional features, enhancing informative regions and weakening noisy parts.Source: §I · p. 2
  • Temporal-resolution regularization enforces consistency between representations from randomly selected subsequences, aiming to learn compact, temporally invariant video features.The subsequence representations are encouraged to be close in metric space.Source: §I · p. 2
  • Experiments on iLIDS-VID, PRID2011, MARS, and DukeMTMC-VideoReID demonstrate the proposed method’s effectiveness.The introduction reports evaluations across all four benchmarks but gives no comparative scores here.Source: §I · p. 2

II. RELATED WORK

Prior video Re-ID work models sequence dependencies and selects or aggregates informative frames and regions, while graph-based Re-ID often represents relationships among images or videos. This paper distinguishes its approach by dynamically learning a graph over regional features with prior structure.

II. RELATED WORK

  • RNN-based video Re-ID methods model inter-sequence dependencies, while other approaches use attention for frame or region selection and 3D CNNs to aggregate spatial and temporal cues.The cited examples include RNN methods,, attention methods,,,,,, and 3D CNN methods,,.Source: §II · p. 2
  • A closely related method combines GAT with feature extraction to relate frames and regional variation, but learns weights without prior knowledge.Its regions are based on strongly activated feature-map points.Source: §II · p. 3
  • Graph-based Re-ID methods use image or video nodes to model relationships among training samples, including graph matching, similarity updates, and label estimation.The cited approaches include –.Source: §II · pp. 2–3
  • Unlike sample-level graph methods, this approach dynamically learns a prior-informed graph over regional features and propagates local, global, and structural information.The architecture also concatenates graph-branch and global-branch features for testing.Source: §II · p. 3, Figure 2 · p. 3

III. THE PROPOSED METHOD

The method builds a graph over video regions using pose alignment and feature affinity, then propagates neighbor information to refine regional features. It also regularizes representations from different temporal resolutions to be consistent.

A. Overview

  • The overview combines ResNet50 features, a global branch, and a graph branch whose regional features are updated, attended, and supervised with identification and triplet-ranking losses.The graph branch uses pose and feature affinity to construct its adjacency graph.Source: A. Overview · p. 3

B. Adaptive Graph Feature Propagation

  • The adaptive graph connects regional nodes through pose alignment and feature affinity to capture relations among regions across frames.The graph combines the two complementary relation types into an adaptive structure-aware adjacency graph.Source: B. Adaptive Graph Feature Propagation · p. 4
  • Pose alignment connects regions containing the same human part, with estimated keypoints grouped as head, trunk, and leg.For a region, its part set is determined by the body parts located within it.Source: B. Adaptive Graph Feature Propagation · p. 4, Figure 3 · p. 4
  • Feature affinity adds adaptive links intended to capture fine relations between regional features, complementing pose alignment's coarse relations.The combined edge weights use the pose alignment adjacency matrix and feature affinity matrix.Source: B. Adaptive Graph Feature Propagation · p. 4
  • Stacked graph-neural-network layers aggregate neighbor information and iteratively update regional features into refined representations.The update balances aggregated and original features; α is set to 0.1 in the experiments.Source: B. Adaptive Graph Feature Propagation · pp. 4–5
  • Temporal resolution regularization makes video representations from randomly selected subsequences close to one another in the metric space.The subsequences are formed by randomly selecting T − i frames from the image sequence.Source: B. Adaptive Graph Feature Propagation · p. 5

C. Loss Functions

The training objective combines cross-entropy and soft-hard triplet losses across the global and graph branches. The four branch-specific losses are summed.

C. Loss Functions

  • Training uses cross-entropy loss and soft-hard triplet loss to supervise the model parameters jointly.Source: C. Loss Functions · p. 5
  • The soft-hard triplet loss uses anchor, positive, and negative features, with distance measured by the L2 norm.P and K denote identities and sampled images per identity in a mini-batch.Source: C. Loss Functions · p. 5
  • The global and graph branches each have two losses, and the total loss sums the four branch losses.Source: C. Loss Functions · p. 5

A. Datasets

The evaluation uses four video person Re-ID datasets: iLIDS-VID, PRID2011, MARS, and DukeMTMC-VideoReID.

A. Datasets

  • PRID2011 contains videos from two camera views, with only the first 200 identities appearing in both cameras.The evaluation uses sequences longer than 21 frames.Source: A. Datasets · p. 5
  • iLIDS-VID contains 600 sequences of 300 people, with two videos per person and an average sequence duration of 73 frames.Source: A. Datasets · p. 5
  • MARS contains 1,261 identities and around 20,000 video sequences across six cameras, plus 3,248 distracter sequences.Source: A. Datasets · p. 5
  • DukeMTMC-VideoReID contains 4,832 tracklets and 1,812 identities, with training, testing, and distraction splits.It has 369,656 training frames and 445,764 testing and distraction frames.Source: A. Datasets · p. 5

B. Evaluation Metrics

Evaluation uses CMC and mAP, with the reported metrics depending on the dataset protocol. PRID2011 and iLIDS-VID report CMC, while MARS and DukeMTMC-VideoReID report both metrics.

B. Evaluation Metrics

C. Implementation Details

Implementation uses pose estimates to build regional connections and refine features; training samples eight frames per video, while testing offers a faster or more accurate sequence strategy.

C. Implementation Details

  • The implementation uses ImageNet-pretrained ResNet50 and resizes input images to 256 × 128.Experiments use PyTorch and four TITAN X GPUs.Source: C. Implementation Details · p. 5
  • The network randomly samples T = 8 frames from every video into a tracklet and is trained with ADAM at learning rate 1×10−4 and weight decay 5×10−4 for 300 epochs.Source: C. Implementation Details · p. 6
  • Testing either represents a video with its first image or averages representations from chunk-based image sequences; the second strategy is more accurate, while the first is faster.Cosine distance ranks the resulting video representations.Source: C. Implementation Details · p. 6
  • AlphaPose keypoints define body parts and pose-alignment connections between vertically partitioned regional features, which are combined with feature-affinity connections for propagation.For multiple people in a frame, the selected pose is chosen using bounding-box size, central position, and average keypoint confidence.Source: C. Implementation Details · p. 6, Figure 5 · p. 6

D. Comparison with State-of-the-art Methods

The method outperforms prior approaches on MARS and DukeMTMC-VideoReID, and reports state-of-the-art results on PRID2011 and iLIDS-VID, with optical-flow comparisons qualified on the smaller datasets.

D. Comparison with State-of-the-art Methods

E. Ablation Study

Ablations on MARS report gains from the graph and consistency components, stable performance across propagation depths, and qualitative improvements under clutter and occlusion.

E. Ablation Study

  • Rank-1 remains above 89.0% and mAP above 80.5% with one, two, or three propagation layers; K = 2 performs best and is used in the experiments.The propagation-layer comparison is reported in Table IV.Source: Table IV · p. 7, E. Ablation Study · p. 7
  • 89.8% Rank-1 and 81.1% mAP with all proposed components, up from the baseline’s 87.8% Rank-1 and 78.0% mAP.The ablation table reports Rank-1 and mAP among its metrics.Source: Table V · p. 7, E. Ablation Study · p. 7
  • +Ap improves Rank-1 and mAP by 0.7% and 0.4% over +Attention; +Af reaches 88.5% Rank-1 and 79.8% mAP.Combining +Ap and +Af reaches 89.3% Rank-1 and 80.4% mAP; consistency loss improves Rank-1 and mAP by 0.5% and 0.7%.Source: E. Ablation Study · p. 7, Table V · p. 7
  • Grad-CAM examples show graph-based models weakening background distractions and avoiding wrong attentive regions in the illustrated occlusion cases.The figure compares the baseline without graph learning with models using feature-affinity, pose-alignment, or combined graphs.Source: E. Ablation Study · p. 7, Figure 6 · p. 8

V. CONCLUSION

The conclusion presents adaptive regional graph learning and temporal-resolution consistency as contributions, with results across four benchmarks and ablations supporting the approach.

V. CONCLUSION

  • The method learns an adaptive structure-aware graph over spatial person regions and propagates neighbor messages to enrich person representations.The conclusion describes the propagated information as complementary contextual information.Source: §V · p. 8
  • A regularization term enforces consistency across temporal resolutions to learn compact, discriminative representations.Source: §V · p. 8
  • Results on four standard benchmarks and ablation studies are reported as evidence of the scheme’s effectiveness and component feasibility.Source: §V · p. 8
Loading 1909.02240v2…