PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixed High-Order Attention Network for Person Re-Identification

Binghui Chen, Weihong Deng, Jiani Hu

arXiv:1908.05819v1cs.CV

TL;DR

Person ReID commonly uses coarse or first-order attention, while higher-order attention for subtle cross-camera identity differences remains underexplored. The paper proposes HOA and the mixed MHN framework, which combines different attention orders and adversarial training for zero-shot ReID. Experiments report superiority over a wide range of state-of-the-art methods on three large benchmarks.

  • Problem

    Existing ReID attention methods are mainly coarse or first-order, limiting modeling of complex high-order relationships and subtle pedestrian differences in a zero-shot setting.

  • Method

    The paper proposes HOA with a high-order polynomial predictor and MHN, which mixes different HOA orders and adds an adversarial constraint to prevent order collapse.

  • Results

    MHN outperforms a wide range of state-of-the-art ReID models on Market-1501, DukeMTMC-ReID, and CUHK03-NP.

  • Takeaways & Limitations

    The approach provides a model-agnostic way to enrich attention information for person ReID across three large benchmarks.

  • Takeaways & Limitations

    In zero-shot settings, a high-order attention module can collapse to a lower-order module because of biased learning behavior.

Abstract

from arXiv · show

Attention has become more attractive in person reidentification (ReID) as it is capable of biasing the allocation of available resources towards the most informative parts of an input signal. However, state-of-the-art works concentrate only on coarse or first-order attention design, e.g. spatial and channels attention, while rarely exploring higher-order attention mechanism. We take a step towards addressing this problem. In this paper, we first propose the High-Order Attention (HOA) module to model and utilize the complex and high-order statistics information in attention mechanism, so as to capture the subtle differences among pedestrians and to produce the discriminative attention proposals. Then, rethinking person ReID as a zero-shot learning problem, we propose the Mixed High-Order Attention Network (MHN) to further enhance the discrimination and richness of attention knowledge in an explicit manner. Extensive experiments have been conducted to validate the superiority of our MHN for person ReID over a wide variety of state-of-the-art methods on three large-scale datasets, including Market-1501, DukeMTMC-ReID and CUHK03-NP. Code is available at http://www.bhchen.cn/.

1. Introduction

Person ReID must match pedestrians across cameras despite viewpoint, pose, lighting, and occlusion changes. The paper addresses limitations of coarse or first-order attention with high-order attention and a mixed network designed for zero-shot identity recognition.

  • Person ReID matches pedestrian images across videos captured by different cameras, where viewpoints, lighting, poses, and occlusions make recognition challenging.
  • Spatial attention produces masks and channel attention produces scale vectors, but both are limited in capturing complex high-order interactions among visual parts.
  • High-Order Attention uses a high-order polynomial predictor to model relationships among visual parts and produce discriminative attention results that capture subtle pedestrian differences.
  • MHN combines multiple HOA modules with different orders to preserve diverse attention knowledge and reduce biased learning toward identities seen during training.
  • An adversarial learning constraint is introduced to prevent order collapse during MHN training.
  • MHN is model-agnostic, applies to architectures such as IDE and PCB, and outperforms a wide range of state-of-the-art models on three large ReID benchmarks.

2. Related work

This section reviews attention-based person ReID, high-order statistics for visual representation, and zero-shot learning as related foundations. It positions person ReID as matching the seen/unseen identity setting of zero-shot learning while noting limited attention to that perspective.

  • Person ReID & Attention Mechanism: Person ReID matches pedestrian images across videos captured by different cameras using approaches including pairwise constraints, metric learning, deep embeddings, re-ranking, and attribute learning.
  • Person ReID & Attention Mechanism: Attention methods allocate computational resources toward informative input regions, including hard attention for locating parts and soft attention for enhancement.
  • High-order statistics: High-order statistics provide powerful representations, and high-order pooling integrated with deep CNNs has shown promise in fine-grained visual categorization.
  • High-order statistics: High-order representations have been developed through bilinear pooling, Tensor Sketch approximations, iterative compression, and aggregation of hierarchical convolutional responses.
  • Zero-Shot Learning: Zero-shot learning trains on seen classes and uses learned knowledge to distinguish unseen classes; person ReID fits this setting because training and testing identities do not intersect.
  • Zero-Shot Learning: Existing person ReID works largely ignore its zero-shot learning setting, despite zero-shot learning also being studied in image classification, video recognition, and image retrieval or clustering.

3. Proposed Approach

The approach generalizes attention from spatial or channel reweighting to high-order statistics, then combines HOA modules of different orders in MHN to prevent order collapse and diversify attention knowledge.

  • 3.1. Problem Formulation: Attention reweights convolutional activations element-wise, with spatial and channel attention arising as special cases of the general formulation.A(X) supplies values in [0, 1]; spatial masks replicate across channels, while channel scales replicate across spatial dimensions.
  • 3.2. High-Order Attention Module: High-order attention addresses the inability of coarse spatial and channel attention to model complex interactions among visual parts and subtle pedestrian differences.The module models A(X) using high-order statistics to produce more discriminative attention proposals.
  • 3.2. High-Order Attention Module: HOA represents local descriptors with polynomial statistics, using low-rank tensor approximations and trainable projections to form vector-like attention maps.The r-th order tensor contains degree-r monomials; factorization controls parameters and overfitting for orders r > 1.
  • 3.2. High-Order Attention Module: A sigmoid-based HOA map can include a ReLU nonlinearity, producing attention values in [0, 1] before applying the map to convolutional features.Shared weights across spatial locations allow implementation with 1x1 convolutions and element-wise operations.
  • 3.3. Mixed High-Order Attention Network: MHN uses several HOA modules with different orders and an adversarial constraint that maximizes discrepancies among their encoded features to suppress order collapse.Because streams share the downstream encoder, feature diversity is attributed to the HOA modules, encouraging distinct attention knowledge.

4. Experiments

Experiments evaluate MHN on three zero-shot person-ReID benchmarks, comparing it with baselines and attention methods while analyzing module count, adversary constraints, nonlinearity, placement, and model size.

  • Experimental Setup: MHN is evaluated on Market-1501, DukeMTMC-ReID, and CUHK03-NP under zero-shot learning settings using ResNet50-based IDE and PCB architectures.The experiments use SGD training with 70 epochs and apply MHN to both baseline architectures.
  • Comparison with State-of-the-Art Methods: MHN-6 (PCB) improves over PCB by 2%/6.4% R-1/mAP on Market-1501 and 5.2%/7.5% R-1/mAP on DukeMTMC-ReID.The paper reports that MHN-6 (PCB) achieves new state-of-the-art performance on all three benchmarks.
  • Component Analysis: R-1 on CUHK03-NP Labeled rises from 71.2% to 77.2% when PCB uses six rather than two HOA modules.Increasing the number of HOA modules generally improves performance, while increasing to k = 8 yields few additional gains.
  • Component Analysis: The adversary constraint further improves MHN-6 over unconstrained variants on DukeMTMC and Market by preventing HOA order collapse.Without the constraint, multiple HOA modules already improve R-1 and mAP over IDE and PCB; adding Ladv produces further gains.
  • Component Analysis: MHN outperforms spatial and channel attention methods, while adding nonlinearity further improves high-order attention performance.The comparisons include HA-CNN, SENet50, SpaAtt+Q, and CASN+IDE; the paper attributes MHN’s advantage to modeling complex high-order information.
  • Component Analysis: Placing HOA modules after layer2 performs best because lower layers contain noisy texture information and higher layers may lose useful knowledge for unseen identities.The layer2 configuration is used for both IDE and PCB throughout the experiments.
  • Component Analysis: MHN parameter counts increase with HOA order but remain relatively small compared with SENet50, while each MHN outperforms SENet50.The authors characterize the resulting model as light and sweet.

5. Conclusion

The paper combines high-order attention with a zero-shot learning perspective to address biased visual learning in person ReID. MHN uses mixed-order HOA modules and an adversary constraint, with experiments on three benchmarks validating the method’s necessity and effectiveness.

  • HOA models complex, high-order statistics of visual parts to produce more discriminative attention proposals.The module is designed to capture subtle pedestrian differences.
  • MHN uses HOA modules at different orders to reduce partial or biased learning focused only on seen identities.This design targets the zero-shot nature of person ReID.
  • An adversary constraint is introduced to prevent order collapse in the HOA module.
  • Experiments on three popular benchmarks validate the necessity and effectiveness of the proposed method.
Loading 1908.05819v1…