PaperlayerView exact revision on arXiv

Source-linked AI summary

Hard-Aware Point-to-Set Deep Metric for Person Re-identification

Rui Yu, Zhiyong Dou, Song Bai, Zhaoxiang Zhang, Yongchao Xu, Xiang Bai

arXiv:1807.11206v1cs.CV

TL;DR

Person re-ID is challenging because images vary in pose, viewpoint, illumination, and occlusion, while triplet-loss performance is limited by sampling. HAP2S uses point-to-set triplet loss with soft, adaptive hard-sample weighting. It reports strong results across re-ID and generic metric-learning benchmarks, along with robustness to outliers and flexibility across weight functions.

  • Problem

    Person re-ID is challenging under pose, viewpoint, illumination, and occlusion variation, and metric-loss performance is limited by sampling.

    Source: §1 · pp. 1–2

  • Method

    HAP2S extends triplet loss to point-to-set distances and adaptively gives harder samples greater weights through soft hard-mining.

    Source: §1 · p. 3

  • Results

    HAP2S reports state-of-the-art results on three person re-ID benchmarks and on CUB-200-2011 and Cars196, and greater robustness to outliers than other losses.

    Source: §1 · pp. 3–4, §5 · p. 14

  • Takeaways & Limitations

    The reported effectiveness spans person re-ID and generic metric-learning benchmarks, and two HAP2S weight-function instantiations achieve similar re-ID accuracies.

    Source: §1 · p. 4

Abstract

from arXiv · show

Person re-identification (re-ID) is a highly challenging task due to large variations of pose, viewpoint, illumination, and occlusion. Deep metric learning provides a satisfactory solution to person re-ID by training a deep network under supervision of metric loss, e.g., triplet loss. However, the performance of deep metric learning is greatly limited by traditional sampling methods. To solve this problem, we propose a Hard-Aware Point-to-Set (HAP2S) loss with a soft hard-mining scheme. Based on the point-to-set triplet loss framework, the HAP2S loss adaptively assigns greater weights to harder samples. Several advantageous properties are observed when compared with other state-of-the-art loss functions: 1) Accuracy: HAP2S loss consistently achieves higher re-ID accuracies than other alternatives on three large-scale benchmark datasets; 2) Robustness: HAP2S loss is more robust to outliers than other losses; 3) Flexibility: HAP2S loss does not rely on a specific weight function, i.e., different instantiations of HAP2S loss are equally effective. 4) Generality: In addition to person re-ID, we apply the proposed method to generic deep metric learning benchmarks including CUB-200-2011 and Cars196, and also achieve state-of-the-art results.

1 Introduction

Person re-ID is difficult because of variation across pedestrian images, and conventional triplet sampling leaves many selected triplets useless. HAP2S uses point-to-set distances with adaptive hard-sample weighting and reports strong results across person re-ID and generic metric-learning benchmarks.

1 Introduction

  • Triplet sampling is costly to exhaustively enumerate, and many selected triplets already satisfy the training constraint and provide no useful training signal.The introduction illustrates the issue with two useless triplets among three selected examples.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • Uniform weighting treats hard and easy samples alike, while extreme mining keeps only the hardest sample and can ignore other hard examples or amplify outliers.HAP2S instead uses soft hard-mining rather than a select-or-unselect rule.Source: §1 · p. 3
  • HAP2S replaces point-to-point triplets with point-to-set distances over the positive and negative samples in a mini-batch, weighting harder samples more heavily.For positives, harder samples are farther from the anchor; for negatives, they are nearer.Source: Figure 2 · p. 2, §1 · p. 3
  • HAP2S reports state-of-the-art performance on three person re-ID benchmarks and on CUB-200-2011 and Cars196, plus greater robustness to outliers than other losses.The paper also reports that two HAP2S instantiations achieve similar re-ID accuracies, without dependence on a specific weight function.Source: §1 · pp. 3–4

2 Related Works

Related work distinguishes identification and metric losses for single-image representations, and reviews sampling-sensitive metric losses and hard-sample mining. HAP2S is presented as a point-to-set metric loss that adaptively weights harder samples.

2 Related Works

  • Single-image re-ID methods use identification losses such as softmax [47] or metric losses such as triplet loss, with HAP2S belonging to the metric-loss category..Source: §2 · p. 4
  • Unlike earlier point-to-point metric losses, HAP2S includes all mini-batch samples and adaptively gives harder samples greater weight.Unlike Zhou et al.'s point-to-set loss, which weights marginal-set samples equally in its triplet term, HAP2S weights harder samples more heavily.Source: §2 · pp. 4–5
  • Prior hard-mining methods select samples in a select-or-unselect manner, whereas HAP2S adaptively assigns larger weights to harder samples.The related-work discussion describes methods including hard-negative selection and hardest-positive and hardest-negative selection.Source: §2 · pp. 4–5

3 Proposed Method

The method uses a deep network to produce image features for metric-loss supervision, then extends triplet learning from individual points to sets with adaptive weighting of harder samples. It also proposes combining HAP2S with softmax loss to use class labels in training.

3.1 Overview

  • ResNet-50 and two fully connected layers produce image features used to compute HAP2S loss in each mini-batch.The network uses 1024- and 128-neuron fully connected layers and Euclidean distance as its point-to-point metric.Source: §3.1 · p. 5

3.2 Revisit Triplet Loss

  • Triplet loss pushes same-person distances below different-person distances by a margin, but many triplets become useless once they satisfy the constraint.Hard mining selects the hardest positive and negative for each anchor in a mini-batch.Source: §3.2 · p. 6

3.3 Hard-Aware P2S Loss

  • P2S triplet loss represents all same-label and different-label mini-batch samples as positive and negative sets, generalizing point-to-point triplet loss.The conventional triplet loss selects only the hardest sample to represent each set.Source: §3.3 · pp. 6–7
  • HAP2S assigns adaptive weights to set members so harder samples contribute more, addressing the exclusion of other hard samples and vulnerability to outliers in simple hard mining.Exponential and polynomial weighting are offered as alternative instantiations; α adjusts the polynomial weight distribution.Source: §3.3 · pp. 6–8

3.4 Multi-loss Training

  • Combining HAP2S with softmax loss adds class-ID supervision, which metric loss alone does not fully use.The paper reports that details of the multi-loss network, algorithm, and experiments are in the supplementary material.Source: §3.4 · p. 8

4 Experiments

Experiments compare HAP2S with other losses and state-of-the-art methods across person re-identification and generic metric-learning benchmarks. They also examine robustness to outliers, parameter settings, and implementation and evaluation protocols.

4.3 Comparison with Other Losses

  • HAP2S E and P outperform other tested losses across three re-ID datasets; mAP gaps exceed +2.2% on Market-1501 and DukeMTMC-reID.Feature visualizations show larger inter-class and smaller intra-class variances, while example retrievals show more correct top-ranked matches for HAP2S.Source: Table 1 · p. 9, §4.3 · pp. 10–11, Figure 4 · p. 10

4.1 Datasets and Evaluation Protocols

  • Evaluation uses standard protocols: Market-1501 and DukeMTMC-reID report CMC and mAP, while CUHK03 reports average CMC across 20 train/test splits.Experiments include both labeled and automatically detected CUHK03 bounding boxes.Source: §4.1 · pp. 8–9

4.2 Implementation Details

  • Training uses Adam with HAP2S loss, 256-image mini-batches, and resized, cropped, and flipped images; testing uses unaugmented features ranked by Euclidean distance.The model is trained for 150 epochs, with the learning rate decaying from 4 × 10^-4 to 4 × 10^-7.Source: §4.2 · p. 9

4.5 Robustness to outliers

  • With 1,000 outliers, HAP2S E achieves 64.48% mAP and is least affected; experiments with false-negative labels also support its robustness.For hard triplet loss, training is prone to collapse when the number of outliers exceeds 200.Source: Figure 6 · p. 12, §4.5 · pp. 12–13

4.4 Comparison with State-of-the-arts

  • On DukeMTMC-reID, MLFN and HA-CNN are state of the art; both use softmax, whereas this study uses pretrained ResNet-50.The authors suggest HAP2S could outperform current state-of-the-art methods with a stronger backbone, but do not report that comparison here.Source: §4.4 · p. 12

4.6 Parameter Analysis

  • On Market-1501, high accuracy and training stability occur when σ ≤1.5 or α ≥6; uniform weighting tends to produce no loss and fail in training.HAP2S approaches hard triplet loss as σ→0 or α→∞, and uniform weighting as σ→∞ or α=0.Source: §4.6 · p. 13, Table 3 · p. 13

4.7 Generic Deep Metric Learning

  • On CUB-200-2011, HAP2S outperforms other methods by about +2% on both NMI and Recall@K; on Cars196, it leads Recall@K and has comparable NMI.HAP2S E and P perform similarly; the reported state-of-the-art comparison uses HAP2S E.Source: §4.7 · pp. 13–14, Table 4 · p. 14

5 Conclusions

The paper proposes HAP2S loss to address sample selection in deep metric learning for person re-ID by adaptively weighting harder samples. It reports strong results on person re-ID and generic metric-learning benchmarks, alongside robustness to outliers.

5 Conclusions

  • HAP2S loss adaptively assigns greater weights to harder samples within the point-to-set triplet loss framework.The method addresses sample selection by weighting samples rather than focusing on selecting them.Source: §5 · p. 14
  • HAP2S achieves state-of-the-art re-ID accuracies on three large-scale benchmarks and is more robust than alternatives when training data include outliers.Source: §5 · p. 14
  • HAP2S also achieves state-of-the-art performance on generic deep metric-learning benchmarks.The paper uses an off-the-shelf ResNet-50 and suggests bespoke re-ID networks could further boost performance.Source: §5 · p. 14

Supplementary Material of Hard-Aware Point-to-Set Deep Metric for

The supplement extends the paper's HAP2S re-ID results with further experiments, gradient analysis, and a multi-loss training method. It also identifies the authors and their institutional affiliations.

Supplementary Material

1 Further Experimental Analysis

The supplementary experiments examine HAP2S across distance measures and batch sizes. They report that its variants outperform hard triplet loss in the tested settings, while larger batches improve accuracy at greater computational cost.

1.1 Robustness to Distance Measure

  • Hard triplet training collapses with squared Euclidean and smooth ℓ1 distances, while both HAP2S variants perform well under all three tested distance measures.Source: Table 1 · p. 18, §1.1 · p. 18

1.2 Impact of Batch Size

  • HAP2S variants consistently outperform hard triplet loss across all tested batch sizes on Market-1501.Source: Figure 1 · p. 19, §1.2 · p. 19
  • As batch size increases, mAP and rank-1 accuracy rise steadily for all three losses, but larger batches demand more computing resources.Source: §1.2 · pp. 18–19

2 Gradient Analysis of HAP2S Loss

The gradient analysis shows how HAP2S weights harder samples more heavily in its parameter updates. Its intuitive analysis contrasts that adaptive weighting with uniform weighting and hard triplet selection, which can yield easy or unstable gradients.

2.1 Detailed Gradient Analysis

  • In HAP2S E, farther positive samples and nearer negative samples receive greater derivative weights, so harder samples contribute more to the loss gradient.Source: §2.1 · p. 20

2.2 More Intuitive Analysis

  • The analysis treats weighting as sampling: uniform weighting mainly selects negative pairs at distances of 0.5–2, which are generally easy and induce no loss.Source: §2.2 · p. 21, Figure 2 · p. 21
  • Hard triplet loss always selects the nearest negative, which noisy gradients may make prone to collapse; HAP2S focuses adaptively on hard examples while maintaining stable training.Source: §2.2 · p. 21, Figure 3 · p. 21

3 Multi-loss Training

The section combines HAP2S metric learning with softmax classification in a multi-task network and evaluates the joint training approach on three re-ID datasets. The combined variants outperform competing multi-losses, while softmax also improves HAP2S performance; the authors relate the gains to complementary feature-space objectives.

3.1 Network and Algorithm

  • Softmax uses class-ID labels to classify features into predefined categories, unlike HAP2S, which verifies sample-label pairs; combining the losses may reduce overfitting risk.The joint objective weights HAP2S and softmax losses with λ ∈ [0, 1].Source: §3.1 · pp. 22–23
  • The network shares a pre-trained ResNet-50 backbone between a C-class softmax branch and a 128-neuron feature branch that computes HAP2S loss.The classification branch uses FC layers of 2,048 and C neurons with dropout rate 0.5; the HAP2S branch uses FC layers of 1,024 and 128 neurons, with Euclidean distance as its point-to-point metric.Source: §3.1 · p. 22

3.2 Experimental Evaluations

  • +4.73% mAP and +5.53% rank-1 accuracy on Market-1501, and +3.04% mAP and +3.00% rank-1 accuracy on DukeMTMC-reID, are gains for HAP2S E+Softmax over HAP2S E.On CUHK03, the reported HAP2S E rank-1 gain is +0.44%; HAP2S P's improvements are described as on par.Source: §3.2 · p. 23
  • More than +2.14% in mAP on Market-1501 and DukeMTMC-reID, and more than +2% in CUHK03 rank-1 accuracy, separate HAP2S+Softmax from competing multi-losses.Both HAP2S E+Softmax and HAP2S P+Softmax outperform Quadruplet+Softmax and Hard Triplet+Softmax on all three datasets.Source: Table 2 · p. 24, §3.2 · p. 24
  • More than +3% mAP gains on Market-1501 and DukeMTMC-reID are reported when quadruplet loss and hard triplet loss are combined with softmax.On CUHK03, multi-loss learning slightly improves these losses from already high re-ID accuracies.Source: §3.2 · p. 24

3.3 Further Comments on Multi-loss Training

  • HAP2S explicitly learns feature-space geometry, while softmax separates probability-space decision boundaries and can regularize against vanishing gradients.The paper attributes their interaction to back-propagating both branches' error differentials to the shared backbone.Source: §3.3 · p. 24, Figure 5 · p. 25
Loading 1807.11206v1…