Source-linked AI summary
Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification
Qiqi Xiao, Hao Luo, Chi Zhang
TL;DR
Person re-identification is difficult because of substantial visual variation and relies increasingly on deep metric learning. This paper proposes MSML, which mines hard positive and negative samples, and reports superior performance to most state-of-the-art methods across four benchmark datasets.
Problem
Person re-identification must match people across images despite pose, viewpoint, illumination, background, occlusion, and similar appearances.
Method
MSML forms batches containing K images from each of P identities, computes an N × N distance matrix, and uses the hardest positive and negative pairs.
Results
MSML achieves the best accuracy on most evaluated datasets and outperforms most state-of-the-art methods on Market1501, MARS, CUHK03, and CUHK-SYSU.
Takeaways & Limitations
Hard sample mining filters easy samples and is reported to improve model robustness and produce a finer metric in feature-embedding space.
Takeaways & Limitations
A related loss suffers poor generalization and is difficult to apply to tracking tasks when evaluated with the same probe image.
Abstract
from arXiv · showhide
Person re-identification (ReID) is an important task in computer vision. Recently, deep learning with a metric learning loss has become a common framework for ReID. In this paper, we also propose a new metric learning loss with hard sample mining called margin smaple mining loss (MSML) which can achieve better accuracy compared with other metric learning losses, such as triplet loss. In experi- ments, our proposed methods outperforms most of the state-of-the-art algorithms on Market1501, MARS, CUHK03 and CUHK-SYSU.
1. Introduction
Person re-identification is a challenging computer-vision task affected by appearance and imaging variations. The paper proposes MSML, a hard-sample metric-learning loss that selects difficult positive and negative pairs and reports strong results across four datasets.
- ReID supports applications such as tracking people across cameras and searching large galleries, but pose, viewpoint, illumination, background, occlusion, and similar appearances make it difficult.
- CNN-based representation learning provides high-level features, while metric-learning losses address pairwise similarity between same- and different-person images.
- MSML minimizes positive-pair distances and maximizes negative-pair distances using a batch distance matrix with K images from each of P identities.
- The method selects the most dissimilar positive pair and most similar negative pair in each batch as the hardest pairs for training.
- MSML outperforms other metric-learning losses on person ReID and shows significant performance across Market1501, MARS, CUHK03, and CUHK-SYSU.
2. Related Work
Related work covers deep convolutional feature extractors, metric learning, hard sample mining, and approaches using unsupervised learning, additional modalities, video sequences, or re-ranking.
- 2.1. Deep convolutional networks: ResNet and its variants use residual or grouped convolutional building blocks to balance representation capability and computational cost.
- 2.2. Deep metric learning: Deep metric-learning methods transform images into embedding features and compute similarities or distances in Euclidean space.
- 2.2. Deep metric learning: Triplet and quadruplet losses structure positive and negative pairs, while hard mining selects dissimilar positive pairs and similar negative pairs for training.
- 2.3. Other proposed ReID methods: Unsupervised and transfer-learning methods address dataset bias and limited labeled data, though models trained on one dataset can perform poorly on another.
- 2.3. Other proposed ReID methods: Other ReID methods improve global or local features with pose alignment, language or GAN-generated inputs, sequence information, attention, and re-ranking.
3. Our Method
The method develops MSML, a metric-learning loss that mines extreme positive and negative pairs within each batch. It combines relative and absolute distance constraints while addressing the dominance of easy samples in conventional quadruplet-based training.
- Motivation: MSML addresses the rapid growth of quadruplets and the overwhelming number of easy samples, which can limit the influence of hard examples.The issue is especially pronounced for negative pairs, whose number is much larger than that of positive pairs.
- Benefits: Hard sample mining filters easy samples and improves model robustness, while MSML is reported to outperform other metric-learning losses on person ReID.The paper also describes MSML as easy to implement and combine with other methods.
- Margin Sample Mining Loss: MSML selects the most dissimilar positive pair and the most similar negative pair in the whole batch for hard-sample mining.The batch contains K images from each of P persons, and distances are computed in an N × N matrix with N = K × P.
- Margin Sample Mining Loss: MSML covers both relative distances sharing a probe identity and absolute distances involving different probe identities.The two edge-mining cases correspond to triplet-style relative distance and the second constraint of quadruplet loss.
- Margin Sample Mining Loss: The loss uses the upper bound of positive-pair distances and the lower bound of negative-pair distances within a batch.This pushes the bounds of the two pair sets apart rather than separating pairs independently around each sample.
4. Experiments
Experiments evaluate MSML across multiple ReID benchmarks, network backbones, metric-learning losses, and comparisons with representative methods. MSML achieves the best accuracy in most datasets and base-model settings, while its distance distributions provide finer feature-space separation than TriHard.
- Experimental setup: Experiments use CUHK03, CUHK-SYSU, Market1501, and MARS, with rank-1, rank-5, rank-10 accuracy, and mAP as evaluation metrics.The study also compares different network backbones and loss functions before benchmarking against representative ReID methods.
- Loss comparisons: The experiments compare classification loss alone with triplet, hard-mined triplet, quadruplet, and MSML losses combined with classification loss.Table 1 defines Cls, Tri, TriHard, Quad, and MSML as the compared methods.
- Loss comparisons: Approximately 10.0% gains in mAP and rank-1 accuracy over classification-only baselines are reported for triplet loss.TriHard and Quad also outperform triplet loss, with TriHard slightly better on Market1501, MARS, and CUHK-03, while Quad performs better on CUHK-SYSU.
- Loss comparisons: MSML achieves the best accuracy on most experimental datasets across the different base models.The comparison includes Resnet50, Inception-v2, and Resnet50-X as base networks.
- Results analysis: TriHard and MSML both obtain high accuracy, but negative-pair distances can remain below positive-pair distances with TriHard, whereas MSML produces a finer feature-space metric.Figure 3 visualizes distance distributions for positive and negative image pairs.
- Results analysis: Quad and TriHard do not consistently reach their best accuracy under the same setting and can perform worse than triplet loss with Inception-v2 and Resnet-X.The passage contrasts these results with MSML’s consistently best performance in the reported experiments.
5. Conclusion
The paper introduces MSML, a metric-learning loss that uses hard sample mining for person ReID and reports stronger performance than most existing methods across several benchmarks.
- MSML improves robustness by filtering easy samples during hard sample mining.The method is presented as a new metric-learning loss for person re-identification.
- MSML selects the most dissimilar positive pair and most similar negative pair to compute the final loss.It first calculates a distance matrix, then uses the hardest positive and negative pairs for training.
- The experiments compare MSML with different metric-learning losses using ResNet50, Inception-v2, and ResNet50-X base models.
- MSML outperforms most existing methods on CUHK03 and CUHK-SYSU.The cited tables report comparisons with existing methods on these two datasets.