PaperlayerView exact revision on arXiv

Source-linked AI summary

Proxy Anchor Loss for Deep Metric Learning

Sungyeon Kim, Dongwon Kim, Minsu Cho, Suha Kwak

arXiv:2003.13911v1cs.CVcs.LG

TL;DR

Pair-based losses capture fine-grained data-to-data relations but are computationally expensive, whereas proxy-based losses converge quickly but discard those relations. The paper introduces Proxy-Anchor loss, which combines proxies with batch-wide data interactions. Across four public benchmarks, it achieves state-of-the-art performance and the fastest convergence without careful data sampling.

  • Problem

    Pair-based losses exploit rich data-to-data relations but have high training complexity, while proxy-based losses converge faster but cannot use those relations.

  • Method

    Proxy-Anchor loss uses each proxy as an anchor and associates it with all batch data, weighting gradients by relative hardness from data-to-data relations.

  • Results

    Across four public benchmark datasets, the model achieves state-of-the-art performance and converges most quickly without careful data sampling.

  • Takeaways & Limitations

    The loss combines fast, reliable proxy-based convergence with rich data-to-data relations during training.

Abstract

from arXiv · show

Existing metric learning losses can be categorized into two classes: pair-based and proxy-based losses. The former class can leverage fine-grained semantic relations between data points, but slows convergence in general due to its high training complexity. In contrast, the latter class enables fast and reliable convergence, but cannot consider the rich data-to-data relations. This paper presents a new proxy-based loss that takes advantages of both pair- and proxy-based methods and overcomes their limitations. Thanks to the use of proxies, our loss boosts the speed of convergence and is robust against noisy labels and outliers. At the same time, it allows embedding vectors of data to interact with each other in its gradients to exploit data-to-data relations. Our method is evaluated on four public benchmarks, where a standard network trained with our loss achieves state-of-the-art performance and most quickly converges.

1. Introduction

Pair-based losses capture fine-grained data-to-data relations but incur high complexity and slow convergence, while proxy-based losses converge faster yet lose those relations. Proxy-Anchor loss combines proxies with batch-wide data interactions, achieving high accuracy and faster convergence than baselines.

  • Results: On Cars-196, the proposed loss achieves the highest Recall@1 accuracy and converges faster than baselines in epochs and actual training time.All methods use batch size 150 on one Titan Xp GPU.
  • Motivation: Pair-based losses provide rich data-to-data supervision but can require O(M^2) or O(M^3) computation, slowing convergence.Tuple sampling can require manual tuning and may increase overfitting risk.
  • Motivation: Proxy-based losses reduce complexity because the number of proxies is substantially smaller than the training dataset.They also tend to be more robust against noisy labels and outliers.
  • Motivation: Proxy-based losses cannot exploit rich data-to-data relations because each data point is associated only with proxies.
  • Proposed method: Proxy-Anchor loss associates every proxy with all data in a batch, weighting gradients by relative hardness while pulling same-class data closer and pushing other data away.This design combines proxy-based efficiency with data-to-data interactions.

2. Related Work

Related work contrasts pair-based losses, which model fine-grained data relations at high computational cost, with proxy-based losses, which reduce complexity but discard those relations. Proxy-Anchor retains proxy efficiency while allowing batch-wide data interactions and relative-hardness weighting.

  • Pair-based Losses: Pair-based losses model higher-order data relations and hardness, but some methods sample only selected examples instead of using the entire batch.
  • Pair-based Losses: Pair-based training complexity grows polynomially with dataset size, making convergence slow and motivating tuple sampling with tunable hyperparameters.Sampling may increase overfitting risk because it relies largely on local batch relations.
  • Proxy-based Losses: Proxy-based methods represent subsets of training data with learned proxies and reduce complexity by relating each example to proxies instead of other examples.
  • Proxy-based Losses: Proxy-NCA assigns one proxy per class, while SoftTriple uses multiple class proxies and Manifold Proxy replaces Euclidean distance with a manifold-aware distance.
  • Proxy-based Losses: Proxy methods improve convergence but cannot access rich data-to-data relations when examples are associated only with proxies.
  • Proxy-based Losses: Proxy-Anchor lets embedding vectors interact during training because its gradients reflect the relative hardness of data.

3. Our Method

Proxy-Anchor loss uses proxies as anchors while associating them with all batch embeddings, preserving proxy-based efficiency while enabling data-to-data interactions through gradients. Its training complexity remains O(MC), matching Proxy-NCA and lower than pair-based alternatives.

  • 3.1. Review of Proxy-NCA Loss: Proxy-NCA pulls every positive with constant power and pushes harder negatives more strongly, but cannot exploit fine-grained data-to-data relations.Its data-to-proxy-only associations limit the capability of the trained embedding network.
  • 3.1. Review of Proxy-NCA Loss: Proxy-based losses converge faster and are more robust to outliers and noisy labels because proxies represent groups of training data.Their proxy count is generally substantially smaller than the number of training data.
  • 3.2. Proxy-Anchor Loss: Proxy-Anchor uses each proxy as an anchor and associates it with all data in a batch, allowing embeddings to interact through the proxy during training.The loss assigns one proxy per class and separates each batch into positive and negative embeddings for each proxy.
  • 3.2. Proxy-Anchor Loss: Relative hardness changes gradient strength: harder positives are pulled more strongly, while harder negatives are pushed more strongly.Gradients depend on other embeddings in the batch, so the loss reflects relative hardness and richer data-to-data relations.
  • 3.3. Training Complexity Analysis: O(MC) is the training complexity of Proxy-NCA and Proxy-Anchor, compared with O(M 2) or O(M 3) for pair-based losses when C ≪M.Proxy-Anchor compares every proxy with all positive or negative examples in a batch, while pair-based losses compare tuples of data.

4. Experiments

Experiments evaluate Proxy-Anchor on four benchmarks, showing state-of-the-art retrieval performance, robust qualitative retrieval, and sensitivity to batch size, embedding dimension, and hyperparameters.

  • Comparison to Other Methods: Proxy-Anchor is evaluated on four benchmark datasets using three embedding and image-size settings.The settings are 64/128 dimensions at 224×224, 512 dimensions at 224×224, and 512 dimensions at 256×256.
  • Comparison to Other Methods: 2.7% Recall@1 improvement over the previous best is reported on CUB-200-2011, while Proxy-Anchor outperforms prior methods across the three settings.The comparison includes ensemble methods.
  • Comparison to Other Methods: Proxy-Anchor achieves state-of-the-art performance on SOP and the best scores in all three settings on In-Shop.On SOP, it is second best only for Recall@10 and Recall@100 with 64-dimensional embeddings.
  • Comparison to Other Methods: Larger crops with 512-dimensional embeddings achieve state-of-the-art performance on all datasets, while lower dimensions often outperform existing higher-dimensional models.The results suggest a compact yet effective embedding space, alongside greatly improved convergence speed.
  • Qualitative Results: Retrieval remains robust despite pose variation, background clutter, distinct object colors, and viewpoint changes across the benchmarks.Figure 4 presents top-4 retrievals for each query; red boundaries mark failures that remain visually similar to the query.
  • Impact of Batch Size: Batch sizes of 300 or larger produce the best performance, consistent with the loss using more within-batch examples and relations.Smaller batches slightly reduce performance because fewer data-point relations are available.
  • Impact of Embedding Dimension: Performance is stable for embedding dimensions of at least 128, and Proxy-Anchor outperforms MS loss at every tested dimension without high-dimensional degradation.Dimensions from 32 to 1,024 are examined on Cars-196.
  • Impact of Hyperparameters: On Cars-196, accuracy is high and stable for α greater than 16, while increasing δ improves performance with a relatively small effect at large α.The reported hyperparameters were not tuned to optimize test accuracy, yet still outperform existing methods.

5. Conclusion

The paper introduces Proxy-Anchor loss to combine proxy-based efficiency with pair-based data-to-data relations. Across four benchmarks, it reports state-of-the-art performance and fastest convergence without careful data sampling.

  • Conclusion: Proxy-Anchor loss combines fast, reliable convergence from proxies with rich data-to-data relations from pair-based losses.Its gradients allow embedding vectors in a batch to affect one another.
  • Conclusion: The model achieves state-of-the-art performance on four public benchmark datasets and converges most quickly without careful data sampling.The paper proposes exploring extensions to deep hashing networks for more efficient testing and training.

A. Appendix

The appendix adds experiments on backbone networks, input-image sizes, t-SNE visualizations, and qualitative retrieval results.

  • Appendix: Additional experiments analyze backbone networks and input-image sizes, while t-SNE visualizations and more qualitative retrieval results are provided for four benchmarks.These results were omitted from the main paper because of space limits.

A.1. Impact of Backbone Network & Image Size

The appendix studies backbone architectures and larger input images for Proxy-Anchor. Increasing image size consistently improves retrieval accuracy and can matter more than using a more powerful architecture.

  • Impact of Backbone Network: Backbone comparisons use CUB-200-2011 and Cars-196 with 512-dimensional embeddings and fixed α and δ settings.The evaluated backbones include Inception-BN, GoogleNet, ResNet-50, and ResNet-101.
  • Impact of Image Size: 8.9% Recall@1 improvement is achieved with 448×448 images over the default 224×224 size.Accuracy improves consistently as input-image size increases.
  • Impact of Image Size: With enough GPU memory, larger images are reported as more effective for improving performance than a more powerful architecture.Larger images reduce the allowable batch size.

A.2. Additional Qualitative Results

Across four benchmarks, Proxy-Anchor produces more accurate qualitative retrievals than Proxy-NCA, including difficult viewpoint, appearance, and fine-grained similarity cases. t-SNE visualizations further show relevant nearest neighbors across the test splits.

  • Proxy-Anchor retrieves more accurate matches than Proxy-NCA across CUB-200-2011, Cars-196, SOP, and In-shop examples.The comparison uses the same backbone network; examples include improved bird, car, object, and clothing retrievals.
  • On CUB-200-2011, Proxy-Anchor correctly retrieves birds in examples where both models find visually similar birds but Proxy-NCA fails.
  • On Cars-196, Proxy-Anchor succeeds despite viewpoint changes and color differences in the retrieved vehicles.
  • On SOP, Proxy-Anchor retrieves the same object despite extreme viewpoint changes, while on In-shop it avoids a short-dress confusion made by the baseline.
  • t-SNE visualizations of all four test splits show relevant nearest neighbors, suggesting semantic similarity generalizes to unseen test data.The visualizations cover CUB-200-2011, Cars-196, SOP, and In-shop.
Loading 2003.13911v1…