Source-linked AI summary
Deep metric learning using Triplet network
Elad Hoffer, Nir Ailon
TL;DR
The paper addresses the limited explicit learning of representations when deep models are trained primarily for classification. It introduces a triplet network that learns representations through distance comparisons and reports useful embeddings across datasets, while noting that broader Siamese comparisons were inconclusive.
Problem
Deep representations and their induced metrics are often learned only implicitly through classification, motivating explicit representation learning.
Method
A triplet network uses a deep embedding and three shared-parameter network instances to compare a reference with positive and negative samples.
Results
Across various datasets, the learned representations were useful for classification and showed semantic clustering, while MNIST accuracy exceeded the Siamese implementation.
Takeaways & Limitations
Because training requires knowing which two of three images share a class rather than the class identity, the approach may support unsupervised learning.
Takeaways & Limitations
Meaningful Siamese-network comparison results were unavailable for three of the four datasets.
Abstract
from arXiv · showhide
Deep learning has proven itself as a successful set of models for learning useful semantic representations of data. These, however, are mostly implicitly learned as part of a classification task. In this paper we propose the triplet network model, which aims to learn useful representations by distance comparisons. A similar model was defined by Wang et al. (2014), tailor made for learning a ranking for image information retrieval. Here we demonstrate using various datasets that our model learns a better representation than that of its immediate competitor, the Siamese network. We also discuss future possible usage as a framework for unsupervised learning.
1 INTRODUCTION
The paper argues that deep representations and induced metrics should be learned explicitly rather than emerging only as side effects of classification. It proposes triplet-based comparison learning as an alternative to Siamese pairwise metric learning.
- Deep networks can learn useful data representations, but these representations and their induced metrics are often incidental to classification.
- Siamese networks explicitly learn metrics by using contrastive loss to bring similar pairs closer and separate dissimilar pairs.
- The proposed task uses triplets labeled by relative similarity, requiring S(x, x1) > S(x, x2) whenever r(x, x1) > r(x, x2).
- For the experiments, x+ shares x’s class and x− has a different class, while a deep network learns an L2 embedding F(x).
- The triplet network is presented as a strong competitor to the Siamese approach and is studied more comprehensively than an earlier image-retrieval application.
2 THE TRIPLET NETWORK
A triplet network uses three shared-parameter feedforward subnetworks to compare positive and negative samples against a reference. Training turns these distance outputs into a two-class comparison problem.
- A triplet network comprises three instances of the same feedforward network with shared parameters.
- For inputs x, x+, and x−, the model outputs the L2 distances from x’s embedding to the embeddings of x+ and x−.
- The two distances encode comparisons between each candidate and the reference x.
- 2.1 TRAINING: Training samples pair x and x+ from one class with x− from another, making the task a two-class decision about which candidate shares x’s class.
- 2.1 TRAINING: Shared parameters let backpropagation update the model using all three samples simultaneously.
3 TESTS AND RESULTS
The experiments evaluate Triplet network embeddings on four datasets and visualize their semantic structure, while comparing classification performance with Siamese representations.
- 3.1 DATASETS: The study used four datasets: Cifar10, MNIST, SVHN, and STL10.Cifar10, MNIST, and SVHN include 32x32 or 28x28 images, while STL10 uses 96x96 images and fewer training examples.
- 3.1 DATASETS: Training used uniformly sampled triplets with two same-class images and one different-class image, without augmentation or whitening.All datasets used global normalization to zero mean and unit variance.
- 3.2 THE EMBEDDING NET: The Cifar10 and SVHN embedding network used convolutional and max-pooling layers, ending in a 128-dimensional representation without a classification layer.STL10 used the same architecture with a stride change, while MNIST used a smaller network.
- 3.4 2D VISUALIZATION OF FEATURES: PCA projections showed significant semantic clustering in the learned Euclidean embeddings, and a subsequent linear classifier achieved high classification accuracy.The visualization examined whether the embedding captured meaningful properties of the images.
- 3.5 COMPARISON WITH PERFORMANCE OF THE SIAMESE NETWORK: On MNIST, TripletNet representations achieved higher accuracy than representations learned by the Siamese network.The Siamese comparison used the same embedding network with contrastive loss between pairs of samples.
- 3.5 COMPARISON WITH PERFORMANCE OF THE SIAMESE NETWORK: Comparable Siamese-network results could not be obtained for the other three datasets.The authors conjecture that this may relate to a context problem and leave its resolution to future work.
4 FUTURE WORK
The paper proposes extending Triplet networks beyond direct labels by using comparative information from spatial, temporal, and crowdsourced sources.
- 4 FUTURE WORK: Triplet networks may support unsupervised learning because they learn from comparisons between samples instead of direct data labels.The proposed future direction uses comparative relationships as the training signal.
- 4 FUTURE WORK: Spatial proximity between objects or patches could serve as a rough similarity oracle in unsupervised learning.Nearby regions of the same image are expected to be semantically similar.
- 4 FUTURE WORK: Temporal proximity could provide comparative information because consecutive video frames are expected to depict the same object more often than frames separated by ten minutes.The authors suggest Triplet networks may improve classification in an unsupervised environment.
- 4 FUTURE WORK: Crowdsourcing may be suitable because people tend to provide comparative labels more accurately, and similarity comparisons can be easier to collect than direct labels.Examples include images taken at the same location and shared annotations.
5 CONCLUSIONS
The paper concludes that Triplet networks explicitly learn useful representations from comparative relationships, with performance comparable to explicitly trained classification networks.
- 5 CONCLUSIONS: Triplet networks explicitly learn useful representations with a deep network.The model learns from comparative measures rather than requiring direct class labels.
- 5 CONCLUSIONS: Across various datasets, the learned representations support classification comparably to representations from a network trained explicitly for classification.The conclusion frames this as evidence that the learned embeddings are useful for classification.
- 5 CONCLUSIONS: The method requires knowing which two of three images share a class rather than knowing the class identity itself.This comparison-based supervision may support future learning from data with unclear or hierarchical labels.
- 5 CONCLUSIONS: The authors suggest that future embedding-network enhancements could benefit Triplet networks similarly to classification tasks.Examples include Network-in-Network and Inception models.