Source-linked AI summary
Deep Supervised Hashing with Triplet Labels
Xiaofang Wang, Yi Shi, Kris M. Kitani
TL;DR
Traditional hashing separates visual-feature extraction from hash-code learning, while off-the-shelf features may be poorly compatible with the latter. This paper proposes end-to-end deep hashing that maximizes triplet-label likelihood, and reports outperforming all baselines on CIFAR-10 and NUS-WIDE, including DPSH and prior triplet-based methods.
Problem
Off-the-shelf visual features may be incompatible with hash-code learning, producing sub-optimal codes; existing deep hashing uses pairwise or triplet supervision.
Method
The proposed method simultaneously learns image features and hash codes by maximizing the likelihood of given triplet labels.
Results
The method outperforms all baselines on CIFAR-10 and NUS-WIDE, including DPSH and previous triplet-label deep hashing methods.
Takeaways & Limitations
Triplet-label supervision provides the basis for end-to-end deep hashing that targets query-positive proximity and query-negative separation in the learned code space.
Takeaways & Limitations
The triplet-label likelihood uses α as a positive hyper-parameter margin.
Abstract
from arXiv · showhide
Hashing is one of the most popular and powerful approximate nearest neighbor search techniques for large-scale image retrieval. Most traditional hashing methods first represent images as off-the-shelf visual features and then produce hashing codes in a separate stage. However, off-the-shelf visual features may not be optimally compatible with the hash code learning procedure, which may result in sub-optimal hash codes. Recently, deep hashing methods have been proposed to simultaneously learn image features and hash codes using deep neural networks and have shown superior performance over traditional hashing methods. Most deep hashing methods are given supervised information in the form of pairwise labels or triplet labels. The current state-of-the-art deep hashing method DPSH~\cite{li2015feature}, which is based on pairwise labels, performs image feature learning and hash code learning simultaneously by maximizing the likelihood of pairwise similarities. Inspired by DPSH~\cite{li2015feature}, we propose a triplet label based deep hashing method which aims to maximize the likelihood of the given triplet labels. Experimental results show that our method outperforms all the baselines on CIFAR-10 and NUS-WIDE datasets, including the state-of-the-art method DPSH~\cite{li2015feature} and all the previous triplet label based deep hashing methods.
1 Introduction
The introduction motivates end-to-end supervised deep hashing because separately learned off-the-shelf features can produce sub-optimal codes, then proposes maximizing triplet-label likelihood and reports superior benchmark performance.
- Hashing maps high-dimensional image features into compact binary codes whose Hamming distances approximate feature-space Euclidean distances.This supports efficient approximate nearest neighbor search through computational and storage savings.
- Supervised hashing can achieve competitive performance with fewer bits than unsupervised methods by using labeled information.Fewer bits benefit search speed and storage efficiency.
- Off-the-shelf visual features may be incompatible with hash-code learning, causing image similarities to be imperfectly preserved and codes to become sub-optimal.Deep hashing addresses this by learning image features and hash codes simultaneously.
- DPSH is presented as the state-of-the-art pairwise-supervised deep hashing method, maximizing the likelihood of pairwise similarities modeled through Hamming distance.The introduction uses DPSH as the pairwise-label reference point for the proposed method.
- Triplet labels encode relative similarity by requiring a query to be close to a positive image and far from a negative image simultaneously.Pairwise labels impose only one such constraint at a time, while triplets can be decomposed into two pairwise labels but are not generally reconstructed from arbitrary pairs.
- The proposed method jointly learns features and hash codes end-to-end by maximizing triplet-label likelihood, and it outperforms baselines on CIFAR-10 and NUS-WIDE.Its stated components are image feature learning, hash-code learning, and a loss function evaluating how well triplet labels are satisfied.
2 Related Work
The related-work section organizes hashing by dependence on data and supervision, identifies limitations of prior feature-learning and triplet-based methods, and positions the proposed likelihood formulation as the new contribution.
- Hashing methods are categorized as data-independent or data-dependent, with data-dependent methods further divided into unsupervised and supervised approaches.Data-independent methods generally require longer codes for satisfactory performance.
- Unsupervised hashing learns hash functions from training data without supervised information, including classical and deep methods such as Spectral Hashing and Semantic Hashing.The passage lists representative examples rather than comparing their performance.
- Supervised hashing uses labeled data through point-wise, pairwise, or ranking labels; representative methods include CCA-ITQ, KSH, LFH, and related approaches.The taxonomy distinguishes methods by the form of supervision provided.
- Prior methods often learn hash codes independently of off-the-shelf features, which may be incompatible with the code-learning procedure and yield sub-optimal codes.This limitation motivates deep methods that jointly learn image features and hash codes.
- CNNH introduced deep hashing with triplet labels without off-the-shelf features but could not learn image features and hash codes simultaneously.NINH, DSRH, DRSCH, and DSCH addressed simultaneous learning, while the proposed work introduces a new triplet-label likelihood formulation.
- DPSH jointly learns features and codes with pairwise labels and is reported to achieve the highest performance among other deep hashing methods.The proposed method instead uses triplet labels because they are described as containing richer information than pairwise labels.
3 Approach
The approach jointly learns image features and hash codes from triplet labels in an end-to-end Siamese-triplet network. It maximizes triplet-label likelihood while incorporating quantization error into a differentiable training objective.
- 3.1 Problem Definition: Triplet labels specify that each query should be closer to its positive image than its negative image in Hamming space.The method seeks hash codes satisfying these relative-similarity constraints as much as possible.
- 3.2 Learning the Hash Function: The model combines a CNN-based image-feature learner, a fully connected hash-code learner, and a triplet-label likelihood loss.These components are integrated into a Siamese-triplet network with shared-weight sub-networks that take image triplets as input.
- 3.2 Learning the Hash Function: The triplet-label likelihood uses pairwise hash-code inner products and a positive margin α to encourage the query-positive distance to be smaller than the query-negative distance.The margin regularizes the distance gap and can help speed training by preventing derivatives from becoming too small.
- 3.2 Learning the Hash Function: Because discrete optimization is intractable, binary codes are relaxed to real-valued vectors and a quantization-error term is added to the loss.The hyper-parameter λ balances negative log triplet likelihood against quantization error, with binary codes recovered using the sign function.
- 3.2 Learning the Hash Function: The resulting loss is differentiable with respect to the network outputs, enabling minimization by back-propagation and hash-code generation for new images after training.For a new image, the trained sub-network output is converted to a binary code with the sign function.
4 Experiment
Experiments on CIFAR-10 and NUS-WIDE evaluate the proposed method against traditional and deep hashing baselines using MAP. The method outperforms competing approaches across the reported settings, while ablations examine margin, loss weighting, and training-set size.
- Datasets and Evaluation Protocol: Experiments use CIFAR-10 and NUS-WIDE, with MAP as the evaluation metric and dataset-specific similarity definitions.CIFAR-10 similarity is category membership; NUS-WIDE similarity requires at least one shared label.
- Baselines: The baselines include traditional unsupervised and supervised hashing, CNN-F feature variants, pairwise deep hashing, and triplet-label deep hashing methods.Hand-crafted-feature experiments use GIST for CIFAR-10 and concatenated visual descriptors for NUS-WIDE.
- Comparison to Traditional Hashing Methods: The proposed method significantly outperforms previous hashing methods using hand-crafted features on both datasets.The reported comparisons use the first experimental setting and the same evaluation protocol as the cited baseline results.
- Comparison to Deep Hashing Methods: About 2%: the method outperforms DPSH on both CIFAR-10 and NUS-WIDE in the first experimental setting.On NUS-WIDE, the comparison uses DPSH*, obtained by rerunning the authors' code because image links produce dataset differences.
- Comparison to Deep Hashing Methods: More than 10% on CIFAR-10 and about 5% on NUS-WIDE: the method outperforms DPSH under the second experimental setting.After rerunning DPSH on CIFAR-10 under the same setting, the proposed method still outperforms DPSH* by about 1%.
- Ablation Studies: Very small or very large λ values reduce MAP, while larger margins improve performance within the same number of training epochs and more training images improve results.λ balances negative log triplet likelihood against quantization error; the training-image study uses 12-bit codes on CIFAR-10.
5 Conclusion
The paper proposes a triplet-label deep hashing method that jointly learns image features and hash codes, and reports outperforming all baselines in benchmark experiments.
- The proposed method simultaneously learns image features and hash codes under triplet-label supervision.
- Experiments on standard benchmark datasets show that the method outperforms DPSH and previous triplet-label deep hashing methods.
- Retrieval examples compare top images produced by the proposed method against those retrieved using DPSH hash codes.
- The ablation studies examine how model performance changes across selected design and training-data settings.