Source-linked AI summary
Feature Learning based Deep Supervised Hashing with Pairwise Labels
Wu-Jun Li, Sheng Wang, Wang-Cheng Kang
TL;DR
Existing hashing methods often rely on hand-crafted features, and deep supervised methods commonly use triplet rather than pairwise labels. DPSH introduces an end-to-end approach that jointly learns image features and hash codes from pairwise labels, outperforming other methods with state-of-the-art image-retrieval performance.
Problem
Existing hashing methods may use hand-crafted features incompatible with hash learning, while pairwise-label applications lack methods for simultaneous feature and hash-code learning.
Method
DPSH is an end-to-end deep hashing framework integrating image representation learning, hash-function learning, and a pairwise-label-guided loss.
Results
DPSH outperforms other methods and achieves state-of-the-art performance in image retrieval applications.
Takeaways & Limitations
DPSH provides simultaneous feature learning and hash-code learning for supervised hashing with pairwise labels.
Takeaways & Limitations
The discrete optimization problem is hard to solve, and directly relaxing binary codes to continuous values may not achieve satisfactory performance.
Abstract
from arXiv · showhide
Recent years have witnessed wide application of hashing for large-scale image retrieval. However, most existing hashing methods are based on hand-crafted features which might not be optimally compatible with the hashing procedure. Recently, deep hashing methods have been proposed to perform simultaneous feature learning and hash-code learning with deep neural networks, which have shown better performance than traditional hashing methods with hand-crafted features. Most of these deep hashing methods are supervised whose supervised information is given with triplet labels. For another common application scenario with pairwise labels, there have not existed methods for simultaneous feature learning and hash-code learning. In this paper, we propose a novel deep hashing method, called deep pairwise-supervised hashing(DPSH), to perform simultaneous feature learning and hash-code learning for applications with pairwise labels. Experiments on real datasets show that our DPSH method can outperform other methods to achieve the state-of-the-art performance in image retrieval applications.
1 Introduction
Hashing supports efficient large-scale image retrieval, but existing approaches often separate feature construction from hash learning or rely on triplet labels. DPSH addresses pairwise-label applications with an end-to-end framework that jointly learns representations and hash codes, achieving state-of-the-art image-retrieval performance.
- Hashing is widely used for approximate nearest-neighbor image retrieval because it offers fast query speed and low memory cost.
- Hand-crafted features may be incompatible with hash-function learning, motivating deep methods that jointly learn features and hash codes.
- Most supervised deep hashing methods use triplet labels, while pairwise-label applications have few feature-learning methods.
- DPSH integrates a deep image-representation network, a hash function, and a pairwise-label-guided loss into one end-to-end architecture.The integrated components can provide feedback to one another during learning.
- DPSH is presented as the first method to simultaneously learn features and hash codes for applications with pairwise labels.
- DPSH outperforms unsupervised methods, supervised hand-crafted-feature methods, and deep feature-learning baselines in image retrieval, achieving state-of-the-art performance.
2 Notation and Problem Definition
The problem setting represents images with feature vectors and pairwise similarity labels, then learns binary codes whose Hamming distances preserve those relationships.
- 2.1 Notation: The notation uses bold lowercase letters for vectors, bold uppercase letters for matrices, ZT for transpose, ∥·∥2 for Euclidean norm, and sgn(·) for the elementwise sign function.
- 2.2 Problem Definition: Each image xi is represented by a feature vector that may be hand-crafted features or raw image pixels.
- 2.2 Problem Definition: Pairwise labels sij indicate whether image pairs are similar, with sij = 1 for similar pairs and sij = 0 for dissimilar pairs.
- 2.2 Problem Definition: The goal is to learn c-bit binary codes whose Hamming distances are low for similar pairs and high for dissimilar pairs.
3 Model and Learning
DPSH integrates CNN-based feature learning, hash-code learning, and pairwise-label supervision in one end-to-end framework. Its optimization alternates between discrete hash-code updates and backpropagation-based updates of the network and hash-function parameters, while supporting out-of-sample prediction.
- Model: DPSH addresses CNNH’s two-stage limitation by performing simultaneous feature learning and hash-code learning in an end-to-end framework.The model integrates feature learning and hash-code learning rather than training them in separate stages.
- Feature Learning Part: The architecture uses a seven-layer CNN feature-learning component, with five convolutional layers and two fully connected layers.The paper uses CNN-F for illustration; other CNN architectures can substitute for it, but are left for future study.
- Objective Function: Pairwise-label supervision is modeled through a likelihood objective that encourages small Hamming distances for similar points and large distances for dissimilar points.The binary codes satisfy bi ∈ {−1, 1}^c, and the optimization objective is derived from the negative log-likelihood of observed pairwise labels.
- DPSH Model: A fully connected layer links the learned representation to the hashing objective through weight matrix W and bias vector v.The feature-learning and objective-function parts are connected within the same framework, allowing the integrated model to learn from image pixels to pairwise labels.
- Learning Algorithm: DPSH uses alternating optimization: binary codes are updated with a sign operation, while W, v, and θ are learned by backpropagation.Training uses minibatches and summarizes the complete procedure in Algorithm 1.
- Out-of-Sample Extension: After training, DPSH predicts hash codes for unseen points through forward propagation, providing an out-of-sample extension.The learned deep hashing framework directly computes a query point’s hash code without requiring it to be in the training set.
4 Experiment
The experiments evaluate DPSH against diverse hashing baselines on CIFAR-10 and NUS-WIDE using MAP, with pairwise labels constructed from shared class labels. Across the reported comparisons, DPSH outperforms competing methods, including models using hand-crafted or fixed deep features and methods with triplet labels.
- Experimental setup: DPSH is evaluated on CIFAR-10 and NUS-WIDE against unsupervised, supervised, deep-feature, pairwise-label, and triplet-label hashing baselines.The comparisons include hand-crafted-feature methods, CNNH, NINH, and other deep hashing methods.
- Accuracy: DPSH dramatically outperforms unsupervised methods, supervised hand-crafted-feature methods, and deep hashing methods with feature learning under the Table 2 setting.The authors state that the comparison uses the same experimental setting and evaluation metric as the cited CNNH and NINH work.
- Accuracy: Simultaneous feature learning and hash-code learning improves performance: DPSH outperforms both CNNH, which lacks simultaneous learning, and DPSH0, which keeps CNN-F feature layers fixed.DPSH0 receives no feedback from hash-function learning to its feature-extraction layers.
- Comparison to non-deep baselines with deep features: DPSH outperforms non-deep hashing baselines that use deep features extracted by an ImageNet-pretrained CNN-F network.The comparison includes methods denoted by combinations such as FastH+CNN, where the non-deep hashing method operates on CNN-F features.
- Comparison to baselines with ranking labels: Under a separate evaluation setting, DPSH with pairwise labels dramatically outperforms baselines with triplet labels, including methods that also learn features and hash codes end-to-end.The authors caution that DPSH values in Table 4 differ from Table 2 because the experimental settings differ.
5 Conclusion
The paper proposes DPSH for pairwise-label settings and reports simultaneous feature and hash-code learning with state-of-the-art image-retrieval performance. It also examines sensitivity to the hyper-parameter η.
- DPSH is proposed as a deep hashing method for applications with pairwise labels.
- Figure 2 examines DPSH sensitivity to the hyper-parameter η.
- DPSH performs simultaneous feature learning and hash-code learning for pairwise-label applications.
- Experiments on real datasets show that DPSH achieves state-of-the-art performance in image retrieval applications.