Source-linked AI summary
Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
Hanjiang Lai, Yan Pan, Ye Liu, Shuicheng Yan
TL;DR
Existing image hashing may separate hand-crafted feature extraction from coding, leaving their compatibility limited. The paper proposes a one-stage deep supervised architecture that jointly learns image representations and binary codes, and reports superior retrieval performance across benchmark datasets against supervised and unsupervised methods.
Problem
Hand-crafted image features followed by separate projection and quantization may not be optimally compatible with binary coding.
Method
A one-stage deep architecture jointly learns image representations and hash codes using shared convolution layers, divide-and-encode modules, and triplet ranking loss.
Results
The proposed method reports substantially better search accuracies than baseline hashing methods and superior gains over CNNH and CNNH⋆ across three datasets.
Takeaways & Limitations
Simultaneously learning useful image representations and similarity-preserving hash codes can benefit each other in image retrieval.
Takeaways & Limitations
CNNH⋆ comparisons on CIFAR-10 are affected by different network configurations and optimization frameworks between implementations.
Abstract
from arXiv · showhide
Similarity-preserving hashing is a widely-used method for nearest neighbour search in large-scale image retrieval tasks. For most existing hashing methods, an image is first encoded as a vector of hand-engineering visual features, followed by another separate projection or quantization step that generates binary codes. However, such visual feature vectors may not be optimally compatible with the coding process, thus producing sub-optimal hashing codes. In this paper, we propose a deep architecture for supervised hashing, in which images are mapped into binary codes via carefully designed deep neural networks. The pipeline of the proposed deep architecture consists of three building blocks: 1) a sub-network with a stack of convolution layers to produce the effective intermediate image features; 2) a divide-and-encode module to divide the intermediate image features into multiple branches, each encoded into one hash bit; and 3) a triplet ranking loss designed to characterize that one image is more similar to the second image than to the third one. Extensive evaluations on several benchmark image datasets show that the proposed simultaneous feature learning and hash coding pipeline brings substantial improvements over other state-of-the-art supervised or unsupervised hashing methods.
1. Introduction
Learning-based hashing targets efficient nearest-neighbor search by encoding images into compact binary codes. The paper addresses incompatibility between hand-crafted visual features and separate coding steps with a one-stage deep architecture that jointly learns representations and hash codes.
- Learning-based hashing encodes input data points such as images into similarity-preserving binary codes for large-scale nearest-neighbor search.
- Existing image-hashing pipelines separately use hand-crafted visual descriptors, projection, and quantization, which may produce features poorly matched to coding.
- The proposed one-stage supervised method maps input images to binary codes through a deep architecture that jointly learns image representations and hash codes.
- Its pipeline combines shared convolution layers, divide-and-encode branches assigning one hash bit per branch, and a triplet ranking loss preserving relative similarities.
2. Related Work
Related hashing work is organized around unsupervised and supervised methods, with supervised approaches using side information to learn compact representations. Deep hashing removes hand-crafted features, but the paper positions its one-stage learning as an alternative to CNNH’s staged process.
- Learning-based hashing methods are categorized as unsupervised or supervised according to whether supervised information is used.
- Unsupervised methods learn hash functions from training data alone, including kernelized, semantic, graph-based, and iterative-quantization approaches.
- Supervised methods use labels, pairwise similarities, or relative similarities to learn compact bitwise representations.
- Most supervised image-hashing methods begin with hand-crafted visual features before projection and quantization generate hash codes.
- CNNH uses raw pixels but decomposes hash learning into approximate-code learning followed by a deep-network stage, whereas the proposed method learns representations and codes in one stage.
3. The Proposed Approach
The proposed approach jointly learns image representations and binary hash codes with a shared deep architecture trained on relative-similarity triplets. Convolutional features are divided into bit-specific branches, while a triplet ranking loss encourages the desired Hamming-distance ordering.
- Goal: Hash learning maps each image I into a q-bit binary code F(I) while preserving image similarities.
- Triplet ranking loss: Training uses triplets (I, I+, I−), where I should be closer to I+ than I− in the learned hash space.Relative similarities can be represented by triplets and incorporated into a triplet ranking objective.
- Shared sub-network: A shared convolutional sub-network encodes each triplet into intermediate feature vectors with a common representation.The shared design also reduces the number of parameters in the architecture.
- Shared sub-network: The shared sub-network uses stacked convolution and pooling layers, including 1 × 1 filters and average pooling instead of traditional fully-connected layers.
- Divide-and-encode module: The divide-and-encode module splits intermediate features into q slices, maps each slice to one dimension, and concatenates the outputs into a q-bit approximate code.Each branch corresponds to one hash bit, and the input dimension can be distributed across unequal slices when it is not divisible by q.
- Divide-and-encode module: Compared with a fully-connected alternative using the whole feature vector for every bit, separate feature slices are intended to reduce redundancy among hash bits.
- Binary outputs: A sigmoid followed by a piece-wise threshold function encourages approximate hash outputs toward binary values during training.Values below 0.5 − ϵ are truncated to 0, values above 0.5 + ϵ to 1, and intermediate values remain unchanged.
4. Experiments
Experiments evaluate the proposed hashing method on SVHN, CIFAR-10, and NUS-WIDE using multiple search-accuracy metrics and comparisons against supervised and unsupervised baselines. Results show gains over hand-crafted-feature methods, deep two-stage competitors, and architectural alternatives.
- Experimental Settings: Experiments use SVHN, CIFAR-10, and NUS-WIDE, comparing the proposed method with eight state-of-the-art supervised and unsupervised hashing methods.The evaluation includes LSH, SH, ITQ, CNNH, KSH, MLH, BRE, and ITQ-CCA.
- Experimental Settings: Search quality is measured by MAP, precision-recall curves, precision within Hamming distance 2, and precision across different numbers of returned samples.Methods use identical training and test sets for fair comparison.
- Results of Search Accuracies: On all three datasets, the proposed method substantially improves MAP, Hamming-radius precision, precision-recall, and top-return precision over baselines using traditional hand-crafted visual features.The comparisons cover multiple bit lengths and retrieval settings.
- Results of Search Accuracies: 9.6% ∼14.0% / 3.9% ∼9.2% relative MAP increases over the corresponding second-best competitor occur on CIFAR-10 / NUS-WIDE, respectively.The proposed method generally outperforms the related deep two-stage methods CNNH and CNNH⋆ across most metrics.
- Divide-and-Encode Module: 0.581 accuracy with 48 bits on CIFAR-10 represents a 19.7% improvement for the divide-and-encode module over the fully connected alternative.The alternative replaces divide-and-encode with a fully connected layer followed by sigmoid activation while keeping other layers unchanged.
- Shared versus Independent Sub-Networks: The shared-sub-network architecture outperforms an alternative with two independent sub-networks, possibly because the available training samples are insufficient for larger parameterized networks.The paper notes examples of 500 training images per class on CIFAR-10 and NUS-WIDE.
5. Conclusion
The paper develops a one-stage supervised hashing architecture that generates image hash codes through deep networks and triplet-based relative-similarity learning. Its shared convolutional representation and divide-and-encode modules support superior image-retrieval performance over state-of-the-art methods.
- The proposed one-stage supervised hashing method generates bitwise image codes through a carefully designed deep architecture.
- Shared stacked convolution layers convert input images into unified intermediate representations before hash-code encoding.
- Triplet ranking loss preserves relative similarities, while divide-and-encode modules transform intermediate representations into hash codes.
- Empirical image-retrieval evaluations report superior performance gains over state-of-the-art methods.