Source-linked AI summary
Deep Multi-View Enhancement Hashing for Image Retrieval
Chenggang Yan, Biao Gong, Yuxuan Wei, Yue Gao
TL;DR
Binary hashing enables fast large-scale image retrieval but can reduce accuracy, while single-view methods overlook complementary multi-view information. D-MVE-Hash introduces supervised deep multi-view enhancement hashing with learned view relations, Hamming-space fusion, and a memory network, and reports significant improvements over state-of-the-art single-view and multi-view hashing methods.
Problem
Binary hashing supports efficient large-scale retrieval but can reduce accuracy, and single-view hashing does not exploit complementary information across multiple visual views.
Method
D-MVE-Hash jointly learns multi-view hash codes and a robust view-relation matrix, fuses them with binary codes in Hamming space, and uses a memory network during retrieval.
Results
D-MVE-Hash significantly outperforms state-of-the-art single-view and multi-view hashing methods across evaluations on CIFAR-10, NUS-WIDE, and MS-COCO.
Takeaways & Limitations
The framework provides an end-to-end approach for exploiting multi-view information in efficient supervised image retrieval while retaining binary-code retrieval advantages.
Abstract
from arXiv · showhide
Hashing is an efficient method for nearest neighbor search in large-scale data space by embedding high-dimensional feature descriptors into a similarity preserving Hamming space with a low dimension. However, large-scale high-speed retrieval through binary code has a certain degree of reduction in retrieval accuracy compared to traditional retrieval methods. We have noticed that multi-view methods can well preserve the diverse characteristics of data. Therefore, we try to introduce the multi-view deep neural network into the hash learning field, and design an efficient and innovative retrieval model, which has achieved a significant improvement in retrieval performance. In this paper, we propose a supervised multi-view hash model which can enhance the multi-view information through neural networks. This is a completely new hash learning method that combines multi-view and deep learning methods. The proposed method utilizes an effective view stability evaluation method to actively explore the relationship among views, which will affect the optimization direction of the entire network. We have also designed a variety of multi-data fusion methods in the Hamming space to preserve the advantages of both convolution and multi-view. In order to avoid excessive computing resources on the enhancement procedure during retrieval, we set up a separate structure called memory network which participates in training together. The proposed method is systematically evaluated on the CIFAR-10, NUS-WIDE and MS-COCO datasets, and the results show that our method significantly outperforms the state-of-the-art single-view and multi-view hashing methods.
1 INTRODUCTION
D-MVE-Hash addresses the limitations of single-view hashing by learning and enhancing relationships among multiple visual feature views within a supervised deep hashing framework. It combines view-relation evaluation, flexible Hamming-space fusion, and memory-based retrieval, with experiments reporting superiority over state-of-the-art hash methods.
- Motivation: Single-view convolutional hashing improves retrieval but does not exploit the complementary information available across multiple feature views.
- Method: The memory network learns to reproduce the view-relation matrix during training, allowing retrieval to use the learned matrix without repeatedly performing stability evaluation.
- Motivation: D-MVE-Hash defines multi-view data as multiple visual representations in different feature spaces and directly models their relationships in the original spaces.
- Method: The model computes a view-relation matrix from continuously optimized multi-view features, using feature fluctuations to regulate view relevance and support view-independent, robust relations.
- Method: Its end-to-end framework fuses arbitrary view combinations with binary codes through multiple Hamming-space enhancement methods while automatically learning hash functions by iterative training.
- Evaluation: Extensive experiments on three datasets and detailed ablations report that D-MVE-Hash outperforms several state-of-the-art hashing methods in image retrieval.
2 RELATED WORK
Related work develops efficient supervised and deep hashing methods, while multi-view hashing seeks to preserve complementary information across visual feature spaces. Existing approaches use spectral, graph, kernel, or deep-learning techniques, often relying on feature extraction and predefined or optimized view weights.
- Image Retrieval: Content-based image retrieval searches for visually similar or same-category images and commonly compresses multimedia data for similarity search.
- Hashing: Supervised hashing uses labeled datasets and generally produces more compact binary representations with better retrieval performance than unsupervised hashing methods.
- Deep Hashing: Deep learning and convolutional neural networks have improved image retrieval, motivating their integration with hashing to combine learned representations and efficient binary search.
- Multi-view Hashing: Multi-view hashing preserves data structure across multiple global feature spaces using spectral, graph, kernel, and deep-learning techniques.
- Multi-view Hashing: Prior multi-view methods often extract features through traditional processing and assign view weights that may be preset or optimized as parameters.
3 DEEP MULTI-VIEW ENHANCEMENT HASHING
D-MVE-Hash jointly learns multi-view binary codes and view-relation information, then enhances traditional global features through Hamming-space fusion. A memory network replaces stability evaluation during testing to reduce retrieval computation while preserving the hashing pipeline.
- Problem Definition and MV-Hash: D-MVE-Hash maps multiple visual views into a shared binary representation using view-specific feature networks and an integrated hash function.The formulation represents each object with q-bit codes and combines view outputs through a view-relation-dependent mapping.
- View Stability Evaluation: Each view network is pretrained for classification before view stability evaluation constructs the view-relation matrix.The view outputs are organized as Q, normalized, and used to derive the relation representation E.
- Joint Learning: The pairwise hash objective labels similar images with y = 1 and dissimilar images with y = −1 while relaxing binary outputs to [−1, 1]^q during training.The objective uses an L1-based constraint with a margin threshold and a regularizer intended to avoid gradient vanishing.
- Joint Learning: View-relation information directly enters the global loss, so it affects the gradient-descent optimization direction during network training.The network is trained by back-propagation with mini-batch gradient descent.
- Enhancement and Fusion: The enhancement stage first applies shared Hamming-space hash constraints, then integrates multi-view codes, view relations, and global features using replication, view-code, or probability view pooling fusion.The constraints distinguish clustering and hash-dispersion strengths, while the fusion methods provide alternative ways to combine the representations.
- View-code Fusion: View-code fusion fills dynamically sized code segments with per-view random codes and fully uses the view-relation matrix, but its performance is limited by view stability evaluation.The method is contrasted with replication fusion, which repeats view codes according to manually controlled fusion parameters.
- Joint Learning: A memory network learns the view-relation matrix during training and supplies it during testing instead of recomputing view stability evaluation.Because retrieval uses preconverted hash codes, model complexity and retrieval-time complexity remain separate; Hamming ranking requires O(1) time per query.
4 EXPERIMENTS
Experiments evaluate D-MVE-Hash across three datasets, ablations, code lengths, and comparisons with supervised hashing baselines. The results report stronger retrieval performance, robustness to partial views, and more relevant retrieved images, while requiring additional training time.
- Experimental Setup: Experiments use CIFAR-10, NUS-WIDE, and MS-COCO to compare D-MVE-Hash with supervised single-view and multi-view hashing methods.The compared methods include KSH, MvDH, CMH, CNNH, HashNet, DCH, AGAH, and MV-Hash.
- Ablation Experiment for View-relation Matrix E: With complete multi-view information and view-relation calculation, D-MVE-Hash gains 8.13% over the baseline.Training takes about 1.711 hours, which is 0.757 hours slower than the baseline.
- Comparisons and Retrieval Performance: As code length increases, unenhanced codes cover neighborhoods less effectively, whereas D-MVE-Hash clusters similar objects within smaller Hamming radii.At shorter lengths, the fixed basic-code component can cause multi-view-code misclassification and allow unenhanced codes to outperform enhanced codes.
- Comparisons and Retrieval Performance: The model converges to a stable optimum after about 50,000 iterations, and visualization shows more relevant retrieved results.
5 CONCLUSION
D-MVE-Hash exploits multi-view information and view relations for hash-code generation through MV-Hash, fusion methods, and a memory network. Experiments and visualizations demonstrate effectiveness for image retrieval.
- Conclusion: D-MVE-Hash uses MV-Hash to calculate view relations from object stability across views and merges view-relation matrices with binary codes through three enhancement methods.
- Conclusion: A memory network avoids excessive retrieval-time computation for view-stability evaluation, while experiments and visualizations demonstrate D-MVE-Hash and MV-Hash effectiveness.