PaperlayerView exact revision on arXiv

Source-linked AI summary

Discrete Latent Factor Model for Cross-Modal Hashing

Qing-Yuan Jiang, Wu-Jun Li

arXiv:1707.08322v1cs.IR

TL;DR

Cross-modal hashing must balance retrieval accuracy against training cost, because continuous relaxation methods are faster while discrete methods are typically more accurate. DLFH directly learns binary codes with a discrete latent factor model, and experiments report better accuracy with training speed comparable to continuous methods and faster than existing discrete methods.

  • Problem

    Existing CMH methods face a trade-off between faster training with continuous relaxation and typically better accuracy but slower training with discrete learning.

  • Method

    DLFH is a supervised discrete latent factor model for CMH that directly learns binary hash codes without continuous relaxation.

  • Results

    Experiments report that DLFH significantly outperforms relaxation-based continuous methods in accuracy with comparable training speed, and existing discrete methods in both accuracy and training speed.

  • Takeaways & Limitations

    DLFH combines discrete code learning with efficient training for cross-modal similarity search on large-scale datasets.

  • Takeaways & Limitations

    The paper demonstrates linear out-of-sample hash functions; stronger functions such as deep neural networks are left for future study.

Abstract

from arXiv · show

Due to its storage and retrieval efficiency, cross-modal hashing~(CMH) has been widely used for cross-modal similarity search in multimedia applications. According to the training strategy, existing CMH methods can be mainly divided into two categories: relaxation-based continuous methods and discrete methods. In general, the training of relaxation-based continuous methods is faster than discrete methods, but the accuracy of relaxation-based continuous methods is not satisfactory. On the contrary, the accuracy of discrete methods is typically better than relaxation-based continuous methods, but the training of discrete methods is time-consuming. In this paper, we propose a novel CMH method, called discrete latent factor model based cross-modal hashing~(DLFH), for cross modal similarity search. DLFH is a discrete method which can directly learn the binary hash codes for CMH. At the same time, the training of DLFH is efficient. Experiments on real datasets show that DLFH can achieve significantly better accuracy than existing methods, and the training time of DLFH is comparable to that of relaxation-based continuous methods which are much faster than existing discrete methods.

1 Introduction

Cross-modal hashing supports efficient similarity search when a query supplies one modality and the database contains another. DLFH directly learns binary codes while targeting the accuracy–training-speed trade-off between continuous and discrete CMH methods.

  • 1 Introduction: Hashing reduces storage costs and supports efficient retrieval by representing data points as binary codes in Hamming space.The codes aim to preserve similarity from the original space.
  • 1 Introduction: Cross-modal hashing enables searches in which only one modality is available for a query, unlike multi-source hashing.This makes CMH applicable when all modalities are not observed in real applications.
  • 1 Introduction: Existing CMH methods trade training speed against accuracy: continuous relaxation methods train faster, whereas discrete methods are typically more accurate but time-consuming.DLFH is proposed to address this trade-off.
  • 1 Introduction: DLFH directly learns binary hash codes for cross-modal similarity search without continuous relaxation.It is a supervised CMH method based on a discrete latent factor model.
  • 1 Introduction: DLFH experiments on real datasets report significantly better accuracy than existing methods, with training speed comparable to continuous methods and faster than existing discrete methods.The comparison includes both relaxation-based continuous and discrete baselines.

2 Notations and Problem Definition

The problem setup uses paired feature representations from two modalities and supervised cross-modal similarity labels. It learns binary codes that preserve cross-modal similarity and hash functions for unseen queries, under stated data-observation assumptions.

  • 2 Notations and Problem Definition: The two modalities are represented by feature matrices X and Y for n training points, with x_i and y_i denoting each point’s modality-specific features.The feature dimensions are d_x and d_y, respectively.
  • 2 Notations and Problem Definition: Training assumes both modalities are observed for every training point, but queries need only one observed modality in the cross-modal setting.The data are also assumed to be zero-centered, and missing training modalities are left for future study.
  • 2 Notations and Problem Definition: The supervised similarity matrix S marks whether each cross-modal pair is similar, with S_ij = 1 indicating similarity and otherwise dissimilarity.The formulation assumes all elements of S are observed, while adaptation to missing elements is noted.
  • 2 Notations and Problem Definition: Supervised CMH learns binary codes U and V for two modalities so similar cross-modal pairs have small Hamming distance.The setup also learns hash functions for producing codes for unseen query points.

3 Discrete Latent Factor Model based Cross-Modal Hashing

DLFH formulates cross-modal hashing as a discrete latent-factor model that directly learns binary codes while preserving supervised similarity. Its alternating optimization is convergent, and stochastic updates reduce computational cost for larger datasets.

  • 3.1 Model Formulation: The objective maximizes the log-likelihood of the supervised similarity matrix over binary code matrices U and V.The formulation uses a logistic likelihood and seeks codes whose inner-product relationships preserve the similarity matrix S.
  • 3.1 Model Formulation: DLFH directly learns binary cross-modal hash codes without continuous relaxation, modeling supervised similarity with a discrete latent factor model.The method is designed for cross-modal supervised hashing and preserves similarity information in the learned codes.
  • 3.2 Learning Algorithm: Alternating column-wise updates optimize U and V with the other matrix fixed, using surrogate optimization to avoid exhaustive binary search.Learning one column reduces the direct search cost from O(2^n) to an update with O(n^2) complexity.
  • 3.2 Learning Algorithm: The learning algorithm converges because each U or V update does not decrease the objective, which is upper-bounded, although the non-convex problem may reach a local optimum.The monotonic objective behavior provides the stated convergence guarantee.
  • 3.2 Learning Algorithm: Stochastic DLFH samples m rows or columns when computing gradients, reducing computational cost from O(n^2) to O(nm) when m is far less than n.The stochastic strategy is introduced because full DLFH becomes intractable for large training sets.
  • 3.3 Out-of-Sample Extension: For unseen queries, DLFH uses linear hash functions learned by fitting the observed binary codes, while stronger functions such as deep neural networks are left for future work.The paper presents the linear function as a demonstration of effectiveness rather than its central focus.

4 Experiments

Experiments on MIRFLICKR-25K and NUS-WIDE compare DLFH and KDLFH with six CMH baselines using MAP, convergence, training-time, and hyper-parameter analyses. DLFH achieves higher retrieval accuracy while retaining training efficiency, whereas KDLFH trades slower training for further accuracy gains.

  • 4.3 Convergence Analysis: DLFH and its stochastic variant converge quickly, with stochastic sampling causing some vibration but preserving an overall convergent trend.The objective value for DLFH-Full does not decrease with iterations, and MAP also converges overall; only a small number of iterations is needed.
  • 4.4 Accuracy: DLFH significantly outperforms all baselines on both image-to-text and text-to-image retrieval tasks, while KDLFH further improves retrieval accuracy.Table 1 reports MAP results on both datasets; the comparison includes supervised, unsupervised, continuous, discrete, and kernel-based methods.
  • 4.5 Training Speed: DLFH’s training speed is comparable to relaxation-based continuous methods, while existing discrete methods are much slower; KDLFH is faster than the kernel baseline SePH.CCA-ITQ is fastest but has low accuracy, whereas DLFH offers a stronger accuracy–efficiency balance.
  • 4.5 Training Speed: KDLFH improves accuracy over DLFH but trains much more slowly, giving users a speed-versus-accuracy choice between the two methods.The paper positions DLFH for faster training and KDLFH for higher accuracy in practical applications.
  • 4.6 Sensitivity to Hyper-Parameter: DLFH is insensitive to λ when 1 < λ < 16, while increasing sampled points improves accuracy at higher computational cost; m = c provides a tradeoff.The sensitivity study uses 16-bit codes and reports results for λ in [10^-4, 32].

5 Conclusion

DLFH is a discrete cross-modal hashing method that directly learns binary codes while maintaining efficient training, outperforming continuous and discrete alternatives in accuracy and training speed.

  • DLFH significantly outperforms relaxation-based continuous methods in accuracy with comparable training speed, and surpasses existing discrete methods in both accuracy and training speed.
Loading 1707.08322v1…