Source-linked AI summary
One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective
Jiun Tian Hoe, Kam Woh Ng, Tianyu Zhang, Chee Seng Chan, Yi-Zhe Song, Tao Xiang
TL;DR
Deep hashing typically combines multiple losses to obtain discriminative, balanced, and low-quantization-error binary codes, making optimization difficult. OrthoHash replaces these objectives with cross-entropy maximizing cosine similarity to binary orthogonal targets, while using normalization and label smoothing for the supported settings. Experiments report stronger retrieval performance than prior methods across category- and instance-level benchmarks, with scope limitations for target construction and code-center guarantees.
Problem
Deep hashing must jointly learn discriminative binary codes and minimize quantization error, while additional balance and orthogonality constraints often require more than four losses.
Method
OrthoHash uses cross-entropy to maximize cosine similarity between L2-normalized continuous codes and binary orthogonal targets, with BN for code balancing and label smoothing for multi-label classification.
Results
OrthoHash outperforms previous deep hashing methods across three instance-level retrieval datasets at all evaluated code lengths, including gains of 0.6%, 9.1%, and 17.1% with 128-bit codes.
Takeaways & Limitations
A single cosine-similarity-based classification objective can unify discriminativeness and quantization-error minimization while avoiding explicit loss-weight tuning.
Takeaways & Limitations
When 2^K < C, sampled binary targets can contain identical nearest rows and degrade performance, and the optimization does not guarantee final codes equal target-class centers.
Abstract
from arXiv · showhide
A deep hashing model typically has two main learning objectives: to make the learned binary hash codes discriminative and to minimize a quantization error. With further constraints such as bit balance and code orthogonality, it is not uncommon for existing models to employ a large number (>4) of losses. This leads to difficulties in model training and subsequently impedes their effectiveness. In this work, we propose a novel deep hashing model with only a single learning objective. Specifically, we show that maximizing the cosine similarity between the continuous codes and their corresponding binary orthogonal codes can ensure both hash code discriminativeness and quantization error minimization. Further, with this learning objective, code balancing can be achieved by simply using a Batch Normalization (BN) layer and multi-label classification is also straightforward with label smoothing. The result is an one-loss deep hashing model that removes all the hassles of tuning the weights of various losses. Importantly, extensive experiments show that our model is highly effective, outperforming the state-of-the-art multi-loss hashing models on three large-scale instance retrieval benchmarks, often by significant margins. Code is available at https://github.com/kamwoh/orthohash
1 Introduction
Deep hashing must learn discriminative binary codes while controlling quantization error, yet existing methods often add several auxiliary losses. OrthoHash unifies these goals with one cosine-similarity-based loss and uses BN for balancing.
- Hashing converts image features into binary codes for efficient storage and accurate retrieval.
- A CIFAR10 visualization shows CE alone produces unbounded codes, BN balances them, and the proposed objective reduces intra-class variance and quantization error.
- Deep hashing requires both discriminative binary codes and minimized quantization error, but the quantization layer introduces vanishing gradients and sub-optimal relaxed codes.
- Existing methods commonly add bit-balance, Hamming-distance, orthogonality, and gradient-related losses, often exceeding four losses and complicating optimization.
- OrthoHash maximizes cosine similarity between L2-normalized continuous codes and binary orthogonal targets, unifying discriminativeness and quantization-error minimization in one cross-entropy loss.
2 Related Work
Related hashing methods address binary optimization, gradient difficulties, and predefined targets through relaxations, modified coding layers, or target-centering losses. OrthoHash instead emphasizes a one-loss design based on cosine similarity.
- Conventional hashing methods emphasize code balance, uncorrelated bits, and similarity preservation, while data-dependent methods learn compact dataset-specific codes.
- Binary optimization is NP-hard, and continuous relaxations or penalty terms introduce hyperparameters that must balance competing learning objectives.
- Straight-through estimators and Bi-half layers bypass vanishing gradients but modify the computational graph and complicate end-to-end optimization.
- DPN and CSQ learn continuous codes near predefined targets, using randomly assigned maximally separated vectors or Hadamard hash centers.
- Cosine similarity provides a lens for relating continuous-code similarity to approximate binary representations and quantization error.
3 OrthoHash: One Loss for All
OrthoHash reformulates retrieval and quantization through cosine similarity, then uses one classification objective with binary orthogonal targets. Batch normalization supplies code balancing, while target construction and code length determine practical constraints.
- 3.1 Reformulating Deep Hashing in the Lens of Cosine Similarity: Cosine similarity captures both hash-code retrieval similarity and quantization error, allowing the two objectives to be unified.Retrieval depends on code direction, and maximizing similarity between continuous and binary codes lowers quantization error.
- 3.2 Discriminative Hash Codes with Orthogonal Target: The method maximizes cosine similarity between continuous codes and binary orthogonal targets using a single classification objective.The targets encode classes, and the resulting loss replaces a separately weighted quantization term.
- 3.2 Discriminative Hash Codes with Orthogonal Target: Orthogonal binary targets provide maximal inter-class Hamming distance, while an angular margin further reduces intra-class variance.The paper sets the cosine margin to m = 0.2 in its experiments unless stated otherwise.
- 3.2.1 Binary Orthogonal Target: Random Bern(0.5) targets approximate orthogonality, but when 2K < C, identical nearest target rows can degrade performance.The stated remedy is to increase K; heuristic maximum-distance target generation helps more at lower K than at higher K.
- 3.2 Discriminative Hash Codes with Orthogonal Target: The optimization does not guarantee that final hash codes become the centers of their target classes.The paper instead leaves the optimization algorithm to find the best hash codes.
4 Experiment
Experiments evaluate the proposed hashing variants across category-level and instance-level retrieval benchmarks, analyzing retrieval performance, code balance, margins, orthogonal transformation, and distance-based factors.
- Setup: The experiments compare seven deep hashing methods using the same learning rate, Adam optimizer, and 100 training epochs.The evaluated datasets include ImageNet100, NUS-WIDE, MS-COCO, GLDv2, ROxf, and RPar.
- Category-level retrieval: OrthoCos+BN and OrthoArc+BN outperform recent state-of-the-art methods DPN and CSQ; OrthoCos+BN reaches 0.850 and 0.856 on NUS-WIDE with 64- and 128-bit codes.On MS-COCO, the proposed variants perform best with at most 1% improvement over previous deep hashing methods.
- Code balance: 5-20% improvement follows from appending BN to cross-entropy models across datasets and code lengths, while Bihalf adds 0.1-4.9% over CE+BN.The results support code balance as an important factor and show that the proposed method achieves it without explicitly engineering the computational graph.
- Cosine and angular margin: Cosine margin slightly outperforms angular margin by about 0.2% on average.The comparison is between OrthoCos+BN and OrthoArc+BN.
- Instance-level retrieval: On all three instance-level datasets and every tested bit length, the proposed method outperforms previous deep hashing methods, with 128-bit gains of 0.6%, 9.1%, and 17.1%.The authors interpret this pattern as evidence of better generalization to unseen instances.
- Analysis: Orthogonal transformation increases cosine-distance separability from 0.142 to 0.167 for GLDv2 2048-bit codes and improves performance by 1.1% over R50-DELG-C.The analysis attributes performance improvements to quantization error, Hamming-distance separability, and orthogonality; reducing quantization error increases separability and improves hash-center orthogonality.
- Domain shifting with BN: Testing BN statistics across domains can cause a large performance drop, so ROxf and RPar use running means and variances recomputed from their respective databases.The issue arises because the model is trained on GLDv2 and tested directly on different datasets.
- Analysis: For 64-bit ImageNet100, larger separability between intra-class and inter-class Hamming distances corresponds to better retrieval performance.Figure 3 normalizes histogram frequencies so all bins sum to 1, while Figure 4 analyzes quantization error, separability, and orthogonality.
5 Conclusion & Future Work
The paper unifies deep hashing training under a single classification objective based on cosine similarity, while using L2 normalization, label smoothing, and batch normalization to support end-to-end, multi-label, balanced-code learning.
- The method unifies deep hashing objectives under a single classification objective using cosine similarity between continuous codes and binary orthogonal targets.
- L2 normalization enables end-to-end deep hashing without extra sophisticated constraints.
- Label smoothing supports multi-label classification, while batch normalization supports code balancing under the unified objective.
- Experiments validate the method on category-level and instance-level retrieval benchmarks.
Broader Impact
The work targets difficult-to-train deep hashing frameworks by replacing complex loss designs with a single cosine-similarity-based objective, reporting competitive common-dataset performance and significant instance-retrieval improvements.
- Complex loss designs make state-of-the-art hashing frameworks difficult to train and reproduce because they require hyperparameter tuning and multi-stage training.
- The proposed single-objective formulation aims to simplify training and improve reproducibility through cosine-similarity-based hashing.
- The method reports competitive performance on common datasets and significant improvements over state-of-the-art methods for instance-level retrieval.