PaperlayerView exact revision on arXiv

Source-linked AI summary

Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification

Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, Lei Zhang

arXiv:1508.04535v2cs.CV

TL;DR

Existing hashing methods often separate feature learning from hash-function learning and use preset code lengths, limiting adaptability. This paper jointly learns raw-image features and weighted hashing functions with regularized triplet similarity learning, and reports strong retrieval and re-identification results with shorter codes.

  • Problem

    Existing methods often learn hash functions from hand-crafted features and use preset code lengths, while different applications may require different code lengths.

  • Method

    A supervised CNN jointly learns raw-image features and weighted binary hash functions using regularized triplets that separate matched and mismatched pairs while preserving appearance adjacency.

  • Results

    The framework outperforms state-of-the-art hashing methods on image-retrieval benchmarks and achieves comparable person-re-identification performance while preserving accuracy with shorter codes.

  • Takeaways & Limitations

    Unequally weighted bits make hashing codes flexible across code lengths, including shorter representations obtained by truncating insignificant bits.

Abstract

from arXiv · show

Extracting informative image features and learning effective approximate hashing functions are two crucial steps in image retrieval . Conventional methods often study these two steps separately, e.g., learning hash functions from a predefined hand-crafted feature space. Meanwhile, the bit lengths of output hashing codes are preset in most previous methods, neglecting the significance level of different bits and restricting their practical flexibility. To address these issues, we propose a supervised learning framework to generate compact and bit-scalable hashing codes directly from raw images. We pose hashing learning as a problem of regularized similarity learning. Specifically, we organize the training images into a batch of triplet samples, each sample containing two images with the same label and one with a different label. With these triplet samples, we maximize the margin between matched pairs and mismatched pairs in the Hamming space. In addition, a regularization term is introduced to enforce the adjacency consistency, i.e., images of similar appearances should have similar codes. The deep convolutional neural network is utilized to train the model in an end-to-end fashion, where discriminative image features and hash functions are simultaneously optimized. Furthermore, each bit of our hashing codes is unequally weighted so that we can manipulate the code lengths by truncating the insignificant bits. Our framework outperforms state-of-the-arts on public benchmarks of similar image search and also achieves promising results in the application of person re-identification in surveillance. It is also shown that the generated bit-scalable hashing codes well preserve the discriminative powers with shorter code lengths.

I. INTRODUCTION

The paper addresses limitations of separately learned features and hash functions, preset code lengths, and costly storage of multiple code versions. It proposes supervised bit-scalable deep hashing for image retrieval and person re-identification.

  • Motivation: Traditional systems separately learn hand-crafted feature representations and hashing functions, so features may not match the hashing objective.Hand-crafted feature engineering also requires domain knowledge and heavy tuning.
  • Motivation: Preset hash lengths limit flexibility because different scenarios may require shorter codes for constrained devices or longer codes for higher accuracy.Storing multiple code versions causes extra computation and storage.
  • Proposed framework: The proposed supervised framework maps raw images to binary codes with a CNN that jointly learns image features and hash functions.The framework targets general image retrieval and person re-identification across disjoint camera views.
  • Bit scalability: Each hash bit receives a significance weight, allowing insignificant bits to be truncated to obtain arbitrary shorter codes without extra computation.The codes are intended to preserve matching accuracy across varying lengths.
  • Similarity learning: Triplet-based regularized similarity learning maximizes matched–mismatched margins in Hamming space while encouraging similar-appearance images to receive similar codes.Triplets contain two same-label images and one different-label image.
  • Reported contributions: Experiments report preserved instance-level similarity, improved retrieval performance over state-of-the-art hashing approaches, and application to surveillance person re-identification.The paper frames the formulation as extendable to other similar tasks.

III. BIT-SCALABLE DEEP HASHING FRAMEWORK

The framework learns weighted binary codes directly from raw images through regularized triplet similarity learning. Unequal bit weights support flexible code truncation, while a smooth approximation enables gradient-based optimization.

  • Code representation: A deep CNN jointly learns a nonlinear raw-image transformation and a weight vector representing the significance of each output hash bit.The weighted Hamming affinity measures code similarity as a weighted combination of bit agreements.
  • Bit scalability: Weighted hashing improves code effectiveness, supports truncating low-weight bits, and degenerates naturally to conventional Hamming distance.Truncation adapts code lengths to different computational scenarios.
  • Triplet formulation: Triplets contain two same-label images and one different-label image, and a max-margin loss separates matched from mismatched pairs in Hamming space.The loss is intended to preserve ranking according to annotated semantics.
  • Regularization: A regularization term encourages the learned codes to preserve adjacency relations from the original appearance space.The similarity matrix assigns larger values to similar image pairs and smaller values to dissimilar pairs.
  • Optimization surrogate: Because binary objectives are discontinuous and nondifferentiable, a tanh-like sign approximation is used during learning and the sign function is used at test time.The smoothness parameter β increases from 2 to 1000 during training.
  • Retrieval: A lookup-table approach rapidly returns weighted affinity values for truncated binary codes.For code length l, the table has length 2^l.

B. Deep Architecture

The deep architecture takes raw images as input and outputs weighted binary hashing codes. Its convolutional, fully connected, activation, and element-wise weighting layers support end-to-end retrieval learning.

  • Deep Architecture: The network uses six convolution-pooling layers followed by two fully connected layers, a tanh-like hash layer, and an element-wise bit-weighting layer.The first fully connected layer has 512 units, while the second matches the hash-code length.
  • Deep Architecture: The architecture produces hashing codes from raw images and trains its retrieval representation with triplet-based similarity learning.The retrieval example measures similarity using Hamming affinity.

IV. LEARNING ALGORITHM

The learning algorithm optimizes the network and bit-weight parameters jointly by differentiating the continuous loss. It computes image-level gradients through forward and backward propagation and combines triplet contributions.

  • Joint Optimization: The algorithm optimizes the network parameters for the continuous regularized triplet-learning loss.The same optimization procedure applies to varied-length hashing.
  • Joint Optimization: Network parameters are treated jointly with bit weights, and partial derivatives of the objective are computed for backpropagation.The parameter vector combines the hashing-function parameters and weight parameters.
  • Gradient computation: Gradients for triplet terms are computed through forward and backward propagation for each image participating in a triplet.The procedure derives image-specific gradients before aggregating them.
  • Gradient computation: The regularization component is optimized with respect to each approximated hash representation using matrix-based derivatives.The derivation rewrites the second objective term with respect to r_j.
  • Optimization procedure: The overall process forwards an image, computes derivatives, and sums the resulting gradients to update the model.The summarized steps include calculating the representation and its corresponding gradient.

B. Acceleration

The framework accelerates training by expressing the total triplet loss through image-based gradients, while accounting for triplet interactions in the derivative calculation.

  • The second loss term has a partial derivative depending on a single image, whereas the first requires triplet-based gradients.
  • The total loss can be represented through the outputs of the images appearing in the triplets.
  • The derivative is computed with respect to the distinct images adopted in the triplet set, whose count is M.
  • Image-based gradient calculation accounts for the outputs of all images in the triplets rather than only one image.
  • The resulting method provides the gradient-computation procedure used by Algorithm 1 and Algorithm 2.

C. Batch Process Implementation

The deep hashing algorithm prepares training by collecting distinct images from the triplets and iteratively updating the network parameters.

  • The training procedure takes training triplets D as input and outputs network parameters ω.
  • It first collects all distinct images {I_j} appearing in the triplet set D.
  • The algorithm then repeats its optimization steps after this preparation stage.

1. Calculate outputs (rj ew

Training computes network outputs and partial derivatives for images in sampled triplets, then applies backpropagation and parameter updates within repeated iterations.

  • The procedure computes image outputs by forward propagation before evaluating derivatives.
  • Backpropagation is used to propagate the computed derivatives through the network.
  • The implementation updates parameters iteratively and increments t until t exceeds T.
  • Partial derivatives are calculated for an image using the image-based partial-derivative algorithm.
  • Triplets are generated from randomly selected categories and images, with different-label images selected from the remaining categories.

A. Dataset and Experimental Setting

Experiments evaluate the hashing framework on public image-retrieval datasets using held-out query procedures, ranking and lookup metrics, variants, and established baselines.

  • Experiments use MNIST, CIFAR-10, CIFAR-20, and NUS-WIDE, splitting each dataset into training and query sets.
  • The query image is searched within the query set itself under a leave-one-out procedure.
  • The study evaluates DRSCH, DSCH, Euclidean, BS-DRSCH, and BS-DSCH variants to examine framework components and bit-scalable versions.
  • Comparisons include eight state-of-the-art hashing approaches, including LSH, SH, ITQ, PCA-RR, MLH, BRE, and KSH.
  • Evaluation uses Hamming ranking and hash lookup, with MAP and precision@500 among the reported performance metrics.

B. Network and Parameter Setting

Experiments evaluate the hashing framework across four benchmark datasets using multiple code lengths and retrieval measures. The reported results show strong performance, while also identifying reduced gains on the larger CIFAR-20 setting.

  • B. Network and Parameter Setting: The framework resizes inputs and trains with batches of approximately 200 images, using λ = 0.001 across experiments.NUS-WIDE uses 64 × 64 inputs; MNIST, CIFAR-10, and CIFAR-20 use 28×28, 32×32, and 32 × 32 inputs, respectively.
  • MNIST: MNIST retrieval uses precision curves, Precision@500, 64-bit Precision@k, and MAP across code lengths from 8 to 64 bits.DRSCH and DSCH outperform the other methods in all cases, with DRSCH gaining at least 10% over traditional methods using CNN features.
  • CIFAR-10: CIFAR-10 evaluation reports Hamming-radius precision, Precision@500, 64-bit Precision@k, and MAP across different code lengths.The approach outperforms traditional cascade methods and achieves a relative increase of 1.67% compared with DSRH.
  • NUS-WIDE: NUS-WIDE evaluation uses GIST and CNN representations for traditional methods and reports three precision views across varied code lengths.The dataset is multi-label, and semantic similarity is based on category-label information; MAP is calculated within the first 50K searched neighbors.
  • CIFAR-20: CIFAR-20 results report Hamming-radius precision, Precision@500, 64-bit Precision@k, and MAP across code lengths, with DRSCH performing best.As dataset scale grows, the performance gain becomes insignificant, which the authors relate to more populated classes and more complicated manifold estimation for triplet comparison.

D. Efficiency Analysis

The bit-scalable framework supports variable-length hashing by selecting the highest-weight bits, while maintaining competitive retrieval performance across code lengths and datasets. A single training run supports varied code-length requirements.

  • Bit-scalable hashing: BS-DRSCH learns a maximum 64-bit weighted hash code and selects any k ≤ 64 highest-weight bits for testing.This design makes the hashing codes scalable to any requested length up to the trained maximum.
  • Practical efficiency: BS-DRSCH requires only one training process, making it suitable when different application scenarios require different hashing-code lengths.The selected code length can be changed at testing time by truncating to the bits with largest weights.
  • Retrieval performance: BS-DRSCH achieves very competitive retrieval results relative to its fixed-length DRSCH and DSCH versions across varied hash-code lengths.Performance is evaluated using mean average precision across multiple bit lengths on benchmark datasets.
  • Retrieval performance: For code lengths of 32 bits or fewer, BS-DRSCH generally outperforms DRSCH on MNIST, NUS-WIDE, and CIFAR-20.The reported advantage is dataset-dependent and is specifically stated for shorter codes.
  • Retrieval performance: At sufficiently large code lengths, weighted and non-weighted hash codes obtain similar performance because the weighted code can be approximated by a longer unweighted code.The performance gains from weighting become insignificant as the number of bits increases.

F. Application to Person Re-Identification

The paper applies deep hashing to person re-identification across disjoint surveillance-camera views, evaluating it on CUHK03 against re-identification and hashing baselines. The framework achieves comparable performance to state-of-the-art re-identification methods and larger margins over other hashing methods on Rank-1 and Rank-5.

  • Task motivation: Person re-identification across disjoint camera views remains challenging because viewpoint and pose changes create dramatic variations.The task is relevant to surveillance applications that restrict the use of face recognition.
  • Dataset and protocol: The CUHK03 dataset contains 13,164 images of 1,360 pedestrians from six surveillance cameras, split into training, validation, and test identities.The reported partition uses 1,160 training persons, 100 validation persons, and 100 test persons.
  • Dataset and protocol: The evaluation compares the framework with three person re-identification methods, four hashing methods, and Euclidean distance using Hamming ranking.Hashing-based methods are evaluated with 64- and 128-bit codes.
  • Results: The deep hashing framework achieves comparable performance to state-of-the-art person re-identification methods on CUHK03.The comparison is reported in Table X using manually labeled pedestrian bounding boxes.
  • Results: The framework outperforms other hashing methods with large margins on Rank-1 and Rank-5 identification rates.These results are reported for the CUHK03 person re-identification evaluation.
  • Future directions: The conclusion identifies more semantic image information and feedback learning as directions for extending the framework.The paper specifically mentions multiple attributes as an example of additional semantics.
Loading 1508.04535v2…