Coupled CycleGAN: Unsupervised Hashing Network for Cross-Modal Retrieval

Introduction

• Heterogeneous image and text data make cross-modal search difficult, while hashing supports compact binary codes and fast bit-wise retrieval.The central challenges are capturing cross-modal correlation and learning reliable hash codes.Source: Introduction · p. 1
• Existing unsupervised deep hashing methods can struggle to model accurate modality correlations and separate hash-code generation from common-representation learning.The paper identifies these as limitations of existing approaches.Source: Introduction · pp. 1–2
• UCH couples outer-cycle representation learning with inner-cycle hash-code generation in one unsupervised framework.The authors report that the two learning processes interact and are optimized together.Source: Introduction · p. 2

Related Work

• Shallow methods exploit image-text co-occurrence through approaches including CCA, CVH, PDH, CMFH, LSSH, and FSH.Their strategies include mapping paired data to latent spaces, preserving cross-modal similarity, and learning unified codes.Source: Related Work · p. 2
• UGACH builds a similarity matrix from deep features and learns modality correlation through manifold structure, while UDCMH iteratively updates features using hash codes as supervision.The paper notes that UGACH's matrix may be inaccurate and time-consuming, and that UDCMH's short codes provide insufficient supervision for discriminative representations.Source: Related Work · p. 3
• UCH instead learns bidirectional transformations between modalities and iteratively updates hash codes within a coupled-cycle framework.The authors characterize the resulting codes as more reliable.Source: Related Work · p. 3

Proposed UCH

• UCH combines an outer-cycle representation module with an inner-cycle hashing module, trained jointly to learn common representations and compact hash codes.The outer cycle maps modality data through a shared space to generate other-modality data; the inner cycle uses hashing layers to produce codes.Source: Proposed UCH · p. 3

Problem Definition

• The task takes image-text pairs as input and seeks image and text codes in {-1, 1}^n×K, with K denoting code length.The hashing-layer outputs H* are converted to binary codes B* using a sign function.Source: Problem Definition · p. 3

Representation Learning

• The outer-cycle GAN learns cross-modal representations through bidirectional modality generation, with adversarial, reconstruction, and similarity losses.The similarity loss regulates paired representations in a shared common space; discriminators provide adversarial loss for the generators.Source: Representation Learning · pp. 3–4

Hashing Learning

• The inner-cycle GAN generates hash codes using adversarial, reconstruction, and similarity losses, with the total hashing objective combined with representation loss.The algorithm updates the hash-code matrix using the sign of the summed image and text outputs.Source: Hashing Learning · p. 4

Optimization

• To address gradient vanishing from minimax GAN loss, UCH optimizes in stages, first training the outer cycle and then the inner cycle.The updates separately optimize adversarial terms and reconstruction-plus-similarity terms while other parameters are fixed.Source: Optimization · p. 4

Datasets

• The evaluation uses MIRFlickr-25K, IAPR TC-12, and Microsoft COCO, with 20,015, 18,571, and 122,218 image-text pairs, respectively.For all unsupervised methods on COCO, the retrieval set is also used for training.Source: Datasets · p. 5

Baselines and Evaluation

• The comparison includes five unsupervised shallow methods, two supervised shallow methods, and the deep method UGACH; retrieval is evaluated with MAP, PR curves, and Precision@N.The shallow methods use CNN-F features for fair comparison.Source: Baselines and Evaluation · p. 5

Implementation Details

• Image features are CNN-F fc7 outputs, while a 300-node embedding layer projects text bag-of-words vectors into continuous features.The implementation uses TensorFlow and two NVIDIA TITAN X GPUs.Source: Implementation Details · pp. 5–6

Experiment Results

• UCH achieves the highest MAP at all code lengths across MIRFlickr-25K, IAPR TC-12, and Microsoft COCO, and outperforms UGACH across code lengths on MIRFlickr and IAPR TC-12.The reported MAP comparison attributes UCH's performance to jointly learning common representations and hash codes.Source: Experiment Results · pp. 6–7, Table 2 · p. 7, Table 3 · p. 7
• The 64-bit precision-recall and top-1,000 precision curves show conclusions similar to the MAP comparisons.PR curves vary the Hamming radius from 0 to 64 in steps of 1; Precision@1000 measures precision among the top 1,000 retrieved instances.Source: Experiment Results · pp. 6–7, Figure 2 · p. 6, Figure 3 · p. 7
• UCH outperforms UGACH in MAP across code lengths on both MIRFlickr and IAPR TC-12.The comparison uses the same CNN-F network to extract deep features for UGACH.Source: Experiment Results · p. 7, Table 2 · p. 7, Table 3 · p. 7

• UCH jointly learns common representations and reliable hash codes without labels, and achieves state-of-the-art performance on three widely used cross-modal retrieval datasets.Source: Conclusions · p. 7