PaperlayerView exact revision on arXiv

Source-linked AI summary

Relevance-Resolution Transfer via Scale-Decomposable Fractional Diffusion for Multi-Length Cross-Modal Hash Retrieval

Xi Chen, Xu Chen, Xiangyang Jia, Ting Gan, Xu Zhang, Shuquan Wei, Sitong Fan

arXiv:2609.34393v1cs.IR

TL;DR

Multi-label methods create fine-grained relations, but do not ensure they survive as consistent rankings in finite, multi-length Hamming spaces. MULTIBIT constructs scale-decomposable fractional-diffusion supervision and aligns nested-prefix rankings to it; a single nested model achieves the highest mean mAP across four code lengths on all three datasets and the highest cross-length ranking agreement among compared methods.

  • Problem

    Fine-grained label relations do not ensure that candidate rankings remain consistent in finite Hamming spaces across code lengths.

    Source: §1 · p. 1

  • Method

    MULTIBIT combines a fractional relation teacher, scale-aware bit subblocks, and nested-prefix alignment to transfer multi-label relevance into multi-length Hamming rankings.

    Source: §1 · p. 2

  • Results

    A single nested model achieves the highest Mean mAP averaged over four code lengths on all three datasets and the highest cross-length ranking agreement among compared methods.

    Source: §5 · pp. 9–10

  • Takeaways & Limitations

    The results indicate that organizing relations by propagation scale is a practical route to consistent multi-length binary retrieval.

    Source: §5 · p. 10

Abstract

from arXiv · show

Cross-modal hashing enables efficient retrieval by encoding heterogeneous data into compact binary codes. Recent methods exploit fine-grained relations encoded in multi-label training structure, yet none of them constrains how those relations survive as consistent candidate rankings in finite, multi-length Hamming spaces, which we term the relevance resolution bottleneck (RRB). To address the RRB, we propose MultiBit, which transfers relevance resolution from multi-label structure to multi-length Hamming spaces. MultiBit first constructs a scale-decomposable fractional relation teacher from dataset-level label co-occurrence and label specificity, and models dependencies from local to long-range over continuous diffusion scales. It then maps the discretized diffusion scales and their quadrature weights to scale-aware bit subblocks of the maximum-length code, organizes the target code lengths as nested prefixes, and aligns their Hamming candidate rankings with the teacher relations. Experiments on multiple benchmarks demonstrate improved retrieval accuracy. Code is available in the supplementary material.

1 INTRODUCTION

Recent multi-label methods produce increasingly fine-grained semantic relations, but do not ensure those distinctions persist as consistent rankings in finite, multi-length Hamming spaces. MULTIBIT addresses this relevance resolution bottleneck by organizing scale-aware supervision and aligning nested-code rankings with it.

1 INTRODUCTION

  • Fine-grained label-side relations do not determine how propagation ranges map to finite bits or ensure candidate ordering survives across code lengths.The remaining limitation is transferring relations into finite, multi-length codes.Source: §1 · p. 1
  • The relevance resolution bottleneck occurs when semantically different candidates collide, invert order, or exchange ranks across code lengths.The paper defines relevance resolution as preserving distinctions between a query and its candidates, rather than increasing bit count.Source: §1 · p. 1
  • MULTIBIT combines a fractional relation teacher, scale-aware bit subblocks, and nested-prefix alignment to preserve multi-label relevance in multi-length Hamming rankings.The framework's contributions also formalize the bottleneck and evaluate how much ordering survives sign quantization at all four code lengths.Source: §1 · p. 2

2 RELATED WORK

Prior cross-modal hashing methods build semantic supervision from binary relevance, label overlap, or structured label dependencies, while multi-length hashing supports deployment at several code lengths. The paper identifies a remaining gap: these approaches do not constrain ranking consistency across lengths.

Cross-Modal Hashing

  • Cross-modal hashing supervision ranges from binary pairwise or hard-negative objectives to multi-label overlap and structured label dependencies (Zhang et al., 2026c; Su et al., 2025; Peng et al., 2026; Huo et al., 2024b; Yao et al., 2026; Cheng et al., 2026b; Zhang et al., 2026b; Cao et al., 2025; Zhang et al., 2026a; Meng et al., 2026).The structured approaches incorporate dataset-level co-occurrence and higher-order dependencies.Source: §2 · p. 2

Fine-Grained Relevance Modeling

  • Fine-grained relevance methods capture label overlap and frequency effects, with structured approaches extending supervision to correlations, graph neighborhoods, and indirect paths.Some produce continuous relation scores for pairwise supervision or representation regularization.Source: §2 · p. 2

Multi-Length and Bit-Scalable Hashing

  • Central-coding, multi-bit, and adaptive-bit methods support deployment across code lengths, but do not constrain ranking changes when codes are truncated.The paper's stated gap is that relations are not organized by propagation scale and finite bit capacity or constrained to remain consistent across lengths.Source: §2 · pp. 2–3

3 METHOD

MULTIBIT trains a shared maximum-length representation whose scale-specific subblocks and nested prefixes preserve teacher-derived relevance rankings. Its method constructs fractional-diffusion supervision, allocates bits by scale, and aligns rankings at each target length.

3.1 PROBLEM FORMULATION: RELEVANCE RESOLUTION BOTTLENECK

  • A shared maximum-length representation supports target code lengths through prefix selection, while modality-specific hash functions map image and text inputs into a common Hamming space.Training uses tanh as a continuous relaxation because the discrete sign function is non-differentiable; inference uses encoding, prefix truncation, and Hamming ranking.Source: §3.1 · p. 3

3.2 OVERVIEW

  • The framework constructs fractional-diffusion relational supervision, allocates scale-specific bit subblocks, and aligns nested-prefix Hamming rankings to a shared teacher ordering.The overview names these components FRT, SBO, and MPA, respectively.Source: §3.2 · p. 3, Figure 2 · p. 4

3.3 SCALE-DECOMPOSABLE FRACTIONAL RELATION TEACHER (FRT)

  • FRT builds a PPMI label graph and specificity-weighted labels, then uses fractional diffusion to represent dependencies across scales and derive integrated and scale-specific teachers.Quadrature nodes and weights expose finite diffusion scales for the bit-allocation stage.Source: §3.3 · pp. 3–4

3.4 SCALE-AWARE BIT ORGANIZATION (SBO)

  • SBO allocates the maximum code budget using normalized quadrature weights, assigns at least one bit to each retained scale, and concatenates subblocks from local to long-range scales.The allocation ratio is a kernel-mass prior governed by α rather than η, not a measured per-scale contribution.Source: §3.4 · pp. 4–5

3.5 MULTI-LENGTH PREFIX ALIGNMENT (MPA)

  • MPA aligns both retrieval directions' Hamming distributions at each nested-prefix length with the integrated teacher distribution, while scale losses supervise individual subblocks.At inference, the teacher branch and listwise objectives are removed; requested lengths use prefix extraction and standard Hamming ranking.Source: §3.5 · pp. 5–6
  • When the listwise divergence vanishes, the continuous Hamming surrogate preserves the teacher's strict relevance order as a distance order at that length.The stated implication is conditional on zero divergence and concerns the continuous surrogate, not necessarily sign-quantized codes.Source: §3.5 · p. 6

4 EXPERIMENTS

Across three benchmarks, MultiBit leads in most reported retrieval settings and combines competitive accuracy with one nested model serving four code lengths. Ablations, relevance-resolution measures, cross-length rankings, and examples examine how its components and representations behave.

4.2 COMPARISON WITH STATE-OF-THE-ART METHODS

  • MultiBit wins 16 of 24 benchmark settings, including every 64- and 128-bit cell on MIRFlickr-25K and MS-COCO; RCMH and VH-ABL remain below it throughout.The comparison covers I2T and T2I retrieval, three datasets, and four code lengths; CMCL takes the other eight settings.Source: §4.2 · p. 6, Table 1 · p. 6

4.3 ABLATION STUDY

  • Each component improves Mean mAP in all twelve dataset–length settings, while 43.5% to 70.1% of queries improve against the alternative designs.At 128 bits, FRT adds 0.888, 1.087, and 2.072 pp on MIRFlickr-25K, IAPR TC-12, and MS-COCO, respectively; Figure 4 displays query-level AP-difference distributions.Source: §4.3 · p. 7, Figure 4 · p. 7

4.4 RELEVANCE RESOLUTION ANALYSIS

  • MultiBit achieves the highest NDCG@100 and Kendall’s τb and the lowest collision and inversion rates; validation surfaces show a broad high-performance region at intermediate α and η.The comparison is under an IDF-weighted evaluator, and the same leading pattern holds with a frequency-free label-name reference.Source: §4.4 · pp. 7–8, Table 3 · p. 8, Figure 5 · p. 8

4.5 MULTI-LENGTH CONSISTENCY AND EFFICIENCY

  • Averaged across four lengths, MultiBit has the highest Mean mAP on all three datasets and stays above MultiBit-Sep in all twelve settings.CMCL leads five individual-length cells by at most 0.92 pp; one nested model replaces four at roughly 74% and 47% less model and code storage.Source: §4.5 · p. 8, Table 4 · p. 9
  • From 16 to 128 bits, MultiBit’s rank displacements concentrate closer to zero than Naive Prefix, with higher tie-corrected τb.Against CMCL, τb is 0.760 versus 0.633 at 16–128 bits on MIRFlickr-25K.Source: Figure 6 · p. 8, §4.5 · p. 9

4.6 QUALITATIVE RETRIEVAL ANALYSIS

  • Illustrative 16-bit examples show a baseline distance inversion in I2T and a collision in T2I; MultiBit brings highly relevant candidates closer through distance separation, not re-ranking.The examples are qualitative and do not estimate how prevalent these cases are.Source: §4.6 · p. 9, Figure 7 · p. 9

5 CONCLUSION

The conclusion presents MultiBit as a transfer framework for relevance resolution across multi-length Hamming spaces. It reports leading average Mean mAP and cross-length ranking agreement among compared methods.

5 CONCLUSION

  • MultiBit transfers relevance resolution using a scale-decomposable fractional relation teacher, scale-aware bit organization, and multi-length prefix alignment.Source: §5 · p. 9
  • A single nested model has the highest Mean mAP averaged over four code lengths on all three datasets and the highest cross-length ranking agreement among compared methods.Source: §5 · pp. 9–10
  • Generative AI tools were used only for manuscript readability and language edits, which the authors reviewed.Source: AI USE STATEMENT · p. 10

A.1 DATA SPLIT, EVALUATION METRICS, AND IMPLEMENTATION DETAILS

The appendix specifies the shared evaluation protocol and implementation, then formalizes relevance resolution as preserving the label-derived ranking at each target length. Its formal statement notes a finite-distance capacity bound for B-bit codes.

A.1 DATA SPLIT, EVALUATION METRICS, AND IMPLEMENTATION DETAILS

  • All methods share a fixed-seed data split; evaluation reports I2T, T2I, and their arithmetic mean using mAP@all over the complete retrieval database.The 1,000 validation pairs select checkpoints and are disjoint from the 5,000 queries; experiments use CLIP ViT-B/32, 16-, 32-, 64-, and 128-bit codes, and validation-selected checkpoints.Source: §A.1 · p. 14

A.3 FORMAL STATEMENT OF THE RELEVANCE RESOLUTION BOTTLENECK

  • Relevance resolution is preserved at a code length when Hamming-distance ranking agrees with the descending graded-relevance ranking; ideal transfer requires this at every target length.A B-bit code has only B + 1 distinct Hamming distances, so the formal statement treats this as a capacity bound and measures collisions and inversions.Source: §A.3 · p. 15
  • The formal statement distinguishes collisions, inversions, and cross-length inconsistency as three ways relevance resolution can fail.Source: §A.3 · p. 15

B DETAILED FORMULATIONS AND DERIVATIONS

The appendix explains the fractional teacher’s scale organization, its mapping to code subblocks, and diagnostics of ranking resolution and quantization. Additional results examine quadrature fidelity and retrieval on NUS-WIDE.

B DETAILED FORMULATIONS AND DERIVATIONS

  • The 16-, 32-, and 64-bit prefixes occupy the first subblock, while the 128-bit code combines multiple scales and receives the largest cross-scale SBO gain.With equal bit splits as a same-capacity control, Full MultiBit improves on 47.5–63.7% of queries, with mean ∆AP of 0.11–0.85 pp.Source: B DETAILED FORMULATIONS AND DERIVATIONS · p. 15, Table B-1 · p. 17
  • Resolution improves monotonically with code length; on MIRFlickr-25K, the last 64 bits add 0.032 NDCG@100 and 0.028 Kendall’s τb.The diagnostic increases code length, scale coverage, and bit count together, so it isolates code length rather than long-range scales alone.Source: B DETAILED FORMULATIONS AND DERIVATIONS · pp. 15, 17, Table B-2 · p. 17
  • The fractional-kernel rationale treats quadrature weights as a prior over scales, not as measured per-scale contributions to the normalized teacher score.PPMI emphasizes co-occurrences beyond independent expectation, while label specificity keeps rare labels informative; diffusion ranges from local neighborhoods to longer-range dependencies.Source: B DETAILED FORMULATIONS AND DERIVATIONS · p. 17

B.1 QUANTIZATION STATISTICS

  • Soft and discrete rankings agree with τb of 0.896–0.964, and measured agreement stays within 0.007–0.011 of the tie-only bound.The analysis attributes the bound mainly to the discrete code’s B + 1 possible distances, with sign quantization adding only a small perturbation.Source: §B.1 · p. 18, Table B-4 · p. 18, Table B-3 · p. 18

B.2 QUADRATURE FIDELITY OF THE SCALE ORGANIZATION

  • The four-scale Gauss–Laguerre organization has signal-weighted error of 6.1%–18.6% and τb ≥0.93; at eight scales, error is at most 1.7% and τb ≥0.99.The authors adopt four scales as the smallest configuration exposing the local-to-long-range structure used by SBO.Source: §B.2 · p. 19, Table B-5 · p. 19

C.1 COMPARISON ON NUS-WIDE

  • On NUS-WIDE, CMCL leads MultiBit by 0.07–0.15 pp at 16 and 32 bits, while MultiBit leads by 0.23–0.35 pp at 64 and 128 bits.The comparison uses the same protocol for the fixed-length and multi-length baselines; the table reports I2T and T2I mAP@all.Source: §C.1 · pp. 19–20, Table C-1 · p. 19

C.2 PERFORMANCE–EFFICIENCY MEASUREMENTS

The efficiency analysis separates model complexity, training cost, and online hash-head encoding time. The reported encoding-time mean excludes the backbone and search stages.

C.2 PERFORMANCE–EFFICIENCY MEASUREMENTS

  • 0.0014 ms is MultiBit’s smallest mean encoding time among compared methods, covering only hash heads and binarization.At 128 bits, image and text encoding times are 0.0013 and 0.0015 ms; the shared CLIP backbone, database search, and end-to-end retrieval latency are excluded.Source: §C.2 · p. 20, Table C-2 · p. 20
  • At 128 bits, MultiBit uses 2.71M trainable parameters and 0.0054G FLOPs, with 0.542 h of training, the highest time among these methods.Its best validation checkpoint occurs at epoch 39; Best Epoch denotes the highest-validation-Mean-mAP checkpoint within 100 epochs.Source: §C.2 · p. 20, Table C-2 · p. 20

D.1 COMPONENT ABLATION INTERPRETATION

The ablations report gains from adding FRT, SBO, and MPA, with FRT strongest among the tested relation teachers. Query-level comparisons also show positive mean AP differences for Full MultiBit across all nine comparisons.

D.1 COMPONENT ABLATION INTERPRETATION

  • Mean mAP increases monotonically as FRT, SBO, and then MPA are added in each of twelve settings, with complementary rather than interchangeable gains.The increments are small in several MIRFlickr-25K and IAPR TC-12 settings and larger on MS-COCO.Source: §D.1 · p. 20
  • Short-code SBO gains reflect scale-specific supervision; bit reallocation between scales takes effect only when multiple subblocks are combined.The first 16, 32, and 64 bits lie within the first subblock, so only the 128-bit code combines scales.Source: §D.1 · p. 20
  • FRT attains the highest Mean mAP in all twelve dataset–length settings, with its margin over the strongest remaining variant growing from 0.02–0.44 pp at 16/32 bits to 0.22–1.32 pp at 64/128 bits.The teacher comparison holds the backbone, SBO, MPA, and training schedule fixed; FRT’s longest-code margin is widest on MS-COCO.Source: §D.1 · p. 20, Table D-1 · p. 21

D.2 QUERY-LEVEL GAIN STATISTICS

  • Full MultiBit has positive mean ∆AP in all nine query-level comparisons, ranging from 0.03 to 1.18 pp; 70.13% improve against the single-scale teacher on MS-COCO.Figure 4 plots the cumulative distribution of ∆AP by dataset and alternative design, with query groups of 1,500 comparisons.Source: §D.2 · p. 21, Table D-2 · p. 21

E.1 RELEVANCE RESOLUTION EVALUATION

This section compares relation-teacher variants and evaluates whether MultiBit preserves relevance resolution across code lengths. MultiBit leads the compared multi-length methods on the reported resolution metrics, while FRT also leads alternative teachers under both evaluators.

E.1 RELEVANCE RESOLUTION EVALUATION

  • FRT’s four-scale weights follow from one order parameter via Gauss–Laguerre quadrature, unlike independently trained mixture weights.The compared variants also include binary, Jaccard, single-scale graph, and uniform multi-scale relations.Source: §E.1 · p. 22
  • FRT has the highest NDCG@100 under both evaluators on all three datasets, leading the learned-weight mixture by 0.008–0.011 on every dataset.Under matched normalization, FRT leads UKM by 0.005–0.014 under the IDF evaluator and 0.006–0.012 under the semantic evaluator.Source: §E.1 · p. 22, Table E-1 · p. 22, Table E-2 · p. 23
  • MultiBit has the highest NDCG@100 and Kendall’s τb and the lowest collision and inversion rates in all 21 dataset–method rows across three code lengths.The comparison covers all evaluated multi-length methods; Table E-3 reports the IDF-weighted evaluator results.Source: §E.1 · p. 22, Table E-3 · p. 23
  • VH-ABL exceeds CMCL in NDCG@100 in all nine settings, but its averaged mAP@all is 2.7–3.6 pp lower and up to 6.9 pp lower at 16 bits on MS-COCO.The section attributes this difference to NDCG@100 evaluating the ranking top while mAP integrates over the full ranking length.Source: §E.1 · pp. 22–23, Table E-3 · p. 23
  • NDCG@100 uses the full retrieval database, while collision, inversion, and Kendall’s τb use the candidate pool formed with the full model.The protocol averages query metrics separately for I2T and T2I and reports their arithmetic mean in the resolution tables.Source: Table E-3 · p. 23, §E.1 · p. 24

E.2 DECOUPLED SEMANTIC EVALUATOR

The section tests MultiBit with a label-name semantic evaluator, examines sensitivity to FRT hyperparameters, and describes the cross-length ranking evaluation. MultiBit remains first under the semantic evaluator, while the reported sensitivity grid peaks at dataset-specific settings.

E.2 DECOUPLED SEMANTIC EVALUATOR

  • MultiBit has the highest NDCG@100 and Kendall’s τb and the lowest collision and inversion rates in all 21 dataset–method rows under the semantic evaluator.It is also best on both no-shared-label strata across all three datasets, including pairs in the top quartile of label-graph similarity.Source: §E.2 · p. 24, Table E-4 · p. 25, Table E-5 · p. 25
  • The IDF-weighted and semantic reference vectors have query-level τb values of 0.728, 0.588, and 0.655 on MIRFlickr-25K, IAPR TC-12, and MS-COCO, respectively.The references are correlated but not interchangeable, yet both rank MultiBit first on all four metrics in every dataset–length setting.Source: §E.2 · p. 25

E.3 FRT HYPERPARAMETER SENSITIVITY

  • The hyperparameter grid peaks at (η, α) = (0.20, 1.20) on MIRFlickr-25K and (0.20, 0.90) on IAPR TC-12, MS-COCO, and NUS-WIDE.Nearby combinations remain competitive, while edge values are generally lower.Source: §E.3 · p. 25, Table E-6 · p. 26

F.1 MULTI-LENGTH EVALUATION

  • Cross-length agreement compares rankings for the same query and candidate identities using tie-corrected Kendall’s τb; the evaluation reports the 16–128 pair and the six-pair mean.The described configurations include Naive Prefix, MultiBit-Sep, MultiBit without MPA, and the full model; τb = 1 denotes identical pairwise ordering.Source: §F.1 · p. 26

F.2 FOUR-PANEL RANK-DISPLACEMENT VISUALIZATION

The visualization compares candidate ranks at 16 and 128 bits, while the accompanying evaluation measures agreement across code lengths. MultiBit has the strongest reported cross-length agreement, and removing MPA reduces that agreement.

F.2 FOUR-PANEL RANK-DISPLACEMENT VISUALIZATION

  • Figure 6 plots normalized candidate rank at 16 bits against rank at 128 bits, with shading for candidates in the 16-bit top 20% and contours for empirical density.The panels show five I2T queries with 500 candidates each per dataset–method combination; the annotated values are sample values, not the full-evaluation agreement.Source: §F.2 · p. 27, Table F-1 · p. 27
  • A rank displacement of zero means perfect rank preservation, while larger absolute displacement indicates greater change in candidate ordering.The figure’s mean absolute displacement and tie-corrected Kendall’s τb summarize rank changes and agreement, respectively.Source: §F.2 · p. 27
  • MultiBit has the highest Kendall’s τb for every dataset and retrieval direction on both the 16–128 pair and the six-pair mean.Without MPA, the six-pair mean falls from 0.855 to 0.820 on MIRFlickr-25K, 0.817 to 0.801 on IAPR TC-12, and 0.839 to 0.833 on MS-COCO.Source: §F.2 · pp. 27–28, Table F-2 · p. 27

Related papers

Loading 2609.34393v1…