PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Multiresolution Relevance for Hierarchical Generative Retrieval

Weihao Shen, Wei Chen, Fuwei Zhang, Guojun Liu, Qingsong Hua, Wei Lin, Fuzhen Zhuang

arXiv:2609.39312v1cs.IR

TL;DR

Relevant documents may share SID prefixes and diverge at later refinements, while full-SID supervision treats them as separate paths. RARS derives consistent conditional relevance targets across the hierarchy and trains a shared query representation to predict branch allocations. It improves over matched training and supervision controls across three ESCI locales, with gains also reported across SID constructions and relevance definitions.

  • Problem

    Full-SID supervision treats relevant documents as separate paths, leaving their relevance allocation among competing branches at shared prefixes implicit.

    Source: §1 · pp. 1–2

  • Method

    RARS aggregates document relevance over SID prefixes and uses parent-mass-weighted, prefix-conditioned prediction to supervise branch allocations through a shared query representation.

    Source: §1 · p. 2, §4 · p. 4

  • Results

    RARS improves over matched full-SID training under autoregressive decoding across three ESCI locales and outperforms grouped soft-target, decoder soft-target, and sampled-tree controls.

    Source: §5.2 · pp. 7–8

  • Takeaways & Limitations

    Gains persist across alternative SID constructions and training relevance definitions, including Exact-only supervision.

    Source: §5.2 · p. 8, §6 · p. 10

    Incomplete relevance judgments can obscure branching ambiguity: under subsampling, many queries appear resolved despite spanning multiple branches in the full observed set.

    Source: §5.2 · p. 8

Abstract

from arXiv · show

Generative retrieval with semantic identifiers (SIDs) makes successive decisions over a document hierarchy. Relevant documents for the same query may share coarse prefixes and diverge at finer depths, with branching patterns varying across queries. These paths reveal how relevance is distributed across successive refinements, yet standard full-SID supervision treats them as separate training targets. To make this allocation explicit, we formulate multiresolution relevance as consistent conditional distributions induced by a single document-level relevance measure across the SID hierarchy. We introduce \textbf{RARS}, \textbf{R}esolution-\textbf{A}ligned \textbf{R}elevance \textbf{S}upervision, which uses the resulting refinement-level distributions to supervise a shared query representation. RARS aggregates document relevance over prefixes and trains a prefix-conditioned predictor to allocate relevance among sibling branches. All relevance-bearing children participate in local competition, and each local loss is weighted by the relevance mass reaching its parent. This objective trains the query encoder to capture both the coarse structure shared by relevant documents and their finer branch allocations. The predictor is discarded after training, preserving standard autoregressive retrieval at inference. Experiments on three multilingual ESCI locales show consistent improvements over matched full-SID training under autoregressive decoding. RARS also outperforms grouped soft-target, decoder soft-target, and sampled-tree supervision under a common retrieval rule. The gains persist across alternative identifier structures and relevance definitions. Code is available at: https://github.com/Nevaeh7/RARS

1 INTRODUCTION

Full-SID supervision treats relevant documents as separate paths, leaving their shared-prefix branch allocations implicit. RARS makes these multiresolution allocations explicit through consistent relevance targets and improves retrieval across three ESCI locales and several supervision controls.

1 INTRODUCTION

  • Relevant documents can share coarse SID prefixes and diverge later, but full-SID training treats them as separate paths, leaving shared-parent relevance structure implicit.A query may concentrate relevance on one branch at one depth and divide it among several children at the next.Source: §1 · pp. 1–2
  • A single document-level relevance measure induces consistent conditional targets across SID refinements by aggregating relevance over nested prefixes.Parent mass equals the total mass of its children, and conditioning child mass on the parent yields local refinement distributions.Source: §1 · p. 2
  • RARS uses a shared query representation and prefix-conditioned child scores to learn relevance allocations across supported refinements.It retains relevance-bearing branches, weights each local loss by the relevance mass reaching its parent, and includes competing legal siblings in local normalization.Source: §1 · p. 2
  • Controlled experiments on three multilingual ESCI locales show gains over grouped soft-target, decoder soft-target, and sampled-tree supervision.Further comparisons report improvements under autoregressive retrieval and across alternative SID constructions and relevance definitions.Source: §1 · pp. 2–3

2 RELATED WORK

Related work spans structured identifiers, relevance-aware supervision, and hierarchical prediction. RARS focuses on using document relevance to supervise local decisions across successive SID refinements.

Generative retrieval and structured identifiers

  • Structured identifiers organize documents into progressively finer codes, while prior product-search methods incorporate category structure, relevance, or query-conditioned tries.RARS builds on CaLIR and addresses how document-level relevance directly supervises refinement decisions at multiple retrieval resolutions.Source: §2 · p. 3

Relevance-aware supervision

  • Prior relevance-aware methods use rankings, prefix preferences, graded relevance, feedback, distillation, or multiple positives, while RARS allocates document relevance over SID prefixes and child branches.Its projections preserve relevance mass across depths, and shared query representations learn conditional allocations across relevant paths.Source: §2 · p. 3

Hierarchical prediction

  • Hierarchical prediction methods factorize paths or incorporate tree structure into probabilities and losses, whereas RARS uses the SID hierarchy to supervise query encoders while retaining the full-SID decoder for retrieval.RARS also uses refinement entropy to characterize observed relevance across the hierarchy.Source: §2 · p. 3

3 PRELIMINARIES

Semantic identifiers form nested document partitions, and autoregressive retrieval resolves a query through successive SID refinements. Standard full-SID supervision trains on complete relevant paths, which can share prefixes before diverging.

Semantic identifiers

  • A semantic identifier is a length-L code whose prefixes form nested corpus partitions, from coarse document groups to finer distinctions.The SID trie defines valid prefixes at each depth and valid child branches beneath each prefix.Source: §3 · p. 3

Autoregressive generative retrieval

  • Autoregressive retrieval factorizes a document SID into successive decisions conditioned on the query and the previously generated prefix.Trie-constrained decoding excludes invalid continuations, so each prefix decision determines which documents remain reachable.Source: §3 · pp. 3–4

Full-SID supervision

  • Full-SID supervision treats each relevant query–document pair as a target path, and multiple paths may share a prefix before diverging at a later refinement.These paths specify different child branches below the same parent.Source: §3 · p. 4

4 RARS: RELEVANCE ACROSS RETRIEVAL RESOLUTIONS

RARS projects one document-level relevance measure onto nested SID partitions, then trains a shared query representation to predict local relevance allocations. Parent-mass weighting aligns refinement-level supervision while the full-SID decoder remains in use for inference.

4.1 RELEVANCE ACROSS IDENTIFIER RESOLUTIONS

  • Nested SID partitions turn one document-level relevance measure into consistent conditional allocations across depths, while a refinement-entropy profile describes where relevance branches.The profile depends on observed relevance and the SID hierarchy, not model predictions.Source: §4.1 · p. 4

4.2 QUERY-LEVEL HIERARCHICAL PREDICTION

  • A shared query state and depth-specific projections score children conditioned on their parent prefix, normalizing over relevance-bearing children and competing legal siblings.Local normalization avoids a softmax over the full identifier space.Source: §4.2 · p. 5

4.3 RESOLUTION-PRESERVING OBJECTIVE

  • The resolution-preserving objective matches projected conditional relevance distributions with local KL losses weighted by the relevance mass reaching each parent.All relevance-bearing children participate; the full-SID decoder and retrieval procedure retain their original objectives.Source: §4.3 · p. 5

4.4 SUPERVISION AT BRANCHING PREFIXES

  • Explicit distributional targets match relevance-weighted target sampling in expected local gradient while removing target-draw variance, coupling shared-query learning across coarse prefixes and finer relevant branches.The variance term is Gini impurity, whereas the refinement ambiguity uses Shannon entropy.Source: §4.4 · p. 6

4.5 INFERENCE AND RETRIEVAL SCORING

  • At inference, autoregressive identifier scoring uses the top-C category-admitted set, but category probabilities do not enter identifier scores.A compatibility-augmented beam rule adds compatibility only to legal SID continuations; matched scoring rules distinguish decoding effects from training gains.Source: §4.5 · p. 6

5 EXPERIMENTS

Experiments on three ESCI locales compare RARS with matched training controls and evaluate robustness across relevance definitions and SID structures. RARS improves retrieval under standard autoregressive decoding, with further analyses examining evidence coverage and prediction alignment.

5.1 EXPERIMENTAL SETUP

  • The evaluation uses category-filtered US, ES, and JP ESCI subsets with graded query–product judgments and matched retrieval settings.Exact, Substitute, and Complement judgments receive training weights 3, 2, and 1, normalized per query; evaluation retains the original graded judgments.Source: §5.1 · pp. 6–7

5.2 EXPERIMENTAL RESULTS AND DISCUSSION

  • Across all three locales, RARS improves Recall@100 and NDCG@10 over matched Base models under autoregressive retrieval; all-level scoring further improves both variants.The matched comparisons fix the backbone and decoding settings; Table 1 reports five-seed means.Source: §5.2 · p. 7, Table 1 · p. 7
  • Under a common all-level retrieval rule, explicit conditional targets outperform sampled one-hot targets, while RARS also gains over grouped soft-target supervision on both primary metrics across locales.The comparisons distinguish target aggregation, prediction site, and target stochasticity.Source: §5.2 · pp. 7–8, Table 2 · p. 8
  • At 25% judgment retention, more than half of queries appear resolved at L1 despite spanning multiple full-set branches, while roughly 70% of L4 relevance mass lacks retained support.Observed ambiguity is common at coarse and intermediate depths and becomes less prevalent toward the leaves; target fidelity therefore depends on judgment coverage.Source: §5.2 · p. 8, Figure 3 · p. 8
  • Across tested SID constructions and training relevance definitions, RARS improves both primary metrics over matched Base, including with Exact-only training.The tested constructions include hierarchical k-means and a deeper RQ-VAE; richer relevance sets yield larger gains, but their source is not isolated.Source: §5.2 · p. 8, Table 3 · p. 8
  • Under fixed model parameters, candidate set, retrieval budget, and normalized graded-relevance targets, all-level scoring reduces Jensen–Shannon divergence and assigns more probability mass to relevant children than AR-only scoring.Source: §5.2 · p. 8
  • Examples show RARS improving rankings for relevant products that share prefixes or diverge across coarse SID branches, while distinguishing relevance grades.Cases include promoting an Exact product over a Substitute and retrieving a previously missed Exact item.Source: §5.2 · p. 9, Figure 6 · p. 9

6 CONCLUSION

The conclusion presents RARS as supervision for relevance allocation across SID refinements, with its auxiliary predictor removed at inference. It reports gains across ESCI controls and SID and relevance choices, plus supplementary NQ320K retrieval results.

6 CONCLUSION

  • RARS supervises relevance allocation from coarse prefixes to finer SID branches, then discards its auxiliary predictor to retain standard autoregressive retrieval.The conclusion also reports consistent ESCI gains over matched controls across SID constructions and relevance definitions, including Exact-only supervision.Source: §6 · p. 10
  • On NQ320K, controlled retrieval-scoring comparisons show strong gains, including for relevant Wikipedia documents absent from training annotations.The conclusion identifies the comparison as further evidence of retrieval-scoring effectiveness on Wikipedia documents.Source: §6 · p. 10

A EXPERIMENTAL SETUP AND IMPLEMENTATION

A.1 DATASETS AND EVALUATION METRICS

  • Eligible ESCI test sets contain 6,014 US, 1,656 ES, and 1,883 JP queries.The datasets are category-filtered US, ES, and JP subsets.Source: §A.1 · p. 13

A.2 BASELINES

  • Table 1 includes sparse, dense, and generative retrieval baselines; the matched Base is CaLIR with full-SID likelihood, category classification, and multi-positive category alignment.The Base rows distinguish autoregressive scoring from compatibility-augmented scoring.Source: §A.2 · pp. 14–15

A.3 IMPLEMENTATION AND OPTIMIZATION

The implementation retains CaLIR’s reference objectives and adds RARS supervision through the shared query encoder. Sparse relevance targets drive parent-weighted local prediction, while retrieval scoring uses category-constrained SIDs and compatibility signals.

A.3 IMPLEMENTATION AND OPTIMIZATION

  • RARS adds SID-hierarchy supervision through the shared query encoder to CaLIR’s HSR and QRE objectives, each weighted by 0.1.The reference setup fixes λtree = 0.5; five-seed comparisons use final epoch-300 checkpoints.Source: §A.3 · p. 16

A.4 SPARSE TARGET CONSTRUCTION

  • Sparse targets sum document relevance at every ancestor; the predictor scores local child distributions, and each refinement loss is weighted by the relevance mass reaching its parent.Candidate sets include all relevant children and selected high-scoring sibling negatives; prediction cost is linear in active query-tree edges, apart from hard-negative selection.Source: §A.4 · p. 16

A.5 RETRIEVAL AND COMPATIBILITY SCORING

  • Retrieval admits SID sequences from the top-three category predictions and adds compatibility scores based on query attributes and lexical features.Compatibility is propagated to SID prefixes by taking the maximum item score in each subtree; up to 100 identifiers are generated with beam 100.Source: §A.5 · pp. 16–17

B PROOFS AND DERIVATIONS

The appendices derive properties of hierarchical relevance objectives and report supplementary retrieval and uncertainty analyses. They connect local conditional supervision to full-path matching and compare NQ320K retrieval performance across seen and unseen documents.

B.1 HIERARCHICAL CONSISTENCY

  • Direct projection of leaf relevance to depth j equals successive marginalization from depth L to j.This establishes consistency between projected targets at a given depth and repeated marginalization through the hierarchy.Source: §B.1 · p. 17

B.2 TARGET-INDUCED GRADIENT VARIANCE

  • At fixed local predictions, sampling one-hot targets induces logit-gradient variance around the gradient from the relevance distribution.The derivation identifies the centered gradient as μ − eY.Source: §B.2 · p. 17

B.3 ENTROPY AND TARGET-SAMPLING VARIANCE

  • The target-sampling variance is Gini impurity, bounded above by Shannon entropy in nats.For normalized empirical entropies, the corresponding scale factor is required.Source: §B.3 · p. 17

B.4 CROSS-ENTROPY DECOMPOSITION

  • Because target entropy is model-independent, cross-entropy and KL divergence in Eq. (7) have the same minimizer on target support: pℓ = μℓ.Source: §B.4 · p. 17

B.5 FULL-PATH CONSISTENCY AND GROUPED SUPERVISION

  • With full legal-child normalization, parent-mass-weighted local KL recovers full-path matching; restricted candidate sets instead define a local discriminative objective allocating relevant mass among children.The grouped objective differs from the tree objective by the entropy of the leaf relevance distribution.Source: §B.5 · pp. 17–18

C SUPPLEMENTARY EXPERIMENTAL RESULTS

  • The supplementary experiments examine NQ320K retrieval and vary training objectives, SID and supervision choices, local prediction alignment, and retrieval scoring.Training-run variability is reported separately from query-level uncertainty.Source: C SUPPLEMENTARY EXPERIMENTAL RESULTS · p. 18

C.1 RETRIEVAL EFFECTIVENESS ON NQ320K

  • On the full NQ320K test set, RARS gains 4.65 points in Recall@10, 2.50 in Recall@100, and 1.79 in MRR@100 over the strongest baseline for each metric.RARS leads all three baselines on every reported metric across the full, seen, and unseen test sets; the largest Recall@10 gain is on unseen queries.Source: Table 6 · p. 18, §C.1 · p. 18

C.2 RETRIEVAL EFFECTIVENESS AND STATISTICAL UNCERTAINTY

  • Diagnostic intervals condition on exploratory checkpoint and scoring-weight choices, while paired significance tests compare predictions on the same queries.Training-run statistics instead measure variation across seeds under the fixed training protocol.Source: §C.2 · p. 19

C.3.1 TRAINING OBJECTIVES AND VARIATION ACROSS RUNS

The objective comparison evaluates five training objectives under a common all-level retrieval rule. RARS is compared with matched full-SID, grouped soft-target, decoder soft-target, and sampled-tree supervision.

C.3.1 TRAINING OBJECTIVES AND VARIATION ACROSS RUNS

  • The controls distinguish targets applied to complete-SID likelihoods, autoregressive decoder predictions, or the shared query representation through the RARS predictor.The decoder normalizes over all legal children, whereas RARS uses its local candidate-set objective.Source: §C.3.1 · p. 19
  • RARS improves five-run means of both primary metrics over the matched base, grouped soft targets, and sampled tree.Grouped soft targets weight complete identifiers by query-conditioned relevance mass; sampled tree and RARS share an auxiliary predictor but use one-hot and explicit local-distribution targets, respectively.Source: §C.3.1 · pp. 19–20
  • NDCG@10 gains over grouped soft targets were 0.72, 0.71, and 0.52 percentage points on US, ES, and JP; Recall@100 gains were 0.74, 0.47, and 0.41 points.These are five-run mean gains; Figure 7 summarizes paired within-seed differences.Source: Table 7 · p. 19, Figure 7 · p. 20, §C.3.1 · p. 20

C.3.2 TRAINING AND RETRIEVAL RESOLUTION

RARS improves retrieval under autoregressive decoding, while compatibility scoring adds gains under matched retrieval settings. Its advantage persists across tested SID constructions and training relevance definitions.

C.3.2 TRAINING AND RETRIEVAL RESOLUTION

  • Comparisons use matched Base and RARS models under four retrieval settings, with the auxiliary prediction head absent during retrieval.Comparisons between training conditions use the same retrieval mask; within-model comparisons measure compatibility scoring.Source: §C.3.2 · p. 20
  • RARS raises AR-only Recall@100 by 1.58/0.71/0.70 points and NDCG@10 by 1.24/1.22/0.57 on US/ES/JP.All-level scoring adds 0.88/0.48/0.92 NDCG@10 points for Base and 0.64/0.54/0.89 for RARS; training and compatibility scoring both improve retrieval under matched conditions.Source: Table 8 · p. 20, §C.3.2 · p. 20

C.4 IDENTIFIER AND TRAINING SENSITIVITY

  • Sensitivity experiments vary SID construction, training positives, tree-loss weight, temperature, sibling candidates, and head dimension one component at a time.For identifier geometry and relevance-definition comparisons, Base and RARS use the same data partitions, optimization settings, and five random seeds; evaluation judgments remain fixed across training-positive variants.Source: §C.4 · p. 21, §C.4.1 · p. 21

C.4.1 IDENTIFIER GEOMETRY AND RELEVANCE DEFINITIONS

  • RARS improves all five metrics across tested SID constructions, with NDCG@10 gains of 0.74, 0.88, and 1.00 points for hierarchical k-means, five-level RQ-VAE, and reference RQ-VAE.Under Exact-only, Exact+Substitute, and Exact+Substitute+Complement training evidence, Recall@100/NDCG@10 gains are 0.57/0.49, 0.88/0.73, and 1.38/1.00 points, respectively.Source: Table 9 · p. 21, Table 10 · p. 21, §C.4.1 · p. 21

C.4.2 TRAINING HYPERPARAMETERS AND SIBLING CANDIDATES

The analyses test RARS’s training sensitivity, prediction alignment, retrieval-resolution scoring, and the structure and recoverability of relevance across SID depths. Results show benefits across configurations, alongside limits from ambiguous or incomplete evidence.

C.4.2 TRAINING HYPERPARAMETERS AND SIBLING CANDIDATES

  • Every tested nonzero tree weight improves both primary metrics over λtree = 0, while sibling negatives and larger prediction heads also improve metrics across locales.At a parent with one relevant child, relevant-child-only normalization has zero local gradient; sibling negatives provide competing alternatives there.Source: §C.4.2 · pp. 21–23

C.5 PREDICTION ALIGNMENT AND PREFIX COVERAGE

  • Among ambiguous parents, one standard deviation more refinement ambiguity is associated with a 0.0267 increase in baseline JS divergence after controls.The coefficient for JS reduction is −0.0073, with 95% CI [−0.0120, −0.0026].Source: §C.5 · p. 23
  • Prefix coverage improves significantly at L2–L4 in all three locales; L4 gains are 2.22, 1.40, and 1.13 percentage points on US, ES, and JP.L1 intervals include zero, and L4 coverage equals SID Recall@100.Source: §C.5 · p. 24

C.6 ABLATION OF RETRIEVAL RESOLUTIONS

  • Removing L2 or L3 compatibility terms produces the largest decreases in R@100 and NDCG@100, with significant paired differences across locales.Full scoring achieves the highest values in all three locales; L1 differences are positive in every locale and significant on both metrics in US.Source: §C.6 · p. 24, Table 14 · p. 24

D.1 RELEVANCE ACROSS IDENTIFIER RESOLUTIONS

  • Nonzero refinement entropy appears in 68.8%–78.9% of queries at L1, falling to 8.5%–18.0% at L4; semantic SIDs place more entropy at L3–L4 than controls.Queries with identical E/S/C counts and total SID entropy still show distinct depth profiles.Source: §D.1 · pp. 24–25, Table 16 · p. 25

D.1.1 MEASURES AND CONTROLS

  • Disjoint positive subsets from the same query have lower depth-profile distances than matched queries under binary and graded evidence in all locales.Graded within-query distance reductions are 18.2%, 23.8%, and 28.8%.Source: §D.1.1 · p. 27
  • Branching persists with Exact-only evidence, and fixing positive count preserves early-depth entropy concentration; binary and graded relevance have highly correlated global ambiguity.The binary/graded mean normalized ambiguity difference is at most 0.0031, with correlations of 0.9986–0.9991.Source: §D.1.1 · pp. 26–27

D.2 INCOMPLETE EVIDENCE AND RESOLUTION LIMITS

  • At 25% retention, incomplete judgments can make 54.3%–70.0% of queries appear resolved at L1 while 70.7%–72.9% of L4 relevance mass lacks retained support.With candidate counts matched per query, expected L2 coverage is 2.56%–3.12% for semantic assignments versus 0.17%–0.24% after within-L1 shuffling.Source: §D.2 · p. 27

D.2.1 EVIDENCE SUBSAMPLING AND STATISTICAL ANALYSIS

This analysis subsamples observed relevance evidence and compares how well SID assignments preserve withheld relevance coverage. It finds that semantic assignments retain more fine-level coverage than within-L1 shuffling at matched candidate counts.

D.2.1 EVIDENCE SUBSAMPLING AND STATISTICAL ANALYSIS

  • The design crosses three locales, five retention levels, and three identifier assignments with 20 paired replicates per condition, evaluating four SID depths and individual products.Primary comparisons use linear relevance weights, uniform masking, and 25% retention.Source: §D.2.1 · p. 28
  • Withheld coverage measures withheld positive mass in groups containing a retained positive, while catalog expansion measures the catalog share in those supported groups.Figure 10 plots withheld relevance coverage against catalog expansion across SID depths and identifier assignments.Source: §D.2.1 · p. 29, Figure 10 · p. 29
  • At 25% retention, L4 coverage gains over global assignment are 0.0055/0.0349/0.0065 points on US/ES/JP, with Holm-adjusted p-values of 0.2476/0.0042/0.9969.The primary test family covers TV reduction and coverage gain at all four depths and three locales, with Holm correction over 24 tests.Source: Table 20 · p. 29, §D.2.1 · p. 29
  • At 25% retention, 71%–73% of reference L4 relevance mass belongs to groups with no retained positives.Coverage is reported by positive-count stratum under the semantic hierarchy.Source: Table 21 · p. 30, §D.2.1 · p. 30
  • At matched candidate counts, semantic L2 coverage is 2.64%, 2.56%, and 3.12% in US, ES, and JP, versus 0.17%, 0.23%, and 0.24% after within-L1 reassignment.The shared candidate cap averages 8.12, 19.56, and 15.33 candidates per query in US, ES, and JP.Source: §D.2.1 · p. 31

E REAL-PRODUCT CASE STUDIES

Six product-query examples show that relevance branches at different SID depths and allocates different conditional mass among sibling products. Both hierarchy structure and relevance grades shape the local supervision targets.

E REAL-PRODUCT CASE STUDIES

  • Relevant products branch at query-dependent depths, from the first SID level to the final refinement.Examples include curtain and PSP products separating at L1, bitters at depth 3, and air filters at the final token.Source: E REAL-PRODUCT CASE STUDIES · p. 32
  • Conditional relevance mass depends on grades as well as branching: the binder and refill below a133/b49 receive masses 3/4 and 1/4.The two Complement refills below a126/b136/c35 each receive mass 1/2.Source: E REAL-PRODUCT CASE STUDIES · p. 32, Figure 12 · p. 32
  • SID structure determines which relevant products compete at a refinement, while relevance grades determine their relative local mass.The examples include equal local targets for same-grade products and different targets for products with different grades under one parent.Source: E REAL-PRODUCT CASE STUDIES · p. 32

Related papers

Loading 2609.39312v1…