PaperlayerView exact revision on arXiv

Source-linked AI summary

Disentangling Paradigm, Identifier, and Decoding in Generative Retrieval

Hicham Randrianarivo, Logan Renaud, Alexia Allal

arXiv:2610.08716v1cs.IRcs.CL

TL;DR

Generative-retrieval comparisons of autoregressive and diffusion models vary identifiers, training settings, and decoding together, making paradigm attribution unclear. This paper crosses those factors on shared trained models and tests one-pass scoring for diffusion retrieval. Decoding substantially shifts diffusion Hit@1, and one-pass scoring nearly always matches or beats generate-and-match, while autoregressive models still lead in Hit@1.

  • Problem

    Recent comparisons change identifiers, training settings, and decoding alongside the generation paradigm, so Hit@1 differences cannot be credited to paradigm alone.

    Source: §1 · p. 1

  • Method

    The study crosses paradigms and identifiers on fixed-budget models, then evaluates several decodings, including one-pass corpus scoring for diffusion retrievers.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    One-pass scoring matches or beats generate-and-match in 11 of 12 settings, while autoregressive beam remains ahead of every diffusion decoding in Hit@1.

    Source: §5.2 · pp. 4–5

  • Takeaways & Limitations

    Decoding choice materially affects diffusion Hit@1, so paradigm comparisons need to account for decoding alongside the model and identifier.

    Source: §1 · p. 1

    The paradigms use different backbones and training recipes, leaving recipe and paradigm partly entangled.

    Source: §6 · p. 6

Abstract

from arXiv · show

Generative retrieval trains a language model to generate the identifier of a relevant document. Recent work replaces the autoregressive decoder with diffusion, but changes identifiers, training recipe and decoding at once, so differences cannot be credited to the paradigm. On NQ320K and MS300K, we train autoregressive, masked-diffusion and block-diffusion models with residual-quantised, product-quantised and random identifiers. With identifier length and training budget fixed, we decode each model in several ways. Decoding alone moves a diffusion model's Hit@1 by 6.6 to 13.7 points. Our reference diffusion decoding, generate-and-match, generates an identifier, then retrieves the closest corpus identifiers. The generated identifier is right for 14-21% of NQ320K queries. We test one-pass scoring to decode diffusion retrievers: the model reads a fully masked identifier once, and each document is scored by its codes' probabilities. It matches or beats generate-and-match in 11 of 12 settings. Autoregressive models still lead in Hit@1; on NQ320K, the lead comes from the model, not beam search. Starting from one sampled identifier, one-pass scoring removes 46-83% of masked diffusion's deficit to beam search; from generate-and-match, at most a quarter. On NQ320K, every paradigm largely memorises which identifier answers which query: random identifiers keep 83-90% of the Hit@1 of residual-quantised ones. There, product-quantised identifiers lead residual-quantised ones by 3.4 points in the autoregressive model and by -0.7 to +3.6 in diffusion models; across decodings, AR's gap exceeds diffusion's by 1.5-2.3 points, around our 2-point threshold. Paradigm comparisons must report each paradigm at its own recipe and best decoding.

1 Introduction

Recent diffusion comparisons change identifiers, training settings, and decoding alongside the generation paradigm, so Hit@1 differences cannot be attributed to paradigm alone. This study crosses these factors on fixed trained models and finds that decoding alone substantially moves diffusion Hit@1.

1 Introduction

  • The study crosses paradigm, identifier, and decoding on NQ320K and MS300K while fixing docid length, vocabulary, and training budget.Source: §1 · p. 1
  • 6.6 to 13.7 points: decoding alone moves a diffusion model’s Hit@1 in the controlled comparisons.Source: §1 · p. 1
  • One-pass scoring applies a parallel-decoding recommender rule to diffusion retrieval and matches or beats generate-and-match in 11 of 12 settings.It scores every corpus docid in one forward pass on a fully masked identifier.Source: §1 · p. 1

2 Related Work

Prior work studies docid construction, diffusion retrieval, or decoding, but does not cross generation paradigm with identifier under controlled conditions. This paper varies the factors on shared trained models and tests one-pass scoring for diffusion retrievers.

2 Related Work

  • Generative retrieval work spans text and semantic docids, but earlier docid-construction studies held the paradigm fixed [8].Source: §2 · p. 1
  • Related diffusion-language-model comparisons depend on decoding, including reported advantages linked to inaccurate sampling.Diffusion models can have lower per-token error yet higher whole-sequence error.Source: §2 · p. 2
  • Earlier diffusion-retrieval and recommendation comparisons change codes, recipes, and decoders together, limiting attribution of gains to paradigm [49] [32] [22] [33].Source: §2 · p. 2
  • One-pass per-item scoring was used in parallel-prediction recommendation; this study applies it to a diffusion denoiser without an extra head, beam, or denoising.Source: §2 · p. 2
  • Prior controlled analyses focus on autoregressive retrieval, and none crosses generation paradigm with identifier at a shared model scale, corpus, and budget.Source: §2 · p. 2

3 Design Space

The design varies docid construction, generation paradigm, and decoding while holding trained models fixed for decoding comparisons. It tests semantic and random identifiers, autoregressive and diffusion models, and corpus-wide scoring as a diffusion decoding.

3 Design Space

  • RQ encodes residuals from coarse to fine, PQ quantises embedding slices independently, and random codes have no semantic meaning.Reversing or permuting levels preserves codes and collisions but removes their coarse-to-fine order.Source: §3 · p. 2
  • AR commits one docid level at a time, block diffusion commits blocks left to right, and MDLM treats all levels as one block.The backbones differ: AR uses pretrained T5, while MDLM and BD3-LM share a DiT architecture.Source: §3 · pp. 2–3
  • The diffusion models receive the query as a clean prefix while docid levels are independently masked; block diffusion also conditions on clean earlier blocks.Source: §3 · p. 3
  • Training learns docid probabilities from query–docid pairs using each paradigm’s own loss; block diffusion splits identifiers into blocks with separate noise levels.With one block, the diffusion loss trains MDLM.Source: §3 · p. 3
  • AR retrieval uses trie-constrained beam search, while diffusion decoding ranges from sampling and argmax to matching generated codes against corpus identifiers.Generate-and-match applies argmax, code matching, then a rerank; best-of-8 pools nearest documents from stochastic samples.Source: §3 · p. 3
  • One-pass scoring skips generation and ranks the corpus using per-level probabilities from a fully masked identifier; each document requires L table lookups.For MDLM, this score is the t=1 term of its likelihood bound; for block diffusion, it is not.Source: §3 · p. 3

4 Experimental Setup

The experiments use NQ320K and MS300K, fixed-length code identifiers, and a common example budget, then evaluate retrieval with Hit@k. Analyses also define query exposure and miss categories, while the reported reference comparison uses different decoding procedures for AR and diffusion.

4 Experimental Setup

  • NQ320K contains 320k Wikipedia query–document pairs, with unseen-document gold queries reported separately; MS300K uses 320k documents and has no unseen documents.Source: §4 · p. 3
  • Docids use 16 levels of 512 codes; RQ and PQ quantise GTR-T5-base embeddings, while random codes use the same seed across corpora.Source: §4 · p. 3
  • Models train on 16.4M examples, and Hit@k measures the share of queries with the gold document in the top k.The setup evaluates final rather than EMA weights; paired 95% bootstrap intervals resample queries.Source: §4 · pp. 3–4
  • The reference Table 1 reports Hit@1 with trie beam for AR and generate-and-match for diffusion, and marks non-converged cells in grey.Its caption says AR leads or ties every learned-code column; on NQ320K, random codes retain most paradigms’ Hit@1.Source: Table 1 · p. 4
  • The analyses define paradigm–identifier interaction beyond ±2 Hit@1 points and classify misses by duplicate, code, topical, or unrelated status.They also measure gap closed relative to AR beam and whether a miss’s gold remains among reranked candidates.Source: §4 · p. 4

5 Results

Across five results subsections, the study compares retrieval paradigms under controlled identifiers and decoding, showing that decoding, identifier choice, and query exposure shape outcomes. Diffusion gains substantially from stronger decoding, while autoregressive models lead in Hit@1 and diffusion reaches farther at ranking depth.

5.1 What Each Paradigm Learns

  • 83–90%: on NQ320K, random identifiers retain this share of RQ Hit@1, while on MS300K they retain less than half; AR leads or ties every learned-code cell.The authors give AR’s first- and last-level accuracy, training outputs per example, and backbone size and pretraining as possible reasons for its lead.Source: §5.1 · p. 4
  • Hit@1 rises with the gold document’s training-query count in every cell; content matters most when queries are scarce, and AR’s lead and PQ–RQ gap on NQ320K come from seen documents.For one training query, random codes reach 4% in every diffusion model versus 19–22% for PQ; above twelve queries, they catch up.Source: §5.1 · p. 4

5.2 What Diffusion Generates and How to Decode It

  • 14–21%: raw diffusion generations exactly match the gold identifier on NQ320K; matching and reranking raise Hit@1 to 37–45%.Wrong generations differ at eight or more of 16 levels in 54–60% of cases, so matching against corpus identifiers carries much of Hit@1.Source: §5.2 · p. 4
  • 6.6–13.7 points: diffusion Hit@1 rises from one sample to its best decoding, yet no diffusion setting reaches AR beam; one-pass matches or beats generate-and-match in 11 of 12 settings.On NQ320K, MDLM one-pass ties AR beam at Hit@10 on PQ and leads at Hit@100; it also retrieves 41% of unseen documents against 21%.Source: §5.2 · pp. 4–5, Table 2 · p. 4

5.3 Does the Identifier Favour a Paradigm?

  • On NQ320K, AR’s PQ–RQ advantage is 3.4 points, versus −0.7 to +3.6 for diffusion; the AR–diffusion gap difference ranges from 1.5 to 2.3 points across decodings.On MS300K, point estimates instead favor PQ under AR beam and RQ under diffusion chain-rule rescoring.Source: §5.3 · p. 5
  • After a miss, the gold PQ document remains among candidates about 2.3 times as often as the gold RQ document when generation first diverges within four levels.The paper attributes recovery to the identifier rather than the paradigm; PQ’s lead persists when scoring the document, though it shrinks.Source: §5.3 · pp. 5–6

5.4 Where Misses Land

  • Across learned-code cells, the four paradigms share one profile of miss causes; on NQ320K, an oracle over eight cells reaches 62.6% Hit@1 versus 48.8% for the best cell.On MS300K, the corresponding figures are 49.1% and 24.9%.Source: §5.4 · p. 6

5.5 A Recipe for Diffusion GR

  • For diffusion retrieval, the recommended recipe is paradigm-specific training followed by one-pass scoring; block diffusion on RQ can also benefit from chain-rule rescoring on the top 20.That rescoring adds 0.6–1.8 points on both corpora but costs MDLM 3.1 points on PQ; comparisons should report each paradigm’s best decoding and seen/unseen Hit@1 where applicable.Source: §5.5 · p. 6

6 Limitations and Conclusion

The conclusions are bounded by choices in model scale and pretraining, training recipes, data and identifiers, and decoding comparisons. The study reports findings and future directions within those bounds.

6 Limitations and Conclusion

  • AR’s T5 is pretrained on about twenty times more text than the diffusion models, and its lead may partly come from pretraining; model sizes are comparable but not matched.Models are kept below 500M parameters, so whether the ranking holds at larger scales remains open.Source: §6 · p. 6
  • Because AR’s recipe collapses diffusion, each paradigm uses its own training recipe, leaving recipe and paradigm partly entangled.Source: §6 · p. 6
  • RQ3 covers one code shape and numeric docids, while RQ1 and RQ3 rely on NQ320K; random codes do not converge within budget on MS300K.On MS300K, identifier gaps change sign across decodings.Source: §6 · p. 6
  • One-pass scoring is compared only with generative decodings, its cost grows with corpus size, and why MDLM loses 3.1 points on PQ under chain-rule rescoring remains open.Source: §6 · p. 6
  • On NQ320K, random codes retain 83–90% of RQ’s Hit@1 in every paradigm, while the identifier separates AR from diffusion by 1.5 points at each paradigm’s best decoding, within the authors’ threshold.Source: §6 · p. 6
  • The authors identify two next steps: search the one-pass L×K code-score table without scanning the corpus, and test whether training that reaches documents without training queries changes the paradigm ranking.Source: §6 · p. 6

A Data, Identifiers and Settings

The appendix specifies grid-cell settings, corpus construction and evaluation, and identifier-entropy measures. It also defines the table’s prefix-group and collision quantities.

A Data, Identifiers and Settings

  • Table 3 reports settings for every grid cell; learning rates swept on RQ codes are reused for PQ and random codes.All cells use mixed bfloat16, dropout 0.1, one RTX 6000 Ada, and final weights; MDLM’s EMA is unused.Source: Table 3 · p. 9
  • Table 4 defines unseen documents as those gold for no training query and describes MS300K as a DDRO-script subset with a DDRO dev set.The corpus uses title and content truncated to 4,000 characters; its evaluation set is not the standard 5,193-query dev set.Source: Table 4 · p. 9
  • Table 5 defines identifier entropy, collision rate, and U@k, and reports that MS300K RQ has 499 live level-1 codes.Its 16×512 identifiers have 1-based levels, with level 1 coarsest; PQ has about 8.96 bits at every level.Source: Table 5 · p. 9

B Query Conditioning

All three paradigms attend to the whole query, but use different attention patterns over docid positions. The figure contrasts causal generation, full attention, and blockwise diffusion.

B Query Conditioning

  • AR cross-attends to the encoded query and uses causal attention over the docid.Source: Figure 6 · p. 10
  • MDLM places the query and partly masked docid in one sequence with full attention.Source: Figure 6 · p. 10
  • With block size 2, block diffusion’s doubled stream lets each noisy docid block attend to itself, the clean query, and clean earlier blocks.The figure identifies the doubled stream as BD3-LM [1].Source: Figure 6 · p. 10

C What the Models Learn

The tables describe per-token accuracy under matched contexts and NQ320K Hit@k by whether a gold document was seen during training. They also specify the decodings and generated-query condition used for the latter comparison.

C What the Models Learn

  • On training queries, AR is near 100% at every level, while diffusion PQ accuracy rises from 88–90% at level 1 to 90–97% at level 16 and RQ falls from 88–89% to 79–85%.Table 6 defines level 1 as prediction from the query alone and level 16 as prediction with the other 15 levels given; diffusion ranges over MDLM, BD8, and BD4.Source: Table 6 · p. 11
  • Table 7 splits NQ320K Hit@k by seen gold documents (6,075 queries) and unseen ones (1,755 of 7,830), using each paradigm’s best decoding.Those decodings are AR trie beam, MDLM one-pass scoring, and BD8 and BD4 chain-rule rescoring.Source: Table 7 · p. 11
  • Adding g10 generated queries per training document at the same 16M examples yields paired changes against MDLM above of −13.5 [−14.8, −12.1] for seen and +1.1 [+0.2, +2.1] for unseen.The caption says real queries are seen about 4.5× less often in this condition.Source: Table 7 · p. 11

D Generation and Decoding

The section compares decoding methods on fixed models and reports supporting measurements for block-diffusion generation and rescoring.

D Generation and Decoding

  • Table 8 compares AR’s trie beam and diffusion’s generate-and-match on the same trained models.Source: Table 8 · p. 12
  • Figure 7 reports raw-generation exact match against Hit@1 after code matching and reranking, wrong-level counts, and per-token accuracy under two samplers.Source: Figure 7 · p. 12
  • Table 9 reports NQ320K masked per-level accuracy, wider-shortlist MDLM rescoring, alternate chain-rule orders, and associated Hit@1 changes.Source: Table 9 · p. 12

E Error Taxonomy

The section defines a rank-1 miss taxonomy and describes groups of documents sharing a code.

E Error Taxonomy

  • Table 10 separates outcome by gold-document rank from causes of rank-1 misses, where the first matching cause applies.Source: Table 10 · p. 13
  • Rank-1 miss causes are near-duplicate of the gold (at most 2 of 16 levels wrong), topical, or unrelated.Source: Table 10 · p. 13
  • Table 11 lists largest shared-code groups by size, title/body shares, mean pairwise TF-IDF cosine (random pairs in parentheses), and topic.Source: Table 11 · p. 13
Loading 2610.08716v1…