PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning Reranking with Semantic IDs as the Sole Item Representation

Zhenwei Tang, Jintao Liu, Di Bai, Xidong Wu, Xinwu Cheng

arXiv:2610.22797v1cs.IRcs.CLcs.LG

TL;DR

Whether SID-pretrained models can reason over a supplied slate using only identifiers remains untested, despite SIDs’ compactness and potential behavioral signal. The paper builds a controlled SID-only reranking testbed and compares post-training methods. Its strongest model matches text-reading LLMs with much longer prompts, but its item descriptions remain less accurate.

  • Problem

    Prior SID reasoners generate the next item and listwise rerankers use readable metadata, leaving untested whether pretrained SID tokens support ranking a supplied slate alone.

    Source: §1 · pp. 1–2

  • Method

    RERANKID evaluates rationale-then-ranking from SID-only histories and slates, varying item rendering, history length, and slate size one axis at a time while comparing common post-training methods.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    8B RLVR with a graded ranking reward matches text-reading LLMs with prompts about 20× longer, ranks above random without history where a caption reader does not, and keeps gains at long histories where the text reference stops improving.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The tested post-training methods elicit ranking from SID preference signal and history evidence, but not faithful item descriptions, motivating post-training that also grades reasoning.

    Source: §1 · p. 2, §6 · p. 9

    The study uses the OpenOneRec research release, and deployment of its rankings would require platform-side review of fairness and exposure effects.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Reasoning rerankers write a rationale before ranking a candidate slate, and existing systems read item metadata to do so, even though generative recommenders increasingly represent items as compact Semantic IDs (SIDs). Ranking from SIDs alone has two appeals: SID prompts are far shorter than the matched item text, and although item codes are fixed from content, the backbone learns their token embeddings in part from user-item interaction sequences, so the tokens can carry behavioral signal that item text lacks. That training targets next-item generation, and whether it supports ranking a supplied slate from SIDs alone is untested. We build ReRankID, a controlled testbed in which models reason and rank from a history and slate shown only as SIDs, varying item rendering, history length, and slate size one at a time. On this testbed, such models often copy the input order instead of reranking, and readable metadata does not repair the failure. We compare supervised fine-tuning, rejection-sampled self-training, on-policy distillation, and verifier-based reinforcement learning as post-training methods and find that they elicit the missing behavior. The strongest post-trained model, the 8B model trained with a graded ranking reward, gives the best SID-only ranking and matches strong text-reading LLMs whose prompts are about 20x longer. It ranks above random with no history at all, where a caption-reading reference does not, and it keeps its gains at long histories, where text prompts pass 50k tokens and the caption-reading reference stops improving. In a paired rationale audit, this model argues trade-offs better than that reference but describes items less accurately. The common post-training methods we test thus elicit ranking from SIDs but not faithful item descriptions, which calls for post-training that improves the reasoning itself, not only the ranking it produces.

1 INTRODUCTION

The paper tests whether SID-pretrained models can reason over and rerank a supplied slate using only SIDs, a setting not established by next-item generation or metadata-based reranking. Its testbed exposes frequent input-order copying before post-training, while common post-training methods elicit SID-only ranking, with item-description accuracy still limited.

1 INTRODUCTION

  • The open question is whether SID tokens learned during next-item generation can support ranking a supplied slate, since prior SID reasoners generate items and listwise rerankers use readable metadata.SIDs are compact, and their embeddings can carry preference signal learned from user-behavior sequences that titles and captions do not contain.Source: §1 · pp. 1–2
  • RERANKID tests rationale-then-ranking from SID-only histories and slates, with paired text views, one-axis changes to item rendering, history length, and slate size, and diagnostics for interface failures.Rankings refer to candidate indices, allowing the same users and slates to be rendered as SIDs, text, or both.Source: §1 · p. 2
  • Before post-training, existing models often return the input order unchanged, do not gain sustainably from longer histories, and are not rescued by readable captions.At the anchor setting, both models trail text-view references; SID prompts are roughly 20× shorter than matched text prompts and nearly 28× shorter at the longest history.Source: §1 · p. 2
  • 8B RLVR with a graded ranking reward improves on SFT, matches text-reading LLMs with prompts about 20× longer, ranks above random without history, and keeps gains at long histories where the text reference stops improving.In a paired rationale audit, it argues trade-offs better than the text reference but misstates items it has never seen as text.Source: §1 · p. 2
  • The study compares four common post-training methods under one protocol and finds that the best post-trained model uses information a text reader does not, while its rationale audit finds item descriptions unreliable.The testbed and paired rationale audit support evaluating both ranking and what the model says about candidate comparisons.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work separately studies semantic-ID generation, metadata-based listwise reranking, and post-training for reasoning or ranking. This paper’s related-work framing identifies the untested intersection: reasoning over a supplied slate from identifiers alone.

2 RELATED WORK

  • SID recommender work studies token representations and reasoning for next-item generation, while this paper removes generation to test whether pretrained SID tokens support reasoning over a supplied slate.Prior work notes that SID semantics may not be usable merely by attaching tokens to a language model and that token capacity and decoding structure constrain expression.Source: §2 · p. 3
  • Listwise LLM rerankers order supplied candidates, but the reviewed systems use readable item text and therefore do not test whether identifiers alone support preference reasoning or accurate rationales.Recommendation rerankers are sensitive to history length, candidate-set size, and presentation order (Hou et al., 2024; Bito et al., 2026).Source: §2 · p. 3
  • Prior post-training work includes rejection self-training, on-policy distillation, and verifier-based reinforcement learning, while this study compares SFT, RFT, OPD, and RLVR on a pure-SID task.Related recommendation methods distill rationales or apply rank-aware rewards, but the cited Rec-R1 comparison uses text-based product search.Source: §2 · p. 3

3 THE RERANKID TESTBED

RERANKID provides a controlled way to test whether models can reason and rerank a supplied slate from SIDs alone. It holds users and slates fixed across item renderings and varies history length, slate size, and rendering one axis at a time.

3 THE RERANKID TESTBED

  • The testbed compares SID-only, text-only, and combined item renderings on the same users, slates, and labels, so rendering differences are isolated.Models return a rationale and ten candidate indices, scored by the observed positive’s rank.Source: §3 · p. 4
  • The test set uses 400 exact-one-positive users and nested slates from a fixed priority pool, avoiding ambiguous credit and keeping growing slates comparable.Users are balanced between explicit-engagement and long-view positives; histories use the same nested recent subsequence for each user.Source: §3 · p. 4
  • The evaluation starts from k=64 history items and N=30 candidates, then varies history length, slate size, or rendering one axis at a time.Histories and slates are nested, and evaluation users are disjoint from post-training pools.Source: §3 · p. 4
  • Invalid outputs score zero in end-to-end NDCG@5, so interface failures count against ranking performance rather than disappearing from a valid-only average.The protocol also records identity-order rankings, reasoning loops, validity, and use of a fixed extraction fallback.Source: §3 · p. 4
  • Figure 2 tracks prompt-token counts, NDCG@5 across history lengths, and NDCG@5 across slate sizes, with each intervention varying one axis from the anchor setting.The figure compares 8B Base, its RLVR model, Gemini-3.6, and seeded Random.Source: §3 · p. 4, Figure 2 · p. 5

4 DIAGNOSING THE EXISTING MODELS

Before post-training, existing OpenOneRec models struggle with the slate-ranking interface: they often copy candidate order, and readable captions do not consistently improve ranking. Longer histories provide no sustained benefit despite much shorter SID prompts.

4 DIAGNOSING THE EXISTING MODELS

  • At the anchor setting, the 8B model beats seeded Random from SID-only inputs by +.063 (CI [+.027, +.099]), while the 1.7B margin is within noise; both trail every text-view reference.With no history, 8B Base’s +.035 margin over Random is also within noise, while Gemini is statistically indistinguishable from Random.Source: §4 · p. 6
  • For the 8B model, longer histories do not help: its ranking drifts with no sustained trend; with no history (k=0), SID candidates alone yield a +.035 margin over seeded Random (CI [−.001, +.071]).Source: §4 · p. 6
  • Existing models return the literal candidate order for between a fifth and nearly half of users, through empty-reasoning and history-transcription failures.Readable captions do not consistently improve ranking; the 8B scores stay within a few points across SID, text, and combined views.Source: §4 · p. 6

5 ELICITING SID-ONLY REASONING RERANKING

The paper compares four post-training methods for eliciting SID-only reasoning reranking, then evaluates ranking, history and slate interventions, and rationale quality. The 8B model trained with a graded ranking reward gives the strongest SID-only result, while its rationales remain less accurate at describing items than Gemini’s.

5.1 POST-TRAINING SETUP

  • The four methods differ in supervision: SFT imitates a caption-view hindsight teacher, RFT filters student responses, OPD distills teacher distributions on student trajectories, and RLVR uses a ranking verifier.RFT and RLVR continue from SFT; SFT and OPD start from Base, with training users disjoint from evaluation users.Source: §5.1 · pp. 6–7

5.2 RESULTS

  • 8B RLVR reaches .277 NDCG@5 from SID-only inputs, on par with Gemini and GPT-5.6-Terra text references whose prompts are about 20× longer.RLVR is the only method that improves on SFT, and only at 8B; its paired gains over the text references are +.029 and +.021.Source: Table 2 · p. 7, §5.2 · p. 7
  • SFT eliminates identity permutations and reasoning loops at both scales, with large significant gains over Base; 8B RLVR then adds a significant paired gain over SFT.Across SFT, RFT, and RLVR, extraction fallback is invoked for at most 1.0% of anchor users and adds at most +.004 NDCG@5.Source: Table 2 · p. 7, §5.2 · p. 7
  • At 8B, graded ranking reward beats binary top-one reward by +.045 (CI [+.013, +.077]); at 1.7B, the rewards are indistinguishable.The same RLVR recipe stays within noise of SFT at 1.7B.Source: §5.2 · p. 8
  • RFT matches SFT at 1.7B but loses ground at 8B, while OPD removes interface failures and recovers 73% of the teacher’s gain at 1.7B and 61% at 8B without exceeding it.Every trained model at both scales ranks above the caption-centroid ranker (.184).Source: §5.2 · p. 8
  • As history grows, 8B RLVR’s gains plateau through k=256, while Gemini stops improving as its text prompt exceeds 50k tokens; at N=100, RLVR ends level with Gemini.At the anchor setting, SID prompts average 845 tokens against 16,722 for matched text prompts.Source: §5.2 · p. 8
  • With no history, 8B RLVR beats Random by +.079 (CI [+.039, +.118]); on caption-rendered slates, Gemini is statistically indistinguishable from Random.The no-history prompt contains no user identifier, leaving candidate SIDs as the source of the observed margin.Source: §5.2 · p. 8
  • In the rationale audit, RLVR wins the trade-off aspect 50:0, but loses item-level claims 0:66 and user portraits 0:37; Gemini names nearly all ten ranked candidates accurately.The audit used 100 anchor users and counted only verdicts consistent across both presentation orders.Source: §5.2 · pp. 8–9, Figure 3 · p. 9

6 CONCLUSION

The conclusion finds that post-training elicits SID-only ranking, with 8B RLVR matching text-reading LLMs on much shorter prompts while remaining limited in item description accuracy. It proposes separating SID-based ranking from metadata-based explanations and documents the study’s ethics, AI use, and reproducibility materials.

6 CONCLUSION

  • 8B RLVR ranks well and argues trade-offs better than the text reference, but misidentifies items; the conclusion suggests ranking from SIDs and explaining from retrieved metadata.It also proposes the text rendering as a public evaluation of general LLMs’ modeling of real user behavior.Source: §6 · p. 9

AI USE STATEMENT

  • The authors used generative AI for SFT hindsight rationales, rationale judging, text-reference systems, experimental code, and manuscript drafting, and reviewed the work and take responsibility for it.The AI use statement continues by identifying the authors’ review of scientific claims, data handling, figures, citations, and final manuscript.Source: AI USE STATEMENT · pp. 9–10

ETHICS STATEMENT

  • The study uses the OpenOneRec research release under its access terms, collects no new data, and requires platform-side fairness and exposure review before deployment.Caption text goes to commercial LLM APIs only for text references, teacher annotations, and rationale judging, with no-training and zero-retention settings.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

A BENCHMARK CONSTRUCTION

The benchmark holds users, labels, and candidate indices fixed while comparing matched SID-only, text, and combined item views. It varies history length and slate size separately and uses interaction data and reference systems to frame what the evaluation can test.

A.1 TASK AND RENDERINGS

  • The task gives the last k eligible history items and N candidates, then requires a rationale and an ordered array of ten distinct candidate indices.SID-only uses three-codebook SIDs; text and combined views change item rendering without changing users, items, labels, display order, or index-valued actions.Source: §A.1 · p. 15

A.2 USERS AND SLATES

  • The benchmark uses 400 development and 400 test users drawn from 162,074 source users, with disjoint partitions and histories preceding targets.Users qualify with a renderable positive and at least 256 history items; slates and histories are nested, with anchor k=64 and N=30.Source: §A.2 · p. 15, Table A.1 · p. 15

A.3 LABEL CONSTRUCTION

  • Each slate has one observed positive; competing positives are excluded, while exposure-hard negatives record items shown but not engaged with.All systems rank the same slates under the same labels, and comparisons use paired per-user results.Source: §A.3 · pp. 15–16

A.4 PRETRAINING EXPOSURE

  • The evaluated backbones were pretrained on interaction sequences involving the items, but their training tasks did not include reasoned ordering of candidate slates.Evaluation users are disjoint from post-training users, and the k=0 condition removes history as a lookup key for user-item associations.Source: §A.4 · p. 16

A.5 SCOPE OF THE REFERENCE SET

  • The references anchor different comparisons: Random sets the score scale, Caption is a text baseline, and API systems represent strong general text readers on the same slates.Scoring the same slates keeps the main comparisons paired.Source: §A.5 · p. 16

B.1 METRICS AND INFERENCE

The evaluation scores ranking quality end to end, retaining invalid outputs and documenting the reasoning-to-index interface. The supplied section also describes shared prompts and the setup of SFT and rejection-sampled fine-tuning.

B.1 METRICS AND INFERENCE

  • NDCG@5 is primary, with NDCG@10, Recall@5, Recall@10, and MRR also reported; an example still invalid after the complete inference protocol receives zero, and valid-only averages are never substituted for end-to-end scores.Source: §B.1 · p. 16

B.2 SHARED EVALUATION PROMPT

  • The shared prompt asks models to analyze user interests and candidate fit, then return ten distinct candidate indices ordered by relevance.SID inference permits free reasoning first and invokes a fixed extraction stage only if that response is invalid; failures remain in the end-to-end score.Source: §B.1 · p. 16, §B.2 · p. 18

C POST-TRAINING RECIPES

  • SFT, RFT, OPD, and RLVR share the SID-only task and benchmark, but differ in initialization, supervision, and response source.RFT and RLVR continue from SFT; OPD starts from Base and uses a frozen, scale-matched SFT teacher on student-generated SID trajectories.Source: C POST-TRAINING RECIPES · p. 19

C.1 SUPERVISED FINE-TUNING

  • SFT trains on accepted teacher responses that pair a rationale with an index array, using a text-view hindsight teacher and SID-only student prompts.The SFT source pool has 12,000 users; quality checks leave 11,991 examples, split into 11,031 training and 960 development examples.Source: §C.1 · p. 19

C.2 REJECTION-SAMPLING FINE-TUNING

  • RFT samples eight SID-only responses per prompt and retains at most the first response that passes a verifier for a valid, nonidentity ranking with the positive first.The verifier rejects invalid or incomplete answers, truncations, empty rationales, lower-ranked positives, and identity collapse.Source: §C.2 · p. 22

N Source 1.7B retained 1.7B coverage 8B retained 8B coverage

RFT’s selected 8B checkpoint performs below its SFT initializer on test, while the 1.7B result is nearly unchanged. The supplied OPD material describes its student-teacher procedure and identifies the design dimensions that differ from other recipes.

N Source 1.7B retained 1.7B coverage 8B retained 8B coverage

  • At 8B, RFT reaches .211 on test against .237 for SFT, a paired difference of −.026 with 95% CI [−.047, −.005].At 1.7B, RFT reaches .223 against .225 for SFT, a paired difference of −.002 with 95% CI [−.028, +.023].Source: N Source 1.7B retained 1.7B coverage… · p. 22

C.3 ON-POLICY DISTILLATION

  • OPD trains a Base student on its own SID-only responses while a frozen, same-scale SFT teacher scores each student prefix; neither receives captions, labels, or answer hints.The teacher’s retained top-32 token probabilities are renormalized.Source: §C.3 · p. 23
  • OPD’s feedback and initialization differ from other methods, while its task, source prompt keys, and evaluation benchmark remain shared.The table caption identifies which recipe dimensions are contrasted and which are held fixed.Source: Table C.5 · p. 23

C.4 REINFORCEMENT LEARNING WITH VERIFIABLE REWARDS

RLVR applies a verifier-based ranking reward to an SFT-initialized model, with GRPO updates driven by group-relative rewards. At 8B, graded-reward RLVR significantly improves on SFT, while the 1.7B gain is not significant; at 8B, graded reward also beats binary top-one reward.

C.4 REINFORCEMENT LEARNING WITH VERIFIABLE REWARDS

  • RLVR uses the shared SID prompt without a teacher prompt and completes 100 updates, selecting checkpoints using 400 development examples without policy-gradient updates.Both model scales share the data, reward, rollout, and per-update optimization settings; five RLVR checkpoints are evaluated in development selection.Source: §C.4 · pp. 23–24, Table C.7 · p. 25
  • The verifier gives the exact single-positive NDCG@5 reward to valid outputs placing the positive in ranks 1–5; invalid outputs or lower ranks receive zero.A valid trajectory must close its reasoning block and emit exactly ten distinct in-range indices; other ranking metrics are logged only as diagnostics, with no rationale-based shaping.Source: §C.4 · p. 24
  • GRPO samples eight responses per prompt and standardizes their rewards within the group, so only within-group reward differences drive updates.Each response token shares its response’s advantage; the update uses a clipped surrogate with a KL term relative to the SFT initializer.Source: §C.4 · p. 24
  • At 8B, RLVR reaches .277 against SFT at .237, a paired gain of +.040 (95% CI [+.006, +.076]); at 1.7B, .231 against .225 is not significant.The 8B RLVR model’s cumulative gain over Base is +.109 (95% CI [+.069, +.149]); all reported RLVR outputs are valid, with no identity permutation or stage-1 loop.Source: §C.4 · p. 25
  • At 8B, graded reward beats binary top-one reward by .045, while the two rewards are indistinguishable at 1.7B.At 8B, the top-one model’s difference from the graded-reward model is −.045 (95% CI [−.077, −.013]); at 1.7B, the difference is −.002 (95% CI [−.022, +.018]).Source: §C.4 · p. 25

D.1 ANCHOR METRICS AND PAIRED CONTRASTS

Additional paired comparisons show that the 8B graded-reward model outperforms binary top-one reward, while comparisons against text references and across scales remain within noise. The supplementary results also examine prompt length, stability under changed prompts and display orders, and rationale quality.

D.1 ANCHOR METRICS AND PAIRED CONTRASTS

  • At 8B, graded NDCG@5 reward finishes +.045 above binary top-one correctness (95% CI [+.013, +.077]); at 1.7B, the rewards are indistinguishable.Other added comparisons against text references or across scales remain within noise.Source: §D.1 · p. 25

D.2 RENDERING, TOKENS, AND ENGAGEMENT STRATA

  • Mean anchor text prompts contain 17,815 tokens for Gemini-3.6-Flash, 21,317 for GPT-5.6-Terra, and 31,209 for Claude-Sonnet-5, versus 16,722 under the OpenOneRec tokenizer.The provider-tokenizer counts make the reported compactness ratios conservative; OpenOneRec token counts are a common yardstick, not commercial API billing counts.Source: §D.2 · p. 26, Table D.3 · p. 27
  • At both model scales, SFT gains are larger for grade-3 positives than grade-2 positives.Grade-3 positives are explicit likes, follows, and forwards; the accepted SFT set contains 5,994 grade-2 and 5,997 grade-3 examples.Source: §D.2 · p. 27, Table D.4 · p. 27

D.3 DISPLAY-ORDER AND PROMPT-WORDING STABILITY

  • Across three display orders, the text reference’s NDCG@5 is .247, .234, and .245; the original-order difference from the permutation average is +.005 (95% CI [−.007, +.018]).Three prompt rewordings also stay within the rerun range, with paired differences of +.009, −.011, and −.002, each confidence interval straddling zero.Source: §D.3 · p. 27

D.4 RATIONALE-QUALITY JUDGE

  • The rationale audit separates item-level claims from user portraits and checks verdict stability across presentation order; controlled corruptions generally reversed targeted wins.The 100-user judge saw caption-rendered histories and slates with anonymized rationales, and agreed with all 98 programmatic coverage wins.Source: §D.4 · p. 28
  • Gemini’s rationale names 9.7 of ten ranked candidates against 3.0 for RLVR, while ranking-order fidelity is .983 for Gemini and 1.0 for RLVR.Fidelity is calculated only for candidates both mentioned and ranked, excluding unmentioned candidates rather than placing them at the bottom.Source: §D.4 · p. 28

D.5 QUALITATIVE CASES

The qualitative cases illustrate both interface failures in base models and the trade-off in the strongest SID-only model: it can rank relevant items and reason about trade-offs without reliably identifying their contents. A long-history example contrasts its stable ranking with the base model and text reference.

D.5 QUALITATIVE CASES

  • An 8B Base model’s valid identity ranking drops the positive at display position 29; identity rankings occur for 82 of 400 anchor users.For 1.7B Base, identity rankings occur for 169 of 400 users, often after an empty reasoning block.Source: §D.5 · p. 29
  • Base models can fail before ranking: one 8B SID response copies history tokens to the 2,048-token cap and never closes its reasoning block.All 149 looping anchor users of 8B Base stop at that cap, after which forced extraction returns the identity list and loses the positive.Source: §D.5 · p. 29
  • SID-only students see no captions, making instance-level item claims the expected failure when they imitate text-grounded teacher reasoning.The accompanying teacher example grounds item picks in readable captions that are absent from the student’s input.Source: §D.5 · p. 29
  • In one case, the 8B RLVR model ranks the positive third while Gemini ranks it eighth, but RLVR misidentifies the novel and Gemini describes all ten returned candidates accurately.RLVR discusses only three candidates, and its narrative order matches its ranking; the judge favors Gemini on item claims but RLVR on trade-off argument.Source: §D.5 · p. 30
  • For a 441-item history, RLVR ranks the positive second at k=64 and k=256, while Base falls out of the top ten at both lengths.At k=0 RLVR misses the positive; Gemini ranks it first at k=64 but seventh at k=256, while Base moves from seventh at k=0 to outside the top ten.Source: §D.5 · p. 30
  • A 1.7B Base text-view output scores zero after forced extraction repeats four indices, despite including the positive among them.Deduplication leaves only four distinct indices where the format requires ten, illustrating an invalid-output failure rather than a missed target.Source: §D.5 · p. 30

E ARTIFACTS AND COMPUTE

The release includes the benchmark, training data, run materials, and per-user results behind reported numbers. It also records computational settings and excludes credentials and identifying machine details.

E ARTIFACTS AND COMPUTE

  • The release contains benchmark and training-data builders, parsers, reward code, prompts, model revisions, environment specifications, run commands, and per-user result files.Result files record rendering, split, history length, slate size, display seed, decoder, and selected checkpoint where applicable.Source: E ARTIFACTS AND COMPUTE · p. 30
  • The release excludes API credentials, account identifiers, and machine paths, while released data derived from OpenOneRec-RecIF remain subject to its terms.Source: E ARTIFACTS AND COMPUTE · p. 30
  • Training and inference use bf16 on A100 GPUs; SFT, RFT, and RLVR use four GPUs, while OPD uses two.Checkpoints are selected on 400 development users before scoring on test users; supervision budgets and retention counts are recorded in release metadata.Source: E ARTIFACTS AND COMPUTE · p. 30
Loading 2610.22797v1…