PaperlayerView exact revision on arXiv

Source-linked AI summary

JevRec: Linguistic Encoding with Jev-like Decision Modeling for LLM-based Generative Recommendation

Qiyong Zhong, Jiajie Su, Ming Yang, Li Zhang

arXiv:2610.36937v1cs.CL

TL;DR

Generative recommendation usually assigns equal-length semantic identifiers despite unequal item frequencies, leaving a trade-off between repeated encoding costs for popular items and semantic detail for long-tail items. JevRec allocates SID lengths by frequency and separates length prediction from content generation; on three public datasets, it outperforms all compared baselines across evaluated recommendation metrics.

  • Problem

    Fixed-length semantic identifiers cannot match representation detail and encoding cost to unequal item frequencies: fine-grained codes are repeatedly costly for popular items, while shortening codes compresses long-tail representations.

    Source: §1 · p. 1

  • Method

    JevRec assigns shorter semantic-ID prefixes to popular items and richer descriptions to long-tail items, using separate length and content heads that share history representations and condition content generation on length.

    Source: §1 · p. 2

  • Results

    Across three public datasets, JevRec outperforms all compared baselines on every evaluated recommendation metric.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support treating SID length as both a representation choice and an explicit prediction decision for accurate and efficient generative recommendation.

    Source: §5 · p. 9

    The item-to-SID mapping is context-invariant, and item frequency is computed from training interactions only.

    Source: §3.2 · p. 3

Abstract

from arXiv · show

Generative recommendation (GR) represents items as semantic identifiers (SIDs) and autoregressively generates the next item's SID from interaction history. However, most methods assign the same SID length to all items despite highly unequal interaction frequencies. This uniform design repeatedly incurs the cost of fine-grained codes for popular items, while shortening codes to reduce this cost also compresses the semantic representations of long-tail items. Inspired by the compact forms of frequent expressions in natural language, we propose JevRec to encode items as variable-length words and generate the next item from interaction context. Linguistic encoding retains frequency-dependent semantic prefixes from shared codebooks, producing shorter SIDs for popular items and richer descriptions for long-tail items. Yet predicting a sequence-level length and generating token-level content impose different optimization requirements that may conflict under a shared output head. Our Jev-like decision modeling component therefore predicts a length distribution with a dedicated head and generates content conditioned on length through a separate head while sharing the interaction-history representation. Experiments on three public datasets show that JevRec outperforms all compared baselines across the evaluated recommendation metrics. Further analyses show lower training and inference time than TIGER and reduced gradient conflict between length and content prediction.

1 INTRODUCTION

Generative recommendation predicts the next item’s semantic identifier from interaction history, but fixed-length codes cannot allocate representation detail according to item frequency. JevRec varies SID lengths by frequency and separates length decisions from content generation; experiments report gains over all compared baselines on every evaluated recommendation metric.

1 INTRODUCTION

  • Generative recommendation represents items as discrete semantic-ID token sequences and autoregressively generates the next item’s SID from interaction history.Unlike methods with a separate item embedding for every item, it uses a compact shared token vocabulary.Source: §1 · p. 1
  • Fixed-length SIDs cannot balance semantic detail against encoding cost according to item frequency: fine-grained codes repeatedly cost more for frequent items, while shortening all codes compresses long-tail representations.Recommendation data typically have many low-frequency items and a small set of popular items accounting for a large share of interactions.Source: §1 · pp. 1–2
  • Across three public datasets, JevRec outperforms all compared baselines on every evaluated recommendation metric.The efficiency analysis also reports shorter training time and faster convergence.Source: §1 · p. 2
  • Linguistic encoding assigns popular items shorter SIDs and long-tail items richer semantic descriptions, preserving shared semantics and unique identities while reducing repeated encoding costs for frequent interactions.It retains frequency-dependent prefixes from shared semantic codebooks.Source: §1 · p. 2
  • Jev-like decision modeling uses separate length-classification and content-generation heads that share history representations, with content generation conditioned on length.Length probabilities guide candidate selection across decoding branches to form a length-aware recommendation list.Source: §1 · p. 2

2 RELATED WORK

Prior generative-recommendation work spans varied item tokenizers and studies adaptive representations, while linguistic encoding coordinates frequency-aware SID lengths with explicit target-length decisions during generation.

2 RELATED WORK

  • Generative recommendation predicts discrete identifier sequences rather than scoring separate item embeddings, and prior work spans quantization, clustering, language-model identifiers, and multimodal tokenization.Other lines of work study joint tokenizer-recommender optimization, hierarchical identifiers, compression, memorization, and alternative generation or retrieval methods.Source: §2 · p. 2
  • Tokenization balances representational detail against sequence length, but item frequency determines how often an item’s encoding cost is incurred in recommendation.Prior variable-length item representations emphasize adaptive encoding, history compression, or decoding-tree structure.Source: §2 · p. 2
  • JevRec coordinates frequency-aware encoding with explicit target-length decisions, retaining shorter prefixes for popular items and finer-grained descriptions for long-tail items.Length decisions condition subsequent content generation.Source: §2 · p. 3

3 METHOD

JevRec combines frequency-dependent item SIDs with separate length and content prediction heads that share a history representation. Joint training and length-aware inference use the resulting length distribution to generate and rank recommendations.

3.2 Problem Formulation

  • The task predicts the next item from chronological interaction history, representing each item with a uniquely identifying SID whose length may vary.The history is serialized with length prefixes; generated SIDs are used to recover an ordered list of distinct items.Source: §3.2 · p. 3

3.3 Linguistic Encoding

  • Items receive variable-length SIDs by retaining frequency-dependent prefixes from shared semantic codebooks and appending collision-resolution tokens for unique identification.The default setting yields SID lengths of three, four, and five for high-, medium-, and low-frequency groups, respectively.Source: §3.3 · p. 3

3.4 Jev-like Decision Modeling

  • The shared encoder–decoder represents interaction history, while separate heads predict SID length and generate content conditioned on that length.The length decision is sequence-level, whereas content generation predicts successive tokens from an evolving prefix.Source: §3.4 · pp. 3–4

3.1 Overview

  • The overview combines frequency-aware SID construction with length decision-making, conditional decoding, quota-based selection, and cross-length ranking.The figure’s blue blocks represent complete SIDs, and its item examples are illustrative.Source: §3.1 · p. 3, Figure 2 · p. 4

3.5 Joint Training

  • Length and content losses are jointly optimized with unit weights, with each training example’s content generation conditioned on its ground-truth length.The shared backbone is updated to learn history representations for both length selection and fine-grained item discrimination.Source: §3.5 · p. 4

3.6 Length-Aware Inference

  • JevRec’s length-aware inference allocates candidate quotas across length branches, decodes within each branch, then ranks selected items by joint score.If a branch falls short of its quota, unselected candidates from other branches fill the vacancies when enough valid candidates exist; smaller cutoffs use the same ranked list.Source: §3.6 · p. 5

4 EXPERIMENTS

Across three Amazon Reviews domains, JevRec is evaluated against sequential and generative baselines, through ablations and analyses of optimization, efficiency, sensitivity, and item-level SID lengths. It leads all compared baselines on every evaluated recommendation metric, while analysis examines its design trade-offs and runtime.

4.1 Experimental Setup

  • The evaluation uses Amazon Reviews Musical Instruments, Industrial & Scientific, and Video Games, with leave-one-out splits and full-catalog evaluation.Users and items with fewer than five interactions are excluded; histories are capped at 20 items.Source: §4.1 · p. 5

4.2 Overall Performance (RQ1)

  • JevRec achieves the best result in all 12 dataset–metric combinations, averaging 6.52% relative improvement over the strongest baseline per column and 12.04% over TIGER.The advantage spans three domains, Recall and NDCG, and stronger tokenization baselines.Source: Table 2 · p. 6, §4.2 · p. 6

4.3 Ablation Study (RQ2)

  • Random frequency-independent SID lengths trail JevRec by 8.70% on average, while reversed-frequency lengths perform worse than random lengths on every metric.The ablations also show that a shared LM head lowers every metric by 7.50% on average; removing length tokens causes the largest average degradation, while a history-only length classifier stays within 1%.Source: §4.3 · pp. 6–7, Table 3 · p. 7

4.4 Optimization Conflict Analysis (RQ3)

  • Separating length and content heads shifts pooled gradient cosine from −0.105 to −0.015, reducing negative samples but not eliminating directional interference.With a shared head, mean cosine remains negative at every recorded checkpoint.Source: §4.4 · p. 7, Figure 3 · p. 7

4.5 Cost Efficiency Analysis (RQ4)

  • Training time per epoch falls by 15.75% on average, and JevRec also reduces total inference time on all three datasets.On Instrument, it reaches its best validation NDCG@10 20 epochs earlier than TIGER.Source: §4.5 · pp. 7–8, Figure 4 · p. 8

4.6 Hyperparameter Analysis (RQ5)

  • Across the tested settings, NDCG@10 stays within 3.2% of default for the full N sweep and M values 2–4, while raising M above 3 lowers it on all datasets.The decline is sharper from M=4 to M=5; increasing N beyond 3 yields small, non-monotonic changes.Source: §4.6 · p. 8, Figure 5 · p. 8

4.7 Case Study (RQ6)

  • At M=N=3, Grand Theft Auto V receives a three-token SID with 2,142 interactions, while Sekiro: Shadows Die Twice gets five tokens with seven interactions.Warcraft III: The Frozen Throne has 25 interactions and receives a four-token SID; each SID includes a final collision-resolution token.Source: §4.7 · p. 8, Figure 6 · p. 8

5 CONCLUSION

JevRec combines frequency-aware SID lengths with separate, length-conditioned prediction heads, and experiments report gains over baselines alongside lower training and inference time than TIGER.

5 CONCLUSION

  • Experiments on three public datasets show consistent gains over compared baselines and lower training and inference time than TIGER.Ablations support the component contributions.Source: §5 · pp. 8–9
  • Linguistic encoding gives popular items shorter SIDs and long-tail items richer semantic descriptions while preserving shared semantics and unique identities.It uses shared codebooks and collision-resolution tokens, reducing encoding cost for frequent interactions without requiring equally short representations for all items.Source: §5 · p. 8
  • Jev-like decision modeling uses separate length and content heads that share user-history representations, with content generation conditioned on length.Length-aware inference uses predicted length probabilities for candidate allocation and cross-length ranking rather than committing to one predicted length.Source: §5 · p. 8
  • Gradient analysis shows that separate heads alleviate conflict between the length and content objectives.The findings support treating SID length as both a representation choice and an explicit prediction decision.Source: §5 · p. 9

B FREQUENCY GROUP ALLOCATION

Frequency groups are sized using a smooth prior over item proportions, then assigned progressively longer SIDs from popular to long-tail items.

B FREQUENCY GROUP ALLOCATION

  • Frequency grouping uses training interactions to sort items by frequency and partitions the sorted catalog into contiguous groups sized by sampled proportions.The proportions come from discrete support points of a normalized Gamma(Kγ, θ=1) distribution; allocation is shared across users and excludes validation and test interactions.Source: B FREQUENCY GROUP ALLOCATION · p. 12
  • A smaller Kγ places more items in leading short-code groups, while a larger Kγ shifts items toward later long-code groups.The proportions apply to item counts, not cumulative interactions or distinct frequency values.Source: B FREQUENCY GROUP ALLOCATION · p. 12
  • Each successive frequency group receives an SID one token longer, retaining its semantic-code prefix and appending a collision-resolution token.With M=3, N=3, and Kγ=2.1, the three SID lengths are 3, 4, and 5, assigned to approximately 44.6%, 35.2%, and 20.2% of items, respectively.Source: B FREQUENCY GROUP ALLOCATION · p. 12

C EXPERIMENTAL DETAILS

The experiments follow prior-work settings and directly use reported results for most baselines.

C EXPERIMENTAL DETAILS

C.1 Implementation Details.

Implementation uses a shared residual-quantization tokenizer, a four-layer recommendation backbone, separate prediction heads, and length-branch decoding with a fixed recommendation budget.

C.1 Implementation Details.

  • Item text is encoded with sentence-t5-base, followed by whitened PCA [37] and FAISS residual quantization [5] with 256-entry codebooks.Each item retains a frequency-dependent semantic prefix and a collision-resolution token.Source: §C.1 · p. 12
  • The encoder and decoder each have four layers, hidden size 128, feedforward dimension 512, and four attention heads of dimension 64.The length and content heads share the backbone; the native T5 content projection is tied to token embeddings, while the length head is an MLP.Source: §C.1 · p. 12
  • JevRec uses Kmax=10 as both the beam width per length branch and the total recommendation-list budget, allocating slots and merging candidates once at that cutoff.Recall@5 and NDCG@5 use the first five entries of the same list; probabilities include EOS and are not length-normalized.Source: §C.1 · p. 12
  • All datasets use M=N=3 and Kγ=2.1, yielding SID lengths of 3, 4, and 5 with a four-level shared residual quantizer.Source: §C.1 · p. 12

Related papers

Loading 2610.36937v1…