PaperlayerView exact revision on arXiv

Source-linked AI summary

LESSRec: Level-Specific History Modeling for Semantic ID-Based Recommendation

Shangxin Yang, Zongwei Wang, Jing Long, Junliang Yu, Min Gao

arXiv:2610.37286v1cs.IR

TL;DR

Semantic ID recommendation leaves open whether history from every semantic level helps predict a target token. LESSRec aligns historical access to the target level while adding complementary item-level semantics; experiments report recommendation effectiveness for LESSRec and LESSRec+ across three public datasets and three tokenizers.

  • Problem

    It is unclear whether historical information from all semantic levels is equally useful when predicting a token at a particular level.

    Source: §1 · p. 1

  • Method

    LESSRec separately encodes level-specific histories, aligns decoder access with the target level, and fuses that context with an item-level semantic sequence through dual cross-attention.

    Source: §1 · p. 2, §3.4 · pp. 3–4

  • Results

    LESSRec and LESSRec+ demonstrate recommendation effectiveness across three public datasets and three tokenizers.

    Source: §6 · p. 9

  • Takeaways & Limitations

    The reported results support combining level-specific historical evidence with complementary item semantics, while LESSRec+ reduces encoder computation at the same total parameter count as LESSRec.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

Semantic ID-based recommendation represents each item as a sequence of semantic tokens obtained through residual quantization. This paradigm generates the next item's tokens autoregressively from coarse to fine, entailing each prediction attending to historical tokens at all semantic levels. However, our inference-time analysis reveals that historical tokens at the target-token's semantic level tend to be more informative, whereas other levels can introduce interference. Motivated by these findings, we propose LESSRec that separately encodes the historical token sequence at each semantic level and restricts attention over historical tokens to the sequence at the target token's semantic level during prediction. To complement these level-specific histories, LESSRec explicitly provides an item-level semantic sequence that represents each historical item's overall semantics in chronological order. The decoder accesses the level-aligned token sequence and the item-level semantic sequence through two separate cross-attention branches and fuses their contexts. We further introduce LESSRec+, which uses separate encoders for different semantic-level histories with the same total parameter count to reduce encoder computation. Extensive experiments across three public datasets and three tokenizers demonstrate the recommendation effectiveness of LESSRec and LESSRec+, with efficiency analyses highlighting the computational benefits of LESSRec+.

1 Introduction

Semantic ID recommenders generate item tokens from coarse to fine, but it is unclear whether history from every semantic level helps predict each target token. LESSRec addresses this by aligning historical access with the target level while retaining complementary item semantics.

1 Introduction

  • With visible-token counts held constant, increasing same-level access improves performance, and full alignment slightly outperforms original TIGER while using one quarter of the history tokens.The inference-time masking experiment keeps model parameters fixed and changes only the decoder cross-attention mask.Source: §1 · p. 1
  • 11.59% under standard encoding and 49.46% under level-isolated encoding: fully aligned access improves Recall@10 over fully mismatched access on Games.Figure 1 compares same-level and cross-level access configurations; the larger contrast under level-isolated encoding strengthens the evidence for same-level history.Source: §1 · p. 2, Figure 1 · p. 2
  • LESSRec separately encodes each semantic level, aligns decoder access to the target level, and fuses that context with an item-level semantic sequence through a separate cross-attention branch.The item-level sequence combines contextualized states across levels within each historical item.Source: §1 · p. 2
  • LESSRec+ assigns level-specific encoder groups to process their own histories, reducing encoder computation and intermediate activation storage while retaining LESSRec’s total parameter count.The groups partition the existing encoder layers.Source: §1 · p. 2
  • Experiments cover three datasets and three tokenizers; component ablations support the design, and efficiency analyses report computational benefits for LESSRec+.Source: §1 · p. 2

2 Preliminaries

Each historical item is represented by residual-quantized semantic tokens, and the input concatenates those tokens in interaction order. The decoder then generates the next item's semantic IDs autoregressively.

2 Preliminaries

  • Each historical item has an ordered sequence of semantic IDs obtained through residual quantization, with one semantic token at each level.Source: §2 · p. 2
  • The input concatenates historical items’ semantic IDs in interaction order and appends EOS.Source: §2 · p. 2
  • The decoder autoregressively generates the next item’s semantic IDs followed by EOS, conditioning each target position on history and preceding target tokens.Source: §2 · p. 2
  • For each semantic level, the same-level historical sequence collects that level’s tokens across historical items in interaction order.Source: §2 · p. 2
  • The valid input positions comprise the positions of tokens in the level-specific history plus the end-token position, excluding padding.Source: §2 · p. 2

3 Method

LESSRec separates historical semantic tokens by level, aligns each prediction with same-level history, and supplements it with chronological item-level context. LESSRec+ assigns level-specific histories to separate encoder groups to reduce computation while retaining the same total parameter count.

3 Method

  • The method organizes same-level historical associations and complementary within-item semantics for prediction at each semantic level.LESSRec uses attention masks while sharing encoder layers across levels.Source: §3 · p. 2

3.1 Level-Isolated Self-Attention (LIS)

  • LIS restricts historical tokens to self-attention within their semantic level, keeping encoded level-specific histories isolated.The shared EOS state can attend to valid positions, but historical token states cannot attend to EOS; Figure 2 shades levels from coarse to fine and marks visible positions with checkmarks.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2 Level-Aligned Cross-Attention (LAC)

  • LAC lets a level-r target token’s aligned branch attend only to historical level-r tokens, complementing LIS’s isolation during encoding.Together, LIS and LAC prevent other-level historical tokens from entering the aligned history context used for prediction.Source: §3.2 · p. 3

3.3 Item-Level Semantic Sequence (ILS)

  • ILS averages each historical item’s LIS-encoded semantic-token states and applies a residual projection to form chronological item-level representations.The pooled states already contain same-level historical context, so the sequence needs no additional self-attention step.Source: §3.3 · p. 3

3.4 Dual Cross-Attention Fusion

  • Dual cross-attention gives the same decoder query separate branches over level-aligned token history and the item-level semantic sequence, then combines their contexts.A weighted sum controls the branches’ relative contributions; item-level positional bias supplies positions aligned with chronological interactions.Source: §3.4 · pp. 3–4

3.5 Training and Inference

  • Training minimizes average autoregressive negative log-likelihood over history–target pairs, conditioning each prediction on preceding ground-truth target tokens.No auxiliary objective is required, and inference uses trie-constrained beam search over valid identifiers.Source: §3.5 · p. 4

3.6 LESSRec+: Level-Specific Encoders

  • LESSRec+ assigns encoder-layer groups to separate level-specific histories, reducing encoder computation while retaining LESSRec’s total parameter count.Each group processes only its assigned level’s history; the shorter inputs also reduce intermediate activation storage and allow parallel execution.Source: §3.6 · p. 4

4 Experiments

Experiments compare LESSRec and LESSRec+ with conventional and semantic-ID recommenders across three datasets and tokenizers, then test component contributions, design settings, efficiency, and history length. The proposed models outperform baselines, while LESSRec+ reduces encoder computation and shows particular efficiency gains on longer histories.

4.1 Datasets, Baselines, and Evaluation Settings.

  • Evaluation uses Games, Software, and Beauty with 5-core filtering, histories capped at 20, leave-one-out testing, and Recall/NDCG at cutoffs 5 and 10.The study compares conventional sequential and semantic-ID generative recommenders; proposed-model comparisons hold data partitions, IDs, histories, decoding, and checkpoint selection fixed within each tokenization setting.Source: §4.1.1 · p. 5, §4.1.2 · p. 5, §4.1.3 · p. 5

4.2 Overall Performance (RQ1)

  • The best LESSRec or LESSRec+ configuration leads every dataset–metric column, and both models beat corresponding baselines across all three tokenizers.Relative gains range from 6.2% to 39.2% for LESSRec and 6.0% to 54.3% for LESSRec+; LESSRec+ improves over LESSRec in 31 of 36 comparisons.Source: §4.2 · pp. 5–6, Table 2 · p. 6

4.3 Ablation Study (RQ2)

  • ILS alone improves TIGER but trails variants with LIS; adding LAC further helps, and adding ILS to LIS+LAC gives the best ablation results on both datasets.The item-level sequence supplies overall item semantics alongside same-level information; LIS helps even with unrestricted decoder access.Source: §4.3 · p. 6

4.4 Effect of Different Settings (RQ3)

  • Intermediate fusion coefficients yield the best results on Games and Beauty, between token-history-only α=0 and item-sequence-only α=1.The other settings are held fixed while α varies.Source: §4.4.1 · p. 7, Figure 4 · p. 7

4.4.2 Number of Semantic Tokens per Item; 4.4.3 Encoder Depth Allocation

  • Both models outperform TIGER with four, five, or six semantic tokens per item; on Games, all five tested LESSRec+ depth allocations also beat TIGER.No allocation is best across all metrics: [1] leads R@5 and N@5, [1] leads R@10, and the two tie for the highest N@10 at reported precision.Source: §4.4.2 · p. 7, §4.4.3 · p. 7, Figure 5 · p. 7

4.5 Complexity and Resource Efficiency (RQ4)

  • LESSRec+ reduces encoder self-attention operations by a factor of L^2 relative to LESSRec and TIGER, while retaining the same total encoder depth.Both proposed models add an item-level decoder cross-attention branch whose cost scales linearly with history length.Source: §4.5 · p. 7, Table 5 · p. 7
  • LESSRec+ cuts per-epoch time by 33.7% and peak training memory by 39.2%, while both proposed models add 0.4% parameters and converge faster than TIGER.Time to convergence decreases by 34.0% for LESSRec and 65.2% for LESSRec+; LESSRec takes 5.8% longer per epoch.Source: §4.5 · p. 7, Table 6 · p. 8

4.6 Effect of History Length (RQ5)

  • Both models outperform TIGER in all three history-length buckets, with larger gains for longer histories; LESSRec+ reduces encoding time by 13.4% and 29.5% in the middle and longest buckets.LESSRec+ incurs a 9.2% encoding-time overhead in the shortest bucket; LESSRec’s encoding-time changes range from a 1.6% reduction to a 5.1% increase.Source: §4.6 · p. 8

5 Related Work

Related work covers semantic item tokenization, generative recommendation with semantic IDs, and structure-aware sequence modeling. LESSRec’s focus is modeling historical semantic levels and aligning access with the target level while retaining item-level context.

5.1 Semantic Item Tokenization

  • Tokenization work ranges from text-embedding quantization and hierarchical RQ-VAE IDs to learnable, collaborative-aware, tree-structured, textual, and action-context identifiers.LESSRec keeps semantic IDs fixed and evaluates RQ-VAE, RQ-Kmeans, and LETTER tokenization.Source: §5.1 · p. 8

5.2 Semantic ID-Based Generative Recommendation

  • Semantic-ID recommenders vary in learning, information integration, and generation; LESSRec instead focuses on modeling historical semantic levels.Examples include alternating tokenizer–recommender learning, dual-source item indices, parallel generation of unordered IDs, and query-anchored reasoning.Source: §5.2 · p. 8

5.3 Structure-Aware Sequence Modeling

  • Structure-aware methods encode hierarchy or item boundaries through positional encodings, intra-item attention, anchor tokens, or hierarchical decoding; LESSRec aligns history access with target levels and keeps item-level context.DiscRec is used as a representative structure-aware baseline for comparison.Source: §5.3 · p. 8

6 Conclusion

The conclusion presents same-level history as generally more informative for target-token prediction and describes LESSRec’s level-aligned and item-level context branches. It reports recommendation effectiveness across three public datasets and three tokenizers, with LESSRec+ reducing encoder computation at unchanged parameter count.

6 Conclusion

  • LESSRec separately encodes semantic-level histories, aligns decoder access to the target level, and adds complementary item-level semantics; LESSRec+ reduces encoder computation at the same parameter count.Experiments across three public datasets and three tokenizers support both models’ recommendation effectiveness and LESSRec+’s computational benefits.Source: §6 · p. 9

A Why Same-Level History Helps

The analysis compares same-level and full-history access under assumptions about residual quantization and temporal dependence, then examines prediction and learning effects. It concludes that same-level history can retain predictive information while cross-level mixing can weaken the initial learning signal.

Predictive information

Assumptions

  • The predictive-information analysis assumes successive residual components are recoverable by quantization and level trajectories are independent first-order Markov chains.RQ-VAE training alone does not guarantee the assumed temporal dependence.Source: A Why Same-Level History Helps · p. 10

Predictive information

  • Under the stated assumptions, same-level history retains the full history’s predictive information, with a positive gap when the previous same-level token predicts the target.The learning comparison is limited to initialization; learned projections and content-dependent scores may change it.Source: A Why Same-Level History Helps · p. 10

Learning signal

  • With equal scores, mixing levels reduces the expected gradient magnitude by a factor of L, even when predictive information remains available.The analysis omits layer normalization, feed-forward sublayers, EOS, and item memory to isolate level mixing.Source: A Why Same-Level History Helps · pp. 10–11

B Residual Quantization for Item Tokenization

The paper uses offline tokenizers to construct fixed semantic IDs, with tokenizer parameters trained separately from the recommendation model.

B Residual Quantization for Item Tokenization

B.1 RQ-VAE

RQ-VAE maps item content embeddings into coarse-to-fine semantic IDs by repeatedly quantizing residuals. Its encoder, decoder, and codebooks are trained jointly with reconstruction, codebook, and commitment losses.

B.1 RQ-VAE

  • RQ-VAE encodes an item’s content embedding into a latent vector, then repeatedly selects and subtracts a nearest codeword to quantize residuals.This follows residual quantization for image generation and its use in TIGER.Source: §B.1 · p. 11
  • Earlier codewords approximate the latent representation, while later codewords refine the remaining residual into a coarse-to-fine semantic ID.Source: §B.1 · p. 11
  • The encoder, decoder, and codebooks are jointly trained with reconstruction, codebook, and commitment losses; implementation uses β = 0.25, λ = 1, and K-means initialization.Codebooks are subsequently updated by gradient descent, so K-means initialization alone does not make the tokenizer residual K-means.Source: §B.1 · p. 11

B.2 RQ-Kmeans

RQ-Kmeans builds hierarchical semantic IDs by clustering residual representations at successive levels. The described implementation combines residual quantization with an autoencoder and K-means-guided conditional subcodebooks.

B.2 RQ-Kmeans

  • RQ-Kmeans constructs hierarchical semantic IDs by clustering residual representations at successive levels [10].Source: §B.2 · p. 11
  • At each level, the nearest centroid is assigned and subtracted before quantizing the next residual, so earlier levels capture larger-scale variation and later levels refine it.Source: §B.2 · p. 11
  • The implementation combines a learned autoencoder with K-means-guided conditional subcodebooks, where earlier tokens route each level’s subcodebook.The first-level subcodebook is shared across items, and selected vectors are summed and decoded to reconstruct the content embedding.Source: §B.2 · p. 11

C Additional Experimental Analysis

Design ablations find benefits from learned item-level fusion, while tests of EOS and QKV variants clarify which parts of LESSRec contribute to its performance.

C Additional Experimental Analysis

  • Removing EOS leaves performance close to LESSRec in both metrics and produces no consistent gain.Historical tokens cannot attend to EOS under LIS, so their encoded states remain level-isolated when EOS is retained.Source: C Additional Experimental Analysis · p. 12
  • Splitting encoder-side QKV computation has little effect, whereas splitting decoder-side QKV computation leads to a clear performance decline.The authors suggest decoder-side splitting may weaken how level-specific histories complement the shared item-level semantic sequence.Source: C Additional Experimental Analysis · p. 12
  • Cat.+MLP outperforms mean-pooling-based LESSRec in both metrics, while removing the MLP reduces performance in both.The ablation suggests retaining individual semantic-level representations before learned fusion can benefit item-level modeling.Source: C Additional Experimental Analysis · pp. 11–12, Figure 7 · p. 12

Related papers

Loading 2610.37286v1…