PaperlayerView exact revision on arXiv

Source-linked AI summary

DiscRec: Disentangled Semantic-Collaborative Modeling for Generative Recommendation

Chang Liu, Yimeng Bai, Xiaoyan Zhao, Yang Zhang, Fuli Feng, Wenge Rong

arXiv:2506.15576v2cs.IR

TL;DR

Generative recommendation predicts items by generation, but existing approaches can miss item-level structure and entangle semantic and collaborative signals. DiscRec introduces item-aware positions and a fused dual-branch design, and the paper reports consistent gains over state-of-the-art baselines on four real-world datasets.

  • Problem

    Generative recommenders can ignore item boundaries and entangle semantic and collaborative signals with distinct distributions, creating incompatible optimization objectives and representational interference.

    Source: §I · p. 2

  • Method

    DiscRec combines item-level position embeddings with semantic and localized-attention collaborative branches, then adaptively fuses their outputs through a gate.

    Source: §I · p. 2

  • Results

    DiscRec is reported to decouple semantic and collaborative signals and consistently outperform state-of-the-art baselines across four real-world datasets.

    Source: §I · p. 2

  • Takeaways & Limitations

    DiscRec combines embedding-layer disentanglement with deeper cross-signal interaction, unlike two-stream designs that keep the signals separate throughout the model.

    Source: §I · p. 2, §VI · p. 12

Abstract

from arXiv · show

Generative recommendation is emerging as a powerful paradigm that directly generates item predictions, moving beyond traditional matching-based approaches. However, current methods face two key challenges: token-item misalignment, where uniform token-level modeling ignores item-level granularity that is critical for collaborative signal learning, and semantic-collaborative signal entanglement, where collaborative and semantic signals exhibit distinct distributions yet are fused in a unified embedding space, leading to conflicting optimization objectives that limit the recommendation performance. To address these issues, we propose DiscRec, a novel framework that enables Disentangled Semantic-Collaborative signal modeling with flexible fusion for generative Recommendation. First, DiscRec introduces item-level position embeddings, assigned based on indices within each semantic ID, enabling explicit modeling of item structure in input token sequences. Second, DiscRec employs a dual-branch module to disentangle the two signals at the embedding layer: a semantic branch encodes semantic signals using original token embeddings, while a collaborative branch applies localized attention restricted to tokens within the same item to effectively capture collaborative signals. A gating mechanism subsequently fuses both branches while preserving the model's ability to model sequential dependencies. Extensive experiments on four real-world datasets demonstrate that DiscRec effectively decouples these signals and consistently outperforms state-of-the-art baselines. Our codes are available on https://github.com/Ten-Mao/DiscRec.

I. INTRODUCTION

Generative recommendation predicts the next item by generating semantic-ID tokens, but existing methods can overlook item boundaries and entangle semantic and collaborative signals. DiscRec adds item-aware alignment and a flexible dual-branch design; the paper reports improved performance across four real-world datasets.

I. INTRODUCTION

  • Generative recommendation replaces query-candidate matching with autoregressive generation of the next item from semantic-ID sequences built from interaction histories.The paradigm has a tokenization stage followed by a recommendation stage.Source: §I · p. 1
  • Uniform token-level modeling overlooks item boundaries, weakening collaborative-signal learning, while entangled signals with distinct distributions create incompatible optimization objectives and representational interference.The paper describes semantic signals as hierarchically decaying across semantic-ID indices, unlike collaborative signals.Source: §I · pp. 1–2, Figure 1 · p. 1
  • DiscRec combines item-level position embeddings with a dual-branch module to align item structure and disentangle semantic and collaborative signals with flexible fusion.The semantic branch uses original token embeddings; the collaborative branch uses localized attention within each item, and a gating mechanism fuses their outputs.Source: §I · p. 2
  • DiscRec is incorporated into TIGER and LETTER and is reported to decouple the signals and achieve superior recommendation performance on four real-world datasets.The paper presents these experiments as a validation of the framework's generalizability.Source: §I · p. 2

1) Task Formulation:

  • In the sequential setting, the task is to predict the next item from a user's historical interaction sequence, with items represented by hierarchically quantized tokens.RQ-VAE's residual quantization produces semantic IDs whose shared prefixes capture collaborative semantics among related items.Source: §II · p. 2, 1) Task Formulation · p. 3

2) Model Optimization:

The model optimization section describes how item semantic embeddings are encoded and quantized into hierarchical tokens, then used to represent interaction histories and train the tokenizer.

2) Model Optimization:

  • The tokenizer encodes an item's semantic embedding into a latent representation, then queries L distinct codebooks to quantize it into discrete tokens.Each codebook maps continuous latent vectors to symbolic tokens by selecting the nearest code in Euclidean space.Source: 2) Model Optimization · p. 3
  • At each quantization level, the residual from the previous level is encoded, and the resulting quantized representation is decoded to reconstruct the original semantic embedding.The first residual vector is the encoded latent representation.Source: 2) Model Optimization · p. 3
  • The tokenization loss combines reconstruction loss with two quantization losses, while stop-gradient and β balance learning between the encoder and codebooks.Reconstruction loss encourages the reconstructed embedding to approximate the input; quantization losses reduce discrepancies between residuals and codebook entries.Source: 2) Model Optimization · p. 3
  • Applying discretization to each historical item converts the interaction history into a token sequence, with each item represented by L tokens.The target next item is represented similarly.Source: 2) Model Optimization · p. 3

B. Recommendation Stage

The recommendation stage frames next-item prediction as autoregressive generation over tokenized interaction histories. A T5-based encoder-decoder processes the history and target tokens, then predicts the target sequence.

1) Task Formulation:

  • The next-item task generates each target token autoregressively, conditioned on the tokenized history and previously generated target tokens.This setup uses semantic IDs to extract semantic signals and interaction transitions to capture collaborative signals.Source: 1) Task Formulation · p. 3

2) Model Optimization:

  • The T5-based recommender embeds the tokenized history and an end-of-sequence token, then feeds the sequence into the Transformer encoder.The embedding sequence is obtained by looking up the token embedding table.Source: 2) Model Optimization · p. 3
  • For decoding, the model prepends a beginning-of-sequence token to the target embeddings and uses the encoder output alongside them to model user preference.The Transformer decoder takes the encoder output and target embedding sequence as inputs.Source: 2) Model Optimization · pp. 3–4
  • The decoder output is projected into token-embedding space to predict target tokens, and recommendation training minimizes their sequence-to-sequence negative log-likelihood.Auxiliary losses, including collaborative regularization, may also be included in frameworks such as LETTER.Source: 2) Model Optimization · pp. 3–4

C. Distribution Analysis of Semantic and Collaborative Signals

Embedding-norm distributions motivate disentangling semantic and collaborative signals, while DiscRec addresses item-token misalignment with item-aware positions and a dual-branch module.

C. Distribution Analysis of Semantic and Collaborative Signals

  • Code-embedding norms show hierarchical decay across semantic-ID positions, unlike the more uniform token-embedding norms, motivating signal disentanglement.The analysis treats embedding-vector norms as proxies for information capacity and averages them across items.Source: C. Distribution Analysis of Semantic… · p. 4

III. METHODOLOGY

  • DiscRec’s methodology combines Item-level Position Embedding with a Dual-Branch Module for disentangling and adaptively fusing semantic and collaborative signals.The framework also includes a complexity analysis.Source: §III · p. 4

A. Disentangled Collaborative-Semantic Modeling

  • DiscRec addresses item-token misalignment and signal entanglement through item-position structure and parallel semantic and collaborative encoding with adaptive embedding-level fusion.The two modules target the two challenges through complementary designs.Source: A. Disentangled Collaborative-Semantic… · p. 4

1) Item-level Position Embedding:

  • The Item-level Position Embedding assigns shared embeddings by each token’s position within its semantic ID, capturing item structure without item-specific position tables.The table reserves its final two rows for [EOS] and [BOS].Source: 1) Item-level Position Embedding · p. 4

2) Dual-Branch Module:

The Dual-Branch Module separates semantic and collaborative representations, then adaptively fuses them into a unified representation for downstream tasks.

2) Dual-Branch Module:

  • The module combines original token embeddings with position embeddings to form disentangled encoder and decoder representations.Figure 3 depicts the larger recommendation pipeline and the module’s branches and fusion gate.Source: 2) Dual-Branch Module · pp. 4–5, Figure 3 · p. 5

Semantic Branch

  • The semantic branch outputs the original token embeddings without additional processing, keeping this branch independent of collaborative information.Source: 2) Dual-Branch Module · p. 5

Collaborative Branch

  • The collaborative branch feeds token and position embeddings into a Transformer to aggregate collaborative information from sequential interaction patterns.The encoder uses bidirectional self-attention and the decoder uses unidirectional attention.Source: 2) Dual-Branch Module · p. 5
  • Localized attention restricts aggregation to tokens within the same item, aligning collaborative-signal modeling with item granularity.The mask assigns 0 to same-item token pairs and −∞ otherwise; [EOS] and [BOS] are assigned to the nearest adjacent item.Source: 2) Dual-Branch Module · pp. 5–6

Branch Fusion

  • A Dynamic Fusion Gate learns branch-specific gate vectors and softmax-normalized weights to adaptively combine the branch outputs.Disentanglement is confined to the embedding level, preserving interaction between signal types in higher model layers.Source: 2) Dual-Branch Module · p. 6

B. Training Procedure

DiscRec is trained in two sequential stages: first learning a tokenizer, then training the recommender using tokenized inputs and disentangled representations.

B. Training Procedure

  • Training proceeds through tokenization and recommendation stages, each updating its model parameters until convergence.Source: B. Training Procedure · p. 6
  • In the tokenization stage, the tokenizer learns to discretize item representations into semantic IDs by iteratively computing its loss and updating its parameters.Source: B. Training Procedure · p. 6
  • In the recommendation stage, the learned tokenizer creates inputs, the dual-branch module forms encoder and decoder representations, and the recommender is updated using recommendation loss.The process repeats until convergence.Source: B. Training Procedure · p. 6

C. Complexity Analysis

DiscRec’s total time complexity is in the same order as TIGER’s, and the experiments use four Amazon subsets with a standardized sequential-recommendation setup.

C. Complexity Analysis

  • DiscRec’s overall time complexity is O(TD^2 + T^2D + TLKD), matching TIGER’s order and similar inference time complexity.Tokenization over T items costs O(TD^2 + TLKD); the recommendation phase includes self-attention, feed-forward, collaborative-branch, and fusion costs.Source: C. Complexity Analysis · p. 6

IV. EXPERIMENT

  • The experiment asks about comparative performance, component contributions, disentanglement, generalization, and alternative architectures.The research questions frame the subsequent evaluation across these five topics.Source: §IV · p. 6

1) Datasets:

  • Beauty, Musical Instruments, Arts, and Toys are evaluated after 5-core filtering, with histories fixed at 20 interactions and leave-one-out splits.Each user’s most recent interaction is held out for testing, the second most recent for validation, and earlier interactions for training.Source: 1) Datasets · p. 7, Table I · p. 7

2) Baselines:

  • Traditional baselines include MF, LightGCN, Caser, HGN, and SASRec.These methods cover latent-factor, graph-convolution, convolutional, gating, and self-attention approaches.Source: 2) Baselines · p. 7
  • Generative baselines include TIGER, LETTER, and EAGER.EAGER uses separate decoding paths for semantic and collaborative signals.Source: 2) Baselines · p. 7

3) Evaluation Metrics:

DiscRec improves on its TIGER and LETTER bases, with DiscRec-L achieving the best overall results. Evaluation uses full-item ranking and shared generative-model settings.

3) Evaluation Metrics:

  • Evaluation uses Recall@K and NDCG@K at K = 5 and 10, with full ranking over the item set and constrained decoding at beam size 20.The constrained decoder uses a prefix tree to filter invalid semantic-ID prefixes.Source: 3) Evaluation Metrics · p. 7

4) Implementation Details:

  • Generative baselines share a four-layer T5 configuration, while DiscRec-T and DiscRec-L add a two-head collaborative branch to TIGER and LETTER, respectively.The shared recommender uses model dimension 128; the collaborative branch uses attention dimension 64 and MLP hidden size 1024.Source: 4) Implementation Details · p. 7

B. Performance Comparison (RQ1)

  • Traditional methods generally underperform generative approaches; the paper attributes this partly to ID embeddings’ limited ability to capture hierarchical semantic structure.RQ-VAE-based generative models can represent finer distinctions among semantically similar items.Source: B. Performance Comparison (RQ1) · pp. 7–8
  • DiscRec-T consistently beats TIGER, DiscRec-L beats LETTER, and DiscRec-L achieves the best overall results.Table II compares baselines with DiscRec-T and DiscRec-L; “RI” denotes relative improvement over TIGER and LETTER.Source: B. Performance Comparison (RQ1) · p. 7, Table II · p. 8
  • LETTER outperforms TIGER, which the authors largely attribute to collaborative-signal supervision during tokenization.LETTER’s auxiliary loss based on collaborative embeddings bridges items with similar semantics but divergent interaction patterns.Source: B. Performance Comparison (RQ1) · p. 8
  • EAGER’s two-stream design has limited performance and falls below traditional methods such as MF on Instruments.The authors suggest that independently processing the signals may overseparate them and hinder integration of their complementary strengths.Source: B. Performance Comparison (RQ1) · p. 8

C. Ablation Study (RQ2)

The ablations support the contributions of item-level position embeddings, localized attention, and adaptive gating, while showing that standard positional embeddings and added model capacity do not deliver comparable gains.

C. Ablation Study (RQ2)

  • Increasing TIGER to five Transformer layers adds more parameters than DiscRec but does not produce significant performance gains.The authors attribute DiscRec’s improvements to task alignment and signal disentanglement rather than parameter growth alone.Source: C. Ablation Study (RQ2) · pp. 8–9
  • Removing IPE, localized attention, or adaptive gating degrades performance, with IPE and attention removals causing larger declines.The ablated variants remove IPE, the localized-attention Transformer, or gating fusion; Table III reports the study on Beauty.Source: C. Ablation Study (RQ2) · p. 8, Table III · p. 9
  • IPE and item-ID embeddings improve performance consistently, whereas standard positional embeddings yield negligible change.The item-ID embedding table imposes scalability constraints; IPE is presented as a more scalable, cost-effective alternative.Source: C. Ablation Study (RQ2) · p. 9

D. Effectiveness Analysis (RQ3)

DiscRec shows item-aligned attention patterns and distinct semantic and collaborative embedding distributions. Across varied sequence lengths, popularity levels, and a user-level random split, it consistently improves performance over TIGER and LETTER.

1) Item-Aware Alignment:

2) Semantic-Collaborative Signal Disentanglement with Flexible Fusion:

  • Semantic token embeddings decline across token indices like code embeddings, while collaborative embedding norms increase, indicating distinct signal distributions.The reported norm analysis uses Beauty and treats average embedding norm as a measure of information capacity.Source: 2) Semantic-Collaborative Signal… · p. 10, Figure 6 · p. 10

E. Generalization Analysis (RQ4)

  • Across varying sequence lengths and item popularity levels, DiscRec consistently achieves performance improvements over both TIGER and LETTER, attributed to balanced integration of collaborative and semantic signals.Source: Figure 7 · p. 10, E. Generalization Analysis (RQ4) · p. 11
  • Under a user-based split, DiscRec continues to deliver consistent performance improvements, suggesting its effectiveness is not contingent on specific user behavior patterns and indicating robustness across diverse user distributions.Table IV reports a performance comparison on Beauty under a user-level random split.Source: E. Generalization Analysis (RQ4) · p. 11, Table IV · p. 11

F. Potential Design Analysis (RQ5)

Exploratory variants test where and how DiscRec’s disentanglement and fusion components should be applied. The results favor embedding-layer decoupling with token-level collaborative representations and find no significant performance difference for Self-Gating versus DiscRec-T.

F. Potential Design Analysis (RQ5)

  • Applying DiscRec to all Transformer layers causes a significant decline, placing it below TIGER and suggesting decoupling is most suitable at the embedding layer.The authors suggest higher layers primarily facilitate information interaction, where decoupling may be detrimental.Source: F. Potential Design Analysis (RQ5) · p. 11, Table V · p. 11
  • OneQuery performs slightly worse than TIGER, while TokenAvg performs slightly worse than DiscRec-T.The authors associate the OneQuery result with the challenge of learning a unified collaborative query and the TokenAvg result with the value of preserving token-level personalized collaborative information.Source: F. Potential Design Analysis (RQ5) · p. 11, Table V · p. 11
  • Self-Gating shows no significant performance difference from DiscRec-T.The authors describe directly learning a gating vector as a simple and effective approach for adaptively fusing the two information types.Source: F. Potential Design Analysis (RQ5) · p. 11, Table V · p. 11

V. RELATED WORK

The related-work section surveys sequential, generative, and collaborative-information recommendation approaches, noting limits in generalization and unresolved signal-modeling challenges. The conclusion presents DiscRec as addressing token–item misalignment and semantic–collaborative entanglement, with LLM-based recommendation and stronger disentanglement strategies left for future work.

A. Sequential Recommendation

  • Sequential recommendation progresses from Markov chains to recurrent, convolutional, and self-attention models, but the surveyed methods remain discriminative.Examples include GRU4Rec, Caser, SASRec, and BERT4Rec.Source: A. Sequential Recommendation · p. 11

B. Generative Recommendation

  • Generative recommendation tokenizes item semantics and generates predictions; surveyed directions enhance tokenization or recommendation architectures.TIGER uses RQ-VAE for semantic tokens and an encoder–decoder; LETTER adds collaborative and diversity regularization, while EAGER uses two streams.Source: B. Generative Recommendation · p. 12

C. Collaborative Information Modeling

  • Collaborative-information approaches include LLM-based systems and two-stage generative methods, but prior approaches do not resolve token–item misalignment and signal entanglement.CoLLM maps collaborative representations into an LLM embedding space; LETTER and ETEGRec add collaborative supervision to tokenization, while EAGER and SC-Rec [17] use dual-branch designs.Source: C. Collaborative Information Modeling · p. 12

VI. CONCLUSION

  • DiscRec combines item-level position embeddings with embedding-layer semantic and collaborative branches and adaptive gating, retaining cross-signal interactions in deeper layers.The conclusion characterizes it as a lightweight solution for item-aware alignment and flexible signal disentanglement and fusion.Source: §VI · p. 12
  • Future work will adapt DiscRec to LLM-based recommendation and investigate contrastive learning or mutual-information minimization for disentanglement.Source: §VI · p. 12

Related papers

Loading 2506.15576v2…