PaperlayerView exact revision on arXiv

Source-linked AI summary

Token-level Collaborative Alignment for LLM-based Generative Recommendation

Fake Lin, Binbin Hu, Zhi Zheng, Xi Zhu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Tong Xu

arXiv:2601.18457v1cs.IR

TL;DR

LLM recommenders have difficulty incorporating collaborative filtering because CF models preferences at the item level while LLMs optimize token-level predictions. TCA4Rec maps CF logits into token distributions and uses them with one-hot labels in soft next-token training. The paper reports consistent improvements over strong baselines and stronger alignment with CF signals.

  • Problem

    CF models learn item-level preferences, whereas LLMs optimize token-level next-token prediction, leaving no natural objective-level interface for collaborative alignment.

    Source: §1 · pp. 1–2

  • Method

    TCA4Rec converts item-level CF logits into token-level distributions and combines them with one-hot supervision to optimize a soft next-token prediction loss.

    Source: §1 · p. 2

  • Results

    Experiments show consistent improvements over strong baselines and validate that TCA4Rec enhances alignment with CF signals.

    Source: §6 · p. 9

  • Takeaways & Limitations

    TCA4Rec is presented as a model-agnostic, plug-and-play approach compatible with traditional CF models and decoder-based generative recommenders, with an explicit trade-off between collaborative alignment and semantic fluency.

    Source: §1 · p. 2

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated strong potential for generative recommendation by leveraging rich semantic knowledge. However, existing LLM-based recommender systems struggle to effectively incorporate collaborative filtering (CF) signals, due to a fundamental mismatch between item-level preference modeling in CF and token-level next-token prediction (NTP) optimization in LLMs. Prior approaches typically treat CF as contextual hints or representation bias, and resort to multi-stage training to reduce behavioral semantic space discrepancies, leaving CF unable to explicitly regulate LLM generation. In this work, we propose Token-level Collaborative Alignment for Recommendation (TCA4Rec), a model-agnostic and plug-and-play framework that establishes an explicit optimization-level interface between CF supervision and LLM generation. TCA4Rec consists of (i) Collaborative Tokenizer, which projects raw item-level CF logits into token-level distributions aligned with the LLM token space, and (ii) Soft Label Alignment, which integrates these CF-informed distributions with one-hot supervision to optimize a soft NTP objective. This design preserves the generative nature of LLM training while enabling collaborative alignment with essential user preference of CF models. We highlight TCA4Rec is compatible with arbitrary traditional CF models and generalizes across a wide range of decoder-based LLM recommender architectures. Moreover, it provides an explicit mechanism to balance behavioral alignment and semantic fluency, yielding generative recommendations that are both accurate and controllable. Extensive experiments demonstrate that TCA4Rec consistently improves recommendation performance across a broad spectrum of CF models and LLM-based recommender systems.

1 Introduction

LLM recommenders face a mismatch between semantic modeling and collaborative behavior, while existing approaches lack a direct optimization-level interface for CF signals. TCA4Rec projects CF preferences into token distributions and soft-label next-token training while preserving generative learning.

1 Introduction

  • Soft-prompt methods face behavioral–semantic space mismatch and may require multi-stage or curriculum training, while semantic-ID codebooks can suppress user-specific preferences.The codebooks primarily reflect semantic similarity rather than collaborative structure.Source: §1 · p. 2
  • CF item-ranking objectives and LLM token-level next-token prediction differ in granularity, leaving no natural objective-level interface for collaborative alignment.CF used only as contextual hints or representation bias cannot directly steer the optimization trajectory.Source: §1 · p. 2
  • TCA4Rec projects item-level CF logits into token-level distributions and combines them with one-hot supervision for soft next-token training.The framework is presented as model-agnostic and plug-and-play.Source: §1 · p. 2
  • TCA4Rec integrates CF from traditional models with decoder-based generative recommenders and offers an explicit way to trade collaborative alignment against semantic fluency.The paper names SASRec and BERT4Rec as CF examples and TallRec, LLaRA, CoLLM, and MSL as compatible architectures.Source: §1 · p. 2
  • Experiments on three public datasets show TCA4Rec outperforming baselines and enhancing alignment with CF signals.The paper frames these experiments as evidence of the framework's effectiveness.Source: §1 · p. 2

2 Preliminaries

Sequential recommendation predicts a user's next item from interaction history. The LLM-based task reformulates this as target-item-title prediction, and Figure 1 outlines how CF signals enter the generative training pipeline.

2.1 Sequential Recommendation Task

  • Sequential recommendation trains a model to infer a user's next item from the user's profile and interaction history.The sequential recommender's parameters are denoted θ_Rec.Source: §2.1 · p. 2

2.2 LLM-based Recommendation Task

  • LLM-based recommendation reformulates next-item prediction as predicting the target item's title from a prompt populated with historical-item metadata.The target title is represented as a token sequence and trained with the standard autoregressive next-token objective.Source: §2.2 · pp. 2–3
  • Figure 1 shows CF item logits becoming token-level distributions, which are fused with one-hot supervision before soft next-token fine-tuning.The sequence links item-level collaborative signals to the LLM's generative objective.Source: Figure 1 · p. 3

3 Methodology

TCA4Rec converts item-level collaborative-filtering signals into token-level distributions and combines them with next-token supervision. Its two components—the Collaborative Tokenizer and Soft Label Alignment—make collaboration part of the generative training objective while preserving next-token prediction.

3 Methodology

  • The Collaborative Tokenizer transfers collaborative signals from item-level logits to token-level distributions, making them compatible with language-model training.The framework pairs this transformation with Soft Label Alignment, which fuses the distributions with one-hot labels for a soft next-token prediction loss.Source: §3.1 · p. 3, §3 · p. 3

3.1.1 Collaborative Logits Derivation.

  • User and item embeddings feed a scoring function that produces collaborative logits reflecting user preference over items.The scoring function depends on the recommendation model; dot product is used for SASRec to rank items by user preference.Source: §3.1.1 · pp. 3–4

3.1.2 Token-level Distribution Transformation.

  • At each decoding position, the tokenizer retains items matching the generated prefix, softmax-normalizes their logits, and aggregates probabilities for items sharing the same next token.This yields a normalized token-level distribution from the collaborative signals.Source: §3.1.2 · p. 4

3.2 Soft Label Alignment

  • Soft Label Alignment fuses the collaborative token distribution with the one-hot ground-truth label and trains with a soft cross-entropy objective.The resulting objective retains ground-truth next-token supervision while accounting for collaborative knowledge.Source: §3.2 · p. 4

3.3.1 Auxiliary KL Formulation.

  • The alternative auxiliary KL objective explicitly encourages the LLM predictive distribution to align with the collaborative distribution.The notation names the collaborative token-level distribution p_j(v) and the model prediction P_j(v).Source: §3.3.1 · p. 5

3.3.2 Gradient Analysis.

  • Soft-NTP uses an adaptive distribution q_j depending on the ground-truth target and current model prediction, unlike auxiliary KL’s fixed target weighting.The authors describe this adaptivity as balancing collaborative signals with the LLM’s world knowledge.Source: §3.3.2 · p. 5

4 Experiment

Across three datasets, TCA4Rec surpasses strong baselines and improves diverse LLM-based recommenders. Further experiments show its gains depend on CF signal quality and the balance between collaborative consistency and performance.

4.1.1 Dataset.

  • The experiments use Toys, Sports, and Office data, filtered to 5-core interactions and requiring at least 10 historical items per interaction.Chronologically ordered data is split 8:1:1 into training, validation, and test sets.Source: §4.1.1 · p. 5

4.1.2 Baselines.

  • The study compares varied recommendation baselines and enhances four representative LLM-based recommendation algorithms with TCA; detailed implementation settings and metrics are in Appendix B.The detailed baseline descriptions are in Appendix A.Source: §4.1.2 · p. 5

4.2 Overall Performance

  • TCA4Rec surpasses strong and state-of-the-art baselines and consistently improves four LLM-based recommendation backbones across three datasets.The enhanced backbones are TallRec, LLaRA, CoLLM, and MSL.Source: Table 3 · p. 6, §4.2 · p. 6

4.3 Model-Agnostic Capability

  • TCA improves SID-based methods across all three datasets, while title-only SID generation limits their performance and LETTER outperforms TIGER.The authors attribute LETTER's advantage to its use of CF signals during Semantic ID generation.Source: Table 4 · p. 6, §4.3 · p. 6

4.4 Collaborative Consistency

  • Collaborative consistency rises with α, but performance peaks and then declines as α grows further.The authors attribute the decline at excessive α to noise and errors in CF signals; fine-tuning also raises consistency without collaborative signals.Source: Figure 2 · p. 7, §4.4 · p. 7

4.5 Ablation Study

  • Removing the Collaborative Tokenizer causes a substantial performance drop, while Soft-Label Alignment outperforms the auxiliary KL-loss variant.Without the tokenizer, the token-level distribution becomes uniform; the KL-loss variant still outperforms the MSL baseline.Source: Table 5 · p. 7, §4.5 · p. 7

4.6.1 Training Size.

  • As SASRec training data increases, TCA4Rec improves with the stronger CF signals, whereas CoLLM shows slight performance degradation.Only SASRec's training size varies; CoLLM and TCA4Rec remain trained on 10K samples.Source: Table 6 · p. 8, §4.6.1 · p. 8

4.6.2 CF Models.

  • Stronger CF models yield greater TCA performance improvements, while weaker models can supply unreliable signals that impair performance.The comparison uses BERT4Rec under the MSL framework alongside the other CF model.Source: §4.6.2 · p. 8

5 Related Work

Related work spans generative LLM recommenders and LLMs used as recommendation enhancers; hybrid approaches also inject collaborative signals, though they retain limitations in modeling them explicitly.

5.1 LLM-based Recommendation

  • LLM-based recommendation methods are grouped into systems that generate recommendation results and systems that use LLMs as multimodal encoders for downstream tasks.The first category includes generative recommendation approaches; the second uses LLM-derived representations in existing recommender systems.Source: §5.1 · p. 8, §5.1.1 · p. 8, §5.1.2 · p. 8

5.1.1 LLM as Recommender.

  • Generative LLM recommenders progressed from in-context learning to fine-tuning and interest modeling, but still struggle to explicitly capture user–item collaborative signals.Examples include TallRec [3], iLoRA [12], MSL [30], and HatLLM [4].Source: §5.1.1 · p. 8

5.1.2 LLM as enhancer.

  • LLM-enhancer methods supply fine-grained representations to downstream recommenders, but overlook LLM reasoning capabilities.Examples include LLMEmb [20] and AlphaRec [26].Source: §5.1.2 · p. 8

5.2 Collaborative Signals in LLMRec

  • Hybrid methods inject collaborative signals through prompts, binary codes, or semantic-ID tokenizers, yet limitations remain in explicitly and sufficiently modeling those signals.Examples include Collm [41], LLaRA [14], BinLLM [40], TIGER [24], LC-Rec [43], LETTER [35], and ETEGRec [19].Source: §5.2 · pp. 8–9

6 Conclusion

TCA4Rec injects collaborative filtering signals into LLMs through Collaborative Tokenizer and Soft Label Alignment, bridging item-level preferences and token-level supervision while preserving generative ability.

6 Conclusion

  • TCA4Rec combines Collaborative Tokenizer and Soft Label Alignment to bridge item-level CF preferences with token-level supervision while preserving LLM generative ability.Experiments show consistent improvements over strong baselines and enhanced alignment with CF signals.Source: §6 · p. 9

A Baselines Details

The baselines include traditional collaborative and sequential recommenders, models aligning text and collaborative representations, and LLM-based generative recommendation systems.

A Baselines Details

  • Traditional baselines include LightGCN [8], SASRec [11], and BERT4Rec [28], covering neighborhood aggregation and sequential attention or masked modeling.AlphaRec [27] maps textual metadata representations into collaborative space for recommendation.Source: A Baselines Details · p. 10
  • RLMRec [25] aligns collaborative relational embeddings and language-model semantic representations through contrastive and generative techniques.TIGER [24] uses RQ-VAE semantic IDs as autoregressive retrieval targets, while LETTER [35] integrates collaborative signals into item tokenization.Source: A Baselines Details · p. 10
  • LLM-based generative baselines include TallRec [3], LLaRA [14], Collm [41], and MSL [30], using instruction tuning, hybrid prompts, CF embeddings, or token masking.Source: A Baselines Details · p. 10

B.1 Implementation Details

Implementation uses Llama3.2-3B with SASRec, LoRA fine-tuning, and specified training and evaluation settings.

B.1 Implementation Details

  • Experiments use Llama3.2-3B [7] as the backbone and SASRec [11] as the base recommender, fine-tuning the LLM with LoRA [9].LoRA uses rank 8, alpha 16, and dropout 0.05; experiments run on an RTX A6000.Source: §B.1 · p. 10
  • Traditional ID-based methods use embedding dimension 64, while large-model methods use AdamW [21] with a linear learning-rate schedule initialized at 1e-4.Source: §B.1 · p. 10
  • LLM-related methods train for at most 10 epochs with evaluation every 2 epochs and early stopping after two consecutive evaluations without improvement.Historical item sequences are length 10 and maximum token length is 2048.Source: §B.1 · p. 11

B.2 Evaluation Metrics

Performance is assessed with HR and NDCG at k in [10], while inference constrains generated text to items in the candidate list.

B.2 Evaluation Metrics

  • HR measures whether the ground-truth item appears within the top-k results.Source: §B.2 · p. 11
  • NDCG evaluates the quality of the ground-truth item's ranking position.Source: §B.2 · p. 11

B.3 Inference Setting

  • A constrained prefix tree ensures inference text corresponds to an item in the candidate list.Source: §B.3 · p. 11

C Derivation of Auxiliary KL Loss

The appendix outlines single-position derivatives for the auxiliary KL and soft-NTP losses. It defines token softmax probabilities and logits and gives the soft-NTP derivative in terms of p_j(k) and q_j(k).

D Gradient of Auxiliary KL

  • The auxiliary KL derivation starts at a single position and proceeds to a derivative with respect to a token logit.The passage defines the softmax probability and token logit used in the derivation.Source: D Gradient of Auxiliary KL · p. 11

E Gradient of Soft NTP

Related papers

Loading 2601.18457v1…