PaperlayerView exact revision on arXiv

Source-linked AI summary

Implicit Reasoning for Large Language Model-based Generative Recommendation

Yinhan He, Liam Collins, Bhuvesh Kumar, Jundong Li, Neil Shah, Donald Loveland

arXiv:2606.14142v3cs.CLcs.AI

TL;DR

Reliable access to pretrained knowledge remains poorly understood in SID-based generative recommendation. This paper analyzes explicit reasoning pipelines and proposes PAUSEREC, whose latent reasoning replaces rationales; explicit CoT underperforms simple next-item training unless followed by costly reinforcement learning.

  • Problem

    How to efficiently and effectively access LLMs’ pretrained knowledge for generative recommendation remains poorly understood, while existing multi-stage pipelines offer limited insight into their necessity.

  • Method

    PAUSEREC uses trainable <pause> tokens for latent reasoning, retaining continual pretraining and next-item fine-tuning while replacing rationale training and reinforcement learning.

  • Results

    Explicit CoT consistently underperforms simple next-item fine-tuning, with gains emerging only after expensive reinforcement-learning post-training.

  • Takeaways & Limitations

    PAUSEREC provides an effective and efficient alternative to explicit rationales by using latent reasoning to bridge language and Semantic IDs.

  • Takeaways & Limitations

    The evaluation uses offline next-item prediction and does not exhaustively tune pause placements, initialization schedules, or decoding variants.

Abstract

from arXiv · show

Large Language Models (LLMs) are increasingly adopted as backbones for Generative Recommendation (GR), promising access to pretrained world knowledge. Yet reliably invoking this knowledge for GR remains poorly understood. A key obstacle is that LLM-based GR typically represents items with Semantic IDs (SIDs), disrupting LLMs' natural-language reasoning interface because these tokens are unseen by the LLM during pretraining. Existing approaches address this with expensive multi-stage pipelines that ground SIDs and elicit explicit rationales, but offer limited insight into when and why each stage is necessary. In this work, we systematically decompose explicit reasoning training pipelines for LLM-based GR, revealing three key limitations: weakened world-knowledge verbalization, misalignment between SID and natural-language token embedding spaces, and sensitivity to rationale quality, all of which hurt explicit reasoning performance. To circumvent these issues, we propose PauseRec, a lightweight implicit reasoning paradigm tailored for GR. PauseRec is exceptionally practical, avoiding costly reasoning trace acquisition and reasoning alignment training, leading to a multitude of benefits: (1) it outperforms standard explicit CoT methods by up to 6.22%, (2) it reduces training cost by up to 65% GPU hours, and (3) it speeds up inference by up to 71.3%. These results position PauseRec as a lightweight alternative to explicit rationale generation, enabling more effective and efficient LLM-based GR.

1 Introduction

LLM-based generative recommendation relies on pretrained world knowledge but faces a fundamental mismatch between natural-language reasoning and Semantic IDs. The paper identifies limitations of explicit reasoning pipelines and proposes PAUSEREC, which uses trainable pause tokens for implicit reasoning and improves recommendation performance while reducing overhead.

  • LLM-based GR generates the next item from user history, seeking to use pretrained world knowledge for semantic inference and intent-to-item mapping.
  • Explicit GR pipelines combine continual pretraining, supervised next-item finetuning, rationale-generation training, and reinforcement-learning post-training.
  • Analysis finds that grounding recovers coarse item categories but often misses titles and fine-grained categories, while CoT SFT can make pretrained knowledge harder to verbalize and separate text from SID embeddings.
  • PAUSEREC inserts trainable <pause> tokens before SID generation, optimizing them through the final next-item objective to provide latent computation without rationale supervision.
  • 6.22% improvement over standard SFT and CoT-based methods is achieved across three Amazon review datasets while reducing training and inference overhead.

2 Preliminaries

The paper formulates generative recommendation as predicting a user’s next item from chronologically ordered interaction history, with items represented by Semantic IDs and generated conditionally from a natural-language prompt. Existing explicit-reasoning pipelines add continual pretraining, next-item SFT, rationale-generating CoT SFT, and optionally computationally expensive reinforcement-learning post-training.

  • GR predicts the next item i_n+1 from a user history H = [i_1, ..., i_n] of chronologically ordered interactions.
  • Each item i is represented by a length-L Semantic ID sequence added to the LLM vocabulary.
  • Recommendation uses conditional generation from Prompt(H), which converts interaction history into a natural-language prompt listing past items and optionally metadata.All methods share this generative formulation but differ in how reasoning is inserted before SID prediction.
  • Explicit pipelines ground item semantics through CPT, learn next-item SID prediction through SFT, and then generate natural-language rationales before the target SID through CoT SFT.CPT trains on interleaved SIDs and item descriptions with only SID token embeddings trainable; rationale construction uses templates or a teacher LLM.
  • Existing methods may add RL post-training to optimize recommendation rewards directly, although this stage is computationally expensive.

3 Contributions of the Training Stages

The training-stage analysis finds that CPT grounds SIDs only partially, recovering coarse category structure but not precise item-level semantics. CoT SFT with explicit rationales does not reliably improve SID prediction over simple next-item SFT, while performance recovery may require costly RLVR.

  • CPT: CPT recovers coarse category structure but provides only partial SID grounding rather than precise item-level understanding.One-level categories are recovered almost perfectly, whereas item titles and fine-grained categories remain weakly recovered.
  • CPT: 99.6% 1-level category accuracy and 40.5% 2-level accuracy show that CPT captures broad categorical structure.The results indicate that LLMs associate SIDs with semantics from pretraining primarily at a coarse level.
  • CoT SFT: CoT SFT alone does not reliably improve SID prediction across template-based, teacher-generated, rejection-sampled, and format-restricted rationale variants.The study evaluates these rationale types after CPT and SFT on Amazon Beauty.
  • CoT SFT: 0.0524 vs. 0.0533 Hit@5 shows the strongest reported CoT SFT result remains below the simple next-item SFT baseline.Weaker teacher-generated variants lose over 20% relative Hit@5.
  • CoT SFT: RLVR can recover performance after CoT SFT, but it requires multiple rollout trajectories per step and is therefore costly.The passage contrasts this requirement with CoT’s success on language tasks and the limited gains from CoT SFT alone.

4 Diagnosis of CoT SFT Limitations

Diagnostic studies identify three limitations of explicit CoT SFT for generative recommendation: weakened verbalization of retained world knowledge, geometric misalignment between text and SID embeddings, and brittle dependence on rationale wording. Together, these failures limit rationale influence on SID logits and motivate implicit reasoning in latent space.

  • Weakened world-knowledge verbalization: CoT SFT preserves answer information in logits but makes world knowledge difficult to verbalize, causing text-match accuracy to collapse while logit-based accuracy remains near the base model.The diagnosis uses Qwen3-1.7B on MMLU, HellaSwag, PIQA, and ARC-Challenge with exact-choice generation and highest-logit evaluation.
  • Text–SID embedding misalignment: SID and natural-language token embeddings diverge across training stages, making it difficult for the LLM to unify language and SIDs under a coherent rationale.PCA comparisons after SID initialization, CPT, SFT, and CoT SFT show increasing separation, while natural-language rationales only weakly shift SID logits.
  • Rationale sensitivity: Recommendation performance becomes highly sensitive to rationale text, with removing the target category more than halving performance and small word-level edits also changing accuracy.Evaluations on Amazon Beauty measure Hit@5 and NDCG@5 under category removal, five-word deletion, and five-word noise insertion.
  • Text–SID embedding misalignment: Explicit CoT has limited leverage on final recommendation because natural-language rationales only weakly affect logits over SID tokens.This limitation follows from the geometric separation between text and SID embedding spaces.
  • Motivation for implicit reasoning: These failures motivate implicit latent-space reasoning, where learned <pause> tokens bridge language and SIDs without decoding brittle intermediate reasoning text.The proposed bridge addresses weakened decoding, text–SID misalignment, and sensitivity to small rationale edits.

5 Methodology: PAUSEREC

PAUSEREC replaces explicit CoT SFT and RL with latent pause-based computation while retaining CPT and next-item SFT. It pretrains a bridge token between language and SID representations, then optimizes only target SID prediction without rationale supervision.

  • Core design: PAUSEREC retains CPT and next-item SFT but replaces CoT SFT and RL with pause-based latent computation.This design is proposed as an implicit reasoning method for LLM-based generative recommendation.
  • Pause pretraining: The method pretrains <pause> on CPT text with random insertions so it learns semantic transitions between language and SID tokens.The pause branch runs in parallel with next-item SFT from the same CPT checkpoint, with updates concentrated on the pause embedding while other parameters remain frozen.
  • Implicit reasoning SFT: Implicit reasoning SFT inserts k pauses between user history and the target SID, masks pause-position loss, and optimizes only target SID tokens.This lets pause slots serve as task-specific latent scratch space without imitating a fixed teacher rationale distribution.
  • Motivation: PAUSEREC addresses CoT failures through latent computation without verbalization, CPT-grounded embedding-space bridging, and rationale-free supervision.Loss masking on pause positions preserves computation while avoiding direct rationale supervision.
  • Implementation: 10% of each sequence is covered by randomly inserted <pause> tokens during pretraining, and only the pause embedding is trainable.The pause embedding is initialized at the vocabulary-center mean after CPT with variance scaled by 10^-9.

6 Experiments

Across three Amazon review datasets, PAUSEREC improves recommendation effectiveness over next-item SFT, RL-based CoT, and non-LLM baselines while substantially reducing training and inference costs. Ablations and attention analysis show that pretrained pause tokens bridge text and SID spaces, moderate pause counts work best, and latent computation proceeds in stages.

  • Effectiveness: 8.85%: PAUSEREC’s largest relative gain over next-item SFT occurs on Toys Hit@5, while improving every reported metric.The improvement is measured relative to the next-item SFT baseline.
  • Effectiveness: 6.22%: PAUSEREC’s largest relative improvement over OneRec-Think occurs on Toys Hit@5, with gains on 10 of 12 metrics.OneRec-Think remains higher on Beauty Hit@10 and NDCG@10; PAUSEREC wins all Sports and Toys metrics and consistently outperforms non-LLM baselines.
  • Efficiency: 65%: PAUSEREC reduces training GPU hours and is roughly 3.5× faster per inference sample than OneRec-Think on Amazon Beauty.The savings come from avoiding RL post-training and natural-language rationale generation, and grow with longer generated rationales.
  • Ablations: Pretrained pause-token initialization achieves the best Hit@5 and NDCG@5, outperforming text-only, SID-only, and default initializations.This supports using CPT-grounded pause pretraining to bridge text and SID embedding spaces.
  • Ablations: k=5: Moderate pause counts work best, while increasing to k=10 does not consistently improve performance, suggesting latent computation saturates.The same pause count is used during implicit SFT and inference.
  • Attention analysis: Early pause tokens attend broadly to the prompt and history boundary, whereas later pauses focus on fewer historical SID tokens.This staged attention transition provides an explanation for how latent pause computation improves generative recommendation.

7 Related Work

Prior work uses LLMs for ranking, feature extraction, and generative recommendation with Semantic IDs (SIDs), typically grounding SIDs through continued pretraining before next-item supervised fine-tuning. This work builds on that foundation to examine when pretrained world knowledge improves SID prediction beyond standard training.

  • LLM-based GR: LLM-based generative recommenders commonly output Semantic IDs, alongside using LLMs as rankers or feature extractors.The cited prior work includes Hou et al. (2023), Rajput et al. (2024), and Hua et al. (2024).
  • LLM-based GR: Typical LLM-based GR pipelines apply continued pretraining on item-text corpora to ground SIDs, followed by next-item supervised fine-tuning.This foundation is associated with Bao et al. (2023).
  • LLM-based GR: The paper investigates when pretrained world knowledge improves SID prediction beyond standard training.

8 Conclusion

The paper concludes that explicit rationales are a poor interface for SID-based generative recommendation because weakened verbalization, text–SID embedding mismatch, and rationale sensitivity limit CoT SFT. PAUSEREC instead uses trainable <pause> tokens to enable latent reasoning that bridges language and SIDs, yielding an effective and efficient approach.

  • 8 Conclusion: Explicit rationales limit CoT SFT for SID-based generative recommendation through weakened verbalization, text–SID embedding mismatch, and rationale sensitivity.LLMs retain useful signals, but these three limitations make explicit rationales a poor interface.
  • 8 Conclusion: PAUSEREC replaces rationales with trainable <pause> tokens that enable latent reasoning bridging language and SIDs.The approach is designed to connect the language interface with semantic item identifiers without relying on explicit rationales.
  • 8 Conclusion: Extensive experiments show that PAUSEREC is effective and efficient.The conclusion characterizes PAUSEREC as achieving both effectiveness and efficiency.

A Limitations and Potential Risks · B AI Usage · C Artifacts

The paper identifies remaining evaluation and design limitations, clarifies the bounded use of AI assistance, and describes an artifact release covering PAUSEREC’s implementation and experiments.

  • A Limitations and Potential Risks: PAUSEREC uses a compact pause-token design and reports sensitivity to pause length.The study does not exhaustively tune all possible pause placements, initialization schedules, or decoding variants.
  • A Limitations and Potential Risks: The evaluation follows the standard offline next-item prediction protocol rather than complementary user-facing studies.Future studies could examine perceived usefulness, diversity, and recommendation presentation.
  • A Limitations and Potential Risks: The paper leaves pause placement, initialization schedules, and decoding variants as natural extensions for future work.These design dimensions are not exhaustively tuned in the reported study.
  • B AI Usage: AI writing assistance was limited to polishing grammar, clarity, and sentence flow.The authors developed, checked, and approved the research ideas, designs, analyses, results, and final claims.
  • C Artifacts: The artifact release includes code and scripts for SID-based training-data construction, PAUSEREC training stages, constrained SID decoding, and reported tables and figures.These materials support the paper’s training and evaluation workflow.
  • C Artifacts: The implementation extends the open-source OneRec-Think repository with pause-token pretraining, implicit-reasoning finetuning, evaluation utilities, and experiment orchestration.The passage identifies OneRec-Think as the implementation base and notes its release under the Ap… text truncates before the license is completed.

D Theoretical Analysis of Text–SID Separation … F.3 SID Metadata Decoding Prompt

The paper theoretically bounds how text–SID separation limits explicit rationale effects on SID ranking, while empirical and implementation details show how PauseRec uses pause tokens, prompt formats, attention analysis, and metadata decoding. Together, these sections characterize PauseRec’s bridge between natural-language and SID spaces and specify its evaluation procedures.

  • D Theoretical Analysis of Text–SID Separation: Smaller text–SID coupling ρ indicates stronger separation between rationale-induced hidden-state movement and SID-discriminative directions.The coupling coefficient is defined through projection onto the SID-discriminative subspace.
  • D Theoretical Analysis of Text–SID Separation: Theorem 1 shows that text-induced hidden-state changes can alter relative SID logits only through their projection onto SID-discriminative directions, up to the residual bound.The rationale perturbation is decomposed as ∆= ∆text + r, with ∆text ∈Utext and ∥r∥≤ϵ.
  • D Theoretical Analysis of Text–SID Separation: If the target SID initially trails a competitor by margin γ exceeding the rationale’s maximum margin change M, the rationale cannot make the target outrank that competitor.The resulting margin remains negative, so the target SID receives a lower logit than the competing SID.
  • E PAUSEREC Embedding Visualization: Across CPT, pause-token pretraining, next-item SFT, and implicit-reasoning SFT, the <pause> token remains at the boundary between natural-language and SID embedding clusters.This boundary placement provides empirical evidence that <pause> connects semantic information across the two embedding spaces.
  • F.1 Sample Prompts for PAUSEREC Training and Inference: During pause pretraining, <pause> tokens are inserted at random word boundaries, while implicit-reasoning SFT uses pause-token blocks before the target SID.The supplied example uses 10% random <pause> insertion with seed 42 and an implicit-reasoning block containing k=5 pause tokens.
  • F.2 Prompt for Qualitative Attention Analysis: The qualitative attention example examines whether later pause steps retrieve history items related to the target through hair-care intent.Attention increases toward Natures Bounty Optimal Solutions Hair, Skin and Nails Gummies, whose SID begins with <s_a_206><s_b_60>.
  • F.3 SID Metadata Decoding Prompt: Metadata recovery uses the first two items from each dataset’s pretraining file as in-context examples and evaluates remaining-item predictions by exact string match.No judge model is used; generated Title: and Category: fields are parsed, and decoding is greedy with do_sample=False and max_new_tokens=256.

F.4 Sample Rationales Used in CoT SFT … G.1 Inference Speed of CoT SFT Variants

The supplementary sections document CoT rationale templates and prompts, diagnose generated reasoning behavior, summarize benchmark scale, and show that PAUSEREC substantially reduces inference latency. They also specify the fixed-pause-token design and its selected tradeoff across datasets and metrics.

  • F.4 Sample Rationales Used in CoT SFT: CoT SFT samples include category-only, free-form, rejection, and restricted rationales, with examples linking beauty purchase histories to mirrors or hair clips.The restricted example structures the rationale into three explicit steps from purchase history to grooming tools and a lighted cosmetic mirror.
  • F.5 Teacher Model Prompts for Reasoning Generation in CoT SFT: The teacher-model prompts request concise purchase-prediction reasoning from history and ground truth, or require inferring need before verifying the target among candidates.The two prompt formats are free-form reasoning and format-restricted predictive rationale generation.
  • F.6 General Language Benchmark Prompts and Generated Reasoning: The diagnostic language benchmark uses chat-style multiple-choice prompts that open a <think> block, with five MMLU demonstrations and zero-shot HellaSwag, PIQA, and ARC-Challenge.HellaSwag adds a sentence-completion prefix, PIQA presents two solutions, and ARC-Challenge normalizes options to A/B/C/D.
  • F.6 General Language Benchmark Prompts and Generated Reasoning: 27,094 target examples produced non-empty reasoning fields, but generations usually contained short answer-likelihood statements rather than multi-step rationales.In 27,074 of 27,094 examples, text after </think> began with SID tokens; the four most frequent reasoning strings covered answer letters C, B, D, and A.
  • F.7 Dataset Statistics: The three Amazon benchmarks contain 11.9K–18.4K items and 167K–296K interactions after preprocessing, covering recommendation tasks at different scales.These statistics are reported in Table 8.
  • G.1 Inference Speed of CoT SFT Variants: CoT SFT variants are substantially slower than PAUSEREC because they autoregressively generate rationale tokens before constrained SID decoding.PAUSEREC inserts a fixed number of <pause> tokens and proceeds directly to constrained SID decoding.
  • G.1 Inference Speed of CoT SFT Variants: 3.5× is the approximate slowdown for the shortest template rationale versus PAUSEREC, while longer-template and teacher-generated variants are roughly 5.5–7.1× slower.The benchmark used 500 Beauty samples on a single NVIDIA A100-SXM4-80GB GPU with batch size 16 and greedy decoding.
  • G.1 Inference Speed of CoT SFT Variants: k = 5 gives the best overall tradeoff among evaluated pause-token counts, although no single value dominates every dataset and metric.This setting is used in the main experiments.

G.2 Parameter Analysis

PauseRec’s pause-count analysis finds that k=5 is the most robust setting across datasets and metrics, while even one or three pauses are competitive and ten pauses offer no consistent gains.

  • Pause-count analysis: k=5 pauses are best or tied for best on 9 of 12 metrics and remain close to the best result on the others.The optimal pause count varies across datasets and metrics, but k=5 is the most robust setting.
  • Pause-count analysis: One or three pauses are already competitive, indicating that a small latent computation window is useful.Pause-based reasoning does not require a large number of pause tokens to perform well.
  • Pause-count analysis: Ten pauses provide no consistent additional gains and sometimes slightly hurt performance.Increasing the pause count beyond the robust setting can reduce performance on some metrics.
Loading 2606.14142v3…