PaperlayerView exact revision on arXiv

Source-linked AI summary

DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendation

Yu Wang, Zhiwei Liu, Jianguo Zhang, Weiran Yao, Shelby Heinecke, Philip S. Yu

arXiv:2312.11336v1cs.IRcs.AI

TL;DR

Sequential recommendation requires handling collaborative signals, noisy and personalized preferences, and interests that evolve over time—challenges existing prompting strategies do not fully address. DRDT combines collaborative demonstrations, multi-aspect preference analysis, and dynamic reflection; it improves performance across six LLMs and three datasets.

  • Problem

    Existing prompting strategies do not fully address sequential recommendation’s collaborative signals, noisy personalized interactions, and evolving user preferences.

    Source: §1 · p. 2, §4.4 · p. 7

  • Method

    DRDT uses a retriever-reranker framework with collaborative demonstrations, multi-aspect preference analysis, and iterative probing, critiquing, and reflecting.

    Source: §1 · p. 2, §4.4 · p. 8, §6 · p. 13

  • Results

    DRDT produced notable performance improvements across six LLMs and three datasets, and the evaluation reports the principle’s effectiveness.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The findings underscore the importance of developing tailored reasoning strategies for sequential recommendation.

    Source: §1 · p. 3, §6 · p. 13

    Divergent thinking alone can oversimplify nuanced preferences, and flawed internal reasoning can lead to hallucinations.

    Source: §4.4 · p. 8

Abstract

from arXiv · show

The rise of Large Language Models (LLMs) has sparked interest in their application to sequential recommendation tasks as they can provide supportive item information. However, due to the inherent complexities of sequential recommendation, such as sequential patterns across datasets, noise within sequences, and the temporal evolution of user preferences, existing LLM reasoning strategies, such as in-context learning and chain-of-thought are not fully effective. To address these challenges, we introduce a novel reasoning principle: Dynamic Reflection with Divergent Thinking within a retriever-reranker framework. Our approach starts with a collaborative in-context demonstration retriever, which collects sequences exhibiting collaborative behaviors as in-context examples. Following this, we abstract high-level user preferences across multiple aspects, providing a more nuanced understanding of user interests and circumventing the noise within the raw sequences. The cornerstone of our methodology is dynamic reflection, a process that emulates human learning through probing, critiquing, and reflecting, using user feedback to tailor the analysis more effectively to the target user in a temporal manner. We evaluate our approach on three datasets using six pre-trained LLMs. The superior performance observed across these models demonstrates the efficacy of our reasoning strategy, notably achieved without the need to fine-tune the LLMs. With our principle, we managed to outperform GPT-Turbo-3.5 on three datasets using 7b models e.g., Vicuna-7b and Openchat-7b on NDCG@10. This research not only highlights the potential of LLMs in enhancing sequential recommendation systems but also underscores the importance of developing tailored reasoning strategies to fully harness their capabilities.

1 INTRODUCTION

The introduction identifies gaps in existing prompting for sequential recommendation and presents DRDT, a retriever-reranker approach that combines collaborative signals with temporal preference reasoning. It evaluates the principle across six LLMs and three datasets.

  • Sequential recommendation requires collaborative signals, personalized aspect handling despite noisy interactions, and modeling preferences that change over time.Source: §1 · p. 2
  • The collaborative demonstration retriever finds sequences ending with the target user’s last item and supplies their next-item choices as in-context references.Source: §1 · p. 2
  • Divergent thinking analyzes user engagement across multiple aspects rather than relying on a single predefined reasoning path or item similarity.Source: §1 · p. 3
  • Dynamic reflection repeatedly probes a next-item prediction, critiques it using user feedback, and revises the analysis as the sequence advances.Source: §1 · p. 3
  • DRDT combines a retriever-reranker framework with collaborative signals and temporal preference reasoning to address these challenges.Source: §1 · pp. 2–3
  • DRDT produced notable performance improvements across six LLMs and three datasets, and the evaluation reports the principle’s effectiveness.Source: §1 · p. 3

2 RELATED WORK

Related work covers LLM-based sequential recommendation, chain-of-thought prompting, and self-verification. The authors position their work as a deeper investigation of prompt design for sequential recommendation.

2.1 LLM FOR FEW-SHOT SEQUENTIAL RECOMMENDATION

  • Prior LLM-based sequential recommendation uses zero-shot and few-shot prompting, preference analysis, candidate ranking, graph enrichment, or fine-tuned models.The approaches include historical interactions with candidates, masked-history completion, and preference summaries used to select items.Source: §2.1 · p. 4

2.2 COT REASONING IN LARGE LANGUAGE MODELS

  • Chain-of-thought methods guide answers along reasoning paths, including zero-shot, sampled, ensembled, and retrieval-interleaved variants.Source: §2.2 · pp. 4–5

2.3 SELF-VERIFICATION IN LARGE LANGUAGE MODELS

  • Self-verification methods refine or evaluate model outputs, but LLM-generated feedback can hallucinate and self-criticism is not always beneficial.The authors argue that recommendation sequences provide human feedback for correcting generation; they identify deeper prompt design as an open focus.Source: §2.3 · p. 5

4 METHODOLOGY

The methodology combines a collaborative retriever with a reranker that analyzes preferences over time, using multi-aspect abstraction and feedback-driven reflection to guide recommendations.

4.1 RETRIEVER-RERANKER FRAMEWORK

  • The retriever supplies relevant collaborative behavior as in-context references, while the reranker analyzes user preferences and reorders a preselected candidate set.The framework addresses noise and LLM context limits by selecting informative examples and reranking candidates rather than the full item set.Source: §4.1 · pp. 5–6

4.2 COLLABORATIVE IN-CONTEXT DEMONSTRATION RETRIEVER

  • The collaborative retriever finds other users whose histories end with the target user’s last item and uses their next purchases as ranking cues.The retrieved sequences provide collaborative context and indicate which candidates should rank higher.Source: §4.2 · pp. 6–7

4.3 DIVERGENT THINKING (DT)

  • Divergent Thinking abstracts preferences across multiple aspects to guide prediction and reduce the influence of noisy raw interactions.The LLM identifies relevant aspects for the dataset, then generates a preference analysis for each aspect for use during reranking.Source: §4.3 · p. 7

4.4 DYNAMIC REFLECTION WITH USER FEEDBACK (DR)

  • DT can oversimplify complex, evolving histories, hallucinate from flawed reasoning, and leave prioritization among preference aspects unresolved.These limitations motivate the feedback-driven reflection scheme.Source: §4.4 · p. 8
  • Dynamic reflection iteratively probes, critiques, and reflects on choices from preceding items, revising aspect emphasis with observed engagements before reranking the target.The process repeats for T steps, using the actual next item as feedback and asking which aspect may have influenced the choice.Source: §4.4 · p. 8

5 EXPERIMENTS

Experiments compare DRDT with plain-text, ICL, and COT on three datasets using pre-trained LLM rerankers, then examine component effects, reflection-step counts, and a case study.

5 EXPERIMENTS

  • The evaluation uses Movielens-1M and Amazon Games and Luxury Beauty, with Recall@K and NDCG@K at K=10 and 20.The study samples 200 users per dataset and reranks 20 candidates: the target item plus 19 randomly sampled items.Source: §5 · pp. 8–9

5.1 OVERALL COMPARISON

  • ICL and COT preferences vary across models and datasets; Vicuna-13b benefits from ICL on ML-1M and Games but not Luxury.For Mistral-7b and Longchat-7b, ICL and COT generally underperform plain-text except on Luxury.Source: §5.1 · p. 9
  • DRDT significantly outperforms plain-text, ICL, and COT across all datasets, improving ranking for all LLMs on almost all metrics.Openchat-7b also surpasses GPT-Turbo-3.5 with plain-text prompting on Recall@10, NDCG@10, and NDCG@20 across ML-1M, Games, and Luxury.Source: §5.1 · p. 9, Table 2 · p. 10

5.2 ABLATION STUDY

  • Removing Dynamic Reflection, Collaborative In-Context, or Divergent Thinking causes a significant performance drop.The ablation results identify each component as important to overall performance.Source: §5.2 · p. 10
  • Combining Collaborative In-Context with Divergent Thinking consistently improves Luxury performance but has lesser or negative effects on ML-1M and Games.Without Dynamic Reflection to regulate analysis through probing, the combination can perform worse than either component alone.Source: §5.2 · p. 10
  • Divergent Thinking combined with Dynamic Reflection improves performance on ML-1M and Luxury, with a lesser impact on Games.The authors suggest collaborative information may matter more than evolving behavior patterns in some contexts.Source: §5.2 · p. 10

5.3 HYPERPARAMETER SENSITIVITY

  • Performance metrics trend upward as reflection steps increase, with the tested maximum capped at three steps.One step is a prediction-probing and analysis-reflection cycle using a ground-truth item at a specific sequence position; the cap reflects context-length and sequence-length constraints.Source: §5.3 · p. 12

5.4 CASE STUDY

  • In the case study, DRDT ranks the ground-truth item Judge Dredd first, surpassing the other prompting strategies.Plain-text and COT tended to regenerate historically similar items, while ICL improved task comprehension but did not fully capture personalized reranking nuances.Source: §5.4 · pp. 12–13

6 CONCLUSION

The conclusion presents DRDT as a retriever-reranker reasoning principle for sequential recommendation. It combines collaborative demonstrations, multi-aspect preference abstraction, and temporal reflection to improve ranking without fine-tuning.

6 CONCLUSION

  • DRDT uses a Retriever-Reranker framework whose collaborative in-context demonstration retriever collects collaborative sequences from the dataset.Source: §6 · p. 13
  • Divergent thinking abstracts high-level user preferences across multiple aspects, while dynamic reflection iterates through probing, critiquing, and reflecting on preference analysis.The iteration is intended to encourage temporal reasoning.Source: §6 · p. 13
  • The reasoning principle improves LLM ranking performance without requiring fine-tuning.Source: §6 · p. 13
Loading 2312.11336v1…