PaperlayerView exact revision on arXiv

Source-linked AI summary

Lusifer: LLM-based User SImulated Feedback Environment for online Recommender systems

Danial Ebrat, Eli Paradalis, Luis Rueda

arXiv:2405.13362v4cs.IRcs.LG

TL;DR

Static benchmarks do not capture evolving user preferences, motivating Lusifer, an LLM-based simulator that updates user profiles and explains changes for RL recommender systems. Baselines show stronger standard predictive metrics, while Lusifer performs better for most metrics in a cold-start setting with Gemma:12B and users with fewer than 10 prior interactions.

  • Problem

    Static benchmarks do not capture how user preferences shift as people encounter new item attributes or styles.

    Source: §I · p. 2

  • Method

    Lusifer simulates evolving user feedback by updating profiles at each interaction and providing explanations, in a platform for training and evaluating RL recommender agents.

    Source: §I · p. 2

  • Results

    Traditional baselines achieve stronger standard predictive metrics, while Lusifer with Gemma:12B outperformed most baselines across metrics for users with fewer than 10 prior interactions.

    Source: §V · pp. 4–5

  • Takeaways & Limitations

    Lusifer offers a dynamic, explainable environment for testing and training RL-based recommender systems under real-world constraints.

    Source: §V · p. 5

    On standard predictive metrics, Lusifer has higher RMSE and MAE and lower Pearson correlation than the baselines.

    Source: §V · p. 4

Abstract

from arXiv · show

Reinforcement learning (RL) recommender systems often rely on static datasets that fail to capture the fluid, ever changing nature of user preferences in real-world scenarios. Meanwhile, generative AI techniques have emerged as powerful tools for creating synthetic data, including user profiles and behaviors. Recognizing this potential, we introduce Lusifer, an LLM-based simulation environment designed to generate dynamic, realistic user feedback for RL-based recommender training. In Lusifer, user profiles are incrementally updated at each interaction step, with Large Language Models (LLMs) providing transparent explanations of how and why preferences evolve. We focus on the MovieLens dataset, extracting only the last 40 interactions for each user, to emphasize recent behavior. By processing textual metadata (such as movie overviews and tags) Lusifer creates more context aware user states and simulates feedback on new items, including those with limited or no prior ratings. This approach reduces reliance on extensive historical data and facilitates cold start scenario handling and adaptation to out of distribution cases. Our experiments compare Lusifer with traditional collaborative filtering models, revealing that while Lusifer can be comparable in predictive accuracy, it excels at capturing dynamic user responses and yielding explainable results at every step. These qualities highlight its potential as a scalable, ethically sound alternative to live user experiments, supporting iterative and user-centric evaluations of RL-based recommender strategies. Looking ahead, we envision Lusifer serving as a foundational tool for exploring generative AI-driven user simulations, enabling more adaptive and personalized recommendation pipelines under real world constraints.

I. INTRODUCTION

Existing recommender simulations and benchmarks have trade-offs in interactivity, realism, scalability, and generalization. Lusifer introduces an LLM-based environment that tracks evolving user preferences and provides explanations, with a focus on RL evaluation under limited data.

I. INTRODUCTION

  • Existing environments trade customizability, behavioral complexity, and domain realism against computational efficiency, simulation complexity, or domain applicability.Source: §I · p. 1
  • Related LLM simulation work explores user-item interactions and complex behavior, but is reported to face accuracy, generalization, reproducibility, and synthetic-data-quality challenges.Source: §I · p. 1
  • Lusifer updates user profiles at each interaction and explains preference changes, making evolving simulated feedback inspectable.The paper presents it as an LLM-based environment for generating users and behaviors in recommender systems.Source: §I · p. 2
  • Lusifer is presented as a more realistic, interactive platform for training and evaluating RL recommender agents than static benchmarks.Source: §I · p. 2
  • The work emphasizes recent, partial user histories and claims adaptability to data-scarce, cold-start, and out-of-distribution scenarios.Source: §I · p. 2
  • Lusifer is proposed as a controllable, scalable alternative to live user trials for systematically exploring recommendation strategies.Source: §I · p. 2

II. METHODOLOGY

The methodology covers data preprocessing, user-profile creation, and simulated-rating generation.

II. METHODOLOGY

  • The methodology has three stages: data preprocessing, creating user profiles, and generating simulated ratings.Source: §II · p. 2

A. Data preprocessing

The study uses MovieLens data enriched with movie descriptions and represents users and movies as text features for evaluation.

A. Data preprocessing

  • Movie overviews and other metadata were retrieved through TMDB to enrich the MovieLens dataset.Source: A. Data preprocessing · p. 2
  • MovieLens 100K and 1M ratings are divided into five folds, with 80% for training and 20% for testing in each fold.Source: A. Data preprocessing · p. 2
  • User age, gender, and occupation are combined into user_info, while movie titles, genres, and overviews form movie_info.Source: A. Data preprocessing · p. 2

B. Phase 1: Creating User Profiles

Lusifer builds user profiles from recent, chronological ratings in sequential batches, retaining intermediate states and explanations as preferences evolve.

B. Phase 1: Creating User Profiles

C. Phase 2: Generating Simulated Ratings

In Phase 2, Lusifer uses updated user summaries and recent rated movies to simulate ratings for recommended test-set movies, then evaluates those ratings against actual test ratings.

C. Phase 2: Generating Simulated Ratings

D. Baseline Methods for Comparative Analysis

The comparison uses four established recommendation algorithms and limits their training data to each user's last 40 interactions, aligning that usage with Lusifer.

D. Baseline Methods for Comparative Analysis

  • The baselines are ALS, SVD++, neural collaborative filtering (NCF), and RNN4REC.The paper identifies these as established recommendation algorithms for modeling user preferences and item relationships.Source: D. Baseline Methods for Comparative… · p. 3
  • All baselines are limited to each user's last 40 interactions, matching Lusifer's training-set usage.Source: D. Baseline Methods for Comparative… · p. 3
  • Latent-factor models use 16-dimensional embeddings, while NCF and RNN4REC use specified neural architectures and negative sampling.ALS uses L2 regularization λ=5.0 and confidence weighting α=10; SVD++ includes user-specific biases and implicit interactions. NCF uses hidden dimensions (128, 64, 32), and RNN4REC uses LSTM hidden sizes (16, 16) over recent_num=10 interactions.Source: D. Baseline Methods for Comparative… · p. 4

III. PROMPT ENGINEERING

Lusifer uses prompts to create initial user summaries, update them, and generate simulated ratings, with validation and recovery procedures for model outputs and interruptions.

III. PROMPT ENGINEERING

  • Prompts guide initial user summaries, summary updates, and simulated ratings, using one-shot examples to guide output formats.Source: §III · p. 4
  • Validation layers check simulated ratings and profile updates, with retries for outputs that parsing protections cannot validate.The retry mechanism resends prompts a specified number of times and resolved most unexpected outputs, according to the passage.Source: §III · p. 4
  • The system stores recent user data after each step so processing can resume from the last successful point after interruptions.Source: §III · p. 4

IV. SCALABILITY AND ADAPTABILITY

Lusifer is configurable across supported LLMs, input features, and batch sizes, and returns simulated ratings while incorporating them into updated user behavior.

IV. SCALABILITY AND ADAPTABILITY

  • Researchers can use the OpenAI API or an Ollama-integrated open-source LLM, and can change the model through configuration.This study uses gpt-4o-mini, Gemma:3B, and Gemma:12B.Source: §IV · p. 4
  • The setup lets researchers adjust the number of prior interactions and summary-update batch sizes, with examples of 40 movies and batches of 10.User and item textual features, including user_info and item_info, are supplied with prompts to provide context for simulating behavior.Source: §IV · p. 4
  • For RL training, Lusifer accepts recommended item IDs and returns simulated ratings while incorporating new ratings into updated user behavior.Batch sizes can control how frequently the RL agent's behavior is updated.Source: §IV · p. 4

V. EXPERIMENTAL RESULTS AND DISCUSSION

Lusifer trails collaborative-filtering baselines on standard accuracy metrics, but performs better than most baselines in the reported cold-start setting. Textual metadata and dynamic profiles support its cold-start handling and explainable feedback.

V. EXPERIMENTAL RESULTS AND DISCUSSION

  • With Gemma:12B, Lusifer outperforms most baselines across all metrics for users with fewer than 10 prior interactions.Table II covers this MovieLens 100K cold-start setting.Source: §V · p. 5, Table II · p. 5
  • Unstructured metadata improves accuracy, while limiting numeric profile exposure to the last 10 ratings helps balance user history, contextual clues, and explainability.The paper reports that including numeric ratings may sometimes reduce prediction accuracy, likely because LLMs reason less effectively over numeric inputs.Source: §V · p. 5
  • For user cold starts, Lusifer builds profiles from textual metadata and a small set of recent interactions; for item cold starts, it can predict from item metadata without prior ratings.The paper presents these capabilities as useful where new users and content continually enter recommendation systems.Source: §V · p. 5
  • Despite lower predictive precision than baselines, Lusifer offers a flexible, explainable environment for testing and training RL-based recommenders under real-world constraints.The authors also characterize it as ethically viable and useful with sparse data and dynamic, step-by-step user updates.Source: §V · p. 5

VI. CONCLUSION

Lusifer generates dynamic user behaviors for RL-based recommenders by focusing on recent interactions and iteratively updating profiles. It is comparable to traditional methods on standard accuracy metrics, while its stated strengths include adaptability, explainability, and cold-start handling.

VI. CONCLUSION

  • Lusifer focuses on recent user interactions and iteratively updates profiles to model evolving preferences, with performance comparable to traditional methods on standard accuracy metrics.The conclusion identifies adaptability, scalability, explainability, and cold-start handling as Lusifer’s strengths.Source: §VI · p. 5

VII. FUTURE WORK

The authors propose improving Lusifer’s accuracy and expanding its applications through model, feedback, integration, and evaluation work.

VII. FUTURE WORK

  • The authors plan to test advanced LLM architectures, refine prompts, and use richer metadata to improve precision and capture subtle user behaviors.These are proposed accuracy improvements.Source: §VII · p. 5
  • They plan to integrate Lusifer with RL recommender agents in continuous training loops and add reviews, browsing behavior, and session data beyond explicit ratings.The proposed integration would directly assess adaptive strategies under shifting user profiles.Source: §VII · p. 5
  • The authors propose extending Lusifer beyond movies to other domains and benchmarking it against established user-simulation platforms.The benchmarking is intended to further validate its effectiveness.Source: §VII · p. 5
Loading 2405.13362v4…