Source-linked AI summary
RecoWorld: Building Simulated Environments for Agentic Recommender Systems
Fei Liu, Xinyu Lin, Hanchao Yu, Mingyuan Wu, Jianyu Wang, Qiang Zhang, Zhuokai Zhao, Yinglong Xia, Yao Zhang, Weiwei Li, Mingze Gao, Qifan Wang, Lizhu Zhang, Benyu Zhang, Xiangjun Fan
TL;DR
Offline evaluation can reflect exposure bias, while online A/B tests have slow feedback loops and require care with real users. RecoWorld proposes a simulated user and agentic recommender that interact over multiple turns, with users able to issue instructions and the recommender adapting its lists. The paper presents this as a blueprint for safer testing and training of agentic recommenders.
Problem
Offline evaluation from historical behavior can introduce exposure bias, while online A/B tests are slow and must be handled carefully with real users.
Source: §1 · p. 1Method
Results
RecoWorld is presented as a blueprint for simulated environments that model user feedback and support agentic recommender development.
Source: §1 · p. 2Takeaways & Limitations
Simulated online environments allow recommenders to test bold, different strategies without compromising real-user experience.
Source: §1 · p. 1The relationship between retention signals and NDCG is framed as hypotheses about exploitation, disengagement, and exploration, not as experimental findings.
Source: §2 · pp. 3–4
Abstract
from arXiv · showhide
We present RecoWorld, a blueprint for building simulated environments tailored to agentic recommender systems. Such environments give agents a proper training space where they can learn from errors without impacting real users. RecoWorld distinguishes itself with a dual-view architecture: a simulated user and an agentic recommender engage in multi-turn interactions aimed at maximizing user retention. The user simulator reviews recommended items, updates its mindset, and when sensing potential user disengagement, generates reflective instructions. The agentic recommender adapts its recommendations by incorporating these user instructions and reasoning traces, creating a dynamic feedback loop that actively engages users. This process leverages the exceptional reasoning capabilities of modern LLMs. We explore diverse content representations within the simulator, including text-based, multimodal, and semantic ID modeling, and discuss how multi-turn RL enables the recommender to refine its strategies through iterative interactions. RecoWorld also supports multi-agent simulations, allowing creators to simulate the responses of targeted user populations. It marks an important first step toward recommender systems where users and agents collaboratively shape personalized information streams. We envision new interaction paradigms where "user instructs, recommender responds," jointly optimizing user retention and engagement.
1 Introduction
RecoWorld proposes simulated environments for agentic recommenders, where virtual users provide multi-turn feedback and instructions so systems can be tested and refined without risking real users. The framework also supports simulations of targeted populations and varied recommendation goals.
1 Introduction
- Offline metrics can inherit exposure bias, while online A/B tests have slow feedback loops and require careful handling with real users [15] [60].The paper motivates simulated online environments as a way to test bold strategies without compromising user experience.Source: §1 · p. 1
- The framework supports instruction-following evaluation, creator strategy experiments, and interest exploration using simulated feedback.Simulated user groups with similar features can provide collective exploration feedback for contextual bandits.Source: §1 · p. 2
- RecoWorld is a blueprint for simulated environments that models long-term value and collective impacts, rather than proposing a new agentic recommender.Its user-simulation and reward-signal design is presented as potentially useful for modeling short- and long-term interests in existing and generative recommenders.Source: §1 · p. 2
- RecoWorld generates multi-turn user–recommender trajectories and engagement statistics that can serve as reward signals for RL-based agent training.When a user may disengage, the simulator can issue instructions; the recommender responds with new item lists intended to re-engage them.Source: §1 · p. 2
2 Agentic Environments for RecSys
RecoWorld pairs a user simulator with an agentic recommender in a feedback loop: users act on recommendations and may issue instructions, while the recommender adapts its lists. The section describes how these interactions can support training and simulator evaluation.
2 Agentic Environments for RecSys
- RecoWorld’s dual-view architecture combines a user simulator that produces feedback or instructions with a recommender that delivers personalized suggestions.Instruction-following recommenders dynamically update lists in response to user instructions; agents may converse to clarify preferences.Source: §2 · p. 3
- During a session, users click or skip items and may issue instructions when considering leaving; the recommender can refresh its list and the interaction trajectory continues until exit.The user may review the updated list and provide further feedback before ending the session.Source: §2 · p. 3
- Trajectory-level metrics such as total time spent can reward agentic recommenders, while an LLM judge retains trajectories that meet predefined task rubrics for training.The paper proposes multiple trajectories per user and initial recommendation set, with RL techniques using rewards and self-critique.Source: §2 · p. 3
- Retention signals are described as orthogonal to offline metrics such as NDCG, with interpretations of high or low NDCG and retention framed as hypotheses.For example, the authors hypothesize that low NDCG with high retention may indicate effective exploration.Source: §2 · pp. 3–4
- User preferences can be conveyed implicitly through behavior as well as explicit instructions, with similar behavioral patterns offering shared signals.Examples include interpreting continued sessions as likely satisfaction and non-return as possible disengagement.Source: §2 · p. 4
- The simulator can be assessed by comparing session-level interaction statistics with human annotators and by comparing revised recommendations with dataset ground truth.The section also describes variants using human users, traditional recommenders, or disabled instructions, alongside benefits such as safe RL training and natural-language feedback.Source: §2 · p. 4
3 User Simulation
RecoWorld simulates multi-turn user behavior to study recommendation engagement and retention, with user actions shaped by context and interaction history. It offers alternative content representations and memory models for handling multimodal, growing histories and changing preferences.
3 User Simulation
- For each recommended item, users may click, comment, share, like, watch, skip, or leave, with decisions conditioned on context and past interactions.Leaving can prompt reflection and an instruction to improve recommendations.Source: §3 · p. 4
- The simulator optimizes long-term retention through session duration and return frequency, modeling each item's reasoning, action, and mindset update.A session comprises multiple recommendation turns and ends when the user exits.Source: §3 · p. 4
3.1 Multimodal User Engagment Modeling
- Multimodal histories are difficult to unify, and growing interaction histories require long-context handling that gives recent interactions greater weight.The paper motivates compression as one way to summarize extended histories.Source: §3.1 · p. 4
- Text descriptions offer flexible, compressible inputs but can miss video mood, sarcasm, or musical cues; multimodal models ingest media directly but face weaker reasoning and context-efficiency challenges.Demographic attributes and textual metadata can supplement multimodal inputs.Source: §3.1 · p. 5
- Semantic IDs compactly encode item meaning while preserving LLM reasoning, but require continuous pre-training and periodic retraining for new content.This alternative does not perform continual pre-training, supervised fine-tuning, or reinforcement learning on the backbone at inference.Source: §3.1 · p. 5
- Dynamic engagement memory filters interaction history using context-conditioned importance scores and a retention threshold, while organizing behavior at interaction and session levels.Session summaries capture trajectory-level signals such as mindset shifts and cross-session preference transitions.Source: §3.1 · p. 6
- Preference state evolves through a temporal update based on memory, context, reasoning traces, and system corrections, enabling the simulator to represent preference shifts.The update is framed as adding a mindset adjustment to the prior state.Source: §3.1 · p. 6
3.2 Candidate Items
- Candidate items may use real content and metadata, represented as text descriptions, audiovisual features, semantic IDs, or combinations of these inputs.The listed item types include short-form videos, text posts, mixed media, and friend suggestions.Source: §3.2 · p. 6
4 Instruction-Following RecSys
RecoWorld frames instruction-following recommendation as responding directly to user commands, rather than proactively suggesting items during conversation. It describes an autonomous-agent design and an iterative policy-learning formulation for using user feedback to update recommendations.
4 Instruction-Following RecSys
- An instruction-following recommender can use perception, reasoning and planning, tool-mediated action, and memory to detect user state, interpret commands, deliver recommendations, and inform later interactions.Planning can decompose instructions into subtasks delegated to retrieval and ranking modules.Source: §4 · pp. 6–7
- The recommender-user interaction is modeled as an MDP in which user mindset is the state, recommendations are agent actions, and a simulated user supplies state transitions.The recommender policy is updated iteratively to maximize expected reward, which may combine session time, clicks, and instruction-following scores.Source: §4 · p. 7
- Rollouts can estimate cumulative reward for on-policy PPO with asynchronous rewards or off-policy DPO, using trajectory-level feedback rather than only action-level rewards.Reward options include total session time, click counts, self-critique scores, or combinations measuring instruction following and engagement.Source: §4 · p. 7
- Voice interfaces provide commands about topics, creators, and feed content that can serve as feedback signals for agentic recommenders.Examples include requesting more F1 content or less political content.Source: §4 · pp. 6–7
- Instructions can be ambiguous: a request for UFC content may reflect a local-event interest or a longer-term combat-sports interest, requiring intent-sensitive alignment.The paper proposes reasoning-intensive retrieval to infer intent and refine ambiguous queries.Source: §4 · p. 7
- Instruction-following recommendation responds directly to commands and refines top results by adding, removing, or reordering items, unlike conversational recommenders' proactive suggestions.The recommender uses the conversation and engagement history to adjust what is shown.Source: §4 · p. 7
5 Multi-Agent Simulators
Multi-agent simulators model individuals interacting over time, while RecoWorld describes user-state and environment updates and creator-facing simulations for estimating audience responses.
5 Multi-Agent Simulators
- Multi-agent simulations model decision-making and interaction over time, with applications spanning social-media dynamics and computational social experiments.Examples include studying information diffusion, polarization, inflammatory content, universal basic income, and responses to shocks such as hurricanes.Source: §5 · pp. 7–8
- These simulators range from thousands of agents to millions, including OASIS and AgentScope.AgentTorch examines the tradeoff between simulation scale and individual agency.Source: §5 · pp. 7–8
- RecoWorld models each user's evolving state from interactions with neighbors and the environment, while the environment evolves in response to users' actions and states.User-to-user influence is specified by environment parameters.Source: §5 · p. 8
- Creators can expose content to simulated followers, general populations, or targeted demographics and track predicted responses from 6 hours to 1 week.Simulations can be reset and rerun to measure variability and improve confidence in predictions.Source: §5 · p. 8
6 Related Work
Related work spans generative recommenders, recommender simulators, and LLM-based user simulators; the section highlights challenges in evaluating powerful recommenders and supporting instruction-driven interaction.
Evaluation of Recommender Systems
- HSTU achieves up to 65% better ranking metrics than baselines, while its 1.5-trillion-parameter version boosts online A/B metrics by 12.4%; OneRec yields a +1.6% increase in watchtime.OneRec replaces the traditional retrieve-then-rank pipeline with an end-to-end generative model and was deployed to millions of users.Source: §6 · p. 8
- Traditional offline metrics can fail to predict real-user gains, while A/B tests are costly, slow, and impractical or risky for every experiment.The paper identifies these constraints as motivating interest in large-scale simulated environments for recommender evaluation.Source: §6 · p. 8
Simulated Environments for Training Agentic Systems
- RecSys simulation environments include KuaiSim's multi-behavior and cross-session feedback for RL training and AgentRecBench's standardized world APIs over multi-domain datasets.These environments provide sandboxes for agents to plan, explore, and learn complex behaviors.Source: §6 · p. 8
User Simulation with LLMs
- Non-conversational LLM user simulators typically provide feedback but cannot proactively instruct recommenders, whereas conversational simulators can issue context-aware instructions.The instructions can signal fatigue or boredom and a desire to shift away from prior preferences.Source: §6 · p. 8
7 Industry Use and Conclusion
RecoWorld introduces an environment for instruction-driven interaction between users and agentic recommenders. The paper presents a framework for building simulated environments and notes industry interest.
7 Industry Use and Conclusion
- RecoWorld introduces an environment for instruction-driven interactions between users and agentic recommender systems.The work prioritizes presenting a comprehensive framework for building simulated environments.Source: §7 · p. 8
- RecoWorld was internally released in September 2025 and is presented as a foundation for instruction-following recommenders.Source: §7 · p. 8
- The paper reports interest from Google and Kuaishou RecSys teams, suggesting potential impact in real-world applications.Source: §7 · p. 8
June 29-July 3, 2026 Dubai, UAE
The document identifies a workshop, its organizing committee, and an outstanding-paper recognition for RecoWorld.
June 29-July 3, 2026 Dubai, UAE
- The recognition is attributed to the HCRS@TheWebConf 2026 Organizing Committee.Source: June 29-July 3, 2026 Dubai, UAE · p. 11
- The event is identified as the 2nd Workshop on Human-Centered Recommender Systems.Source: June 29-July 3, 2026 Dubai, UAE · p. 11
- RecoWorld: Building Simulated Environments for Agentic Recommender Systems is identified as an outstanding paper.Source: June 29-July 3, 2026 Dubai, UAE · p. 11