Source-linked AI summary
AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, Ji-Rong Wen
TL;DR
AgentCF addresses the limited modeling of non-verbal recommender interactions and user-item relations by treating users and items as jointly optimized language agents. Through autonomous interaction, collaborative reflection, and memory-based preference propagation, it effectively simulates personalized and diverse behaviors in recommender settings. The approach remains bounded by its fixed LLM and the paper’s stated need to explore more real-world scenarios.
Problem
Existing studies mainly simulate dialogue, while non-verbal recommender interactions and two-sided user-item relations remain insufficiently modeled.
Method
AgentCF models users and items as memory-equipped agents that autonomously interact and collaboratively reflect on mismatches with real interaction records.
Results
AgentCF effectively simulates user-item interactions and produces personalized behaviors across user-item, user-user, item-item, and collective interactions.
Takeaways & Limitations
The framework supports diverse interaction behaviors and propagates preferences among agents, implicitly modeling collaborative filtering.
Takeaways & Limitations
The LLM is fixed without explicit gradient-based learning, and future work must examine more types of real-world scenarios and interactions.
Abstract
from arXiv · showhide
Recently, there has been an emergence of employing LLM-powered agents as believable human proxies, based on their remarkable decision-making capability. However, existing studies mainly focus on simulating human dialogue. Human non-verbal behaviors, such as item clicking in recommender systems, although implicitly exhibiting user preferences and could enhance the modeling of users, have not been deeply explored. The main reasons lie in the gap between language modeling and behavior modeling, as well as the incomprehension of LLMs about user-item relations. To address this issue, we propose AgentCF for simulating user-item interactions in recommender systems through agent-based collaborative filtering. We creatively consider not only users but also items as agents, and develop a collaborative learning approach that optimizes both kinds of agents together. Specifically, at each time step, we first prompt the user and item agents to interact autonomously. Then, based on the disparities between the agents' decisions and real-world interaction records, user and item agents are prompted to reflect on and adjust the misleading simulations collaboratively, thereby modeling their two-sided relations. The optimized agents can also propagate their preferences to other agents in subsequent interactions, implicitly capturing the collaborative filtering idea. Overall, the optimized agents exhibit diverse interaction behaviors within our framework, including user-item, user-user, item-item, and collective interactions. The results show that these agents can demonstrate personalized behaviors akin to those of real-world individuals, sparking the development of next-generation user behavior simulation.
1 INTRODUCTION
AgentCF addresses the limited simulation of non-verbal recommender interactions by modeling users and items as interacting agents. It collaboratively optimizes both sides through autonomous interaction and reflection, producing personalized simulated behaviors.
- Motivation: Existing LLM-agent studies emphasize dialogue, while recommender interactions can reveal preferences but are difficult to model from language alone.Verbalizing interaction records may miss underlying behavioral patterns such as correlated purchases.
- Motivation: Modeling both user preferences and item adopters is crucial for representing two-sided user-item relations.
- Approach: AgentCF represents both users and items as agents equipped with memories of preferences, features, and behavioral information.
- Results: Experiments on real-world datasets show effective user-item simulation and promising recommendation performance against classical and LLM-based recommenders.The framework also supports user-user, item-item, and collective interactions with personalized behaviors.
- Approach: User and item agents are collaboratively optimized by autonomous interaction and reflection on disparities between simulated decisions and real interaction records.The method mutually updates user and item memories rather than optimizing only one side.
2 METHODOLOGY
AgentCF integrates LLM-powered user and item agents into recommender systems, where memories support autonomous interaction, collaborative reflection, and preference propagation. The framework thereby models user-item relations and enables diverse simulated interactions.
- Task Setting: The task extends traditional recommendation by representing users and items as agents that simulate original interactions autonomously.User agents capture corresponding user preferences, while item agents reflect item characteristics and adopter preferences.
- Implicit Preference Propagation: Preference propagation updates both user and item memories so later agents can use information from previous adopters and interacted items.This implements the collaborative-filtering idea through memory updates rather than gradient-based parameter learning.
- Memory Design: User memories store evolving preferences through short-term and long-term components, whereas item memories record stable characteristics and adopter preferences.Item memories are continuously updated with user preferences to support propagation.
- Collaborative Optimization: AgentCF equips agents with memory mechanisms and uses autonomous interaction followed by collaborative reflection against real-world interaction records.Because the LLM remains fixed, optimization updates associated memories rather than model gradients.
- Diverse Interactions: Optimized agents simulate user-item, social, collective, and item-to-item interactions within recommender systems.Item-to-item interaction is proposed as potentially useful for item cold-start scenarios by propagating collected user preferences.
3 EXPERIMENTS
Experiments evaluate AgentCF on sampled Amazon review datasets using ranking benchmarks and ablations. The results indicate that collaborative optimization of user and item agents improves recommendation performance, personalization, bias stability, and alignment with real users.
- Datasets: Two dense and two sparse subsets of the “CDs and Vinyl” and “Office Products” Amazon datasets contain 100 users each.Dense subsets are used for further analysis because they better demonstrate interactions among agents.
- Evaluation: NDCG@K at K=1, 5, and 10 is measured with leave-one-out ranking against one ground-truth item and nine sampled candidates.Each test instance is repeated three times and averaged.
- Overall Performance: AgentCF outperforms other baselines in most scenarios, with AgentCFB demonstrating the effectiveness of preference propagation for personalized collaborative filtering.AgentCFB+R improves personalized inference through specialized candidate preferences, while AgentCFB+H helps in sparse Office scenarios by incorporating historical interactions.
- Overall Performance: AgentCF is superior or comparable to traditional recommenders trained on sampled datasets and comparable to full-data traditional models in sparse scenarios.The approach achieves these results using approximately 0.07% of the complete dataset, although it has room for improvement in other scenarios.
- Ablation Study: Removing autonomous interaction, user-agent optimization, or item-agent optimization worsens performance, supporting collaborative reflection and behavior-based two-sided modeling.Autonomous interaction provides feedback about simulated-to-real misalignment; item agents update memories with user preferences and propagate them to new agents.
- Bias Analysis: Compared with LLMRank, AgentCF shows enhanced stability against position and popularity bias while ranking candidates according to personalized preferences.LLMRank tends to select popular items and higher-positioned items.
- Optimization Progress: Around 95% of user agents make correct choices after collaborative reflection, while continuous optimization increases correct initial choices over optimization steps.The experiment tests whether agents select the positive item from two candidates before and after each optimization step.
3.4 Simulations on Other Types of Interactions
AgentCF simulates user-user, item-item, preference-propagation, and collective interactions among language agents. These interactions reproduce socially informed purchasing, support cold-start items, propagate preferences with decay, and improve advertisements through collaboration.
- User-user Interaction Simulation: Figure 4 measures the proportion of agents buying items after reviews, distinguishing similar-preference reviewers from negative reviews.“Similar” refers to reviewers with preferences similar to test users, while “Neg” denotes negative reviews.
- User-user Interaction Simulation: User agents increasingly purchase after positive reviews, refrain after negative reviews, and trust reviews from users with similar preferences.The simulation compares decisions before and after test users view reviews written by other agents.
- Item-item Interaction Simulation: Interactions between cold-start and popular item agents alleviate the cold-start problem even when their identity information differs.The result suggests new item agents relate identity information to personalized memory rather than simply copying another agent’s memory.
- Preference Propagation: Continuous interactions propagate a seed user’s preferences to increasingly many agents with similar behaviors, while broader interaction scopes reduce the proportion expressing them.The pattern indicates information decay during propagation.
- Collaborative Advertisements Creation: Collaborating advertiser agents critique individual-LLM advertisement drafts and combine personalization, creativity, and attractiveness to generate more appealing advertisements.The agents are specialized in different aspects of advertising and provide suggestions for improving a draft.
4 RELATED WORK
Prior work uses LLM-powered agents for human-like interaction and recommendation, but recommendation studies largely focus on users and neglect user-item relations. AgentCF instead treats items as agents to model the two-sided relation central to collaborative filtering.
- LLM-powered Agents: LLM-powered agents have been used to simulate human-like daily life, debate, game-playing, and other interactions.These agents draw on reasoning, planning, memory, and reflection capabilities.
- Language Model For Recommendation: Language-model recommendation studies infer preferences from historical interactions, augment models with LLM knowledge, or fine-tune language models on recommendation data.The related approaches include both direct prompting and model specialization.
- Language Model For Recommendation: Recent agent-based recommender studies facilitate recommendations or simulate user behavior, but mainly neglect modeling user-item relations.AgentCF differs by explicitly regarding items as agents and proposing agent-based collaborative filtering.
5 CONCLUSION AND FUTURE WORK
AgentCF models two-sided user-item relations by collaboratively optimizing user and item agents through autonomous interaction and reflection. The resulting agents show human-like behaviors across user-item, user-user, item-item, and collective interactions, while broader scenarios remain future work.
- Conclusion: AgentCF jointly optimizes user and item agents by comparing autonomous decisions with real interaction records and collaboratively reflecting on misleading simulations.The process aligns preferences and propagates information to other agents in later interactions.
- Conclusion: The framework produces human-like behaviors across user-item, user-user, item-item, and collective interactions.These behaviors are presented as evidence of AgentCF’s effectiveness.
- Future Work: Future work will explore more types of real-world scenarios and their corresponding interactions.The paper presents AgentCF as an initial step toward simulating inanimate objects with language-based agents.
A IMPLEMENTATION DETAILS
The implementation uses RecBole for AgentCF and baseline systems, with different language models assigned to item selection and collaborative reflection and inference.
- Implementation Details: AgentCF and baseline systems are implemented with the RecBole recommendation framework.LLMRank uses gpt-3.5-turbo-16k-0613; AgentCF uses textdavinci-003 for autonomous item selection and gpt-3.5-turbo-16k-0613 for reflection and inference.
- Implementation Details: The LLM API temperature is fixed at 0 for the experiments.This setting applies to the language-model calls described in the implementation.
B TRAINING DETAILS
This section presents the details and insights of collaborative agent optimization to clarify the proposed approach.
- Collaborative agent optimization is introduced to help readers understand the approach.
- The optimization process is discussed as a central component of the approach.
- The section focuses on explaining how collaborative optimization supports comprehension of AgentCF.
B.1 Training Data Format
Training uses real users’ historical interaction sequences and simulates those interactions chronologically, pairing each observed item with a negative candidate.
- Real users’ historical interaction records serve as training data for optimizing agents and updating memory.
- Historical behavioral sequences are arranged in chronological order for stepwise optimization.
- At each step, the observed item is treated as a positive item and paired with a negative item for the user agent to choose between.
B.2 Candidate Selection Process
Negative candidates are selected with popular and position biases to make choices more difficult and encourage user agents to reflect more comprehensively.
- Popular and position biases are intentionally introduced when selecting negative candidate items.
- Increasing discrimination difficulty makes user agents more likely to select negative items.
- Negative-item selections are used to encourage more comprehensive reflections by user agents.
B.3 Memory Module
The memory module separates short-term reflection from long-term storage, while the current full-history design may become impractical for much longer interaction histories.
- Agents reflect on and adjust misleading conceptions in short-term memory during optimization.
- User agents’ long-term preferences are not retrieved during optimization to support more diverse preference exploration through collaborative reflection.
- Appending each previous short-term memory produces long-term memory matching the length of the real interaction history.
- Storing all past experiences as a list may be impractical for histories with hundreds or thousands of interactions.
B.4 Prompt Disign
The collaborative reflection process iteratively addresses mismatches between agent decisions and real-world interaction records. Optimization stops when choices align with the records or when the iteration limit is reached.
- Collaborative reflection examines inconsistent decisions, derives preferences and dislikes from positive and negative examples, and updates agent memories.
- Optimization alternates autonomous item selection with collaborative reflection until choices match real-world records or the maximum rounds are reached.
- The maximum number of optimization rounds per interaction step is 2.
B.5 Computational Cost and Efficiency
The study limits experiments to 100 sampled users per dataset because LLM API calls and agent communication are costly. It also explores staged models and merged optimization steps to improve efficiency.
- 100 users and their historical interaction records are sampled per dataset because LLM API calls and agent communication remain costly and inefficient.
- Different LLMs are assigned to optimization stages according to task complexity, using a parallel-capable model for simpler selection and a stronger model for reflection.
- Reflection and memory updating can be merged into one optimization step to reduce computational expense and improve efficiency.
- The integrated reflection-and-update approach does not reduce performance relative to separating the two processes.
- Future work targets lower computational costs and more efficient agent interaction to scale simulations to larger datasets.
C PROMPTS AND RESPONES
The appendix illustrates AgentCF prompts for autonomous choice, collaborative memory optimization, review-based preference transfer, and advertisement refinement. These examples show agents using personalized descriptions to make and revise decisions.
- C.1 Agent-based Collaborative Filtering: AgentCF represents users and items as agents whose memories describe user preferences and item characteristics for autonomous interaction.
- C.1 Agent-based Collaborative Filtering: The forward prompt asks a user agent to choose between two CDs by extracting preferences, evaluating candidate features, and explaining the choice.
- C.1 Agent-based Collaborative Filtering: The backward process revises user memory after an unsuitable choice by combining newly identified preferences with prior descriptions and removing conflicts.
- C.1 Agent-based Collaborative Filtering: Item-agent prompts likewise update CD descriptions so distinguishing features align with the user’s preferences without contradicting inherent item characteristics.
- C.2.1 Users Reading and Writing reviews.: The review experiment tests whether users can understand previously unseen items by reading positive and negative reviews written by other user agents.
- C.2.1 Users Reading and Writing reviews.: The experiment samples 100 test users, optimizes them on earlier interactions, and uses five prior item-interacting agents to generate reviews for each ground-truth item.
- C.2.1 Users Reading and Writing reviews.: Before reading reviews, test users receive item characteristics and decide whether a recommended CD matches their preferences.
- C.5 Collaborative Advertisements Creation: Advertisement refinement changes a draft rejected for insufficient artist and track detail into an advertisement accepted for matching genre, vocal, and lyrical preferences.