PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models Empowered Personalized Web Agents

Hongru Cai, Yongqi Li, Wenjie Wang, Fengbin Zhu, Xiaoyu Shen, Wenjie Li, Tat-Seng Chua

arXiv:2410.17236v2cs.CLcs.AIcs.IR

TL;DR

Existing LLM-based Web agents overlook personal data that can help interpret instructions and customize actions, and the task lacks a comprehensive benchmark. The paper formulates personalized Web agents, constructs PersonalWAB, and proposes PUMA; PUMA significantly outperforms existing Web agents on single-turn and multi-turn personalized tasks.

  • Problem

    LLM-based Web agents overlook profiles and historical behaviors that can reveal implicit preferences, and the task lacks a comprehensive training and evaluation benchmark.

    Source: §1 · p. 2

  • Method

    The paper constructs PersonalWAB across three personalized Web tasks and proposes PUMA, which retrieves relevant historical behaviors and aligns LLMs for personalized function execution.

    Source: §1 · pp. 2–3

  • Results

    PUMA significantly outperforms existing Web agents on single-turn and multi-turn personalized Web tasks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    On PersonalWAB, PUMA aligns better with personalized user instructions and preferences than existing Web agents.

    Source: §1 · p. 3

    The study focuses on shopping scenarios and abstracts Web environments as functions rather than GUIs; extending the approach to broader Web environments requires further investigation.

    Source: §3.2.3 · p. 4, §7 · p. 8

Abstract

from arXiv · show

Web agents have emerged as a promising direction to automate Web task completion based on user instructions, significantly enhancing user experience. Recently, Web agents have evolved from traditional agents to Large Language Models (LLMs)-based Web agents. Despite their success, existing LLM-based Web agents overlook the importance of personalized data (e.g., user profiles and historical Web behaviors) in assisting the understanding of users' personalized instructions and executing customized actions. To overcome the limitation, we first formulate the task of LLM-empowered personalized Web agents, which integrate personalized data and user instructions to personalize instruction comprehension and action execution. To address the absence of a comprehensive evaluation benchmark, we construct a Personalized Web Agent Benchmark (PersonalWAB), featuring user instructions, personalized user data, Web functions, and two evaluation paradigms across three personalized Web tasks. Moreover, we propose a Personalized User Memory-enhanced Alignment (PUMA) framework to adapt LLMs to the personalized Web agent task. PUMA utilizes a memory bank with a task-specific retrieval strategy to filter relevant historical Web behaviors. Based on the behaviors, PUMA then aligns LLMs for personalized action execution through fine-tuning and direct preference optimization. Extensive experiments validate the superiority of PUMA over existing Web agents on PersonalWAB.

1 Introduction

LLM-based Web agents use instructions to complete Web tasks, but existing agents overlook user profiles and behavioral histories that can reveal implicit preferences. The paper formulates personalized Web agents, builds PersonalWAB, and proposes PUMA to align agents with those preferences.

1 Introduction

  • Existing LLM-based Web agents overlook profiles and historical behaviors that can reveal implicit preferences for instruction comprehension and personalized action execution.The paper also identifies the lack of a comprehensive benchmark for training and evaluation as a primary challenge.Source: §1 · p. 2
  • The paper formulates personalized Web agents as inferring requirements and preferences from instructions and personal data to select Web functions and formulate their parameters.Personal data includes profiles and historical Web behaviors.Source: §1 · p. 2
  • PersonalWAB evaluates personalized search, recommendation, and review generation using 1,000 simulated users, Web functions, and single-turn and multi-turn paradigms.It derives ground truth from liked items and real reviews, and uses an LLM-based user simulator for multi-turn evaluation.Source: §1 · p. 2
  • PUMA stores long-term Web behaviors in a memory bank, retrieves task-relevant behaviors and features, and combines them with user instructions.PUMA also uses heuristic pseudo-label parameters for SFT and DPO to optimize function parameters.Source: §1 · pp. 2–3
  • PUMA significantly outperforms existing Web agents on single-turn and multi-turn personalized Web tasks.It uses retrieved behavior and features to guide personalized actions, with SFT and DPO strategies for function parameters.Source: §1 · pp. 2–3

2 Related Work

Prior Web-agent work covers single-turn tasks, interactive navigation, and varied training approaches, while personalized LLM research spans content generation and user-facing applications. The paper identifies personalized function calls tailored to individual needs as a gap in this work.

Web agents

  • Prior Web-agent studies cover single-turn and interactive tasks, with fine-tuning and prompting among the approaches used to automate them.Tasks include multi-domain, multi-hop, real-time Web interaction, and visual UI understanding.Source: §2 · p. 3

Personalization gap

  • Prior Web-agent work overlooks personalized function calls, while a cited study's predefined platform roles do not require preference understanding or preference-driven execution changes.This paper addresses the gap with a benchmark and framework for personalized Web-agent optimization and evaluation.Source: §2 · p. 3

Personalized LLMs

  • Personalized LLM research includes personalized content generation and user-facing applications, including dialogue systems and memory mechanisms.Applications also include healthcare, education, and robotics.Source: §2 · p. 3

3 Task and Benchmark

The section defines personalized Web agents as selecting Web functions and parameters from user instructions and profile/history data, then introduces PersonalWAB and its evaluation tracks. PersonalWAB uses Amazon Review data to construct profiles, personalized instructions, and a function-based Web environment.

3.1 Task Formulation

  • Agents use instructions, user profiles, and historical behaviors to select a Web function and parameters that produce personalized results.Profiles include static attributes, while historical behaviors are represented as a time-ordered sequence.Source: §3.1 · p. 3

3.2 Benchmark Construction

  • PersonalWAB uses Amazon Review data and a three-step pipeline: personalized data construction, instruction creation, and Web environment implementation.The benchmark construction uses existing behavior data because collecting real data from Web-agent users is challenging.Source: §3.2 · p. 4, Figure 2 · p. 4

3.2.1 Personalized Data Construction.

  • The benchmark samples 1,000 users across five product categories, splits chronological interactions 80/10/10, and generates profiles from each user’s full behavior history.Profiles summarize demographics, shopping preferences, and behavioral tendencies.Source: §3.2.1 · p. 4

3.2.2 User Instruction Creation.

  • Personalized search, recommendation, and review instructions are generated from user profiles and relevant product or behavior information.Search instructions vary by profile; recommendation instructions are shorter and broader, while review instructions incorporate target-product information and actual review text.Source: §3.2.2 · p. 4

3.2.3 Web Environment Implementation.

  • The environment abstracts Web interaction as functions, and evaluation combines single-turn accuracy with multi-turn simulator feedback and action-step efficiency.The single-turn track measures function and result accuracy; the multi-turn track adds average steps.Source: §3.2.3 · p. 4, §3.3 · p. 5

4 Benchmark Analysis

PersonalWAB’s generated profiles outperform Apollonion [5] on profile-behavior and profile-product consistency evaluations. Its users span demographic and behavioral categories, while review instructions are somewhat more complex than search and recommendation instructions.

4.1 Statistic Analysis

  • Users span gender, age, and occupation categories, while most fall into medium behavioral-attribute categories and fewer into high or low categories.The behavioral attributes include price sensitivity, diversity preference, and interaction complexity, preserving typical and edge cases.Source: §4.1 · p. 5
  • Review instructions are slightly more complex than search and recommendation instructions because they contain users’ initial evaluations.Recommendation instructions have the fewest tokens, reflecting their exploratory nature and fewer user-information requirements.Source: §4.1 · p. 5

4.2 Profile Consistency Evaluation

  • Constructed profiles improve both profile-product and profile-behavior consistency over Apollonion [5].The profile-product task ranks previously interacted and randomly sampled items; the profile-behavior task matches profiles to users’ past behaviors.Source: Figure 5 · p. 6, §4.2 · p. 6

5 PUMA Framework

PUMA identifies Web functions, then generates their parameters using task-specific memory and parameter optimization. Its retrieval filters user history for the current instruction, while fine-tuning and preference optimization align parameter generation with user preferences.

5 PUMA Framework

  • PUMA first fine-tunes an LLM to identify Web functions, then retrieves relevant user behaviors and optimizes parameters with heuristic fine-tuning and DPO.The two stages are Web function identification and function parameter generation.Source: §5 · p. 6, Figure 6 · p. 6

5.1 Task-specific Memory Retrieval

  • Task-specific retrieval ranks long-term memory entries by instruction similarity, then extracts features tailored to the identified function.Search retains product title, category, price, and store; recommendations retain title, category, and product ID; reviews retain ratings and comments.Source: §5.1 · p. 6

5.2 Function Parameter Optimization

  • Heuristic SFT labels vary by function: generated queries for search, recent same-category product IDs for recommendations, and dataset review text for review functions.The SFT inputs combine the instruction, task-specific memory, and identified function.Source: §5.2 · p. 6
  • DPO samples diverse parameter candidates, evaluates their result accuracy, and optimizes preference pairs to favor better-performing parameters.Candidate parameters are generated through high-temperature sampling and beam search.Source: §5.2 · p. 7

6 Experiments

The experiments compare memory, reasoning, and recommendation-specific baselines with PUMA across single-turn and multi-turn tracks. PUMA leads the single-turn comparison, while multi-turn results cover its task-specific memory method rather than the full framework; the efficiency analysis reports faster completion for PUMA.

6 Experiments

  • The evaluation compares memory-retrieval, enhanced-reasoning, and recommendation-specific baselines across single-turn and multi-turn tracks.All methods use the same prompt template, differing only in the memory component.Source: §6 · p. 7, §6.1 · p. 7

6.1 Main Results

  • Recommendation instructions often receive the wrong search function; relevant memory and ReAct improve function accuracy, while baseline result accuracy remains similar to No Memory.The authors report that PUMA achieves the highest function accuracy across tasks and higher result accuracy.Source: §6.1.1 · p. 7

6.1.1 Single-turn Track.

  • In multi-turn tests, Task-specific Memory performs strongly on search and recommendation with fewer steps, but the full PUMA approach was not evaluated.The authors attribute this method's performance to extracting relevant information while filtering redundant data.Source: §6.1.2 · p. 7, Table 4 · p. 8
  • PUMA achieves the highest function accuracy across tasks and higher result accuracy, with best overall performance using shorter memory and a smaller LLM.The single-turn comparison is reported in Table 3.Source: §6.1.1 · p. 7, Table 3 · p. 8

6.2.1 Analysis on efficiency.

  • 2.8 seconds was PUMA’s average completion time, versus 6.5 to 6.9 seconds for GPT-based methods across 100 randomly selected single-turn tasks.Figure 7 reports average task completion time in seconds.Source: §6.2.1 · p. 7, Figure 7 · p. 8

7 Conclusion and Future Work

The paper formulates personalized Web agents, introduces PersonalWAB and PUMA, and reports that PUMA surpasses existing Web agents. It also identifies scope limits and directions for future work.

7 Conclusion and Future Work

  • PUMA consistently surpasses existing Web agents on PersonalWAB, aligning better with personalized instructions and preferences.The paper says its task, benchmark, and framework may broaden research and inspire new methods in Web-agent scenarios.Source: §7 · p. 8
  • Future work includes broader task scenarios, dynamic preference learning, and user-in-the-loop agents that incorporate feedback and request missing information.The authors propose these directions to improve adaptability and task completion effectiveness and efficiency.Source: §7 · p. 8
  • The study currently focuses on shopping scenarios, and extending the approach to broader Web environments presents complexities requiring further investigation.The authors also flag ethical and privacy considerations and potential biases in personalization.Source: §7 · p. 8

A.1 Details of Profile Evaluation Experiments

This appendix describes profile-consistency evaluations and baseline approaches, then reports implementation details and analyses of PUMA components and memory length.

A.1 Details of Profile Evaluation Experiments

  • The profile evaluations report significant improvements in user prediction and recommendation, indicating profiles are more distinct and better aligned with user behaviors.Profile-behavior consistency uses top-1 accuracy; profile-product consistency ranks positive over negative items using NDCG@5 and Recall@5.Source: §A.1 · p. 10

A.2 Details of Baselines

  • Baselines span memory retrieval, enhanced reasoning, and recommendation-specific memory, including ReAct [56], Reflexion [42], RecMind [49], and InteRecAgent [17].Memory variants include no, random, recent, and instruction-relevant history; Reflexion and InteRecAgent are evaluated only in the multi-turn setting.Source: §A.2 · pp. 10–11

A.3 Implementation Details

  • Benchmarking uses gpt-4o-mini-2024-07-18 for profiles and multi-turn user messages, but claude-3-5-sonnet@20240620 for instructions; PUMA fine-tunes LLaMA2-7B with LoRA at memory-token lengths 256, 512, and 768.PUMA uses generated function parameters as initial SFT labels and then performs DPO.Source: §A.3 · p. 11

A.4 More Analysis

  • Removing memory or SFT causes a substantial accuracy decline across tasks, while removing DPO causes a smaller decrease.The authors interpret the SFT result as difficulty generating user-aligned function parameters without fine-tuning; they say DPO helps align outputs with preferences.Source: §A.4.1 · p. 11

A.4.1 Ablation study.

  • Table 6 varies memory-token length and measures both function accuracy and result accuracy.The supplied passage introduces the comparison but does not provide its full findings.Source: §A.4.2 · p. 11

A.4.2 Analysis on memory length.

The analysis examines how memory length relates to task accuracy and describes action transitions in multi-turn search and recommendation.

A.4.2 Analysis on memory length.

  • Memory length has little effect on function accuracy but substantially affects result accuracy in recommendation; shorter memory limits stored products, reducing recommendation accuracy.Search and review are less sensitive because the agent relies more on the instruction than on memory.Source: §A.4.2 · p. 12

A.4.3 Analysis on action transitions.

  • Search alternates between search and respond actions, while recommendation shows more entangled transitions and greater difficulty in adapting to user intent.Figure 8 maps interaction steps horizontally and uses color-band width for the agent’s focus on each function.Source: §A.4.3 · p. 12, Figure 8 · p. 12
  • The multi-turn attempt analysis measures result accuracy and solved-task counts as attempts increase.Figure 9 presents performance across multiple attempts in the multi-turn track.Source: §A.4.4 · p. 12, Figure 9 · p. 12

A.4.4 Analysis of multi-turn performance variation.

The analysis reports how multi-turn performance changes over attempts and notes the user-centered importance of returned-item relevance.

A.4.4 Analysis of multi-turn performance variation.

  • Most tasks are completed within the first five attempts, while result accuracy declines in later attempts as harder tasks remain; the authors suggest limited multi-turn training data may hinder effective use of later feedback.Review tasks are typically finished within two attempts, and higher later-step accuracy occurs only in rare outliers.Source: §A.4.4 · p. 12
  • For users, the relevance of returned items matters more than whether the agent uses search or recommendation.The passage frames search and recommendation as distinct functions whose distinction may not be meaningful from a user’s perspective.Source: §A.4.5 · p. 12

A.4.5 Analysis on function usage and outcome accuracy.

Outcome Accuracy evaluates returned-result correctness independently of the function used for search and recommendation. PUMA achieves the highest Outcome Accuracy across both tasks.

A.4.5 Analysis on function usage and outcome accuracy.

  • Outcome Accuracy measures returned-result correctness independently of the function used for search and recommendation.The authors present it as a more balanced perspective because function and result accuracy vary significantly between the two tasks.Source: §A.4.5 · p. 12
  • PUMA achieves the highest Outcome Accuracy across both search and recommendation tasks.This evaluates result relevance without restricting correctness to the function selected.Source: §A.4.5 · pp. 12–13
  • The section identifies Outcome Accuracy as a way to prioritize relevance over strict function adherence in evaluating these tasks.The authors connect this priority to real-world user needs.Source: §A.4.5 · p. 13

A.4.6 Analysis on search function implementation.

PUMA outperforms all baselines with both tested retrieval models and in sparse-history scenarios, though task effects vary. The section also discusses fairness, privacy, benchmark scope, and prompting details.

A.4.6 Analysis on search function implementation.

  • PUMA consistently outperformed all baselines with both BM25 and Sentence-BERT retrieval, despite slight result-accuracy degradation across baselines with dense retrieval.The benchmark evaluates agents’ use of provided search functions rather than optimizing the functions themselves.Source: §A.4.6 · p. 13
  • Among 139 users with fewer than 10 historical records (16.2% of the test set), PUMA remained best across tasks, while sparse history affected search, recommendation, and review differently.Search stayed stable or improved slightly, recommendation improved, and review declined; the analysis is in the single-turn track.Source: §A.4.7 · p. 13

B.1 Ethical and Privacy Considerations

  • Personalization may amplify popularity or historical-data biases and discriminate against user groups; the authors call for fairness-aware and diversity-promoting approaches.They also flag data-security risks, including exposure or exploitation of browsing and purchase histories if not properly secured.Source: §B.1 · p. 13

B.2 Scope of This Work

  • The benchmark focuses on shopping functions, while the authors describe the framework as extendable to domains such as news recommendation and social-media curation.Shopping was selected for its structured behavior data and diverse interactions, including search, recommendation, and reviews.Source: §B.2 · p. 13

C Prompting Details

  • Prompting details cover profile generation, search, recommendation, review instructions, user simulation, and execution templates for single- and multi-turn tracks.The profile-generation prompt asks a simulated shopper to summarize basic information using historical purchases and corresponding reviews.Source: C Prompting Details · pp. 13–14, Figure 10 · p. 14, Figure 11 · p. 15

Related papers

Loading 2410.17236v2…