PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Faithful Simulation of Human Shopping Behavior

Jiakai Tang, Yan Mi, Jing Yu, Yang Zhang, See-Kiong Ng, Qi Cao, Fei Sun, Xu Chen, Wen Chen, Jian Wu, Han Zhu, Bo Zheng

arXiv:2608.20707v1cs.IR

TL;DR

Realistic shopping simulation is limited by weak long-session memory and step-wise optimization that misses whole-trajectory behavior. RecVerse combines hierarchical memory with trajectory-level RL in a screenshot-grounded agent and releases USB; experiments report improved behavioral fidelity and intent consistency over GUI-based baselines.

  • Problem

    Existing simulators struggle to preserve long-range session context and optimize trajectory-level behavioral fidelity beyond individual logged actions.

  • Method

    RecVerse uses screenshot-based GUI grounding, hierarchical Working, Episodic, and Preference Memory, learned memory updates, and macro- and micro-level trajectory-aligned rewards.

  • Results

    RecVerse improves intent consistency while preserving realistic behavior, producing trajectories closer to real users than strong GUI-based baselines.

  • Takeaways & Limitations

    USB provides an interactive benchmark with real GUI trajectories for offline evaluation, counterfactual analysis, and reinforcement-learning research on user simulation.

Abstract

from arXiv · show

Simulating realistic user shopping behavior underpins offline evaluation and reinforcement learning in e-commerce scenarios. While recent LLM- and VLM-based simulators have made encouraging progress, reproducing a real browsing session remains difficult for two reasons. (i) Memory Challenge: a shopping session spans dozens of pages, yet existing agents either discard long-range observation histories, losing the evolving user state, or naively concatenate them, overwhelming the context window and even degrading simulation quality. (ii) Optimization Challenge: current user simulators are typically supervised to match each logged action via imitation or step-level rewards; the resulting sessions often display unrealistic patterns, such as over-exploration or excessive passivity, which per-step supervision can neither detect nor correct. To address the above challenges, we present RecVerse, a GUI-grounded simulation agent that perceives pages through screenshots and produces faithful multi-turn trajectories. For the memory challenge, RecVerse adopts a cognitive-inspired hierarchical memory: Working Memory for short-term focus, Episodic Memory for in-session traces, and Preference Memory for high-level intent, with memory updates treated as actions so that the agent adaptively learns when and what to memorize. For the optimization challenge, RecVerse is optimized with a trajectory-level RL objective that scores entire sessions, aligning both macro-level action-type distributions and micro-level shopping intent with real users. We further release USB (User Simulation Benchmark), an interactive e-commerce GUI trajectory dataset for multi-turn user simulation. Experiments show that RecVerse significantly outperforms existing baselines in both behavioral fidelity and intent consistency.

1 Introduction

Faithful e-commerce user simulation must preserve long-range browsing context and optimize whole-session behavior, not merely imitate individual actions. RecVerse addresses these challenges with hierarchical memory, trajectory-level reinforcement learning, and the USB benchmark.

  • Faithful shopping simulation supports offline and counterfactual policy evaluation and RL-based recommendation training as recommender systems become more interactive.
  • Existing simulators fall short because rule-based methods use rigid action spaces, while LLM-based agents ignore image-rich, spatially structured pages.
  • Memory problem: Long sessions create a memory problem: pruning history breaks cross-page dependencies, whereas concatenating it overwhelms context and can degrade simulation quality.
  • Optimization problem: Step-wise imitation creates an optimization problem because behavioral noise and per-action supervision do not enforce realistic session-level clicking and exploration patterns.
  • RecVerse: RecVerse combines pixel-level screenshots, hierarchical memory, learned memory updates, and trajectory-level rewards for macro behavioral distributions and micro-level intent.
  • Contributions: The paper releases a GUI-grounded benchmark with pixel-level observations and action annotations for long-horizon reinforcement learning, alongside RecVerse’s reported gains in behavioral fidelity and intent consistency.

2 Preliminary

The task is to generate process-oriented, GUI-grounded browsing trajectories that reflect users’ exploration and cognitive states over long sessions. The preliminary discussion contrasts step-wise imitation with reinforcement learning for optimizing behavior beyond single-action correctness.

  • Task Definition: The simulator observes page-level screenshots and generates user actions conditioned on the current observation and historical context.
  • Task Definition: Unlike goal-oriented GUI agents, user simulation models how people explore, hesitate, compare, and act rather than only whether they complete a task.
  • Task Definition: Faithful simulation targets multimodal trajectory distributions and long-horizon cognitive states instead of a single optimal path or explicit subgoals.
  • Imitation Learning: Imitation learning maximizes the likelihood of logged user actions and provides a stable behavioral prior for later reinforcement-learning optimization.
  • Reinforcement Learning: Reinforcement learning optimizes behavior through reward signals beyond token-level correctness, with GRPO using group-relative comparisons without a learned value function.
  • Reinforcement Learning: Prior reinforcement-learning approaches still center rewards on step-wise imitation, leaving human-like memory and trajectory-level behavioral fidelity largely unaddressed.

3 RecVerse

RecVerse combines hierarchical memory with trajectory-level reinforcement learning to simulate faithful multi-turn shopping behavior. It also introduces USB, a GUI trajectory benchmark for interactive e-commerce user simulation.

  • Cognitive-Inspired Hierarchical Memory: Memory writing is part of the action space, allowing reinforcement learning to determine when and what Episodic and Preference Memory should store.Episodic records support behavioral continuity and provide grounding for preference distillation.
  • Cognitive-Inspired Hierarchical Memory: Its three-level memory separates recent visual context, session-level events, and distilled high-level preferences.Working Memory is capacity-limited, Episodic Memory records selected interaction events, and Preference Memory captures attributes, comparison criteria, and dislikes.
  • Trajectory-Aligned RL: The trajectory objective combines macro-level behavioral-distribution alignment with micro-level intent-weighted category alignment.Macro reward targets item-related action frequencies while micro reward evaluates item choices through hierarchical product categories and intent-strength weights.
  • Trajectory-Aligned RL: RecVerse models user simulation as trajectory generation, producing reasoning, environment actions, and memory updates at each step.The policy is warm-started with imitation learning and refined using GRPO with complete multi-turn rollouts.
  • USB Benchmark: USB provides overview statistics for trajectories, items, and users as the benchmark accompanying RecVerse.The supplied passage identifies Table 1 as organizing USB statistics across these three dimensions.

4 Experiments

Experiments evaluate USB’s interactive GUI benchmark and RecVerse across behavioral fidelity, intent consistency, ablations, and scaling. RecVerse combines GUI perception, hierarchical memory, and trajectory-level rewards to achieve realistic behavior and strong intent alignment.

  • Experimental Setup: USB contains 5,274 screenshot-action browsing trajectories with eight action types, three taxonomy levels, user profiles, and an interactive environment for online multi-turn RL.The benchmark includes complete GUI-grounded interactions and real page observations as feedback.
  • Experimental Setup: USB is the first benchmark described here to combine visual trajectories, diverse actions, user profiles, and interactive feedback for online multi-turn agentic RL.Existing benchmarks lack one or more of these capabilities, while OPeRA provides only static offline GUI logs.
  • Experimental Setup: Evaluation jointly measures behavioral fidelity against real-user distributions and intent consistency at item and category levels.Behavioral metrics include ATL, CTR, IPVR, ACR, and CVR; intent metrics include item-level HR, Precision, Recall, F1, and category-level similarity.
  • Overall Performance: RecVerse-GUI with RL improves item-level F1 from 4.27 to 7.19, HR from 5.92 to 10.45, and HCO from 23.11 to 32.64 versus STA.The model also lies near the real-user high-fidelity band, indicating joint behavioral realism and intent alignment.
  • Ablation Study: The full memory hierarchy outperforms removing Preference Memory or retaining only Working Memory, while Episodic and Preference Memory serve complementary abstraction levels.Episodic Memory organizes session events, whereas Preference Memory accumulates higher-level interests guiding later decisions.
  • Ablation Study: Removing the micro-level reward causes the largest degradation in item- and category-level intent metrics, while removing the macro-level reward weakens trajectory-level behavioral constraints.The reward analysis supports combining distributional behavior matching with intent-aware optimization.
  • Further Analysis: Scaling from 2B to 4B improves intent metrics consistently while maintaining broadly realistic session behavior, although some behavioral dimensions remain mismatched.Precision, Recall, F1, HR, and category metrics rise, while behavioral effects are non-uniform across ATL, CTR, ACR, CVR, and IPVR.

5 Related Work

RecVerse extends GUI-agent research from goal completion to faithful process-oriented shopping simulation. It addresses the need to model visual browsing, long-horizon behavior, and intent-consistent evaluation.

  • GUI Agents: GUI agents perceive graphical interfaces from screenshots, but conventional benchmarks mainly evaluate completion of prescribed tasks.RecVerse instead targets how users browse, hesitate, compare, and decide.
  • GUI Agents: RecVerse repurposes GUI perception for process-oriented simulation, requiring behavioral coherence, intent exploration, and faithful action distributions.This shift also motivates long-horizon memory and trajectory-level reinforcement learning.
  • User Behavior Simulation: Classical and LLM-based simulators model structured features or textual histories but ignore the visual layout that drives real browsing decisions.Recent methods add supervision over real trajectories, yet the related-work discussion identifies remaining limitations.
  • User Behavior Simulation: Rule-based simulators use probabilistic models over vectorized item features, whereas LLM-based agents add reasoning and persona modeling over textual observations.These approaches differ in observation modality from GUI-grounded simulation.

6 Conclusion

The paper introduces RecVerse and USB for GUI-grounded e-commerce user simulation. Experiments report trajectories closer to real users, with improved intent consistency while preserving realistic behavior.

  • Contributions: RecVerse combines cognitive-inspired hierarchical memory with trajectory-aligned reinforcement learning for long-horizon context and behavioral fidelity.The system targets both realistic browsing dynamics and user intent.
  • Contributions: USB is an interactive benchmark containing real GUI trajectories, diverse actions, user profiles, and pixel-level observations.It supports multi-turn user behavior simulation in e-commerce recommendation.
  • Findings: RecVerse improves intent consistency while preserving realistic behavior relative to strong GUI-based baselines.The reported trajectories are closer to real users.
  • Implications: The work aims to support offline evaluation, counterfactual analysis, and reinforcement-learning-based recommender optimization.These are presented as applications for more reliable user simulators.

A Evaluation Metric Definitions

The evaluation separates whether a simulator behaves like a real user in aggregate from whether it engages with content matching the real user’s interests. Behavioral metrics therefore compare production-style funnel statistics directly with real-user references.

  • Evaluation Design: Evaluation asks whether simulated aggregate behavior resembles real users and whether interacted content matches what real users would engage with.These are treated as complementary questions for long-term user behavior simulation.
  • Behavioral Fidelity: Behavioral fidelity uses click-through, add-to-cart, and conversion rates compared directly against real-user references.The design avoids normalized divergences that can conceal absolute interaction volume.
  • Behavioral Fidelity: Direct funnel-statistic comparison prevents an over-active agent with realistic action proportions from appearing aligned solely through KL or JS divergence.The rationale concerns action-type statistics rather than the specific items acted upon.

A.1 Behavioral Fidelity Metrics

Behavioral fidelity measures how closely simulated action statistics match real-user behavior. The metrics aggregate trajectory actions into funnel rates and trajectory length, while intent metrics separately assess item and category alignment.

  • Behavioral Fidelity: Behavioral fidelity rates are computed from aggregated counts across all evaluated trajectories.The framework defines rates using total exposed items and total actions of each type.
  • Behavioral Fidelity: CTR, IPVR, ACR, and CVR measure click, post-click item-page view, post-click add-to-cart, and post-click conversion rates.ATL reports the average number of steps per trajectory.
  • Behavioral Fidelity: Faithfulness is evaluated by minimizing divergence from real-user statistics rather than maximizing or minimizing any individual metric.Closer metric values to real users indicate more realistic simulation.
  • Intent Consistency: Item-level intent alignment compares agent and real-user interaction sets using exact-match Hit Rate, Precision, Recall, and F1.These metrics are computed per trajectory and averaged over the test set.
  • Intent Consistency: Category-level alignment uses Category Precision, Category Recall, and Hierarchical Category Overlap based on similarity in the category hierarchy.The category metrics can be evaluated across all active actions or restricted to sub-action types.

B.1 Category Distribution Analysis

USB spans three taxonomy levels, combining a dominant Apparel category with broad long-tail coverage and fine-grained shopping intents. Its analysis uses category-frequency visualizations to characterize this distribution.

  • Category Distribution: Figure 3 reports the top-10 product categories at each of USB’s three taxonomy levels alongside a word cloud of sampled categories.The visualizations summarize category coverage at coarse, intermediate, and fine-grained levels.
  • L1 Categories: Apparel dominates the dataset at L1 with 51.0%, while Daily Necessities, Jewelry, Food & Drink, Beauty Products, and Luggage & Bags provide long-tail coverage.The leading non-apparel categories range from 3.3% to 5.7%.
  • L2 Categories: At L2, Clothing accounts for 40.2% and Shoes for 8.5%, followed by Food, Bags, Dining Utensils, and Skincare.These figures show that non-apparel categories remain represented below the dominant apparel-related groups.
  • L3 Categories: At L3, Tops is largest at 17.8%, followed by Pants, Dresses, Clothing Sets, and additional footwear and lifestyle categories.The fine-grained distribution exposes specific shopping intents such as Sneakers, Casual Shoes, Backpacks, Ready-to-eat, and Sandals.
  • Distribution Pattern: Overall, USB exhibits a natural long-tail pattern intended to capture both dominant consumer interests and diverse niche shopping intents.The word cloud provides an intuitive overview of category prevalence, with font size corresponding to frequency.

C.2 Implementation Details

RecVerse training combines full-parameter fine-tuning with a frozen visual encoder, teacher-generated imitation data, truncated interaction history, and smoothed action resampling. Rollout prompts are dynamically populated with user, memory, observation, and action information.

  • Training Setup: Experiments use Qwen3.5-2B with full-parameter fine-tuning for imitation and reinforcement learning while freezing the visual encoder.Training uses Megatron-LM for distributed training and vLLM for rollout inference.
  • Training Data Construction: Imitation data are synthesized from real trajectories by a Qwen3.5-397B-A17B teacher to warm-start policies with RecVerse’s reasoning and memory-update format.The teacher reconstructs latent user state and memory updates from the profile, hierarchical memory, visual observation, and logged next action.
  • Context Management: Historical click and interaction sequences are truncated to at most 20 entries to avoid excessive context.The memory state is updated from generated memory content and used to synthesize subsequent steps until trajectory termination.
  • Action Resampling: Power-law smoothed resampling mitigates long-tailed action imbalance by increasing minority-action sampling while preserving the original distribution’s relative structure.The experiments set α=0.3 and apply the same resampling strategy to trainable baselines for fair evaluation.
  • Rollout Generation: The rollout prompt is a condensed template whose placeholders are filled dynamically with the user profile, historical records, current memory, GUI observation, and executable actions.The dynamic fields support page-specific rollout generation.

D Additional Analysis

Additional analyses show that category-level matching provides stronger intent guidance than strict item matching, while a case study illustrates aligned browsing and purchase behavior supported by hierarchical memory.

  • Matching Granularity: Strict item matching gives credit only when the simulated action reaches a logged item, whereas category matching grants partial credit to semantically related items.The matching hierarchy can relate products through categories such as clothing, shoes, and sandals.
  • Matching Granularity: Category matching improves F1 from 4.90 to 7.19, HR from 6.51 to 10.45, and HCO from 27.41 to 32.64 over strict item matching.It also yields smaller deviations in CTR, CVR, and IPVR, although strict item matching is closer on ATL and ACR.
  • Case Study: The case study shows RecVerse and a real user following a similar pattern of product-list browsing, a low-price household-item click, and purchase.The simulated rollout preserves short exploration followed by a focused purchase decision despite slightly different scroll counts.
  • Case Study: Mindset and Preference Memory frame the purchase as a useful, low-risk kitchen addition and abstract it as interest in cost-effective kitchen gadgets.Recent visual context supports immediate action, while accumulated preference evidence provides continuity for longer-term decisions.

E Human Evaluation Interface

The human-evaluation interface compares two anonymized trajectories conditioned on the same user information and asks annotators to identify which appears machine-generated.

  • Evaluation Interface: Each evaluation task presents the user profile and historical interactions alongside two anonymized trajectories displayed side by side.The trajectories are conditioned on the same user profile.
  • Annotation Procedure: Annotators inspect action sequences, corresponding GUI observations, and interacted items highlighted on the page before selecting the machine-generated trajectory.They also write a brief difference summary as part of the task.
Loading 2608.20707v1…