Source-linked AI summary
LiveSim: Simulating Environment-Shaped Users in Multi-Agent Live-Stream Ecosystems
Jiaqi Xu, Yiran Qiao, Jing Chen, Qiwei Zhong, Xiang Ao, Xueqi Cheng
TL;DR
Static profiles are inadequate for live-stream users whose behavior changes with interaction environments, limiting realistic ecosystem simulation. LiveSim instead shapes editable behavioral hypotheses from trajectory mismatches and transfers recurring environment–behavior patterns through collective memory. Experiments on real-world risk-control data show improved user-level behavioral fidelity and support ecosystem-level analysis of risk evolution and interventions.
Problem
Live-stream interaction dynamics continuously reshape user behavior, while static profiles inferred from historical observations provide underspecified characterizations for unseen conditions.
Method
LiveSim progressively refines editable behavioral hypotheses using trajectory-grounded mismatches, extracts transferable environment–behavior patterns, and accumulates them in collective behavioral memory.
Results
LiveSim substantially improves user-level behavioral fidelity and supports ecosystem-level simulation of risk evolution and platform interventions.
Takeaways & Limitations
Explicitly modeling how environments reshape users supports both individual behavioral fidelity and ecosystem-level live-stream analysis.
Takeaways & Limitations
Experiments use live-streaming risk trajectories from one platform, and inferred behavioral hypotheses are simulation constructs rather than users’ true intent or psychological states.
Abstract
from arXiv · showhide
User behavior simulation with large language models~(LLMs) is increasingly used to support multi-agent ecosystem simulation. Existing simulators typically rely on static user profiles inferred from historical observations, which become inadequate in socially intensive environments such as live streaming where interaction dynamics continuously reshape user behavior. We propose \textbf{LiveSim}, an LLM-based framework for live-stream ecosystem simulation. It represents users as editable behavioral hypotheses and progressively refines them through trajectory-grounded interactions, where discrepancies between simulated and observed trajectories reveal missing environmental shaping effects. These signals are further extracted as transferable environment-behavior patterns and accumulated in a collective behavioral memory to improve user-level behavioral fidelity and support ecosystem-level simulation. Experiments on real-world live-stream risk-control data validate the effectiveness of LiveSim in improving user-level behavioral fidelity and enabling ecosystem-level analysis of risk evolution and platform intervention effects.
1 Introduction
LiveSim addresses the limits of static user profiles in live-stream environments, where interaction dynamics continuously reshape behavior. It models users as editable behavioral hypotheses and uses trajectory discrepancies to extract transferable environmental shaping patterns for ecosystem simulation.
- Motivation: Existing simulators commonly represent users with predefined descriptions, but live-stream interaction environments dynamically reshape behavioral tendencies within sessions.Relevant signals include streamer persuasion, real-time viewer feedback, and evolving session dynamics.
- Motivation: Historical logs record users’ eventual actions but provide sparse evidence about how behavior forms under unseen interaction conditions.User characterizations inferred from history are therefore underspecified for predicting responses to new environments.
- Core idea: LiveSim reconceptualizes user characteristics as editable behavioral hypotheses that are revised as interactions unfold.The hypotheses are provisional assumptions inferred from available evidence rather than fixed profiles.
- Core idea: Discrepancies between simulated and observed trajectories diagnose missing environmental shaping effects, while recurring conditions motivate transferable environment–behavior patterns.These patterns generalize across users and sessions instead of treating every mismatch as an isolated error.
- Implications: LiveSim uses shaped user behavior to support ecosystem-level simulation for evaluating risk evolution and platform intervention strategies.The framework is intended to connect individual behavioral fidelity with trustworthy, controllable ecosystem analysis.
2 LiveSim Framework
LiveSim hierarchically shapes individual user hypotheses from sparse histories and trajectory-grounded mismatches, then deploys the evolved agents in closed-loop live-stream ecosystems. Its RBHS process converts mismatches into environment-conditioned patches, evaluates them through reflection, and transfers recurring patterns through collective memory.
- Individual-level shaping: LiveSim starts each user from a behavioral hypothesis inferred from sparse history and progressively shapes it through interaction evidence.The hypothesis captures identity cues, interests, and risk-relevant tendencies while remaining editable during simulation.
- Ecosystem-level simulation: Evolved user agents interact with fixed streamer and shill agents in closed-loop sessions to study fraud evolution and platform interventions.Simulated sessions replay the membership of real historical sessions, with each audience member instantiated from its evolved hypothesis.
- Individual-level shaping: Behavioral probes use rich session context and sampled same-trajectory history to predict one next action without replaying the full serial trajectory.Each mismatch compares the probed action with the observed action at the corresponding step.
- Behavioral shaping loop: Mismatch signatures localize environmental conditions that the current hypothesis fails to explain, enabling targeted environment–behavior patches.Downstream shaping consumes the summarized signature rather than the raw prediction error.
- Behavioral shaping loop: RBHS evolves each hypothesis as P_0u ⊕ ∆u, where ∆u is an accumulated set of mismatch-driven additive patches.Patches encode how specific session conditions shift latent user state and action propensity.
- Behavioral shaping loop: Reflection re-probes patched hypotheses, classifies patches as effective, regressive, or neutral, and routes still-failing evidence into further refinement.Each user runs the shaping loop for three rounds, balancing computational cost with patch-convergence depth.
- Collective memory: Collective Behavioral Memory pools recurring reflections across users and supplies transferable priors to subsequent users during cold start.A relevance gate selects memory entries that fit the target user’s initial hypothesis and probe evidence.
3 Experiments
LiveSim is evaluated for individual behavioral fidelity and ecosystem-level dynamics using real live-stream sessions, with trajectory-grounded and closed-loop protocols. Results show that reflective shaping improves fidelity, while the ecosystem analysis reveals nonlinear fraud dynamics and benefits from state-aware intervention.
- Experimental Setup: The dataset contains 14,391 user-session pairs from 1,963 users, with 3,124 held out for trajectory-grounded testing.Users without historical logs are outside the study’s scope.
- Evaluation Protocols: Trajectory-grounded evaluation pins the historical session context and tests whether an evolved user hypothesis reproduces the target user’s actions step by step.The closed-loop protocol jointly simulates streamers and viewers to study emergent session dynamics and counterfactual interventions.
- Overall Performance: HIT can overstate fidelity: Qwen2.5 7B reaches 54.39 HIT yet trails raw Doubao models on distributional and judge metrics.The paper attributes this pattern to generic comments that increase exact matches while distorting action distributions.
- Overall Performance: RBHS improves balanced fidelity across metric families, ranking first on Doubao 1.8 for A-JSD, Conv-Acc, and all three LLM-Judge metrics.Its largest reported gains are A-JSD −43% and Conv-F1 +13%, while HIT changes modestly.
- Ablations: The two reflection mechanisms are complementary: LiveSim reaches Conv-F1 55.33 and A-JSD 0.0302, exceeding either local or global reflection alone.Direct alone improves Conv-F1 from 48.89 to 51.77 and A-JSD from 0.0534 to 0.0404 over Basic Profile.
- Fraud Evolution: In closed-loop fraud simulations, increasing shill density raises final conversion from 20.61% to 56.97%, but gains diminish as density increases.The first increase adds 19.54 percentage points, whereas the 0.3-to-0.4 increase adds 6.25 points.
- Fraud Evolution: The first 10 rounds produce 62%–77% of final conversions, identifying a front-loaded intervention window.Later signals have less room to change outcomes because the remaining audience has either converted or become resistant.
- Fraud Evolution: At fixed shill density 0.4, empathetic-converter shills raise conversion from 56.97% to 59.55% compared with uniform supporters.The staged empathy-then-conversion behavior provides a stronger persuasion signal at the same headcount.
4 Related Work
Recent work uses LLMs to simulate users and multi-agent ecosystems across dialogue, recommendation, virtual communities, and other human-centered settings. These approaches commonly construct simulators from personas, historical observations, or profile information, but are not directly comparable under the paper’s protocol.
- LLM-based user simulation spans dialogue systems, recommendation, and broader human-centered scenarios.
- Existing approaches commonly build user simulators from persona descriptions, historical observations, or profile information for specific interaction contexts.
- Multi-agent ecosystem simulation studies collective behavior and emergent social dynamics in virtual communities, large-scale social systems, and domain-specific ecosystems.
- Because prior methods use different tasks and feedback signals, the paper re-implements their core mechanisms as controlled baselines rather than comparing them directly under their original protocols.
5 Conclusion
LiveSim models live-stream users as editable behavioral hypotheses rather than static profiles inferred from history. It uses trajectory discrepancies to capture environmental shaping, improving user-level fidelity and supporting ecosystem-level studies of fraud evolution and interventions.
- LiveSim is an LLM-based framework for simulating environment-shaped users in live-stream ecosystems.
- The framework progressively refines editable behavioral hypotheses through interactions instead of treating user characteristics as static historical profiles.
- Discrepancies between simulated and observed trajectories reveal missing environmental shaping effects for hypothesis refinement.
- LiveSim improves user-level behavioral fidelity and supports ecosystem-level simulation of fraud evolution and intervention strategies.
7 Limitations
The experiments use live-streaming risk trajectories from one specific platform setting, and the framework’s effectiveness in other domains remains unvalidated. LiveSim’s behavioral hypotheses are simulation constructs, not ground-truth user profiles or psychological states.
- Experiments are conducted on live-streaming risk trajectories from a specific platform setting.
- Effectiveness in e-commerce chat, short-video comments, open social networks, and other domains remains to be validated.
- Behavioral hypotheses inferred from sparse interaction logs should not be interpreted as users’ true intents or psychological states.
Ethical Considerations
LiveSim uses de-identified interaction logs under the source platform’s terms of service, removes or hashes identifying fields, and restricts intended use to safety-oriented analysis. The paper explicitly prohibits optimizing persuasive manipulation or engagement with risky content.
- LiveSim is built on de-identified live-streaming interaction logs collected under the source platform’s terms of service.
- Identifiers, usernames, and free-text fields are removed or hashed before prompt construction.
- The framework should be used only for safety analysis, moderation evaluation, and protective intervention design.
- It should not be used to optimize persuasive manipulation or increase engagement with risky content.
A.1 Dataset
The dataset represents behavior at both user-session and session levels, combining interaction logs with metadata and structured behavioral evidence. LiveSim uses these inputs to initialize and refine user behavioral hypotheses through trajectory-grounded probes and local patches.
- Dataset units: Each user-session pair contains session metadata, visible context, and the target user’s action sequence, while closed-loop simulation operates at the session level.The dataset contains 7,243 unique sessions and 14,391 user-session pairs across the stated splits.
- Session labels: The dataset primarily targets platform-annotated fraud sessions and adds normal sessions as white samples for distinguishing risk-driven from ordinary engagement.White samples comprise 11.62% of history pairs and 13.96% of test pairs.
- Behavioral hypothesis inputs: Initial hypotheses combine behavioral statistics, scene distributions, representative events, identity, psychological needs, language features, and anchor preferences.The psychological facet infers motivational needs and visit goals, while language features include tone, question rate, and emotion intensity.
- Behavioral hypothesis inputs: Representative events preserve context by pairing prioritized user actions with the four preceding actions from the same session.At most six representative events are retained per user, with priority given to follow, group-join, private-message, and comment events.
- Trajectory-grounded shaping: LiveSim converts historical sessions into coverage-ranked interaction and conversion probes, then evaluates action alignment and patch locality.Candidates are grouped by low-stake interaction versus high-stake conversion steps, and up to six probes are selected using contextual richness and action coverage.
- Trajectory-grounded shaping: Patches are self-contained local corrections with triggers and effects on interest, trust, desire, and fatigue, plus replacement and locality metadata.This schema supports targeted behavioral updates rather than unrestricted persona rewrites.
B.1 RBHS Robustness Study
The robustness study evaluates RBHS under an identical trajectory-grounded protocol across diverse LLM backbones. Results show consistent distributional and judge-based improvements, with larger gains for weaker backbones and reduced performance dispersion.
- Cross-backbone robustness: Across eight open and closed backbones spanning small and large scales, A-JSD and Tr-JSD decrease while all three judge axes improve.Evaluation uses a random 10% test subset and the same metric definitions as the main study.
- Cross-backbone robustness: RBHS transfers through cognitive-state updates rather than direct action-token biasing, allowing the shaping signal to work across different action heads.The updated dimensions are interest, trust, desire, and fatigue.
- Backbone differences: The bottom-half backbones gain +13.9 Consist on average, compared with +11.5 for the top half.The bottom half is defined by base Consist values ranging from 55.61 to 60.54 in the cited comparison.
- Backbone differences: After shaping, the inter-backbone Consist spread shrinks from 11.2 to 7.6 points.The reported pattern indicates that weaker backbones benefit disproportionately and narrow their gap with stronger ones.
B.2 Fine-Grained LLM Behavior Analysis
The fine-grained analysis examines how RBHS changes action distributions across backbones. It finds that shaping preserves substantial backbone-specific default behavior rather than collapsing agents into one common action policy.
- Distribution comparison: The analysis compares each backbone’s initial-hypothesis action distribution with its post-patch distribution under the trajectory-grounded protocol.The comparison focuses on distributional shifts produced by applying environment-behavior patches.
- Backbone-specific behavior: Backbones retain distinct default action mixes after RBHS, spanning comment-heavy, balanced, and conversion-eager behaviors.The reported spectrum ranges from comment-dominant gpt4omini and Qwen models to conversion-eager GPT-5.4-mini.
- Backbone-specific behavior: gpt4omini and Qwen series produce comments in 91.7% and 86.4% of actions, respectively, while GPT-5.4-mini has the highest follow-plus-join-group share.GPT-5.4-mini’s conversion-oriented share changes from 22.4% to 19.7% after patching.
B.3 Why is Trajectory-grounded Protocol
Trajectory grounding keeps simulated histories coherent by evaluating behavior within observed interaction trajectories instead of allowing deviations to compound against static responses. The study reports severe conversion degradation without grounding and uses three reflection rounds as the preferred trade-off.
- Protocol rationale: Teacher-forcing the history channel while keeping the current decision free avoids off-policy inconsistency caused by simulated actions diverging from static logged responses.Without grounding, incoherent feedback can compound into low-trust question loops.
- Protocol ablation: Without trajectory grounding, Conv-F1 on Doubao 1.8 collapses to 7.08.The comparison is made against the same backbone under the trajectory-grounded protocol.
- Protocol ablation: Free-rollout RBHS still improves every reported distributional, conversion, and judge metric, but its absolute gains are not comparable because the baseline begins at 7.08.The paper therefore evaluates RBHS against the trajectory-grounded baseline throughout.
- Reflection rounds: Direct patching captures most improvement, while later reflection rounds provide smaller additional gains and Round 3 gives the best overall trade-off.Round 4 slightly reduces Conv-Acc and F1, indicating mild over-refinement.
- Incremental updating: RBHS supports incremental updates by adding effective new patches and retiring ineffective replaced patches without recomputing the full history.This permits lightweight continuous updates as new sessions arrive.
C.3 Future Work Deployment Scenarios
LiveSim is positioned for environments where user actions are shaped over time by observable stimuli, with extensions beyond live-streaming simulation. It also supports intervention-agent training and synthetic data generation for rare or emerging scenarios.
- RBHS is especially suited to scenarios where user actions are temporally shaped by observable environmental stimuli.
- The approach can extend to online communities, interactive social platforms, and e-commerce live-streaming settings.
- LiveSim can provide counterfactual trajectories and reward signals for post-training intervention agents.These signals include intervention success rate and disruption felt by users.
- LiveSim can also serve as a data synthesizer for rare or emerging scenarios.