PaperlayerView exact revision on arXiv

Source-linked AI summary

PsyMemBehav: Psychology-Informed and Memory-Grounded Two-Level Identity-to-Action Reasoning for Social-Media Behavior Simulation

Wenji Zhang, Xilong Cheng, Yunxiao Qin, Mai Tu, Tongye Zhu, Yuhao Zhou, JingBo Ji, Yuting Tan, Duoyin Li, Hongjiang Xiao, Yuan Zhang

arXiv:2610.14570v1cs.AIcs.SI

TL;DR

Predicting real users’ social-media behavior requires modeling both platform decisions and text from partial behavioral histories and thread context. PsyMemBehav combines profile and memory grounding with target-first reasoning and trajectory-level training. Its 27B model ranks first in 11 of 12 individual metrics and achieves the lowest aggregate group-level deviation among simulated systems.

  • Problem

    Social-media behavior prediction must infer target selection, platform actions, and textual responses from partial behavioral histories and current thread context.

    Source: §1 · p. 1

  • Method

    PsyMemBehav integrates psychology-informed profiles, longitudinal memories, and thread context through identity activation and target-first action decisions, with rubric-guided alignment of complete trajectories.

    Source: §1 · p. 2

  • Results

    PsyMemBehav-27B ranks first in 11 of 12 individual metrics and achieves the lowest aggregate group-level deviation among simulated systems.

    Source: §1 · p. 2, §6 · p. 7

  • Takeaways & Limitations

    The results support evidence-grounded user modeling for evaluating thread-level public-opinion simulation in observed social-media contexts.

    Source: §1 · p. 1, §6 · p. 7

    The benchmark is conditional on recorded engagements, has no explicit no-action label, and cannot distinguish unseen threads from viewed-but-ignored threads.

    Source: §1 · p. 2

Abstract

from arXiv · show

Large language models (LLMs) support social simulation, but faithfully reproducing real users requires grounding both structured platform decisions and language in sparse, noisy behavioral evidence. We introduce PsyMemBehav, which maps psychology-informed profiles and retrieved memories to target-first platform behavior through two-level identity-to-action reasoning. To support this approach with comprehensive behavioral traces, we construct a dataset PsyMemBehav-bench from 5.57M observed behaviors of 5,862 real users (approximately 950 per user) across Reddit, Bluesky, and Twitter. Each recorded engagement is modeled as target selection followed by Comment, Like, or Repost and optional response generation. PsyMemBehav combines 41-dimensional user descriptors, top-5 BM25 memories, reverse-CoT supervision, and rubric-guided DPO over complete reasoning-action trajectories. Experiments show that PsyMemBehav 27B ranks first in 11 of 12 reported individual-level metrics, improving average response quality over GPT-5.4 from 54.28 to 59.68 (9.95% relative), and obtains the lowest group-level MAE (0.081) and MSE (0.009) among simulated systems. These results support evidence-grounded user modeling for observed-thread continuation; the benchmark remains conditional on recorded engagement and does not model no-action decisions.

1 Introduction

PsyMemBehav addresses the challenge of predicting real users’ structured platform decisions and language from behavioral histories and thread context. It combines profile and memory grounding with target-first reasoning and trajectory-level alignment, and evaluates individual behavior and aggregate thread outcomes.

1 Introduction

  • The benchmark predicts target, action, and optional text only after a recorded engagement; it has no no-action label and cannot distinguish unseen threads from viewed-but-ignored threads..Source: §1 · p. 2
  • The framework activates relevant profile traits and longitudinal memories before selecting a target and predicting the platform action and response.It uses 41-dimensional psychology-informed user profiles and thread context.Source: §1 · p. 2
  • PsyMemBehav-bench contains 5.57M behaviors from 5,862 users across Reddit, Bluesky, and Twitter, combining descriptors, history, context, and observed behavior.The dataset supports joint evaluation of structured decisions and text generation.Source: §1 · p. 2
  • Reverse-CoT supervision and rubric-guided DPO optimize complete reasoning-action trajectories, including action fidelity, memory grounding, and persona consistency.Interactions are modeled as Comment, Like, or Repost, with text generated when required.Source: §1 · p. 2
  • PsyMemBehav-27B ranks first in 11 of 12 individual metrics, improves average response quality over GPT-5.4 by 9.95%, and yields the lowest aggregate group-level deviation.The reported improvement is relative.Source: §1 · p. 2

2 Related Work

Prior work explores persona simulation, preference alignment, and group-level social simulation, while PsyMemBehav focuses on real accounts, platform-specific behavior, and historical traces. Its evaluation connects account-level modeling with aggregate thread simulation.

2.1 Social-Media Account Simulation

  • Existing account-simulation approaches often use generic personas or aggregate populations, while PsyMemBehav models real accounts using psychological and platform-specific traits, retrieved memory, and thread context.Social-media simulation also requires selecting interaction targets and platform-specific actions, not only generating text.Source: §2.1 · p. 2

2.2 Rubric-Guided Alignment for Social-Behavior Simulation

  • Rubric-guided preference pairs align complete reasoning-action trajectories rather than optimizing only final textual responses.The rubric addresses simulation requirements such as profile faithfulness, memory grounding, platform-appropriate actions, and thread coherence.Source: §2.2 · pp. 2–3

2.3 Group-Level Thread Simulation

  • PsyMemBehav improves account-level identity-to-action modeling and evaluates whether those improvements transfer to group-level thread simulations.The evaluation perspective extends beyond single-turn plausibility to interaction dynamics, stance distributions, network structures, conversational diversity, and thread-level realism.Source: §2.3 · p. 3

3 Methodology

PsyMemBehav predicts observed social-media behavior by selecting a target before choosing an action and generating any required text. Its method makes relevant identity evidence explicit and trains complete reasoning-action trajectories in two stages.

3.1 Problem Formulation and Overview

  • The simulator predicts a target, then Comment, Like, or Repost, with a comment when required; the task is conditional on recorded engagement.Raw interactions are normalized into those three actions, while original posts and past interactions serve as evidence rather than prediction labels.Source: §3.1 · p. 3

3.2 Two-Level Identity-to-Action Reasoning

  • Profiles combine 22 general psychological dimensions with 19 platform-behavior dimensions, while BM25 retrieves the five highest-ranked pre-time behaviors using the visible thread as its query.The 41 LLM-inferred descriptors are history-derived prediction features, not validated psychometric or clinical measurements; retrieval uses neither dense similarity nor a recency prior.Source: §3.2 · p. 3
  • Identity activation selects profile and memory evidence relevant to the thread before action decision predicts the target, action, and response.The explicit intermediate trace separates history-derived profile information from context-specific action selection.Source: §3.2 · p. 3

3.3 Two-Stage Training

  • Stage 1 reconstructs behavior-conditioned reasoning traces and uses accepted traces for supervised fine-tuning; these traces are structured supervision, not ground-truth cognition.A validation judge retains traces consistent with profile, memory, context, identity activation, and action decision.Source: §3.3 · p. 4
  • Stage 2 applies rubric-guided DPO to complete trajectories, comparing SFT rollouts with references synthesized from observed behavior.The rubric is induced from critiques of simulation weaknesses, consolidated into ten criteria, verified by human annotators, then frozen; action fidelity is up-weighted and preference pairs are filtered for confidence, margin, duplication, and length.Source: §3.3 · pp. 4–5, Figure 2 · p. 4

4 Experimental Setup

The experimental setup evaluates two PsyMemBehav model sizes against closed- and open-weight baselines at individual and group levels. PsyMemBehav-bench provides user-disjoint chronological data with profiles, retrieved memories, thread context, candidate targets, and observed behavior.

4.1 Base Models

  • Experiments compare PsyMemBehav 9B and 27B, trained with the same pipeline, with 27B serving as the stronger main-experiment backbone.The 9B model tests whether the framework remains effective with a smaller simulator.Source: §4.1 · p. 5

4.2 Baselines

  • Baselines include five named proprietary models and Qwen3.5-9B and Qwen3.5-27B; all receive the same profiles, retrieved memories, thread context, and candidate targets.Gold target and action labels are withheld, and models select a target before predicting action and branch-specific text.Source: §4.2 · p. 5

4.3 Evaluation Metrics

  • Individual-level evaluation measures target accuracy, action accuracy, seven comment-quality dimensions, and their average.Gemini-3-Flash judges semantic and intent alignment, emotional and stance consistency, style fidelity, persona memory, and naturalness.Source: §4.3 · p. 5
  • Group-level rollouts are evaluated using reply reciprocity, stance deviation, Top1 KOL, Top5 PageRank, and MAE and MSE across thread-level statistics.Profiled simulators continue the same observed thread seed through target-first actions.Source: §4.3 · p. 5

4.4 Dataset and Data Split

  • PsyMemBehav-bench contains 5.57M behaviors from 5,862 users across Reddit, Bluesky, and Twitter, with approximately 950 behaviors per user.The user-disjoint chronological split assigns 5,000 users to training and 862 to validation and testing; collection may overrepresent active users and underrepresent low-activity, private, or new users.Source: §4.4 · p. 5

5 Experimental Results and Analyses

PsyMemBehav 27B leads most individual-level metrics, while ablations differentiate the contributions of reverse-CoT supervision and rubric-guided DPO. In observed-thread continuation, it achieves the lowest aggregate error, though influence-centralization comparisons remain mixed.

5.1 Individual-Level Results

  • PsyMemBehav 27B ranks first in 11 of 12 metrics; average response quality is 59.68 versus GPT-5.4’s 54.28, a 9.95% relative increase.It also records target accuracy of 65.34 and action accuracy of 71.40.Source: §5.1 · p. 6

5.3 Group-Level Thread Simulation

  • MAE is 0.081 and MSE is 0.009 for PsyMemBehav 27B, the lowest among simulated systems; Top1 KOL and Top5 PageRank comparisons favor other models.Its reciprocity is 0.121 versus the real value of 0.043, and pairwise preference favors PsyMemBehav over GPT-5.4 in 64.0% of continuations versus 34.0%, with 2.0% ties.Source: Table 4 · p. 7, §5.3 · p. 7

5.2 Ablation Study

  • R-CoT plus R-DPO raises target accuracy from 38.79 to 48.22, action accuracy from 51.61 to 59.52, and average response quality from 35.60 to 48.78 over Base.Naive CoT prompting degrades performance, while output-only SFT improves response quality without improving action prediction.Source: §5.2 · p. 7
  • R-CoT contributes most target-accuracy and Trait-fidelity gains, while R-DPO adds further action-accuracy and response-quality gains but reduces Trait fidelity by 1.36 points.R-CoT adds 7.85 target-accuracy and 5.54 Trait-fidelity points; R-DPO adds 5.91 action-accuracy and 5.41 response-quality points.Source: §5.2 · p. 7
  • Removing identity activation most reduces response quality (−4.60), whereas removing action decision most reduces target accuracy (−6.02) and action accuracy (−3.14).Platform-specific traits contribute more to target accuracy and Trait fidelity, while general traits contribute more to response quality.Source: §5.2 · p. 7

5.3 Group-Level Thread Simulation

  • PsyMemBehav 27B’s absolute reciprocity gap is 0.078 versus GPT-5.4’s 0.139, but its Top1 KOL gap is 0.019 versus GPT-5’s 0.013.Its Top5 PageRank gap is 0.072 versus GPT-5.4’s 0.067, so the strongest improvements concern reciprocity, stance composition, and aggregate fidelity rather than centralization.Source: §5.3 · p. 7

6 Conclusion and Limitations

PsyMemBehav combines user profiles, retrieved memories, target-first reasoning, reverse-CoT supervision, and rubric-guided DPO. The supplement details its profile and memory construction, reasoning traces, and evaluation rubric, alongside analyses of profile diversity and rubric activation.

6 Conclusion and Limitations

  • PsyMemBehav combines 41-dimensional user descriptors, BM25 memories, target-first reasoning, reverse-CoT supervision, and rubric-guided DPO to support evidence-grounded user modeling for observed-thread continuation.The conclusion notes limitations including missing exposure/no-action labels, LLM-inferred profiles, answer-conditioned traces, and aggregate rather than per-instance human validation.Source: §6 · p. 7

A.1 Trait-based User Representation

  • Profiles contain numerical trait scores and evidence-grounded summaries, with general psychological traits separated from platform-specific online tendencies.The general profile covers 22 dimensions across four categories, while the platform-specific profile covers 19 dimensions across four categories.Source: §A.1 · p. 10

A.2 Memory Retrieval

  • The method uses BM25 lexical retrieval, querying with the current post and visible comment context against historical behaviors observed strictly before t; its default retrieval size is k = 5, balancing relevant evidence against redundancy or noise.Source: §A.2 · p. 10

A.3 User Profile Distribution

  • The PCA profile visualization shows a dense population center alongside peripheral low-density regions representing users with distinctive profile combinations.The distribution is described as compact but noncollapsed, supporting profile-conditioned simulation of a population with individual heterogeneity.Source: §A.3 · pp. 10–11, Figure 3 · p. 11

B Reasoning Schema and Trace Construction

  • Identity activation selects context-relevant traits, memories, stance evidence, and behavioral constraints; action decision then selects a target before predicting the action and any required response.Visible text is planned only for Comment, not Like or Repost.Source: B Reasoning Schema and Trace… · p. 11
  • Reverse-CoT traces explain why an observed action is plausible given the profile, retrieved memory, and thread, providing behavior-explanation supervision rather than a free-form alternative action.Only traces accepted by the trace validator are used as training supervision.Source: B Reasoning Schema and Trace… · p. 11

C Evaluation Rubric and DPO Preference Construction

  • The ten-criterion rubric covers action fidelity, profile alignment, content quality, and format/reasoning validity, with action fidelity double-weighted in the score.The DPO judge receives the high-level action label but not the gold visible text.Source: C Evaluation Rubric and DPO Preference… · p. 11

C.1 Rubric Activation Analysis

  • Persona consistency, CoT plausibility, and context grounding are the most frequently activated rubric dimensions, while naturalness, specificity, and memory consistency are also highly active.The supplement describes additional analyses of platform-level actions, thread structures, retrieval size, judge robustness, and pairwise group-simulation preferences.Source: §C.1 · p. 11, Figure 4 · p. 12, D Additional Analyses · p. 12

D.1 Platform-level Simulation Statistics

The simulated action distribution favors comments, while thread statistics show a mix of mostly moderate discussions and a smaller number of deeper, more active ones. Generation quality peaks with five retrieved memories for both model sizes.

D.1 Platform-level Simulation Statistics

  • Comments account for the largest share of simulated actions, followed by reposts and likes.The figure presents the distribution of simulated agent actions.Source: §D.1 · p. 12, Figure 5 · p. 12
  • Most simulated discussions are moderate in scale, while a small subset has greater thread depth and activity.The reported thread statistics include comments per thread, maximum thread depth, and unique participants.Source: §D.1 · p. 12

D.2 Effect of Memory Retrieval Size

  • Both 9B and 27B models achieve their best average generation quality at k = 5 retrieved past comments.The authors attribute lower quality at smaller k to insufficient history and at larger k to redundant or noisy memories.Source: §D.2 · p. 12

D.3 Judge Robustness

PsyMemBehav 27B ranks highest under all four reported judge models, and the supplement describes a pairwise group-simulation preference test against GPT-5.4.

D.3 Judge Robustness

  • PsyMemBehav 27B receives the highest score under all four reported judges, addressing whether the ranking depends on one evaluator or judge-family preference.The table’s rows are judge models and its columns are evaluated models.Source: §D.3 · p. 13, Table 6 · p. 13
  • The pairwise test gives both models the same real thread seed and selects the better continuation according to fidelity to real discussion dynamics.The accompanying table compares GPT-5.4 with PsyMemBehav 27B.Source: §D.3 · p. 13, Table 7 · p. 13

D.4 Human–Gemini Evaluation Agreement

Human and Gemini overall scores are close, but their dimension-level judgments differ; aggregate similarity does not establish per-instance agreement.

D.4 Human–Gemini Evaluation Agreement

  • 54.40 and 55.46 are the human and Gemini overall means, respectively, a human-minus-Gemini difference of −1.06 points.Humans score semantic alignment, style fidelity, and naturalness slightly higher; Gemini scores intent, emotion, stance, and profile/memory consistency higher.Source: §D.4 · p. 13
  • Stance (4.20 points) and emotion (3.60 points) have the largest absolute gaps, while other dimensions differ by at most 3.73 points.Because gap signs vary, Gemini is not uniformly more lenient or stringent; the authors recommend human evaluation alongside LLM-judge results and cross-judge robustness.Source: §D.4 · p. 13

D.5 Dataset Scale and Longitudinal Coverage

PsyMemBehav-bench characterizes scale through real-user coverage, behavioral volume, and per-user history density, not platform-wide coverage. Its collection favors users with rich observable histories, limiting generalization beyond the sampled users and contexts.

D.5 Dataset Scale and Longitudinal Coverage

  • Real-user coverage counts distinct users represented, behavioral evidence combines total data and per-user history density, and prediction target distinguishes user-level prediction from aggregate simulation.Source: §D.5 · p. 13
  • PsyMemBehav-bench combines 5,862 real users, 5.57 million observed behaviors, and approximately 950 behaviors per user; related-work comparisons consider users, behavioral evidence, and prediction target.The paper describes this as a large-scale, longitudinal, user-centered dataset rather than platform-wide coverage.Source: §D.5 · p. 13, Table 8 · p. 14
  • Sampling favors observable users with rich histories and may overrepresent highly active accounts while underrepresenting low-activity, private, or newly created accounts.Platform access, observation period, and activity-based sampling also shape the population, so generalization beyond studied users and contexts requires caution.Source: §D.5 · p. 14

E Evaluation Metric Definitions

The appendix defines profile fidelity and group-level metrics, then illustrates how models differ in single replies and group continuations. Profile fidelity reflects agreement under a fixed annotator, not absolute psychometric validity.

E.1 Profile Fidelity: Normalized Edit Distance

  • Profile fidelity is 100 × (1 − NED) between discretized profile sequences; NED uses Levenshtein edit distance and sequence length, and scores measure consistency under a fixed annotator.The score is not absolute psychometric validity; general psychological traits and platform-specific behavioral traits are reported separately.Source: §E.1 · p. 14

E.2 Group-Level Simulation Metrics

  • Reply reciprocity measures mutual replies among interacting pairs; overly high reciprocity signals an unrealistic group-chat pattern rather than broadcast discussion.Source: §E.2 · p. 14
  • Stance deviation is the ℓ1 distance between simulated and real Pro, Anti, and Neutral distributions, with lower values indicating closer collective-opinion composition.Source: §E.2 · p. 14
  • Top1 KOL measures reply concentration on the most-addressed user, while Top5 PR sums PageRank for five top-ranked users to assess attention and influence concentration.Source: §E.2 · p. 14
  • MAE and MSE aggregate absolute and squared per-thread deviations from real statistics; lower values indicate greater group-level fidelity, and MSE penalizes large mismatches more.Source: §E.2 · p. 14

F Case Studies

  • Case studies pair a single-reply comparison with replay-style group simulation, showing that plausible individual replies may still fail to reproduce collective discussion dynamics.Source: F Case Studies · pp. 14–15

F.1 Single-reply Comparison

  • PsyMemBehav 27B remains more hedged and balancing than several strong general-purpose models, whose replies are more socially reactive or stance-bearing than the real user's cautious, low-commitment reply.Source: §F.1 · p. 15, Table 9 · p. 15

F.2 Group Simulation Comparison

In a representative thread replay, PsyMemBehav 27B most closely matches the real stance distribution among simulated systems. The prompt templates specify how reasoning is grounded and how candidate behaviors are judged.

F.2 Group Simulation Comparison

  • The replay starts from the same real thread seed and simulates multiple agents; stance classes are mutually exclusive, charged share equals Pro plus Anti, and lower stance distance is better.Entropy measures stance diversity, so closeness to the real entropy—not a higher or lower raw value—indicates greater fidelity.Source: §F.2 · p. 15
  • 0.266 stance distance makes PsyMemBehav 27B closest to the real continuation; it also has the highest neutral share (0.697), lowest charged share (0.303), and entropy (0.744) closest to the real value (0.523).The real continuation is predominantly neutral (0.830), with a charged share of 0.170; this is a representative case, complementary to aggregate results.Source: Table 10 · p. 15, §F.2 · p. 15

G Prompt Templates

  • The reference CoT validation prompt requires profile, context, dialogue, memories, or target grounding, limits reasoning to the current turn, prevents information leakage, and checks whether memory is needed.When memory is needed, the reasoning should identify relevant memory ids and explain their effect on current behavior.Source: G Prompt Templates · p. 16
  • Pairwise DPO judging prioritizes target grounding, then conditioned action, persona fit, memory use, stance, naturalness, specificity, format, and CoT plausibility.Source: G Prompt Templates · p. 19
  • The seven-dimension evaluation judges semantic and intent alignment, emotion, stance, style, persona/memory consistency, and naturalness rather than lexical overlap alone.Source: G Prompt Templates · p. 20
Loading 2610.14570v1…