PaperlayerView exact revision on arXiv

Source-linked AI summary

Socio-Foundation: A Model for Generalizable Individual Behavior Simulation via Hierarchical Capability Distillation

Liang Wang, Wenxuan Xie, Xinyi Mou, Yixin Luo, Zhongyu Wei

arXiv:2610.08967v1cs.AI

TL;DR

General-purpose language models can flatten distinctive personas, while task-specific tuning may fragment skills and transfer poorly. Socio-Foundation organizes simulation capabilities and integrates specialists through hierarchical distillation; it improves over Qwen3-8B across five dimensions and approaches frontier models.

  • Problem

    General-purpose models may flatten distinctive personas, while task-driven post-training has limited transfer to unseen settings and pooled corpora can suffer objective interference.

    Source: §1 · p. 2

  • Method

    Socio-Foundation organizes simulation into five FONTS capabilities and uses a three-stage pipeline to train task experts, consolidate capability experts, and integrate them into a unified simulator.

    Source: §1 · p. 2, §4 · p. 4

  • Results

    61.7 average (∼11-point gain): Socio-Foundation outperforms Qwen3-8B across all five dimensions, approaches GLM-5.2, and transfers zero-shot to three OOD benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that hierarchical expert distillation consolidates task-specific expertise into a unified individual simulator that improves over its Qwen3-8B base across five capabilities.

    Source: §6 · p. 10

    The stated learning objective is to capture individual behavioral distributions across roles and interaction contexts while maintaining consistency, contextual sensitivity, and diversity.

    Source: §3 · p. 3

Abstract

from arXiv · show

Simulating individual behavior requires large language models (LLMs) to preserve persona traits while adapting to dynamic social contexts. However, general-purpose LLMs often flatten distinct personas, while task-specific tuning suffers from fragmentation and generalization. To overcome these challenges, we organize individual simulation into the \textbf{FONTS Taxonomy}, comprising five complementary capability dimensions: \emph{persona fidelity} (\textbf{F}), \emph{outcome realization} (\textbf{O}), \emph{behavioral naturalness} (\textbf{N}), \emph{trajectory coherence} (\textbf{T}), and \emph{social grounding} (\textbf{S}). Grounded in this taxonomy, we curate a standardized training corpus library of approximately 10 million instances across 14 representative datasets and present \textbf{Socio-Foundation}. Socio-Foundation decouples specialization from integration via a three-stage pipeline: learning task experts via DAPO, consolidating them into capability experts via off-policy distillation, and unifying them via multi-teacher on-policy distillation (MOPD). We also establish \textbf{IndiEval}, consolidating 29 metrics across the FONTS dimensions. Experiments show that Socio-Foundation outperforms its \textit{Qwen3-8B} base by 11.0 points and approaches frontier models such as \textit{GLM-5.2}, with ablations and out-of-distribution evaluations further demonstrating the effectiveness and generalization of our model.

1 Introduction

The introduction frames individual simulation as balancing persona fidelity with adaptation across social contexts, while avoiding homogenization and fragmented task skills. It presents FONTS, a hierarchical expert-distillation pipeline, and IndiEval as the foundation for a unified simulator.

Motivation

  • General-purpose models can flatten distinctive personas, while prompting leaves behavioral priors unchanged and task-driven tuning transfers poorly to unseen settings.Pooling heterogeneous corpora can also trigger objective interference that corrupts task-specific behaviors.Source: §1 · p. 2

Contributions

  • FONTS organizes simulation into five capabilities, and the authors curate a standardized corpus library of approximately 10 million instances from 14 representative datasets.The dimensions are persona fidelity, outcome realization, behavioral naturalness, trajectory coherence, and social grounding.Source: §1 · p. 2
  • Socio-Foundation separates specialization from integration by training task experts with DAPO, distilling capability experts, then unifying them through MOPD.The pipeline is designed to circumvent cross-task interference.Source: §1 · p. 2

Results

  • 61.7 average (∼11-point gain): Socio-Foundation outperforms Qwen3-8B across all five dimensions, approaches GLM-5.2, and transfers zero-shot to three OOD benchmarks.The introduction also reports that MOPD outperforms tested weight-merging and off-policy alternatives, and that removing any capability expert lowers the overall average.Source: §1 · p. 2

Contributions

  • IndiEval consolidates 29 task-specific metrics under the five capabilities while preserving their original evaluation protocols.The experiments report consistent gains over Qwen3-8B across all five dimensions.Source: §1 · p. 2

2 Related works

Related work describes individual simulation as conditioning behavior on person-specific information, then refining those characteristics through prompting, retrieval, adaptation, or post-training. Recent foundation-model approaches learn reusable behaviors, but organizing heterogeneous experts around shared capabilities remains underexplored.

Individual simulation

  • Individual simulation conditions behavior on person-specific signals such as personas, interviews, memories, and interaction histories, using prompting, retrieval, adaptation, or reward-based post-training.A remaining challenge is preserving persona fidelity while transferring across social contexts and behavioral requirements.Source: §2 · pp. 2–3

Foundation models for individual simulation

  • Role-playing, user-simulation, cognition, and behavioral foundation models learn reusable behavior across roles and contexts from dialogue, experiments, social-science data, and interaction corpora.Common consolidation approaches include model merging and trajectory-level SFT distillation; recent work also uses on-policy and multi-teacher distillation.Source: §2 · p. 3
  • How to organize heterogeneous experts around the cross-task capabilities required for individual simulation remains underexplored.This is the gap identified amid existing merging, trajectory-level distillation, and on-policy integration approaches.Source: §2 · p. 3

3 Preliminaries

The preliminaries formulate individual simulation as conditional generation in multi-party interactions and describe the optimization and distillation tools used in the paper. They define an objective spanning behavioral consistency, contextual sensitivity, and diversity.

Problem setup

  • Individual simulation is formulated as conditional generation over multi-party interaction episodes, where each response depends on persona, interaction history, and social context.The simulator autoregressively approximates the target behavior distribution.Source: §3 · p. 3
  • The stated learning objective captures an individual's behavioral distribution across roles and interaction contexts while maintaining consistency, contextual sensitivity, and behavioral diversity.Source: §3 · p. 3

Group-based policy optimization

  • DAPO extends GRPO with asymmetric clipping, token-level loss normalization, dynamic sampling, KL-penalty removal, and reward shaping for overlong responses.It focuses updates on response groups with informative reward differences and aims to reduce premature loss of diversity in varied social scenarios.Source: §3 · p. 3

Policy distillation

  • Off-policy distillation evaluates teachers on fixed-dataset prefixes, whereas on-policy distillation evaluates them on histories sampled from the student's own rollout distribution.MOPD extends on-policy distillation to multiple capability-specific teachers; weighted forward-KL encourages coverage of teacher-supported behavioral modes.Source: §3 · p. 3

4 Socio-Foundation: Hierarchical Expert Distillation

Socio-Foundation organizes individual-simulation capabilities into FONTS and develops a unified simulator through a three-stage post-training pipeline. The stages train task experts, consolidate them into capability experts, and integrate those experts into a student.

4.1 Five Dimension Capability Organization

  • FONTS consolidates individual-simulation requirements into five complementary capability dimensions that guide expert consolidation and provide a shared evaluation structure.Tasks may contribute to multiple dimensions when their behavioral requirements and evaluation metrics span different aspects of simulation.Source: §4 · p. 4, §4.1 · pp. 4–5

4 Socio-Foundation: Hierarchical Expert Distillation

  • The pipeline trains task experts with DAPO, consolidates experts sharing a capability into teachers through off-policy distillation, then integrates those teachers into a student through multi-teacher on-policy distillation.The final student is trained on histories sampled from its own rollouts, and simulation uses only the resulting policy.Source: §4.2 · p. 5, §4.3 · p. 5, §4.4 · p. 5

4.2 Task-Expert policy optimization

  • Each task expert starts from a common checkpoint and is optimized for its task using DAPO on task-specific rewards; dynamic sampling filters zero-variance groups.The objective increases the likelihood of relatively high-reward behavior while limiting policy updates.Source: §4.2 · p. 5

4.3 Capability-Expert consolidation through off-policy distillation

  • Task experts associated with the same capability are consolidated into a shared teacher, with fixed dataset prefixes transferring each source expert’s predictive distribution to the capability student.Overlapping capability membership lets a task supervise multiple teachers without decomposing its reward into capability-specific scores.Source: §4.3 · p. 5

4.4 Unified simulation through on-policy distillation

  • Multi-teacher on-policy distillation trains the unified student against capability teachers on histories sampled from the student’s own rollouts.Teachers score continuations from the same student-generated prefix, and only the resulting policy is needed for simulation.Source: §4.4 · p. 5

4.5 Implementation Details

  • The training library selects 14 datasets and approximately 10 million instances from a collection of more than 40 datasets, using scale, quality, and capability coverage criteria.All three stages share a Qwen3-8B backbone; task experts use rank-32 LoRA, and the final student is trained against frozen capability teachers.Source: §4.5 · pp. 5–6

5 Experiments

The experiments evaluate Socio-Foundation across five individual-simulation capabilities using IndiEval, then compare its results, pipeline alternatives, and out-of-distribution performance. The results report gains over Qwen3-8B, benefits from the hierarchical pipeline, and stronger performance across capability dimensions and additional benchmarks.

5.2 Main results

  • Socio-Foundation gains 11.0 points over Qwen3-8B on IndiEval’s five-dimension average, with gains on 24 of 29 metrics.The comparison is reported in Table 1.Source: §5.2 · pp. 6, 8, Table 1 · p. 7
  • Socio-Foundation substantially outperforms larger Qwen3 models and DeepSeek-V4-Flash, slightly outperforms Kimi-K2.6, and remains close to GLM-5.2 and GPT-5.5.General-purpose models are consistently weak on behavioral naturalness, where Socio-Foundation performs better on 4 of 7 metrics.Source: §5.2 · p. 8
  • Against similarly sized specialized simulators, Socio-Foundation has the best or second-best result on 86% of metrics and the highest overall average.The passage reports that specialized models often generalize unevenly beyond their training tasks.Source: §5.2 · p. 8

5.3 Pipeline ablations

  • The full hierarchical pipeline scores 61.7 Avg.; capability-expert off-policy distillation scores 59.3, and capability-expert averaging scores 56.8.Capability-expert averaging also exceeds task-expert averaging, 56.8 versus 54.5; direct task-expert SFT and OPD score 59.5 and 54.2, while pooled-data SFT and reverse-KL OPD score 43.0 and 50.9.Source: §5.3 · p. 9, Table 2
  • DAPO task experts generally outperform their SFT counterparts across tasks based on dataset-level scores aggregated over normalized metrics.The comparison motivates using DAPO for task-expert training.Source: §5.3 · p. 9, Figure 7 · p. 9

5.4 Mechanism and generalization analyses

  • On the three additional OOD benchmarks, Socio-Foundation scores 57.00 on Mistakes, 62.00 on HiToM, and 69.31 on τ-USI, outperforming both baselines on all three.These benchmarks are outside IndiEval and are excluded from Table 1 scores and Avg.Source: §5.4 · pp. 9–10, Figure 6 · p. 9
  • Socio-Foundation exceeds five 8B specialized baselines and Qwen3-8B across all five dimensions, with its largest gains over the base model in trajectory coherence and behavioral naturalness.Figure 5 presents the five-dimensional scores aggregated from Table 1.Source: Figure 5 · p. 9, §5.4 · p. 10
  • Removing any capability expert lowers overall Avg. and its corresponding dimension score; removing the naturalness expert causes the largest decline.Removing F, N, T, or S raises outcome realization while lowering overall Avg.Source: §5.4 · p. 10, Table 3 · p. 10

6 Conclusion

Socio-Foundation organizes individual simulation around FONTS and integrates task expertise through hierarchical distillation; the paper reports gains across capabilities and on out-of-distribution benchmarks. Its survey annotations classify dataset–task–role configurations by behavioral requirements, not by mutually exclusive task categories.

6 Conclusion

  • Socio-Foundation improves over Qwen3-8B across all five capabilities, performs strongly among same-scale simulators, and approaches frontier models.Ablations favor on-policy integration over direct expert merging and the tested off-policy alternative; results on three out-of-distribution benchmarks support generalization beyond the main suite.Source: §6 · p. 10

A Dataset Survey, Capability Annotations, and Clustering

  • The survey inventory records 44 dataset/task entries and 10 supplementary benchmark or protocol entries, which are not 54 independent dataset families.The survey inventory is distinct from the selected training corpus.Source: A Dataset Survey, Capability… · p. 14

A.1 Annotation unit and organization

  • Each annotation unit is a dataset–task–role configuration, and requirements are grouped into eight clusters mapped to the five FONTS dimensions.Assignments represent capability demands, not mutually exclusive task categories or independent scores for every label.Source: §A.1 · p. 14, Table 4 · p. 14

A.2 Overview of dataset coverage

The survey organizes dataset/task entries by capability requirements across five FONTS dimensions and eight semantic clusters. Entries may appear in multiple clusters, and their annotations follow the stated task and role interpretations.

A.2 Overview of dataset coverage

  • BehaviorChain is assigned to F for persona-conditioned node behavior and T for progression; UserLM-S3 and UserLM-LiC are separated by their intent-fidelity versus outcome-focused interpretations.The former UserLM-S3 interpretation includes natural user behavior, while UserLM-LiC concerns intent coverage and task outcomes.Source: §A.2 · p. 15
  • HiToM, τ-USI, and Mistakes are separate out-of-distribution benchmarks, and their presence in the survey does not imply they were used for training.Source: §A.2 · p. 15
  • Figure 8 groups entries into eight clusters within five colored FONTS regions, with each chip identifying a dataset/task entry that may recur across clusters.The diagram summarizes Table 5 annotations; Table 6 separately records dimensions measured by selected main-table metrics.Source: Figure 8 · p. 15

A.3 Detailed inventory and annotation results

The appendix documents how dataset/task entries are interpreted and mapped to capability requirements, how selected metrics align with those assignments, and how collected data are cleaned.

A.3 Detailed inventory and annotation results

  • Table 5 separates dataset/task names, task interpretations, requirements, clusters, and FONTS dimensions; daggered entries mark local identifiers or derivatives with unestablished conversions.Its supplementary entries complete coverage of the paper’s benchmarks and protocols.Source: §A.3 · p. 16, Table 5 · p. 16

A.4 Concordance with the evaluation dimensions

  • The selected metric assignments differ from broader dataset-level requirements: SOTOPIA believability maps to F, UserLM-S3 adherence and termination F1 to N, and UserLM-LiC metrics to O.AgentSense and UserLM-LiC are OOD tasks in the main table; HiToM, τ-USI, and Mistakes are in a separate OOD analysis.Source: §A.4 · p. 17, Table 6 · p. 18

B.1 Cleaning Method

  • Cleaning repairs formatting and prompt–target consistency, while filtering addresses unsuitable instances such as role confusion, ambiguous answers, duplicates, and excessive length.Reported rules also address construction artifacts, profile noise, empty targets, and unexpanded questionnaire templates.Source: §B.1 · p. 18, §B.2 · p. 18

B.2 Cleaning Procedure

  • Repairs include correcting prompt–target mismatches and template inconsistencies, including aligning targets with requested output formats.Examples include removing chain-of-thought text when only an answer is requested and reconciling numeric with alphabetic answer labels.Source: §B.2 · p. 18
  • Filtering removes instances with role confusion, conflicting or ambiguous answers, duplicates, or lengths above 32,768 tokens.Source: §B.2 · pp. 18–19

B.3 Cleaning Results

The cleaning results report retained, removed, and transformed counts across 45 sources, distinguishing repaired instances from filtered-out rows.

B.3 Cleaning Results

  • 19,678,715 input rows yielded 19,018,645 retained and 660,070 removed; 52,615 transformations were reported separately.Transformed counts cover repaired instances, while removed counts cover filtering; transformations are not subtracted again from retained rows.Source: Table 7 · p. 19, §B.3 · p. 19

C.1 Task-level reinforcement learning

Task experts are trained independently with DAPO, then distilled into capability experts and a unified student. The appendix selects DAPO based on its final five-task average and specifies training and sampling configurations.

C.1 Task-level reinforcement learning

  • Each task expert uses an independently optimized LoRA adapter on the shared Qwen3-8B backbone, with a 200-step training budget.The recipe uses rank 32, scaling 64, and a learning rate of 5 × 10−6.Source: §C.1 · p. 20

C.2 Capability consolidation and final integration

  • Capability experts are distilled from filtered task-expert demonstrations, and the final student learns on its own generated prefixes against frozen capability teachers.The reference hierarchy uses a sampled forward-KL surrogate for the final distillation loss.Source: §C.2 · p. 20

C.3 Choice of DAPO for task-expert training

  • DAPO achieves the highest final five-task average among the compared RL methods, with particularly strong late-stage performance on FANToM and BehaviorChain.The authors therefore adopt DAPO for task-expert training.Source: §C.3 · p. 20, Figure 9 · p. 20

C.4 Data sampling

  • Task-expert DAPO samples instances from the training corpus; subsequent off-policy distillation uses 2,048 cases per task or all cases when fewer are available.Table 8 reports per-dataset DAPO sample counts, but the supplied passage does not include the values.Source: §C.4 · pp. 20–21, Table 8 · p. 21

D.1 Benchmark inventory and capability mapping

The appendix documents IndiEval’s benchmark inventory, FONTS metric mapping, scoring procedures, sampling, and representative evaluation prompts. It also describes benchmark selection and a repeated-run protocol for characterizing evaluation variability.

D.1 Benchmark inventory and capability mapping

  • IndiEval covers 13 benchmark entries with 29 selected metrics across F/O/N/T/S, while AgentSense and UserLM-LiC form its OOD component relative to training.HiToM, τ-USI, and Mistakes provide additional OOD evaluations outside the IndiEval average.Source: §D.1 · p. 21

D.2 Main-table metric definitions and FONTS mapping

  • The metric inventory assigns selected metrics to F/O/N/T/S and orients displayed scores so that higher is better.Examples include termination F1 and human-likeness for naturalness, story consistency for trajectory coherence, and private-information accuracy for social grounding.Source: §D.2 · pp. 21–22, Table 9 · p. 21

D.3 Score normalization and aggregation

  • Scores are direction-aligned using fixed protocol bounds, then aggregated by dataset, dimension, and an equally weighted five-dimension average.Metrics from different dimensions are not pooled, even when they come from the same dataset.Source: §D.3 · p. 22

D.4 Evaluation sampling and case counts

  • The 13 IndiEval entries contain 6,297 cases; adding 165 τ-USI cases gives 6,462 cases, with τ-USI excluded from the IndiEval average.In-domain train and test sets are kept separate to prevent overlap and train–test leakage.Source: §D.4 · p. 23, Table 10 · p. 23

D.5 Representative prompts in IndiEval

SYSTEM — RUBRIC EXCERPT

P5. MirrorBench — GTEval judge prompt

E.1 Benchmark selection and evaluation pathways

  • The variability study selects LifeChoices, HUMANUAL, and MirrorBench to represent automatic scoring, LLM-based scoring, and interactive evaluation pathways.Five evaluations use a fixed checkpoint and test sets; reported dispersion is the standard deviation across run-level scores, not a standard error or confidence interval.Source: §E.1 · p. 28, §E.2 · p. 28, Table 11 · p. 28

E.3 Evaluation Variability and Implications for Statistical Significance

Five repeated evaluations of a fixed checkpoint show limited but metric-dependent variation across LifeChoices, HUMANUAL, and MirrorBench. The authors use the results to contextualize small score differences, not to establish model-comparison significance.

E.3 Evaluation Variability and Implications for Statistical Significance

  • Standard deviations across five evaluations are below 0.6 percentage points on the 0–100 scale for LifeChoices accuracy, HUMANUAL response/state alignment, and MirrorBench GTEval.The repeated-run study covers a fixed checkpoint and test sets.Source: §E.3 · p. 29
  • Variability is metric-dependent: MirrorBench GTEval has a 0.396% normalized standard deviation, while PI and RNR have 1.690% and 1.723%.The fixed-human-reference RNR control varies across runs, but does not quantify how much of the main RNR variance comes from the judge.Source: §E.3 · p. 29
  • Repeated evaluations of one checkpoint do not establish whether model improvements are statistically significant; that requires matched comparisons accounting for repeated measurements and case dependencies.The study contextualizes small score differences rather than assigning significance from a single checkpoint’s repeated evaluations.Source: §E.3 · p. 29
Loading 2610.08967v1…