Source-linked AI summary
AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society
Jinghua Piao, Yuwei Yan, Jun Zhang, Nian Li, Junbo Yan, Xiaochong Lan, Zhihong Lu, Zhiheng Zheng, Jing Yi Wang, Di Zhou, Chen Gao, Fengli Xu, Fang Zhang, Ke Rong, Jun Su, Yong Li
TL;DR
Existing approaches to understanding society face costly social experiments, simplified agent-based models, and limited platforms for realistic large-scale LLM-agent simulation. AgentSociety integrates LLM-driven agents, a realistic societal environment, and a large-scale simulation engine, with experiments demonstrating its potential as a testbed for social research and policy evaluation.
Problem
Understanding society requires explanation and prediction, but social experiments are costly and challenging, while existing simulations simplify individual behavior and environments.
Method
AgentSociety combines LLM-driven social agents, a realistic societal environment, and a simulation engine supporting large-scale interactions and agent messaging.
Results
Experiments demonstrate AgentSociety’s performance and potential as a testbed for social experiments, while successful replication of real-world social experiments supports its authenticity and practicality.
Takeaways & Limitations
AgentSociety provides a tool for analyzing, predicting, and intervening in complex social systems, with potential use by social scientists and policymakers.
Takeaways & Limitations
The economic simulator omits explicit goods and labor markets, limiting realism in modeling detailed market dynamics and real-world economic fluctuations.
Abstract
from arXiv · showhide
Understanding human behavior and society is a central focus in social sciences, with the rise of generative social science marking a significant paradigmatic shift. By leveraging bottom-up simulations, it replaces costly and logistically challenging traditional experiments with scalable, replicable, and systematic computational approaches for studying complex social dynamics. Recent advances in large language models (LLMs) have further transformed this research paradigm, enabling the creation of human-like generative social agents and realistic simulacra of society. In this paper, we propose AgentSociety, a large-scale social simulator that integrates LLM-driven agents, a realistic societal environment, and a powerful large-scale simulation engine. Based on the proposed simulator, we generate social lives for over 10k agents, simulating their 5 million interactions both among agents and between agents and their environment. Furthermore, we explore the potential of AgentSociety as a testbed for computational social experiments, focusing on five key social issues: polarization, the spread of inflammatory messages, the effects of universal basic income policies, the impact of external shocks such as hurricanes, and urban sustainability. These five issues serve as valuable cases for assessing AgentSociety's support for typical research methods -- such as surveys, interviews, and interventions -- as well as for investigating the patterns, causes, and underlying mechanisms of social issues. The alignment between AgentSociety's outcomes and real-world experimental results not only demonstrates its ability to capture human behaviors and their underlying mechanisms, but also underscores its potential as an important platform for social scientists and policymakers.
1 Introduction
Traditional social experiments can be costly and difficult to conduct, while existing simulated agents often struggle to reproduce human-like behavior and social experience. AgentSociety addresses these gaps by combining LLM-driven agents with a realistic environment and large-scale interactions.
- Social-science research seeks both explanations of causal mechanisms and predictions of future social behavior.Explanation focuses on why social outcomes occur, whereas prediction forecasts events or emergent behaviors from data.
- Traditional social experiments can be costly to implement and pose substantial practical and ethical challenges.
- Rule-, equation-, and machine-learning-based agents remain limited in generating human-like behavior, especially when communication is represented numerically rather than through natural language.
- Human-like minds and behaviors alone do not create a social being; social experience and activity through interactions with agents and the environment are also crucial.
- AgentSociety combines LLM-driven agents, a realistic societal environment, and large-scale interactions among agents and between agents and the environment.Its agents model emotions, needs, motivations, and cognition, which dynamically drive mobility, employment, consumption, and social interactions.
2 AgentSociety: Design and Overview
AgentSociety is organized as a large-scale social simulator that links human-like agents, a realistic environment, and a scalable simulation engine. Its evaluation framework examines agent design, environmental realism, engine scale, and support for social-science research.
- Design rationale: Society’s nonlinear interactions generate emergent and unpredictable collective behaviors that require bottom-up simulation of agents, environments, and their interactions.
- Evaluation framework: The evaluation framework assesses LLM-driven agents through their minds, social behaviors, and mind-behavior coupling.Mind-behavior coupling concerns how behaviors are generated from agents’ internal states.
- Agent behaviors: Complex behaviors such as socializing, economic activity, and movement require interaction with other agents or the environment, unlike simpler behaviors such as sleeping.
- Societal environment: Dataset-based environments use pre-existing data but lack dynamic, real-time feedback to agents’ behaviors.
- Simulation engine: The simulation engine’s scale is evaluated across four population levels: < 100, 100-1k, 1k-10k, and > 10k agents.The framework treats scale as a key factor in supporting research on complex social systems.
- Evaluation framework: Figure 1 presents the evaluation framework for LLM-driven social simulators.
- System overview: AgentSociety integrates LLM-driven social agents, a realistic societal environment, and a simulation engine supporting large-scale interactions.The platform is designed to support advanced social-science research and social experiments.
3 LLM-driven Social Generative Agents
AgentSociety designs LLM-driven social agents with interconnected mental processes, social behaviors, and memory, situated within a broader social-simulation framework. The framework targets comprehensive, adaptive behavior by linking internal states, past experiences, and interactions with environments and other agents.
- Overview: AgentSociety addresses the difficulty of creating LLM-driven agents that simulate comprehensive social beings across interdependent behaviors.The paper identifies gaps in connecting minds to behaviors and modeling dependencies among mobility, employment, consumption, and social interaction.
- Overview: The proposed agents combine stable profiles and dynamic statuses with emotions, needs, and cognition.Emotions respond to stimuli, needs motivate actions, and cognition represents understanding of the external world.
- Overview: Three explicitly modeled social behaviors—mobility, social interactions, and employment & consumption—bridge agents’ internal minds and external environments.Other simple behaviors, such as sleeping, are handled directly by LLMs.
- Emotion, Needs, and Cognition: Emotion updates from profiles, statuses, and interactions, then influences actions, motivations, and cognitive processes.The model represents six core emotions with intensities rated from 0 to 10.
- Emotion, Needs, and Cognition: Needs provide sustained motivational mechanisms, while cognition supports reasoning, planning, decision-making, and attitude updates.The needs module informs action plans, and cognitive updates are linked to emotional and attitudinal changes.
- Workflow and Social Behaviors: Memory links psychological states to behavior by integrating current emotions, cognition, and past experiences for continuous adaptation.The simulator’s social module also represents relationships and interactions, supporting study of information and influence spread.
4 Real-world Societal Environment
AgentSociety models objective societal conditions through urban, social, and economic spaces, supporting agent mobility, interaction, and economic activity. This environment provides feedback from real-world constraints while enabling social propagation and policy-oriented analysis.
- The societal environment comprises urban, social, and economic spaces supporting mobility, social interaction, and economic behaviors.It is designed as a virtual mapping of objective aspects of the world so agents can focus on subjective behavioral logic.
- Urban Space: Urban space combines road networks, AOIs, POIs, and multimodal transportation to provide positional feedback and travel-time and monetary costs.Supported modes include driving, walking, public transit, and taxi services.
- Urban Space: Static infrastructure, dynamic mobility, and geospatial data are harmonized into a high-fidelity decision-making sandbox for agents.The environment uses real-world or real-world-aligned data and interfaces for agent interaction.
- Social Space: The social space models relationships and connection strengths, enabling agents to select interaction targets and supporting offline and online interactions.Its message-based design also provides intervention capabilities over social propagation.
- Economic Space: The economic space represents firms, agents, government, and banks through income generation, consumption, savings, taxation, and policy adjustments.It supports analysis of links between micro-level decisions, market behavior, and policy interventions.
- Economic Space: The economic simulator omits explicit goods-market and labor-market dynamics, limiting realism and motivating future refinement.The model simplifies price adjustment and does not represent unemployment or worker–firm negotiation.
5 Large-scale Social Simulation Engine
The large-scale engine treats agents as independent units that communicate through messaging while grouping agents into processes for scalable asynchronous execution. Its architecture combines distributed execution, LLM services, messaging, storage, metrics, and experiment tools, including surveys and interviews.
- Execution Architecture: Independent agents exchange information through messaging rather than fixed execution dependencies or orders.Agent groups balance communication costs with parallel acceleration as simulation scale increases.
- System Architecture: The architecture combines shared services, experiment-specific simulation tasks, and an optional GUI.Shared services include LLM APIs, MQTT messaging, databases, and metric recording.
- System Architecture: The engine uses Ray-created agent groups and subprocess-managed environment simulators to execute experiments at scale.Open-source software supplies distributed computing, LLM access, message transmission, data storage, and metric management.
- Execution Architecture: Agent groups execute multiple agents within single processes, addressing TCP port exhaustion while retaining distributed parallelism.The design combines asynchronous and parallel execution to prevent execution failures caused by port limits.
- Agent Messaging System: MQTT provides publish/subscribe messaging that connects large numbers of agents and supports reliable transmission and GUI-based monitoring.The system is intended to deliver messages to hundreds of thousands of agents by ID.
- Toolbox for Social Experiments: The toolbox supports direct interviews and structured surveys by distributing user questions to agents through MQTT.Survey responses follow predefined formats and are compiled for analysis, while interviews use agents’ internal states and environments.
6 Performance Evaluation
Experiments evaluate environmental concurrency, messaging throughput, and end-to-end simulator scalability. Results indicate minimal environment degradation and benefits from parallelization, while LLM API calls remain the principal scaling constraint.
- Evaluation Design: The evaluation addresses societal-environment performance, MQTT messaging versus alternatives, and large-scale execution with LLM-driven agents.Experiments were conducted under controlled cloud-server settings with attention to LLM API rate-limit interference.
- Societal Environment: 1,000 to 1,000,000 individuals were used to test environment load, with queries modeled on request distributions in agent simulations.Performance was measured using time per simulation step and queries per second.
- Societal Environment: Performance degradation remained minimal as individual counts and query rates increased, indicating support for extensive agent–environment interaction.The environment was reported to handle massive interactions without significant degradation.
- Agent Messaging System: MQTT, Redis Pub/Sub, and RabbitMQ met the 20,000 msg/s requirement under the tested extreme conditions, while Kafka failed to initialize 100,000 agents within five minutes.MQTT was selected despite approximately half Redis Pub/Sub’s throughput because its GUI tools support monitoring, debugging, and testing.
- Social Simulator: With 10k agents, token-usage distributions remain stable across group configurations, whereas LLM API call time is more sensitive to parallelization.Environment responsiveness also varies with the number of groups.
- Social Simulator: Total time per round decreases as group count increases, while environment time remains in the millisecond range and LLM time remains the primary bottleneck.The evaluated configurations used 8, 16, and 32 groups, with API-call variability attributed to server-side load.
- Limitations: For simulations exceeding 10^4 agents, private LLM inference may improve stability but requires substantial GPU and model-configuration costs.The study identifies LLM API performance as the main constraint on large-scale execution efficiency.
7 Exemplary Social Experiments
AgentSociety supports computational experiments across polarization, inflammatory-message propagation, universal basic income, hurricanes, and urban sustainability. The experiments combine interventions, surveys, interviews, and behavioral observations to examine social patterns and responses.
- 7.2 Polarization: 52% of agents became more polarized under homophilic interactions, compared with 39% in the control group.In heterogeneous interactions, 89% became more moderate and 11% adopted opposing viewpoints.
- 7.3 Spread of Inflammatory Messages: Inflammatory messages achieved substantially higher information reach and emotional responses than regular content.Node-level intervention was more effective than edge-level intervention for containing spread and moderating emotional intensity.
- 7.4 Universal Basic Income: The UBI policy increased consumption levels and reduced depression levels over the following 24 simulation steps.Depression was assessed using surveys based on the CES-D, and the pattern resembled reported effects of Texas’ UBI policy.
- 7.5 External Shocks of Hurricane: Hurricane arrival reduced average activity levels across CBGs from 70%-90% to approximately 30%, followed by gradual recovery.The simulated mobility response broadly matched real visit trends, though deviations appeared during the hurricane’s peak.
- 7.6 Urban Sustainability: All six eco-normative systems increased pro-environmental normative alignment and reduced mobility-related CO2 emissions relative to baseline.Teams producing stronger normative alignment also tended to induce larger shifts toward walking, cycling, and public transit.
8 Related Works
Related work spans LLM-driven agents and social simulation, including agent-based models of cognition, interaction, collective behavior, and system dynamics. Existing approaches remain limited by simplified environments, narrow problem focus, computational inefficiency, and inaccurate user behaviors.
- 8.1 LLM-driven Agents: LLM-driven agents use language models as “brains” with memory management, interactive interfaces, and expanded action spaces.Research includes decision-making assistants and agents intended to reproduce or explain human responses.
- 8.1 LLM-driven Agents: AgentSociety addresses the lack of a platform that can deploy LLM agents for large-scale real-world simulation.The paper positions this gap against rapid progress in using LLMs to simulate real humans.
- 8.2 Social Simulation: Social simulation includes macrosimulation of macro-level variables and microsimulation of emergent phenomena through granular agent behaviors.Microsimulation, often termed agent-based simulation, has become the more widely adopted approach.
- 8.2 Social Simulation: Early agent-based models such as cellular automata, Game of Life, and Sugarscape expanded the study of interactions, collective behavior, and system dynamics.Applications include cooperation, information propagation, crowd dynamics, macroeconomic systems, and market dynamics.
- 8.2 Social Simulation: Existing LLM-driven social simulations often use isolated tasks and simplified environments, while larger-scale efforts face computational inefficiency and inaccurate user behaviors.AgentSociety combines LLM-driven generative agents, a realistic societal environment, and a large-scale simulation engine to address these limitations.
9 Discussion
AgentSociety is presented as a fourth computational-social-science paradigm built around highly realistic agents, large-scale interaction, and intervention. The discussion extends its relevance from social experimentation and policy evaluation to dynamic risk management and future societal planning.
- 9.1 Three Levels of Social Simulator: The simulator combines high-fidelity individual behavior generation with short- and long-term simulations of individual behavior and group evolution.Experiments also assess evolution under different intervention conditions.
- 9 Discussion: AgentSociety advances agent-based modeling toward a fourth paradigm centered on highly realistic human-like agents for analysis, prediction, and high-precision bottom-up simulation.It supports arbitrary selection, intervention, and control of experimental subjects.
- 9.3 Social Simulator for Social Governance: The simulator addresses limitations of historical-data-based governance by supporting continuously updated, dynamic, cross-domain, and holistic risk simulation.Its stated applications include tracking evolving risks and modeling cascading effects across social and economic domains.
- 9.3 Social Simulator for Social Governance: AgentSociety provides a platform for parallel counterfactual experiments that compare alternative policy outcomes and explore multidimensional intervention combinations.This exploration can identify superior strategies and suggest composite policy solutions.
- 9.3 Future Societal Applications: Future applications include digital human society infrastructure and a sandbox for comparing alternative urban architectures and coupled energy, transportation, and housing systems.The discussion gives vertical megacities and distributed satellite cities as example alternatives.
- 9.3.3 Social simulator for the future human-AI society: The simulator’s current focus on LLM agents leaves future human-AI societies, including differing AI adoption rates and AI participation in governance, as open application areas.The paper identifies issues such as unemployment, AI legislators, and AI rights as future concerns.
10 Conclusion
AgentSociety integrates LLM-driven agents, a realistic societal environment, and large-scale interactions to simulate human behavior and societal dynamics. The paper presents it as an experimental testbed and policy-evaluation platform whose real-world experiment replication supports its authenticity and practicality.
- 10 Conclusion: AgentSociety integrates LLM-driven agents, a realistic societal environment, and large-scale interactions for authentic simulations of human behavior and societal dynamics.It is positioned as advancing generative social science and computational social science 2.0.
- 10 Conclusion: Successful replication of real-world social experiments supports AgentSociety’s authenticity and practicality as a testbed for social scientists and a policy-evaluation platform for policymakers.The simulator is also described as a low-cost, low-risk environment for testing and refining macro-level policy interventions.