Source-linked AI summary
GOD: Govern, Observe, and Direct - A Real-Time Control Room for Agent Societies
Yige Luo, Ran Guan
TL;DR
Generative-agent runs are difficult to inspect, intervene in, and package when controls, logs, replays, and configuration are separated. GOD provides a local-first control room and artifact workflow that links commands to replay state. Across 14 intervention runs, 78 of 84 target-agent checks recorded the commanded destination.
Problem
Generative-agent runs expose configuration, live processes, logs, and replays through separate interfaces, making command-to-state inspection and run packaging awkward.
Method
GOD combines a browser control room with Ask and Intervene commands, replay inspection, shared command records, and portable experiment, map, and agent packs.
Results
78 of 84 target-agent checks recorded the commanded destination across 14 intervention runs.
Takeaways & Limitations
GOD connects local scenario setup and live operator commands to replay inspection and portable packs for generative-agent societies.
Takeaways & Limitations
The evaluation used one model configuration with synthetic profiles, and results may differ with other models, scenarios, and users.
Abstract
from arXiv · showhide
Generative-agent systems are easier to start than to inspect. A run can contain many agents, locations, messages, commands, and model calls, yet the operator often gets either a finished replay or raw logs. That makes it hard to ask why an agent moved, test a small intervention, or package a run for another researcher. GOD is a local-first control room for agent societies. From the same browser workflow, an operator can issue targeted questions or interventions and inspect the resulting replay state. The system combines a setup wizard, Agent Studio, Map Studio, a spatial replay interface, Ask and Intervene commands, and portable experiment, map, and agent packs. Its technical contribution is the command and artifact loop: live controls and replay evidence share the same operator command model, while package contracts separate scenario, map, and profile data from local runtime state. The public release includes hosted Smallville-style and PKU replays, the open-source repository, and downloadable packs. We evaluate this path on 15 completed run slots. Across the 14 intervention runs, 78 of 84 target-agent checks recorded the commanded destination, and 169 of 182 state answers matched a saved location or action string.
1 Introduction
GOD addresses the difficulty of operating and inspecting generative-agent runs by linking browser controls, replay evidence, and portable artifacts. Its central design combines an operator command loop, a shared command record, and separate artifact contracts.
- Operator workflow: GOD connects scenario authoring, world inspection, targeted commands, run advancement, and preserved replay state in one operator workflow.Agents remain visible on the map, and ASK and INTERVENE actions are recorded beside their corresponding replay steps.
- Audience and artifacts: The system targets researchers, educators, designers, and builders who need credential-free inspection and locally editable runs.Hosted replays record what happened, while downloadable packs preserve scenario, map, and agent-profile data for another run.
- Operator workflow: The browser control room combines mapped state, temporal controls, targeted questions, and next-step interventions.
- Command and evidence model: A shared command schema keeps live controls and replay views attached to the state transitions they precede.
- Portable artifacts: Separate artifact contracts keep runnable experiments, maps, and agent packs apart from replay history, credentials, and machine-local runtime state.
2 Related Work
GOD builds on systems for agent conversation, spatial simulation, and social evaluation while focusing on operator control and artifact handling. Its comparisons are capability-level rather than agent-quality benchmarks.
- Comparison scope: The comparison concerns feature-level capabilities and portable artifacts, not agent quality or benchmark scores.
- Comparison scope: Table 1 compares publicly described system affordances using R, P, and NR labels, with NR meaning the cited source did not report the capability.
- Positioning: GOD builds on multi-agent conversation, simulation, and evaluation systems, then focuses on a spatial operator and artifact workflow.
- Developer tools: AutoGen Studio and AgentScope provide interfaces for workflow construction, debugging, evaluation, monitoring, or deployment, whereas GOD records live commands beside spatial replay state.
- Distinct contribution: GOD exports scenario data separately and contributes integration, authoring views, and import/export contracts rather than a new policy, planner, memory architecture, or base simulator.
3 System Overview
GOD combines a React/Vite control room and FastAPI backend with local execution, live and replay controls, and portable package contracts. The system separates static replay inspection from editable local runs.
- Architecture: GOD uses a React/Vite control room and FastAPI backend around AgentSociety and JiuwenClaw to connect commands, execution, replay storage, and pack exchange.
- Operator control room: The control room keeps world state, temporal controls, run status, resident information, and command input visible together for live and replayed runs.
- Artifact contracts: Experiments, maps, agents, replay stores, and runtime state are separated, while public packs exclude API keys, local configuration, runtime snapshots, and replay databases.
- Packs and replays: A browser replay is a static record, whereas an installable experiment pack is a runnable seed containing scenario context, map, residents, locations, skills, and step schedule.Readers can inspect a run without installation, then download packs to modify or reproduce the setup.
- Simulation loop: Each simulation step combines map observations, resident profiles, world context, mounted skills, and pending interventions before the runtime executes a structured skill result.
- Operator commands: Ask sends questions to agents without changing the environment, while Intervene injects instructions or event and movement commands into the next step.
- Execution boundary: Hosted replays support credential-free browsing, but authoring, live commands, and new simulation steps require local execution.
4 Demonstration Plan
The demonstration follows an operator cycle in the GOD Town scenario, moving from replay inspection to questioning, intervention, automatic stepping, and authoring. The same setup supports hosted inspection and local modification.
- Scenario: GOD Town contains 10 residents, 10 semantic locations, and 65 location-scoped interactions, advancing from a weekday morning in 30-minute ticks.
- Operator cycle: The operator inspects replay summaries, messages, residents, and a profile before distinguishing stored-frame playback from a live step.
- Operator cycle: A read-only question to Alice records her current answer without changing the world.
- Operator cycle: A group intervention directs all 10 residents to gather at the library while automatic live steps advance the world and preserve the command beside new map frames.
- Execution paths: The hosted replay supports read-only inspection, while editing maps, changing packs, running new steps, and comparing variants require local execution.
- Intended use: The workflow is intended for controlled scenario comparisons, classroom walkthroughs, and interactive-world design, not as a substitute for human-behavior studies.
5 Evaluation
The evaluation tests whether GOD connects operator commands to replay evidence and portable artifacts across completed intervention runs. Results show measurable command routing, state-answer agreement, replay comparison, and package validation, with explicit scope limits.
- Evaluation setup: 15 completed run slots comprised one no-event baseline and 14 intervention runs across 10 scenario templates.Four scenario templates were run twice, using the same PKU map, 22 profiles, initial locations, and fixed Qwen-Plus configuration.
- Command routing: All 14 event commands reached the event or targeted-instruction path after fixing a parser trigger collision.The 14/14 result is a regression check on the corrected command path, not a held-out estimate of routing generalization.
- Command routing: 78 of 84 target-agent checks recorded the commanded destination, with all six misses occurring in the gymnasium variant because pathfinding marked the destination unreachable.The 23 non-target matches were reported separately because they included agents already at the commanded destination.
- Trace evidence: 212 of 308 agent-run evidence windows contained strong event terms, while the staff-only notice produced none in its two-step window despite targeted acceptance records.Routing and trace evidence were therefore treated as separate measures.
- Interview checks: 169 of 182 state answers matched a saved location alias or action substring, and 144 of 182 event-boundary answers passed the stated term rule.The event-boundary total combines 70/70 pre-event absence checks with 74/112 post-intervention presence checks; the separate leakage audit was 0/140 for seven listed unsupported locations.
- Artifact checks: The mean pairwise final-location JSD across four repeated scenario pairs was 0.011, with three pairs at 0 and the diplomatic-visit pair at 0.045.Four pairs support compact replay comparison but do not establish deterministic behavior, stability, or a timing effect.
- Comparison: The runner assumes GOD’s command, replay, profile, and pack schemas, so other systems would require adapters and no cross-system scores are reported.Table 1 instead compares capabilities reported in cited system descriptions.
- Artifact checks: The selected backend suite passed 82 tests covering replay export, package import/export, live endpoints, operator commands, and setup routing.The static build also passed replay and package validation.
6 Conclusion
GOD connects local scenario setup and live operator commands to replay inspection and portable packs for generative-agent societies. The release provides hosted replays, downloadable packs, and an Apache-2.0 repository.
- Conclusion: GOD contributes an operator and artifact workflow over integrated simulator and agent-runtime components, rather than a new town simulation or agent policy.The workflow maps browser commands to execution and replay records and provides authoring and import/export contracts.
- Conclusion: The selected backend suite passed 82 tests covering the released replay and package paths.The live benchmark used one model configuration and four repeated scenario pairs, while operator usability was not evaluated.
- Availability and licensing: GOD is released under Apache-2.0 with a public site offering hosted replays and downloadable experiment, map, and agent packs.Hosted replays require no credentials, whereas new runs require a local model endpoint.
7 Limitations
The reported evaluation is limited to one model configuration and repeated synthetic profiles. Its checks assess command and replay consistency rather than semantic grounding, completed trajectories, or operator usability.
- Scope: The evaluation tested one model configuration with repeated synthetic profiles, so results may differ with other models, scenarios, and users.The repeated profiles are not independent subjects.
- Measurement boundary: Target destination recorded@1 checks state after one movement step rather than completed trajectory execution or arrival.The measure therefore concerns recorded command-state consistency at that step.
- Measurement boundary: The string checks measure command and replay consistency, not semantic grounding or operator usability.The paper also states that the evaluation does not establish hallucination or persona quality.
8 Ethics and Broader Impact
GOD frames simulations involving real places or people as authored, synthetic artifacts rather than evidence about actual institutions or individuals. Its public packs also exclude private runtime state and credentials.
- The main demo uses a fictional town and synthetic residents, while real-world scenarios use synthetic profiles as packaging and replay stress tests.The paper keeps its qualitative case study on the Smallville-style town and labels generated behavior as simulated.
- Reviewers can inspect replay data and packs without access to local runtime state because packs exclude API keys, logs, private replay databases, and machine-specific configuration.
A Authoring Views
The authoring views prepare local runs by configuring the model, selecting or creating experiments, calibrating maps, and editing agent profiles. These views support importing packs and publishing locally runnable packages.
- The setup wizard records the model endpoint and local ports before the operator selects a bundled experiment, imports a pack, or creates a new run.
- Map Studio converts generated or imported maps into runnable packages by calibrating locations, walkable paths, and blocked regions.
- Agent Studio and agent packs support reusable resident profiles and sprites, including editing identity and personality fields and placing agents in the current world.
B Benchmark and Artifact Details
The benchmark exercises GOD’s browser-facing commands across controlled event scenarios and evaluates answers, movement, and replay traces from completed runs. Scoring distinguishes pooled results from scenario-level averages and uses lexical replay checks.
- The benchmark runner invokes the live ASK, INTERVENE, and RUN STEP endpoints and computes metrics from each run’s SQLite replay and command transcript.Only transcripts containing a completed-run marker enter scoring.
- All runs use one PKU campus map, 22 agent profiles, and identical initial locations, with one baseline and three controlled variants changing notification target, notification time, or related factors.The event block includes warnings, a diplomatic visit, a rumor, a lecture, and a traffic blockade.
- Each non-baseline run asks five role-diverse agents before and after an injected event, then intervenes on a destination and queries three target agents after another step.The roles are student, teaching assistant, librarian, reporter, and coordinator.
- Pooled numerators and denominators cover 14 completed intervention runs, while scenario macro-averages average repeats within scenarios before averaging the 10 intervention scenarios.Replay-state and event-boundary metrics use saved locations, action substrings, and lexical checks around event transitions.
B.1 Artifact and Reproducibility Checks
Artifact checks validate public replays, downloadable packs, and their boundary with local runtime state. The release exposes replay and pack inventories while excluding private execution data from portable artifacts.
- The public site provides two replay pages with 29 timeline frames, 10 and 22 agent entries, four completed operator commands, and eight release-backed replay downloads.
- The pack library exposes 10 experiment packs, 10 map packs, and 10 agent packs containing 141 profile entries, 104 location entries, and 173 interaction entries.
- The validator checks manifest fields, timeline length, step files, command completion, profile counts, map metadata, and release-backed download links.
- Backend tests cover replay export, package import and export, packs, live experiment endpoints, operator commands, and setup-wizard routing across portable-data and runtime-state boundaries.Packs exclude local logs, SQLite replay stores, runtime snapshots, API keys, model credentials, and machine-specific paths.