Source-linked AI summary
Synthetic Computers at Scale for Long-Horizon Productivity Simulation
Tao Ge, Baolin Peng, Hao Cheng, Jianfeng Gao
TL;DR
Long-horizon productivity agents need to use rich, user-specific computer context effectively. This paper creates artifact-rich synthetic computers and runs tailored simulations, finding improved performance on held-out synthetic computers and an external productivity benchmark.
Problem
Productivity agents need to use rich, user-specific files, history, and evolving work context effectively over long horizons.
Method
The paper creates diverse synthetic computers with realistic hierarchies and artifacts, then runs tailored objectives through separate setup and work agents.
Results
Synthetic-computer simulations significantly improve agent performance on both held-out synthetic computers and an external productivity benchmark.
Takeaways & Limitations
Synthetic computers provide a promising substrate for agent self-improvement in long-horizon productivity scenarios.
Takeaways & Limitations
The extracted skill set may eventually become too large to carry as external instructions, reducing its usefulness and increasing agent burden.
Abstract
from arXiv · showhide
Realistic long-horizon productivity work is strongly conditioned on user-specific computer environments, where much of the work context is stored and organized through directory structures and content-rich artifacts. To scale synthetic data creation for such productivity scenarios, we introduce Synthetic Computers at Scale, a scalable methodology for creating such environments with realistic folder hierarchies and content-rich artifacts (e.g., documents, spreadsheets, and presentations). Conditioned on each synthetic computer, we run long-horizon simulations: one agent creates productivity objectives that are specific to the computer's user and require multiple professional deliverables and about a month of human work; another agent then acts as that user and keeps working across the computer -- for example, navigating the filesystem for grounding, coordinating with simulated collaborators, and producing professional artifacts -- until these objectives are completed. In preliminary experiments, we create 1,000 synthetic computers and run long-horizon simulations on them; each run requires over 8 hours of agent runtime and spans more than 2,000 turns on average. These simulations produce rich experiential learning signals, whose effectiveness is validated by significant improvements in agent performance on both in-domain and out-of-domain productivity evaluations. Given that personas are abundant at billion scale, this methodology can in principle scale to millions or even billions of synthetic user worlds with sufficient compute, enabling broader coverage of diverse professions, roles, contexts, environments, and productivity needs. We argue that scalable synthetic computer creation, together with at-scale simulations, is highly promising as a foundational substrate for agent self-improvement and agentic reinforcement learning in long-horizon productivity scenarios.
1 Introduction
Synthetic Computers at Scale creates diverse, artifact-rich, user-specific computer environments to support realistic long-horizon productivity simulations. Preliminary experiments show that these simulations generate experiential learning signals associated with improved in-domain and out-of-domain productivity performance, while the methodology can scale broadly with sufficient compute.
- Motivation: Productivity work is context-heavy and requires agents to use files, project history, prior decisions, collaborator feedback, and evolving work state over long horizons.The paper argues that synthetic data must synthesize realistic user context rather than only generic tasks.
- Method: Synthetic Computers at Scale progressively elaborates personas into user-specific computer environments populated with realistic, artifact-rich content.The environments are designed to support productivity work involving structured artifacts such as documents, spreadsheets, presentations, reports, and supporting materials.
- Method: Each simulation uses a setup agent to create tailored objectives and a separate work agent to act as the user across the computer until completing multiple professional deliverables corresponding to about a month of human work.The work agent navigates the computer, coordinates with simulated collaborators, and produces professional artifacts.
- Preliminary experiments: 1,000 synthetic computers were instantiated, with each simulation requiring over 8 hours of agent runtime and spanning more than 2,000 turns on average.The simulations record process and outcome signals, including searching, planning, revision, coordination, feedback incorporation, and completed artifacts.
- Results and implications: Significant improvements in agent performance were observed on both in-domain and out-of-domain long-horizon productivity evaluations.The authors position scalable synthetic computer creation and simulation as a promising substrate for agent self-improvement and agentic reinforcement learning.
- Resources: The release includes 100 synthetic computers—50 Windows-style and 50 macOS-style—and retrospective analysis reports for 500 long-horizon simulations.These resources support research on synthetic computer creation, experiential learning from long-horizon trajectories, and scalable productivity simulation.
2 Synthetic Computer Creation
Synthetic computers are created by progressively elaborating personas into detailed user profiles, user-specific filesystem plans, and populated directory hierarchies containing realistic, content-rich artifacts. The planning process specifies organization, files, artifact types, timestamps, and cross-file dependencies before content generation.
- 2.3 Artifact Instantiation: The planned filesystem is instantiated as a directory hierarchy populated with realistic, content-rich artifacts after folder organization, naming conventions, and inter-artifact relationships are planned.This completes the progression from persona and profile to a populated synthetic computer environment.
- 2.1 User Profile: Each synthetic computer begins with a persona that is expanded into a detailed user profile serving as its semantic foundation.The profile captures professional context and computer-use behavior, including responsibilities, collaborators, work products, tools, document habits, naming preferences, and organization style.
- 2.2 Filesystem Planning: The user profile determines a filesystem plan covering directory organization, a virtual time axis, project structure, file inventory, artifact types, and cross-file dependencies.A filesystem policy first establishes system start time, drive layout, default paths, storage patterns, naming style, and usage patterns.
- 2.2 Filesystem Planning: Planned files are organized into logical paths with artifact types, descriptions, timestamps, origins, and content modes inferred from projects, responsibilities, collaborators, and document habits.The plan specifies both which files exist and how they are arranged within the directory structure.
- 2.2 Filesystem Planning: A directed dependency graph links files that reference, derive from, represent later versions of, or extract from other files, enabling conditioned artifact instantiation.This graph models correlations among artifacts so later files can reuse, summarize, revise, or cite earlier ones rather than being generated independently.
3 Long-Horizon Productivity Simulation
The simulation uses a setup agent to create user-specific productivity objectives and collaboration settings, then a work agent acts as the user to complete those goals on the synthetic computer. Objectives span about a month, require multiple professional deliverables, and may involve coordination with simulated collaborators rather than independent task completion.
- Simulation design: Two agents divide the simulation: the setup agent creates tailored objectives and collaboration settings, while the work agent operates the synthetic computer as the user.This design separates scenario construction from execution across the user’s computer.
- Objective generation: The setup agent conditions objectives on the user profile and computer state, including role, responsibilities, active projects, file inventory, and existing artifacts.It infers realistic and valuable objectives for about a month of work rather than sampling generic tasks.
- Objective generation: Objectives are organized into deliverable work packages specifying professional outcomes, expected progress, and artifacts to create or revise, with dependencies between some deliverables.Examples include analysis workbooks, internal memos, presentation decks, supporting materials, and final PDF packages.
- Collaboration setting: The setup agent models productivity as coordination with other people, allowing the work agent to clarify requirements, request data, and obtain source materials during execution.This distinguishes the simulations from standalone tasks where all information is provided upfront.
- Collaboration setting: Simulated collaborators represent managers, peers, clients, compliance officers, or external partners, each with roles, backgrounds, communication styles, relevant knowledge, and sometimes private reference materials.These materials can be shared with the user later through collaboration when needed.
4 Experiments
Experiments show that month-long simulations create realistic, content-rich computer environments through sustained artifact production and collaboration. Rubric evaluations indicate that trajectory-derived skills improve work-agent performance both in-domain and out-of-domain.
- Synthetic computer statistics: 197 files per computer after simulation, up from 112 before, while directory depth remains stable, indicating artifact growth within established organizational structures.The simulations primarily extend existing environments by creating and revising artifacts rather than generating unrealistic directory structures.
- Artifact composition: 67.8% of files are DOCX, XLSX, PDF, or PPTX, with DOCX alone comprising 34.8%, alongside supporting code, text, image, and structured-data files.Artifact-size statistics further indicate that presentations, PDFs, collaborator references, and final deliverables are content-rich rather than lightweight placeholders.
- Simulation scale: 2,272 turns and 8.59 hours per simulation on average, with 5.5 collaborators and about 31 communications, demonstrate sustained execution and coordination.Most runtime comes from daily execution rather than weekly planning, extending beyond one-shot task completion.
- Deliverable evaluation: Scores for most evaluated computers fall between 60% and 80%, showing that the work agent completes many required deliverables but still leaves substantial room for improvement.A Claude Opus 4.6 judge scores final deliverables using computer- and objective-specific rubrics merged from five simulation runs.
- In-domain skill transfer: 68.6% mean rubric score versus 61.6% baseline, with the skill-augmented agent outperforming the baseline on 83 of 100 held-out computers.The skills are extracted from 900 training simulations, and trajectory analysis supplies experiential signals about successful behaviors and failure modes.
- Out-of-domain transfer: 105 wins and 67 losses in the primary Sonnet out-of-domain setting yield significant sign tests, with p = 0.002 and p = 0.005; transfer to Haiku and Opus is positive but weaker.The skills were extracted from Sonnet-based simulation trajectories, and both one-sided and two-sided tests are significant.
5 Discussion
Synthetic computers support a self-improving loop in which realistic long-horizon simulations generate experiential signals that improve agent behavior. Their value can grow through scaling simulations, agents, models, and persona-conditioned contexts for productive work.
- Self-improving loop: Synthetic computers enable large-scale realistic simulations without private user data, producing experiential signals from planning, filesystem use, collaboration, revision, failure recovery, and completed deliverables.These signals capture both process and outcomes.
- Self-improving loop: Figure 9’s loop creates synthetic computers, runs long-horizon simulations, extracts lessons and work patterns, converts signals into skills, and eventually distills them into model weights.The skill set can then be reset for the next simulation round.
- Self-improving loop: Skill-based improvement is fast, inspectable, and useful for validation, but an ever-growing skill set can become too large as external instructions and burden the agent.Updating underlying model weights is identified as a natural next step.
- Scaling dynamics: Synthetic computers become more valuable as they scale because more simulations enrich environments, stronger agents produce better trajectories, and stronger models extract better learning signals.This creates a favorable scaling dynamic for realistic, high-signal productivity data.
- Broader coverage: Persona-scale generation can create differentiated synthetic computers spanning diverse professions, organizations, workflows, work styles, and productivity needs.The methodology scales realistic contexts rather than merely generating more tasks.
- Broader coverage: Synthetic computers provide accumulated documents, project states, feedback, preferences, constraints, and decisions for practicing long-horizon planning, grounding, coordination, revision, and recovery.As agents improve, these contexts can support useful work across realistic professional settings.
6 Conclusion and Future Work
Synthetic Computers at Scale provides realistic, user-specific environments for long-horizon productivity simulation, producing experiential signals that improve agents on held-out synthetic computers and an external benchmark. Future work targets more personalized artifacts, noisier filesystems, and dynamic collaborators to better reflect evolving human contexts.
- Conclusion: The methodology creates realistic, user-specific computer environments that support long-horizon productivity simulation and improve agents on held-out synthetic computers and an external productivity benchmark.These simulations provide useful experiential signals for agent self-improvement.
- Future work: Artifact design should infer user- or organization-specific preferences so artifacts vary in visual style, layout, formatting, and content.Current artifacts reflect users’ roles, projects, and files but remain too visually uniform across computers.
- Future work: Filesystems should model everyday noise and accumulated history, including temporary downloads, duplicate drafts, abandoned files, screenshots, web saves, and outdated materials.Such traces would make synthetic computers less clean and more human-like.
- Future work: Collaboration should become more dynamic by simulating collaborators’ independent work, files, meetings, emails, deadlines, and evolving organizational state.Richer multi-agent simulation could make coordination and organizational context closer to real productivity work.
- Future work: These directions are challenging but achievable with current modeling and agent capabilities, requiring careful design and sustained investment for substantial potential payoff.The envisioned payoff is bridging increasingly capable agents with high-value productivity work grounded in rich, evolving human contexts.
A Retrospective Analysis Report
The retrospective finds strong planning and analytical execution, but severe cross-document inconsistencies and uncorrected known errors undermined the deliverables.
- Overall assessment: 71.5% overall, with deliverable scores ranging from 54.8% for the ESG overlay to 88.2% for Castellano onboarding.Strengths included upfront planning, collaborator communication cadence, and core analytical work.
- Strengths: All 8 output files were created and properly located, with documented 10,000-path, 10-year Monte Carlo analysis and complete 12-asset-class delta tables.The package also cited 2025-12-31 data-as-of information and three external sources.
- Specification adherence: The Capital Markets Outlook final PDF was 11 pages versus the specified 18–22 pages, with no evidence the agent recognized or addressed the shortfall.The weekly plan correctly called for an 18–22 page outlook, but execution produced a condensed version.
- Cross-document consistency: US Large Cap weights differed across documents—14%, 20%, 27.4%, and 19%—while cash varied from 5% to 9%.The agent built documents independently without a reconciliation pass.
- Cross-document consistency: EM exposure was trimmed from 12% to 10% in the workbook and presentation but increased from 10% to 11.1% in the final PDF and rollout memo, reversing the IC-approved direction.The rollout memo therefore instructed advisors to describe EM exposure as increasing.
- Failure modes: 3 Day 17 stress-test data errors remained uncorrected despite Sandra’s explicit instruction not to submit the workbook with current values.The agent sent Sandra a blank message on Day 19, and her reply confirmed the corrections were never made.