Source-linked AI summary
Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction
Yipeng Li, Ashutosh Hathidara, Jane Lo, Harshavardhan Abichandani, Gunraj Singh, Atin Ghosh
TL;DR
Privacy, compliance, and restricted schemas limit access to the enterprise data needed to train and evaluate tool-calling agents, while existing synthesis approaches leave gaps in validity or authoring demands. STS generates data through policy-enforcing APIs, and GP operates from the tool registry without schema access. Across ten environments, GP matches or exceeds a schema-privileged baseline, and STS-populated snapshots outperform seed snapshots in 10 of 12 model–metric pairs.
Problem
Enterprise agent training and evaluation need realistic database snapshots, but privacy, compliance, and proprietary schemas restrict access to production data and existing synthesis approaches have limitations.
Source: §1 · p. 1Method
STS generates data through policy-enforcing APIs, while GP populates environments through the raw tool registry without database-schema access or per-environment configuration.
Source: §1 · p. 2, §4.2 · p. 5Results
Across ten environments, GP matches or exceeds the schema-privileged EnvScaler baseline, and agents trained on STS-populated snapshots outperform seed-snapshot agents in 10 of 12 model–metric pairs.
Source: §6 · p. 10Takeaways & Limitations
The downstream comparisons support the utility of STS-populated snapshots for agent training.
Source: §6 · p. 10STS’s validity guarantee depends on complete API-level encoding of business rules; its scope is relational, API-mediated databases with explicit validation, not unstructured content or probabilistic enforcement.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Tool-calling agents have become central to enterprise AI, yet training and evaluating them at scale remains severely constrained due to business and legal restrictions on enterprise systems, data, and database schemas. Tabular data synthesis offers a natural alternative, but its effectiveness is fundamentally limited by structural validity and schema availability, while procedure-based approaches yield the opposite weakness, typically lacking distributional fidelity without per-domain authoring. We introduce **Synthesis Through Simulation** (STS), a **schema--free** data synthesis paradigm in which an LLM agent generates data by executing operations against policy-enforcing APIs within simulated enterprise environments. Because data is generated through the same environment that defines what is valid, STS guarantees structural validity by construction while decoupling validity enforcement from distribution modeling, allowing each to be addressed independently. The **Generalist Populator** (GP), STS's domain-agnostic agent, addresses the remaining challenges of distributional fidelity and synthesis scalability: GP achieves **0.88** average marginal fidelity and **100\% constraint satisfaction** across all ten environments *without access to DB schemas*, while statistical synthesizers are inapplicable to seven due to necessary seed data requirements, and schema-privileged agents fail 82\% of trajectories on airline environment's tightly coupled workflows due to brittle task composition. We open-source the full framework, all ten environments, and generated datasets at https://github.com/SAP/synthesis-through-simulation.
1 Introduction
Enterprise agents need realistic, coherent database snapshots, but privacy and compliance restrict access to production data and schemas. STS generates data through policy-enforcing APIs, while GP targets distributional fidelity without schema access or per-environment authoring.
1 Introduction
- Production enterprise data and schemas are restricted by privacy, compliance, and proprietary-domain concerns, complicating agent training and evaluation at scale.Such agents need realistic, coherent pre-populated database snapshots on which to act.Source: §1 · p. 1
- Statistical synthesizers optimize distributional fidelity but lack policy-defined relational enforcement and require large seed corpora, whereas procedure-based methods encode validity without typicality and require per-domain authoring.Enterprise validity depends on business policies checked against live database state, not solely on individual record values.Source: §1 · p. 2
- STS generates data by executing LLM-agent operations against policy-enforcing APIs, guaranteeing structural validity by construction and separating validity enforcement from distribution modeling.Accepted API trajectories produce coherent database snapshots, leaving distributional realism to the agent.Source: §1 · p. 2
- GP achieves validation pass rate VPR = 1 across ten environments and marginal fidelity above 0.93 in five, using only the raw tool registry.It requires no underlying database schema, hand-authored intent library, sampler, or per-environment configuration.Source: §1 · p. 2
- The paper frames enterprise synthesis around structural validity, distributional fidelity, and scalability, and introduces STS and the zero-authoring Generalist Populator to address them.The decomposition separates API-enforced validity from the agent’s distribution modeling task.Source: §1 · p. 2
2 Related Work
Prior synthesis methods address distributions or selected constraints, while agentic environment and benchmark work targets different outputs. STS instead generates database snapshots through API-level business-rule enforcement.
2 Related Work
- Tabular generators reproduce training-corpus distributions but treat records independently and face cold start; referential-integrity methods do not enforce dynamic business rules checked against live state.Constraint-aware synthesis handles static declarative constraints, whereas STS generates through the enforcement layer itself.Source: §2 · p. 2
- Agentic synthesis and environment-scaling systems generate trajectories or tool-interactive environments, rather than enterprise database snapshots with API-enforced business rules.These systems do not make write-time structural validity a design goal or measurement target.Source: §2 · pp. 2–3
- Tool-use benchmarks measure agent capabilities, while STS makes the populated database snapshot its deliverable and enforces constraints programmatically rather than through a post-hoc LLM judge.τ-bench shares an enterprise and policy-constrained setting, but evaluates agent pass/fail instead.Source: §2 · p. 3
3 Problem Formulation
The formulation represents an enterprise environment through its database state, typed tools, transition function, and validation suite. It assigns validity to API enforcement and leaves synthesis to optimize distributional fidelity and scalability.
3.1 Enterprise Environments
- An enterprise environment comprises database states, a typed tool registry, a transition function, and a suite of programmatic business-rule checks.Each tool has an input schema and a precondition over the current state and arguments.Source: §3.1 · p. 3
- The tool registry is the only environment interface, and the database schema remains inaccessible; trajectories induce state sequences from an initial seed state.The seed may be empty or contain pre-populated records.Source: §3.1 · p. 3
- A tool call commits a deterministic state transition when its write is accepted, while the tool precondition encodes business rules checked at write time.The validation suite evaluates programmatic business-rule checks over the terminal state.Source: §3.1 · p. 3
3.2 Synthesis Objectives
- For any trajectory executed through the environment, validation pass rate is VPR = 1 by construction.Writes commit only when the environment’s preconditions hold; rejected calls leave the state unchanged.Source: §3.2 · pp. 3–4
- Figure 1 depicts GP’s N-trajectory loop through EXPLORE, PLAN, and EXECUTE, with an ExplorationManifest shared across trajectories.The manifest accumulates structural knowledge, distribution weights, and reusable heuristics.Source: Figure 1 · p. 4
- With validity guaranteed, synthesis targets distributional fidelity and scalability without per-environment intent authoring.Fidelity is measured at marginal per-column, pairwise within-table, and cross-table child-count levels.Source: §3.2 · p. 4
4 The STS Framework
STS combines policy-enforcing enterprise environments with the Generalist Populator, which synthesizes data through tool registries rather than direct record generation. GP explores, plans, and executes tasks while accumulating reusable environment knowledge.
4 The STS Framework
- STS pairs policy-enforcing environments that guarantee structural validity with GP, a zero-authoring agent that synthesizes data from the tool registry alone.The framework’s components separate environment-level validity enforcement from distribution-faithful data generation.Source: §4 · p. 4
4.1 Policy-Enforcing Environments
- The ten environments span enterprise domains and expose typed operations through a tool registry, without exposing their database schemas.Each is a SQLite-backed mock system with a validation suite.Source: §4.1 · pp. 4–5
4.2 The Generalist Populator
- GP populates databases exclusively through API calls, requiring no schema access, hand-authored task specifications, or per-environment configuration.Across trajectories it maintains an ExplorationManifest containing structural knowledge and reusable information.Source: §4.2 · p. 5
- EXPLORE builds a model of tools, entity creation order, and plausible workload weights during the first K trajectories, then later trajectories reuse the manifest.Exploration uses read calls and tool-signature inspection to infer this knowledge.Source: §4.2 · p. 5
- PLAN uses manifest context, task history, and operation-coverage signals to select a concrete task and promote coverage across trajectories.Attribute targets use reference distributions when available and otherwise rely on the language model’s world knowledge.Source: §4.2 · pp. 5–6
- EXECUTE resolves identifiers through database reads, writes in dependency order, adapts to rejected calls, and stores discovered shortcuts as reusable heuristics.Its agentic loop receives the manifest, planned task, and accumulated heuristics as context.Source: §4.2 · p. 6
5 Experiments
The experiments assess GP’s fidelity, validity, and scalability across simulated and real enterprise systems, and compare it with statistical and schema-privileged synthesis. Ablations examine exploration, planning, and heuristics, while a downstream test evaluates training data generated from STS-populated snapshots.
5.1 Metrics
- MDF measures per-column marginals, PDF within-table column-pair distributions, CDF child-count distributions across FK relationships, and WO successful state-changing API calls per trajectory.VPR reports structural validity; the distributional metrics use JSD-based similarity, with numeric columns using the complement of the Kolmogorov–Smirnov statistic for MDF.Source: §5.1 · p. 6
5.2 GP Performance Across All Environments
- VPR = 1.0 across ten environments; MDF exceeds 0.93 in five—Airline 0.973, IT Mgmt 0.966, Forum 0.945, HR 0.939, and E-commerce 0.983.HR and Banking have the highest mean WO (3.32 and 4.59).Source: Table 2 · p. 7, §5.2 · p. 7
5.3 Generalization to Real Enterprise Systems
- On Payroll and MintHCM, GP achieves VPR = 1.0 and MDF ≥0.90 without schema access; MintHCM’s PDF is 0.344 at N=100.The passage attributes MintHCM’s lower PDF to insufficient coverage of 119 possible candidature-table pairs.Source: §5.3 · p. 7, Table 3 · p. 8
5.4 LLM Backbone Sensitivity
- Backbone sensitivity was tested with three LLMs across Airline, HR, and Cloud using GP at K = 3 and N = 100.The supplied passages identify the comparison setup and Table 4, but not its results.Source: Table 4 · p. 8, §5.4 · p. 8
5.5 Coverage Advantage: GP vs. Tabular Data Synthesizers
- Seven environments lack seed data; among the three seed-backed comparisons, GP achieves VPR = 1.0, leads on airline fidelity, but trails slightly on retail and banking.GP’s per-trajectory latency is 41–118s, considerably slower than offline synthesizers per row; the authors note that the use cases differ.Source: §5.5 · p. 8, Figure 3 · p. 9
5.6 GP vs. Schema-Privileged Synthesis
- EnvScaler has 82% zero-write trajectories on airline, mostly attributed to stale-ID collisions; schema access favors its cloud CDF, while HR fidelity differences are minimal.The authors do not treat the full airline failure rate as a capability gap; cloud CDF is 0.649 vs. 0.610, and HR CDF is 0.505 vs. 0.509 for GP and EnvScaler, respectively.Source: §5.6 · pp. 8–9, Table 5 · p. 9
5.7 Ablation Study
- Exploration, planning, and heuristics affect fidelity and workflow productivity: K = 3 captures the PDF gain, while removing PLAN or heuristics degrades different outcomes.Without PLAN, fidelity drops despite more bookings per trajectory; without heuristics, banking WO and PDF fall, HR zero-write failures rise, and airline Calls/WO nearly doubles.Source: §5.7 · pp. 9–10
5.8 Downstream Agent Training
- SFT from STS-populated snapshots beats seed-snapshot SFT in 10 of 12 model–metric pairs across four backbones on the τ 2-bench airline evaluation.The two exceptions are both Gemma-4-E2B-it results, with margins ≤0.017.Source: Table 6 · p. 10, §5.8 · p. 10
6 Conclusion
STS guarantees structural validity by construction, and GP operates without schema access or per-environment configuration. The ten environments span domains and validation complexity, from Forum’s 10 tools and 10 checks to HR’s 30 tools and 61 checks.
6 Conclusion
- Across ten environments, STS guarantees VPR = 1.0 by construction, and GP matches or exceeds EnvScaler despite a more constrained information budget.STS-populated snapshots also outperform pre-populated seed snapshots in 10 of 12 model–metric pairs across four backbones.Source: §6 · p. 10
- GP is a zero-authoring agent that requires neither schema access nor per-environment configuration.The framework deliberately limits the synthesizer to the tool registry, reflecting the information boundary available to an external agent in many enterprise deployments.Source: A Enterprise Environments · p. 13
A Enterprise Environments
- The environments range from Forum, with 10 tools and 10 validation checks, to HR, with 30 tools and 61 checks across complex business policies.The domains vary in subject matter and complexity.Source: A Enterprise Environments · p. 13
- The environment table covers ten enterprise settings with typed tool registries and programmatic business-rule checks, while keeping database schemas inaccessible to the synthesizer.Source: Table 7 · p. 13, A Enterprise Environments · p. 13
B Graph Similarity Metrics for Manifest Comparison
Manifest comparison uses label-independent graph metrics to compare entity-dependency structure. In the airline comparison, the obfuscated graph differs from the semantic graph by two edge edits.
B Graph Similarity Metrics for Manifest Comparison
- The comparison projects directed graphs to undirected form and sets substitution costs to zero, so GED measures topology rather than labels or direction.Node and edge insertions and deletions each cost one.Source: B Graph Similarity Metrics for… · p. 14
- Under these costs, GED is the number of edge insertions and deletions needed to make the unlabeled graphs structurally identical.A GED of zero indicates isomorphic topology.Source: B Graph Similarity Metrics for… · p. 14
- Normalized GED scales the edit distance by the edge count of the larger graph and lies between zero and one.The section describes exact GED computation with branch-and-bound for the studied 11-node graphs.Source: B Graph Similarity Metrics for… · p. 14
- GED = 2 and GEDnorm = 0.154 for the two airline manifest graphs, which each have 11 nodes; the difference is two edge edits.The obfuscated graph adds passenger-to-booking edges inferred from argument schemas without semantic cues.Source: B Graph Similarity Metrics for… · p. 14
C Metric Definitions
The metrics compare synthesized records with fixed reference distributions at marginal, within-table pairwise, and cross-table relational levels. Separate measures track successful writes and exploration outcomes across environments.
C.1 Common Foundation: JSD-Based Similarity
- The three distributional metrics—MDF, PDF, and CDF—share the same foundation; each reports 95% confidence intervals using non-parametric bootstrap resampling (B = 1000, α = 0.05).Source: §C.1 · pp. 14–15
- Baseline subtraction counts only records added by the agent, using live counts minus snapshot counts clipped at zero; this assumes seed rows are not modified or deleted during synthesis.The stated STS workflows are creation-oriented; structural metrics instead track added rows by primary-key set subtraction.Source: §C.1 · p. 14
C.2 Marginal Distribution Fidelity (MDF)
- MDF measures whether each registered column’s agent-generated value distribution matches its reference marginal, using a Kolmogorov–Smirnov statistic for numeric columns.Source: §C.2 · p. 15
C.3 Pairwise Distribution Fidelity (PDF)
- PDF compares joint distributions for registered within-table column pairs, measuring correlation structure beyond what MDF captures.The reference joint distributions are marginally consistent with the MDF reference distributions.Source: §C.3 · p. 15
C.4 Cross-table Distribution Fidelity (CDF)
- CDF compares child-record counts per agent-created parent with a reference distribution for each registered foreign-key relationship.Counts above nmax are folded into the last bucket.Source: §C.4 · p. 15
C.5 Average Successful Write Operations (WO)
- Average successful write operations counts successful write-tool calls per trajectory and averages those counts across trajectories; API-rejected calls are excluded.Tables enumerate registered marginal columns and within-table pairs by environment.Source: §C.5 · pp. 15–16, Table 8 · p. 16, Table 9 · p. 16
Airline ablation: full numerical results
- Across all ten environments, K = 3 increases zero-write fractions over K = 0 in Forum (0.30 versus 0.13) as exploration shifts sampling toward more complex workflows.The passages report persistently elevated zero-write rates in HR, Pharmacy, and University, although exploration reduces but does not eliminate them.Source: Zero-write failure rate (zero_frac)… · p. 17
- Exploration’s effects vary by environment: for complex workflows, K = 3 raises successful writes over K = 0 in HR (+1.98), Banking (+2.32), and Retail (+2.30), while MDF falls in lifecycle-ordered settings.MDF falls from 0.896 to 0.709 to 0.573 in Retail and from 0.939 to 0.864 to 0.802 in Banking; complex multi-step environments also retain elevated zero-write rates.Source: MDF, WO, and zero_frac across K and… · p. 17, Zero-write failure rate (zero_frac)… · p. 17
F Implementation Details
Implementation uses a fixed agent and evaluation setup across environments, with GP’s primary configuration set to K = 3. The supplied table captions identify the scopes of the registered-FK and ablation tables.
F Implementation Details
- GP’s primary evaluation uses K = 3, N = 100 independent trajectories per environment, clean databases with fixed seed snapshots, and Claude 4.5 Opus at temperature 1.0.The tool loop uses LangGraph; the PLAN coverage window is max(10, 2 × the number of write tools).Source: F Implementation Details · p. 17
- The statistical baselines use SDV CTGAN and TVAE trained for 300 epochs, SDV HMA with Gaussian-copula fitting, and REaLTabFormer with 20 parent rows.Generated row counts and parent-row settings vary by baseline.Source: F Implementation Details · p. 17
- The supplied captions identify Table 10 as registered foreign-key relationships and Table 11 as the airline exploration-budget ablation with 100 runs per condition.Table 12 is captioned as zero-write trajectory fractions across environments and exploration budgets, also with 100 runs each.Source: Table 10 · p. 17, Table 11 · p. 18, Table 12 · p. 18
G Ablation Tables: Heuristic Extraction and Plan Phase
The ablations show that heuristic extraction and planning contribute in different ways: heuristics reduce execution cost in some environments, while planning improves distributional alignment on Airline.
Heuristic Extraction
- Without heuristics, Banking mean WO drops by 1.49 (33%) and tool-call overhead per write rises by 54%; Airline WO drops slightly (−5%) while Calls/WO nearly doubles (+96%).HR is an outlier: heuristics improve WO marginally (+7%) while slightly increasing calls per write, and do not eliminate failures tied to data-dependency complexity.Source: G Ablation Tables: Heuristic… · p. 18, Table 14 · p. 19
Plan Phase
- Removing PLAN degrades Airline’s distributional metrics (∆MDF −0.065, ∆PDF −0.062, ∆CDF −0.071) and reduces round-trip bookings from 52.5% to 7.9%, against a 55% reference prior.Without a plan, task composition emerges opportunistically from the current database state rather than targeting reference marginals.Source: G Ablation Tables: Heuristic… · pp. 18–19, Table 15 · p. 19
H Plan Phase Ablation: Trajectory Analysis (Airline)
On Airline, removing the plan changes trajectories from targeted user activity to broad database population and strongly skews bookings toward one-way trips.
H Plan Phase Ablation: Trajectory Analysis (Airline)
- Without planning, 48 of 50 trajectories process all 10 users in one run, unlike the few-user, few-action trajectories produced with planning.The no-plan pattern resembles a direct database seeding script rather than simulated user activity.Source: H Plan Phase Ablation: Trajectory… · p. 19, Table 16 · p. 20
- Without an explicit plan instruction, round-trip bookings fall from 52.5% to 7.9%, against a 55% reference prior, as the agent defaults to one-way bookings.Round trips require composing outbound and return flights into one reservation call; the plan embeds the target attribute in the task description.Source: H Plan Phase Ablation: Trajectory… · p. 19, Table 16 · p. 20
- With a plan, one trajectory books one round-trip economy flight for one gold-tier user in 18 tool calls; without one, the agent makes 10–22 one-way bookings in 93 calls on average.The examples contrast one write for one user with 10–22 writes across 10 users.Source: H Plan Phase Ablation: Trajectory… · p. 19
I Computational Cost and Throughput
GP’s computational profile varies with environment complexity and workflow structure; comparison with direct row insertion shows that the API enforcement layer, rather than uniformly higher fidelity, distinguishes STS.
I Computational Cost and Throughput
- HR (30 tools, 61 checks) takes 2.9× longer per trajectory than IT Mgmt (13 tools, 10 checks); Cloud’s 100 s standard deviation reflects occasional long VPC-provisioning trajectories.Table 17 reports per-trajectory latency, throughput, and estimated cost across all environments.Source: I Computational Cost and Throughput · p. 20, Table 17 · p. 20
- Banking and retail achieve the highest write throughput (5.10 and 3.85 writes/min) despite different latency profiles, because their workflows involve compact multi-step transactions.A cost-per-write measures one fully validated transactional episode, not one row; an Airline write creates 3–6 rows across three tables with all foreign-key constraints satisfied.Source: Table 17 · p. 20, I Computational Cost and Throughput · p. 21
J Isolating the Simulation Loop: LLM-Direct
- VPR is 1 for STS but 0.846 / 0.820 for LLM-Direct on Airline and HR; fidelity is not uniformly higher, and LLM-Direct has higher HR CDF (0.588 vs. 0.505).LLM-Direct uses the same backbone but inserts rows directly, bypassing API policy enforcement.Source: J Isolating the Simulation Loop:… · p. 21, Table 18 · p. 21
K Statistical Synthesizer Training Configuration
This appendix section documents GP execution, statistical-synthesizer constraints, and further evaluation details, including additional structural-validity results and distributional and safety qualifications. It also records broader-impact and release disclosures covering licensing, documentation, human-subject status, and LLM use in the core method.
K Statistical Synthesizer Training Configuration
- Statistical synthesizers apply only to airline, retail, and banking, the environments with seed rows; single-table CTGAN and TVAE cannot model cross-table consistency.CTGAN and TVAE use 300 epochs and sample 100 rows per table; HMA models parent-child cardinalities, while REaLTabFormer generates children conditioned on parents, with 20 parent rows sampled per environment.Source: K Statistical Synthesizer Training… · pp. 21–22
- HMA’s banking VPR is 0.800 (8/10 checks), while VPR is N/A for CTGAN, TVAE, and REaLTabFormer because of seed-ID reuse or primary-key collisions.Synthesized rows are inserted into a clean SQLite copy, bypassing the API, and checked with the same constraint suite used for GP.Source: K Statistical Synthesizer Training… · pp. 21–22
EnvScaler: multi-subtask task that produces zero writes
- 82 of 100 EnvScaler trajectories produce zero net writes because stale entity state causes the first write to fail.Independently generated tasks can overlap in destructive writes, leaving schema- and ID-specific tasks without a fallback.Source: EnvScaler: multi-subtask task that… · p. 22
GP: focused single-intent task that succeeds
- 4 successful write operations result when GP discovers live entities at execution time and completes a focused booking task.The task avoids hard-coded entity IDs; the broader comparison notes that this design remains resilient as entity state evolves.Source: GP: focused single-intent task that… · p. 22, Summary · p. 23
M Real-World Enterprise Generalization: Details
- Both Payroll and MintHCM attain VPR = 1.0 with GP given only tool repositories and no schema, business-rule encodings, or per-system configuration.MintHCM’s PDF is 0.344, attributed to sparse sampling of a high-cardinality candidatures pair at N = 100 rather than modeling failure.Source: M Real-World Enterprise… · p. 23
N Backbone Sensitivity: Full Confidence Intervals
- 10.60 mean and 48.29 standard deviation describe Gemini 2.5 Pro Cloud WO; it still attains VPR = 1.0 and produces 120 valid records.Across all nine backbone–environment pairs, quantitative variation is comparable to the cross-environment spread reported in Table 2.Source: N Backbone Sensitivity: Full… · pp. 23–24
O Broader Impacts
- STS requires write access to a live policy-enforcing API, and accepted trajectories are policy-compliant, so it adds data-generation convenience rather than new access or attack surface.The passage frames this as bounding the marginal risk of producing training data for agents that operate without authorization.Source: O Broader Impacts · p. 24
NeurIPS Paper Checklist
- Proposition 1 states that GP-generated databases satisfy all validation checks by construction, and the paper provides its full proof immediately after the proposition.The checklist also states that the result’s assumptions and proof are provided.Source: NeurIPS Paper Checklist · p. 26
10. Broader impacts
- The paper identifies privacy-preserving research infrastructure as a societal impact and states that its released framework and simulated environments pose no high misuse risk beyond standard software-release concerns.The release consists of a data synthesis framework and ten simulated environments, not pretrained language models or scraped internet data.Source: §10 · p. 29
- The paper credits SDV, REaLTabFormer, and LangGraph, and lists their licenses as Business Source License, MIT, and MIT, respectively.Source: §10 · p. 29
- The paper reports that its ten simulated environments and generated datasets include a step-by-step README and datasheet-format documentation.Source: §10 · p. 30
- The research involves no crowdsourcing or human subjects.Source: §10 · p. 30
Related papers
This paper cites
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?2023
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs2023
- $τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains2024
- Deep Learning with Differential Privacy2016
- ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training2026
- ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas2026