Source-linked AI summary

Evolving Agent Interfaces

Saaket Agashe, Yuzhe Yang, Gonzalo Gonzalez-Pumariega, Bhuvan Chandra Koduru, Chih-Lun Li, Jiachen Yang, Ang Li, Xin Eric Wang

arXiv:2610.28601v1cs.AI

TL;DR

Computer-use interfaces are designed by humans and fixed before execution, although task diversity creates mismatches and requires manual redesign. Agent S4 evolves repositories of interfaces and usage knowledge from synthetic task trajectories, then selects repositories using held-out validation unseen by the meta-agent. The evolved repositories outperform the cited fixed-interface and model baselines on OSWorld V2 and ScienceBoard.

  • Problem

    Human-designed, fixed interfaces require manual redesign as tasks and models change and can mismatch the operations and feedback needed for a task.

    Source: §1 · p. 2
  • Method

    Agent S4 uses a meta-agent to revise interface repositories from synthetic-task trajectories without success labels, while held-out validation guides selection and the task agent stays fixed.

    Source: §1 · p. 2
  • Results

    21.3% binary success on OSWorld V2 for Qwen3.8-27B with evolved interfaces exceeds GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%); ScienceBoard success is 58.6% versus 41.4% for Qwen CUA.

    Source: §1 · p. 3, §9 · p. 15
  • Takeaways & Limitations

    The results support interface design as an agent capability, with repositories that evolve both available interfaces and knowledge about when and how to use them.

    Source: §1 · pp. 2–3

    Task attempts are bounded by each task’s budget and the harness’s permissions.

    Source: §2 · p. 4

Abstract

from arXiv · show

Computer-use agents are limited by the interfaces humans give them. An agent may type a spreadsheet cell by cell when a single line of code would do, or print all two thousand lines of a file through a shell to find one function. Both the interfaces and the knowledge of when to use each are built by hand, which cannot keep pace with new applications, workflows, and models. We argue that interface design should itself be an agent capability. We introduce Agent S4, an autonomous computer-use scientist that builds Evolving Agent Interfaces from task experience. A meta-agent examines task agents' trajectories on synthetic training tasks, without success labels, and constructs and revises interfaces and usage knowledge in a repository. Population-based evolution then selects which repositories to develop further by their performance on validation tasks the meta-agent never sees, favoring changes that generalize beyond the seen tasks. On OSWorld V2, Agent S4, with Qwen3.8-27B and the evolved repository of interfaces, sets a new state of the art in open-source computer-use agents, reaching 21.3% binary score, 3.8 times the same model with fixed interfaces. This is above both GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their respective costs. On ScienceBoard, the evolved interfaces improve the success rate from 41.4% to 58.6% with the same model. These results suggest that useful interfaces can be evolved from experience, making autonomous interface design a source of improvement complementary to advances in models and harnesses. We open-source our code at https://github.com/simular-ai/agent-s.

1. Introduction

Human-designed, fixed interfaces can require manual redesign as tasks and models change, and may mismatch the work agents need to do. Agent S4 evolves interface repositories from task trajectories, then uses held-out validation to select repositories for development.

1. Introduction

  • Human-designed interfaces require researchers to diagnose failures and create new primitives, observations, or memory mechanisms, a process the paper says does not scale with task diversity.Static interfaces can mismatch tasks, while reusable skills remain limited to the operations and observations allowed by the preset interface.Source: §1 · p. 2
  • Agent S4 builds repositories of mixed interfaces and usage knowledge from synthetic task trajectories, without success labels, while keeping the task agent’s model and harness fixed.The meta-agent studies trajectories to hypothesize changes to actions, observations, or usage knowledge, then mutates or crosses over repositories.Source: §1 · pp. 2–3
  • 21.3% binary success on the full 108-task OSWorld V2 set puts Qwen3.8-27B with evolved interfaces above GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their respective costs.On ScienceBoard, a separate evolution run reaches 58.6% success versus 41.4% for Qwen CUA with a human-designed interface.Source: Figure 2 · p. 2, §1 · p. 3
  • Held-out validation tasks, which the meta-agent never sees, guide repository selection so that construction uses training experience while selection favors generalization beyond it.Validation instructions, trajectories, diagnostics, and scores remain hidden from the meta-agent.Source: §1 · p. 3

2. Problem Formulation

The paper frames interface evolution as revising a persistent repository of executable interfaces and supporting knowledge using task trajectories. The objective is to improve an agent’s expected task score across a target task distribution.

2. Problem Formulation

  • A repository contains executable interfaces and supporting knowledge such as capability descriptions, reusable workflows, configuration, and usage notes, and is intended to support performance across tasks.The objective is to maximize expected task score across a target task distribution, beyond the experience used to construct the repository.Source: §2 · pp. 3–4
  • An interface connects an agent to computer components through requests and observations; the harness executes requests and places resulting observations in the model’s context.The agent chooses an interface and request based on the task instruction and interaction history.Source: §2 · p. 4
  • Task attempts are bounded by the task budget and harness permissions, and the score averages evaluator scores over the resulting interactions.Interfaces implemented as programs can combine operations, return selected information, and be redesigned by reading trajectories and writing code.Source: §2 · pp. 4–5
  • Interface evolution maps a repository and the trajectories produced while using it to a revised repository, with task experience guiding further changes.Repeatedly reconstructing calculations can motivate a reusable operation, while difficulty verifying changes can motivate a new observation.Source: §2 · p. 5

3. Agent S4

Agent S4 evolves repositories while holding the task agent fixed: it selects parent repositories, revises or combines them using training trajectories, evaluates descendants, and retains alternatives in a population. Validation affects selection but is hidden from the meta-agent during construction.

3. Agent S4

  • Agent S4 keeps the task agent’s model and harness fixed while population-based evolution selects parents, creates a descendant, evaluates it, and retains it alongside its parents.Training experience guides repository changes; held-out validation guides later selection.Source: §3 · p. 5

3.1. Population and selection

  • Binary tournament selection chooses eligible parent repositories using validation scores, and the population retains candidate records and their associated training trajectories.For crossover, the first selected parent is excluded from the second selection.Source: §3.1 · pp. 5–6

3.2. Mutation and crossover

  • The meta-agent studies parent trajectories to hypothesize useful changes; mutation adds, revises, or removes interfaces and knowledge, while crossover makes one coherent descendant from two distinct parents.Crossover can choose between overlapping interfaces, port complementary capabilities, or combine ideas; the descendant need not be the union of its parents.Source: §3.2 · p. 6

3.3. Evaluation and population update

  • Each frozen descendant is evaluated on training and disjoint validation tasks, then added alongside its parents regardless of validation score; the final repository is selected by validation performance.The meta-agent observes none of the target test tasks during construction.Source: §3.3 · pp. 6–7
  • Training trajectories have no ground-truth success labels and are exposed to the meta-agent, whereas validation tasks, trajectories, diagnostics, and scores remain inaccessible to it.Validation therefore guides parent selection without providing direct feedback for repository edits.Source: §3.3 · p. 6

4. Experimental Setup

The experiments use synthetic training tasks and official validation tasks on OSWorld V2 and ScienceBoard, with the meta-agent’s access to a scratch virtual machine varied across evolution baselines. The selected repositories are compared with fixed-interface and other repository baselines on full-benchmark tasks.

4. Experimental Setup

  • OSWorld V2 uses 30 synthetic training tasks and ScienceBoard uses 48, with 15 official validation tasks per benchmark; full-benchmark results include 108 and 169 tasks, respectively.ScienceBoard tests generalization to niche scientific computer-use applications and workflows.Source: §4 · p. 7
  • The task agent and meta-agent use Qwen3.8-27B in a modified Prime Agent harness, with model and harness fixed within each search.The harness uses a persistent IPython kernel, Medium reasoning, a 131,072-token context window, and a 32,000-token output limit.Source: §4 · p. 7
  • Evolution baselines share a seed repository, while population search alternates mutation and crossover and linear evolution mutates only the latest descendant.Population-search parent tournaments use mean partial validation score on OSWorld V2 and validation success on ScienceBoard; a scratch VM lets the meta-agent inspect applications and try interface code.Source: §4 · p. 8
  • Repository selection uses the most fully solved OSWorld V2 validation tasks, breaking ties by mean partial score, and the highest validation success rate on ScienceBoard.ScienceBoard additionally reevaluates its top three variants after every five steps; OSWorld V2 omits this denoising procedure.Source: §4 · p. 8
  • Synthetic tasks follow each benchmark’s methodology but use independently authored scenarios, and task-agent attempts are not graded or given reference solutions.The ScienceBoard synthetic set was planned across six scientific applications; OSWorld V2 task targets reflected its challenge-phenomenon taxonomy.Source: §4 · p. 9

5. Main Results

On OSWorld V2, Agent S4’s evolved interfaces raise Qwen3.8-27B’s binary success above both GPT-5.5 and Claude Opus 4.8 at a fraction of their costs. The same model also substantially outperforms its fixed-interface baseline.

  • 21.30% binary success at $1.10 per task exceeds GPT-5.5’s 13.00% and Claude Opus 4.8’s 20.60% on full OSWorld V2.Those frontier-model results cost $25.50 and $72.40 per task, respectively; the comparison is with the repository evolved without a scratch VM.Source: §5 · p. 9, Table 2 · p. 10
  • With a scratch VM, the evolved repository reaches a 51.06% partial score on OSWorld V2.Binary success measures complete task execution, while partial score measures intermediate progress.Source: §5 · p. 9
  • Binary success rises from 5.56% to 21.30% and partial score from 20.69% to 46.90% when Qwen3.8-27B uses evolved rather than fixed interfaces.Task-solving cost falls from $1.39 to $1.10 per task.Source: §5 · p. 10

6. Generalization

Agent S4’s fresh evolution run transfers to ScienceBoard, a benchmark spanning six scientific applications. The evolved repository improves success over the seed repository and adds application-specific interfaces and tools.

6. Generalization

  • ScienceBoard covers 169 tasks across six scientific applications, including astronomy, molecular visualization, geographic analysis, theorem proving, and typesetting.The listed applications are Celestia, ChimeraX, GRASS GIS, KAlgebra, Lean, and TeXstudio.Source: §6 · p. 10
  • The meta-agent learns from 48 synthetic training tasks, while 15 validation tasks guide selection over 30 construction steps.Source: §6 · p. 11
  • Success rises from 55.03% with the seed repository to 58.58% with the evolved repository, solving 99 rather than 93 of 169 ScienceBoard tasks.Average model calls per task decline from 61.7 to 58.9.Source: Table 3 · p. 10, §6 · p. 11
  • The selected repository grows from 15 to 64 functions, adding a ChimeraX command interface, live-session GRASS GIS modules, and a multi-expression KAlgebra calculator.Source: §6 · p. 11
  • The gains are largest on KAlgebra, where success rises from 48% to 68%.Source: §6 · p. 11

7. Analysis

Evolution improves repository performance while expanding the interfaces available to agents. The analyses compare evolution strategies, describe the resulting interfaces, and show why repeated validation matters.

7.1. Evolution Dynamics

  • Evolution without a scratch VM raised binary success to 21.30% and partial score to 46.90% at $1.10 per task, improving on the seed repository’s 19.44% binary success and $1.11 cost.The seed repository raised binary success from 15.74% without a repository while leaving partial score nearly unchanged.Source: §7.1 · p. 11
  • 21.30% binary success was highest without a scratch VM; with a scratch VM, population search matched linear evolution at 19.44% but scored 51.06% versus 47.17% partial.The table compares the seed repository and evolution strategies on the full OSWorld V2 benchmark.Source: §7.1 · p. 11, Table 4 · p. 11

7.2. Qualitative Study of Evolved Interfaces

  • Without a scratch VM, evolution added four interfaces and 127 extensions; with one, it added 12 interfaces and 253 extensions, including more cross-app extensions.The counts by interface label can overlap; the authors suggest the additional application interfaces likely explain the higher partial score with a scratch VM.Source: §7.2 · p. 12
  • ScienceBoard evolution added seven interfaces and 91 extensions, mostly app-specific capabilities for scientific software, plus task-specific procedures for GRASS GIS and Lean.The three task-specific extensions package procedures for displaying a GRASS GIS map and completing a Lean proof.Source: §7.2 · p. 12
  • Across runs, evolved interfaces support observing and acting on desktops, application objects, shared cross-app data, and goal completion in a single call.The browser case study shows browser.api retrieving mailbox and bank data as text, with curl and pdftotext used to fetch and read receipt PDFs.Source: §7.2 · pp. 13–14, Figure 5 · p. 13

7.3. Repeated Validation during Evolution

  • Repeated validation changed repository rankings: if-016 averaged 71.67% across four evaluations, while if-003 fell from 73.33% initially to a 60.00% mean.The three highest-scoring repositories were reevaluated every five steps because long-horizon task scores vary between runs.Source: §7.3 · p. 14

8. Related Work

Related work studies how interaction design, reusable skills, and automated agent design shape computer-use systems. Agent S4 differs by evolving complete interface repositories across tasks and selecting them with held-out validation.

Automated agent design and evolutionary search

  • Figure 6 contrasts best first-evaluation scores with best mean scores over repeated ScienceBoard evaluations.Repeated evaluation separates a high first score from consistent performance.Source: Figure 6 · p. 14

Computer-use agents and interaction design

Skills, memory, and tool construction

  • Prior work establishes reusable textual feedback, procedures, and executable tools; Agent S4 searches complete repository versions whose components can change together [Shinn et al., 2023; Wang et al., 2025b; Cai et al., 2024; Zheng et al., 2025; Chen et al., 2026].Its repositories combine interfaces, observation routines, configuration, and usage knowledge.Source: §8 · p. 14

Automated agent design and evolutionary search

  • Agent S4 combines population search with asymmetric access to evaluation evidence: the meta-agent studies synthetic-task trajectories without success labels, while held-out validation selects repository variants.The task harness remains fixed and retains control of model invocation, execution, permissions, and budgets.Source: §8 · p. 15

9. Conclusion

Agent S4 evolves reusable computer-use interfaces and usage knowledge from synthetic-task trajectories, then uses held-out validation to select repositories. With the task agent’s model and harness fixed, the evolved repository achieves strong results on OSWorld V2 and ScienceBoard.

9. Conclusion

  • 21.3% binary success on full OSWorld V2 exceeded GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their respective task-solving costs.The benchmark contains 108 tasks.Source: §9 · p. 15
  • 58.58% success on full ScienceBoard compared with 41.4% for Qwen CUA.This was a separate evolution run on the 169-task benchmark.Source: §9 · p. 15
  • The findings support learning interfaces from synthetic experience as a way to improve computer use, complementary to advances in models and harnesses.The task agent’s model and harness stay fixed while the repository evolves.Source: §9 · p. 15

A. Experimental Design

The experiments evaluate repository evolution on OSWorld V2 and ScienceBoard, using held-out validation and fixed task-agent interfaces for reference comparisons. The settings describe the harness, repository access, model service, and runtime budgets.

A.1. Benchmark Detail

  • OSWorld V2 covers 108 long-horizon desktop and web workflows; ScienceBoard covers 169 human-curated scientific tasks involving professional software and graphical or command-line interfaces.The benchmarks assess complete task execution, with OSWorld V2 also reporting partial scores.Source: §A.1 · p. 19

A.2. Settings

  • Evolution used all 30 authored OSWorld V2 synthetic tasks and 48 of 50 ScienceBoard tasks; each benchmark used 15 official validation tasks.Two ScienceBoard tasks were excluded because initialization failed before task-agent execution.Source: §A.2 · p. 19
  • External reference results use the OSWorld V2 screenshot-and-action harness, while Qwen CUA uses structured mouse, keyboard, and screenshot actions and Prime Agent exposes persistent code execution.Reference interfaces are fixed; Prime Agent conditions are extensible, with repository interfaces added through the same tool.Source: §A.2 · p. 19
  • Prime Agent executes model-issued Python in a persistent IPython kernel, preserving variables, imports, and function definitions across calls; automatic harness refinement is disabled.Task-specific harness notes are deleted before each task, leaving the repository as the only state carried between tasks.Source: §A.2 · pp. 19–20
  • Repositories are installed at a fixed path and invoked through capabilities, documentation, and call commands; all evolution runs begin from a shared seed of 15 basic interfaces.The seed contains about 550 lines of Python and shell and has no benchmark-application-specific interfaces.Source: §A.2 · p. 20, Table 5 · p. 21
  • Qwen3.8-27B serves as task agent, GUI grounder, and interface builder; OSWorld V2 uses a 300-response task budget, while ScienceBoard uses a 60-minute wall-clock limit.Reported model-call cost estimates assume an 80% input-cache hit rate and exclude repository-evolution and infrastructure costs.Source: §A.2 · pp. 20–21

B.1. Evolution Lineage

Population evolution on OSWorld V2 and ScienceBoard retains and recombines repository branches using validation performance to select promising variants. The best reported variants improve on their seeds, although performance does not rise monotonically.

B.1. Evolution Lineage

  • 50.2% mean partial score at OSWorld V2 step 24, versus 35.8% for the seed; its 26.7% binary success ties the best variants, versus 6.7% for the seed.Crossover combines repositories from steps 2 and 6; the two validation metrics distinguish different aspects of performance.Source: §B.1 · p. 21, Figure 7 · p. 22
  • Earlier OSWorld V2 branches remain useful beyond their construction step, and repositories selected by partial score need not lead on binary success.At step 24, the partial-score high is tied on binary success by steps 5, 25, and 29.Source: §B.1 · p. 21
  • 71.7% mean ScienceBoard validation success for if-016 over four evaluations of 15 tasks, versus 53.3% for the seed’s initial evaluation.if-016 crosses if-014 and if-009; no later variant exceeds its mean in the 30-step run.Source: §B.1 · pp. 21, 23, Figure 8 · p. 23

B.2. Execution Efficiency

The evolved repository uses fewer calls on its full-credit OSWorld V2 tasks, but aggregate call counts remain similar. Its examples show interfaces that combine observations and operations in reusable calls.

B.2. Execution Efficiency

  • 7.1% fewer tool calls on average for the evolved repository’s full-credit tasks, while it receives full credit on more tasks.Full-credit subsets differ between repositories; the all-task mean remains similar, so this is not a uniform speedup on matched tasks.Source: §B.2 · pp. 23–24, Table 6 · p. 24
  • On the 11 tasks where both repositories receive full credit, mean calls rise from 87.8 to 103.2; excluding an interrupted evolved attempt, all-task means are 163.2 and 163.4.The shorter full-credit average describes each repository’s successful trajectories, not a uniform reduction across the same tasks.Source: §B.2 · p. 24, Table 6 · p. 24

B.3. Examples of Evolved Interfaces

  • Evolved interfaces change what the agent can observe and do in one call, including extensions to seeded desktop-control and web-app interfaces and newly added interfaces.A fixed-interface agent instead observes through screenshots and acts through clicks and keystrokes.Source: §B.3 · p. 24

B.4. Additional Case Study

The client photo-delivery case contrasts a seed repository that builds browser access during the task with an evolved repository that reuses browser helpers. The evolved trajectory reads structured page data and shows reduced setup overhead, though the case does not isolate the effect of CDP alone.

B.4. Additional Case Study

  • The case task requires reading TeamChat instructions, reconciling VaultBank payments, and preparing and emailing finished photos or watermarked previews.The seed provides browser target discovery but lacks a page-evaluation helper.Source: §B.4 · p. 24
  • Reusable helpers expose page text, page evaluation, and calls to the page’s own backend, with backend requests carrying the page’s cookies, headers, and origin.The browser interface includes page_text and page_eval; the API helper is described as avoiding reimplementation of authentication.Source: §B.4 · pp. 24–25
  • The evolved agent retrieves the requested TeamChat message at tool cell 11 using existing helpers, while the seed builds and debugs its own CDP client and retrieves it at cell 37.The evolved interface encapsulates target selection and WebSocket communication, moving browser setup and debugging out of the task trajectory.Source: §B.4 · p. 24, Figure 11 · p. 27
  • The evolved agent loads additional bank transactions and extracts 81 table rows from the page DOM, then aggregates payments by client name in Python.Returning page text and structured records supports bulk reading through the same browser session.Source: §B.4 · p. 27
  • The evolved trajectory is shorter, with fewer tool calls and image-bearing results, alongside a higher task score; reduced browser setup and direct page access are the clearest mechanisms described.Both agents eventually use CDP, and other repository interfaces and file, image, and email operations contribute to the observed case outcome.Source: §B.4 · p. 27, Table 8 · p. 27

B.5. Tasks With and Without Evolved Interfaces

Selected cases contrast GUI-heavy seed trajectories with evolved helpers that package application operations, while the accompanying harness prompts direct builders to use synthetic-task trajectories without treating dummy scores as success labels. These examples illustrate mechanisms rather than controlled effects.

B.5. Tasks With and Without Evolved Interfaces

  • The episodes were selected to illustrate mechanisms, not as controlled comparisons; synthetic s_* tasks are ungraded, and several evolved repositories come from separate runs.Call counts exclude interface-building cost.Source: §B.5 · p. 28
  • In MailHub, if-011 uses page_text and one cdp_eval call to fetch the inbox with full email bodies in 12 calls, versus 60 GUI calls for the seed.The seed’s early clicks miss the email list, and its code attempt to read the page is refused with 403 Forbidden.Source: Figure 12 · p. 28
  • In XMind, if-029 builds and saves the map with helper functions, then renders a PNG in one call; the seed’s menu attempt ends at the call limit without an image.The seed’s export click opens a JPEG panel rather than PNG.Source: Figure 13 · p. 29
  • Other cases show application-specific helpers: Zotero lists collections and items in one call, ReviewSphere submits a review through browser.chrome_js, and Word accepts eight and rejects six tracked changes in one call.The Zotero interface is read-only on the described build; its API rejects writes.Source: Figure 14 · p. 30, Figure 15 · p. 31, Figure 16 · p. 32
  • In ScienceBoard cases, evolved KAlgebra and GRASS GIS helpers reduce model steps while both seed and evolved episodes pass; the calculator case also requires a tab switch and retry to confirm p=1.58496250072.The plotting task takes 68 versus 11 steps, and the calculation task 40 versus 21; the raster-statistics task also takes 40 versus 21.Source: Figure 18 · p. 34, §B.5 · p. 34, Figure 19 · p. 35

C. Harness Prompts

  • The builder prompt says synthetic tasks have no verifiers and dummy evaluators return 0.0, so those values must not be treated as success or reward.Builders are instructed to inspect trajectories for repeated work, missing capabilities, and fragile workflows, distinguishing observed evidence from hypotheses.Source: C. Harness Prompts · p. 37
  • The prompts prioritize general helpers and clear signatures and docstrings so agents can discover and choose interfaces; mutation and crossover instructions are fixed by the harness.Crossover combines capabilities from two parent repositories, choosing between differing approaches according to evaluation results.Source: C. Harness Prompts · pp. 37–38

D. OSWorld V2 Synthetic Training Tasks

The OSWorld V2 synthetic training tasks span desktop applications and local web workflows, exposing interaction failures through varied, constraint-rich work without labeling agent attempts. Across applications, they ask agents to produce finished, editable deliverables and, in some cases, reconcile source evidence or validate outputs before handoff.

D. OSWorld V2 Synthetic Training Tasks

  • 30 synthetic tasks—21 desktop and nine involving local mock websites—provide construction experience across document, spreadsheet, media, modeling, research, file-management, and web workflows.Tasks include an instruction, input materials, and an initial application state. Authors and independent reviewers assessed task design, while a constant-zero evaluator served only as a runner-compatible placeholder; held-out validation, not training success labels, determines selection.Source: D. OSWorld V2 Synthetic Training Tasks · pp. 38–39, Table 9 · p. 39

D.1. Task Instructions

  • The instructions span publication and media editing, resource planning, and technical audio work, each requiring agents to reconcile supplied materials and produce editable deliverables.Examples include updating a two-page visitor guide, editing a volunteer thank-you video, preparing a repair handoff, and shaping a rehearsal mix.Source: §D.1.1 · p. 40, §D.1.2 · p. 41, §D.1.3 · p. 41
  • Research and evidence tasks require structured decisions from source packets, including separating evidence, permissions, and release states, or curating records and evaluating claims.The seed-library task also requires dependency relationships between decision branches; the reminder-evidence handoff distinguishes independent evidence from related analyses and association from causation.Source: §D.1.5 · p. 41, §D.1.7 · p. 43
  • Technical and creative production tasks include plotting a cartridge-screening study, resolving enclosure clearances, preparing linked musical parts, and creating a true-size return mat.The screening task specifies a 3.00 L/min duty window and conservative one-standard-uncertainty thresholds; the enclosure task calls for editable Blender modeling and review renders.Source: §D.1.6 · p. 42, §D.1.8 · p. 43, §D.1.9 · p. 43
  • Administrative scenarios require agents to reconcile records and act within stated authority across supplier correspondence, CRM handoffs, and a simulated company’s migration intake.The tasks include preserving history, distinguishing planning or filing from acceptance, and updating customer-facing company information and communications.Source: §D.1.10 · p. 44, §D.1.11 · p. 44, §D.1.13 · p. 45
  • Other instructions test document repair, formula-driven spreadsheet modeling, and a single irreversible diversion-form submission conditioned on checking a final emailed inspection result.The handbook task preserves supplied prose while repairing structure and references; the cup model exercises cases and input changes, while the diversion task keeps the form open during consultation and bars a second application.Source: §D.1.14 · p. 45, §D.1.15 · p. 45, §D.1.16 · p. 46

D.1.17–D.1.18. Offline field kit and gripper demonstration

  • The field-kit task requires repairing and packaging an offline inspection kit, while the animation task requires building and checking a silent gripper demonstration.The kit must preserve issued route content, include linked companions and a manifest, and pass extraction checks. The animation must show grasp, transfer, set-down, and release in an editable Blender scene.Source: §D.1.17 · p. 46

D.1.19. Explain Loopline’s accessible episode release

  • The Obsidian task asks for a non-hierarchical producer induction board that explains episode release, parallel work, approvals, correction loops, and listener deliverables.It must retain access to the three source notes, use labeled groups and connections, and keep the normal route and correction loops legible without relying on color alone.Source: §D.1.19 · p. 47

D.1.20–D.1.21. Recording-day calendar and scan-station help

  • The calendar and presentation tasks require reconciling a recording-day schedule and building an interactive scan-station guide, then checking the resulting handoffs.The calendar task calls for evidence-based local event edits under operating rules; the presentation task requires exercising all navigation branches and documenting checked routes and unverified behavior.Source: §D.1.20 · p. 47, §D.1.21 · p. 47

D.1.22–D.1.23. Learning-study review and request form

  • The review and form tasks require a sourced assessment of a fictional learning study and a reusable, accessible, fillable equipment-request form.The review must include numerical comparisons, distinguish resolvable questions from research needs, and be confirmed after submission. The form must be tested in both editable and PDF formats, with temporary answers removed.Source: §D.1.22 · p. 48, §D.1.23 · p. 48

D.1.24. Explain Mere Studio’s spring-cart damping comparison

  • The video task requires an editable, silent comparison of two simulated damping recordings, with synchronized releases, matched physical scale, a half-speed replay, and a qualified observation.The prompt specifies an 18–24 second, 1280×720, 30 fps video; the footage is illustrative rather than laboratory measurement, and the ending must discuss overshoot and settling without inventing times or claiming certification.Source: §D.1.24 · pp. 48–49

D.1.25–D.1.26. Interruption report and editorial review

  • The interruption-report and editorial-review tasks require auditable, source-grounded analysis while preserving editable native work for verification or further review.The Calc report must reconstruct episodes overlapping the reporting window and check pivot totals; the Writer review must adjudicate inherited revisions and leave new tracked changes and anchored comments reviewable.Source: §D.1.25 · p. 49, §D.1.26 · pp. 49–50

D.1.27–D.1.28. Drying-trial assessment and listening outing

  • The trial-assessment and outing tasks ask agents to evaluate supplied evidence for an engineering recommendation and complete a six-person local mock-event booking.The trial report must include numerical evidence, limitations, selected-run line charts, and a proposed next experiment; the outing task requires comparing events and seats before ordering within the mock checkout timer.Source: §D.1.27 · p. 50, §D.1.28 · p. 50

D.1.29–D.1.30. Rain pathways and individualized home-practice letters

  • The rain-poster and home-practice tasks require editable visual and personalized teaching deliverables grounded in supplied briefs and learner records.The poster needs three grouped, editable pathway diagrams in Draw. The letters workflow must retain learner states and evidence, apply consent and incomplete-observation rules, and produce a reusable database-backed merge master plus ordered recipient files.Source: §D.1.29 · p. 50, §D.1.30 · pp. 50–51

E. ScienceBoard Synthetic Training Tasks

This section describes 48 synthetic ScienceBoard tasks used to give Agent S4 construction experience across six scientific applications, with trajectories recorded without success labels, including numerical and graphing work in KAlgebra, Lean proof completion, and TeXstudio document production.

E. ScienceBoard Synthetic Training Tasks

  • 48 tasks span Celestia, ChimeraX, GRASS GIS, KAlgebra, Lean, and TeXstudio; two authored tasks were excluded because initialization failed before agents started.Tasks had instructions, initial application states and, where needed, input materials; the meta-agent examined attempts without evaluators or success labels.Source: E. ScienceBoard Synthetic Training Tasks · p. 51, Table 10 · p. 51

E.1. Task Instructions

  • Celestia tasks probe viewpoint-dependent lunar phases, changing daylight, saved observing states, scale and motion comparisons, stellar surveys, and horizon geometry.The instructions require agents to manipulate views or simulation state and support comparisons with displayed readings or geometric reasoning.Source: §E.1.1 · p. 52, §E.1.2 · p. 53, §E.1.3 · p. 53
  • ChimeraX tasks use explicitly invented point geometries to test scaffold alignment, alternate-location comparison, residual-and-coverage screening, and cross-chain contact mapping.The tasks request measurements or selections while preserving the supplied coordinates and, in some cases, saving a native comparison.Source: §E.1.11 · p. 55, §E.1.12 · p. 55, §E.1.13 · p. 55
  • Other ChimeraX tasks require sampled-trajectory distance analysis and snapshot export, fitted-plane and probe measurements, or anchored torsion editing.The instructions distinguish sampled frames from unsampled motion, specify fitting and measurement conditions, and require preserving input geometry where applicable.Source: §E.1.15 · p. 56, §E.1.16 · p. 56, §E.1.17 · p. 56
  • GRASS GIS tasks cover spatial access and raster contrast, station filtering, resampling, terrain derivatives, travel-time reach, and connected habitat patches.They specify data semantics and boundary conditions, including preserving valid zeros, survey gaps, grid alignment, and inclusive thresholds.Source: §E.1.18 · p. 57, §E.1.19 · p. 57, §E.1.20 · p. 57
  • Further GRASS GIS tasks require retaining jointly surveyed cells for cut-and-fill accounting and classifying terrain visibility from two observation posts.The visibility task uses native viewsheds and reports the plot area visible from both posts.Source: §E.1.25 · pp. 58–59, §E.1.26 · p. 59
  • The supplied KAlgebra tasks ask agents to minimize an open-channel coating cost, plot a delayed actuator profile, and construct a corrugated-sheet height surface.They require a positive-width minimum, complete piecewise profiles, and a 3D function reflecting specified crest orientation and spacing.Source: §E.1.27 · p. 59, §E.1.28 · p. 59, §E.1.29 · pp. 59–60

E.1.30–E.1.32. KAlgebra tasks

  • The KAlgebra tasks cover ambiguous calibration, vector decomposition, and finding a dilution threshold cycle.E.1.30 asks for all admissible inputs and the real domain of a calibration curve; E.1.31 asks for parallel and perpendicular velocity components; E.1.32 asks for the first cycle whose tracer concentration falls strictly below a threshold.Source: §E.1.30 · p. 60, §E.1.31 · p. 60, §E.1.32 · p. 60

E.1.33–E.1.35. KAlgebra tasks

  • Further KAlgebra tasks require correcting for recording bias, constructing a focus-defined reflector, and fitting a colorimeter calibration.E.1.33 seeks expected kinetic energy after detection-weighted sampling; E.1.34 requires graphing a reflector defined by equal distances from focus and directrix; E.1.35 fits an affine model with a free intercept and estimates an unknown concentration.Source: §E.1.33 · p. 60, §E.1.34 · p. 61, §E.1.35 · p. 61

E.1.37–E.1.40. Lean tasks

  • The Lean tasks formalize a weighted inequality, rollback for shifts and reflections, compatible projections, and preservation of greatest common divisors.The proofs must retain supplied theorem statements and definitions and eliminate unfinished proofs; the weighted bound also characterizes equality, while the projection task identifies settle outputs with states fixed by both functions.Source: §E.1.37 · p. 61, §E.1.38 · p. 62, §E.1.39 · p. 62

E.1.41–E.1.42. Lean tasks

  • Additional Lean tasks prove a route-cost formula after outlet replacement and characterize when adjacent integer scans cover a window without repeats.The route-cost claim applies to all tree shapes and link costs, including zero; the scan equivalence must hold for arbitrary integer endpoints, including negative ones.Source: §E.1.41 · p. 62, §E.1.42 · pp. 62–63

E.1.44–E.1.46. TeXstudio tasks

  • TeXstudio tasks produce a normal-mode note, a torsion figure and caption, and a density note with propagated measurement uncertainty.The coupled-cart task calculates and checks two modes; the toy-chain task reports a signed torsion relative to ideal references; the cylinder task combines measurement and calibration uncertainty and identifies the largest contribution.Source: §E.1.44 · p. 63, §E.1.45 · p. 63, §E.1.46 · pp. 63–64

E.1.47–E.1.48. TeXstudio tasks

  • The remaining TeXstudio tasks make cross-references survive renumbering and replace manual citations and references with a generated numeric bibliography.The exchange note uses shared automatic numbering for paired balances and a separate conservation equation; the thermometer lesson uses key-based citations and BibTeX while preserving uncited records in the supplied bibliography file.Source: §E.1.47 · p. 64, §E.1.48 · p. 64

E.1.36. sb_kalgebra_010 Compare interior and transition errors in harmonic reconstruction

The KAlgebra task asks for signed reconstruction errors at interior and near-transition times for two harmonic counts. The Lean task asks for a finite-step disturbance-bound proof.

E.1.36. sb_kalgebra_010 Compare interior and transition errors in harmonic reconstruction

  • For N = 4 and N = 12, calculate signed voltage errors at t = T/4 and t = T/(4*N), then compare how the errors change with more harmonics.Report all four errors to one decimal place and retain the calculations in KAlgebra’s Calculator history.Source: §E.1.36 · p. 61

E.1.43. sb_lean_007 Bound the accumulated error of a disturbed iteration

  • Prove the finite-step error bound around p for every natural-number step, including step zero, under disturbances of magnitude at most 𝜀 and a contraction factor 0 ≤ q < 1.The disturbances may vary in magnitude and sign at each step; q = 0 and 𝜀 = 0 are allowed.Source: §E.1.43 · p. 63
Loading 2610.28601v1…