Interlock: From Local Correctness to Task Completion in Long-Horizon Agents

1 INTRODUCTION

• Task completion depends on relationships among environmental evidence, consequential decisions, and required outcomes, since a correct decision can still leave necessary consequences unfinished.Evidence and obligations may need to be carried through many intervening reads, computations, and edits.Source: §1 · p. 1
• INTERLOCK treats the complete set of conditional obligations for an outcome as its organizing unit, determining both unresolved work in the environment and evidence needed to accept results.The introduction situates this approach alongside prior work on task synthesis, environment generation, and consistency among instructions, environments, and verifiers.Source: §1 · p. 2
• INTERLOCK compiles the graph into a workspace, task specification, and rubric, then reviews whether typed relations survive generation and whether criteria are supported by solver-visible materials.Completion-filtered training retains executions that realize every applicable effect.Source: §1 · p. 2
• Both backbones improve across external benchmarks, with AutomationBench task pass rate gains of 4.00–4.50 percentage points, JobBench weighted score gains of 3.20–6.64 points, and Harvey LAB criterion pass rate gains of 3.89–4.16 percentage points.The evaluation uses completion-filtered fine-tuning on each backbone’s own accepted rollouts.Source: §1 · pp. 2–3
• A typed completion graph links evidence to supported decisions through grounding, jointly constrained components through coupling, and decisions to required downstream effects through realization.Early findings can constrain work many interactions later, while revised decisions can require coordinated changes to existing artifacts.Source: §1 · p. 2

2 RELATED WORK

• Tool-use and task-synthesis work spans verified tool-use data, generated environments and scenarios, extended interactions, composed tasks, and curricula derived through exploration.Examples include ToolAlpaca (Tang et al., 2023), APIGen (Liu et al., 2024), AgentGen (Hu et al., 2025), and BUTTON (Chen et al., 2025).Source: §2 · p. 3
• A separate line aligns generated instructions, environments, and verifiers, including approaches based on completion tests, workflows, execution checks, or common constraint programs.Examples include Endless Terminals (Gandhi et al., 2026), RST (Li et al., 2026b), CodeMidas (Ye et al., 2026), and Anchor (Ivanov & Rana, 2026).Source: §2 · p. 3
• Graph-guided approaches organize generation around tool connectivity, grounded queries, or skill paths, whereas INTERLOCK constrains synthesis through evidence-grounded decisions, joint conditions, and required downstream effects.The comparison names ScaleEnv (Tu et al., 2026), EnvFactory (Xu et al., 2026b), and SkillSynth (Fan et al., 2026).Source: §2 · p. 3

3 METHODOLOGY

• Each task packages a public request, initial workspace, and private rubric, while the solver receives only the request and workspace.Each backbone is fine-tuned on its own accepted trajectories using rejection-sampling fine-tuning.Source: §3 · p. 3, §3.4 · p. 6

3.1 TYPED COMPLETION GRAPH

• Grounding ties decisions to governing evidence and rules, coupling imposes shared conditions on components, and realization links decisions to required downstream effects.Conditional effects specify their public basis, applicable population, trigger, and required outcome.Source: §3.1 · p. 4

3.2 RELATION-CONSTRAINED TASK SYNTHESIS

• Reusable task families, archetypes, and scenario parameters define graph requirements, which specialized agents translate into a workspace, request, contract, and rubric.Generation keeps decisions unresolved, makes shared conditions constrain combined outputs, and requires observable effects and outcome checks for downstream recipients.Source: §3.2 · pp. 4–5

3.3 REQUIREMENT REVIEW AND TASK ADMISSION

• Source-coverage review traces original requirements through generated artifacts, while public-input review checks rubric support, consistency, and judgeability from the request and workspace.Admission also requires a normally terminated validation execution, complete judgments, and a sound package assessment; the validation solver need not complete the task.Source: §3.3 · p. 5

3.4 COMPLETION-FILTERED SELF-SUPERVISION

• An agent-as-judge inspects execution artifacts and workspace changes, applying rubric criteria to retain accepted trajectories for training.A trajectory is accepted only when all required criteria pass and any separately declared quality threshold is met.Source: §3.4 · p. 6

4.1 EXPERIMENTAL SETUP

• Evaluation uses held-out AutomationBench, JobBench Main/Easy, and Harvey LAB; each backbone trains on 8,957 of its own accepted trajectories, totaling about 154M supervised tokens.The benchmarks respectively report task pass rate, task-averaged weighted score, and criterion pass rate.Source: §4.1 · p. 6

4.2 MAIN RESULTS

• 4.00/4.50 pp AutomationBench, 4.35/3.20 JobBench Main, 6.64/4.31 JobBench Easy, and 3.89/4.16 pp Harvey LAB gains over original checkpoints.Values are for Qwen3.6-27B/Qwen3.6-35B-A3B, respectively; Interlock-RFT leads every Table 1 evaluation column for both backbones, also outperforming ClawGym-SFT.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 TASK DIVERSITY AND TRANSFER

• At 8,957 trajectories, expanding training coverage from 16 to 128 domains raises all four metrics at each successive setting.The 128-domain setting corresponds to the main Interlock-RFT result.Source: §4.3 · p. 7, Figure 3 · p. 7

4.4 AGENTIC VERIFICATION FOR COMPLETION-FILTERED TRAINING

• At matched budgets, agent-based filtering improves all four downstream metrics; it inspects workspace snapshots and can execute verification scripts, unlike text-only judging.Average judge output rises from approximately 11.03k to 12.28k tokens per candidate rollout, an 11.3% increase.Source: §4.4 · p. 8, Table 2 · p. 8

4.5 LONG-HORIZON EXECUTION ON SYNTHESIZED TASKS

• 84,293 tokens, 27.63 turns, and 42.10 tool calls are the INTERLOCK trajectory means, versus 14,940 tokens, 11.92 turns, and 14.76 calls for ClawGym.The figure reports mean, median, and 95th percentile for recorded tokens, assistant turns, and tool calls.Source: §4.5 · p. 8, Figure 4 · p. 9

4.6 GENERALIZATION ACROSS TASK ENVIRONMENTS

• INTERLOCK spans 128 domains and 484 task families with dispersed task-statement groups; the benchmarks are independently authored OOD tasks excluded from synthesis and RFT.The task-statement projection uses normalized BGE-M3 embeddings reduced with PCA followed by t-SNE.Source: §4.6 · p. 9, Figure 5 · p. 9

4.7 CASE STUDIES OF TASK COMPLETION

• In recovery planning, the successful attempt respects supply, shared-tray, whole-order, and action-cost constraints; the schedule attempt retains all six due trainees within a shared shift window and valid credentials.The contrasting recovery attempt exceeded supply and tray limits, partially served an unsplittable order, and omitted an expedite cost; the failed schedule omitted two trainees and violated shift and credential constraints.Source: §4.7 · pp. 9–10

• Typed completion graphs use grounding, coupling, and realization relations to constrain how evidence, decisions, joint conditions, and required effects are compiled into tasks and criteria.Source: §5 · p. 10
• Dual-view review checks graph preservation and public support, while completion-filtered RFT selects executions that realize every applicable graph effect.Source: §5 · p. 10
• Training improves both Qwen3.6 backbones across AutomationBench, JobBench Main and Easy, and Harvey LAB, supporting the value of complete synthetic execution experience for transfer.Source: §5 · p. 10

AI USE STATEMENT

• AI tools assisted manuscript preparation and task synthesis, execution, and assessment; the authors report manually reviewing AI-assisted work and taking responsibility for the final content.The statement lists prompt design, language polishing, bilingual editing, figure and caption preparation, generated task materials, and execution and evaluation records.Source: AI USE STATEMENT · p. 11

A ACCEPTANCE AND OUTCOME SCORING

• Each mandatory obligation is represented by public evidence, an applicability condition, an outcome condition, and an object scope.Applicability depends on the request, initial materials, and delivered choices; unconditional obligations have applicability equal to 1.Source: A ACCEPTANCE AND OUTCOME SCORING · p. 14

A.1 DIAGNOSTIC OUTCOME SCORING

• Diagnostic scoring propagates criterion credit through declared dependencies, excludes already-satisfied change criteria, and preserves scored preservation criteria.Critical groups account for at most half the rubric weight, and an essential-criterion failure zeros its group; partial score summarizes fulfillment, whereas acceptance requires every mandatory obligation and any declared quality threshold.Source: §A.1 · p. 14

B BEHAVIORAL DIAGNOSTIC PROTOCOL

• The behavioral protocol measures whether correct upstream decisions are followed by all applicable downstream effects, including across paired task versions with relevant public-fact interventions.A composition-transfer extension reserves selected domain–relationship combinations, and measurements include joint success, updates to affected effects, and preservation of unaffected conditions.Source: B BEHAVIORAL DIAGNOSTIC PROTOCOL · pp. 14–15

C TRAJECTORY STATISTICS AND COMPARISON

• The INTERLOCK horizon population comprises 32,370 normally terminated trajectories from 3,278 tasks, while the comparison uses all 24,497 supplied ClawGym trajectories.Each backbone independently selected 8,957 accepted trajectories from its own normally terminated rollouts for its own RFT training.Source: C TRAJECTORY STATISTICS AND COMPARISON · p. 15
• Assistant turns count recorded assistant messages, while multiple tool calls in one message count as one turn and multiple calls; retries and error responses do not add turns.Token counts use the same Qwen3.6-27B tokenizer and count recorded interaction content and one copy of the tool schema, with each trajectory equally weighted.Source: C TRAJECTORY STATISTICS AND COMPARISON · p. 15
• INTERLOCK trajectories average 84,292.51 recorded tokens, 27.63 assistant turns, and 42.10 tool calls, versus ClawGym means of 14,939.62 tokens, 11.92 turns, and 14.76 calls.The corresponding mean ratios are 5.64×, 2.32×, and 2.85×; Figures 7 and 8 show cumulative distributions and bin shares for the same trajectory pools.Source: C TRAJECTORY STATISTICS AND COMPARISON · p. 15, Figure 7 · p. 16, Figure 8 · p. 16
• Figures 7 and 8 compare cumulative horizon distributions and trajectory-length and assistant-turn bin shares using shared counting rules for both corpora.In Figure 7, curves farther right indicate larger values at the same percentile; Figure 8 plots bin shares, not probability densities.Source: Figure 7 · p. 16, Figure 8 · p. 16

D STATISTICS OF THE RETAINED TASK CORPUS

• The median retained task contains 17 source files and 30 criteria, across a corpus of 3,278 tasks spanning eight archetypes.Figure 9 summarizes archetype shares, source-workspace file counts, and rubric criterion counts; the corpus was filtered for at least one accepted rollout from each backbone.Source: D STATISTICS OF THE RETAINED TASK CORPUS · p. 17, Figure 9 · p. 17

D.1 VERIFICATION METHODS

• Content inspection accounts for 44.04% of criteria and recomputation for 40.87%, with initial–final comparison at 10.83% and text matching at 4.25%.The methods provide complementary checks of delivered content, derived results, changes from the initial state, and specified text.Source: §D.1 · p. 17, Figure 10 · p. 18

D.2 VERIFICATION ACROSS ARCHETYPES

• Verification mixes differ by archetype: content inspection is 53.6% of Design & Construct criteria, while recomputation is 56.4% for Classify, Route & Act and 58.1% for Transform & Migrate.Plan & Optimize allocates 46.3% of its criteria to recomputation; Figure 11 normalizes criterion counts within each archetype.Source: §D.2 · pp. 17–18, Figure 11 · p. 18

E TASK STRUCTURE AND EXECUTION LENGTH

• Atomic criterion count correlates with recorded trajectory tokens at ρ = 0.336 and assistant turns at ρ = 0.359, while initial file-count correlations are ρ = −0.036 and ρ = 0.041.The criterion-count correlations have 95% confidence intervals of [0.304, 0.367] for tokens and [0.325, 0.392] for turns; binned medians rise with criterion count for both measures.Source: E TASK STRUCTURE AND EXECUTION LENGTH · p. 18, Figure 12 · p. 19
• The analysis uses 32,370 normally terminated rollouts across 3,278 tasks, averaging available solver-specific medians for each task before computing correlations.Confidence intervals use 2,000 bootstrap resamples clustered by source task pack, retaining associated instances together.Source: E TASK STRUCTURE AND EXECUTION LENGTH · p. 19
• Figure 12 summarizes task-level execution measures with points, binned-median curves, and 25th–75th percentile bands; token axes are logarithmic.Its annotations report Spearman correlations and bootstrap confidence intervals clustered by source task.Source: Figure 12 · p. 19

F.1 FIELD-TRIAL PROTOCOL: DECISIONS MUST REMAIN EXECUTABLE AND INTERPRETABLE

• Attempt 1 reverses the field-trial risk condition, conflicts with an AP7 timing constraint, lacks dated resource allocations, misstates payer responsibility, and changes endpoint roles.Attempt 2 preserves the risk condition, dated activities, payer boundary, and endpoint roles while using four sites with AP5 and meeting the minimum of two sites per environment.Source: §F.1 · p. 20, Figure 13 · p. 21

F FURTHER CASE STUDIES OF SYNTHESIZED TASKS

• 22/22 for field-trial attempt 2, 41/44 for charging attempt 2, and 29/29 for payroll attempt 2; the payroll audit still finds a shared off-cycle tax error.The corresponding first-attempt counts are 15/22, 35/44, and 20/29.Source: §F.1 · p. 20, Figure 13 · p. 21, Figure 14 · p. 22

F.2 CHARGING NETWORKS: CAPACITY MUST STAY ATTACHED TO ITS RESOURCE

• The charging plan’s 53 kW PA-1 headroom calculation does not justify assigning 88.5 kW to that panel, and later repairs leave capacity and load obligations unresolved.Attempt 2 preserves nine demand dispositions but leaves SUR-118 at 612 kWh against 462 kWh planned capacity, omits existing peak loads, and retains a cost-summary mismatch.Source: §F.2 · p. 22, Figure 14 · p. 22

F.3 PAYROLL CLOSE: AMOUNTS, AUTHORITY, AND ACCOUNTING EFFECTS

• Payroll attempt 1 misstates E1004’s pending exposure, treats E1011’s refund as authorized without relevant authority, and omits E1005’s gross-cost row.Attempt 2 adds the $1,760 gross-cost row without releasing the $1,296.80 held cash; both runs nevertheless use 19.5% instead of the required 22% tax rate.Source: §F.3 · p. 23, Figure 15 · p. 23

G.1 TASK SYNTHESIS AND TRAINING ALGORITHM

• Algorithm 1 builds completion graphs, compiles and reviews task packs, admits eligible tasks, and trains backbone-specific policies on retained data.The supplied algorithm fragments identify graph construction, task-pack compilation, review and admission, and per-backbone training.Source: §G.1 · p. 24, §F.3 · p. 24

G.2 DETAILED EXPERIMENTAL SETUP

• The evaluation compares original, ClawGym-SFT, and Interlock-RFT models on three external benchmarks, whose evaluated instances are excluded from Interlock synthesis inputs and RFT data.The two backbones are Qwen3.6-27B and Qwen3.6-35B-A3B; the setting tests out-of-domain generalization.Source: §G.2 · p. 25
• The screened corpus contains 3,278 tasks; five rollout attempts per task and backbone yield 32,370 normally terminated trajectories after 410 anomalous or incomplete attempts are excluded.Each backbone independently selects 8,957 accepted trajectories from its own rollout pool for RFT.Source: §G.2 · p. 25
• Reported metrics are AutomationBench task pass rate, JobBench-Main and JobBench-Easy weighted scores, and Harvey LAB criterion pass rate; JobBench scores average task-level total_score/max_score.AutomationBench averages binary task-pass decisions. Harvey LAB pools passed binary criterion verdicts over the total evaluated across corresponding tasks and runs.Source: §G.2 · p. 26

H PROMPTS AND CONTRACTS FOR GRAPH-GROUNDED SYNTHESIS

• Appendix H presents selected translated prompt excerpts and compact data-contract summaries for Interlock’s graph-grounded task synthesis.The excerpts condense Chinese implementation templates and omit repeated self-checks and runtime boilerplate.Source: H PROMPTS AND CONTRACTS FOR… · p. 27

H.1 FROZEN SOURCE AND COMPLETION GRAPH

• A frozen source retains the full task requirements, while a typed graph declares evidence, decisions, effects, grounding, coupling, and realization relations.The graph encodes requirements rather than execution order; checks enforce reference integrity and stage coverage, including rubric coverage for every effect.Source: §H.1 · pp. 27–28

H.2 SELECTED CONSTRUCTION AND REVIEW PROMPTS

• Construction prompts preserve the shared graph across workspace, task, and rubric stages, while independent reviews check source coverage and whether rubric criteria are supported by public inputs.The source reviewer sees the frozen specification and artifacts; the public-input critic sees only the task, workspace, and rubric.Source: §H.2 · pp. 28–30
• The review prompt limits the reviewer to rubric_critique.json and supported defect guidance, requiring a clean return when no defect is supported.It prohibits editing the rubric, constructing reference deliverables, proving solvability, or predicting difficulty.Source: §H.2 · p. 30

H.3 CARRIER AND RUBRIC CONTRACTS

• Carrier mappings link graph requirements to evidence, deliverables, and rubric criteria, with code checking references and a source judge assessing preservation.Mappings are recorded in separate evidence-index, deliverable-contract, and rubric-carrier files; future solver outputs need not exist in the initial workspace.Source: §H.3 · p. 31
• The compact rubric specifies each criterion’s target, public basis, verification method, importance, and credit mode; the compiler writes it and code renders rubric.md.Private graph mappings remain in rubric_carriers.json, while criterion dependencies are distinct from the completion graph.Source: §H.3 · p. 31
• An accepted trajectory passes every required criterion and meets any separately declared quality threshold; weights and diagnostic scoring labels do not waive mandatory obligations.Source: §H.3 · pp. 31–32

H.4 REVIEW RESULTS AND TASK ADMISSION

• The source judge reports individual judgments, while code checks coverage of requirements and relation–stage pairs and requires explicit dispositions for prior findings.The report schema separates requirements, relations, findings, and reviewed findings; preservation can be marked preserved, defective, or undetermined.Source: §H.4 · p. 32
• The source result is classified as passed, defective, or undetermined, and unresolved findings require an associated disposition.The post-execution judge evaluates the public task against initial and final files and their differences without access to the source graph, carrier maps, source findings, or solver trajectory.Source: §H.4 · p. 32
• Admission requires passed source-preservation and clean public-input reviews, normal execution, complete judgments, and a sound package assessment with no outstanding defects.Confirmed defects return to their responsible generation stage; source-specification and graph defects require offline correction and keep the candidate internal.Source: §H.4 · p. 32
• Trajectory acceptance is recorded separately and is not required for task admission.The two review views share at most one generation-time rubric recompilation; post-execution revision follows a separately bounded repair budget.Source: §H.4 · p. 32

I BENCHMARKS FOR LONG-HORIZON AGENTS

• WebArena and OSWorld test realistic web and desktop tasks with execution-based outcome checks, while AppWorld and ToolSandbox emphasize stateful applications and implicit dependencies.These settings connect task success to environmental state and applicable rules.Source: I BENCHMARKS FOR LONG-HORIZON AGENTS · pp. 32–33
• WorkArena++, TheAgentCompany, Terminal-Bench, and Long-Horizon-Terminal-Bench evaluate compositional planning, workplace workflows, command-line tasks, or extended workflows with graded subtasks.Their emphasis on coordinated work and outcome evaluation complements INTERLOCK’s focus on preserving completion requirements during task synthesis.Source: I BENCHMARKS FOR LONG-HORIZON AGENTS · p. 33
• AutomationBench covers cross-application orchestration, API discovery, and policy compliance; JobBench evaluates expert-prioritized reference-workspace delegations; Harvey LAB evaluates legal work in document-rich matters.The benchmarks use their own evaluation protocols and independently authored tasks.Source: I BENCHMARKS FOR LONG-HORIZON AGENTS · p. 33