Contract monitoring: governing AI via separation of powers

1 Introduction

• The framework addresses contract specification, scalable compliance checks, and worker optimization while targeting scalable safety guarantees for settings such as code security and escape-the-box.The motivating questions include how much compute compliance checking needs relative to worker training and deployment.Source: §1 · pp. 1–2, §1.1 · p. 2

1.1 Our contributions

• The principal specifies a contract through a report judge that accepts or rejects evidence of a violation, so compliance means no valid violation report exists.This lets the principal specify how to verify purported violations rather than provide an efficient procedure for directly checking every action sequence.Source: §1.1 · p. 3
• A monitor acts as a prosecutor, generating a violation report for the judge; adversarial self-play trains it to find violations while workers try to evade detection.The judge automatically evaluates reports, providing a training signal without continual human oversight.Source: §1.1 · pp. 2–4
• Validation against adversarial workers trained with bounded compute estimates evasion risk and whether a frozen monitor is strong enough for deployment.The framework can help determine how much monitor training is needed for a contract and whether to amend a contract that is too costly to enforce.Source: §1.1 · p. 4, Figure 2 · p. 4

1.2.1 Constitutional AI and direct monitoring

• Unlike constitutional AI, contract monitoring separates enforcement from the worker’s learned behavior, and unlike direct monitoring, it separates the safety specification from enforcement.Monitor failures become search failures against an independently specified contract; adversarial self-play can train against independently verifiable evasion.Source: §1.2.1 · pp. 5–6

1.2.2 Verifiable AI and AI debate

• Contract monitoring reverses verifiable AI’s proof obligation: compliance requires that no judge-accepted violation report exists, giving a coNP characterization under polynomial restrictions.The paper also contrasts statistical validation of monitors with debate, where a favorable verdict on an observed transcript does not establish the intended quantified claim.Source: §1.2.2 · pp. 6–8

2.1 Defining the contract with a report judge

• With bounded report length, exhaustive search can find an accepted report whenever one exists; bounded-compute monitors can instead receive statistical safety guarantees against bounded-compute adversaries.The judge must faithfully express the principal’s intended safety boundary; monitor misses then constitute search failures relative to that independently specified contract.Source: §2.1 · pp. 8–9
• A judge may only approximate the intended boundary, so the principal can require the allowed-action set to lie between a strict contract and a loose contract known to be safe.The stricter set must still leave enough actions for the worker to perform its task.Source: §2.1 · p. 9

2.2 Examples: contracts defined by judges

• A judge can also assess evaluator changes, unauthorized permissions, or a high-level argument that accumulated access and influence create a credible route to prohibited control.For dangerous positioning, the principal sets the prohibited outcomes and caution level; the report need not provide an executable exploit or complete low-level plan.Source: §2.2 · pp. 9, 11
• Examples use concrete violation evidence: differing public outputs for two secrets, a forbidden sandbox state within a horizon, or a prohibited path in a logged provenance graph.The judge runs the submitted program on paired inputs, replays sandbox inputs, or checks path endpoints and recorded transfers; each check concerns its specified outputs, environment, or logs.Source: §2.2 · p. 10

2.1 Defining the contract with a report judge

• A judge defines a contract by accepting reports that demonstrate violations, so the boundary is specified separately from the monitor’s ability to find them.The principal may instead sandwich the judge-defined contract between a stricter set of allowed actions and a looser set known to be safe.Source: §2.1 · p. 9, §2.2 · p. 10

3 Training a contract monitor with self-play

• In each interaction, an adversary produces an action transcript and private violation report, while the monitor sees the transcript and attempts its own report.The judge checks both reports, so evasion requires an accepted adversary report and a monitor failure to find an accepted report.Source: §3 · pp. 11–12
• The adversary maximizes successful evasion while the monitor minimizes it, forming a zero-sum game trained through adversarial self-play.The paper compares this training setup to the GAN framework.Source: §3 · p. 12
• An unrestricted monitor can achieve zero evasion against every adversary by finding an accepted report whenever the adversary supplies one.The authors state that ideal monitoring is not necessary for safety and propose determining the self-play training needed for safe deployment.Source: §3 · p. 12

• Validation tests a frozen monitor against adversarial workers trained with increasing RL compute to find violations that evade it while supplying valid self-reports.Runs hold environment, tool access, and inference budgets fixed; adversaries are evaluated on held-out interactions across independent runs.Source: §4 · pp. 12–13
• If an RL adversary is truly optimal at its compute level, its observed failure supports deploying the monitor against workers from the same base model trained with up to that compute.The allowed evasion probability is set by the principal.Source: §4 · pp. 12–13
• The procedure experimentally estimates how much monitor training a contract needs, and an excessive compute requirement can motivate amending the contract to make it easier to compute.Source: §4 · p. 13
• The guarantee applies only when workers are aware of their contract violations; blind violations are ruled out only under the stated assumption about the relative difficulty of hiding a violation and producing an accepted report.The paper expects this assumption to hold in hacking scenarios.Source: §4 · p. 13
• Scaling-law extrapolation may estimate very small evasion probabilities that red-teaming cannot measure directly, though the proposed relationship remains to be established empirically.Figure 3 depicts an idealized relationship between monitor self-training and deployment against an RL-trained worker.Source: §4 · p. 13, Figure 3 · p. 13

5 Discussion

• The framework trains monitors through self-play and uses validation against adversarial workers to bound monitor evasion after deployment against RL-trained workers.Validation tests whether adversarial workers can evade the monitor before deployment.Source: §5 · p. 14
• The authors characterize collusion during validation or deployment as possible only through monitor jailbreaking, and say validation should flag both this and attempts to disable monitoring.During deployment the monitor is fixed and only post-processes worker actions; contracts can also prohibit hacking activities.Source: §5 · p. 14
• Separating judge and monitor makes the enforced contract objective, but monitors may be incentivized to alter the environment so workers violate it.The discussion says validation should reveal this behavior and recommends contracts enforceable by a less capable, more trusted monitor.Source: §5 · p. 14
• Validation may miss worst-case adversaries, so the authors recommend a wide safety margin and regularly branching adversarial workers from deployed workers.Branching tests whether workers gain new insight into evasion after learning through task completion.Source: §5 · pp. 14–15
• Judge LLMs may fail to verify contracts faithfully if prompt attacks such as model hypnosis exploit meaning-preserving changes with distributed signals.Robust AI judges could potentially use white-box access to worker activations to catch otherwise missed violations.Source: §5 · p. 15
• Training adversarial workers carries a risk of teaching bad actions, mitigated by self-reporting and contracts whose violations are not themselves directly catastrophic.For catastrophe-prevention contracts, a breach should open possibilities of catastrophe only if left unreported.Source: §5 · p. 15
• Ambiguous contract cases can be escalated to a more nuanced contract or human, while contracts should prohibit workers from training themselves to evade monitoring.Validation is also proposed to detect simulated adversarial co-training when its adversaries are worst-case.Source: §5 · p. 15
• Bounding rare evasion probabilities through modular contracts remains an open direction, since reusing monitors across workers may require validation compute at least equal to deployment compute.The framework is phrased for single workers; multi-agent extensions are future work.Source: §5 · pp. 15–16

AI Use

• AI helped draft and edit some sections from conceptual bullet points, and those contributions were subsequently heavily edited.Source: AI Use · p. 16
• AI was used for literature search but not for the paper's conceptual contributions, except to help devise contract examples in Section 2.2.Source: AI Use · p. 16
• AI generated the idealized visualization of scaling laws in Section 4.Source: AI Use · p. 16