PaperlayerView exact revision on arXiv

Source-linked AI summary

Five Primitives for Governing Autonomous AI Agents at Runtime

Jiten Oswal, John Cadeddu

arXiv:2608.26696v1cs.AIcs.CRcs.SE

TL;DR

Autonomous agents defeat controls designed for onboarded or provisioned principals because they are ephemeral and choose their own actions. The paper treats governance as a runtime problem and proposes a decomposition whose implementation reports four primitives built and exercised, without measured deployment-population outcomes.

  • Problem

    Autonomous agents' ephemerality and model-selected actions defeat controls designed for principals that are onboarded or provisioned.

  • Method

    The paper frames agent governance as a runtime problem and develops a five-part decomposition of the controls required around agent actions.

  • Results

    Four of the five primitives are implemented and exercised, while the paper does not claim measured outcomes across a deployment population.

  • Takeaways & Limitations

    Runtime governance must address the properties that make autonomous agents useful but defeat controls designed for onboarded or provisioned principals.

  • Takeaways & Limitations

    Synchronous mediation adds policy, network, and durable-write costs on the critical path, while fail-closed mediation turns control-plane failures into governed-agent outages.

Abstract

from arXiv · show

Enterprise deployments of autonomous AI agents inherit a control model built for human users and long-lived services, and the fit fails in three specific ways: agent principals are ephemeral, appearing and vanishing faster than provisioning; their actions are selected by a model rather than programmed, so the set of things they may attempt is not known in advance; and the population is discovered rather than provisioned, because anyone who can call an API can create one. We argue that governing such agents is a runtime problem -- not a model-alignment problem and not a build-time problem -- and we derive five primitives from the questions that must be answered before an action takes effect and after it has: discovery, identity, governance, attestation, and supply chain. For each we state what fails if it is absent and why the others cannot structurally supply it. We describe an implementation in which an agent's action is mediated against policy before it takes effect, authorised against a per-tenant action vocabulary, and recorded in a hash-linked signed ledger a third party can verify with the vendor out of the loop. We report what the architecture costs: the enforcement point sits on the request's critical path, identity requires a sidecar per workload, and fail-closed mediation converts availability incidents into denial. We are explicit about implementation status: four primitives are built and running in private pilots, and the fifth is built as separate tooling and not yet integrated into the request path. We keep it in the set deliberately: a five-part decomposition that exactly matches what its authors happened to build is not a taxonomy but a description of a codebase.

1 Introduction

Autonomous agents do not fit controls designed for human users and long-lived services because they are ephemeral, choose actions at runtime, and are discovered rather than provisioned. The paper frames governance as a runtime problem and decomposes it into five necessary primitives, while reporting implementation costs and status.

  • Agents may exist only for a task, so registration-dependent controls risk being skipped or becoming a bottleneck that drives agents outside control.
  • Agents select actions and parameters at runtime, so the governable action set is not known when policy is written.
  • Because anyone who can call an API can create an agent, governance must discover the population rather than rely on a maintained inventory.
  • The paper treats agent governance as a runtime problem rather than primarily a model-alignment or build-time problem.
  • The proposed decomposition contains discovery, identity, governance, attestation, and supply chain primitives derived from questions before and after actions.
  • The authors argue the five are irreducible, with distinct agent-specific design consequences including aggregate discovery and value-free storage.
  • The architecture incurs critical-path latency, a sidecar per workload, and fail-closed denials during availability incidents.
  • Four primitives are built and running, while the fifth is built separately and not yet integrated into the request path; the paper is not an evaluation.

2 Why the existing controls do not fit

Existing identity, gateway, application, and logging mechanisms each address narrower principal or trust assumptions than autonomous agents require. The implementation therefore places mediation, identity, attestation, and an independently verifiable export around a single authorization call.

  • Existing controls fail because they were designed for principals whose lifecycle, actions, or population can be provisioned and enumerated.
  • Human IAM cannot preserve attribution when agents use service accounts or human credentials, while workload identity does not decide which self-selected actions are authorised.
  • API gateways and network policies operate at route or reachability granularity, while agent governance often depends on payload parameters.
  • Application guardrails share a failure domain with the constrained agent and cannot produce evidence a third party will accept.
  • Conventional audit logs are controlled by the same party whose behaviour they evidence, making them inadequate for disputes.
  • The deployed shape centers on a mediator and action catalog, workload identity, attestation components, an append-only chain, and per-tenant signing keys.
  • The governed-agent wire contract is POST /v1/actions/authorize over mutual TLS with a workload credential, producing an evidence package independently verifiable by a customer.

3 Deriving the five

The paper derives five primitives by tracing one agent action from permission to retrospective proof. Each answers a distinct question, and the first four operate around request time while supply chain spans requests.

  • A single agent action requires four sequential request-time questions plus a fifth question operating across requests.
  • Discovery determines whether the agent exists and is known; without it, the other controls govern only agents already visible to them.
  • Identity establishes the acting subject at the moment of action, which discovery alone cannot authenticate.
  • Governance must authorize an action before its effect, and identity alone supplies no policy decision for an open-ended action set.
  • Attestation provides an independently verifiable record because the governance layer cannot credibly attest to its own decisions.

4. What happened, in a form someone else can check?

The five-primitives decomposition separates behavioral evidence from supply-chain provenance and treats each primitive as independently necessary. Discovery uses denied actions as signals while constraining recording so agent retries cannot turn observability into an outage.

  • What happened, in a form someone else can check?: Attestation concerns a record others can verify, while supply chain operates between requests and tests the agent’s composition rather than its behavior.
  • 3.1 Why not fewer, and why not more: The set is minimal because removing any primitive leaves a failure the remaining four cannot address, but it is not claimed to be maximal.
  • 3.1 Why not fewer, and why not more: The primitives are separable and load-bearing because they have different trust models, failure modes, and timescales.
  • 3.1 Why not fewer, and why not more: Merging primitives inherits the worst failure mode of each, so the separation is an architectural consequence rather than presentation.
  • 4 Discovery: Capability discovery treats an undefined action attempt as both a deny-by-default refusal and a signal for operators to define policy.
  • 4.1 Discovering what agents attempt: The discovery surface aggregates one row per organisation and action type because retrying agents can otherwise amplify failures into an unbounded table.
  • 4.1 Discovering what agents attempt: Discovery stores parameter names but no values, making unsafe retention structurally unrepresentable; recording failures must not affect authorization.
  • 4.2 Discovering the agents themselves: Population discovery remains a library rather than a running-system component, and reconciliation focuses on disagreements among code, deployment, and credential sources.

5 Identity

Identity must prove which ephemeral workload is acting without relying on prior human provisioning or caller-supplied claims. The design uses short-lived attested workload credentials and canonical mappings across system representations.

  • Identity: Attestation-based workload identity issues short-lived credentials from observable workload properties rather than manual provisioning.Credentials are presented over mutual TLS, expire on the order of minutes, and are revoked by withholding reissuance.
  • Identity: Transport-layer identity binds policy evaluation to the credential proved by the connection, not an identity asserted in the payload.Payload-forwarded identities would let the agent choose its own principal.
  • Identity: A canonical mapping lets one principal remain traceable across storage, audit, and policy representations.Separate vocabularies preserve each layer’s constraints without coupling permanent audit records to other schemas.
  • Identity: Platform-internal identities should be derived when they are deterministic functions of already stored values.For per-tenant instances of a small class set, storing a row per tenant and class adds no information.

6 Governance

Governance mediates actions before they take effect, but agent actions cannot be governed by a vendor-fixed vocabulary. The design therefore combines customer-authored action definitions with deny-by-default mediation and runtime revocation.

  • Governance: The mediator evaluates each agent request on the request path and allows the action to proceed only when policy permits it.This makes authorization a pre-effect control rather than an after-the-fact report.
  • Governance: A vendor-fixed action enumeration leaves customer-defined agent actions ungovernable until a release adds them.Model-selected actions drawn from customer-defined tools cannot be exhaustively enumerated by the vendor.
  • Governance: The action vocabulary is per-tenant and customer-authored, with refused unknown attempts surfacing missing definitions for later governance.The loop is attempt → refused → visible → defined → governable.
  • Governance: Immutably versioned action definitions let each attested record preserve which meaning governed it at the time.Without versioning, changing definitions makes older audit records harder to interpret.
  • Governance: Authorization denial does not constrain how much action data crosses the policy boundary for evaluation.Disclosure is a separate decision governed by the same tables, so deny-by-default provides no safety net for it.
  • Governance: A kill switch checked on every authorization revokes looping principals at their next action rather than at credential expiry.Because the check is on every request, its latency and cost constrain the design.

7 Attestation

Attestation makes decisions independently verifiable, while supply-chain provenance addresses whether the governed agent is actually the composition the operator believes it to be. The paper presents the latter as necessary but not yet integrated into the request path.

  • Attestation: Attestation records every authorization decision in a hash-linked, signed append-only ledger for independent verification.The ledger uses canonicalized rows and per-tenant keys held in a key-management service.
  • Attestation: Verification must occur through an exported evidence package and independent tool, with the vendor’s systems out of the loop.Otherwise the verification question is routed through the party being evaluated.
  • Attestation: Completeness requires an export to state its covered window and resolve indeterminate completeness to false.Individual record validity does not establish that the exported window is the whole record.
  • Attestation: An operator with database and signing access could rewrite and re-sign history; prior exports provide evidence of rewriting rather than prevention.This is an explicit residual of the trust model.
  • Supply chain: The first four primitives can faithfully govern a compromised composition, making supply-chain provenance necessary to identify what the agent is made of and whether it changed.A replaced dependency can leave identity, authorization, and attestation apparently correct while the governed thing is compromised.
  • Supply chain: Supply-chain checks must account for models, prompts, and tool descriptions because modifying a tool description can change behavior without changing code or package versions.Conventional supply-chain tooling does not inspect that surface.
  • Supply chain: Provenance analysis exists as separate tooling but does not yet run inside the described control plane or condition authorization on the request path.The planned integration would expose a provenance verdict for mediator consultation.
  • Supply chain: The supply-chain primitive remains in the taxonomy because it answers a necessary governance question even though its integration is incomplete.The paper distinguishes the primitive’s necessity from the current implementation roadmap.

9 The control plane governs its own agents

The control plane applies its own identity, authorization, revocation, and attestation controls to internal agents. Separate policy namespaces and transport-layer identity prevent self-governance from becoming a naming convention or customer-configured exception.

  • The control plane governs its own agents: Internal agents use the same workload credentials, authorization endpoint, deny-by-default mediator, kill switch, and attestation chain as customer agents.The design provides no privileged bypass for the platform’s own automation.
  • The control plane governs its own agents: Internal action types require a separate policy namespace so customer policies cannot authorize them or determine the platform’s own behavior.The separation prevents internal actions from appearing on customer-facing API surfaces.
  • The control plane governs its own agents: Transport-layer credential checks enforce internal-action separation because namespaces alone are only naming conventions.The authorization route accepts internal action types only from workloads whose credentials identify them as internal agents.
  • The control plane governs its own agents: Internal agents reuse an existing closed principal type in the permanent audit envelope, carrying forensic distinction in a separate field.This avoids changing the append-only ledger schema for a new feature.
  • The control plane governs its own agents: Self-governance demonstrates generality and removes privileged-exception questions, but it does not establish that the controls are correct.The authors also use their own integration to expose ergonomic problems before customers encounter them.

10 What the architecture costs

The architecture imposes four concrete costs: synchronous mediation adds critical-path latency, attested identity requires a sidecar, fail-closed behavior turns mediator outages into denials, and integration is front-loaded. Its intended scope is open action sets, undiscovered populations, or independently verifiable evidence; implementation is running in private pilots, with discovery-by-scanning still unwired.

  • Critical-path cost: Synchronous mediation adds a network round trip, policy evaluation, and durable write before an action takes effect, inflating tail latency.Completing signatures under bounded lag changes when records become tamper-evident relative to actions, making the remedy a security decision.
  • Deployment cost: Attestation-based identity requires a sidecar, shared volume, registration entry, and deployment change for every governed workload.The paper identifies this as its single largest adoption obstacle and treats it as intrinsic to attested identity.
  • Availability cost: Fail-closed mediation converts governance-layer outages into outages for every governed agent by refusing actions when the mediator is unreachable.This raises the control plane’s availability requirement to that of the most critical system it governs.
  • Adoption and scale: Integration effort is front-loaded: the first governed agent requires several times the code and additional deployment resources, while later agents incur a small fraction of that marginal cost.The first figure determines whether teams integrate; the marginal figure determines whether the programme scales.
  • Scope: The architecture earns its cost when action sets are open, populations are not enumerable, or evidence must satisfy someone who does not trust the operator.For small, fixed, enumerable action sets in one trust domain without third-party evidence needs, in-process guardrails are sufficient.
  • Implementation status: The system runs in a small number of private pilots, while code-scanning population discovery exists as tested tooling without an execution path.The paper distinguishes four primitives implemented and exercised from the fifth, whose integration remains pending.

12 Related work

The paper positions runtime agent governance as complementary to identity, policy, logging, supply-chain, alignment, and code-validation work. Its central contribution is a five-primitive decomposition in which each requirement is necessary, structurally distinct, and fails differently.

  • Identity and alignment: Attested workload identity supplies caller identity and zero-trust assumptions, but does not authorize actions selected by the caller.The paper treats identity as one primitive rather than the whole architecture because agent actions may not be enumerable in advance.
  • Policy and alignment: Policy engines provide governance decisions, but commonly assume that policy authors know the action and resource vocabulary in advance.The paper argues that this assumption fails when models compose actions and parameters at runtime.
  • Attestation: Hash-linked logs and transparency architectures provide attestation construction, while this paper insists that verification operate outside the vendor’s systems.That separation makes evidence independently checkable by another party.
  • Supply chain: Conventional provenance and artifact-attestation frameworks address software composition but do not currently cover models, prompt templates, and tool descriptions.The paper therefore extends the relevant supply-chain surface for agents.
  • Alignment and runtime control: The paper is orthogonal to alignment: it constrains what a running deployment may cause, whereas code validation constrains what agent code may do.A well-aligned model can still be deployed with more authority than intended.
  • Five-primitive contribution: The proposed decomposition follows five requirements—discovery, identity, governance, attestation, and supply chain—that are necessary, non-substitutable, and differently failure-prone.The paper’s practical ordering is to ask what exists, what it is, may it act, what happened, and what it is made of.

A One action, through four primitives

A support agent’s email action passes through identity, model-selected parameters, policy mediation, discovery for unknown action types, and independent attestation. The sequence makes authorization a pre-effect control while leaving supply-chain composition unresolved.

  • Identity: A sidecar obtains a short-lived workload credential from attested platform properties before the support agent sends any request.No person provisions the credential, and it expires in minutes.
  • Agent decision: The model selects send_email and composes recipient, subject, and body parameters at runtime, so the concrete action was not enumerable beforehand.This illustrates why runtime governance must handle actions that arise only when the model decides.
  • Governance: The mediator evaluates the proven mutual-TLS identity against tenant policy, action attributes, and kill-switch state before returning permit or deny.A revoked agent is stopped at authorization rather than waiting for credential expiry.
  • Discovery: An unknown action type is refused by deny-by-default and recorded as one parameter-name-only row per tenant and action type.The discovery record has no column capable of storing parameter values, and refusal and discovery are the same event.
  • Governance: For known action types, the decision returns before the agent proceeds, making mediation a control rather than a report.The key ordering is decision first, effect second.
  • Attestation: The decision is hash-linked, canonicalized, and signed by the tenant’s key, then later checked by an independent verifier without vendor-system involvement.A background verifier rechecks signatures and linkage, while customers can export the evidence package.
  • Supply chain: The sequence does not establish which model, tool descriptions, or dependency versions constituted the agent at the time of action.That unresolved composition gap is the missing supply-chain primitive.
Loading 2608.26696v1…