Source-linked AI summary
Can AI Agents Deliver Verifiable Network-Wide Outcomes Across Authority Boundaries?
Tianzhu Zhang, Chih-Kai Huang, Meikang Qiu
TL;DR
Distributed authority scopes leave no single agent with sufficient visibility to establish a network-wide outcome, making completion admission a distinct assurance problem. EvidenceNet collects contract-required observations and admits completion only when their provenance, freshness, coverage, and task checks succeed. Live-network experiments show that post-change evidence distinguishes successful outcomes from action records alone and rejects wrong-source, substituted, or stale observations.
Problem
Distinct authority scopes fragment the evidence needed to determine whether coordinated network operations achieved the operator’s intended network-wide outcome.
Method
EvidenceNet uses a trusted completion contract, brokered observations bound to source and epoch, deterministic admission checks, and verifier assessment.
Results
Post-change evidence admitted successful outcomes that configuration-action records alone could not establish, while source-binding and freshness checks rejected wrong-source, substituted, or stale observations.
Takeaways & Limitations
Trusted runtime assurance should retain control over evidence and machine-checkable completion conditions while agents drive planning, configuration, interpretation, and recovery.
Abstract
from arXiv · showhide
AI agents are increasingly involved in network automation, where they can initiate configuration changes through mediated operational interfaces and assess the resulting state. Nonetheless, operational networks usually span many devices and administrative domains. Realizing an operator's intent requires coordinating agents with distinct authority scopes that define the resources they can access, the operations they can invoke, and the network state they can observe. This division limits the blast radius of an erroneous action but fragments the evidence needed to assess the network-wide outcome. Successful execution of a configuration action proposed by one agent does not establish that remote devices responded as intended or that routing changes reached the required devices. A valid observation may also become stale after a subsequent change. Before the coordinated operation can be declared complete, a trusted assurance layer must collect current observations from the required scopes and determine whether they collectively support the operator's intended network-wide outcome. To address the completion admission problem, we present EvidenceNet, a runtime assurance layer for deciding whether coordinated agent operations have achieved an operator's network intent. Its broker collects the post-change observations required by a completion contract, and its admission gate checks that the evidence comes from the required scopes, remains current, and satisfies the task rules. A verifier agent provides an additional assessment of the observation content. Experiments on live routing networks show that post-change state checks recognize successful outcomes that configuration-action records alone cannot establish. Controlled interventions further show that EvidenceNet rejects completion when otherwise satisfactory observations have the wrong source, have been substituted, or are stale.
I. INTRODUCTION
Network-wide intent assurance is difficult when agents operate across distinct authority scopes with fragmented observations. EvidenceNet addresses completion admission by requiring current, properly sourced evidence before coordinated operations are accepted.
- I. INTRODUCTION: Authority scopes limit erroneous-action blast radius but fragment visibility into whether remote devices and network policies reached the intended state.
- I. INTRODUCTION: Local configuration success does not establish that the coordinated operation achieved its intended network-wide outcome.
- I. INTRODUCTION: Completion admission combines evidence from required scopes to decide whether observations collectively support the operator’s intent.
- I. INTRODUCTION: EvidenceNet is a trusted runtime layer that collects contract-required observations, binds them to source and network epoch, and admits completion only after deterministic checks and verifier assessment succeed.
- I. INTRODUCTION: The paper defines completion contracts for required network properties and implements provenance- and freshness-aware evidence controls before completion.
- I. INTRODUCTION: Live-network experiments and controlled evidence interventions evaluate evidence acquisition, deterministic controls, verifier judgment, and repair.
II. RELATED WORK
Prior assurance, verification, and agent-governance systems address adjacent problems but do not jointly determine whether distributed post-change evidence supports completion. EvidenceNet targets this combined authority, provenance, freshness, coverage, and admission problem.
- II. RELATED WORK: Assurance monitors delivered services, while verification checks configurations or forwarding state against specified invariants.
- II. RELATED WORK: Existing LLM network-management systems improve task-level correctness but do not directly address completion assurance across agents with distinct authority scopes.
- II. RELATED WORK: Agent policy, access-control, and execution-provenance systems govern actions or traceability without establishing that post-change observations are sufficiently scoped, current, and complete.
- II. RELATED WORK: The paper identifies no prior work jointly enforcing action authority, evidence origin, freshness, required-observation coverage, and live-network completion acceptance.
III. EVIDENCENET DESIGN
EvidenceNet separates agent proposals from trusted completion admission through a contract that specifies required observation atoms and deterministic evidence checks. Admission is fail-closed and requires complete, valid, current, relationally consistent, and verifier-supported evidence.
- III. EVIDENCENET DESIGN: Agents produce plans, configurations, repairs, and interpretations as proposals, while admission depends on current observations from every required network location.
- A. Completion contract: A completion contract maps each required network property to observation atoms specifying source, exact query, parser success condition, and freshness rule.
- A. Completion contract: The orchestrator registers each atom’s request and directs its source scope to execute the declared query.
- A. Completion contract: The evidence broker creates records containing broker-issued identity, requester and collector identities, source, command, output hash, collection time, and network epoch.
- A. Completion contract: The fail-closed gate requires every declared atom, successful deterministic checks, declared relationships, per-atom verifier support, and a positive global recommendation.
- A. Completion contract: Verifier support is necessary but cannot override deterministic failure, and admission is evaluated separately from the external outcome classification.
B. Orchestration, scoped execution, and evidence collection
EVIDENCENET orchestrates scope-aligned proposals through authority wrappers, then collects contract-defined observations from authorized source scopes. The broker binds each returned result to its request, source, command, atom, hash, and current epoch.
- B. Orchestration, scoped execution, and evidence collection: Trusted components mediate orchestration, scoped execution, evidence collection, epoch tracking, and submission, while network agents remain untrusted proposal and assessment providers.
- B. Orchestration, scoped execution, and evidence collection: The coordinator decomposes operator intent into plans aligned with participating authority scopes and receives structured proposals from scope-constrained agents.
- B. Orchestration, scoped execution, and evidence collection: Scope wrappers validate target routers and operation types before forwarding authorized operations, rejecting cross-scope actions and mutating commands presented as reads.
- B. Orchestration, scoped execution, and evidence collection: Each observation atom identifies its requester scope, authorized source scope, exact command, and network property to support.
- B. Orchestration, scoped execution, and evidence collection: The broker verifies each request against the contract and binds the returned evidence to request, source, command, atom, output hash, and current epoch.
C. Verifier assessment and one-round repair
After evidence collection, EVIDENCENET serializes the completion dossier, combines deterministic admission checks with verifier assessment, and permits at most one coordinated repair round before rechecking the network.
- The verifier receives the operator intent, serialized dossier, and freshness metadata, then returns per-atom decisions and a global recommendation.
- The submission gate checks coverage, record binding, freshness, task predicates, and cross-observation relationships against the current network epoch.
- EVIDENCENET allows at most one repair round, after which the workflow stops if admission still fails.The repair coordinator uses failed atoms, relevant evidence, candidate scopes, and the original task to propose coordinated repairs.
- Successful repair commands advance the global epoch, so earlier evidence becomes stale and the system collects new observations before repeating verification and admission.
IV. EVALUATION
The evaluation tests whether EVIDENCENET admits successful outcomes, rejects provenance and freshness violations, identifies content defects beyond deterministic predicates, and restores outcomes through one-round repair.
- The evaluation is organized around four questions covering post-change evidence, provenance and freshness, verifier judgment, and one-round repair.
- The prototype runs on NetAgentBench with an external evaluator unavailable to agents and outside the completion-admission path.
- Six custom tasks span OSPF adjacency and reachability, multi-area OSPF, BGP propagation and filtering, BGP policies, and reachability policies.
Q1: evidence acquisition
The evidence-acquisition experiment shows that post-change observations distinguish successful network outcomes from failures, whereas execution records alone do not establish fulfillment.
- 45 fulfillment traces were replayed: 32 produced network states satisfying every task requirement, while 13 failed at least one requirement.
- EVIDENCENET agreed with the external evaluator in all 45 cases.
- ADF-CLAIMS rejected all 13 failures but also rejected all 32 successful outcomes when completion was assessed from execution records alone.
Q2: Binding and freshness
Controlled evidence interventions show that broker-enforced provenance and freshness checks reject observations that appear satisfactory but come from the wrong source, were substituted, or are stale.
- Each of three successful traces was replayed under clean, wrong-source, substituted-record, stale, and current-after-fault conditions.
- Across 27 wrong-source, substituted, or stale cases, the content-only baseline admitted every dossier, while the deterministic gate and EVIDENCENET rejected all.
- The deterministic gate and EVIDENCENET made identical decisions in this experiment, so the verifier added no further separation.
- The blinded defect study used six clean and 24 altered dossiers containing outdated, misattributed, contradicted, or misrepresented observations.
- Both verifiers accepted all six clean and all 24 altered dossiers, whereas the rule-based checker accepted the clean dossiers and rejected every altered dossier.
Q4: one-round repair
One repair round restored every injected single-router fault and produced evidence that passed completion admission, although the repairs changed all three routers.
- Q4: one-round repair: 9 of 9 runs recovered with one repair round, whereas all 9 runs without repair failed both evaluation and completion admission.The study injected three fault types and tested each three times under both conditions.
- Q4: one-round repair: Every repaired network passed all evaluated properties and completion admission after new evidence was collected.
- Q4: one-round repair: Every successful repair issued configuration commands to all three routers, making the repairs broader than necessary.
V. LIMITATIONS
EvidenceNet’s prototype depends on a trusted runtime and limited evidence model, while its freshness and contract mechanisms do not establish complete or atomic network state.
- V. LIMITATIONS: The prototype assumes its orchestrator, wrappers, broker, client, gate, and observation sources are uncompromised.Router observations are assumed not to be falsified, and scope enforcement uses in-process wrappers rather than isolation or cryptographic capabilities.
- V. LIMITATIONS: Broker records verify consistency within the trusted runtime but do not authenticate actual router state or detect changes bypassing wrappers.
- V. LIMITATIONS: The global epoch invalidates all evidence after each successful configuration command, while sequential collection prevents an atomic network snapshot.A fixed stabilization wait also does not prove convergence.
- V. LIMITATIONS: Admission covers only contract-represented properties, so omitted requirements or weak predicates may allow incomplete outcomes.The verifier cannot establish properties for which no evidence was collected.
C. Repair and evaluation scope
The repair and evaluation results support trusted evidence-based completion, but remain bounded by broad repair actions and a small, controlled experimental scope.
- C. Repair and evaluation scope: The prototype lacks minimal changes, staged deployment, rollback, and guarantees under simultaneous faults.
- C. Repair and evaluation scope: The evaluation used Containerlab, one FRRouting release, small topologies, six tasks, and two verifier backends.
- C. Repair and evaluation scope: Controlled interventions demonstrate enforcement, but the results do not establish behavior across vendors, larger deployments, adversarial settings, or concurrent changes.
- C. Repair and evaluation scope: Trusted runtime assurance retains control over evidence and machine-checkable conditions while agents handle planning, configuration, interpretation, and recovery.
- C. Repair and evaluation scope: One-round repair restored every evaluated fault, but each repair issued commands to all three routers.