PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenAgentFlow: Enabling System-Wide Safety Boundaries for Heterogeneous AI Agent Fleets

Dongsheng Chen, Xiangyu Zhao, Xin Yao, Xuetao Wei

arXiv:2609.00015v1cs.AIcs.CR

TL;DR

Heterogeneous AI agent fleets create safety risks when concrete actions depend on shared state, provenance, and composed workflows beyond any single agent-local check. OpenAgentFlow governs those actions at a shared commit boundary using normalized AgentEvents and a control-plane/action-plane architecture. Android evaluations report strong results across action-event, dynamic-policy, and emulator suites, while the prototype remains limited to instrumented PEP-mediated actions and practical rather than cryptographically attested provenance.

  • Problem

    Heterogeneous agents and execution paths lack a shared enforcement view over concrete actions, session state, and provenance, limiting system-level governance across multi-step workflows.

  • Method

    OpenAgentFlow normalizes pending GUI, API, tool, and LLM-planned actions into AgentEvents, checks them through a shared pre-execution PEP, and maintains governance state in a separate control plane.

  • Results

    Across Android action-event, dynamic-policy, and emulator evaluations, OpenAgentFlow achieved 94.0% accuracy and 95.3% attack block rate, matched 27/30 dynamic-policy cases, and reached a 92.9% trace-adjusted pass rate.

  • Takeaways & Limitations

    OpenAgentFlow provides a practical shared enforcement boundary for exposing composed action-flow risks, applying updated policies, and recording decisions before actions affect shared state.

  • Takeaways & Limitations

    Its guarantee applies to instrumented actions passing through the PEP, using practical enforcement-observed provenance rather than cryptographic or OS-attested information flow.

Abstract

from arXiv · show

AI agents powered by large language models are evolving from isolated assistants into heterogeneous systems in which multiple agents, planners, controllers, and execution backends operate over the same user or enterprise environment. In such settings, safety becomes a system-level action-governance problem: deciding whether concrete agent-generated actions should be committed before they modify shared state. Existing safeguards cover prompts, tool calls, GUI actions, and agent-local behavior, but often leave enforcement fragmented, obscure risks that emerge across multi-step action flows, and provide limited support for auditability and policy evolution. We present OpenAgentFlow, a control-plane/action-plane architecture that enforces safety at the action-commit boundary. It normalizes pending GUI actions, API calls, tool calls, and LLM-generated invocations into a unified AgentEvent stream, routes each event through a shared pre-execution Policy Enforcement Point, and maintains provenance, session state, audit records, and updatable policies in the control plane. This creates a shared governable action stream and allows new rules to take effect without modifying agents, prompts, models, or execution paths. We instantiate OpenAgentFlow on Android. On a 300-case action-event benchmark, it achieves 94.0% accuracy and a 95.3% attack block rate. On a 30-case dynamic-policy suite, it matches expected behavior in 27 cases after new rules are installed. Across 98 traced cases from a 100-case Android emulator suite, it achieves 90.8% raw accuracy and a 92.9% trace-adjusted pass rate across GUI, API, and LLM-planned cases. These results show that OpenAgentFlow provides a practical shared enforcement boundary for heterogeneous AI agent fleets.

1 INTRODUCTION

OpenAgentFlow frames heterogeneous-agent safety as governance of concrete actions at a shared action-commit boundary, addressing risks that emerge across agents, endpoints, and composed session flows. Its Android evaluation reports strong benchmark, dynamic-policy, and emulator results.

  • Motivation: Heterogeneous agents can create risks across shared resources, execution surfaces, endpoints, and multi-step workflows that no single prompt or local decision captures.The relevant unit is the concrete action about to be committed, interpreted with session state and provenance.
  • Problem: Fragmented policies across agents, prompts, tools, applications, and wrappers weaken auditability, complicate debugging, and hinder consistent governance over time.Different components expose different policy, enforcement, and logging points, making whole-system governance brittle.
  • Approach: OpenAgentFlow converts GUI actions, API calls, tool invocations, and LLM-planned calls into unified AgentEvents checked by a shared pre-execution Policy Enforcement Point.The control plane maintains policies, provenance, audit records, session state, and rule updates outside the agents.
  • Evaluation: 94.0% overall accuracy and a 95.3% attack block rate were achieved on the 300-case action-event benchmark.Prompt-only advisory blocks no attack cases, while the deterministic T1/T2 common path has P99 latency below 1 ms.
  • Evaluation: 27/30 dynamic-policy cases matched expected behavior after new control-plane rules were installed without changing the agent, model, prompt, or execution path.The suite included denial of all core matching actions after rule updates.
  • Evaluation: Across 98 traced cases from a 100-case Android emulator suite, raw accuracy was 90.8% and trace-adjusted pass rate was 92.9% across GUI, API, and LLM-planned cases.The same governance layer covered action-event benchmarks, policy updates, multi-agent propagation, and real Android execution.

2 RELATED WORK

Prior work places safeguards on local prompts, tools, GUI controllers, runtimes, or traces, whereas OpenAgentFlow treats the cross-channel pending action stream as the shared enforcement object. It combines this boundary with enforcement-observed provenance and a separate control-plane rule store.

  • Acting agents and GUI/mobile benchmarks: Acting-agent research spans tool use, web and desktop navigation, mobile GUI manipulation, realistic benchmarks, and increasingly composed multi-agent systems.The cited systems include ReAct, Toolformer, WebArena, OSWorld, AndroidWorld, AndroidLab, AutoGen, CAMEL, MetaGPT, and AgentScope.
  • Prompt-injection and tool-using agent safety: Prompt-injection research studies how untrusted tool or retrieved content can steer later agent actions away from user intent.The passage situates these attacks within tool-integrated agents and related defenses.
  • Runtime safeguards for acting agents: Runtime safeguards increasingly mediate acting agents before execution, but OpenAgentFlow interposes specifically on executor-facing actions across GUI, API, tool, and LLM-planned paths.Its evaluation combines offline action-event suites with real Android emulator runs.
  • Behavioral policies, provenance, and control-plane design: OpenAgentFlow applies source–sink policy ideas through provenance observed from GUI representations, API or tool results, and session memory at the pending-action boundary.The governed unit is an AgentEvent, while policy state remains separate from action executors.
  • Boundary distinction: Unlike application-, process-, backend-, or controller-specific mechanisms, OpenAgentFlow uses one PEP, provenance memory, audit trail, and policy store across incompatible enforcement surfaces.The pending action serves as the stable interface across executors.
  • Positioning: Table 1 positions OpenAgentFlow as targeting a shared action stream across GUI, API, tool, and LLM-planned invocations with session provenance and a control-plane rule store.The comparison concerns mediation units and policy surfaces among runtime systems.

3 PROBLEM SETTING AND SAFETY GOALS

OpenAgentFlow frames safety around intended actions crossing heterogeneous execution channels within a shared session, where risks can emerge from prior history, provenance, and composed actions. It sets goals for mandatory, framework-independent, session-aware, auditable, and updateable mediation, scoped to instrumented actions passing through the PEP.

  • 3 PROBLEM SETTING AND SAFETY GOALS: Heterogeneous agents can affect the same environment through GUI, API, tool, and LLM-generated execution channels.The setting distinguishes decision-making agents from the applications, tools, APIs, and services they operate.
  • 3 PROBLEM SETTING AND SAFETY GOALS: OpenAgentFlow treats the intended action, target, payload, channel, and session context as the enforcement unit before commitment.Pending actions are normalized into AgentEvents for PEP evaluation before reaching executors or changing persistent state.
  • 3.2 SESSION-LEVEL ACTION RISKS: Session-level action risks arise when individually reasonable actions violate policy through composed data movement, scope violations, or direct effects on the environment.Examples include source-to-sink propagation, unauthorized API use, deletion, payments, uploads, and settings changes.
  • 3.3 SAFETY GOALS AND SCOPE: Mandatory mediation requires every instrumented agent-mediated action routed through the enforcement boundary to be checked at the action-commit point.The stated guarantee applies to actions passing through instrumented GUI controllers, tool wrappers, API executors, or operating-system-facing interfaces.
  • 3.3 SAFETY GOALS AND SCOPE: Framework independence and session-aware enforcement let one PEP use common events and prior session observations across agents and execution channels.This addresses local agents’ partial visibility into other agents’ reads, tool results, payloads, provenance, and audit state.
  • 3.3 SAFETY GOALS AND SCOPE: Auditability and policy update require recording action decisions and enforcing newly installed rules through the same governance layer.The control plane maintains policies, provenance, audit records, session state, and rule updates outside the agents.

4 OPENAGENTFLOW ARCHITECTURE

OpenAgentFlow places a shared policy enforcement point between heterogeneous agents and executors, normalizing pending actions into AgentEvents before commit. Its control plane maintains provenance, session state, audit records, and updateable rules, while staged enforcement handles routine and ambiguous cases.

  • Action Plane: OpenAgentFlow intercepts intended actions before commit and checks them through a shared Policy Enforcement Point.The action plane mediates actions headed to GUI controllers, API executors, tool backends, or operating-system-facing interfaces.
  • Action Plane: AgentEvent unifies GUI, API, tool, and LLM-planned actions using session, agent, channel, target, action, payload, and contextual evidence.The representation places otherwise different actions on the same governance and audit path.
  • Session State and Trust Boundary: Session state stores enforcement-observed provenance, policy context, and prior audit state outside individual agents.The PEP uses bounded memory to preserve policy-relevant observations across a session and support later decisions.
  • Session State and Trust Boundary: Provenance is collected from instrumented GUI representations, API or tool results, event builders, and PEP-maintained memory.GUI extraction records visible text and application context, while API and tool wrappers record sensitive fields with their source.
  • Session State and Trust Boundary: Reliable provenance requires relevant values to pass through instrumented observation points or PEP-maintained memory.Obfuscation, partial rewrites, ambiguous fields, and short numeric overlaps are evaluated as boundary cases.
  • Policy Enforcement Path: The T1–T4 pipeline combines deterministic fast-path checks with fallback handling for incomplete provenance and rare policy-conditioned escalation.T1 and T2 cover flow rules, scopes, high-risk operations, payload patterns, provenance matching, and source-to-sink policies; T3 and T4 address ambiguous cases.
  • Policy Enforcement Path: Administrators or users can add or edit FlowRules that the action-plane PEP refreshes and enforces without changing agent code, models, prompts, or executors.Rules specify match fields, decisions, and audit reasons.

5 EVALUATION

The evaluation tests whether execution-time mediation, fast-path checks, policy updates, and shared enforcement work across offline and Android settings. OpenAgentFlow achieves strong benchmark performance, updates decisions after deployment, and applies the same PEP across GUI, API, and LLM-planned paths.

  • Pre-execution mediation: 94.0% overall accuracy and a 95.3% attack block rate are achieved on the 300-case action-event suite.The comparison includes lower bounds, named baselines, internal ablations, and the full control-plane design.
  • Fast-path contribution: 128/300 to 280/300 is the improvement from T1-only to T1+T2 on the broad suite.T2 adds payload and provenance matching, while T3/T4 change little because most cases contain inspectable evidence.
  • Fast-path contribution: T3/T4 recover controlled no-provenance cases that T1+T2 miss, while T1/T2 remain the primary inspectable safety boundary.The stress suite tests direct sensitive API writes without a provenance chain.
  • Control-plane policy updates: 27/30 dynamic-policy cases match expected behavior after structured FlowRules are installed in the control plane.All 6 core matching actions are denied, 11/12 negative controls remain allowed, and 10/12 boundary cases match expectation.
  • Threat and provenance governance: Cross-app, cross-tool, and cross-agent information-flow cases require shared provenance and source–sink policy state beyond current-action checks.Named baselines can handle several current-action risks, but the retained failures include cross-channel and other boundary cases.
  • Real Android execution: 90.8% raw accuracy and a 92.9% trace-adjusted pass rate are reported across 98 traced Android executions.The same PEP semantics are exercised in GUI, API, and LLM-planned execution paths.

6 DISCUSSION AND CONCLUSION

OpenAgentFlow complements existing safeguards by governing agent-produced actions at a shared commit boundary. Its guarantees apply to instrumented actions passing through the PEP, while deployment requires mechanisms for user intent and exceptions.

  • Scope and role: OpenAgentFlow complements prompt guardrails, tool allowlists, operating-system permissions, sandboxing, and traditional information-flow systems.It focuses on the point where an agent-produced action is about to modify shared user or enterprise state.
  • Scope and role: The PEP evaluates pending actions using their targets, payloads, and accumulated session state before commitment.This shared boundary supports governance across heterogeneous execution paths.
  • Guarantee boundary: The guarantee covers instrumented agent-mediated actions that pass through the PEP and relies on enforcement-observed provenance.The prototype does not provide cryptographic or OS-attested information flow; incomplete or ambiguous provenance triggers fallback or escalation.
  • Deployment considerations: Production deployments should pair action-commit enforcement with confirmation, task-specific exceptions, administrative overrides, and audit trails.These mechanisms address user-authorized disclosures or high-impact operations that a default policy may block.
  • Conclusion: OpenAgentFlow turns heterogeneous GUI, API, tool, and LLM-generated actions into a governable stream through AgentEvents and shared session state.The architecture separates policy management from action execution.

A EVALUATION PROTOCOL AND METRICS

The evaluation protocol separates offline policy semantics from Android execution and distinguishes raw task labels from behavior observed in traces. Suites cover broad threats, policy updates, provenance boundaries, and emulator integration.

  • Metrics and labels: Offline cases pass when the PEP decision matches an expected ALLOW or DENY outcome.Attack block rate measures blocked expected-DENY cases, while false block rate measures incorrectly blocked expected-ALLOW cases.
  • Metrics and labels: Android raw accuracy compares decisions with task-script labels, whereas semantic accuracy compares them with behavior observed in execution traces.The distinction accounts for GUI-control task drift.
  • Evaluation design: The offline suites test controlled AgentEvent semantics, provenance matching, and rule updates, while Android runs exercise GUI, API, and LLM-planned paths.This separates policy evaluation from GUI task drift and then tests integration in the emulator.
  • Suite composition: The broad offline suite retains hard misses and hard false positives to expose matcher limits rather than removing them from aggregate results.DENY labels cover source-to-sink propagation, high-risk operations, and direct sensitive writes; ALLOW labels cover safe actions under the configured policy.
  • Suite composition: The 200-case threat evaluation spans benign and high-risk intra-app actions, cross-app, cross-tool, and cross-agent flows, prompt-injection consequences, payment boundaries, and randomized variants.Focused suites separately test direct sensitive writes without provenance, dynamic policy updates, and provenance-boundary behavior.
  • Android suite: The Android emulator suite contains 100 executions, with 98 traced cases used for the main traced analysis.It includes 60 GUI cases and 40 API or LLM-planned cases.

C NAMED GUARDRAIL BASELINES

Named baselines are evaluated at their native information boundaries, while OpenAgentFlow additionally uses shared cross-channel provenance and control-plane policy state. The comparison shows why current-action abstractions differ from session-level enforcement.

  • Baseline interfaces: Llama Guard receives textualized current actions, AgentSpec receives runtime facts at its DSL boundary, and VeriSafe uses action-local predicate abstraction with CHC verification.None receives OpenAgentFlow’s shared cross-channel provenance store.
  • Prompt-only advisory: Prompt-only advisory has no independent decision point after the agent emits an action.Its instruction is placed before action generation, so it does not mediate the emitted action at execution time.
  • Llama Guard: Llama Guard’s text-only interface excludes structured sources, targets, provenance identifiers, source-to-sink rules, FlowRules, and session memory.Serialized context adds text but does not become a provenance store.
  • Llama Guard: Serialized text context can perform worse than current-action-only classification because the model must infer value-source, sink, and policy relationships.The context variant receives prior source-phase text but lacks explicit provenance binding.
  • AgentSpec: AgentSpec checks current AgentEvent facts and runtime scope expressible at its DSL/interpreter boundary, without OpenAgentFlow session state.Its representative rules cover scope violations, high-risk operations, and normal calendar creation.
  • VeriSafe: VeriSafe’s fixed predicates distinguish ordinary edits, read checks, external sends, high-impact operations, and authorization using only current-action information.Its local CHC rules deny external sends or unauthorized high-impact actions.

D OFFLINE EFFECTIVENESS, LATENCY, AND FALLBACK

The offline evaluation examines broad-suite effectiveness, latency, semantic fallback, and dynamic policy updates. It shows strong broad-suite performance, lightweight fast-path checks, controlled fallback recovery, and mostly correct post-installation rule changes, with matcher calibration failures remaining.

  • Offline suite design: The broad suite includes ordinary cases, boundary cases, deliberate misses, and benign values resembling sensitive data.
  • Broad-suite outcomes: The two hard categories account for most residual errors: obfuscated leaks are missed, while benign identifier-like values can be over-blocked.
  • Architecture comparison: 166/300 accuracy cases passed, with 55.3% accuracy, 37.8% attack block rate, and 21.1% false block rate for the VeriSafe GPT-to-CHC adaptation.
  • Architecture comparison: 1206.1 ms average latency across 500 offline cases was dominated by the GPT-4o predicate-abstraction and local CHC verification path.
  • Fast-path latency: T1 captures explicit scope, high-risk, and installed fast-path rules, while T2 provides the largest gain because pattern- and provenance-based propagation dominates the broad suite.
  • Semantic fallback: T1+T2 pass benign no-provenance cases but miss direct sensitive writes; T3/T4 recover these controlled fallback cases without claiming arbitrary semantic understanding.
  • Dynamic policy updates: Six core dynamic-policy insertion cases switch from ALLOW to DENY after rule installation, while 11/12 negative controls remain allowed.
  • Dynamic policy updates: The three retained failures are matcher calibration issues involving short digits, partial phone suffixes, and case-sensitive email comparison.

F THREAT COVERAGE AND PROVENANCE BOUNDARIES

The threat and provenance evaluations test whether policy abstraction extends beyond a single contact-to-calendar flow and expose the enforcement boundary of observed provenance. Residual failures include misses, ambiguities, obfuscated leaks, and over-tainting of benign values.

  • Scope: The threat and provenance suites test generalization beyond a single contact-to-calendar flow and expose the trust boundary of enforcement-observed provenance.
  • Threat scenario outcomes: The threat evaluation retains high-risk intra-app misses, one prompt-injection consequence miss, one purchase-boundary ambiguity, obfuscated leaks, and benign public contact-like values.
  • Provenance boundaries: Metadata-only provenance forgery is ignored because fast-path enforcement uses evidence observed by controllers, wrappers, or PEP memory rather than agent claims.
  • Provenance boundaries: When provenance is stripped, direct sensitive-payload inspection can still catch obvious values, but numeric reuse can over-taint benign values and non-standard formats remain boundary cases.

G END-TO-END TRACE EXAMPLES

The end-to-end traces show the PEP converting pending actions into AgentEvents, consulting session memory and policy state, and allowing, denying, or rewriting actions before execution. The examples demonstrate value-specific provenance, sink blocking, and remediation without erasing prior observations.

  • Trace processing: Each trace turns a pending action into an AgentEvent, consults session memory and policy state, then allows, denies, or rewrites it before execution.
  • Unsafe endpoint chain: Contacts lookup is allowed, but later Calendar and Mail actions are denied when their payloads reuse recorded Contacts-derived phone or email values.
  • Safe variant: The safe chain remains allowed because it writes only agenda text and meeting time, showing that provenance memory is value-specific rather than session-wide poisoning.
  • Remediation and later reuse: A rewrite removes a Contacts-derived phone number and permits the cleaned Calendar action, while later reuse of that number remains denied.

H ANDROID RAW AND SEMANTIC LABELING

The Android suite distinguishes raw accuracy from trace-adjusted accuracy because some cases have divergent raw and semantic labels or no available trace. Among 98 traced cases, trace adjustment improves the reported pass rate.

  • Labeling cases: Table 25 presents representative Android cases where raw and semantic labels diverge or where no trace is available.
  • Accuracy: 90.8% raw accuracy was achieved on 89/98 traced Android cases.
  • Accuracy: 92.9% trace-adjusted accuracy was achieved on 91/98 traced cases, while two UNKNOWN/no-trace executions were excluded from accuracy, block-rate, and false-block denominators.
  • Timing boundary: The 244.8-second average emulator task duration measures task execution time, not PEP decision latency.

I KNOWN FAILURE CASES AND BOUNDARY ANALYSIS

The evaluation retains known failures to characterize the prototype’s current boundary rather than removing them from the reported suites. These failures distinguish matcher, provenance, and real-execution integration limitations, while identifying needs for production deployment.

  • Known failures remain in the evaluation suites rather than being excluded from the reported results.Table 26 summarizes the main failure types.
  • The failure taxonomy separates prototype matcher limits, provenance limits, and real-execution integration issues.
  • The prototype’s boundary includes needs for stronger normalization, broader endpoint coverage, confirmation for ambiguous purchases, and more robust GUI execution traces.These requirements follow from the retained failures identified in the analysis.
Loading 2609.00015v1…