PaperlayerView exact revision on arXiv

Source-linked AI summary

Trustworthy Agentic AI: A Comprehensive Cybersecurity and Systems Survey on Threat Landscapes, Defense Architectures, and Open Challenges

Seyedakbar Mostafavi

arXiv:2609.13731v1cs.AI

TL;DR

Autonomous agent systems blur security boundaries because untrusted natural-language inputs can influence control and tool execution. This survey synthesizes 206 studies and governance standards into a systems-security framework, formalizing agent architecture, threats, trustworthiness, and defenses. It presents a multi-layered zero-trust blueprint and maps technical controls to evaluation benchmarks and governance frameworks.

  • Problem

    Agent tool access expands prompt-injection consequences to arbitrary host takeover, remote code execution, and cloud privilege escalation.

    Source: B. The Cybersecurity Dilemma in… · p. 2

  • Method

    The survey synthesizes research on agent architectures, threat surfaces, trustworthiness, defenses, benchmarks, and governance into a systems-level framework.

    Source: §X · pp. 20–21

  • Results

    The survey formalizes agents as a stateful 5-tuple, establishes a six-dimensional taxonomy, analyzes threats across execution and interaction planes, and formulates a zero-trust defense blueprint.

    Source: §X · p. 20

  • Takeaways & Limitations

    The blueprint combines isolation, capability-based access control, sandboxing, kernel monitoring, signed memory provenance, and Byzantine fault-tolerant consensus.

    Source: §X · pp. 20–21

    PBFT across agent nodes incurs quadratic token consumption and minutes of decision latency, limiting real-time cluster throughput.

    Source: C. Scalable Byzantine Swarm Consensus… · p. 19

Abstract

from arXiv · show

The transition from passive foundation models to autonomous, goal-directed agentic AI systems has introduced unprecedented capabilities by coupling recursive cognitive reasoning loops, persistent memory architectures, live tool execution planes, and multi-agent collaboration topologies. However, granting probabilistic neural cores execution authority across filesystems, networks, and cloud infrastructure dissolves classical security perimeters: natural language simultaneously serves as input data, internal control code, and communication protocols, exposing a Turing-complete blast radius where untrusted data represents executable instructions. This survey delivers a comprehensive systems-security reference framework for trustworthy agentic AI, synthesizing 206 foundational studies and regulatory standards. We formalize the general agent architecture as a stateful 5-tuple and establish a 6-dimensional trustworthiness taxonomy covering security, safety, privacy, explainability, fairness, and accountability. We systematically analyze threat surfaces across intra-execution loops and interaction planes, formulate a multi-layered zero-trust defense-in-depth architecture integrating Dual-LLM isolation, Capability-Based Access Control, kernel eBPF probes, and sandboxed runtimes, review standardized evaluation benchmarks, and map technical controls to international AI governance frameworks.

I. INTRODUCTION

Agentic AI combines autonomous reasoning, persistent memory, tool execution, and collaboration, but natural language blurs the boundary between untrusted input and executable control. This survey maps the resulting security gaps and reviews a 206-publication corpus.

I. INTRODUCTION

  • Natural language serves as input, internal control logic, and communication, so autonomous execution exposes systems to a Turing-complete blast radius.The survey describes risks including remote code execution, data exfiltration, and cloud compromise.Source: §I · p. 1

A. Taxonomy of Agent Execution Archetypes

  • Agent deployments span software engineering, web interaction, cybersecurity, robotics, and enterprise workflows, each with distinct execution surfaces.Software agents can hold broad filesystem privileges, while web agents encounter untrusted third-party content.Source: A. Taxonomy of Agent Execution… · p. 2

B. The Cybersecurity Dilemma in Agentic Autonomy

  • Three structural tensions define the cybersecurity dilemma: expanded blast radius, stochastic control flow, and machine-speed dual-use cyber operations.The survey contrasts prompt injection against conversational models with potential host takeover and cloud privilege escalation in agents.Source: B. The Cybersecurity Dilemma in… · p. 2

C. Four Critical Knowledge Gaps in Agentic AI Security

  • Four knowledge gaps concern multi-step input unpredictability, complex reasoning, variable operational environments, and interactions with untrusted external entities.Associated threats include goal hijacking, reasoning drift, sandbox escape, indirect prompt injection, vector poisoning, and Byzantine subversion.Source: C. Four Critical Knowledge Gaps in… · p. 2

D. Systematic Survey Methodology & Corpus Identification

  • A PRISMA-S-guided review and multi-stage filtering protocol produced a corpus of 206 verified foundational publications.The corpus spans agent reasoning, systems sandboxing, vector-memory integrity, and regulatory governance.Source: D. Systematic Survey Methodology &… · pp. 2–3

E. Comparison with Related Surveys and Our Contributions

The survey positions itself against work on general LLM applications, classical ML security, and narrow prompt injection, which it says lack a unified systems-security architecture for autonomous agents. It organizes its response around five research questions and four principal contributions.

E. Comparison with Related Surveys and Our Contributions

  • Prior surveys address general LLM applications, classical ML security, or narrow prompt injection, but do not unify security across agent loops, persistent memory, tools, and multi-agent consensus.The paper points to Table I for its comparison with related surveys.Source: E. Comparison with Related Surveys and… · p. 3, Table I · p. 4

F. Formal Survey Research Questions (RQs)

  • Five research questions cover systems formalization, trustworthiness dimensions, threat modeling, zero-trust defenses, and empirical evaluation with regulatory compliance.The questions map to Sections II through IX, including future research horizons.Source: F. Formal Survey Research Questions… · p. 3

G. Core Contributions of this Survey

  • Four contributions comprise a stateful 5-tuple and adversarial POMDP model, a six-dimensional taxonomy, dual-plane threat analysis, and a zero-trust defense blueprint.The blueprint synthesizes isolation, access control, kernel telemetry, sandboxing, memory provenance, and Byzantine consensus while comparing operational costs and deployability.Source: G. Core Contributions of this Survey · pp. 3–4

H. Paper Organization

  • The paper proceeds from agent architecture and trustworthiness through threats, defenses, benchmarks, governance, and open challenges before concluding.Table II summarizes key abbreviations.Source: H. Paper Organization · p. 4, Table II · p. 4

A. Unified Conceptual Framework and Formal System Tuple

The framework models an agent as a stateful system comprising cognition, memory, tools, an operating environment, and an execution policy. It places that system in a partially observable setting and describes how planning and observations shape subsequent actions.

A. Unified Conceptual Framework and Formal System Tuple

  • The stateful 5-tuple comprises a cognitive core, hierarchical memory, tool and action plane, external environment, and execution policy.Tools define schemas, runtimes, and permission envelopes; the policy maps interaction traces and memory to thoughts and tool invocations.Source: A. Unified Conceptual Framework and… · pp. 4–5

B. Partially Observable Markov Decision Process (POMDP) Formulation

  • The POMDP formulation captures agent operation without direct access to the complete underlying system state.Its state, action, transition, reward, observation, observation-probability, and discount components describe the environment and agent’s available signals.Source: B. Partially Observable Markov… · p. 5
  • The agent updates a belief distribution from its execution history and observations, then uses its policy to choose tool actions.Poisoned observations can divert belief away from the true state and toward unauthorized or policy-violating actions while the model remains confident.Source: B. Partially Observable Markov… · p. 5

C. Cognitive Reasoning & Planning Paradigms

  • The cognitive core decomposes high-level objectives, while ReAct interleaves reasoning and actions to incorporate tool outputs and environmental feedback.The passage labels the surrounding stages as perception, cognitive planning, and tool actuation, with memory context, self-reflection, and JSON-schema decisions also identified.Source: C. Cognitive Reasoning & Planning… · pp. 5–6
  • Tree-of-Thoughts and Graph-of-Thoughts search candidate reasoning states with heuristic evaluation and BFS, DFS, or MCTS lookahead.The survey says these methods improve task success in mathematical and software-engineering domains but expand the attack surface and pathways for biasing the search.Source: C. Cognitive Reasoning & Planning… · p. 5
  • Reflexion and Self-Refine use natural-language self-critique after failed validation or tests, storing reflections in episodic memory to steer later planning.The passage also names grounded planning systems and says embodied systems extend this principle with evolving skill libraries and curricula.Source: C. Cognitive Reasoning & Planning… · p. 5

Cloud APIs (S3, SQL, REST)

Agent systems rely on persistent memory, structured tool execution, and collaboration across distinct topologies. These capabilities introduce operational risks that motivate evaluating trustworthiness at the systems level.

D. Hierarchical Memory Subsystems

  • Hierarchical memory spans working context, episodic records retrieved by semantic similarity, and semantic or procedural knowledge in model weights and structured graphs.MemGPT-style virtual context paging moves memory blocks between context tiers through function calls.Source: D. Hierarchical Memory Subsystems · p. 6

E. Tool Integration Plane & Model Context Protocol (MCP)

  • Structured function calling validates model-generated JSON or XML against schemas before dispatching requests to local binaries or remote services.The action plane turns reasoning into computational side effects.Source: E. Tool Integration Plane & Model… · p. 6
  • MCP standardizes tool discovery, resource access, and operations through prompts, resources, and tools over local stdio or remote HTTP/SSE.Rogue servers may publish poisoned schemas or manipulate responses to hijack downstream planning.Source: E. Tool Integration Plane & Model… · pp. 6–7
  • Distributed agent clusters use isolation ranging from Linux namespaces and browser engines to micro-virtual machines.The survey identifies deterministic boundary isolation across worker pools as a systems requirement.Source: E. Tool Integration Plane & Model… · p. 7

F. Multi-Agent Collaboration Topologies

  • Multi-agent systems use hierarchical delegation, peer-to-peer negotiation, or shared blackboards and consensus mechanisms.These topologies can expose systems to cascading hallucinations, AI-worm propagation, and Byzantine consensus subversion.Source: F. Multi-Agent Collaboration Topologies · p. 7

III. THE 6-DIMENSIONAL TRUSTWORTHINESS TAXONOMY

  • Agent trustworthiness is framed as a dynamic systems-level guarantee because failure in one cognitive or execution module can compromise the whole socio-technical system.The paper introduces a six-dimensional trustworthiness taxonomy to structure its analysis.Source: §III · p. 7

A. Dimension 1: Security (Confidentiality, Integrity, Availability)

The taxonomy treats security as protection of agent components and actions, while safety, privacy, and explainability address operational harm, data exposure, and verifiability.

A. Dimension 1: Security (Confidentiality, Integrity, Availability)

  • Security protects the cognitive core, memory, tools, and inter-agent channels, with confidentiality, integrity, and availability requiring boundary filtering, authorization gates, and runtime limits.Confidentiality includes protecting prompts, schemas, credentials, and private memories; integrity covers reasoning, memory, and tool parameters; availability addresses denial-of-service risks.Source: A. Dimension 1: Security… · p. 7

B. Dimension 2: Safety & Operational Robustness

  • Safety constrains irreversible tool actions through authorization or rollback and uses schema validators to suppress hallucinated endpoints, parameters, and paths.Robustness also requires stable decisions and graceful handling of tool exceptions and unexpected observations.Source: B. Dimension 2: Safety & Operational… · p. 8

C. Dimension 3: Privacy & Data Protection

  • Privacy protection spans the data lifecycle, combining sanitized and zero-leakage tool dispatches with protected vector memories and isolated, purged session contexts.Embedding inversion and membership inference can expose private content, while shared caches and vector stores can leak data across users.Source: C. Dimension 3: Privacy & Data… · p. 8

D. Dimension 4: Explainability, Interpretability & Verifiability

  • Explainability requires action-linked audit traces, formal safety checks of symbolic plans, and tests that intermediate reasoning causally reflects action selection.Post-hoc rationalization alone is insufficient in high-stakes systems; symbolic plans can be checked with SMT solvers before execution.Source: D. Dimension 4: Explainability,… · p. 9

E. Dimension 5: Fairness & Non-Discrimination

This section extends trustworthiness to fairness, accountability, and cross-dimensional trade-offs, then surveys intra-execution threats from perception through action.

E. Dimension 5: Fairness & Non-Discrimination

  • Fairness controls include demographic-parity testing and counterfactual audits of tool policies, while multi-agent markets require verifiable auction rules and anti-collusion monitoring.The risks include unequal service quality or errors across protected groups, as well as tacit collusion, price-fixing, and predatory bidding.Source: E. Dimension 5: Fairness &… · p. 9

F. Dimension 6: Accountability, Auditability & Provenance

  • Agent actions need attributable, non-repudiable records, including DID-bound cryptographic signatures and immutable Merkle audit trees aligned with governance requirements.The survey names the NIST AI RMF 1.0, ISO/IEC 42001, and EU AI Act as frameworks for logging, risk management, and human oversight.Source: F. Dimension 6: Accountability,… · p. 9

G. Cross-Dimensional Pareto Trade-off Dynamics

  • Trustworthiness dimensions create engineering trade-offs: defense mechanisms add latency, differential-privacy noise reduces retrieval precision and task success, and restrictive safety filters can block benign work.The survey identifies balancing safety guarantees with problem-solving autonomy as an open challenge.Source: G. Cross-Dimensional Pareto Trade-off… · p. 9

IV. INTRA-EXECUTION SECURITY: PERCEPTION, BRAIN, AND ACTION THREATS

  • Intra-execution threats arise within the perception, brain, and action stages, and an attack on any stage can compromise the agent’s decision trajectory.The section’s threat taxonomy includes action-plane threats such as sleeper backdoors, reasoning poisoning, and hallucination cascades.Source: §IV · p. 9, §3 · p. 10, Figure 5 · p. 10

A. Perception Threats: Injections, Suffixes & Steganography

1) Direct Prompt Injection and Goal Hijacking:

  • Direct prompt injection can replace the original objective or elicit confidential prompts, tool definitions, API tokens, and credentials.The agent processes concatenated tokens without a native execution privilege hierarchy, so injected goals may receive authority equal to or greater than developer constraints.Source: 1) Direct Prompt Injection and Goal… · p. 10

2) Adversarial Suffix Optimization (GCG and AutoDAN):

  • GCG optimizes suffix tokens toward an affirmative target prefix, while AutoDAN produces readable, stealthy jailbreak prefixes that transfer across black-box models.GCG’s unreadable sequences may be detected by perplexity filters; AutoDAN uses genetic-algorithm-based extensions.Source: 2) Adversarial Suffix Optimization… · p. 10

3) Automated Black-Box Red-Teaming (PAIR and TAP):

The section contrasts automated and multimodal attacks with brain-level backdoors and planning failures, showing how reasoning topology shapes error propagation and recovery.

3) Automated Black-Box Red-Teaming (PAIR and TAP):

  • PAIR iteratively refines prompts against a target agent, while TAP searches a pruned branching tree and achieves over 80% jailbreak success in fewer than 20 model queries.Both frameworks automate black-box jailbreak discovery without white-box gradient access.Source: 3) Automated Black-Box Red-Teaming… · p. 10

4) Multimodal Steganography and Visual Jailbreaks:

B. Brain Threats: Backdoors, Hallucinations & Planning Vulnerabilities

1) Backdoors and Sleeper Agent Activation:

  • Sleeper-agent triggers can remain concealed during alignment evaluation and activate only when a deployment condition, such as a timestamp or secret phrase, appears.The passage states that standard safety alignment does not purge these triggers.Source: 1) Backdoors and Sleeper Agent… · p. 11

2) Chain-of-Thought Backdoor Injection (BadChain):

3) Hallucination Snowballing Mechanisms:

4) Planning Vulnerabilities Across Reasoning Topologies:

  • Linear CoT and ReAct propagate errors without branching recovery, whereas ToT, GoT, and LATS permit backtracking but expose heuristic scoring to manipulation that can favor malicious branches.An attacker who manipulates the score can cause safe paths to be pruned.Source: 4) Planning Vulnerabilities Across… · p. 11

C. Action Plane Threats: Parameter Injection, SSRF & Supply Chain

Action and interaction surfaces expose agents to command injection, confused-deputy abuse, malicious tool servers, environmental attacks, and persistent memory compromise.

1) Command Injection and Shell Parameter Smuggling:

  • Shell commands built through string interpolation let attacker-controlled filenames inject metacharacters and execute arbitrary commands with the agent’s host privileges.Source: 1) Command Injection and Shell… · p. 11

2) The Confused Deputy Problem in Tool Execution:

3) Model Context Protocol (MCP) and Supply Chain Vectors:

D. Empirical Case Studies and Real-World CVE Dissections

  • CVE-2023-34541 exposed database credentials through unparameterized SQL construction, while CVE-2024-21513 showed unconstrained AutoGPT subprocesses could enable complete host takeover.The cases demonstrate breaches stemming from failed isolation boundaries and uncontained execution privileges.Source: D. Empirical Case Studies and… · pp. 11–12

V. INTERACTION SECURITY: ENVIRONMENT, MEMORY, AND AGENT-TO-AGENT THREATS

  • Interaction security covers external environments, episodic memory, and peer agents, where data streams crossing network perimeters can become channels for remote compromise.Agents ingest unstructured content from the web, filesystems, email, and APIs, creating external vulnerabilities.Source: §V · p. 12, A. Threats on Agent-to-Environment… · p. 12

A. Threats on Agent-to-Environment Interfacing

  • Indirect prompt injection embeds instructions in retrieved external content, which can cause goal hijacking, unauthorized tool use, and automated propagation.Markdown rendering can also trigger zero-click exfiltration, while SSRF can expose cloud credentials; resource-exhaustion and sensor-spoofing attacks are additional risks.Source: A. Threats on Agent-to-Environment… · p. 12

B. Threats on Agent-to-Memory (RAG & Vector Poisoning)

  • RAG poisoning can make malicious documents rank among top-k results; fewer than five poisoned documents in a database of over one million chunks achieved over 90% top-1 attack success.This result is reported by Zou et al.. Vector embeddings can also be inverted to reconstruct over 70% of original private text.Source: B. Threats on Agent-to-Memory (RAG &… · p. 13
  • Shared vector stores and KV caches can leak one user’s data to another, while asynchronous memory updates can leave distributed agents with stale or divergent state.Source: B. Threats on Agent-to-Memory (RAG &… · p. 13

C. Threats on Agent-to-Agent (Multi-Agent Swarms)

Multi-agent systems can amplify errors and enable attacks that propagate across swarms. The survey pairs these risks with layered defenses for cognitive inputs and persistent memory.

C. Threats on Agent-to-Agent (Multi-Agent Swarms)

  • Upstream hallucinations or incorrect code can be treated as verified by downstream agents, amplifying errors and producing collective swarm failure rates above 65%, [151].This occurs in sequential or hierarchical handoffs.Source: C. Threats on Agent-to-Agent… · p. 13
  • A self-replicating Morris II worm can spread through agent messages, execute its prompt, extract contacts, and send newly poisoned messages to peers [143].The passage models propagation using epidemic differential equations.Source: C. Threats on Agent-to-Agent… · p. 13
  • A single adversarial image in a shared multi-agent RAG repository can jailbreak an entire swarm, while compromising at least n/3 agents breaches classical Byzantine fault-tolerance boundaries [152], –,.The stated outcomes include subverted consensus, falsified decisions, and manipulated decentralized markets.Source: C. Threats on Agent-to-Agent… · pp. 13–14
  • Sybil identities can manipulate reputation, voting, and market auctions, while agents in strategic games have demonstrated premeditated, multi-turn deception,, –.The passage identifies false alliances as one form of deception.Source: C. Threats on Agent-to-Agent… · p. 14

VI. ZERO-TRUST DEFENSE-IN-DEPTH ARCHITECTURE

  • The proposed four-layer zero-trust architecture combines mathematical, cryptographic, and kernel-level isolation rather than relying on prompt engineering or heuristic filters.The supplied layer descriptions cover cognitive guardrails and memory integrity.Source: §VI · p. 14

A. Layer 1: Pre-Execution & Cognitive Guardrails

  • An unprivileged Inspector converts external content into sanitized typed JSON, while the privileged Executor receives only verified entities inside immutable system-prompt envelopes [158].The Inspector has no tool-execution privileges and strips embedded natural-language commands.Source: A. Layer 1: Pre-Execution & Cognitive… · p. 14
  • Instruction-data attention delimiters are proposed to prevent external data tokens from influencing instruction decoding,.The passage describes directional visibility masks as the mechanism.Source: A. Layer 1: Pre-Execution & Cognitive… · p. 14

B. Layer 2: Memory Integrity & Cryptographic Provenance

  • Signed memory embeddings bind source hashes, timestamps, and author identity; recursively updated Merkle logs provide append-only provenance for forensic audits and non-repudiation,.Differential-privacy noise is also proposed to bound mutual information between stored vectors and source documents,,.Source: B. Layer 2: Memory Integrity &… · p. 14

C. Layer 3: System Sandboxing & Least Privilege Execution

The architecture restricts tool authority and isolates execution, then applies authentication and resilience controls to agent swarms. Its controls are allocated with operational cost and task criticality in view.

C. Layer 3: System Sandboxing & Least Privilege Execution

  • Tool access uses cryptographically signed, fine-grained, short-lived capability tokens, while micro-VMs or WebAssembly sandbox tools and eBPF probes monitor sensitive system calls,, –,,.Tokens are restricted to one operation with bounded parameters and a time-to-live under 60 seconds; the gateway validates scope before issuing a system call.Source: C. Layer 3: System Sandboxing & Least… · pp. 14–15

D. Layer 4: Multi-Agent Zero-Trust Protocols

  • Swarm messages use unique DIDs and mutually authenticated TLS, while mission-critical decisions use threshold signatures and PBFT consensus,, –.Automated circuit breakers can sever routes and roll back shared blackboard states when anomalous behavior is detected.Source: D. Layer 4: Multi-Agent Zero-Trust… · pp. 15–16

E. Hardware-Assisted Confidential Computing (TEEs)

  • Hardware trusted execution environments can protect model weights, active context, and keys from host-level compromise and untrusted cloud hypervisors –.The passage names Intel SGX, AMD SEV-SNP, ARM TrustZone, and NVIDIA Confidential GPUs as examples.Source: E. Hardware-Assisted Confidential… · p. 16

F. Comparative Trade-Off Synthesis of Defensive Controls

  • Defensive controls impose computational, financial, and operational trade-offs, so cloud deployments should co-schedule them according to task criticality and data classification.The passage contrasts continuous lightweight eBPF and CapBAC protections with reserving PBFT and Dual-LLM inspection for high-blast-radius nodes.Source: F. Comparative Trade-Off Synthesis of… · p. 16

VII. EMPIRICAL BENCHMARKS, AUDITING & EVALUATION FRAMEWORKS

The survey reviews interactive benchmarks, automated red-teaming, and quantitative metrics for assessing agent safety, adversarial resilience, and utility across execution trajectories.

A. State-of-the-Art Empirical Benchmark Suites

  • In InjecAgent, leading models including GPT-4 have attack success rates of 24.1%–45.3% across 17 tools and over 1,000 scenarios; BIPIA reports injection rates above 50% across standard LLMs [21], [144].Other suites assess tool selection and parameter security, API use, and multi-step database retrieval.Source: A. State-of-the-Art Empirical… · p. 16
  • AgentBench examines multi-turn decisions across operating systems, databases, web interfaces, and games; R-Judge evaluates safety and risk awareness over 1,624 interaction trajectories,.AgentBench reports that agents frequently execute destructive bash commands under ambiguous instructions.Source: A. State-of-the-Art Empirical… · p. 16
  • GAIA reports 92% completion for human baselines and under 10% for leading commercial agents on complex multimodal tasks; CyberSecEval, Machiavelli, and AgentHarm cover cyber capabilities, deception, power-seeking, and jailbreak resilience [28], [29],,, [185].Source: A. State-of-the-Art Empirical… · p. 16

B. Automated Adversarial Red-Teaming Methodologies

  • Automated attacker-target loops refine prompts using refusal signals to discover multi-step jailbreak vectors that bypass static heuristic filters [186]–[189].The reviewed methods also include MCP schema and grammar fuzzing and dynamic vector-poisoning probes.Source: B. Automated Adversarial Red-Teaming… · pp. 16–17

C. Standardized Quantitative Metric Formulations

  • The proposed metrics cover swarm cascade probability, unauthorized-action attack success, benign-task utility preservation, false safety rejections, and weighted blast radius.Blast radius weights data exfiltration, filesystem changes, and privilege escalation; the metric set is presented to support cross-benchmark comparability.Source: C. Standardized Quantitative Metric… · p. 17

VIII. AI GOVERNANCE, SAFETY STANDARDS, AND REGULATORY COMPLIANCE

The survey connects agent governance requirements to technical controls and identifies unresolved verification challenges. It emphasizes the limits imposed by translating natural-language goals into formal safety checks and exploring interactive state spaces.

A. International Regulatory Frameworks and Standards

  • NIST AI RMF functions GOVERN, MAP, MEASURE, and MANAGE frame lifecycle risk management, while its Generative AI Profile calls for asset discovery, tool-authorization tracking, red-team auditing, and pre-deployment risk quantification,.Source: A. International Regulatory Frameworks… · p. 18
  • The EU AI Act classifies agents in specified critical settings as high-risk and requires lifecycle risk management, data governance, event logging, and human oversight.Article 14 specifies human-in-the-loop, human-on-the-loop, and human-in-command oversight modalities.Source: A. International Regulatory Frameworks… · p. 18

B. Comparative Analysis: Technical Enforcement of Global AI Mandates

  • The survey maps traceability mandates to Merkle logging, least-privilege mandates to capability tokens and micro-VMs, and red-team requirements to CI/CD-integrated adversarial testing.The mappings cite EU AI Act Article 12 and ISO/IEC 42001, NIST AI RMF MANAGE 2.4 and OWASP LLM06, and NIST AI RMF MEASURE 2.6, respectively.Source: B. Comparative Analysis: Technical… · p. 18

IX. OPEN RESEARCH CHALLENGES & FUTURE HORIZONS

  • Probabilistic models cannot currently provide mathematical proof that proposed tool-action sequences satisfy safety invariants before execution,,.The survey frames this as a foundational open problem for trustworthy agents.Source: A. Formal Verification of… · p. 18

A. Formal Verification of Neural-Symbolic Action Chains

  • Natural-language variability makes translation into formal logic vulnerable to hallucination, while interactive environment state spaces are unbounded and model checking is undecidable generally and PSPACE-complete under bounded horizons –.The passage identifies grammar-constrained decoding and bounded symbolic abstract interpreters as a research frontier, with sub-50ms execution budgets.Source: A. Formal Verification of… · pp. 18–19

B. Lifelong Dynamic Alignment & The Non-Invertible Side-Effect Dilemma

Agentic systems face security challenges from long-horizon alignment drift, irreversible external actions, and costly swarm consensus. The survey connects these problems to defense trade-offs, decentralized value networks, and open research directions.

B. Lifelong Dynamic Alignment & The Non-Invertible Side-Effect Dilemma

  • Long-horizon exposure to external observations can produce alignment drift, catastrophic forgetting, and sleeper backdoors, while emails, transfers, and webhook changes may be impossible to roll back.Shadow execution and staged verification for irreversible actions, plus memory scrubbing that preserves benign recall, remain open challenges.Source: B. Lifelong Dynamic Alignment & The… · p. 19

C. Scalable Byzantine Swarm Consensus in High-Latency Agent Clusters

  • PBFT tolerates up to f < n/3 adversarial nodes, but its all-to-all voting creates quadratic token costs and minutes of latency in model-based clusters.Real-time detection and isolation of compromised peers also remains open; graph telemetry with optimistic consensus is proposed as a research direction.Source: C. Scalable Byzantine Swarm Consensus… · pp. 19–20

D. Dual-Use Machine-Speed Cyber Warfare and Automated Patch Synthesis

  • Cyber agents can accelerate offensive and defensive operations, but defenders must secure every execution path while attackers need only one unmodeled edge case.The section identifies formal verification of generated patches as necessary to check for secondary security regressions and broken legacy logic.Source: D. Dual-Use Machine-Speed Cyber… · p. 20

E. Distributed Cluster Scheduling & Blast-Radius Pareto Optimization

  • Defense-in-depth controls impose computational, latency, and financial overhead, motivating risk-aware scheduling that varies guardrail rigor with an action’s blast radius.The proposed direction routes low-risk read-only queries through lightweight checks and high-impact actions through hardware-accelerated trusted execution environments.Source: E. Distributed Cluster Scheduling &… · p. 20

F. Decentralized Value Ecosystems, Cryptographic Provenance & ZK-ML

  • Decentralized agent commerce requires recognized cryptographic identities and secure settlement, while ZK-ML could prove policy-compliant tool actions without revealing weights, user data, or system prompts.The proposed proof links an action to a certified, untampered model adhering to developer safety policy.Source: F. Decentralized Value Ecosystems,… · p. 20

X. CONCLUSION

  • Agentic systems combine planning, persistent memory, tool actuation, and collaboration, collapsing the boundary between natural-language data and executable control.The survey characterizes untrusted external input as uncompiled executable code and formalizes the resulting closed-loop system with an adversarial POMDP model.Source: §X · p. 20
  • The survey formalizes agent architecture and trustworthiness, analyzes threats across execution and interaction surfaces, and proposes zero-trust defenses alongside benchmarks, governance mappings, and open research horizons.Its framework synthesizes 206 studies and regulatory standards, defining a stateful 5-tuple and a six-dimensional taxonomy. The defense blueprint combines isolation, access control, sandboxing, monitoring, cryptographic provenance, and Byzantine consensus.Source: §X · pp. 20–21
  • The survey recasts evaluation across six trustworthiness dimensions and synthesizes a zero-trust blueprint with comparative analysis of operational latency, token costs, computation, and deployability.It maps technical defenses to governance mandates and identifies open horizons including shadow execution, optimistic swarm consensus, and zero-knowledge proofs of agency.Source: §X · pp. 20–21
Loading 2609.13731v1…