PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges

Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin, Prasant Mohapatra

arXiv:2510.23883v3cs.AI

TL;DR

Agentic AI’s autonomy, tool use, and coordination raise security risks, while current research on those risks and defenses needs systematization. This survey organizes threats, reviews defenses and evaluation approaches, and synthesizes findings that include vulnerabilities and reliability challenges in agent systems.

  • Problem

    Agentic AI combines base-LLM vulnerabilities with new risks from agent interactions, while the growing security literature needs systematization across threats, defenses, and open challenges.

    Source: §II · p. 3

  • Method

    The survey organizes agentic AI attacks into threat categories and reviews defenses, security controls, benchmarks, evaluation metrics, and open challenges.

    Source: §III · p. 3, §VII · p. 21

  • Results

    The surveyed evidence reports vulnerabilities to prompt injection, retrieval-based backdoors, and inter-agent trust exploits, alongside brittle task performance and interface-dependent outcomes.

    Source: §II · p. 3, D. INTERFACE AND ENVIRONMENT RISKS · pp. 11–12

  • Takeaways & Limitations

    Security and trustworthiness are prerequisites for safe deployment because autonomy and persistence expand the attack surface, tool integration magnifies misuse, and coordination can introduce unpredictability.

    Source: §I · p. 2

Abstract

from arXiv · show

Agentic AI systems powered by large language models (LLMs) and endowed with planning, tool use, memory, and autonomy, are emerging as powerful, flexible platforms for automation. Their ability to autonomously execute tasks across web, software, and physical environments creates new and amplified security risks, distinct from both traditional AI safety and conventional software security. This survey outlines a taxonomy of threats specific to agentic AI, reviews recent benchmarks and evaluation methodologies, and discusses defense strategies from both technical and governance perspectives. We synthesize current research and highlight open challenges, aiming to support the development of secure-by-design agent systems.

I. INTRODUCTION

AI has expanded from bounded, task-specific systems to LLM-powered agents that plan, use memory and tools, and act across changing environments. These capabilities enable multi-step automation across domains while creating security risks that motivate this survey’s review of attacks, defenses, benchmarks, and open challenges.

I. INTRODUCTION

  • LLMs can generate responses across many domains, but most deployments remain passive and do not independently pursue goals, maintain memory, or act without human supervision.Earlier AI systems are described as optimized for constrained datasets and bounded task-specific applications.Source: §I · p. 1
  • Agentic AI extends LLMs with goal-directed reasoning, planning, persistent memory, and tools for acting in digital or physical environments.Unlike passive LLMs, agents can deliberate over time, coordinate with other agents, and adapt to changing contexts.Source: §I · p. 1, Figure 1 · p. 2
  • Tooling ecosystems and research prototypes have made agent architectures more accessible for multi-step and open-ended tasks.Examples include LangChain, AutoGPT, Voyager, and enterprise deployments in customer support and data analysis.Source: §I · p. 1
  • 13.86% of real-world issue-resolution tasks were solved autonomously by Devin, compared with 1.96% by prior models.The section also describes agentic systems for procurement, personalized support, research, collaboration, and physical systems.Source: §I · p. 2
  • Agentic systems introduce security risks because autonomy and persistence expand the attack surface, tool integration magnifies potential misuse, and agent coordination can introduce unpredictability.The section presents security and trustworthiness as prerequisites for safe deployment in societal applications.Source: §I · p. 2

II. MOTIVATION AND CONTRIBUTIONS

The survey organizes agentic AI security threats and reviews prompt-injection variants alongside autonomous cyber-exploitation and tool abuse. It frames threat categories as analytical groupings that can overlap, and highlights the need to systematize threats and defenses while identifying open challenges.

III. TAXONOMY OF SECURITY THREATS

  • The taxonomy groups threats into prompt injection, autonomous cyber-exploitation, multiagent and protocol-level threats, interface and environment risks, and governance and autonomy concerns.These categories organize attacks by primary distinguishing properties rather than defining mutually exclusive classes.Source: §III · p. 3

A. PROMPT INJECTION AND JAILBREAKS

  • Prompt injection uses malicious instructions to divert an agent, while indirect attacks place them in external data; adaptive attacks penetrated eight IPI defenses with 50% success.For indirect attacks, the attacker may be the owner or supplier of third-party information processed by the agent [57].Source: A. PROMPT INJECTION AND JAILBREAKS · pp. 3, 5
  • Unintentional prompt injection can arise from ambiguous requests or drift in long chat histories, making alignment under shifting inputs a security challenge.The survey distinguishes these benign-interaction risks from deliberate adversarial injections.Source: A. PROMPT INJECTION AND JAILBREAKS · pp. 5–6
  • Prompt injections span text and code, images, video, audio, and combinations of modalities; image perturbations can manipulate agents, while audio perturbations can embed malicious instructions.The survey also notes video concerns because video can be decomposed into image key frames.Source: A. PROMPT INJECTION AND JAILBREAKS · p. 6
  • Cross-modal prompt injection combining aligned vision and text signals boosted attack effectiveness by at least 30.1% across various tasks.Adversarial image patches in screenshots have also hijacked multimodal OS agents into executing harmful commands regardless of screen layout or user request [90].Source: A. PROMPT INJECTION AND JAILBREAKS · p. 6
  • Attack propagation ranges from non-propagating data extraction to recursive compromise across future interactions and autonomous spread across agents or system boundaries.Obfuscation can hide instructions in encodings or low-resource languages, while payload splitting hides them across benign-seeming inputs until aggregation.Source: A. PROMPT INJECTION AND JAILBREAKS · pp. 6–7
  • Agents with code-execution or system tools can identify and carry out cyberattacks without direct human supervision, including sandboxed website compromises and one-day vulnerability exploitation.GPT-4 achieved an 87% success rate when given CVE descriptions and generally outperformed the examined models and conventional scanners [98].Source: A. PROMPT INJECTION AND JAILBREAKS · pp. 7–8
  • Website-hacking agents use multi-step exploits, and their performance drops markedly without document access or detailed prompting; cooperative tool use also motivates containment and runtime monitoring.Reported exploit chains include XSS+CSRF, server-side template injection, and blind SQL union injections [81], –.Source: A. PROMPT INJECTION AND JAILBREAKS · p. 8

C. MULTI-AGENT AND PROTOCOL-LEVEL THREATS

Multi-agent systems introduce protocol-mediated and cross-domain threats that can compromise coordinated workflows, not just individual agents. The section surveys MCP and A2A attacks and organizes broader risks from the threat actor’s perspective.

Protocol-Level MCP and A2A Attacks

  • Protocol-mediated interactions let adversaries compromise coordinated workflows through message tampering, role spoofing, and protocol exploitation.The section argues that compromising discovery, authentication, or task orchestration can escalate local manipulation to system-level compromise.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · pp. 8–9
  • MCP threats include flooding and replay that can cause denial of service, credential compromise enabling impersonation, and attacks on retrieved context, training, or confidentiality.These attacks exploit both the protocol and agents’ interactions with MCP-mediated resources.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · p. 8
  • Figure 6(a) illustrates MCP response tampering: a compromised token lets an agent accept manipulated contact details, while poisoned data can influence later actions.Attackers may also access the consulted database or file system and poison its underlying data.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · pp. 8–9, Figure 6 · p. 9
  • A2A threats include fake agent advertisements, unauthorized registration, and recursive delegation that can trigger denial of service; forged discovery responses can introduce malicious agents.Agent-to-agent attacks also include transitive prompt injection, context tampering, and memory manipulation.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · p. 9

Threat Actor Perspective

  • The threat-actor taxonomy names six classes: impersonation and role abuse, coordination manipulation, knowledge and learning manipulation, inference and policy evasion, accountability obfuscation, and confidentiality and integrity breaches.Impersonation exploits weak identity and trust management; runtime team formation and recursive delegation can spread adversarial influence, while manipulated reward signals can propagate misaligned behavior.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · p. 10
  • Cross-agent inference can reconstruct restricted data from individually innocuous outputs, while fragmented records and hidden cross-domain causality complicate accountability.The section notes that conventional single-agent safety assumptions and several existing interpretability tools provide limited coverage in these settings.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · p. 10
  • Encrypted, distributed workflows reduce direct exposure but still leave confidentiality and integrity gaps; available cryptographic techniques offer partial mitigation and can be computationally expensive.The section describes risks from modifying decrypted results or interfering with intermediate computations across domains.Source: C. MULTI-AGENT AND PROTOCOL-LEVEL… · pp. 10–11

D. INTERFACE AND ENVIRONMENT RISKS

Interface and environment risks arise from mismatches and variability in how agents perceive and act in web-based or embodied settings. The section describes fragility in action spaces, state tracking, stopping decisions, generalization, and reproducibility.

Observation and Action Space Misalignment

  • Misalignment between text-trained LLMs and grounded actions can impair perception and execution; streamlining action spaces and improving observations can enhance trajectory stability and task success.Examples include removing low-utility commands, abstracting compound actions, trimming duplicate histories, and supplying complete page states.Source: D. INTERFACE AND ENVIRONMENT RISKS · p. 11

Perception-Action Fragility in Realistic Environments

  • In WebArena, GPT-4 may repeat prior inputs, reflecting difficulty incorporating short-term state and action history; dialogue-oriented training is offered as a likely contributor.The passage links these shortcomings to training that prioritizes short-term instruction-response behavior over long-term embodied state tracking.Source: D. INTERFACE AND ENVIRONMENT RISKS · p. 11

Premature Termination and Achievability Misjudgement

  • Removing WebArena’s explicit unachievable-task hint raises GPT-4 task success by 14.41% but lowers true-positive detection of impossible tasks to 44.44%; 54.9% of feasible tasks are mislabeled impossible under the instruction setting.The comparison illustrates sensitivity of stop/continue behavior to instruction changes.Source: D. INTERFACE AND ENVIRONMENT RISKS · p. 11

Brittleness of Templates, Feedback, and Memory

  • GPT-4 succeeds consistently on only 4 of 61 WebArena task templates, and other benchmarks report sharp drops or instability across task variations and interface states.The section connects this brittleness to the need for memory, feedback, and improved planning primitives.Source: D. INTERFACE AND ENVIRONMENT RISKS · p. 11

Dynamic Content, Localization, and Robot Detection

  • Localization, dynamic page elements, and robot checks make web-agent trials less consistent; advanced multimodal agents achieve at best 40% success on CAPTCHAs, versus nearly 100% for humans.The section identifies reproducibility, reliability, and scalability as persistent challenges in realistic web environments.Source: D. INTERFACE AND ENVIRONMENT RISKS · pp. 11–12

E. GOVERNANCE AND AUTONOMY CONCERNS

As agents become more independent and receive less oversight, the paper identifies safety, security, trust, and responsibility concerns, especially in high-stakes applications. It recommends human-in-the-loop control, structured autonomy levels, and governance frameworks that bound self-directed behavior.

Governance and Autonomy Concerns

  • Reduced human oversight and control raise safety, security, and trust concerns for autonomous agents that can write and run code.The section also notes potential unpredictable actions and consequences such as disinformation and hijacking.Source: E. GOVERNANCE AND AUTONOMY CONCERNS · p. 12
  • In high-stakes settings such as autonomous weapons, ineffective oversight raises ethical questions about power and responsibility.Source: E. GOVERNANCE AND AUTONOMY CONCERNS · p. 12
  • The paper reports recommendations for human-in-the-loop control and structured autonomy levels, alongside governance frameworks defining acceptable bounds for self-directed behavior.Source: E. GOVERNANCE AND AUTONOMY CONCERNS · p. 12

IV. DEFENSES AND SECURITY CONTROLS

The paper reviews proposed agentic AI defenses and emphasizes that evolving threats require continued optimization and evaluation. It also introduces a comparison of secure-by-design coverage across defense approaches.

Defenses and Security Controls

  • Because attack vectors and threats continually evolve, agentic AI defenses need ongoing optimization and evaluation.Source: §IV · p. 12
  • The section reviews existing defense strategies and frameworks while identifying potential shortcomings.Source: §IV · p. 12
  • Table 1 presents secure-by-design coverage of security-critical defense components across defense approaches.Source: §IV · p. 12

A. PROMPT-INJECTION-RESISTANT DESIGNS

Prompt-injection defenses operate within the agent, rely on users, or apply external system controls; the approaches trade off robustness, automation, generality, and deployment cost.

Agent-Focused

  • Agent-focused defenses use prompt engineering, runtime behavior constraints, or injection-aware fine-tuning to help models recognize and resist malicious instructions.Examples include instruction hierarchies, StruQ, and SecAlign [177].Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · p. 12

Training-Based and Training-Free Defenses

  • Training-based defenses can degrade general-purpose capability without providing significant protection against adaptive attacks.Defensive fine-tuning evaluations reported this trade-off.Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · p. 12

User-Focused

  • User-focused defenses enlist human verification, attribution, or control-flow signals, but can reduce automation and reliability.Human confirmation may invite inattentive approvals; known-answer detection uses cryptographic tokens to signal compromise,.Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · pp. 12–13

System-Focused

  • System-focused defenses add external verification, controls, or constrained execution, including planning safeguards and runtime checks.TrustAgent uses pre-, in-, and post-planning safeguards, while Melon uses masked re-execution and tool-call similarity checks [70].Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · p. 13
  • System-level detection, isolation, prompt augmentation, and quality checks screen inputs or outputs, restrict actions, or flag anomalous language signals.Prompt augmentation is simple to deploy but can be bypassed by adaptive attacks; perplexity is one anomaly signal.Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · pp. 13–14

Trade-off comparison of agent-, user-, and system-focused defenses

  • Hybrid designs are favored because agent-, user-, and system-focused defenses each have distinct trade-offs in robustness, automation, and deployment cost.User checks can limit automation and fail when the user is the attacker; agent-focused defenses may require retraining or falter against adaptive attacks, while system controls add complexity and remain vulnerable to sophisticated attacks.Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · p. 14

Training-Based and Training-Free Defenses

  • Training-based methods add learning or auxiliary detectors, whereas training-free methods modify prompts or constrain behavior; lightweight tool filters can reduce utility and remain vulnerable to tailored attacks.Known-answer detection is post-hoc, while tool filtering limits calls to predefined sets [188].Source: A. PROMPT-INJECTION-RESISTANT DESIGNS · p. 14

B. POLICY FILTERING AND ENFORCEMENT

Policy enforcement either governs agent actions and decision sequences at runtime or scans inputs and outputs for violations.

Policy Filtering and Enforcement

  • Policy enforcement proactively restricts, intervenes in, or adjusts agent actions to keep behavior within established safety and security limits.Deployment guidance emphasizes layered, modular guardrails [196],.Source: B. POLICY FILTERING AND ENFORCEMENT · p. 14

Governance-Centric (Runtime) Enforcement

  • Governance-centric enforcement embeds policies in the agent loop or delegates oversight, enabling runtime validation of actions and decision sequences.GuardAgent applies configured constraints [198], AgentSpec specifies runtime policies [199], ShieldAgent reviews action sequences [200], and R²-Guard combines detection with logical inference [201].Source: B. POLICY FILTERING AND ENFORCEMENT · p. 14

Signal-Centric (Non-runtime) Enforcement

  • Signal-centric enforcement scans inputs or outputs and blocks harmful prompts or unsafe responses without restricting internal agent behavior.Examples cover text, images, and video [193],, [203]; another framework rejects or sanitizes outputs containing prompt injections [95].Source: B. POLICY FILTERING AND ENFORCEMENT · pp. 14–15

C. SANDBOXING AND CAPABILITY CONFINEMENT

Sandboxing contains agent risks by isolating execution contexts and limiting capabilities, while formal approaches seek guarantees beyond practical containment.

Sandboxing and Capability Confinement

  • Sandbox tests and containerized environments exercise generated or third-party code under constrained conditions before deployment.Examples include handcrafted unsafe-code scenarios, unit-test execution in gVisor or Docker, and plugin isolation [204]–.Source: C. SANDBOXING AND CAPABILITY CONFINEMENT · p. 15
  • Least-privilege architectures confine agent capabilities, including dual-agent code evaluation and isolated execution layers.A deterministic sandbox can also envelop REST calls, database changes, and file operations during reinforcement-learning reward computation.Source: C. SANDBOXING AND CAPABILITY CONFINEMENT · p. 15
  • Formal sandboxing can provide provable safety and security for continuous-control systems independently of the underlying AI model.This approach separates enforcement from controller design rather than shaping rewards during training.Source: C. SANDBOXING AND CAPABILITY CONFINEMENT · p. 15
  • Sandboxing isolates execution contexts to contain risks, prevent privilege escalation, and limit damage if an agent is compromised.It differs from policy enforcement at the reasoning or output-filtering level.Source: C. SANDBOXING AND CAPABILITY CONFINEMENT · p. 15
  • Sandboxing can incur efficiency and collaboration costs, and implementation or dependency-pipeline flaws can permit leakage or code execution.The survey recommends combining sandboxing with policy enforcement or runtime monitoring.Source: C. SANDBOXING AND CAPABILITY CONFINEMENT · p. 15

D. DETECTION AND MONITORING

The survey reviews monitoring and organizational frameworks alongside capability and security benchmarks, which probe reliability, safety, control, and agent failure modes.

D. DETECTION AND MONITORING

  • Monitoring approaches include proactive reachability analysis, decentralized runtime enforcement, and adaptive attacker-defender modeling.The survey notes that static monitoring can be insufficient against adaptive adversaries; decentralized enforcement supports local adaptations in multi-agent settings, [216].Source: D. DETECTION AND MONITORING · p. 16

E. STANDARDS AND ORGANIZATIONAL MEASURES

  • Organizational guidance includes NIST, NCCoE, OWASP, and CSA frameworks addressing risk management, cybersecurity controls, vulnerabilities, and lifecycle threat modeling.Together these frameworks place agent safeguards within broader regulation, compliance, and risk-management structures, –.Source: E. STANDARDS AND ORGANIZATIONAL MEASURES · p. 16

V. BENCHMARKS AND EVALUATION

  • DoomArena provides configurable cross-environment attack composition and fine-grained threat modeling for probing agent failures and defense limitations.The evaluation section covers capability benchmarks, security-specific frameworks, and methods intended to improve fidelity, comparability, and reproducibility.Source: §V · p. 16

A. LANDSCAPE BENCHMARKS FOR CAPABILITY

  • BrowserGym unifies observation and action spaces across web tasks, enabling like-for-like comparison while showing performance sensitivity to models and environments.Its interface includes tasks from MiniWoB++, WorkArena, and WebArena [160].Source: A. LANDSCAPE BENCHMARKS FOR CAPABILITY · pp. 16–17
  • τ-bench reports that recent models fail on over half of realistic-domain tasks, with pass∧8 below 25% in retail.Its pass∧k metric measures consistency across repeated runs of the same task [223].Source: A. LANDSCAPE BENCHMARKS FOR CAPABILITY · p. 17
  • ST-WebAgentBench separates task success from unsafe or noncompliant decisions, which prior work often undermeasures.It evaluates enterprise web agents using Completion under Policy and Risk Ratio metrics [234].Source: A. LANDSCAPE BENCHMARKS FOR CAPABILITY · pp. 17, 19
  • AgentHarm finds that simple jailbreak templates can markedly increase harmful-request compliance while largely preserving multi-step tool execution ability.It uses synthetic tools to simulate actions safely across multiple harm categories [44].Source: A. LANDSCAPE BENCHMARKS FOR CAPABILITY · p. 19
  • OS-Harm evaluates desktop-agent accuracy and safety adherence, while R-Judge assesses risk recognition and ToolEmu probes risky behavior in an emulated tool sandbox.These benchmarks examine side effects, trajectory risk assessment, and potential negative outcomes [208], [235],.Source: A. LANDSCAPE BENCHMARKS FOR CAPABILITY · p. 19

C. EVOLUTION OF AGENTIC SECURITY EVALUATION

The section proposes evaluation practices that look beyond end-state success to capture agent behavior, reliability, and judgment across repeated trials. It also emphasizes safe, reproducible testing and transparent reporting of benchmark conditions.

Process-aware evaluations

  • Trajectory-level scoring can reveal near-misses, rolled-back unsafe tool calls, and brittle plans that end-state success/failure metrics miss.Trace-level judges can detect policy violations or side effects, providing finer-grained feedback for training guardrails and control measures.Source: C. EVOLUTION OF AGENTIC SECURITY… · p. 19

Repeated trial metrics

  • Security evaluations should report performance distributions, including pass∧1 and pass∧k across several k values, rather than a single average.The τ-bench pass∧k metric measures reliability under repeated trials; pass@k instead fits tasks such as code generation where one correct generation can suffice.Source: C. EVOLUTION OF AGENTIC SECURITY… · p. 19

Standardizing judges and reducing judge bias

  • LLM judges can be biased by prompt design, trace format, or model choice, so safety evaluations should report human agreement, format sensitivity, and stability.Agent-as-a-Judge uses an evaluative agent to reason over trajectories and provide structured critique and scoring.Source: C. EVOLUTION OF AGENTIC SECURITY… · p. 19

Sandboxing and emulation to contain risk

  • Emulation and VM-backed sandboxes enable safer, deterministic, repeatable security experiments, but benchmark reports should state their fidelity assumptions.Results are more useful when emulated APIs, latency, and error modes closely match the real use case.Source: C. EVOLUTION OF AGENTIC SECURITY… · p. 19

Reproducibility and comparability

  • Comparable security benchmarks should use unified logging, seeded randomness, and fixed observation/action spaces, while documenting releases, costs, latency, determinism, and dataset hygiene.These practices address apples-to-oranges comparisons, safety-system overhead, dynamic environments, and contamination risks.Source: C. EVOLUTION OF AGENTIC SECURITY… · pp. 19–20

VI. OPEN CHALLENGES

The open challenges include measuring safety over long agent episodes and securing collaborations among agents. The section identifies specific evaluation gaps and avenues for improving resilience in both settings.

A. LONG-HORIZON SECURITY

  • Long-horizon evaluations should test temporal robustness across subgoals, interruptions, and distribution shifts, and detect latent misbehavior that may activate only under triggers.Long workflows expose performance and reliability gaps, while memory errors and partial observability can compound security risks beyond those seen in shorter tasks.Source: A. LONG-HORIZON SECURITY · p. 20

B. NOVEL MULTI-AGENT SECURITY CONSIDERATIONS

  • Multi-agent security work should strengthen authenticated messaging, disagreement and rollback mechanisms, and sentinel agents that regulate worker actions.A single faulty or adversarial agent can cascade failures, with resilience varying by organizational structure; reviewers or challenge mechanisms can improve error recovery.Source: B. NOVEL MULTI-AGENT SECURITY… · p. 20

C. IMPROVED SAFETY AND SECURITY BENCHMARKS

The open challenges span richer security benchmarks, adaptive attacks, physical agents, and human-agent interfaces. The survey concludes by synthesizing threats, defenses, evaluation approaches, and directions for future work.

C. IMPROVED SAFETY AND SECURITY BENCHMARKS

  • Security benchmarks should score trajectory segments and performance distributions, not only task completion or average performance from a single run.The section also calls for standardized judges robust to adversarial influence and benchmarks for test-time reasoning capabilities.Source: C. IMPROVED SAFETY AND SECURITY… · p. 20
  • Sandbox and emulated environments need fidelity assessment against real-world conditions to support rigorous evaluation of attack damage.Security evaluations should also examine whether current environments are sufficiently faithful and propose alternatives when they are inadequate.Source: C. IMPROVED SAFETY AND SECURITY… · p. 20
  • Agentic security research could extend influence analysis to trace decisions to reference data, past states, or model components for security analysis.For static models, influence analysis has traced predictions to training data, model layers, and few-shot demonstrations.Source: C. IMPROVED SAFETY AND SECURITY… · p. 20

E. SAFETY AGAINST ADAPTIVE ATTACKS

  • Adaptive attacks remain underexplored in agent defense evaluations, motivating methods to assess them, test current defenses, and develop defenses that co-evolve.Attackers in this threat model know the defense employed; defenses initially reported as effective have often been undermined by later adaptive evaluation.Source: E. SAFETY AGAINST ADAPTIVE ATTACKS · p. 21

F. AGENTIC AI SECURITY IN THE PHYSICAL WORLD

  • Physical agents face sensor and hardware risks, while end-to-end models trade defensive separation for zero-shot generalization without corresponding robustness gains.The survey also identifies systematic evaluation of smart-home agent security as underexplored.Source: F. AGENTIC AI SECURITY IN THE PHYSICAL… · p. 21

G. HUMAN-AGENT SECURITY INTERFACES

  • Human-agent interfaces need manipulation-resistant interactions and auditing tools that make long traces interpretable without overburdening human verification.The section also calls for studying exploitation of user trust and for empirical work on human factors alongside technical defenses.Source: G. HUMAN-AGENT SECURITY INTERFACES · p. 21

VII. CONCLUSION

  • The survey synthesizes agentic security threats, defenses, benchmarks, and open challenges to support research on safer, more secure frameworks for large-scale societal use.It aims to provide a distilled introduction to the field and galvanize further research progress.Source: §VII · p. 21
Loading 2510.23883v3…