Source-linked AI summary
Securing Generative AI Agentic Workflows: Risks, Mitigation, and a Proposed Firewall Architecture
Sunil Kumar Jang Bahadur, Gopala Dhar
TL;DR
GenAI agentic workflows create security risks across data, models, autonomous agents, and system interactions. The paper proposes a centralized GenAI Security Firewall integrating security services and GenAI-assisted defenses, and describes potential reductions in policy drift and prevalent attacks.
Problem
Method
The paper proposes a centralized GenAI Security Firewall that integrates security services, workflow monitoring, input and output checks, and GenAI-assisted defense.
Source: §4 · p. 2, §4.2 · p. 4, §5 · p. 5Results
The proposed centralized firewall claims 60-80% reduced policy drift and prevention of 80-90% of prevalent attacks such as prompt injection.
Source: §4.4 · p. 4Takeaways & Limitations
The paper concludes that robust security measures are essential for trustworthy and reliable deployment of GenAI-based agentic systems.
Source: §5 · p. 5
Abstract
from arXiv · showhide
Generative Artificial Intelligence (GenAI) presents significant advancements but also introduces novel security challenges, particularly within agentic workflows where AI agents operate autonomously. These risks escalate in multi-agent systems due to increased interaction complexity. This paper outlines critical security vulnerabilities inherent in GenAI agentic workflows, including data privacy breaches, model manipulation, and issues related to agent autonomy and system integration. It discusses key mitigation strategies such as data encryption, access control, prompt engineering, model monitoring, agent sandboxing, and security audits. Furthermore, it details a proposed "GenAI Security Firewall" architecture designed to provide comprehensive, adaptable, and efficient protection for these systems by integrating various security services and leveraging GenAI itself for enhanced defense. Addressing these security concerns is paramount for the responsible and safe deployment of this transformative technology.
1. Introduction
Agentic workflows enable autonomous AI tasks but create new security risks, amplified when multiple agents interact. The paper identifies these risks and proposes a comprehensive architecture to mitigate them.
1. Introduction
- Autonomous agentic workflows create a new attack surface, with unforeseen vulnerabilities and unintended-consequence risks amplified by multi-agent interaction.The paper calls for a multilayered approach spanning data security, model validation, ethical development, and continuous monitoring.Source: §1 · p. 1
- The paper identifies security risks in GenAI-based agentic workflows and proposes a comprehensive solution architecture to mitigate them.Source: §1 · p. 1
2. Problem Statement
The section groups agentic-workflow security concerns across data privacy, model vulnerabilities, agent autonomy, and network and system integration. These risks range from sensitive-data exposure to compromised models, unauthorized agent actions, and insecure communications.
2. Problem Statement
- Agentic workflows raise security concerns that developers and users must address across data, models, agent control, and system interactions.Source: §2 · p. 1
2.1. Data Privacy and Confidentiality
- Sensitive data processed throughout the workflow lifecycle faces leakage, unauthorized access or manipulation, and privacy-regulation compliance risks.Source: §2.1 · p. 1
2.2. Model Vulnerabilities
- Model vulnerabilities include prompt injection, evasion of workflow controls, poisoning of training data, and theft of models or training data.Source: §2.2 · pp. 1–2
2.3. Agent Autonomy and Control
- Autonomous agents may act harmfully or outside their programming, while their decisions can be difficult to trace and their access may escalate.Source: §2.3 · p. 2
2.4. Network and System Security
- Agent interactions with APIs, networks, and existing IT infrastructure introduce exploitable vulnerabilities, interception risks, and integration flaws.Source: §2.4 · p. 2
3. Problem Statement
The paper presents technical and procedural safeguards for agentic-workflow security, including data protection, prompt controls, monitoring, sandboxing, and audits. It also proposes using GenAI to assist these defenses and motivates a centralized firewall layer as an alternative to scanning every LLM interaction.
3. Problem Statement
- Safeguards include encryption, access control, prompt design and sanitization, model monitoring, agent sandboxing, and security audits.The listed measures address sensitive data, manipulation, anomalous behavior, compromised agents, and workflow or infrastructure vulnerabilities.Source: §3 · p. 2
- GenAI can assist with encryption identification, access control, prompt refinement, model monitoring, sandbox management, and parts of security audits.The paper states that incorporating GenAI into security practices can bolster workflow defenses.Source: §3 · p. 2
- Scanning every LLM interaction with an input/output security layer can increase overhead and latency and negatively affect performance, motivating an independent centralized firewall.Source: §3 · p. 2
4. Proposed Solution: GenAI Security Firewall
The proposed GenAI Security Firewall is a distinct central service layer intended to protect potentially multi-agent, self-optimizing workflows. Its architecture combines security services with workflow interaction, validation, and feedback, and the paper describes adaptability and potential operational benefits.
4. Proposed Solution: GenAI Security Firewall
- The proposed firewall provides a distinct central security layer for potentially multi-agent and self-optimizing GenAI workflows.Source: §4 · p. 2
4.1. Architecture Overview
- The architecture places firewall services alongside a multi-agent workflow, its memories and data stores, connectors, and a secret and access manager.Firewall components include input scanning, DDoS guarding, model monitoring, vulnerability knowledge, data auditing, output validation, and historical event memory.Source: §4.1 · pp. 2–4
4.2. Workflow
- The workflow scans inputs, analyzes them through security services, categorizes them, validates safe outputs, and releases them; threats trigger blocking and oversight alerts.A feedback mechanism tracks output relevance and reward to refine policies and detection accuracy.Source: §4.2 · p. 4
4.3. Key Features
- The firewall is described as providing end-to-end input, model, data, and output protection through a modular, adaptable, and potentially more efficient independent service layer.Its design uses LLMs/RAG and learning capabilities to adapt to threats and contexts.Source: §4.3 · p. 4
4.4. Benefits of a Centralized GenAI Firewall:
- 60-80% lower policy drift and prevention of 80-90% of prevalent attacks such as prompt injection are claimed for centralized enforcement.Source: §4.4 · p. 4
- 15-25% maintenance-cost savings and an average incident-response-time reduction of at least 30-40% are claimed for centralized security operations.Source: §4.4 · p. 4
5. Conclusion
GenAI agentic workflows face security risks across data privacy, model integrity, agent control, and system interactions. The proposed firewall integrates security services and GenAI-based defenses, while future work remains important for trustworthy deployment.
- Security risks span data privacy, model integrity, agent control, and system interactions, requiring specialized defenses alongside encryption, access control, and monitoring.Source: §5 · p. 5
- The GenAI Security Firewall combines input scanning, DDoS protection, model monitoring, data auditing, and output validation with GenAI-based anomaly detection and threat analysis.A vulnerability knowledge base and feedback mechanisms support adaptation to evolving threats.Source: §5 · p. 5
- Robust security measures are described as essential for trustworthy deployment, with continued work needed on model robustness, explainability for agent actions, and secure multi-agent coordination.Source: §5 · p. 5