PaperlayerView exact revision on arXiv

Source-linked AI summary

Architecting the Secure AI-SOC: A Neurosymbolic Framework for Pipeline Integrity and Threat Mitigation

Anna Gazani, Spyridon Kounoupidis, Panagiotis Katsaros, Nikolaos Kekatos, Grigorios Tsoumakas, Georgios Koutidis

arXiv:2609.10707v1cs.CRcs.AI

TL;DR

LLM adoption in SOCs creates an indirect-prompt-injection problem because malicious logs and external content can hijack model operations, while single-layer defenses trade semantic coverage against efficiency and reliability. The paper proposes a neurosymbolic architecture that combines deterministic SIEM sanitization, NeMo Guardrails, and HITL telemetry, and reports ATLAS-mapped mitigation of the promptware kill chain with verifiable constraints and operational visibility.

  • Problem

    LLM-enabled SOCs are vulnerable to indirect prompt injection through poisoned logs and external content, while deterministic and purely neural defenses each have important limitations.

  • Method

    The framework combines deterministic SIEM decoders and rules for structural pre-filtering with NeMo Guardrails for semantic self-checking and validated LLM actions.

  • Results

    The synergistic architecture dismantles the promptware kill chain and bounds LLM stochasticity with verifiable constraints across MITRE ATLAS techniques.

  • Takeaways & Limitations

    The architecture provides a resilient AI-SOC defense with closed-loop telemetry that maintains Human-in-the-Loop visibility into thwarted attacks.

  • Takeaways & Limitations

    The threat model assumes attackers can embed whatever content is needed for a successful prompt injection and can compromise external content processed by the SOC.

Abstract

from arXiv · show

The integration of Large Language Models (LLMs) into Security Operations Centers (SOCs) streamlines threat intelligence but introduces critical vulnerabilities, notably indirect prompt injection via log poisoning. Adversaries exploit this vector to execute multistep ``promptware'' kill chains by embedding malicious payloads within system logs to hijack the LLM's operational logic. Securing this pipeline presents a dichotomy: deterministic defenses are computationally efficient yet semantically blind, while purely neural evaluations introduce prohibitive latency and probabilistic flaws. To address this, we propose a novel neurosymbolic defense-in-depth architecture that ensures end-to-end pipeline integrity. The primary layer employs customized SIEM decoders as a deterministic pre-filter, performing immediate structural sanitization to neutralize volumetric padding and signature-based injections at the ingestion edge. The secondary layer leverages NeMo Guardrails to enforce strict semantic boundaries through self-checking validation on the structured SIEM alerts prior to LLM processing. Furthermore, the framework integrates a closed-loop telemetry system, providing critical Human-in-the-Loop (HITL) visibility into thwarted attacks directly within the SOC dashboard. We present a comprehensive experimental evaluation mapped to the MITRE ATLAS taxonomy, assessing the framework against diverse prompt injections. Our results demonstrate that this synergistic approach effectively dismantles the promptware kill chain - bounding LLM stochasticity with verifiable constraints, and delivering a resilient, highly observable defense mechanism for next-generation AI-SOCs.

1 Introduction

LLM-enabled SOCs improve threat-analysis workflows but create an indirect-prompt-injection attack surface, exposing a gap between efficient deterministic filters and semantically capable neural defenses. The paper proposes a neurosymbolic architecture combining SIEM pre-filtering with NeMo Guardrails, telemetry feedback, and MITRE ATLAS evaluation.

  • 1 Introduction: LLM integration streamlines SOC analysis but enables indirect prompt injection through malicious payloads embedded in logs and processed by the LLM.The payload can hijack the model’s operational logic after log correlation and ingestion.
  • 1 Introduction: Deterministic defenses efficiently block volumetric attacks but remain semantically blind, whereas purely neural defenses are probabilistic and costly.The paper frames these limitations as the central motivation for defense in depth.
  • 1 Introduction: The architecture combines deterministic SIEM pre-filtering with semantic NeMo Guardrails to protect the AI-SOC pipeline end to end.The design distributes defensive responsibilities across the ingestion and LLM-interface stages.
  • 1 Introduction: The SIEM layer sanitizes payloads at ingestion, while the neural layer validates structured alerts and autonomous tool use before and after LLM processing.NeMo Guardrails supplies input and output boundary checks around the cognitive engine.
  • 1 Introduction: The framework reports dismantling the promptware kill chain through ATLAS-mapped evaluation while adding closed-loop HITL telemetry for analyst visibility.Thwarted injection attempts are structured and fed back into the SIEM dashboard.

2 Background and Threat Model

The paper models the AI-SOC as trust-separated telemetry, SIEM, and cognitive domains, with the SIEM-to-LLM boundary most exposed to indirect prompt injection. Attackers can compromise external content or logs without direct access to the SIEM backend or LLM credentials.

  • 2 Background and Threat Model: The AI-SOC pipeline separates untrusted telemetry, semi-trusted SIEM correlation, and a trusted-but-vulnerable cognitive domain.The SIEM-to-LLM interface is identified as the most critical trust boundary.
  • 2 Background and Threat Model: LLMs struggle to distinguish trusted system instructions from untrusted logs or CTI indicators when both share one context window.This ambiguity creates the core condition for indirect prompt injection.
  • 2 Background and Threat Model: The threat model assumes attackers compromise external content processed by the SOC rather than directly accessing SIEM backends or LLM API keys.This reverses the traditional direct-LLM attack model.
  • 2 Background and Threat Model: Log poisoning crosses the first trust boundary and can carry malicious payloads into LLM alerts, establishing promptware Initial Access.The assumed attacker can embed the content needed for a successful injection.

B. Poisoned CTI Feeds and API Compromise (External Intelligence

External CTI integrations add supply-chain and API risks because attackers can poison public feeds or modify responses returned to the SIEM. Retrieved malicious instructions may remain dormant until a benign event triggers enrichment and LLM processing.

  • B. Poisoned CTI Feeds and API Compromise (External Intelligence: Compromised CTI providers or poorly secured APIs can alter JSON enrichment responses delivered to the SIEM.The paper identifies malicious indicator injection and API interception as external intelligence vectors.
  • B. Poisoned CTI Feeds and API Compromise (External Intelligence: Retrieval-dependent persistence lets malicious instructions remain dormant in external CTI databases until a benign event triggers their retrieval.The poisoned data is then incorporated into an alert and executed by the LLM.
  • B. Poisoned CTI Feeds and API Compromise (External Intelligence: Once poisoned data reaches the context window, promptware can attempt privilege escalation and actions on objective such as exfiltration or denial of service.The consequences depend partly on autonomous capabilities granted to the LLM.

3 Secure AI-SOC Methodology and Proposed Architecture

The proposed architecture assigns structural filtering to a deterministic SIEM layer and semantic validation to NeMo Guardrails, creating a closed-loop defense around the LLM. The design also preserves forensic information, bounds stochastic outputs, supports HITL visibility, and maps mitigations to ATLAS attack stages.

  • 3.1 The Neurosymbolic Defense-in-Depth Framework: The symbolic pre-filter normalizes and bounds incoming data before LLM access, while semantic guardrails catch manipulations that static rules miss.The two layers are designed to complement structural and semantic coverage.
  • 3.2 Layer 1: Deterministic Pre-filtering via SIEM: The SIEM pre-filter extracts payloads, applies atomic rules, strips actionable injections, and retains forensic metadata before event correlation.This early intervention produces sanitized JSON alerts with minimal computational overhead.
  • 3.3 Layer 2: Self Checking via Semantic Guardrails: NeMo Guardrails screens structured alerts through self-checking input and output rails that enforce policy, validate JSON actions, and block unauthorized commands.The output rail restricts actions to block_ip, delete_file, kill_process, or none.
  • 3.3 Layer 2: Self Checking via Semantic Guardrails: The dual-rail configuration bounds LLM stochasticity by making actions and outputs verifiable, predictable, and aligned with security policies.This constraint applies even when a zero-day injection crosses the SIEM trust boundary.
  • 3.4 Telemetry Feedback and Human-in-the-Loop (HITL) Integration: The telemetry feedback loop addresses the black-box problem by returning thwarted injection attempts to analysts through structured SOC visibility.The HITL design preserves situational awareness when inputs are dropped or sanitized.
  • 3.5 Mapping to MITRE ATLAS Taxonomy: ATLAS mapping shows why both layers are needed: symbolic filtering misses semantic jailbreaks, while isolated neural evaluation remains exposed to volumetric resource exhaustion.The framework maps these complementary protections across the promptware kill chain.

4 Prototype Implementation and AI-SOC Configuration

The prototype implements a two-layer AI-SOC defense in which Wazuh extracts and sanitizes suspicious payloads before NeMo Guardrails applies semantic validation. Structured telemetry preserves forensic context and exposes thwarted attacks through the SOC dashboard.

  • Layer 1: Deterministic Pre-filtering: Wazuh custom decoders establish the primary trust boundary by extracting suspected prompt-injection text into the llm_injection_payload field before correlation.The decoder uses PCRE2 patterns for explicit override phrases and preserves contextual metadata for traceability.
  • Layer 1: Deterministic Pre-filtering: Deterministic workflow rules inspect payload length, encoding, role impersonation, and known promptware patterns with predictable execution times.Atomic rules independently represent security conditions, including Level 10 Rule ID 100050 for extracted injection payloads.
  • Layer 1: Deterministic Pre-filtering: Active Response truncates oversized payloads and replaces executable linguistic semantics with immutable markers, mitigating volumetric and signature-based attacks before LLM processing.The procedure bounds computational complexity and transforms malicious instructions into inert forensic artifacts.
  • Pipeline Integration: Sanitized payloads and integrity metadata are reinserted into bounded JSON alerts, allowing NeMo Guardrails to evaluate trusted telemetry with reduced overhead.The resulting alert remains operationally visible to Human-in-the-Loop analysts.
  • Layer 2: Semantic Guardrails: NeMo Guardrails wraps the cognitive engine with self-checking input and output rails that validate prompts, responses, and tool use against operational policies.The semantic layer screens instruction overrides, role manipulation, prompt extraction, and unauthorized action requests.
  • Telemetry and HITL Visibility: Structured telemetry records mark detected attacks and populate the SOC dashboard with signatures and network indicators for direct analyst oversight.The dashboard can identify PROMPT_INJECTION_ATTEMPT events while preserving source parameters such as src_ip.

5 Experimental Evaluation

The evaluation processes poisoned Wazuh alerts through the complete orchestrated pipeline and tests semantic guardrails against diverse MITRE ATLAS-mapped promptware vectors. Layer 2 detects most attacks but retains residual failures on sophisticated semantic manipulations.

  • Evaluation Setup: The prototype orchestrator sequentially ingests Wazuh JSONL alerts and processes their structured fields through the full defense pipeline.Alerts include fields such as full_log, previous_output, source IP metadata, and decoder-extracted payloads.
  • Results: 18 of 20 adversarial payloads were identified and neutralized by Layer 2 semantic guardrails, while two sophisticated attacks evaded interception.The failures comprised one Unauthorized Tool Abuse/Lateral Movement vector and one complex Instruction Overriding jailbreak.
  • Results: 3 of 20 prompt-injection attempts were intercepted by Layer 1 deterministic pre-filtering, reflecting its focus on structural anomalies and explicit signatures.The experiment used a log-poisoning scenario containing an override directive and randomized padding intended to exhaust context.
  • Future Hardening: The authors propose enriching Colang intents, tightening similarity thresholds, adding few-shot adversarial examples, and enforcing Output Rails to address residual jailbreaks.These refinements target subtle context shifting and role-playing exploits before unauthorized SOAR execution or data exfiltration.

6 Related Work

Related work spans LLM-enabled SOC operations, agent-level isolation, structural defenses, and programmable guardrails. The paper positions its contribution as moving protection toward ingestion by combining deterministic SIEM filtering with semantic validation.

  • AI-SOC and Agent Defenses: Prior work uses execution isolation and real-time agent-intent monitoring, whereas this framework shifts defensive focus leftward toward the telemetry ingestion pipeline.The comparison distinguishes IsolateGPT and AgentSentinel from the proposed ingestion-centered architecture.
  • Prompt-Injection Defenses: Prompt engineering and isolation techniques remain susceptible to context exhaustion and semantic obfuscation, motivating heuristic, classifier, structural, and programmatic defenses.The related approaches include Rebuff, StruQ, ACE, and instruction-data separation by design.
  • Architectural Defenses and Semantic Guardrails: NeMo Guardrails advances policy-based intent classification over structured inputs, but standalone programmatic rails or plan-then-execute systems can impose overhead on high-volume raw SIEM data.The paper addresses this operational concern by placing deterministic SIEM decoders before semantic evaluation.
  • Architectural Defenses and Semantic Guardrails: The proposed neurosymbolic framework uses customized deterministic decoders as a prerequisite filter so downstream guardrails process clean, structured alerts and resist volumetric attacks.This arrangement is presented as a bridge between efficient structural filtering and semantic protection.

7 Conclusions and Further Work

The paper concludes that combining SIEM pre-filtering with neural guardrails provides verifiable pipeline integrity, operational efficiency, and HITL telemetry visibility. It also identifies stateful semantic attacks and multimodal promptware as boundaries for future work.

  • Conclusions: The defense-in-depth architecture combines deterministic SIEM pre-filtering with semantic neural guardrails to address volumetric padding and complex jailbreaks before LLM evaluation.The conclusion frames the combination as a response to the limitations of monolithic defenses.
  • Conclusions: MITRE ATLAS evaluation supports verifiable pipeline integrity and operational efficiency while structured telemetry preserves Human-in-the-Loop visibility.The conclusion links technical protection with analyst observability rather than silent event suppression.
  • Limitations and Further Work: The current implementation evaluates alerts independently, leaving potential exposure to multi-turn context shifting and memory poisoning when cross-session state is retained.This is an explicitly stated scope boundary of the isolated-alert design.
  • Limitations and Further Work: Future work proposes stateful session-aware intent classification and lightweight computer vision to address cross-session persistence and multimodal promptware.The proposed extensions target attacks encoded in image or audio CTI artifacts.

A Appendix

The appendix contains the complete prompts used by NeMo Guardrails to self-check input and output data.

  • The appendix provides the full NeMo Guardrails prompts used for self-checking input and output data.

A.1 Input Rail Prompt

The input rail uses a security-classification prompt to identify prompt-injection attempts against an LLM-based SOC assistant. It blocks manipulation attempts while preserving normal security discussions.

  • The input rail classifies whether a user message manipulates, overrides, or bypasses the SOC assistant’s intended behavior.
  • It blocks malicious instructions embedded in retrieved logs, documents, or external data, including fake system messages and hidden commands.
  • Normal security queries, examples, explanations, and prompt-injection discussions remain unblocked unless they attempt to execute an attack.

A.2 Output Rail Prompt

The output rail validates whether the SOC assistant’s response is safe, compliant, and formatted as expected. It constrains supported response actions to an explicit set.

  • The output rail evaluates whether an assistant response is safe and compliant with the expected SOC response format.
  • The validation rules restrict the action field to block_ip, delete_file, kill_process, or none.
  • The appendix labels this specification as the NeMo Guardrails self-check output prompt.

A.3 Active Response Prompt

The active response prompt directs the LLM to analyze structured security alerts and recommend an action. It treats alert data as untrusted and requires explicit handling of injection attempts.

  • The active response prompt asks the SOC analyst model to analyze an alert and recommend an action using alert context and available actions.
  • The prompt supplies structured alert metadata including severity, rule, groups, system, manager, source, and MITRE information.
  • All alert data is treated as untrusted, and the model must not follow instructions contained inside the alert.
  • Detected instruction manipulation or a DECODER_INJECTION_PAYLOAD sets injection_attempt to true.
  • The appendix identifies this specification as the active response prompt used to generate an action from the LLM.
Loading 2609.10707v1…