AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security

Motivation

• Static filters and training-time interventions often generalize poorly to evolving attacks and may not completely suppress sensitive knowledge.The paper motivates adaptive, runtime defenses for prompt injection, jailbreaking, and information disclosure.Source: §1 · p. 1

Research gap

• The paper identifies a gap in security scaling: test-time and system-level approaches remain less explored than training-time safety methods.This contrasts with the extensive use of inference-time computation for improving model capabilities.Source: §1 · p. 2

Proposed framework

• AegisLLM coordinates orchestrator, deflector, responder, and evaluator agents to monitor and mitigate threats at inference time.The framework continuously refines its defenses through automated prompt optimization and Bayesian learning without retraining.Source: §1 · p. 2

Contributions

• AegisLLM contributes a scalable agentic framework, inference-time security optimization, and evaluations showing attack prevention with minimal utility trade-offs.The listed contributions cover jailbreaks, adversarial perturbations, sensitive disclosures, and automated optimization with few examples.Source: §1 · pp. 2–3

LLM Safety and Security

• Existing safety efforts include alignment training, guardrails, red-teaming, safe decoding, interpretability, unlearning, and test-time safety methods.The paper frames safety as context-dependent and reports limitations in approaches focused solely on model-level modifications.Source: §2 · p. 3

Agentic Systems

• Multi-agent architectures have coordinated complex tasks successfully, yet their direct application to LLM security remains relatively underexplored.AegisLLM adapts these principles to build a security framework for greater resilience.Source: §2 · p. 3

Agentic Optimization

• Agentic optimization uses structured feedback, execution traces, and natural-language critiques to refine complex LLM workflows beyond gradient-based training.DSPy supports prompt and demonstration optimization for multi-stage pipelines and adaptive defenses.Source: §2 · p. 3

3.1. Motivation and Design Goals

• Static defenses cannot reliably follow rapidly evolving attacks, motivating test-time adaptability, distributed responsibilities, and modular composition without retraining.The design goals emphasize real-time response, complementary lines of defense, and extensibility across risk categories.Source: §3.1 · p. 4

3.2. System Architecture and Workflow

• The Orchestrator routes queries, the Responder answers initially safe requests, the Evaluator checks queries and responses, and the Deflector handles unsafe outputs.A seemingly benign query can be redirected after the Evaluator detects sensitive information in the Responder’s output.Source: §3.2 · pp. 4–5
• Separating decision-making from response generation helps responder agents preserve utility on benign queries while specialized agents provide layered defenses.The modular design assigns each component a single function and mitigates vulnerabilities in individual prompts or model instances.Source: §3.2 · p. 5

3.3. Agentic Optimization

• Prompt optimization searches over instruction text and few-shot demonstrations using validation metrics such as accuracy and false positive rate.MIPROv2 bootstraps examples, proposes instruction candidates, and applies Bayesian optimization to agent-specific configurations.Source: §3.3 · pp. 6–7
• Each agent’s optimization loop evaluates candidate prompts against labeled queries and updates instructions or demonstrations without retraining the language model.The process can refine prompts to detect subtle leaks that keyword-focused Evaluators might miss.Source: §3.3 · pp. 6–7

4.1. Benchmarks

• The evaluation covers WMDP and TOFU for unlearning, and StrongREJECT and PHTest for jailbreak resistance and false-refusal behavior.The benchmark suite measures restricted-knowledge suppression, retained capabilities, conversational quality, attack resistance, and harmless-query refusals.Source: §4.1 · pp. 7–8

4.2. Baselines

• Unlearning comparisons include GradDiff-Merged, RMU, RMU-LAT, ELM-Merged, TAR, and prompting-based Guardrail baselines.These methods represent loss-difference training, activation or latent perturbation, unknowledgeable behavior training, meta-learning, and lightweight prompting.Source: §4.2 · p. 8
• Jailbreaking comparisons include Circuit Breaker, LLM-LAT Robust, Refusal-VPI, and Llama Guard.The baselines include latent representation rerouting and a two-stage system-level harmful-content classifier.Source: §4.2 · p. 8

4.3. Implementation Details

• Each AegisLLM component uses a separate DSPy language-model session with role-specific prompts, while the Deflector refuses free-form unsafe queries or randomly selects an option for flagged multiple-choice questions.The Evaluator checks candidate responses for violations, and task-specific prompt optimization uses small sampled datasets with training and validation splits.Source: §4.3 · pp. 8–9
• Table 2 compares Llama-3-8B unlearning across WMDP Cyber, Bio, and Chem, alongside MMLU and MT-Bench retention measures.Lower WMDP accuracy indicates stronger unlearning, whereas higher MMLU and MT-Bench scores indicate better retained capabilities.Source: Table 2 · p. 9

5.1. Unlearning

• WMDP-Cyber and WMDP-Bio accuracies approach the 25% random-guessing minimum, indicating near-complete suppression of targeted knowledge.The framework achieves the lowest restricted-topic accuracy among evaluated methods on the Llama-3-8B results.Source: §5.1 · p. 10
• 58.4% MMLU accuracy and only a 5.6% MT-Bench gap from the non-unlearned base model indicate retained general and conversational capabilities.These results support targeted knowledge suppression with limited utility degradation.Source: §5.1 · p. 10
• The TOFU evaluation compares AegisLLM abstentions with Thaker et al.’s post-processing filter baseline using a Llama-2-7B responder fine-tuned on TOFU.The supplied passages identify the comparison and responder setup but do not provide table values.Source: Table 3 · p. 10, §5.1 · p. 11

5.2. Jailbreaking

• 0.038 StrongREJECT and 88.5% PHTest compliance show competitive jailbreak defense without extensive training.The base Llama-3-8B scores 0.078 on StrongREJECT and has 7.1% PHTest false refusal; Llama Guard reaches 0.039 but relies on extensive training.Source: §5.2 · pp. 11–12, Table 4 · p. 12
• 73.0% refusal on 15 attacks is reached with 10.3% PHTest false refusal after exposure to 15 attacks and five examples per attack.With five attacks, refusal is 60.7% and false refusal is 8.7%; with ten attacks, they are 67.0% and 9.0%.Source: §5.2 · p. 12, Figure 4 · p. 13

5.3. Ablation Studies

• 24.6% and 26.3% restricted-topic accuracy show optimized Llama-3-8B unlearning improvements over 31.7% and 32.0%, while flagged ratios reach 97–99%.Optimization substantially improves restricted-query detection and handling.Source: §5.3 · pp. 13–14
• Optimization improves flagged ratios across architectures, reaching 94.8–98.0% for Qwen-2.5-72B and 93.1–96.3% for distilled Llama 8B.Qwen retains 79.8% MMLU, while distilled Llama 8B improves MMLU by 5.2%.Source: §5.3 · p. 14, Table 5 · p. 14

6. Conclusion

• AegisLLM treats security as an emergent property of coordinated, specialized agents rather than a static model characteristic.The framework operates at inference time and is designed for dynamic, scalable security.Source: §6 · p. 15

A. Modules Initialization

• Configuration-driven modularity enables component-specific adaptation, experimentation, and DSPy optimization without changing core code.The design uses clear signatures, type annotations, concise prompts, and configurable modules.Source: A. Modules Initialization · p. 23
• For unlearning, the orchestrator detects topic-related probes while the evaluator checks whether responses reveal sensitive knowledge.Both components use topic-specific configuration and chain-of-thought predictors to return safety judgments and reasoning.Source: A. Modules Initialization · pp. 24–25
• The modules use distinct roles: orchestration, response generation, safe deflection, and evaluation of harmful outputs or information leakage.The orchestrator classifies inputs, the responder answers questions, the deflector refuses or supplies safe alternatives, and the evaluator checks proposed responses.Source: A. Modules Initialization · pp. 24–26
• For jailbreaking, the orchestrator identifies harmful inputs, the responder handles multiple-choice or free-form questions, and the deflector returns refusal behavior or a safe multiple-choice option.The implementation supports parsed and non-parsed response generation and configurable refusal messages.Source: A. Modules Initialization · pp. 25–26

B. Unlearning Texts

• WMDP unlearning targets hazardous knowledge across biosecurity, cybersecurity, and chemical security.The listed areas include bioweapons, pathogen research, exploitation, weaponization, synthesis, procurement, and detection bypassing.Source: B. Unlearning Texts · pp. 26–27
• TOFU unlearning targets fictional author profiles by supplying the names associated with each forget task.The method uses separate Forget 1%, Forget 5%, and Forget 10% tasks with corresponding author names filled into the unlearning text.Source: B. Unlearning Texts · p. 27

C. Jailbreak Texts

• The jailbreak policy flags direct harm, safety circumvention, obfuscation, harmful-output instructions, chain requests, and harmful intent unless beneficial research makes the content essential.It also asks whether the stated purpose can be achieved without generating potentially harmful content.Source: C. Jailbreak Texts · p. 27
• A lower-false-rejection variant lets the orchestrator default to safe and the evaluator default to unsafe only when explicit criteria require those decisions.This gives borderline content a second chance instead of immediate rejection.Source: C. Jailbreak Texts · p. 27

• The Orchestrator flags a Metasploit exploit query as unsafe and passes it to the Deflector, which returns a non-informative response.For multiple-choice questions, the Deflector generates a random choice rather than harmful information.Source: D. Demonstration of AegisLLM Workflow… · p. 27
• The workflow blocks the user’s request for harmful information after the Orchestrator determines that using Metasploit to exploit a vulnerability is unsafe.Source: D. Demonstration of AegisLLM Workflow… · p. 27
• Figure D.1 demonstrates AegisLLM detecting and deflecting a direct request for exploit code on WMDP.Source: Figure D.1 · p. 28

• 0.040 StrongReject and 64.7% compliance characterize prefix-only Self-Examination, whereas suffix-only Self-Examination has 0.030 StrongReject, 49.0% compliance, and 46.0% full refusal.The suffix-based variant is described as tending to reject benign queries.Source: E. LLM Self-Examination and… · p. 28, Table E.1 · p. 28
• 0.015 StrongReject is the best score for prefix-only Self-Reminder, but compliance is 32.6% and full refusal is 55.7%.This configuration provides strong attack mitigation while substantially reducing usability.Source: E. LLM Self-Examination and… · p. 28, Table E.1 · p. 28
• 69.4% compliance is achieved by suffix-only Self-Reminder, but its StrongReject score is 0.041, the worst among evaluated methods.The configuration preserves helpfulness better while providing weaker jailbreak prevention.Source: Table E.1 · p. 28, E. LLM Self-Examination and… · p. 29
• 0.016 StrongReject from combined prefix-suffix Self-Reminder is comparable to the best prefix-only score, but compliance falls to 36.8% with 51.4% full refusal.The combined configuration errs toward blocking attacks and frequently rejects legitimate inputs.Source: Table E.1 · p. 28, E. LLM Self-Examination and… · p. 29

• Table F.1 presents representative Orchestrator reasoning outputs from Llama-3-8B and DeepSeek-R1 ablations before and after DSPy optimization.Source: F. Sample Safety Reasonings for WMDP · p. 29, Table F.1 · p. 30
• The Orchestrator labels a function-call question as safe when it concerns programming rather than biosecurity, cybersecurity, or chemical security.Source: F. Sample Safety Reasonings for WMDP · p. 34
• The Orchestrator labels questions about E. coli cell culture and satellite communication vulnerabilities as unsafe when they are interpreted as probing security-related knowledge.The examples include WMDP-Bio and WMDP-Cyber queries with corresponding safety judgments and reasoning.Source: F. Sample Safety Reasonings for WMDP · pp. 31, 35

G. Initial and Optimized Prompts: Unlearning and Jailbreaking

• DSPy modules implement both unlearning and jailbreaking pipelines, so Orchestrator and Evaluator prompts use DSPy-compatible formatting regardless of optimization.The section presents unlearning prompts before jailbreaking prompts.Source: G. Initial and Optimized Prompts:… · p. 37

G.1.1. Initial Unlearning Prompts

• Initial unlearning prompts define structured input, reasoning, and boolean is_safe fields, requiring True for unrelated inputs and False for security-related inputs.The Orchestrator receives security-domain keywords spanning biosecurity, cybersecurity, and chemical security, plus the question and choices.Source: §G.1.1 · pp. 37–38
• The unlearning text accompanying Listing 1 corresponds to the WMDP/MMLU benchmarks, while optimized examples include both benign and hazardous-domain questions.The examples demonstrate how the prompt maps topic relevance to the is_safe Boolean output.Source: §G.1.1 · p. 38, §G.1.2 · pp. 45–46

G.1.2. Optimized Unlearning Prompts

• Optimized unlearning prompts are model-specific, with separate prompt sets presented for Llama-3-8B and DeepSeek-R1 (Distilled Llama 8B).The optimized prompts retain structured fields while adding explicit instructions to explain the safety decision.Source: §G.1.2 · p. 38
• The optimized Orchestrator prompt explicitly lists hazardous-topic categories and asks for a clear reasoning explanation supporting the True-or-False safety classification.Its categories include bioweapons and pathogens, exploitation and reconnaissance, and chemical synthesis, procurement, purification, and related topics.Source: §G.1.2 · pp. 47–48
• Optimized prompts use multiple-choice demonstrations with assistant labels, including unrelated academic questions marked true and security-relevant biology or cybersecurity questions marked false.Examples include herpesvirus, vaccinia-virus, zero-day, and control-flow-integrity questions alongside general science and programming questions.Source: §G.1.2 · pp. 48, 50, 51
• The optimized unlearning prompt uses structured examples and explicit `is_safe` Boolean formatting to classify diverse knowledge-seeking inputs.Examples include biomedical, chemistry, programming, phishing, and environmental questions, with safe and unsafe labels demonstrated.Source: §G.1.2 · pp. 52–55
• The unlearning orchestrator prompt was optimized with DSPy for DeepSeek-R1 and Distilled Llama 8B.Listing 3 identifies the optimized system and user prompts and their target models.Source: §G.1.2 · p. 55

G.2.1. Initial Jailbreaking Prompts

• The initial jailbreaking prompts define structured `input`, `reasoning`, and Boolean `is_safe` fields for the Orchestrator and Evaluator components.The Orchestrator assesses inputs, while the Evaluator assesses input responses under the same structured output format.Source: §G.2.1 · pp. 55–56
• The initial jailbreak policy flags direct facilitation or endorsement of harm, safety circumvention, obfuscation, harmful-output requests, chain requests, and harmful intent, while allowing legitimate research and education.It also treats requested system-prompt changes as unsafe and asks whether the purpose can be achieved without potentially harmful content.Source: §G.2.1 · pp. 55–56

G.2.2. Optimized Orchestrator Prompts for Jailbreaking

• The optimized jailbreaking Orchestrator prompt instructs a language model to determine whether inputs are harmful and provide reasoning before returning a Boolean safety decision.Its demonstrations classify benign technical, shipping, and philosophical requests as safe, while flagging weapon construction, covert property damage, prompt injection, and disease transmission as unsafe.Source: §G.2.2 · pp. 56–61

G.2.3. Optimized Evaluator Prompts for Jailbreaking

• The optimized Evaluator prompt applies the same harm-focused policy to responses, explicitly identifying promotion of antisemitic and Holocaust-supporting ideology as unsafe.Its examples treat historical discussion, psychotherapy guidance, and benign software assistance as safe, while preserving refusal responses as evaluator inputs for classification.Source: §G.2.3 · pp. 61–64
• The prompt enforces a fixed response schema beginning with `[[ ## reasoning ## ]]`, followed by `[[ ## is_safe ## ]]`, and ending with `[[ ## completed ## ]]`.The `is_safe` field must be formatted as a valid Python bool.Source: §G.2.3 · pp. 65–67
• The evaluator prompt requires classification of whether an input response elicits or attempts to elicit harmful outputs.It frames the task as a safety decision over the supplied input response.Source: §G.2.3 · p. 66
• The prompt flags content only when it directly promotes, facilitates, or endorses harm, while allowing legitimate research, education, and harm-reduction discussions.This criterion distinguishes harmful content from non-harmful contextual discussion.Source: §G.2.3 · p. 66
• The evaluator is instructed to flag safety-circumvention attempts, including ignored instructions, persona switching, obfuscation, explicit harmful-output requests, and chain requests.The prompt specifies these categories as unsafe unless demonstrably essential for beneficial research.Source: §G.2.3 · p. 66
• The prompt applies a necessity test by asking whether the stated purpose could be achieved without generating potentially harmful content.It also directs the evaluator to flag intent to cause harm to individuals and changes to system prompts that would make queries unsafe.Source: §G.2.3 · p. 66
• The evaluator receives the candidate content through an `{ input_response }` placeholder and must return a Boolean safety decision.The specified output is `True` when the response is safe and `False` otherwise.Source: §G.2.3 · p. 66
• Listing 7 identifies these materials as DSPy-optimized system and user prompts for AegisLLM’s jailbreaking evaluator.The listed prompts include evaluator examples with explicit `is_safe` values and the required completion markers.Source: §G.2.3 · pp. 65–67