SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety

Zhe Liu; Zonghao Ying; Wenxin Zhang; Quanchen Zou; Deyue Zhang; Dongdong Yang; Xiangzheng Zhang; Hao Peng

Source-linked AI summary

SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety

Zhe Liu, Zonghao Ying, Wenxin Zhang, Quanchen Zou, Deyue Zhang, Dongdong Yang, Xiangzheng Zhang, Hao Peng

arXiv:2605.05704v3May 7, 2026cs.CRcs.AI

TL;DR

LLM agents’ tool use and reasoning create security risks, while existing defenses can over-refuse benign requests. SAFEHARBOR combines adversarial rule generation, hierarchical memory, and context-aware projection to define more precise boundaries, reporting 63.6% peak benign utility on GPT-4o and harmful refusal above 93%.

Problem
Existing defenses struggle to distinguish benign from malicious agent behavior in context, creating a safety–utility trade-off and over-refusal of benign requests.
Method
SAFEHARBOR generates adversarially enhanced safety rules, stores them in hierarchical memory, and uses contrastive projection with gated retrieval for context-aware decisions.
Results
SAFEHARBOR reports state-of-the-art performance, including 63.6% peak benign utility on GPT-4o and harmful refusal exceeding 93%.
Takeaways & Limitations
The framework supports precise boundary enforcement while mitigating false refusals and maintaining strict safety standards.
Takeaways & Limitations
The paper limits its impact analysis to defensive evaluation and avoids deployment-oriented attack guidance.

Abstract

from arXiv · show

Recent advances in foundation models have transformed LLMs from passive conversational systems into autonomous agents capable of reasoning and tool execution. While these capabilities unlock substantial practical value, they also introduce new security risks, as adversaries can manipulate agents into performing harmful actions in real-world environments. Existing defense strategies mitigate such threats but frequently struggle to balance safety and utility, resulting in over-refusal of benign user requests. To mitigate this trade-off, we propose SafeHarbor, a novel framework designed to establish precise decision boundaries for LLM agents. Unlike static guidelines, SafeHarbor extracts context-aware defense rules through enhanced adversarial generation. We design a local hierarchical memory system for dynamic rule injection, offering a training-free, efficient, and plug-and-play solution. Furthermore, we introduce an information entropy-based self-evolution mechanism that continuously optimizes the memory structure through dynamic node splitting and merging. Extensive experiments demonstrate that SafeHarbor achieves state-of-the-art performance on both ambiguous benign tasks and explicit malicious attacks, notably attaining a peak benign utility of 63.6\% on GPT-4o while maintaining a robust refusal rate exceeding 93\% against harmful requests. The source code is publicly available at https://github.com/ljj-cyber/SafeHarbor.

1. Introduction

SAFEHARBOR targets the boundary ambiguity that makes existing defenses over-refuse benign requests. It combines adversarial rule generation, hierarchical memory, and contrastive safety projection to improve safety–utility balance.

Motivation: Existing defenses often require retraining or resource-intensive monitoring proxies, while static safety margins struggle with contextual distinctions.These limitations introduce latency and contribute to coarse-grained decisions between benign and malicious intents.
Framework: SAFEHARBOR generates diverse safety policies through adversarial enhancement and organizes them in a dynamic hierarchical memory.The memory is designed to capture latent vulnerabilities while keeping retrieval scalable as the knowledge base grows.
Framework: Contrastive projection jointly assesses semantic and contextual risks to mitigate over-refusal during tool invocation.The framework uses a fast path for clearly benign queries and reserves dual-score analysis for ambiguous contexts.
Framework: SAFEHARBOR synthesizes robust safety rules by applying adversarial enhancement to harmful trajectories within adaptive clustering.This produces dynamic clusters of rules rather than relying solely on fixed safety guidance.
Results: 63.6% peak benign utility on GPT-4o accompanies a harmful refusal rate exceeding 93%.The reported result supports the framework’s intended balance between preserving benign-task utility and refusing harmful requests.

2. Related Work

Prior work spans intrinsic alignment, external guardrails, dynamic validation, and evolving memory architectures. SAFEHARBOR instead constrains memory self-evolution around safety-rule refinement.

Safety defenses: Intrinsic alignment can require costly retraining, whereas external guardrails monitor interactions without changing the base model.The related-work discussion contrasts supervised safety fine-tuning with external content-safety classification.
Safety defenses: External guardrails such as Llama-Guard-3 classify content safety but lack agency in tool execution.This distinction motivates defenses designed for agents that can perform actions through tools.
Memory mechanisms: Recent memory research emphasizes capacity, structural organization, temporal tracking, and evolving knowledge networks.These approaches primarily extend context capabilities and refine understanding over time.
SAFEHARBOR: Figure 2 summarizes a three-stage workflow spanning adversarial rule generation, dual knowledge storage, and scoring and retrieval.The stages connect dynamic safety-rule construction with memory organization and gated inference routing.
Memory mechanisms: SAFEHARBOR uses constrained, time-independent memory self-evolution to consolidate safety rules through evolutionary refinement.This differs from unconstrained memory accumulation, which can introduce attack surfaces and misaligned information.

3. Methodology

SafeHarbor defines context-aware safety boundaries through trajectory modeling, hierarchical memory, adversarial rule generation, and geometry-aware risk projection. Its memory stores paired prohibitions and benign exemptions while entropy-based evolution and contrastive learning refine retrieval and boundary precision.

Problem Formulation: SafeHarbor models each agent interaction as a trajectory and distinguishes refusal from execution outcomes for harmful and benign queries.The evaluation score compares generated trajectories with optimal references, where correct refusal and perfect benign execution represent successful outcomes.
Hierarchical Memory: Queries are mapped into a latent space, then organized with harmful and benign data in a hierarchical memory tree for similarity-based retrieval.Internal nodes route searches across broad risk categories, while leaf nodes store fine-grained attack patterns and safety knowledge.
Dual Knowledge Storage: Each leaf stores a prohibition and a benign exemption, protecting valid instructions near harmful centroids from misclassification.The dual-policy unit couples Rharm from harmful clusters with Ebenign synthesized from benign trajectories.
Adversarial Rule Generation: Adversarial generation mutates harmful trajectories and cycles through goal decomposition and contextual reframing to cover structural and semantic attack vectors.The pipeline uses generated variants to challenge safety boundaries without overfitting to a single attack pattern.
Memory Evolution: Information Gain measures the entropy shift from adding an embedding to its nearest cluster and governs dynamic memory-topology evolution.A significant gain indicates that the incoming instance introduces variance the current cluster cannot adequately resolve.
Safety Projection: A lightweight two-layer projector uses benign and harmful prototypes, while margin-based contrastive learning structures latent geometry beyond binary classification.The contrastive objective pulls samples toward their class centers and pushes them from opposing centers by margin ∆.

4. Experimental Setup

The experiments evaluate data-independent defense memory and generalization across benchmarks covering harmful-task execution, safety risks, and common agent failure modes. Comparisons include prompting, memory-augmented, and vector-retrieval baselines, using official benchmark grading frameworks.

Datasets: The dynamic defense memory is built exclusively from AgentAlign, which contains 18,749 instances spanning 4,956 harmful and 13,793 benign samples.The benign portion includes neutral cases, supporting evaluation on unseen queries.
Datasets: AgentAlign evaluates whether agents retain the capability to execute complex harmful tasks after bypassing safety filters.This benchmark is paired with another benchmark for assessing generalization on unseen queries.
Datasets: AgentSafetyBench contains 2,000 test cases across 349 interaction environments, covering 8 safety risk categories and 10 common failure modes.Its broader coverage evaluates robustness across varied agent interactions.
Baselines: Baselines span explicit prompting, dynamic memory augmentation, and standard vector retrieval, including Rule Traverse, A-Mem, and standard RAG.Rule Traverse embeds 14 Llama Guard safety categories in the system prompt, while A-Mem uses a local LLM to manage memory.
Metrics: AgentHarm is reported with Score, Full Score, Refusal, and Non-Ref Score, while AgentSafetyBench uses refusal rates for defense success.The AgentHarm metrics respectively capture harm severity, maximally successful attacks, explicit rejections, and non-refused execution performance.

5. Experimental Results

SAFEHARBOR delivers a strong safety–utility balance across backbones and evaluation settings, while its hierarchical memory and inference components support both precise defense and efficient retrieval.

Overall performance: 93.2% harmful refusal on GPT-4o closely trails specialized guardrails, while Qwen2.5-7B benign refusal falls to 9.1% versus LlamaGuard’s 22.7%.The evaluation excludes over-defensive and under-defensive methods using predefined utility and safety thresholds.
Environment-based safety: 62.05% Refusal-Env is SAFEHARBOR’s highest absolute score on environment-based interaction scenarios, achieved with GPT-4o.SAFEHARBOR surpasses A-Mem by 14.7% on GPT-4o and RAG by 16.0% on Qwen2.5-7B with a Qwen2.5-72B verifier.
Ablation study: 48.9% harmful refusal after flattening the hierarchy shows that hierarchical clustering is essential for precise retrieval.Using raw trajectories raises benign refusal to 25.0%, while removing benign exemptions triples benign refusal to 25.0%.
Efficiency analysis: 306.67 ms average latency gives SAFEHARBOR an approximately 20-fold speed advantage over GuardAgent.The fast path offloads most safety checks from full-chain reasoning; LlamaGuard records 379.30 ms and AgentAlign 1728.20 ms.
Retrieval effectiveness: 25.8% Top-3 noise ratio is lower than standard RAG’s noise ratio exceeding 78%, while retrieval latency remains within the millisecond range.SAFEHARBOR also surpasses other retrieval-based methods on Intent Match.
Attack enhancement: Enhanced adversarial trajectories reduce safety-filter detection performance, including LlamaGuard’s drop from 90.36% to 29.84%.The enhancement injects stealthy permutations while retaining features that trigger model execution.

6. Conclusion

SAFEHARBOR reconciles robust safety with high utility in LLM agents by combining adversarial rule evolution with hierarchical knowledge retrieval. Experiments indicate reduced false refusals while maintaining strict safety standards.

SAFEHARBOR integrates adversarial rule evolution with hierarchical knowledge retrieval to mitigate over-defense without sacrificing inference efficiency.

Impact Statement

The work advances LLM safety and alignment by developing SAFEHARBOR as a defensive mechanism for harmful-query risks while preserving utility on benign tasks. The authors describe the research as focused on defensive evaluation and avoiding deployment-oriented attack guidance.

SAFEHARBOR is designed to mitigate malicious exploitation risks during generative-AI execution while preserving utility on benign tasks.
The authors frame their harmful-prompt and attack-pattern analysis as defensive evaluation rather than deployment-oriented attack guidance.

C. Hyperparameter Sensitivity Analysis

The sensitivity analyses identify operating points that balance memory-tree quality, safety-projector performance, and noise. The selected configurations emphasize high intent matching and minimal noise.

Safety Projector: The safety projector reaches peak performance at λ = 0.3, reflecting a trade-off between insufficient regularization and an over dominant auxiliary objective.
Dynamic Memory Evolution: τsim = 0.5 provides the best memory-tree balance, with peak Intent Match and minimal noise between aggressive merging and excessive fragmentation.
Analysis Setup: Figure 3 evaluates contrastive loss weight λ and safety margin ∆, while Figure 4 evaluates similarity and gain thresholds using Intent Match, Noise Ratio, and system overhead.
Dynamic Memory Evolution: τgain = 0.7 achieves the highest Intent Match with minimal noise and is selected as the optimal operating point.

D. Safety Projector Bypass Analysis

The Safety Projector exposes a trade-off between safety assurance and efficiency: lenient benign thresholds increase harmful leakage, whereas conservative thresholds suppress it.

6.25% harmful leakage occurs at a benign threshold of 0.3, showing that a lenient decision boundary compromises defense.
The deployed configuration sets the benign threshold to at least 0.6 and the harmful threshold to at most 0.3, keeping leakage below 0.5% while limiting fast-path acceleration to approximately 23% to 25%.

E. Analysis of Online Adaptation

SAFEHARBOR’s online adaptation exhibits a memory-size trade-off: performance peaks at 1,000 injected samples, while threshold settings balance harmful leakage against benign fast-path efficiency.

Memory Scaling: At 1,000 injected samples, SAFEHARBOR reaches its optimal performance peak on AgentHarm with Qwen2.5-7B.The experiment progressively injects raw AgentAlign attacks while ablating the Safety Projector and Attack Enhancement modules to isolate memory scaling.
Threshold Sensitivity: The Safety Projector exposes a trade-off between safety assurance and system efficiency through harmful and benign threshold selection.Harmful Leak Rate measures malicious queries bypassing the filter, while Benign Fast Path Rate measures safe queries processed without heavy model invocation.
Memory Scaling: 3.0% harmful score and 48.5% benign score define the favorable balance at the 1,000-sample threshold.Beyond this point, excessive raw attacks increase benign refusals and degrade benign task execution despite marginally increasing harmful refusals.

F. Judge Model Sensitivity

SAFEHARBOR’s retrieval-based rules reduce over-defensiveness across judge backbones, with stronger reasoning models generally achieving better safety-utility balance than smaller baselines.

Backbone Sensitivity: GPT-4o and Qwen2.5-72B consistently outperform smaller baselines in balancing low refusal rates with high overall accuracy.Their stronger reasoning capabilities improve comprehension and application of complex safety rules.
Backbone Sensitivity: Llama-Guard shows only a marginal performance shift after retrieval-based rules are added.The passage attributes this to its existing fine-tuning for safety classification.

G. Implementation of Adversarial Generation

SAFEHARBOR evaluates adversarial generation through three social-engineering paradigms and uses generated harmful-benign contrasts, hierarchical rules, and semantic refinement to support context-sensitive judgments.

Attack Generation: The adversarial generator implements Goal Decomposition, Privilege Escalation, and Contextual Reframing as distinct social-engineering paradigms.The templates are provided in raw form, with {{TOPIC}} populated from the seed prompt category.
Goal Decomposition: Goal Decomposition rewrites a restricted objective into seemingly benign sub-steps embedded within a natural narrative.The stated purpose is to bypass intent detection.
Privilege Escalation: Privilege Escalation reframes a query as a mandatory directive from an authoritative stakeholder to test access-control boundaries.The template names roles such as Chief Security Officer, Lead Developer, and External Auditor.
Adversarial Case Study: SAFEHARBOR distinguishes legitimate tool use from malicious intent by retrieving context-aware boundary rules and applying deep semantic reasoning.In the social-engineering case, it blocks an attempt to report a target to her workplace after Instagram and Google lookups.
Contextual Reframing: Contextual Reframing places a sensitive topic in benign, hypothetical, educational, or fictional contexts to test context recognition.The strategies include safety-training examples, story or script settings, and explicit statements that the context is hypothetical and safe.
False-Positive Mitigation: A retrieved maintenance exemption lets the LLM Verifier override superficial “SSH” and “backup” keyword matches and permit system maintenance.The case study reports a low Projector Harm Score of 0.0853 for the administrative backup context.
Rule Evolution and Judgment: The evolutionary rule generator contrasts harmful attacks with benign usage to derive nuanced exemption clauses without over-generalizing.Rule refinement merges similar safety-rule nodes, while the judgment prompt combines dynamic safety signals with retrieval-augmented exemptions.