PaperlayerView exact revision on arXiv

Source-linked AI summary

VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails

Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra, Shuo Li, Krishnamurthy Dj Dvijotham, Mohit Bansal, Tomas Pfister, Long T. Le

arXiv:2610.12813v1cs.AIcs.CLcs.CRcs.LG

TL;DR

Guardrails for LLM agents must make inspectable decisions while adapting to risks that emerge after deployment. VEPA evolves formal policies from feedback using deterministic verification and evidence-gated rule activation. Across three benchmarks, it outperforms LLM-only and adaptive memory guardrails, with advantages under challenging deployment conditions.

  • Problem

    Adaptive guardrails can learn from deployment feedback, while formal guardrails provide deterministic enforcement but typically rely on fixed or task-specific policies.

    Source: §1 · p. 1, F. Limitations and Future Work · p. 28

  • Method

    VEPA uses typed events and executable symbolic rules, with LLM-proposed policy patches checked deterministically and rule authority gated by supporting evidence.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    VEPA consistently outperforms LLM-only and memory-based adaptive guardrails across multiple safety and security benchmarks, with advantages persisting under heterogeneous streams, noisy feedback, longer horizons, and different model backbones.

    Source: §1 · p. 3, §6 · p. 13

  • Takeaways & Limitations

    The results suggest formal guardrails can adapt over time while retaining a deterministic post-parsing authority boundary.

    Source: §6 · p. 13

    VEPA’s guarantees apply after mapping interactions to a fixed typed event; its parser remains neural, and its policy language supports binary allow/refuse decisions rather than richer temporal or relational requirements.

    Source: F. Limitations and Future Work · p. 28

Abstract

from arXiv · show

LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrails should therefore make inspectable, reproducible, and deterministic allow or refuse decisions before potentially irreversible agent actions are executed. However, safety and security requirements are rarely complete before deployment, so guardrails need to keep adapting as new risks emerge. Existing adaptive guardrails learn from deployment experience, but they usually apply safety policies through LLM inference, which can lead to inconsistent decisions and ambiguous policy interpretations. Formal guardrails, on the other hand, enforce policies deterministically, but they usually rely on pre-deployment or task-specific policies and have no mechanism to adapt to newly observed risks. We introduce VEPA, a framework for the verified evolution of formal agent guardrails. VEPA represents the guardrail as a formal policy of executable symbolic rules over typed agent events. During deployment, we use an LLM guided by deployment feedback to propose policy patches that fix newly observed failures while preserving previously correct behavior. Then, a deterministic verifier only accepts policies that are structurally valid and conflict-free. Finally, an evidence-aware authority layer activates a verified rule only when it has enough supporting evidence, so uncertain updates cannot directly affect guardrail decisions. Across ASSEBench-Safety, ASSEBench-Security, and ATBench, VEPA consistently outperforms baselines while expanding the scope of deterministic formal-policy enforcement. These gains also hold under more challenging deployment conditions, including mixed streams, corrupted feedback, and longer horizons.

1. Introduction

VEPA addresses the tension between deterministic, auditable enforcement and the need to adapt guardrails as new risks emerge. It evolves executable symbolic rules from feedback while constraining their deployment authority.

1. Introduction

  • High-stakes guardrails must provide inspectable, reproducible decisions while adapting to risks that emerge after deployment.Agent actions can involve sensitive resources and may be irreversible once taken.Source: §1 · p. 1
  • Adaptive neural guardrails can produce ambiguous, non-deterministic policies, whereas formal guardrails often remain fixed and struggle to incorporate newly observed risks.VEPA aims to bridge these limitations through evolving formal policies.Source: §1 · p. 2
  • VEPA combines LLM-proposed policy patches with deterministic verification and evidence-aware authority, balancing failure repair against preservation of previously correct behavior.Only structurally valid, conflict-free candidates are admitted, and rules need sufficient supporting evidence before affecting deployment.Source: §1 · pp. 2–3
  • VEPA achieves the strongest F1 across ASSEBench-Safety, ASSEBench-Security, and ATBench on events covered by its active policy, with advantages persisting under challenging deployment conditions.The comparison is against LLM-only and adaptive natural-language-memory guardrails.Source: §1 · p. 3

2. Related Work

Prior work includes neural and formal guardrails, as well as approaches that adapt safety policies or memories from deployment experience. VEPA focuses on evolving a persistent formal policy while separating update generation, verification, and authority.

Neural and Formal Guardrails

  • Neural guardrails rely on probabilistic decisions, while formal guardrails enforce executable policies; VEPA focuses on continually evolving persistent formal policy from deployment feedback.The paper describes this as complementary to existing formal-enforcement work.Source: §2 · p. 3

Evolving Safety and Security Policy

  • Prior memory approaches include adaptive checks, structured policy memory, and evolving safety-oriented memories.The paper also notes broader memory work and reports that experience can propagate unsafe behavior.Source: §2 · pp. 3–4
  • Adaptive guardrails update external policies or memories from experience, but VEPA evolves executable formal rules and gates which rules may affect deployment decisions.Its update loop separates LLM-proposed updates, verification, and evidence-aware activation.Source: §2 · p. 4

3. Method

VEPA represents agent interactions as typed events evaluated by executable symbolic rules. Its policy evolves from deployment feedback through constrained search, while evidence-aware authority determines which verified rules can affect decisions.

3.1. Problem Settings

  • VEPA models events with typed contextual fields and applies formal rules to produce deterministic decisions; deployment updates must address failures without introducing conflicts or regressions.The event representation uses a fixed schema, and deterministic guarantees apply after parsing.Source: §3.1 · pp. 4–5

3.2. VEPA: Verified Evolving Policy for Agent Guardrails

  • The framework separates policy evolution from deployment authority: LLMs propose patches, deterministic checks verify candidates, and only sufficiently supported rules become active.The overview describes two layers: policy-patch generation and verification, then evidence-gated activation.Source: §3.2 · p. 5, Figure 2 · p. 6

3.2.1. Constrained Policy Evolution

  • Policy evolution balances repairing newly observed failures against preserving correct behavior, using LLM-proposed edits and a feasible archive that searches non-dominated trade-offs.Deterministic verification admits only well-formed, conflict-free candidates; the Pareto frontier retains alternatives across the two objectives.Source: §3.2.1 · pp. 6–7

3.2.2. Evidence-Aware Rule Authority

  • Verified rules gain or lose deployment authority according to accumulated evidence, while active rules remain conflict-free and make deterministic decisions on fixed parsed events.New or modified rules are evaluated against accumulated evidence for their current condition; rules without sufficient evidence remain unavailable for inference.Source: §3.2.2 · pp. 8–9

4. Experiments

Across three benchmarks, VEPA’s formal-policy decisions improve and its coverage expands; it also leads under mixed risks, noisy feedback, and hybrid fallback deployment.

4.1. Experimental Setup

  • Experiments used three safety and security benchmarks, adaptive-memory, LLM-judge, and rule-learning baselines, and cumulative prequential macro-F1 averaged across five shuffled-stream seeds.Methods received the same data, feedback, labels, and update schedule; the benchmark events were mapped to allow/refuse decisions.Source: §4.1 · pp. 9–10, §4 · p. 9

4.2. RQ1: Reliability and Scope of Formal Authority

  • VEPA achieved the highest F1 across all three datasets, reaching 0.81 on ASSEBench-Safety and 0.76 on ATBench and beating the strongest baselines by 0.02 and 0.09, respectively.Baselines were evaluated on the same events covered by the active formal policy.Source: Figure 3 · p. 10, §4.2 · p. 11
  • Active-policy coverage grew monotonically, reaching approximately 80%, 89%, and 88% on ASSEBench-Safety, ASSEBench-Security, and ATBench, respectively.Coverage is the fraction of newly observed events for which the pre-update policy returns allow or refuse.Source: Figure 4 · p. 10, §4.2 · p. 11

4.3. RQ2: Robustness under Realistic Deployment Conditions

  • On the mixed stream, VEPA reached approximately 0.86 F1, versus 0.78 for AGrail and 0.80 for LiSA, and led at every step.The stream combined the three benchmarks and shifted the prevalence of safety and security risks.Source: Figure 5 · p. 11, §4.3 · p. 11
  • With adaptation labels independently flipped at probability ρ=0.1, VEPA reached approximately 0.81 F1, versus 0.77 for AGrail and 0.76 for LiSA.VEPA remained the strongest method and continued improving throughout deployment under corrupted feedback.Source: Figure 5 · p. 11, §4.3 · p. 11
  • In hybrid deployment, the formal policy decided covered events and an LLM judge handled uncovered events; the system consistently achieved the best performance.VEPA surpassed all baselines after the first step and maintained its lead, while formal guarantees remained limited to covered events.Source: Figure 5 · p. 11, §4.3 · p. 12

5. Analysis

The analysis tests VEPA’s components, its behavior over a longer mixed stream, and whether its advantage persists with alternative LLM backbones.

5.1. Ablation Study

  • Removing verification caused the largest final-round drop, 0.13 F1; greedy search reduced F1 by 0.04, and removing the authority gate reduced it by 0.03.The ablation variants respectively omitted verification, replaced archive-based Pareto search with greedy search, or activated every verifier-admitted rule immediately.Source: Figure 6 · p. 12, §5.1 · p. 12

5.2. Long-Horizon Deployment

  • The long-horizon evaluation measures performance over a 20-round mixed deployment stream.The analysis also examines longer deployment streams; the supplied text does not report the figure’s outcome.Source: Figure 7 · p. 12, §5 · p. 12

5.3. Generalization across LLM Backbones

  • VEPA maintained an advantage over baselines when either the update-time proposer or deployment-time parser and inference model was replaced.The proposer replacement was Gemini-3.6-Flash; the deployment-time model replacement was Gemini-3.5-Flash-Lite.Source: Figure 8 · p. 13, Figure 9 · p. 13, §5.3 · p. 13

6. Conclusion

VEPA evolves executable symbolic guardrail policies while retaining deterministic, post-parsing enforcement, and the paper situates this approach against neural, memory-based, and formal alternatives.

6. Conclusion

  • VEPA combines typed events, LLM-guided archive-based Pareto search, deterministic verification, and evidence-aware activation to evolve symbolic guardrail policies.The search balances repairing newly observed failures against preserving previously correct behavior.Source: §6 · p. 13

A.1. Detailed Comparison of Guardrails

  • Neural and memory-based guardrails can adapt but rely on LLM-mediated decisions, while formal guardrails typically use fixed or task-specific policies rather than persistent evolution.The comparison defines deterministic enforcement for VEPA as applying to covered decisions after parsing.Source: §A.1 · p. 18

A.2. Choice of Empirical Baselines

  • Empirical comparisons were limited to methods that can use the same online interaction stream, feedback, and update schedule without deployment-specific starting policies.The authors compare against AGrail, LiSA, Hoeffding Tree, and RIPPER; several other methods require different inputs or supervision settings.Source: §A.2 · pp. 18–19

B.1. Datasets

The experiments use shared sequential deployment streams with feedback revealed after each round, comparing VEPA from an empty formal policy against adaptive and symbolic-learning baselines. The setup also specifies the policy-evolution objective, authority thresholds, and rule semantics.

B.1. Datasets

  • The mixed-stream experiments combine ASSEBench-Safety, ASSEBench-Security, and ATBench, with examples shuffled into sequential deployment rounds.Methods within a seed receive the same event ordering and feedback stream; adaptive methods start without deployment feedback.Source: §B.1 · p. 19
  • Feedback is revealed for 50% of examples after each round by default, with uniform sampling and shared revealed examples across adaptive methods.Noisy-feedback labels are independently flipped with probability 0.1; sparse ratios and targeted prompt-injection corruption are also evaluated.Source: §B.1 · p. 19

B.2. Baselines and Hyperparameters

  • Baselines include memory-based AGrail and LiSA, incremental Hoeffding Tree, batch-retrained RIPPER, and LLM No Memory.All adaptive baselines receive the same stream, feedback, labels, and update schedule as VEPA.Source: §B.2 · p. 20
  • VEPA searches at most K=30 LLM-proposed patches per update and retains feasible candidates alongside all observed failures and correct examples.Its macro-F1 objective rewards repairing observed failures while penalizing losses on examples correctly handled by the prior policy.Source: §B.2 · p. 20
  • Rule authority depends jointly on supporting and contradicting evidence, with contradictory examples increasing the support required for activation.Under the stated default settings, zero, one, or two contradictions require approximately 13, 32, or 51 supporting examples, respectively.Source: §B.2 · p. 20
  • Deployment risk preferences can be encoded with separate allow and refuse thresholds, and stricter settings require stronger evidence before permissive rules activate.The experiments use the same threshold values for allow and refuse decisions.Source: §B.2 · p. 20
  • The policy DSL evaluates conjunctions of categorical predicates deterministically, treating a missing field as the ordinary value unknown.To stop missing information from satisfying a negative predicate, unknown must be included explicitly in its not_in set; oversized proposer feedback packets may be subsampled while archives remain intact for candidate evaluation.Source: §B.2 · p. 21

B.3. Detailed Experimental Setup

The experiments evaluate deployment-time decisions before feedback is revealed, then test sparse and targeted-corrupted feedback conditions. VEPA leads the reported comparisons under both conditions.

B.3. Detailed Experimental Setup

  • When no formal rule applies, the hybrid system uses Gemini-3.1-Flash-Lite as fallback; Gemini-3.1-Pro proposes policy updates by default.The fallback receives the event and current policy as context.Source: §B.3 · p. 21
  • Evaluation reports macro-averaged F1 on allow and refuse decisions prequentially, before feedback for each unseen batch is revealed.Results are means across five independently shuffled seeds, with learning-curve shading denoting standard deviation.Source: §B.3 · p. 21
  • Covered-policy comparisons evaluate every baseline on the same unseen events covered by VEPA’s pre-update active policy.The ablation protocol instead scores each variant across the full unseen batch without fallback, while the full-stream hybrid evaluation routes uncovered events to an LLM fallback.Source: §B.3 · p. 22

C.1. Sparse and Adversarial Feedback

  • 0.79 F1 with 5% feedback and 0.81 with 10%: VEPA has the highest F1 at every tested feedback ratio.Its gains are 0.04 F1 at 25% feedback and 0.06 at 50%.Source: §C.1 · p. 22, Table 3 · p. 22
  • 0.83 F1 final-round mean: VEPA improves from 0.68 under targeted prompt-injection feedback corruption and exceeds the strongest baseline’s 0.81.The figure concerns corruption concentrated on prompt-injection examples; VEPA exceeds LiSA and AGrail by 0.05 and 0.08 F1, respectively.Source: §C.1 · p. 22, Figure 10 · p. 22

C.2. Analysis of Adaptation-Regression Trade-off

VEPA achieves the strongest reported adaptation-regression trade-off, while its verified rule archive grows faster than the subset authorized for deployment. The distinction reflects that verification alone does not grant deployment authority.

C.2. Analysis of Adaptation-Regression Trade-off

  • 0.86 F1 with 3.8% regression: VEPA has the highest adaptation F1 and lowest regression rate in the comparison.Regression is the percentage of previously correct examples made incorrect after updates; LiSA reports 0.80 F1 and 8.5% regression, while RIPPER reports 0.79 F1 and 12.7%.Source: §C.2 · p. 23, Figure 11 · p. 23

C.3. Analysis of Rule Coverage during Deployment

  • The active rule set grows more conservatively than the verified archive because deployment authority requires sufficient evidence beyond structural and conflict checks.Rules with limited or contradictory evidence remain inactive while accumulating support, and the final active set remains substantially smaller than the archive while preserving VEPA’s covered-event performance advantage.Source: §C.3 · p. 23

C.4. Results on Open-Source Model

With Qwen-3.8-27B, VEPA retains advantages across mixed-stream, noisy-feedback, and LLM-fallback settings. Its advantage also persists over a 30-step deployment horizon.

C.4. Results on Open-Source Model

  • With Qwen-3.8-27B, VEPA maintains an advantage across mixed-stream, noisy-feedback, and LLM-fallback settings, including the highest final-round F1 in the mixed stream.The hybrid system with Qwen fallback achieves the strongest full-stream performance.Source: §C.4 · p. 23, Figure 13 · p. 24

C.6. Long-Horizon Streams

  • 0.85 ± 0.07 mean F1: VEPA leads the 30-step evaluation, exceeding LiSA by 0.04 F1 and the non-adaptive LLM baseline by 0.14 F1.Its improvements are 0.05 over AGrail, 0.07 over RIPPER, and 0.08 over Hoeffding Tree.Source: §C.6 · p. 24, Table 4 · p. 24

D. Detailed Derivations of Guarantee Boundary in VEPA

VEPA’s guarantee boundary rests on two properties: verifier-preserved structural invariants and evidence-based authority for active rules. The derivations explain what those checks establish and how deployment evidence updates rule authority.

D.1. Structural Invariants

  • Conflict-freedom is checked symbolically over the fixed event schema, rejecting opposite-decision rules whose typed conditions can overlap.This rules out opposite-decision overlap for events representable by the DSL.Source: §D.1 · p. 25
  • If the initial policy is well-formed and conflict-free and each update passes both checks, the active policy preserves those properties at every round.Removing rules from a checked policy does not invalidate remaining rules or create new conflicts.Source: §D.1 · p. 25
  • A fixed parsed event covered by the active policy receives a deterministic, reproducible symbolic decision.The guarantee is conditioned on the parsed event and applies to events covered by the policy.Source: §D.1 · p. 25

Under the activation rule

  • Rule authority is evaluated from covered deployment observations under a fixed-scope working model, with posterior uncertainty governing activation and later demotion.Unchanged rules update incrementally; a newly added or behaviorally modified rule starts from the fixed prior and uses evidence within its new scope.Source: §D.2 · p. 25, Under the activation rule · p. 26
  • The authority criterion is operational rather than a selection-adjusted frequentist guarantee, and repeated evaluation gives neither a time-uniform guarantee nor one for a rule’s first activation round.Retrospective evidence reuse can make support–contradiction statistics optimistic; held-out or prospective evidence could avoid direct reuse but may slow activation and reduce coverage.Source: Under the activation rule · p. 27

E.1. Other Related Work

  • VEPA differs from offline rule-induction approaches by evolving policies online from traces, using an LLM for proposals while deterministic verification and replay-based evaluation preserve authority.Its updates must repair new failures while preserving previously correct behavior.Source: §E.1 · p. 27

E.2. Method Design

VEPA’s deterministic guarantee applies after interactions have been mapped to typed events, not end to end from raw interactions. Adaptation is also bounded by a fixed schema, while archive compression remains outside this work’s focus.

E.2. Method Design

  • Deterministic verification and rule execution begin after an LLM parser maps raw interactions to typed events; parser errors and adversarial inputs remain outside the guarantee.For the same typed event and active policy, the symbolic decision is deterministic and reproducible.Source: §E.2 · p. 27
  • A fixed event schema bounds adaptation to risks expressible using its existing attributes; failures requiring new attributes or concepts need schema and verifier extensions.Novel combinations of existing attributes can still represent previously unseen interactions or attacks.Source: §E.2 · pp. 27–28
  • The experiments retain all observed examples, leaving long-term archive compression to future work.The paper suggests bounded coresets as a possible way to reduce policy-search cost while preserving representative examples.Source: §E.2 · p. 28

F. Limitations and Future Work

VEPA’s limitations include neural parsing before enforcement, a fixed schema, and a binary allow/refuse policy language. The appendix prompts operationalize trace parsing and policy refinement within those representation and evidence constraints.

F. Limitations and Future Work

  • VEPA guarantees deterministic enforcement only after typed-event parsing; its LLM parser can make extraction errors or face adversarial inputs, and binary decisions over a fixed schema may miss richer requirements.The stated future directions include trusted execution hooks, parser verification, temporal and relational rules, and richer enforcement actions.Source: F. Limitations and Future Work · p. 28

G. Prompts

  • The trace prompt directs the parser to produce one typed event for the agent-owned outcome, treating requests, tool definitions, and environment messages as context unless adopted or acted upon.The parser prompt also says to treat ignored prompt injections as context, not agent behavior.Source: G. Prompts · pp. 28–29
  • The refinement prompt asks for focused patches that repair failures while preserving correctly classified examples and minimizing changes to the parent policy.It permits add, delete, rewrite, split, or merge operations and requires checking supplied verifier feedback and schema constraints.Source: G. Prompts · p. 31
  • Patch proposals must use only schema-supported fields and values; when typed fields cannot distinguish repair from keep examples, the prompt requires explaining that representational limit.Unknown is an ordinary categorical value, so prompts caution against unsupported inference and unintended matches in negative conditions.Source: G. Prompts · pp. 29, 31, 32, 33
Loading 2610.12813v1…