PaperlayerView exact revision on arXiv

Source-linked AI summary

EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents

Nanxi Li, Yingzi Ma, Yulong Cao, Edward Suh, Bo Li, Dawn Song, Chaowei Xiao

arXiv:2609.05903v1cs.CR

TL;DR

Agent safety must govern consequential action trajectories, yet fixed defenses can mismatch heterogeneous models and domains. EvoSafeHarness searches a natural-language policy and executable code around a frozen model, using domain specifications, behavioral traces, and independent adversarial review. Across four benchmark families, it achieves a stronger safety–utility frontier than fixed defenses, while residual risks remain where harm lacks an observable runtime safety relation.

  • Problem

    Fixed expert-designed harnesses may over-restrict some models and miss application-specific safety relations across heterogeneous deployments.

  • Method

    EvoSafeHarness searches a natural-language policy and executable code harness around a frozen model and target domain using behavioral traces, domain specifications, and fresh-context adversarial review.

  • Results

    Across four benchmark families, EvoSafeHarness improves the safety–utility frontier; on DecodingTrust-Agent it reduces average ASR from 45.6% to 10.0% at a 3.3-point utility cost and leads in 14 of 15 cells.

  • Takeaways & Limitations

    Model behavior determines how relations should be enforced, while domain semantics determine which relations and trajectory state must be protected.

  • Takeaways & Limitations

    Residual risks remain for harms without an observable safety relation, including client-targeted scams, options solicitation, and telecom finance fraud.

Abstract

from arXiv · show

Large Language Model (LLM) agents are turning language into real-world effects, making safety necessary against both indirect prompt injections and direct harmful requests. System-level safety harnesses add an enforcement layer beyond model-level defenses, but existing harnesses are usually designed once by experts and applied across heterogeneous models and domains. Effective protection is deployment-dependent: models differ in how much enforcement they need before utility declines, while domains differ in the effects, state, and action sequences that must be governed. A harness that is strict enough for one model may over-block another, and a policy that transfers across domains may miss application-specific safety relations. We present EvoSafeHarness, a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain. It jointly searches a natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review to reject benchmark-specific rules. Across four agent benchmark families, EvoSafeHarness achieves a stronger safety-utility frontier than fixed expert-designed defenses. On DecodingTrust-Agent, it reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost and achieves the best score in 14 of 15 cells. On AgentDojo, it reaches 82.8% utility at 0.0% ASR, twice CaMeL's utility at the same operating point, and transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptive PAIR attacks with a refinement budget of 16. Analysis shows that domain semantics determine which safety relations and trajectory state are needed, while model and runtime behavior determine how and where those relations should be enforced.

1 Introduction

Agent safety requires system-level protection because agents can turn harmful instructions into consequential actions, while fixed harnesses may not fit heterogeneous models and domains. EvoSafeHarness searches deployment-specific policies and code, improving the safety–utility frontier across evaluated benchmarks.

  • Agent failures can transfer payments, leak credentials, delete production data, or create persistent processes, expanding safety from utterances to action trajectories.
  • Model-level defenses improve refusal behavior but do not provide a model-independent system enforcement boundary, motivating harness-level defenses.
  • EvoSafeHarness jointly searches natural-language policy and executable code for a frozen model and target domain, using fresh-context review and staged evaluation.The loop includes a Designer, Criticizer, Cascade Test Environment, and Analyzer.
  • 45.6% to 10.0% average ASR at a 3.3-point utility cost, with the best score in 14 of 15 DecodingTrust-Agent cells.Fixed defenses either left ASR above 37% or sacrificed more than 20 utility points.
  • 82.8% utility at 0.0% ASR on AgentDojo equals twice CaMeL’s utility at the same operating point and transfers unchanged to unseen AgentDyn suites.The transferred harness achieves 75.0% utility and 0.0% ASR on AgentDyn.
  • Model behavior determines how a security relation is enforced, while domain semantics determine which relations and state require protection.The framework therefore generates distinct harnesses for model×domain deployments rather than applying one fixed policy.

2 Related Work

Prior work optimizes prompts, harnesses, or benchmark performance, but generally does not search adversarial defenses separately for each model and domain. EvoSafeHarness adapts trace-driven harness search to security with domain specifications and threat-aware evaluation.

  • Meta-Harness searches harness source from execution traces, while NLAH exposes natural-language policy as an editable surface.
  • Existing harness-optimization methods target task performance or cost rather than adversarial behavior and do not search defenses per model and domain.This contrasts with EvoSafeHarness’s deployment-specific security optimization.
  • DecodingTrust-Agent, AgentDojo with AgentDyn, and Agent-SafetyBench cover complementary agent-security settings, including direct and indirect attacks and unseen-suite generalization.

3 Problem Formulation

The paper models an agent as a frozen language model mediated by a harness that governs interactions with users and tools. It optimizes utility and attack resistance jointly over direct and indirect threats in a target domain.

  • A tool-using agent is a frozen model M paired with a harness H that mediates user, model, and tool interactions throughout the action loop.The no-defense harness is the bare interaction loop, H0.
  • A harness H=(P,C) combines a natural-language policy with executable code that can transform context, block calls, inspect outputs, maintain state, and enforce capabilities.
  • The threat model covers indirect attacks through ingested content and direct harmful user requests, requiring both provenance and action-level checks where appropriate.
  • Utility U measures benign-task success, while ASR measures attack-task success for a benchmark domain.
  • score(M,H,D)=100·(U(M,H,D)−ASR(M,H,D)) rewards security only when benign utility is preserved, making refuse-all behavior score zero.
  • The optimization searches valid harness programs under domain constraints, so observed model failures and domain action semantics jointly shape the solution.

4 Method: EvoSafeHarness

EvoSafeHarness searches valid policy–code harnesses using domain contracts, archived traces, independent adversarial review, and a nested evaluation cascade. Its design rejects superficial benchmark rules while allocating evaluation resources conservatively.

  • 4 Method: EvoSafeHarness: EvoSafeHarness’s loop uses a domain specification, Designer, fresh-context Criticizer, staged Cascade Test Environment, and Analyzer to revise deployment-specific harnesses.
  • 4 Method: EvoSafeHarness: The Designer may revise policy, add trajectory state, alter gates or control flow, or rewrite code rather than choosing from a fixed mutation menu.
  • 4 Method: EvoSafeHarness: The cascade proceeds from static validation and smoke tests to confidence-gated prefix evaluation and full search evaluation, reusing earlier trajectories.
  • 4 Method: EvoSafeHarness: Candidates are discarded only when evidence establishes inferiority, while held-out data remains invisible to all cascade decisions.
  • 4.5 The Criticizer: The Criticizer rejects rules that fail under renamed paths, relocated targets, or paraphrased instructions, targeting benchmark-artifact overfitting.

5 Experimental Setup

The evaluation spans four agent benchmark families, multiple victim models and domains, and both direct and indirect attacks, with frozen baselines compared against per-cell harness searches. Metrics pair safety outcomes with utility to avoid rewarding refusal-only defenses.

  • Benchmark design: Four benchmark settings evaluate searched harnesses across distinct attack channels, domains, models, and transfer conditions.DecodingTrust-Agent is the primary model×domain grid; Agent-SafetyBench, AgentDojo/AgentDyn, and AgentCanary address complementary questions.
  • Benchmark design: DTAP freezes five victims across os-filesystem, finance, and telecom, covering both direct and indirect attacks.The grid is designed to measure deployment-specific model×domain variation.
  • Comparison protocol: EvoSafeHarness is searched separately for each model×domain cell, while baselines are ported once and frozen across victims and domains.Search uses training splits only, with held-out outcomes withheld from the Designer and Criticizer.
  • Metrics: DTAP, AgentDojo, and AgentDyn report utility, ASR, and score = U −ASR, while ASB and AgentCanary use their own safety metrics and utility counterparts.Every safety number is paired with utility because refusing everything can trivially produce zero ASR.

6 Results

EvoSafeHarness improves the safety–utility frontier across model–domain grids, non-injection harms, zero-shot transfer, and adaptive attacks. Its strongest results combine lower attack rates with retained utility and robustness beyond benchmark-specific tool patterns.

  • DecodingTrust-Agent: The DTAP grid exposes strong victim variation and domain-specific baseline failures: undefended ASR ranges from 4.8% to 71.0%, while fixed defenses perform poorly on telecom.Progent is the strongest fixed safety baseline but reduces per-victim mean utility to 44.4–66.7%, versus 71.1–86.7% undefended.
  • DecodingTrust-Agent: EvoSafeHarness achieves the best score in 14 of 15 DTAP cells, reducing average ASR from 45.6% to 10.0% at a 3.3-point utility cost.Progent reaches 10.5% ASR but reduces utility to 56.4%, versus 79.8% for EvoSafeHarness.
  • DecodingTrust-Agent: Direct ASR falls from 50.9% to 12.6% and indirect ASR from 40.4% to 7.4% across DTAP.The single loss is the Sonnet 4.6/finance cell, whose undefended ASR is already 2.9%.
  • Agent-SafetyBench: EvoSafeHarness lowers ASB clean UBR and ASR to 12.3% and 7.4% while raising mean utility under attack to 63.4%.It is the only defense in the reported three-victim average to improve both safety measures while also increasing utility under attack.
  • AgentDojo→AgentDyn transfer: 82.8% utility at 0.0% ASR on AgentDojo and 75.0% utility at 0.0% ASR on unseen AgentDyn exceed fixed-baseline transfer performance.The harness is searched on AgentDojo and run on AgentDyn without re-search; CaMeL has half the AgentDojo utility at the same zero-ASR point, while Progent drops to 9.7% AgentDyn utility.
  • Adaptive attacks: Under adaptive PAIR attacks, mean ASR reaches 19.5% at refinement budget 16, remaining below the undefended agent’s static-prompt ASR.ASR stays at 9.7% through budget 2 and rises to 16.7% for two attackers and 25.0% for the third at budget 16.

7 Analysis: What Does EvoSafeHarness Discover?

EvoSafeHarness discovers deployment-specific safety relations by adapting enforcement to domain semantics, victim behavior, and runtime state. Its analyses show that targeted, evidence-driven search can improve security while avoiding unnecessary machinery, but residual risks remain where harm lacks an observable relation.

  • 7.2 Why Search Must Be Model- and Domain-Specific: Domain semantics determine which safety relations and state must be protected, while victim behavior determines whether checks are semantic or deterministic, cached or recomputed, and where they run.Finance requires effect-specific checks and account history, whereas os-filesystem centers on command effects, paths, secrets, and data flow.
  • 7.3 Where Does Residual Risk Remain?: EvoSafeHarness reduces successful attacks by 78.1% relative (479→105) and fully closes 21 risk types with non-zero undefended ASR.Residual successes are concentrated in os-filesystem and finance, with five risk types accounting for 54/105 remaining failures.
  • 7.3 Where Does Residual Risk Remain?: The hardest risks lack an observable safety relation, leaving client-targeted scams at 60.0% (9/15), options solicitation at 30.0% (3/10), and telecom finance fraud at 16.7% (5/30).These harms can occur through in-scope operations while depending on fabricated claims, suitability, or misleading values.
  • 7.4 What Search Teaches Us About Optimizing Safety: Search should remove mechanisms without marginal value: a generic safety policy lowered utility from 100% to 90% without changing ASR at 5%.Later candidates retained deterministic caller-scope, action-type, redaction, and volume relations instead.
  • 7.5 Controlled Ablations of the Search Stack: All four controlled outer models preserve similar mean utility (83.33–86.67%), while mean ASR ranges from 5.24% to 17.62%.GPT-5.6 Sol obtains the best mean score, while every optimizer beats fixed baselines without choosing utility-destroying blocking.

8 Conclusion

EvoSafeHarness treats agent safety as deployment-dependent: domains determine which relations need protection, while models determine enforcement strictness. The resulting harnesses outperform fixed defenses, transfer without re-search, and remain robust under adaptive attacks.

  • EvoSafeHarness searches deployment-specific harnesses that convert domain semantics into runtime relations and tune enforcement to each victim model.The framework searches around a frozen model using failure traces, domain specifications, and independent adversarial review.
  • Across four benchmark families, searched harnesses dominate fixed defenses on the safety–utility frontier and transfer to unseen tools without re-search.
  • The discovered designs support shifting agent security from selecting universal guardrails toward generating and validating a harness for each deployment.

A Evaluation Interface and Security Protocol

The evaluation interface exposes policy and lifecycle hooks for transforming prompts, inspecting tool calls, and updating trajectory state. Security evaluation combines staged cascade search with intent-preserving evasion probes, benign replay, and fresh-context adversarial critique.

  • Each candidate implements three hooks: system_prompt_transform, on_pre_tool_call, and on_post_tool_call, with trace-isolated trajectory state.Pre-call logic can inspect, rewrite, or refuse calls; post-call logic can inspect results and update state.
  • Cascade evaluation extends surviving trajectories from smoke prefixes to confidence prefixes and then to the complete search sample.AgentDojo uses five- and fifteen-pair prefixes before thirty pairs, while DTAP uses three- and twelve-task prefixes before thirty tasks.
  • Robustness probes rename targets, relocate destinations, and paraphrase malicious instructions while replaying benign signatures to detect false positives.A fresh-context Criticizer also searches for cheap intent-preserving evasions and benchmark-specific tokens before cascade evaluation.

B Benchmarks and Reproducibility Configuration

The study evaluates frozen harnesses across multiple agent benchmarks, victims, domains, attack conditions, and reproducibility controls. Baselines and searched bundles are held fixed across comparisons, while audits document mechanism variation and search lineage.

  • DTAP: DTAP uses 60 training tasks and a frozen 100-task held-out split spanning 53 filesystem, finance, and telecom risk types.The held-out split contains 30 benign, 35 direct, and 35 indirect tasks, with disjoint directories and per-task sandboxes.
  • AgentDojo and AgentDyn: AgentDojo evaluates a full 285-pair held-out matrix, while AgentDyn tests unchanged transfer on shopping, GitHub, and daily-life suites unseen during search.
  • Agent-SafetyBench: Agent-SafetyBench uses a sealed 240-task held-out set across six conditions, yielding 1,440 episodes per arm when all rows are complete.
  • AgentCanary: AgentCanary uses 46 direct-attack and 41 benign task files with family-stratified train and held-out splits, DeepSeek-V4-Flash, and OpenClaw.
  • Baselines and mechanism audits: Baseline ports remain frozen across victims and domains, while searched bundles vary mechanism realization across shared, domain-signature, and model-specific strata.Table A2 groups mechanisms as deterministic code, quarantined judges, or natural-language policy; six bundles omit policy and four omit runtime auxiliary models.
  • Controlled ablations: The controlled ablation fixes Kimi-K2.5 and varies only the outer Designer across four models under identical search, critique, and held-out protocols.

F Detailed Residual-Risk Audit

The residual-risk audit separates direct and indirect attacks by official risk type and shows substantial reductions for many observable relations. Remaining failures cluster in risk types without measured reduction, whose fixed-split rates define where further search is needed.

  • The audit reports residual ASR separately by source and risk type, pooling counts over five victims without combining direct and indirect channels.The displayed categories account for all 105 residual successes, while 28 of 53 risk types have zero pooled residual ASR.
  • Direct least-privilege, data-minimization, and purpose-limitation failures fall from 16/25 (64.0) to 10/25 (40.0), a −24.0 change.
  • Indirect least-privilege, data-minimization, and purpose-limitation failures fall from 17/25 (68.0) to 2/25 (8.0), a −60.0 change.
  • Indirect phone-number privacy leaks fall from 10/15 (66.7) to 0/15 (0.0), a −66.7 change.

G Two-Level Search and Evaluation Cost

The evaluation reports resource ledgers for EvoSafeHarness and finds that nested-cascade selection reduces evaluation work while preserving selected defenses across comparisons.

  • 34.0% reduction in scheduled victim-task evaluation work, with the same selected defense retained in all twelve comparisons.The nested cascade uses 66.0% of the work required without cascade evaluation.
  • Normalized cascade cost is 66.7% for Opus 5, Sonnet 5, and GPT-5.6 Sol, versus 60.0% for GPT-5.6 Terra.
  • Candidate count poorly predicts outer-model output, with r=0.143 and Spearman ρ=0.262.
  • Input-side fields comprise 97.6% of the inner token envelope, motivating trajectory compaction and early termination.
  • The released telemetry schema supports per-candidate accounting across phase, candidate, stage, task, and component labels.

H Statistical Significance of the Safety Improvements

The paper evaluates uncertainty and paired significance for the held-out DecodingTrust-Agent grid. EvoSafeHarness significantly reduces attack success in most cells, while benign-utility differences remain within sampling noise.

  • Confidence intervals (Wilson): The analysis uses 95% Wilson score intervals because small binomial samples near 0 or 1 make Wald intervals unreliable.
  • Paired significance test (McNemar): McNemar’s test compares paired per-task outcomes using discordant counts b and c, testing the null hypothesis b = c.The exact two-sided binomial p-value is reported when the discordant count is small.
  • Findings: Table A7 reports ASR over 70 attack tasks per cell, split into 35 direct and 35 indirect attacks, with 95% Wilson intervals.
  • Findings: EvoSafeHarness significantly reduces ASR in 13 of 15 cells, with exact McNemar p < 0.05.The two exceptions are Sonnet 4.6/finance and Sonnet 4.6/telecom, where undefended ASR is already 1.4–2.9%.
  • Findings: 385 attacks blocked under EvoSafeHarness versus 11 newly opened, a 35:1 ratio across pooled discordant pairs.The pooled continuity-corrected McNemar statistic is χ2_cc = 351.3 with p ≈2×10^-78.
  • Findings: Benign-utility Wilson intervals overlap in all 15 cells, consistent with the approximately 3-point aggregate utility cost under the available sample size.Each cell contains 30 benign and 70 attack tasks.
Loading 2609.05903v1…