Source-linked AI summary
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
Junjie Zhang, Hui Liu, Kecheng Chen, Xianbo Mo, Changsheng Chen, Haoliang Li
TL;DR
Jailbreaks against product-level agents can cause harmful tool use and persistent state changes, while trajectory-based red-teaming may reuse misleading experiences and impose context and interpretability costs. RedEvoAgent distills cross-case trajectories into an evolving human-readable attack skill using tool-effectiveness profiling, Deciding-Tool Attribution, and validation-gated updates. Across multiple benchmarks, target models, and execution harnesses, it outperforms fixed and agentic baselines, improves tool efficiency, and transfers across attacker models and harnesses.
Problem
Product-level agents require scalable black-box red-teaming because jailbreaks can produce harmful tool use and persistent state changes, while trajectory retrieval suffers from bias, unclear tool credit, context overhead, and poor interpretability.
Method
RedEvoAgent distills cross-case attack trajectories into a concise human-readable skill, profiles tool effectiveness, attributes deciding tools, and accepts skill updates only after validation improvement.
Results
Across multiple benchmarks, target models, and execution harnesses, RedEvoAgent outperforms fixed and agentic baselines, improves attack-tool efficiency, and transfers across attacker models and harnesses.
Takeaways & Limitations
The evolved skill records tool priorities and attack strategies, supports directed tool selection, and can be transferred across attacker models and target execution harnesses.
Takeaways & Limitations
Trajectory-based alternatives remain constrained by retrieval bias, unclear individual tool credit, context overhead, and low-level records that hinder human auditing.
Abstract
from arXiv · showhide
LLM-based agents are increasingly deployed in product-level execution harnesses, where jailbreaks can trigger harmful tool use and persistent state changes, creating greater risks than unsafe text generation alone. Existing automatic red-teaming methods often rely on fixed attacks, while recent agentic attackers coordinate multiple jailbreak tools and show stronger potential through trajectory-based retrieval. However, such retrieval can reuse misleading experiences due to retrieval bias and unclear tool credit, and full trajectories add context overhead while reducing interpretability. We propose RedEvoAgent, a black-box red-teaming agent that distills cross-case attack trajectories into a concise, human-readable attack skill. The attack skill adaptively evolves through tool-effectiveness profiling and Deciding-Tool Attribution for skill updates, and a validation ratchet that retains only updates improving validation performance. Experiments on multiple benchmarks, target models, and target execution harnesses show that RedEvoAgent outperforms fixed and agentic baselines, improves tool efficiency, and transfers across attacker models and target execution harnesses.
1 INTRODUCTION
RedEvoAgent addresses the security risks of jailbreaks against product-level agents by evolving a concise attack skill from attack experience. It combines tool profiling, trajectory analysis, and validation-based acceptance to improve red-teaming effectiveness and transferability.
- Motivation: Product-level agent jailbreaks can trigger destructive tool use and persistent system-state changes, motivating scalable adversarial evaluation.These risks extend beyond unsafe text generation because agents can modify files, transfer data, and call external APIs.
- Motivation: Fixed jailbreak mechanisms explore limited attack spaces, potentially missing vulnerabilities exposed by alternative attack strategies.Examples include prompt transformation, adversarial optimization, and other predefined mechanisms.
- Motivation: Trajectory-based retrieval can reuse misleading experiences because retrieval bias and unclear tool credit destabilize optimization, while full trajectories increase context overhead and reduce interpretability.Low-level action records also hinder human auditing of strategy evolution.
- Approach: Candidate skill updates are retained only when they improve validation performance, with the test split locked for final evaluation.This validation ratchet restricts evolution to revisions that improve held-out attack performance.
- Approach: RedEvoAgent distills successful and failed attack trajectories into a concise, human-readable skill while profiling tool effectiveness and extracting tool priorities and orchestration strategies.The framework uses an extensible toolbox including methods such as GCG and FlipAttack.
- Results: Experiments across ASB, AgentHarm, multiple target models, and Claude Code and Codex harnesses show stronger performance than fixed and agentic baselines, with transfer across attackers and harnesses.The reported evaluation also demonstrates improvements in attack-tool efficiency.
2 RELATED WORK
Prior work spans fixed jailbreak mechanisms, multi-tool automatic red-teaming, and experience-based skill learning. RedEvoAgent extends this direction by evolving a single natural-language tool-orchestration skill through incumbent-relative held-out validation.
- Jailbreak Attacks and Automatic Red-Teaming: Earlier jailbreak attacks typically use one predefined mechanism, including gradient-based, evolutionary, iterative-refinement, or tree-structured black-box search.These methods optimize or refine attacks within specified mechanisms rather than evolving a shared orchestration skill.
- Jailbreak Attacks and Automatic Red-Teaming: Recent automatic red-teaming methods select or combine multiple attack strategies, including tool-based selection modeled as a contextual bandit.This line of work moves beyond single fixed attacks toward adaptive coordination across cases.
- Skill Learning and Evolution from Experience: Experience-based skill learning converts past runs into reusable natural-language instructions, while skill evolution revises them using later-run feedback.Trace2Skill combines recurring trajectory lessons, and SkillOpt validates candidate edits on held-out data.
- Skill Learning and Evolution from Experience: RedEvoAgent is presented as the first automatic red-teaming method to evolve one natural-language tool-orchestration skill through incumbent-relative held-out validation.The method contrasts with retrieving individual past episodes at test time.
3 METHOD
RedEvoAgent evolves a concise attack skill for black-box red-teaming by combining tool-efficacy measurements with attributed attack trajectories and validation-gated revisions. Its attacker uses a multi-tool toolbox and updates only the skill while target and attacker components remain fixed.
- 3.1 PROBLEM FORMULATION: RedEvoAgent models black-box red-teaming as learning an attack skill that guides an attacker agent against a target model and execution harness.The attacker selects jailbreak-tool calls or QUERYTARGET actions within a bounded execution budget.
- 3.2 ATTACKER AGENT ARCHITECTURE: The attacker toolbox integrates seven complementary jailbreak tools, including gradient-based, evolutionary, generator-based, template, disguise, role-playing, and prompt-substitution methods.Prompt Substitution rephrases failed prompts while preserving their intent.
- 3.2 ATTACKER AGENT ARCHITECTURE: The attack skill is a compact Markdown document inserted into the attacker’s system prompt, summarizing measured tool efficacy and reusable orchestration heuristics.The attacker repeatedly reasons over the skill and execution history to choose tool calls or target queries.
- 3.3 EXPERIENCE COLLECTION: RedEvoAgent collects isolated tool evaluations and complete training trajectories, which together provide complementary inputs for skill distillation.Trajectories record tool calls, candidate prompts, target queries, responses, and case scores.
- 3.3 EXPERIENCE COLLECTION: Deciding-Tool Attribution assigns each successful trajectory to the tool immediately preceding its first successful target query, reducing self-reinforcing tool-selection bias.The attributed trajectories and empirical tool profile are supplied to the distiller to produce candidate skills.
- 3.4 VALIDATION RATCHET: Candidate skills are synthesized from the current skill, tool profile, training trajectories, and rejection context, then accepted only when validation performance improves.Rejected candidates and their validation scores remain in the rejection context, while accepted updates trigger fresh training-trajectory collection and clear that context.
4 EXPERIMENTS
Experiments across benchmarks, target models, and execution harnesses show that RedEvoAgent improves attack effectiveness and efficiency over fixed and agentic baselines, while its learned skills transfer across settings. Ablations identify skill guidance, experience components, and validation-ratchet behavior as important to performance.
- Experimental setup: Experiments use ASB and AgentHarm across multiple target models and Claude Code and Codex execution harnesses, reporting attack success, refusal, harm, and tool-call efficiency.The attacker uses GPT-4o mini with a 20-step interaction budget; skill evolution runs for four rounds.
- Comparison with attack baselines: RedEvoAgent matches or exceeds the strongest isolated attack tool in every tested setting and outperforms fixed and agentic baselines.On AgentHarm with DeepSeek-V4-Flash under Claude Code, it reaches 74.3 HarmScore versus 67.9 for FlipAttack; on ASB with MiniMax-M2.5 under Codex, it reaches 92.8 ASR versus 81.1 for FlipAttack.
- Skill contribution: RedEvoAgent achieves the highest attack performance in every evaluated skill comparison while using the fewest attack-tool calls per case.The comparison includes No Skill, generic Human Skill, and the evolved attack skill.
- Experience components: Removing the Tool-Effectiveness Profile, trajectory collection, or Deciding-Tool Attribution lowers ASB test ASR from 93.2% to 76.9%, 85.0%, and 87.5%, respectively.The corresponding degradations are −16.3, −8.2, and −5.7 percentage points.
- Ratchet depth: Validation-ratchet rounds generally raise test scores when performance remains below the isolated-tool ceiling, but validation gains can reduce test performance under distribution mismatch.For AgentHarm/MiniMax, R4 raises validation HarmScore from 18.5 to 28.1 but lowers test HarmScore from 28.8 to 20.8 relative to R2.
- Zero-shot transfer: A transferred attack skill improves over No Skill across attacker models and execution harnesses, reaching 90.5 ASR on Codex versus 80.8 for No Skill.This retains most of the 92.8 ASR achieved by a skill evolved directly for Codex.
5 CONCLUSION
RedEvoAgent learns and evolves a human-readable attack skill from past attack experience. Across experimental settings, it improves attack effectiveness and tool efficiency, transfers across models and harnesses, and depends on toolbox strength and diversity.
- RedEvoAgent learns and evolves an attack skill document from past attack experience.
- Across all experimental settings, RedEvoAgent outperforms the strongest isolated attack tool baseline and existing automatic red-teaming methods.
- The evolved skill achieves the highest attack effectiveness with the fewest attack tool calls in the skill ablation.
- The skill document records tool priorities and attack strategies and transfers across attacker models and target execution harnesses.
- New attack tools can be added to the same evaluation and evolution process, but performance depends on toolbox strength and diversity.