Source-linked AI summary
ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing
Yi Ting Shen, Kentaroh Toyoda, Alex Leung
TL;DR
Existing LLM attackers and defenses are largely tested in isolation, with scores that may not distinguish genuine leakage from hallucination or separate attack strength from defense effectiveness. ACEA provides a pluggable, model-agnostic head-to-head arena with verifiable secrets, counterfactual scoring of blocked attacks, live reporting, and optional advisory feedback. The paper presents these components as a unified way to make red-team and blue-team evaluation inspectable and actionable.
Problem
Attackers and defenders are developed separately, while existing scores may conflate hallucinated leaks, attack potency, and defense effectiveness.
Method
ACEA connects pluggable red- and blue-team adapters to a shared target LLM, uses synthetic secrets and blocked-payload scoring, and provides live reports plus optional in-context hints.
Results
ACEA decomposes each round into attack potency and defense effectiveness while supporting model-agnostic adapter participation and observable head-to-head contests.
Takeaways & Limitations
ACEA turns one-sided, post-hoc red-teaming into a head-to-head evaluation with inspectable scores and feedback for improving red or blue projects.
Takeaways & Limitations
Scores inherit LLM-judge biases, counterfactual potency measurements may overstate deployed risk, and the empirical comparison covers four defenses, four targets, and thirty rounds each.
Abstract
from arXiv · showhide
Automated red-team attacks and blue-team defenses for large language models (LLMs) are advancing quickly. However, attackers and defenders are built and tested in isolation, and the resulting scores are hard to trust. To tackle this, we present ACEA (Adversarial Co-Evolution Arena), a platform that connects a pluggable red-team adapter and a pluggable blue-team adapter to a shared target LLM and scores their attack and defense rates with an LLM judge. ACEA contributes four components. First, a pluggable, model-agnostic arena. Any red or blue project connects over a minimal HTTP protocol, which we call the ACEA Standard Adapter Protocol (ASAP). It can be written in any language, and a project that exposes nothing but the protocol is a full participant. Second, an evaluation methodology built for adversarial rounds. Seeding the target with canonical secrets gives verifiable ground truth that separates real leakage from hallucination. We also send each attack to the target even when the defense blocks it, which measures the attack's raw potency independently of whether it was stopped. Together these yield a per-round decomposition of attack strength and defense effectiveness. Third, a real-time, game-style visualization with a detailed end-of-battle report that localizes each failure. The evaluation thus becomes an actionable signal for improving a red or blue project. Fourth, an optional in-context improvement loop that turns each round's outcome into advisory hints for the next. An adapter can then adapt across rounds without keeping state, provided it reads the hints. We describe the design of ACEA and the metrics through which red and blue teams are scored head to head.
1 Introduction
ACEA addresses the difficulty of comparing attackers and defenders by placing pluggable red- and blue-team adapters against a shared target LLM. Its protocol, grounded scoring, and live reporting are designed to make adversarial evaluation directly comparable and inspectable.
- Current red-team and defense efforts are largely developed and evaluated separately, making particular attacker–defender matchups difficult to assess.
- LLM-judge scores may confuse genuine data leaks with hallucinations, while blocked attacks conceal the attacker’s strength.
- ACEA connects arbitrary red and blue projects to a shared target through the minimal ACEA Standard Adapter Protocol.
- Known synthetic secrets let the judge verify real leakage, while scoring blocked payloads separately measures attack potency independently of defense decisions.
- ACEA adds live contest visualization and detailed reporting so users can observe breakthroughs, defensive holds, and localized failures.
2 Related Work
Prior work advances automated attacks, defenses, co-evolution, and evaluation tooling, but these directions leave complementary gaps. ACEA positions itself as a model-agnostic arena that directly matches arbitrary attackers with arbitrary defenders using verifiable scoring and live observation.
- Automated and agentic LLM red-teaming: Automated red-teaming has progressed from single-input optimization to multi-turn, agentic, and learning-based attackers.
- LLM defenses and guardrails: LLM defenses range from moderation guards to model-hardening methods, but are typically shipped as fixed artifacts and tested on fixed benchmarks.
- Self-play, co-evolution, and self-improving agents: Self-play and co-evolution research improves agents through feedback, reflection, prompt evolution, self-judgment, or code evolution.
- Evaluation frameworks and benchmarks: Evaluation frameworks standardize red-teaming with behavior sets, leaderboards, scorers, probe libraries, and agent-harm benchmarks.
- Positioning: ACEA closes three gaps together: arbitrary attacker–defender pairing, verifiable leakage-aware scoring that separates blocked attacks from attack strength, and live contest views.
3 The ACEA Platform
ACEA is a model-agnostic arena that pits pluggable red- and blue-team adapters against a shared target through a fixed, inspectable battle loop. It combines verifiable leakage scoring, independent attack-potency measurement, per-round outcome decomposition, and optional hint-based adaptation.
- Objective and positioning: ACEA connects arbitrary red and blue systems through a common arena while keeping implementations and model weights hidden from one another.Adapters communicate with the arena core over HTTP, and the arena executes each round against the target.
- Arena core and battle loop: Each round generates an attack, evaluates the defense, queries the target, optionally filters the output, and scores the exchange.The loop records whether the defense blocked input, altered output, the target refused unaided, or the attack succeeded.
- Arena core and battle loop: Blocked attacks still trigger one target generation, preserving a measure of raw attack potency without delivering the response.This counterfactual scoring uses the undefended payload and avoids a second target query.
- Target and scoring: Canonical secrets distinguish genuine leakage from hallucinated sensitive-looking content, while participant-declared evidence can resolve some verdicts without a model call.The target inventory contains fixed synthetic records whose material classes are declared by the owner.
- Target and scoring: ACEA separates objective success, incidental confidential disclosure, and defense attribution so attack and defense outcomes are not collapsed into one score.Three-level verdicts classify rounds as success, failure, or partial when the objective fails but confidential material is disclosed; defense rates also distinguish unaided target refusal from defense contribution.
- In-context improvement loop: Optional evolution hints let stateless adapters adapt across rounds and battles, but only when adapters read them; no weights are updated.The wrapper derives hints from verdicts and trace analysis, while adaptation remains bounded by the hint channel and its budget.
4 Experiments
ACEA evaluates red-team attacks and blue-team defenses head to head across configurable defense frameworks, models, targets, and repeated adversarial rounds. The experiments use seeded secrets, explicit target-model recovery, and fixed-run settings to assess attack success while exposing important measurement caveats.
- Experimental design: ACEA tests whether its scoring changes as expected by running attack success rate across defense frameworks, defense models, and targets.Table 3 covers one attacker over thirty rounds per defense-target pair.
- Experimental design: The bundled red team composes attacks from independent technique, context, and pressure layers, then synthesizes prompts combining them.The stated objective is to find combinations unseen by the defender.
- Experimental design: ACEA evaluates a three-stage classifier defense and a minimal one-call filter, each backed by large and small models, yielding four defenses.The default pipeline uses input lexical and intent checks plus output confidential-data scanning; the minimal filter provides a near-inactive reference.
- Experimental design: Each defense runs against four targets seeded with synthetic canonical secrets, allowing suspected leaks to be checked against known secret inventories.Target 4 is the smallest target model, while the platform did not record target model identities at runtime.
- Experimental settings: The experiments use thirty rounds per defense-target pair with in-context hints enabled, attacker memory reset between pairs, and mostly single-run columns.Target 1 is repeated three times; Target 3 is incomplete, and independently published tools are evaluated in separate runs without claiming hint-loop activation.
- Results and caveats: Attack success varies substantially by target and run: the weakest defense reaches 21/30 and 19/30 on a permissive target, while the same pair ranges from 0.000 to 0.700 across targets.Repeated runs of one weakest-defense pair against the same target returned 0.200, 0.000, and 0.200.
5 Real-Time Visualization and Reporting
ACEA makes red/blue battles observable during execution and interpretable afterward. Its live interface exposes round-level activity, while the persistent trace produces localized, actionable reports.
- Examples: Figure 2 depicts round 14 of 26 with the attacker ahead 10 to 3 in a minimal-defense matchup against a permissive target.The figure provides a mid-battle example of the interface described above.
- Live visualization: ACEA renders each battle as a live game-style contest showing attacks, defense decisions, judge verdicts, and running ASR and DR indicators.The event stream supports monitoring, replay, and inspection of completed rounds.
- Live visualization: The interface separates the red team, shared target, and blue team into lanes while displaying the round, score, verdict, report status, and event log.Speech bubbles report each role’s own step rather than invented captions.
- Actionable reporting: The end-of-battle report localizes failures by payload, turn, and verdict, verifies evidence when possible, and separates attack potency from defense effectiveness.It also distinguishes the defense’s contribution from refusals produced by the target itself.
- Actionable reporting: Technique-level analysis explains why attacks succeeded or failed and supplies concrete suggestions for improving a participant.The report includes per-round turning points, strategic assessments, and red-team or blue-team priorities.
- Examples: Figure 3 presents excerpts from the corresponding report, where the attacker succeeded in 23 of 26 rounds.The report example connects the live battle view to post-battle diagnosis.
6 Discussion, Limitations, and Ethics
ACEA is intended for authorized defensive testing with synthetic secrets and operator-controlled targets, but its measurements have important scope and reliability limits. These include judge dependence, target dependence, a small comparison, coarse saturation detection, and an optional improvement loop.
- Ethics and assumptions: ACEA is designed for defensive evaluation using a self-contained chatbot with synthetic canonical secrets and a target under the operator’s control.The authors position the platform for authorized safety testing because it is dual-use.
- Measurement limits: LLM-judge scores inherit model bias and inconsistency, while open-ended harm remains an estimate rather than an exact measurement.Declared evidence markers provide verifiable ground truth only when the engagement author can specify them.
- Target dependence: The target’s alignment determines whether attacks can succeed, so defense comparisons across different targets do not isolate the defenses.Against a fully hardened target, target refusals can dominate defense-rate measurements and flatten improvement curves.
- Empirical scope: The empirical comparison covers four defenses, four targets, and thirty rounds per setting, with repetitions on only one target.The framework’s effect exceeds the serving model’s effect in these measurements, so results primarily describe frameworks.
- Empirical scope: ACEA neither detects nor displays saturation, leaving early-versus-late comparison as its only coarse signal for whether improvement has plateaued.The operator decides when to stop a run.
- Improvement and reporting limits: The in-context improvement loop is optional and helps only when an adapter reads evolution_hints; its Layer 3 meta-optimization requires several sessions per variant.The reported attack-success definition also differs from an earlier definition, making those figures incomparable.
7 Conclusion
ACEA combines pluggable red and blue adapters, verifiable per-round scoring, live observability, and advisory cross-round hints in a shared-target arena. The authors frame it as a head-to-head alternative to isolated, post-hoc red-teaming, while noting limits around single-target contests and model-judged harm.
- Conclusion: ACEA connects pluggable red-team and blue-team adapters around a shared target LLM under an LLM judge.Adapters interact through a protocol that requires them to expose only their behavior.
- Conclusion: Each round combines verifiable leakage ground truth with separate measures of attack potency and defense effectiveness.This addresses the problem of treating blocked attacks as indistinguishable outcomes.
- Conclusion: The arena makes the contest observable as it unfolds and can feed round outcomes back as advisory hints for stateless adapters.Adapters can adapt across rounds without retaining state when they read the hints.
- Open issues: ACEA remains limited to one red team, one blue team, and one target, excluding multi-party dynamics and transfer across targets.Declared markers provide verifiable ground truth only for engagements whose authors can state them; open-ended harm still depends on model judgment.