ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?

1 INTRODUCTION

• Existing benchmarks provide less systematic coverage of exploitation effects that depend on interactions among protocol participants, especially without target implementation details.Such vulnerabilities can arise from inconsistent message interpretation, timing, state, or trust assumptions across components.Source: §1 · p. 2
• ProtoArena comprises 80 controlled cross-component exploitation environments spanning 11 protocol and application domains and 7 attack categories, accessed without target source code.Agents pursue observable security effects through scoped interfaces, with traffic, logs, and trajectories assessed by a hybrid verifier.Source: §1 · p. 2
• Claude Opus 4.8 reaches the highest aggregate success rates in reported runs: 48.8% at L1 and 12.5% under effect-directed black-box L3.Across five models, rates vary by protocol domain, attack category, and information level; DNS and TLS/PKI have its highest observed rates.Source: §1 · p. 2
• The evaluation separately studies attack-strength optimization using a disclosed DNSBomb mechanism.The contribution list describes three information levels that vary attack-context disclosure while retaining the objective and target instance.Source: §1 · p. 2

2 RELATED WORK

• Prior agentic work spans code inspection and exploitation, including CyberGym, ExploitGym, and reports of agents performing consequential multi-step security operations.These developments motivate reproducible evaluation with explicit access conditions and validation criteria.Source: §2 · p. 2
• Table 1 compares benchmark scope and task design, while its feature key defines black-box access and cross-component evaluation criteria.The key also describes mechanism withholding, runtime-state exploitation, working exploits, and anti-contamination.Source: Table 1 · p. 3, §2 · p. 3
• Benchmarks cover generated-code security, interactive CTFs, source-assisted proof-of-concept generation, deployed web exploitation, and verified exploit capabilities.Examples include CyberSecEval, InterCode-CTF, CyberGym, CVE-Bench, ExploitGym, and ExploitBench.Source: §2 · p. 3
• Existing benchmarks provide limited coverage of black-box exploitation of cross-component protocol vulnerabilities, which require security effects to depend on participant interactions.Source-assisted reproduction need not infer hidden component relationships, and network-facing access alone does not establish interaction-driven failures.Source: §2 · p. 3

3 PROTOARENA BENCHMARK

• ProtoArena contains 80 controlled environments across 11 protocol and application domains and 7 attack categories, primarily based on documented cross-component vulnerabilities.The required effect depends on interaction among distinct protocol participants.Source: §3 · p. 3

3.1 SCOPE AND EVALUATION UNIT

• A cross-component task requires the security effect to depend on interaction among participants, such as message interpretation, timing, shared state, or authentication assumptions.Multiple deployed services alone do not establish that the target effect arises from a security-relevant interaction.Source: §3.1 · p. 4

3.2 AGENT INPUTS AND INTERACTION

• Tasks provide scoped access instructions, an observable effect objective, and execution constraints, while information levels control disclosure of protocol identity, mechanism, and topology.Agents may probe endpoints, execute programs, and instantiate attacker-side participants only where explicitly permitted; they submit executable proofs of concept and supporting artifacts.Source: §3.2 · p. 4

3.3 SUCCESS CONDITIONS

• A run succeeds when independently captured traffic and runtime logs confirm the specified effect within the authorized scope and budget.Rules check events, time windows, thresholds, and constraints; LLM review interprets evidence against the requested effect.Source: §3.3 · pp. 4–5

3.4 INFORMATION LEVELS

• The three information levels progressively reduce guidance while keeping the instance, effect objective, verifier, and budgets fixed.L1 gives a documented procedure, L2 provides mechanism and topology context without steps, and L3 supplies only scoped access, budgets, and the effect objective.Source: §3.4 · p. 5

4 PROTOARENA CONSTRUCTION

• The construction process combines protocol implementations with task-specific configuration and supporting services, informed by documented attack chains and threat models.The section covers attack selection, benchmark coverage, and environment realization.Source: §4 · p. 5

4.1 SOURCES AND TASK SELECTION

• Task selection targets externally triggerable, observable cross-component effects, and experts validate vulnerability reproducibility using proofs of concept.Each environment receives one primary attack-category label for coverage analysis.Source: §4.1 · pp. 5–6
• The benchmark contains 80 environments across 11 protocol and application domains, each requiring an executable exploit whose effect depends on interaction among protocol participants.The seven primary attack categories include amplification/DoS, poisoning/deception, parsing discrepancies, authentication bypass, downgrade, disclosure, and integrity/injection.Source: §4.1 · p. 5

4.2 ENVIRONMENT REALIZATION AND ATTACKER CONTROL

• Environment realization separates target, attacker-controlled, and evaluation components, and limits agents to explicitly authorized roles and operations.Victim filesystems, management interfaces, hidden configurations, and verifier artifacts remain outside agent control; runs start clean and operate within traffic and resource limits.Source: §4.2 · p. 6
• The verifier combines rule-based checks with LLM review of captured traffic and runtime logs to determine whether the target effect occurred.Evidence is collected outside the agent-writable workspace, with provenance identifiers; attacker-controlled content cannot alter evaluation policy.Source: §4.2 · p. 6
• In the isolated DNS response-pulse deployment, Claude Opus 4.8 exceeds a 170,000× peak-to-average byte-rate ratio when given mechanism and topology.The measurement does not quantify effects on an Internet-facing service.Source: §4.2 · p. 6

5 EXPERIMENTAL EVALUATION

• The evaluation compares five models on confirmed success across three information levels, protocol domains, and attack categories, plus a separate DNSBomb optimization experiment.L1 supplies detailed attack guidance, L2 supplies mechanism and topology without a procedure, and L3 provides scoped black-box access and an effect-level objective.Source: §5 · pp. 6–7

5.1 PERFORMANCE ACROSS INFORMATION LEVELS

• 26.3%–48.8% at L1, 21.3%–35.0% at L2, and 8.8%–12.5% at L3; Claude Opus 4.8 leads each level but drops 36.3 percentage points from L1 to L3.Even with detailed guidance, the best-performing model fails on more than half of the environments.Source: Table 3 · p. 7, §5.1 · p. 7

5.2 MODEL PERFORMANCE ACROSS PROTOCOL DOMAINS

• Domain rankings vary: Claude leads DNS and TLS/PKI at L1 and L2, while L3 leaders differ across domains.At L3, the highest rates are 14.3% for HTTP/Web and 12.5% for email; CDN has no successes.Source: Table 4 · p. 7, §5.2 · p. 8

5.3 MODEL PERFORMANCE ACROSS ATTACK CATEGORIES

• Claude Opus 4.8 has the highest L1 rate, including ties, in all seven attack categories, but no model leads every category at L3.At L3, DeepSeek, GLM, and Kimi tie on smuggling/parsing, while Qwen leads forgery/authentication bypass.Source: Table 5 · p. 8, §5.3 · p. 8

5.4 ATTACK-STRENGTH OPTIMIZATION UNDER A KNOWN MECHANISM

• 171,267.65 was Claude Opus 4.8’s highest peak-to-average ratio; DeepSeek V4 Pro recorded 5,951.7, and all five models passed the auxiliary criterion.DeepSeek recorded 60 qualifying intervals in a 60-second window, while Claude recorded six; the measures capture temporal coverage and peak response concentration, respectively.Source: Figure 3 · p. 8, §5.4 · p. 9, Figure 4 · p. 9

5.5 DNS CROSS-ZONE BENCHMARK: OUT-OF-ZONE CACHE POISONING

• GLM-5.3’s L3 run returned forged out-of-zone address 6.6.6.6 for two names, replacing the baseline 93.184.216.34.The Authority-section NS test triggered a cache-acceptance event, but its subsequent query failed to parse, leaving the served effect unconfirmed.Source: Figure 5 · p. 9, §5.5 · p. 9

• Agents construct executable exploits and achieve verifiable cross-component security effects in some controlled environments.Source: §6 · p. 9
• Reliable exploitation from limited external observations remains challenging, with performance varying across protocol domains and attack categories.Source: §6 · p. 9
• The evaluation covers 80 environments with one retained outcome per model–environment–information-level combination, while stochasticity and runtime timing may affect success rates.Repeated runs and broader coverage are needed to establish stable rankings and generalization.Source: §6 · p. 9
• Controlled deployments cannot fully capture real-world configurations, workloads, and defenses; the DNSBomb optimization experiment uses only a disclosed mechanism in a controlled deployment.Source: §6 · p. 10
• Future work will expand coverage, analyze agent trajectories, and repeat evaluations under varied runtime conditions to assess reliability and timing and configuration sensitivity.The planned analyses target failures in probing, reasoning, implementation, and execution.Source: §6 · p. 10

• ProtoArena is a benchmark for evaluating LLM agents' ability to exploit cross-component network protocol vulnerabilities under black-box access.Source: §7 · p. 10
• Current models construct executable exploits and achieve verifiable security effects in some tasks, but success declines as guidance is reduced.Source: §7 · p. 10
• Success varies across protocol domains and attack categories, and reliable exploitation from limited external observations remains challenging.The benchmark provides a foundation for evaluating and advancing protocol-security reasoning and exploitation capabilities for cybersecurity assessment.Source: §7 · p. 10

• Timely discovery, validation, and remediation of cross-component vulnerabilities are important because their impact may extend across services and applications.Source: ETHICAL CONSIDERATIONS · p. 10
• The 80 environments are primarily informed by publicly documented vulnerabilities or attack mechanisms and are not intended to reproduce or disclose previously unknown zero-days.Source: ETHICAL CONSIDERATIONS · p. 10
• Experiments assess security effects only in controlled environments and do not test production systems or third-party services.Source: ETHICAL CONSIDERATIONS · p. 10
• Attack traffic, payloads, and security effects remain confined to benchmark-managed containers and local endpoints, with external communication limited to model API requests.Network access, traffic volume, and resource usage are restricted; experiment processes, containers, and temporary state are removed afterward.Source: ETHICAL CONSIDERATIONS · p. 10

• Section 3 specifies task interfaces, information levels, access boundaries, and success criteria, while Section 5 reports models and outcomes.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendices reproduce the DNSBomb prompts and specify byte accounting, temporal aggregation, task criteria, and evidence adjudication with judge-agreement information.Source: REPRODUCIBILITY STATEMENT · p. 10
• The agent runtime uses OpenHands v1 SDK 1.34.0 with Python 3.12, and service-state files, judge source code, launch logs, and Docker introspection are unavailable to the agent.Source: REPRODUCIBILITY STATEMENT · p. 10
• Outbound communication passes through a local proxy that permits only model API domains.Model API communication is the sole permitted external-network exception to local task interaction.Source: REPRODUCIBILITY STATEMENT · p. 11

AI ASSISTANCE STATEMENT

• LLMs assisted with language polishing and proofreading.The authors manually reviewed and verified all AI-assisted content, citations, and factual claims against original sources.Source: AI ASSISTANCE STATEMENT · p. 11
• LLMs broadened the authors’ related-work search to reduce the chance of overlooking relevant studies.Source: AI ASSISTANCE STATEMENT · p. 11
• The authors take full responsibility for the final content.Source: AI ASSISTANCE STATEMENT · p. 11

A VERIFICATION AND ANALYSIS RECORDS

• Confirmed successes require the intended effect, mechanism-specific conditions, independent corroboration, and no disqualifying rule violation; agent claims alone do not suffice.Runs retain interaction traces, PoCs, harness measurements, service logs, and case identifiers; affected attempts are rerun before results enter the matrix.Source: A VERIFICATION AND ANALYSIS RECORDS · p. 13

B DNSBOMB PROMPT EXAMPLES AT THREE INFORMATION LEVELS

• The DNSBomb prompts retain objectives, thresholds, budgets, and task wording while varying how much attack context is disclosed.L1 supplies attack family, mechanism, topology, and steps; L2 omits the paper name and steps; L3 gives the effect and thresholds without protocol or mechanism context.Source: B DNSBOMB PROMPT EXAMPLES AT THREE… · p. 14

B.3 L3: EFFECT-DIRECTED BLACK-BOX EXPLORATION

• L3 withholds protocol, attack mechanism, and topology while specifying at least 20 distinct pulses at 2000x send rate within 60 seconds.Source: B DNSBOMB PROMPT EXAMPLES AT THREE… · p. 14, §B.3 · p. 15

B.1 L1: GUIDED REPRODUCTION

• L1 guides agents to delay authoritative replies, accumulate queries, and release them in repeated pulses, targeting at least 20 qualifying pulses at 2000x send rate within 60 seconds.The prompt limits interactions to 20 attempts and confines activity to the container.Source: §B.1 · p. 14

B.2 L2: MECHANISM-INFORMED EXPLOITATION

• L2 supplies the resolver-forwarding mechanism, topology, and controllable authoritative server, then requests at least 20 pulses at 2000x send rate within 60 seconds.Source: §B.2 · p. 15

C DNSBOMB ATTACK-STRENGTH OPTIMIZATION PROMPT

• The separate DNSBomb optimization prompt maximizes a continuous response-to-query outcome by accumulating and releasing replies, minimizing query input, and exploiting attack-span averaging.The scoring denominator is floored at 10 bytes per second.Source: C DNSBOMB ATTACK-STRENGTH OPTIMIZATION… · p. 16

D DNSBOMB OPTIMIZATION MEASUREMENT PROTOCOL

• Byte totals use logged wire-format query and client-facing response lengths, with downstream responses measured at the resolver-to-client outlet.The peak metric uses the largest fixed epoch-aligned one-second response bin, not a sliding window; a burst split across bins can have a lower measured peak.Source: D DNSBOMB OPTIMIZATION MEASUREMENT… · p. 17, Table 6 · p. 17
• The peak ratio divides that bin’s response rate by max(query-input rate, 10 B/s), while qualification uses the unfloored rate and counts consecutive seconds separately.Authoritative response bytes are excluded from the denominator, so the ratio is not total downstream traffic relative to all attacker-generated bytes.Source: D DNSBOMB OPTIMIZATION MEASUREMENT… · p. 17

E HYBRID EVIDENCE ADJUDICATION

• Hybrid adjudication applies deterministic rejection checks before LLM review, which requires evidence of both the task mechanism and its observable effect with independent corroboration.Agent plans and claimed scores express intent but do not independently establish success.Source: E HYBRID EVIDENCE ADJUDICATION · p. 18
• The two LLM judges’ final verdicts agreed with expert verification, with no observed differences attributable to judge selection across verified cases.Source: E HYBRID EVIDENCE ADJUDICATION · p. 18