Source-linked AI summary
SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system
Yuhan Meng, Shaofei Li, Jionghao Huang, Jiandong Jin, Puyi Wang, Hanlin Jiang, Anis Yusof, Peng Jiang, Zhenkai Liang, Yao Guo, Ding Li
TL;DR
Cybersecurity research lacks an integrated way to address autonomous attack and predominantly human-intensive defense. SysEvolve creates a safe co-evolutionary loop, with evaluation showing zero-loss collection at 2.1% overhead, attack success improvements over 25%, and 10–1000× greater precision than prior systems.
Problem
Cybersecurity research lacks an integrated approach to the asymmetry between increasingly autonomous attacks and predominantly human-intensive defense.
Method
SysEvolve co-designs realistic cyber ranges, safe attack generation, and interpretable real-time defense so adversarial outputs drive mutual capability evolution.
Results
Across its three components, SysEvolve achieves zero-loss collection at 2.1% overhead, improves attack success by over 25%, and delivers 10–1000× greater precision than prior systems.
Takeaways & Limitations
The evaluation supports autonomous, safe attack-defense co-evolution as a unified system-level approach spanning deployment, attack, and defense capability evolution.
Takeaways & Limitations
The high-configuration experiment is associated with greater execution cost and weaker progression, but its design does not isolate decoys as the sole cause.
Abstract
from arXiv · showhide
The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous execution while defense remains predominantly human-intensive. Despite substantial prior work across cyber ranges, AI-driven attack, and AI-driven defense, this asymmetry persists. We trace it to a deeper root cause, that evolution itself has stalled on both sides at three layers. To overcome this, we propose co-evolution as the integrating insight, where attack and defense AI agents autonomously and safely drive each other's evolution through adversarial confrontation. Based on this insight, we present \sysevolve, comprising three co-designed components, \sysfield, \sysspear, and \sysarmor. \sysfield constructs realistic multi-host ranges. \sysspear generates efficient, safe attack schemes. \sysarmor performs real-time, interpretable defense. Together they form a self-driven adversarial loop restoring evolution at all three layers. In evaluation, \sysfield achieves zero-loss collection at 2.1\% overhead and orchestrates 257 CVEs into 1,148 ranges, \sysspear improves attack success by over 25\% over baseline LLMs, and \sysarmor achieves 10--1000$\times$ greater precision than prior systems and detects real APT attacks in production at Huawei and Sangfor. Our evaluation also reveals three findings about LLM agent capabilities. First, multi-step composition and larger topologies expose agent capability gaps hidden by single-step evaluations. Second, the bottleneck lies after initial access in post-compromise state utilization. Third, LLM agents are susceptible to environmental interference. When decoy endpoints are deployed in the range, agent timeouts triple and downstream completion disappears despite the success rates of initial accesses are unchanged.
1 Introduction
LLM-driven cyberattacks are advancing toward autonomous multi-stage execution while defense remains asymmetric, reflecting stalled evolution in deployment, capability, and adversarial layers. SysEvolve addresses this gap through a safe, self-driven co-evolution loop linking SYSFIELD, SYSSPEAR, and SYSARMOR.
- Motivation: Attack has progressed to autonomous multi-stage execution, including zero-day exploitation, large-scale penetration testing, and breaches of real-world protected systems, while defense remains comparatively human-intensive.The supplied introduction frames this imbalance as a persistent cybersecurity asymmetry.
- Problem: Evolution has stalled across deployment, capability, and adversarial layers, leaving both sides without reliable infrastructure, trustworthy assessment, or sustained cross-side pressure.The paper identifies stalled evolution as the deeper root cause rather than merely a coordination failure.
- Co-evolution paradigm: Co-evolution restores these layers by having attack agents generate schemes that train defense detection, while defense feedback guides attack refinement and escalation.This forms a closed-loop mutual-driving process rather than independent attack and defense improvement.
- System contribution: SysEvolve combines SYSFIELD for realistic multi-host ranges and assessment, SYSSPEAR for safe verified attack generation, and SYSARMOR for real-time interpretable defense.Together, the three components form a self-driven adversarial co-evolution loop.
- Evaluation: 2.1% system overhead accompanies SYSFIELD’s zero-dropout log collection, while the system orchestrates over 257 CVEs into 1,148 distinct ranges and SYSSPEAR improves attack success by over 25%.The evaluation also reports that zero commercial LLMs escaped the range or bypassed its verification mechanism.
- LLM agent capability findings: LLM agents struggle with multi-step compositions and larger topologies, chiefly after initial access in post-compromise state utilization, and remain susceptible to environmental interference.Decoy endpoints triple agent timeouts and eliminate downstream completion without changing initial-access success rates.
2 Background & Related Work
Cybersecurity-agent benchmarks span distinct workflow segments and success endpoints, from knowledge and vulnerability research to exploitation and multi-host cyber ranges. Interpretable evaluation also depends on harness and environment controls, external verification, and separating offensive progression from defensive observation.
- Benchmark scope: Benchmarks form four complementary research lines, each delegating a different workflow portion and using a distinct endpoint rather than defining a difficulty ranking.The focused comparisons emphasize multi-host, multi-stage execution and continuity rather than aggregate benchmark quality.
- Benchmark scope: CyberSecEval, Cybench, and InterCode range from knowledge and hazardous-capability items to executable feedback and flag-based completion.These benchmarks differ in whether agents receive executable feedback and whether success is established through flags.
- Vulnerability research and exploitation: 1,507 historical vulnerabilities from 188 open-source projects: CyberGym verifies generated PoCs across pre- and post-patch versions, while SEC-Bench Pro requires rediscovery without original proofs or patches.ExploitBench and ExploitGym instead verify progressively finer exploit capabilities, including arbitrary read/write, program-counter control, and unauthorized code execution.
- Cyber ranges: Cyber ranges emphasize externally verified initial access and subsequent network operations, including chained vulnerability, configuration, credential, and lateral-movement steps across simulated or enterprise-like multi-host environments.CVE-Bench probes real web applications remotely, while AgentCyberRange separates web exploitation from post-exploitation and provides common execution and verification infrastructure.
- Evaluation methodology: Offensive oracles establish triggers, flags, or target states, whereas security signals establish observed behavior; neither alone establishes both successful compromise and runtime defense observation.This boundary motivates evaluating offensive progression and defensive observation as distinct claims.
- Evaluation methodology: Without environment qualification, factor isolation, and external verification, infrastructure failures, harness effects, tools, execution policy, or agent self-report can distort benchmark outcomes.The execution environment also creates security and measurement boundaries through egress, credentials, isolation, monitoring, reset behavior, and service availability.
3 Overview of SYSEVOLVE
SYSEVOLVE is an AI-native autonomous attack-defense co-evolutionary system built from three co-designed components. Its bidirectional adversarial feedback loop turns attack and defense outcomes into continual training and refinement signals, restoring evolution across deployment, capability, and cross-side pressure.
- System components: SYSEVOLVE comprises SYSFIELD for realistic multi-host cyber ranges and capability assessment, SYSSPEAR for verified attack-scheme generation, and SYSARMOR for real-time interpretable threat detection.SYSFIELD serves as the deployment substrate for both attackers and defenders.
- Attack-to-defense feedback: SYSSPEAR’s attacks stress-test SYSARMOR, while successful detections reinforce coverage and missed detections drive model fine-tuning.This converts red-team exercise outcomes from one-time reports into continual training signals.
- Defense-to-attack feedback: SYSARMOR feeds detection results back to SYSSPEAR, whose Strategy Board uses action outcomes and detection confidence to guide attack refinement.High-confidence detections indicate approaches that are easily visible to the defender.
- Three-layer co-evolution: The adversarial loop restores evolution across all three layers by combining SYSFIELD’s deployment substrate, self-evolving attack and defense capabilities, and cross-side pressure.Each side improves in response to the other’s evolving capability rather than in isolation.
4 SYSFIELD
SYSFIELD is a trustworthy, process-grounded substrate for evaluating autonomous cyber agents through loss-resilient evidence collection, reproducible cyber-range construction, and verified multi-stage attack orchestration. Its mechanisms validate topology, attack paths, audit completeness, and containment rather than trusting deployment success or final flags alone.
- System overview: SYSFIELD integrates loss-resilient log collection, CRADLE-guided range construction, and context-aware weakness orchestration with multi-layer verification.These planes provide execution evidence, reproducible deployment, and process-grounded evaluation of multi-stage attacks.
- Cyber-range construction: CRADLE uses a declarative language separating stable environment topology from ordered, timed, dependency-aware setup and experiment events.The environment section specifies networks, endpoints, systems, software, and configurations; the events section specifies actions and dependencies.
- Attack-path orchestration: Context-aware weakness orchestration validates capability and runtime dependencies to construct executable multi-weakness attack paths.Deploying multiple vulnerable services alone does not ensure that later stages can use capabilities obtained earlier.
- Trusted evaluation: A trusted evaluation boundary and component isolation prevent agents from gaining reward through range escape, external resources, verifier attacks, or private benchmark state.These controls address Hack Rewarding in executable attack environments.
- Verification: Dedicated mechanisms establish topology validity, vulnerability-chain validity, log completeness, and agent capability instead of inferring them from deployment success, metadata, collector status, or final flags.This shifts SYSFIELD from a passive benchmark harness to a trustworthy security-evaluation substrate.
5 SYSSPEAR · 5.1 Design Overview
SYSSPEAR autonomously generates efficient, knowledge-enhanced, verified-safe multi-stage attack schemes to restore attack capability evolution. Its design centers on the Strategy Board and three co-designed subsystems for generation, knowledge-enhanced execution, and static verification.
- 5.1 Design Overview: SYSSPEAR aims to autonomously generate multi-stage attack schemes that are efficient, knowledge-enhanced, correct, and safe.The design addresses attack capability evolution within the co-evolution loop.
- 5.1 Design Overview: Realistic campaign search spaces are large, with many branches revealed as dead ends only after execution.This makes autonomous attack-scheme generation difficult.
- 5.1 Design Overview: The Strategy Board is an inference-memory abstraction that aligns and evolves knowledge across SYSSPEAR’s subsystems.It forms the center of the design shown in Figure 6.
- 5.1 Design Overview: The attack scheme generator uses task slicing to mediate collaboration among isolated solvers.It produces attack-scheme candidates around the Strategy Board.
- 5.1 Design Overview: SecFlow codifies over 200 expert skills from more than a decade of security operations into a dependency-structured library.It also orchestrates heterogeneous agent runtimes through a uniform protocol.
- 5.1 Design Overview: The checking layer uses SMT solving to verify dependency satisfiability and checks software safety through fine-grained symbolic analysis.Static checking yields a verified attack scheme as SYSSPEAR’s output.
5.2 High-Efficiency Attack Scheme Generator
SYSSPEAR generates multi-stage attack schemes by treating an evolving typed evidence graph, the Strategy Board, as both the artifact and collaboration medium. It combines adaptive task slicing, bounded evidence projection, and isolated solver execution to preserve discoveries, coordinate searches, and recover from failures.
- The Strategy Board links typed resources through explicit relations, allowing isolated executors to contribute evidence to one evolving attack scheme.Unlike naive task-tree designs, evidence is connected into the whole scheme rather than passed only through parent-agent summaries.
- Assignments are durable work units with explicit instructions, checkable success criteria, focused Board entries, and reserved evidence questions.Adaptive slicing lets local searches advance, refute, or delimit routes, while complementary focus regions enable bounded parallelism.
- Context Projection builds token-bounded, connected evidence neighborhoods from each Assignment’s focus entries, instruction, success criterion, and Board relations.It prioritizes confirmed evidence and selected relation types while preserving the recorded entries’ semantic truth and status.
- Each Assignment runs in an isolated Solver with independent model, context, and runtime state, avoiding another unbounded shared reasoning transcript.Solvers share only a reusable-file workspace operationally; files acquire semantic weight only after being added to the Board.
- Versioned Board updates preserve attribution, support disagreement, reject stale modifications, and retain evidence when failed executors are replaced.Concurrent claims coexist through explicit supports or contradicts relations rather than being overwritten by the latest report.
5.3 SecFlow: Knowledge-Enhanced Agent Runtime
SecFlow is SYSSPEAR’s execution substrate, combining a structured library of expert attack skills, a uniform agent protocol, and filesystem-authoritative state to make agent execution knowledgeable, reliable, and isolated.
- Knowledge enhancement: SecFlow codifies over 200 expert skills from more than a decade of security operations across 30,000+ production machines into a structured library.Each skill provides domain-specific procedures that guide agent decisions beyond parametric memory.
- Knowledge enhancement: A dependency graph links skill preconditions to evidence-producing postconditions, enforcing prerequisite-aware ordering for multi-stage attack campaigns.The graph makes later skills applicable only after required evidence, such as privileged sessions or credentials, is produced.
- Capability orchestration: ACP decouples reasoning execution from task control by standardizing session lifecycle, tool invocation, permission requests, and subtask delegation.Resumable ACP sessions allow interrupted Solvers to recover by replaying filesystem transcripts rather than restarting from scratch.
- State and isolation: The filesystem is SecFlow’s authoritative state, enabling crash recovery, concurrency isolation, and real-time consistency across long-running tasks.Task descriptions, queues, event streams, execution records, and reports are persisted as produced, while snapshots and JSON Patches reconstruct task evolution.
- State and isolation: A layered process architecture separates concurrent Solvers across Gateway, WorkspaceHost, Task, and Agent Runtime boundaries, with workspace-scoped resource scheduling.This design prevents resource contention, file races, and cross-session state leakage during concurrent execution.
5.4 Static-Analysis-Verified Attack Scheme Correctness and Safety
SYSSPEAR statically verifies attack schemes for dependency correctness and software safety before execution. It combines ORAX’s SMT-based dependency checking with EUNOMIA’s semantic symbolic execution and SYMGX’s SGX-specific analysis.
- Dependency correctness: ORAX models skill preconditions and postconditions as SMT constraints to determine whether a valid execution order exists.Unsatisfiable schemes are rejected and returned for revision before execution.
- Dependency correctness: ORAX handles oracle-dependent constraints through feedback between the SMT solver and oracle handler.External tools, network services, and library functions have observable behavior but unknown implementations.
- Software safety: EUNOMIA uses AES to specify local search strategies for different code regions, avoiding interference from shared variable contexts.This semantic-enhanced symbolic execution checks referenced exploitation tools, scripts, and libraries for vulnerabilities before execution.
- Specialized-environment safety: SYMGX analyzes SGX targets for cross-boundary pointer vulnerabilities that conventional symbolic execution cannot detect.Its context-aware Global State Transition Graph models multi-entry arbitrary-order execution and maintains enclave state across transitions.
6 SYSARMOR
SYSARMOR is a provenance-driven endpoint detection and response system combining scalable streaming analysis, graph anomaly detection, TTP-level reasoning, and automated incident response. Its components support interpretable alerts and closed-loop recovery over evolving incidents.
- SYSARMOR architecture: SYSARMOR integrates rule-based detection, provenance-graph anomaly detection, and TTP-level reasoning over high-volume telemetry using a distributed Apache Flink processing engine.Its unified architecture supports scalable monitoring, real-time alert generation, and post-detection investigation.
- NODLINK: NODLINK formulates online APT detection as an Online Steiner Tree Problem, incrementally expanding suspicious terminal nodes into causal hopsets rather than analyzing isolated events.It maintains a compact evolving provenance graph and preserves attack-related causal structure during expansion.
- KNOWHOW: KNOWHOW converts suspicious provenance subgraphs into candidate attack lifecycles and precise, interpretable alerts that combine structural evidence with CTI-derived TTP explanations.The lifecycle representation helps analysts understand attack progression and prioritize response actions.
- STAIR: STAIR organizes incident response as a closed loop that tracks incident state, identifies recovery stages, generates actions, and incorporates execution feedback.Persistent state tracking and stage-aware planning address state loss and stage mismatch during long-horizon response.
7 Evaluation · 7.1 Evaluation of SYSFIELD
SYSFIELD provides a low-loss, low-overhead evaluation substrate whose multi-step, large-topology ranges expose capability gaps and whose stage-aligned measurements locate post-entry failures and environmental interference. The evaluation also separates offensive completion from defensive observability, showing that partial or failed trajectories can still generate meaningful evidence.
- 7.1.2 RQ1: Measurement Fidelity and Cost: SYSFIELD records 0% event loss with 2.1% overhead, while Sysdig and LTTng lose 98.5% and 60.0% of events and HARDLOG and OMNILOG impose 23.2% and 33.0% overhead.The event-loss and overhead comparisons use separate baselines and are not a single collector ranking.
- 7.1.3 RQ2: Range Difficulty and Scale: Multi-step composition lowers attack success by 18.1–21.6 percentage points across Kimi K3, GLM-5.2, DeepSeek-V4-Pro, and GPT-5.6 Luna.The reduction is consistent across all four evaluated models rather than driven by one outlier.
- 7.1.3 RQ2: Range Difficulty and Scale: Increasing topology from 7 to 50 nodes lowers attack success by 9.8–10.6 percentage points across the same four models.Model ordering remains unchanged, while the larger search space and longer dependencies add difficulty.
- 7.1.3 RQ2: Range Difficulty and Scale: SYSFIELD orchestrates more than 257 CVEs and 1,148 multi-vulnerability ranges, supporting more than 300 system nodes and more than 150 attack behaviors under a 256-GB memory limit.In the reported commercial-LLM trials, no model escaped the isolated range or bypassed external verification.
- 7.1.4 RQ3: Multi-stage Progression under Controlled Conditions: Kimi K3 reaches targets 1, 2, and 3 in 22, 18, and 16 of 50 cases, versus 19, 10, and 6 for DeepSeek-V4-Pro, widening the gap after initial access.The stage-level pattern indicates that sustained foothold use, continuation-target selection, and recovery are harder than initial vulnerability triggering.
- 7.1.4 RQ3: Multi-stage Progression under Controlled Conditions: SYSSPEAR reaches target 1 in 34/50 cases (68%), target 2 in 15/50 (30%), and target 3 in 7/50 (14%), with gains over DeepSeek-V4-Pro shrinking from 30 to 10 to 2 percentage points.Among its 34 initial footholds, only 7/34 (20.6%) reach the final target, shifting attention toward post-compromise state utilization.
- 7.1.4 RQ3: Multi-stage Progression under Controlled Conditions: Under high interference, target-1 completion remains 2/50, but downstream completion disappears; timeouts rise from 6/50 to 19/50 and mean runtime from 23.6 to 40.5 minutes.Every high-configuration run records a decoy interaction, but worker parallelism, execution order, and topology hints also differ, so decoys are not isolated as the sole cause.
- 7.1.5 RQ4: Offensive Progression and Defensive Evidence: Defensive observability is distinct from attack completion: Kimi K3 completes all targets in 16/50 cases but produces signals in 42/50, while DeepSeek-V4-Pro completes 6/50 but signals in 30/50.Across direct-run arms, 27 failed executions cover all expected rules and 7 successful executions have incomplete strict coverage; missing evidence is concentrated around execution-oriented network activity.
7.2 Evaluation of SYSSPEAR
SYSSPEAR’s coordinated multi-agent architecture improves penetration-testing success over single-agent execution, especially on hard multi-stage cases. Its task slicing, context projection, and runtime isolation work together to preserve actionable evidence while keeping Solver sessions bounded.
- Evaluation setup: SYSSPEAR is evaluated on 13 XBow web penetration-testing cases, comprising 9 Level-2 and 4 Level-3 tasks involving realistic vulnerabilities and automated verification.The comparison uses identical containerized tooling and a 10-minute timeout for SYSSPEAR and Claude Code.
- Architecture effectiveness: With GLM-5.1, SYSSPEAR solves 8/13 tasks versus 7/13 for Claude Code and completes the Level-3 HTTP-smuggling task XBEN-066-24 that the baseline does not.This same-model comparison provides architecture-aligned evidence, though it does not isolate individual mechanisms.
- Mechanism analysis: In XBEN-018 and XBEN-066, Strategy Board entries and projected context let later Assignments resume from confirmed facts and refuted hypotheses instead of repeating exploration.The cases cover an XSS tag-filter bypass and HTTP smuggling/SSRF, respectively.
- Architecture effectiveness: 76.9% (10/13 tasks) is the highest recorded success rate, achieved by skill-enabled SYSSPEAR, which solves 3 of 4 Level-3 cases while every other configuration solves at most one.DeepSeek-V4-Pro-Flash and Opus 4.6 Claude Code each solve 9/13 tasks (69.2%), but only one Level-3 case.
- Mechanism analysis: Task slicing defines Assignment boundaries, context projection transfers relevant evidence, and runtime isolation keeps each Solver session bounded yet informed.The mechanisms are co-designed rather than independent and jointly contribute to the observed gains.
7.3 Evaluation of SYSARMOR
SYSARMOR demonstrates real-world detection, enterprise-scale throughput, and moderate stable memory use. Its combined rule-based and provenance-driven approach identifies multi-stage APT activity, including evasive behaviors that conventional defenses may miss.
- Detection Effectiveness: SYSARMOR recognized six attack phases and preserved ATT&CK mappings with underlying command-line context, while the campaign exhibited masquerading and obfuscation.No Data Exfiltration or Impact stage was observed, suggesting interruption before later-stage objectives.
- Detection Effectiveness: 38,588 alerts across 11 rule categories covered 0.47% of the processed event stream during the operational assessment.The rule-based engine used an existing Falco rule set containing more than 36 signatures.
- Detection Effectiveness: The NODLINK + KNOWHOW pipeline detected a three-day APT campaign, mapping it to six attack stages in a provenance graph with 201 nodes and 1,982 edges.The campaign was first surfaced on its first day and the alert remained active until activity ceased on the third day.
- Scalability: 12,000 events per second were sustained with single-job configurations, while concurrent detection processed about 9,000 events per second.The reduction was primarily attributed to Kafka duplicating the event stream to two independent consumer groups, increasing broker egress load and I/O overhead.
- Scalability: More than 150 endpoints could be supported in real time by one server at an average event rate of 57 events per second per endpoint.The normalization stage was identified as the primary bottleneck.
- Resource Consumption: 3.8 GB and 4.8 GB were consumed after warm-up by the rule-based and machine-learning jobs, respectively, remaining stable throughout evaluation.The machine-learning job had a higher initial footprint while NODLINK, KNOWHOW, and graph state were loaded.
8 Conclusion
The paper proposes co-evolution to address the asymmetry between autonomous attack and static defense, realizing it through SYSEVOLVE’s self-driven adversarial loop of three co-designed components.
- 8 Conclusion: SYSEVOLVE integrates SYSFIELD, SYSSPEAR, and SYSARMOR into a self-driven adversarial co-evolution loop.SYSFIELD constructs realistic multi-host ranges through declarative topology-driven construction and provides complete attack-trace collection and trustworthy assessment.