Source-linked AI summary
Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing
Weizhe Wang, Yitong Zhang, Yao Zhang, Xiaoqiang Di, Zhigang Li, Bin Wu, Guangquan Xu
TL;DR
Long-horizon penetration testing challenges LLM agents to retain early facts and causal links across extended interactions. Intentest externalizes state in a fact-intent DAG and adds scheduling and intent retrieval mechanisms. On web CTF challenges, it improves success over VulnBot, while intent guidance reduces rounds on successful medium and hard tasks.
Problem
LLM penetration-testing agents can forget early facts and causal links during long interactions, undermining reasoning across extended tasks.
Source: §3 · p. 3Method
Intentest stores verified network facts and predecessor-constrained exploration intents in a persistent DAG, alongside task scheduling and intent retrieval layers.
Source: §1 · p. 2, §4.2 · p. 4, §4.1 · p. 4Results
Intentest achieves 88.2% overall success and 75.0% success on hard tasks, improving over VulnBot by approximately 44 and 50 percentage points, respectively.
Source: §8 · p. 16Takeaways & Limitations
Intent retrieval and prediction reduce average rounds on successful medium and hard tasks by about 33% and 48%, respectively, without changing the set of solvable tasks.
Source: §8 · p. 16The reported results support effectiveness within web-based CTF environments, while generalization to production enterprise intranets and untested vulnerability types or protocols remains to be validated.
Source: §7 · p. 15
Abstract
from arXiv · showhide
Large language model (LLM) based agents are increasingly applied to cybersecurity tasks such as vulnerability discovery and automated penetration testing. On long-horizon security tasks, however, such agents remain limited by context forgetting and intent drift: early critical facts and causal reasoning chains are lost over extended interactions, and the agent falls into aimless, repetitive exploration. This paper proposes Intentest, an intent-graph-guided automated penetration testing agent that externalizes long-horizon state from the LLM's context window onto a persistent fact-intent directed acyclic graph (DAG), thereby substantially reducing invalid transitions. We evaluate Intentest on automated penetration testing of web applications, a representative long-tail task in cybersecurity. In the DAG, verified network states are stored as immutable fact nodes, and exploration directions are constrained as intent edges bounded by predecessor facts. The system adopts a three-layer architecture, in which the fact-intent mapping layer maintains the global state, the task scheduling and allocation layer ensures execution stability through two-phase degradation recovery and multi-dimensional adaptive load balancing, and the intent retrieval and prediction layer provides tactical priors through a top-down five-stage filtering algorithm. On a benchmark of real CTF challenges covering more than ten vulnerability types across three difficulty levels, Intentest achieves an overall success rate of 88.2% and a success rate of 75.0% on hard tasks, improving over the baseline by approximately 44 and 50 percentage points. Ablation experiments further show that the intent retrieval and prediction reduce the average number of rounds on successful medium and hard tasks by about 33% and 48%, respectively, without changing the set of solvable tasks.
1. Introduction
Intentest addresses context forgetting and intent drift in long-horizon penetration testing by storing verified facts and constraining exploration in a fact-intent DAG. On real web CTF challenges, it improves success over the strongest baseline, with further contributions targeting execution stability and intent retrieval.
1. Introduction
- LLM penetration-testing agents can lose early critical facts and their causal connections during long interactions, leading to aimless exploration and ineffective loops.The task requires finding a vulnerable path among scattered facts gathered across network layers.Source: §1 · pp. 1–2
- Intentest stores verified network states as immutable fact nodes and constrains exploration directions as intent edges governed by predecessor facts.The graph is intended to limit divergent, aimless exploration.Source: §1 · p. 2
- The evaluation compares Intentest with baselines on real web CTF challenges spanning mainstream vulnerabilities and easy, medium, and hard difficulty levels.The study also disables intent retrieval and prediction in an ablation to quantify intent guidance.Source: §1 · p. 2
- A global fact-intent control blackboard supports asynchronous multi-agent collaboration, while two-phase degradation recovery and adaptive load balancing target execution failures and computational overload.Recovery intercepts tasks before termination and recovers partial facts from residual logs.Source: §1 · p. 2
- A top-down five-stage filtering algorithm uses structural constraints to retrieve tactical logic and reduce causal hallucinations caused by word overlap.The method is presented as an alternative to noisy retrieval-augmented generation.Source: §1 · p. 2
- Intentest improves overall success by approximately 44 percentage points and hard-task success by approximately 50 percentage points over VulnBot, the baseline with the highest overall success rate.The comparison uses real-world CTF challenges covering multiple mainstream web vulnerabilities and a clear difficulty gradient.Source: §1 · p. 2
2. Background and Related Work
Prior penetration-testing agents address long-horizon reasoning through structured task trees, multi-agent collaboration, classical planning, or feedback learning, but each approach has stated constraints. The paper motivates explicit architectural constraints for preserving facts across long time spans.
2. Background and Related Work
- The paper frames long-horizon state memory and logical reasoning as core challenges, arguing that implicit model memory remains unreliable when early micro-facts must be associated over long spans.It motivates explicit architectural constraints for complex enterprise-scale intranet penetration.Source: §2 · p. 2, §2.2 · p. 3
2.1. Evolution of automated penetration testing architectures
- PentestGPT maintains testing state in a structured natural-language task tree, but human-in-the-loop operation and context-window limits constrain its automatic use in fact-rich environments.The text says these limits can lead to forgetting early critical facts and losing strategic intent.Source: §2.1 · p. 2
- PentestAgent and VulnBot [23] use heterogeneous agents and shared test memory, yet fully automatic success declines on composite long-tail vulnerability scenarios.The passage also notes that word-overlap matching struggles with causal inheritance across many time steps and can introduce distracting historical features.Source: §2.1 · p. 3
- CheckMate [14] delegates global fact maintenance and logical constraints to classical planning, but predefined predicates and deterministic actions can deadlock on uncertain or non-standard vulnerabilities.Source: §2.1 · p. 3
2.2. Agent alignment and dynamic trajectory generation
- Pentest-R1 [28] uses two-stage reinforcement learning to acquire attack logic and then learn error correction in interactive CTF environments; Cyber-Zero [29] simulates troubleshooting sequences at low cost.The paper presents these as approaches to resilient reasoning and scalable intent-data generation.Source: §2.2 · p. 3
3. Motivation and Challenges
The paper identifies long-horizon fact retention, safe operational control, and noisy retrieval or static fine-tuning as bottlenecks for LLM penetration testing. Intentest responds with graph-based state, isolation and adaptive allocation, and structure-validated intent prediction.
3. Motivation and Challenges
- In partially observable penetration testing, agents must update beliefs through exploration, while LLMs struggle to preserve facts needed across long interaction sequences.A fact discovered dozens of steps earlier may be essential to a later decision, but context truncation and redundant logs can cause drift and ineffective operations.Source: §3 · p. 3
- Classical planning can deadlock on unfamiliar vulnerabilities, while unconstrained tool access exposes agents to prompt-injection-driven destructive actions.Source: §3 · p. 3
- Text-similarity retrieval can add redundant attack reports and causal-association hallucinations, while fine-tuning on purified reports can impair error correction when target conditions change.Source: §3 · p. 4
- Intentest replaces implicit memory with fact-intent DAG state, uses container-level isolation and adaptive allocation, and cross-validates intent predictions against graph structure.The stated aims include avoiding infinite loops and filtering redundant historical features and retrieval-induced causal-association hallucinations.Source: §3 · p. 4
4. Design of Intentest
Intentest organizes persistent fact-intent state, task execution, and tactical guidance in a three-layer architecture. Its verified-fact DAG constrains exploration, while scheduling and graph-based intent retrieval support execution and decision-making.
4.1. Overview
- Intentest separates persistent fact-intent state, execution scheduling, and intent retrieval and prediction across three architectural layers.The retrieval layer is triggered by successful exploration and new facts to recommend subsequent actions.Source: §4.1 · p. 4
4.2. Fact-Intent DAG Mapping
- Verified immutable facts connect to predecessor-constrained intent edges, externalizing belief state and restricting actions to causally supported paths.Concurrent agents share the DAG as a communication medium, and an intent can be jointly triggered by multiple predecessor facts.Source: §4.2 · pp. 4–5
4.3. Task Scheduling and Allocation Mechanism
- Bootstrap scans initially, Explore verifies single-step intents at highest regular priority, and Reason plans new tactical chains when no unclaimed Explore tasks remain.Reason tasks also require new facts or external expert prompts and use a single-project mutex lock.Source: §4.3 · p. 6
- Task dispatch filters workers by tool dependencies, capacity, and health before choosing the lowest-priority-weight and lightest-load eligible node.A pseudo-random jitter breaks ties among equally weighted nodes.Source: §4.3 · p. 6
- After an Explore or Bootstrap timeout, a 90-second degradation phase salvages partial facts from terminal logs instead of destroying the task container.Reason timeouts instead fail and release the lease without degradation.Source: §4.3 · p. 6
4.4. Intent-Graph Construction and Retrieval Prediction
- Intentest merges an offline historical intent graph with a live attack graph of verified facts and executed steps to recommend attack chains.The offline graph is built from threat intelligence and exercise documents; the live graph records explored topology independently of prior knowledge.Source: §4.4 · p. 7
- Five-stage retrieval screens similar tactics, checks predecessor-graph structure, falls back to Jaccard matching, aligns candidates with the authorized goal, then uses LLM reasoning if needed.The algorithm recommends candidate intents but does not authorize execution; each must pass fact-based DAG precondition validation.Source: §4.4 · pp. 7–8
5. Experimental Evaluation
Intentest is evaluated on real CTF web challenges across difficulty levels, against automated and semi-automated baselines. Results show the highest success rates across difficulty levels, lower successful-task exploration costs, and efficiency gains from intent guidance, with case studies illustrating its role in path selection and cross-step fact association.
5.1. Experimental Setup
- The evaluation uses real CTF challenges covering more than ten vulnerability types and three difficulty levels, comparing Intentest with automated, semi-automated, and general-purpose agents.All agents used DeepSeek V4 Pro; the public challenge writeups were excluded from the offline intent-graph corpus.Source: §5.1 · pp. 8–9
5.2. RQ1: Task Success Rate
- 88.2% overall success versus VulnBot’s 44.1%, and 75.0% on hard tasks versus 25.0%, give Intentest its largest advantage on hard challenges.The overall differences against baselines were statistically significant.Source: Table 1 · p. 9, §5.2 · p. 10
5.3. RQ2: Exploration E
- About 7 rounds per successful task for Intentest compares with about 20 for Pentest-R1, 12 for Claude Code, and 23.6 for VulnBot.Failed-task rounds averaged about 34 for Intentest, versus 40 for Pentest-R1 and about 32.8 for VulnBot.Source: §5.3 · pp. 10–11, Table 2 · p. 11
5.4. RQ3: Ablation Study
- Intent guidance leaves the solvable-task set unchanged but reduces successful-task rounds by about 33% on medium tasks and 48% on hard tasks.The overall average drops from 11 to 7 rounds per successful task.Source: §5.4 · pp. 11–12, Table 3 · p. 11
5.5. Case Studies
- The case studies compare Intentest–noIntent on the left with Intentest on the right, highlighting the successful attack path between the origin and goal against ineffective branches.They examine medium- and hard-difficulty SQL injection tasks.Source: §5.5 · p. 12, Figure 3 · p. 12, Figure 4 · p. 13
5.5.1. Medium-di
- On the medium SQL injection task, Intentest–noIntent spends rounds on WAF-detected time-based injection before succeeding with XOR and Boolean-based blind injection; intent guidance finds the bypass path sooner.Intent guidance uses tactical priors to shift exploration away from low-probability branches.Source: §5.5.1 · pp. 12–13, Figure 3 · p. 12
5.5.2. Hard-di
- On the hard SQL injection task, Intentest links source-code leakage to an open-source version, locates differentiating code, bypasses the filter, and retrieves the flag.Without intent guidance, the agent does not connect the early leakage discovery to later exploitation and scans pages without injection vulnerabilities.Source: §5.5.2 · p. 13, Figure 4 · p. 13
6. Discussion
The discussion attributes baseline failures largely to reasoning and state defects, and presents graph-based state as decisive for solvability while qualifying that evidence as corroborating rather than conclusive. It also distinguishes the graph’s role in sustaining long-horizon reasoning from the intent graph’s efficiency benefits and notes that offline priors depend on coverage and freshness.
6.1. Failure Analysis
- VulnBot’s rigid playbook repeated scans and injection attempts, while completed stages and already obtained flags were rechecked as tasks lengthened.The authors link these patterns to poor scenario adaptation and unreliable state memory, rather than budget exhaustion.Source: §6.1 · p. 14
- VulnBot’s failures reflect reasoning and state defects rather than budget exhaustion, including premature success judgments, host confusion, and missed flags in long outputs.Its failed-task average was about 32.8 rounds against a 40-round budget.Source: §6.1 · pp. 13–14
- Intentest’s failures concentrate on expert-level vulnerability chains requiring framework-internal knowledge, while one medium-task failure occurred at the final privilege-escalation step.In that medium task, the agent had located the flag file but could not read it after escalation failed.Source: §6.1 · p. 14
6.2. Evidence for the Necessity of Graph State
- 44.1% vs 88.2% overall success for VulnBot and Intentest, respectively, support graph-based state as decisive for solvability, but the comparison is corroborating rather than conclusive.VulnBot also differs in orchestration and tooling, leaving residual attribution uncertainty.Source: §6.2 · p. 14
6.3. Role and Applicability Boundary of the Intent Graph
- On attack surfaces without historical features, the graph supplies no exploitable prior; DAG constraints and heuristic reasoning still support exploration, while prior quality depends on graph freshness.The offline graph omits vulnerability classes, 0-day variants, and attack patterns absent from its historical corpus.Source: §6.3 · pp. 14–15
7. Threats to Validity
The evaluation is limited by the gap between CTF benchmarks and production environments, coarse outcome measures, and constraints on attribution, contamination control, scalability, and security validation.
7. Threats to Validity
- CTF results may not generalize to production networks, which have larger heterogeneous topologies, dynamic defenses, and stricter operational constraints.Source: §7 · p. 15
- Success rate and rounds are coarse proxies that omit vulnerability severity, access practicality, and agent time or API cost.Source: §7 · p. 15
- A controlled ablation of the fact-intent DAG is infeasible, and the VulnBot comparison is corroborating rather than conclusive because system orchestration and tooling differ.Source: §7 · p. 15
- Challenge and writeup blacklisting reduces leakage risk, but paraphrased or variant public writeups may still introduce test-related patterns.Source: §7 · pp. 15–16
- Although Top-k screening limits matching to local candidate subgraphs, end-to-end scalability on very large networks remains unmeasured.Source: §7 · p. 16
- Structural validation and container isolation mitigate adversarial risks, but robustness to graph manipulation remains to be evaluated.Source: §7 · p. 16
8. Conclusion and Future Work
Intentest uses a fact-intent DAG, task scheduling, and intent retrieval to address context forgetting and intent drift in long-tail penetration testing. On real web-based CTF challenges, it reports higher success than VulnBot, while future work includes broader environments and tasks.
8. Conclusion and Future Work
- 88.2% overall task success and 75.0% on hard tasks, improving over VulnBot by approximately 44 and 50 percentage points, respectively.Source: §8 · p. 16
- Intentest combines immutable fact nodes, predecessor-constrained intent edges, degradation recovery, adaptive load balancing, and five-stage intent retrieval.Source: §8 · p. 16
- The intent retrieval and prediction module reduces rounds on successful medium and hard tasks by about 33% and 48%, respectively, without changing solvable tasks.Source: §8 · p. 16
- Future work includes broader vulnerability coverage, dynamic graph updates, larger-scale non-CTF networks, and transfer to other long-horizon cybersecurity tasks.Source: §8 · p. 16