Source-linked AI summary
PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs
Rushabh Vipulkumar Patel, Dipo Dunsin, Mohammed Almaiah, Mohamed Chahine Ghanem
TL;DR
Premium-model token costs limit continuous automated penetration testing, especially for organisations already facing workforce constraints. PentestChain combines a deterministic exploit map, a cost-aware local and free-tier cascade, and MCP orchestration; across heterogeneous legacy engagements, it reports comparable effectiveness at zero measured paid-API cost, while acknowledging limits on hardened infrastructure and client-side MCP security.
Problem
Premium frontier-model costs make continuous automated penetration testing unaffordable for smaller organisations, while free-tier and local-model effectiveness remains undercharacterised.
Method
PentestChain combines a deterministic exploit map with a local-first, free-tier AI cascade, deterministic fallback, and eleven-tool MCP orchestration.
Results
Across ten engagements on five heterogeneous targets, PentestChain enriched 7–85 CVEs per target, confirmed exploitation on three platforms, produced 180+ web findings, and used zero paid-API spend.
Takeaways & Limitations
A deterministic backbone can make a 7B-parameter model sufficient for automated penetration testing while keeping dollar cost measurable and zero in the reported engagements.
Takeaways & Limitations
Four of five evaluation targets are deliberately vulnerable legacy systems, and client-side MCP remote-code execution cannot be fully closed by the engine alone.
Abstract
from arXiv · showhide
AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller organisations that need it most. This paper presents PentestChain, a ten-phase automated penetration testing framework that couples a curated, deterministic exploit map with a cost-aware AI cascade-a local Ollama model (qwen2.5-7b) first, then free-tier OpenRouter and Cerebras, with a rule-based fallback that always produces output-and exposes the full pipeline through a Model Context Protocol (MCP) server with eleven tools. We make three contributions. First, we treat US-dollar cost per engagement as a measured, first-class evaluation metric and show that a 7B-parameter local model, kept off the critical path by a deterministic backbone, sustains end-to-end operation at zero measured paid-API cost. Second, we analyse the attack surface that an MCP-exposed offensive engine introduces, grounding a four-position threat model in the 2025 MCP incident record (the CVE-2025-6514 remote-code-execution flaw in mcp-remote, the postmark-mcp supply-chain backdoor, and the tool-poisoning-rug-pull-line-jumping class), and contribute four mitigations. Third, we specify a reproducible, containerised evalua-tion protocol aligned with the standardised testbeds now expected at top-tier venues-AutoPenBench, a Cybench subset, and the PentestGPT 182-sub-task benchmark-with multi-trial statistics (more than 10 trials per configuration, pass-at-k, non-parametric significance tests and effect sizes) and direct, same testbed reproduction of the PentestGPT and PentestAgent baselines rather than citation of their published numbers. On the legacy targets measured to date, the framework detected 26 services, enriched 34 CVEs, produced
1 Introduction
PentestChain addresses the cost and accessibility limits of premium-model penetration testing with a deterministic, free-tier-first architecture. It combines bounded LLM roles, fallback protection, and MCP orchestration to target comparable effectiveness at zero measured cost.
- Motivation: Premium-model token costs make continuous penetration testing unaffordable for organisations with limited resources, while local and small-model evaluation remains undercharacterised.Published offensive-security prototypes evaluate local or small models in only about a quarter of cases.
- Research question: The paper asks whether free-tier and locally hosted LLMs, backed by deterministic fallback, can match premium-model effectiveness while reducing per-engagement cost to zero.The question treats dollar cost as an engagement-level evaluation target rather than merely a general affordability concern.
- Approach: PentestChain uses a ten-phase pipeline with a curated exploit map, nine bounded LLM roles, a local-first cascade, and an eleven-tool MCP server.The framework runs in a Kali Linux Docker container with PostgreSQL; the LLM is kept off the exploitation critical path.
- Contributions: The paper contributes a measured cost model, an MCP threat model with four mitigations, and a reproducible evaluation protocol using standardised testbeds and same-testbed baselines.The protocol includes AutoPenBench, a Cybench subset, and the PentestGPT benchmark, with multi-trial statistics.
2 Related Work
Prior work spans reinforcement-learning systems, LLM agents, standardised offensive-security benchmarks, and MCP-based pentest tools. PentestChain positions itself around deterministic execution, measured cost, and reproducible comparison rather than premium-model dependence.
- Reinforcement learning: Reinforcement-learning approaches can optimise attack paths in simulation but remain limited by training cost, brittleness, and weak transfer to exact real-world banners.PentestChain instead uses a curated exploit map with an AI fallback, avoiding training data and prioritising explainable selection.
- LLMs in offensive security: LLM offensive-security systems expose a knowledge–execution gap: smaller models may describe valid attacks yet fail to execute them reliably through tools.PentestChain assigns deterministic execution to the exploit map and limits the model to labelling and ranking tasks.
- Benchmarks: The field is converging on containerised benchmarks including Cybench, AutoPenBench, and the 182-sub-task PentestGPT benchmark for reproducible offensive-security evaluation.These instruments support task-level scoring across CTF, milestone, real-world CVE, and OWASP-oriented settings.
- Cost and accessibility: Most published frameworks prioritise effectiveness with premium models, leaving per-engagement cost and free-tier operation comparatively underexamined.The cited systems include GPT-4 or GPT-4.1 evaluations, while the literature identifies accessibility as a practical concern for under-resourced organisations.
- Positioning: Compared with MCP-based and local-capable systems, PentestChain places offensive logic in a deterministic map, reports measured zero cost, analyses its exposed MCP surface, and reproduces baselines on the same testbeds.It requires no training and retains a deterministic correctness floor beneath the model.
3 Threat Model
PentestChain treats MCP exposure as an attack surface involving the engine, host, and target, and models four adversary positions. It maps these risks to layered controls while retaining residual client-side exposure.
- Assets and adversaries: The threat model protects exploitation capability, decision integrity, and engagement-data confidentiality across four adversary positions.The positions are malicious client, hostile or poisoned server, client remote code execution, and man-in-the-middle or confused deputy.
- Attack surface: MCP tool poisoning can inject instructions through tool metadata, while CVE-2025-6514 demonstrates that authorization handling can enable arbitrary command execution on the client.The postmark-mcp backdoor illustrates supply-chain risk, and rug pulls can silently redefine an approved tool after first use.
- Specification-level considerations: OAuth 2.1 with PKCE and resource binding provide protocol protections, but incomplete deployment and vulnerable authorization discovery mean OAuth alone is insufficient.The cited client-side RCE makes the authorization flow itself part of the attack surface.
- Mitigations: PentestChain mitigates the modelled risks with localhost binding, start-time target allowlisting, exclusion of write and execute tools, and tamper-evident audit logging.The controls respectively reduce remote and replay exposure, prevent unauthorized targeting, limit poisoned-tool impact, and support post-hoc detection.
- Residual risk: The controls are defence-in-depth rather than a completeness proof: client-side A3 cannot be fully closed by the engine alone.The paper therefore recommends pinning client versions and treating tool descriptions as untrusted input.
4 System Architecture
PentestChain separates orchestration, execution, persistence, and AI assistance while using a deterministic exploit backbone and a cost-ordered cascade to keep every phase productive.
- Layered Architecture: The architecture uses Flask presentation, a run_full_scan state machine, structured subprocess wrappers, PostgreSQL persistence, and an eleven-tool FastMCP interface.The MCP layer exposes scan, service, CVE, exploitation, remediation, reporting, session, and stop operations.
- Cost-Aware AI Cascade: The cascade tries local Ollama, then free-tier OpenRouter and Cerebras, and finally a total deterministic fallback that guarantees a valid result.Providers are skipped after rate limits, timeouts, or invalid responses, with a 90-second dead-provider interval.
- Deterministic Exploit Map: The exploit map deterministically selects and configures Metasploit modules on service-port and banner matches, consulting validated AI suggestions only after misses.The map contains 41 entries and distinguishes exploits requiring LHOST from bind-shell or auxiliary modules.
- Cost-Aware AI Cascade: Because no single AI call is load-bearing, each phase still produces complete output through the rule-based path when cloud providers are exhausted.The deterministic branch makes the system operational rather than merely inexpensive during free-tier availability.
- Ten-Phase Pipeline: The ten-phase pipeline covers discovery, service and vulnerability analysis, exploitation, remediation, and report export, with AI confined to nine bounded roles.Its tools include Nmap, Metasploit, Nikto, sqlmap, and WPScan, while CVEs are enriched from four sources.
5 Implementation
The implementation packages PentestChain in Docker with PostgreSQL, preserves stateful exploitation sessions, and supports web-only, MCP-only, and combined deployment modes.
- Deployment Environment: Docker Compose runs the Kali application and PostgreSQL containers against an isolated Metasploitable 2 target, with pinned tool and eventlet versions.The environment includes Nmap, Metasploit, sqlmap, Nikto, WPScan, Python 3.11, and project requirements.
- Engineering Challenges: The implementation fixed JSON parsing, LHOST configuration, and cross-phase session loss through manual request parsing, three-field exploit entries, and an interactive Metasploit console.A 30-second deadline keeps sessions available for remediation verification.
- Deployment Modes: The system supports web-UI-only, MCP-only, and combined operation, with MCP-only running locally through Ollama without browser use or outbound traffic beyond the isolated target network.All three modes share one codebase without divergent execution paths.
- Instrumentation: PostgreSQL records scan state and each AI call’s provider, model, tokens, latency, and outcome, making fallback rate and dollar cost directly computable.The six persistence tables cover scans, services, CVEs, exploitation, remediation, and AI calls.
6 Experimental Design
The evaluation combines measured legacy-target runs with a planned reproducible protocol using standardised testbeds, reproduced baselines, multi-trial statistics, and explicit capability and cost metrics.
- Standardised Evaluation: The planned protocol covers AutoPenBench, a Cybench subset, and the 182-sub-task PentestGPT benchmark using pinned container images and task or milestone success measures.AutoPenBench is the primary head-to-head instrument because its milestone scoring matches the staged pipeline.
- Statistical Protocol: Each configuration is planned for at least 10 independent trials with pass@k for k ∈ {1, 3, 5}, confidence intervals, non-parametric tests, and effect sizes.Seeds, model strings, and full prompts are logged and released.
- Baselines and Ablations: The protocol reproduces PentestGPT and PentestAgent on identical testbeds and ablates orchestration, routing, model scale, and the deterministic map, but these comparisons are not yet measured.Table 2 distinguishes measured legacy metrics from planned standardised-testbed, baseline, and ablation results.
- Cost Measurement: The cost model defines engagement cost from recorded token usage and price coefficients, with the evaluated local and free tiers yielding C_eng = 0 structurally unless a paid tier is enabled.Replacing them with a premium tier would make cost scale with measured token volume, which averaged 181 ± 23K tokens per engagement.
- Measured Results: Across four runs, the framework detected 26 services and enriched them to 34 unique CVEs, while confirmed exploit sessions were retained for remediation verification.Successful exploit actions ranged from 5 to 23 per run depending on scan profile.
6.11 Planned Testbed and Ablation Results
PentestChain evaluates cross-target generalisation, cascade reliability, and planned benchmark and ablation protocols while reporting zero-cost operation and explicit security boundaries.
- Planned evaluation: The planned evaluation spans AutoPenBench, a Cybench subset, and the PentestGPT benchmark, pairing task success with cost and pass@k across systems.The protocol also specifies multi-trial statistics and same-testbed reproduction of PentestGPT and PentestAgent baselines.
- Planned ablations: The four-part ablation compares orchestration, provider routing, model scale, and map+AI versus AI-only execution, with map removal expected to reduce confirmed exploitation most.Stage-level completion covers information gathering, vulnerability discovery, and exploitation.
- Cross-platform results: Across five environments, the exploit map transferred without modification, while web scanning scaled to BWA and zero-cost operation held across ten runs.The evaluation includes legacy Linux, web, Windows, containerised PHP, and contemporary Debian targets.
- Modern-target result: The modern Debian spot check produced 163 web findings and remediation for all five remediable services at zero paid-API cost.The modern-target result extends detection and remediation beyond deliberately vulnerable legacy systems.
- Reliability and security: The architecture maintains effective AI-layer reliability because its deterministic branch always returns a valid result, while the MCP interface retains a client-side residual risk.The four mitigations address server-side positions but do not fully close client-side A3.
- Limits: Generalisability depends on substring matching against Nmap banners, limiting confidence for suppressed, falsified, or unmapped services and for long-horizon reasoning.The paper scopes capability claims as projected until the strengthened standardised evaluation is complete.
8 Ethics and Responsible Use
The paper frames PentestChain as dual-use offensive tooling and restricts its experiments to owned, isolated systems under institutional research-ethics requirements.
- Experimental scope: All reported experiments use systems owned by the researchers, including deliberately vulnerable virtual machines, containers, and a self-hosted Debian host on isolated networks.The work follows standard phased penetration-testing methodology and the institution’s research-ethics code.
9 Threats to Validity
Threats to validity concern stochastic model behaviour, legacy-target bias, incomplete MCP security closure, host-state variability, and the limited control provided by containerisation.
- Stochasticity: AI-response nondeterminism is addressed through multi-run averaging, prompt and response logging, fallback execution, fixed seeds, variance, effect sizes, and separation of refusals from infrastructure errors.The strengthened protocol uses at least 10 trials per configuration.
- External validity: Four of five evaluation targets are deliberately vulnerable legacy systems, so results should not be extrapolated to modern hardened production infrastructure.The modern Debian host and standardised benchmarks are intended to probe this gap.
- Threat-model completeness: The four MCP mitigations provide defence in depth rather than a security proof, and client-side RCE remains an unresolved A3 risk requiring pinned versions and untrusted tool descriptions.The residual risk lies outside what the engine can fully close.
10 Future Work
Future work strengthens evaluation and extends PentestChain while preserving its zero-cost, deterministic-fallback architecture.
- Evaluation: The immediate next step is the strengthened evaluation required before projected capability claims can be treated as settled.The roadmap preserves the zero-cost and deterministic-fallback design while expanding evidence.
- Benchmark expansion: A scaled VulHub study will test at least five CWE categories in a two-arm free-tier versus premium design.This directly targets broader capability comparison under controlled conditions.
- System extensions: Planned engineering extensions include signed, idempotent remediation with rollback and air-gapped deployment using a tuned local model.These extensions retain the framework’s cost-aware architecture.
11 Conclusion
PentestChain combines deterministic exploit selection, cost-aware AI orchestration, and MCP exposure to deliver comparable penetration-testing effectiveness at zero measured cost. Its architectural design and evaluation protocol are intended to make the accessibility claim reproducible and testable.
- Comparable penetration-testing effectiveness was achieved at zero measured cost across heterogeneous targets, with CVE enrichment, confirmed exploitation, and extensive web findings.The reported evaluation covered ten engagements against five targets, including confirmed exploitation on three platforms and more than 180 web-application findings on OWASP BWA.
- A deterministic exploit-map backbone and bounded, fallback-protected LLM roles make a 7B-parameter model sufficient without making the system model-centric.The architecture places offensive logic in the deterministic map while confining the LLM to nine bounded roles.
- The paper treats dollar cost per engagement as a measured first-class metric and specifies a standardised, statistically grounded protocol for testing the claim.The protocol is positioned for comparison on standardised testbeds and includes reproducible evaluation artifacts and statistical analysis.
- The paper is scheduled for ACM ASIACCS 2027 in Macau, and the authors state that scientific claims and experimental work were their own despite limited AI-assisted editing.
--script ftp-vsftpd-backdoor against the target as the check
The section documents representative results, reproducibility artifacts, ethical safeguards, and the dual-use boundary of PentestChain. It presents the system as a controlled orchestration of public offensive tools rather than a source of new exploits.
- Results: 26 services, 34 unique CVEs, 11.5 ready exploits, 12 confirmed exploits, and a 10.0/10 risk score were reported for Metasploitable 2 at zero measured paid-API cost.These are means from four runs, with a failed Heartbleed attempt remaining as a residual AI-suggester case.
- Reproducibility: The artifact release enables direct regeneration of measured results through pinned environments, prompts, seeds, instrumentation, and exported AI-call data.It includes the engine, container configuration, evaluation targets, and ai_calls tables underlying the cost model.
- Reporting: Reports are exported in HTML, PDF, DOCX, JSON, and CSV, linking headline risk to the chain from banner through CVE, session, and verified fix.
- Ethical Considerations: The framework introduces no new exploit and therefore presents incremental risk primarily through greater accessibility and scale of publicly documented offensive tools.The authors assess publication as net-positive while acknowledging risks to operators, unauthorised target owners, and the wider ecosystem.
- Ethical Considerations: Experiments used only owned vulnerable systems on isolated host-only networks, with no third-party or production targets, human subjects, or personal data.The work nevertheless applied institutional ethics and coordinated-disclosure practices.
- Ethical Considerations: Four safeguards ship enabled by default, including a start-time target allowlist and exclusion of file-write and remediation-execution tools from the MCP surface.The safeguards are described as defence-in-depth because the client-side A3 flaw cannot be fully closed by the engine.