Source-linked AI summary
LLM Agents can Autonomously Exploit One-day Vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta, Daniel Kang
TL;DR
Prior work had not established whether LLM agents could autonomously exploit real-world one-day vulnerabilities. This paper builds a benchmark and evaluates a tool-using ReAct agent, finding that GPT-4 succeeds on most vulnerabilities with CVE descriptions but performs far worse without them.
Problem
Prior autonomous-hacking studies focused on toy vulnerabilities, leaving the ability of LLM agents to exploit real-world deployments unresolved.
Method
The authors collect 15 reproducible real-world one-day vulnerabilities and evaluate a tool-using ReAct agent across language models and vulnerability scanners.
Results
GPT-4 achieves a 87% success rate with CVE descriptions, compared with 0% for every other tested model and scanner; without descriptions, GPT-4 achieves 7%.
Takeaways & Limitations
The findings indicate an emergent capability in GPT-4 and motivate careful consideration of LLM-agent integration in cybersecurity and widespread deployment.
Takeaways & Limitations
The agent omits subagents and a separate planning module, and the authors withhold prompts from the public for ethical reasons.
Abstract
from arXiv · showhide
LLMs have becoming increasingly powerful, both in their benign and malicious uses. With the increase in capabilities, researchers have been increasingly interested in their ability to exploit cybersecurity vulnerabilities. In particular, recent work has conducted preliminary studies on the ability of LLM agents to autonomously hack websites. However, these studies are limited to simple vulnerabilities. In this work, we show that LLM agents can autonomously exploit one-day vulnerabilities in real-world systems. To show this, we collected a dataset of 15 one-day vulnerabilities that include ones categorized as critical severity in the CVE description. When given the CVE description, GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit). Fortunately, our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities. Our findings raise questions around the widespread deployment of highly capable LLM agents.
1 Introduction
Prior autonomous-hacking studies focused on toy vulnerabilities, leaving real-world deployments understudied. This work evaluates one-day vulnerabilities and finds that GPT-4 can exploit most of them when given CVE descriptions.
- Prior work primarily examined LLM assistance for humans, offense-versus-defense speculation, or autonomous hacking of toy websites.
- The study asks whether LLM agents can autonomously hack real-world deployments.
- The benchmark contains 15 real-world one-day vulnerabilities drawn from reproducible CVEs and academic papers.
- GPT-4 achieves a 87% success rate, while GPT-3.5, 8 open-source models, and open-source scanners achieve a 0% success rate.
- Without the CVE description, GPT-4’s success rate drops to 7%, indicating greater ability to exploit vulnerabilities than to find them.
2 Background on Computer Security and LLM Agents
Computer security vulnerabilities can enable severe unwanted actions, while LLM agents increasingly use tools, planning, and documents to perform complex tasks. The paper extends autonomous-hacking research from toy exercises toward real-world one-day vulnerabilities.
- Computer programs can be misused to obtain root access, execute code remotely, or exfiltrate private data.
- Vulnerabilities are disclosed and often added to the CVE database after providers can patch them, while open-source cases can be reproduced in sandboxes.
- LLM agents can use tools, react to tool outputs, plan, create subagents, and read documents.
- GPT-4 strongly outperforms the other tested models in tool use.
- One-day vulnerabilities are disclosed but remain unpatched, leaving deployments exposed while attackers have the vulnerability description.
3 Benchmark of Real-World Vulnerabilities
The benchmark targets reproducible, real-world vulnerabilities rather than toy capture-the-flag tasks. It contains 15 cases spanning websites, containers, and Python packages, including high- and critical-severity examples.
- The benchmark begins with real vulnerabilities from CVEs and academic papers describing real systems.
- The authors focus on open-source software because many closed-source CVEs cannot be reproduced.
- Open-source vulnerabilities may still be irreproducible because of unspecified dependencies, broken containers, or underspecified CVE descriptions.
- After filtering, the dataset contains 14 CVE vulnerabilities plus ACIDRain, emulated on WooCommerce because of its complexity and severity.
- 8/15 vulnerabilities are categorized as high or critical severity, and 11/15 (73%) postdate GPT-4’s knowledge cutoff.
- The dataset therefore emphasizes real-world, high-severity vulnerabilities instead of toy capture-the-flag vulnerabilities.
4 Agent Description
The agent combines a base LLM, prompt, ReAct framework, and tool access to evaluate reproducible vulnerabilities. Only GPT-4 exploits vulnerabilities in the evaluation, while the implementation omits subagents and a separate planning module.
- The agent consists of a base LLM, prompt, agent framework, and tools.
- The vulnerability set includes descriptions of issues such as container escape, CSRF with arbitrary code execution, SQL injection, and cross-site scripting.
- Table 1 lists the considered vulnerabilities and descriptions, defining ACE as arbitrary code execution and RCE as remote code execution.
- Table 2 records each vulnerability’s CVE number, publication date, and CVE severity, including the WooCommerce ACIDRain emulation.
- Only GPT-4 is capable of exploiting vulnerabilities in the dataset; every other evaluated method fails.
- The agent uses the ReAct framework and provides tools for browsing, terminal access, web search, file editing, and code interpretation.
- The implementation uses 91 lines of code and omits subagents and a separate planning module.
5 LLM Agents can Autonmously Exploit One-Day Vulnerabilities
The evaluation tests LLM agents and vulnerability scanners on reproduced real-world one-day vulnerabilities, finding GPT-4 substantially more capable than all other tested methods. GPT-4’s performance depends strongly on receiving the CVE description, while the authors also examine cost and generalization beyond the knowledge cutoff.
- 5.2 End-to-end Hacking: 87%: GPT-4’s success rate, compared with 0% for every other tested model and scanner.The comparison includes GPT-3.5, eight open-source models, ZAP, and Metasploit.
- 5.2 End-to-end Hacking: GPT-4 fails on Iris XSS and Hertzbeat RCE, with JavaScript navigation and a Chinese description identified as contributing factors.The agent struggles to access elements made available through JavaScript, while the English prompt may be confused by Hertzbeat’s Chinese description.
- 5.2 End-to-end Hacking: 82%: GPT-4’s success rate on vulnerabilities released after its knowledge cutoff date.This corresponds to 9 of 11 vulnerabilities.
- 5.3 Removing CVE Descriptions: 7%: GPT-4’s success rate after removing the CVE description, down from 87%.The description-assisted task tests exploitation, whereas the modified task also requires finding the vulnerability.
- 5.3 Removing CVE Descriptions: 33.3%: GPT-4 identified the correct vulnerability, but exploited only one of the vulnerabilities it detected.For vulnerabilities past the knowledge cutoff, identification success was 55.6%.
- 5.4 Cost Analysis: $8.80 per exploit: the estimated GPT-4 cost at a 40% overall success rate, versus $25 for estimated human labor.The average run cost $3.52, and the authors estimate LLM agents are 2.8× cheaper than human labor.
6 Understanding Agent Capabilities
The qualitative analysis examines why GPT-4 can exploit diverse vulnerabilities and why performance degrades without descriptions. Successful attacks often require many coordinated tool actions, code writing, and broader exploration or planning.
- 6 Understanding Agent Capabilities: 48.6 steps: Wordpress XSS-2 requires many actions on average, including extensive navigation through a complex layout.One successful attack used 100 steps, including 70 navigation steps; some pages exceeded the 512 kB tool-response limit.
- 6 Understanding Agent Capabilities: CSRF + ACE requires both a CSRF attack and code execution, but without the description the agent typically commits to one vulnerability type without backtracking.The authors connect this behavior to the absence of subagents in their implementation.
- 6 Understanding Agent Capabilities: ACIDRain spans website navigation, checkout-field extraction, Python race-condition code, and terminal execution.The exploit requires operating several tools and writing code based on website actions.
- 6 Understanding Agent Capabilities: GPT-4 also exploits non-web vulnerabilities by writing code for the Astrophy Python-package RCE and container-management software.Both examples were published after GPT-4’s knowledge cutoff date.
- 6 Understanding Agent Capabilities: The authors suggest planning, subagents, and larger tool-response sizes could further improve the agent’s capabilities.This proposal follows qualitative analysis of the agent’s tool use and failures.
7 Related Work
Prior cybersecurity research has largely examined human-assisted use, societal implications, or toy environments rather than autonomous exploitation of real-world vulnerabilities. This work instead focuses on agents’ concrete ability to hack real-world one-day vulnerabilities.
- Prior studies mainly use LLMs to assist human operators in penetration testing or malware generation.
- Other cybersecurity-and-AI research examines societal implications rather than autonomous exploitation.
- Automatic vulnerability detection and exploitation is the cybersecurity subarea most closely related to this work.
- ZAP, Metasploit, and Burp Suite are common vulnerability scanners, but the open-source scanners tested cannot find the studied vulnerabilities.
- Research on securing LLM agents addresses attacks such as indirect prompt injection and removal of model protections, making it orthogonal to this work.
8 Conclusions
The paper concludes that LLM agents can autonomously exploit real-world one-day vulnerabilities, with GPT-4 succeeding when given CVE descriptions. The findings indicate both an emergent capability and a need for careful consideration of agent integration and deployment.
- GPT-4 with the CVE description is currently the only tested configuration capable of exploiting these real-world one-day vulnerabilities.
- The results suggest that uncovering a vulnerability is more difficult than exploiting it.
- The findings highlight the need to consider carefully how LLM agents are integrated into defensive measures and deployed widely.
9 Ethics Statement
The authors frame the ability to hack real-world systems as potentially harmful while arguing that studying it academically is important. They report using sandboxed environments and withholding prompts for ethical reasons.
- The demonstrated capabilities could be used for black-hat hacking, which the authors characterize as immoral and illegal.
- The experiments were conducted only in sandboxed environments to prevent harm.
- The authors withheld their prompts publicly after disclosing the findings to OpenAI, citing ethical reasons and prior practice.