Source-linked AI summary
Augmenting Greybox Fuzzing with Generative AI
Jie Hu, Qian Zhang, Heng Yin
TL;DR
Structured inputs that fail format parsing cannot reach deeper program states, while existing mutation-based and generative approaches face challenges producing useful format-conforming inputs. CHATFUZZ adds a ChatGPT mutator to greybox fuzzing, improving average edge coverage by 12.77% over AFL++ across 12 programs.
Problem
For structured inputs, broken formats are rejected at parsing, while generating format-conforming inputs remains a major limitation for mutation-based and generative approaches.
Source: §1 · pp. 1–2, §2.2 · p. 3Method
CHATFUZZ augments an evolutionary greybox fuzzer with a Chat Mutator that prompts a ChatGPT endpoint for seed variations.
Source: §3 · p. 4, §7 · p. 17Results
12.77% higher average edge coverage than original AFL++ across 12 text-input programs; performance is particularly strong for programs with explicit syntax.
Source: §1 · p. 2Takeaways & Limitations
CHATFUZZ generated more syntactically valid seeds that explore deeper program space than AFL++ in the reported evaluation.
Source: §7 · p. 17CHATFUZZ supports text-based formats; for text inputs without explicit syntax rules, the chat mutator may contribute little or nothing.
Source: §5 · p. 15
Abstract
from arXiv · showhide
Real-world programs expecting structured inputs often has a format-parsing stage gating the deeper program space. Neither a mutation-based approach nor a generative approach can provide a solution that is effective and scalable. Large language models (LLM) pre-trained with an enormous amount of natural language corpus have proved to be effective for understanding the implicit format syntax and generating format-conforming inputs. In this paper, propose ChatFuzz, a greybox fuzzer augmented by generative AI. More specifically, we pick a seed in the fuzzer's seed pool and prompt ChatGPT generative models to variations, which are more likely to be format-conforming and thus of high quality. We conduct extensive experiments to explore the best practice for harvesting the power of generative LLM models. The experiment results show that our approach improves the edge coverage by 12.77\% over the SOTA greybox fuzzer (AFL++) on 12 target programs from three well-tested benchmarks. As for vulnerability detection, \sys is able to perform similar to or better than AFL++ for programs with explicit syntax rules but not for programs with non-trivial syntax.
1 Introduction
Structured inputs can be rejected at format parsing, limiting exploration of deeper program states. CHATFUZZ adds an LLM-based mutator to greybox fuzzing and reports higher average edge coverage than AFL++ across 12 programs.
1 Introduction
- Inputs with broken formats are rejected at parsing, restricting exploration of deeper program states.Fuzzing seeks broad execution-path coverage, but arbitrary mutations often fail to produce meaningful inputs for highly structured formats.Source: §1 · p. 1
- Mutation-based fuzzers often produce syntactically invalid inputs, while generative approaches can require manual grammars and lack runtime feedback.For an XML parser, AFL++ and SymSan produced valid-document ratios of 0.82% and 16.93%, respectively; TreeFuzz generated 96.3% syntactically conforming JavaScript programs.Source: §1 · p. 2
- LLMs can generate variants that retain the structure of provided examples, motivating their use for AI-based input mutations.CHATFUZZ prompts ChatGPT for variants of a seed and lets the greybox fuzzer evaluate them, retaining some as seeds for further exploration.Source: §1 · p. 2
- The chat mutator's seed quality depends on five choices: model endpoint, prompt design, response length, completion count, and sampling temperature.The paper reports experiments exploring configurations of these hyper-parameters.Source: §1 · p. 2
- 12.77% higher average edge coverage than original AFL++ across 12 text-input programs; CHATFUZZ contributes 12.77% of queue seeds and performs particularly well on explicit syntax.Its performance is stronger for programs with explicit syntax than for those with non-trivial syntax.Source: §1 · p. 2
2 Background and Related Work
The background describes LLMs and prompts, then uses XML to show why format-valid inputs matter and how ChatGPT-generated variants can retain valid syntax.
2.1 Large Language Models
- LLMs are autoregressive text models pretrained to predict next tokens; prompts pair natural-language instructions with input, and ChatGPT is based on GPT-3.5.The paper identifies gpt-3.5-turbo as an available GPT-3.5 model through the OpenAI API.Source: §2.1 · p. 3, §2.2 · p. 3
2.2 A Motivating Example
- For libxml2, format-valid XML is needed to reach post-parsing logic; gpt-3.5-turbo produced distinct valid variants, motivating ChatGPT as an additional mutator.XML rules include properly nested elements, a root element, and quoted attribute values; Table 1 presents the examples.Source: §2.2 · pp. 3–4
3 Design and Implementation
CHATFUZZ augments AFL++ with a Chat Mutator that prompts a ChatGPT endpoint for variations on a seed, then evaluates those variations and retains seeds that reach new code regions. The section explores model and prompt settings to balance seed quality, coverage, and generation efficiency.
3 Design and Implementation
- CHATFUZZ combines an AFL++ evolutionary greybox fuzzer with a Chat Mutator that generates seed variations from a selected pool seed for evaluation and possible retention.New-coverage seeds are periodically imported into the fuzzer's seed pool.Source: §3 · p. 4
3.1 Hyper-Parameter Selection
- Five settings are explored: model endpoint, prompt design, response length, completion-choice count, and sampling temperature.The hyper-parameter evaluation uses jq, php, mujs, and xml, measuring coverage growth, seed uniqueness, and syntactic validity.Source: §3.1 · p. 5
3.1.1 Model Endpoint Choice
- At present, the evaluation uses gpt-3.5-turbo as chat model CT ($0.0002 per 1,000 tokens) and text-curie-001 as completion model CP ($0.0020 per 1,000 tokens); CP is more efficient but more expensive than CT.Source: §3.1.1 · p. 5
3.1.2 Prompt Design
- Prompt configurations vary whether they include a sample input, the expected input format, or both.AI noINPUT tests omitting the sample, while AI noFORM tests omitting format information.Source: §3.1.2 · pp. 5–6
3.1.3 max tokens - Maximum number of tokens
- For CP, latency generally rises with max tokens; the study sets max tokens to 256, selecting the largest power of two below 5 seconds per query.For CT, the latency trend is not clear; the section uses a 5-second query-latency threshold for its choice.Source: Figure 2 · p. 6, §3.1.3 · pp. 6–7
3.1.4 n - Completion choices number
- Increasing n raises query cost but lowers average cost per generated testcase; subsequent experiments use n=20 to balance turnaround time and generation efficiency.Latency is evaluated across n values from 1 to 128 for both endpoints.Source: Figure 3 · p. 7, Figure 4 · p. 7, §3.1.4 · p. 7
3.1.5 temperature - Sampling Temperature
- Higher temperature increased CT's unique-seed ratio, while validity fell as temperature rose; CT's valid-seed ratio exceeded CP's for jq, mujs, and xml, but not php.For CP, uniqueness rose up to 1.25 then dropped for jq and xml; above that range, more very short, ill-formatted seeds appeared.Source: §3.1.5 · pp. 7–8, Figure 5 · p. 8, Figure 6 · p. 8
- Code coverage improvement was highest at temperature 1.50 for AI CT and 1.25 for AI CP; CP generated 15.91–99.31× more seeds than CT in two hours.Despite CP's higher seed throughput, CT outperformed CP in coverage improvement.Source: Figure 7 · p. 8, §3.1.5 · p. 9
4 Evaluation
The evaluation tests CHATFUZZ against AFL++ and format-agnostic and endpoint variants on 12 text-input programs from three benchmarks. CHATFUZZ finds more edges overall, while its seed contribution and bug-detection results vary with input syntax.
4.1 Evaluation Plan
- The evaluation covers 12 text-input programs selected from 45 programs across UniBench, Fuzzbench, and LAVA-M, and compares CHATFUZZ with AFL++ and format-agnostic and endpoint variants.Each fuzzing trial was assigned one dedicated core for fair comparison.Source: §4.1 · pp. 10–11
4.2 RQ1: Coverage Efficiency
- 6.11% more coverage than AFL++ was found on average by format-agnostic CHATFUZZ-F; CHATFUZZ-C and CHATFUZZ-CF found 7.61% and 6.89% more edges, respectively.The reported endpoint variants’ results were close to CHATFUZZ-F.Source: §4.2 · p. 12, Table 9 · p. 13
4.3 RQ2: Mutation Effectiveness
- 12.77% of CHATFUZZ's fuzzer-queue seeds were imported from its chat mutator on average, with over 30% for mujs, ossfuzz, and lua.Its contribution was low to none for programs with relatively non-trivial input formats.Source: Table 10 · p. 13, §4.3 · p. 13
4.2 RQ1: Coverage Efficiency
- 11.04% more edges across 12 target programs were found by CHATFUZZ than by AFL++.The evaluation also reports higher coverage efficiency than AFL++.Source: §4.2 · pp. 11–12, Figure 10 · p. 14
4.4 RQ3: Security Application
- In 8-hour trials, neither AFL++ nor CHATFUZZ triggered bugs in jq, mujs, ossfuzz, or md5sum; CHATFUZZ found more bugs than AFL++ for cflow.CHATFUZZ-CF found 12 bugs for md5sum.Source: Table 11 · p. 14, §4.4 · pp. 14–15
4.3 RQ2: Mutation Effectiveness
- The syntactically valid seed ratio was higher in CHATFUZZ than AFL++ for the reported targets, excluding curl, openssl x509, base64, and md5sum.For jq, the queues contained 393 valid seeds for CHATFUZZ and 392 for AFL++; 28.24% of CHATFUZZ's valid seeds were imported from the chat mutator.Source: §4.3 · pp. 13–14, Figure 11 · p. 15
5 Limitations and Discussions
CHATFUZZ’s scope is constrained by input-format support and the capabilities of its LLM service, while adaptive parameter tuning remains future work.
Unsupported Input Format
- CHATFUZZ supports text-based formatted inputs, but non-explicit syntax such as hashes can leave the mutator contributing little or nothing.Image, video, audio, PDF, WOFF, and compressed-file inputs are also outside its current support.Source: §5 · p. 15
LLM Model Limitations
- The OpenAI API’s latency, context length, and query rate affect performance; long-input-dependent states and stateful targets such as network protocols are not directly supported.The model cannot generate extremely long inputs or automatically remember past prompts.Source: §5 · p. 15
Automated Parameter Finetuning
- Dynamic parameter adjustment could improve coverage efficiency, but automated chat-mutator parameter tuning is left for future work.Low sampling temperature yields smaller differences between the sample input and new seed than high temperature.Source: §5 · p. 15
6 Related Work
Related work spans feedback-guided fuzzing, grammar-based input generation, and AI-based testing; CHATFUZZ distinguishes itself by using LLMs as mutation operators and generating valid inputs from seeds.
Grey-box Fuzzing
- Grey-box fuzzers mutate seeds using coverage feedback, while prior approaches also target rare branches, symbolic execution, and seed selection.AFLFast and EcoFuzz adjust seed-selection strategies to expand path coverage.Source: §6 · pp. 15–16
Grammar-Based Test Generation
- Grammar-based techniques generate legal structured inputs, but often require users to understand or manually specify grammars that may not transfer across formats.Prior examples include grammar- and token-based generation for structured inputs.Source: §6 · p. 16
- CHATFUZZ leverages LLMs to infer grammar from seed inputs and generate valid test cases, mitigating the need for manual grammar specification.The paper also identifies a gap in existing techniques’ use of AI or language models alongside software monitors.Source: §6 · p. 16
AI-Based Testing
- Prior AI-based testing applies learned models to branch behavior, test generation, or program synthesis, whereas CHATFUZZ uses LLMs as mutation operators in fuzzing.The approaches differ in how generated tests are incorporated into testing workflows.Source: §6 · p. 16
7 Conclusion
The paper presents CHATFUZZ for structured-input greybox fuzzing and reports higher coverage efficiency than AFL++, alongside more valid seeds that explore deeper program space.
7 Conclusion
- Across 12 target programs from three benchmarks, CHATFUZZ achieves higher coverage efficiency than the state-of-the-art greybox fuzzer AFL++.The evaluation targets real-world programs that take highly structured data as input.Source: §7 · p. 17
- CHATFUZZ uses generative large language models to generate well-formatted inputs for greybox fuzzing.The prototype was evaluated on 12 target programs across three benchmarks.Source: §7 · p. 17
- CHATFUZZ generates more syntactically valid seeds that explore deeper program space.Source: §7 · p. 17