PaperlayerView exact revision on arXiv

Source-linked AI summary

Examining Zero-Shot Vulnerability Repair with Large Language Models

Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, Brendan Dolan-Gavitt

arXiv:2112.02125v3cs.CRcs.AI

TL;DR

Security bugs remain costly to repair, motivating a test of whether general-purpose LLMs can generate vulnerability fixes without specialized training. The paper evaluates prompt designs and multiple commercial, open-source, and local models across synthetic, hand-crafted, and real-world scenarios. The models repaired 100% of the synthetic and hand-crafted scenarios collectively, but real-world evaluation exposed difficulties with functionally correct repairs.

  • Problem

    The paper asks whether off-the-shelf LLMs can generate safe, functional vulnerability fixes despite not being trained specifically on security repairs.

  • Method

    The authors evaluate multiple black-box and local LLMs using varied prompts and settings across synthetic, hand-crafted, and historical real-world security-bug scenarios.

  • Results

    LLMs collectively repaired 100% of the synthetic and hand-crafted scenarios, while real-world evaluation found repaired programs in 8-out-of-12 projects.

  • Takeaways & Limitations

    Carefully prompted general-purpose LLMs can generate security fixes in simple scenarios, but the approach is not yet ready to deliver real value in program repair frameworks.

  • Takeaways & Limitations

    Regression tests and crashing-input checks can show that observed failures no longer occur without proving that the underlying vulnerability has been eliminated.

Abstract

from arXiv · show

Human developers can produce code with cybersecurity bugs. Can emerging 'smart' code completion tools help repair those bugs? In this work, we examine the use of large language models (LLMs) for code (such as OpenAI's Codex and AI21's Jurassic J-1) for zero-shot vulnerability repair. We investigate challenges in the design of prompts that coax LLMs into generating repaired versions of insecure code. This is difficult due to the numerous ways to phrase key information - both semantically and syntactically - with natural languages. We perform a large scale study of five commercially available, black-box, "off-the-shelf" LLMs, as well as an open-source model and our own locally-trained model, on a mix of synthetic, hand-crafted, and real-world security bug scenarios. Our experiments demonstrate that while the approach has promise (the LLMs could collectively repair 100% of our synthetically generated and hand-crafted scenarios), a qualitative evaluation of the model's performance over a corpus of historical real-world examples highlights challenges in generating functionally correct code.

I. INTRODUCTION

This paper asks whether off-the-shelf LLMs can generate safe, functional repairs for security vulnerabilities without specialized training. It examines prompt design and model behavior across synthetic, hand-crafted, and real-world scenarios.

  • Research questions: The study investigates whether code-completion LLMs can generate safe and functional replacements for identified security bugs.The research questions also examine prompt context, real-world challenges, and repair reliability.
  • Research questions: The paper focuses on zero-shot repair by off-the-shelf LLMs rather than specialized models trained to predict human-written fixes.The goal is to assess whether general-purpose models can use broader source-file context to generate replacement code.
  • Approach: The evaluation spans synthetic, hand-crafted, and real-world buggy scenarios to characterize LLM vulnerability-repair performance.The study evaluates recent models across multiple scenario types rather than a single benchmark setting.
  • Approach: The authors compare prompt wording, contextual cues, and generation settings to encourage code that is both functional and secure.Prompt inputs can include comments, function names, and code, while temperature, sampling, length, and stop words affect generation.
  • Motivation: Security vulnerability repair is difficult because developers must detect bugs, localize and understand their causes, and create and test patches.Existing static tools and runtime sanitizers can help identify bugs, but patch creation remains costly manual work.

C. Studied Off-the-Shelf Large Language Models

The study evaluates commercial and local code LLMs whose outputs depend on prompt construction and decoding settings. It also describes the local models and the practical reasons for including one trained by the authors.

  • Studied models: The evaluated models include OpenAI Codex, AI21 Jurassic-1, polycoder, and the locally trained gpt2-csrc model.The off-the-shelf models were trained on large quantities of open-source code, while polycoder and gpt2-csrc can run locally.
  • Studied models: Local models provide unrestricted sampling and allow researchers to inspect and control training and generation procedures.The authors cite avoiding API rate limits as a practical benefit and experimental control as a scientific benefit.
  • Studied models: The authors trained gpt2-csrc on approximately 17GB of deduplicated C/C++ source code from popular Debian packages.They filtered non-source files, removed non-text files, deduplicated the data, and trained a BPE tokenizer.
  • Prompting: Prompt construction can steer LLMs toward different tasks, but no prompt guarantees that the model will follow the user’s intended task.Finite context windows also require selecting which source-file material enters the prompt.
  • Prompting: Prompt styles vary in comment verbosity, naming conventions, and contextual information, making prompt engineering an open empirical problem.The study explores comment types and prompt patterns to understand how these choices affect generated repairs.

IV. RQ1 / RQ2: SYNTHETIC EXPERIMENTATION

The study automates synthetic vulnerability generation and repair, evaluating LLM outputs with compilation, functional tests, and CodeQL across CWE-787 and CWE-89. Codex produced valid repairs for both scenarios, but repair rates varied substantially and parameter effects differed by vulnerability.

  • Experimental design: The framework derives repair prompts from CodeQL reports and original vulnerable programs, then evaluates merged LLM completions with functional and security tests.The study also includes a model-parameter sweep and investigates prompt patterns with increasing bug context.
  • Experimental design: The synthetic study targets CWE-787 out-of-bounds writes and CWE-89 SQL injection across lower- and higher-level programming contexts.These CWEs were selected because they are high-impact, directly identifiable from code, and span different abstraction levels.
  • Synthetic generation: 500 Codex suggestions were generated from two models across 25 temperature/top p combinations, producing 250 suggestions per model.Each program was compiled before functional and security correctness checks.
  • Synthetic generation: 95 unique CWE-787 and 22 unique CWE-89 functional-but-vulnerable programs became repair scenarios.The repair prompts incorporated the faulty code identified by CodeQL and contextualized the need for a fix.
  • Results: 2.2% of 22034 valid CWE-787 programs and 29.6% of 10796 valid CWE-89 programs were repaired.The corresponding counts were 491 and 3197 suggested patches that fixed the bugs.
  • Results: No single temperature/top p setting performed best across all scenarios and engines.Higher temperatures helped CWE-787 but hurt CWE-89, motivating an ensemble of temperatures with top p fixed at 1.00.

D. Prompt Engineering and Hand-Crafted Vulnerable Code

The study varies repair-prompt context and generation settings across hand-crafted software vulnerabilities. Results vary widely, but every scenario is repaired by at least one prompt–engine combination, with richer context often supporting functional correctness.

  • Experimental design: The experiments vary prompt templates, model settings, and software scenarios to study functional and secure zero-shot repair.Templates range from no information to extensive comments and hints, while scenarios cover selected high-impact, concrete, and varied CWEs.
  • Experimental design: 50 possible programs per scenario were generated for most black-box models across five temperatures with top p fixed at 1.00.AI21 models were sampled at half this amount, producing 25 possible programs per scenario.
  • Results: Every hand-crafted scenario was successfully repaired by at least one combination of template and engine, despite wide variation across prompts, scenarios, and LLMs.The authors emphasize that one correct completion is sufficient to fix a given bug.
  • Prompt effects: High-context templates generally produced better results because additional technical detail helped difficult cases pass both security and functional tests.Low-context prompts sometimes omitted the output or functionality requirements needed for valid repairs, as illustrated by CWE-79.
  • Model effects: Codex models consistently outperformed the other models, although every evaluated model repaired at least some programs.The proposed explanation is that Codex’s broader English-text training may improve understanding of verbose comments, but this remains a hypothesis.

E. Repairing Hardware CWEs

The hardware experiments evaluate LLM repair of two straightforward Verilog CWEs using functional and security tests. Unlike software repair, the models appeared to perform better with less prompt context, while Verilog generation was less proficient overall.

  • Scenario design: The study designs two straightforward Verilog vulnerability scenarios based on Hardware CWEs, using simple code whose security properties are relatively easy to test.The scenarios target CWE-1271 and CWE-1234 and contain 14 and 17 lines of Verilog, respectively.
  • Results: LLMs were less proficient at producing Verilog than C or Python, so the study swept temperatures and top p for Codex but not all other models.API restrictions prevented sweeping Jurassic models, while polycoder and gpt2-csrc lacked Verilog support.
  • Evaluation: Verilator checks both generated-code functionality and security, including reset locking for CWE-1271 and access control for CWE-1234.The framework also post-processes generated Verilog to add missing or remove redundant end and endmodule keywords.
  • Results: The hardware repair examples seemed to perform better with less prompt context than with more context.The authors suggest this may reflect the conceptual simplicity and straightforward specifications of the hardware components.

A. Overview

The real-world evaluation extends zero-shot vulnerability repair to larger and more realistic open-source projects. Unlike earlier short programs, these cases cannot provide the full source-file context within model token limits.

  • Overview: The study investigates real-world CVE scenarios to characterize LLM repair performance on larger and more realistic software projects.This evaluation introduces the challenge that complete source-file context often exceeds model input limits.

B. ExtractFix Dataset

The ExtractFix evaluation uses localized vulnerabilities from three open-source projects and supplies LLMs with reduced, targeted context. Developer patches localize the repair site, while sanitizers and regression tests assess security and functionality.

  • Dataset: The study collects 12 real-world vulnerabilities across three projects from the ExtractFix dataset.Cases require a proof-of-concept trigger, a single-file developer patch, and a reasonably comprehensive test suite.
  • Dataset preparation: Each vulnerability is prepared by identifying the fixing commit, its vulnerable parent, a triggering input, suitable sanitizers, and regression-test procedures.ASAN covers memory-safety issues, while UBSAN covers integer overflows, divide-by-zero, and related errors.
  • Bug localization: Developer-provided patches serve as an oracle for localizing each vulnerability and prompting the model at the original repair location.Root-cause identification and localization are treated as separate from the repair questions studied here.
  • Context constraints: Model token limits constrain both the code supplied and the code generated, making complete real-world files too large to present in full.This constraint did not affect the shorter handcrafted examples to the same extent.
  • Example: A generated repair for EF01 was semantically equivalent to the human patch after the repaired code was reassembled with the original file.The example uses a patched line generated by code-davinci-001 and matched surrounding safe code during consolidation.
  • Code reduction: The code-reduction pipeline includes file defines, the vulnerable function up to the bug, and a prompt template, then removes lines until the token budget fits.The prompt length is checked with each model’s tokenizer and an estimate of output tokens.
  • Suggestion consolidation: Suggestion consolidation seeks at least 30 characters of overlap with the original file after the vulnerable location to graft only the repaired code into the file.Prompts include at least two safe-code lines to encourage the model to continue with matching source text.

D. Findings

On real-world vulnerability scenarios, LLM-generated patches sometimes passed compilation, functional tests, and security checks, but qualitative review found many patches implausible or potentially buggy. OpenAI models generally performed best, while patch suitability varied across prompts and engines.

  • Real-world repairs: 8 / 12 selected projects were repaired under the study’s compilation, functional-test, and ASAN/UBSAN criteria.A project counted as repaired only when the replacement compiled, passed included functional tests, and no longer crashed on the triggering input.
  • Comparison and patch quality: The ensemble’s performance appeared comparable to ExtractFix, which repaired 10 / 12 projects.The authors cautioned that some LLM patches passed tests while appearing implausible or introducing inadequately tested bugs.
  • Experimental presentation: Across ExtractFix scenarios, the study evaluated 50 possible programs for most black-box LLM settings and 25 for AI21’s model.Figure 12 reports safe and functional or compiling programs across temperature settings from 0.00 to 1.00.
  • Failure characteristics: Real-world patches included cases where complex context, long edits, or tricky loop-bound semantics challenged the models.The discussion identifies EF02, EF18, and EF22 as examples involving substantial additions, long changes, or difficult semantics.
  • Evaluation caveat: Many successful results were successful only under the study’s narrow testing-based definition of repair.The evaluation used functional tests plus absence of an ASAN/UBSAN crash for the original problematic input.

VI. RQ4: DISCUSSION ON LLMS’ RELIABILITY

The study finds mixed reliability: LLMs convincingly repaired all synthetic and hand-crafted scenarios, yet qualitative review of real-world patches exposed serious correctness concerns. Their repairs are also constrained to one location within one file, which is not universal for security bugs.

  • Experimental results: 3,688 of 58,500 synthetic patches repaired programs across 117 scenarios, while 2,796 patches repaired 100% of seven hand-crafted scenarios.The study also generated 19,600 real-world patches, of which 982 ‘repaired’ programs in 8 of 12 projects.
  • Reliability: Real-world results were not sufficiently reliable to replace automatic program repair, despite strong synthetic and hand-crafted performance.Qualitative analysis found that many real-world patches appearing successful under tests may be unreasonable.
  • Scope boundary: LLM repairs are restricted to a single place within a single file, although security bugs are often more localized than other bugs.The authors explicitly note that this localization is not universal.

VII. STUDY LIMITATIONS

The study’s conclusions are bounded by potentially weak tests, dependence on security-analysis tools or triggering inputs, limited scenario coverage, and prompt-engineering constraints. No disclosure was required because the study used synthetic, hand-crafted, and historic vulnerabilities.

  • Potentially Inadequate Functional Tests: Regression tests may be weak proxies for correctness, so patches that appear repaired may still be incorrect.The authors suggest stronger evaluation approaches such as fuzzing.
  • Security Tool Test Dependency: Synthetic and hand-crafted evaluations depended on CodeQL’s existing queries for security analysis and repair assessment.All such scenarios could be analyzed by CodeQL, and repaired bugs were built from CodeQL-generated reports.
  • Security Tool Test Dependency: For real-world scenarios, passing the original ASAN/UBSAN crashing input shows that the failure case is repaired but does not prove vulnerability absence.Testing cannot prove that a bug has been completely eliminated.
  • Scenario Design: The scenarios do not cover all programming languages or security weaknesses.The authors attempted broad variety but acknowledge remaining coverage gaps.
  • Prompt Engineering: The study did not examine patches requiring multiple files, and choosing context under finite token limits remains open.The authors frame prompt design and context selection as continuing challenges.
  • Vulnerability Disclosure: No vulnerability disclosure was required because the study used synthetic, hand-crafted, and historic vulnerabilities.

VIII. RELATED PRIOR WORK

Prior work spans security-bug research, automated repair, code-completion tools, and vulnerability detection, while this study examines general-purpose LLMs without specialization for security-bug fixing.

  • Security patches are generally more localized and require fewer source-code modifications than non-security bug patches.
  • Automated repair research addresses programs violating specifications, with security-bug repair treated as one specialization.
  • NMT-based repair models learn human-designed patches from curated bug-fix pairs, including approaches reporting 9.22% on a test set.
  • Prior approaches often impose restrictions such as single-line fixes or language specificity.
  • This study probes larger general-purpose LLMs as zero-shot security-bug-fix recommenders without fine-tuning.

APPENDIX

The appendix presents synthetic-generation and real-world repair tables, figure captions, vulnerability examples, and representative repair patches.

  • Table VIII reports results for synthetic vulnerable-program generation.
  • Figures 14–16 document aggregate repair totals and Verilog repairs for CWE-1271 and CWE-1234, with highlighted code generated by LLMs.
  • Table IX reports real-world program-repair template performance using valid repair percentage, defined as functional and safe patches divided by valid compilable patches.
  • Figures 17–19 show the highest-confidence successful repair patches for EF15, EF01, and EF20.
  • The appendix includes code-diff examples for wrbmp.c and tools/tiffcrop.c.
Loading 2112.02125v3…