PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing

Ana Nunez, Nafis Tanveer Islam, Sumit Kumar Jha, Peyman Najafirad

arXiv:2409.10737v2cs.SEcs.AI

TL;DR

LLM code generation often produces functionally correct but insecure code, while existing approaches may neglect dynamic runtime vulnerabilities. AutoSafeCoder addresses this gap with Coding, Static Analyzer, and Fuzzing Agents that iteratively generate, inspect, test, and repair code. On SecurityEval, it reduced vulnerabilities by 13% versus baseline LLMs while maintaining high functionality with a 3% decrease versus GPT-4o.

  • Problem

    LLM-generated code often contains security vulnerabilities despite functional correctness, and existing approaches may lack dynamic execution-based assessment.

    Source: §1 · pp. 1–2

  • Method

    AutoSafeCoder uses Coding, Static Analyzer, and Fuzzing Agents in an iterative feedback loop to generate, detect, and repair vulnerabilities.

    Source: §1 · p. 2, §3.1 · p. 4, §3.2 · p. 4

  • Results

    13% fewer vulnerabilities were reported on SecurityEval than for baseline GPT-4o, while functionality decreased by 3% compared with GPT-4o.

    Source: §4.3 · p. 5, §5 · p. 6

  • Takeaways & Limitations

    Combining static and dynamic security analysis can improve the security of LLM-generated code while retaining high functionality in the evaluated setting.

    Source: §4.3 · p. 5, §5 · p. 6

    Fuzzing performance was affected by incompatible dependencies and privilege management, and the authors are exploring a functional testing agent.

    Source: §4.3 · p. 6

Abstract

from arXiv · show

Recent advancements in automatic code generation using large language models (LLMs) have brought us closer to fully automated secure software development. However, existing approaches often rely on a single agent for code generation, which struggles to produce secure, vulnerability-free code. Traditional program synthesis with LLMs has primarily focused on functional correctness, often neglecting critical dynamic security implications that happen during runtime. To address these challenges, we propose AutoSafeCoder, a multi-agent framework that leverages LLM-driven agents for code generation, vulnerability analysis, and security enhancement through continuous collaboration. The framework consists of three agents: a Coding Agent responsible for code generation, a Static Analyzer Agent identifying vulnerabilities, and a Fuzzing Agent performing dynamic testing using a mutation-based fuzzing approach to detect runtime errors. Our contribution focuses on ensuring the safety of multi-agent code generation by integrating dynamic and static testing in an iterative process during code generation by LLM that improves security. Experiments using the SecurityEval dataset demonstrate a 13% reduction in code vulnerabilities compared to baseline LLMs, with no compromise in functionality.

1 Introduction

LLM-generated code often achieves functional correctness while failing security standards, creating a need for security-focused generation and repair. AutoSafeCoder addresses this gap with a collaborative system combining coding, static analysis, and fuzzing feedback.

  • LLM-generated code can contain security issues despite strong functional correctness, with vulnerabilities reported in around 40% of cases and below-minimal security in 16 of 21 ChatGPT cases.Source: §1 · pp. 1–2
  • Existing approaches often identify vulnerable code but overlook repair or dynamic execution-based assessment of runtime vulnerabilities.Source: §1 · p. 2
  • AutoSafeCoder uses Coding, Static Analyzer, and Fuzzing Agents to generate, detect, and repair vulnerabilities while preserving functionality.Source: §1 · p. 2
  • The system combines static-analysis and dynamic-fuzzing feedback in a continuous loop, using few-shot and in-context learning for vulnerability identification.Source: §1 · p. 2

2 Related Work

Prior multi-agent code-generation systems emphasize collaborative functional improvement, while security assessment requires both static detection and runtime testing. AutoSafeCoder’s framework combines these complementary analyses to address their distinct limitations.

2.1 Multi-Agent Systems for Code Generation

  • Existing multi-agent code-generation approaches assign specialized roles and iteratively refine outputs, but mainly evaluate functionality rather than security.Source: §2.1 · p. 2

2.2 Static and Dynamic Analysis

  • Static analysis detects source-level issues without execution but may miss runtime vulnerabilities, whereas dynamic analysis captures input- and condition-dependent issues with runtime overhead.Source: §2.2 · p. 2
  • AutoSafeCoder integrates Coding, Static Analyzer, and Fuzzing Agents so code auditing and mutation-based crash detection feed revisions back into generation.Source: Figure 1 · p. 3, §2.2 · p. 3

3 Methodology

AutoSafeCoder iteratively generates, statically audits, and dynamically fuzzes code, returning analyzer and crash feedback to the Coding Agent for repair. Its three specialized agents use GPT-4-driven generation, CWE-oriented analysis, and type-aware mutation testing.

3 Methodology

  • The methodology divides generation into iterative Coding, Static Analyzer, and Fuzzing phases, with validated code progressing from static review to runtime testing.Source: §3 · p. 3

3.1 Coding Agent

  • The Coding Agent uses GPT-4 and few-shot prompts to generate and repair code, incorporating specialized-agent feedback until issues are resolved or the iteration limit is reached.Source: §3.1 · p. 4

3.2 Static Analyzer Agent

  • The Static Analyzer Agent identifies vulnerabilities using MITRE CWE guidance, returns CWE codes and remediation suggestions, and communicates with the Coding Agent for up to four iterations.Source: §3.2 · p. 4

3.3 Fuzzing Agent

  • The Fuzzing Agent generates LLM-based seed inputs, applies type-aware mutations, executes the code, and sends crash-causing inputs and error contexts back for repair and retesting.Source: §3.3 · p. 4

4 Experiments

Experiments use SecurityEval for security and HumanEval for functionality, with GPT-4o as the baseline and sandboxed execution for fuzzing. AutoSafeCoder reduces vulnerabilities while retaining high functionality, though executable-code and dependency issues constrain fuzzing performance.

4.1 Dataset

  • SecurityEval contains 121 Python samples linked to 69 CWE vulnerabilities, while HumanEval supplies unit tests and pass@k measures functional accuracy.Source: §4.1 · p. 4

4.2 Experimental Setup

  • Experiments used GPT-4o, four static-analysis communication rounds, 150 fuzzing iterations, and a sandbox restricted by system calls and a 6-second time limit.Source: §4.2 · p. 5

4.3 Results and Discussions

  • 13% fewer vulnerabilities were detected for AutoSafeCoder than GPT-4o, while Bandit identified recurring CWE-94 vulnerabilities in 25 of 44 vulnerable samples.Source: §4.3 · p. 5, Table 1 · p. 5
  • The Static Analyzer Agent corrected vulnerabilities in 53% of cases within 1 to 4 iterations, while the Fuzzing Agent processed 65 samples and found crashes in 5.Source: §4.3 · p. 5, Table 2 · p. 5
  • AutoSafeCoder’s HumanEval functionality was 3% lower than GPT-4o, indicating retained high functionality alongside improved security.Source: §4.3 · p. 5, Table 1 · p. 5
  • Fuzzing performance was affected by incompatible dependencies and privilege management, motivating exploration of a functional testing agent and more executable generated code.Source: §4.3 · pp. 5–6

5 Conclusions

AutoSafeCoder combines static analysis and mutation-based fuzzing in a multi-agent framework for safer LLM code generation. On SecurityEval, it reduced vulnerabilities by 13% compared with baseline LLMs.

  • 13% fewer vulnerabilities were observed on SecurityEval compared with baseline LLMs.The reported reduction was achieved without compromising functionality.Source: §5 · p. 6

6 Social Impact Statement

AutoSafeCoder seeks to reduce vulnerabilities in LLM-generated code through integrated static and dynamic analysis. The authors also identify risks from incorrect outputs and malicious misuse.

  • AutoSafeCoder aims to reduce vulnerabilities that could compromise user data and privacy.Source: §6 · p. 6
  • Incorrect results could introduce new vulnerabilities, security breaches, or system failures.Source: §6 · p. 6
  • Malicious actors could misuse the technology to develop harmful software more efficiently.Source: §6 · p. 6
  • The authors recommend responsible implementation, ethical guidelines, and collaboration with cybersecurity experts.Source: §6 · p. 6

A Appendix / supplemental material

The appendix provides prompt templates for the agents’ code generation, vulnerability detection, fuzzing-input generation, and feedback exchanges.

  • Figure 2 provides the Coding Agent’s prompt template for code generation.Source: Figure 2 · p. 9
  • Figure 3 provides the Static Analyzer Agent’s prompt template for detecting vulnerabilities.Source: Figure 3 · p. 9
  • Figure 4 provides the prompt template for generating initial fuzzing inputs.Source: Figure 4 · p. 10
  • Figure 5 provides the Coding Agent’s prompt template for receiving Static Analyzer feedback.Source: Figure 5 · p. 10
  • Figure 6 provides the Coding Agent’s prompt template for receiving Fuzzing Agent feedback.Source: Figure 6 · p. 10

Related papers

Loading 2409.10737v2…