PaperlayerView exact revision on arXiv

Source-linked AI summary

CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generation

Jinjun Peng, Leyi Cui, Kele Huang, Junfeng Yang, Baishakhi Ray

arXiv:2501.08200v1cs.SEcs.CLcs.LG

TL;DR

LLM-generated code can be functionally correct yet vulnerable, while existing benchmarks inadequately assess both properties. CWEVAL combines clear security-critical tasks with outcome-driven tests for simultaneous functionality and security evaluation, revealing a substantial functional-versus-secure performance gap.

  • Problem

    Existing secure-code benchmarks inadequately evaluate functionality and security together, while vulnerabilities in functionally correct code can go unnoticed.

  • Method

    CWEVAL uses human-verified security-critical tasks, comprehensive specifications, reference implementations, and outcome-driven oracles for simultaneous functionality and security evaluation.

  • Results

    Across LLMs, requiring both functionality and security reduces the func@10 pass rate by around 30%, with a maximum 35.79% drop for Gemini 1.5 Flash.

  • Takeaways & Limitations

    CWEVAL reveals a significant gap between functional code and code that is both functional and secure, and identifies serious inaccuracies in prior evaluations.

Abstract

from arXiv · show

Large Language Models (LLMs) have significantly aided developers by generating or assisting in code writing, enhancing productivity across various tasks. While identifying incorrect code is often straightforward, detecting vulnerabilities in functionally correct code is more challenging, especially for developers with limited security knowledge, which poses considerable security risks of using LLM-generated code and underscores the need for robust evaluation benchmarks that assess both functional correctness and security. Current benchmarks like CyberSecEval and SecurityEval attempt to solve it but are hindered by unclear and impractical specifications, failing to assess both functionality and security accurately. To tackle these deficiencies, we introduce CWEval, a novel outcome-driven evaluation framework designed to enhance the evaluation of secure code generation by LLMs. This framework not only assesses code functionality but also its security simultaneously with high-quality task specifications and outcome-driven test oracles which provides high accuracy. Coupled with CWEval-bench, a multilingual, security-critical coding benchmark, CWEval provides a rigorous empirical security evaluation on LLM-generated code, overcoming previous benchmarks' shortcomings. Through our evaluations, CWEval reveals a notable portion of functional but insecure code produced by LLMs, and shows a serious inaccuracy of previous evaluations, ultimately contributing significantly to the field of secure code generation. We open-source our artifact at: https://github.com/Co1lin/CWEval .

I. INTRODUCTION

CWEVAL addresses weaknesses in existing secure-code benchmarks by combining clear, reproducible security-critical tasks with simultaneous functionality and security testing. Its benchmark suite enables empirical evaluation of LLM security and reveals that functional correctness does not ensure secure code.

  • Motivation: Existing secure-code benchmarks suffer from unclear specifications, infeasible functionality evaluation, and unstable static-analysis security judgments.CyberSecEval reproduces fewer than a third (562/1916) of vulnerable samples, while SecurityEval produces false negatives and false positives.
  • CWEVAL: CWEVAL uses human-verified tasks with comprehensive specifications, functionality and security test oracles, and secure and vulnerable reference implementations.These components support reproducibility, clarify expected behavior, and verify that vulnerabilities exist and can be mitigated without sacrificing functionality.
  • CWEVAL: CWEVAL’s outcome-driven oracles observe dynamic code properties, enabling more flexible and accurate security evaluation than traditional static analysis.The approach evaluates diverse implementations by their execution outcomes rather than by matching fixed insecure patterns.
  • CWEVAL-BENCH: CWEVAL-BENCH provides 119 security-critical coding tasks covering 31 CWEs across 5 programming languages and is designed for expansion.The authors open-source the complete benchmark suite and evaluation pipeline.
  • Study: Evaluations of four LLM families show security risks in generated code and inaccuracies in previous evaluations.The benchmark supports empirical study of both functional and security outcomes.

II. RELATED WORK

Prior code-generation benchmarks largely emphasize functional correctness, while secure-code benchmarks remain limited by unreliable specifications, testing, and security analysis. These shortcomings motivate more comprehensive and reproducible evaluation of secure code generation.

  • General code-generation benchmarks: HumanEval and similar code-generation benchmarks primarily assess functional correctness without considering security.Other general-purpose datasets extend evaluation along different dimensions but do not specifically address vulnerability detection.
  • Security benchmarks: SecurityEval covers 40 CWE-related categories, but poor specifications, missing functionality tests, and static analysis make its results unreliable.CyberSecEval likewise evaluates secure code generation but is identified among prior approaches with limitations.
  • Secure-generation methods: SVEN and SafeCoder target secure generation through fine-tuning, yet mainly retain previous benchmarks with minor changes.The unresolved limitations in benchmark quality, reproducibility, and accuracy leave a need for more comprehensive evaluation resources.

III. OUR METHOD: CWEVAL

CWEVAL models evaluation around a task specification, an implementation, and test oracles that independently assess functionality and security. Its outcome-driven design evaluates execution outcomes instead of relying on static vulnerability patterns.

  • Problem formulation: CWEVAL defines each task through a specification P, implementation I, and test oracles T that produce pass-or-fail functionality and security results.Functional correctness is represented as f+ or f−, while security is represented as s+ or s−.
  • Problem formulation: The framework rigorously assesses whether an LLM implementation is both functionally correct and secure for a security-critical task.This directly exposes the risk of accepting code that works functionally but remains vulnerable.
  • Framework components: CWEVAL coding tasks include specifications, secure and insecure reference implementations, and separate functionality and security oracles.The secure reference passes all oracles, whereas the functional but insecure reference fails the security oracles.

A. Coding Task Specifications

CWEVAL task specifications are designed to make security semantics explicit without revealing security cues, while keeping functional expectations sufficiently clear for meaningful evaluation. This separates security performance from basic task comprehension.

  • Security-semantics existence: CWEVAL requires each task specification to contain security-related semantics, such as file operations or explicitly user-provided values.Without these semantics, secure and insecure implementations cannot be meaningfully distinguished.
  • Security-semantics existence: Earlier task designs sometimes failed to clarify whether values originated from user input, leaving security semantics vague.This ambiguity is identified as a limitation of previous designs.
  • No security-awareness leakage: The specifications avoid explicit security hints, including safe or unsafe variable names and direct instructions to perform tasks safely.This simulates practical use in which users may have limited vulnerability awareness.
  • Expectation unambiguity: CWEVAL keeps functionality specifications clear and straightforward so functionality tests measure alignment tax rather than confusion about task requirements.Otherwise, failure to understand the task could be conflated with refusal caused by security alignment.

B. Test Oracles

CWEval evaluates functionality through input–output expectations and security through execution outcomes beyond return values. Unlike prior benchmarks, it uses dynamic outcome analysis rather than static analyzers to assess security.

  • Functionality evaluation checks whether generated implementations produce expected values for specified input–output pairs.
  • Security evaluation additionally examines execution time, memory-access validity, and data side effects or integrity.These properties target denial-of-service, memory-related, and SQL-injection vulnerabilities.
  • CWEval is outcome-driven and uses dynamic analysis to observe secure or insecure execution outcomes instead of relying on static analyzers.The framework avoids hardcoded static-analysis rules that can produce false positives and false negatives across diverse implementations.

C. Reference Implementations

CWEval uses reference implementations to validate task specifications and test oracles. Separate implementations demonstrate functional-but-insecure behavior and fully functional, secure behavior.

  • The (Iref)P f+,s− implementation passes all functionality tests but fails at least one security test.This demonstrates a reproducible security issue in an implementation that is functionally acceptable.
  • Reference implementations support cross-checking between coding-task specifications and test oracles.They also provide a basis for differential testing and test-oracle augmentation in future extensions.
  • The (Iref)P f+,s+ implementation passes both functionality and security tests, showing that a secure functional implementation exists.

IV. OUR BENCHMARK: CWEVAL-BENCH

CWEVAL-BENCH is a high-quality benchmark suite for secure code generation built around self-contained, security-focused tasks. Its language-independent design supports multilingual evaluation.

  • Coding tasks design: CWEVAL-BENCH tasks are guided by CWE documentation and designed to be self-contained for straightforward evaluation.
  • Coding tasks design: Security-semantics isolation minimizes dependencies on third-party libraries while preserving essential security-critical semantics.
  • Coding tasks design: Language-independent task specifications, test oracles, and reference solutions can be translated across supported languages for broader security evaluation.LLM translations are manually reviewed and refined to ensure validity.

V. EVALUATION

The evaluation benchmarks LLMs on functionality and joint functionality–security performance using adapted pass@k metrics. It examines performance gaps, model scale, and security prompting or fine-tuning.

  • The evaluation asks how large the gap is between functional correctness and security on CWEVAL-BENCH.
  • It investigates whether larger models generate more secure and functional code.
  • It studies whether security instructions and fine-tuning improve security while affecting functionality performance.
  • Metrics: func@k measures the likelihood that at least one of k generated implementations is functionally correct.It follows the pass@k definition and requires passing all functionality test oracles.
  • Metrics: func-sec@k measures the likelihood that at least one of k generated implementations is both functionally correct and secure.It requires passing both functionality and security test oracles and uses the unbiased pass@k estimator.
  • Metrics: c is the number of sampled implementations that are both functional and safe.

B. Setup

The study evaluates four popular LLMs, using random sampling across four temperatures and greedy decoding. It reports best-performing pass rates for multiple functional and security-functionality metrics.

  • Model selection: The evaluation studies GPT-4o mini, Claude 3.5 Haiku, Gemini 1.5 Flash, and Llama 3.1 70B Instruct.Additional model-size variants are evaluated for RQ.2.
  • Experimental settings: For RQ.1, each model produces 100 samples at temperatures 0.2, 0.4, 0.6, and 0.8.The study reports the best-performing ·@k for k = 1, 10, 50.
  • Experimental settings: Greedy-search decoding reports the pass rate of one deterministic sample as ·@k∗.Random sampling and greedy decoding are presented with separate metric labels.
  • Experimental settings: For RQ.2 and RQ.3, the evaluation uses random sampling with n = 100 at temperature 0.8 because of limited budget.

C. Results

CWEVAL-BENCH reveals a substantial gap between functional correctness and simultaneous functionality-security performance across LLMs. Larger models and simple security instructions generally improve secure-functionality scores, while SafeCoder can reduce performance under simultaneous evaluation.

  • RQ.1. Performance of Leading LLMs: From func@10 to func-sec@10, performance drops around 30% across all models, reaching 35.79% for Gemini 1.5 Flash.The gap indicates functional but insecure generations in security-critical coding scenarios.
  • RQ.2. Model-size comparison: Larger models almost always achieve higher func-sec@k scores than smaller models within the same family.For GPT-4o and Gemini 1.5, func@k differences are small while func-sec@k differences are larger.
  • RQ.3. Improving CWEVAL-BENCH performance: Security instruction prompting improves func-sec@k for almost all LLMs, with up to a 9.8% gain on func-sec@10 for Claude 3.5 Haiku.The prompting approach adds a simple instruction that code should be secure and contain no vulnerability.
  • RQ.3. Improving CWEVAL-BENCH performance: SafeCoder produces much more secure code without functionality loss under separate evaluations, but shows significant functionality degradation on CWEVAL-BENCH.On CWEVAL-BENCH, it generates less functional and secure code than the original CodeLlama-7b-hf model.
  • RQ.3. Improving CWEVAL-BENCH performance: CWEVAL-BENCH evaluates functionality and security simultaneously, distinguishing secure coding from avoiding security-sensitive code.A high func-sec@k requires generated code to satisfy both properties.

VI. CONCLUSION AND FUTURE WORK

The paper introduces CWEVAL and CWEVAL-BENCH for simultaneous, rigorous evaluation of LLM-generated code functionality and security. Empirical results expose a significant functionality-security gap and a severe pitfall in separate evaluations.

  • CWEVAL and CWEVAL-BENCH provide simultaneous evaluation of security and functionality on high-quality security-critical coding tasks.The framework and benchmark aim to improve the accuracy and rigor of secure code-generation assessment.
  • Empirical evaluation reveals a significant gap between writing functional code and writing code that is both functional and secure.The paper also identifies a previously ignored pitfall in separate functionality and security evaluations.
  • Future work includes automating benchmark creation and expansion to improve CWEVAL's scalability and efficiency.
Loading 2501.08200v1…