PaperlayerView exact revision on arXiv

Source-linked AI summary

BaxBench: Can LLMs Generate Correct and Secure Backends?

Mark Vero, Niels Mündler, Victor Chibotaru, Veselin Raychev, Maximilian Baader, Nikola Jovanović, Jingxuan He, Martin Vechev

arXiv:2502.11844v3cs.CRcs.AIcs.LGcs.PL

TL;DR

LLMs have shown skill at function-level coding, but it remains unclear whether they can autonomously generate larger deployment-ready code that is both correct and secure. BaxBench evaluates end-to-end backend generation with functional tests and practical security exploits. Flagship models achieve no more than a 37% correct-and-secure generation rate, underscoring the challenge of generating secure, correct backends for production use.

  • Problem

    It remains unclear whether LLMs can autonomously generate larger-scale, deployment-ready code that is both correct and secure.

    Source: §1 · pp. 1–2

  • Method

    BaxBench evaluates 392 backend-generation tasks across scenarios and frameworks using end-to-end functional tests and practical security exploits.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Flagship LLMs do not exceed a 37% correct-and-secure generation rate on BaxBench, and models struggle with functional correctness in about 40% of cases.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Secure, correct backend generation is a minimal requirement before LLMs can be used to generate production code, and BaxBench enables rigorous evaluation toward that goal.

    Source: §7 · p. 9

    BaxBench's coverage could be expanded with more exploits and tests, additional frameworks, and more numerous, diverse, and complex scenarios.

    Source: §5 · p. 9

Abstract

from arXiv · show

Automatic program generation has long been a fundamental challenge in computer science. Recent benchmarks have shown that large language models (LLMs) can effectively generate code at the function level, make code edits, and solve algorithmic coding tasks. However, to achieve full automation, LLMs should be able to generate production-quality, self-contained application modules. To evaluate the capabilities of LLMs in solving this challenge, we introduce BaxBench, a novel evaluation benchmark consisting of 392 tasks for the generation of backend applications. We focus on backends for three critical reasons: (i) they are practically relevant, building the core components of most modern web and cloud software, (ii) they are difficult to get right, requiring multiple functions and files to achieve the desired functionality, and (iii) they are security-critical, as they are exposed to untrusted third-parties, making secure solutions that prevent deployment-time attacks an imperative. BaxBench validates the functionality of the generated applications with comprehensive test cases, and assesses their security exposure by executing end-to-end exploits. Our experiments reveal key limitations of current LLMs in both functionality and security: (i) even the best model, OpenAI o1, achieves a mere 62% on code correctness; (ii) on average, we could successfully execute security exploits on around half of the correct programs generated by each LLM; and (iii) in less popular backend frameworks, models further struggle to generate correct and secure applications. Progress on BaxBench signifies important steps towards autonomous and secure software development with LLMs.

1. Introduction

BaxBench addresses the gap between function-level coding benchmarks and the challenge of generating deployment-ready backends that are both functional and secure. It evaluates end-to-end backend generation across scenarios and frameworks, and finds that flagship models still struggle to produce correct and secure applications.

1. Introduction

  • Existing benchmarks largely focus on function-level writing, bug fixing, or domain-specific coding, while security evaluations often separate security from correctness or remain function-level.This leaves deployment-ready generation of correct and secure applications under-tested.Source: §1 · pp. 1–2

Key Contributions

  • BaxBench comprises 392 tasks combining 28 backend scenarios with 14 frameworks, and evaluates generated applications for both functionality and security.The benchmark also studies security-specific prompting, scenario complexity, and framework choice.Source: §1 · p. 2, Key Contributions · p. 3

1. Introduction

  • BaxBench tests API functionality with functional tests and probes security by sending malicious queries to generated backends.Its security exploits are developed by experts and refined on generated and human-written solutions; both evaluation types are framework- and language-agnostic.Source: §1 · p. 2
  • Flagship LLMs do not exceed a 37% correct-and-secure generation rate, and models fail the functional task in approximately 40% of cases.The authors also report that LLM agents do not provide decisive improvements.Source: §1 · p. 2

Key Contributions

  • The authors release BaxBench as a modular framework that can be extended with more challenging tasks to support ongoing evaluation of deployment-ready code generation.The benchmark evaluates 11 state-of-the-art LLMs with functional tests and security exploits.Source: §1 · p. 2, Key Contributions · p. 3

2. Construction of BAXBENCH

BAXBENCH combines realistic backend scenarios with diverse frameworks to evaluate end-to-end functionality and security. Its framework-agnostic tests and exploits run against generated applications in an isolated evaluation pipeline.

2. Construction of BAXBENCH

  • 392 tasks pair 28 backend scenarios with 14 frameworks across six programming languages.Each task combines a scenario and target framework.Source: §2 · p. 3, §2.2 · p. 4

2.1. Scenarios

  • Scenarios are selected for real-world relevance, implementation complexity, security risk, and feasibility in existing frameworks.The authors manually verified that the final 28 scenarios met these criteria.Source: §2.1 · p. 3
  • Functional tests check each endpoint against the scenario’s OpenAPI specification and are reusable across frameworks because they are API-level.Tests were manually created and verified on human-reviewed solutions.Source: §2.1 · p. 3
  • Dynamic security testing may leave code paths unexplored, although the exploits target specification-derived deployment-time vulnerabilities rather than abstract program states.The authors note that static analysis can reason about all possible execution paths.Source: §2.1 · p. 4
  • Security exploits are expert-written and framework-agnostic, covering black-box API attacks and whitebox-like inspection of application artifacts.Exploit candidates were refined using GPT-4O-generated and human-written solutions, manual review, and Snyk-Code; exploits target multiple threats per scenario.Source: §2.1 · p. 4

2.2. Constructing and Evaluating Tasks

  • Each task prompts a model with an OpenAPI scenario and framework details, then runs the generated code and tests in Docker.Frameworks were selected to span popular and niche options, using the StackOverflow Developer Survey and GitHub stars as guides.Source: §2 · p. 3, §2.2 · p. 4

3. BAXBENCH Statistics

BAXBENCH spans REST API backends and a range of security weaknesses across its tasks. Results show that passing programs often remain exploitable, and secure pass rates remain limited.

3. BAXBENCH Statistics

  • BAXBENCH covers 28 HTTP REST API scenarios and 54 endpoints, averaging about two endpoints per scenario, across 14 frameworks and six languages.OpenAPI specifications average about 430 tokens, while plaintext specifications average about 280 tokens.Source: §3 · pp. 4–5
  • Around 50% of each model’s passing programs are exploitable, and OPENAI O3-MINI reaches only 35% sec_pass@1.Figure 3 distinguishes sec_pass@1 from pass@1 using full bars and shaded extensions.Source: Figure 3 · p. 5
  • Each scenario targets an average of 3.3 CWEs, with up to five; the benchmark organizes exploits under 13 non-overlapping high-severity CWEs.Nine of the 13 CWEs appear in MITRE’s 2024 Top 25, and ten are included in four OWASP Top 10 2025 risk groups.Source: §3 · p. 5

4. Evaluation

The evaluation tests 11 LLMs on backend generation, measuring functional correctness and exploit-based security. Results expose challenges in both dimensions and show that security prompting, framework choice, scenario complexity, and specification format affect performance.

Experimental Setup

  • The evaluation covers 11 models, sampling 10 solutions per task for non-reasoning models and one for reasoning models, and reports pass@1 and sec_pass@1.pass@1 counts programs that pass all tests; sec_pass@1 counts programs that pass all tests with no successful exploit.Source: §4 · p. 5

Main Results

  • OPENAI O1 reaches 62% pass@1, but OPENAI O3-MINI, DEEPSEEK-R1, and CLAUDE-3.5 SONNET outperform it on sec_pass@1; O3-MINI is 6% higher.The best-performing model for functional correctness is not the best for secure correctness.Source: §4 · pp. 5–6

Prompting and Agents

  • Security reminders can improve secure generation, while OpenHands yields no security gain with the normal prompt but a larger gain under the oracle prompt.Oracle reminders give the highest sec_pass@1 but generally reduce passing programs; OpenHands improves sec_pass@1 by 7.2% under the oracle prompt, versus 2.5% without the agent scaffolding.Source: §4 · pp. 6–7, Table 2 · p. 6

Impact of the Backend Framework

  • Framework choice affects correctness and security: models perform better with popular languages and struggle more with lower-resource, complex, and multi-file frameworks.For OPENAI O1, Figure 5 covers all prompt types; results for other models appear in the appendix.Source: §4 · p. 6, Figure 5 · p. 7

Differences Across Scenarios

  • Larger OpenAPI specifications correlate with lower correctness, but short specifications can also describe scenarios that models find difficult.The paper attributes difficulty both to many interacting endpoints and to complex logic within individual endpoints.Source: Figure 6 · p. 7, §4 · p. 7

Added Complexity of Security

  • 5% average token overhead separates correct but exploitable solutions from correct, unexploited solutions.The overhead varies by model, framework, and scenario: it is 3.5% for CLAUDE-3.5 SONNET and 8.2% for QWEN2.5 7B; Calculator is 15.1%, partly because safe evaluation requires sanitization and safety checks.Source: §4 · pp. 7–8

Plaintext Prompt vs. OpenAPI Specification

  • OpenAPI specifications produce significantly more functionally correct backends than their plaintext transcriptions for the three tested models.The authors use this comparison to support their choice of precise OpenAPI specifications in the main experiments.Source: Table 3 · p. 8, §4 · p. 8

Exploit Coverage

  • Snyk missed 64 vulnerable programs found by the exploits, while flagging 25 programs the exploits did not catch.Manual review found 16 false positives and two rate-limit cases, leaving seven additional correct flags across three CWEs; the authors argue that practical exploits provide a sound upper bound on security.Source: §4 · p. 8

5. Discussion

BaxBench is intended to guide model improvement: current models underperform on representative backend tasks, and analysis identifies both routine implementation errors and a security–correctness trade-off. The benchmark itself could be extended with more tests, frameworks, and scenarios.

How to improve your model on BAXBENCH?

  • BaxBench’s tasks are described as representative of simpler real-world security-critical backend coding, while current models heavily underperform on them.The authors analyze model errors to suggest targeted improvements.Source: §5 · p. 8
  • OpenAI o1’s incorrect programs often fail on boilerplate requirements, formats, response codes, file handling, CLI flags, or compilation and execution.The authors believe these errors may reflect training’s focus on algorithmic coding and popular languages, and suggest diverse, high-quality training data for these frameworks.Source: §5 · p. 8
  • Security prompting raises models’ security rates but costs functional correctness; reasoning models’ correctness rates decrease much less.The authors suggest that security prompting and post-training could steer models toward secure implementations, potentially because base models already possess relevant knowledge.Source: §5 · pp. 8–9

Limitations

  • The benchmark could be extended with more exploits and tests, additional frameworks, and more numerous, diverse, and complex scenarios.The authors say these changes could tighten performance bounds, broaden domain relevance, and help sustain the benchmark’s challenge; public release also leaves contamination impossible to prevent entirely.Source: §5 · p. 9

6. Related Work

Prior benchmarks cover function-level, domain-specific, security, and repository-based coding tasks, but BaxBench focuses on generating complete backend applications from scratch and evaluating their security. Its top-down scenarios, multi-function and multi-file tasks, and backend focus distinguish it from security benchmarks built around individual weaknesses.

Benchmarking Correctness

  • Earlier correctness benchmarks emphasize short algorithmic tasks, while later work includes domain-specific and open-domain assessments.Examples include HumanEval, MBPP, and APPS; DS-1000 and Sketch2Code; and ODEX and BigCodeBench, respectively.Source: §6 · p. 9
  • BaxBench targets complete application generation from scratch, unlike repository-based benchmarks focused on code edits, snippets, or implementation plans.The cited repository-based benchmarks include SWE-Bench, RepoBench, and SWE-Lancer.Source: §6 · p. 9

Benchmarking Security

  • BaxBench builds from real-world end-to-end scenarios and identifies potential CWEs in generated code, whereas existing security benchmarks typically craft tasks around individual CWEs.It also evaluates multiple functions and files, with a specialized focus on backend applications.Source: §6 · p. 9
  • Other cybersecurity benchmarks assess malicious code generation or exploit generation, tasks orthogonal to BaxBench’s secure code-generation focus.RedCode studies malicious code generation; NYU CTF and Cybench evaluate exploit generation given vulnerable software.Source: §6 · p. 9

7. Conclusion

BaxBench evaluates standalone backend application generation across scenarios and frameworks. The authors report that flagship models rarely generate code that is both correct and secure, and position this capability as a minimal requirement for production code generation.

7. Conclusion

  • BaxBench combines 28 scenarios and 14 frameworks into 392 evaluation tasks for standalone backend application development.The authors describe this target as practically relevant and challenging in functionality and security.Source: §7 · p. 9
  • Across an evaluation of 11 state-of-the-art LLMs, flagship models rarely produce code that is both correct and secure.Source: §7 · p. 9
  • The authors frame secure and correct backend generation as a minimal requirement before LLMs can be used to generate production code.They present BaxBench as enabling rigorous evaluation toward automated software development.Source: §7 · p. 9

Impact Statement

The impact statement presents BaxBench as a contribution to safer and more reliable LLM-driven software development, while acknowledging that its findings could be misused. The authors judge the potential positive impact to outweigh that risk.

Impact Statement

  • The benchmark evaluates both correctness and security of code generated by LLMs.Source: Impact Statement · p. 10
  • By highlighting vulnerabilities and errors, the work aims to contribute to safer and more reliable LLM-driven software development.Source: Impact Statement · p. 10
  • The authors acknowledge that the benchmark could also be used to refine malicious strategies, but believe its positive impact outweighs this risk.Source: Impact Statement · p. 10

A. Summary of BAXBENCH Scenarios and CWEs

BAXBENCH describes 28 backend scenarios and covers 13 CWEs. Its summaries document scenario scale and relate covered CWEs to MITRE Top 25 and OWASP Top 10 lists.

A. Summary of BAXBENCH Scenarios and CWEs

  • 28 scenarios specify backend tasks, with 54 endpoints total and an average of 3.3 potential CWEs per scenario.The scenarios cover 13 unique CWEs.Source: Table 4 · p. 13, A. Summary of BAXBENCH Scenarios and… · p. 13
  • Scenario summaries report endpoint counts, OpenAPI and plaintext specification lengths, and potential CWEs.Average specification lengths are 432.2 GPT-4O tokens for OpenAPI and 281.6 tokens for plaintext.Source: Table 4 · p. 13
  • A separate summary lists the covered CWEs and their relationships to MITRE Top 25 and OWASP Top 10.Source: Table 5 · p. 14

B. Full Example and Case Study: Calculator

This section presents the Calculator case study alongside scenario-specific backend implementation constraints. It covers prompt requirements for security, formatting, environment, permitted utilities, file locations, and expected output behavior.

B. Full Example and Case Study: Calculator

  • Without security-specific hints, QWEN2.5 72B generated an eval-based solution flagged as CWE-94 because the code-injection test succeeded; with such hints, it generated a correct, more complex solution ensuring the request expression was a valid arithmetic expression.Source: B. Full Example and Case Study:… · p. 15

C. Prompts

  • Prompts combine scenario descriptions and OpenAPI or text specifications with applicable database and secret requirements, allowed packages, environment instructions, and security guidance.Security settings range from no additional instructions to generic reminders and oracle-based reminders naming potential CWEs.Source: C. Prompts · p. 19

C.1. Formatting Instructions

  • Single-file frameworks request code inside <CODE> tags, whereas multi-file frameworks request file paths and code for each file.Multi-file formatting instructions are supplied individually through environment instructions.Source: §C.1 · p. 20

C.2. Environment-specific Instructions

  • Environment-specific prompts vary by framework, specifying project or file setup, storage constraints, and commands that must start the application successfully.Examples include SQLite-only requirements for Lumen and Rails, and file-creation and server-start instructions for Django and NestJS.Source: §C.2 · pp. 20–22

C.3. Scenario-specific Instructions

  • Only scenarios with additional constraints receive scenario-specific instructions; examples direct FileSearch to retrieve matching files from /data and permit ffmpeg for FrameExtract.Scenarios absent from the list have no specific instructions.Source: §C.3 · p. 22
  • ImageTransfer permits ImageMagick's convert command for image-format conversion.Source: §C.3 · p. 22
  • ImageTransfer and Logger instructions specify browser viewing without download recommendations and a text log file with one message per line.Logger's file must be named logs.txt.Source: §C.3 · p. 22
  • Monitor notes that ps does not support the command parameter on this OS, while PDFCat and PDFToText specify CLI-only pdfunite and the pdftotext utility.The PDFCat example concatenates file1.pdf and file2.pdf into concatenated.pdf.Source: §C.3 · pp. 22–23
  • RegexSearch uses ./files relative to the server directory, SongDownloader streams songs from /var/songs, and ZipToTxt combines multiple zip-contained text files into one text file.The ZipToTxt service must accept a zip file and include the contents of all text files it contains.Source: §C.3 · p. 23

D. Additional Results

The appendix adds pass@5 results, CWE-occurrence reporting, and further performance visualizations. It defines pass@k and sec_pass@k and notes that reasoning models could not be reliably evaluated at k=5.

D. Additional Results

  • The additional-results appendix extends the main results with pass@5, reports CWE occurrence, and adds scenario- and framework-level visualizations.It directs readers to D.1 for pass@5, D.2 for CWE occurrence, and D.3–D.4 for performance visualizations.Source: D. Additional Results · p. 23

D.1. Pass@5 and SecPass@5

  • pass@k measures the likelihood that at least one of k model attempts passes all functional tests.The appendix uses a low-variance unbiased estimator; n is the number of sampled solutions and c the number of correct ones.Source: §D.1 · p. 24
  • sec_pass@k counts solutions that pass all functional tests and are not compromised by the security exploits.Because exploits may miss vulnerabilities, measured sec_pass@k is a strict upper bound on true sec_pass@k.Source: §D.1 · p. 24
  • Figures 7–9 compare pass@1 and pass@5 across prompts without security instructions, with a generic reminder, and with oracle-based security instructions.The figures do not include OpenAI o3-mini, OpenAI o1, or DeepSeek-R1 because those models were run only once per task, preventing reliable pass@5 and sec_pass@5 calculations.Source: Figure 7 · p. 23, §D.1 · p. 24, Figure 8 · p. 24

D.2. CWE Occurrence

The appendix reports CWE occurrence among functionally correct generated backends and gives per-scenario performance breakdowns across models and prompt settings.

D.2. CWE Occurrence

  • CWE occurrence is reported as the ratio of functionally correct backends with a specific CWE to those that could have that CWE.The appendix organizes occurrence reports by scenario, framework, and model.Source: §D.2 · p. 24
  • A slash marks cases where no code could have the corresponding CWE, while 0.00 means none or very few backends had it after rounding.Source: §D.2 · p. 25
  • Scenario-level CWE occurrence tables cover prompts without security instructions, with a generic reminder, and with oracle-based instructions.Source: Table 6 · p. 26, Table 7 · p. 27, Table 8 · p. 28
  • Framework-level CWE occurrence tables cover prompts without security instructions and with a generic security reminder.Source: Table 9 · p. 29, Table 10 · p. 29
  • Model-level CWE occurrence tables cover prompts without security instructions, with a generic reminder, and with oracle-based instructions.Source: Table 12 · p. 30, Table 13 · p. 31, Table 14 · p. 31

D.3. Model Performance across Scenarios

  • Figures 10–20 break down pass@1 and sec_pass@1 by scenario for all 11 evaluated models and all three prompt settings.Source: §D.3 · p. 32

D.4. Model Performance across Frameworks

This section provides per-framework performance breakdowns for the evaluated models, reporting pass@1 and sec_pass@1 across three prompt settings. The figures complement the OPENAI O1 results with breakdowns for the other ten models.

D.4. Model Performance across Frameworks

  • The per-framework figures report pass@1 and sec_pass@1 for the other 10 evaluated models across all three prompt settings.They complement the OPENAI O1 results shown in Fig. 5.Source: §D.4 · p. 32
  • The reported framework comparisons evaluate functionality and security performance across different frameworks for six models: OPENAI O3-MINI, DEEPSEEK-R1, GPT-4O, CLAUDE-3.5 SONNET, LLAMA-3.3 70B, and DEEPSEEK-V3.Source: Figure 21 · p. 42, Figure 22 · p. 43, Figure 23 · p. 43
  • The figures are organized as individual model breakdowns across frameworks, rather than as a single pooled model comparison.Source: Figure 21 · p. 42, Figure 22 · p. 43, Figure 23 · p. 43

Related papers

Loading 2502.11844v3…