PaperlayerView exact revision on arXiv

Source-linked AI summary

Applying Security by Design at the Point of Execution: How Governed Security Requirements Affect the Security of AI-Generated Code

Pedro Farinha

arXiv:2610.10659v1cs.SEcs.CR

TL;DR

Secure-code benchmarks score generators on unseen security tests without supplying the applicable requirements, leaving specification failures difficult to distinguish from capability failures. This study delivers governed requirements at generation time through MCP; security improved on both benchmarks, although the joint secure-and-functional metric remained inconclusive.

  • Problem

    Secure-code benchmarks omit security requirements and score code against unseen tests, so low scores cannot distinguish specification failures from capability failures.

    Source: §1 · p. 1

  • Method

    The study delivered task-selected requirements from a governed security-by-design knowledge base through MCP, using agent and direct-call setups.

    Source: §1 · p. 1, §2 · p. 3

  • Results

    Security improved on both benchmarks; on DualGauge, tasks passing all security tests rose from 44.1% to 78.0%.

    Source: §10 · p. 10

  • Takeaways & Limitations

    Benchmarks detect evaluator-targeted weaknesses but do not establish conformance with applicable requirements, motivating a conformance measure.

    Source: §7 · p. 8, §10 · p. 10

    Experiments used one model and one generation per task, and the joint secure-and-functional metric was inconclusive.

    Source: §9 · p. 9

Abstract

from arXiv · show

Security by design asks that security requirements are defined before code is written. Secure-code benchmarks typically measure the opposite situation: the agent receives the task without requirements and is scored with security tests it has not seen. We measured what changes when security requirements, selected from a governed security-by-design knowledge base (SbD-ToE) and delivered through a Model Context Protocol (MCP) server, are given to the generator at the point of execution. On DualGauge, which scores with a language-model judge, we ran 59 Python tasks: the share of tasks passing all security tests rose from 44.1% to 78.0% and the share of security tests passed from 77.4% to 93.0%. On BaxBench, which runs functional tests and real exploits in containers, we ran 28 backend scenarios: among functionally correct solutions, the share with no successful exploit rose from 65% to 86%, comparable to the authors' Oracle Security Reminder (85%), which they call an unrealistic upper bound; the requirement delivery reached it without knowing the tests. The study has limits. The pre-specified joint secure-and-functional metric did not improve significantly on either benchmark. The code written with the governed requirements is stricter, and it failed functional tests that fix values the task never states; we attribute those failures case by case in a post-hoc analysis. Each experiment used one model and one generation per task. The requirements come from the author's open-source manual; we do not assess its completeness. Governed security requirements delivered at the point of execution appear to improve the security of generated code. Current benchmarks cannot tell whether that code meets the security requirements that apply to it, only whether it shows the weaknesses their evaluators were built to detect. We propose measuring conformance with the applicable requirements.

1. The problem

Secure-code benchmarks give generators tasks without security requirements and score them against unseen security tests, so low scores can conflate capability and specification failures. This study delivers governed requirements at the point of execution and compares generation with and without that delivery.

1. The problem

  • The point of execution is the moment a coding agent generates code, when delivered requirements can be applied without relying on prompt authors’ memory.The authors call this a way for coding agents to apply security by design.Source: §1 · p. 1

2. What we did

  • The MCP server exposes the versioned SbD-ToE manual and graph, whose selector returns task-matched requirements, but selector completeness and content were not evaluated.The selector uses task text, generation mode and risk level; the knowledge base’s content had not been reviewed by domain experts.Source: §2 · pp. 2–3
  • All benchmark tasks used the manual’s L2 risk level because the benchmarks did not classify them; other levels, which return different requirements, were not tested.The manual classifies applications by exposure, sensitive data and impact; L2 is the medium level.Source: §2 · pp. 2–3

1. The problem

  • Benchmarks omit governed requirement selection: generators must infer which requirements apply while evaluators probe for common weaknesses.A low score cannot distinguish inability to implement a property from failure to infer that it was required; stating requirements may help without changing the model.Source: §1 · pp. 1–2, §2 · p. 3

2. What we did

  • The study used both an agent that retrieves requirements before coding and a BaxBench direct-call setup that adds requirements to its baseline prompt.In the direct-call setup, a separate agent selects requirements while BaxBench retains its single model API call.Source: §2 · p. 3
  • Comparisons used each benchmark’s baseline prompt and, on BaxBench, the Oracle Security Reminder, an unrealistic test-matched prompting upper bound.The Oracle prompt asks developers to anticipate scenario vulnerabilities; requirement delivery selects from task text without knowing the tests.Source: Table 1 · p. 3, §2 · p. 4

3. Results: security improves on both benchmarks

Delivering security requirements raised both DualGauge security metrics and moved BaxBench security in the same direction, but the joint secure-and-functional metric was inconclusive. DualGauge gains covered multiple weakness types, while its functional non-inferiority margin was not met.

3. Results: security improves on both benchmarks

  • DualGauge tasks passing all security tests rose from 44.1% to 78.0%, and security tests passed rose from 77.4% to 93.0%; BaxBench exploit-free correct solutions rose from 65% to 86%.BaxBench security was comparable to the authors’ Oracle Security Reminder; requirement delivery did not know the tests.Source: §3 · p. 4, Figure 1 · p. 5

4. DualGauge: both security metrics rise

  • On DualGauge secure@1, 23 tasks changed from failing to passing and 3 changed from passing to failing; the paired difference was +33.9 percentage points.The 95% confidence interval was +18.6 to +48.1 pp, with exact McNemar p < 0.001.Source: §4 · p. 4
  • DualGauge gains included path traversal (16 to 30 of 34 tests), log injection (0 to 9 of 9) and cross-site scripting (14 to 19 of 19).The per-task test-share change had a median of 0 and mean of +18.8 pp, with about half of tasks accounting for the pooled gain.Source: §4 · p. 4
  • The DualGauge joint metric rose from 12 to 18 tasks but was inconclusive (p = 0.109); the share passing all functional tests fell from 25 to 20.The pre-set 5 pp functional non-inferiority margin was not met.Source: §4 · p. 4
  • Most DualGauge tests and intervals added after the run lacked multiple-testing correction, and the joint metric depended on an executor repair.The security conclusion held with and without the repair.Source: §4 · p. 4

5. BaxBench: security comparable to the authors’ “unrealistic” upper bound for prompting

BaxBench requirement delivery improved security among functionally correct solutions versus No Security Reminder and was comparable to the Oracle Security Reminder. The joint secure-and-functional metric did not consistently improve, and stricter requirements caused some functional failures.

5. BaxBench: security comparable to the authors’ “unrealistic” upper bound for prompting

  • BaxBench runs functional tests and real exploits against backend applications in containers, and a solution is correct and secure only if it passes both.The benchmark has 28 scenarios per run.Source: §5 · p. 5

5.1 Against the No Security Reminder baseline

  • 86% of requirement-delivery solutions were secure among correct, versus 65% with No Security Reminder in the direct-call run.This is descriptive because the functionally correct subsets differ in size.Source: Table 3 · p. 5, §5.1 · p. 6
  • 83% (FastAPI) and 84% (Flask) were secure among correct with requirement delivery, versus 70% and 75% with No Security Reminder.The joint metric did not improve in either agent run.Source: §5.1 · p. 6

5.2 Against the Oracle Security Reminder

  • In the direct-call run, 86% of correct requirement-delivery solutions and 85% of correct Oracle Security Reminder solutions were secure among correct.The study could not test equivalence; it detected no security difference between the two.Source: §5.2 · p. 6
  • The Oracle Security Reminder scored higher on the joint metric in two of three runs; cited-source test values changed the comparison to 21 versus 20 on FastAPI and 18 versus 19 on Flask.The cited-source test-value analysis was designed after the results were seen.Source: §5.2 · p. 6

5.3 Where the functional loss comes from

  • Requirement delivery failed six direct-call tasks that No Security Reminder passed; four failures involved stricter validation and two were code errors.Examples of stricter validation include rejecting the benchmark’s 11-character secret, an unknown field, pass1, or a request without a content type.Source: §5.3 · p. 7

5.4 Plausibility and power

  • With 28 pairs, power to detect a +12.5 percentage-point joint-metric difference was about 0.14, so the results cannot confirm or exclude an effect that size.The published Oracle Security Reminder effect was +12.5 percentage points.Source: §5.4 · p. 7

6. What the benchmarks do not measure

The benchmarks test selected known weaknesses, not all applicable security requirements or additional controls. Their evaluations can also penalize stricter implementations and produce false passes.

6. What the benchmarks do not measure

  • Each BaxBench scenario runs one to four security tests, and the benchmark checks 13 weakness types; untested weaknesses do not affect its score.In one scenario, neither No Security Reminder nor requirement delivery passed the malicious-regular-expression test, while the Oracle Security Reminder did because it named that weakness.Source: §6 · p. 7
  • The tests do not score additional controls, although requirement-delivery code contains several controls that other arms almost never include.The Table 4 audit counts files containing controls, not whether the controls work.Source: §6 · p. 7, Table 4 · p. 8
  • Among 23 DualGauge tasks that gained secure@1, 17 failed at least one functional test; in seven, the code enforced a requirement the test did not expect.Figure 3 attributes causes post hoc, with one annotator; some attributions are marked probable.Source: §6 · p. 8, Figure 3 · p. 8
  • Two DualGauge tasks received security passes despite code not running, and two BaxBench scenarios’ security tests end without a result by design.The DualGauge judge passed every security test by reading the code in those two cases.Source: §6 · p. 8
  • The Oracle Security Reminder’s gain is measured only on the tests whose weaknesses it names.Its published correct-and-secure rate for Claude Opus 4.5 rose from 0.561 to 0.686.Source: §6 · p. 8

7. What is left unmeasured

Benchmarks show whether selected weaknesses were detected, not whether code meets its applicable security requirements. The paper proposes requirement-conformance measurement and relates it to prior benchmark, method, standards, and measurement work.

7. What is left unmeasured

  • Security probes provide necessary but insufficient evidence: they do not establish whether code meets the requirements applicable to its task.The proposed measure adds conformance alongside evidence that tested weaknesses were not found.Source: §7 · p. 8
  • Conformance is the share of applicable requirements satisfied by the code, traced from each requirement to its implementation.The authors report that this trace can be measured, though benchmarks do not currently score it.Source: §7 · pp. 8–9
  • 192 of 202 citations (95%) for 15 single-control identifiers occur in files where the corresponding control is present; four citations name identifiers absent from the delivered text.Requirement identifiers appear in 27 of 28 delivery files, with 468 citations total.Source: §7 · p. 9

8. Related work

  • CWEval [8], SecureVibeBench [9], BaxBench [10], and DualGauge [1] score functionality and security without giving generators the requirements their tests check.Earlier studies report weaknesses in generated code [6] and reduced security among developers using assistants.Source: §8 · p. 9
  • RESCUE retrieves guidelines and examples [13], while requirement delivery states which security properties apply; other approaches include training, risk cues, critics, repair loops, and orchestration [11] [12] [14] [15] [16].The paper connects requirement definition and evaluation to SSDF and ISO/IEC 25010, and distinguishes its security-outcome study from the registered report.Source: §8 · p. 9

9. Limits

The authors qualify the security findings by the study’s small sample and inconclusive joint metric. They identify future evaluation work, including a requirement-conformance evaluator, alongside extending the apparatus to more models, generations, and benchmarks.

9. Limits

  • Each experiment used one model and one generation per task, across 59 or 28 tasks; the joint metric was inconclusive and DualGauge missed its functional non-inferiority margin.The paper also lists judge reliability, requirement selection and knowledge-base coverage among its limits.Source: §9 · pp. 9–10

10. Conclusion

  • The conclusion describes a part of the benchmark-reported security gap as missing requirements at generation, while noting stricter code can fail functional tests that assume unstated values.The joint secure-and-functional metric did not improve significantly on either benchmark.Source: §10 · p. 10
  • Future work prioritizes applying the fixed, reusable apparatus to other models, multiple generations per task, and other benchmarks.More generations and tasks would increase statistical power beyond the 59- and 28-task samples.Source: §9 · p. 9, §10 · p. 10
  • The proposed evaluator would log each applicable requirement as delivered, cited, and satisfied, using pinned tools and blinded expert checks for different kinds of judgments.It would report delivery completeness (Mrecall) and execution fidelity (MSbD), the share of applicable requirements satisfied by the artefact.Source: §10 · p. 10

Ethics

The paper describes its security study as defensive and states the author’s relationship to the evaluated artefacts. It also records generative-AI use and where the study artefacts are deposited.

Ethics

  • The study releases no exploit code, payloads, or attack tools; its knowledge payload states requirements and audit logs show benchmark test information did not reach the generator.The author designed the manual, knowledge graph, and MCP server evaluated in the study.Source: Ethics · p. 10
  • The manual and server are openly licensed, but the repository that builds the knowledge graph is not public; the versions used are pinned by hash.The knowledge base and selector were fixed before contact with the benchmarks, and attribution records are published and contestable.Source: Ethics · p. 10

Generative AI use

  • AI tools supported harness and analysis scripts and drafting under the author’s direction; the author made design decisions, checked claims, and takes responsibility for the text.Source: Generative AI use · p. 11

Availability

  • The artefact is deposited in Zenodo and figshare, which hold the same archive; data and text use CC BY 4.0 and scripts use Apache 2.0.The archive includes protocols, outputs, evaluations, audit logs, attribution records, and scripts.Source: Availability · p. 11

Appendix A. Two worked examples

The worked examples show requirement delivery adding a computation bound in a Fibonacci task and strict input validation in a login service. The latter can conflict with functional tests that accept undeclared fields.

Appendix A. Two worked examples

  • The Fibonacci example links a controlled-failure requirement to rejecting values that could trigger unbounded computation, with the model choosing the bound of 100,000.The requirement name itself does not mention a bound.Source: Appendix A · pp. 11–12
  • All 12 Fibonacci-task tests pass in both conditions, although the requirement-backed bound stops input 1015 in 190 ms while the baseline runs until a 60-second timeout.The judge scores both implementations as passing, despite the baseline timeout.Source: Appendix A · p. 11
  • For the login service, the delivered validation instructions call for strict Pydantic models, email-format validation, length limits, and rejection of unknown fields.The scenario’s login specification declares only email and password fields.Source: Appendix A · p. 12
  • The generated login code rejects unknown fields and returns 401 for every /login validation error, but fails a functional test whose request includes an undeclared field.The security of that test-failing cell cannot be assessed.Source: Appendix A · p. 12

Appendix B. DualGauge forensics

The forensic analysis attributes functional failures among tasks gaining secure@1 to several causes, with some cases remaining uncertain. It also reports executor-repair and judge-scoring issues alongside a presence audit of security patterns.

  • The post-hoc attribution grouped failures as requirement–fixture conflicts, tests fixing unstated details, executor or environment issues, and generator defects.One annotator reviewed the 17 tasks that gained secure@1 and failed a functional test.Source: Appendix B · p. 12
  • Among the 17 tasks that gained secure@1 but failed a functional test, some executor and unstated-detail attributions remain probable rather than confirmed.Without the generated test harness, two executor cases cannot be distinguished from generator defects; two unstated-detail cases were not re-executed.Source: Appendix B · p. 12
  • After a three-line, condition-blind executor repair and re-execution, grounded secure@1 was 46 of 59, up from 44 of 59 before repair; pre-repair joint secure-pass@1 was 11 of 59 versus 12 for baseline.Before repair, 22 of 59 grounded samples failed to start because the executor placed an import before from __future__ imports.Source: Appendix B · pp. 12–13
  • Fourteen grounded security verdicts were judge abstentions counted as failures, including 12 on one task where SQL injection was correctly rejected.The judge also inconsistently scored timeouts, and two cryptographic failures were judge errors on code identical to its baseline twin.Source: Appendix B · p. 13
  • The presence audit counted files containing security patterns, including structured logging, input allowlists, defensive limits, and requirement identifiers, but did not assess correctness.Grounded versus baseline counts were 2 versus 27 for structured logging and 3 versus 24 for input allowlists.Source: Appendix B · p. 13

Appendix C. Instructions

The appendix gives distinct instructions for activating governed security guidance and formatting generated code in DualGauge, and for retrieving task-specific requirements in BaxBench. The BaxBench instructions require a standalone developer-facing summary plus a separate audit-notes block.

  • For DualGauge, the grounded instruction invokes the security-by-design skill and MCP context tool, then asks the agent to apply returned requirements while preserving specified behavior.The tool call uses the exact task, mode=codegen, and risk_level=L2.Source: Appendix C · p. 13
  • Both DualGauge conditions require the complete final implementation in one fenced Python code block.The instruction says that a reply merely referencing files is invalid.Source: Appendix C · p. 13
  • For BaxBench, the retrieval agent must pass the unchanged task to the MCP, select applicable requirements, and return developer instructions followed by an audit-notes line and omitted requirement ids.The retrieval agent prepares context for a developer and must not implement the task or include code.Source: Appendix C · pp. 13–14

Appendix D. BaxBench under test values that follow cited sources (post hoc)

The post-hoc BaxBench reevaluation changed test values to follow cited sources and found different correct-and-secure counts across arms. The apparatus appendix records the experimental configurations and how returned requirements were handled.

Appendix D. BaxBench under test values that follow cited sources (post hoc)

  • With cited-source test values, requirement delivery scored 20 against 17 for No Security Reminder on FastAPI, and 19 against 19 on Flask.The Oracle Security Reminder scored 21 on FastAPI and 18 on Flask; the edits were chosen after seeing the results.Source: Appendix D · p. 14

Appendix E. Apparatus

  • The DualGauge confirmatory setup used Claude Code with Claude Opus 4.7; its instructions and sampling settings were revised during exploratory runs, 33 of which were confirmatory tasks.The grounded agent made 59 skill invocations and 72 knowledge-server calls; the baseline made no tool calls.Source: Appendix E · p. 14
  • The BaxBench evaluator was unmodified, while the direct-call retrieval agent averaged 1.64 server calls per task and retained a median of 18 of 39 returned requirements.BaxBench's current main branch changed three scenarios' tests without changing the direction of comparisons.Source: Appendix E · p. 14
  • With default settings, the agent saved the server response to a file in 41 of 59 DualGauge sessions and 24 of 28 FastAPI sessions, then read part of it.The server response was 45,000 to 76,000 characters.Source: Appendix E · p. 14

Appendix F. The two benchmarks

The appendix describes the scope and evaluation of DualGauge and BaxBench, then details the SbD-ToE manual, knowledge graph, and MCP server. The benchmarks differ in task scale and security evaluation, while the server returns task-activated security context.

Appendix F. The two benchmarks

  • DualGauge covers short Python, C++, and JavaScript tasks, scoring functional pass@1, secure@1, and joint secure-pass@1 through an executor and language-model judge.The sample used 59 Python tasks, 31 involving a web framework.Source: Appendix F · p. 15
  • BaxBench evaluates 28 backend scenarios across 14 environments with functional tests and containerized exploits probing 13 weakness types.A solution is correct and secure when it passes functional tests and no exploit succeeds; the authors call Oracle Security Reminder an unrealistic upper bound.Source: Appendix F · p. 15

Appendix G. The SbD-ToE manual, knowledge graph and MCP server

  • The SbD-ToE manual organizes requirements, controls, and policies across three application criticality levels and relates requirements to OWASP ASVS and NIST SSDF.The knowledge graph stores stable requirement identifiers, applicability levels, controls, evidence patterns, and related concepts.Source: Appendix G · p. 15
  • The MCP context tool takes task text, mode, and risk level, then returns activated scope, an identifier citation map, a completeness report, and usage rules.The activated scope includes requirements, controls, mechanisms, practices, artefact types, and evidence patterns.Source: Appendix G · pp. 15–16
Loading 2610.10659v1…