PaperlayerView exact revision on arXiv

Source-linked AI summary

SecReason: Vulnerability Detection and In-Stream Self-Repair for LLM Code Generation

Ruidong Han, Zhensu Sun, Chengyan Ma, Yuqing Niu, Siqi Ma, David Lo

arXiv:2610.14453v1cs.CRcs.SE

TL;DR

LLM-generated code can remain vulnerable even when it passes functional verification, and existing defenses do not inspect code as it is generated. SecReason scans reasoning-stream drafts and inserts warnings for in-place repair; on tasks where unassisted generation introduces vulnerabilities, it raises secure-and-functionally-correct delivery from below 15% to 72.8%–98.1%.

  • Problem

    Existing defenses do not inspect code during generation, while functionally correct but vulnerable output can persist when verification focuses on functionality.

    Source: §1 · p. 1

  • Method

    SecReason incrementally scans reasoning-stream code with statement-local and context-dependent checks, then inserts warnings that prompt repair in the same trace.

    Source: §1 · p. 2

  • Results

    72.8%–98.1% of runs deliver secure and functionally correct code with SecReason, up from below 15% under unassisted generation on tasks where it introduces vulnerabilities.

    Source: §1 · p. 2

  • Takeaways & Limitations

    In-stream repair uses fewer tokens than post-hoc repair and limits functional regressions when security rules misfire on secure code.

    Source: §1 · p. 2

    Self-repair was evaluated on two of six models, and the 173 tasks cover unit-level generation rather than repository-scale development or multi-step agentic pipelines.

    Source: §6 · p. 18

Abstract

from arXiv · show

Large language models (LLMs) may introduce security vulnerabilities into generated code, which can persist into production when verification focuses on functional correctness. Existing defenses act before, during, or after generation, but none inspects the code during generation. Security prompts and retrieved security knowledge act before any code exists, constrained decoding limits token choices but cannot guide repair, and post-hoc repair must regenerate the full response. Reasoning LLMs draft code early in their streamed chain of thought, and in this pre-delivery window the code exists only inside the reasoning stream. We present SecReason, an in-stream defense that monitors this window. Dual-track incremental scanning applies security rules, checking each statement as it is emitted and tracing dataflows across the accumulated code. When a security rule fires, SecReason interrupts generation, inserts an in-stream warning, and resumes decoding, so the model repairs the draft within the same reasoning trace without regenerating the full response. On tasks where unassisted generation introduces vulnerabilities, \system raises the share of runs delivering secure and functionally correct code from below 15\% to 72.8\%--98.1\% on \tool{CWEval} and \tool{SecCodeBench}. \system consumes fewer tokens than post-hoc repair, and when security rules misfire on secure code, it limits the functional regressions from false warnings.

1 Introduction

Functionally correct LLM-generated code can still carry vulnerabilities into software repositories, while existing defenses do not inspect code during generation. SecReason monitors reasoning-stream drafts and warns the model to repair them before delivery; evaluations report higher secure-and-functional delivery with fewer tokens than post-hoc repair.

1 Introduction

  • Functionally correct but vulnerable code can persist because developers primarily verify functionality and rarely screen for security.Existing defenses act before, during, or after generation, but none inspects code during generation; post-generation repair requires regenerating the full response and risks breaking working code.Source: §1 · p. 1
  • Reasoning models draft code before delivery, creating a pre-delivery window in which an analyzer can inspect the stream and guide in-place repair.Security concerns in reasoning do not prevent vulnerable drafts from reaching delivered code.Source: §1 · pp. 1–2
  • SecReason incrementally recognizes completed statements, combines statement-local checks with context-dependent dataflow scans, and inserts warnings that prompt repair in the ongoing trace.It operates at the inference serving layer without modifying model weights.Source: §1 · p. 2
  • 72.8%–98.1% of runs deliver secure and functionally correct code with SecReason, up from below 15% under unassisted generation on vulnerable tasks.The gains use fewer tokens than post-hoc repair and limit functional regressions when rules misfire on secure code.Source: §1 · p. 2

2 Reasoning Models Draft Insecure Code Early

Reasoning models can draft insecure code early and deliver it despite self-review, leaving a window for in-stream analysis before execution. The section motivates that opportunity and identifies the practical challenges of applying static analysis and feeding findings back during generation.

2.1 Motivating Example

  • A URL-generation trace identifies # and ? as invalid prefix characters, but the draft checks only slashes and empty strings, allowing evil.com# to direct requests to an untrusted host.The model delivers the draft unchanged after its own review.Source: §2.1 · p. 3

2.2 Empirical Analysis

  • 52.4%–99.8% of delivered lines appear verbatim in reasoning-stream code blocks, while insecure generations occur on both benchmarks and drafts close 45.2%–89.1% through the trace.Security-term mentions do not reliably ensure secure delivery; 5.4% to 61.1% of generations whose traces mention security terms still fail security tests.Source: Table 1 · p. 4, §2.2 · p. 4

2.3 Opportunity for Security Analysis Before Delivery

  • In automated environments, vulnerable generated code can reach execution without intermediate human review, so post-generation defenses act after the delivery boundary.Because delivered code is drafted well before delivery, an analyzer can inspect it as the reasoning trace unfolds and insert warnings for repair before delivery.Source: §2.3 · pp. 4–5

2.4 Challenges of In-Stream Analysis and Self-Repair

  • In-stream analysis must scan completed statements rather than arbitrary fragments or wait for a code fence to close, balance latency against cross-statement dataflow coverage, and turn structured findings into actionable guidance without restarting generation.Rescanning all accumulated code on each statement can stall the stream, while scanning only the new statement misses cross-statement taint flows.Source: §2.4 · p. 5

3 Method

SecReason reconciles static analysis with streaming generation by parsing code incrementally, scanning it at statement and accumulated-context scopes, and inserting warnings that let the model repair drafts in place.

3.1 Overview

  • SecReason monitors streamed reasoning between the inference engine and downstream clients, using streaming detection and in-stream self-repair before delivery.When a rule triggers, it interrupts generation, inserts a warning, and resumes from the existing token prefix.Source: §3.1 · p. 6

3.2 Streaming Detection

  • Tree-sitter incrementally parses fenced code into an AST, while semgrep-core matches security rules across the five evaluated languages.Completed statements are cleanly parsed; trailing partial syntax remains in localized ERROR nodes.Source: §3.2 · p. 6, §3.2.1 · p. 6

3.2.1 Statement-Unit Extraction.

  • SecReason extracts each cleanly parsed statement once, skipping ERROR nodes and previously seen statement end positions across parse rounds.A call is extracted only after it becomes complete and has an unseen end position.Source: §3.2.1 · p. 6

3.2.2 Dual-Track Scanning via Rule Partitioning.

  • Statement-local rules run on each emitted unit, while context-dependent rules scan accumulated code at function or class closure, code-fence closure, and generation termination.The tracks cover, respectively, evidence within one statement and evidence or scope spanning statements.Source: §3.2.2 · p. 7

3.3 In-Stream Self-Repair

  • A match produces a structured finding with the rule identifier, CWE diagnostic, offending snippet, and remediation guideline for the self-repair stage.The finding is passed to the stage that guides revision without discarding ongoing reasoning.Source: §3.2.2 · p. 7, §3.3 · p. 8

3.3.1 Mid-Stream Interception and Warning Insertion.

  • On a rule trigger, SecReason interrupts token production and inserts a warning pairing the diagnosis and offending snippet with remediation guidance and a functionality-preserving constraint.A repeated trigger for the same rule escalates the warning header to say the issue persists.Source: §3.3.1 · p. 8
  • A configurable per-rule interruption cap prevents repeated interventions from looping indefinitely while monitoring continues for other rule violations.After the cap is reached, further interruptions for that rule are suppressed and generation continues.Source: §3.3.1 · p. 8

3.3.2 Resuming Generation.

  • After inserting a warning, SecReason resumes decoding from the existing token prefix and rebuilds detection state by replaying the stream and warning through statement extraction.Replay preserves earlier taint sources in the accumulated buffer, while discarding matches that end inside the replayed region.Source: §3.3.2 · p. 8

4 Evaluation

The evaluation examines incremental detection, live-stream detection and repair, and robustness to false positives using static-code and executable security benchmarks.

4 Evaluation

  • The evaluation measures incremental scanning accuracy, live-stream detection reliability, lead time and runtime cost, in-stream repair efficacy, and robustness to false positives.The implementation uses SGLang, Tree-sitter, and semgrep-core on an NVIDIA RTX PRO 5000 and AMD Ryzen 9 9950X3D server running Ubuntu 24.04.Source: §4 · pp. 8–9, §4.1.1 · p. 9

4.1.1 Implementation and Environment.

  • The Multi-Language Static Corpus contains 2,878 balanced vulnerable and benign samples across Python, JavaScript, Java, and C, including complete programs and incomplete snippets.C++ is omitted because the official Semgrep registry has no default security rules for it.Source: §4.1.2 · p. 9

4.1.2 Benchmarks and Corpora.

  • The executable-benchmark evaluation covers 173 tasks in five languages: 97 from CWEval and 76 from SecCodeBench.Six open-weight models supply reasoning streams; two locally deployed models are used for self-repair and robustness experiments.Source: §4.1.2 · p. 9, §4.1.3 · p. 9

4.2 Detection Accuracy

Incremental scanning performs near batch scanning on complete code and improves detection on incomplete snippets, where batch parsing can miss vulnerabilities.

4.2 Detection Accuracy

  • On incomplete snippets, SecReason raises recall by 12.5 to 15.2 percentage points at 95.6% to 100% precision, lifting overall F1 by +1.5 to +17.4 points over batch scanning.The advantage occurs where syntax errors prevent batch parsing and vulnerability discovery.Source: Table 2 · p. 9, §4.2 · p. 10
  • On syntactically complete code, SecReason's F1 differences from batch scanning stay within 2.0 points in all four languages.The comparison uses official registry rules.Source: §4.2 · p. 10
  • Limited registry-rule coverage explains low absolute F1 on JavaScript and Java, with batch recall of about 10% and SecReason recall of at most 23.5%.The registry rules cover only a small share of vulnerable samples in these languages.Source: §4.2 · p. 10

4.3 Streaming Detection Performance

SecReason’s corpus-calibrated ruleset supports reliable in-stream detection: it flags nearly all insecure generations, often well before the stream ends. Monitoring adds little latency, though some findings are false positives and one insecure generation is missed.

4.3.1 Rule Coverage and Synthesis.

  • 98.2% precision, 99.7% recall, and 98.9% F1 were achieved by corpus-calibrated rules across 888 generations, covering all 20 CWE categories.Official Semgrep rules covered 13 categories, with 59.8% precision, 36.7% recall, and 45.5% F1; the calibrated rules were synthesized from two locally deployed models’ generations, with four other models held out.Source: §4.3.1 · p. 10, Table 3 · p. 10

4.3. Streaming Detection Performance

  • The replay measured detection reliability and lead time on insecure generations, and recorded whether secure generations triggered findings inside the reasoning trace.For each insecure generation, lead time was the share of the stream remaining at the first finding; replay used corpus-calibrated rules without intervention.Source: §4.3.2 · p. 10, §4.3 · p. 10

4.3.2 Detection Rate and Lead Time.

  • 323 of 324 insecure generations were flagged (99.7%), with 54%–82% of the stream remaining at the median first finding across models.299 findings fired inside the reasoning trace and 24 only while writing the final response.Source: §4.3.2 · p. 10, Table 4 · p. 11
  • 25 of 70 findings on secure generations were false positives caused by rule patterns matching partial code across separate draft fences.The single missed insecure generation resulted from insufficient pattern coverage in the calibrated rules.Source: §4.3.2 · pp. 10–11
  • Monitoring load averaged 0.9% of generation latency and never exceeded 3.2%, although the monitor scanned each generation 42–196 times on average.A scan took about the generation time of two to three tokens on the fastest model; serializing all scans would add at most 3.2% to generation time.Source: §4.3.3 · p. 11

4.4 Self-Repair Efficacy

SecReason is evaluated as an in-stream self-repair method against prompt-based and post-generation alternatives. It substantially increases secure, functionally correct deliveries, while outcomes depend on rule coverage and repair can introduce functional failures.

4.4 Self-Repair Efficacy

  • The study tested Qwen and Gemma on screened tasks, comparing SecReason with Hardened Prompt, Post-Hoc Repair, INDICT, and oracle-retrieved RAG-Guided Prompt under matched warning and attempt budgets where applicable.The screened pool retained tasks with at least one insecure but functionally valid unassisted run, and excluded tasks whose five unassisted runs all failed functional verification.Source: §4.4 · pp. 11–12

4.4.1 Repair Effectiveness.

  • 72.8%–98.1% of SecReason runs delivered Secure & Func. code, versus fewer than 15% for RAW; Insecure & Func. completions fell to at most 3.1%.Repaired runs accounted for 67.2%–93.5% of SecReason deliveries, while at most 12.8% were Initially Secure.Source: §4.4.1 · p. 12, Table 5 · p. 13
  • Functional failure stayed at or below 10.8% on CWEval but reached 17.7% for Gemma and 24.1% for Qwen on SecCodeBench, primarily from out-of-task or version-incompatible APIs.SecReason averaged 0.98–1.19 repairs per run, increasing mean token consumption by 10%–60% over RAW generation.Source: §4.4.1 · p. 12
  • SecReason had the highest Secure & Func. rate in every Table 5 cell, while up to 29.7% of Hardened Prompt runs remained Insecure & Func.In 72% of 187 insecure Hardened Prompt runs, the model identified a potential vulnerability in its trace but still delivered insecure code.Source: §4.4.2 · p. 12, Table 5 · p. 13

4.4.2 Comparison with Alternatives.

  • Post-Hoc Repair came within six points of SecReason in Secure & Func. rate but used 1.5×–2.4× its tokens per run; INDICT fell below Post-Hoc Repair in every cell and had the table’s highest token cost.RAG-Guided Prompt improved over Hardened Prompt in every cell, but insecurity persisted on SecCodeBench; unlike in-stream warnings, prompt-carried knowledge was not checked against delivered code.Source: §4.4.2 · pp. 12–14, Table 5 · p. 13

4.4.3 Ablation Studies.

  • With official registry rules, Secure & Func. rates were 27.7% on Qwen and 23.2% on Gemma, below the calibrated-ruleset rates of 80.5% and 88.3%.Most insecure deliveries did not trigger a warning; ablations also found that removing remediation guidance lowered Secure & Func. rates by 20.2 points on Qwen and 25.9 on Gemma.Source: §4.4.3 · p. 14, Table 6 · p. 14

4.4.4 Failure Analysis.

  • Of 110 functionally failed SecReason runs, 67 failed to compile, most often because of missing or incompatible library APIs; 31 compiled but failed functional tests, and 12 hit the token limit.Ten runs remained insecure: three involved detection misses and seven went unresolved.Source: §4.4.4 · pp. 14–15
  • Warnings inside unclosed code fences were associated with 72.4% Secure & Func. outcomes versus 89.4% when no warning landed inside a fence, with functional failure rates of 26.3% versus 9.3%.Insecure rates were the same at 1.3%, so the reported difference was functional rather than a security regression.Source: §4.4.4 · p. 15

4.5 Robustness to Rule False Positives

The false-positive evaluation weakened rules on vulnerability-free tasks to test their effects on repair methods. SecReason showed smaller functional regressions than the post-generation alternatives, and its in-stream context was associated with different responses to warnings.

4.5 Robustness to Rule False Positives

  • The study sampled 15 vulnerability-free tasks across five languages and tested essential and incidental false-positive settings on Post-Hoc Repair, INDICT, and SecReason.Each setting and model used five independent runs per task; Hardened Prompt and RAG-Guided Prompt were excluded because they do not act on rule warnings.Source: §4.5 · p. 15, Table 7 · p. 15

4.5.1 Impact of False Positives on Correctness and Efficiency.

  • Under false positives, SecReason’s functional failure rate rose by at most 4.0 points, while median token use remained 0.9×–1.6× its baseline.Post-Hoc Repair’s functional failure rate rose by 5.3–9.3 points on Qwen and 5.3–10.7 points on Gemma, while its median token use grew to 2.8×–4.4×.Source: §4.5.1 · p. 16, Table 8 · p. 16
  • In 45% of 93 triggered SecReason runs, the model explicitly refuted the warning as spurious; Post-Hoc Repair accepted warnings in 75% of 92 fired rounds and sometimes altered surrounding logic during full regeneration.Those post-hoc attempts to eliminate benign constructs caused compilation errors and runtime failures.Source: §4.5.2 · p. 16

5 Related Work

Prior work constrains generation, monitors reasoning text, or analyzes completed code, but the cited approaches do not inspect drafted code during generation. Related repair methods use feedback or security knowledge, generally before generation or after code is complete.

5.1 Vulnerability Detection for LLM-Generated Code

  • Generation-time defenses enforce syntax, execution-state consistency, or textual safety, while reasoning-trace monitors inspect natural language rather than generated code.The cited examples include grammar-guided token masks [49], EG-CFG, and monitoring for reward hacking [2], leaked sensitive data, or inconsistencies [19].Source: §5.1 · pp. 16–17

5.2 Feedback-Driven Code Repair

  • Post-generation checks such as GPTScan and IRIS analyze completed code, while feedback-driven security patching guides repair after generation and requires regenerating the full response.Other feedback-driven repair uses REPLs, compiler errors, execution results, or other feedback.Source: §5.1 · p. 17, §5.2 · p. 17
  • Security Course puts distilled vulnerability knowledge into a retrieval corpus before generation, enriching prompts without inspecting the code ultimately delivered.Source: §5.2 · p. 17

6 Threats to Validity

The evaluation has limitations in ruleset generalization, model and task coverage, and the interpretation of qualitative analyses. Replaying the intervention under official registry rules nevertheless shows that its effect does not depend on rule calibration.

6 Threats to Validity

  • Manual qualitative classifications and chain-of-thought interpretation may be fallible, while primary metrics come from executable test suites that can misclassify some defensive behavior.The authors treat lexicon-matched security-term statistics as correlational; pre-built suites are shared across methods.Source: §6 · pp. 17–18
  • Task-level splitting reduced ruleset recall from 97.0% on the synthesis half to 7.0% on the held-out half, limiting the calibrated ruleset to this benchmark task pool.Despite this leakage risk, official-registry replay showed insecure deliveries below RAW in every benchmark and model combination.Source: §6 · p. 17
  • In-stream interception requires raw reasoning tokens, excluding closed-source APIs that return only thought summaries.Source: §6 · p. 17
  • Self-repair experiments covered only two of six reasoning models, leaving efficacy on the other four untested.The tested models were Qwen3.6-35B-A3B and Gemma-4-26B-A4B.Source: §6 · p. 18
  • The 173 tasks cover unit-level generation, not repository-scale development or multi-step agentic pipelines.Source: §6 · p. 18
  • Code extraction relies on markdown code fences, so code drafted outside fences is not analyzed.Most delivered lines were drafted inside fences in the empirical analysis.Source: §6 · p. 18

7 Conclusion

SecReason monitors reasoning streams to detect vulnerable drafts and guide repair before delivery. Across CWEval and SecCodeBench, it improves secure, functionally correct delivery on tasks where unassisted generation is vulnerable, with lower token cost than post-hoc repair.

7 Conclusion

  • SecReason detects vulnerable drafts in reasoning streams and guides the model to repair them before delivery.Source: §7 · p. 18
  • On tasks where unassisted generation introduces vulnerabilities, secure and functionally correct code rises from below 15% to 72.8%–98.1% on CWEval and SecCodeBench.Source: §7 · p. 18
  • In-stream repair costs fewer tokens than post-hoc repair and limits functional regressions from rule false positives.Source: §7 · p. 18

Ethics Statement

The work evaluates a defensive mechanism on public security benchmarks and corpora, without real-world systems or attack targets. SecReason is an optional inference-layer monitor that leaves model weights and training data untouched.

Ethics Statement

  • The evaluation uses public security benchmarks CWEval and SecCodeBench and public corpora Purple Llama and Rosetta Code.Source: Ethics Statement · p. 18
  • The study involves no real-world systems or attack targets; vulnerabilities correspond to publicly documented CWE categories.Source: Ethics Statement · p. 18
  • Exploitation is confined to benchmark security test suites running in isolated containers, and the optional inference-layer monitor leaves model weights and training data untouched.Source: Ethics Statement · p. 18

Related papers

Loading 2610.14453v1…