PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsaid, Unsafe? Implicit Security Obligations in LLM-Based RTL Code Generation

Guang Yang, Xing Hu, Xiang Chen, Xin Xia

arXiv:2608.26588v1cs.CRcs.SE

TL;DR

LLM-generated RTL can be functionally correct yet insecure when functional specifications omit security obligations, a costly gap because silicon cannot be patched after tape-out. The paper builds SECRTL-GEN and introduces RTL-Obliger, which symbolically infers signal-level obligations from specifications and CWE patterns before revising RTL. Across five models and four HDLs, RTL-Obliger raises mean all-pass to 61.6% from 49.6–51.4% for SecV/RESCUE.

  • Problem

    Existing RTL-generation evaluation emphasizes functional correctness, while security obligations are often omitted from functional specifications despite insecure RTL being unpatchable after tape-out.

  • Method

    SECRTL-GEN benchmarks real SoC IP across four HDLs, and RTL-Obliger uses an LLM-extracted functional-semantic graph, symbolic CWE matching, and two-stage obligation-guided RTL revision.

  • Results

    Across five models and four HDLs, RTL-Obliger raises mean all-pass from 49.6–51.4% for SecV/RESCUE to 61.6%.

  • Takeaways & Limitations

    Vanilla generation is functionally strong but insecure; CWE knowledge helps, unaided self-thinking cannot close the gap, and security prompting trades away functionality.

  • Takeaways & Limitations

    SECRTL-GEN covers only five port-observable resource-access CWE families, so results may not transfer to side channels, fault injection, closed-source SoCs, or full-chip integration.

Abstract

from arXiv · show

Large Language Models (LLMs) generate register-transfer-level (RTL) code with rapidly improving functional correctness. Security of LLM-generated code, however, has been studied mainly for software, where flaws can still be patched after deployment. Insecure RTL offers no such remedy once taped out into silicon. We construct SECRTL-GEN, a multi-language resource-access security benchmark grounded in real SoC IP: 392 tasks over five CWE families and four HDLs (Verilog, SystemVerilog, VHDL, and Python), each with black-box functional and security testbenches. Functional specifications intentionally omit security obligations, matching how obligations are often kept out of functional docs in practice. An empirical study of five frontier LLMs shows a sharp gap: under vanilla prompts they pass functional tests in about 73-79% of cases but security tests in only 14-35%, and stronger functional models are not safer. Adding CWE knowledge raises security, while unaided self-thinking helps less and both security-oriented prompts cut functional pass rates, showing that the bottleneck is missing weakness awareness in the specification, not an inability to write defensive RTL. We present RTL-Obliger, a neuro-symbolic framework that infers these implicit obligations. An LLM extracts a functional-semantic graph from the specification; a symbolic engine then matches it against a CWE pattern ontology to surface mitigation-evidence gaps and signal-level obligations; the LLM finally revises RTL under those obligations in a functionality-preserving two-stage generation. Across five models and four languages, RTL-Obliger raises mean all-pass from 49.6-51.4% (SecV/RESCUE) to 61.6%, with higher security and functional rates than these secure-generation baselines.

1 INTRODUCTION

The paper exposes a security gap in LLM-generated RTL under specifications that omit implicit security obligations, then introduces RTL-Obliger to infer and apply those obligations. Its benchmark and evaluation show strong functional performance but poor security, while RTL-Obliger improves combined functional-security success.

  • Motivation: Insecure RTL cannot be patched after tape-out, making omitted hardware security obligations more consequential than many software vulnerabilities.Functional specifications often exclude defenses documented separately in security configurations, threat models, or checklists.
  • Empirical gap: Five frontier LLMs pass functional tests in 73–79% of cases but security tests in only 14–35% under vanilla prompts.The study finds that stronger functional capability does not guarantee security.
  • Benchmark: SECRTL-GEN contains 392 tasks across five CWE families and four HDLs, using real SoC IP with black-box functional and security tests.The benchmark intentionally withholds security obligations from functional specifications.
  • Empirical gap: CWE knowledge improves security, whereas self-thinking helps less and security-oriented prompting reduces functional pass rates.These findings identify missing weakness awareness in the specification as the central bottleneck.
  • RTL-Obliger: RTL-Obliger combines functional-semantic graph extraction, symbolic CWE matching, and two-stage obligation-guided RTL revision.The framework is designed to infer signal-level obligations before revising the generated RTL.
  • Results: Mean all-pass rises from 49.6–51.4% for SecV/RESCUE to 61.6% with RTL-Obliger.The authors also release datasets and source code to support replication.

2 BACKGROUND

The paper defines security obligations as requirements for avoiding hardware weaknesses that functional specifications may not entail. Security testing provides evidence for exercised obligations, but not proof that every obligation is satisfied.

  • RTL generation: RTL generation maps natural-language functional specifications to RTL programs, with functional correctness checked by simulation testbench Tfunc.The notation c |= Tfunc means all functional assertions pass.
  • Weaknesses and obligations: CWE catalogs recurring hardware flaws, while each selected weakness family w has an abstract obligation class Ow derived from MITRE mitigation guidance.The paper scopes its analysis to MITRE’s Most Important Hardware Weaknesses.
  • Weakness example: For CWE-226, obligations include clearing every secret-holding element before completion, including error and abort paths, without exposing residual values.These requirements can be absent from a functional specification because functional ciphertext output does not require them.
  • Security evaluation: Passing Tsec demonstrates only the exercised obligations, not that every obligation in Ow holds.The evaluation approximates the obligation set through security-testbench assertions.
  • Implicit obligations: A specification is security-incomplete when its stated constraints do not entail the obligations required to avoid a weakness.For example, requiring a buffer to read as zero after completion entails one CWE-226 obligation even without mentioning security.

3 EMPIRICAL STUDY

SECRTL-GEN measures security-incomplete RTL generation across real SoC designs, five CWE families, four HDLs, and black-box testbenches. The controlled study finds a large functional-security gap, substantial benefit from CWE knowledge, and a functionality-security trade-off under prompting.

  • 3.1 Case construction: SECRTL-GEN comprises real SoC IP cases selected for isolated simulation, port-observable security properties, and alignment with five resource-access CWE families.The designs come from OpenTitan, Hack@DAC 2021, CVA6/Ariane, and PULP Platform.
  • 3.1 Case construction: Each case includes a functional specification, golden implementation, functional testbench, and security testbench exercising the relevant obligation class Ow.The specification intentionally states behavior while withholding how the module should be secured.
  • 3.1 Case construction: The benchmark yields 392 language-specific instances from 98 designs across Verilog, SystemVerilog, VHDL, and Python via Amaranth.The same security property must hold across languages.
  • 3.2 Baseline results: Under L0, functional pass rates are 73.4%–79.4% while security pass rates are 14.5%–35.4% across five models.MiniMax-M3 has the highest average functional rate, 79.4%, but near-bottom security performance, 20.1%.
  • 3.3 Finding 1: Mean security rises from 23.3% under L0 to 59.4% under L2, indicating that CWE knowledge substantially mitigates the functional-security gap.The improvement is reported consistently across all models and languages.
  • 3.3 Finding 2: Mean security increases from 23.3% to 54.2% and all-pass from 19.0% to 45.6% under L1 versus L0.Self-thinking activates latent defenses but remains behind explicit CWE knowledge.
  • 3.3 Finding 2: Mean functional pass falls from 76.3% under L0 to 63.4% under L1 and 60.1% under L2.The resulting trade-off caps overall all-pass at roughly 50% for these prompting methods.

4 RTL-OBLIGER: INFERRING IMPLICIT SECURITY OBLIGATIONS

RTL-Obliger treats implicit security requirements as mitigation-evidence gaps and reduces their discovery to symbolic signal-level matching. This inference guides later RTL edits rather than asking the LLM to invent obligations.

  • Obligation inference: RTL-Obliger matches specification evidence of weakness preconditions against a reusable mitigation ontology to identify missing signal-level defenses.For example, a written reusable secret register without a clear-at-done mitigation becomes a deterministic gap check.

4.1 Overview and Design Goals

RTL-Obliger is designed to infer implicit security obligations, make them auditable, and revise RTL while preserving functional behavior. Its pipeline combines symbolic inference with LLM-based obligation filtering and generation.

  • Design goals: G1 requires inferring obligations from specification structure and an explicit security ontology rather than guessing them end-to-end with an LLM.
  • Design goals: G2 makes each obligation traceable to matched structures, uncovered elements, and a CWE scenario as a reproducible witness.
  • Design goals: G3 patches the design to satisfy inferred obligations while changing functional behavior as little as possible.
  • Pipeline: RTL-Obliger binds a specification’s Functional-Semantic Graph to a CWE pattern ontology, symbolically identifies mitigation gaps, then uses an LLM for filtering and obligation-guided RTL revision.

4.2 Evidence Model

The evidence model represents specification structure and reusable mitigation knowledge as graphs over a shared finite vocabulary. This separation lets deterministic matching connect observed design evidence to expected security evidence.

  • Evidence model: RTL-Obliger encodes present security-relevant structure and expected mitigation evidence as graphs over a shared closed vocabulary Σ.
  • Closed vocabulary: Σ uses six finite dimensions: Kind, Lifetime, Sens, Op, Guard, and Exit.
  • Closed vocabulary: Kind, Lifetime, and Sens describe hardware elements by type, data persistence, and security sensitivity.
  • Closed vocabulary: Op, Guard, and Exit describe transfers by operation, firing condition, and control-state phase.
  • Spec graph Gspec: The FSG records stateful elements, transfers, and lifecycle phases extracted from one functional specification, while its well-formedness constraint requires each transfer exit to belong to S or be ε.
  • Ontology graph Gonto: An LLM translates specification text into a JSON FSG constrained by Σ, whereas Gonto stores CWE-specific element patterns, transfer patterns, scenarios, and hint templates.
  • Evidence model: Shared vocabulary Σ enables deterministic matching between Gspec’s observed evidence and Gonto’s complete mitigation evidence, supporting inferability and auditability.

4.3 Symbolic Inference

The symbolic core matches specification evidence against CWE scenarios and emits signal-level obligations when required mitigations are missing. Matching is deterministic and deliberately high-recall, with later LLM filtering handling structural false positives.

  • Inference: The symbolic core takes Gspec and Gonto and deterministically outputs signal-level security obligations without making LLM calls.
  • Scenario triggering: A scenario fires when its structural preconditions match and at least one written element lacks the expected mitigation.
  • Scenario triggering: Scenario triggering requires matches for required elements, required transfers, and contextual transfers, plus a non-empty uncovered-element set U.
  • Scenario triggering: Algorithm 1 evaluates every ontology scenario against Gspec and retains scenarios whose preconditions hold and U ≠ ∅.
  • Matching: Transfer matching checks operation, guard tags, exit path, and target binding; finite matching predicates terminate and produce reproducible structural decisions.
  • Uncovered elements: The engine computes uncovered written elements by comparing required-transfer targets with elements covered by matching mitigation transfers.
  • Obligation instantiation: Instead of a generic CWE warning, RTL-Obliger can identify concrete signals such as plaintext_reg and key_reg lacking a clear at done.
  • Obligation instantiation: Each obligation records its scenario, CWE identifier, uncovered-element set U, and filled natural-language text; all structurally supported candidates are retained for downstream filtering.

4.4 Obligation-Guided RTL Generation

RTL-Obliger separates functional drafting from security revision. An LLM judge filters structurally generated candidates, after which local obligation-guided patches or a secure fallback revise the draft while preserving its function and interface.

  • Generation strategy: The final stage filters candidate obligations and generates RTL while changing functional behavior as little as possible.
  • Obligation filtering: An LLM-as-Judge re-checks each candidate against the specification and FSG summary because some structurally matched candidates are false positives.
  • Obligation filtering: The judge may only keep or reject candidates, and parse failure preserves the full candidate set rather than silently dropping obligations.
  • Two-stage generation: Stage 1 creates a functional draft from the specification alone, avoiding the functional penalty associated with one-shot secure generation.
  • Two-stage generation: Stage 2 revises the draft using the retained obligations when available.
  • Two-stage generation: Named uncovered elements allow local patches; if obligations are absent or extraction fails, a secure fallback hardens the draft while preserving function and interface.

5 RESULTS ANALYSIS

Across five models and four HDL settings, RTL-Obliger outperforms SecV and RESCUE on joint functional-security success. Ablations and failure analysis identify symbolic matching as the main contributor, while obligation-guided generation remains the dominant failure stage.

  • All-pass: 61.6% average all-pass exceeds SecV’s 49.6% and RESCUE’s 51.4%, with RTL-Obliger leading on every model and language.The gains are +12.0 and +10.2 percentage points, respectively.
  • Functional pass rate: 71.9% average functional pass exceeds SecV’s 55.0% and RESCUE’s 56.9%, consistent with generate-then-revise preserving a clean functional draft.Security content is applied through local obligation-guided edits rather than inserted into one generation pass.
  • Security pass rate: 69.5% average security pass exceeds SecV’s 60.6% and RESCUE’s 63.8%, although RESCUE is higher in two DeepSeek-v4 cells.RTL-Obliger still has higher all-pass in those cells because RESCUE loses more on functional pass rate.
  • Statistical significance: All-pass gains are significant for every model against both baselines, while all three overall metric gains are significant at p < 10^-4.The model-level all-pass tests report pA ≤8.5×10−3.
  • Component ablation: 57.8% full-pipeline average all-pass falls to 49.7% without symbolic matching, compared with 55.6% without the Judge and 54.8% without two-stage generation.The ablation uses DeepSeek-v4-Pro, with one run per ablation and the Full row reusing the RQ1 five-run mean±std.
  • Failure analysis: 73.2% of RTL-Obliger failures retain obligations but fail during obligation-guided generation, while 21.2% miss the gold CWE during symbolic matching.Judge over-filtering accounts for 5.4% and FSG extraction for 0.2%, directing future work toward Stage 2 revision and matching recall.

6 DISCUSSION

The discussion compares RTL-Obliger with coding-agent pipelines and examines evaluation validity. RTL-Obliger achieves stronger security and all-pass performance at substantially lower token cost, while several threats constrain interpretation.

  • Evaluation design: The agent study uses one run with DeepSeek-v4-Pro on all 392 benchmark instances, making its absolute rates less stable than the five-run main comparison.The same model and first run are used for RTL-Obliger and the agent baselines for cost comparison.
  • Comparison with coding agents: 58.7% average all-pass versus 7.9% for MAGE and 24.2% for Sec-MAGE, RTL-Obliger leads across all four languages.Security averages 70.4% versus 14.0% and 30.6%, while functional correctness is mixed across languages.
  • Comparison with coding agents: 3.8M tokens and 1,536 calls make RTL-Obliger substantially cheaper than MAGE and Sec-MAGE in the cost-controlled comparison.MAGE uses 13.7M tokens and 3,116 calls; Sec-MAGE uses 33.2M tokens and 5,178 calls.
  • Threats to validity: The adapted baseline implementations may differ from their original artifacts, despite faithful re-implementation intended to reduce bias.This limitation applies to SecV, RESCUE, and MAGE adaptations to SECRTL-GEN.
  • Threats to validity: SECRTL-GEN covers five resource-access CWE families selected for port observability, so results may not transfer to side channels, fault injection, closed-source SoCs, or full-chip integration.Newer models or other languages may also shift absolute rates, although the obligation-inference design is described as model-agnostic.
  • Threats to validity: All-pass depends on black-box functional and security testbenches that may miss some insecure behaviors despite manual review and golden-implementation checks.Black-box, port-observable evaluation avoids teaching to internal signals but does not establish complete security coverage.

7 RELATED WORK

Related work has improved functional RTL generation and examined security risks, but existing approaches do not fully address security-obligation omission in real SoC IP across multiple HDLs. SECRTL-GEN and RTL-Obliger target that gap with benchmarked obligation inference.

  • Functional RTL generation: Functional RTL benchmarks such as VerilogEval, RTLLM, and CVDP primarily evaluate simulation correctness against golden designs.Domain-adapted models and iterative multi-agent pipelines further raise functional pass rates.
  • LLM security: Software-security studies document vulnerable LLM outputs, while secure-generation methods inject CWE knowledge through security tuning, retrieval, or secure examples.These studies generally assume vulnerabilities can be addressed in a post-deployment patching model.
  • Hardware security: Hardware benchmarks and audits find that LLM-generated RTL can pass functional checks while failing security tests, whereas repair studies begin with identified buggy RTL.This differs from generation from specifications that omit security obligations.
  • This work: SECRTL-GEN evaluates real SoC IP across four HDLs, and RTL-Obliger symbolically infers signal-level obligations from a functional-semantic graph and CWE ontology.Together, they address obligation-omitted multi-HDL security evaluation and design-time obligation inference.

8 CONCLUSION

The paper argues that functional evaluation alone misses security obligations in LLM-generated RTL, and introduces benchmark and generation methods to expose and address that gap. Its scope remains limited to resource-access properties checked by black-box tests.

  • Conclusion: SECRTL-GEN evaluates real SoC IP with black-box functional and security oracles while intentionally omitting security obligations from functional specifications.The benchmark supports multi-HDL resource-access security evaluation.
  • Conclusion: RTL-Obliger symbolically infers signal-level obligations from a functional-semantic graph and CWE ontology, then applies them through two-stage generation.The framework targets missing weakness awareness rather than merely defensive RTL-writing ability.
  • Conclusion: The study finds vanilla generation functionally strong but insecure; CWE knowledge improves security, while unaided self-thinking does not close the gap and security prompting reduces functionality.These conclusions are drawn within the benchmark’s resource-access scope.
  • Scope and future work: The work is limited to resource-access properties checked by black-box tests, while side-channel and fault defenses generally require white-box oracles.Future work plans to broaden the ontology within resource access and study repair workflows for white-box properties.
Loading 2608.26588v1…