PaperlayerView exact revision on arXiv

Source-linked AI summary

When Verified Source Becomes Attack Input: Defending Smart Contracts Against LLM-Based Vulnerability Scanning

Mingyuan Huang, Zimo Ji, Yifan Mo, Shuai Wang

arXiv:2608.28400v1cs.CRcs.SE

TL;DR

Publicly disclosed smart-contract source enables transparency but can also support scalable LLM-based vulnerability scanning. DeLLMGuard separates source disclosure from runtime execution through multiple contract addresses and verifies that the original implementation is preserved. Across 387 vulnerable contracts and three agents, it reduces overall root-cause correctness from 23.5% to 6.6%, although cross-contract recovery remains challenging.

  • Problem

    Public source disclosure supports auditing but can become scalable input for malicious LLM-based vulnerability discovery.

  • Method

    DeLLMGuard uses multi-address deployment with proxy, delegate, and factory relations, plus a Verification Layer that checks deployment, bytecode, source, and state preservation.

  • Results

    23.5% to 6.6% overall root-cause correctness on 387 real-world vulnerable contracts evaluated with three LLM agents.

  • Takeaways & Limitations

    Cross-contract recovery can reduce automated scanning without hiding source code, because agents often recover downstream contracts yet still fail to identify vulnerabilities.

  • Takeaways & Limitations

    The defensive effect may diminish as future agents or analysis tools improve at recovering multiple contract relations.

Abstract

from arXiv · show

Smart contracts are financial programs deployed on blockchains to manage digital assets. To build trust with users and investors, smart contract projects typically publish their source code on blockchain explorers and verify it against the deployed bytecode, making the on-chain program accessible through a human-readable implementation. However, LLM agents are changing the threat model of this disclosure mechanism. By leveraging publicly disclosed source code, recent agent workflows make it increasingly practical to scan contract vulnerabilities for exploits at large scale. In this paper, we propose DeLLMGuard, a smart contract deployment framework that defends against malicious LLM-based vulnerability scanning while preserving public source disclosure and authorized auditing. DeLLMGuard can separate disclosed source code from runtime execution through multiple contract addresses in a real-world blockchain environment. LLM agents must therefore recover additional proxy, delegate, and factory relations before vulnerability analysis. A built-in Verification Layer checks deployment relations, runtime bytecode, source code, and state changes to ensure that the transformation preserves the original business implementation. We evaluate DeLLMGuard on 387 real-world vulnerable contracts with three LLM agents in an environment derived from SCONE-bench. DeLLMGuard reduces overall root-cause correctness from 23.5% to 6.6% and outperforms the closed-source bytecode baseline on the primary non-proxy set. Trace and ablation analyses further show that agents often recover downstream contracts but still fail to identify the vulnerability, indicating that cross-contract recovery remains a major challenge for automated LLM scanning.

1 Introduction

Publicly verified smart-contract source code supports transparency and auditing but is increasingly usable by LLM agents for scalable vulnerability discovery. DeLLMGuard addresses this threat with deployment-level indirection that preserves disclosure while complicating automated analysis.

  • Motivation: Verified source code makes deployed smart-contract implementations human-readable and supports external security audits.DeFi projects commonly publish source code on blockchain explorers and verify it against deployed bytecode.
  • Motivation: Frontier LLM agents can autonomously discover and validate exploits in real-world contracts, including previously unknown vulnerabilities.Tool-mediated execution lets agents discard failed attempts and continuously scan contracts.
  • Motivation: Public disclosure can therefore become scalable input for malicious vulnerability discovery and exploitation.Attackers need only one profitable vulnerability, while defenders must manually verify potential findings.
  • Research challenge: Smart-contract defenses must preserve public source access and auditing while remaining effective against attackers controlling their own models and tools.Existing defenses often assume specially trained models or controlled execution environments, unlike public blockchains.
  • Approach: DeLLMGuard separates disclosed source addresses from runtime execution through multiple on-chain contract relations, requiring agents to reconstruct execution context.The framework uses proxy, delegate, and factory relations without modifying business logic or restricting source access.

2 Preliminaries

Blockchain execution records transactions and state changes immutably while charging gas for computation and storage. Smart-contract source verification makes difficult-to-inspect bytecode accessible for auditing and transparency.

  • Blockchain execution: Blockchains record transactions and contract state in an immutable sequence of blocks.Transactions can transfer assets or invoke contracts, causing on-chain state changes.
  • Blockchain execution: Blockchain execution charges gas for computation and storage.
  • Smart contracts: Developers compile Solidity contracts into EVM bytecode, which is publicly accessible but difficult to inspect directly.
  • Transparency and auditing: Publishing and verifying source code on blockchain explorers lets users and auditors inspect the corresponding implementation.Source access complements audit reports, which provide evidence of independent security review.

3 Research Problem

The research problem is how to reduce automated LLM-based vulnerability scanning without sacrificing public source disclosure and security auditing. DeLLMGuard must operate through standard blockchain mechanisms and accommodate real-world deployment relationships.

  • Problem: Public source disclosure supports transparency and auditing but also gives LLM agents scalable input for automated vulnerability discovery.
  • Problem: The paper asks how deployment can defend against automated LLM scanning while preserving public source disclosure and security auditing.
  • Setting: The evaluation inputs combine DeFiHackLabs incidents with recovered victim addresses, verified source, compiler metadata, and bytecode.
  • Requirements: Real-world deployments require handling proxy relationships and different Solidity compiler versions because source and addresses are not always one-to-one.
  • Requirements: DeLLMGuard must use standard contract, deployment, and on-chain-state interfaces without changing Solidity or the EVM.
  • Requirements: The framework must identify proxies, preserve business runtime and logic verifiably, and keep original source publicly available.

4 Methodology

DeLLMGuard analyzes deployment relationships, applies architecture-specific guard components, and verifies the transformed system before evaluation. Its core strategy breaks the direct link between public entry points and business source while preserving execution context.

  • Overview: The Proxy Analyzer identifies proxy relationships and compiler versions before guard application.The Verification Layer checks transformed relations, runtime bytecode, and source-code preservation before evaluation.
  • Core design: DeLLMGuard separates public entry addresses from disclosed business source through additional proxy, delegatecall, or factory relations.Agents must reconstruct relevant contracts and execution contexts before vulnerability analysis.
  • Guard components: PDC diversifies proxy-resolution cues, DLC separates entry from implementation through delegatecall, and FIC separates disclosed source from factory-deployed runtime.BIC is evaluated separately as a closed-source baseline that prevents standard source verification.
  • Proxy Analyzer: The Proxy Analyzer starts at the public entry address, determines whether it is a proxy, and recovers its implementation from forwarding logic and storage.Cases with resolved proxies lacking source code are excluded from paired evaluation.
  • Delegate Layering Component: DLC places a guard at the entry address and executes the original business bytecode from a separate implementation through delegatecall.Delegatecall preserves the guard contract’s storage and call context while retaining disclosed source in the business contract.
  • Delegate Layering Component: DLC adds bytecode, decompilation, deployment-transaction, and transaction-analysis steps before agents retrieve source and analyze vulnerabilities.

4.4 Proxy Diversification Component (PDC)

PDC protects confirmed proxy deployments by adding shadow identities to unused proxy-related storage slots while preserving the effective implementation and forwarding logic. The component increases the cross-contract and storage analysis required before LLM vulnerability analysis.

  • PDC operation: PDC targets confirmed proxy deployments and writes shadow identities to unused proxy-related slots without changing the original implementation slot or forwarding logic.The selected slots are checked before writing, and the original proxy does not read them during forwarding or business execution.
  • Supported proxy conventions: PDC covers EIP-1967 admin and beacon slots, the EIP-1822 PROXIABLE slot, and the legacy ZeppelinOS admin slot.Inserted identities match expected slot roles, including admin EOAs and implementation contracts.
  • LLM analysis workflow: PDC adds get_invalid and detect_invalid so LLM workflows inspect additional proxy candidates before identifying the effective implementation address.The original workflow uses get_hex and detect_slot before source retrieval and vulnerability detection.
  • Factory indirection: FIC separates disclosed source code from the deployed business address by deploying the business contract through a factory using CREATE or CREATE2.DeLLMGuard verifies the creation relation between the factory and deployed business contract.
  • Factory indirection: FIC requires bytecode and transaction analysis to recover the creating factory before source-based vulnerability analysis.The public entry address does not directly reveal the factory creation relation.

4.6 Verification Layer

The Verification Layer validates that transformations preserve the original deployment, routing, runtime, source, and state behavior. It compares an expected deployment graph with evidence reconstructed from blockchain artifacts and rejects transformations when applicable checks fail.

  • Verification purpose: The Verification Layer prevents reduced LLM accuracy from being attributed to an incorrect transformation rather than the intended protection.It checks business-source and business-runtime identities before and after transformation, along with deployment relations, execution routes, and state changes.
  • Evidence-graph validation: The verifier compares an expected deployment graph with an evidence graph reconstructed from bytecode, storage, transaction traces, and compiler outputs.Graph nodes represent deployed contracts, while edges encode proxy forwarding, delegatecall, CREATE, and CREATE2 relations.
  • Deployment and routing verification: Deployment and routing checks validate factory creation relations, forwarding paths, delegatecall targets, effective proxy implementations, and shadow-slot behavior.Pdep verifies CREATE or CREATE2 address derivation, while Proute checks relations from deployed code and storage.
  • BIC baseline: BIC preserves deployed runtime bytecode but exposes only wrapper source, making it a closed-source comparison baseline rather than a DeLLMGuard guard component.It removes direct source-level information while retaining bytecode, storage, transaction, and RPC access for analysis.
  • Composite configuration: The composite configuration combines proxy, delegate, factory, and comment interventions, requiring invalid-candidate checks, decompilation, and transaction analysis before vulnerability detection.CPC changes disclosed comments without changing deployed bytecode or execution.
  • Analysis limitation: Bytecode decompilation does not eliminate cross-contract recovery challenges because agents may misinterpret local instructions and their relations.Correct analysis links bytecode with storage and transaction information to determine effective proxy, delegate, and factory relations.

5 Evaluation

The evaluation tests DeLLMGuard across real vulnerable contracts, LLM agents, and controlled replay environments, finding substantial reductions in root-cause identification. Agents often traverse added relations but fail at later recovery or reasoning stages.

  • Evaluation Setup: The evaluation asks whether DeLLMGuard prevents vulnerability identification, where agents stop, how components contribute, and whether the defense remains effective when its mechanism is disclosed.PARSEC provides paired original and transformed deployments with identical historical-chain conditions and address-discovery workflows.
  • Evaluation Setup: 387 real-world vulnerable contracts and three LLM agents were evaluated in an environment derived from SCONE-bench.Agents operated through PARSEC’s local replay RPC rather than live vulnerable contracts.
  • Effectiveness: 74.9%: DeLLMGuard’s mean relative reduction in strict correct on the larger non-proxy set, with proxy cases moving in the same direction.Model-specific non-proxy reductions are 59.6%, 87.5%, and 77.6% for GLM, GPT, and Claude.
  • Effectiveness: DeLLMGuard preserves business source code and outperforms the closed-source BIC baseline on the primary non-proxy dataset.On non-proxy cases, BIC strict correctness is 13.49%, 9.09%, and 7.33%, versus DeLLMGuard’s 12.90%, 3.52%, and 3.23%.
  • Effectiveness: 23.5% to 6.6%: complete DeLLMGuard reduces overall Correct, while Correct+Partial falls from 42.3% to 13.4% across 387 cases and three models.The defense reduces root-cause correctness while preserving the transformed analysis environment.
  • Failure Analysis: Agents frequently follow added runtime indirections, yet their downstream bottlenecks differ: GPT struggles with business-code recovery, Claude with decompilation, and GLM with semantic reconstruction.FIC is especially difficult: only 43 agent-cases find the Factory address, 15 follow it, and none query it for source code.
  • Ablation Study: FIC produces the largest consistent non-proxy reduction, DLC adds further reduction, while PDC and CPC have smaller or inconsistent effects.CPC alone has a small and inconsistent effect across models and settings.
  • Adversarial Robustness: Mechanism knowledge does not restore baseline performance overall, although some informed proxy attacks improve isolated partial or strict outcomes.The reported above-baseline proxy values are characterized as isolated changes rather than stable reversal.

6 Related Work

Related work applies LLM-based vulnerability detection to general software repositories and smart contracts, combining autonomous exploration, execution, program analysis, fine-tuning, and multi-agent methods.

  • LLM-based Vulnerability Detection in General Software: General-software systems have progressed from isolated snippets to repository-level auditing and reproducible executable-proof-of-concept generation.RepoAudit performs autonomous repository exploration and path validation, while CyberGym evaluates agents in reproducible environments built from real vulnerabilities.
  • LLM-based Vulnerability Detection in Smart Contracts: Smart-contract vulnerability systems combine LLM semantic reasoning with program analysis, domain-specific fine-tuning, multi-agent collaboration, or rule-based methods.GPTScan, iAudit, and ABAuditor represent these complementary approaches.

7 Limitations

The evaluation is bounded by adaptive attackers, isolated simulated deployments, and coverage of 387 contracts from SCONE-bench with three LLM agents.

  • Adaptive attackers: Future agents or analysis tools may reduce DeLLMGuard’s defensive effect because its introduced relations remain recoverable through decompilation, tracing, and transaction analysis.The adversarial setting also discloses the complete transformation mechanism.
  • Simulated deployment environment: Transformed vulnerable contracts are evaluated only in isolated blockchain environments rather than public networks for safety and ethical reasons.PARSEC reproduces historical state, execution, RPC access, and address-based interaction without live user assets.
  • Evaluation coverage: Evaluation coverage is limited to 387 real vulnerable contracts from SCONE-bench and three LLM agents.

8 Conclusion

The paper presents DeLLMGuard as a deployment framework that preserves source disclosure while separating disclosed source from runtime execution through multiple guard contracts. Across 387 vulnerable contracts and three LLM agents, it reduces overall vulnerability-root-cause correctness from 23.5% to 6.6%, with cross-contract recovery remaining a key challenge.

  • DeLLMGuard preserves public source disclosure while separating disclosed source code from the runtime execution path through multiple guard contracts.
  • 23.5% to 6.6%: DeLLMGuard reduces overall correctness on 387 real-world vulnerable contracts evaluated with three LLM agents.
  • Agents often recover downstream contracts but still fail to identify vulnerabilities, showing that cross-contract recovery can reduce automated scanning without hiding source code.

Ethical Considerations

The evaluation uses previously disclosed vulnerabilities and isolated local environments rather than new live-contract attacks. DeLLMGuard preserves public review information and supports authorized auditing while avoiding source hiding and access control.

  • The evaluation uses historical vulnerabilities and exploit proofs of concept from DeFiHackLabs and SCONE-bench rather than undisclosed targets or new vulnerabilities in live contracts.
  • All transformed contracts and agent interactions run in isolated PARSEC environments that reproduce historical chain state without exposing public blockchains or real user assets.
  • DeLLMGuard targets large-scale malicious LLM scanning, not legitimate security analysis, while keeping source code, bytecode, storage, transactions, and traces accessible.
  • Authorized auditors can obtain the deployment relations introduced by DeLLMGuard, preserving information needed for security review without source hiding or access control.

A Adaptive-Adversary Prompt

The adaptive-adversary prompt describes mechanically rewritten source layouts and deployment scaffolding while asserting semantic equivalence to the historical runtime. Depending on whether the original entry point was a proxy, the transformations add factory disclosure, delegatecall indirection, comment injection, and possibly proxy storage-slot shadowing.

  • Adaptive-Adversary Prompt: The transformed deployed runtime is asserted to remain semantically equivalent to the historical contract at the historical fork block.
  • Adaptive-Adversary Prompt: The disclosed source layout, deployment scaffolding, runtime indirection, and selected proxy-storage entries may differ from the historical deployment.
  • Non-proxy originals: Non-proxy originals receive CREATE2 factory disclosure, cross-contract delegatecall splitting, and comment injection.
  • Proxy originals: Proxy originals receive the same three transformations plus proxy storage-slot shadowing.
  • Transform 1 — CREATE2 factory disclosure: CREATE2 factory disclosure presents the business contract as a generated factory's child, while the factory contains deployment scaffolding rather than business logic or vulnerabilities.
  • Transform 2 — Cross-contract delegatecall split: Delegatecall shells add an indirection hop and forward calls to a private-constant address without adding business logic or vulnerabilities.
  • Transform 3 — Comment injection: Injected ARSD comments are perturbations rather than operator instructions and do not change the analysis task.
  • Transform 4 — Proxy storage-slot shadowing: Decoy proxy slots point to plausible-looking scaffolding contracts rather than the business logic.
Loading 2608.28400v1…