PaperlayerView exact revision on arXiv

Source-linked AI summary

Checking Smart Contracts with Structural Code Embedding

Zhipeng Gao, Lingxiao Jiang, Xin Xia, David Lo, John Grundy

arXiv:2001.07125v1cs.SE

TL;DR

High-value smart contracts attract attackers, while existing bug detectors often rely on manually specified patterns. SMARTEMBED represents Solidity code and bugs as structural embeddings and compares them for clone detection, bug detection, and validation; the authors report clone, bug-detection, and validation results, including a clone ratio around 90%.

  • Problem

    Many smart-contract bug detectors require expert-defined patterns or specifications, which can be costly and slow to update as new bugs and exploits arise.

    Source: §1 · p. 1

  • Method

    SMARTEMBED encodes Solidity code and known bug patterns as vectors that represent lexical, syntactic, and some semantic information, then checks similarity among code elements.

    Source: §1 · p. 1

  • Results

    SMARTEMBED reports a clone ratio around 90%, more than 1,000 clone-related bugs identified, and results comparable to Deckard and SmartCheck across its tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The approach can support efficient contract checking as code and bug patterns change and can help validate contracts against known bugs.

    Source: §7 · p. 16

    The bug database contains 52 contracts covering 10 bug types, and the collected Ethereum source-code set is incomplete.

    Source: §6 · p. 16

Abstract

from arXiv · show

Smart contracts have been increasingly used together with blockchains to automate financial and business transactions. However, many bugs and vulnerabilities have been identified in many contracts which raises serious concerns about smart contract security, not to mention that the blockchain systems on which the smart contracts are built can be buggy. Thus, there is a significant need to better maintain smart contract code and ensure its high reliability. In this paper, we propose an automated approach to learn characteristics of smart contracts in Solidity, which is useful for clone detection, bug detection and contract validation on smart contracts. Our new approach is based on word embeddings and vector space comparison. We parse smart contract code into word streams with code structural information, convert code elements (e.g., statements, functions) into numerical vectors that are supposed to encode the code syntax and semantics, and compare the similarities among the vectors encoding code and known bugs, to identify potential issues. We have implemented the approach in a prototype, named SmartEmbed. Results show that our tool can effectively identify many repetitive instances of Solidity code, where the clone ratio is around 90\%. Code clones such as type-III or even type-IV semantic clones can also be detected accurately. Our tool can identify more than 1000 clone related bugs based on our bug databases efficiently and accurately. Our tool can also help to efficiently validate any given smart contract against a known set of bugs, which can help to improve the users' confidence in the reliability of the contract. The anonymous replication packages can be accessed at: https://drive.google.com/file/d/1kauLT3y2IiHPkUlVx4FSTda-dVAyL4za/view?usp=sharing, and evaluated it with more than 22,000 smart contracts collected from the Ethereum blockchain.

1 INTRODUCTION

High-value smart contracts create incentives for attackers, while many existing detectors depend on expert-written rules that can be costly to update. The paper proposes structural code embeddings and similarity checking for Solidity code.

1 INTRODUCTION

  • Existing bug detectors often require human-defined patterns or specifications, making new checkers costly and slow to build as bugs and exploits emerge.The paper frames this as a challenge given the high stakes of smart contracts and the race between attackers and defenders.Source: §1 · p. 1
  • SMARTEMBED encodes code and bug patterns, including lexical, syntactic, and some semantic information, as vectors, then checks similarity among code elements at different granularities.The approach targets clone detection, bug detection, and validation against known bugs.Source: §1 · pp. 1–2
  • Across clone detection, bug detection, and validation, SMARTEMBED reports results comparable to Deckard and SmartCheck, a clone ratio around 90%, and more than 1,000 clone-related bugs.The evaluation used 22,725 contracts collected from Ethereum.Source: §1 · p. 2
  • The study collected 22,725 verified Solidity contracts and 17 well-known buggy contracts for its prototype evaluation.The contracts were collected from the Ethereum blockchain.Source: §1 · p. 2
  • SMARTEMBED reports low false-positive rates when checking contracts for bugs similar to known ones, with efficient queries for clones or bugs.The approach is designed to check against known bug patterns.Source: §1 · p. 2

2 RELATED WORK

Related work spans smart-contract security tools, program embeddings, and similarity-based clone and bug detection. The paper positions structural Solidity embeddings as a way to find similar code and to check against known bugs without handcrafted features.

2.1 Smart Contract and Security Problems

  • Most prior smart-contract security tools target specific vulnerabilities and require manually constructed patterns or specifications; this work proposes structural code embeddings for bug detection and validation by similarity.The authors describe their approach as the first to apply this technique to smart contracts.Source: §2.1 · p. 2

2.2 Word Embedding and Code Similarity

  • Unlike prior code-embedding methods discussed here, SMARTEMBED serializes Solidity parse trees to embed program elements at different levels.The paper presents this as its first application of code embeddings to Ethereum smart contracts.Source: §2.2 · p. 2

2.3 Clone Detection, Bug Detection, and Code Validation

  • Clone detection, bug detection, and validation can each be framed as searching for similar code, while existing clone and conventional bug tools use varied representations or require curated patterns.The paper situates its embedding approach alongside text-, token-, tree-, graph-, semantic-, and deep-learning-based clone methods.Source: §2.3 · pp. 2–3
  • SMARTEMBED learns code and bug representations from known bugs without handcrafted features, but its bug detection and validation remain limited to instances similar to those known bugs.With sufficiently comprehensive code and known bugs, the approach may be applicable to both tasks; conventional tools' potential bugs can also supplement the database.Source: §2.3 · p. 3

3 APPROACH

SmartEmbed parses Solidity contracts into normalized, structurally informed token streams, embeds code fragments as vectors, and compares those vectors for clone detection, bug detection, and contract validation. Its pipeline applies this process to a dataset of 22,725 verified contracts.

3.1 Data Collection

  • 22,725 verified contracts were collected from EtherScan, averaging around 6 individual contracts, 27 functions, 85 statements, and 323 lines of code per smart contract.The reported collection was current as of April 20, 2018.Source: §3.1 · p. 4

3.2 Parsing

  • AST-based serialization captures structural relations, including containment and neighboring nodes, and some data-flow information for downstream applications.Serialization varies by contract, function, and statement level, using relevant tree nodes and identifiers to capture branch or loop conditions and defining/using relationships.Source: §3.2 · pp. 4–5

3.3 Normalization

  • Normalization removes selected semantic-irrelevant details by replacing single-character variables and constants, removing non-essential punctuation, and splitting camel-case identifiers while retaining them as tokens.The resulting streams covered 22,725 contracts, 631,261 functions, and 1,944,513 statements; 1.2 GB of clean text contained 119,568 tokens.Source: §3.3 · p. 5

3.4 Code Embedding Learning

  • SmartEmbed applies Word2Vec and FastText to normalized streams, selecting FastText for later experiments because it performed better on syntactic tasks and can generate vectors for out-of-vocabulary words.FastText represents a word by summing its character n-gram vectors; the training corpus contained 119,568 unique tokens.Source: §3.4 · p. 5

3.4.1 Token Embedding

  • Token embeddings are fixed at 150 dimensions, half the 308 Solidity grammar node types, and serve as pretraining for higher-level code embeddings.The authors leave sensitivity analysis of vector dimensions for future work.Source: §3.4.1 · p. 5

3.4.2 Higher Level Embedding

  • A code fragment’s embedding sums the embeddings of its constituent tokens, producing a fixed-length vector for each contract, function, or statement.The authors chose summation to capture semantic features as well as code size.Source: §3.4.2 · p. 6

3.5 Embedding Matrix Building

  • SmartEmbed stacks contract-, function-, and statement-level vectors into separate embedding matrices, with one row per code element.The reported matrices contain 22,725 contract rows, 631,261 function rows, and 1,944,513 statement rows, each with 150 dimensions.Source: §3.5 · p. 6

3.6 Similarity Checking

  • Similarity scores are compared with task-specific thresholds to identify clone pairs, code resembling known bugs, or vulnerable statements in a new contract.The thresholds may differ across clone detection, bug detection, and contract validation.Source: §3.7 · p. 6, §3.6 · p. 6

4 EMPIRICAL EVALUATION

The evaluation examines SmartEmbed’s code embeddings and similarity checking for clone detection, bug detection, and contract validation. The reported embedding visualization clusters tokens with similar syntactic and lexical meanings.

4.1 Code Embedding Evaluation

  • Tokens with similar syntactic and lexical meanings cluster together in the embedding visualization, including operators and related token pairs.Examples include arithmetic and comparison operators, as well as “args” and “dynargs” and “StringLiteral” and “decimals.”Source: §4.1 · p. 6, Figure 3 · p. 7

4.2 Similarity Checking Evaluation

  • The similarity evaluation compares SmartEmbed with Deckard for clone detection and SmartCheck for bug detection and contract validation.The authors frame the evaluation around those three tasks.Source: §4.2 · p. 7
  • The evaluation asks six questions covering clone detection, bug detection, distinguishing bugs from fixes, structural and semantic information, contract validation, and efficiency.These questions define the reported evaluation scope; this request’s passages do not provide their results.Source: §4.2 · p. 7

4.3 RQ-1: Clone Detection Evaluation

Clone detection is important for smart-contract quality assurance because contracts are often irreversible and self-contained. The evaluation compares SmartEmbed with Deckard at contract, function, and statement levels, then examines detected clone pairs.

4.3.1 Experimental Setup

  • The evaluation compares similarity-based clone detection at three granularities, using 200 sampled contracts for function- and statement-level queries because exhaustive pairwise comparisons were too expensive.Those samples contained 5,307 functions and 16,350 statements, compared against all functions and statements in the repository; contract-level comparison used a 22,718 × 22,718 matrix after seven parsing-error cases were removed.Source: §4.3.1 · p. 7

4.3.2 Experimental Results

  • SMARTEMBED clone ratios were 0.39/0.85 at contract level, 85%/91% at function level, and 0.82/0.93 at statement level for thresholds 1.0/0.95, with fewer clones overall than Deckard.Deckard’s clones involved more than 6.6 million lines of code at threshold 1.0 and more than 96% clone ratio at threshold 0.95; SmartEmbed’s function-level counts were 23,087 (85%) and 24,640 (91%).Source: Table 2 · p. 8, §4.3.2 · p. 8
  • Most SmartEmbed-detected clones also appeared in Deckard’s results, and overlap was more stable at function level, where SmartEmbed tolerates non-essential syntactic differences.The comparison uses the reported overlap summary and Venn graphs for contract-, function-, and statement-level clones.Source: Figure 4 · p. 8, Figure 5 · p. 8, Figure 6 · p. 8
  • After excluding same-creator pairs, the contract-level clone ratio remained 51% at threshold 0.95; 68.3% of contracts contained code blocks supporting ERC20 compliance.The paper attributes the latter finding to template contracts contributing to cloning in Ethereum.Source: §4.3.2 · p. 9

4.3.3 Examples of clone detection

  • SmartEmbed detected examples of type-III or even type-IV semantic clones that Deckard can miss, while preserving identifier differences helped avoid an accidental clone match.The examples were manually checked; the paper describes the differing data structures in Deckard’s unmatched “addCompany” functions.Source: §4.3.3 · p. 9, Figure 7 · p. 9, Figure 8 · p. 9

4.4 RQ-2: Bug Detection Evaluation

SMARTEMBED searches statement embeddings against a database of known buggy statements, reporting similar statements for manual validation. Its results show high precision for simpler clone types, while broader candidate recovery trades precision against volume.

4.4.1 Experimental Setup

  • For each of 63 buggy statements from 32 vulnerable contracts, the system searches contract statements above a similarity threshold and manually checks reported candidates.The 52 known vulnerable contracts covered 10 vulnerability types; 20 contracts were reserved for contract validation.Source: §4.4.1 · p. 10
  • The bug database is built from known buggy statements, whose vectors are composed by summing the vectors for relevant tokens.The 63 buggy statements are mapped to 150-dimensional vectors.Source: §4.4.1 · p. 10
  • The evaluation treats exact fragments, syntactically equivalent fragments, and syntactically similar fragments with edits as Type-I, Type-II, and Type-III/IV clone-related bugs.Reported candidates were validated by comparing them with real bug lines.Source: §4.4.1 · p. 11

4.4.2 Experimental Results

  • Most SMARTEMBED bug candidates are Type-II clones, consistent with developers introducing clone-related bugs by copying and pasting code.The authors also report that SMARTEMBED can achieve 100% precision for Type-I and Type-II clone-related bugs.Source: §4.4.2 · p. 11
  • SMARTEMBED achieves 100% precision for Type-I and Type-II clone-related bugs, but performance drops for Type-III/IV clones.Lowering the similarity threshold to find more Type-III/IV matches may also introduce false positives.Source: §4.4.2 · p. 11
  • SMARTEMBED reports more bugs than SmartCheck in the comparison: SmartCheck flags 697 of the 1,163 manually validated statements.SmartCheck was run only on manually validated contracts associated with those statements, using the collected bug patterns.Source: §4.4.2 · p. 11
  • At similarity threshold 0.91, SMARTEMBED reports 1,052 statements as potential bugs with 95% precision.The authors suggest a threshold between 0.90 and 0.91 may suit the task; at 0.90, it reports 1,311 potential bugs, 1,163 validated as real bugs, at 88.7% precision.Source: §4.4.2 · p. 11, Table 6 · p. 12

4.4.3 Examples of bug detection

  • SMARTEMBED can use a sample bug to generate embeddings for recognizing similar bugs, whereas encoding a new bug type in SmartCheck requires extra specification work.The authors qualify this advantage as dependent on embedding relevant structural and semantic code information well.Source: §4.4.3 · p. 12

4.5 RQ-3: Practical Analysis

The practical analysis tests whether SMARTEMBED distinguishes fixed statements from their original buggy versions. In the five contracts with identified fixes, the authors report that a 0.90 threshold correctly identified every fixed contract as not vulnerable.

4.5.1 Experimental Setup

  • The authors found fixed versions for 5 of 52 buggy smart contracts by investigating contract creators’ histories and comparing corresponding buggy and fixed statements.They manually pinpointed fixed statements and estimated their similarity to the original buggy statements.Source: §4.5.1 · p. 12

4.5.2 Experimental Results

  • At similarity threshold 0.90, SMARTEMBED correctly identifies all five fixed smart contracts as not vulnerable.The authors attribute this to statement-level embeddings: even small changes within a buggy statement reduce similarity scores.Source: §4.5.2 · p. 12, §4.5.1 · p. 12
  • The analysis reports a significant drop in similarity scores between fixed statements and their original buggy statements.The authors interpret this as further evidence that SMARTEMBED separates the real buggy statement from the fixed statement.Source: §4.5.2 · p. 12

4.5.3 Bug and Bug Fix Examples for Practical Analysis

  • In the batchTransfer() example, the fix replaces unchecked multiplication with SafeMath multiplication, changing the buggy statement and function signatures.The original multiplication could overflow and transfer tokens exceeding the sender’s balance.Source: §4.5.3 · p. 12

4.6 RQ-4: Ablation Analysis

The ablation compares SMARTEMBED with BASICEMBED, which removes structural and semantic relations from statement tokenization. The reported results favor SMARTEMBED, while the BASICEMBED example illustrates how statement context can help distinguish bug matches from false positives.

4.6.2 Experimental Results

  • BASICEMBED reports over 30,000 bugs with overall precision around 5%, leaving most reported candidates as false positives.Because validating all candidates was too expensive, the evaluation manually sampled 300 contracts.Source: §4.6.2 · p. 13
  • SMARTEMBED stably and substantially outperforms BASICEMBED in precision across similarity thresholds.BASICEMBED removes structural and semantic relations, retaining only the simple statement token sequence.Source: §4.6.2 · p. 13, §4.6.1 · p. 13

4.6.3 Bug Detection Example for the Ablation Analysis

  • In the Rubixi example, BASICEMBED reports 10,679 potential bugs for “owner = msg.sender,” often matching identical tokens without relevant context.SMARTEMBED can encode the function signature and contract ancestor in the statement embedding to reduce false positives and identify real bugs.Source: §4.6.3 · pp. 13–14
  • The authors conclude that structural and semantic information added to SMARTEMBED has significant benefits for overall performance.They also report that 87% of BASICEMBED’s reported bugs have a similarity threshold of 1.0, mostly corresponding to Type-I clone-related bugs.Source: §4.6.2 · p. 13, §4.6.3 · p. 14

4.7 RQ-5: Contract Validation Evaluation

The evaluation tests whether SMARTEMBED can identify bugs resembling known bugs in buggy and audited bug-free contracts. Results indicate that it captures clone-related bugs with low false-positive rates, with detection and false positives changing as the similarity threshold changes.

4.7 RQ-5: Contract Validation Evaluation

  • The evaluation targets pre-deployment checking because deployed contracts are immutable, and the bug database can be extended with additional known bugs.The authors clarify that their approach is not formal verification.Source: §4.7 · p. 14

4.7.1 Experimental Setup

  • The test set comprised 20 buggy contracts and 20 audited bug-free contracts, with 2,857 statements and 45 labelled bugs; SMARTEMBED and SmartCheck were compared.Any vulnerability reported in the audited contracts was counted as a false positive.Source: §4.7.1 · p. 14, Table 9 · p. 15

4.7.2 Experimental Results

  • SMARTEMBED captured known-like bugs with low false-positive rates and identified clone-related bugs more accurately than SmartCheck; at threshold 0.90 it reported 8 false positives, falling to 0 at 0.95.The comparison enabled SmartCheck only for the bug types listed in Table 4; even then, SmartCheck had a 9.9% false-positive rate.Source: §4.7.2 · pp. 14–15, Table 10 · p. 15
  • Lowering the threshold from 0.95 to 0.90 increased the number of clone-related bugs discovered from 27 to 36.The authors suggest that code changes made to cloned code may reduce its similarity, making lower thresholds useful for detection.Source: §4.7.2 · p. 14
  • At threshold 0.85, false negatives fell to 0 while false positives rose to 116; the authors report that its F1 score remained much better than SmartCheck’s.The authors state that this threshold identified all labelled bugs in the evaluation.Source: §4.7.2 · pp. 14–15, Table 10 · p. 15

4.8 RQ-6: Time Cost Analysis

The time-cost analysis measures embedding training, pairwise similarity, and query costs for clone detection, bug detection, and contract validation. The authors conclude that clone and bug queries are efficient for practical use.

4.8 RQ-6: Time Cost Analysis

  • Checking one contract took 0.26 ms for clone detection, 2.3 ms for bug detection, and 4.7 ms for contract validation.Computing the corresponding similarity matrices took 6.05 s, 53.22 s, and 4.7 ms, respectively.Source: §4.8 · p. 15
  • Training the code embeddings took about a day on the reported dataset, while an average pairwise similarity calculation took around 250 ns.The reported machine used an Intel Xeon CPU E5-2640 v4 at 2.40 GHz.Source: §4.8 · p. 15
  • The authors conclude that SMARTEMBED’s clone or bug queries are efficient for practical uses.They note that runtime depends on the sizes of the contract codebase and bug database.Source: §4.8 · p. 15

5 DISCUSSION

The discussion reports developer feedback on clone and bug reports and describes a standalone web application. It also considers integrating SMARTEMBED into familiar tools and development environments.

5 DISCUSSION

  • SMARTEMBED can report code clones at contract, function, or statement level, whereas Etherscan’s cited “find similar contract” function finds only exact matches.One developer said function-level reports might be useful; another saw potential value for authors of widely cloned contracts.Source: §5 · p. 15
  • Some developers saw bug reports as useful for checking vulnerabilities, while another cautioned that insecure-looking patterns can be appropriate depending on contract intent.The comments distinguish a reported pattern from a judgment about whether it makes a contract insecure in context.Source: §5 · p. 15
  • The authors implemented SMARTEMBED as a standalone web application where Solidity developers can paste source code to find repetitive code and clone-related bugs.The source code and experimental contract data are available in the authors’ GitHub repository.Source: §5 · p. 15
  • Developers suggested integration with Etherscan or IDEs such as Remix and Visual Studio Code; the authors say similarity checking could support real-time IDE use.The authors state that they plan to pursue these suggestions.Source: §5 · p. 15

6 THREATS TO VALIDITY

The authors identify threats related to code representation, dataset coverage, and external evaluation. These boundaries include possible size-related bias, incomplete contract and bug data, and reliance on one comparison tool for validating clone-related bugs.

Internal Validity

  • The intuitive code-snippet representation may bias clone detection across code sizes, and the authors plan to explore alternative embedding approaches.Examples include representations based on control-flow graphs, AST paths, execution traces, and API sequences.Source: §6 · p. 16

Data Validity

  • The 22,725-contract Etherscan source-code dataset was incomplete, and the bug database covered 10 Solidity-relevant bug types; omitted types may cause false negatives.The authors also note that the selected contracts may not be sufficiently diverse or representative.Source: §6 · p. 16

External Validity

  • Clone-related bugs were validated only against SmartCheck, which can itself have false positives and false negatives; the authors plan broader evaluations.The authors also acknowledge that their user-study sample was insufficient.Source: §6 · p. 16

7 SUMMARY

SMARTEMBED uses structural code embedding and similarity checking for clone detection, bug detection, and contract validation. Evaluated on more than 22,000 Ethereum Solidity contracts, it identifies repetitive code, semantic clones, and clone-related bugs.

7 SUMMARY

  • Around 90% clone ratio: SMARTEMBED identifies many instances of repetitive Solidity code and detects more semantic clones accurately than Deckard.The evaluation used more than 22,000 Solidity smart contracts from the Ethereum blockchain.Source: §7 · p. 16
  • SMARTEMBED applies structural code embedding and similarity checking to clone detection, bug detection, and contract validation for smart contracts.Source: §7 · p. 16
  • More than 1,000 clone-related bugs are identified efficiently and accurately using the authors’ bug databases.The authors say this can enable efficient checking as code and bug patterns change.Source: §7 · p. 16
  • The authors say SMARTEMBED’s capabilities can facilitate contract validation in practice.Source: §7 · p. 16

Related papers

Loading 2001.07125v1…