PaperlayerView exact revision on arXiv

Source-linked AI summary

Defining Smart Contract Defects on Ethereum

Jiachi Chen, Xin Xia, David Lo, John Grundy, Daniel Xiapu Luo, Ting Chen

arXiv:1905.01467v3cs.SE

TL;DR

Smart contracts are immutable after deployment, while Ethereum’s distinctive features create defect types that require systematic study beyond security-focused analyses. The paper analyzes StackExchange posts and real-world contracts, defines and validates 20 defects, and releases a labeled dataset. More than 99% of 587 manually labeled contracts contain at least one defined defect, and the study summarizes five impacts to guide understanding and removal priority.

  • Problem

    Because deployed smart contracts cannot be patched and contain Ethereum-specific features, prior work lacked a comprehensive, practitioner-validated account of defects across five quality aspects.

  • Method

    The authors manually analyzed Ethereum StackExchange posts and real-world contracts, defined 20 defects, surveyed practitioners, labeled 587 contracts, and released the dataset.

  • Results

    More than 99% of 587 manually labeled smart contracts contained at least one defined defect, and the study summarized five impacts of contract defects.

  • Takeaways & Limitations

    The defect definitions, impacts, and dataset can help developers prioritize defect removal and help researchers develop smart-contract defect-detection tools.

  • Takeaways & Limitations

    The keyword-based collection may miss contract defects, and the manually labeled dataset may still contain errors from misunderstanding contracts.

Abstract

from arXiv · show

Smart contracts are programs running on a blockchain. They are immutable to change, and hence can not be patched for bugs once deployed. Thus it is critical to ensure they are bug-free and well-designed before deployment. A Contract defect is an error, flaw or fault in a smart contract that causes it to produce an incorrect or unexpected result, or to behave in unintended ways. The detection of contract defects is a method to avoid potential bugs and improve the design of existing code. Since smart contracts contain numerous distinctive features, such as the gas system. decentralized, it is important to find smart contract specified defects. To fill this gap, we collected smart-contract-related posts from Ethereum StackExchange, as well as real-world smart contracts. We manually analyzed these posts and contracts; using them to define 20 kinds of contract defects. We categorized them into indicating potential security, availability, performance, maintainability and reusability problems. To validate if practitioners consider these contract as harmful, we created an online survey and received 138 responses from 32 different countries. Feedback showed these contract defects are harmful and removing them would improve the quality and robustness of smart contracts. We manually identified our defined contract defects in 587 real world smart contract and publicly released our dataset. Finally, we summarized 5 impacts caused by contract defects. These help developers better understand the symptoms of the defects and removal priority.

1 INTRODUCTION

Smart contracts run on Ethereum’s blockchain and cannot be patched after deployment, making pre-deployment defect detection critical. This study systematically defines contract defects across five aspects, validates practitioners’ views, and examines defects in real-world contracts.

  • Smart contracts are Turing-complete programs whose compiled bytecode resides on Ethereum’s blockchain, where consensus protocols ensure execution.
  • Because blockchain data is immutable, deployed smart contracts cannot be patched when bugs are detected.
  • The study defines contract defects across security, availability, performance, maintainability and reusability, extending prior work beyond security-focused analysis.
  • The authors analyzed 17,128 Ethereum StackExchange posts, conducted an online survey with 138 responses from 32 countries, and manually labeled 587 contracts.
  • More than 99% of the 587 labeled smart contracts contained at least one defined defect, and five impacts were summarized to clarify symptoms and removal priority.
  • The released dataset is intended to support future smart-contract analysis and testing, while the findings provide a basis for practical defect-detection tools.

2 BACKGROUND

Ethereum smart contracts combine blockchain execution, gas-based resource accounting, Solidity language features, and token standards. The background illustrates these mechanisms through a Solidity gambling contract containing multiple defects.

  • Smart contracts: Ethereum smart contracts are immutable blockchain programs executed by the Ethereum Virtual Machine and identified by unique 160-bit hexadecimal addresses.
  • Features of Smart Contracts: Gas cost equals gas cost multiplied by gas price, while gas limits terminate execution with an out-of-gas exception when the limit is exceeded.
  • Solidity: Solidity supports version pragmas, modifiers, constructors, fallback functions, and Ether-transfer APIs with different failure and gas-limit behavior.
  • Solidity example: The example Gamble contract includes defects such as transaction-state dependency, hard-coded addresses, strict balance equality, block-information dependency, nested calls, and missing return statements.
  • ERC-20 Token: ERC-20 standardizes token behavior through nine interfaces, three optional interfaces, and two events that compliant tokens must implement consistently.

3.1 Motivation

Smart contracts cannot be patched after deployment, and their gas system and decentralized features create defect types that differ from those in traditional software. The paper therefore defines Ethereum-specific defects with definitions, examples, and possible solutions.

  • Smart contracts cannot be patched after deployment, so detecting and removing defects is used to support contract robustness.
  • Ethereum’s gas system and decentralized features motivate defining smart-contract defects that are specialized beyond traditional software defects.
  • The paper presents definitions, examples, and possible solutions for the newly defined Ethereum-specific contract defects.

3.2 Approach

The study combines keyword-filtered Ethereum StackExchange posts, card sorting, and manual contract labeling to define and organize smart contract defects.

  • Keyword Filtering: 17,128 StackExchange posts were crawled, then 66 keywords filtered them to 4,141 posts for further analysis.
  • Keyword Filtering: Researchers removed posts unrelated to Solidity contract defects, including topics about web3, development environments, wallets, or functionality.
  • Open Card Sorting: Each retained post became a card containing its defect title, description, and comments for collaborative categorization.
  • Open Card Sorting: Two authors independently categorized the remaining 80% of cards, reconciled differences, and achieved an overall Cohen’s Kappa of 0.82.
  • Defining Contract Defects: The analysis summarized 6 high-level root causes and 16 detailed contract defects from the StackExchange cards.
  • Defining Contract Defects: Examples included Deprecated APIs, Block Info Dependency, and four additional defects identified by comparing similar real-world contracts.
  • Defining Contract Defects: The researchers examined 578 real-world contracts sampled from 17,013 verified Etherscan contracts and ultimately defined 20 contract defects.

3.3 Results

The paper defines 20 contract defects across security, availability, performance, maintainability, and reusability, illustrating their symptoms and consequences. The defects include exploitable vulnerabilities, gas inefficiencies, locked funds, maintenance risks, and barriers to code reuse.

  • Five defect categories: The study organizes 20 contract defects into five categories: security, availability, performance, maintainability, and reusability.The security category covers defects that may be exploited for financial gain or attacks, while availability defects can cause errors or financial loss for callers.
  • Security defects: Unmatched type assignment can cause overflow because Solidity does not throw an exception when integer values exceed their maximum.The passage contrasts uint8 and uint256 ranges and identifies unchecked increments as a common source of overflow.
  • Security and performance defects: Re-entrancy permits interrupted execution through external calls, while nested calls inside unbounded loops can exceed the gas limit.A CALL operation costs 9000 gas for a non-zero value transfer, and high iteration counts make total cost difficult to bound.
  • Availability defects: Availability defects include greedy contracts that receive Ether but provide no withdrawal mechanism, unmatched ERC-20 behavior, missing reminders, and missing return statements.Missing return statements can cause callers to rely on compiler-added defaults and produce unpredictable execution behavior.
  • Performance defects: Performance defects include unused statements, public functions that unnecessarily copy array arguments to memory, and byte[] usage that costs more gas than bytes.Replacing byte[] with bytes can save a small amount of gas per call and potentially much more across repeated calls.
  • Maintainability and reusability defects: Maintainability defects include hard-coded addresses and missing interrupters, while reusability defects include deprecated APIs and unspecified compiler versions.Hard-coded addresses cannot be modified after deployment; deprecated APIs and unspecified compiler versions can create future refactoring or compilation problems.

4 RQ2: PRACTITIONERS’ PERSPECTIVE

Practitioners broadly regarded the defined contract defects as important, although views varied for defects tied to internal use, compiler versions, and newer Solidity releases. Survey feedback supported the defects’ practical relevance and suggested that removing them could improve smart-contract quality and robustness.

  • Survey design and respondents: 138 responses from 32 countries produced an average defect-importance score of 4.22.The survey also collected 84 comments from respondents.
  • Overall assessment: Almost all contract defects received scores above 4, indicating broadly positive practitioner feedback.“Unspecified Compiler Version” scored 3.84 but was still discussed as important for code reuse.
  • Variation in perceived importance: “Missing Interrupter,” “Missing Reminder,” and “Unspecified Compiler Version” received the most negative feedback.Respondents linked some lower ratings to internal company use, distrust of back-door mechanisms, or perceived similarity among minor compiler versions.
  • Variation in perceived importance: Developers still considered several defects potentially harmful because deployed contracts may use older Solidity versions or operate on permissionless Ethereum networks.The authors argue that defects removed from newer Solidity versions can remain relevant when developers deploy contracts using older versions.
  • Practical feedback: Respondents described the defect summary and suggested controls as useful for practitioners, researchers, and avoiding flaws.Comments also characterized the defects as capable of causing serious problems and identified compiler-tool integration as useful.

5 RQ3: DISTRIBUTION AND IMPACT OF CONTRACT DEFECTS

The authors labeled 587 real-world Ethereum contracts to characterize defect distributions and assign five impact levels. Defects were widespread, although critical unwanted behaviors were less common than lower-level maintainability, reuse, or user-impact problems.

  • Dataset and labeling: 587 real-world contracts were manually labeled to establish defect distributions and ground truth for future detection studies.Two experienced authors independently labeled the contracts and reached an overall Kappa value of 0.71 before resolving disagreements.
  • Impact framework: The impact framework combines contract severity, attacker behavior, and external user influence into five impact levels.User influence includes caller errors, gas waste, and difficulties understanding or reusing code.
  • Impact framework: Impact levels 1–2 represent critical unwanted behavior, level 3 represents major unwanted behavior such as lost Ether, and levels 4–5 represent trivial problems.The lowest levels include issues such as low readability that do not affect normal contract execution.
  • Distribution and impact: Impact 1–5 defects occurred in 2.90%, 7.16%, 27.09%, 93.86%, and 99.14% of contracts, respectively.A single contract can contain multiple defects with different impact levels.
  • Distribution and impact: Unspecified Compiler Version and Missing Interrupter were highly prevalent, appearing in 90.63% and 89.1% of contracts, respectively.The former increased code-reuse workload, while the latter raised questions about emergency control and user trust.
  • Distribution and impact: 99.82% of contracts contained at least one impact-4 or impact-5 defect, whereas 7.33% contained defects capable of critical unwanted behavior.Overall, 32.03% contained defects at impact levels 1–3, which can lead to unwanted behaviors.
  • Distribution and impact: ERC-20 contracts represented 36.11% of the dataset, while 21.22% of ERC-20-related contracts did not strictly follow the standard.The authors found no High Gas Consumption Data Type defects, but caution that the limited dataset may miss them.

6 DISCUSSION

The discussion presents implications for research, practitioners, and educators, then outlines detection challenges and possible methods for bytecode and source code. It emphasizes that bytecode detection is valuable but constrained by lost semantic information and irregular defect patterns.

  • Implications: The study reviews existing tools that detect some of the 20 identified contract defects.The review covers papers published from 2016 to 2019 and reports the results in Table 6.
  • Implications: Detection tools could be integrated into Solidity compilers and development tools because surveyed company managers expressed interest in them.The paper frames such tools as a direction for future development.
  • Challenges in Detection: Bytecode-level analysis loses semantic information, making some defects difficult to detect automatically.EVM optimization can remove or alter source-level information, such as whether a function explicitly includes a return value.
  • Challenges in Detection: Bytecode detection is important because Ethereum exposes contract bytecode while fewer than 1% of contracts have open source code.Regular bytecode patterns can support detection of defects such as Nested Call, whereas Missing Interrupter lacks a regular pattern.
  • Detection Methods: The proposed bytecode methods use control-flow graphs, opcode patterns, function hashes, and instruction relationships to detect selected defects.Examples include identifying Nested Call through unbounded loops containing CALL instructions and Strict Balance Equality through BALANCE followed by EQ at a conditional jump.
  • Detection Methods: Seven remaining defects require source-code detection because compilation removes or optimizes information needed for their identification.Suggested methods include checking deprecated APIs and compiler pragmas, comparing optimized output with source code, and analyzing functions through an abstract syntax tree.

7 THREATS TO VALIDITY

The paper identifies threats involving incomplete defect coverage, labeling errors, differing interpretations, unreliable survey responses, and evolving smart-contract technology. It describes mitigation steps but retains boundaries on the stability and completeness of the findings.

  • Internal Validity: Keyword-based StackExchange filtering may have missed some contract defects, and the study defined only 20 defects because of time and resource limits.The authors state that researchers can extend the defect list using their methods.
  • Internal Validity: Manual labeling of 587 contracts may contain errors caused by misunderstanding smart contracts, despite independent labeling by experienced authors.The study used card sorting and selected authors with three years of smart-contract development experience to reduce errors.
  • Internal Validity: Different researchers and developers may understand the impact of a contract defect differently.The authors consulted related posts, real-world examples, developers, and survey feedback to reduce this threat.
  • Internal Validity: Some survey feedback may be incorrect because respondents may not understand every defect or may answer uniformly across questions.The authors added an “I don’t understand” option, removed such responses, and allowed respondents to skip questions.
  • External Validity: Changes in Solidity and Ethereum may create new defects or alter existing defect distributions over time.The authors state that their method can be applied to new developments but requires further effort.

8 RELATED WORK

Related work has largely addressed security vulnerabilities and automated detection, while this study broadens coverage to contract defects across five dimensions and validates practitioners’ views of their harmfulness.

  • Security-Focused Studies: Prior work introduced Ethereum smart-contract security vulnerabilities from Solidity, EVM, and blockchain perspectives.Atzei et al. presented 12 kinds of security vulnerabilities and described attacks that could enable attackers to profit.
  • Security-Focused Studies: Oyente uses symbolic execution and control-flow analysis to detect four security issues in smart contracts.Its checks include mishandled exceptions, transaction-ordering dependence, timestamp dependence, and reentrancy.
  • Security-Focused Studies: Oyente’s analysis identified 8,519 contracts containing the defined security problems.This result is reported as part of the prior evaluation discussed in the paper.
  • Detection Tools: Other tools extend security detection through symbolic analysis, fuzzing, and clone detection using source code or bytecode.The surveyed approaches include Zeus, ContractFuzzer, MAIAN, and SMARTEMBED.
  • Study Positioning: This study differs by covering security, availability, performance, maintainability, and reusability rather than focusing only on security problems.It also validates whether practitioners consider the identified defects harmful.

9 CONCLUSION AND FUTURE WORK

The paper reports an empirical study that defines and characterizes smart-contract defects using StackExchange posts, surveys, and real-world contracts. It releases a labeled dataset and presents practical uses for developers and researchers.

  • Conclusion: The study selected 4,141 warning-related StackExchange posts from 17,128 posts and manually defined 20 contract defects across five aspects.The aspects are security, availability, performance, maintainability, and reusability.
  • Conclusion: Survey feedback indicated that the defined defects are important and that addressing them can improve smart-contract quality.The study also analyzed each defect’s impacts and labeled 587 real-world Ethereum contracts.
  • Future Work: Developers can use the five impacts to build more robust, better-designed contracts and prioritize defect removal.Researchers can use the released dataset as ground truth for developing smart-contract defect-detection tools.
Loading 1905.01467v3…