Self-Supervised Bug Detection and Repair

• Bug detection requires reasoning over formal structures such as data and control flow and ambiguous information such as names, idioms, and comments.Traditional analyses use formal reasoning and combinatorial search but must be manually coded by experts, a lengthy and costly process that misses ambiguous information in code.Source: §1 · p. 1
• Existing learned approaches lack large supervised corpora for robust detection across common bugs, and prior datasets are limited or unrepresentative of real-code bugs.Prior work used randomly inserted bugs, Cloze tasks, small code edits, or build errors.Source: §1 · p. 1
• BUGLAB co-trains a detector to repair bugs with a selector that creates hard-to-detect bugs as training data.The Python implementation improves over training with randomly inserted bugs on a manually curated test set of 2,374 real-life bugs and identified 19 previously unreported bugs in open-source packages; false-positive rates remained around 98%.Source: §1 · p. 1

• A rewrite rule pairs a match function with a transformation function; it applies at a subtree when the match holds, otherwise leaving that subtree unchanged.For reversible rules, the inverse transformation returns the original syntax tree.Source: §2 · p. 2
• The detector is trained on an unannotated codebase to repair the hardest applicable rewrites, but searching all rewrite-location pairs is practically intractable.The potential rewrite set may be very large or unbounded.Source: §2 · p. 2
• A selector samples rewrite-location pairs to approximate the intractable maximum of detector loss instead of enumerating every possible pair.This turns training into a min-max optimization in which the selector seeks difficult examples and the detector learns to repair them.Source: §2 · p. 2
• The selector and detector predict rewrites with opposing objectives; at test time, BUGLAB discards the selector and uses only the detector.The two models can use the same architecture.Source: §2 · p. 2

• Code is represented as entities and typed relations in a heterogeneous graph, with a projection to token sequences and relations for relational transformer models.The projection deterministically maps entities appearing in relations to token entities.Source: §3 · p. 3
• Entity content is embedded from subtokens and contextualized with either a max-aggregation MLP-based GNN or GREAT over projected tokens and relations.GREAT uses positional encodings and projected relations; other entity-representation models were not explored.Source: §3 · p. 3
• The rewrite probability is factorized into a model for locating the rewrite and a model for choosing the rewrite given its location.The localization distribution includes a special NoBug location to indicate that code is not buggy.Source: §3 · p. 3
• Rewrite selection uses rule-specific score functions over the location representation and, for some rules, additional metadata, then applies a softmax over applicable rewrites.The metadata can include representations of other entities that could be used at the location.Source: §3 · p. 3

4 A Python Implementation

• PYBUGLAB targets a large subset of “stupid simple bugs” and its approach is largely transferable to other languages, with engineering effort required.The targeted bugs require small code changes but can substantially affect correctness; broader or learnable rewrite rules are left for future work.Source: §4 · p. 3
• The implementation extracts syntax, data- and control-flow, type, and documentation entities and relations from Python code.The available entities and relationships include syntax nodes, tokens, and symbols such as variable and function references.Source: §4 · pp. 3–4

4.1 Bug-Inducing PYBUGLAB Rewrite Rules

• The four bug-inducing rewrite classes are variable misuse, argument swapping, wrong operators, and wrong literals.The example in Figure 2 allows 63 rewrites, even though it is a small snippet.Source: §4.1 · p. 4, Figure 2 · p. 4
• Variable-misuse rewrites swap a variable for an in-scope symbol defined earlier, while argument swapping extends to arbitrary argument expressions.The argument-swap score uses representations of the parameter and the two arguments to be swapped.Source: §4.1 · p. 4
• Operator rewrites cover binary Boolean, arithmetic, and comparison operators plus two unary operators, and only substitute compatible operators.Literal rewrites are limited to integers -2, -1, 0, 1, and 2, and swapping True with False.Source: §4.1 · p. 4

4.2 PYBUGLAB Rewrite Rules for Data Augmentation

• Data augmentation includes variable renaming, comment deletion, if-else branch swapping, and comparison mirroring, which may change evaluation order for calls.These rewrites are intended not to change program semantics and are used as augmentation; comparison mirroring can alter semantics when it changes the order of function calls.Source: §4.2 · pp. 4–5

4.3 Implementation Details

• Training alternates between fitting the detector on selector-generated hard samples and fitting the selector to produce samples the detector found hardest.The unmodified NoBug case is always included, and discrete rewrite sampling prevents direct end-to-end training.Source: §4.3 · p. 5
• The implementation runs asynchronously with constantly updated detector and selector data pools; each sample is removed after four training uses.Snapshots of both models generate new pool elements, and the selector’s arg max is approximated using the k sampled candidates per input program.Source: §4.3 · p. 5

5 Evaluation

• The evaluation uses 2,374 manually filtered real-world bugs from PyPI repositories, alongside randomly rewritten functions from held-out packages.Training uses 3.4k of the 4k most downloaded PyPI packages, with the remainder reserved for testing.Source: §5 · pp. 5–6

5.1 Quantitative Evaluation

• PYBUGLAB training produces more robust detectors than other training strategies with both GNN and GREAT models; code augmentation adds no substantial difference.Random selection improves over supervised training mainly on in-distribution RANDOMBUGS samples.Source: §5.1 · p. 6, Table 1 · p. 6, Table 2 · p. 7
• Bug localization is harder than repair at a given location, and no model exceeds 63% accuracy on deciding whether a snippet contains a bug.Incorrect literals are among the hardest bug types; incorrect assignment operators are easier on RANDOMBUGS but harder on PYPIBUGS.Source: §5.1 · p. 7
• Detecting and repairing real-life bugs is significantly harder than handling randomly inserted bugs, and learned selectors outperform random selectors on PYPIBUGS.The authors speculate that learned selectors avoid easy-to-detect bugs; improvements on RANDOMBUGS often correlate with PYPIBUGS improvements.Source: §5.1 · p. 7
• The selector learns to generate harder-to-find bugs, but its sampled bug distribution is not necessarily realistic.Argument swapping, wrong literals, and assignment operators are over-represented, while comparison-operator and variable-misuse bugs are under-represented.Source: §5.1 · p. 8

5.2 Qualitative Inspection of Raised Warnings

• Of 1,000 reported warnings, only 19 were judged real-life bugs; qualitative inspection also found false positives and semantically equivalent proposed repairs.Many false positives involve incorrect literals, and some suggested repairs reflect missing aliasing knowledge or rewrites that do not change behavior.Source: §5.2 · pp. 9–10

6 Related Work

• BUGLAB builds on learned code-pattern analysis, which can use names, comments, control flow, and data flow to identify bugs [2] [24].Traditional formal methods target correctness properties, especially for security- and safety-critical bugs.Source: §6 · p. 10
• Prior methods include DeepBugs’ separate models for operators, operands, and argument swaps [22], and variable-misuse methods using random bug insertion or Cloze-like tests [3] [30].BUGLAB instead uses a structured code representation, one model, self-supervision, and a broader bug set.Source: §6 · p. 10
• BUGLAB resembles ELECTRA [6] and GANs [12], but rewrites structured code, trains directly for bug detection, and uses a non-differentiable objective.Its detector, rather than its selector, is the main object of interest.Source: §6 · p. 10

7 Discussion and Conclusions

• BUGLAB improves on baseline methods and detects bugs in real-life code, while the paper highlights a performance gap between randomly inserted and real-life bugs.The discussion also reports impractical false-positive rates for existing machine-learning bug detectors.Source: §7 · p. 10

A Model Architectures

• The appendix uses shared input and scoring components, subtoken embeddings with max pooling, and pointer-net-like localization followed by softmax rewrite scoring.The subtoken vocabulary has 15,000 entries, with at most six subtokens per token; training uses Adam, dropout, gradient clipping, and epsilon-greedy selector sampling.Source: A Model Architectures · p. 13

A.1 GNN Architecture

• The GNN appendix describes message passing with 256-dimensional node states, edge-type-specific transformations, and an eight-layer residual architecture.It also includes a table captioned as VarMisuse accuracies and another for GREAT-based models in the fully supervised setting.Source: §A.1 · pp. 13–14, Table 7 · p. 14

A.2 GREAT Architecture

• The GREAT reproduction matched the reference implementation on VarMisuse with 6- and 10-layer configurations, each using hidden size 512.Source: §A.2 · p. 14
• GREAT models were usually outperformed by their GNN equivalents in the main PYBUGLAB experiments, contrary to earlier results by Hellendoorn et al..Source: §A.2 · p. 14
• Varying GREAT depth from 5 to 10 layers showed no trend, and ReZero, LayerScale, and LayerNorm placement experiments did not significantly improve results.The authors concluded model capacity was not limiting and ruled out implementation issues overall.Source: §A.2 · p. 14
• The reported GREAT configuration uses 8 layers, hidden size 512, 8 heads, and intermediate size 2048.Training used maximum sequence length 400 and minibatches of 20.Source: §A.2 · p. 14

B Python Code Representation

• PYBUGLAB extracts code entities and relationships using libCST and Jedi, with many entity and relationship types inspired by prior code representations, [3] [4], [33],.Source: B Python Code Representation · p. 15, Table 9 · p. 15, Table 10 · p. 15
• Figure 4 illustrates the entity-and-relationship graph for Figure 2’s synthetic snippet; its displayed bar signature is illustrative.Source: B Python Code Representation · p. 15

• The prose identifies Figure 5 as defining the evaluation metrics used in this work.Source: C PYBUGLAB Evaluation Metrics · p. 15
• Figure 4 shows the entities and relationships for the code snippet in Figure 2.Source: Figure 4 · p. 16
• Figure 5 presents the evaluation metrics used for bug detection and repair.Source: C PYBUGLAB Evaluation Metrics · p. 15, Figure 5 · p. 17

D PYPIBUGS Description

• The dataset was built from commits across PyPI packages in the libraries.io v1.6.0 dataset, with observed changes matching a rewrite treated as candidate bug fixes.Source: D PYPIBUGS Description · p. 17
• Requiring each candidate to be the sole code change in its file yielded around 9,000 changes, excluding multi-bug files and many non-bug fixes.Source: D PYPIBUGS Description · p. 17
• PYPIBUGS contains 2,374 sanitized real-world bug-fixing changes after commit review and removal of three samples with parsing issues.The initial 2,377 reviewed changes were filtered to exclude non-bug fixes, including configuration-like constants and deployment artifacts.Source: D PYPIBUGS Description · p. 17
• PYPIBUGS covers Python bugs representable by the paper’s rewrites and targets high precision through sole-change filtering and manual review.The passage distinguishes it from ManySStuBs [16].Source: D PYPIBUGS Description · p. 17
• Because licensing concerns prevent release of raw data, replication materials include project URLs, commit SHAs, file paths, bug types, and a checkout script.Users must implement code extraction for buggy and fixed versions.Source: D PYPIBUGS Description · pp. 17–18

E RANDOMBUGS Description

• The appendix reports RANDOMBUGS statistics broken down by bug kind.Source: E RANDOMBUGS Description · p. 18, Table 12 · p. 18

F Additional Evaluation Results

• The appendix contains additional evaluation results beyond those reported in the main text.Source: F Additional Evaluation Results · p. 18

F.1 Localization & Repair Assuming Code is Buggy

• With NoBug masked out, the appendix evaluates localization per bug kind on buggy examples; the repair results in Table 3 remain unchanged.Source: §F.1 · p. 18, Table 13 · p. 18

G Detected Bugs in Open-Source Projects

• PYBUGLAB detected real-life bugs in open-source projects and submitted pull requests.Source: G Detected Bugs in Open-Source Projects · p. 18

G.1 Bug in spulec/moto

• In spulec/moto, the fix raises ResourceNotFoundError with identity_pool_id rather than identity_pool, and its pull request was merged.Source: §G.1 · p. 19

G.2 Bug in apache/tinkerpop

• In apache/tinkerpop, the fix checks read_partitions before setting readPartitions, and its pull request was merged.Source: §G.2 · p. 19

G.3 Bug in certbot/certbot

• The certbot example changes the source-address condition in probe_sni, and its pull request was merged.Source: §G.3 · p. 19
• The detected certbot variable-misuse bug required a more nuanced repair than replacing socket_kwargs with source_address.Source: §G.3 · p. 19

G.4 Bug in Polyconseil/aioamqp

• The aioamqp fix passes the original exception to the error callback instead of wrapping it in ChannelClosed.The example also shows the callback being invoked while iterating over channels.Source: §G.4 · p. 19

G.5 Bug in apache/beam

• The Beam test checks that pcoll2, rather than pcoll1, is unbounded.The change corrects which pipeline collection the assertion tests.Source: §G.5 · pp. 19–20

G.6 Bug in sarugaku/requirementslib

• The requirementslib change removes setup_requires bookkeeping identified as dead or unused code.The example removes setup_requires initialization and its update from the retrieved results.Source: §G.6 · p. 20

G.7 Bug in CiscoDevNet/webexteamssdk

• The examples correct a Webex SDK type check to validate title, a redset log message to identify item_str, and a PyTorch name rectification to update extra_input_names.These changes address the checked argument, the value reported on deserialization failure, and the name list used alongside extra_input_sizes.Source: §G.7 · p. 20, §G.8 · p. 20, §G.9 · p. 20

G.10 Two Bugs in saltstack/salt

• Salt's enable and disable branches test the requested enabled state, while boltons stores the supplied cache_value instead of always setting True.The Salt examples change both branches' conditions; the boltons example assigns the constructor argument to self.cache_value.Source: §G.10 · p. 21, §G.11 · p. 21

G.12 Bug in geopy/geopy

• The geopy fix handles a None text value explicitly, and the Allure fix formats link values with link_pattern rather than issue_pattern.The Allure change applies when the kind is a link and the value is not already an HTTP URL.Source: §G.12 · p. 21, §G.13 · p. 21

G.14 Bug in qiniu/python-sdk

• The Qiniu SDK fix assigns default_uc_host to its own configuration field rather than copying default_api_host.Source: §G.14 · p. 22