SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics

1 INTRODUCTION

• Supervised repair lacks project-specific tokens and execution signals that can identify fault type; mined commits may not contain relevant project knowledge.The paper also notes that compiling and testing past commits at scale is difficult.Source: §1 · pp. 1–2
• SelfAPR perturbs a past version of the target project to create synthetic training samples containing project-specific tokens and semantic relationships.The approach is intended to capture rare tokens, idiomatic expressions, and relationships among domain identifiers.Source: §1 · p. 2
• SelfAPR executes perturbed programs and adds each buggy sample’s execution diagnostic to the input, training the model to produce the correct pre-perturbation program.At inference, the failing test case’s error is used as the input diagnostic.Source: §1 · p. 2
• SelfAPR repairs 110 bugs across 17 projects, including 65/388 in Defects4J 1.2 and 45/430 in version 2.0, outperforming supervised repair approaches.The evaluation covers 818 Defects4J bugs; 10 repaired bugs had not been reported as repaired by related supervised approaches.Source: §1 · pp. 2–3
• SelfAPR is presented as the first learning architecture to encode and use test execution diagnostics for program repair.The authors also report that project-specific training samples contribute 20 additional repairs, a 30% increase on Defects4J version 1.2.Source: §1 · pp. 2–3
• The released dataset contains 1 039 873 buggy samples, including 408 858 functional errors and 631 015 compilation errors.The authors make the code and data publicly available for further research.Source: §1 · p. 3

2.1 Automated Program Repair (APR)

• Search-based APR explores patches from edit patterns with heuristics, semantics-based APR synthesizes patches under constraints, and supervised learning ranks data-mined patches but lacks programming-language knowledge.Search and semantics-based approaches have their own trade-offs: large search spaces for the former and typically small-scale edit patterns for tractable synthesis in the latter.Source: §2.1 · p. 3

2.2 Self-supervised Learning

• Self-supervised learning turns unlabeled data into labeled data without human annotation; code-learning work typically masks or replaces tokens, while program repair had only preliminary investigations.The paper frames self-supervision as largely unexplored in program repair.Source: §2.2 · p. 3

3.1 Overview

• SelfAPR perturbs a correct past version of a project, validates variants with its compiler and tests, and trains a model to repair future bugs from that project.At inference, the model receives the buggy program and a failing-test diagnostic; fault localization supplies suspicious statements and their context.Source: §3.1 · pp. 3–4

3.2 Perturbation Model

• Perturbations reverse repair objectives: deleting code trains insertion, while adding superfluous code trains code removal.Rules also target valid identifiers, code reuse, and synthesis from surrounding context; all training samples are checked for uniqueness and bugs.Source: §3.2 · p. 4, §3.2.2 · pp. 4–5

3.2.1 Running Example.

• Multiple perturbations of one AST statement can produce project-specific training samples that help address a new, unseen bug years later.The Closure example describes perturbing similar methods in a past version even though the later buggy method was not present there.Source: §3.2.1 · p. 4

3.2.2 Perturbation Rules.

• Rules cover type, operator, literal, constructor, variable, invocation, boolean-expression, statement-order, and block changes, including donor-code transplantation.The rules target typing and scope, reuse of nearby or similar code, and context-sensitive synthesis; their diverse samples support repair actions beyond the fixed transformations.Source: §3.2.2 · pp. 4–5

3.3 Diagnostic Extraction

• Compiler errors or test failures identify perturbed programs as buggy training samples and yield compiler or test-failure diagnostics.A compilable variant is run against the available test suite; a failing test produces a test execution diagnostic.Source: §3.3 · p. 6
• Diagnostics are included in the neural input because their error signals may guide patch generation toward the bug type.The paper gives NullPointerException as an example of a diagnostic that can indicate a repair action.Source: §3.3 · p. 6
• Each diagnostic is tokenized into a compiler- or test-error marker, exception type, error message, and failing test method name.Compiler-error diagnostics also provide training signals about project-specific typing and scoping, although they are not directly tied to the final goal of repairing functional errors.Source: §3.3 · p. 6

3.4 Neural Architecture and Training

• Each training sample pairs an executed perturbation-based program and its error diagnostic with the correct, unperturbed program as output.The architecture guarantees that every training sample has been executed at least once.Source: §3.4 · p. 6
• SelfAPR uses a transformer neural network to translate perturbation-based programs into the expected correct programs.The paper describes the transformer as state-of-the-art.Source: §3.4 · p. 6
• Training updates model parameters to maximize the probability of generating the correct code given the perturbation-based program and diagnostic.Source: §3.4 · p. 6

3.5 Patch Ranking

• Beam search greedily computes the most likely next tokens when generating patches.Source: §3.5 · p. 6
• Candidate sequences are ranked by the maximum likelihood estimation score of the overall prediction.Source: §3.5 · p. 6
• SelfAPR outputs the top K likely sequences, with K configured as the beam width.Source: §3.5 · p. 6

3.6 Implementation

• The perturbation model uses Spoon, while the patch generator uses a six-layer Transformer with capped input and patch lengths.Training runs for 10 epochs with batch size 32 and vocabulary size 32,128.Source: §3.6 · p. 6

4.1 Research Questions

• RQ1 compares self-supervised training with state-of-the-art repair; RQ2 tests project-specific training, and RQ3 examines component contributions.RQ3 removes diagnostics, functional-error samples, and compiler-error samples in separate ablations.Source: §4.1 · p. 7, §4.5 · p. 8

4.2 Training and Testing Datasets

• SelfAPR perturbs buildable, tested past project versions; its 1,039,873 samples from 17 projects support testing on 818 later Defects4J bugs.The samples comprise 631,015 compiler-error and 408,858 functional-error samples; bugs used for training are excluded from testing.Source: §4.2 · p. 7
• Figure 4 shows that all perturbation rules contribute samples, but Rule11 generates the most and Rule12 the fewest.Rule12 wraps target statements with a block.Source: Figure 4 · p. 7, §4.2 · p. 7
• Figure 5 shows diverse compiler and functional diagnostics, with no error type dominating; the leading types include cannot find symbol, AssertionFailedError, and NullPointerException.Training data filtering retains functional-error samples and semantic compiler errors while removing syntactic compiler errors.Source: Figure 5 · p. 7, §4.2 · p. 7

4.3 Patch Verification

• Patches are checked with the compiler, human-written tests, and independent generated tests, then manually assessed by two authors against developer patches.Correct patches must pass both test suites and match the developer patch or be judged correct through manual analysis.Source: §4.3 · pp. 7–8

4.4 Methodology for RQ1

• RQ1 compares SelfAPR with supervised, semantics-based, search-based, and BugLab approaches under shared benchmark conditions and fault-localization assumptions.The evaluation uses a beam size of 50, ensemble models from 10 epochs, and counts correct repairs and their beam-search rankings.Source: §4.4 · p. 8

4.5 Methodologies for RQ2 & RQ3

• RQ2 removes each test project’s perturbation samples to assess project-specific training, while RQ3 separately ablates diagnostics and both error-sample types.Both questions measure correctly repaired bugs; RQ3 evaluates its ablations on Defects4J v1.2.Source: §4.5 · p. 8

5.1 Answers to RQ1: Effectiveness of Self-supervision

• Fourteen of 16 perturbation rules contribute to at least one repair, and Rule6 repairs the most bugs despite a comparatively small training-sample share.The authors interpret this pattern as evidence that learning occurs across rules.Source: Figure 6 · p. 9, §5.1 · p. 9
• With perfect fault localization, SelfAPR repairs 65 D4J v1.2 bugs and 45 D4J v2.0 bugs; with spectrum-based localization, it repairs 39 and 28, respectively.It outperforms related work except Recoder on D4J v1.2 with spectrum-based fault localization.Source: §5.1 · p. 9
• SelfAPR outperforms BugLab by a large margin; its broader, more generic perturbations produce diverse samples, unlike BugLab’s narrower rules without transplantation or deletion.The authors link BugLab’s omissions to fewer opportunities to learn insertions, deletions, and project-specific usage.Source: §5.1 · p. 9
• Sixty percent of SelfAPR’s correct patches rank in the beam-search top 10, using learned prioritization rather than manually defined heuristics.The authors note that top-10 ranking matters for developer acceptance in practice.Source: Table 3 · p. 9, §5.1 · p. 9
• Despite using fewer samples than CoCoNuT, CURE, and RewardRepair, SelfAPR performs better and uniquely repairs 10 bugs unreported as repaired by other learning-based approaches.The authors attribute sample quality to project-specific knowledge and controlled bug-triggering samples without unrelated changes.Source: §5.1 · p. 9

5.2 Answers to RQ2: Project-specific Training

• Project-specific training raises correct repairs from 45 to 65 across six projects, an overall improvement of 30.8%.Examples show reuse of a frequent Math for-loop statement and learning a Closure token relationship from cooccurrence.Source: Table 4 · p. 10, §5.2 · p. 10

5.3 Answers to RQ3: Ablation Study

• For D4J v1.2, CE-only, FE-only, and no-diagnostic ablations repair 34, 59, and 56 bugs, respectively, versus 65 for full SelfAPR.The authors identify each component as important.Source: Table 5 · p. 10, §5.3 · p. 10
• FE samples outperform CE-only training; CE samples add project-specific knowledge and typing and scoping constraints, while diagnostics contribute nine more correct patches.The Lang-33 example is repaired by FE-trained SelfAPR guided by a NullPointerException diagnostic.Source: §5.3 · p. 10

6.1 Threats to Validity

• The evaluation is limited to Java, leaving performance generalization to other programming languages untested.The authors also identify possible perturbation-tool bugs and manual patch-correctness assessment as internal threats.Source: §6.1 · p. 11

6.2 Multi-location Bugs

• SelfAPR does not repair 420 of 818 multi-location Defects4J bugs; interacting fixes across locations contribute to the challenge.The authors note this challenge is shared with related work.Source: §6.2 · p. 11

7.1 Creation of Perturbed Programs

• Unlike mutation-testing mutants or Patra and Pradel’s embedding-guided low-level perturbations, SelfAPR uses AST transformations and program analysis for code transplantations and deletions with larger functional impact.Source: §7.1 · p. 11

7.2 Self-supervised Learning on Code

• Earlier self-supervised repair perturbs characters, while SelfAPR’s AST-level perturbations can trigger functional errors; BugLab does not execute perturbations or include test diagnostics.Source: §7.2 · p. 11
• Other code self-supervision typically masks or perturbs single tokens, unlike SelfAPR’s AST perturbations that respect language constraints; SelfAPR generates samples from a past project version.Source: §7.2 · p. 11

7.3 Training based on Execution

• Prior execution-trace and compiler-diagnostic approaches address other tasks or compilation errors, while SelfAPR uses test diagnostics for repair, including assertion failures.Source: §7.3 · p. 11

• 110 of 818 Defects4J bugs are correctly repaired by SelfAPR.Source: §8 · p. 11
• 10 of those repairs had not previously been achieved by supervised learning repair approaches.Source: §8 · p. 11
• SelfAPR combines automatically generated perturbation-based self-supervision with execution information that captures failing assertions in the input.Source: §8 · p. 11