TraceFixer: Execution Trace-Driven Program Repair

1 INTRODUCTION

• In the Figure 1 example, the desired state removes 92 from ports, and TraceFixer predicts a replacement line that produces the desired list.The figure contrasts the buggy execution trace with the trace the developer intends.Source: §1 · pp. 1–2
• Prior repair approaches do not use runtime traces or the desired state as inputs, although knowing the intended state can make it easier to determine a fix.Developers first locate where actual behavior departs from desired behavior, then edit the code to prevent that divergence.Source: §1 · p. 1
• TraceFixer takes buggy code, an execution-trace excerpt, and the desired state at the divergence point to predict a code variant that reaches that state.The approach uses a neural model that reasons about code and corresponding execution traces.Source: §1 · pp. 1–2
• TraceFixer addresses the need for training data with neural bug injection and handles multiple input modalities through formatting compatible with sequence-to-sequence models.The injected Python bugs mimic single-line mistakes observed in the wild.Source: §1 · p. 2
• TraceFixer improves over the code-only baseline by 13% to 20%, fixes 57%–63% with its top suggestion, 82%–87% within the top 10, and 10 of 20 real-world bugs.The evaluation covers single-line Python bugs.Source: §1 · p. 2
• Finding the divergence point and obtaining traces are outside the paper’s scope; the authors envision debugger support or fault localization on tests as possible routes.The authors cite fault localization and generate-and-validate repair techniques.Source: §1 · p. 2

2.1 Problem Definition

• Trace-guided repair predicts fixed code from buggy code, its trace, and the desired divergence-point state; traces cover program memory and may contain up to three events.The approach ignores file-system, network, and other-process state.Source: §2.1 · pp. 2–3

2.2 Overview

• The four-stage workflow injects bugs for training, collects execution traces, formats the inputs, and trains a model to repair previously unseen bugs.During training, traces supply the divergence point and desired state; at prediction time, the developer supplies them.Source: §2.2 · p. 3

2.3 Preliminary

• TraceFixer uses a Transformer sequence-to-sequence model, whose encoder summarizes input and whose decoder generates edited code token by token.Byte-pair encoding splits rare or unfamiliar code tokens into vocabulary sub-tokens.Source: §2.3 · pp. 3–4

2.4 Bug Injection

• A neural injector trained on patched PySStuBs examples learns to generate bugs by reversing the bug-fixing process.After filtering invalid files and Python 2 code, the training set contains 47,314 valid samples.Source: §2.4 · p. 4

2.5 Execution and Tracing

• Line-level tracing records in-scope program state, excludes calls into other functions or modules, and retains only examples whose buggy and fixed traces diverge.Non-deterministic examples are dropped, and the desired state is copied from the correct trace at its first divergence from the buggy trace.Source: §2.5 · pp. 4–5

2.6 Constructing and Formatting Input

• The formatted model input combines the buggy line, initial state, trace events, desired state, and code context, while the target is the corrected line.The trace length is configurable, and the example includes only variables that change between states.Source: §2.6 · p. 5

2.7 Deep Learning Model

• After BPE tokenization, a CodeT5-based sequence-to-sequence model is trained to predict the corrected line and decodes its output tokens into a fix.Training compares predictions with ground-truth targets and updates the model weights.Source: §2.7 · pp. 5–6

3 EVALUATION

• RQ1 asks how effective TraceFixer is at fixing bugs.The supplied evaluation passage states the research question, not its answer.Source: §3 · p. 6
• RQ2 compares TraceFixer with a baseline that uses code but no traces.The supplied evaluation passage states the research question, not its answer.Source: §3 · p. 6
• RQ3 asks how effective a multitask model is when trained for both trace-guided and code-only repair.The supplied evaluation passage states the research question, not its answer.Source: §3 · p. 6
• RQ4 examines how repair effectiveness varies across bug types.The supplied evaluation passage states the research question, not its answer.Source: §3 · p. 6

3.1.1 Datasets.

• The study uses PySStuBs for injector training and a 20-bug real-world check, plus injected-bug samples from CodeNet and TheAlgorithms.Manual test setup and project compatibility challenges limit the real-world sample; CodeNet yields 103K data points, and TheAlgorithms yields 8.5K.Source: §3.1.1 · p. 6

3.1.2 Metrics.

• UTOPk measures how often the correct fix appears among the model’s k unique suggestions, counting exact matches except for unnecessary whitespace.The model continues querying until it has produced k unique suggestions.Source: §3.1.2 · pp. 6–7

3.1.3 Baseline.

• The Code-only baseline is a stripped-down TraceFixer variant that receives buggy source code but no execution traces.This controlled comparison isolates the contribution of trace inputs; the authors avoid direct comparisons with prior tools focused on Java.Source: §3.1.3 · p. 7

3.1.4 Implementation and Hardware.

• The implementation instruments Python with standard and community packages and runs experiments using CodeT5-small on P100 and V100 GPUs.One training epoch takes about 2.5 hours on the P100 and 1.5 hours on the V100.Source: §3.1.4 · p. 7

3.2 RQ1: Bug Fixing Effectiveness

• 63%, 57%, and 10% of bugs were fixed by the top prediction; UTOP10 accuracy was 87%, 82%, and 50% across CodeNet, TheAlgorithms, and real bugs.The real-bug set contains 20 bugs.Source: §3.2 · p. 7
• Real-world results may be lower because CodeNet training can shift the data distribution, open-source projects are more complex, and their code uses project or package dependencies.The benchmark datasets mainly contain algorithmic solutions using standard Python functions and data structures.Source: §3.2 · p. 7
• Some real-bug fixes require tokens absent from both the code and trace, which the model typically cannot predict unless it saw such a fix during training.The paper calls this an out-of-vocabulary problem.Source: §3.2 · p. 7

3.3 RQ2: Comparison with Code-Only Approach

• TraceFixer improves over code-only UTOP K accuracy across all three evaluation setups; on CodeNet, its UTOP 1 accuracy improves by 13%.Using three datasets suggests the improvement is independent of dataset.Source: §3.3 · p. 7, Table 1 · p. 8
• Trace-guided repair adds execution traces and desired states, giving the model more guidance than code-only learning about the desired change.The paper says the code-only model has limited knowledge about the desired change.Source: §3.3 · p. 7
• More than 97% of instances fixed by code-only were also fixed by TraceFixer across the three datasets, and TraceFixer fixed additional instances.In the real-bug setup, code-only's fixed instances are strictly included in TraceFixer's.Source: §3.3 · pp. 7–8, Figure 5 · p. 8

3.4 RQ3: Combining Trace-Guided and Code-Only Learning

• The model is trained for trace-guided and code-only repair, using <FFT> and <FFC> prefixes to distinguish the inputs.At query time, trace-guided repair is attempted first; code-only repair follows if the trace-based predictions are incorrect.Source: §3.4 · p. 8
• Code-only and trace-based repair are complementary: code-only reasoning can help when a trace is unavailable or does not help fix the code.The multitask model encodes both repair modes in one model.Source: §3.4 · p. 8
• The multitask variant slightly improves over single-task TraceFixer on CodeNet accuracy.The comparison is reported for the multitask model and single-task variants; the supplied text gives no numeric size for the improvement.Source: §3.4 · p. 8, Table 3 · p. 10

3.5 RQ4: Effectiveness Depending on the Kind of Bug

• Performance across bug categories is close to overall performance except for Functions, where trace guidance is less helpful for constructing the required fix.For example, the desired sorting behavior may require adding a specific key argument, although the divergence is only visible in the list order.Source: §3.5 · p. 10
• The evaluation groups bugs into arithmetic, varmisuse, functions, data structures, and control flow categories.Each category has 20 instances.Source: §3.5 · p. 10

3.6 Threats to Validity

• The implementation is Python-only, so results on other languages remain to be evaluated.The approach is described as language-independent by design, but replication of the results on other languages is untested.Source: §3.6 · p. 10
• TraceFixer assumes the developer can localize the buggy line and provide its desired state, which may take substantial effort for complex programs or large data structures.Source: §3.6 · p. 10

4 RELATED WORK

• Traditional repair pipelines localize faults, search candidate fixes, and check them against correctness criteria; candidate search can be costly, while restricted templates may exclude correct fixes.TraceFixer instead produces fixes directly over the program vocabulary in an inference pass.Source: §4 · p. 10
• Prior learning-based work addresses bug localization or repair, but the discussed repair approaches do not consider dynamic program traces.The paper states that TraceFixer can therefore repair a broader class of bugs than prior works.Source: §4 · pp. 10–11
• Trace information has benefited learned program representations for semantic tasks, and this work applies trace divergence to neural bug repair.Source: §4 · p. 11

• TraceFixer improves over code-only baseline approaches in fixing bugs.Source: §5 · p. 11
• Evaluations on multiple datasets show that TraceFixer is effective at fixing bugs.Source: §5 · p. 11
• TraceFixer allows repair of more real bugs.Source: §5 · p. 11