PaperlayerView exact revision on arXiv

Source-linked AI summary

Less is More: Adaptive Program Repair with Bug Localization and Preference Learning

Zhenlong Dai, Bingrui Chen, Zhuoluo Zhao, Xiu Tang, Sai Wu, Chang Yao, Zhipeng Gao, Jingyuan Chen

arXiv:2503.06510v1cs.SEcs.CL

TL;DR

Existing APR research emphasizes correct patches, while adaptive repairs with minimal modifications and consistency with buggy code have received little attention. The paper introduces AdaPR and AdaPatcher, a two-stage approach combining bug localization and repair strategies. The authors report that AdaPatcher outperforms baselines by a large margin or achieves similar accuracy while improving consistency.

  • Problem

    APR research has focused on correct patches, while consistency with the original buggy code and adaptive repair with minimal modifications have seldom been investigated.

    Source: INTRODUCTION · pp. 1–2

  • Method

    AdaPatcher uses a Bug Locator and a Program Modifier, with location-aware repair learning, hybrid training for selective location use, and adaptive preference learning for fewer changes.

    Source: Adaptive Preference Learning · p. 5, CONCLUSION · p. 8

  • Results

    AdaPatcher outperforms baselines by a large margin in repair accuracy or achieves similar accuracy while significantly improving consistency.

    Source: Experimental Results · p. 6

  • Takeaways & Limitations

    The authors present AdaPR and AdaPatcher as an approach to repair that aims to preserve consistency while reducing modifications.

    Source: CONCLUSION · p. 8

    The study is based on Python, and correctness can be affected when adaptive preference learning is applied.

    Source: FUTURE WORK · p. 7

Abstract

from arXiv · show

Automated Program Repair (APR) is a task to automatically generate patches for the buggy code. However, most research focuses on generating correct patches while ignoring the consistency between the fixed code and the original buggy code. How to conduct adaptive bug fixing and generate patches with minimal modifications have seldom been investigated. To bridge this gap, we first introduce a novel task, namely AdaPR (Adaptive Program Repair). We then propose a two-stage approach AdaPatcher (Adaptive Patch Generator) to enhance program repair while maintaining the consistency. In the first stage, we utilize a Bug Locator with self-debug learning to accurately pinpoint bug locations. In the second stage, we train a Program Modifier to ensure consistency between the post-modified fixed code and the pre-modified buggy code. The Program Modifier is enhanced with a location-aware repair learning strategy to generate patches based on identified buggy lines, a hybrid training strategy for selective reference and an adaptive preference learning to prioritize fewer changes. The experimental results show that our approach outperforms a set of baselines by a large margin, validating the effectiveness of our two-stage framework for the newly proposed AdaPR task.

INTRODUCTION

The paper introduces AdaPR to address the gap between generating correct patches and generating patches that remain consistent with the original code. AdaPatcher combines bug localization and repair strategies to target minimal modifications.

INTRODUCTION

  • Prior LLM-based repair can rewrite much of a function, making patches harder to trace and understand and difficult for developers to accept.The introduction frames consistency with existing code as a practical concern for adopting generated repairs.Source: INTRODUCTION · pp. 1–2
  • AdaPR targets patches that are correct and consistent with the original code, using the fewest possible changes to satisfy the specification.The introduction illustrates this with a one-line code change.Source: INTRODUCTION · p. 2
  • AdaPR presents two challenges: locating the root cause and generating correct patches with as few modifications as possible.The introduction notes that LLMs may rewrite programs from scratch without considering existing structure or semantics.Source: INTRODUCTION · p. 2
  • AdaPatcher uses a Bug Locator to identify bug locations and a Program Modifier to generate repairs based on them.The Bug Locator uses self-debug learning; the modifier combines location-aware repair learning, selective reference to locations, and adaptive preference learning for fewer changes.Source: INTRODUCTION · p. 2
  • The paper introduces AdaPR, builds a dataset with over 50K buggy-code, test-case, and fixed-code samples, and presents AdaPatcher.The authors report that AdaPatcher is effective over a set of baselines while reducing modifications.Source: INTRODUCTION · p. 2

RELATED WORK

Related work describes LLM-based repair approaches using execution feedback, error explanations, and self-improvement, while distinguishing this paper's focus on minimal modifications.

RELATED WORK

  • Prior LLM repair work uses compiler or debugger feedback, execution information, error explanations, and iterative self-improvement, whereas this paper targets minimal modifications.The passage names CoT reasoning loops, NExT, CoFFEE, and other approaches as examples.Source: RELATED WORK · p. 2

METHODOLOGY

AdaPatcher uses a Bug Locator to identify buggy lines and a Program Modifier to propose adaptive fixes that prioritize minimal changes.

METHODOLOGY

  • AdaPatcher's first stage locates the root cause and identifies buggy lines as Code Diff; its second stage proposes fixes for those lines.Code Diff means the differences between buggy and correct code.Source: METHODOLOGY · p. 2
  • The approach prioritizes patches requiring minimal changes to preserve codebase cleanliness and maintainability.Source: METHODOLOGY · p. 3

Task Definition

Given a programming task, buggy code, and correct specification, the objective is to generate a fixed version that satisfies the specification while remaining consistent with surrounding code and requiring minimal modifications.

Task Definition

  • The task takes a programming task, buggy code, and correct specification as input, and seeks a specification-satisfying fixed version consistent with surrounding code and requiring minimal changes.Source: Task Definition · p. 3

Stage I: Bug Locator

The Bug Locator uses a diff-based approach to pinpoint buggy lines and self-debug learning to improve its ability to locate runtime errors.

Stage I: Bug Locator

  • LLMs can comprehend code but often struggle to identify and describe bugs, particularly by pinpointing buggy lines.The Bug Locator addresses this challenge with a diff-based approach that makes bug locations explicit.Source: Stage I: Bug Locator · p. 3
  • The diff aligns bug locations semantically and structurally with buggy lines, using a ‘-’ prefix to mark lines needing deletion or correction.This structured indication helps the Bug Locator focus on relevant code portions.Source: Stage I: Bug Locator · p. 3
  • Self-debug learning is introduced to improve the Bug Locator’s ability to identify and locate runtime errors, which LLMs often struggle to understand.The passage attributes this challenge to LLMs’ limited understanding of program execution at runtime.Source: Stage I: Bug Locator · p. 3

Format of Code Diff.

The section describes a diff format for marking buggy lines and a self-debug learning process that adds execution information to help localize runtime bugs.

Format of Code Diff.

  • A Git diff between buggy code c and corrected code y identifies lines marked ‘-’ as buggy lines L.The resulting diff file d is created by prefixing those buggy lines in c with ‘-’.Source: Format of Code Diff · p. 3
  • The diff file d and code c contain n lines, and d_i denotes its i-th line; · marks string concatenation and <space> denotes a whitespace character.Source: Format of Code Diff · p. 3

Self-Debug Learning.

  • Self-debug learning targets runtime bugs, which can be difficult for LLMs trained on static code text to understand and locate.The approach is inspired by rubber duck debugging (Parreira, Gillet, and Leite 2023; Ni et al. 2024).Source: Self-Debug Learning · p. 3
  • For a buggy program and a failed test case, the program is executed with that test to capture its actual output.The test case t is drawn from the correct specification s.Source: Self-Debug Learning · p. 3
  • The execution information includes test input and expected/actual output, plus variable states at executed lines and their execution order.The states and order form program trace information R_t, with variable states captured using Python’s ‘traceback’ module.Source: Self-Debug Learning · p. 3
  • Trace information is formatted as aligned inline comments showing variables that change after each line, with ellipses compressing large loop traces.The trace aligns structurally with the diff file to help the Bug Locator identify and localize errors.Source: Self-Debug Learning · p. 3
  • Self-debug learning minimizes the negative log-likelihood of the Code Diff file d using the constructed prompt.Source: Self-Debug Learning · p. 3
  • The self-debug training dataset comprises repair instances (q, c, t, d, y), and Pθ represents the LLM’s vocabulary probability distribution.Source: Self-Debug Learning · p. 3

Stage II: Program Modifier

The Program Modifier is trained to repair identified buggy areas while limiting unnecessary edits, including when Bug Locator locations may be incorrect. Its training combines location-aware repair, selective reference, and preferences for fewer modifications.

Stage II: Program Modifier

  • Stage II combines location-aware fixes, selective reference, and adaptive preference learning to reduce unnecessary changes when Bug Locator locations may be incorrect.The Program Modifier is trained to focus on identified buggy areas and prefer fewer changes.Source: Stage II: Program Modifier · p. 3

Location-Aware Repair Learning.

  • Location-aware repair learning trains the Program Modifier to correct identified buggy areas without changing unrelated code.Supervised learning trains it to predict the correct code y.Source: Location-Aware Repair Learning · p. 3
  • The repair instruction marks lines that may need deletion or modification with ‘-’ and supplies the programming task, buggy code, and failed-test execution information.Source: Location-Aware Repair Learning · p. 4

Hybrid Training for Selective Reference.

  • Hybrid training teaches the Program Modifier to selectively reference Bug Locator locations that may be incorrect, rather than modifying code blindly.The strategy is intended to reduce unnecessary modifications and the risk that incorrect locations lead to repair failure.Source: Hybrid Training for Selective Reference · p. 4
  • The dataset is split into D1 and D2 in a 1:k volume ratio, and Bug Locator-generated labels for D2 are added to form D′2.The new instances pair the original bug locations with the Bug Locator’s generated locations.Source: Hybrid Training for Selective Reference · p. 4
  • Joint supervised and selective-learning training is used to maintain repair capability while strengthening selective reference.The loss function applies a loss to each respective dataset.Source: Hybrid Training for Selective Reference · p. 4

Adaptive Preference Learning.

  • Adaptive preference learning uses preference pairs to prioritize correct repairs with fewer modifications over incorrect, more extensive repairs.The pairs are generated by the Program Modifier after unit tests, and DPO-Positive learning trains it to prioritize fewer-change repairs.Source: Adaptive Preference Learning · pp. 4–5
  • The adaptive objective includes a penalty term g intended to maintain a high log-likelihood for preferred code; λ is a hyperparameter.After training, the modifier is optimized to increase the probability of generating preferred code.Source: Adaptive Preference Learning · p. 5

EXPERIMENTS

The experiments evaluate AdaPatcher on ACPR for repair accuracy and consistency, testing its overall performance, components, hybrid training, and bug localization. The results favor the two-stage approach while identifying cases it struggles to handle.

Dataset.

  • ACPR contains 52,168 buggy-code, failed-test, and passed-code triplets, split 8:1:1 by problem and capped at 150/10/20 pairs per problem across train/validation/test.The problem-level split is intended to avoid data leakage; the caps balance the dataset.Source: Dataset · p. 5

Evaluation Metrics.

  • Evaluation measures repair accuracy, additional-test improvement, failures on previously passing cases, and the proportion of code lines preserved after modification.The measures are Code Accuracy Rate, Code Improvement Rate, Failed Repair Rate, and Code Consistency Rate.Source: Evaluation Metrics · p. 5

Baselines.

  • Baselines use GPT-4o for stage-I bug localization and CodeLlama-Instruct-7B for stage-II program repair, alongside end-to-end CoT, few-shot, and LoRA fine-tuning approaches.Source: Baselines · pp. 5–6

Experimental Results

  • 62.22% consistency for AdaPatcherGC exceeded CodeLlama Fine-Tuning's 51.16%; AdaPatcherGC/CG also beat CoT, few-shot, and fine-tuning in accuracy or matched accuracy with higher consistency.GPT-4o in stage two favored accuracy, while CodeLlama favored consistency; CodeLlama first and GPT-4o second gave the optimal performance.Source: Table 2 · p. 6, Experimental Results · p. 6

RQ2. Ablation Study.

  • Removing Self-Debug Learning, Location-Aware Learning, or Hybrid Training lowered accuracy and consistency; Preference Learning slightly reduced accuracy while improving consistency.Table 3 reports the ablation study.Source: Table 3 · p. 6, RQ2. Ablation Study · pp. 6–7

RQ4. Human Study for Bug Localization.

  • Across 900 samples reviewed by two evaluators, trained CodeLlama exceeded GPT-4o by 35.0% in AL and 3.0% in PL, with 15.7% fewer EL and 22.3% fewer NL.The human study compares model-predicted bug locations with human-identified locations.Source: Figure 4 · p. 7, RQ2. Ablation Study · p. 7

RQ3. Why Our Approach Works/Fails.

  • In inspected examples, AdaPatcher localized and slightly changed the buggy line while GPT-4o rewrote most function lines; failures involved uninformative tests or subtle cross-submodule bugs.The authors attribute correctness and consistency to the Bug Locator and Program Modifier working together.Source: RQ2. Ablation Study · p. 7

RQ5. Hybrid Training Analysis.

  • Hybrid training outperformed other training methods on repair-correctness metrics, including supervised training, and remained effective across different amounts of training data.The comparison combines weakly supervised and supervised data during training.Source: Table 4 · p. 7, RQ5. Hybrid Training Analysis · p. 7

FUTURE WORK

The study evaluates AdaPatcher on Python and notes that adaptive preference learning can affect generated-code correctness, leaving reduced changes alongside improved correctness as future work.

FUTURE WORK

  • The study is based on Python, although the authors believe the approach can be adapted to C++ or Java.Source: FUTURE WORK · p. 7
  • Correctness of generated code is affected when the model uses adaptive preference learning.Source: FUTURE WORK · p. 7
  • Future work will explore reducing modifications while further improving repaired-code correctness.Source: FUTURE WORK · p. 7

CONCLUSION

The paper introduces AdaPatcher for fixed code with minimal modifications, combining bug localization and repair strategies. Its experiments show the approach is effective for AdaPR.

CONCLUSION

  • AdaPatcher uses self-debug learning to train a Bug Locator to identify bugs and fix code through their locations.Source: CONCLUSION · p. 8
  • For repair, a Program Modifier uses location-aware repair learning, hybrid training to avoid blindly modifying incorrect locations, and adaptive preference learning to favor fewer changes.Source: CONCLUSION · p. 8
  • The experiments show AdaPatcher is effective for generating fixed code with minimal modifications in the AdaPR task.Source: CONCLUSION · p. 8

Experimental Setups

The experimental setup constructs paired code and bug-location data, configures the two training stages, and evaluates repair consistency alongside baseline prompting and fine-tuning approaches.

Experimental Setups

  • Code pairs are drawn from CodeNet and augmented with AlphaCode tests; only logical-error buggy code is considered, and retained pairs must meet the study’s similarity and submission-order rules.The highest-BLEU qualifying pair is selected; comments are removed before similarity scoring.Source: Experimental Setups · p. 9
  • Bug locations are derived from git diffs between buggy and passed programs, retaining deleted or changed lines while removing headers and added lines.Source: Experimental Setups · p. 9
  • Program traces record execution order and changed variables per line, with loops and arrays truncated or summarized to fit LLM token limits.Source: Experimental Setups · pp. 9–10

Implementation Details.

  • Training uses CodeLlama-instruction-7B, AdamW, batch size 16 in both stages, and greedy decoding; stage learning rates are 5e-5 and 5e-6.Stage one uses WarmupDecayLR and stage two a cosine scheduler; β is 0.1 and λ is 5.Source: Implementation Details · p. 10

Metrics of Code Consistency.

  • Consistency measures the proportion of fixed-code lines preserved after modification, with its range described as 0 to 1.The evaluation also uses git diff to count deleted or changed and added or changed lines.Source: Metrics of Code Consistency · p. 10
  • Correct code produced directly by closed-source models is not counted as a contribution, while unchanged buggy code from open-source models receives a consistency score of zero.Three experienced evaluators determine final results; Git checks whether any lines changed.Source: Metrics of Code Consistency · p. 10
  • Baselines include instruction, chain-of-thought, few-shot, and LoRA fine-tuning; LoRA is applied only to CodeLlama-Instruct-7B.Few-shot prompts add two task, buggy-code, and fixed-code examples; the prompting setups are identified in Figures 5–7.Source: Metrics of Code Consistency · p. 10, Figure 5 · p. 10, Figure 6 · p. 10

Experimental Results

The qualitative experiments show AdaPatcher making targeted repairs while better preserving buggy code’s formatting, logic, and structure than the compared baselines. They also identify localization limits and report gains from self-debug learning and adaptive preference learning in particular settings.

Qualitative Analysis of Baseline Results.

  • CoT can mislocalize bugs and make unnecessary edits, while few-shot examples can leave faults unresolved and introduce new bugs.The examples may also prompt changes to correct code because they do not directly identify bugs in the current program.Source: Qualitative Analysis of Baseline Results · p. 11

Qualitative Analysis of AdaPatcher.

  • At inference, the Bug Locator marks buggy lines in a Code Diff, then the Program Modifier uses those locations for targeted fixes with fewer changes.The described example uses AdaPatcherCC.Source: Qualitative Analysis of AdaPatcher · p. 11

Qualitative Analysis of the Effectiveness.

  • AdaPatcher preserves original naming, execution logic, and code structure better than InstructionGPT-4o, while both compared systems repair the example correctly against Fine-TuningCodeLlama.For the latter comparison, AdaPatcherCC retains the buggy program’s ternary expression rather than exposing the computation as Fine-TuningCodeLlama does.Source: Qualitative Analysis of the… · pp. 11–12

Qualitative Analysis of Failure Cases.

  • Insufficient failure information can obscure bug locations, while complex custom-function calls can lead the Bug Locator to mark call sites instead of root causes, producing superficial repairs.The authors describe these as two common failure situations.Source: Qualitative Analysis of Failure Cases · pp. 12–13

Qualitative Analysis of Bug Localization.

Analysis of Ablation Study.

  • Removing adaptive preference learning lowers Consistency from 46.32% to 44.43% on code samples exceeding 20 lines; self-debug learning raises GPT-4o-stage ACC from 65.71% to 67.57%.The ACC gain is less pronounced with CodeLlama in the second stage.Source: Analysis of Ablation Study · p. 14

Analysis of model performance differences.

  • Large models tend to rewrite buggy code, whereas smaller models tend to copy it without edits, corresponding to different consistency and repair risks.The passage names GPT-4o and Claude-3.5 as examples of larger models and CodeLlama-Instruct-7B as a smaller one.Source: Analysis of model performance… · p. 14
Loading 2503.06510v1…