A Comprehensive Survey of AI-Driven Advancements and Techniques in Automated Program Repair and Code Generation

1 Introduction

• The survey reviews 27 papers on LLM-based automated program repair and code generation, summarizing their goals and approaches.It aims to help readers understand how the tools work, their practical scenarios, and their limitations.Source: §1 · p. 2
• It describes the repair scenarios these tools address and the programming languages on which they work.Source: §1 · p. 2
• It examines how LLMs fit into repair and code-generation workflows, including associated challenges and limitations.Source: §1 · p. 2

2 Survey Methodology

• The methodology describes searching, collecting, and filtering papers, models, and journals relevant to the survey.Source: §2 · p. 2

2.1 Research Questions

• The research questions cover AI-assisted debugging, evaluation benchmarks, code-generation models, and differences in models’ task performance and strengths.Source: §2.1 · pp. 2–3

2.2 Methods Employed

• The review applies inclusion and exclusion criteria, then categorizes selected papers and models under Code Generation and APR.The categories are shown in Fig. 1.Source: §2.2 · p. 3, Figure 1 · p. 3
• The survey develops a taxonomy, compares performance and accuracy, identifies trends and gaps, and reviews benchmarks and evaluation metrics.Source: §2.2 · pp. 3–4

3.1.1 Security Bugs.

• Security repair approaches include template patches, dynamic analysis, search-based and specification-guided repair, input sanitization, and memory-safety fixes.The described methods include testing generated patches on varied inputs and using bounds checks or safer memory allocation.Source: §3.1.1 · p. 4
• APR can add input sanitization or filtering and address memory-safety bugs such as buffer overflows and null dereferences.Proposed memory-safety measures include bounds checking, safer allocation, and replacing raw pointers with smart pointers.Source: §3.1.1 · p. 5

3.1.2 Semantic Bugs.

• Semantic-bug repair uses patterns, greybox fuzzing, evolutionary search, and formal specifications to find or validate fixes.Greybox fuzzing uses partial knowledge of program structure, while specification-guided patches are checked against expected behavior.Source: §3.1.2 · pp. 5–6

3.1.3 Syntactic Bugs.

• Syntactic repair uses mined correction patterns, grammar-aware fuzzing, AST-based mutations, and searches for patches consistent with language syntax.FixMiner mines prior bug-fixing commits for recurring syntax errors.Source: §3.1.3 · p. 6

3.2 What recent trends have emerged for using AI in APR tools?

• Emerging APR practices include pretrained and transfer-learned models, self-supervision, fault localization, AST reasoning, automated tests, explainability, interaction, and multimodal context.Test generation and coverage improvement are described as ways to check repairs and reduce overfitting.Source: §3.2 · pp. 6–7

3.3 What challenges are being faced in APR right now?

• APR tools can misidentify faulty code, need human verification, struggle with large dependent codebases, and demand substantial memory and computing power.The stated resource overhead may disrupt workflows and reduce productivity.Source: §3.3 · p. 7
• Further challenges include poor generalization to unseen or domain-specific bugs, scalability and context limits, security risks, training-data bias, benchmark overfitting, and copyright concerns.Source: §3.3 · p. 8

4.1 Modern Debugging tools

• Debugging tools span fuzzing, learned semantic and structural representations, evolutionary and template-based repair, probabilistic patch prediction, and context-aware or semantic checks.FuzzRepair and AFLNet run patched code against diverse reachable inputs; other tools assess generalization, scalability, patch success probability, program fit, and integrity.Source: §4.1 · pp. 9–10

4.2 Modern Benchmark tools

• Benchmarks provide controlled settings for evaluating bug-fixing efficiency across code generation, fuzzing, debugging, vulnerability localization, real defects, and code translation.Examples include HumanEval and MBPP; ProFuzzBench and SCTBench; DebugBench and VulnLoc; Defects4J; and TransCoder.Source: §4 · p. 8, §4.2 · p. 10

4.3 Common Gaps and Limitations in Modern Debugging Tools and Benchmarks

• Limited or narrow benchmarks can weaken generalization: fuzzing may be incomplete, and results from specific datasets or languages may not transfer to broader programs.The section cites QFuzz, Defects4J, and language-specific datasets as examples of these scope concerns.Source: §4.3 · p. 11
• Nondeterministic repairs impede reproducibility, while patches that pass tests may still be incorrect.The section also notes that test-focused APR is less capable on security vulnerabilities and non-test bugs, including memory-safety defects in C and C++.Source: §4.3 · p. 11
• Limited test suites and simulated bugs may make benchmark evaluations less representative of complex production bugs.External-platform access requirements and rate limits can also restrict testing with some test-case sources.Source: §4.3 · p. 12
• ML-based repair can struggle with rare or out-of-vocabulary tokens, large patches, and incorrect predictions that target the wrong statements.These weaknesses can make such methods ineffective in more complex cases.Source: §4.3 · p. 12

• Codex emphasizes fast, fluent completion; DeepSeek-Coder’s Fill-In-Middle training helps complete unfinished snippets, while StarCoder2 performs better than CodeLlama-34B on HumanEval+.Phind/CodeLlama (34B) is reported to outperform GPT-4 on HumanEval; StarCoder2 struggles with C++ and fill-in-the-middle tasks.Source: §5 · p. 12
• The Open LLM Leaderboard lists 80.48% accuracy; Zephyr scores 90.6% on AlpacaEval and is among the best in bug detection and scalability, yet lags GPT-4 or Claude 3.5 in complex logic handling.Source: §5 · p. 13
• For bug fixing, GraphCodeBERT suits dependency-heavy code, while Codex handles common bugs quickly; GPT-4 leads on complex bugs through broader context handling.DeepSeek-Coder performs well on smaller logic errors but varies by language, and WizardCoder trails GPT-4 in debugging, localization, and generation.Source: §5 · p. 13
• For code summarization, CodeT5 produces coherent structural summaries, while GraphCodeBERT handles difficult dependencies but can be slow on large datasets.GPT-4 handles long, complex code summaries well but is slower on smaller or less complex codebases; StarCoder2 performs better overall than smaller, more secure StableCode-3B but can generate insecure code at 15B parameters.Source: §5 · p. 13
• Codex and GPT-4 are described as strong in translation and refactoring: Codex for basic cross-language translation, GPT-4 for more accurate cross-platform translation, with speed as a drawback.Phind and DeepSeek-Coder support multiple languages, but Phind may struggle with sophisticated language features and DeepSeek-Coder remains limited on niche languages.Source: §5 · pp. 13–14
• Models have task-specific strengths: Codex suits completion and fast debugging, GPT-4 complex tasks, and CodeT5 and GraphCodeBERT structured environments but not real-time adaptation.The survey also notes resource-intensity and security difficulties for newer models.Source: §5 · p. 14
• Mixtral and Phind are described as strongest for multilingual tasks; Smaug’s English-focused testing and the resource demands of GPT-4 and GraphCodeBERT constrain multilingual use.Mixtral beats Claude-2.1 and Llama 2 70B on instruction fine-tuning and multilingual tasks.Source: §5 · p. 14

General-language pretraining

• General-language-pretrained models use different strategies: CodeT5 combines programming and natural language with developer-assigned identifiers, while GraphCodeBERT represents code semantics through data-flow graphs.The survey notes that CodeT5 may need to learn structural relationships, while GraphCodeBERT may fall short on complex logic requiring broader control-flow structures.Source: §6 · pp. 15–16
• 73.8% pass rate on HumanEval is reported for Phind, while CodeT5 is described as a top performer on various CodeXGLUE tasks.Phind is fine-tuned from CodeLlama-34B on about 80,000 structured programming problems.Source: §6 · pp. 15–16

Specialized-code pretraining

• Specialized-code models use varied training: OpenCodeInterpreter builds multi-turn dialogues and erroneous-code examples, StarCoder2 uses interactive feedback, and SPT Code follows pretraining with task-specific fine-tuning.Magicoder uses OSS-INSTRUCT to create realistic code instructions, but may inherit biases from its seed snippets.Source: §6 · p. 16

Self-supervised and bootstrapped approaches

• DeepSeek-Coder uses instruction fine-tuning, Fill-In-the-Middle training, and a 128K-token context window, while WizardCoder’s Evol-Instruct automatically generates complex instructions.The survey says Mixtral activates only some of its 47 billion parameters at inference to conserve resources, although its structure may complicate GPU load balancing; Smaug’s DPO-Positive approach may struggle with low-edit-distance preference data.Source: §6 · pp. 16–17

Distillation and feedback

• Zephyr uses direct teacher-to-student distillation and human feedback, with zero-shot learning intended to support generalization to tasks not explicitly taught.The survey presents these methods as examples of how different training techniques yield different model performance.Source: §6 · p. 17

Conclusion

• The survey curates recent AI research on automated program repair and code generation, comparing tools to aid open-source model selection for use cases without reliable AI fixes.Its stated purpose also includes familiarizing researchers with state-of-the-art developments, use cases, and current challenges in AI-supported code tasks.Source: §7 · p. 17