PaperlayerView exact revision on arXiv

Source-linked AI summary

From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence

Jian Yang, Xianglong Liu, Weifeng Lv, Ken Deng, Shawn Guo, Lin Jing, Yizhi Li, Shark Liu, Xianzhen Luo, Yuyu Luo, Changzai Pan, Ensheng Shi, Yingshui Tan, Renshuai Tao, Jiajun Wu, Xianjie Wu, Zhenhe Wu, Daoguang Zan, Chenchen Zhang, Wei Zhang, He Zhu, Terry Yue Zhuo, Kerui Cao, Xianfu Cheng, Jun Dong, Shengjie Fang, Zhiwei Fei, Xiangyuan Guan, Qipeng Guo, Zhiguang Han, Joseph James, Tianqi Luo, Renyuan Li, Yuhang Li, Yiming Liang, Congnan Liu, Jiaheng Liu, Qian Liu, Ruitong Liu, Tyler Loakman, Xiangxin Meng, Chuang Peng, Tianhao Peng, Jiajun Shi, Mingjie Tang, Boyang Wang, Haowen Wang, Yunli Wang, Fanglin Xu, Zihan Xu, Fei Yuan, Ge Zhang, Jiayi Zhang, Xinhao Zhang, Wangchunshu Zhou, Hualei Zhu, King Zhu, Bryan Dai, Aishan Liu, Zhoujun Li, Chenghua Lin, Tianyu Liu, Chao Peng, Kai Shen, Libo Qin, Shuangyong Song, Zizheng Zhan, Jiajun Zhang, Jie Zhang, Zhaoxiang Zhang, Bo Zheng

arXiv:2511.18538v5cs.SEcs.CL

TL;DR

Code LLMs support tasks such as code completion, translation, repair, and generation, while professional software engineering still poses challenges in domain constraints and security. This paper surveys and experimentally analyzes the code-model lifecycle from data curation through deployment; one reported comparison finds hybrid search near state of the art with less compute than an equivalent large model.

  • Problem

    General-purpose LLMs can struggle with domain constraints and large-system invariants, while code that works can still contain security vulnerabilities.

    Source: §2.1.4 · p. 14

  • Method

    The paper analyzes code LLMs across the lifecycle from data curation and pre-training through fine-tuning, reinforcement learning, and agent deployment, with experiments on training methodologies.

    Source: Conclusion · p. 191

  • Results

    Hybrid search achieved near-SOTA results on benchmarks like LiveCodeBench [444] while using far less compute than an equivalent large model.

    Source: §5.1.2.1 · p. 108

  • Takeaways & Limitations

    Verifiable-reward datasets support large-scale reinforcement learning without human-in-the-loop labeling and rigorous evaluation of coding ability.

    Source: §4.6.1 · p. 96

    General-purpose LLMs may fail to satisfy subtle API contracts or security policies and struggle to maintain invariants across large systems.

    Source: §2.1.4 · p. 14

Abstract

from arXiv · show

Large language models (LLMs) have fundamentally transformed automated software development by enabling direct translation of natural language descriptions into functional code, driving commercial adoption through tools like Github Copilot (Microsoft), Cursor (Anysphere), Trae (ByteDance), and Claude Code (Anthropic). While the field has evolved dramatically from rule-based systems to Transformer-based architectures, achieving performance improvements from single-digit to over 95\% success rates on benchmarks like HumanEval. In this work, we provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs, systematically examining the complete model life cycle from data curation to post-training through advanced prompting paradigms, code pre-training, supervised fine-tuning, reinforcement learning, and autonomous coding agents. We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder), critically examining the techniques, design decisions, and trade-offs. Further, we articulate the research-practice gap between academic research (e.g., benchmarks and tasks) and real-world deployment (e.g., software-related code tasks), including code correctness, security, contextual awareness of large codebases, and integration with development workflows, and map promising research directions to practical needs. Last, we conduct a series of experiments to provide a comprehensive analysis of code pre-training, supervised fine-tuning, and reinforcement learning, covering scaling law, framework selection, hyperparameter sensitivity, model architectures, and dataset comparisons.

6 Code for Generalist Agents

Section 6 organizes code for generalist agents around interaction protocols, agentic capabilities, and environment interfaces.

6.1 Code as Interaction Protocols

  • Interaction protocols are covered through tool use, the Model Context Protocol, and multi-agent coordination.Source: §6 · p. 5

6.2 Code as Agentic Capabilities

  • Agentic capabilities are organized around thinking in code, acting in code, and memory with code.Source: §6 · p. 5

6.3 Code as Environment Interfaces

  • Environment interfaces are treated through code as a simulation gym and computer-use agents.Source: §6 · p. 5

7 Safety of Code LLMs

Section 7 covers code LLM safety across pre-training, post-training, red-teaming, and agent-system mitigation.

7.1 Safety Pre-training for Code LLMs

  • Safety pre-training is divided into data provenance and license compliance, auditing and cleaning, standards, adversarial transformations, privacy, and bias.Source: §7 · p. 5

7.2 Safety Post-training for Code LLMs

  • Safety post-training covers training datasets, supervised fine-tuning, preference optimization for localized flaws, and reinforcement learning alignment.Source: §7 · p. 5

7.3 Red-teaming Techniques for Code LLMs

  • Red-teaming is organized around prompt-level, semantic and contextual, and agentic workflow manipulation.Source: §7 · p. 6

7.4 Mitigation Strategies for Coding and Behavioral Risks in AI Agent Systems

  • Mitigation strategies include secure execution environments, pre-execution validation, and runtime oversight with intent grounding.Source: §7 · p. 6

8 Training Recipes for Code Large Language Model

Section 8 presents training recipes for code LLMs, with guidance spanning distributed training, pre-training, supervised fine-tuning, and reinforcement learning.

8.1 Distributed Training Framework Introduction

  • The training guidance begins with an introduction to distributed training frameworks.Source: §8 · p. 6

8.2–8.4 Training Guidelines

  • Separate guidelines address pre-training, supervised fine-tuning, and reinforcement learning.Source: §8 · p. 6

9 Code Large Language Model for Applications

Section 9 surveys code LLM applications across development assistants, coding platforms, autonomous agents, repair, and review.

9.1 IDE-integrated Development Assistants

  • IDE-integrated development assistants are one application area.Source: §9 · p. 6

9.2–9.3 Cloud-native Coding Platforms and Terminal-based Autonomous Agents

  • The applications also include cloud-native coding platforms and terminal-based autonomous agents.Source: §9 · p. 6

9.4–9.5 Code Repair and Verification Applications; Pull Request Review and Quality Assurance

  • Code repair and verification applications are covered alongside pull request review and quality assurance.Source: §9 · p. 6

1. Introduction

Code LLMs have moved from brittle rule-based systems to widely adopted tools that translate natural-language intent into code. This survey synthesizes the field’s technical pipeline and applications, addressing gaps in prior surveys and reporting experiments across training stages.

1. Introduction

  • Code LLMs translate natural-language intent into functional code across generation, completion, translation, and repair, with commercial tools such as GitHub Copilot [321].The introduction also names Cursor, CodeGeeX, CodeWhisperer, Claude Code, and Gemini CLI as commercial or open-source examples.Source: §1 · p. 7
  • General models such as GPT, Claude, and LLaMA offer breadth, while specialized code LLMs are reported to outperform them on code-specific benchmarks through focused pretraining and architectural optimization.The passage names StarCoder, Code LLaMA, DeepSeek-Coder, CodeGemma, and QwenCoder as specialized examples.Source: §1 · p. 7
  • The field evolved from brittle rule- and grammar-based approaches to transformers, while the 2025 landscape shifts toward reinforcement learning, software engineering agents, and diffusion-based code models.The timeline also depicts terminal tools, IDE integrations, and plugins emerging alongside model development.Source: §1 · p. 7, Figure 2 · p. 8
  • Prior surveys often take panoramic or older-model views; technical coverage varies, few address reinforcement learning, and recent advances remain scattered across publications.The survey contrasts this literature gap with the need to synthesize data curation, instruction tuning, prompting, agents, retrieval, and evaluation.Source: §1 · p. 8
  • The survey unifies a taxonomy and end-to-end treatment of data, pretraining, fine-tuning, prompting, retrieval-augmented generation, agents, and evaluation beyond functional correctness.It also discusses code quality, maintainability, and efficiency as evaluation concerns.Source: §1 · p. 9
  • The paper reports experiments across code pretraining, supervised fine-tuning, and reinforcement learning, examining scaling laws, frameworks, hyperparameters, architectures, and datasets.The survey aims to identify emerging trends and open challenges for researchers and practitioners.Source: §1 · p. 9

2. Code Foundation Models

This section surveys the evolution of code foundation models and coding systems, from general-purpose LLMs and code-understanding models to generative models, repository-scale agents, and diffusion-based coding. It examines model families, training objectives and data, retrieval embeddings, long-context and tool interaction, and limitations and future directions including specialization, agentic training, and scaling laws.

2.1.1. The Rise of General LLMs

  • Transformer pretraining unified previously specialized AI systems; Codex demonstrated natural-language code generation, while AlphaCode [578] used large-scale sampling and filtering for competitive programming.Tool-augmented methods such as ReAct [1209] and Toolformer [868] further extend LLMs toward planning and API use, but professional software settings expose accuracy, security, and system-reliability limits [927, 928, 930].Source: §2.1.1 · pp. 9–10

2.1.2. Model Architectures

  • Architecture has diversified beyond dense transformers to sparse experts, recurrent/state-space, diffusion, and hybrid designs that trade off capacity, latency, and context length.For example, Qwen3-Next combines gated linear operators, attention, and MoE to target 256K+ contexts with low active parameters per token.Source: §2.1.2 · pp. 11–13

2.1.4. Limitations of General LLMs

  • General LLMs often lack the domain depth and robustness for professional engineering: they can miss API contracts, security policies, and invariants, while repository-scale issue resolution remains difficult.More than 100 models across 80 tasks had about 45% of generations contain known vulnerabilities; long-context models also struggle with middle-position information and cross-file reasoning, GUI hierarchy, and reliable long-horizon tool use.Source: §2.1.4 · p. 14

2.2.1. Closed-source Code Large Language Models

  • Closed-source coding models have progressed from generation toward repository-level and agentic software engineering, with GPT, Gemini, and Claude families adding reasoning, multimodality, long context, and tool use.The section describes GPT-5-Codex [756] as using reinforcement learning on engineering tasks, sandboxed execution, and controlled tools; Claude 4/4.5 [63–65] adds deliberative reasoning and a computer-use toolchain.Source: §2.2.1 · pp. 15–17

Others

  • Other model lines span Grok’s longer-context, tool-grounded coding; PanGu-Coder’s execution-aware ranking feedback [885]; PyMT5/JuPyT5’s feature- and notebook-aware generation; and AlphaCode’s sampling-and-filtering pipeline [577].AlphaCode 2 [950] adds learned scoring and reranking, while AlphaEvolve [737] uses iterative code edits, execution, and selection for program search.Source: Others · p. 19

2.2.2. Open-source Code Large Language Models

  • The open-source-model subsection introduces classical encoder-centric NL–PL embedding methods for retrieval and code understanding, alongside encoder-only, encoder–decoder, and decoder-only architectures.Table 2 is identified as a comparison of open-source code-specialized LLMs.Source: §2.2.2 · p. 19, Figure 6 · p. 19, Table 2 · p. 20

2.2.3. Evolution of Open-Source Code Large Language Models

  • The section organizes open-source code LLM development into four evolutionary stages based on architectural innovations and functional capabilities.The taxonomy is presented as a framework for understanding technological progression in open-source code intelligence.Source: §2.2.3 · p. 20
  • The field moved from encoder models for code understanding to generative models and then large language models with complex generation, conversational programming, and instruction following.Early encoders aligned code and text for classification, vulnerability detection, and semantic search; later models added generation and completion, followed by open models such as StarCoder, CodeLlama, DeepSeek-Coder, and CodeQwen.Source: Functional icons · pp. 22–23
  • StarCoder [563] combines long context, fill-in-the-middle training, and governed permissive-code data, while CodeGen2 [731] finds a CLM-plus-span-corruption mixture effective but notes infilling trade-offs.CodeGen2 reports no consistent Prefix-LM gains, benefits from NL+PL mixing under constrained compute, and scaling gains from continued multi-epoch pretraining.Source: Functional icons · pp. 23–24
  • Repository-level and long-context designs support cross-file reasoning: DeepSeek-Coder [363] uses repository pretraining, while StarCoder2 [643] and Qwen2.5-Coder [435] extend model scale and context.DeepSeek-Coder reports strong HumanEval and MBPP results; StarCoder2-3B surpasses similar-size peers, and StarCoder2-15B matches or exceeds larger models.Source: Functional icons · pp. 24–25
  • 62.4% on SWE-Bench Verified is reported for KAT-Dev-32B [1257], while Skywork-SWE [1252] benefits from validated execution trajectories and test-time scaling and DeepSWE [277] uses RL and verifier-guided patch selection.Skywork’s ablations find log-linear gains with more trajectories and emphasize execution-grounded data and framework quality over parameter count.Source: Functional icons · pp. 26–27
  • DiffuCoder [334] explores masked diffusion for code generation; its coupled-GRPO training improves EvalPlus by +4.4% using approximately 21K RL samples.The approach uses paired completions with complementary mask noise to reduce variance in likelihood estimates and exploit non-autoregressive search.Source: Functional icons · p. 28

Functional icons

  • Open code-retrieval embeddings now use varied architectures and contrastive training to support text–code search, code similarity, and repository retrieval.A 7B model [741] reports state-of-the-art CodeSearchNet performance, while Nomic Embed Code, CodeSage [1262], and other families use curricula, hard negatives, or multi-stage training for retrieval tasks.Source: Functional icons · pp. 28–29

2.2.4. Model Pre-Training Tasks

  • Next-token prediction trains causal language models on preceding context, while multi-token prediction targets several future tokens to improve generation efficiency and coherence.Fill-in-the-Middle conditions on both prefix and suffix to restore missing text; Code LLaMA [858] uses it for code completion without additional fine-tuning.Source: §2.2.4 · pp. 30–31, Figure 9 · p. 31

2.2.5. Model Training Stages

  • Model development proceeds from pre-training to post-training, with continued pre-training adapting existing models and annealing gradually reducing the learning rate.Post-training includes supervised fine-tuning on annotated input-output pairs and reinforcement learning guided by rewards or a reward model.Source: §2.2.5 · pp. 32–35

2.3. Open-source Code Pre-training Data

  • Open-source code pre-training datasets have shifted from prioritizing scale toward quality, licensing, and governance.The survey identifies dataset curation as foundational to building powerful, safer, and more reliable open-source models.Source: §2.3 · p. 35

2.3.1. The Github Datasets

  • The Stack v2 expands The Stack v1’s scale and language coverage while adding a formal developer opt-out process.The Stack v1 [493] provided 3.1 TB across 358 languages with deduplication; The Stack v2 [642] reaches about 900 billion tokens and over 600 languages.Source: §2.3.1 · pp. 35–36

2.3.2. StarCoderData

  • StarCoderData [563] adds cleaning and benchmark decontamination to Stack-derived code data, covering 86 languages alongside GitHub issues and commits.The dataset totals 783 GB and was curated for training the StarCoder model series.Source: §2.3.2 · p. 36

2.3.3. Others

  • Other influential resources include broad open datasets, permissively licensed code, and language-specific corpora; their curation highlights deduplication and transparent governance.The Pile [308] supported early models, RedPajama [206] includes a 59-billion-token code slice, and CodeParrot [279] removed nearly 70% of raw data during processing.Source: §2.3.3 · p. 36

2.4. Future Trends

  • Future code intelligence is expected to become more specialized, more capable of autonomous multi-step engineering, and more systematically optimized through scaling laws.The survey anticipates domain-specific gains on repository tasks and debugging, agent training with execution feedback, curricula, and tools, and scaling strategies balancing model size, data quality, compute, and efficiency.Source: §2.4 · p. 37

3. Code Tasks, Benchmarks, and Evaluation

This section surveys coding tasks, benchmarks, evaluation metrics, and resources across scales—from statement, function, and class levels to repositories and agentic systems—covering code efficiency, preference, reasoning, translation, test generation, and evaluation of functional correctness, semantic understanding, human alignment, and whole-project performance.

3. Code Tasks, Benchmarks, and Evaluation

  • The taxonomy organizes coding evaluation into statement/function/class tasks, repository-level work, and agentic systems spanning tool use, research, GUI interaction, and terminal use.Its leaves consolidate established and newer benchmarks for capabilities and scales across these branches.Source: §3 · pp. 37–38, Figure 13 · p. 38

3.1.1. Extensions Based on Traditional Metrics

  • CodeBLEU [846] captures lexical, syntax, and data-flow similarity but can miss functional correctness; Pass@k [161, 501] tests execution, while CodeBERTScore [1323] and RTC [45] assess semantic similarity.Pass@k samples k completions and scores whether at least one passes unit tests, accommodating multiple valid solutions.Source: §3.1 · p. 39, §3.1.1 · p. 39

3.1.2. LLM-as-a-Judge Paradigm

  • Rubric-guided ICE-Score [1334] and slow-thinking CodeJudge [978] assess code quality, while BigCodeReward [1340] tests reward-model preferences using real-world execution feedback.BigCodeReward includes logs, screenshots, application states, and plots; LLM judgments can also be unstable when models produce verbose explanations.Source: §3.1.2 · pp. 39–40

3.1.3. Execution-Based Metrics

  • ProbeGen [43] and REFUTE [907] use generated probes or counterexamples to expose inequivalent or buggy code; EvaLooop [284] measures robustness through average sustainable loops.EvaLooop repeatedly transforms outputs into prompts using generation/summarization and cross-language translation, tracking tasks that remain correct across iterations.Source: §3.1.3 · p. 42

3.1.4. Multi-Agent & Advanced Reasoning Framework

  • MCTS-Judge [1050] and CodeVisionary [1038] add search- and multi-LLM reasoning; Incoherence [991], MAD [705], SBC [800], and arena or critique benchmarks assess consistency, alignment, and preferences.MAD tests evaluator sensitivity to presentation changes; SBC reverse-engineers requirements, while arenas collect preference votes and CodeCriticBench evaluates correctness and critique quality.Source: §3.1.4 · p. 42, §3.1.5 · p. 42, §3.1.6 · pp. 42–43

3.2.1. Code Completion and Code FIM

  • Completion benchmarks cover statement continuation, function recovery, class methods, and FIM; class tasks test stateful dependencies, while FIM evaluates missing-code prediction from left and right context.Class-level evaluations report performance drops relative to function-level tasks, and FIM benchmarks assess syntax-aware infilling across languages.Source: §3.2.1 · p. 43, Code Completion · p. 44

3.2.2. Code Generation

  • Generation benchmarks span unit-tested function synthesis, stateful class tasks, and specialized competition, scientific, systems, efficiency, verification, and retrieval-grounded domains.HumanEval-XL scales multilingual evaluation to 22,080 problems across 23 natural languages and 12 programming languages; competition benchmarks probe increasingly difficult challenges and the gap to expert programmers.Source: §3.2.2 · pp. 46–47

3.2.4. Code Efficiency

  • GPT-4 solutions can be up to 3× slower and use 14∼44× more memory than optimized human baselines on EffiBench [419].Code-efficiency benchmarks also measure runtime, memory, energy, and algorithmic complexity, including cross-language comparisons and input scaling.Source: §3.2.4 · p. 48
  • Code efficiency methods optimize runtime and memory during fine-tuning, jointly train for correctness and efficiency, or refine inefficient code after generation.EffiCoder incorporates runtime and memory signals into its training objective; ACECode uses reinforcement learning, while Rethinking Code Refinement iteratively detects and improves inefficient code.Source: §3.2.4 · p. 49

3.2.5. Code Preference

  • CodeArena combines difficulty-weighted correctness with runtime ranking, while CodePrefBench and automatic arena judges evaluate whether model preferences align with human developers.AutoCodeArena replaces costly human preference collection with LLM judgments, comparing outputs against a fixed baseline and aggregating pairwise comparisons with the Bradley-Terry model; it samples 600 prompts across six topics and runs generated code in a Docker sandbox.Source: §3.2.5 · p. 49

3.2.6. Code Reasoning and Question Answering

  • Code reasoning benchmarks span question-answer datasets and tasks such as execution prediction, program equivalence, static analysis, and multimodal visual-programming reasoning.CodeQA contains over 100,000 question-answer entries, while CodeSense targets fine-grained semantic reasoning in real-world projects; SpecEval, CRUXEval-X, EquiBench, CORE, and ScratchEval assess distinct reasoning capabilities.Source: §3.2.6 · pp. 49–50

3.2.7. Code Translation

  • Translation research progresses from AST-based and unsupervised sequence models to execution feedback, reasoning prompts, agent workflows, and specialized safety-critical migration.Approaches use tests, compiler feedback, and runtime analysis to catch semantic errors; C/C++-to-Rust tools add differential fuzzing, reference oracles, and static analysis. Data-scarcity methods include aligned multilingual snippets, back-translation, and federated learning, while evaluation expands beyond BLEU to complexity and error taxonomies.Source: §3.2.7 · pp. 50–51

3.2.8. Test-Case Generation

  • Test-case benchmarks progress from software-engineering coverage and bug-discrimination tasks to competitive-programming evaluations of whether tests reject wrong submissions while passing correct ones.SWT-Bench and TestGenEval use buggy code and fixes, with TestBench extending the approach to Java; competitive-programming benchmarks move beyond TestEval’s coverage metrics toward functionality-driven discrimination.Source: §3.2.8 · pp. 52–53

3.3. Repository-Level Tasks

  • Repository-level benchmarks draw on repository-based commit messages, issues, and pull requests to assess models across software-development tasks.Repository-level completion and generation use broader project context—including multiple files, dependencies, and cross-file relationships—rather than only the immediate file or function.Source: §3.3 · p. 53, §3.3.1 · p. 53

3.3.1. Code Generation and Completion

  • RepoCoder, which leverages full-repository context, outperforms other methods in 90% of RepoEval tests, but benchmarks expose persistent difficulties with non-standalone code, dependencies, and incomplete suffixes.Other evaluations span multilingual cross-file context, repository dependency reasoning, secure generation, and manually annotated tasks; current models still perform poorly on several of these challenging settings.Source: §3.3.1 · pp. 53–54

3.3.2. Domain-Specific and Complex Code Generation

  • Complex-generation benchmarks cover bioinformatics, paper reproduction, library implementation, multi-file projects, and data-science tasks, where current agents often struggle to complete systematic projects.On Commit0, agents pass some unit tests but cannot reproduce entire libraries, and interactive feedback helps them pass more tests; ProjectEval and DA-Code also find current agents perform poorly on repository-level and complex data-science tasks.Source: §3.3.2 · pp. 54–55

Code Editing, Refactoring, and Agent Collaboration

The section surveys benchmarks and methods spanning code edits, repository understanding, and software-development agents. Evaluations increasingly target realistic, multi-step work, while reported results show persistent difficulties in several settings.

3.3.3. Code Editing, Refactoring, and Agent Collaboration; 3.3.4. Commit Message Generation

  • Editing benchmarks span small Python changes, long-method refactoring, and multilingual exercises; commit-message research has shifted toward pretrained, retrieval-augmented transformers, with developer-intent alignment still challenging.HumanEvo reports that ignoring project evolution overestimates performance by 10.0% to 61.1%.Source: §3.3.3 · pp. 55–56, §3.3.4 · p. 57

3.3.5. Software Engineering Tasks

  • SWE-bench extensions cover multiple languages, multimodal issues, evolving repositories, performance optimization, and feature work, while newer evaluations also test economic value and long-term agent maintenance.FEA-Bench’s best models solve only around 10% of cases, and Multi-SWE-bench reports strongest results in Python and weakest in C/C++/Rust.Source: §3.3.5 · pp. 57–58

3.3.6. Comprehensive Software Development

  • Newer benchmarks extend evaluation beyond code snippets to project documentation, issue resolution, developer tools, and evolving repositories; Stack-Repo highlights the importance of curated, deduplicated source-code data.OmniGIRL includes images in bug reports, where current LLMs significantly struggle; GitGoodBench finds failures on complex version-control tasks such as merge conflicts.Source: §3.3.6 · pp. 58–59

3.3.7. Repository-Level and Long Context Understanding

  • Up to 5.6× compression without degrading performance, LongCodeZip’s training-free framework compresses code contexts for completion, summarization, and question answering.Its two-stage strategy selects function-level chunks, then segments retained functions and optimizes the token budget; it also reduces API costs and latency.Source: §3.3.7 · p. 59

3.4.1. Agent Tool Use; 3.4.2. Deep Research Benchmarks

  • Tool-use benchmarks progress from single calls to multi-step workflows, while deep-research evaluations demand analyst-grade synthesis and expose a gap from tasks humans solve with over 90% accuracy.Source: §3.4.1 · p. 60, §3.4.2 · p. 60

3.4.3. Web Search Benchmarks

  • Web-search benchmarks test fragmented evidence gathering and systematic exploration, yet reported top models achieve near-zero success on end-to-end tasks.Source: §3.4.3 · pp. 60–61

3.4.4. Benchmarking Agents for Graphical User Interfaces

  • GUI benchmarks address both interface navigation and interface development, including cross-platform generalization, long-horizon interaction, and multi-file website creation.Foundational navigation evaluations reveal difficulty with long-horizon reasoning and precise UI grounding; newer development tasks model sequential project workflows.Source: §3.4.4 · pp. 61–62

3.4.5. Terminal Use

  • Terminal-Bench makes the entire system environment the problem space, requiring agents to configure systems, manage dependencies, and complete workflows such as deploying a server.Source: §3.4.5 · p. 62

4. Alignment

This section surveys how code LLMs are aligned with coding objectives and developer expectations, covering instruction and reasoning-data preparation, reinforcement learning and preference optimization, reward design, and evaluation across code and software tasks. It also addresses multimodal generation, tool-grounded feedback, and persistent challenges in robustness and feedback quality.

4.1. Supervised Fine-tuning (SFT)

  • SFT instruction data covers generation, repair, and translation, but human-derived Natural-Instruct data can be inconsistent or poorly formatted, motivating synthetic instruction-data approaches.Self-Instruct [662] and CodeAlpaca [152] use model-generated demonstrations; broader task coverage is intended to improve adaptability to real-world scenarios [599].Source: §4.1 · pp. 62–63

4.1.1. Single-Turn Supervised Fine-tuning

  • Single-turn data methods address complexity, diversity, and instruction sensitivity through iterative augmentation, human-data rewriting, and minimal-edit tests of whether outputs follow changed constraints.Evol-Instruct [662] increases task complexity; Semi-Instruct [657] and OSS-Instruct [1067] use human-written data or code snippets, while CTF-Instruct [659] tests sensitivity with automated metrics and human judgments.Source: §4.1.1 · pp. 63–64

4.1.2. Multi-Turn Supervised Fine-tuning

  • Multi-turn SFT uses compiler or interpreter feedback to improve failed code, then self-distillation can teach models to produce optimized solutions directly from the original prompt.AIEV-Instruct [528] pairs questioner and programmer agents; scheduled-sampling-style masking gradually emphasizes the final single-turn response during training [844].Source: §4.1.2 · p. 64

4.1.3. SFT for Repository Tasks

  • Repository-level SFT targets cross-file dependencies and long-context work; datasets span software-engineering tasks and cybersecurity, while scaling and diversity remain promising amid costly, difficult-to-quality-control data.SWE-smith [1187] scales GitHub task instances, while SWE-Synth [794] adds verifiable bug-fix trajectories; Cyber-Zero [1341] synthesizes CTF trajectories, and CTF-Dojo [1342] offers execution-ready challenges with CTF-FORGE runtime construction success above 98%.Source: §4.1.3 · pp. 64–65

4.1.4. Reasoning-based Methods

  • Reasoning-trace SFT teaches structured problem solving, but RLVR mainly sharpens patterns established by SFT and rejection-sampling fine-tuning rather than discovering wholly new reasoning paths.CoT acts as a scratchpad, and trace structure may matter more than factual correctness; RFT filters sampled candidates for correctness, making diverse, high-quality initial training important.Source: §4.1.4 · pp. 66–67

4.1.5. Training Strategies

  • Training strategies include data deduplication, difficulty and correctness filtering, multitask balancing, and denoising; remaining dataset challenges include leakage, simple-task bias, and limited language coverage.The language gap involves differences in semantics and ecosystem conventions, not just syntax; broader task-complexity coverage is needed to improve performance on demanding programming work.Source: §4.1.5 · p. 67, §4.1.6 · pp. 67–68

4.2.2. Data Cleaning and Decontamination

  • Reasoning-data pipelines balance small, high-quality sources against scale, then use semantic decontamination, verifiability checks, and model-aware filtering to improve data integrity and reward reliability.OpenThoughts3 [354] found that mixing a few high-quality sources can outperform relying on sheer source diversity; DeepMath-103K [395] uses semantic comparisons and filters for consistent answers across teacher solutions.Source: §4.2.1 · pp. 68–69, §4.2.2 · p. 69

4.2.3. Question Filtering and Quality/Difficulty Assessment

  • Question filtering depends on the goal: intermediate-difficulty “Goldilocks” sets favor sample efficiency, while high-difficulty verifiable problems can push RL performance; reasoning chains range from teacher traces to tool-integrated pipelines.OpenMathReasoning [707] generates tool-integrated reasoning with Python execution, while AceReason-Nemotron [170] and Skywork-OR1 [389] format solutions for rule-based verification.Source: §4.2.3 · p. 70, §4.2.4 · pp. 70–71

4.2.4. Reasoning Chain Generation; 4.2.5. Solution Filtering and Refinement

  • Reasoning datasets use specialized generation and filtering, but OpenThoughts3 [354] found that answer filtering did not significantly outperform training on all generated answers.LIMO [1216] scores meta-reasoning features, OpenMathReasoning [707] retains tool-use solutions with novel, significant code execution, and DeepMath-103K [395] requires three solutions to agree on a verifiable answer. Reasoning traces may also be generated through iterative tool use or formatted for reinforcement learning.Source: §4.2.5 · pp. 71–72, §4.2.4 · p. 71

4.2.6. Final Dataset Construction

  • Final dataset scale reflects project goals: LIMO [1216] and s1 [714] use hundreds to a thousand examples, while OpenThoughts3 [354] uses 1.2 million.OpenMathReasoning [707] combines three tasks in a 5.5-million-sample dataset, while DeepMath-103K [395] and OpenMathReasoning [707] also target broad reasoning with over 100,000 and 500,000 unique problems, respectively.Source: §4.2.6 · p. 72

4.3.1. Multilingual Code LLMs

  • Multilingual code models grew from small, single-language systems to ultra-large models covering hundreds of languages, alongside expanding data sources and architectural diversity.The passage cites DeepSeek-Coder-V2 at 236B parameters and 338 languages, and StarCoder2 at 600+ languages trained on 4TB+ data; corpora evolved toward multi-source fusion, with encoder, encoder-decoder, decoder, and graph-based architectures.Source: §4.3.1 · pp. 73–74

4.3.2. Multilingual Code Evaluation

  • Code evaluation expanded from Python-focused generation benchmarks to multilingual, multimodal, and repository-level software-engineering tasks.Examples include HumanEval-XL’s 23 natural languages × 12 programming languages and recent SWE-bench Multimodal and Multi-SWE-bench efforts.Source: §4.3.2 · p. 75

4.4.1. Vision-Language Foundation Models for Code; 4.4.2. Core Challenges and Technical Positioning

  • Vision-language foundations use visual encoders and lightweight adapters, while multimodal code generation must balance visual and functional fidelity with executable output.The survey describes connector approaches such as BLIP-2 [558], interleaved image-text models such as Flamingo [32], and instruction-tuned LLaVA [532, 606, 607]; MCG aims to turn visual design intent into executable, renderable code.Source: §4.4.1 · pp. 76–77, §4.4.2 · p. 78

4.4.3. Frontend Interface Generation

  • Frontend code generation progressed from screenshot-to-code toward design, sketch, and interaction tasks, with hierarchical generation and automated feedback supporting more complex interfaces.Design2Code [900] evaluates screenshot-to-HTML across 43,000 pairs, while Interaction2Code [1118] covers 127 webpages, 374 interactions, and 31 event types; UICoder [1086] uses compile-render-CLIP feedback to filter synthetic training data.Source: §4.4.3 · pp. 78–79

4.4.4. Web-Embodied Intelligence; Methodological Development Timeline

  • Web-embodied intelligence requires agents to observe, reason, and act in live environments, combining code generation with planning and interaction.ReAct [1209] interleaves reasoning and action, while WebShop [1205] and WebArena [1325] provide progressively more realistic environments for evaluating multi-step tasks.Source: §4.4.4 · p. 80, Methodological Development Timeline · p. 80

Methodological Development Timeline

  • Later agent work spans screenshot-based web navigation, open-ended game learning, hierarchical planning, multi-agent collaboration, and tool-augmented multimodal reasoning.WebVoyager [383] uses screenshots as input, Voyager [1014] learns reusable skills in Minecraft, and Agent-E [6] separates planning from browser navigation; long-horizon perception-to-action remains challenging.Source: Methodological Development Timeline · pp. 80–81

4.4.5. Software Engineering Artifact Generation

  • Software-engineering artifacts—including diagrams, charts, and documentation—support understanding across requirements, design, development, and maintenance.These artifacts include UML, data charts, flowcharts, and architecture diagrams.Source: §4.4.5 · p. 81

Sub-task Classification and Progress

  • Multimodal code-generation work spans chart-to-code, software-diagram generation, visual bug fixing, and programming benchmarks with visual inputs.ArtifactsBench renders outputs and uses MLLM judges to assess code correctness, visual fidelity, and interaction completeness in an automated, scalable evaluation loop.Source: Sub-task Classification and Progress · pp. 81–83

4.4.6. Technical Trends and Future Outlook

  • Multimodal generation increasingly uses agentic planning, iterative self-correction, code-in-the-loop feedback, and hierarchical generation, while real-world robustness and fine-grained UI understanding remain difficult.Future directions include stronger long-term planning and self-healing, lower-cost automated evaluation, and expansion to mobile and desktop applications.Source: §4.4.6 · pp. 83–84

4.5.1. Reinforcement Learning (RL) Algorithms

  • Code LLM reinforcement learning uses executable rewards such as tests, compiler errors, and static-analysis signals; PPO, DPO, and RLAIF offer distinct online, offline, and AI-feedback pathways.PPO uses KL-regularized policy updates and repair loops; DPO learns from preference pairs without online rollouts, while RLAIF can use verifiers or critic models for structured feedback.Source: §4.5.1 · p. 84

Proximal Policy Optimization Methods and Variants

  • 50% accuracy on AIME 2024 with a 32B model let DAPO surpass DeepSeek-R1 at half the training cost.DAPO combines Clip-Higher, dynamic sampling, token-level loss, and overlong-reward shaping; related variants address critic overhead, length bias, and value-model challenges.Source: Proximal Policy Optimization Methods… · p. 85

Direct Preference Optimization (DPO) and Veriants

  • DPO removes the explicit reward-model and on-policy sampling steps, but its effectiveness varies with the base model, data, and hyperparameters.DPO-VP uses verifiable correctness signals for preferences, while CodeDPO methods construct preference pairs through self-generation and validation; RLAIF scales feedback with AI evaluators but faces bias, reward-hacking, and cost risks.Source: Direct Preference Optimization (DPO)… · pp. 85–87

4.5.2. RL for Code Generation

  • Code-generation RL ranges from PPO-based synthesis and preference optimization to verifiable-reward training, while completion methods use retrieval-oriented learning and developer feedback.DeepCoder matched o3-mini with 14B parameters on LiveCodeBench; RLCoder improved CrossCodeEval and RepoEval by 12.2%, and aiXcoder-7B-v2 achieved up to 44% improvement in exact match scores.Source: §4.5.2 · pp. 87–89

4.5.3. RL for Code Understanding

  • For code understanding, actor-critic and reward-guided methods address exposure bias in summarization, while comment-generation approaches reward semantic relevance and practical utility.Methods incorporate AST structure, hierarchical attention, statement-level traversal, and rewards for comments useful in code refinement.Source: §4.5.3 · pp. 89–90
  • RL improves code-search alignment, using annotation and query-enrichment rewards to outperform traditional matching-based methods.CoaCor [249, 436, 874, 1212] rewards annotations that distinguish relevant snippets, while QueCos [1010] uses search performance to guide query enrichment.Source: §4.5.3 · p. 90

4.5.4. RL for Software Engineering

  • RL supports autonomous issue repair and review, while Refactor-RL improves BLEU by 11.96% and CodeBLEU by 16.45% over supervised fine-tuning.Refactor-RL uses compilation success and refactoring presence as rewards, increasing successful unit-test passes from 41 to 66.Source: §4.5.4 · pp. 90–91

4.5.5. RL for Code Security

  • RL-guided bug isolation and localization improve security-oriented detection, while repair methods use iterative feedback, discriminative tests, and transferred reasoning.RLocator optimizes ranking metrics directly; RePair, Repair-R1, and Repairity apply process-based supervision, joint test-and-repair optimization, or staged reasoning transfer.Source: §4.5.5 · pp. 91–92

4.5.6. Code Testing

  • RL-based test generation and fuzzing use interaction, adversarial rewards, coverage, and vulnerability signals to discover faults and improve code-generation evaluation.CovRL-Fuzz identified 58 real-world security bugs, including 50 previously unknown bugs.Source: §4.5.6 · p. 92

4.6. Applying Reinforcement Learning with Verifiable Rewards

  • RLVR uses an external verifier, often returning a binary pass/fail reward, and GRPO estimates advantages across generated samples without a learned critic.These properties suit code synthesis and other reasoning tasks whose correctness can be checked computationally.Source: §4.6 · p. 93

4.6.1. RLVR-Suitable Datasets for Code Tasks

  • Verified test suites and checkers provide deterministic correctness signals that support large-scale RLVR training and rigorous evaluation without human labeling.The datasets span programming challenges and enable gains in code-generation accuracy and reasoning depth through self-play with verifiable rewards.Source: §4.6.1 · p. 96

4.6.2. Representative RLVR-Trained Open-Source Code LLMs

  • 63% pass@1 on LiveCodeBench (Aug 2024–Feb 2025) was achieved by Skywork-OR1’s 32B model, while DeepCoder-14B reached 60.6% pass@1, matching o3-mini-2025.Code-R1’s 7B Qwen2.5 model gained 5-6% pass@1 on HumanEval+ over its SFT base; the surveyed models also use varied RLVR strategies and emphasize high-quality verified data and strong base models.Source: §4.6.2 · pp. 98–99

4.6.3. Reward Shaping in Code Post-training

  • Reward shaping combines correctness, task alignment, security, and structural quality with human preference modeling, outcome-supervised rewards, and process rewards.Together, these approaches target functional soundness and engineering robustness while aligning outputs with developer expectations.Source: §4.6.3 · p. 99, §4.6.4 · p. 102
  • Correctness rewards use static analysis to detect code issues and execution against test cases to assess functional behavior.Static tools include linters and AST parsers; tests can be generated from function signatures or natural-language descriptions.Source: §4.6.3 · pp. 99–100

4.6.4. Quality-Oriented Rewards

  • Quality-oriented rewards assess task intent, discourage vulnerabilities and weak access controls, and promote modularity, reusability, and interface compatibility.Security signals can use static analysis or taint tracking, while structural rewards favor clear responsibilities and adherence to existing APIs and contracts.Source: §4.6.4 · p. 100
  • Human preference models learn scalar rewards from ranked code completions, capturing readability and idiomatic style that objective correctness and quality criteria may miss.Preference pairs can train reward models with pairwise ranking loss or DPO, which can then guide post-training through methods such as PPO or DPO.Source: §4.6.4 · p. 101
  • PRM assigns stepwise rewards using local validity and global trajectory coherence, through token-level predictors or state tracking with structures such as ASTs or control-flow graphs.Token-level PRM integrates efficiently with decoding; state-tracking PRM requires symbolic reasoning modules but offers deeper contextual awareness.Source: §4.6.4 · p. 101
  • PRM is reported to reduce failure rates versus end-of-sequence rewards, particularly on multi-step tasks, while enabling progressive construction from validated components.The passage gives the reduction as “TODO XX%,” so no quantitative magnitude is available.Source: §4.6.4 · pp. 101–102

5. Software Engineering Agents

The section organizes software engineering agents by lifecycle tasks and relates their applications to data curation, fine-tuning, and reinforcement-learning strategies. It surveys requirements engineering and program implementation, including feedback-driven and repository-level agents.

5. Software Engineering Agents

  • SWE agents are organized across lifecycle phases, and the section examines training paradigms including data curation, fine-tuning, and reinforcement learning.The framework covers requirements analysis and design, development, testing, and deployment.Source: §5 · p. 102

5.1.1. Requirements Engineering

  • Requirements-engineering agents address acquisition, reconciliation, modeling, and assurance, with approaches ranging from phase-specific systems to integrated multi-stage pipelines.The integrated systems connect stage outputs so they can inform subsequent work and validation.Source: §5.1.1 · pp. 102, 104, 105
  • End-to-end requirements systems coordinate collectors, analyzers, modelers, and validators in shared environments for traceable, feedback-driven work across stages.MARE links acquisition, modeling, and validation; iReDev uses six domain-specific agents and shared event-based repositories.Source: §5.1.1 · pp. 104–105

5.1.2.1. Program Implementation

  • Program-synthesis agents generate programs from specifications and tests, using either single-agent self-improvement or multi-agent collaboration across coding, testing, and debugging.Single-agent loops are simpler but offer less multi-perspective collaboration; multi-agent designs divide work among specialized roles.Source: §5.1.2.1 · pp. 105–106
  • Feedback turns program generation into guided search through sampling, refinement, hybrid search, and consistency-based reranking; one to three refinement rounds usually yield the largest gains.Additional rounds can plateau or regress when feedback is noisy or misleading, while unbounded sampling can be inefficient with incomplete verifiers or heuristic ranking.Source: §5.1.2.1 · pp. 106–107
  • Hybrid sampling and refinement achieved near-SOTA results on LiveCodeBench while using far less compute than an equivalent large model.The approach samples multiple candidates, refines them independently, then selects among them using additional tests, including AI-generated adversarial inputs.Source: §5.1.2.1 · p. 108
  • Repository-level agents extend coding loops with planning, tool use, memory, and dependency-aware retrieval to manage cross-file consistency and long-context tasks.Dependency-aware generation and retrieval shorten context length while retaining semantic coherence.Source: §5.1.2.1 · pp. 108–109
  • Text-to-SQL has evolved from rule-based and template systems to LLM-driven pipelines using schema linking, execution feedback, and multi-agent collaboration.Source: §5.1.2.1 · p. 109

TABLE Match

These passages survey LLM-based agents and methods across software-engineering tasks, including code generation, editing, testing, maintenance, and development workflows. They cover prompting, retrieval, tool use, feedback, supervised fine-tuning, and reinforcement learning, as well as directions for integrated, context-aware, collaborative, and verifiable systems.

TABLE Match

  • A cited result reports accuracy about 10 percentage points above the previous best, but the supplied passage does not identify the system or benchmark for that comparison.The review also discusses chain-of-thought prompting [933], schema linking and retrieval, and multi-agent SQL generation and refinement.Source: TABLE Match · pp. 109–111

5.1.2.2. Program Analysis

  • CodeAgent [938] reports a 41% higher vulnerability detection rate than GPT-4, illustrating agent-based code review with hierarchical roles.Comment-generation methods progress from code-comment mapping and neural models to LLMs, retrieval, and agents using self-reflection or role-specialized review and refinement. Review systems also use expert agents, dynamic routing, and hybrid quality-enhancement methods.Source: §5.1.2.2 · pp. 111–113
  • Fault-localization research spans end-to-end neural repair, targeted fine-tuning, external-tool integration, and multi-agent collaboration for debugging.Methods use validation feedback and static analysis, retrieval, or repository tools to refine localization; multi-agent systems divide tasks such as context extraction, ranking, debugging, and patch generation.Source: §5.1.2.2 · pp. 113–114
  • Agent-based documentation systems target long-form software documentation by maintaining it as code evolves, coordinating specialized roles, and checking it against program behavior.Earlier approaches used retrieve-then-generate, while LLMs improved fluency but still faced technical-accuracy and code-change challenges; RepoAgent [655], DocAgent [1173], and METAMON [524] respectively emphasize maintenance, role collaboration, and metamorphic verification.Source: §5.1.2.2 · pp. 115–116

5.1.1.3. Program Editing

  • Patch-generation approaches combine model adaptation and task reformulation with static analysis, retrieval, execution feedback, and conversational or multi-agent repair.Methods include grammar-constrained editing, reinforcement-style execution feedback, diagnostic-guided or template-constrained repair, and retrieval of bug-fix examples; agentic systems add iterative validation, repository-level search, and memory or reviewer components.Source: §5.1.1.3 · pp. 116–118
  • SWE-Agent [1184] and OpenHands [1040] established dedicated agent–computer interfaces that improve LLM agents’ operation in programming environments.These interfaces simplify commands, manage feedback and history, and support editing safeguards such as rollback; issue repair then connects problem understanding, fault localization, patch generation, and validation.Source: §5.1.1.3 · p. 118
  • Modular repair workflows divide complex, multi-file issues into specialized tasks, while simpler staged designs aim to improve efficiency, stability, and deployability.Approaches span role-based collaboration and staged pipelines for localization, repair, and validation.Source: §5.1.1.3 · pp. 119–120
  • Repair agents progress from graph-based repository knowledge to semantic intent modeling and experience-driven self-evolution for more structured, robust, and adaptive repair.Knowledge graphs support dependency reasoning; semantic methods combine program, code, and execution signals; self-evolution methods learn from prior repair trajectories.Source: §5.1.1.3 · pp. 121–122

5.1.3. Software Testing

  • LLM test generation has shifted from producing syntactically valid cases toward improving correctness, coverage, semantic relevance, and iterative execution-driven refinement.Systems use intent inference, documentation and error feedback, coverage or mutation feedback, and multi-step generation-and-repair pipelines.Source: §5.1.3 · pp. 122–123
  • Agentic testing extends beyond unit tests to fuzzing, mutation testing, bug reproduction, interactive feature evaluation, and autonomous test execution across varied environments.These approaches use continuous feedback, generated mutation operators, coordinated agents, and environment construction with iterative error recovery.Source: §5.1.3 · pp. 123–124

5.1.4. Software Maintenance

  • Log analysis has moved from deterministic parsing and neural methods toward agents that reason iteratively, use tools, and coordinate specialized analyses.Single-agent systems refine hypotheses or escalate difficult cases, while multi-agent frameworks coordinate evidence gathering and validation.Source: §5.1.4 · pp. 124–126
  • Compiler optimization progresses from heuristic search and static prediction to agents that adapt strategies through feedback, reinforcement learning, and simulated compiler outcomes.World-model-driven optimization uses offline reinforcement learning to explore strategies and support generalization across languages and architectures.Source: §5.1.4 · pp. 126–127
  • Decompilation has evolved from neural translation to LLM- and agent-based reconstruction that combines semantic reasoning with structured program analysis.Agentic approaches fuse disassembly outputs and control-flow structure; deobfuscation work ranges from probabilistic and neural methods to LLMs using contextual information and developer symbol preferences.Source: §5.1.4 · pp. 127–128
  • In software maintenance, agents iteratively combine obfuscation identification, specialized tools, and semantic reconstruction to simplify code and recover logic.ALFREDO [724] selects deobfuscation strategies by obfuscation type, while Androidmeda [10] uses contextual reasoning to infer identifiers and comments.Source: §5.1.4 · p. 129
  • Agentic DevOps coordinates task-specific agents and tools, using execution feedback to adapt workflows, testing, resource allocation, and deployment responses.AutoDev [988] delegates subtasks to agents in Docker sandboxes; CI/CD agents can select tests, manage concurrency, and trigger rollbacks or alerts.Source: §5.1.4 · pp. 129–130

5.1.5. End-to-End Software Agents

  • End-to-end software agents use role-specialized workflows or iterative agile loops to cover development stages from requirements through testing, incorporating feedback and shared experience.Waterfall-inspired systems include ChatDev [812] and MetaGPT [402], while LCG [593], AgileCoder [725], and CodeS [1246] use sprint-style cycles.Source: §5.1.5 · p. 130

5.2. General Code Agents in Software Engineering

  • General code agents aim to support complete engineering workflows through multi-turn context tracking and tool use, with designs ranging from executable-code actions to repository-level understanding.CodeAct [1039] supports self-debugging through executable Python actions, while Augment [951] uses repository indexing and developer-specific memory and rules.Source: §5.2 · pp. 130–131

5.3.1. Fine-tuning SWE Agents

  • Fine-tuning SWE agents combines successful-trajectory filtering, verified or synthesized data, code-structure-aware objectives, and scalable adaptation such as PEFT and continual learning.RFT filters failed samples; methods also use execution-checked augmentation, structured task objectives, and LoRA or QLoRA to adapt a small fraction of model parameters.Source: §5.3.1 · pp. 132–134

5.3.2. Reinforcement Learning for SWE Agents

  • RL trains SWE agents through reward feedback on action outcomes, particularly when tasks are multi-step and trajectory correctness is hard to specify.Rewards may derive from passing tests or program performance, or from human preferences and the quality of intermediate reasoning.Source: §5.3.2 · p. 134, RL Algorithm Selection · pp. 135–136

RL Algorithm Selection

  • Policy-gradient methods suit large sequential action spaces, offline RL avoids costly or risky live interaction, and preference-based methods address rewards that resist objective measurement.Examples include AutoPhase for compiler-pass selection, CompilerDream for learning from a world model, and RLHF for subjective tasks such as code review.Source: RL Algorithm Selection · pp. 134–135
  • Execution-based rewards reinforce validated repairs and faster compiler outputs, while RLHF and reasoning-focused rewards shape review quality, interpretability, and collaboration.RewardRepair [1215] uses test outcomes to guide fixes; CodeMentor [723] aligns review tone and relevance with human ratings, and R-Log [633] rewards sound reasoning traces.Source: RL Algorithm Selection · pp. 135–136
  • RL could guide interactive repair by choosing whether to retry patches, request information, or take another action based on prior feedback and repair history.Reflexion [895], conversational APR systems [1111, 1112], and RepairAgent [120] illustrate trial-and-error, dialogue, and stateful feedback loops.Source: RL Algorithm Selection · p. 137
  • RL could optimize long-horizon CI/CD decisions, including deployment adjustments, subtask delegation, and recovery commands based on evolving project or failure states.The passage discusses CodeAgent [938], AutoDev [988], and ExecutionAgent [119] as settings for feedback-driven policies.Source: RL Algorithm Selection · p. 137
  • SFT bootstraps RL by constraining sparse, vast action spaces, while successful RL trajectories can feed later SFT rounds to consolidate discoveries and improve agents iteratively.Automated repair examples include AlphaRepair [1110], T5APR [319], and RewardRepair [1215]; SWE-Exp [163] and SE-Agent [594] illustrate experience and reasoning improvements that can become fine-tuning data.Source: RL Algorithm Selection · p. 138
  • A comprehensive SFT phase followed by targeted RL balances a strong task foundation with goal-oriented refinement; too much SFT limits exploration, while too little destabilizes RL.The passage identifies the SFT-to-RL allocation as a critical hyperparameter and says RL can be shorter when it refines behavior rather than teaching the task from scratch.Source: RL Algorithm Selection · p. 138

5.4. Future Trends: Towards Integrated and Autonomous Software Engineering Ecosystems

  • Future SWE agents could orchestrate the full lifecycle, from requirements refinement and code generation through testing, deployment, and maintenance.ChatDev [812], MetaGPT [402], and AgileCoder [725] are described as early blueprints for cross-lifecycle integration.Source: §5.4 · p. 139
  • Persistent, dynamic project knowledge graphs could extend repository understanding beyond static structure to include evolutionary history, runtime behavior, and implicit design principles.KGCompass [1165] links issue descriptions to code entities, while CGM [944] combines graph retrieval with generation.Source: §5.4 · p. 139
  • Future multi-agent ecosystems could combine specialized roles with self-evolution through historical experience and internal debate to refine strategies over time.CodeAgent [938] and MAGIS [947] handle complex tasks through role division; SWE-Exp [163], SWE-Debate [540], and SE-Agent [594] explore learning and self-iteration.Source: §5.4 · p. 139
  • Practical agents should pair mixed-initiative human oversight with security-by-design, including vulnerability mitigation and formal verification for safe, trustworthy deployment.Aider [949], ChatRepair [1112], and AutoDev [988] illustrate human-guided interaction; multi-agent fuzz testing [739], ESBMC-AI [972], and ContractTinker [1012] illustrate security and verification directions.Source: §5.4 · pp. 139–140

6. Code for Generalist Agents

The section examines code as a medium for generalist agents across interaction protocols, agent reasoning and action, memory, simulated tasks, and computer-use environments.

6.1.1. Tool Use

  • Function calling turns tool descriptions and user requests into parameterized executable calls, with tuning-free prompting and fine-tuned approaches offering alternative ways to align invocations.ReAct interleaves calls with observations, while ReWOO plans calls before execution and DERA pauses for clarification when instructions are ambiguous.Source: §6.1.1 · pp. 141–142

6.1.2. Model Context Protocol; 6.1.3. Multi-Agent Coordination

  • MCP manages tool interactions through request, response, state update, and re-invocation, while A2A delegates work among agents that exchange information and integrate results.MCP structures model-tool context; A2A coordinates specialized agents through message passing and shared context.Source: §6.1.2 · p. 142, §6.1.3 · p. 143

6.2.1. Thinking in Code

  • Code supports executable mathematical reasoning and structured semantic control flow, while code- and pseudocode-based plans represent multi-step tasks, including multi-agent execution.PAL executes intermediate steps; CoC uses code for numerical tasks and pseudocode for semantic reasoning, while planning methods encode task trajectories.Source: §6.2.1 · pp. 143–144

6.2.2. Acting in Code

  • ToC builds hierarchical code trees and uses execution feedback and answer voting; compared with CodeAct, it achieves higher accuracy with fewer execution rounds.The framework evaluates execution success through self-supervised feedback and validates answers through majority voting.Source: §6.2.2 · p. 145
  • Code-interpreter use varies with model reasoning capability, and consistent, parallel, and confidence-based modality-selection strategies each improve reasoning accuracy.Less capable models were more inclined to use the interpreter, while more capable models more often preferred textual reasoning and showed increased errors on moderately difficult problems.Source: §6.2.2 · p. 145

6.2.3. Memory With Code

  • Agents can store validated skills as executable code for later reuse, while virtual-memory-inspired read and write calls manage internal and external context.The described approaches support skill reuse without repeated model inference and real-time modification of contextual content.Source: §6.2.3 · pp. 145–146

6.3.1. Code as Simulation Gym

  • CodeGYM environments support continuous interaction, state updates, and feedback or rewards across puzzle, spatial-reasoning, and strategy-game tasks.Its platforms include rule-based puzzle generation, synthetic spatial datasets, and interactive GameArena settings for long-term planning and strategic reasoning.Source: §6.3.1 · pp. 146–147

6.3.2. Computer-Use Agents

  • Computer-use agents span GUI, terminal, and cross-environment systems; GUI agents use text, visual, or multimodal perception and interact through simulated inputs or direct tools and APIs.Cross-environment systems can combine browsers, terminals, code executors, and file systems.Source: §6.3.2 · pp. 147–149

7. Safety of Code LLMs

This section surveys code LLM safety risks across training, adversarial attacks, and deployment, organizing safeguards around safety pre-training and post-training, red-team evaluation, and coding-agent safety. It covers feedback-based training, preference alignment, reinforcement learning, context- and agent-focused attacks, and defense in depth through secure execution, pre-execution validation, and runtime oversight.

7. Safety of Code LLMs

  • Code LLMs can produce functionally correct but insecure code, and studies find insecure outputs from open-source models such as Qwen3 and DeepSeek-R1.Advanced adversarial prompts can also elicit subtle vulnerabilities, motivating safety measures across the model lifecycle.Source: §7 · p. 149

7.1. Safety Pre-training for Code LLMs

  • Pre-training on largely uncurated public code teaches models to imitate prevalent vulnerable patterns, so safety interventions must shape both training data and learning objectives.Proposed measures include corpus auditing and cleaning, synthetic refusal examples, and adversarial augmentation.Source: §7.1 · pp. 150–151

7.1.1. Data Provenance, Security, and License Compliance; 7.1.2. Training-data Auditing and Cleaning; 7.1.3. The Regulatory and Standards in Data Security

  • Data governance remains difficult because license detection can err, code deduplication is unresolved, and AI training corpora lack widely adopted security standards and direct regulations.The Stack was filtered to 6.4TB of permissively licensed code from a raw corpus exceeding 102TB; the passage reports that, as of 2025, no direct regulations govern AI-generated code security.Source: §7.1.1 · p. 152, §7.1.2 · p. 152, §7.1.3 · p. 153

7.1.4. Robustness Against Adversarial Code Transformations

  • Code-specific obfuscations can bypass natural-language filters because semantically equivalent malicious programs may look different at the token level.Adversarial training with obfuscated examples aims to improve robustness, but increasing augmentation diversity can undermine fidelity and functional correctness.Source: §7.1.4 · pp. 153–154

7.1.5. Privacy Risk Assessment and Mitigation in Pre-training Data

  • Pre-training data can expose models to personal information, credentials, and proprietary code, while deduplication and detect-and-mask sanitization reduce—but do not eliminate—privacy risks.Residual risks include missed secret formats and memorization through model embeddings, warranting post-training audits and awareness of deployment vulnerabilities.Source: §7.1.5 · p. 154

7.1.6. Bias Assessment and Mitigation

  • Code LLMs exhibit social biases, and conversational safety fine-tuning does not reliably transfer to code generation; scalable mitigation that preserves coding ability remains open.Assessment approaches include sensitive-attribute prompts and AST analysis; model editing has been explored to remove specific gender associations.Source: §7.1.6 · p. 155

7.2.1. Pre-training Limitations and the Necessity of Post-training Alignment

  • Likelihood-based pre-training can reproduce vulnerabilities and hallucinate APIs, with studies reporting that approximately 40% of generated programs contain security flaws.A single path-traversal pattern has appeared in over 1,700 projects; targeted post-training alignment, including SFT and RLHF, is presented as necessary for secure generation.Source: §7.2.1 · pp. 156–157

7.2.2. Data as the Cornerstone: Constructing Safety-related Training Datasets

  • Security-alignment datasets draw on real vulnerability fixes, synthetic examples, and preference pairs, but real-world data requires rigorous cleaning to address noise and redundancy.CVEfixes [104] and BigVul [385] use repository fixes; HexaCoder [377] and ProSec [1148] synthesize targeted examples, while preference triplets support DPO [828].Source: §7.2.2 · pp. 157–158

7.2.3. Safety Supervised Fine-Tuning for Code LLMs; 7.2.4. Advanced Preference Optimization for Localized Flaws

  • Feedback-based SFT uses tools and critic dialogue to iteratively validate code, while LPO targets pivotal vulnerability tokens and uses an SFT regularizer to preserve general coding.The discussion frames security training as progressing from explicit examples to global sequence preferences and token-targeted preferences, with AST-level alignment proposed as a next step.Source: §7.2.3 · pp. 158–159, §7.2.4 · p. 159

7.2.5. Coding Safety Alignment via Reinforcement Learning

  • RL safety alignment combines human or AI preference feedback with rewards from tools and AI judges, with hybrid S-GRPO rewards covering compilation, security compliance, and format correctness.Human-feedback RL trains an SFT policy, a reward model from paired preferences, then optimizes the policy; RLAIF scales feedback with a teacher LLM, while Safe RLHF separates helpfulness rewards from harmlessness costs.Source: §7.2.5 · pp. 159–160
  • RL alignment can invite reward hacking and an alignment tax, motivating constrained optimization and safety-editor policies, yet insecure code generation persists and requires layered defenses.Coding reward hacking can include altering tests or exploiting the execution environment; narrow safety signals may also cause refusals or inefficient code.Source: §7.2.5 · pp. 160–161

7.3. Red-teaming Techniques for Code LLMs; 7.3.1. Prompt-Level Manipulation: Subverting Input-Output Behavior

  • Red-teaming progresses from heuristic prompt jailbreaks to optimized, generated, and multi-turn attacks that probe model alignment and expose weaknesses across interaction.Methods include role-play and prefix injection, GCG adversarial suffixes, LLM-generated fuzzing prompts, and adversarial self-play that weakens guardrails over a conversation.Source: §7.3 · p. 161, §7.3.1 · pp. 161–162

7.3.2. Semantic and Contextual Manipulation: Exploiting the Interpretation Layer

  • Semantic attacks exploit the gap between benign-looking syntax and harmful meaning through trusted-data boundary violations, indirect prompt injection, obfuscation, and cross-lingual inputs.Poisoned external documents can embed instructions that agents mistake for user intent, while low-resource languages can evade classifiers trained mostly on high-resource languages.Source: §7.3.2 · p. 162

7.3.3. Agentic Workflow: Subversion of Agent Systems and Tool Use

  • Agentic red-teaming targets actions and infrastructure through malicious tool arguments, sandbox escapes, and autonomous vulnerability exploitation, marking a shift toward attacks on entire workflows.The section characterizes the trajectory as moving from unsafe-output prompts to attacks spanning data sources, tools, and operational environments.Source: §7.3.3 · pp. 162–163

7.4. Mitigation Strategies for Coding and Behavioral Risks in AI Agent Systems; 7.4.1. Foundations in Secure Execution Environments

  • Defense in depth contains agent capabilities through isolation, but container, sandbox, and virtualization choices trade performance and compatibility against security, while static policies can be too restrictive or permissive.Dynamic containment adjusts isolation to observed behavior and risk; Progent exemplifies programmable privilege control, while SandboxEval and InterCode support evaluation of containment approaches.Source: §7.4 · p. 164, §7.4.1 · pp. 164–165

7.4.2. Proactive Defense and Pre-Execution Validation; 7.4.3. Runtime Oversight and Intent Grounding

  • Pre-execution validation combines tool-in-the-loop code repair, multi-agent review, and formal intent constraints, while runtime oversight monitors actions, enforces policies, and can intervene.Runtime approaches include guardian agents, verifiable policies, and resampling unsafe actions; future work aims to ground action monitoring in causality and intent.Source: §7.4.2 · pp. 165–166, §7.4.3 · p. 166

8. Training Recipes for Code Large Language Model

This section presents training recipes for code LLMs spanning pre-training, supervised fine-tuning, and reinforcement learning, including guidance on distributed systems and optimization. It examines multilingual pre-training, practical SFT choices, and RLVR for code generation, deriving recommendations that depend on whether the target is single-sample correctness or multi-sample performance.

8.1. Distributed Training Framework Introduction

  • Framework choice should match infrastructure and scale: Megatron-LM suits high-bandwidth clusters, DeepSpeed prioritizes memory efficiency, PyTorch FSDP and TorchTitan integrate with PyTorch, and Colossal-AI offers flexible parallelism.Megatron-LM combines tensor, pipeline, and sequence parallelism; DeepSpeed ZeRO partitions optimizer states, gradients, and parameters, while TorchTitan composes four-dimensional parallelism.Source: §8.1 · pp. 167–168

8.2. Pre-Training Guidelines

  • Python has the largest model-size and data scaling exponents, while compiled Rust and Go have smaller exponents; irreducible loss ranges from C# (0.288) to Python (0.566).Lower-loss C#, Java, and Rust saturate sooner, while higher-loss Python and JavaScript retain greater potential returns from extended training.Source: §8.2 · pp. 169–170, Table 24 · p. 169
  • Related-language mixtures can improve pre-training: Java-C# reduces loss by over 20%, while Python can harm some targets when it is the target language but help as an auxiliary language.Document-level pairing of parallel implementations improves translation and code understanding, and multilingual models show zero-shot transfer to unseen language pairs.Source: §8.2 · p. 170

8.3. Supervised Finetune Training Guidelines

  • Under the 64-GPU setup, MS-Swift reached comparable accuracy to ZeRO-3 in approximately 20 minutes, versus roughly 50 minutes for LLaMA-Factory and 2 hours for VERL.LLaMA-Factory and MS-Swift offered efficient large-scale SFT, while VERL provided full-sharding generality and PyTorch integration at the cost of runtime.Source: §8.3 · p. 172, Table 25 · p. 172
  • Global batches above roughly 256 hurt code SFT: Qwen3-30B-A3B MBPP falls from 0.860 at 64 to 0.556, 0.254, and 0.169 at 512, 1024, and 2048.The recommended batch range is 64–256; suggested learning rates are 2 × 10−6–5 × 10−6 for 14B and 5 × 10−6–1 × 10−5 for 30B, with 3–5 epochs for 14B and 3–10 for 30B.Source: §8.3 · p. 173, Table 26 · p. 174
  • Dense Qwen2.5-Coder-14B is more robust to SFT tuning, while MoE Qwen3-30B-A3B is more sensitive and benefits from longer training and precise optimization.The MoE model’s performance declines with batches above 512 or learning rates below 1 × 10−6; constant schedules tend to outperform cosine or linear schedules for it.Source: §8.3 · pp. 173, 176, Figure 43 · p. 175
  • Execution-grounded, function-level datasets transfer best to MBPP and MBPP+, contest-style corpora favor HumanEval tasks, and runnable supervision outperforms pure instructional chat data.Under a fixed 50K-sample budget, executable or test-based supervision is most robust; limited contest data can help HumanEval-type reasoning, and supervision quality matters more than raw data volume.Source: §8.3 · p. 176

8.4. Reinforcement Learning Training Guidelines

  • rloo leads estimator scores (Pass@1: 0.322; Pass@5: 0.389 at step 400), while rf++baseline converges about 30% faster at step 280 with Pass@1: 0.318 and Pass@5: 0.356.The authors recommend rf++baseline for practical large-scale training and rloo when maximum performance takes priority over training efficiency.Source: §8.4 · pp. 177, 178, 180, Figure 45 · p. 178, Figure 44 · p. 178
  • 16K tokens achieves the highest Pass@1 (0.336 at step 340), 2K achieves the highest Pass@5 (0.398 at step 380), and 4K is the balanced default; 8K underperforms.Response lengths beyond 16K yield diminishing returns while raising computational cost; longer runs use checkpointing, offloading, tensor parallelism, and more training nodes to mitigate memory and inference burdens.Source: §8.4 · pp. 178, 179, 181, Figure 47 · p. 179, Figure 46 · p. 179
  • N=512 reaches Pass@5 0.388 but is prohibitively slow; among practical settings N=8 achieves Pass@5 0.368, while Pass@1 stays within 0.315–0.319 for N=4–64.N=16 is the recommended default for compute/performance balance, while N=8 suits Pass@5-critical use with sufficient budget; larger rollout counts offer little Pass@1 gain and add computation.Source: §8.4 · pp. 180–182, Figure 48 · p. 180, Figure 49 · p. 181
  • Across lcb-v6 experiments, Pass@1 favors 16K-token responses and rf++baseline, while Pass@5 favors 2K-token responses, N=8 rollouts, and rloo.The guidelines were validated using codecontest_plus with 64 H20 GPUs and FSDP2 distributed training.Source: §8.4 · pp. 177, 182, Figure 50 · p. 181

9. Code Large Language Model for Applications

The section organizes code LLM applications into six domains, describing their architectures, deployment strategies, capabilities, development, and limitations.

9. Code Large Language Model for Applications

  • The section presents a six-domain taxonomy of code LLM applications, organized by architecture, deployment strategy, and functional capability.Each application is also reviewed for its history, technical innovations, and limitations.Source: §9 · p. 182, Figure 51 · p. 183

9.1. IDE-integrated Development Assistants

  • IDE-integrated assistants are described as the most widely deployed category, emphasizing contextual understanding, cross-file reasoning, and workflow integration.Examples include Copilot's multi-model and coding-agent features, Cursor's project-wide editing, and TRAE's plan-then-apply workflow.Source: §9.1 · pp. 183–184

9.2. Cloud-native Coding Platforms

  • Cloud-native platforms target centralized, cloud-integrated development, with AWS- and GCP-specialized assistants, browser-based Replit, and Chinese-market Tongyi Lingma illustrating distinct niches.Their described limitations include adoption challenges, constrained enterprise workflows, and limited international uptake.Source: §9.2 · pp. 186–187

9.3. Terminal-based Autonomous Agents

  • Terminal agents span repository-aware editing and feedback in Aider, extensible tool-based workflows in Claude Code, cached CLI access in Gemini CLI, and reviewable branching plans in Plandex.The section also notes limitations such as API dependence, early-stage ecosystems, and added planning overhead.Source: §9.3 · pp. 187–188

9.4. Code Repair and Verification Applications

  • 164 out of 357 bugs (45.9%) were fixed by RepairAgent on Defects4J; related approaches synthesize specifications, generate template-guided patches, or localize faulty tokens.RepairAgent uses a five-stage hypothesis-driven process; AutoSpec validates generated specifications with theorem provers.Source: §9.4 · pp. 189–190

9.5. Pull Request Review and Quality Assurance

  • Pull request review tools automate summaries and feedback; PR-Agent uses specialized review prompts, while CodeRabbit supports long diffs and multi-file impact analysis.The section notes that feedback may require human filtering and that complex business logic can be difficult to understand without extensive context.Source: §9.5 · pp. 190–191

Conclusion

The paper examines code-generating LLMs across their lifecycle, from data curation and pretraining through post-training and agent deployment. It also bridges academic benchmarks and real-world software-development challenges through systematic experiments.

Conclusion

  • The paper analyzes code-generating LLMs across data curation, pretraining, supervised fine-tuning, reinforcement learning, and deployment as autonomous agents.It examines general-purpose and code-specialized model families.Source: Conclusion · p. 191
  • The analysis connects academic benchmarks with real-world software-development challenges through experiments on scaling laws, architectures, and training methods.The experiments also compare frameworks, hyperparameters, datasets, and manual coding practices.Source: Conclusion · p. 191
  • The experiments cover code pretraining, supervised fine-tuning, and reinforcement learning across multiple dimensions, including scaling laws, framework selection, and dataset comparisons.The paper also examines model architectures, hyperparameter sensitivity, and manual coding practices.Source: Conclusion · p. 191

10. Contributions and Acknowledgements

This section lists the paper's authors and contributor groups and records institutional and training-program support.

10. Contributions and Acknowledgements

  • The author roster lists researchers alongside their institutional affiliations.The listed names include Fei Yuan, Jiayi Zhang, Xinhao Zhang, Xiantao Zhang, and others.Source: §10 · pp. 192–193
  • Core contributors and contributors appear in separate groups, with names and affiliations listed for each.The contributor roster includes researchers affiliated with universities, companies, and independent research.Source: §10 · p. 192
  • The organization and senior advisory committee is listed alphabetically, and acknowledged support includes SKLCCSE and a UKRI-funded doctoral training program.The UKRI grant number given is EP/S023062/1.Source: §10 · p. 193

Related papers

Loading 2511.18538v5…