Source-linked AI summary
AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
Huanting Wang, Jingzhi Gong, Huawei Zhang, Jie Xu, Zheng Wang
TL;DR
This survey addresses the lack of a clear conceptual landscape and standard ways to classify and evaluate a fast-emerging field. It reviews AI coding-agent architectures, techniques, and benchmarks, synthesizes the growth and capabilities of the research, and identifies challenges for safe and reliable real-world use.
Problem
The field lacks a clarified conceptual landscape and standard taxonomy, benchmark suite, or evaluation methodology for its diverse systems.
Source: §1 · p. 2Method
The survey develops a taxonomy and reviews coding-agent architectures, techniques, evaluation practices, benchmarks, challenges, and research directions.
Source: §1 · p. 2Results
The review includes 152 academic references, with publication shares rising from 5% in 2022 to 53% in 2024 before 20% in 2025.
Source: §3.3 · p. 12Takeaways & Limitations
The survey identifies context handling, structured memory, user-intent alignment, human-AI collaboration, and behavior verification as challenges for safe, reliable, usable real-world agents.
Source: §7 · p. 26Existing benchmarks often use small, self-contained problems and miss realistic large-codebase work and interactive, iterative agent behavior.
Source: §6.3 · p. 23
Abstract
from arXiv · showhide
AI agentic programming is an emerging paradigm where large language model (LLM)-based coding agents autonomously plan, execute, and interact with tools such as compilers, debuggers, and version control systems. Unlike conventional code generation, these agents decompose goals, coordinate multi-step processes, and adapt based on feedback, reshaping software development practices. This survey provides a timely review of the field, introducing a taxonomy of agent behaviors and system architectures and examining relevant techniques for planning, context management, tool integration, execution monitoring, and benchmarking datasets. We highlight challenges of this fast-moving field and discuss opportunities for building reliable, transparent, and collaborative coding agents.
1 Introduction
AI agentic programming shifts coding from one-shot code generation to LLM-based agents that plan, execute, refine, and coordinate software-development tools. This survey maps the emerging field and its architectural, technical, evaluation, and reliability challenges.
- Unlike one-shot code tools, LLM-based agents can plan and refine development tasks and use compilers, debuggers, profilers, and version control across end-to-end workflows.They can generate programs, diagnose bugs using compiler or test feedback, write tests, and refactor code.Source: §1 · p. 1
- Some state-of-the-art coding agents have demonstrated hours-long work while maintaining task consistency, avoiding deadlocks, and recovering from failed actions.The cited examples include Anthropic’s Claude Opus 4.Source: §1 · p. 1
- The field lacks a standard taxonomy, benchmark suite, or evaluation methodology, while reliability, code errors, language coverage, safety, trust, and accountability remain challenges.The survey also identifies a mismatch between human-centric development tools and agents’ need for structured internal-state access and rich feedback.Source: §1 · p. 2
- Human-centric tools often hide internal states and transformation traces, limiting agents’ ability to diagnose failures, understand code changes, and recover through iterative reasoning.Agents may need structured access to compiler representations, transformation traces, symbolic information, and execution metadata.Source: §1 · p. 2
- The survey provides a conceptual foundation and taxonomy, reviews architectures and techniques, summarizes evaluation and benchmarks, and discusses challenges and research directions.Its scope includes opportunities spanning programming languages, software engineering, AI, and human-computer interaction.Source: §1 · p. 2
2.1 AI Agentic Programming
AI agentic programming is goal-directed, multi-step software development by LLM-based agents that use tools and feedback to refine their outputs. A REST-task example illustrates how planning, tool use, and iterative refinement combine in a working development workflow.
2.1 AI Agentic Programming
- LLM-based agents perform software-development tasks through goal-directed, multi-step reasoning, tool use, and iterative refinement with limited or no human intervention.They can plan action sequences, adapt strategies, and coordinate complex workflows based on feedback.Source: §2.1 · p. 3
- A typical agent embeds an LLM in an execution loop: it gathers prompt and workspace context, reasons over subgoals, invokes tools, uses their outputs as feedback, and returns results.The loop continues until the task is complete or a stopping condition is reached.Source: §2.1 · p. 3
- Autonomy, interaction, iterative refinement, and goal orientation distinguish agentic systems from one-shot prompts and rule-based or classical machine-learning automation.These capabilities support code assistance, debugging and testing, maintenance, and potentially self-improving software systems.Source: §2.1 · p. 3
2.1.1 Working example.
- For the REST task, the agent plans log parsing and URL counting, implements an endpoint, writes and runs tests, fixes failures, and generates documentation.The example uses tools such as a Python interpreter, pytest, and documentation generators; testing repeats until cases pass.Source: §2.1.1 · p. 4
2.2 Historical Context and Motivations
The paper places AI agentic programming in a longer history of automating software development and traces its emergence from earlier code-generation approaches. It links the shift to LLM capabilities, multi-step reasoning techniques, and increasingly available development tools.
2.2 Historical Context and Motivations
- Software-development automation has long been pursued in AI and software engineering, and the paper relates coding agents’ emergence to operating-system evolution.Source: §2.2.1 · p. 4
2.2.1 A systems perspective.
- Earlier systems included active objects with their own control and asynchronous messaging; coding agents extend this trajectory through proactive planning and coordination across contexts.The paper contrasts them with passive objects and fixed handwritten policies.Source: §2.2.1 · p. 5
- Program synthesis targeted correct-by-construction programs from formal specifications, while code-completion tools predicted likely snippets from context; both preceded LLM-driven agentic workflows.Source: §2.2.2 · p. 5
2.2.2 Algorithm evolution of agents.
- Pretrained LLMs enabled code generation, translation, and programming assistance beyond completion, opening the way to agents that plan, use tools, and refine outputs.Examples include documentation generation, test synthesis, and bug detection and fixing.Source: §2.2.2 · p. 5
- Iterative real-world development, multi-step prompting methods, and the availability of APIs, command-line tools, and language-server protocols motivated semi-autonomous coding agents.The cited prompting methods include ReAct, chain-of-thought, and scratchpads.Source: §2.2.2 · p. 6
2.3 Agency in AI Systems
In AI coding, agency concerns autonomous action guided by objectives, feedback, and learned knowledge. LLM-based agents exhibit classical agency behaviors while operating probabilistically and in a language-driven manner.
- Agency in coding agents is the capacity to select actions autonomously based on internal objectives, external feedback, and learned knowledge.Source: §2.3 · p. 6
- Coding agents can interpret open-ended tasks, plan development steps, coordinate tools, and adapt to compiler or test feedback across iterations.These behaviors align with classical reactivity, proactivity, social ability, and autonomy, despite probabilistic, language-driven operation rather than symbolic world models and search planning.Source: §2.3 · p. 6
2.4 Key Enablers of Agentic Programming
Agentic programming systems combine language models with prompting, tool interfaces, and context-management mechanisms to support goal-driven, iterative software work. Together, these enablers help agents plan, act on feedback, and maintain coherence across tasks.
2.4.1 Large language models.
- LLMs provide core reasoning for coding tasks, while model capabilities and specialization are important design choices.The survey describes models supporting code generation, planning, debugging, and multi-turn interaction; some are optimized for coding through instruction tuning, extended context, tool use, or retrieval.Source: §2.4.1 · pp. 6–7
2.4.2 Prompt engineering and reasoning strategies.
- Structured prompting scaffolds multi-step reasoning and tool use through explicit traces, action interleaving, working memory, or specialized roles.The survey describes chain-of-thought, ReAct, scratchpad, and modular prompting as ways to decompose tasks, preserve intermediate state, and revise behavior.Source: §2.4.2 · p. 7
2.4.3 Tool use and API integration.
- Tool interfaces range from natural-language instructions to structured schemas and domain-specific protocols, with machine-readable schemas helping agents produce well-formed commands and parse responses.Agent systems use compilers, debuggers, tests, linters, and version control; interfaces include command-line tools, LSP, and REST APIs.Source: §2.4.3 · pp. 8–9
2.4.4 State and context management.
- External memory stores plans, tool outputs, and progress to compensate for fixed context windows and maintain coherence across multiple steps.Approaches include vector stores, scratchpads, structured logs, retrieval, and summarization; persistent memory supports recall but adds storage and retrieval overhead.Source: §2.4.4 · p. 9
2.4 Key Enablers of Agentic Programming
- Feedback loops let agents revise outputs using failed tests, compiler errors, or linter warnings, with some systems also using planning, retries, or learning updates.The survey characterizes this closed-loop design as supporting robustness and adaptability in complex programming tasks.Source: §2.4.5 · p. 9
2.5 Comparison to Related Paradigms
AI agentic programming differs from related automation paradigms through multi-step, adaptive work in software environments. Its contrasts with synthesis, completion, DevOps, AutoML, and multi-agent systems clarify its scope and distinctive demands.
2.5.1 Program synthesis.
- Unlike one-shot program synthesis focused on targeted code generation, agentic programming handles multi-step workflows with planning, tool use, and iterative refinement.The passage contrasts classical synthesis, often limited to single-function generation from formal specifications, with continuous interaction in development environments.Source: §2.5.1 · pp. 9–10
2.5.2 Code completion tools.
- Code-completion tools react to developer input with context-aware suggestions, while agentic systems plan proactively and extend into testing, debugging, deployment, and maintenance.The passage notes that completion tools can suggest functions, classes, or small modules, but typically do not handle those broader activities.Source: §2.5.2 · p. 10
2.5.3 DevOps automation.
- DevOps automation runs predefined delivery and infrastructure workflows, whereas agentic programming emphasizes adaptive problem-solving and creative solution generation.The passage also says agentic programming can potentially orchestrate and improve DevOps processes.Source: §2.5.3 · p. 10
2.5.4 Automated machine learning and automated development.
- AutoML optimizes statistical models within structured, well-defined workflows, whereas agentic programming addresses broader software development challenges with creative problem-solving and multimodal reasoning.AutoML automates processes such as model selection, hyperparameter tuning, and feature engineering.Source: §2.5.4 · p. 10
2.5.5 Multi-Agent systems and human-AI collaboration.
- Modern agentic programming extends traditional multi-agent systems with human-in-the-loop collaboration, adaptive roles, tool use, environmental interaction, and persistent memory.The passage contrasts these capabilities with traditional systems' fixed roles and rigid protocols, and notes that contemporary systems can maintain context and learn from past interactions.Source: §2.5.5 · pp. 10–11
- Coding agents share feedback-driven perception and action with robotics and reinforcement-learning agents, but operate in structured, symbolic software environments with long-range dependencies and abstract goals.Their actions are language-based, and success requires coherence across multiple steps as well as reasoning about software goals and dependencies.Source: §2.5.5 · p. 11
3 Survey Methodology
The survey uses a systematic literature review to gather and screen research on AI agentic programming, supplemented by citation chaining and coverage of industry tools. Its final corpus supports a behavioral and architectural taxonomy.
3 Survey Methodology
- The survey follows a systematic literature review methodology to provide comprehensive coverage of AI agentic programming research.Figure 5 is identified as illustrating the methodology.Source: §3 · p. 11
3.1 Search Strategy
- Searches covered major academic databases, selected conference proceedings, and relevant open-source and industry releases because the fast-evolving field is industry-dominated.Sources included Google Scholar, ACM Digital Library, IEEE Xplore, SpringerLink, and arXiv.org.Source: §3.1 · p. 12
3.2 Study Selection
- Two researchers screened titles and abstracts, reviewed full texts with discussion to resolve disagreements, and used backward and forward citation chaining.Included studies focused on autonomous or semi-autonomous software-development systems, demonstrated agentic behavior, and offered novel contributions with evaluation, case studies, or substantial implementation detail.Source: §3.2 · p. 12
3.3 Results
- 152 academic papers were included after full-text evaluation and citation chaining, alongside a review of state-of-the-art coding agents and language models.The search retrieved 7,700 papers, 395 proceeded to full-text review, and 141 met criteria before citation chaining expanded the final corpus.Source: §3.3 · p. 12
- The 152-paper corpus spans 2022–2025, with 53% published in 2024, and the survey develops a taxonomy of agentic systems by behavior and architecture.The reported year distribution is 5% in 2022, 22% in 2023, 53% in 2024, and 20% in 2025.Source: §3.3 · pp. 12–13, Figure 7 · p. 13
4 Taxonomy of AI Agentic Programming
The survey organizes agentic programming systems by behavioral and architectural dimensions, then compares system categories and examines how reasoning strategies trade cost against performance. Its taxonomy distinguishes interactive assistants from more autonomous approaches, while the cost discussion emphasizes choosing strategies within a project’s budget and time constraints.
4 Taxonomy of AI Agentic Programming
- The taxonomy classifies systems by behavioral traits and architecture, providing a structure for comparing the diverse agentic programming landscape.It introduces behavioral dimensions and organizes systems by core functionality and architectural patterns.Source: §4 · p. 13, §4.1 · p. 13, §4.2 · p. 14
4.2.1 Interactive code assistants.
- Interactive code assistants provide editor-integrated completions and suggestions but typically lack autonomy for complex, multi-step development tasks.Some newer systems are adding session-level memory, persistent context, and structured task execution.Source: §4.2.1 · p. 14
4.2 Agent System Categories
- Autonomous task-oriented agents can manage multi-step programming workflows with minimal human intervention, using tools and revising plans as requirements or results change.Examples include agents that generate and debug code over extended tasks or sandbox repositories and verify changes.Source: §4.2.2 · p. 15
4.2.2 Autonomous task-oriented agents.
- Planning-centric agents decompose goals into steps and monitor execution, while collaborative systems assign specialized roles to agents coordinating through dialogue or shared memory.These approaches are typically multi-turn and memory-enabled, trading speed for robustness.Source: §4.2.3 · p. 15, §4.2.4 · p. 15
4.3 Summary and Comparative Table
- Table 5 compares representative systems across the behavioral dimensions and categories introduced in the taxonomy.The comparison is intended to help readers understand existing approaches’ capabilities and limitations and guide future system design.Source: §4.3 · pp. 15–16, Table 5 · p. 16
4.4 Cost and Token Consumption Model
- LLM usage costs include input and output tokens, with additional expense from extended reasoning and tool-augmented workflows.Providers’ rates vary across model families and service tiers.Source: §4.4 · p. 16, §4.4.1 · p. 16
4.4.2 Impact of reasoning strategies.
- Short reasoning minimizes token use and latency, while standard CoT uses roughly twice as many tokens and tool-augmented iteration increases accuracy and robustness at substantially greater token and time cost.The survey notes that a hybrid approach can balance cost and performance when project budgets or timelines are constrained.Source: §4.4.2 · p. 17
5 Challenges
The survey identifies challenges to deploying robust, scalable, and trustworthy agentic systems, spanning evaluation, communication, domain adaptation, safety, toolchains, and memory. It argues that current benchmarks and human-oriented development tools do not fully capture or support the demands of realistic autonomous workflows.
5.1 Evaluation and Benchmarking
- Existing coding benchmarks often miss the complexity of real software workflows, including broader language coverage, multi-turn interaction, tool integration, and build-pipeline management.SWE-Bench evaluates project-level repositories with tests and CI, but remains Python-restricted and offers limited interaction beyond binary pass/fail outcomes.Source: §5.1 · pp. 17–18
5.2 Communication Protocols for Multi-Agent Systems
- Multi-agent systems still lack a common communication protocol, while adapters can add latency, bandwidth overhead, and scalability limits.Dynamic multi-agent sessions also remain difficult to manage.Source: §5.2 · p. 18
5.3 Domain-specific Models for Agents
- Generic coding agents struggle in specialized domains that require strict operational constraints, specialized toolchains, APIs, and domain knowledge underrepresented in general-purpose training data.Domain-adapted models and task-specific learning strategies are proposed as directions for these settings.Source: §5.3 · p. 18
5.4 Safety and Privacy
- Greater agent autonomy raises safety and privacy risks, including subtle bugs, unsafe patterns, security violations, and harmful actions prompted by compromised inputs or tools.Proposed research priorities include secure collaboration protocols, tool-output validation, anomaly detection, and controls for sensitive data access.Source: §5.4 · pp. 18–19
5.5 Toolchain Integration and Programming Language Design
- Human-oriented languages and tools provide coarse, unstructured feedback that can hinder agents’ diagnosis of failures and understanding of code transformations.The survey calls for richer intermediate representations, transformation traces, structured feedback, and agent-aware language interfaces.Source: §5.5 · p. 19
5.6 Scalable Memory
- Fixed context windows and limited persistent memory make it difficult for agents to maintain coherent reasoning across long tasks and multiple sessions.Retrieval-augmented generation and memory summarization offer partial relief but remain inadequate for complex multi-session workflows.Source: §5.6 · p. 19
6 Opportunities and Future Directions
The survey outlines future directions for making coding agents more capable across tools, long-running context, and evaluation. It emphasizes agent-aware infrastructure and benchmarks that better reflect realistic, interactive software development.
6 Opportunities and Future Directions
- Future work spans agent-aware tools, scalable memory, stronger benchmarks, human collaboration, specialization, safety, multi-agent coordination, and systems support.The overview frames these as opportunities for a fast-evolving research frontier.Source: §6 · p. 19, Figure 8 · p. 20
6.1 Integrating Coding Agents with Tools
- Coding agents need modular architectures and more structured access to compiler feedback, intermediate representations, language interfaces, and runtime profiling.Compiler feedback could identify why optimizations fail and track which edits introduce errors or regressions; standardized introspective interfaces could expose program structure.Source: §6.1 · pp. 19–21
6.2 Scalable Memory and Context Management
- Long-running software tasks require persistent, structured memory for task context, code changes, tool feedback, and version-control history.Proposed directions include code-structure-guided attention, hierarchical memory, summarization, and retrieval conditioned on task state; current context windows and lack of persistent memory constrain agents.Source: §6.2 · pp. 21–22
6.3 Evaluation and Benchmarking
- Current benchmarks omit important real-world conditions, including diverse languages, large modular codebases, long-range dependencies, and interactive tool use.The surveyed results show task-dependent performance: models perform poorly on software optimization and well on code generation, with gaps across benchmarks; future evaluations should include realistic workflows, tool and human feedback, and metrics beyond functional correctness.Source: §6.3 · pp. 22–23
6.4 Human-AI Collaboration
The survey presents human-AI collaboration, domain specialization, safety, multi-agent communication, and systems infrastructure as linked directions for developing coding agents. It emphasizes collaboration with human oversight and mechanisms for transparent, dependable operation.
6.4 Human-AI Collaboration
- Near-term agent collaboration can augment rather than replace developers, with humans retaining strategic oversight in interactive workflows.Mixed-initiative designs let developers set goals and constraints, inspect or revise agent outputs, and interrupt and resume work while preserving shared state.Source: §6.4 · pp. 23–24
6.5 Domain Specialization and Adaptability
- Domain specialization could adapt prompting, training, planning, and tool use to programming areas with distinct constraints and tooling.Examples include embedded systems, where agents must reason about memory layout, timing, and hardware-specific APIs, and safety-critical domains that require stricter validation.Source: §6.5 · p. 24
6.6 Safety, Alignment, and Trust
- Safety and trust research should ground actions in user intent and constraints while addressing insecure outputs, sensitive-data leakage, and explainability.Suggested mechanisms include structured intent, verification tools, behavioral audits, human feedback, confidence signaling, and reversible edits with audit trails.Source: §6.6 · p. 25
6.7 Multi-agent Collaboration
- Multi-agent collaboration needs standardized, semantics-aware protocols for low-latency coordination, dynamic participation, conflict resolution, and consistency.The survey cites IEEE P3394 as proposing session, channel, and message abstractions across diverse transports.Source: §6.7 · pp. 25–26
6.8 System Support for AI Coding Agents
- Scaling coding agents requires systems that adaptively schedule resources, reduce tool-data overhead, persist workflow state, and coordinate concurrent work.The infrastructure must support interruption and resumption by persisting goals, annotations, and intermediate representations, alongside source code.Source: §6.8 · p. 26
7 Conclusion
The survey reviews AI agentic programming as a shift toward software workflows in which coding agents plan, use tools, and refine work from feedback. It presents a taxonomy and synthesis of techniques, benchmarks, and open challenges, and identifies opportunities for future systems.
7 Conclusion
- The survey describes coding agents that generate and test code, interact with development tools, decompose tasks, and adapt based on feedback.It presents these capabilities as bringing software development closer to autonomous workflows.Source: §7 · p. 26
- The survey introduces a taxonomy of agents and architectures, reviews techniques and benchmarks, and identifies challenges to safe, reliable real-world use.The identified concerns include context handling, persistent structured memory, user-intent alignment, human-AI collaboration, and verification.Source: §7 · p. 26
- The survey sees potential for agents to augment developer productivity, reduce maintenance costs, and expand access to programming.It frames collaborative workflows and autonomous long-running tasks as directions for this potential.Source: §7 · p. 26
Related papers
This paper cites
- Attention Is All You Need2017
- Training language models to follow instructions with human feedback2022
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks2020
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?2023
Cited by
- From Cosmetic to Structural: A Paired Experiment on Metric-Guided Agentic Refactoring2026
- MemoryWalker: Stop Training Agents on Contexts They Never Saw2026
- Graphectory Viewer: A Tool for Process-Centric Analysis of Agentic Software Trajectories2026
- FeatureBench: Benchmarking Agentic Coding for Complex Feature Development2026
- AIDev: Studying AI Coding Agents on GitHub2026