Source-linked AI summary
A Phased Workflow for Operating LLM-Based Coding Agents
Ante Kapetanovic, Tomislav Duricic, Andro Mercep, Emanuel Lacic
TL;DR
Practitioners need reliable ways to structure non-trivial coding-agent work because harness configuration, context length, and irrelevant context affect outcomes. The paper reports a four-phase Infobip workflow that front-loads human review and applies four context-management strategies, while identifying unresolved gaps in workflow metrics and workflow-level context patterns.
Problem
Reliable coding-agent performance on non-trivial tasks depends on practitioner configuration, while unstructured use, long inputs, multi-turn interaction, and irrelevant context can reduce effectiveness and productivity.
Method
The paper reports a four-phase workflow—research, planning, task definition, and implementation—with decreasing human involvement as artifacts mature and four context-management strategies applied across phases.
Results
The authors’ practitioner experience indicates that upstream research and planning errors can propagate into later plans and code, while code correction can introduce bloat and fragility.
Takeaways & Limitations
The workflow emphasizes front-loaded human review, persistent artifacts, disposable conversations, and deliberate context management during agent-assisted development.
Takeaways & Limitations
The paper identifies no established metrics for workflow effectiveness and a gap between formalized context components and workflow-level patterns.
Abstract
from arXiv · showhide
LLM-based coding agents combine a foundation model with a harness that shapes agent behavior. For non-trivial tasks, how practitioners structure their work with the coding agents determines whether reliable results follow. We report on a phased workflow for operating coding agents developed by the AI research team at Infobip. The workflow structures agent-assisted development into four phases where human effort is front-loaded and delegation increases as artifacts mature. Context management is the central concern, addressed through four strategies applied at each phase to counter known failure modes. From practitioner experience, we observe that upstream errors in research and planning can compound across later phases, while correcting generated code can introduce bloat and fragility. This motivates front-loading human review. We identify two open problems: the absence of metrics for workflow effectiveness and the gap between formalized context management components and the workflow-level patterns that practitioners need.
1 Introduction
Reliable performance on non-trivial coding-agent tasks depends strongly on how practitioners configure and operate the agent harness. The paper therefore motivates structured, context-aware workflows for agent-assisted development.
- Agent effectiveness on non-trivial tasks depends less on the foundation model than on practitioners’ configuration of the agent harness.The user harness includes project instructions, tool selection, session structure, and context management.
- Longer inputs, multi-turn interaction, and irrelevant context consistently degrade coding-agent performance and accuracy.Multi-turn settings show an average 39% performance drop, while irrelevant context can reduce accuracy even when the correct answer is present.
- Unstructured use of AI coding tools can reduce developer productivity, motivating structured approaches to agentic development.
- The paper reports an Infobip workflow that increases delegation as artifacts mature while front-loading human effort and review.The workflow extends the research-plan-implement paradigm and addresses error propagation across research, planning, and code correction.
2 Phased Workflow
The workflow divides agent-assisted development into four phases, with human involvement highest during research and planning and decreasing as artifacts mature. Context is managed through writing, selecting, compressing, and isolating information across phases and sessions.
- Phased Workflow: The workflow comprises research, planning, task definition, and implementation, with upstream phases performed once and downstream phases repeated as tasks evolve.When implementation exposes a poorly defined task, the practitioner returns to task definition rather than patching the implementation.
- Phased Workflow: Human involvement is highest upstream because errors in research and planning carry greater downstream cost as artifacts mature.
- Context Management: The workflow applies write, select, compress, and isolate strategies to counter distraction, confusion, poisoning, and clash in agent context.The strategies can address multiple failure modes; for example, fresh sessions reduce poisoning and distraction, while summaries reduce confusion and distraction.
- Research: Research restricts the agent to read operations, codebase search, and domain-specific tools while producing a curated document of validated findings.The practitioner guides research direction, withholds opinions to avoid anchoring, and validates findings before proceeding.
- Planning: Planning replaces the full discussion with a manually curated artifact containing validated decisions, discarding abandoned reasoning and earlier missteps.Dedicated planning modes were found to finalize prematurely before the problem was sufficiently understood.
- Task Definition: Tasks record identifiers, goals, dependencies, code paths, acceptance criteria, and validation methods, and are split when their context exceeds effective session range.
- Implementation: Implementation uses fresh sessions and persistent instructions, registries, logs, and Git artifacts so conversation state remains disposable and work can resume after interruption.The agent implements, validates, logs, updates task status, commits, and terminates under manual supervision or in an autonomous loop.
- Context Management: Project instructions evolve through concise, targeted corrections when the agent repeats mistakes across sessions.The practitioner reviews output rather than writing it while retaining ownership of the overall process.
3 Open Problems
The paper identifies unresolved gaps in evaluating workflow effectiveness and in formalizing workflow-level patterns that compose context-management components. It also notes that inconsistent terminology hinders comparison and transfer across teams and domains.
- No established metrics show whether phased workflows outperform unstructured delegation or whether task granularity and instructions are well calibrated.Candidate measures include task success rate per session, instruction correction rate, and context utilization at task completion, but these remain unexplored.
- Existing coding-agent benchmarks have limitations that make them unreliable proxies for workflow effectiveness.
- Research formalizes individual context-management components but not the workflow-level patterns that compose them across practitioner workflows.Reported benchmark gains for structured context management are measured at the component level rather than across practitioner workflows.
- The absence of shared terminology makes approaches difficult to compare or transfer across teams and domains.
Speaker Bio
Ante Kapetanovic is a Senior Researcher at Infobip working on human-AI interaction, conversational AI, information retrieval, and trustworthiness.
- Ante Kapetanovic is a Senior Researcher at Infobip.
- His work includes human-AI interaction and conversational AI.
- His work also includes information retrieval and trustworthiness.
Relevance
The proposal addresses three CIKM Industry Day topics through a phased workflow for operating coding agents, context-management challenges, and industry–academia collaboration on open problems.
- Relevance: The proposal presents a phased workflow for operating coding agents from an industry-practitioner perspective.
- Relevance: It addresses context management as a domain-specific challenge in agent-assisted development.
- Relevance: It identifies open problems concerning workflow metrics and practitioner patterns for collaboration between industry and academia.
GenAI Usage Disclosure
Generative AI tools supported the article’s conceptualization and assisted in drafting its accompanying repository, while Google Scholar Labs helped discover relevant references.
- GenAI Usage Disclosure: Generative AI tools supported the conceptualization of the article.
- GenAI Usage Disclosure: Generative AI tools assisted in drafting the accompanying repository.
- GenAI Usage Disclosure: Google Scholar Labs was used to discover relevant references.