PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle

Happy Bhati

arXiv:2609.04681v1cs.SEcs.AI

TL;DR

As agentic coding makes plausible patches cheaper, organizations face the question of how to operate development at scale when proving changes fit to ship matters more. This structured synthesis finds that coding gains attenuate before release, shifting the measure of progress from code generated to production-qualified value delivered.

  • Problem

    The paper asks how organizations can operate agentic development at scale as producing plausible patches gets cheaper while proving changes deserve shipment grows more important.

    Source: §1 · p. 1

  • Method

    The paper presents a structured systems synthesis and a reliability-gate ladder, framing repository-benchmark performance as only part of production assurance.

    Source: §7 · p. 6, §19 · p. 16

  • Results

    The synthesis concludes that gains attenuate before releases, while agent traces show human correction and discarded code and long-horizon agents struggle with self-verification.

    Source: §20 · p. 16

  • Takeaways & Limitations

    The paper argues that progress should be measured as production-qualified value delivered rather than code generated.

    Source: §20 · p. 16

    This is not a PRISMA-style exhaustive systematic review, and some 2026 evidence consists of preprints or organization reports rather than mature peer-reviewed research.

    Source: §19 · p. 16

Abstract

from arXiv · show

AI coding systems are moving from autocomplete and chat toward agents that can inspect repositories, edit multiple files, run tools, write tests, open pull requests, and work for long periods with limited supervision. This capability changes the bottleneck in software delivery. Recent field studies show meaningful gains in coding activity, but newer evidence also shows that those gains attenuate sharply between writing code and shipping reliable software. Review, integration, testing, security, deployment, and production operations remain constraining stages, while the economics are shifting from predictable per-seat licensing toward variable token, tool, sandbox, CI, and rework costs. This paper synthesizes peer-reviewed software-engineering research, university studies, benchmark audits, production reports from major technology companies, developer telemetry, and cost-management evidence released primarily from 2024 through September 2026. No new model experiment is claimed; numerical findings remain attributed to their original studies. The synthesis proposes four engineering concepts: the Agentic SDLC Throughput Paradox, Production-Qualified Change (PQC), the Verification Tax, and an Agentic SDLC Control Plane that allocates autonomy subject to cost, reliability, and human-attention budgets. An evidence-based horizon then maps today's supervised agents to future policy-bounded software factories. The central research question shifts from how much code an agent can generate to how much production-qualified value an engineering system can deliver per dollar, per reviewer-hour, and per unit of operational risk.

1 Introduction · 2 Scope, Evidence Selection, and Claim Boundary

Agentic coding can raise local output while verification, coordination, human judgment, and operational risk constrain what reaches production. This synthesis examines that system-level challenge through evidence from 2024 to September 3, 2026, while keeping prior findings distinct from its proposed concepts.

1 Introduction

  • 26.08% pooled task gain and agent coding-activity gains of 180% at commit, 50% at project, and 30% at release levels show output gains vary by delivery stage.Source: §1 · p. 1
  • Google reports about 60 minutes of active author shepherding per change, while DORA associates higher AI adoption with higher throughput but lower delivery stability.In SWE-chat, only 44% of agent-produced code survived into user commits, and users pushed back in 44% of turns; that dataset also found more security vulnerabilities in agent-written code [8]. SWE-Marathon averaged 27.2 million tokens, with no tested configuration above 30% pass@1 and reward hacking in 13.8% of rollouts [9].Source: §1 · pp. 1–2
  • The Agentic SDLC Throughput Paradox names the gap between faster change generation and downstream validation and release, where system constraints become part of runtime.Agents can now search repositories, modify code, run commands, and iterate on failures; the paper treats verification, coordination, human judgment, and operational risk as constraints on shipped value.Source: §1 · pp. 1–2
  • Production-Qualified Change requires relevant review, test, security, deployment, and operational gates; the proposed Agentic SDLC Control Plane allocates execution and verification by risk, reliability, budget, and capacity.The paper presents the control plane as a systems model to test, not as a complete system already implemented by a company.Source: §1 · p. 2
  • The Verification Tax extends costs beyond model bills to context construction, retries, tools, CI, security analysis, reviewer attention, rework, incidents, and infrastructure.Northeastern researchers report token use varies across languages partly because agents make non-compiling attempts, revise passing solutions, and distrust tests [11]. FinOps reports 98% of surveyed practitioners manage AI spend, up from 31% two years earlier; Gartner forecasts coding costs could exceed average developer salary by 2028, a forecast rather than a measured inevitability.Source: §1 · p. 2

2 Scope, Evidence Selection, and Claim Boundary

  • The review emphasizes work from 2024 through September 3, 2026, selecting sources against six questions and mixing evidence types because benchmarks alone do not describe production engineering.Table 1 makes evidence tiers visible, and its caption specifies that tiers describe evidence type, not a universal ranking of research quality.Source: §2 · p. 2, Table 1 · p. 3
  • This paper narrows prior A-SDLC work [1] from architecture to reliability, verification capacity, delivery economics, and operating models, with prior observability work [2] used for continuity.Source: §2 · p. 3
  • The paper reports no original benchmark, productivity, review, security, market, or company-deployment measurements; those results remain attributed to their original sources.Source: §2 · p. 3

3 From Coding Capability to Production Capability · 4 The Agentic SDLC Throughput Paradox

Repository-scale agent benchmarks made coding capability measurable, but benchmark success and faster code generation do not establish production readiness. The supplied evidence shows that delivery depends on broader engineering work and that coding acceleration can attenuate before changes reach users.

3.1 Benchmarks established the capability leap

  • SWE-bench [14], SWE-agent [15], and SWE-Lancer [16] extended evaluation to repository issues, agent-computer interaction, and more than 1,400 freelance tasks representing roughly US$1 million in historical payouts.These benchmarks made agent progress legible and pushed models toward repository-scale work.Source: §3.1 · p. 3
  • OpenAI’s 2026 SWE-bench Verified audit found material test-design or problem-description issues in 59.4% of 138 tasks a frontier model did not consistently solve, and identified contamination risk.A separate position paper argues that benchmark scores conflate the model, harness, context, execution environment, and feedback signals [18]; passing hidden tests is not a release-readiness certificate because production also entails security review, deployment, monitoring, user behavior, and maintenance.Source: §3.1 · p. 3

3.2 Software engineering is broader than patch generation

  • Software engineering extends beyond patch generation: Gu et al. [19] identify testing, debugging, review, security, and other activities, while a 2025 study of 19 developers and 33 repository issues found roughly half resolved and incremental collaboration better than one-shot use.Developers still struggled with trust, debugging, and testing; the organization-level question is whether the human-agent system moves changes from intent to production without adding more validation work than it removes.Source: §3.2 · p. 4

4 The Agentic SDLC Throughput Paradox

  • Demirer, Musolff, and Yang estimate cumulative commit effects of 40%, 140%, and 180% for autocomplete, interactive, and autonomous agents; autonomous-agent effects are 50% for projects and 30% for releases.Figure 1 visualizes the attenuation that this paper calls the Agentic SDLC Throughput Paradox.Source: Figure 1 · p. 4, §4 · p. 4
  • DORA’s 2025 study associates AI adoption with higher delivery throughput but lower delivery stability; its observational, organization-level findings do not establish causation.DORA frames AI as a mirror and multiplier of the surrounding delivery system, underscoring the need for testing, version control, feedback loops, and loosely coupled architecture.Source: §4 · p. 4
  • In METR’s early-2025 randomized trial, 16 experienced developers completing 246 tasks in familiar mature repositories were 19% slower with AI, a result not generalizable to all developers or later tools [21].The study highlights a gap between expected acceleration and measured end-to-end task time under high domain knowledge and strict quality standards.Source: §4 · p. 4
  • Generation throughput measures candidate-change production, while delivery throughput measures trustworthy changes reaching users; review, integration, testing, release, and downstream demand remain complementary inputs.The paper’s synthesis is that agentic engineering raises generation throughput faster than it automatically raises delivery throughput.Source: §4 · pp. 4–5

5 Code Review Becomes a Capacity Problem · 6 Testing: More Generated Tests Are Not Automatically More Assurance

These sections argue that agentic software development is constrained by review capacity and assurance, not simply code or test generation: review should be risk-adaptive, while tests and verifiers need independent, objective checks. Evidence from industrial systems, agent-use studies, and benchmark audits shows both the value and limits of automation.

5 Code Review Becomes a Capacity Problem

  • At Google, authors receive millions of reviewer comments yearly, and an average change requires about 60 minutes of active shepherding; ML edits and AutoCommenter automate parts of review.AutoCommenter was deployed to tens of thousands of developers.Source: §5 · p. 5
  • Because model-generated changes approved by another model raise a recursive assurance problem, requiring humans to read every line would cap agentic throughput at human review capacity.The proposed scalable direction is to separate kinds of evidence.Source: §5 · p. 5
  • Review should be risk-adaptive: high-risk changes may need independent static analysis, generated and human-authored tests, security checks, ownership review, and a canary, while documentation changes may need less.Code review also serves defect detection, design alignment, security, knowledge transfer, ownership, maintainability, and accountability; larger or more numerous agent proposals can create a review queue.Source: §5 · p. 5
  • Only 44% of agent-produced code survived into commits, while users pushed back in 44% of turns; sessions split between agents authoring virtually all committed code (41%) and humans authoring all of it (23%) [8].These findings suggest interaction and filtering remain part of the work, so generated lines or agent-authored diffs overstate realized engineering output.Source: §5 · p. 5

6 Testing: More Generated Tests Are Not Automatically More Assurance

  • In Meta’s Instagram evaluation, 75% of generated test cases built correctly, 57% passed reliably, and 25% increased coverage; engineers accepted 73% of recommendations for production deployment.Across Meta test-a-thons, the system improved 11.5% of classes to which it was applied; objective verification checks rejected unhelpful output.Source: §6 · p. 5
  • In a 2026 SWE-bench-trajectory study, resolved and unresolved tasks had similar test-writing frequency, and prompting agents to write more or fewer tests did not significantly change outcomes [24].Many generated tests behaved more like observational probes than assertion-rich regression tests; Microsoft Research explored reinforcement learning from automatic feedback partly because generated tests can contain test smells and weak assertions.Source: §6 · p. 5
  • Test volume is a poor assurance metric: useful tests add independent discriminatory power against plausible faults, so the verifier itself must be treated as a first-class artifact.Generated tests can be filtered through objective checks before recommendation, as in Meta’s TestGen-LLM.Source: §6 · p. 5
  • Benchmark audits warn that narrow, wide, or contaminated hidden tests can misjudge functionally sound patches, while SWE-Marathon observed 13.8% reward hacking as agents found verifier shortcuts [9].The shortcut can fail the intended engineering objective even when it satisfies the verifier.Source: §6 · p. 5

7 Reliability Is a Chain of Gates, Not a Model Score · 8 Production-Qualified Change: A Better Unit of Output · 9 The Economics of Agentic Software Delivery

These sections frame reliable agentic delivery as a sequence of evidence gates, count output only after changes meet risk-appropriate qualification requirements, and account for costs beyond model usage. They also identify coordination failures and token-consuming behavior as reliability and economic concerns.

7 Reliability Is a Chain of Gates, Not a Model Score

  • Figure 2 frames reliability as accumulated independent evidence: repository benchmarks cover only part of production assurance, system reliability depends on harness and execution state [25], and scores can shift with harness changes [18].The gate ladder is a synthesis framework, not an industry standard.Source: Figure 2 · p. 6, §7 · p. 6
  • GitHub pairs agent autonomy with mandatory CodeQL, dependency-advisory, and secret-scanning checks before pull-request finalization; it reports analogous Copilot validation prevented hundreds of potential leaks and vulnerabilities.The reported result is vendor-reported and concerns Copilot cloud-agent validation since its 2025 release.Source: §7 · p. 6

7.1 Coordination is a separate reliability dimension

  • Collaborative coding agents were on average about 30% less successful than when performing both tasks alone, with communication, commitment, and partner-model failures motivating explicit contracts and coordination [10].Proposed contracts include ownership boundaries, structured messages, shared state, conflict resolution, and a coordinator to judge authoritative evidence.Source: §7.1 · p. 6

8 Production-Qualified Change: A Better Unit of Output

  • Lines of code, commits, pull requests, and task counts are fragile output measures; optional severity, customer-value, or risk weights should not collapse into an opaque developer score.PQC is presented as a bridge between agent telemetry and delivery outcomes, not a replacement for DORA metrics or engineering judgment.Source: §8 · p. 7
  • PQC counts a change only after relevant qualification gates pass—not at generation; low-risk edits can require fewer gates than migrations, authentication, or payment changes.The relevant qualification vector is defined by the required gates for that change class.Source: §8 · p. 7

9.1 Token price is only the visible part of cost

  • Total Cost to Deliver includes model, context, tools, sandbox, CI, review, security, rework, and incident costs; it is a taxonomy, not a promise of per-pull-request allocation.The model is intended to prevent optimizing visible costs while downstream costs grow unnoticed.Source: §9.1 · p. 7

9.2 Token consumption is behavior, not just model pricing

  • 27.2 million tokens: SWE-Marathon initial rollouts averaged this amount; language-study trajectories also found retries, unnecessary revisions, test distrust, and fallback prototyping [9] [11].The language study links wasted inference to software-engineering quality, including harness quality and language competence.Source: §9.2 · p. 8
  • 98% of surveyed FinOps practitioners now manage AI spend; GitHub tracks AI-credit costs against developer, payroll, and pull-request metrics; Gartner forecasts coding costs could exceed average developer salary by 2028.The paper notes the forecast may prove wrong, while emphasizing that seat counts alone do not govern variable agent costs.Source: §9.2 · p. 8

10 The Verification Tax

The Verification Tax is a measurable quantity for agent-generated changes whose ratio varies across repositories and task types. It should be diagnosed and optimized against reliability, not minimized in isolation.

10 The Verification Tax

  • The Verification Tax varies by repository and task type; Cgeneration includes model, context, and agent execution costs.The passage introduces the Verification Tax for sets of agent-generated candidate changes, but does not provide its defining equation.Source: §10 · p. 8
  • A high Verification Tax may reflect risky work, weak candidates, work beyond the agent’s reliability envelope, inefficient assurance infrastructure, or redundant policy requirements.These causes call for diagnosis rather than treating the metric as a target to game.Source: §10 · p. 8
  • The goal is to reduce verification cost at a fixed reliability target or improve reliability within a fixed assurance budget, not simply minimize verification.A low Verification Tax can be dangerous if it results from skipped tests or rubber-stamped reviews.Source: §10 · p. 8
  • AI can increase coding output in randomized enterprise settings, slow experienced maintainers in a mature-repository setting [21], and increase commits far more than releases in broad telemetry.The passage attributes these differences to studies observing different ratios between generation gains and verification burden.Source: §10 · p. 8

11 The Agentic Autonomy Budget

The section reframes agent autonomy as a budgeted privilege rather than a capability measured by run length. Execution plans must balance compute cost, reliability risk, and human attention, subject to mandatory security, policy, and SLO constraints.

11 The Agentic Autonomy Budget

  • Autonomy is a budgeted privilege, and the largest model or longest run is not automatically the best operating point.The feasible autonomy region is jointly constrained by money, reliability, and human review capacity.Source: §11 · p. 8, Figure 4 · p. 9
  • Each task consumes money and compute, reliability and risk, and human-attention budgets.These include tokens, tools, sandbox and CI costs; failure impact, security exposure, and blast radius; and clarification, review, debugging, approval, and incident response.Source: §11 · p. 8
  • The feasible autonomy region is the intersection of those budgets, with execution plans subject to mandatory security, policy, and SLO constraints.The control policy weighs expected production-qualified value against monetary/compute cost, expected failure loss, and expected human attention; those weights are organizational policy, not universal constants.Source: §11 · p. 9, Figure 4 · p. 9
  • Budgeting autonomy helps avoid under-delegating safe, repetitive, well-verified work and over-delegating poorly verifiable or high-blast-radius tasks.Over-delegation can consume millions of tokens and create large review queues.Source: §11 · p. 9

12 An Agentic SDLC Control Plane

The section proposes an Agentic SDLC Control Plane to observe work, assign execution policies, and record qualification evidence. It can operate as a policy and telemetry layer around existing development systems rather than replacing them.

12 An Agentic SDLC Control Plane

  • The proposed Agentic SDLC Control Plane centralizes execution policy and qualification evidence because developer-by-developer choices fragment reliability and cost policy across IDEs.Specialized agents are optional; the core proposal is centralized policy, evidence gates, and feedback connecting cost and reliability to future autonomy decisions.Source: §12 · p. 9, Figure 5 · p. 10
  • The layer can wrap existing developer infrastructure, classify task risk, select an execution plan, and bind each risk class to required verification and deployment gates.Risk classification considers consequence, reversibility, data sensitivity, ownership, and affected components; execution planning selects the model, context strategy, harness, permissions, parallelism, and budget.Source: §12 · p. 10
  • It should stop or escalate cycling work when progress no longer warrants cost, while attributing model, tool, CI, reviewer, and rework costs across task-to-repository levels.Cost attribution is recorded at task, PR, team, and repository levels.Source: §12 · p. 10
  • The control plane turns AI observability from dashboard reporting into policy enforcement, extending prior work on joining cost and code-quality signals [2].It updates policies using PQC rate, escaped defects, rollbacks, review corrections, token use, and human escalation.Source: §12 · p. 10

13 What Companies Are Actually Struggling With · 14 Metrics for an Agentic Engineering Organization · 15 Future Horizon: From Supervised Agents to Policy-Bounded Software Factories

Companies’ bottlenecks include verification capacity and agent-harness quality, while the proposed metrics emphasize system-level cost, qualification, and reliability. The future horizon moves from human-accountable agents toward policy-bounded software factories, but reliable composition remains an open systems challenge.

13 What Companies Are Actually Struggling With; 13.1 The hidden organizational issue: verification capacity

  • Companies may buy AI licenses without funding verification capacity; Table 3 maps recurring problems to controls, and assurance redesign should automate low-risk evidence while protecting human attention for high-risk changes.When generation is cheap, work can simply queue at inspection, integration, or shipping; unlimited reviewer hiring is not the proposed response.Source: §13 · p. 10, §13.1 · p. 11, Table 3 · p. 11

13.2 The hidden technical issue: harness quality

  • Agent success and cost depend on harness components as well as the model, so organizations need versioned harnesses and reproducible runs, not only records of which foundation model was called.Tool permissions, repository retrieval, context compression, test selection, sandbox state, and retry logic can materially change outcomes; SWE-agent’s interface results [15] and newer reliability work [25] make this dependency explicit.Source: §13.2 · p. 11

14 Metrics for an Agentic Engineering Organization

  • The proposed metrics are system-level research and operating measures, not individual developer scores, and should connect cost with qualification and reliability because rework is not equivalent to safe shipping.GitHub’s 2026 ROI dashboard connects AI credits with cost per developer and pull-request output; the proposed next step is linking cost to qualification and reliability.Source: §14 · pp. 11–12, Table 5 · p. 12

15 Future Horizon: From Supervised Agents to Policy-Bounded Software Factories; 15.1 H0: supervised agents with hard CI boundaries

  • The capability horizon uses maturity stages rather than dates: H0 describes documented practice, while H1–H3 are less mature horizons, not guaranteed timelines.At H0, agents implement features and fixes, generate or improve tests, respond to review feedback, and open pull requests; production policy remains externally enforced and human-accountable.Source: §15 · p. 12, Figure 6 · p. 13, §15.1 · p. 13

15.2 H1: control-plane engineering

  • H1’s operational shift is standardizing harnesses, permission tiers, token budgets, independent verifiers, model routing, and evidence requirements, with AI cost allocated to repositories and business services.FinOps data and GitHub ROI tooling already point toward this control-plane direction.Source: §15.2 · p. 13

15.3 H2: multi-agent and long-horizon execution

  • H2 research exposes unresolved reliability limits: CooperBench shows a coordination penalty [10], while SWE-Marathon reports low ultra-long-task success, high token use, weak self-verification, and reward hacking [9].These failures identify needs for durable state, structured coordination, adversarial verification, bounded permissions, progress accounting, and recovery from partial failure. METR reports increasing capability on longer tasks, while longer runs can enlarge both the change surface and cost tail.Source: §15.3 · p. 13

15.4 H3: autonomous software factories remain a systems research problem

  • H3 remains a systems research problem: a software factory must safely decide, implement, verify, deploy, observe, roll back, and learn within policy, requiring reliable composition of separately evaluated capabilities.Harvard’s work and teaching discussions motivate correctness-by-construction methods as automatic generation scales; Boston University’s curriculum emphasizes correctness, security, and maintainability in AI-generated code. These are educational and research-direction signals, not performance evidence.Source: §15.4 · p. 13

16 Research Agenda · 17 Implications for Engineering Leaders

The research agenda proposes tests of what predicts production qualification, how to allocate verification and cost, and how agents and organizations can sustain reliability. Engineering leaders are urged to treat agentic development as a constrained production technology, measuring shipped reliability and costs while bounding autonomy, review demands, and risk.

16.1 RQ1: What predicts Production-Qualified Change better than benchmark score?; 16.2 RQ2: Can verification budgets be allocated adaptively without increasing escaped defects?

  • Studies should test whether PR-level signals predict Production-Qualified Change better than benchmark scores and whether risk-adaptive gates preserve escaped-defect risk while reducing delivery costs.Instrument real agentic PRs across repositories and model versions; measure cost, latency, reviewer time, and qualification rate under adaptive verification.Source: §16.1 · p. 14, §16.2 · p. 14

16.3 RQ3: When does an additional agent increase reliability rather than correlated error?; 16.6 RQ6: Does AI-generated test code improve real defect detection?; 17 Implications for Engineering Leaders

  • Research should test whether additional agents reduce errors despite correlated failures, and whether AI-generated tests detect real defects beyond test quantity [24].Measure generator-verifier error correlation across model families, prompts, contexts, and tools; evaluate mutation score, fault localization, regression detection, specification coverage, and post-release defects. Leaders should prefer independent oracles, properties, mutation, integration checks, and production signals over tests that repeat generator assumptions.Source: §16.3 · p. 14, §16.6 · p. 14, §17 · p. 15

16.4 RQ4: What is the optimal stop rule for long-horizon coding agents?; 17 Implications for Engineering Leaders

  • Long-horizon agents need stop policies based on progress and remaining work, since trajectories can consume millions of tokens with little progress [9].Candidate signals include verifier improvement, edit churn, repeated failures, context growth, and estimated remaining work; leaders should impose token and time ceilings with progress checkpoints.Source: §16.4 · p. 14, §17 · p. 15

16.5 RQ5: How should agentic cost be attributed to delivered value?; 17 Implications for Engineering Leaders

  • Agentic cost research should compare cost per PR, merged PR, PQC, escaped-defect-free change, and total delivery cost, with the best metric potentially varying by product class.Leaders should include CI, sandbox, reviewer, security, rework, and incident costs; a control plane should expose marginal value, such as expert time avoided versus retries that add review burden.Source: §16.5 · p. 14, §17 · p. 15

16.7 RQ7: What organization design converts code acceleration into release acceleration?; 17 Implications for Engineering Leaders

  • Organizational experiments should test which interventions convert code acceleration into release acceleration, given the 180%-to-30% attenuation observed by Demirer et al..Candidate interventions include review-agent triage, platform engineering, smaller changes, contract testing, modular architecture, ownership routing, and protected reviewer capacity. Leaders should reserve expert review for architecture, security, ambiguous requirements, and high-blast-radius changes, while varying autonomy by change class.Source: §16.7 · p. 14, §17 · p. 15

16.8 RQ8: How should software agents coordinate?; 17 Implications for Engineering Leaders

  • Because current agents do not automatically become effective teammates, research should compare structured coordination protocols with free-form chat [10].Test explicit commitments, shared plans, role ownership, transactional state, conflict detection, and coordinator architectures; leaders should establish ownership and protocols before multiplying agents.Source: §16.8 · p. 14, §17 · p. 15

16.9 RQ9: How can reliability evidence survive model and harness changes?; 17 Implications for Engineering Leaders

  • Reliability evaluations must separate model, harness, environment, retrieval, and policy effects, using versioned evidence and replayable trajectories to assess change [18] [25].Leaders should version the model, harness, prompts or policy, context construction, tool permissions, and verifier so failures can be diagnosed.Source: §16.9 · p. 15, §17 · p. 15

16.10 RQ10: What skills should senior engineers retain when routine implementation is delegated?

  • As routine implementation is delegated, longitudinal research should test whether heavy delegation erodes the expertise needed to review and recover from agent failures.The human role shifts toward decomposition, architecture, risk judgment, verification, and incident reasoning; Harvard and BU curricula emphasize correctness, evaluation, testing, CI/CD, and production reliability in AI-assisted software work.Source: §16.10 · p. 15

18 Implications for Researchers and Universities

Software-engineering research is moving toward evaluating socio-technical systems, while agentic-software education must extend beyond prompt-writing to engineering foundations, operational practice, and human-agent work. As implementation becomes cheaper, engineering judgment becomes more leveraged, making deeper instruction in foundations important.

Implications for Researchers and Universities

  • Research is shifting toward socio-technical evaluation, spanning realistic SWE tasks [19], real-world traces [8], release output, token economics [11], generated-code correctness, and verification and maintainability education.The cited work ranges from broader tasks and realistic environments to adoption outcomes, language- and behavior-dependent costs, correctness-by-construction, and curriculum.Source: §18 · p. 15
  • A durable curriculum should teach architecture and change impact, testing methods, secure coding and least privilege, plus CI/CD, observability, incident response, and rollback.The listed testing topics include property-based testing, mutation, and test-oracle design; operations also include SLOs.Source: §18 · pp. 15–16
  • Students also need model and harness evaluation, benchmark design, cost attribution and resource budgets, and human-agent interaction, review, accountability, and organizational design.The curriculum extends beyond prompting to cover how systems are evaluated and resourced and how people work with and oversee agents.Source: §18 · pp. 15–16
  • When implementation becomes cheaper, engineering judgment becomes more leveraged, which supports teaching foundations more deeply, not less.Source: §18 · p. 16

19 Limitations and Responsible Claims · 20 Conclusion

The paper concludes that agentic software development must be managed for production-qualified value rather than code generation alone, while treating its synthesis and proposed measures as provisional. Reliable delivery depends on downstream verification and controls, and evidence across studies and domains requires careful interpretation.

19 Limitations and Responsible Claims

  • The paper is a structured synthesis, not an exhaustive systematic review; emerging 2026 evidence, company-reported outcomes, and market forecasts have important limits.Some 2026 sources are preprints or organization reports; company outcomes may reflect selection, product design, or incentives, and Gartner cost forecasts are planning signals, not facts about 2028.Source: §19 · p. 16
  • 26.08% productivity, 19% slowdown [21], and observational production-hierarchy results describe different populations and must not be treated as one experiment.The studies differ in settings, treatments, outcomes, and causal strength.Source: §19 · p. 16
  • PQC, Verification Tax, Agentic Autonomy Budget, and Control Plane are unvalidated hypotheses, and PQC should not become an individual productivity score.Using PQC as a score could encourage gaming and penalize engineers working on difficult, high-risk systems.Source: §19 · p. 16
  • Agent autonomy should vary by domain and depend on reversibility, blast radius, regulation, and human consequence.Internal scripts should not inherit the same policy as changes to authentication, financial transactions, medical devices, or critical infrastructure.Source: §19 · p. 16
  • The paper does not claim that stronger agents inevitably reduce engineering employment; human-AI complementarity is reported, while labor outcomes depend on demand, organization, education, and distribution.Demirer et al. reports strong complementarity between AI output and human effort in the production chain.Source: §19 · p. 16

20 Conclusion

  • The Agentic SDLC depends on downstream requirements, discriminating tests, review and security, deployment controls, and production telemetry to deliver reliable software.Generating candidate code is no longer the only scarce activity.Source: §20 · p. 16
  • Agentic SDLC gains attenuate before release, while human correction, discarded code, costly self-verification, verifier gaming, and multi-agent coordination failures persist.Long-horizon agents consume enormous inference budgets, making cost management inseparable from reliability engineering.Source: §20 · p. 16
  • The paper proposes measuring production-qualified value through PQC, Verification Tax, and Agentic Autonomy Budget, asking what trusted change reaches users, at what cost, attention, and risk.These are prompts for better questions, not final metrics; they also ask when an agent should continue, change strategy, or stop.Source: §20 · pp. 16–17

Related papers

Loading 2609.04681v1…