PaperlayerView exact revision on arXiv

Source-linked AI summary

Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting

Sandeco Macedo

arXiv:2607.00038v1cs.SE

TL;DR

Coding-agent practice lacks a reviewable account of what “designing the loop” means and how it relates to prompting and harness internals. The paper defines and taxonomizes bounded loop specifications, codes fifty public examples, and finds strong adoption of autonomous verification and named terminal states but weaker automation and durable memory. It frames loop engineering as complementary to prompt engineering while documenting the limits of its descriptive evidence.

Abstract

from arXiv · show

In mid-2026 a slogan reorganized how practitioners talk about coding agents: stop prompting your agent, start designing the loop that prompts it. We take this claim seriously and give it a careful treatment. We call the object of the new practice the loop specification: a bounded, reusable artifact, made of a trigger, a goal, a verification step, a stopping rule and a memory, that a human hands to an agent harness (such as Claude Code or Codex) so the agent pursues a goal on its own, in place of step-by-step prompting. We distinguish this external loop specification from two things it is often confused with: an ordinary programming loop, and the internal perceive-act-observe cycle that the harness already provides as plumbing. We position loop engineering as a new layer in the progression from prompt to context to harness to loop, and we argue, against the stronger headlines, that it does not retire prompt engineering; loop and prompt are distinct tools with distinct uses. We offer four contributions: a definition and scope for the discipline; an anatomy and taxonomy of loop specifications organized around trigger, goal type, a five-level verification ladder, architecture, and named terminal states; a descriptive analysis of the Loop Library, a public corpus of fifty real loops that we code by hand; and a set of design principles and anti-patterns grounded in the scientific literature on self-correction, reward hacking and model-as-judge fragility. The corpus shows that practice has matured most where the discipline says it matters: seventy percent of loops verify in the autonomous zone of the ladder and seventy-four percent name their terminal states, while automated triggering and durable memory remain comparatively underdeveloped. We close with the limits the practice must respect, including the verification burden, comprehension debt and the risk of cognitive surrender.

1 Introduction

The paper treats loop engineering as a distinct, reviewable layer for coding agents: humans design bounded external loops that operate harnesses rather than prompting every step. It defines the object, situates it alongside prompting, and grounds the practice in a hand-coded corpus and scientific literature.

Research gap

  • The paper asks what loop engineering builds, how it relates to internal agent loops, what real loops contain, and which practitioner claims withstand scientific scrutiny.The concept emerged through threads, talks, and blog posts rather than a reviewable account.Source: §1 · pp. 1–2

Definition and scope

  • A loop specification is an external, bounded, reusable artifact that combines a trigger, goal, verification, stopping rule, and memory for an agent harness.It is distinct from both ordinary programming loops and the harness’s internal perceive-act-observe cycle.Source: §1 · p. 2

Positioning the practice

  • The paper defines loop engineering as a new layer from prompt to context to harness to loop, with each layer subsuming rather than replacing the previous one.The loop specification operates the harness, so loop engineering complements prompt engineering rather than ending it.Source: §1 · p. 2, Figure 1 · p. 2

Contributions

  • It defines the discipline, develops a taxonomy, hand-codes fifty Loop Library entries, grounds design guidance in literature, and releases sandeco-loop as a loop-authoring skill.The paper presents these contributions as vocabulary and evidence for examining a fast-moving practice, not as a new algorithm.Source: §1 · pp. 2–3

2 Background: From Prompting to Loops

The paper places loop engineering above the harness’s internal agent cycle, whose lineage spans iterative reasoning, tool use, search, feedback, memory, and multi-agent coordination. The external loop specification determines when the harness runs, what counts as done, and when control returns to a human.

Workflow versus agent, and where the loop specification sits

  • The internal agent cycle is harness plumbing: a model repeatedly reasons, calls tools, reads results, and decides again over a stop condition.Its lineage includes MAPE-K, sense-plan-act, and observe-orient-decide-act cycles.Source: §2 · p. 3

From single prompts to iterative reasoning

  • Iterative reasoning methods improve single-pass prompting by adding intermediate reasoning, decomposition, planning, self-questioning, search, or task-specific reasoning structures.The cited methods include chain-of-thought, self-consistency, least-to-most, plan-and-solve, and Self-Discover.Source: From single prompts to iterative… · p. 3

Acting, not just reasoning

  • Tool-using systems shape the action phase by deciding which tools exist, when calls fire, how results return, and how sub-tasks are delegated.Examples span ReAct, Toolformer, ReWOO, formal planners, routing systems, and API-scale tool use.Source: Acting, not just reasoning · p. 3

Searching, learning, coordinating

  • Search and feedback methods replace a single trajectory with trees or graphs, critique-and-revise cycles, execution feedback, self-generated lessons, and accumulated skills.Frameworks package these mechanisms through memory, reflection, planning, conversational agents, procedures, and staged development.Source: Searching, learning, coordinating · p. 4

Workflow versus agent, and where the loop specification sits

  • The loop specification adds a further layer around the harness, deciding when the agent is invoked, what counts as completion, and when control returns to the human.It does not alter the model’s inner while-loop; the harness is the engine and the external specification is the pilot.Source: Workflow versus agent, and where the… · p. 4

3 Loop Engineering: Definition and Scope

Loop engineering studies an external artifact that makes agent work bounded, verifiable, stateful, and repeatable. Its central criterion is feedback that changes the next action, while its scope remains descriptive rather than experimentally benchmarked.

3 Loop Engineering: Definition and Scope

  • A loop specification is a human-designed artifact with a trigger, preferably verifiable goal, execution phase, verification, named stopping state, and persistent memory.The harness supplies the engine, while loop engineering writes the external pilot.Source: §3 · p. 4

Three senses of “loop,” and why scope matters

  • The paper distinguishes the external loop specification from ordinary control flow and the internal agent cycle that the harness provides by default.This separation prevents the practice’s external design problem from being confused with the academic lineage of agent plumbing.Source: Three senses of “loop,” and why scope… · pp. 4–5

The central skill is the check, not the prompt

  • Verification is the discipline’s central skill because reproducible checks replace blind trust and determine when control returns to the human.A loop without something that pushes back is merely an agent agreeing with itself.Source: The central skill is the check, not… · p. 5

The golden rule: a loop only when feedback changes the next action

The operational anatomy: five pieces plus memory

Scope of the contribution

  • The paper is a position paper anchored in a descriptive corpus study, not a controlled experiment or benchmark evaluation.It does not run loops under a measured budget and treats that study as future work.Source: Scope of the contribution · p. 6

4 Anatomy and Taxonomy of Loop Specifications

The taxonomy organizes loop specifications by trigger, goal, verification rigor, architecture, terminal states, and memory. It emphasizes objective checks, independent judging, explicit stopping states, and durable external state as safeguards for autonomous operation.

Trigger

  • Triggers may be manual, scheduled, or event-driven, with automation removing the human from startup while overly eager cadence wastes cost.Manual triggering is described as most common and least automated.Source: Trigger · p. 6

Goal type

  • Goals may be deterministically verifiable, model-judged, or mixed, but verifiable goals are preferred because they provide their own stopping rule.Purely subjective goals are not loopable because they lack reproducible checks; model-as-judge goals remain fragile [12].Source: Goal type · p. 6

The verification ladder

  • The five-level verification ladder ranges from deterministic checks and rules through delayed field truth to model judgment and human checkpoints.Levels 1–2 form the autonomous zone, levels 1–3 the objective zone, and levels 4–5 assisted flow; level 4 should use a different model as judge [27].Source: The verification ladder · pp. 6–7, Figure 3 · p. 7

Architecture

  • Architectures range from solo agents to maker-checker pairs and manager-led helper systems, with independent checking hardening model-judged verification.Multi-agent debate adds disagreement to counter premature self-trust [22].Source: Architecture · p. 7

Stopping rule and named terminal states

  • Named terminal states distinguish success, clean no-op, blocked, stalled, and exhausted outcomes, while errors and exhausted budgets never count as success.Stopping can follow goal completion, stagnation detection, or a budget ceiling.Source: Stopping rule and named terminal states · p. 7

State and memory

  • Because models forget between runs and deep content receives the least attention, durable state should be externalized in files rather than kept in conversation history [24].The Ralph loop replays the same prompt with fresh context while state lives in files, trading conversation history for versioned artifacts; curated external memory can improve agents without changing weights [52].Source: State and memory · pp. 7–8

5 Evidence from the Loop Library

The fifty-loop corpus shows strong reliance on autonomous, objective verification, but manual triggering, solo architectures, and persistent memory remain less developed. Most loops combine multiple pattern families, while named stopping and verifiable goals are more mature than mechanisms for running without human intervention.

Most loops verify in the autonomous zone

When a judge appears, it rarely judges alone

  • Among eleven level-4 judge loops, most pair judging with maker-checker architecture or a separate critic.This separates generation from verification rather than letting the agent grade its own work.Source: When a judge appears, it rarely judges… · p. 9

Triggers are still mostly manual; architectures mostly solo

  • 78% of loops are manually triggered and 78% use a single agent, while scheduled or event-driven triggers and multi-agent designs remain minorities.Only 12% are scheduled and 10% event-driven; 18% use maker-checker and 4% orchestrate helpers.Source: Triggers are still mostly manual;… · p. 9, Figure 5 · p. 9

Loops blend pattern families, but lean on defining and trusting

  • Loops combine pattern families, with a median of two per loop; defining “done” and trusting the result are more common than persistent state and memory.Family A appears in 44% of loops, C in 40%, B in 38%, and D in 32%; inferred labels do not change the ordering.Source: Loops blend pattern families, but lean… · p. 9, Figure 6 · p. 10

A maturity mismatch

  • 74% name terminal states, 70% verify autonomously, and 66% set verifiable goals, compared with 22% automated triggers, 20% named skills, and 32% persistent memory.The corpus has solved knowing when work is done more fully than running without a person.Source: A maturity mismatch · p. 10

6 Design Principles

The design principles organize robust loops around frozen verification, reversible action, independent checking, and governed memory. They also treat reusable skills as the compositional units that distinguish a durable loop from a brittle script.

Family A: define “done” before anything else

  • Reliable loops use a frozen external yardstick, consecutive successes, named terminal states, and stagnation or budget stopping.The guidance responds to evidence that unaided self-judgment can fail to improve and may degrade without external feedback [16].Source: Family A: define “done” before… · p. 10

Family B: act without breaking what works

  • Changing one thing per turn and rerunning checks keeps verification interpretable and limits regressions.The guidance also recommends clean starting states, surgical edits, baselines, and fixing the worst issue first.Source: Family B: act without breaking what… · p. 11

Family C: earn trust in the result

  • Separating maker and approver roles, using fresh hold-outs, and grounding claims in evidence reduces risks from self-judging and reward hacking [27].Independent review can counter degeneration, but debate must remain on task and does not guarantee correctness [22] [16] [4].Source: Family C: earn trust in the result · p. 11

Family D: sustain the loop over time

  • Durable memory should record progress and decisions while curating lessons by evidence, because unguided accumulation can reduce performance below zero-shot baselines [52].Irreversible actions should be gated behind explicit human approval.Source: Family D: sustain the loop over time · pp. 11–12

Loops should call skills

  • Reusable, named, tested skills make loops compositional, whereas loops without them are brittle repetitions around an unfamiliar system.The paper presents skill reuse as a distinct anti-pattern boundary and develops a skill applying these principles.Source: Loops should call skills · p. 12

7 Anti-Patterns and Evaluation

The paper identifies recurring loop designs that reduce reliability or waste resources, then proposes qualitative checks and a cost-aware metric for evaluating them.

The while-true around a stranger

The self-approving loop (reward hacking)

Specification gaming

  • Specification gaming lets loops satisfy checks without solving the task, including by editing tests, hard-coding outputs, or hacking the environment [6].Hold-out checks, proven verifiers, and preserving failing checks are proposed defenses.Source: Specification gaming · p. 12

Pretending level 4 is level 1

  • Model-based judges are vulnerable to bias, prompt sensitivity, drift, and attack, so their true verification level should be stated and hardened when necessary.Source: Pretending level 4 is level 1 · p. 12

The unattended runaway

  • Missing stopping rules, stagnation detectors, and budget ceilings allow loops to consume resources or take consequential actions unnoticed.Named terminal states, no-progress detection, budget ceilings, and human approval for irreversible actions provide the brakes.Source: The unattended runaway · p. 13

Evaluation criteria

  • The proposed checklist asks whether feedback changes the next action, verification is genuine, maker and checker are separated, terminal states are explicit, and memory persists appropriately.Source: Evaluation criteria · p. 13

Cost per accepted change

  • Cost per accepted change divides tokens or money spent by verified changes that survive, making raw token spend an inadequate health measure.The paper proposes keeping this metric low and flat or falling, while treating budget consumption without approved work as failure.Source: Cost per accepted change · p. 13

8 An Authoring Skill for Loop Specifications

sandeco-loop operationalizes loop-engineering principles by triaging tasks, interviewing authors, hardening specifications, and emitting an actuated document. The deliverable is presented as an authoring procedure rather than an evaluated contribution.

8 An Authoring Skill for Loop Specifications

  • sandeco-loop converts loop-engineering principles into a repeatable procedure that writes a hardened specification rather than running the loop itself.Source: §8 · p. 13

Triage before anything

The interview

  • The interview records the goal, check, trigger, stop states, skills, sub-loops, persistent state, guardrails, and approval points.Source: The interview · p. 14

Hardening against the anti-patterns

  • Hardening separates makers from checkers, forbids errors or exhausted budgets from counting as success, caps nested loops, persists state, and records cost per accepted change.Source: Hardening against the anti-patterns · p. 14

What it emits

  • The skill emits a single <name>-loop.md document containing the goal, verification, turn steps, terminal states, guardrails, memory, sub-loops, rationale, and actuation.Actuation varies with task size, using a /goal command for smaller work and a fresh-context Ralph skeleton for longer work.Source: What it emits · pp. 14–15

A worked example

  • A test-coverage example targets full directory coverage with an exit-zero coverage check, one-test turns, red-before/green-after validation, named stop states, and approval before CI changes.Source: A worked example · p. 15

Status

  • The skill is a released deliverable, not an evaluated contribution, and its effect on real-world outcomes remains an open empirical question.Source: Status · p. 15

9 Discussion and Limitations

The paper places loop engineering on a spectrum of autonomy while emphasizing that verification, human judgment, security controls, and empirical validation constrain what loops can safely claim.

Where the human sits

  • Thirty-six percent of corpus loops require human approval, concentrated around destructive, production, financial, or external actions.The authors argue that checkpoints should target irreversible steps because users may trust plausible-looking plans too readily.Source: Where the human sits · p. 15

What the practice must not overclaim

Security and the cost of autonomy

  • Unattended loops require isolated execution, scoped credentials, network restrictions, and budget caps because plugins can run arbitrary code and web access enables prompt injection.These controls are presented as necessary conditions for autonomous operation, not optional extras.Source: Security and the cost of autonomy · p. 16

When not to use a loop

  • Loops are inappropriate when feedback cannot change the next action, goals lack reproducible checks, direction is ambiguous, or looping costs more than it returns.In those cases, the paper recommends one-shot or scheduled execution, or avoiding the loop.Source: When not to use a loop · p. 16

Limitations

  • The corpus is a qualitative reading of one public catalogue, so its percentages describe that corpus rather than all loop specifications and support no causal conclusions.The authors also characterize transferring findings from internal agent cycles to external loops as an argued analogy, not a measured result.Source: Limitations · p. 16

Future work

  • Future studies should test loops under fixed budgets using cost per accepted change, while tooling should expose trigger, verification, architecture, terminal states, and memory explicitly.The paper presents this work as conceptual and descriptive groundwork for those empirical and tooling directions.Source: Future work · pp. 16–17

10 Conclusion

The conclusion defines loop engineering as a distinct layer that structures autonomous agent work without replacing prompting. It presents the corpus as evidence of a real but uneven practice whose central skill is verification, while emphasizing costs, limits, and the need for measurement.

10 Conclusion

  • Loop engineering is a distinct, deliberately designed practice that adds scaffolding around prompts rather than eliminating prompting.The paper defines loop specifications as bounded artifacts and distinguishes them from programming loops and internal agent cycles.Source: §10 · p. 17
  • The fifty-loop corpus suggests maturity in verification and named stopping, with greater room for improvement in automated triggers, reusable skills, and durable memory.The conclusion frames these observations as descriptive claims appropriate to a position paper, not measured performance results.Source: §10 · p. 17
  • The paper identifies verification as the loop’s central skill and calls for adoption with its costs and limits visible.It proposes cost per accepted change as the yardstick for turning the framing into measured evidence.Source: §10 · p. 17

Declaration on the Use of Generative AI

The author reports using Grammarly for textual agreement and Claude Opus 4.8 for text structuring and English translation, then reviewing and editing the manuscript.

Declaration on the Use of Generative AI

  • The author used Grammarly and Claude Opus 4.8 during manuscript preparation, reviewed the resulting content, and accepts full responsibility.Claude Opus 4.8 supported text structuring and translation into English.Source: Declaration on the Use of Generative AI · p. 17

Related papers

Loading 2607.00038v1…