Cadence: Strategic Guidance for Coding Agents

1 Introduction

• Rule-based guidance misses reasoning errors without fixed patterns, while periodic guidance alternates between unnecessary overhead and insufficient supervision.Frequent checks burden executions with minor lapses; sparse checks can leave severe failures, such as repeated invalid patches, uncorrected until they compound.Source: §1 · p. 2
• Strategic guidance jointly decides when to inspect and how to intervene according to the agent’s runtime execution health.This approach moves beyond rigid schedules and simple rules toward context-aware intervention.Source: §1 · p. 2
• Cadence combines an advisory-or-replacement intervention module with a scheduler that lengthens intervals after advisory guidance and resets them after replacement guidance.Advisory guidance lets the current step proceed; replacement guidance discards the erroneous step and requests regeneration.Source: §1 · p. 2
• 25.33% (+76 resolved tasks) on mini-swe-agent and 15.67% (+47 tasks) on Moatless are Cadence’s resolve-rate gains over unmonitored baselines.Cadence also uses 0.48M tokens per task on Moatless and 1.03M on mini-swe-agent, compared with 0.51M and 1.47M for the state-of-the-art monitor.Source: §1 · p. 2
• Cadence’s evaluation on SWE-bench Lite reports up to 25.33% higher resolve rates than vanilla agents and up to 29.93% token savings versus state-of-the-art monitors.The authors describe it as the first runtime monitor to combine strategic inspection with guidance.Source: §1 · p. 2

2.1 Periodic Guidance: Does One Interval Fit All?

• The periodic-guidance study compares four inspection intervals—3, 5, 9, and 15 steps—using mini-swe-agent trajectories grouped by vanilla-agent task outcomes.Two experts manually inspected all 300 vanilla trajectories to characterize the resolved and unresolved groups’ execution-health profiles.Source: §2.1 · p. 3
• A 9-to-15-step interval reduces resolve rate for compromised executions from 23.87% to 14.19%, while favorable executions stay at or above 88.28% across intervals.Increasing the interval from 3 to 15 steps reduces token consumption in both groups; the study concludes that no single fixed interval fits all executions.Source: §2.1 · pp. 3–4

2.2 Rule-based Guidance: Are Heuristics Sufficient for Misbehavior Detection?

• 35 of 152 failed tasks (23.03%) never received guidance from the rule-based monitor, which missed diagnosis and repair errors not captured by predefined patterns.The undetected cases include wrong repair targets, faulty repair logic or implementation, and incomplete repairs.Source: §2.2 · p. 4

3.1 Problem Formulation

• The runtime-monitoring task models an agent trajectory as sequential thought, action, and observation steps, with a monitor inspecting prefixes and inserting guidance.The agent conditions its subsequent steps on the resulting trajectory, with the aim of mitigating misbehavior and steering task resolution.Source: §3.1 · p. 5

3.2 Cadence Overview

• Cadence pairs a two-tier intervention module, which decides how to intervene, with a scheduler, which decides when to inspect.The cycle starts from a base interval and continues until the task is complete.Source: §3.2 · p. 5

3.3 Two-Tier Intervention Module

• The intervention module uses LLM-based semantic reasoning to assess recent execution health and choose advisory guidance for satisfactory progress or replacement for compromised progress.Advisory guidance allows the proposed action to execute; replacement discards it and prompts regeneration. Health is defined at a high conceptual level to support generalization across agents.Source: §3.3 · pp. 6–7
• Cadence decomposes a failing trajectory into localized execution-health assessments rather than simply re-prompting a secondary LLM with the original problem.This reshapes the prompt toward diagnostic reasoning, while the authors say monitor capability remains necessary.Source: §3.3 · pp. 6–7

3.4 The Inspection Scheduler

• After advisory guidance, the scheduler increases the inspection interval by a fixed growth step up to a maximum; after replacement, it resets the interval to its base value.The intervention level serves as an execution-health signal for backing off during stable execution and intensifying checks during serious misbehavior.Source: §3.4 · p. 7

4.1 Datasets

• SWE-bench Lite contains 300 repository issues, resolved only when all FAIL_TO_PASS and PASS_TO_PASS tests pass after applying the agent’s patch.The task supplies a description, base commit, and tests.Source: §4.1 · p. 8

4.2.1 Monitoring Baselines.

• The monitoring baselines cover fixed-interval guidance, LivePlan’s rule-triggered LLM guidance, static rule-based messages, and an unmonitored Vanilla control.Fixed-Interval uses synchronous inspection to avoid LLM latency confounding intervention timing.Source: §4.2.1 · pp. 8–9

4.2.2 Working Agents.

• The study tests mini-swe-agent, which operates through bash commands, and Moatless, which uses 11 structured tools and semantic search.Moatless builds a repository vector index before the run.Source: §4.2.2 · p. 9

4.3 Evaluation Metrics

• Effectiveness is measured by Resolve Rate; efficiency by Tokens per Task and Token Leverage, with monitor LLM calls per task also reported.Token Leverage is resolved tasks per million total tokens consumed; Tokens per Task includes agent and monitor tokens.Source: §4.3 · p. 9

4.4 Environment and Implementation

• Experiments run in isolated Modal sandboxes using official prebuilt task images and default step limits of 150 for mini-swe-agent and 50 for Moatless.Cadence’s reference settings are base interval 5, growth step 2, and maximum interval 20; Moatless uses linear exploration, and supported models use temperature zero.Source: §4.4 · pp. 9–10

• The evaluation measures Cadence’s overall effectiveness and efficiency.This is RQ1.Source: §5 · p. 10
• The evaluation tests robustness across configurations.This is RQ2.Source: §5 · p. 10
• The evaluation uses ablation and trajectory analysis to examine module contributions, successes, and failures.These are RQ3 and RQ4.Source: §5 · p. 10

5.1 RQ1: How effective and efficient is Cadence compared with existing runtime monitors?

• 4.33% (+13 tasks) over LivePlan on Moatless and 8.33% (+25 tasks) over Fixed-Interval (𝑘= 5) on mini-swe-agent are Cadence’s resolve-rate gains over the strongest baseline.Cadence’s absolute gains over Vanilla are 15.67% on Moatless and 25.33% on mini-swe-agent.Source: §5.1 · p. 10
• 17 unique Moatless tasks and 20 unique mini-swe-agent tasks are resolved by Cadence; these tasks are primarily rescued from incomplete repairs and incorrect bug-root-cause locations.The overlap visualization compares resolved tasks among Cadence and four baselines, using Fixed-Interval (𝑘= 5) as its primary configuration.Source: §5.1 · p. 10, Figure 6 · p. 11
• Cadence significantly outperforms all baselines on both agents except LivePlan on Moatless, where the adjusted p-value is 0.1175.The authors relate the narrower Moatless margin to its structured action space, which enables more reliable mapping of actions to LivePlan rules.Source: §5.1 · p. 11
• On mini-swe-agent, Cadence uses the fewest tokens per task and has the highest token leverage; on Moatless it has the top token leverage with token use comparable to LivePlan (0.48M versus 0.51M).The reported efficiency advantage is attributed to extending inspection intervals during healthy execution and tailoring interventions to error severity.Source: §5.1 · p. 11
• 42.33% is Cadence’s highest resolve rate across the two agents, and its best token leverage exceeds the strongest baseline by up to 105% (0.41 versus 0.20).On Moatless, Cadence’s token leverage is 0.83 versus 0.69 for LivePlan.Source: §5.1 · p. 11

5.2.1 Robustness to Backbone Models.

• Across tested coding-agent and monitor backbones, Cadence beats Vanilla; changing the coding-agent backbone yields a 15.00% resolve-rate gain with Qwen3-Coder-480B-A35B and 4.67% with GPT-5-mini.The reported gains vary partly with trajectory length: 60.0 steps for Qwen and 26.6 for GPT-5-mini.Source: §5.2.1 · p. 12

5.2.2 Robustness to Scheduling Parameters.

• Varying one parameter at a time, the study tests base intervals 3 and 7 around the standard 5-step interval and growth steps 4 and 6, omitting the safety-fallback maximum interval, reached by only 1.7% of inspection steps across all experiments.Source: §5.2.2 · p. 12
• 34.00% is Cadence’s lowest resolve rate across tested schedule settings, matching Fixed-Interval (𝑘= 5) while using 0.96M versus 1.75M tokens per task.This occurs with base interval 5 and growth step 6; token leverage shows no clear dependence on the tested schedule parameters.Source: §5.2.2 · p. 13, Table 3 · p. 13

5.2 RQ2: How robust is Cadence across different backbone models and scheduling parameters?

• Cadence improves resolve rates by up to 20.33% across diverse backbone models and matches the strongest baseline using 45.1% fewer tokens, even under its least favorable parameter configuration.Source: §5.2.2 · p. 13

5.3.1 Ablation of the Inspection Scheduling Module.

• Fixed schedules with more inspections do not consistently improve resolution: the 5-step variant raises inspection and token use but yields virtually no gain over full Cadence.The 3-step variant raises resolve rate by 3.33%, but the increase is not statistically significant and token efficiency is lower.Source: §5.3.1 · p. 14

5.3.2 Ablation of the Two-Tier Intervention Module.

• Advisory interventions can require extra remediation steps after flawed code or deleted files, whereas replacement halts erroneous actions and triggers regeneration immediately.The always-advisory variant consumes 12.2% more tokens than the always-replacement variant.Source: §5.3.2 · p. 14

5.3.1 Ablation of the Inspection Scheduling Module.

• Replacing dynamic scheduling with a fixed interval of 7 steps under a similar token budget lowers resolve rate by 9.00%.The 7-step variant uses 1.025 versus 1.028 million tokens per task; the 5-step variant uses more resources but yields virtually no resolution gain, and the 3-step variant’s 3.33% increase is not statistically significant.Source: §5.3.2 · p. 14, §5.3.1 · p. 14

5.3.2 Ablation of the Two-Tier Intervention Module.

• Replacing two-tier intervention with a single fixed strategy lowers resolve rate by up to 7.33%.Always-advisory and always-replacement variants reduce resolve rates by 7.33% and 4.33%, respectively, versus full Cadence.Source: §5.3.2 · p. 14

5.4.1 Resolved Cases.

• 90 of 112 critical intervention cases (80.4%) failed under the vanilla agent setup.Experts identified 112 cases where misbehavior would likely lead to task failure without monitor correction.Source: §5.4.1 · pp. 14–15
• Among 112 corrected cases, replacement guidance was decisive in 88 (78.6%) and advisory guidance in 24 (21.4%).Reasoning errors made up 72.3% of corrected misbehaviors; tool-use errors made up 27.7%, with 29 of 31 tool-use cases occurring under mini-swe-agent.Source: §5.4.1 · p. 15, Figure 7 · p. 15
• In a Django case, Cadence blocked a repair placed in the wrong deletion path and redirected the agent to the fast path, where it resolved the issue.The baseline agent submitted an invalid patch.Source: §5.4.1 · p. 15, Figure 8 · p. 16

5.4.2 Failed Cases.

• Across 284 analyzed failures, 57.7% were attributed to the monitor and 42.3% to the coding agent.Root-Cause Misdiagnosis was the most prevalent monitor category (26.8%), while Incomplete Guidance Realization was the largest overall category (39.4%).Source: §5.4.2 · p. 16, Table 5 · p. 17
• Inspection Timing Failure accounted for 8.1% of cases and occurred only on Moatless, whose multiple tool calls per turn compress trajectory steps.The authors suggest incorporating tool invocation counts alongside step bounds when determining inspection timing.Source: §5.4.2 · p. 16
• Cadence’s identified shortfalls include untimely inspection, misdiagnosis, inaccurate guidance, and agents failing to fully execute guidance.Proposed improvements include deeper root-cause reasoning and checklists that track whether flagged defects are resolved before submission.Source: §5.4.2 · p. 16

6 Threats to Validity

• Manual annotation may introduce subjectivity; two annotators determined categories and resolved discrepancies through discussion.The study also set generation temperature to 0 whenever supported, used statistical significance tests, and followed official implementations or original papers for baselines.Source: §6 · p. 17
• Evaluation focuses on Python bug-fixing and feature-implementation tasks, not other task types or programming languages.The authors plan to evaluate Cadence on SWT-bench and SWE-bench Pro to further assess generalizability.Source: §6 · p. 17
• Possible exposure of underlying LLMs to evaluation tasks is a threat, although shared backbones mean contamination affects monitors comparably in relative performance comparisons.Source: §6 · p. 17

7.1 Offline Learning

• Offline learning uses historical execution records for model post-training or prompt-based experience reuse, but cannot intervene during runtime misbehavior.Examples include reinforcement learning, imitation learning, and trajectory-derived workflow or experience memories.Source: §7.1 · p. 17

7.2 Online Monitoring

• Online monitoring inspects runtime trajectories and delivers guidance upon detecting misbehaviors.The reviewed triggering mechanisms are rule-based and periodic guidance.Source: §7.2 · p. 18

7.2.1 Rule-based Guidance.

• LivePlan uses action-phase mappings and action graphs to detect skipped or stalled phases and loops, then blocks severe misbehavior or appends guidance for minor cases.Its action-phase mappings require manual adjustment when deployed on a new agent.Source: §7.2.1 · p. 18
• Heuristic rules can detect patterns quickly but miss complex reasoning errors requiring natural-language understanding.LLM-based inspection can cover broader reasoning errors by analyzing raw trajectory text independently of an agent’s action space.Source: §7.2.1 · p. 18, §7.2.2 · p. 18

7.2.2 Periodic Guidance.

• Wink inspects at fixed step intervals and uses asynchronous calls to reduce latency, but severe errors can cascade before results arrive.Its fixed interval gives healthy and misbehaving agents the same supervision frequency regardless of runtime status.Source: §7.2.2 · p. 18

8 Conclusion

• Cadence achieves the highest resolve rate among evaluated monitors while maintaining competitive token efficiency.Evaluation covered 300 SWE-bench Lite tasks.Source: §8 · p. 18
• Cadence gives advisory guidance when execution health is satisfactory and replacement guidance for severe misbehaviors.Source: §8 · p. 18
• Its scheduler increases inspection frequency after replacement-level guidance and reduces it after advisory-level guidance.Source: §8 · p. 18