Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents

1 INTRODUCTION

• The study analyzes 1,200 trajectories from four configurations and evaluates mitigations on 200 held-out Verified and 100 SWE-bench Pro tasks.Repeated runs account for execution stochasticity.Source: §1 · p. 2
• 79.00%–98.00% of tasks exhibit the three identified behaviors, which account for 6.86%–22.75% of task monetary cost.The behaviors are subsumed retrieval, similar script generation, and test re-execution.Source: §1 · p. 2
• Structure-aware retrieval can increase end-to-end cost by up to 28.14% as verbose feedback and altered delegation create retrieval overhead.Its effects on subsumed retrieval are inconsistent across architectures and models.Source: §1 · p. 2
• Developer-designed skills reduce task cost by 7.88%–41.73%, with maximum reductions of 38.57%–88.91% across the three behaviors.The skills provide high-level, trace-agnostic guidance.Source: §1 · p. 3
• Agent-synthesized skills provide low-level, trace-specific guidance and reduce cost by at most 22.32%, roughly half the maximum achieved by developer-designed skills.The paper identifies limited effectiveness and generality as concerns for synthesized skills.Source: §1 · p. 3

2 STUDY SETUP

• RQ1 labels 1,200 trajectories with an author-defined taxonomy and calibrated detectors, while RQ2–RQ3 compare three interventions with baselines on Pass@1, cost, cost-of-pass, and behavior changes.The interventions are CodeGraph, agent-synthesized skills, and developer-designed skills.Source: §2 · p. 3
• The study compares Claude Code with Sonnet 4.6 and Mini-SWE-Agent with Sonnet 4.6, MiniMax-M3, or Qwen-3.5 Plus.Comparisons isolate framework-associated differences under the same main model and model-associated differences within a fixed framework.Source: §2 · p. 3
• The 300-task RQ1 analysis uses the earliest 300 of 500 SWE-bench Verified tasks; mitigation evaluation uses the remaining 200 and 100 sampled SWE-bench Pro tasks.The Verified split preserves repository distribution, and Pro sampling covers all 11 repositories while favoring costlier tasks.Source: §2 · p. 3
• Eight baseline cells are triplicated, and effects are called robust at one-sided p < 0.05; unreplicated approaches inherit matched-baseline noise estimates under an equal-variance assumption.That assumption was checked on nine replicated approach cells; behavior-level changes are reported only as mechanistic evidence because their metrics are noisier.Source: §2 · p. 3

3 RQ1: ANALYZING COST-INEFFICIENT BEHAVIORS IN CODING AGENTS

• At least 79.00% of tasks exhibit one of the three behaviors, which together account for up to 22.75% of task cost.Subsumed retrieval concentrates in the first half of trajectories, test re-execution shifts toward the middle and later stages, and similar script generation occurs more broadly.Source: §3 · pp. 3–4, Figure 2 · p. 4

3.1 SUBSUMED RETRIEVAL

• In Claude Code, cross-agent retrieval accounts for 50.15% of subsumed retrieval; in Mini-SWE-Agent, shell editing and missing line numbers can prompt rereads, as can long debugging loops.Long-distance retrieval is most prominent in Mini-SWE-Agent with MiniMax-M3 (52.40%), whose average trajectory is 77.99 steps, versus 29.99–58.06 for other configurations.Source: §3.1 · pp. 4–5
• Subsumed retrieval affects 64.33–92.33% of tasks and contributes 5.01–11.41% of task cost, making it the most prevalent behavior.Its causes differ by architecture: cross-agent retrieval dominates in Claude Code, while Mini-SWE-Agent encounters low-feedback editing, zoom-in localization, and long debugging loops.Source: §3.1 · p. 5

3.2 SIMILAR SCRIPT GENERATION

• Similar script generation affects up to 68.00% of tasks and contributes up to 9.57% of task cost, occurring 5.91–9.98× more often in Mini-SWE-Agent than in Claude Code.In Mini-SWE-Agent, 42–66 tasks per configuration have at least six occurrences, compared with two in Claude Code.Source: §3.2 · pp. 5–6
• Similar scripts arise during reproduction, editing, and validation; 39.89–48.84% of occurrences immediately follow a similar script, while 21.71–34.60% follow at least six intervening steps.Ephemeral scripts are predominantly inspection probes (81.03–93.45%); persistent files also include validation and, in Mini-SWE-Agent, patch scripts, but can become less salient or harder to edit and reuse.Source: §3.2 · pp. 5–6

3.3 TEST RE-EXECUTION

• Test re-execution affects 49.67–83.00% of tasks and contributes up to 5.39% of task cost; Mini-SWE-Agent with MiniMax-M3 averages 5.29 occurrences per task.Source: §3.3 · p. 6
• Test re-execution commonly follows gaps in repository-specific test knowledge, attempts to recover unclear or truncated test output, or stalled reasoning without patch updates.Source: §3.3 · p. 6

4.1 CROSS-APPROACH COMPARISON

• Across approaches, mitigation generally improves from CodeGraph to agent-synthesized skills to developer-designed skills, with larger gains under Mini-SWE-Agent and on Verified-200 than on Pro-100.For skill-based approaches, the Verified-200 versus Pro-100 difference partly reflects skills derived from Verified trajectories and applied to Pro.Source: §4.1 · p. 6, Table 3 · p. 7
• Changes in task cost are amplified relative to changes in behavior-attributed cost; fitted slopes are 2.83 on Verified-200 and 1.33 on Pro-100.The section attributes this amplification to shorter trajectories reducing repeated context reading and, in some cases, preventing downstream actions such as reasoning about a test rerun.Source: §4.1 · pp. 6–7, Figure 6 · p. 7

4.2 RQ2: EFFECTIVENESS OF STRUCTURE-AWARE RETRIEVAL

• CodeGraph produces no robust cost reductions and four robust cost increases of 8.39–28.14%, despite reducing subsumed retrieval by over 75% under Claude Code on both benchmarks.Under Mini-SWE-Agent, subsumed retrieval increases in four of six settings.Source: §4.2 · pp. 7–8
• CodeGraph's verbose feedback and altered delegation can offset retrieval savings: under Claude Code, queries return 8.2–16.6× more tokens, while cheaper subagent calls fall to zero.Under some Mini-SWE-Agent settings, agents add 5.65–6.75 CodeGraph calls per task, raising cost by 8.39–28.14%; where ordinary retrieval is replaced, savings are indistinguishable from baseline.Source: §4.2 · pp. 7–8

4.3 RQ3: AGENT-SYNTHESIZED VS. DEVELOPER-DESIGNED SKILLS

• Agent-synthesized skills give configuration-specific, low-level operational rules, whereas developer-designed skills provide shared, higher-level principles for reusing context and artifacts and avoiding unwarranted test reruns.The authors suggest that trace-agnostic guidance helps explain the broader and more consistent effectiveness of developer-designed skills.Source: §4.3 · p. 8
• Developer-designed skills robustly reduce average cost in six of eight settings by 7.88–41.73%, versus three settings by 8.86–22.32% for agent-synthesized skills.Developer-designed skills have one small Pass@1 drop (−1.50 pp for Claude Code on Verified-200).Source: §4.3 · p. 8

Agent Trajectory Analysis

• Prior trajectory analyses examine agents’ task approaches, execution failures, and cost accumulation, but recurring inefficient behaviors, their cost contributions, causes, and mitigations remain underexplored.Source: §5 · p. 9

Agent Cost Reduction

• Existing cost-reduction work targets token use, interaction counts, orchestration, runtime supervision, or toolsets; whether structure-aware retrieval yields consistent end-to-end savings remains unclear.This paper evaluates diagnosed behaviors alongside cost and task performance.Source: §5 · p. 9

Agent Skills

• Prior agent-skill work builds reusable guidance from execution experience and evaluates skill effectiveness or cost, while this study derives skills from diagnosed inefficiencies and compares agent-synthesized with developer-designed guidance.Source: §5 · p. 9

Agent and Task Coverage

• The study covers four agent configurations, SWE-bench Verified, and 100 SWE-bench Pro tasks, so behavior and intervention effects may differ elsewhere.The authors also note that skill findings depend on their synthesis pipeline and developer-skill design; other skill formats may differ.Source: §6 · p. 9

Behavior Identification

• Behavior parsing, labeling, and detection may introduce errors despite explicit implementations and manual calibration.Source: §6 · p. 9

Cost and Outcome Measurement

• Monetary cost excludes execution time and hardware costs, while Pass@1 may miss regressions beyond test satisfaction.Source: §6 · p. 9

Execution Randomness

• Only baselines and nine intervention settings were triplicated; baseline-derived variability may not fully establish statistical significance.Other settings were limited to single runs by budget.Source: §6 · p. 9

Structure-aware Retrieval

• Using CodeGraph may limit generalization to other structure-aware retrieval tools.The authors selected CodeGraph for its popularity and robust structural retrieval implementations.Source: §6 · p. 9

Skill Design and Scope

• Findings about skills depend on the agent-synthesis pipeline and developer-skill design; script-based or other formats may differ.Source: §6 · p. 9

7 CONCLUSIONS & FUTURE WORK

• Developer-designed skills provide high-level, trace-agnostic guidance and achieve the strongest and most consistent cost savings.Agent-synthesized skills provide low-level, trace-specific guidance and moderate savings; structure-aware retrieval can increase cost through verbose feedback and altered delegation.Source: §7 · p. 9
• The study identifies subsumed retrieval, similar script generation, and test re-execution as prevalent cost-inefficient behaviors in Claude Code and Mini-SWE-Agent trajectories.Source: §7 · p. 9
• Future work will examine longer-horizon interactive tasks, more diverse agents, and inefficiencies beyond monetary cost.The authors also propose using behavior diagnoses to guide optimizations and generate more efficient trajectories for agent training.Source: §7 · p. 9

8 AI USE STATEMENT

• Experimental methods combine rule-based parsing with LLM-based action labeling under an author-defined taxonomy, and an LLM pipeline generates agent-synthesized skills for RQ3 evaluation.Source: §8 · p. 10
• AI assistants helped debug and refactor author-written code, refine manuscript wording and structure, and improve table and figure layouts.The authors independently designed the research questions, metrics, detection rules, analyses, findings, and conclusions, reviewed AI-assisted work, and took responsibility for the final outputs.Source: §8 · p. 10

A APPENDIX

• The appendix covers agent configurations, action labeling, task splits, execution randomness, additional RQ1 results, CodeGraph integration and RQ2 analyses, and skill implementations and contents for RQ3.Source: A APPENDIX · p. 15

• The setup covers agent versions and configurations, action labeling, benchmark task splits, and execution randomness.Source: §A.1 · p. 15
• Claude Code uses CLI v2.1.169 with per-task limits of $3.00 on SWE-bench Verified and $5.00 on SWE-bench Pro; its model temperature is not exposed.Source: §A.1 · p. 15
• Mini-SWE-Agent uses v2.3.0, temperature 0.0, a 250-step limit, and the same per-task cost limits as Claude Code.Source: §A.1 · p. 15

• Semantic action labels map different framework-specific actions to a shared vocabulary and restrict each detector to relevant steps, with Generate_Patch closing a ReTest group.Source: §A.1.1 · p. 15
• The taxonomy has 18 labels in five categories, and a tool call may receive up to two labels, such as Generate_Tests and Execute_Generated_Tests.Source: §A.1.1 · p. 15
• Rule-based labeling splits MSA shell commands into atomic commands and classifies them by command and path patterns, while CC tools map directly and Bash uses the same classifier.Source: §A.1.1 · p. 16
• An LLM judge labels steps unmatched by rules; fallback handled 7.26% of total steps across four configurations on 300 Verified tasks, at less than 0.2 cents per trajectory.Source: §A.1.1 · p. 16
• Source reading (Retrieve_Code, 25.65–33.12%) and script generation plus execution (23.66–34.07%) dominate every configuration, followed by existing-test execution (12.15–13.57%).Source: §A.1.1 · p. 16, Figure 7 · p. 16

• For each flagged action, token counts cover its reasoning, commands, and tool output using the o200k base tokenizer from tiktoken.Source: §A.1.2 · p. 17
• Generated content is charged at the output-token price, while generated content and tool output carried into later LLM requests incur input-token or cache-token charges.Source: §A.1.2 · p. 17
• The action’s attributed cost is the sum of these charges, priced using the corresponding model’s token prices.Source: §A.1.2 · p. 17

• For Verified, tasks are ordered by repository creation date, with the earliest approximately 60% assigned to analysis and the remainder to Verified-200.Source: §A.1.3 · p. 17
• The Pro sample draws 100 of 731 public tasks, allocating repository quotas by square-root pool size with a minimum of four to represent all 11 repositories.Source: §A.1.3 · p. 17
• Within each Pro repository, tasks are split into cost terciles and sampled at 20%, 30%, and 50% for low, mid, and high costs, respectively.Source: §A.1.3 · p. 17

A.1.4 CONTROLLING AGENT EXECUTION RANDOMNESS

• Pass@1 noise floors range from 0.87 pp to 4.00 pp, and approach runs are not systematically noisier than baselines except one CodeGraph cell.CodeGraph with MSAQ35+ on Pro-100 has 2.58× baseline cost noise, so its own 8.85% cost and 17.05% CoP floors are used; verdicts remain unchanged under a 1.5× noise-floor inflation. Behavior-frequency CVs average 15.68%, so behavior changes are treated as mechanistic evidence without robustness verdicts.Source: §A.1.4 · pp. 18–19

A.2 ADDITIONAL RESULTS FOR RQ1

• CC achieves 76.67% Pass@1 at $0.52 per task, while MSAQ35+ costs $0.10 per task and achieves 65.67%.MSAMM3 trades 1.67 pp of Pass@1 against MSAS46 for a 34.04% lower task cost.Source: Table 9 · p. 19, §A.2 · p. 19

A.2.1 WHY CC EXHIBITS FEWER COST-INEFFICIENT BEHAVIORS THAN MSA

• CC’s prompts and tools suppress file-based similar-script generation and some retrieval inefficiency, but provide no guidance against repeated test runs.CC creates 0.21 draft files per task versus 2.72–7.07 for MSA, while ReTest remains comparable: 2.09 executions per task for CC, 2.51 for MSAS46, and 2.10 for MSAQ35+.Source: §A.2.1 · p. 19

A.2.2 SUBSUMED RETRIEVAL

• Agents re-read 11.05–18.10% of retrieved code lines through subsumed retrieval, whose severity depends more on context management than retrieval volume.Examples include re-reading after an edit fails, zooming into a file already read in full, and re-reading after a long debugging stall.Source: Table 10 · p. 20, §A.2.2 · p. 20
• Similar-script generation recurs in both ephemeral probes and file-based patching, as agents regenerate near-duplicate scripts instead of reusing existing logic.CC emitted four 23–29-line scripts with largely repeated setup, while MSAMM3 created three patch scripts sharing the same scaffold; the section classifies scripts by AST-derived function.Source: §A.2.2 · p. 21, Figure 11 · p. 21

A.2.4 TEST RE-EXECUTION

• ReTest is long-tailed: 107 of MSAMM3’s 249 affected tasks contain at least six actions, versus 39 of CC’s 149.Repeated executions arise from repository-specific test-runner gaps, attempts to recover usable test signals, and progress stalls; MSAS46, for example, reran tests 22 and 18 times without another patch.Source: §A.2.4 · pp. 21–22

A.3.1 CODEGRAPH INTEGRATION

• CodeGraph is integrated as an AST-based, read-only symbol index in both frameworks, but cannot run on 13 Pro-100 tasks.It is accessed through an MCP server by CC and shell commands by MSA; the evaluation compares its 87-task Pro subset with the baseline on the same tasks.Source: §A.3 · p. 22, §A.3.1 · p. 23

A.3.2 CODEGRAPH USAGE

• Agents use CodeGraph in 74.00–97.50% of tasks, with source reading and symbol lookup accounting for most queries.Together these functions make up 99.33% and 89.46% of CC queries on Verified-200 and Pro-100, and 80.84–95.30% of MSA queries.Source: §A.3.2 · p. 23, Table 12 · p. 24
• File-structure queries are rare on Verified-200 (0.11–4.35%) but more common on Pro-100 (7.32–14.79%), while index-status queries are almost never issued.The passage attributes the difference to Pro's larger, multi-language repositories.Source: §A.3.2 · p. 24

A.4 ADDITIONAL DETAILS FOR RQ3

• Because each configuration has a single rule-only skill set and no selection decision, the entire set is preloaded into the system prompt for every task.The authors state that full preloading keeps the rules consistently available rather than conditionally loaded and avoids on-demand discovery of bundled scripts or resources.Source: §A.4 · p. 24

A.4.1 AGENT-SYNTHESIZED SKILLS

• Since full agent analysis costs nearly as much per trajectory as solving the task, the pipeline analyzes the top 25% by diagnosed inefficiency cost share with the agent and uses one LLM call for the remaining affected trajectories.The single-call mode receives compact step summaries, and Trace2Skill hierarchical consolidation merges per-trajectory rules into one skill set per configuration.Source: §A.4.1 · pp. 24–25
• After amortization, only MSAMM3 on Pro-100 retains a robust SynSkills cost reduction (-9.65%); MSAQ35+ has robust cost increases of +11.55% and +8.48% on both benchmarks.Because DevSkills have no synthesis cost, they reduce cost more than amortized SynSkills in seven of eight settings; the exception is MSAMM3 on Pro-100 (-7.88% versus -9.65%). The synthesis surcharge would fall below 2.00% of baseline cost if applied over 3,000 tasks.Source: §A.4.1 · p. 25, Table 15 · p. 25
• Synthesized rules advise running tests once per code state, reusing already-read source content, and preferring existing tests over ad hoc verification scripts.The supplied skill content also discourages unchanged test reruns and duplicate script variants.Source: §A.4.1 · pp. 25–26
• Use one canonical, self-contained reproduction script and consolidate related probes, scenarios, and checks into a single batched run.The skills favor evolving the script in place rather than creating numbered variants or repeatedly relying on inline commands.Source: §A.4.1 · pp. 28–29
• Cache read content and search findings, use targeted queries, and stop broad exploration once the fix is clear and verified.The guidance says to reread only after an edit, external change, truncation, or genuinely new information need.Source: §A.4.1 · pp. 28–29
• Make localized changes surgically, and after repeated failures diagnose the root cause or switch strategy instead of retrying the same approach.For uncertain fixes, the skills suggest prototyping in a scratch buffer before changing tracked source.Source: §A.4.1 · p. 28
• Define exploration and artifact scope before starting, follow existing project and test conventions, and use selective version-control operations.The skills also advise checking existing tests and source before authoring reproductions, and avoiding broad operations that could affect unrelated work.Source: §A.4.1 · pp. 28–29
• Run focused validation once per code state, capture its output, and rerun only after underlying code or inputs change.The guidance also recommends trusting passing results and limiting regression checks to the changed path and closely related modules.Source: §A.4.1 · pp. 27, 29

A.4.2 FULL TEXT OF THE DEVELOPER-DESIGNED SKILLS

• Before reading a file, state a concise expectation; first check whether its content is already in context and reuse it when possible.The developer-designed guidance advises rereading only when necessary and briefly stating the reason.Source: §A.4.2 · p. 30
• When modifying source code, agents should use targeted edits, identify the exact old and new strings, and make the minimal change.The skill warns against overwriting an entire existing file.Source: §A.4.2 · p. 30
• Before running unfamiliar tests, agents should inspect repository documentation, CI configuration, or test scripts to identify the correct runner and treat loader or dependency failures as harness errors.The skill says to skip this check when the working command is already known.Source: §A.4.2 · p. 30
• Agents should capture test output and extract its signal rather than rerunning tests unless the source code has changed.The skill recommends redirecting full test output to a file for filtering.Source: §A.4.2 · p. 30
• Agents should save reusable or multi-step scripts as workspace files, avoid numbered variants, and revise the original artifact unless separate versions are necessary.The skill says to state the reason before creating a separate version.Source: §A.4.2 · p. 30
• When repeated reads, tests, or edits produce no new evidence, agents should stop, state what remains uncertain, form a hypothesis, and take a materially different action..Source: §A.4.2 · p. 30
• If stuck on a subproblem such as test-environment setup, agents should submit their best current solution rather than exhaust the budget without submission.The skill states that a correct patch that cannot be fully verified is preferable to no submission.Source: §A.4.2 · p. 30