The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents

1 Introduction

• Logical-token counts alone do not determine physical cost because request boundaries and prefix reuse change executed prefill; unified-memory devices also share memory across model and host allocations.The paper tracks host allocation, paging, and latency alongside token measures.Source: §1 · p. 1
• The study asks when logical-token savings predict physical savings in a frozen retrieval-augmented path that can preload ten code blocks.Repository-localization methods already include iterative retrieval, structure-aware search, and graph navigation [4] [5] [7] [8] [9] [10] [11].Source: §1 · p. 1
• A quality-gated protocol pairs task-quality bounds with calls, uncached prefill, KV time-area, wall time, and idle-subtracted GPU energy.It separates executed prefill into logical input minus reused prefixes and reports index construction separately.Source: §1 · p. 2
• On 450 localization cases, both preload policies pass the quality gate, while eager evidence saves work by avoiding calls and anchored reduction adds no further saving.Repeated observation masking can erase prompt savings, consistent with reduced prefix reuse.Source: §1 · p. 2

2.1 Two Paths from Context to Cost

• Prompt savings increase executed prefill when a policy loses more reusable tokens than it removes, while fewer calls can reduce total work even if individual calls are not cheaper.The treatment–control decomposition is ΔTuncached = ΔTprompt − ΔThit.Source: §2.1 · p. 2

2.2 Frozen Code-RAG Supply Path

• The frozen Code-RAG path compiles repository snapshots into CodeNib blocks, uses a revision-pinned encoder for query and code, and retrieves ten spans.Query encoding and search count toward preload-task cost; compilation is amortized separately, and preload inputs are hash-verified.Source: §2.2 · p. 3

2.3 Policy Algebra

• Policies vary evidence supplied before the first turn and visible history serialization; EAGER keeps full history, COMPACT rewrites it after a successful read, and MASK-5 shortens older outputs.The anchor and MASK-5 can change later actions and invalidate reuse; MASK-5 rewrites observations once they leave its five-observation window.Source: §2.3 · p. 3

2.4 Quality-Constrained Resource Objective

• A policy enters the cost comparison only if its lower paired 95% quality endpoint clears the registered non-inferiority threshold; failures remain in denominators and resource totals.The resource vector includes logical tokens, requests, uncached prefill, latency, KV occupancy, unified-memory pressure, and sampled GPU energy.Source: §2.4 · pp. 3–4

Evaluation questions

• RQ1 tests quality-gated delivery and retention policies, RQ2 examines how logical-token ordering translates to physical resource measures, and RQ3 tests whether the pattern recurs in an external agent scaffold.A same-model prefix-cache intervention isolates prefix reuse.Source: §3 · p. 4

Localization cohorts

• 100 resource tasks from 25 pinned repository snapshots yield 900 cells, while a separate 450-case SWE-rebench-derived cohort powers the localization quality gate.The cohorts are not pooled; physical costs use the 100-task cohort.Source: §3 · p. 4

Localization stack and host

• Localization studies use a fixed Qwen3.6-35B-A3B-FP8 model with vLLM on a DGX Spark, with prefix caching disabled in the resource matrix and toggled only for the cache intervention.The setup uses a 65,536-token context cap and one measured request at a time.Source: §3 · p. 4

Cache mechanism study

• 600 cells over the same 100 localization tasks compare cache regimes while holding model, runtime, decoder, task order, and revision fixed.Resets and salts restrict reuse to each four-query block.Source: §3 · p. 4

External patch study

• 100 SWE-bench Verified instances across all 12 Python repositories are evaluated with mini-SWE-agent using one trajectory, a 32-call cap, and a 1,800-second budget per arm.The three seeded arms receive identical top-10 candidates; resolved rate is measured under the fixed call cap.Source: §3 · pp. 4–5

Measurement and analysis

• External-study policies are compared only within their own stack because the target and serving stack differ from localization, preventing separation of model, scaffold, and runtime effects.Resource measurements combine trace, server, KV, power, and host-memory data; bootstrap procedures resample repository groups or shards, and the instrumented study lacked prospective power analysis.Source: §3 · p. 5

4.1 RQ1: Quality-Eligible Policies and Shorter Trajectories

• EAGER and COMPACT both pass the five-point quality gate on 450 localization cases; EAGER differs by +0.8 points with a −2.6 lower endpoint, and COMPACT differs by +2.4 with a −0.9 lower endpoint.Nonconforming outputs score zero in the localization cohort.Source: §4.1 · p. 5, Figure 2 · p. 6
• EAGER saves 46.3k prompt tokens per task (33.9%) in the instrumented primary, while COMPACT uses 8.5% more prompt tokens, 13.3% more calls, 8.2% more wall time, and 9.4% more energy than EAGER.The extra calls account for the gap; the traces do not identify call intent, so the attribution remains open.Source: §4.1 · pp. 5–6

4.2 RQ2: Physical Savings Follow Fewer Calls

• 1.98 kJ per full-recall localization for EAGER compares with 3.00 kJ for GREP and 2.27 kJ for COMPACT; all four primary cost-metric intervals exclude zero.The quality-adjusted ordering matches the per-attempt ordering.Source: §4.2 · p. 6, Table 1 · p. 6, Figure 2 · p. 6
• 767.7 seconds and 36.1 kJ build the 25 repository indexes; amortization adds 3.07 seconds and 0.144 kJ per query at ten queries per snapshot.At 100 queries, index construction is below 1.1% of energy.Source: §4.2 · p. 7
• +8.3% logical prompt for EAGER with caching has a 95% CI spanning zero, while uncached prefill falls by 67.7% (95% CI [−71.0, −63.8]).Both cache-interaction intervals include zero.Source: §4.2 · p. 7

4.3 RQ3: Token Savings Reverse under External Retention

• −33.3% prompt volume under MASK-5 accompanies +608.5% uncached prefill and +125.6% GPU-energy proxy; the policy resolves 26/100 attempts versus EAGER’s 29/100 and misses the quality gate.MASK-5 removes 75.1k logical prompt tokens but loses 135.9k device-hit tokens; the prefill inversion is consistent with reduced prefix reuse, though trajectory changes may also contribute.Source: Figure 2 · p. 6, §4.3 · p. 7

5 Implications for Resource-Aware Context Management

• EAGER uses the fewest resources among eligible localization arms; COMPACT remains eligible but adds no saving because shorter requests do not offset its additional calls.The comparison separates retrieval delivery from history reduction.Source: §5 · p. 7
• MASK-5 removes 75.1k logical tokens but loses 135.9k reusable tokens, so its uncached prefill rises; masking would lower prefill only when removed tokens exceed lost reuse.At the observed hit fraction ρ, the first-order change is −75.1k + 135.9k · ρ tokens per attempt, turning positive above ρ ≈ 0.55.Source: §5 · pp. 7–8
• Apply the quality gate before interpreting resource differences: ELOC and CLOC qualify against GREP, but M5 does not qualify against EPATCH.Token count alone would favor the policy excluded by the gate; other patch contrasts remain descriptive.Source: §5 · p. 8

6 Related Work

• Repository localization progressed from report–code similarity and history to iterative retrieval [4], structure-aware search [5] [7], and code-graph navigation [8] [9] [10] [11].The paper adopts the function/class endpoint and freezes retrieval to study the resources used to deliver and retain evidence.Source: §6 · p. 8
• Context-management work ranges from static prompt compression [13] [14] to externalized memory, observation masking or summarization, pruning, and structured state [15] [16] [17] [34] [35] [36].Repository compression has also been adapted to task [37].Source: §6 · p. 8
• Runtime work reuses KV state or selectively retains tokens [18] [19] [21] [38] [39] [41], while KV compression can lengthen outputs and latency.The paper measures how context policies interact with reuse.Source: §6 · p. 8
• The protocol extends cost-aware evaluation motivated by Green AI, HELM, and Zeus with a prespecified quality gate, uncached prefill, KV time-area, and charges for failures and index construction.Source: §6 · p. 8

• Evaluation covered two stacks on one DGX Spark serving one request at a time; batching, cache pressure, and cloud GPUs remain untested.Source: §7 · p. 8
• The four fixed policies leave window sweeps, LLM summarization, learned pruning, and external memory unmeasured; the GPU-energy proxy also excludes CPU, host-memory, and whole-system energy.Source: §7 · p. 8
• The 32-call patch cap measures fixed-budget performance, not unrestricted SWE-bench ability, and localization quality and detailed cost use separate cohorts of 450 and 100 tasks.Source: §7 · p. 8
• A context policy changes both agent requests and reusable prefixes; resource comparisons should measure executed work and require policies to pass a quality gate.Source: §7 · p. 8
• A 33.3% prompt saving from repeated observation masking coincided with a 608.5% rise in uncached prefill.Source: §7 · p. 8

• The primary configuration pins implementation, datasets, model endpoint, decoding, and policies; host and run metadata are captured, while raw outputs and telemetry remain immutable for table regeneration.Source: A Frozen Protocol and Artifact Notes · p. 13
• The post-run audit verified 897 traces but retained failed status for three cells whose exception records omitted preload-candidate hashes and final trace records.Source: A Frozen Protocol and Artifact Notes · p. 13
• The supplementary studies add 22.55 device-hours and 0.464 kWh, while the accounting excludes model-server residency and other uninstrumented development.Source: A Frozen Protocol and Artifact Notes · p. 14
• The external protocol resets the Radix cache for each policy–shard block, limits generation to 32 calls, and excludes official grading from generation-resource totals while retaining empty patches and test failures in the denominator.Source: A Frozen Protocol and Artifact Notes · p. 14
• A rejected uninitialized Astropy gitlink invalidated and triggered rerunning the full 13-cell EAGER block; the recovery restricted accepted gitlinks and kept networking disabled.Source: A Frozen Protocol and Artifact Notes · p. 14
• For n queries against one snapshot, reported amortized cost is Cquery + Cbuild/n; compilation time and energy are also recorded.Source: A Frozen Protocol and Artifact Notes · pp. 13–14
• The artifact can distribute manifests and rebuild scripts without compiled vectors, avoiding relicensing source-derived index contents; evaluated repositories retain their individual upstream licenses.Source: A Frozen Protocol and Artifact Notes · pp. 14–15