PaperlayerView exact revision on arXiv

Source-linked AI summary

CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents

Trang Nguyen, Eulrang Cho, Bingqing Chen, Tim Dettmers

arXiv:2609.26779v1cs.AIcs.LGcs.SE

TL;DR

Long-horizon coding agents face context limits and the high cost of processing long trajectories. CLIFFCOMPACTION keeps verbatim excerpts while discarding prior compactions, and bounded-context evaluations report preserved or improved benchmark performance at lower cost.

  • Problem

    Coding agents need to manage sessions spanning millions of tokens despite limited context windows and the computational expense and potential effectiveness loss of bloated contexts.

    Source: §1 · p. 2

  • Method

    CLIFFCOMPACTION is a rule-based autocompaction method that retains verbatim excerpts and compacts only the latest live session, discarding prior compacted history.

    Source: §1 · p. 2, §2.3 · p. 5

  • Results

    CLIFFCOMPACTION preserves or improves benchmark performance under bounded context; on Terminal-Bench 2.0, it reduces cost by 50% while achieving higher success rates.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Per-rollout savings make test-time scaling more economical, with three compacted Kimi K2.6 rollouts gaining 10.5 points at 1.9× the cost of a single uncompacted run.

    Source: §1 · p. 3

    Benefits depend on scaffold and task, and are meaningful only for medium-to-long-horizon tasks.

    Source: §8 · p. 12

Abstract

from arXiv · show

Agents often work on complex problems that require millions of tokens of context, which necessitates compacting across sessions due to limited context windows. We develop CliffCompaction, an autocompaction technique that reduces cost by up to 50% under a bounded context while maintaining or improving performance on Terminal-Bench and achieving new levels of efficiency for test-time scaling and state-of-the-art results on KernelBench. The per-rollout savings of CliffCompaction make the performance--cost trade-off of test-time scaling more efficient, adding over 10 percentage points on Terminal-Bench for less than the cost of two full-context runs. Under parallel test-time scaling, CliffCompaction lets Kimi K2.6 match Opus 4.7, and exceed Opus 4.6 and GPT-5.3 Codex at lower cost. The key to CliffCompaction's effectiveness is that it keeps compacted information faithful by only truncating or dropping content, never rephrasing or rewriting it. We never compact a compaction---each pass operates only on original content, and prior compacted output is discarded, preventing context drift from accumulating. These properties sustain continual learning over sessions exceeding a million tokens: on KernelBench, CliffCompaction reaches CUDA kernel speedups of $2.23\times$ after 200 steps and $3.58\times$ after 400 steps, surpassing specialized search algorithms and trained agents despite being a general-purpose compaction technique. We open-source a scaffold-agnostic API-proxy implementation of CliffCompaction usable with Claude Code, Codex and other harnesses.

1 INTRODUCTION

Long-horizon coding agents face costly, potentially bloated contexts, alongside expensive test-time scaling and continual-learning methods that may be task-specific. CLIFFCOMPACTION uses bounded context to preserve or improve performance at lower cost and support economical scaling.

1 INTRODUCTION

  • On Terminal-Bench 2.0, CLIFFCOMPACTION achieves higher success rates while reducing cost by 50%.On SWE-bench Verified, it preserves full-context success rates for GLM-5.1 and Kimi K2.6 at context thresholds of 32K and 16K tokens, respectively.Source: §1 · p. 2
  • Long-horizon agents must manage context when sessions reach their limits while improving performance without incurring the cost of processing ever-longer contexts.Growing contexts can also impair agent effectiveness.Source: §1 · p. 2
  • Continual-learning approaches often rely on complex, task-specific methods with limited generalizability.Source: §1 · p. 2
  • Test-time scaling can require tens of rollouts, and its marginal gains may not justify the expense as gains saturate and parallel scaling creates a selection bottleneck.Source: §1 · p. 2
  • CLIFFCOMPACTION maintains performance on coding and terminal tasks at lower cost, makes test-time scaling economical, and sets a new best KernelBench continual-learning result under matched evaluation.The approach is described as training-free, drop-in, model-agnostic, cache-friendly, and requiring no auxiliary LLM calls, while sacrificing full-history recall.Source: §1 · pp. 2–3, Table 1 · p. 2

2 CLIFFCOMPACTION

CLIFFCOMPACTION preserves verbatim excerpts while discarding earlier compactions, trading full-history recall for precision. It grows context between occasional compactions to limit cache disruption, selectively truncates or drops token-heavy content, and compacts only the latest live session at each pass.

2 CLIFFCOMPACTION

  • CLIFFCOMPACTION retains compact verbatim excerpts and recent turns while dropping token-intensive conversation content, then discards previous compactions at each new pass.It prioritizes compaction precision over full-history recall; summaries risk omission or distortion that can produce ongoing context drift.Source: §2 · p. 3

2.1 CLIFFCOMPACTION AND KV-CACHE

  • Context grows unchanged until a preset budget is exhausted, so the KV-cache remains valid between occasional compactions and each smaller re-prefill is amortized over cache reuse.Modifying context invalidates the cache; uncached-input API costs are roughly 5 to 6 times cached-input costs in the studied models.Source: §2.1 · p. 4, Figure 2 · p. 4

2.2 CLIFFCOMPACTION AND TOKENS

  • Tool results longer than 500 characters are dropped, shorter results are retained, tool calls are reduced to signatures, thoughts are truncated to 300 characters, and prompts and recent turns are kept.The preserved tool-call signatures let the agent reissue calls to recover removed information when needed.Source: §2.2 · pp. 4–5

2.3 CLIFF BY CLIFF

  • Each compaction uses only the latest live session and discards the previous compacted history, avoiding progressively lossier summary-of-summary compression.This keeps each compacted block faithful to the turns it condenses.Source: §2.3 · p. 5
  • Although earlier information is not carried forward directly, it can influence later compactions through the agent’s behavior and reasoning, softening recall loss across sessions.The paper reports this residual knowledge supports million-token sessions with a 128k-token compaction window.Source: §2.3 · p. 5

3 CLIFFCOMPACTION MAINTAINS PERFORMANCE AT LOWER COST

The evaluation tests CLIFFCOMPACTION across coding benchmarks, scaffolds, models, and context thresholds. At moderate thresholds it preserves or improves success, while more aggressive 8K limits can lower performance.

3.1 EXPERIMENTAL SETUP

  • The evaluation covers SWE-bench Verified with mini-swe-agent and OpenHands, and Terminal-Bench with Terminus-2 and Claude Code, across multiple models and context thresholds.Thresholds are 45K, 32K, 16K, and 8K; Claude Code and CLIFFCOMPACTION are matched at approximately 45K mean peak context.Source: §3.1 · p. 6

3.2 RESULTS

  • 61.42% on Terminal-Bench 2.0 for Kimi K2.6 at 32K and 16K exceeds the 59.16% full-context baseline by 2.26 percentage points.At 16K, replacing Terminus-2’s native summarization increases success from 55.45% to 61.42%; GLM 5.1 rises from 49.83% full-context to 54.33%.Source: §3.2 · pp. 6–7, Table 2 · p. 7
  • 76.69% on Terminal-Bench 2.1 exceeds Claude Code autocompaction at a matched ∼45K mean peak context (70.97%) and its default 200K configuration (73.03%).The result has markedly lower seed variance and indicates scaffold-agnostic deployment under these conditions.Source: Table 2 · p. 7, §3.2 · p. 7
  • On SWE-bench Verified, moderate thresholds largely preserve full-context performance, while the 8K setting produces larger drops and is best treated as a stress test.For Kimi K2.6, 32K changes resolution from 73.87% to 73.27%; at 16K it remains within 2.0 percentage points of full context.Source: Table 3 · p. 7, §3.2 · p. 8

4 MAKING TEST-TIME SCALING AFFORDABLE WITH CLIFFCOMPACTION

The section studies whether compaction can make multiple agent rollouts affordable, using a learned trajectory selector and benchmark comparisons. On Terminal-Bench, compacted rollouts improve the accuracy–cost trade-off, including results competitive with proprietary models at lower cost.

4.1 EXPERIMENTAL SETUP

  • The learned selector, SOFT GROUP VERIFICATION (SGV), chooses a trajectory using execution and solution features, including within-group agreement measures based on line overlap.Pass@1 is the mean resolution rate across individual runs without selection; Oracle (pass@k) is the fraction solved by at least one rollout.Source: §4.1 · p. 8

4.2 RESULTS

  • PACTION reaches 65.9% at $38.67 with a 1.3× cost multiplier, exceeding the uncompacted three-rollout baseline while costing 42% as much.It is cheaper than a single run of either GPT 5.3 Codex or Opus 4.6.Source: §4.2 · p. 8
  • At three rollouts, 16K CLIFFCOMPACTION reaches 69.7% for $58.01, matching Opus 4.7 (69.4%) and exceeding GPT 5.3 Codex (64.7%) and Opus 4.6 (62.9%).It gains 10.5 points at 1.9× cost versus a single Kimi K2.6 run, and is 5.7 points more accurate at 37% lower cost than the uncompacted three-rollout baseline.Source: §4.2 · p. 8, Table 4 · p. 9
  • For GLM 5.1, CLIFFCOMPACTION raises practical resolution from 55.1% to 60.7% while reducing scaled-inference cost by 40%, from $158.25 to $95.34.The reported gain narrows much of the gap to proprietary models.Source: §4.2 · p. 9
  • With a separate Gemini-2.5-Flash verifier ranking five rollouts, LLM-as-a-Verifier converts a 5.0× cost increase into +3.4 points, while CLIFFCOMPACTION with SGV yields +10.5 at 1.9× and +6.7 at 1.3×.GLM 5.1 shows the same pattern, with +10.9 points at 1.8× cost.Source: §4.2 · p. 9

5 CONTINUAL LEARNING WITH CLIFFCOMPACTION

The section evaluates whether CliffCompaction can sustain continual kernel optimization across trajectories exceeding one million tokens. On KernelBench Level 3, it supports progress beyond the full-context limit and improves speedup as the step budget grows.

5 CONTINUAL LEARNING WITH CLIFFCOMPACTION

  • KernelBench Level 3 trajectories can exceed one million tokens, or 15–70× the median SWE-bench or Terminal-Bench trajectory.The evaluation lets an OpenHands agent optimize GPU kernels until the step budget is exhausted or, without compaction, the model context window is reached.Source: §5 · p. 9, §5.1 · p. 9

5.1 EXPERIMENTAL SETUP

  • The experiments use Kimi K2.7 for CUDA and GPT-5-mini for Triton on L40S, plus CliffCompaction with Kimi K2.6 on RTX Pro 6000 Blackwell.Reported metrics include geometric-mean speedup against PyTorch Eager, accuracy, and the fraction of problems exceeding a speedup threshold.Source: §5.1 · pp. 9–10

5.2 RESULTS

  • At 200 steps, speedup reaches 2.23× on L40S and 1.87× on RTX Pro 6000; at 400 steps, it reaches 3.58× and 2.48×, respectively.Without compaction, 98% of L40S runs and 76% of RTX Pro 6000 runs terminate early, at median step counts of 99 and 122.Source: §5.2 · p. 10
  • On L40S, kernels exceeding 2× speedup rise from 50% without compaction to 86% at 400 steps, while both 200K and 128K thresholds outperform the uncompacted 256K baseline.On RTX Pro 6000, the fraction exceeding 2× doubles from 32% to 64%, and accuracy reaches 98%.Source: §5.2 · p. 10
  • At equal total cost, CliffCompaction reaches 2.02× speedup against 1.30× for the full-context run.The section frames this as greater sequential optimization per dollar with a smaller context window.Source: §5.2 · p. 11
  • At 200 steps, CliffCompaction reaches 2.09× against 1.78× for AdaExplore and 2.23× against 1.80× for CUDA-Agent.At 200 steps, it exceeds 1.2× speedup on 78% of problems and 2× on 52%; the comparison notes that action units differ across strategies.Source: §5.2 · p. 11

6 CLIFFCOMPACTION VS. EXISTING COMPACTION STRATEGIES

The comparison evaluates context-management strategies across short SWE-bench Verified and longer KernelBench Level 3 trajectories. CliffCompaction is the only compared method reported to pair competitive quality with low cost across all three benchmarks.

6 CLIFFCOMPACTION VS. EXISTING COMPACTION STRATEGIES

  • The comparison includes sliding windows, LLM summarization, and a two-tier summarization-plus-microcompaction pipeline.The two-tier baseline clears older tool observations while retaining tool calls and five recent observations, then summarizes older turns when needed.Source: §6 · p. 11
  • CliffCompaction is the only method reported to maintain competitive quality while remaining inexpensive across all three benchmarks.On KernelBench Level 3, it reaches 3.58× at $8.32 per problem, compared with 3.47× at $8.10 for summarization, 3.33× at $12.84 with microcompaction, and 2.86× at $21.52 for sliding window.Source: §6 · p. 11
  • On SWE-bench Verified, all four methods fall within 2.6 points; sliding window scores highest on both models but raises cost by 9% and 7%.Summarization has the weakest quality for both models.Source: §6 · p. 11, Table 6 · p. 11

7 RELATED WORK

The related work spans context management, test-time scaling, and continual learning for coding agents. The paper positions CliffCompaction at their intersection, studying how context compaction affects scaling with additional inference-time compute.

RELATED WORK

  • Context-management approaches include sliding windows, LLM summaries, trained compression, external memory or retrieval, subagents, and rule-based eviction, each with trade-offs in cache use, loss, transfer, infrastructure, orchestration, or scaffold dependence.Repeated summaries can distort or omit details across rounds; other approaches may require training, retrieval infrastructure, or scaffold-specific instrumentation.Source: §7 · p. 12
  • Prior code-agent test-time scaling combines parallel sampling, sequential refinement, selection, repository-aware exploration, branching, or compact trajectory summaries.The paper studies how an agent equipped with CliffCompaction responds to additional inference-time compute.Source: §7 · p. 12
  • Continual-learning approaches use iterative refinement, external experience memory, or evolutionary candidate programs, but many rely on task-specific training, memory, or optimization procedures.The related-work discussion identifies generalizability to new tasks as a constraint for many such methods.Source: §7 · p. 12

8 LIMITATIONS

The paper limits the scope of its claims by noting that compaction benefits depend on scaffold and task, and that its strategy comparison excludes other context-management directions. Its cost and runtime analyses also report variation across configurations and cache conditions.

8 LIMITATIONS

  • Compaction benefits depend on scaffold and task, and are meaningful only for medium-to-long-horizon tasks.Scaffolds consume different amounts of the context budget with fixed prompts and tool definitions, so their minimum thresholds vary.Source: §8 · p. 12
  • The comparison covers methods acting on conversation history at inference time, not trained context managers or external-memory stores.The paper says resource constraints prevented comparison with those approaches.Source: §8 · p. 12

A.1 COST SAVINGS WITH CLIFFCOMPACTION

  • Cost savings vary by model, scaffold, benchmark, and threshold; the largest reported Terminal-Bench reductions are 25.0%–52.1% for Kimi K2.6 and 35.8%–64.6% for GLM 5.1.At an 8K budget on SWE-bench Verified with mini-swe-agent, Kimi K2.6 is marginally more expensive than full context because re-prefilling offsets context savings.Source: §A.1 · pp. 16–17
  • Cache reads account for 78% of the uncompacted bill; compaction cuts cache-read cost from $0.427 to $0.087 per task while adding $0.06 in uncached-input and output costs.The passage reports that these changes halve total cost.Source: §A.1 · p. 17
  • Observed costs can differ from perfect-cache estimates: Kimi K2.6 estimates match real costs within about 1%, while GLM 5.1's lower cache-hit rates make real costs higher than estimates.Cache-hit rates decline as thresholds tighten, reflecting additional cache invalidation and re-prefilling.Source: §A.1 · p. 17

A.2 MEASURED RUNTIME

  • On SWE-bench Verified, a 32K context cap reduces per-instance wall-clock time by 23% for Kimi K2.6 and 34% for GLM 5.1; a self-hosted GLM-4.7-Flash setup reports speedups of 1.60×, 2.10×, and 2.62× at 32K, 16K, and 8K.The self-hosted setup uses vLLM on RTX PRO 6000 Blackwell.Source: §A.2 · pp. 17–18

A.3 CLIFFCOMPACTION VS. OTHER COMPACTION METHODS

CLIFFCOMPACTION keeps context unchanged between clean compaction resets, producing lower per-step costs than microcompaction with summarization or sliding windows. Its append-only growth and sharp drops contrast with the other methods’ context traces.

A.3 CLIFFCOMPACTION VS. OTHER COMPACTION METHODS

  • On a representative KernelBench task, CLIFFCOMPACTION grows context append-only and drops it at compaction, unlike sliding window’s ceiling or summarization’s sawtooth trace.Figure 5 shows context size over steps.Source: §A.3 · p. 18
  • Per-step costs are lowest for CLIFFCOMPACTION, intermediate for microcompaction + summarization, and highest for sliding window.Figure 4 decomposes the per-step input cost into uncached prefill and cache reads.Source: §A.3 · p. 18, Figure 4 · p. 19
  • Because it leaves earlier tokens untouched between compactions, CLIFFCOMPACTION serves context almost entirely from cache, paying uncached re-prefill only at rare resets.Microcompaction repeatedly invalidates the KV-cache; sliding windows re-prefill close to the full context at nearly every step.Source: §A.3 · pp. 18–19

B EXTENDED CLIFFCOMPACTION DESIGN AND IMPLEMENTATION

CLIFFCOMPACTION compacts by truncating or dropping original content rather than rewriting it. Its action signatures preserve only a short leading fragment, while OpenHands tool-call signatures are listed separately.

B EXTENDED CLIFFCOMPACTION DESIGN AND IMPLEMENTATION

  • CLIFFCOMPACTION retains literal fragments of the original context, truncating rather than rewriting or paraphrasing content.For each action, it keeps only the first 150 characters; the text says commands typically begin with useful information, while lengthy inline file contents often follow.Source: B EXTENDED CLIFFCOMPACTION DESIGN AND… · p. 19
  • In mini-swe-agent and Terminus-2, the signature is the leading 150 characters of each parsed bash command or terminal-keystroke string.For OpenHands, the tool-call signatures are listed in Table 12.Source: B EXTENDED CLIFFCOMPACTION DESIGN AND… · p. 19, Table 12 · p. 20
  • Figure 5 tracks context size over steps on a KernelBench task.The accompanying description characterizes CLIFFCOMPACTION’s context as append-only growth followed by a clean drop at compaction.Source: Figure 5 · p. 20

C.1 CLIFFCOMPACTION HAS HIGH PRECISION

CLIFFCOMPACTION’s high-precision trade-off is reflected in re-reading behavior, while compacted rollouts yield strong SWE-bench scaling results and KernelBench speedups when compared at equal kernel-proposal budgets.

C.1 CLIFFCOMPACTION HAS HIGH PRECISION

  • All compaction strategies add more than four steps except summarization (+1.26); CLIFFCOMPACTION adds +5.04 re-reads, versus summarization’s +2.35.The authors interpret the behavior as corroborating a low-recall, high-precision trade-off: retained information is verbatim, and gaps prompt restoration from the source.Source: §C.1 · p. 20, Figure 6 · p. 21

C.2 TEST-TIME SCALING ON SWE-BENCH VERIFIED

  • 79.4% Practical accuracy for Kimi K2.6 with SGV tops proprietary single-model baselines, including Opus 4.5 (76.8%), and exceeds LLM-as-a-Verifier (78.2%).Table 13 reports test-time scaling results on SWE-bench Verified, ordered by accuracy.Source: Table 13 · p. 21, §C.2 · p. 21
  • At 16K, three compacted Kimi rollouts reach 76.4% for $264.06, within 0.4 points of Opus 4.5 at 70% of its cost; five reach 77.0%, surpassing it.GLM 5.1 at 16K is the cheapest scaled configuration overall: 74.0% at $229.89.Source: §C.2 · p. 21

C.3 SPEEDUP BY KERNELS GENERATED ON KERNELBENCH

  • 1.47× versus AdaExplore’s 0.99× by the tenth kernel; CLIFFCOMPACTION reaches 1.60× in 13 kernels, a level AdaExplore takes 100 kernels to reach.Figure 7 indexes best-so-far speedup by kernels generated.Source: Figure 7 · p. 22, §C.3 · p. 22
  • At equal step budgets, CLIFFCOMPACTION generates fewer candidates, and indexing by proposals widens its advantage over compaction baselines.The authors report that it re-reads from the workspace and converts each proposal into more speedup; kernel proposals are counted as agent turns writing compiled kernel code.Source: §C.3 · p. 22

D EXPERIMENTAL DETAILS

The experimental details describe how the selector is trained and tested, then specify KernelBench agent configurations and the evaluation procedure for kernel speedups.

D.1 ADDITIONAL TEST-TIME SCALING DETAILS

  • For Practical results, SGV uses 30-feature vectors and separate LightGBM resolution classifiers per rollout budget; Kimi’s selector is trained on GLM 5.1 rollouts and vice versa.The same-model training comparison shows no consistent advantage.Source: §D.1 · p. 22, Table 16 · p. 24
  • Across training fractions from 25% to 100%, selector resolution rates remain essentially unchanged across all 89 Terminal-Bench 2.0 tasks.The reported data-efficiency result is for Kimi K2.6, GLM→Kimi, at k=3 with a fixed 30-feature set.Source: §D.1 · p. 22, Table 17 · p. 24

D.2 ADDITIONAL KERNELBENCH DETAILS

  • CUDA trials use Kimi K2.7/K2.6, L40S/RTX Pro 6000 Blackwell hardware, fixed step budgets, and no PyTorch compute fallback; Triton uses GPT-5-mini and permits PyTorch operators.The CUDA setup follows Dai et al. (2026) except for backbone, hardware, stopping condition, and context management.Source: §D.2 · p. 25
  • Each problem’s score is its best correct speedup, capped at 10×; failures receive 0.1, and reported Speedup is the geometric mean across 50 tasks.Kernels are checked against the reference on five random inputs before runtime is measured against PyTorch Eager.Source: §D.2 · p. 25

Related papers

Loading 2609.26779v1…