Self-Compacting Language Model Agents

1 Introduction

• Growing traces accumulate stale content that anchors subsequent reasoning [Laban et al., 2026], producing context rot [Hong et al., 2025, Cheng et al., 2026].The motivation spans lengthy competition-math deliberation and agentic trajectories containing search results, execution outputs, and intermediate plans.Source: §1 · p. 1
• Fixed token thresholds ignore reasoning state, so poorly timed compaction can discard partial results needed to continue rather than merely removing stale work.Existing approaches trigger compaction at token thresholds [Cursor Research et al., 2026] or leave users to invoke /compact [Anthropic, 2025]. The appropriate timing instead depends on whether reasoning is ongoing or a subproblem has resolved.Source: §1 · pp. 1–2
• SelfCompact pairs an inline compaction tool with a rubric that encourages compaction after resolved subtasks or convergence, but suppresses it mid-derivation or when stuck.The model emits <summarize>, the scaffold summarizes, and generation resumes from the summarized prefix; the same model judges timing and summarizes without external verification or fine-tuning. The authors claim the first training-free introduction and evaluation of rubric-based adaptive compaction timing.Source: §1 · pp. 2–3
• Up to 18.1 points on competition math and 5–9 points on agentic search are gained over no compaction; SelfCompact matches or exceeds fixed-interval summarization.Evaluation covers seven open-weight models across competition math and agentic search benchmarks. Agentic search gains accompany 30–70% lower per-question cost.Source: §1 · p. 3
• Ablations show that the tool alone produces uneven behavior: some models compact at unhelpful moments, while others never invoke it; rubric guidance enables effective adaptive compaction.Addressing whether agents can recognize context rot without training, the findings suggest compaction timing can be supplied as an inference-time meta-cognitive capability rather than learned in model weights.Source: §1 · pp. 2–3

Context management in long-horizon agents

• Long-horizon agents accumulate thoughts, tool calls, and observations, increasing cost and degrading performance through context rot.Compaction is therefore presented as a prerequisite for sustained long-horizon capability.Source: §2 · p. 3

Current strategies

• Reactive compaction triggers near the token budget [Anthropic, 2025], allowing stale or erroneous context to pollute generation before compaction occurs.This treats compaction as overflow prevention rather than responding to the trajectory’s state.Source: §2 · p. 3
• Periodic compaction fires every k turns or tokens [Cursor Research et al., 2026, Wu et al., 2026a], potentially erasing needed information mid-subgoal.The illustrated BrowseComp trajectory loses four already-verified facts after a fixed-interval summary interrupts reasoning, leaving the model to guess.Source: §2 · p. 3

Setup

• SelfCompact condenses a possibly partial continuation while retaining the original prompt, then resumes generation from the resulting context.The summarization tool receives both the original prompt and the continuation generated so far.Source: §3 · p. 3
• The same language model performs summarization, requiring no external verifier or auxiliary model.Source: §3 · p. 3

Algorithm 1

• The rubric-gated inference-time compaction algorithm takes a probe interval, step budget, and rubric prompt as inputs.Source: §3 · p. 4

SelfCompact

• SelfCompact combines a model-exposed summarization tool with a decision rubric, running on a single inference engine without fine-tuning.The tool alone produces uneven, often poorly timed use across models. The rubric translates resolved subtasks and avoiding mid-derivation interruption into conditions the model can verify.Source: §16 · p. 4

Summarizer design

• The summarizer appends a user-role instruction to preserve the trajectory's KV cache, then replaces the accumulated trajectory with its generated summary.The summarization call prefills only the appended instruction; subsequent decoding resumes from the original question and summary.Source: §16 · p. 4

Triggering the summarizer / Workflow

• Periodic rubric probes choose COMPRESS or CONTINUE, distinguishing SelfCompact from unconditional fixed-interval compaction and an untruncated no-compaction baseline.Intervals count reasoning tokens for math or tool calls for search; task-specific rubric conditions require verbatim trajectory evidence. COMPRESS triggers summarization, while CONTINUE leaves the trajectory unchanged.Source: §16 · pp. 4–5

Cost analysis

• 5–9 accuracy points and 30–70% lower per-question token cost are SelfCompact's gains over no compaction on agentic search.On competition math, SelfCompact leads on 11 of 12 benchmark/model cells under a token budget matched to fixed-interval summarization.Source: §16 · p. 5
• O(Lℓ) summary generation and O(ℓ^2) new-prefix prefill constitute summarizer overhead; cache reuse avoids naive O(L^2) trajectory re-prefill.L and ℓ denote trajectory and summary token lengths; each probe interval adds a rubric call and, only on COMPRESS, a summarizer call. The cached-versus-prefill break-even is L/ℓ > 10, while the search summarizer reaches 20–80× compression.Source: §16 · p. 5

4.1 Multi-turn Reasoning in Competition Math

• 11 of 12 matched-token math settings favor SelfCompact over both the baseline and fixed-interval summarization.For Qwen3.5-9B, baseline improvements are 16.4 points on IMO-Answerbench, 10.0 on HMMT Nov, and 18.1 on HMMT Feb. The exception is Qwen3-30B-A3B on HMMT Feb, where fixed-interval summarization leads by 1.1 points.Source: §4.1 · p. 5, Table 1 · p. 6
• 52.9% oracle accuracy on IMO-Answerbench for Qwen3-4B-Instruct-2507 exceeds fixed-interval summarization by 11.5 points and the baseline by 14.0 points.Fixed-interval summarization degrades 40.4% of answer transitions despite producing a net gain. The oracle skips summarization when the current answer is correct but cannot change scheduled timing, indicating further adaptive-policy headroom.Source: §4.1 · pp. 5–6, Table 3

4.2 Agentic Search

• 8.5, 9.2, and 5.3 percentage-point BrowseComp-Plus gains over no compaction make SelfCompact strongest across GLM-4.7-Flash, MiniMax-M2.5, and Mimo-V2-Flash, respectively.Across all three models, accuracy follows no compaction < fixed-interval ≤ SelfCompact. Autonomous triggering adds up to 6.3 points over fixed-interval summarization.Source: Table 4 · p. 7, §4.2 · p. 7
• SelfCompact reduces BrowseComp-Plus per-question costs versus the baseline by 67% for GLM-4.7-Flash, 63% for MiniMax-M2.5, and 33% for Mimo-V2-Flash, using an approximation billing all prompt tokens at cached rates.Per-question costs fall from $0.12 to $0.04, $0.19 to $0.07, and $0.24 to $0.16, respectively.Source: §4.2 · pp. 6–7
• SelfCompact’s BrowseComp-Plus summaries fire earlier than the fixed 30%-of-context threshold across all three models.Trigger distributions skew below the threshold rather than concentrating at it. The authors interpret this as fixed intervals retaining stale context after a sub-question has already been resolved.Source: §4.2 · p. 7, Figure 2 · p. 8
• 5–20 percentage-point gains over the fixed-budget Threshold baseline appear in the two hardest BrowseComp-Plus difficulty bins across all three models.Difficulty is approximated by no-compaction output-token consumption, divided into five equal-size bins per model. On easy bins, differences among the three policies remain within sampling noise.Source: §4.2 · p. 8, Figure 3 · p. 8
• 5.4 points of average GLM-4.7-Flash accuracy disappear without rubric gating, dropping SelfCompact from 46.4% to 41.0%.Without rubrics, performance approaches fixed intervals (41.5%) and trails them on BrowseComp (33.6% versus 35.4%); Qwen3-4B-Instruct-2507 math accuracy similarly falls from 45.5% to 40.9%. The rubric protects information that later reasoning steps require.Source: §4.2 · pp. 8–9, Table 5 · p. 9

Context compaction in frontier models

• Frontier API models typically trigger automatic compaction at context-length thresholds, risking information loss when compression occurs at inappropriate positions.A concurrent LangChain feature introduced a summary tool in March 2026, but not rubric-based criteria for deciding when to summarize.Source: §5 · p. 9

Learning to compact during post-training

• SelfCompact elicits adaptive compaction from off-the-shelf tool-using models through a lightweight rubric, without parameter updates.Most prior methods train compaction through SFT, RL, or both, encoding timing implicitly in demonstrations or rewards [Kontonis et al., 2026, Yang et al., 2026, Yan et al., 2026, Cassano and Rush, 2026, Mitra and Ulukus, 2025, Aghajohari et al., 2025, Wu et al., 2026a, Sun et al., 2025, Ye et al., 2025, Shao et al., 2025, Wu et al., 2026b].Source: §5 · p. 9
• SelfCompact’s rubric fires after sub-task resolution or during convergence, and suppresses compaction mid-derivation or when the model is stuck.Fixed token-count triggers ignore trajectory structure and risk discarding context still needed for a derivation [Yan et al., 2026, Aghajohari et al., 2025, Wu et al., 2026a,b].Source: §5 · p. 9
• Zhu et al. [2026a] provide the closest training-free approach, using an external frontier summarizer after full search trajectories rather than dynamically selecting compaction times.SelfCompact replaces these fixed post-trajectory intervals with rubric-based timing.Source: §5 · pp. 9–10

KV Cache Eviction

• KV-cache eviction prioritizes attention efficiency over interpretability, whereas context compaction preserves inspectable, overridable natural-language summaries.Static eviction uses recency [Xiao et al., 2023] or attention scores [Li et al., 2024]; newer approaches often learn richer importance measures [Park et al., 2025, Eyuboglu et al., 2025, Zweiger et al., 2026, Monea et al., 2025, Li et al., 2026]. Some approaches also employ black-box weight updates.Source: §5 · p. 10

Model capability gap

• Only open-weight models were evaluated; frontier systems may detect context rot without a rubric.Frontier systems report stronger metacognition [Lindsey, 2025]. SelfCompact is positioned as a complementary, training-free scaffold that layers onto any model where the capability gap exists.Source: §6 · p. 10

No reinforcement learning

• Training-free interventions isolate the rubric’s contribution, leaving reinforcement learning outside the evaluation scope.Prior work [Kontonis et al., 2026, Zhu et al., 2026b] shows that reinforcement learning can teach models both when and what to compact.Source: §6 · p. 10
• The rubric could supply a behavioral target for reinforcement learning to distill into the model’s policy.The authors propose this as a natural extension rather than an evaluated component.Source: §6 · p. 10

• 30–70% lower cost is achieved by SELFCOMPACT while matching or exceeding fixed-interval summarization across six benchmarks and seven models.Source: §7 · p. 10
• SELFCOMPACT gates trajectory summarization with a rubric, compacting at closed reasoning units rather than fixed intervals.Source: §7 · p. 10
• A lightweight rubric supplies the capability to decide when to compact without training.Source: §7 · p. 10

A Math experimental setup

• 4 open-weight Qwen models cover instruction-tuned Qwen3-4B-Instruct-2507 and Qwen3-30B-A3B-Instruct-2507, plus thinking-disabled Qwen3.5-4B and Qwen3.5-9B.Hugging Face checkpoints are served through vLLM [Kwon et al., 2023]. The 30B-A3B model uses tensor-parallel size 2; each other model runs on one H200 GPU.Source: A Math experimental setup · p. 16
• 3 benchmarks assess competition math: IMO-Answerbench [Luong et al., 2025], HMMT November 2025, and HMMT February 2026.Final answers are extracted and scored using math_verify.Source: A Math experimental setup · p. 16
• 16 samples per question use temperature 1.0 and top-p 0.7, with results reported as mean ± std over samples.Generation rounds are capped at 16,384 tokens, with a per-trajectory budget of 12 rounds. Summarizer output is hard-truncated at 512 tokens.Source: A Math experimental setup · p. 16
• The experiment compares BASELINE, FIXED-INTERVAL SUMMARY, SELFCOMPACT, and SELFCOMPACT W/ RUBRICS, differing in whether and when summarization occurs.BASELINE uses one 16k-token round without summarization; FIXED-INTERVAL SUMMARY summarizes every 16,384 generated tokens. SELFCOMPACT triggers through a model-emitted <tool_call> stop token, while the rubric variant summarizes only on COMPRESS and otherwise retains the unmodified prefix.Source: A Math experimental setup · p. 16
• The rubric compresses after a specific final answer, or when two rounds without new mathematical facts coincide with an exact next step.Each judgment requires a short evidence phrase; unsupported judgments must be N. ANSWER requires a specific final answer, STUCK assesses the last two rounds, and HAS-NEXT requires an explicit next action.Source: A Math experimental setup · p. 16
• Compression preserves final answers or named next steps, retains critical mathematical progress, removes redundant reasoning, and flags answers requiring verification.The final-answer branch locks in the answer; the stuck-with-next-step branch resets while preserving that step. Summaries of unsolved attempts identify what remains to be done.Source: A Math experimental setup · pp. 16–17

### INSTRUCTIONS

• The ReasoningCache scaffold of Wu et al. [2026a] prompts solving from scratch without a summary and improving the previous attempt when one is available.Previous attempts may be incorrect; suggested improvements include verification, alternative proofs or strategies, and continuation of incomplete solutions. Instructions request step-by-step reasoning and a boxed final answer.Source: A Math experimental setup · pp. 16–17, ### INSTRUCTIONS · p. 17

Models and serving

• GLM-4.7-Flash, MiniMax-M2.5, and Mimo-V2-Flash run through OpenRouter and AggAgent [Lee et al., 2026], using the unmodified react_agent_selfcheck scaffold.The rubric and summarizer prompts are reproduced verbatim.Source: B Agentic-search experimental setup · p. 18

Benchmarks

• 150 questions per benchmark are sampled from BrowseComp, BrowseComp-Plus [Chen et al., 2025], and DeepSearchQA.This subsampling follows Sun et al. [2025] and Lee et al. [2026].Source: B Agentic-search experimental setup · p. 18

Sampling parameters

• 1.0 temperature, 0.95 top-p, 10,000 maximum output tokens per call, and 100 LLM calls per trajectory apply across models; parallel tool calls are disabled.Fixed-interval summarization triggers at 30% of context. GLM-4.7-Flash keeps its thinking trace visible to the rubric probe using enable_thinking=True and clear_thinking=False.Source: B Agentic-search experimental setup · p. 18, Table 7 · p. 18

Scaffold details

• 40,000 prompt tokens, iteration ≥3, fewer than one prior summary, and ≥2 rounds since the last probe gate COMPRESS; a 30%-of-context backstop forces compression.The rubric is appended to a trajectory copy at round boundaries, avoiding pollution of the rolling cache.Source: B Agentic-search experimental setup · p. 18
• The ReAct loop alternates reasoning and search or retrieval calls until a final answer, with benchmark-specific retrieval backends.BrowseComp and DeepSearchQA use Serper search and crawl4ai page extraction through visit. BrowseComp-Plus uses search and get_document_bcp against its released corpus.Source: B Agentic-search experimental setup · p. 18

Search rubric probe

• 3–5 citeable essential facts, a closed unit, concrete progress, and no search stagnation are required for rubric-approved compression.Progress means a new fact or refined sub-question; affirmative answers require verbatim trajectory evidence, and compression must preserve dispersed information needed for continuation. Stagnation means at least three of the last four searches produced no new URL or fact; fewer than four searches yields N.Source: B Agentic-search experimental setup · p. 19

Summarizer

• The unmodified webresummer summarizer of Wu et al. [2026b] retains only explicit, relevant, certain information, excluding guesses and unconfirmed content.Compression irreversibly replaces history; continuation retains only the system message, original question, assistant summary, and user continuation.Source: B Agentic-search experimental setup · pp. 19–20

C Cost analysis of summarization

• 5–8% of cumulative prompt tokens are first-time prefills, supporting the single-rate approximation that charges all prompt tokens at the cache rate.For OpenRouter-hosted agents, the estimate pcache · Nprompt + pout · Nout accumulates prompt and completion usage across assistant turns, rubric probes, and summarizer calls.Source: C Cost analysis of summarization · p. 21
• 150 sampled questions per benchmark underpin mean USD-per-question costs, computed from AggAgent’s per-call usage records and each model’s Table 8 prices.Table 8 lists OpenRouter prices in USD per 1,000,000 tokens.Source: C Cost analysis of summarization · p. 21, Table 8 · p. 21
• 20–80× post-compaction prompt shrinkage comes from reducing 50–100k-token trajectories to approximately 1–3k-token summaries, lowering every subsequent call’s cached-input cost.The rubric probe preserves the live prefix and adds approximately 60 output tokens; the summarizer emits its summary once. These overheads are negligible relative to amortized future-call savings.Source: C Cost analysis of summarization · p. 21
• ±3k tokens is the budget-matching tolerance between fixed-interval summarization and SelfCompact for locally served Qwen math runs.These runs use wall-clock rather than dollar comparisons; [Xk] budgets average generated tokens plus refinement tokens across n = 16 samples. Table 10 separates refinement-summary prompts from generated output, with Total = Prompt + Output.Source: C Cost analysis of summarization · p. 21, Table 10 · p. 23

D Trajectory token consumption

• Each generated token incurs one output charge, one uncached-prefill charge on the next call, and repeated cached-input charges until its prefix resets.An assistant turn at round t contributes R − t cached-input charges in an R-round trajectory. A summary follows the same lifecycle after rubric-triggered compaction.Source: D Trajectory token consumption · p. 22
• Two-rate billing modestly increases estimated costs but preserves SelfCompact’s lower cost than the no-compaction baseline.Table 9 separates cumulative prompt tokens into first-time prefill and cache reads, calculating Cost as (pin Nprefill + pcache Ncached + pout Nout)/10^6.Source: D Trajectory token consumption · p. 22, Table 9 · p. 23

E Qualitative trajectories

• 3 BrowseComp-Plus cases on MiniMax-M2.5 were answered correctly by SelfCompact, while fixed-interval compaction and the no-compaction baseline answered all three incorrectly.These qualitative cases distinguish when compaction fires, rather than simply whether it occurs.Source: E Qualitative trajectories · p. 24
• 30% of the context window triggers fixed-interval summaries, whereas SelfCompact periodically checks its rubric and can exceed that threshold while exploration remains unresolved.Approximately 60k prompt tokens triggers fixed-interval compaction on MiniMax-M2.5, including summaries of dead-end searches; SelfCompact waits until a sub-task genuinely resolves.Source: E Qualitative trajectories · p. 24

Case A: Whitesnake — fixed-interval stays anchored, SELFCOMPACT breaks out

• SelfCompact’s constraint-focused summary enabled a fresh David Coverdale search and the correct Whitesnake answer, while fixed-interval summaries repeatedly restored a dead-end shortlist.118.6k → 1.1k tokens at turn 41 replaced the accumulated search history with constraints; fixed-interval compaction occurred four times before an incorrect Wings answer.Source: E Qualitative trajectories · p. 24

Case B: Majida El Roumi — fixed-interval freezes a wrong lead

• SelfCompact identified Majida El Roumi after correcting the classical-piece lead to Albinoni’s Adagio; fixed-interval summaries repeatedly preserved the wrong Rachmaninoff lead.70.2k → 0.9k tokens at turn 24 consolidated the corrected lead; fixed-interval compaction carried the mistaken “All by Myself” hypothesis through four summaries.Source: E Qualitative trajectories · p. 25

Case C: Raheem Sterling — fixed-interval never reaches the match

• SelfCompact kept exploring until identifying Tottenham–Chelsea, then compacted and verified Raheem Sterling’s assist; fixed-interval summaries recycled unsuccessful searches and never reached the match.70k → 0.8k tokens at turn 28 consolidated the identified match; subsequent searches confirmed attendance, the opening goal, and Sterling’s assist.Source: E Qualitative trajectories · p. 25