Source-linked AI summary
Active Context Compression: Autonomous Memory Management in LLM Agents
Nikhil Verma
TL;DR
Long-horizon software-engineering agents face rising context costs, latency, and distraction as interaction histories grow. Focus gives the agent autonomous control to consolidate learnings into persistent knowledge and prune raw history, achieving 22.7% token reduction while preserving benchmark accuracy on five evaluated tasks.
Problem
Growing interaction histories create escalating computational cost, latency, and distraction from irrelevant past errors, while existing solutions often rely on passive external summarization.
Method
Focus lets an agent autonomously consolidate learnings into a persistent Knowledge block and prune raw interaction history during a single task.
Results
22.7% token reduction (14.9M → 11.5M) matched Baseline accuracy at 3/5 = 60% for both agents on N=5 context-intensive instances.
Takeaways & Limitations
Focus provides a practical agent-controlled approach to context management, with exploration-heavy tasks showing 18% to 57% savings across 4/5 instances.
Takeaways & Limitations
The evaluation used only N=5 instances and Claude Haiku 4.5, so broader task and model generalization remains uncharacterized.
Abstract
from arXiv · showhide
Large Language Model (LLM) agents struggle with long-horizon software engineering tasks due to "Context Bloat." As interaction history grows, computational costs explode, latency increases, and reasoning capabilities degrade due to distraction by irrelevant past errors. Existing solutions often rely on passive, external summarization mechanisms that the agent cannot control. This paper proposes Focus, an agent-centric architecture inspired by the biological exploration strategies of Physarum polycephalum (slime mold). The Focus Agent autonomously decides when to consolidate key learnings into a persistent "Knowledge" block and actively withdraws (prunes) the raw interaction history. Using an optimized scaffold matching industry best practices (persistent bash + string-replacement editor), we evaluated Focus on N=5 context-intensive instances from SWE-bench Lite using Claude Haiku 4.5. With aggressive prompting that encourages frequent compression, Focus achieves 22.7% token reduction (14.9M -> 11.5M tokens) while maintaining identical accuracy (3/5 = 60% for both agents). Focus performed 6.0 autonomous compressions per task on average, with token savings up to 57% on individual instances. We demonstrate that capable models can autonomously self-regulate their context when given appropriate tools and prompting, opening pathways for cost-aware agentic systems without sacrificing task performance.
I. INTRODUCTION
Long-horizon agents face cost, latency, and distraction problems as interaction histories grow. Focus addresses this gap through agent-controlled intratrajectory compression that preserves learnings while pruning raw logs.
- Motivation: Growing context creates quadratic re-processing costs, linearly increasing latency, and distraction from failed trials and verbose tool outputs.These effects include the “Lost in the Middle” phenomenon.
- Related Work: Prior approaches manage context through external memory, episodic reflection, attention-level compression, or separate compression models.Examples include MemGPT, Voyager, Reflexion, LATS, StreamingLLM, and LLMLingua.
- Contribution: Focus enables intratrajectory compression by preserving structured learnings in a Knowledge block while actively pruning the agent’s raw history.The approach complements rather than replaces external memory methods.
- Motivation: Append-only histories can exhaust token limits or confuse agents during complex, open-ended exploration.The stated failure mode motivates retaining learned maps rather than complete exploration records.
- Contribution: Focus shifts agents from passive retention to active compression inspired by slime mold exploration, which retracts from dead ends while retaining markers.The agent summarizes a recent trajectory and deletes its raw logs from immediate context.
III. METHODOLOGY
Focus treats exploration as a process that should retain learned maps rather than every low-level interaction detail. This supports discarding obsolete exploration traces while preserving actionable knowledge.
- Design Motivation: Focus’s biological analogy emphasizes retaining a learned map instead of a complete record of movements through an environment.For codebase exploration, the useful result may be a file-location fact rather than raw directory listings.
B. Architecture: The Focus Loop
The Focus Loop lets the agent autonomously checkpoint exploration, consolidate what it learned, and withdraw obsolete messages into persistent knowledge. This creates a sawtooth context pattern rather than monotonically growing history.
- Control: The agent autonomously chooses when to invoke start_focus and complete_focus, without external timers or heuristics forcing compression.These tools add explicit control points to the standard ReAct loop.
- Explore: Start Focus declares the investigation and marks a checkpoint before the agent explores with standard read, edit, and run tools.The checkpoint defines the trajectory later eligible for consolidation.
- Consolidate: During consolidation, the agent summarizes what it attempted, learned, and achieved before withdrawing the raw trajectory.These fields capture actions, facts, file paths, bugs, and outcome.
- Withdraw: The system appends the summary to a persistent Knowledge block and deletes messages between the checkpoint and current step.Context therefore grows during exploration and collapses during consolidation.
IV. EXPERIMENTS
The evaluation compares Baseline and Focus on identical context-intensive SWE-bench Lite tasks using a controlled software-engineering scaffold. Success is verified through isolated Docker execution of the original test suites.
- Evaluation Setup: Focus was evaluated against Baseline on N=5 context-intensive SWE-bench Lite instances using claude-haiku-4-5-20251001.Both agents solved identical tasks in a controlled A/B comparison.
- Evaluation Setup: Task success was verified with the official SWE-bench Docker harness, which applies patches in isolated containers and runs the original test suites.This provides the task-success criterion for the comparison.
- Scaffold: The scaffold used persistent Bash so working directories and environments persisted across tool calls.This was selected to match ordinary developer terminal usage.
- Scaffold: A string-replace editor enabled targeted exact-string file edits instead of full-file rewrites.The system also encouraged extensive tool use and testing before attempting the problem, with a 150-step maximum.
B. Aggressive Compression Prompting
The study revised passive prompting into a directive compression workflow with reminders and structured phases, then tracked success, token use, compression, and efficiency.
- Passive prompting produced only 1–2 compressions per task and 6% token savings, motivating a more directive strategy.
- The revised workflow instructed the agent to start focus before exploration and complete focus after 10–15 tool calls.
- Periodic system reminders reinforced compression after 15 tool calls without a compression event.
- Structured guidance organized work into 4–6 phases: explore, understand, implement, and verify.
- Evaluation tracked task success, total token consumption, compression events, messages dropped, and per-instance efficiency patterns.
C. Results
Across five hard SWE-bench Lite instances, Focus reduced total token consumption while matching Baseline accuracy, with savings on most individual tasks and frequent compressions.
- The A/B comparison covered five hard SWE-bench Lite instances using Haiku 4.5.
- 22.7% total token reduction lowered consumption from 14.9M to 11.5M tokens while maintaining identical accuracy at 3/5 = 60% for both agents.
- Focus reduced tokens on 4 of 5 instances, with per-instance savings ranging from 18% to 57%.
- 6.0 compressions per task, compared with 2.0 under passive prompting, dropped 70.2 messages per task.
E. Case Study: Maximum Savings (matplotlib-26020)
On matplotlib-26020, Focus achieved maximum savings by compressing exploration context before implementation, while the broader sawtooth design periodically reduced accumulated history.
- 57% token savings reduced usage from 4.0M to 1.7M while both agents passed the test suite.
- Focus compressed five times across 71 LLM calls, whereas Baseline used 102 calls without compression.
- After locating relevant files and understanding the bug, Focus summarized exploration and proceeded directly to implementation.
- The sawtooth pattern represents context growth during exploration followed by periodic compressions that preserve learnings in a persistent Knowledge block.
G. Analysis
The analysis finds that compression timing and task structure determine whether Focus improves efficiency: frequent small compressions help exploration-heavy work, while scaffolding remains necessary.
- Frequent small compressions every 10–15 calls preserve recent context while discarding stale exploration logs; infrequent large compressions risk losing implementation details.
- The 22.7% savings required explicit compression instructions and system reminders; passive prompting produced only 6% savings.
V. DISCUSSION
Focus achieves net token savings despite a cognitive tax, but the evidence is limited by small, task-dependent, model- and scaffold-specific evaluation. The results also show that autonomous compression currently depends on aggressive prompting.
- Efficiency: 22.7% net token savings remained after accounting for summary-generation and focus-management overhead.Each compression costs a few hundred tokens but can save thousands by avoiding re-processing stale history.
- Evaluation limits: The evaluation covered only N=5 hard instances, so full SWE-bench Lite validation is needed to characterize which task types benefit most.The authors identify the full N=300 benchmark as a needed validation set.
- Evaluation limits: Compression benefits were task-dependent, ranging from 50-57% savings on exploration-heavy tasks to 110% overhead on one iterative-refinement task.Identifying task characteristics that predict compression benefit remains future work.
- Evaluation limits: Whether similar compression behavior generalizes beyond Claude Haiku 4.5 and the optimized two-tool scaffold remains unknown.The authors note that other models may require model-specific prompting and that different tool configurations may produce different patterns.
VI. CONCLUSION
The paper presents active, model-controlled context compression as a way to reduce token use without sacrificing measured task accuracy. Its conclusion emphasizes aggressive prompting, stronger benefits for exploration-heavy tasks, and several directions for broader validation and automation.
- Main findings: 22.7% token savings reduced total usage from 14.9M to 11.5M while matching Baseline accuracy at 3/5 = 60%.This result was obtained on five context-intensive SWE-bench instances.
- Main findings: Aggressive prompting was key: passive prompting yielded only 6% savings and accuracy degradation.Explicit compression instructions and reminders supported the stronger result.
- Main findings: Explicit instructions increased compressions from 2.0 to 6.0 per task and enabled the 22.7% savings.The prompting schedule directed compression every 10-15 tool calls and used system reminders.
- Scope of benefits: Exploration-heavy tasks benefited most, with savings of 18% to 57% on 4/5 instances, while one iterative-refinement task incurred compression overhead.The conclusion characterizes Focus as best suited to explore-then-implement workflows.
- Future work: Future work includes full-benchmark validation, learning compression heuristics without explicit prompting, structured artifact preservation, and cross-model evaluation.The proposed evaluations include SWE-bench N=300 and models such as GPT-4, Gemini, and open-source systems.