PaperlayerView exact revision on arXiv

Source-linked AI summary

SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning

Sanjay Kariyappa, G. Edward Suh

arXiv:2602.22603v2cs.AIcs.LG

TL;DR

Long-running agentic tasks stress KV-cache memory, and heuristic compression can prematurely remove information that later becomes important. SideQuest uses the LRM to identify stale tool responses through a parallel auxiliary reasoning thread, achieving up to 65% lower peak memory usage with only minor accuracy degradation. The method is evaluated with a small training set and currently applies only to tool responses.

  • Problem

    Agentic contexts grow rapidly, while fixed KV-cache heuristics can prune information needed by later reasoning steps.

  • Method

    SideQuest uses the LRM to reason about stale tool responses and evict them through an auxiliary thread running in parallel with main reasoning.

  • Results

    SideQuest reduces peak memory usage by up to 65% with only a minor accuracy drop and outperforms heuristic baselines on agentic tasks.

  • Takeaways & Limitations

    Model-driven, parallel cache management can adapt context size to evolving agentic tasks without materially disrupting primary reasoning performance.

  • Takeaways & Limitations

    The current model is trained on 215 traces and evicts only tool responses, not arbitrary tokens in the sequence.

Abstract

from arXiv · show

Long-running agentic tasks, such as deep research, require multi-hop reasoning over information distributed across multiple webpages and documents. In such tasks, the LLM context is dominated by tokens from external retrieval, causing memory usage to grow rapidly and limiting decode performance. While several KV cache compression techniques exist for long-context inputs, we find that existing heuristics fail to support multi-step reasoning models effectively. We address this challenge with SideQuest -- a novel approach that leverages the Large Reasoning Model (LRM) itself to perform KV cache compression by reasoning about the usefulness of tokens in its context. To prevent the tokens associated with this management process from polluting the model's memory, we frame KV cache compression as an auxiliary task executed in parallel to the main reasoning task. Our evaluations, using a model trained with just 215 samples, show that SideQuest reduces peak token usage by up to 65% on agentic tasks with minimal degradation in accuracy, outperforming heuristic-based KV cache compression techniques.

1. Introduction

Long-running agentic tasks create rapidly growing KV caches, while fixed heuristics can discard information that later becomes important. SideQuest uses the model’s reasoning to manage its cache through a parallel auxiliary process, reducing cache usage with limited accuracy loss.

  • Motivation: Agentic contexts can span hundreds of thousands of tokens, making the KV cache a primary scaling bottleneck for memory and decode bandwidth.The cache grows linearly with sequence length, reducing effective batch size and increasing memory traffic during token generation.
  • Limitations of Existing Methods: Existing methods retain tokens using fixed importance heuristics, such as attention-based heavy hitters or local information clusters.These methods assume that past token importance predicts future relevance.
  • Limitations of Existing Methods: In multi-step agentic reasoning, heuristic pruning can delete information that appears unimportant early but becomes critical during later synthesis.This dynamic, non-monotonic token utility can cause difficult-to-debug reasoning failures.
  • SideQuest: SideQuest shifts KV cache management from fixed heuristics to a model-driven process that selectively clears segments of its own cache.Compression runs as an auxiliary task in parallel with main reasoning so management tokens do not pollute the primary attention window.
  • Results: 60% reduction in KV cache usage is achieved on complex agentic benchmarks with negligible accuracy degradation.The reported result establishes an efficiency-utility trade-off against heuristic-based techniques.

2. Related Work

Prior KV-cache work reduces memory pressure through architectural, sparse-attention, or eviction techniques, but many eviction methods rely on static token importance. The paper identifies heuristic rigidity as a central problem for evolving multi-step agentic contexts.

  • Existing Efficiency Approaches: Architectural methods reduce per-token KV-state cost, while sparse attention reduces memory bandwidth by accessing only selected tokens.Sparse attention generally leaves the full KV context physically resident in memory.
  • KV Cache Eviction: StreamingLLM, H2O, and SnapKV prune or retain entries using static attention-based heuristics and local information clusters.These methods bound memory by removing entries judged unimportant according to fixed signals.
  • Reasoning-Oriented Methods: Reasoning-focused methods such as R-KV, RaaS, LazyEviction, and ThinKV target token redundancy, milestones, recurrent importance, or hierarchical importance levels.The passage characterizes these methods as primarily addressing single-step Chain-of-Thought tasks with static contexts.
  • Open Limitation: Heuristic rigidity is a fundamental limitation in agentic settings because token importance can change across turns as goals and state evolve.A token with low attention at turn t may become a critical pivot at turn t+n, risking irreversible pruning.

3. SideQuest

SideQuest manages evolving ReAct contexts by identifying stale tool responses with an auxiliary reasoning thread that shares context with the main task. Its training data uses hindsight annotations and joint objectives to teach eviction while preserving primary reasoning behavior.

  • Task Setting: SideQuest targets ReAct tasks in which reasoning traces and tool calls progressively append external results to the context.The objective is to identify stale tool calls and responses and evict their tokens early.
  • Challenge: Tool-response utility is non-monotonic: an initially obsolete search result may be deleted, while another response can regain importance when sources must be cited later.This illustrates why attention-based proxies may not track future usefulness reliably.
  • Core Mechanism: SideQuest analyzes the current ReAct state and problem definition to determine which tool responses are no longer needed.It replaces attention-score proxies with the LRM’s semantic reasoning about task state.
  • Parallel Architecture: At regular intervals, an auxiliary thread analyzes open tool outputs in parallel with the main thread, produces deletion decisions, and clears marked responses from the cache.The shared-context design avoids interleaving management steps into the main task and limits management-token pollution.
  • Auxiliary Control: A trigger phrase and task-specific fine-tuning switch the auxiliary thread into memory-management mode and teach it to emit deletion commands.The trigger separates context management from answering the user’s query.
  • Training Data: Training combines correct main traces with hindsight-generated auxiliary traces, using distillation for the main task and cross-entropy for eviction behavior.Correct traces are annotated by each cursor’s last use, while expired cursors are synthesized into auxiliary supervision; expiration assumes no future reference by tools or the final answer.
  • Training Objective: Joint optimization combines auxiliary cross-entropy and main-trace distillation so the model enters auxiliary mode when triggered while retaining base reasoning behavior.The objective is L = LCE(Daux) + λLdistill(Dmain).

4. Experiments

The experiments evaluate SideQuest on long-context, multi-turn web-browsing tasks using accuracy, memory, reliability, and serving-efficiency metrics. SideQuest substantially reduces memory use while preserving reasoning quality and improving production serving performance.

  • 4.1. Datasets: The evaluation uses FRAMES and BrowseComp, two long-context, multi-turn web-browsing benchmarks requiring synthesis across multiple retrieval steps.FRAMES reports 424 samples, while BrowseComp reports 500 samples from a fixed 100k-document corpus.
  • 4.4. Baselines: SideQuest is compared with an uncompressed full-attention baseline and H2O, SnapKV, and R-KV at 16k and 24k token budgets.The experiments implement the methods in SGLang with prefix caching for context reuse across ReAct iterations.
  • 4.6. Results: 56−65% lower Peak Token Utilization and 53−71% fewer KV cache memory reads come with accuracy degradation of up to 2% on FRAMES and 5% on BrowseComp.At comparable compression levels, heuristic baselines suffer precipitous accuracy drops.
  • 4.6. Results: Task token counts range from a few thousand to over 120k, making fixed-budget compression fail on complex queries while wasting memory on simple ones.SideQuest adaptively evicts cursors that are no longer semantically relevant and selects a context size based on instantaneous task difficulty.
  • 4.6. Results: SideQuest maintains a non-completion rate comparable to the uncompressed baseline, whereas heuristic methods show high rates of unparsable responses and other failures.Some baselines’ lower peak memory usage is partly attributed to early crashes rather than successful completion.

5. Limitations

The current implementation has two principal limitations: modest accuracy degradation, especially on out-of-distribution BrowseComp, and eviction restricted to tool responses rather than the model’s intermediate reasoning.

  • 5. Limitations: SideQuest shows minor performance degradation, especially on out-of-distribution BrowseComp, and the model was fine-tuned on only 215 traces.The authors hypothesize that scaling training data to a larger, more diverse distribution could reduce the remaining gap.
  • 5. Limitations: The eviction strategy is limited to tool responses and does not yet compress the agent’s intermediate reasoning steps.The authors identify thought pruning and combining SideQuest with attention-weight-based methods as future directions.

6. Future Work

Future work extends SideQuest beyond memory management to new domains and auxiliary tasks, including coding, safety, security, and governance.

  • New Domains for Memory Management: SideQuest’s principles are domain-agnostic, with coding agents identified as a promising future application for managing massive codebases and dependency graphs.The proposed use case selectively forgets irrelevant file contents while retaining critical function definitions.
  • SideQuest for Other Auxiliary Tasks: The shared-context parallel architecture can steer LRMs toward auxiliary governance and safety tasks alongside the main user interaction.These checks could use the primary LRM’s multi-turn context awareness instead of relying only on separate smaller guardrail models.
  • SideQuest for Other Auxiliary Tasks: Figure 6 illustrates that SideQuest can support auxiliary tasks beyond memory management.

7. Conclusion

The paper presents SideQuest as a model-driven alternative to static KV-cache compression for long-running agentic reasoning. It uses a parallel auxiliary thread to manage memory without interfering with the main reasoning process.

  • SideQuest addresses the failure of static heuristic compression to capture dynamic token utility in long-running agentic tasks.
  • SideQuest enables Large Reasoning Models to manage their own memory through a parallel auxiliary thread that performs semantic-aware eviction of stale tool outputs.
  • The architecture prevents memory-management tokens from interfering with the primary reasoning process.
  • SideQuest reduces peak memory usage by up to 65% with only a minor accuracy drop and strictly outperforms heuristic baselines.
  • The method removes the need for manual token budgeting by adaptively scaling context size to the query’s instantaneous complexity.

B. Examples of Memory Management Reasoning

The paper provides examples of SideQuest’s auxiliary memory-management reasoning to illustrate how the system decides which context information to retain or remove.

  • These examples demonstrate the sophisticated reasoning used for KV-cache management.
  • Figure 7 presents examples of memory-management reasoning generated by SideQuest.

C.1. Operating System and Retrieval-Based Approaches

Operating-system-inspired and retrieval-based approaches manage what information reaches the model, whereas SideQuest targets the inference-efficiency cost of information already loaded into the KV cache.

  • MemGPT uses hierarchical memory to move text between an active prompt and external storage through function calls, while Cursor uses Dynamic Context Discovery for software-engineering retrieval.
  • MemGPT and Cursor address information retrieval but do not address the linear GPU-memory growth caused by loaded context.
  • SideQuest complements these methods by operating on internal state and garbage-collecting intermediate-reasoning tensors without breaking task continuity.

C.2. Hierarchical and Decomposition-Based Context Management

Hierarchical and recursive methods isolate intermediate reasoning into transient contexts, but local sub-tasks can still accumulate large tool-output and retrieval contexts. SideQuest complements these methods by evicting stale information within each active branch.

  • Hierarchical and Decomposition-Based Context Management: Recursive Language Models and Context Fold restructure reasoning into separate contexts or branches, propagating final results to the main thread.RLMs use externalized recursive inspection, while Context Fold uses dynamic branching and folding.
  • Hierarchical and Decomposition-Based Context Management: These approaches do not eliminate linear context growth within an individual sub-task.A complex sub-problem can still accumulate substantial tool outputs and retrieval artifacts before producing its result.
  • Hierarchical and Decomposition-Based Context Management: SideQuest is complementary because it keeps each branch’s active stream lean by evicting stale tool outputs while preserving necessary ones.It can operate inside RLM or Context Fold sub-tasks, improving the efficiency of individual workers.
Loading 2602.22603v2…