PaperlayerView exact revision on arXiv

Source-linked AI summary

A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression

Jincheng Ren, Siwei Wu, Yizhi Li, Kang Zhu, Shu Xu, Boyu Feng, Ruibin Yuan, Wei Zhang, Riza Batista-Navarro, Jian Yang, Chenghua Lin

arXiv:2604.19572v3cs.CL

TL;DR

Long-horizon terminal agents accumulate redundant observations while still needing exact task-critical signals, making fixed or naive compression unreliable across heterogeneous workflows. TACO uses self-evolving, preservation-aware rules discovered from interaction trajectories and reused across tasks. Across TerminalBench and additional terminal-related benchmarks, it improves or maintains task success while reducing token consumption, including 1%–4% accuracy gains on TerminalBench and 12%–27% lower token use elsewhere.

  • Problem

    Terminal-agent histories contain redundant observations alongside exact feedback needed for later actions, while generic summarization and static heuristics struggle to preserve signals across heterogeneous tasks.

  • Method

    TACO is a plug-and-play, unsupervised test-time adaptation framework that discovers, refines, and reuses preservation-aware compression rules from interaction trajectories.

  • Results

    TACO maintains or improves task success while reducing total token consumption by 12%–27%, and yields 1%–4% absolute accuracy gains across strong models on TerminalBench.

  • Takeaways & Limitations

    Self-evolving, workflow-adaptive observation compression improves task performance and token efficiency across terminal-agent scaffolds, backbone models, and benchmarks.

  • Takeaways & Limitations

    TACO requires multiple cold-start iterations before rule convergence, although the converged Global Rule Pool can then be frozen and reused.

Abstract

from arXiv · show

As terminal agents scale to long-horizon, multi-turn workflows, a key bottleneck is not merely limited context length, but the accumulation of noisy terminal observations in the interaction history. Retaining raw observations preserves useful environment feedback, but also leads to context saturation and high token cost; conversely, naive compression may discard task-critical signals needed for subsequent actions. Because terminal environments are highly heterogeneous across repositories, commands, and execution states, heuristic-based or fixed-prompt compression methods are difficult to generalize. We propose TACO, a plug-and-play, training-free, self-evolving Terminal Agent Compression framework for existing terminal agents. TACO automatically discovers, refines, and reuses structured compression rules from interaction trajectories, enabling workflow-adaptive filtering of low-value terminal outputs while preserving task-relevant observations. Experiments on TerminalBench (TB 1.0 and TB 2.0) and four additional terminal-related benchmarks, including SWE-Bench Lite, CompileBench, DevEval, and CRUST-Bench, show that TACO consistently improves task performance and token efficiency across agent scaffolds and backbone models. On TerminalBench, TACO yields 1%-4% accuracy gains across strong agentic models and improves accuracy by around 2%-3% under the same token budget. On additional terminal-related benchmarks, it reduces total token consumption while maintaining or improving task success rates. These results suggest that self-evolving, workflow-adaptive observation compression is an effective path toward more reliable and efficient long-horizon terminal agents. The code is publicly available at https://github.com/multimodal-art-projection/TACO.

1 Introduction

Terminal agents face a bottleneck from redundant yet potentially task-critical observations in long interaction histories. TACO addresses this with self-evolving compression rules that adapt across workflows while improving the accuracy–token trade-off.

  • Motivation: Terminal agents retain raw command outputs for useful feedback, but growing histories create efficiency and reliability bottlenecks.Terminal observations can include both low-value redundancy and exact evidence needed for later decisions.
  • Motivation: 24.6%–44.1% of raw prompt tokens can be removed from 50 sampled TB 2.0 trajectories through effective-text extraction.The removable content indicates uneven information density, while logs may still contain errors, paths, test names, arguments, and build targets.
  • Motivation: Generic summarization can blur exact terminal signals, whereas static heuristics are brittle across commands, repositories, languages, and domains.These limitations motivate compression that preserves task-critical evidence while adapting to heterogeneous terminal workflows.
  • TACO: TACO discovers, refines, and reuses structured preservation-aware compression rules from interaction trajectories without task-specific training or human-crafted rules.An LLM proposes rules, a conservative executor applies matching rules, and feedback such as full-output requests updates the active set.
  • Results: 1%–4% absolute accuracy gains are achieved across strong backbone models on TerminalBench 1.0 and 2.0.Under matched token budgets, TACO further improves accuracy by around 2%–3%.
  • Results: 12%–27% lower total token consumption is achieved while maintaining or improving task success across additional terminal-related benchmarks.The evaluated benchmarks include SWE-Bench Lite, CompileBench, DevEval, and CRUST-Bench.

2 Self-evolving Compression Agentic Framework

TACO compresses terminal observations through conservative, task-specific rules that evolve during and across workflows. It preserves critical outputs, adapts rules using agent feedback, and reuses empirically effective rules through a global pool.

  • Framework overview: TACO applies dynamically evolving compression rules to terminal observations within a plug-and-play agent adapter.The adapter returns a compressed observation according to the active task-specific rule set.
  • Rule representation: Rules combine applicability conditions with trigger, retention, removal, and conservative-bound parameters for safe filtering.A fixed executor converts structured rules into concrete observation filtering behavior.
  • Observation compression: Critical observations containing explicit errors or failure signals remain unchanged, while non-Critical outputs are compressed by active rules.This design protects signals such as syntax errors and exception traces that may guide subsequent actions.
  • Intra-task evolution: Uncovered observation types trigger LLM-generated rules, and over-compression complaints cause implicated rules to be suppressed and replaced conservatively.Complaints include requesting full output, reporting missing details, or repeating commands to recover information.
  • Global rule evolution: Effective rules are written to a global pool only after successful application and sufficient task-level confidence, while complained-about rules are excluded and down-ranked.Existing rules with complaints receive decayed global confidence, reducing their retrieval probability in later tasks.
  • Global rule evolution: The global ranking score prioritizes rules that are reliable and broadly reusable, and task-conditioned selection filters retrieved candidates for relevance.New rules start with global confidence 1.0 and are activated only after task-specific refinement.
  • Evolution protocol: Multi-round evolution propagates effective rules across parallel tasks, while Retention measures stability of the Top-K rule frontier without reward signals.Evolution stops when Retention stabilizes and uses only observation-level signals rather than answers, hidden tests, or success labels.

3 Experiment Setup

The experiments assess TACO across multiple terminal benchmarks, backbone model families, and agent scaffolds. Results are reported relative to Terminus-2 baselines on TerminalBench with end-to-end token accounting and repeated evaluation runs.

  • Benchmarks: The evaluation covers TerminalBench 1.0 and 2.0, SWE-Bench Lite, CompileBench, DevEval, and CRUST-Bench.This benchmark set tests TACO across several terminal-related task domains.
  • Models and scaffolds: TACO is instantiated with multiple backbone LLMs and evaluated across different agent scaffolds to test generality.The study includes both model-family and agent-implementation variation.
  • Models and scaffolds: The evaluated backbones include closed-source Claude, GPT, and Gemini series models and open-source terminal-capable model families.The open-source set also includes Qwen3 models with fewer than 40B parameters.
  • Agent scaffolds: Terminus-2 serves as the baseline scaffold for TerminalBench, CompileBench, DevEval, and CRUST-Bench, while Mini-SWE-Agent is used for SWE-Bench Lite.Mini-SWE-Agent supports repository-level issue solving and command-line interaction.
  • Reporting: TerminalBench results compare TACO-block gains against corresponding Terminus-2 baselines using the same backbone.The table reports results for both TB 1.0 and TB 2.0.
  • Reporting: In-house results are averaged over five evaluation runs, with end-to-end token counts including backbone execution and auxiliary TACO calls.Run-level standard deviations for main accuracy results are reported in Appendix E.

4 Results and Discussion

Across heterogeneous terminal-agent evaluations, TACO improves accuracy and token efficiency by adaptively evolving reusable compression rules. Its benefits persist under fixed budgets, outperform static compression approaches, and remain effective after rule reuse, while convergence and ablation analyses support the value of self-evolution.

  • Adaptability Across Agentic Models: Approximately 10% lower per-step token cost occurs for models above 200B parameters without substantially changing average steps.For smaller models, TACO can enable longer successful trajectories, increasing total steps and overall consumption relative to early-failure baselines.
  • Efficiency Comparison: Under identical token budgets from 14 million to 120 million tokens, TACO improves accuracy across all six evaluated models.Gains are 1%–2% for several larger models and generally around 2%–3% for models below 32B parameters.
  • Comparison with Static Compression Methods: Static methods that remove more tokens than TACO achieve smaller accuracy gains, showing that compression should preserve task-relevant signals rather than maximize reduction.The comparison includes LLM summarization and 200 human-curated rules, while TACO interactively refines rules using environmental feedback.
  • Rule Reusability and Ablation: TACO rescues more tasks than it regresses across all evaluated models, while its converged Global Rule Pool remains effective when reused on TB 2.0.The cold-start cost requires multiple iterations but is incurred once and can be amortized over later deployments.
  • Rule Reusability and Ablation: Using only either evolutionary component reduces total token consumption but degrades task performance, whereas the full system evolves shared reusable rules.The ablation contrasts fixed initialization without intra-task evolution and task-only evolution without a shared Global Rule Pool.

5 Related Work

Prior work compresses or manages long agent histories through general context-reduction and memory-management techniques. TACO instead compresses terminal observations before they enter agent history and evolves reusable preservation-aware rules rather than task-solving strategies.

  • Context Compression for Code and Terminal: Context-compression research for code and terminal agents includes token pruning, information filtering, retrieval-based selection, and LLM summarization.These approaches target reducing cost and improving reliability in long-horizon software-engineering and terminal tasks.
  • Agent Context Management and Self-Evolving Agents: Agent context-management methods retain, summarize, fold, or remove past messages, actions, observations, and intermediate reasoning traces.Examples include trajectory summarization, memory retrieval, context truncation, observation masking, proactive context folding, and learned history compression.
  • TACO’s Distinction: TACO compresses terminal observations before they enter agent history and evolves reusable, preservation-aware compression rules rather than task-solving strategies.This distinguishes its focus from methods that manage accumulated histories or directly optimize task-solving behavior.

6 Conclusion

TACO provides adaptive, training-free context compression for terminal agents by discovering, refining, and reusing rules from interaction trajectories. Across diverse terminal environments, it improves task performance and token efficiency.

  • TACO automatically discovers, refines, and reuses compression rules from interaction trajectories.These rules support adaptive, training-free context compression across diverse terminal environments.
  • Experiments show that TACO consistently improves task performance and token efficiency across agent frameworks and backbone models.
  • The results support removing redundant terminal context as a practical path toward more efficient and effective long-horizon terminal agents.

A Limitations

TACO compresses observations rather than increasing the intrinsic capability of backbone models, and the study evaluates it on public benchmarks rather than as a standalone deployed system. Results are averaged over five runs, while human annotation is limited to a static comparison baseline.

  • TACO operates at the observation-compression layer and does not modify backbone models or improve their intrinsic capabilities.Its role is to present existing agents with cleaner, less redundant terminal observations.
  • Human annotation was used only to construct the High-Quality Rule static comparison baseline.TACO itself does not rely on manual rule engineering.
  • All reported results are averaged over five independent runs unless otherwise stated.
  • The work studies observational context compression rather than a standalone deployed system.The authors recommend sandboxed execution and human oversight alongside standard safety protocols.

F Additional TerminalBench Results with Closed-Source Models

This section reports closed-source model results on TerminalBench for reference, with starred entries indicating results reported on the TerminalBench leaderboard.

  • Closed-source model results on TerminalBench are reported for reference.
  • Table 10 marks results reported on the TerminalBench leaderboard with an asterisk.

G Hyperparameter Selection

TACO’s hyperparameters balance rule retrieval quality, accuracy, token cost, throughput, and the speed at which newly learned rules become reusable. The main experiments set k = 30 while varying batch size N involves an accuracy–runtime trade-off.

  • Figure 5: Figure 5 reports retrieval-size effects on accuracy and self-evolution token cost, and batch-size effects on accuracy and relative runtime.Token cost and runtime are normalized to k = 30 and N = 4.
  • Hyperparameter effects: The Top-k hyperparameter and batch size N shape interaction between TACO’s Intra-Task Rule Pool and Global Rule Pool.
  • Additional TerminalBench results: TACO consistently improves pass@k across all evaluated settings on TB 2.0 and six models.
  • Rule evolution: TACO updates compression rules from observation-level interaction signals without accessing benchmark correctness signals.
  • Top-k Selection: When k > 30, additional performance gains become marginal or negative while self-evolution token cost continues increasing.The main experiments therefore set k = 30.
  • Batch Size N Selection: Smaller batch sizes tend to improve accuracy by writing newly learned rules back more frequently, whereas larger batches improve parallel throughput and runtime efficiency.Larger batches delay applying newly acquired rules to later tasks.

H Best-of-K Trajectory Sampling

TACO’s TerminalBench evaluation examines pass@k across six models, while trajectory studies show how task-specific rules are initialized, evolved, and reused during execution.

  • H Best-of-K Trajectory Sampling: TACO is evaluated with pass@k for k ∈{4, 5, 6, 7, 8} across six models on TB 2.0, with and without compression.This tests agents’ potential ability under Best-of-K-style inference-time scaling.
  • H Best-of-K Trajectory Sampling: Task-specific active rules are initialized by retrieving ranked rules from a Global Rule Pool, then selecting, adapting, or generating rules with an LLM.The resulting active set is conditioned on the task description.
  • H Best-of-K Trajectory Sampling: Plan-time rules encode domain-specific filtering, such as preserving gcov errors and final linking steps while removing repetitive build output.Other examples target secret-related signals or verbose test-output lines.
  • H Best-of-K Trajectory Sampling: Reactive evolution generates new rules when lengthy observations are not covered by the active set, extending compression to previously unseen output patterns.In one trajectory, partial apt-install compression reduced an initial chunk from 10,071 to 5,371 characters before later continuation output triggered further evolution.
  • H Best-of-K Trajectory Sampling: The sqlite-with-gcov trajectory required no new execution-time rules because its initialized rules covered compiler and git-related output.This indicates online evolution is used when initialized coverage is insufficient.

K.2 Quantitative Compression Analysis

Case studies show that TACO compresses terminal output non-uniformly, concentrating savings on repetitive, high-volume observations while preserving task-relevant signals through semantic filtering.

  • K.2 Quantitative Compression Analysis: 29,464 of 29,636 saved characters in vulnerable-secret came from one reactively evolved objdump rule across 18 applications.Its average compression ratio was 0.445, ranging from 0.146 to 0.999.
  • K.2 Quantitative Compression Analysis: Only 4 of 25 adaptive-rejection-sampler episodes triggered compression, yet those entries saved over 14,000 characters from high-volume apt-get installation steps.The savings were concentrated in installation output rather than distributed evenly across the trajectory.
  • K.2 Quantitative Compression Analysis: 10,071 characters of repetitive R-package installation output were compressed to 73 characters, a ratio of 0.007.The retained status indicator tells the agent whether installation is waiting or progressing.
  • K.2 Quantitative Compression Analysis: The sqlite compiler rule removes 25 file-listing lines while preserving build configuration messages and the final command containing -fprofile-arcs -ftest-coverage.This retains evidence that gcov instrumentation is enabled despite removing verbose copying output.
  • K.2 Quantitative Compression Analysis: The objdump rule achieves an 85.4% reduction while preserving symbolic calls, branch targets, and section headers needed for tracing anti-debugging logic.Across the full trajectory, this rule saved 29,464 characters of redundant disassembly.
  • K.2 Quantitative Compression Analysis: Across the three trajectories, TACO behaves as a domain-adaptive semantic compressor by reusing global rules and evolving rules for uncovered outputs.The case studies present this behavior as an explanation for the performance gains reported in the main experiments.

M.2 Residual Loss Analysis

Residual-loss analysis examines potential critical information loss and documents the prompts and safeguards used to make rule evolution conservative and preservation-aware.

  • M.2 Residual Loss Analysis: Eight potential critical-loss events were manually inspected, with most judged benign; one behaviorally affected case involved a compressed view of a recently written file.The passage states that the affected case occurred after cat pipeline_parallel.py.
  • M.2 Residual Loss Analysis: TACO initializes, selects, modifies, and generates structured rules using task context, historical rules, and terminal state.The prompts require new rules for uncovered high-output command categories and assign explicit trigger, retention, stripping, and truncation parameters.
  • M.2 Residual Loss Analysis: The rule-generation safeguards require complete error preservation and conservative retention, while excluding short-output commands from rule creation.The stated target is high-output commands such as pip, apt, make, pytest, git, and docker.
  • M.2 Residual Loss Analysis: Uncovered long-output commands trigger reusable category-level rules that identify removable noise while preserving errors, results, and actionable information.The prompt explicitly asks for a trigger matching a command category rather than only the exact command.
  • M.2 Residual Loss Analysis: Over-aggressive compression triggers a conservative replacement after feedback such as requests for the full output or command re-execution.The original rule is frozen before revision.
  • M.2 Residual Loss Analysis: Binary-inspection outputs, exact differences, traces, checksums, paths, test results, and program results are listed as content that must not be compressed.Progress bars and repetitive status updates are identified as compressible in the stated policy.
Loading 2604.19572v3…