PaperlayerView exact revision on arXiv

Source-linked AI summary

ACC: Compiling Agent Trajectories for Long-Context Training

Qisheng Su, Zhen Fang, Shiting Huang, Yu Zeng, Yiming Zhao, Kou Shi, Ziao Zhang, Lin Chen, Zehui Chen, Lijun Wu, Feng Zhao

arXiv:2605.21850v2cs.CLcs.AI

TL;DR

Long-context training is costly, and standard agent SFT leaves scattered tool-response evidence weakly supervised. ACC compiles multi-turn agent trajectories into long-context QA pairs for direct answer training, achieving strong long-range benchmark results while preserving general capabilities and exhibiting task-adaptive mechanisms.

  • Problem

    Standard agent SFT masks tool responses and supervises turn-level tool selection, leaving answer-relevant evidence scattered across turns unused for long-context reasoning.

  • Method

    ACC compiles answer-verified search, SWE, and SQL agent trajectories into long-context QA pairs by assembling the original question with multi-turn tool responses and environment observations without additional human annotation.

  • Results

    ACC-trained Qwen3-30B-A3B achieves comparable long-range dependency modeling results to Qwen3-235B-A22B, including 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), while preserving general capabilities.

  • Takeaways & Limitations

    ACC provides scalable supervised fine-tuning data that complements existing long-context extension or training methods, with task-adaptive attention restructuring and expert specialization emerging after training.

  • Takeaways & Limitations

    ACC is evaluated on three agent types and one model, broader generalization and million-token scaling remain unstudied, and teacher-dependent synthesis may propagate bias.

Abstract

from arXiv · show

Recent development of agents has renewed demand for long-context reasoning capacity of LLMs. However, training LLMs for this capacity requires costly long-document curation or heuristic context synthesis. We observe that agents produce massive trajectories when solving problems, invoking tools and receiving environment observations across many turns. The evidence needed to answer the original question is thus scattered throughout these turns, requiring integration of distant context segments. Nevertheless, standard agent SFT masks tool responses and only trains turn-level tool selection, creating a supervision blind spot where these scattered signals go unused. We propose Agent Context Compilation (ACC), which converts trajectories from search, software engineering, and database querying agents into long-context QA pairs that combine the original question with tool responses and environment observations gathered across multiple turns, training the model to answer directly without tool use. This makes the dependencies between the question and the evidence explicit, enabling direct supervision of long-context reasoning over distant segments without additional annotation. ACC is a simple but effective approach that can be combined with any existing long-context extension or training method, providing scalable supervised fine-tuning data. We validate ACC on long-range dependency modeling tasks through MRCR and GraphWalks, challenging benchmarks requiring cross-turn coreference resolution and graph traversal over extended contexts. Training Qwen3-30B-A3B with ACC achieves 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), results comparable to Qwen3-235B-A22B, while preserving general capabilities on GPQA, MMLU-Pro, AIME, and IFEval. Further mechanism analysis reveals that the ACC-trained model exhibits task-adaptive attention restructuring and expert specialization.

1 Introduction

ACC addresses the supervision blind spot in agent SFT by compiling multi-turn trajectories into long-context QA pairs, making scattered evidence directly usable for answer supervision. Across search, software engineering, and SQL agents, ACC improves long-range dependency modeling while preserving general capabilities and revealing task-adaptive mechanisms.

  • Motivation: Agents scatter answer-relevant evidence across tool calls and environment observations, while standard agent SFT masks tool responses and supervises only turn-level tool selection.This leaves distant evidence signals unused for long-context reasoning.
  • Method: ACC compiles the original question with multi-turn tool responses and environment observations into long-context QA training pairs without additional human annotation.The method makes dependencies between questions and scattered evidence explicit for direct answer supervision.
  • Method: ACC uses answer-verified trajectories from search, SWE, and SQL agents, taking answers directly from final outputs without additional annotation.These agent classes retrieve web pages, inspect source files, and query relational tables, respectively.
  • Results: 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6) are achieved by ACC-trained Qwen3-30B-A3B, with results comparable to Qwen3-235B-A22B.MRCR tests cross-turn coreference resolution, while GraphWalks tests graph traversal over extended contexts.
  • Results: ACC training preserves general capabilities and is associated with task-adaptive attention restructuring and expert specialization.The reported general-capability evaluations include GPQA, MMLU-Pro, AIME, and IFEval.

2 Related Work

Long-context research has progressed from retrieval-focused tests toward broader reasoning benchmarks, while existing improvement methods modify model architecture, context handling, or training data.

  • Benchmark evolution: Early benchmarks such as NIAH primarily tested surface-level retrieval by placing specific facts among distractor text.Later benchmarks expanded beyond this setting.
  • Benchmark evolution: RULER added variable tracking, aggregation, and multi-hop reasoning, while LongBench introduced real-world QA, summarization, and code-understanding tasks.These benchmarks broadened evaluation beyond isolated fact retrieval.
  • Method categories: Recent long-context methods modify position embeddings or attention mechanisms, reduce attention complexity, or construct high-quality long documents for pretraining.Examples include RoPE-based scaling, sparse or linear attention, and long-document construction.

3 Method

ACC reframes multi-turn agent trajectories as direct long-context reasoning examples by exposing scattered tool evidence to answer prediction. It replaces turn-level tool-selection supervision with a compiled context containing verified evidence and, when available, retained reasoning traces.

  • 3.1 The Supervision Blind Spot of Agent SFT: Standard agent SFT masks tool responses and supervises only model-generated reasoning and actions, leaving cross-turn evidence integration unsupervised.
  • 3.1 The Supervision Blind Spot of Agent SFT: Intermediate tool responses receive mainly local action-prediction signal, while answer-relevant gradients must traverse later turns and weaken substantially.
  • 3.2 Agent Context Compilation: ACC trains reasoning and final-answer prediction directly from the original question and one compiled context, removing intermediate action terms from the objective.
  • 3.2 Agent Context Compilation: Each answer-verified trajectory becomes a training example whose input combines the original query with compiled context, while the final answer supplies the target.
  • 3.3 Context Construction: ACC extracts visited evidence and distractors by agent type, then randomly permutes self-contained pieces before concatenation within a token budget.Search uses visited pages plus unvisited results; SWE uses patch-relevant files plus inspected distractors; SQL uses queried table contents.
  • 3.3 Context Construction: Answer-verified trajectories receive retained reasoning traces generated by DeepSeek-V3.2-Thinking; SWE instead uses answer-conditioned synthesis because direct rollout accuracy is near 10%.Search and SQL use direct rollouts with pass rates near 100% and 50%, respectively.

4 Experiments

ACC is evaluated on long-range dependency benchmarks, general capabilities, leakage controls, and mechanism analyses. The experiments report strong benchmark gains, preserved general performance, and task-specific changes in attention and expert routing.

  • Main Results: 68.28 on MRCR (+18.09) and 77.51 on GraphWalks (+7.59) demonstrate improved long-range dependency modeling over the Qwen3-30B-A3B-Thinking baseline.MRCR measures multi-round coreference resolution, while GraphWalks measures graph traversal precision.
  • Main Results: ACC results are comparable to Qwen3-235B-A22B despite using nearly 8× fewer active parameters.The comparison concerns the long-range dependency modeling benchmarks.
  • General Capability Preservation: ACC slightly improves GPQA-Diamond (+2.49), MMLU-Pro (+1.50), and AIME’25 (+3.33), while AIME’24 and IFEval remain stable.These evaluations are used to check for negative transfer to general capabilities.
  • Leakage Analysis: Training-query analysis finds domain-level overlap rather than instance duplication, with average nearest-neighbor cosine similarity below 0.36 and separation AUC of 0.9986.Search partially overlaps with general-knowledge benchmarks, whereas SWE and SQL form distinct clusters.
  • Agent-Type Ablation: Single-agent ACC variants improve MRCR over baseline by +8.14 for Search, +4.63 for SWE, and +6.25 for SQL.The ablation indicates that compiling scattered evidence into one context improves cross-turn coreference resolution.
  • Mechanism Analysis: ACC produces task-specific attention restructuring and expert specialization across GraphWalks and MRCR.GraphWalks increases nearby and far-distance attention and distributes distant-token activation across several experts, while MRCR emphasizes nearby attention and a smaller expert set.

5 Conclusion

ACC compiles multi-turn agent trajectories into long-context training data and complements existing long-context methods. ACC-trained Qwen3-30B-A3B reaches comparable performance to Qwen3-235B-A22B on MRCR and GraphWalks while largely preserving general capabilities.

  • ACC compiles multi-turn agent trajectories into long-context training data and can be combined with existing long-context extension or training methods.
  • ACC-trained Qwen3-30B-A3B achieves results comparable to Qwen3-235B-A22B on MRCR and GraphWalks, which test long-range dependency modeling.
  • ACC training largely preserves general capabilities while mechanistic analyses suggest task-specific attention restructuring and task-dependent expert specialization.

6 Limitations and Social Impacts

ACC’s broader generalization and scaling remain open questions, while its reuse of agent trajectories introduces privacy and intellectual-property risks. The SWE and SQL examples illustrate how ACC assembles evidence for direct long-context reasoning.

  • ACC is evaluated on three agent types and one model, leaving broader generalization and scaling to million-token contexts for future study.
  • Reasoning synthesis depends on a strong teacher model, creating a risk of bias propagation.
  • Raw trajectories may leak private information without proper filtering, and compiled contexts may contain copyrighted or proprietary material.
  • SWE agent example: For SWE agents, ACC compiles selectively opened file contents into the background while shuffling in unopened distractors.
  • SQL agent example: For SQL agents, ACC compiles the relevant table into a long-context background, enabling multi-hop relational reasoning without SQL execution.

B Extended Results on General Long-Context Tasks

ACC yields modest gains on general long-context benchmarks spanning multi-hop question answering, long-document understanding, and a comprehensive long-context suite.

  • ACC yields modest gains across HotpotQA and MuSiQue for multi-hop QA, NarrativeQA for long-document understanding, and LongBench-V2.
  • Table 7 reports extended benchmark results as avg@3, with parenthesized values representing improvement over the Qwen3-30B-A3B-Thinking baseline.

C Data Overlap Experiment Details

The overlap analysis cleans and embeds training and benchmark questions, visualizes them with UMAP, and computes similarity and classification metrics. Examples and figures document the SWE and SQL trajectory-compilation settings.

  • Question extraction: Training and benchmark questions are cleaned by retaining problem statements, whitespace-normalizing, and truncating extracted questions to 3,000 characters.
  • Embedding: Cleaned questions are encoded with all-MiniLM-L6-v2 into normalized 384-dimensional vectors, using cosine similarity for distance computations and only leading content beyond 256 tokens.
  • Dimensionality reduction: UMAP projects embeddings into two dimensions using 15 neighbors, minimum distance 0.3, cosine distance, PCA initialization, and random seed 42.
  • Metrics: Quantitative indicators use the original 384-dimensional embeddings, while UMAP coordinates are reserved for visualization.
  • Trajectory examples: SWE compilation retains selectively opened evidence while adding unopened distractors, whereas SQL compilation assembles relevant table contents for graph reasoning.
  • Metrics: The analysis averages each benchmark question’s maximum training-sample cosine similarity, compares centroid cosine distance, and reports logistic-regression AUC for train–benchmark separation.

D Attention Analysis Experiment Details

The attention analysis compares the baseline and ACC-trained models by aggregating causal attention across distance bins and selected layers, then visualizing SFT–baseline changes.

  • Setup: The analysis compares Qwen-30B-A3B-Thinking with the ACC-trained checkpoint using eager attention tensors in bfloat16 precision.The setup forces eager attention implementation so the full 4D attention matrix is accessible.
  • Layer and Distance Binning: Attention distances are divided into 32 equal-width bins, with weights aggregated along lower-triangular off-diagonals and averaged per head and layer.The bins span distances from zero to the sequence length minus one, including the main diagonal.
  • Metrics and Visualization: The heatmap reports SFT–baseline attention deltas for layer–distance pairs, where positive values indicate increased attention mass after ACC training.Top-head analysis separately averages attention over the farthest 25% of distance bins to rank heads with the strongest far-range changes.
  • Statistics and Visualization: Statistics are aggregated across evaluation samples and averaged per head and distance bin before visualization.The reported attention statistics summarize sample-level measurements rather than individual examples.

E Expert Routing Visualization Experiment Details

The expert-routing analysis compares baseline and ACC-trained checkpoints by measuring top-k expert frequencies across token groups and selected layers, then visualizing routing deltas.

  • Models, Layers, and Sampling: The analysis samples 32 GraphWalks and MRCR evaluation examples and collects router statistics from both models in inference mode.Three layers with the largest expert-routing changes are selected for each task.
  • Metric Definition: For each token position, layer, and expert, top-k frequency records whether the expert belongs to the selected top-k gate.The gating indicator is binary, marking selected experts for each token and layer.
  • Metric Definition: The reported heatmap averages per-layer expert frequencies across the selected task layers and displays their routing delta.The selected layers are those with the largest mean absolute expert-routing delta for the target task.
  • Token Grouping: Sequences are divided into 32 equal-length relative-position groups, enabling routing comparisons across variable-length inputs.Each group covers a fixed interval of token positions relative to the full sequence length.
  • Expert Selection: Experts are ranked by mean absolute routing delta, and the 20 experts with the largest changes are visualized.This selection excludes experts whose routing patterns remain nearly unchanged after SFT.
  • Implementation: Router logits are collected by temporarily wrapping selected MoE forward passes and aggregating statistics incrementally with running means.The original forward functions are restored immediately after logits are extracted, avoiding storage of large intermediate tensors.

F SWE Answer-Conditioned Rationale Generation

Because original SWE trajectories lack explicit natural-language reasoning and direct CoT rollouts have low pass rates, ACC synthesizes answer-conditioned reasoning traces from compiled evidence and verified patches.

  • Motivation: Original SWE trajectories contain file-open and edit actions but lack explicit natural-language reasoning, while direct CoT rollout achieves only ∼10% pass rate.This motivates answer-conditioned rationale synthesis for scalable SWE data construction.
  • Synthesis Method: DeepSeek-V3.2-Thinking receives the ACC-compiled question, compiled context, and verified patch to reconstruct a reasoning trace grounded in the evidence.The prompt instructs the model not to reveal the final patch prematurely.
  • Generation Configuration: The generation uses temperature 0.0 and enables the model’s thinking mode through chat-template configuration.The generated trace is supervised reasoning associated with the verified patch rather than the model’s internal reasoning.
  • Prompt Template: The prompt requests step-by-step issue inspection, keyword search, file browsing, distractor elimination, core-file localization, logic analysis, and proposed modifications.It also explicitly asks the model not to reveal the final patch directly in the reasoning trace.
  • Illustration: Figure 8 shows the prompt containing compiled context C_i and verified patch y_i, followed by the final response used as supervised reasoning trace r_i.The figure presents the complete prompt structure and an example output.
Loading 2605.21850v2…