PaperlayerView exact revision on arXiv

Source-linked AI summary

ACC: Compiling Agent Trajectories for Long-Context Training

Qisheng Su, Zhen Fang, Shiting Huang, Yu Zeng, Yiming Zhao, Kou Shi, Ziao Zhang, Lin Chen, Zehui Chen, Lijun Wu, Feng Zhao

arXiv:2605.21850v2cs.CLcs.AI

TL;DR

Long-context training relies on costly document curation or heuristic synthesis, while standard agent SFT masks tool responses. ACC compiles multi-turn trajectories into QA training data; ACC-trained Qwen3-30B-A3B reaches 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), comparable to Qwen3-235B-A22B.

  • Problem

    Long-context training relies on costly document curation or heuristic synthesis, and standard agent SFT masks tool responses while trajectories carry answer-relevant evidence across turns.

    Source: §1 · p. 1, §3.1 · pp. 3–4

  • Method

    ACC compiles multi-turn agent trajectories into long-context QA pairs combining the original question with tool responses and observations, using final answers without additional human annotation.

    Source: §1 · p. 2

  • Results

    ACC-trained Qwen3-30B-A3B reached 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), comparable to Qwen3-235B-A22B while preserving general capabilities.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ACC provides scalable supervised fine-tuning data that can be combined with existing long-context extension or training methods without additional human annotation.

    Source: §1 · p. 2

    ACC is evaluated on three agent types and one model, so broader generalization and scaling to million-token contexts remain unstudied.

    Source: §6 · p. 9

Abstract

from arXiv · show

Recent development of agents has renewed demand for long-context reasoning capacity of LLMs. However, training LLMs for this capacity requires costly long-document curation or heuristic context synthesis. We observe that agents produce massive trajectories when solving problems, invoking tools and receiving environment observations across many turns. The evidence needed to answer the original question is thus scattered throughout these turns, requiring integration of distant context segments. Nevertheless, standard agent SFT masks tool responses and only trains turn-level tool selection, creating a supervision blind spot where these scattered signals go unused. We propose Agent Context Compilation (ACC), which converts trajectories from search, software engineering, and database querying agents into long-context QA pairs that combine the original question with tool responses and environment observations gathered across multiple turns, training the model to answer directly without tool use. This makes the dependencies between the question and the evidence explicit, enabling direct supervision of long-context reasoning over distant segments without additional annotation. ACC is a simple but effective approach that can be combined with any existing long-context extension or training method, providing scalable supervised fine-tuning data. We validate ACC on long-range dependency modeling tasks through MRCR and GraphWalks, challenging benchmarks requiring cross-turn coreference resolution and graph traversal over extended contexts. Training Qwen3-30B-A3B with ACC achieves 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6), results comparable to Qwen3-235B-A22B, while preserving general capabilities on GPQA, MMLU-Pro, AIME, and IFEval. Further mechanism analysis reveals that the ACC-trained model exhibits task-adaptive attention restructuring and expert specialization.

1 Introduction

Agent Context Compilation (ACC) turns multi-turn agent trajectories into long-context question-answer supervision, addressing the unused evidence scattered across tool interactions. The paper reports stronger long-range reasoning benchmarks alongside preserved general capabilities and identifies attention and expert changes after training.

  • Standard agent SFT masks tool responses and supervises only turn-level tool selection, leaving scattered evidence unused despite its importance for answering the original question.Conventional long-context training also relies on costly long-document curation or heuristic context synthesis, limiting scalable supervision.Source: §1 · pp. 1–2
  • ACC assembles the original question with tool responses and environment observations across turns into long-context training data, enabling direct reasoning supervision without human annotation.The approach can be combined with existing long-context extension or training methods.Source: §1 · p. 2, Figure 1 · p. 2
  • ACC compiles answer-verified trajectories from search, software engineering, and SQL agents into training pairs, taking each answer from the trajectory’s final output without additional annotation.The agents retrieve web pages, inspect source files, or query relational tables, respectively.Source: §1 · p. 2
  • 68.3 on MRCR (+18.1) and 77.5 on GraphWalks (+7.6) for Qwen3-30B-A3B are comparable to Qwen3-235B-A22B, while preserving GPQA, MMLU-Pro, AIME, and IFEval.MRCR and GraphWalks test cross-turn coreference resolution and graph traversal over extended contexts.Source: §1 · p. 2
  • ACC training exhibits task-adaptive attention restructuring and expert specialization, reflecting flexible adaptation to distinct long-range reasoning demands.The analysis describes the resulting patterns as flexible and task-specific.Source: §1 · pp. 2–3

2 Related Work

Long-context evaluation has progressed from retrieval and single-turn tasks to harder tests of long-range dependencies. Existing extension and training methods modify architectures, construct pretraining documents, use post-training RL, or manage memory at inference, while ACC trains directly on agent trajectories.

2.1 Long-Context Capacity Evaluation

  • MRCR and GraphWalks test cross-turn coreference and graph traversal over extended contexts, addressing limits of benchmarks focused on localized retrieval or single-turn reasoning.Earlier benchmarks include NIAH, RULER, and LongBench; Musique and NarrativeQA target multi-hop reasoning and long-document understanding.Source: §2.1 · p. 3

2.2 Long-Context Extension and Training

  • Long-context approaches modify architectures, synthesize pretraining documents, combine synthetic data with RL, or manage memory at inference; ACC instead uses agent trajectories for direct reasoning training.Examples include MrRoPe and ROPE++ [12], Longwanjuan and LiteLong, longRLVR and LongPO, and QwenLong-L1.5 and MemAgent.Source: §2.2 · p. 3

3 Method

Standard agent SFT masks tool observations, leaving scattered evidence weakly supervised through local turn decisions. ACC instead compiles evidence into a context for direct answer training, with task-specific evidence selection and context construction.

3.1 The Supervision Blind Spot of Agent SFT

  • Standard agent SFT masks tool responses, so their answer-relevant signals pass through intermediate turns and are weakened by local tool-selection supervision.The model therefore tends to update observations for local action prediction rather than features relevant to the final answer.Source: §3.1 · pp. 3–4

3.2 Agent Context Compilation

  • ACC trains the model to produce a reasoning trace and answer from the original question and compiled context, directly supervising every evidence token without intermediate action terms.This shifts training from local next-tool selection toward integrating scattered evidence into a global answer.Source: §3.2 · p. 4

3.3 Context Construction

  • ACC selects evidence sufficient to answer without tools: visited-page text plus candidate distractors for search, patch-related files plus debugging context for SWE, and queried table contents for SQL.The compiled training example combines the original query and context with the trajectory’s final answer and a reasoning trace.Source: §3.3 · p. 4, §3.2 · p. 4
  • ACC randomly permutes self-contained evidence pieces before concatenating them within a token budget, requiring semantic matching rather than reliance on sequence position.The context length is capped at B tokens.Source: §3.3 · p. 4
  • For Search and SQL, direct-rollout traces have pass rates near 100% and 50%; because SWE direct rollout reaches only near 10% accuracy, answer-conditioned reasoning traces are synthesized instead.SWE synthesis uses compiled context and a verified patch, with all outputs retained (100% yield).Source: §3.3 · p. 5

4 Experiments

ACC improves long-range dependency benchmark performance while preserving general capabilities. Ablations and mechanism analyses show contributions from mixed agent trajectories, distractors, and task-dependent attention and expert changes.

4.2 Main Results

  • ACC scores 68.28 (+18.09) on MRCR and 77.51 (+7.59) on GraphWalks, comparable to Qwen3-235B-A22B with nearly 8× fewer active parameters.It improves both MRCR needle settings and both GraphWalks sub-tasks.Source: Table 2 · p. 6, §4.2 · p. 6

4.1 Experimental Setup

  • ACC compiles 10,802 Search, SWE, and SQL trajectories into training contexts spanning 2K to 128K tokens.The evaluation targets MRCR coreference resolution and GraphWalks traversal, while monitoring GPQA-Diamond, MMLU-Pro, AIME, and IFEval.Source: §4.1 · p. 6, Figure 3 · p. 6

4.3 General Capability Preservation

  • ACC slightly improves GPQA-Diamond (+2.49), MMLU-Pro (+1.50), and AIME’25 (+3.33), while AIME’24 and IFEval remain stable.The reported results indicate no noticeable degradation in general abilities.Source: §4.3 · p. 7, Table 3 · p. 7
  • Training-query analysis finds topical overlap for Search but distinct SWE and SQL clusters, with nearest-neighbor cosine similarity below 0.36 and separation AUC of 0.9986.The authors interpret this as domain-level overlap rather than instance duplication, suggesting gains reflect transferable reasoning rather than leakage.Source: §4.3 · p. 7, Table 4 · p. 7, Figure 4 · p. 7

4.4 Comparison with Long-Context Post-Training Methods

  • QwenLong-L1.5 leads on MRCR, while ACC surpasses it on GraphWalks using standard SFT; LongPO and LongRLVR are listed for reference.LoongRL is omitted because trained checkpoints are not released; QwenLong-L1.5 is evaluated with standard inference, and LongPO instances exceeding its 128K context limit are excluded.Source: §4.4 · pp. 7–8, Table 5 · p. 8

4.5 Ablation Study

  • Single-agent ACC improves MRCR over baseline by +8.14 (Search), +4.63 (SWE), and +6.25 (SQL), but only SQL improves GraphWalks (+5.58).The full mixture surpasses all single-agent variants; raw search trajectories trained with masked observations underperform the base model.Source: §4.5 · p. 8, Table 6 · p. 8
  • Removing Search and SWE distractors lowers MRCR by 3.34 and 3.81 points, while their single-agent GraphWalks variants gain +13.71 and +2.22, respectively.The full ACC mixture still achieves the best overall GraphWalks result, 77.51; the authors attribute the differing effects to localization versus noise filtering.Source: §4.5 · p. 8, Table 6 · p. 8

4.6 Mechanism Analysis

  • ACC restructures attention and expert routing differently for GraphWalks and MRCR, with the strongest shifts in different layers across tasks.GraphWalks gains attention mass at nearby and far distances, while MRCR gains nearby attention and preserves its long-range profile. For distant tokens, GraphWalks distributes activation across several experts; MRCR concentrates activation in one expert across token groups.Source: §4.6 · pp. 8–9, Figure 5 · p. 9

5 Conclusion

ACC compiles multi-turn agent trajectories into long-context training data and can be combined with existing long-context methods. On MRCR and GraphWalks, ACC-trained Qwen3-30B-A3B performs comparably to Qwen3-235B-A22B while largely preserving general capabilities.

5 Conclusion

  • ACC-trained Qwen3-30B-A3B achieves results comparable to Qwen3-235B-A22B on MRCR and GraphWalks while largely preserving general capabilities.Both benchmarks test long-range dependency modeling.Source: §5 · p. 9
  • ACC compiles multi-turn agent trajectories into long-context training data and complements existing long-context extension or training methods.It can be combined with those methods.Source: §5 · p. 9
  • Mechanistic analyses suggest that ACC training produces task-specific attention restructuring and task-dependent expert specialization.Source: §5 · p. 9
  • Future work includes extending ACC to more agent types and scaling it to longer contexts.Source: §5 · p. 9

6 Limitations and Social Impacts · Appendix · A Agent Trajectory Compilation Examples

ACC’s evaluation scope, reliance on a strong teacher, and reuse of agent logs create generalization, bias, privacy, and intellectual-property concerns. The appendix illustrates compilation for SWE and SQL trajectories, turning selected code files or relational records into contexts for direct reasoning.

6 Limitations and Social Impacts

  • ACC is evaluated on three agent types and one model, leaving broader generalization and scaling to million-token contexts unstudied; reasoning synthesis also depends on a strong teacher that may propagate bias.Source: §6 · p. 9
  • Reusing agent logs lowers annotation costs, but raw trajectories may leak private information and compiled contexts may contain copyrighted or proprietary material.Source: §6 · p. 9
  • The authors recommend careful data filtering and safety alignment to address these risks.Source: §6 · p. 9

A Agent Trajectory Compilation Examples

B Extended Results on General Long-Context Tasks

ACC yields modest gains on general long-context benchmarks spanning multi-hop QA, long-document understanding, and a comprehensive long-context suite.

B Extended Results on General Long-Context Tasks

C Data Overlap Experiment Details

The overlap experiment standardizes training and benchmark questions, embeds them for similarity analysis, and uses both embedding-space metrics and two-dimensional visualization to characterize overlap.

Question preparation and embedding

  • Training questions are extracted from user turns, benchmark questions are cleaned to problem statements, and both are normalized and truncated at 3,000 characters before embedding.Questions are encoded with all-MiniLM-L6-v2 as normalized 384-dimensional vectors; the model processes at most 256 tokens, so only each question’s leading segment is used.Source: C Data Overlap Experiment Details · p. 12

Dimensionality reduction and metrics

  • Overlap indicators use the original 384-dimensional embeddings, while UMAP projects them to two dimensions solely for visualization.UMAP uses 15 nearest neighbors, minimum distance 0.3, cosine metric, PCA initialization, and random seed 42.Source: C Data Overlap Experiment Details · p. 12

Overlap metrics

  • The experiment measures average nearest-neighbor cosine similarity, cosine distance between normalized training and benchmark centroids, and logistic-regression AUC for separating training from benchmark samples.AUC is reported for the full training set and the Search subset against all benchmarks.Source: C Data Overlap Experiment Details · pp. 14–15

D Attention Analysis Experiment Details

The attention analysis compares the baseline Qwen-30B-A3B-Thinking model with its ACC-trained checkpoint using accessible causal attention tensors. It measures how attention changes across token distances and identifies heads with the strongest far-range changes.

Setup

Layer and Distance Binning

  • The analysis focused on three layers with the most significant attention changes per task and grouped token distances into 32 equal-width bins.The selected layers were 36, 30, and 42 for GraphWalks, and 15, 45, and 17 for MRCR; attention weights were aggregated along causal-matrix off-diagonals and averaged per head.Source: D Attention Analysis Experiment Details · p. 15

Metric Definition

  • Per-head distance-bin means average attention weights across token positions on the corresponding causal-matrix off-diagonals, including the main diagonal.Per-layer bin means then average these values across all heads in the layer.Source: D Attention Analysis Experiment Details · p. 15
  • The heatmap reports the SFT-minus-baseline attention delta for each layer-distance pair, with positive values indicating increased attention mass after ACC training.Source: D Attention Analysis Experiment Details · p. 15

Top-Head Analysis

  • Top heads are ranked by their tail delta, the change in mean attention over the farthest 25% of distance bins.Attention statistics are aggregated across evaluation samples and averaged per head and distance bin.Source: D Attention Analysis Experiment Details · p. 15

E Expert Routing Visualization Experiment Details

The experiment compares expert-routing patterns in the baseline and ACC-trained models using sampled GraphWalks and MRCR examples, selected task-specific layers, and relative-position token groups. It visualizes the largest routing-frequency changes across experts.

Models and Layers

  • The analysis compares Qwen-30B-A3B-Thinking with the ACC-trained checkpoint in the three layers with the largest routing changes for each task.The selected layers are 42, 40, and 7 for GraphWalks and 17, 16, and 15 for MRCR.Source: E Expert Routing Visualization… · p. 16

Dataset and Sampling

  • 32 evaluation examples per task are run through both models in inference mode to collect router statistics.The examples are randomly sampled from the GraphWalks and MRCR evaluation splits.Source: E Expert Routing Visualization… · p. 16

Metric Definition and Token Grouping

  • The heatmap tracks changes in top-k expert-selection frequency across 32 relative-position token groups and the selected task layers.Sequences are divided into equal-length groups by token index, enabling comparisons across variable-length inputs; per-layer expert frequencies are averaged across the selected layers.Source: E Expert Routing Visualization… · p. 16

Expert Selection

  • The visualization shows the 20 experts with the largest mean absolute routing-frequency changes across groups and layers, omitting nearly unchanged experts.Experts are ranked by mean absolute delta before selecting the top 20.Source: E Expert Routing Visualization… · p. 16

Implementation

  • Router logits are collected by temporarily wrapping the selected MoE layers’ forward passes, then restoring the original functions and aggregating statistics with running means.This avoids modifying model weights and storing large intermediate tensors.Source: E Expert Routing Visualization… · p. 16

F SWE Answer-Conditioned Rationale Generation

Because SWE trajectories lack explicit natural-language reasoning and direct CoT rollouts achieved only ∼10% pass rate, the authors synthesize answer-conditioned rationales from compiled contexts and verified patches. They supervise on the generated final responses, retaining all rationales because the patches are already verified.

F SWE Answer-Conditioned Rationale Generation

  • ∼10% pass rate was achieved by direct rollout of CoT trajectories for SWE, whose original trajectories lack explicit natural-language reasoning.The authors use answer-conditioned rationale generation to scale SWE data construction.Source: F SWE Answer-Conditioned Rationale… · p. 16
  • DeepSeek-V3.2-Thinking receives the compiled context and verified patch to reconstruct an evidence-grounded reasoning trace without revealing the patch prematurely.The prompt combines ACC-compiled question xi with verified patch yi; Figure 8 shows the prompt and generated example.Source: F SWE Answer-Conditioned Rationale… · p. 16, Figure 8 · p. 17
  • The final response, rather than internal reasoning that may reference the ground-truth patch, is used as the supervised trace.Generation uses temperature 0.0 with thinking mode enabled.Source: F SWE Answer-Conditioned Rationale… · p. 16
  • 100% yield is effectively achieved by retaining all generated rationales without correctness filtering because each patch is already verified.The simulated discovery process covers issue inspection, search, file browsing, distractor elimination, core-file localization, logic analysis, and patch proposal.Source: F SWE Answer-Conditioned Rationale… · p. 16

Related papers

Loading 2605.21850v2…