PaperlayerView exact revision on arXiv

Source-linked AI summary

Rivet: Utility-Preserving Context Control for Repository-Level Coding Agents

Ruwei Pan, Linhao Wu, Yakun Zhang, Zixiong Yang, Lu Zhang, Hongyu Zhang

arXiv:2610.34967v1cs.SE

TL;DR

Repository-level coding agents need repository evidence, yet limited context budgets require selecting only part of a repository, and existing selection criteria do not directly measure whether that evidence is present. Rivet grounds task-related code anchors, follows repository relations to collect evidence, and shapes the context to fit the budget. It achieves the highest Pass@1 in all 16 reported settings, improving over the strongest baseline by 3.73–5.15 percentage points on average.

  • Problem

    Context budgets limit repository-level coding agents to selected repository content, while existing selection criteria do not directly measure whether that context contains evidence needed to implement code in its repository.

    Source: §1 · p. 1

  • Method

    Rivet selects task-related code anchors using lexical, semantic, and structural signals, expands them through repository relations, and packs the resulting evidence by priority under the context budget.

    Source: §1 · p. 2

  • Results

    Rivet achieves the highest Pass@1 in all 16 settings, improving over the strongest baseline by 3.73–5.15 percentage points on average.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Under constrained budgets, which evidence enters the context matters more than how much context the agent receives.

    Source: §9 · p. 19

    DevEval and RepoExec focus on function-level generation and do not cover multi-file changes such as feature addition and bug fixing.

    Source: §7 · p. 17

Abstract

from arXiv · show

Repository-level code generation asks an agent to implement a task requirement inside an existing repository. To produce code that works in the repository, the agent needs repository evidence, such as target locations, helper definitions, caller usages, and configuration constraints, which determine how the code should be located, integrated, and executed. Because the context budget cannot hold the entire repository, existing retrieval, compression, and structure-aware methods select context by task similarity, context length, or structural proximity, but these criteria do not directly measure whether the selected context contains this evidence. As a result, the selected context can appear relevant to the task yet miss the evidence that the implementation depends on. We call this failure mode context utility dilution. In this paper, we propose Rivet, a context control framework that preserves repository evidence for coding agents under a fixed context budget. Rivet, first uses Signal Grounding to select task-related code anchors by fusing lexical, semantic, and structural signals. Utility Linking then expands these anchors through dependency, usage, and configuration relations to collect repository evidence. Finally, Budget Shaping packs the evidence by priority and, when the budget is tight, reduces snippets to signatures or summaries before dropping them. We evaluate Rivet, on two executable repository-level benchmarks, DevEval and RepoExec, with four backbone models, two context budgets (4K and 8K tokens), and eight baselines. Rivet, achieves the highest Pass@1 in all 16 settings and improves over the strongest baseline by 3.73--5.15 percentage points on average. Moreover, GPT-5-mini with Rivet, outperforms GPT-5 without Rivet,on both benchmarks while reducing inference cost by 73.3%--84.5%.

1 Introduction

Repository-level coding agents need repository evidence to locate and integrate code, but limited context budgets force them to use only a subset of a repository. Rivet constructs context around task-related anchors and connected evidence; it achieves the highest Pass@1 across the reported settings.

1 Introduction

  • Context utility dilution occurs when selected snippets appear task-relevant but omit repository evidence needed to implement code correctly in its target repository.Existing retrieval, compression, and structure-aware criteria do not ensure that context includes needed helper definitions, caller usages, and configuration constraints.Source: §1 · pp. 1–2
  • Rivet grounds task-related code anchors, follows dependency, usage, and configuration relations to collect evidence, then packs it by priority under the context budget.When the budget is tight, snippets are reduced to signatures or summaries before being dropped.Source: §1 · p. 2
  • Rivet achieves the highest Pass@1 in all 16 settings, improving over the strongest baseline by 3.73–5.15 percentage points on average.Across all settings, Rivet achieves an average Pass@1 of 55.74; GPT-5-mini with Rivet also outperforms GPT-5 without Rivet while reducing inference cost by 73.3%–84.5%.Source: §1 · pp. 2–3
  • Rivet is evaluated on DevEval and RepoExec with four backbone models and 4K- and 8K-token budgets.The comparison covers retrieval, compression, and structure-aware baselines under the same agent scaffold.Source: §1 · p. 2

2 Motivating Example

A concise requirement to implement a JSON-serializability utility does not specify how the function should fit the existing repository. The example motivates collecting repository evidence connected to the task location rather than relying on similarity alone.

2 Motivating Example

  • The example task asks for an is_json_serializable utility in benedict.utils.type_util, but the requirement alone does not provide enough information to implement it for the repository.Useful evidence includes the target location, helper definitions, caller usages, and configuration constraints.Source: §2 · p. 3
  • Keyword-matched snippets can omit helper, caller, and configuration evidence, leaving an implementation that may duplicate project utilities, mismatch usage, or fail in the project environment.Under a limited context budget, relevant but incomplete snippets can crowd out the evidence that constrains implementation.Source: §2 · p. 3
  • The example motivates starting from where the task lands in the repository and collecting connected evidence, as Rivet does through anchor grounding and relation-based expansion.The overview depicts Signal Grounding, Utility Linking, and Budget Shaping as the framework's three stages.Source: §2 · p. 4, Figure 2 · p. 4

3 Approach

Rivet constructs repository context under a fixed budget by locating task-related anchors, linking them to repository evidence, and prioritizing or compacting that evidence. Its stages combine lexical, semantic, and structural signals with dependency, usage, and configuration relations.

3.1 Problem Formulation

  • Rivet models snippets as a typed, weighted evidence graph and uses imports, symbol references, callers, and configuration links to expose repository constraints.The task requirement becomes a weighted query whose explicit code-like mentions receive higher weights.Source: §3.1 · p. 5

3.2 Signal Grounding

  • Signal Grounding fuses lexical matching, semantic similarity, and import-graph propagation to select task-related anchor files.Explicit code mentions receive a deterministic prior; small repositories fall back to TF-IDF cosine similarity for semantic scoring.Source: §3.2 · pp. 5–6

3.3 Utility Linking

  • Utility Linking expands anchors through dependency, usage, and configuration relations, then scores collected snippets by their connection to the required code.Dependency evidence includes definitions and imports, usage evidence includes call sites, and configuration evidence captures integration and execution constraints.Source: §3.3 · pp. 6–7

3.4 Budget Shaping

  • Budget Shaping prioritizes anchors and direct dependencies, then lowers snippet detail to signatures or summaries before dropping selected evidence.It considers evidence tiers first and combines anchor relevance with evidence relevance to order snippets within each tier.Source: §3.4 · p. 7

4 Experimental Setup

The evaluation compares context-construction methods on executable repository-level tasks under matched agent settings and fixed context budgets. It uses two benchmarks, Pass@1, representative baseline families, and four backbone models across 4K and 8K budgets.

4 Experimental Setup

  • The study asks whether Rivet works under constrained budgets, how it compares with larger budgets or stronger models in accuracy and cost, and how its stages and evidence types contribute.The evaluation includes stage and evidence-type ablations.Source: §4 · p. 7

4.1 Benchmarks

  • DevEval provides 1,825 function-level tasks from real-world repositories, while RepoExec provides 355 annotated tasks emphasizing execution within repository context.DevEval is the main benchmark and RepoExec provides external validation across different repositories and test suites.Source: §4.1 · p. 8

4.2 Metrics

  • Pass@1 measures the percentage of tasks whose generated code passes the official tests, with paired bootstrap resampling used to estimate 95% confidence intervals for differences.The resampling compares Rivet with the strongest non-Rivet baseline per benchmark and budget, pooling four backbone models.Source: §4.2 · p. 9

4.3 Baselines

  • Baselines cover retrieval, compression, and structure-aware methods; Target-Local separately tests whether context around a known target location is sufficient.Target-Local receives the benchmark-provided location, whereas other baselines construct or reduce context from the task and repository.Source: §4.3 · p. 9

4.4 Implementation Details

  • All methods share the Mini-SWE-Agent scaffold, task, repository, model, prompt, budget, decoding strategy, and execution environment; only context construction differs.Experiments use four backbones, 4K and 8K budgets, and a consistent Python 3.9 environment.Source: §4.4 · p. 10

5 Experimental Results

Rivet achieves the highest Pass@1 in all 16 settings, outperforming the strongest baseline by 3.73–5.15 percentage points on average. It also compares favorably with larger context budgets and stronger backbones, while ablations show that all three stages contribute.

5.1 RQ1: Accuracy Comparison

  • Rivet leads all 16 settings, improving over the strongest baseline by 3.73–5.15 percentage points on average.Paired bootstrap tests find the Pass@1 gains statistically significant for each benchmark and budget.Source: §5.1 · pp. 10, 12, Table 1 · p. 11
  • Target-Local trails Rivet by 9.02–17.59 percentage points; plain-text LLMLingua2 and LLM Summarize fall below RAG in 30 of 32 comparisons.LongCodeZip, which compresses at function granularity, matches or exceeds RAG on RepoExec in all eight settings.Source: §5.1 · pp. 11–12

5.2 RQ2: Cost-Effectiveness Analysis

  • 54.51 vs. 52.47 Pass@1 on average: with 4K tokens, Rivet beats the strongest 8K baseline in six of eight settings.Rivet at 4K beats the strongest 8K baseline on DevEval for all four backbones; on RepoExec, it wins two settings and trails by 0.28 and 1.97 points in the other two.Source: Figure 3 · p. 12, §5.2 · pp. 12–13
  • GPT-5-mini + Rivet beats GPT-5 + Mini-SWE-Agent by 6.03 points on DevEval and 1.98 on RepoExec, cutting inference cost by 73.3% and 84.5%, respectively.DeepSeek-V3.2 + Rivet improves Pass@1 over DeepSeek-V4-Pro + Mini-SWE-Agent by 8.71 points on DevEval and 4.78 on RepoExec at comparable inference cost.Source: Table 2 · p. 13, §5.2 · p. 13

5.3 RQ3: Ablation Study

  • Removing Signal Grounding, Utility Linking, or Budget Shaping decreases average Pass@1 by 21.72, 16.84, or 2.88 percentage points, respectively.Without Signal Grounding, Rivet falls below RAG in all eight settings; without Utility Linking, it falls below Target-Local in all eight.Source: §5.3 · pp. 13–14, Table 3 · p. 14
  • On RepoExec, disabling dependency, usage, or configuration expansion costs 8.16, 10.14, or 8.16 points; on DevEval, individual drops are only 0.27–0.49.On DevEval, disabling all relations costs 21.53 points; the relation types largely overlap there but provide complementary evidence on RepoExec.Source: §5.3 · pp. 14–15, Table 4 · p. 15

6 Discussion

The discussion illustrates how Rivet links task anchors to repository evidence and examines the contributions and limits of its graph components and target localization. The ablations matter more on RepoExec than DevEval, while localization is highly accurate on both benchmarks.

6.1 How Does Rivet Preserve Repository Evidence?

  • For cookiecutter::get_user_config, Signal Grounding anchors config.py and Utility Linking adds get_config(), defaults, an exception, caller, and YAML dependency; Budget Shaping compresses lower-priority evidence to fit 4K.The requirement’s precedence rule is reflected in code outside the target file, while the baselines miss different parts of that evidence.Source: §6.1 · p. 15, Figure 4 · p. 16

6.2 How Do Graph Components Affect Rivet’s Performance?

  • Static repository relations can miss dynamic behavior such as runtime imports or reflection, reducing evidence Utility Linking collects.Lexical and semantic signals mean missing structural edges alone cannot remove a file from the anchors.Source: §6.2 · p. 16
  • Graph-component variants lose at most 0.44 percentage points on DevEval but 7.60–9.01 on RepoExec; import-and-symbol links alone cost 8.73 points on RepoExec.Removing structural grounding has the largest effect on RepoExec.Source: §6.2 · p. 16
  • The graph ablation measures the graph’s contribution, not Rivet’s robustness to wrong or missing edges; controlled edge-noise evaluation remains future work.Source: §6.2 · p. 16

6.3 How Accurately Does Signal Grounding Localize Target Files?

  • 90.36% of DevEval targets rank first and 97.97% fall within the top five (MRR 0.939); RepoExec records 98.59%, 99.44%, and 0.989, respectively.Localization alone does not explain Rivet’s gains: the ground-truth-location Target-Local baseline trails Rivet by 9.02–17.59 percentage points.Source: Table 6 · p. 17, §6.3 · p. 17

7 Threats to Validity

The evaluation controls context construction across methods, but its scope is limited to function-level generation in Python repositories, with fixed hyperparameters and benchmark-based measures that have stated caveats.

Generalizability

  • Experiments focus on Python, and validation on other languages remains future work despite language-independent grounding, graph propagation, and budget-packing components.Parsing, import resolution, and execution environments are Python-specific.Source: §7 · p. 17

Task representativeness

  • DevEval and RepoExec cover function-level generation, not multi-file feature additions or bug fixes where locating code to change is part of the task.The authors call for evaluation on broader issue-resolution and feature-evolution benchmarks.Source: §7 · p. 17

Comparison fairness

  • All methods share the task, repository files, model, prompt, budget, decoding, scaffold, and execution environment, though method-specific tuning could improve individual baselines.Source: §7 · p. 17

Hyperparameters

  • Rivet’s hyperparameters stay fixed across settings, and their sensitivity is unstudied, so other values may change absolute results.Source: §7 · p. 17

Metric validity and data leakage

  • Pass@1 measures executable success but not full semantic correctness, and public-benchmark pretraining may affect absolute scores while the shared protocol reduces its impact on relative comparisons.Reference implementations, dependencies, tests, and target locations are reserved for evaluation and analysis, not context construction.Source: §7 · pp. 17–18

8 Related Work

Prior repository-context methods select or shorten code using similarity, structure, or agent state. Rivet combines relation-based evidence collection with budget packing that lowers snippet detail before dropping it.

Retrieval and Compression for Code Context

  • Retrieval and compression methods make context more relevant or compact, but can still omit implementation-dependent repository evidence; Rivet follows repository relations and lowers snippet representation before dropping it.These approaches include similarity-based retrieval, iterative retrieval, and token or code-granularity compression.Source: §8 · p. 18

Structure-aware Repository Context Construction

  • Structure-aware methods use repository relations or agent feedback, while Rivet collects dependency, usage, and configuration evidence from shared anchors and packs snippets without LLM calls.Unlike truncating or pruning context, Rivet reduces each snippet’s representation level before dropping it.Source: §8 · p. 18

9 Conclusion

Rivet addresses context utility dilution by linking task-related anchors to repository evidence and packing that evidence under a budget. It leads the reported benchmark settings and enables lower-cost GPT-5-mini to outperform GPT-5, with broader language and task validation left for future work.

Conclusion

  • Rivet achieves the highest Pass@1 in all 16 settings and improves over the strongest baseline by 3.73–5.15 percentage points on average.With a 4K budget, it beats the strongest 8K baseline in six of eight settings; GPT-5-mini with Rivet outperforms GPT-5 at 73.3%–84.5% lower inference cost.Source: §9 · pp. 18–19
  • The authors plan to extend Rivet to other programming languages and evaluate it on multi-file tasks such as issue resolution and feature implementation.Source: §9 · p. 19

Related papers

Loading 2610.34967v1…