PaperlayerView exact revision on arXiv

Source-linked AI summary

REPOFUSE: Repository-Level Code Completion with Fused Dual Context

Ming Liang, Xiaoheng Xie, Gehao Zhang, Xunjin Zheng, Peng Di, wei jiang, Hongwei Chen, Chengpeng Wang, Gang Fan

arXiv:2402.14323v2cs.SEcs.AI

TL;DR

Repository-level code completion draws on broader codebase context to improve prediction, but longer prompts can increase inference time. REPOFUSE fuses analogy and rationale contexts and uses rank truncated generation to condense them; on CrossCodeEval it reports improved accuracy and inference efficiency.

  • Problem

    Repository-level completion uses cross-file context to improve precision and relevance, while richer context can lengthen prompts and increase inference times.

    Source: §1 · p. 1

  • Method

    REPOFUSE fuses code-analogy and semantic rationale contexts, then applies rank truncated generation to select relevant context for a concise prompt.

    Source: §1 · pp. 1–2

  • Results

    40.90% to 59.75% improvement in code exact match (EM) accuracy and up to 24.4% throughput efficiency on CrossCodeEval; inference speed increases by 26.8% while using 75.4% of typical context length.

    Source: §1 · p. 2

  • Takeaways & Limitations

    REPOFUSE has been integrated into a large organization's software-development workflows.

    Source: §1 · p. 2

    Evaluation was limited to smaller Code LMs because of memory constraints and specific usage scenarios; practical implementation with larger models remains necessary to validate the expected performance.

    Source: §6 · p. 9

Abstract

from arXiv · show

The success of language models in code assistance has spurred the proposal of repository-level code completion as a means to enhance prediction accuracy, utilizing the context from the entire codebase. However, this amplified context can inadvertently increase inference latency, potentially undermining the developer experience and deterring tool adoption - a challenge we termed the Context-Latency Conundrum. This paper introduces REPOFUSE, a pioneering solution designed to enhance repository-level code completion without the latency trade-off. REPOFUSE uniquely fuses two types of context: the analogy context, rooted in code analogies, and the rationale context, which encompasses in-depth semantic relationships. We propose a novel rank truncated generation (RTG) technique that efficiently condenses these contexts into prompts with restricted size. This enables REPOFUSE to deliver precise code completions while maintaining inference efficiency. Through testing with the CrossCodeEval suite, REPOFUSE has demonstrated a significant leap over existing models, achieving a 40.90% to 59.75% increase in exact match (EM) accuracy for code completions and a 26.8% enhancement in inference speed. Beyond experimental validation, REPOFUSE has been integrated into the workflow of a large enterprise, where it actively supports various coding tasks.

1. Introduction

Repository-level completion uses cross-file context to improve code-completion precision, but richer prompts can increase inference time. REPOFUSE combines analogy and rationale contexts and uses rank truncated generation to limit prompt size while pursuing accuracy and efficiency.

1. Introduction

  • Repository-level completion extends beyond in-file context because a line's functionality can depend on broader codebase context.The paper describes the trade-off between richer context improving predictions and longer prompts increasing inference times as the Context-Latency Conundrum.Source: §1 · p. 1
  • REPOFUSE fuses analogy context from code similarity analysis with rationale context capturing semantic relationships such as class taxonomies and API interactions.It uses the two cross-file context types to prompt language models for code completion.Source: §1 · pp. 1–2
  • Rank truncated generation (RTG) selects task-relevant context to form a concise prompt within a limited token window.The method is introduced to reduce the prompt-size burden while maintaining high completion accuracy.Source: §1 · p. 2
  • 40.90% to 59.75% improvement in code exact match (EM) accuracy and up to 24.4% throughput efficiency on CrossCodeEval; REPOFUSE uses 75.4% of typical context length and increases inference speed by 26.8%.The paper also reports integration into a large organization's daily software-development workflows.Source: §1 · p. 2
  • The introduction frames REPOFUSE as a way to combine cross-file context with concise prompting for accurate, efficient code completion.It presents comparative evaluation and large-organization workflow integration as contributions.Source: §1 · p. 2

2. Preliminaries

The preliminaries formalize the repository setting and the code units used to describe completion. They define a completion environment, fixed-line code chunks, and a sliding chunk cover that supports specifying the unfinished code and completion goal.

2. Preliminaries

  • A completion environment pairs the edited file's content with the list of source-code files in the repository.The completion task generates and appends tokens to the unfinished chunk ending at the completion point, using that environment.Source: §2 · p. 2
  • A code chunk is a code snippet containing a fixed number of code lines.Chunks capture local syntactic features and can reveal developers' programming intent to some extent.Source: §2 · p. 2
  • A code chunk cover divides source files into chunks bounded by a chunk size and sliding step, with at most one successor sharing ℓck − η lines.The successor illustrates coding outcomes with the lines common to both chunks.Source: §2 · p. 2

3. Methodology

REPOFUSE models repository-level completion on programmers’ use of repository architecture and similar modules. It combines rationale and analogy contexts, then uses rank truncated generation to fit relevant context into a bounded prompt.

3. Methodology

  • REPOFUSE draws on repository architecture and similar modules, reflecting how programmers use both to guide code written for a task.The approach simultaneously leverages completed tokens and similar repository code.Source: §3 · pp. 2–3

3.1. Rationale Context Analysis

  • Rationale context derives available program constructs from imports and represents methods, classes, and packages through their signatures.A class signature concatenates its fields’ and methods’ signatures; a package signature concatenates those of its classes and methods.Source: §3.1 · p. 3

3.2. Analogy Context Retrieval

  • Analogy context retrieves repository chunks similar to the unfinished chunk, whose successor code offers completion guidance.The example identifies similar chunks through shared tokens such as “user” and “service.”Source: §3.2 · p. 3

3.3. RTG-Empowered Completion

  • RTG ranks rationale and analogy items by relevance, selecting a largest set under a token-length bound so both valid constructs and similar implementations can guide completion.Relevance scoring can use different distances or lightweight embedding models.Source: §3.3 · pp. 3–4

4. Experiments and Results

The experiments evaluate REPOFUSE on CrossCodeEval, comparing dual context with single-context baselines across models and truncation settings. They report stronger Code EM alongside more efficient inference, and assess RTG scoring choices.

4.1. Implementation

  • REPOFUSE builds rationale context from a repository code knowledge graph and analogy context from BM25 search results.The graph is pregenerated in the experiment; its update time is reported as roughly 0.04 milliseconds, while performance implications of frequent updates are left for practical scenarios.Source: §4.1 · p. 4

4.2. Benchmark

  • The evaluation uses CrossCodeEval’s Python subset, comprising 471 repositories, 1,368 files, and 2,665 examples, with Code Match and Identifier Match metrics.CrossCodeEval is described as a repository-level completion benchmark spanning Python, Java, TypeScript, and C#.Source: §4.2 · p. 5

4.3. Configuration

  • To avoid potential data leakage, the study selects Code LMs released before mid-2023 and limits evaluation to models supporting 8k input tokens.Rationale-context truncation ranges from 256 to 4096 tokens, while in-file context is capped at 2048 tokens.Source: §4.3 · p. 5

4.4.1. PERFORMANCE

  • 40.90%–59.75% Code EM gains over AC and 32.13%–69.45% over RC are reported for Dual Context across Code LMs.Table 1 presents performance for analogy, rationale, and dual contexts across the evaluated models.Source: §4.4.1 · p. 5, Table 1 · p. 6

4.4.2. INFERENCE EFFICIENCY

  • 24.4% and 57.5% throughput gains over AC and RC, and 26.8% and 95.3% latency speed optimizations, were reported for DC 512 at batch size 8.On a single A100 GPU, DC 512 also raised throughput by 29.2% over AC and 56.5% over RC, while tolerating batch sizes up to 2× and 5× larger, respectively, under memory constraints.Source: §4.4.2 · p. 6

4.4.1. PERFORMANCE

  • Dual Context consistently outstrips AC and RC across truncation sizes; DC at 512 tokens outperforms both baselines at 4096.The authors report that the smaller context also improves inference speed.Source: §4.4.1 · pp. 5–6, Figure 3 · p. 7
  • On DeepSeek-Coder-7B, DC records 744 exact matches versus 553 for RC and 515 for AC, including 38 cases requiring both contexts.The authors attribute DC’s advantage to complementary information and cases not adequately handled by either context alone.Source: §4.4.1 · p. 6, Figure 4 · p. 7

4.4.3. COMPARISON OF RTG SCORE FUNCTIONS

  • Oracle excels at 512 tokens, strategies show no significant differences at 2048, UniXcoder leads non-Oracle strategies, and Random ranks least effective.The compared scoring choices include Oracle, semantic similarity, lexical similarity, and random scores.Source: §4.3 · p. 5, §4.4.3 · pp. 6–7, Figure 5 · p. 7

5. Related Work

Related work spans pretrained code language models and methods for retrieving or integrating repository-level context. The passages also identify efficiency as a need for fast-response development scenarios.

5. Related Work

  • Most pretrained Code LMs focus on individual files; DeepSeek-Coder is noted as an exception that incorporated repository-level data construction during pretraining.The related-work passage also lists models using next-token prediction and, in some cases, fill-in-the-middle training.Source: §5 · p. 8
  • Repository-level approaches retrieve examples, generate prompts, fuse repository information, or compress cross-file context for code generation.The surveyed work also explores private API documentation and similar snippets from a source-code database.Source: §5 · p. 8
  • Existing repository-level methods may not suit fast-response development scenarios, motivating more efficient approaches.Source: §5 · p. 8

6. Conclusion and Future Work

REPOFUSE extracts dual contexts and uses a prompt-generation strategy for repository-level code completion. Future work includes extending its context analysis and evaluating larger Code LMs in practice.

6. Conclusion and Future Work

  • REPOFUSE combines dual contexts from the completion environment with a prompt-generation strategy, and the paper reports its integration into a commercial production line.The system is also described as expected to be released as open source.Source: §6 · p. 8
  • A lightweight program analyzer could remove unnecessary constructs from rationale context to improve completion accuracy and efficiency.The proposed filtering targets constructs that cannot be invoked without specific object types.Source: §6 · p. 8
  • The evaluation was limited to smaller Code LMs; larger-model performance remains an expectation requiring practical validation.The paper also identifies unused suffix content and the relationship between cross-file, prefix, and suffix information as future work.Source: §6 · pp. 8–9

Broader Impact

The broader-impact discussion describes REPOFUSE as refining existing models without directly training them. It links this approach to avoiding the pretraining resource demands and associated environmental impacts.

Broader Impact

  • REPOFUSE adds cross-file context to existing models without directly training them, bypassing the substantial computational resources associated with pretraining.Source: Broader Impact · p. 9
  • The authors acknowledge code-generation risks such as bias and insecurity and report implementing measures to minimize them.Source: Broader Impact · p. 9
  • The paper says its approach enhances developer productivity without the negative broader impacts of increased energy consumption or greenhouse gas emissions.Source: Broader Impact · p. 9

A. Experiment results of REPOFUSE over DeepSeek-Coder-7B on All Truncation Sizes

The supplied captions identify tables comparing analogy, rationale, and dual contexts on DeepSeek-Code-7B.

A. Experiment results of REPOFUSE over DeepSeek-Coder-7B on All Truncation Sizes

B. CODE KNOWLEDGE GRAPH

The CODE KNOWLEDGE GRAPH represents repository entities and their dependencies as a multi-directed graph. Its algorithm retrieves rationale context by collecting related cross-file code or signatures located before the cursor line.

B. CODE KNOWLEDGE GRAPH

  • The CODE KNOWLEDGE GRAPH stores repository entities and dependency relationships in a multi-directed graph of nodes and edges.Nodes represent modules, classes, functions, or variables; edges encode relations such as Imports or Calls.Source: B. CODE KNOWLEDGE GRAPH · p. 14
  • Each node records its location, type, and name, while each edge records its relationship type and location.Node IDs use name:type@location; edge IDs use relation@location when location is available.Source: B. CODE KNOWLEDGE GRAPH · p. 14
  • For Python repositories, Jedi constructs the graph, and the framework allows other tools to extend support to additional languages.Source: B. CODE KNOWLEDGE GRAPH · p. 14
  • The example graph connects class instantiation, variable use, and an import relationship across two Python files.Source: B. CODE KNOWLEDGE GRAPH · p. 14

C. Algorithms

  • The rationale-context algorithm retrieves related edges and imports, then appends cross-file node content or signatures whose start lines precede the cursor.It uses function code text and signatures for other node types.Source: C. Algorithms · p. 16

D. Throughput-Oriented Efficiency

DC 512 delivers higher peak throughput than AC and RC in the reported comparisons, while prompt construction order has similar Code EM performance across the tested strategies. The 3B model under RC does not operate on the NVIDIA A10.

D. Throughput-Oriented Efficiency

E. Comparison of Different Prompt Construction Strategies

F. The token length distribution of different context over benchmark.

The supplied figures cover context token-length distributions, AC/RC/DC convergence, score-function comparisons, and a practical prompt example. Their captions identify these topics but provide no numerical outcomes.

F. The token length distribution of different context over benchmark.

  • Figure 11 presents token-length distributions for different contexts.Source: Figure 11 · p. 19

G. Convergence of AC, RC and DC over 5 Code LMs

  • Figure 12 presents convergence of AC, RC, and DC over five other code LMs.Source: Figure 12 · p. 19

H. Comparison of Code EM values on AC and RC of different Score Functions over Code LMs

I. A Practical Prompt Example

Loading 2402.14323v2…