Source-linked AI summary
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, Dong Yu
TL;DR
Repository-level coding demands understanding dependencies and context across files, a need existing methods can overlook. RepoGraph adds line-level graph structure and ego-graph retrieval to existing workflows. Across four frameworks, it reports an average relative SWE-bench success-rate improvement of 32.8%.
Problem
Repository-level tasks require navigating codebases and understanding inter-file dependencies, while existing approaches can miss structural relationships or global repository context.
Source: §1 · pp. 1–2Method
RepoGraph builds a line-level graph of code definitions, references, and dependencies, then retrieves centered ego-graphs for use in procedural or agent workflows.
Source: §1 · p. 2, §3.1 · p. 5, §3.2 · p. 5Results
32.8% average relative improvement in success rate across four frameworks on SWE-bench; CrossCodeEval is used to test transferability to repository-level coding tasks.
Source: §1 · p. 2Takeaways & Limitations
RepoGraph significantly boosts procedural and agent frameworks on real-world software engineering tasks and transfers to other tasks requiring repository-level understanding.
Source: §6 · p. 10Agent-framework integrations generally increase costs; repeated search_repograph() calls can expand prompt contexts, so the authors advise granular cost control.
Source: §4.2 · pp. 7–8
Abstract
from arXiv · showhide
Large Language Models (LLMs) excel in code generation yet struggle with modern AI software engineering tasks. Unlike traditional function-level or file-level coding tasks, AI software engineering requires not only basic coding proficiency but also advanced skills in managing and interacting with code repositories. However, existing methods often overlook the need for repository-level code understanding, which is crucial for accurately grasping the broader context and developing effective solutions. On this basis, we present RepoGraph, a plug-in module that manages a repository-level structure for modern AI software engineering solutions. RepoGraph offers the desired guidance and serves as a repository-wide navigation for AI software engineers. We evaluate RepoGraph on the SWE-bench by plugging it into four different methods of two lines of approaches, where RepoGraph substantially boosts the performance of all systems, leading to a new state-of-the-art among open-source frameworks. Our analyses also demonstrate the extensibility and flexibility of RepoGraph by testing on another repo-level coding benchmark, CrossCodeEval. Our code is available at https://github.com/ozyyshr/RepoGraph.
1 INTRODUCTION
Repository-level coding requires understanding dependencies across files, beyond the function-level abilities of coding LLMs. RepoGraph adds line-level repository structure to existing workflows, and the paper reports gains across frameworks on SWE-bench.
1 INTRODUCTION
- Repository-level tasks require navigating complex codebases, understanding dependencies among files, and ensuring changes integrate without introducing new issues.SWE-Bench evaluates this setting through real-world GitHub issues requiring repository modification to fix bugs or introduce features.Source: §1 · p. 1
- File-level retrieval can find semantically similar but unrelated snippets, flat-file prompting misses repository structure, and agents may focus narrowly without a global view.The introduction frames these limitations as a reason to move beyond semantic matching toward structural understanding.Source: §1 · pp. 1–2
- RepoGraph represents a repository with line-level nodes and definition-reference dependencies, then retrieves ego-graphs for more comprehensive context in procedural and agent frameworks.The retrieved subgraphs can be integrated into both framework types.Source: §1 · p. 2
- Across four frameworks, RepoGraph achieves an average relative improvement of 32.8% in success rate on SWE-bench and is also tested on CrossCodeEval.The CrossCodeEval test examines transferability to general coding tasks requiring repository-level understanding.Source: §1 · p. 2
2 RELATED WORKS
Prior AI software engineering methods include tool-using agents and procedural localization-and-edit workflows. Repository-level approaches add structural context, and RepoGraph is presented as integrating information across line, file, and repository levels.
2.1 LLM-BASED METHODS FOR AI SOFTWARE ENGINEERING
- Agent frameworks let LLMs iteratively act with tools, observe feedback, and plan; procedural frameworks typically localize relevant code before editing it.Agent tools may include file operations, code search, tests, and shell commands; procedural variants include Search-Expand-Edit.Source: §2.1 · p. 3
- Agentless hierarchically searches a repository representation for suspicious files, while retrieval-based procedural methods search indexed code snippets using issue descriptions.Some retrieval methods use sliding windows for completeness.Source: §2.1 · p. 3
2.2 REPOSITORY-LEVEL CODING CAPABILITY
- Repository-level coding benchmarks extend beyond function- or line-level assessment to tasks requiring understanding and contributing to entire projects.SWE-bench reflects demand for end-to-end automatic solutions to real-world problems.Source: §2.2 · p. 3
- Repository representations have evolved to incorporate file dependencies, retrieval, and graph structures, while methods differ in how they retrieve and use that information.RepoGraph is distinguished as integrating context at line, file, and repository levels; the table compares repository-representation approaches across aspects.Source: §2.2 · pp. 3–4, Table 1 · p. 3
3 REPOGRAPH
RepoGraph is a plug-in for agent and procedural workflows that structures repository code as a graph and retrieves related context. Its construction parses and filters code relations before organizing them into a graph; its utility depends on retrieving subgraphs for workflow use.
3 REPOGRAPH
- Given a repository, RepoGraph construction produces a structured graph of code-line nodes and dependency edges to support tracing issues and gathering dependent code context.The section describes construction as a three-step process.Source: §3.1 · p. 4
3.1 CONSTRUCTION
- Construction parses repository code files with Tree-sitter to identify definitions and references, retaining lines involving function calls and dependencies.The described parsing focuses on functions and classes, excluding less significant details such as individual variables.Source: §3.1 · pp. 4–5
- Project-dependent relation filtering excludes repository-independent calls, including standard or built-in relations and third-party relations identified through imports.The passage gives the built-in call len as an example of a relation excluded from the graph.Source: §3.1 · p. 5
- RepoGraph represents code lines as graph nodes, with definition and reference node types linked by containment and invocation edges.Nodes also carry metadata such as line number, file name, and directory.Source: §3.1 · p. 5
3.2 UTILITY
- For retrieval, RepoGraph centers a k-hop ego-graph on a search term representing a key function or class, capturing nearby dependencies and interactions.Retrieved content explicitly includes method- and line-level information and implicitly groups it at file level.Source: §3.2 · p. 5
- Procedural frameworks receive retrieved ego-graphs in localization and editing prompts, while agents can invoke graph retrieval as an additional action and use its output as an observation.The agent determines when to use the action and chooses the search term.Source: §3.2 · p. 6
4 EXPERIMENTS
The experiments evaluated RepoGraph as a plug-in for procedural and agent frameworks on SWE-bench-Lite. RepoGraph improved results across frameworks, while agent integrations could increase costs.
4.1 SETUP
- Experiments added RepoGraph to RAG, Agentless, SWE-agent, and AutoCodeRover on SWE-bench-Lite, tracking resolve rate, patch application, average cost, and tokens.The study used the same baseline settings and configurations when adding RepoGraph.Source: §4.1 · p. 6
4.2 EXPERIMENT RESULTS
- 29.67 for Agentless + RepoGraph was state-of-the-art among open-source methods; resolve rate rose +2.66 for RAG and +2.34 for Agentless (99.63% and 8.56% relative improvement).Table 2 presents the main evaluation results and costs for open-source baselines with RepoGraph.Source: Table 2 · p. 7, §4.2 · p. 7
- Gains were slightly larger for procedural than agent frameworks; the authors suggest mature, deterministic workflows may integrate plug-ins more smoothly.They also link deterministic workflows to reduced complexity from agents’ dynamic decision-making.Source: §4.2 · p. 7
- RepoGraph’s gains were not mainly due to increased token usage, but agent integrations could cost more as exploration triggers repeated searches.Reported cost increases were +0.13$ for AutoCodeRover and +0.18$ for SWE-agent; the authors recommend granular cost control for agent integrations.Source: §4.2 · pp. 7–8
5 ANALYSIS
The analysis examines localization coverage, REPOGRAPH variants, transfer to CrossCodeEval, and errors in resolved and unresolved cases. Across these analyses, it reports improvements alongside limits tied to context integration and patch generation.
5.1 LOCALIZATION COVERAGE
- File-level localization accuracy improves with all baselines, while line-level gains are more modest; line localization demands more precise code-segment identification.The analysis also reports that agent frameworks have stronger line-level accuracy improvements but lower overall resolve rates than procedural frameworks.Source: §5.1 · p. 8, Table 3 · p. 8
5.2 INVESTIGATION OF REPOGRAPH VARIANTS
- For k=1 and k=2 ego-graphs, direct flattening of the 2-hop variant yields a 26.00% resolve rate; 1-hop summarization lowers the rate to 28.33%.The analysis explores direct flattening and LLM summarization as integration methods; it hypothesizes that summarization can lose information when the 1-hop graph already fits the context window.Source: §5.2 · pp. 8–9
5.3 TRANSFERIBILITY TEST
- On 2,665 Python CrossCodeEval samples, REPOGRAPH improves every metric with both LLMs; original code-match EM was 10.8 and identifier-match EM was 16.7.The evaluation uses GPT-4o and Deepseek-Coder as backbone models.Source: §5.3 · pp. 9–10, Table 5 · p. 9
5.4 ERROR ANALYSIS
- Agent frameworks have 22 uniquely resolved cases versus 12 for procedural frameworks; contextual misalignment is the most prevalent error type across methods.With REPOGRAPH, the proportion of all three error types decreases, especially incorrect localization.Source: §5.4 · p. 10, Figure 3 · p. 10
6 CONCLUSION AND DISCUSSION
REPOGRAPH provides line-level repository navigation by aggregating information through ego-graph retrieval. The paper reports improvements on SWE tasks and transfer to other repository-understanding tasks, while identifying further settings for investigation.
6 CONCLUSION AND DISCUSSION
- REPOGRAPH is a plug-in module that uses ego-graph retrieval to aggregate line-, file-, and repository-level information for codebase navigation.The graph operates at the code-line level.Source: §6 · p. 10
- Experiments on real-world SWE tasks show that REPOGRAPH boosts the performance of both procedural and agent frameworks.Source: §6 · p. 10
- The paper reports transfer to other tasks requiring repository-level understanding and proposes investigating more languages, frameworks, developer environments, and real-time feedback.Source: §6 · p. 10
A DETAILED IMPLEMENTATIONS FOR EACH BASELINE METHOD
The appendix describes how REPOGRAPH is integrated into procedural and agent frameworks. Procedural instructions include flattened graph context, while agents receive a new search action.
A.1 PROCEDURAL FRAMEWORKS
- In procedural frameworks, retrieved ego-graph context is flattened into the Function/Class Dependencies part of both localization and edition instructions.Figures 4 and 5 illustrate the localization and edition instructions, respectively.Source: §A.1 · p. 16, Figure 4 · p. 16
A.2 AGENT FRAMEWORKS
- Agent frameworks add “search repograph” as a command-documented action, with its signature shown for the agent.The system instruction sets the task and response template.Source: §A.2 · p. 16, Figure 7 · p. 18
- With REPOGRAPH, SWE-agent’s maximum completion turn falls from 38 to 35 and its average from 21.47 to 19.12; the new search action is mostly used in the first 15 rounds.The paper describes “search repograph” as more precise than the original search actions.Source: §A.2 · p. 16, Figure 8 · p. 18
B DETAILED IMPLEMENTATIONS FOR REPOGRAPH
The appendix describes an LLM-based summarization variant and points to illustrations of the REPOGRAPH pipeline and one component’s dependency filtering.
B.1 REPOGRAPH VARIANT
- One REPOGRAPH variant uses an LLM to summarize retrieved ego-graph context before further processing; the appendix provides its prompt and examples.Source: §B.1 · p. 18
B.2 REPOGRAPH COMPONENTS
- The appendix provides a simplified illustration of the overall REPOGRAPH integration pipeline.Source: §B.2 · p. 18
- A separate illustration explains how REPOGRAPH excludes project-independent dependencies.Source: §B.2 · p. 18
C.1 RESULTS ON OTHER LLMS
With Claude-3.5-Sonnet on SWE-Bench-Lite, baseline results improved slightly, while adding REPOGRAPH still produced a large performance boost. The authors present this as evidence of effectiveness across different foundational models.
C.1 RESULTS ON OTHER LLMS
- Claude-3.5-Sonnet slightly improves the baseline results on SWE-Bench-Lite.The experiments use the SWE-Bench-Lite test set.Source: §C.1 · p. 18
- Integrating REPOGRAPH with baseline methods still boosts performance by a large margin.The passage does not provide numerical results for this comparison.Source: §C.1 · p. 18
- The authors report that the gains support REPOGRAPH’s effectiveness and extendability across different foundational models.Source: §C.1 · p. 18
C.2 RESULTS ON LOCALIZATION FOR BOTH PASS AND FAIL CASES
The analysis reports lower localization accuracy for failure cases than for the overall average, alongside variation in resolved and unresolved issues across repositories and release years.
C.2 RESULTS ON LOCALIZATION FOR BOTH PASS AND FAIL CASES
- Failure cases have lower localization accuracy than the overall average, and the authors state that correct edit locations are important for resolving issues.Table 7 reports localization accuracy for pass and failure cases at file, function, and line levels; no cell values are supplied here.Source: §C.2 · p. 19, Table 7 · p. 21
C.3 RESOLVE RATE BY REPOSITORY
- Django has the most resolved issues (39), while Django and Sympy have the most unresolved issues (75 and 61, respectively).The passage also describes Sphinx as having 13 resolved and 3 unresolved issues.Source: §C.3 · p. 20, Figure 12 · p. 21
C.4 RESOLVE RATE BY TIME
- In 2020, 47 issues were unresolved and 17 were resolved; by 2021 and 2022, unresolved issues slightly decreased while the resolve rate increased.The passage reports that 19 issues were resolved out of 59 in 2019, and 8 out of 22 in 2023.Source: §C.4 · pp. 20–21
D EXAMPLES FOR ERROR ANALYSES
The paper provides examples for each error category discussed in Section 5.4.
D EXAMPLES FOR ERROR ANALYSES
- Figures 14, 15, and 16 provide one example for each error category listed in Section 5.4.Source: D EXAMPLES FOR ERROR ANALYSES · p. 21
E LIMITATIONS AND FUTURE WORK
The authors limit their main evaluation to proprietary GPT-4-series models, the Lite set, and Python. They also describe REPOGRAPH’s repository-level contribution and potential applications in complex software engineering tasks.
E LIMITATIONS AND FUTURE WORK
- The evaluation explored only proprietary GPT-4-series models; the authors identify open-source models as a future research direction.They cite the poor performance of open-source models on this challenging task as the reason for choosing proprietary models.Source: E LIMITATIONS AND FUTURE WORK · p. 21
- Experiments used only the Lite set because large-scale experiments with proprietary models were costly.The authors say more efficient deployment and cost-effective options will be needed to explore larger datasets.Source: E LIMITATIONS AND FUTURE WORK · p. 21
- The main experiments covered Python only, although REPOGRAPH could be adapted to other languages by changing its parsing schemes.The authors propose evaluating other languages to assess generalizability across programming paradigms.Source: E LIMITATIONS AND FUTURE WORK · p. 21
F IMPACT STATEMENT
- REPOGRAPH is presented as enhancing repository-level code understanding and as having potential applications in debugging, maintenance, and large-scale refactoring.The impact statement says the work highlights repository-level context for code generation and maintenance.Source: F IMPACT STATEMENT · pp. 23–24