PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing repository-level software repair via repository-aware knowledge graphs

Boyang Yang, Jiadong Ren, Shunfu Jin, Yang Liu, Feng Liu, Bach Le, Haoye Tian

arXiv:2503.21710v3cs.SE

TL;DR

Repository-level repair must identify the relevant code behind natural-language bug reports, but existing approaches face grounding, candidate-selection, and interpretability challenges. KGCompass links repository artifacts and code entities, then uses graph paths to guide LLM patch generation. On SWE-bench Lite, it reports 58.3% repair accuracy on Claude-4 Sonnet and gains over pure-LLM baselines across four backbones.

  • Problem

    Repository-level repair must pinpoint buggy functions among thousands, but issue reports often lack explicit location hints and existing methods have grounding and interpretability gaps.

    Source: §1 · p. 2

  • Method

    KGCompass builds a graph linking issues and pull requests to code entities, ranks candidate functions using semantic and structural information, and guides LLM patch generation with entity paths.

    Source: §1 · p. 2, §3.1 · p. 5

  • Results

    58.3% repair accuracy on Claude-4 Sonnet; KGCompass also improves over pure-LLM baselines across four backbones.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    89.7% of successful fault locations require two or more hops, reflecting indirect structural relations that text-only LLMs miss.

    Source: §8 · pp. 18–19

    Evaluation conclusions concern Python projects in SWE-Bench Lite; the modular design is intended to facilitate extension to other languages.

    Source: §6 · p. 17

Abstract

from arXiv · show

Repository-level software repair faces challenges in bridging semantic gaps between issue descriptions and code patches. Existing approaches, which primarily rely on large language models (LLMs), are hindered by semantic ambiguities, limited understanding of structural context, and insufficient reasoning capabilities. To address these limitations, we propose KGCompass with two innovations: (1) a novel repository-aware knowledge graph (KG) that accurately links repository artifacts (issues and pull requests) and codebase entities (files, classes, and functions), allowing us to effectively narrow down the vast search space to only 20 most relevant functions with accurate candidate fault locations and contextual information, and (2) a path-guided repair mechanism that leverages KG-mined entity paths, tracing through which allows us to augment LLMs with relevant contextual information to generate precise patches along with their explanations. Experimental results in the SWE-bench Lite demonstrate that KGCompass achieves state-of-the-art single-LLM repair performance (58.3%) and function-level fault location accuracy (56.0%) across open-source approaches with a single repair model, costing only $0.2 per repair. Among the bugs that KGCompass successfully localizes, 89.7% lack explicit location hints in the issue and are found only through multi-hop graph traversal, where pure LLMs struggle to locate bugs accurately. Relative to pure-LLM baselines, KGCompass lifts the resolved rate by 50.8% on Claude-4 Sonnet, 30.2% on Claude-3.5 Sonnet, 115.7% on DeepSeek-V3, and 156.4% on Qwen2.5 Max. These consistent improvements demonstrate that this graph-guided repair framework delivers model-agnostic, cost-efficient repair and sets a strong new baseline for repository-level repair.

1 Introduction

Repository-level repair must connect natural-language reports to the right code among thousands of functions, while preserving interpretable decisions and controlling candidate-set size. KGCompass links repository artifacts to code entities and uses graph context to guide LLM repair, reporting strong SWE-bench Lite results across multiple models.

Existing approaches

  • Candidate selection trades recall against context size, cost, and hallucination risk, while existing approaches also offer limited insight into repair decisions.A single predicted location can fail when incorrect; adding candidates can inflate the prompt.Source: §1 · p. 2

Challenge

  • Only about 32.7% of SWE-bench Lite issues explicitly name the ground-truth file or function, making cross-modal grounding central to precise fault location.Repository identifiers mentioned in issues can serve as anchors for linking reports to code.Source: §1 · p. 2

Results

  • 58.3% repair accuracy on Claude-4 Sonnet and relative gains of 50.8% on Claude-4 Sonnet, 30.2% on Claude-3.5 Sonnet, 115.7% on DeepSeek-V3, and 156.4% on Qwen2.5 Max.KGCompass reports 56.0% function-level fault location accuracy and costs $0.2 per repair.Source: §1 · pp. 2–3

KGCompass

  • KGCompass links issues and pull requests with files, classes, and functions, narrowing thousands of functions to 20 relevant candidates with graph context for patch generation.The paper reports that the top 20 cover 91.0% of file-level and 66.7% of function-level ground-truth locations.Source: §1 · pp. 2–3

2 Motivating Example

The motivating examples contrast text-only localization and repair with graph-guided alternatives. Multi-hop repository links recover an unmentioned fault location, while an entity path supplies context for a more robust patch.

KG-based Fault Location

  • For astropy-14995, conditioned only on the problem statement, the LLM proposed three candidate locations, none matching the actual function; knowledge-graph traversal from “nddata” via nddata.py and PR #14175 ranked it in the top five.Source: §2 · pp. 3–4

KG-guided Repair

  • For sympy-14814, the path linking issue #14814, PR #14248, and _print_MatAdd supplies context for a defensive getattr-based fix rather than a failing direct attribute access.The path's comments connect _print_MatAdd to the sign-handling pattern in _print_Add.Source: §2 · p. 4

3 Approach

KGCompass mines repository and code relationships to select candidate functions, prompts an LLM with their paths to generate patches, and ranks patches using tests and other criteria.

3 Approach

  • The approach has three phases: knowledge graph mining selects candidate fault locations, patch generation uses candidates and graph context, and patch ranking selects a final patch.The selector takes the 15 highest-scoring graph functions and adds up to five LLM suggestions; ranking uses reproduction and regression tests.Source: §3 · p. 4

3.1 Knowledge Graph Mining

  • The graph links issues and pull requests to code entities and records AST-derived containment, call, import, and attribute relations; function relevance combines semantic and textual similarity with weighted path proximity.Artifacts later than the benchmark issue's creation time are excluded; this filter removes 24.0% of initially mined related artifacts.Source: §3.1 · p. 5

3.2 Patch Generation

  • Patch prompts pair the issue with relevant functions and paths that trace their connections to it; generation also uses deterministic and exploratory sampling and indentation correction.Trying indentation adjustments of ±1 or ±2 levels recovers 2% of invalid patches.Source: §3.2 · pp. 6–7

3.3 Patch Ranking

  • Patch ranking orders candidates by regression-test passing count, reproduction-test passing count, majority voting, and normalized patch size.The system generates at least one reproduction test for 203 of 300 instances (67.7%).Source: §3.3 · pp. 7–8

4 Experimental Setup

KGCompass is evaluated on SWE-bench Lite using top-1 repair and fault-location metrics, with comparisons to leading repair systems and a pure-LLM variant. The setup also examines graph construction, path information, patch ranking, and candidate-pool choices.

4.1 Benchmark

  • SWE-bench Lite contains 300 real GitHub issues from 12 active Python projects, with code snapshots and test suites requiring repair systems to locate faults across repositories.The benchmark includes repositories with thousands of files and functions; a Django snapshot has 2,750 Python files and 27,867 functions.Source: §4.1 · p. 8

4.2 Metrics

  • The evaluation reports resolved rate, file and function fault-location accuracy, and average cost per bug under a top-1 success protocol.File and function accuracy use Jaccard overlap; these scores measure alignment with the ground-truth patch, whereas resolved rate measures end-to-end correctness and can exceed fault-location accuracy.Source: §4.2 · pp. 8–9

4.3 Implementation Details

  • KGCompass uses Neo4j and repository-specific language models and embeddings, with Claude-4 Sonnet as the default repair model and tuned path-decay and similarity-balance parameters.The implementation sets β=0.6 for path-length decay and α=0.3 for embedding-textual similarity balance.Source: §4.3 · p. 9

4.4 Baselines

  • The baselines include leading SWE-bench Lite repair systems and a pure-LLM variant that replaces graph-based candidate selection with up to 20 LLM-proposed functions.The pure-LLM variant keeps subsequent pipeline steps identical to isolate the effect of KG-based fault location.Source: §4.4 · p. 9

4.5 Research Questions

  • The research questions examine repair effectiveness and cost, repository-aware graph contributions to fault location, and the effects of candidate selection, entity paths, patch ranking, and pool size.The graph analysis includes relevant functions, multi-hop relationships, and paths connecting issue descriptions to fault locations.Source: §4.5 · pp. 9–10

5 Experiments and Results

KGCompass is evaluated on SWE-bench Lite for repair performance, fault-location accuracy, and cost, then analyzed for how graph links and repair components contribute to its results. The experiments report strong single-model repair results and examine multi-hop localization, entity-path prompting, and patch selection.

5.1 RQ-1: Effectiveness of KGCompass

  • 58.3% resolved on SWE-bench Lite with Claude-4 Sonnet makes KGCompass the top single-LLM repair system, with 56.0% function-level location accuracy.The average cost is $0.2 per bug.Source: §5.1 · p. 10, Table 1 · p. 10
  • Resolved rates rose 50.8% on Claude-4 Sonnet, 30.2% on Claude-3.5 Sonnet, 115.7% on DeepSeek-V3, and 156.4% on Qwen2.5 Max versus pure-LLM baselines.The gains were larger without explicit location hints: 42.4% for Claude-3.5 and 69.5% for Claude-4, versus 18.5% and 33.7% with hints, respectively.Source: §5.1 · pp. 11–12, Table 2 · p. 12, Table 4 · p. 13

5.2 RQ-2: Contribution of Knowledge Graph to fault location

  • 89.7% of ground-truth functions lie at least one intermediate entity from the issue, while 20 KG-mined functions cover 68.7% of file-level and 40.4% of function-level locations.Only 10.3% are directly connected to the origin issue.Source: Figure 8 · p. 13, §5.2 · p. 14
  • Among 156 intermediate entities on ground-truth paths, 72.4% are files, 16.0% functions, 7.1% pull requests, and 4.5% issues.Repository artifacts account for 11.6% of these entities.Source: §5.2 · p. 14, Figure 9 · p. 14
  • 82.9% of ground-truth functions rank in the top 10 KG candidates, and the 15 highest-scoring functions cover 97.3% of obtained ground-truth locations.KGCompass combines 15 KG candidates with five LLM-generated candidates.Source: §5.2 · pp. 14–15, Figure 10 · p. 14

5.3 RQ-3: Impact of Components on KGCompass’s Repair Performance

  • The hybrid KG+LLM locator exceeds LLM-only coverage: Claude-4 reaches 91.0% file-level and 66.7% function-level, while Claude-3.5 reaches 84.3% and 57.7%.For Claude-3.5, KG-only coverage is 68.7% file-level and 40.4% function-level.Source: §5.3 · p. 15, Table 5 · p. 15
  • With a fixed candidate list at temperature 0.0, entity-path prompting raised resolved bugs from 131 to 142 for Claude-4 and from 102 to 108 for Claude-3.5.The comparisons are against KG-based location without entity paths.Source: §5.3 · p. 16, Table 6 · p. 16
  • With six patch candidates, repair accuracy reached 58.3% for Claude-4 and 46.0% for Claude-3.5; the layered ranking pipeline reports the same results.The Claude-4 ground-truth upper bound is 182 resolved (60.7%), versus ExpeRepair’s 181 (60.3%).Source: §5.3 · pp. 16–17, Table 8 · p. 17

6 Threats to Validity

The authors address validity threats through controls on graph construction and patch evaluation, complementary repair and localization metrics, and comparison with ground-truth patches.

6 Threats to Validity

  • Graph construction uses only issues and pull requests created before each target issue, while evaluation uses regression tests that passed before the bug report.These controls are intended to prevent future-artifact contamination and support fair repair assessment.Source: §6 · p. 17
  • The authors assess generalizability beyond Python SWE-bench Lite projects through thin language adapters, while leaving graph construction, ranking, and repair modules unchanged.They state that adding another language requires modest, self-contained code.Source: §6 · p. 17
  • Repair success, file- and function-level fault-location accuracy, and average cost provide complementary measures, while ground-truth patch comparisons help distinguish correct fixes from coincidental test-passing solutions.The reported metrics are “Resolved,” “File Acc.,” “Func Acc.,” and “Avg Cost.”Source: §6 · p. 17

7 Related Work

Related work spans agentic and procedural repair workflows and code-focused knowledge graphs. KGCompass differs by connecting repository artifacts to code entities and using graph paths to guide repair.

7.1 Repository-level Software Repair

  • Agentic repair coordinates specialized agents, while procedural repair follows expert-designed workflows; KGCompass contrasts these approaches with an explicit graph linking issues to code entities.Agentic examples include OpenHands, AutoCodeRover, Composio, and SWE-agent; Agentless and Moatless exemplify procedural workflows.Source: §7.1 · p. 18
  • Pure-LLM approaches treat repository artifacts and code as separate, whereas KGCompass models both code structure and semantic relationships across artifacts.Source: §7.1 · p. 18

7.2 Knowledge Graph for Repository-level Software

  • Prior knowledge-graph work models code entities and dependencies, including definition-reference links, control flow, and symbol relationships for retrieval or code completion.Examples include RepoGraph [33], GraphCoder, and CodeXGraph [23].Source: §7.2 · p. 18
  • Existing graph-based approaches focus mainly on code structure and overlook repository artifacts and indirect entity relationships; KGCompass combines these in a graph and uses path-guided repair.Source: §7.2 · p. 18

8 Conclusion

KGCompass links repository artifacts to code entities and guides an LLM through ranked multi-hop paths. On SWE-bench Lite, it reports 58.3% repair success and 56.0% function-level fault-location accuracy, with 89.7% of successful localizations requiring multiple hops.

8 Conclusion

  • 58.3% of bugs are repaired on SWE-bench Lite, with 83.6% file-level and 56.0% function-level fault-location accuracy at $0.2 average cost per instance.The paper reports that KGCompass outperforms state-of-the-art single-LLM baselines.Source: §8 · p. 18
  • 89.7% of successful fault locations require two or more hops, and ranked multi-hop paths guide the LLM from linked issues and pull requests to code entities.The graph reduces the search space from thousands of functions to 20 candidates; path-guided prompting attaches an explicit reasoning chain, and ablations show omitting it sharply lowers accuracy.Source: §8 · pp. 18–19
  • The authors identify enriching the graph with domain-specific knowledge and extending path-guided prompting to other reasoning tasks as future directions.Source: §8 · p. 19
Loading 2503.21710v3…