PaperlayerView exact revision on arXiv

Source-linked AI summary

SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair

Quanjun Zhang, Chengyu Gao, Yu Han, Ye Shang, Chunrong Fang, Zhenyu Chen, Liang Xiao

arXiv:2602.23647v2cs.SE

TL;DR

Repository-level repair requires a bridge between identifying a bug and deciding how to fix it, because localization alone may not provide sufficient context or guidance. SGAgent adds a suggestion stage and repository knowledge tools to a three-agent repair workflow. It reports results across repository and vulnerability repair evaluations, including a 51.3% issue resolution rate on SWE-Bench-Lite and 48.0% average resolution on combined VUL4J and VJBench.

  • Problem

    Existing localize-then-fix approaches can lack repository-level semantics and explicit planning between identifying suspicious code and generating a patch.

    Source: §1 · pp. 2–3

  • Method

    SGAgent coordinates localizer, suggester, and fixer agents, using repository knowledge tools and an intermediate suggestion stage to guide patch synthesis.

    Source: §1 · p. 3, §3.2 · p. 6

  • Results

    SGAgent reports 51.3% issue resolution on SWE-Bench-Lite, 60.7% with Claude-4, 65.4% on SWE-Bench-Verified, and 48.0% average resolution on combined VUL4J and VJBench.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The reported evaluations show SGAgent applied to repository-level repair and vulnerability repair across different base models and task settings.

    Source: §1 · p. 3

    The repair suggester assumes provided bug locations are valid and does not question or validate their correctness.

    Source: §3.5.2 · p. 12

Abstract

from arXiv · show

Large Language Models (LLMs) have enabled intelligent agents that autonomously interact with environments and invoke external tools. Recently, agent-based software repair has drawn wide attention, as repair agents can localize bugs, generate patches, and achieve state-of-the-art performance on repository-level benchmarks (e.g., SWE-Bench). However, existing approaches usually adopt a localize-then-fix paradigm, jumping directly from "where the bug is" to "how to fix it", leaving a fundamental reasoning gap. To this end, we propose SGAgent, a Suggestion-Guided multi-Agent framework for repository-level software repair, which follows a localize-suggest-fix paradigm. SGAgent introduces a suggestion phase to strengthen the transition from localization to repair: the suggester starts from the buggy locations, incrementally retrieves relevant context until it fully understands the bug, and provides actionable repair suggestions. We further construct a Knowledge Graph (KG) from the target repository and develop a KG-based toolkit to strengthen SGAgent's global contextual awareness and repository-level reasoning. Three specialized sub-agents (i.e., localizer, suggester, and fixer) collaborate to achieve automated end-to-end software repair. We evaluate SGAgent on SWE-Bench-Lite. SGAgent with Claude-3.5 achieves 51.3% repair accuracy, 81.2% file-level, and 52.4% function-level localization accuracy at an average cost of $1.48 per instance, outperforming all baselines using the same base model. SGAgent also generalizes well across base LLMs, reaching a 60.7% resolution rate with Claude-4. When extended to vulnerability repair, it achieves 48.0% on VUL4J and VJBench, demonstrating strong generalization across tasks and programming languages.

1 Introduction

Repository-level repair requires reasoning across code beyond the suspicious location, but existing approaches often move directly from localization to patch generation. SGAgent inserts a suggestion stage and repository knowledge tools into that transition, and reports results across repository and vulnerability repair benchmarks.

1 Introduction

  • The introduction describes agentic repair as using external tools for iterative planning and codebase interaction, alongside procedural pipelines that sequence repair stages.SWE-Bench evaluates issue resolution with full repository access, and SWE-Bench Lite is its curated lightweight subset.Source: §1 · p. 2
  • Existing localize-then-fix approaches can lack repository semantics and intermediate planning, while inaccurate localization may propagate errors into patch generation.Patches may overfit suspicious snippets, prompting redundant exploration and superficial fixes.Source: §1 · pp. 2–3
  • SGAgent assigns localization, repository-aware repair suggestion, and patch synthesis to three collaborating agents, making suggestions an explicit step between locating and fixing bugs.The suggester reinterprets localization outputs before patching.Source: §1 · p. 3
  • Its repository knowledge graph and 14-tool API toolkit support context retrieval during localization and help the suggester trace related code and refine repair objectives.The graph-and-toolkit contribution is also reported to improve overall performance by 3.1%.Source: §1 · p. 3
  • SGAgent reports 60.7% resolution with Claude-4, 65.4% on SWE-Bench-Verified, and 48.0% average resolution on combined VUL4J and VJBench.These evaluations address repository-level repair and security-related repair scenarios.Source: §1 · p. 3
  • Adding repair suggestions resolves 13.3% more SWE-Bench-Lite instances than the localize-then-fix variant.The paper also situates repository-level repair against function- or file-level settings that miss repository dependencies and contextual complexity.Source: §1 · pp. 2–3
  • 51.3% issue resolution on SWE-Bench-Lite and 81.2% file-level and 52.4% function-level localization accuracy are reported for SGAgent, at $1.48 per instance.The paper reports that it outperforms baselines under the same LLM setting.Source: §1 · p. 4

2 Motivation

The motivation example contrasts direct patching after localization with a suggestion-guided repair. The suggestion identifies a missing cross-function parameter transfer needed for a semantically complete fix.

2 Motivation

  • Without suggestions, the patch changes a condition but omits three conflict-handling parameters in the _insert() call, so it fails the benchmark tests.The omitted parameters are on_conflict, update_fields, and unique_fields.Source: §2 · pp. 4–5
  • With suggestions, the repair identifies the _batched_insert()–_insert() call relationship and passes the three parameters needed for ON CONFLICT DO UPDATE behavior.The resulting patch correctly returns primary-key IDs when update_conflicts=True.Source: §2 · p. 5
  • The example illustrates how suggestions can add cross-function semantic context, where direct patching risks a superficially correct but structurally incomplete change.The authors characterize the suggester as guiding the repair model's fixing process.Source: §2 · p. 5

3 Methodology

The section formalizes the localize-then-fix pipeline and SGAgent’s locate-suggest-fix extension. The new suggester turns identified locations into repair suggestions that accompany the locations into patch generation.

3.1 Problem Statement

  • In localize-then-fix, a localization model identifies bug locations from the bug description and codebase, then a repair model uses those inputs and locations to generate a patch.The formulation denotes the bug as B, codebase as C, locations as L, and patch as P.Source: §3.1 · p. 5
  • SGAgent inserts a suggest phase after localization to generate repair suggestions that help the repair model understand the bug and refine its objective.This changes the pipeline from localize-then-fix to locate-suggest-fix.Source: §3.1 · p. 5
  • The suggester generates a suggestion from the identified location, bug, and codebase, and that suggestion joins the location as input to the repair phase.The suggester is parameterized by γ in the locate-suggest-fix definition.Source: §3.1 · p. 5

3.2 Overview

SGAgent takes a GitHub issue and its codebase and returns a validated patch through a repository graph, a graph-driven retrieval toolkit, and three collaborating agents. The agents locate, suggest, and repair in sequence.

3.2 Overview

  • The repository knowledge graph captures structural and semantic information using three entity types and seven relationship types.It serves as the framework’s knowledge base.Source: §3.2 · p. 6
  • The knowledge-graph toolkit provides retrieval mechanisms for different repair scenarios, with contextual cues to guide retrieval.Agents select tools according to task context.Source: §3.2 · p. 6
  • The localizer retrieves context and proposes candidate locations, the suggester retrieves further context and refines repair objectives, and the fixer generates and ranks patches.The workflow is organized into Locate, Suggest, and Fix phases.Source: §3.2 · p. 7

3.3 Knowledge Graph Construction

SGAgent builds a repair-oriented knowledge graph from repository code, combining structural information with cross-file semantic dependencies to support repository-level understanding and retrieval.

3.3 Knowledge Graph Construction

  • Repository files and their contents are parsed into a graph of nodes and edges, with optional Neo4J storage for querying and visualization.The graph is represented as triples in which entities are nodes and relations are edges.Source: §3.3 · p. 7
  • Tree-sitter extracts tags for each file, with Pygments supplementing definitions when tag parsing fails.ASTs support entity extraction but are not themselves the final graph structure.Source: §3.3 · p. 7
  • File-structure analysis captures class and method containment and variable ownership as the graph’s structural backbone.The graph defines classes, methods, and variables as its three entity categories.Source: §3.3 · p. 7
  • The graph combines AST-based entity extraction with static analysis to connect usage sites, such as function calls, to definitions across files.These cross-file relations let agents trace dependency propagation beyond local syntax.Source: §3.3 · p. 7
  • The graph is designed around relations actionable for repository repair, including containment, invocation, symbol reference, inheritance, and import dependencies.It is not intended to encode every possible form of program semantics.Source: §3.3 · p. 7

3.4 KG-based Toolkit Implementation

SGAgent provides a KG-based toolkit of lightweight, composable retrieval tools organized around common repository-debugging activities, leaving tool selection and sequencing to the agent.

3.4 KG-based Toolkit Implementation

  • The toolkit offers preconfigured, context-aware retrieval utilities for multiple scenarios, with tool functionality summarized in Table 3.Agents choose tools according to the task context.Source: §3.4 · p. 9, Table 3 · p. 9
  • The toolkit groups repository interaction into structure analysis, entity analysis, content search, and filesystem tools.This organization follows a debugging workflow from inspecting repository structure to examining file contents or system state.Source: §3.4 · p. 9
  • Its tools are lightweight, composable primitives, while the agent decides which to invoke, in what order, and how to combine their outputs.This design avoids embedding complex reasoning in a single tool.Source: §3.4 · p. 9
  • Structure tools expose file hierarchies and entity dependencies, while entity tools retrieve classes, methods, variables, and import relationships.Examples include analyzing a Python file’s classes and methods and extracting import statements.Source: §3.4 · p. 9
  • Content-search tools retrieve contextual matches, and filesystem tools support directory traversal, file reading, and command-line execution.The search example returns three lines before and after each keyword match.Source: §3.4 · p. 9

3.5 Multi-Agent Framework

SGAgent divides repository repair among a localizer, suggester, and fixer, supported by a shared toolkit and dynamic memory summarization. The stages pass forward increasingly repair-oriented context, from candidate locations to validated patches.

3.5 Multi-Agent Framework

  • The three specialized agents share a toolkit but pursue distinct stages; the multi-agent design separates task boundaries and limits each stage’s context to condensed outputs.The authors motivate this separation as a way to reduce context dilution and avoid premature patch generation.Source: §3.5 · p. 10

3.5.1 Fault Localizer Agent.

  • The localizer iteratively retrieves graph context and outputs up to five candidate bug locations with a summary for the suggester.Those locations are preliminary candidates, not final repair targets.Source: §3.5.1 · p. 10

3.5.2 Repair suggester Agent.

  • Starting from candidate locations, the suggester expands context until it judges the information sufficient, then sends repair suggestions and an updated context summary to the fixer.The suggester treats the localizer’s locations as given rather than validating or replacing them.Source: §3.5.2 · p. 11
  • The suggester analyzes how candidate locations interact and constrains proposed fixes to remain coordinated and consistent with project framework patterns.The prompt structures this reasoning into interconnection analysis, coordination, and framework preservation.Source: §3.5.2 · pp. 12–13

3.5.3 Issue Fixer Agent.

  • The fixer synthesizes locations, suggestions, and repository context into multiple patch candidates, then validates and selects a final patch.Validation ranks patches by regression tests, then reproduction tests, and finally applies majority voting among normalized remaining patches.Source: §3.5.3 · p. 13
  • The fixer grounds suggestions in repository state, may retrieve additional graph context, and checks precise code details before translating plans into executable changes.Its retrieval is restricted to code-level alignment rather than searching for new bug locations or repair strategies.Source: §3.5.3 · p. 13
  • The fixer prompt requires holistic root-cause analysis, system-aware edits across connected locations, and explicit code-quality and framework-compatibility checks.The fixer is also instructed to explain why the solution addresses the issue and confirm that the patch is expected to pass tests.Source: §3.5.3 · pp. 13–14

3.5.4 Dynamic Memory Summarization.

  • When an agent’s message window exceeds a token threshold, SGAgent summarizes the conversation and retains recent messages while preserving tool-interaction pairs.Existing summaries are integrated into new ones to support continuity while older messages are discarded.Source: §3.5.4 · pp. 14–15

4 Experimental Setup

The experimental setup evaluates SGAgent against repair-system baselines, using SWE-Bench-Lite and metrics for repair, localization, and cost. It also specifies implementation settings for alternative base models and patch sampling.

4.1 Research Questions

  • The study asks how SGAgent compares with baselines, how its components contribute, how it performs across base models, and whether it extends to vulnerability repair.These are the four research questions guiding the evaluation.Source: §4.1 · p. 15

4.2 Benchmark

  • SWE-Bench-Lite supplies 300 reproducible real-world bug instances from twelve actively maintained GitHub Python projects.Each instance includes a repository snapshot, executable test suite, and natural-language issue description.Source: §4.2 · p. 15

4.3 Baselines

  • The baseline comparison includes AutoCodeRover, SWE-Agent, DARS-Agent, OpenHands, Agentless, KGCompass, and ExpeRepair.Table 4 compares SGAgent with representative prior approaches and indicates whether they introduce an explicit intermediate stage between localization and patch generation.Source: §4.3 · p. 15, Table 4 · p. 16

4.4 Evaluation Metrics

  • Evaluation measures issue resolution, file- and function-level localization accuracy, and average cost per bug.Resolution is based on passing all benchmark tests; file accuracy uses average Jaccard similarity between predicted and ground-truth file sets.Source: §4.4 · p. 16

4.5 Implementation Details

  • SGAgent is primarily evaluated with Claude-3.5-Sonnet, while DeepSeek-V3 [9] and Qwen3-235B-A22B [59] assess the framework’s model-agnostic capability, with all models using default hyperparameters.For each issue, four localization sets are sampled, each with up to five candidate bug locations; for each location, patches are sampled once at temperature 0 and nine times at 0.8.Source: §4.5 · pp. 16–17

5 Results and Analysis

SGAgent is evaluated against repair baselines, through component ablations and across base models, on repository and vulnerability repair tasks. Results report repair and localization performance, the effects of individual components, and performance across models and datasets.

5.1.2 Results.

  • SGAgent achieves 51.3% resolution, 81.2% file-level and 52.4% function-level localization accuracy, at $1.48 per instance, outperforming same-model baselines.It ranks fifth overall in resolution and third-lowest in cost among evaluated methods.Source: §5.1.2 · p. 18
  • Against five Claude-3.5-Sonnet baselines, SGAgent uniquely resolves 37, 46, 40, 47, and 43 instances, while they uniquely fix 13, 16, 26, 29, and 32 instances it misses.The comparisons are with DARS-Agent, KGCompass, Lingxi, OpenHands, and Agentless-1.5, respectively.Source: §5.1.2 · p. 18

5.2.2 Results.

  • Majority voting yields a 40.33% resolve rate versus 37.67% for greedy sampling; adding regression and reproduction tests raises it to 44.67% and 51.33%.The authors report improvements of 6.6%, 4.4%, and 2.6% from majority voting, regression tests, and reproduction tests, respectively.Source: Table 8 · p. 19, §5.2.2 · pp. 19–20
  • Removing the Knowledge Graph reduces resolution by 5.3% and localization accuracy by 3.1% (file-level) and 4.0% (function-level); removing Suggest drops resolution by 13.3%.Without Suggest, file-level and function-level localization accuracy decline by 0.9% and 0.5%, respectively.Source: §5.2.2 · pp. 19–20

5.3.2 Results.

  • Claude-4 leads at 60.7% resolution, 88.7% file-level, and 57.2% function-level accuracy; resolution varies more across models than localization, while Suggest and Fix depend more on model capability.Claude-4 surpasses the Claude-4-based KGCompass (58.3%) and ExpeRepair-v1.0 (60.3%) baselines in resolution rate.Source: §5.3.2 · pp. 20–21, Table 9 · p. 20

5.4.1 Design.

  • SGAgent-Vul is tested on VUL4J and VJBench with failure-test feedback for repair, escalation from Fixer to Suggester and Localizer after repeated failures, and a 100-iteration cap.The added failure-guided mechanism motivates using GPT-4o as a more cost-efficient base model than Claude-3.5-Sonnet.Source: §5.4.1 · p. 21

5.4.2 Results.

  • SGAgent-Vul leads both benchmarks, resolving 17 vulnerabilities on VUL4J and 7 on VJBench; it beats specialized repair baselines on both, and SWE-Agent (11) and OpenHands (14) on VUL4J.The evaluations cover 35 VUL4J and 15 VJBench vulnerabilities.Source: §5.4.2 · pp. 21–22
  • Removing Suggest or the Knowledge Graph lowers repair rate from 48.0% to 26.0% or 22.0%; DeepSeek-V3 reaches 52.0%, ahead of GPT-4o at 48.0% and Qwen3 at 40.0%.Across the two repair domains, the Knowledge Graph contributes relatively more to vulnerability repair, while Suggest contributes more to software repair.Source: Table 11 · p. 22, §5.4.2 · p. 22

6 Discussion

SGAgent resolves 65.4% of SWE-Bench-Verified instances, outperforming the compared approaches. A case study contrasts a failing localized patch with a suggester-guided repair, while tool logs show broad use of the repository toolkit.

6.1 Comparison on SWE-Bench-Verified

  • 65.4% of SWE-Bench-Verified instances were resolved by SGAgent, the best performance among compared approaches using Claude-3.5-Sonnet.It resolved 327 of 500 instances, ahead of EPAM AI (62.8%), AutoCodeRover-v2.1 (51.6%), Agentless-1.5 (50.8%), and SWE-Agent (33.6%).Source: §6.1 · p. 23

6.2 Case study: The Role of the Suggester

  • Without the suggester, a hardcoded list check in Request._init_() failed validation tests and missed standalone register_hook calls and tuple inputs.The baseline followed the issue’s initialization-phase clue, producing a plausible but incomplete patch.Source: §6.2 · p. 24, Figure 4 · p. 24
  • The suggester redirected the fix to register_hook, where the fixer produced a test-passing patch supporting lists and tuples while maintaining backward compatibility.The suggester’s plan called for handling single hooks and lists or tuples through type checking and extending or appending as appropriate.Source: §6.2 · pp. 24–25, Figure 4 · p. 24

6.3 Tool Usage Analysis

  • SGAgent made 8,886 toolkit calls; read_file_lines was used 4,512 times, while semantic exploration and validation tools also saw substantial use.All 14 designed tools were invoked at least once; finer-grained entity-analysis tools appeared in the long tail.Source: §6.3 · pp. 25–26, Table 13 · p. 26

7 Related Work

Prior LLM repair work spans prompting, retrieval, interactive agents, fine-tuning, and reinforcement learning. SGAgent distinguishes itself with an explicit suggestion stage and a dynamic toolkit for repository knowledge.

7.1 LLM-based Automated Program Repair

  • Prior LLM repair work spans prompting and retrieval, interactive agents, fine-tuning, and reinforcement learning; SGAgent instead adopts a train-free multi-agent framework.The paper presents the train-free design as a way to minimize training cost and methodological complexity.Source: §7.1 · pp. 26–27

7.2 Repository-level Software Repair

  • Unlike approaches that couple reasoning to patch generation, SGAgent makes the Suggester an intermediate stage for secondary context exploration and dependency analysis before edits.It turns coarse localization signals into actionable, repository-aware repair suggestions.Source: §7.2 · pp. 27–28

7.3 Knowledge-Graph-Guided Repository Retrieval

  • Unlike static graph context such as KGCompass’s precomputed paths, SGAgent’s 14-tool API lets agents retrieve repository knowledge dynamically according to task context.The paper characterizes this as fine-grained, scenario-specific interaction with repository knowledge.Source: §7.3 · p. 28

8 Threats to Validity

The authors identify stochastic model outputs, dataset and language coverage, and base-model dependence as validity concerns. They describe mitigation or additional evaluations for each, while noting that the primary experiments use Python projects.

8.1 Internal Validity

  • To address stochastic outputs, the authors sample multiple localization sets and patches per instance and use fixed seeds and isolated environments.Each instance receives four localization sets; patches are sampled once at temperature 0 and four times at temperature 0.8 for each localization.Source: §8.1 · p. 28

8.2 External Validity

  • Because the primary experiments use Python projects from SWE-Bench-Lite, the authors also evaluate Java vulnerability datasets and report consistent repair rates and localization accuracies.They present these results as evidence of cross-language generalizability.Source: §8.2 · p. 29

8.3 Construct Validity

  • Claude-3.5-Sonnet is SGAgent’s primary base model, so the authors also test DeepSeek-V3 and Qwen3-235B-A22B and report consistent performance trends.They interpret the consistency as support for framework-driven rather than model-specific improvements.Source: §8.3 · p. 29

9 Conclusion

SGAgent combines three specialized agents with a repository knowledge graph and retrieval toolkit for repository-level repair. The authors report strong benchmark performance, cross-model scalability, and performance across programming languages.

9 Conclusion

  • SGAgent coordinates a localizer, suggester, and fixer, adding secondary contextual refinement between localization and patch generation.A repository-level knowledge graph and retrieval toolkit support reasoning about code structure and semantic relationships.Source: §9 · p. 29
  • The authors report that SGAgent outperforms state-of-the-art approaches on resolution rate and file- and function-level localization accuracy while maintaining relatively low cost.The conclusion does not give metric values for these claims.Source: §9 · p. 29
  • The authors report scalability across LLM architectures and robust performance across programming languages, presenting these as support for practical applicability.Source: §9 · p. 29
Loading 2602.23647v2…