PaperlayerView exact revision on arXiv

Source-linked AI summary

SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution

Han Li, Yuling Shi, Shaoxin Lin, Xiaodong Gu, Heng Lian, Xin Wang, Yantao Jia, Tao Huang, Qianxiang Wang

arXiv:2507.23348v1cs.SEcs.CLcs.LG

TL;DR

Agent-based issue resolution can miss patterns spanning complex codebases because agents independently explore limited local regions. SWE-Debate generates dependency-graph fault traces, debates competing reasoning paths, and feeds a consolidated plan into MCTS; it reports state-of-the-art results with 6.7% higher issue resolution and 5.1% higher fault localization accuracy than baselines.

  • Problem

    Independent agent exploration can become stuck in local solutions and fail to identify issue patterns spanning large, complex codebases.

  • Method

    SWE-Debate generates multiple fault propagation traces from a code dependency graph, debates competing localization and modification plans, and uses the consolidated plan for MCTS-based patch generation.

  • Results

    6.7% improvement in issue resolution rate and 5.1% improvement in fault localization accuracy are reported over state-of-the-art baselines.

  • Takeaways & Limitations

    The framework combines graph-based trace generation with structured debate to evaluate competing localization hypotheses and address single-agent perspective limitations.

  • Takeaways & Limitations

    The evaluation is restricted to the open-source DeepSeek-V3-0324 model and a subset of SWE-Bench-Verified, limiting validation across model architectures and larger datasets.

Abstract

from arXiv · show

Issue resolution has made remarkable progress thanks to the advanced reasoning capabilities of large language models (LLMs). Recently, agent-based frameworks such as SWE-agent have further advanced this progress by enabling autonomous, tool-using agents to tackle complex software engineering tasks. While existing agent-based issue resolution approaches are primarily based on agents' independent explorations, they often get stuck in local solutions and fail to identify issue patterns that span across different parts of the codebase. To address this limitation, we propose SWE-Debate, a competitive multi-agent debate framework that encourages diverse reasoning paths and achieves more consolidated issue localization. SWE-Debate first creates multiple fault propagation traces as localization proposals by traversing a code dependency graph. Then, it organizes a three-round debate among specialized agents, each embodying distinct reasoning perspectives along the fault propagation trace. This structured competition enables agents to collaboratively converge on a consolidated fix plan. Finally, this consolidated fix plan is integrated into an MCTS-based code modification agent for patch generation. Experiments on the SWE-bench benchmark show that SWE-Debate achieves new state-of-the-art results in open-source agent frameworks and outperforms baselines by a large margin.

1 INTRODUCTION

Repository-level issue resolution requires fault localization across complex code structures, but independent agents can become trapped in local solutions. SWE-Debate addresses this with graph-guided traces and competitive multi-agent reasoning, reporting improvements in resolution and localization.

  • Motivation: Repository-level issue resolution must localize and modify defective code based on issue reports, requiring reasoning across structural and semantic code properties.The task involves understanding complex dependency graphs and software architecture.
  • Motivation: Independent agent exploration often gets stuck in local solutions and misses issue patterns spanning large, complex codebases.This limited observation scope can lead to inefficient trial-and-error.
  • Approach: SWE-Debate generates multiple fault propagation traces through code dependencies and uses competitive debate to consolidate fault localization.Its pipeline reframes issue resolution through graph-guided localization and structured debate.
  • Results: SWE-Debate achieves new state-of-the-art results under open-source agent frameworks and outperforms baseline methods by a large margin.The evaluation is conducted on SWE-Bench-Verified, and ablations identify multiple chain generation as the largest contributor.
  • Results: 6.7% improvement in issue resolution rate and 5.1% improvement in fault localization accuracy are reported for the competitive debate paradigms.These figures are listed among the paper’s main contributions.

2 MOTIVATION

Single-agent exploration struggles to compare competing fault locations and modification strategies in complex repositories. The Django-11999 example illustrates how debate exposes a structural fix that preserves user-defined methods.

  • 2 MOTIVATION: Complex repository issues become difficult when multiple code locations appear relevant and require architectural comparison of competing modification plans.Individual agents can handle straightforward localization but face perspective limitations on ambiguous issues.
  • 2 MOTIVATION: In the Django-11999 example, isolated exploration focuses on _get_FIELD_display without systematically comparing the base method and field registration mechanisms.The example frames the limited observation scope problem.
  • 2 MOTIVATION: Multi-agent debate compares runtime flexibility with source-level prevention by making agents defend competing localization and modification strategies.This exposes architectural trade-offs and maintainability implications.
  • 2 MOTIVATION: The contribute_to_class approach prevents method overwriting at the source, requires a minimal existence check, and preserves backward compatibility.The described change uses if not hasattr(cls,method_name) around lines 765-767.

3 METHODOLOGY

SWE-Debate combines dependency-graph traversal, competitive debate over localization and modification plans, and MCTS-based patch generation. The pipeline uses structural traces to guide agents toward a consolidated fix plan.

  • Graph-guided localization: Existing exploration overlooks structural relationships between code entities, whereas SWE-Debate reasons over a dependency graph G=(V,E).Localization chains trace code entities along fault propagation paths.
  • Graph-guided localization: The framework uses graph traversal to generate multiple candidate fault propagation traces from semantically matched entry entities.Depth-limited traversal selects promising entities using semantic similarity and structural importance.
  • Graph-guided localization: These localization chains represent plausible propagation paths through calls, inheritance, and data flow, reducing the exploration required by search-based methods.The chains provide structured proposals for the subsequent debate.
  • Multi-agent debate: Competitive debate selects a localization chain and converts it into a modification plan specifying code locations, modification types, and implementation priorities.Agents review and defend alternative proposals before producing the consolidated plan.
  • Patch generation: The final MCTS-based repair stage starts from the debate’s structured plan and iteratively explores, refines, and evaluates code modifications.Its search tree represents codebase states and actions such as Search, Plan, and Edit.

4 EXPERIMENTAL SETUP

The evaluation asks whether SWE-Debate improves issue resolution and fault localization, how its components contribute, and how chain depth affects performance. Experiments use SWE-Bench variants, multiple baselines, and defined resolution and localization metrics.

  • Research questions: The study evaluates repository-level issue resolution, component contributions, fault localization against baselines, and the effect of chain depth.These four research questions structure the experimental analysis.
  • Datasets: Evaluation uses SWE-Bench-Verified with 500 verified issues and SWE-bench-Lite with 300 selected tasks.Both datasets are drawn from SWE-bench.
  • Baselines: Issue-resolution comparisons include Agentless, AutoCodeRover, SWE-Agent, SWE-Search, and SWESynInfer.The baselines represent non-agentic pipelines, code-search systems, repository agents, MCTS exploration, and development-process-centric models.
  • Baselines: Fault-localization comparisons include CodeActAgent, LocAgent, and KGComposs.These baselines use environment search, graph-guided reasoning, or repository-aware knowledge graphs.
  • Metrics: Pass@1 measures first-attempt issue resolution, while Acc@1 (File) measures top-1 file-level localization requiring all required modification paths.The metrics are intended to capture practical patch success and localization accuracy.
  • Implementation: SWE-Debate extends SWE-Search with graph-based localization and multi-agent debate, using static analysis and prompted DeepSeek-V3-0324 agents.The implementation sets graph and debate parameters using a held-out set.

5 RESULTS

SWE-Debate outperforms baselines on repository-level issue resolution and fault localization, with ablations identifying multiple chain generation as its largest contributor. Chain-depth analysis finds that depth 5 best balances reasoning depth and relevance.

  • RQ1: Effectiveness on Issue Resolution: 41.4% Pass@1 solves 207 of 500 SWE-Bench-Verified problems, improving 6.0% over SWE-Search and 2.6% over SWE-Agent with the same model.Both comparisons use DeepSeek-V3-0324; SWE-Debate rises from 35.4% and 38.8%, respectively, to 41.4%.
  • RQ2: Ablation Study: Removing multiple chain generation reduces Pass@1 to 31.4%, a 10.0 percentage point degradation and the largest ablation drop.The resulting single-path exploration frequently misses dependency relationships spanning multiple files or modules.
  • RQ2: Ablation Study: Removing edit plan generation lowers Pass@1 from 41.4% to 35.4%, a 6.0 percentage point drop.Without structured plans, the downstream MCTS-based editing agent lacks strategic direction and explores less effectively.
  • RQ2: Ablation Study: Removing multi-agent debate lowers performance to 37.2%, a 4.2 percentage point drop associated with unresolved competing fix locations.Individual exploration can remain stuck in local solutions when multiple plausible locations exist.
  • RQ3: Localization Performance Comparison: 81.67% file-level localization accuracy surpasses SWE-Agent by 14.67% and LocAgent by 3.93% on SWE-Bench-Lite.Against SWE-Agent with the same DeepSeek-V3-0324 model, accuracy improves from 67.00% to 81.67%; LocAgent reaches 77.74%.
  • RQ4: Impact of the Chain Depth: A chain depth of 5 achieves the highest localization accuracy, reaching 86.7% Acc@1(File), while deeper chains yield diminishing returns or slight degradation.The reported trade-off reflects deeper exploration of code dependencies versus distracting information that reduces debate focus.

6 DISCUSSION

SWE-Debate improves fault localization through graph-based trace generation and competitive debate, while integrating with existing issue-resolution systems. Its main limitations concern scalability, static analysis coverage, and simulated reasoning diversity.

  • 6.1 Strengths: 81.67% file-level accuracy demonstrates the benefit of graph-based localization and multiple fault propagation traces.The method builds dependency graphs from code structure to generate candidate traces.
  • 6.1 Strengths: 14.67% improvement over SWE-Agent with DeepSeek-V3-0324 and 8.31% over SWE-Search with GPT-4o accompany 41.4% Pass@1.The strongest-baseline comparison is a 3.93 percentage point improvement across configurations.
  • 6.1 Strengths: Removing competitive debate causes a 4.2 percentage point drop in resolution rate, supporting its role in evaluating competing fix strategies.The debate uses independent analysis, competitive refinement, and final selection.
  • 6.1 Strengths: SWE-Debate can improve localization modules in systems such as SWE-Search and Agentless without major architectural modifications.The design is described as plug-and-play for downstream repair workflows.
  • 6.2 Limitations and Future Work: Graph construction can be computationally expensive, static analysis may miss runtime relationships, and single-model prompting may limit reasoning diversity.Suggested directions include incremental analysis, heterogeneous models, and domain-specific knowledge bases.

7 THREATS TO VALIDITY

The validity threats concern possible pre-training contamination, restricted experimental scope, and evaluation on one Python-only dataset. These constraints limit confidence in memorization effects and broader generalizability.

  • Internal Threats: DeepSeek-V3-0324 may contain SWE-Bench repositories in pre-training, creating a potential data-contamination threat.The authors plan contamination-free evaluation to further validate the findings.
  • Internal Threats: Evaluation uses only the open-source DeepSeek-V3-0324 model and a subset of SWE-Bench-Verified, limiting validation across model architectures.The authors attribute this scope to time and budget constraints.
  • External Threats: SWE-Bench-Verified contains Python repositories only, so results may not generalize to other programming languages or software domains.The authors identify Multi-SWE-Bench as a future evaluation target.

8 RELATED WORK

Related work spans traditional analysis, LLM-based localization, agent-based systems, and collaborative multi-agent methods. SWE-Debate targets the limited observation scope and modification ambiguity that remain in these approaches.

  • Traditional Fault Localization: Traditional SBFL, MBFL, and learning-based localization rely on test execution and program analysis but face repository-level limitations.These methods require comprehensive test suites and struggle with complex dependency relations.
  • LLM-based Fault Localization: LLM-based methods use code understanding, repository navigation, static analysis, graph representations, scheduling, and context pruning to improve localization.Examples include RCAgent, AgentFL, LocAgent, OrcaLoca, and CoSIL.
  • Automated Repository Issue Resolution: Agent-based systems model issue resolution as sequential interaction with code environments, while pipeline methods organize repository analysis, localization, and patch generation.SWE-Agent, AutoCodeRover, SWE-Search, and CodeR illustrate agent-based designs.
  • Limitations of Existing Approaches: Existing methods often get stuck in local solutions because they lack diverse perspectives for evaluating competing modification plans.This limited observation scope is the localization bottleneck addressed by SWE-Debate.
  • Multi-Agent Systems and Debate: Collaborative multi-agent architectures emphasize consensus and information sharing, whereas debate systems evaluate competing alternatives through structured argumentation.The related work identifies thought degeneration and resistance to modification as limitations of collaborative approaches.

9 CONCLUSION

SWE-Debate combines graph-based fault propagation traces with structured competitive debate to address limited observation scope in repository-level issue resolution. On SWE-Bench-Verified, it improves issue resolution and fault localization over state-of-the-art baselines.

  • 9 CONCLUSION: SWE-Debate combines graph-based fault propagation trace generation with structured debate to evaluate competing localization hypotheses.The framework targets perspective limitations in single-agent methods.

A SWE-BENCH-VERIFIED-S

This section describes SWE-Bench-Verified-S and identifies related benchmark and MCTS-hyperparameter tables.

  • A SWE-BENCH-VERIFIED-S: SWE-Bench-verified-mini4 contains 50 datapoints, compared with 500 in SWE-Bench-Verified, and requires 5GB rather than 130GB of storage.It maintains similar performance, test pass-rate, and task-difficulty distributions to the original dataset.
  • A SWE-BENCH-VERIFIED-S: SWE-Bench-Verified-S augments SWE-Bench-verified-mini with 25 additional instances to better approximate the full dataset’s distribution and performance characteristics.
  • A SWE-BENCH-VERIFIED-S: Table 4 lists instance IDs included in SWE-Bench-Verified-S.
  • A SWE-BENCH-VERIFIED-S: Table 5 presents the hyperparameters used for MCTS.

B HYPERPARAMETERS OF MCTS

The section introduces the MCTS hyperparameters and notes that the workflow’s prompts are enumerated separately.

  • B HYPERPARAMETERS OF MCTS: The MCTS algorithm used in the study employs several hyperparameters.
  • B HYPERPARAMETERS OF MCTS: The supplied passage does not specify the individual MCTS hyperparameter values.
  • B HYPERPARAMETERS OF MCTS: The paper enumerates prompts used throughout the workflow, from initial entity extraction to final plan generation.

Prompt 1: INITIAL ENTITY EXTRACTION PROMPT

The prompts guide entity extraction and graph navigation by emphasizing issue relevance, structural diversity, explicit identifiers, and strategic selection of dependency-graph nodes.

  • Prompt 1: INITIAL ENTITY EXTRACTION PROMPT: The initial prompt asks an analysis expert to identify relevant code entities from an issue description.Target categories include classes, methods, functions, variables, and error types or exceptions.
  • Prompt 1: INITIAL ENTITY EXTRACTION PROMPT: Entity extraction prioritizes directly mentioned or strongly implied names, central components, precise terminology, and a fixed maximum number of entities.The output contains names only, without paths, modules, or extra descriptions.
  • Prompt 1: INITIAL ENTITY EXTRACTION PROMPT: The prompt examples extract issue-specific entities such as Count, DISTINCTCASE, distinct, UserSession, Redis, calculate_discount, and customer_type.The examples demonstrate returning exactly the requested number of simple entity names in relevance order.
  • Prompt 2: CODE SNIPPET ENTITY EXTRACTION PROMPT: A second extraction prompt selects four relevant files, classes, or functions from a problem statement and code snippets.Each selected entity includes an identifier, type, relevance reason, and diversity value.
  • Prompt 2: CODE SNIPPET ENTITY EXTRACTION PROMPT: Code-snippet entity selection prioritizes relevance and diversity across files or modules, avoiding redundant entities from one file when possible.
  • Prompt 3: NEIGHBOR PREFILTERING PROMPT: The neighbor-prefiltering prompt selects up to a maximum number of promising dependency-graph neighbors for further exploration.Criteria include issue relevance, diversity, strategic value for finding root causes or solutions, and variety across entity types.
  • Prompt 3: NEIGHBOR PREFILTERING PROMPT: Neighbor selection uses the issue description, current entity and type, traversal depth, and available neighbor list as inputs.The output is a JSON object containing selected neighbor IDs, selection reasoning, and diversity considerations.
  • Prompt 4: NODE SELECTION PROMPT: The node-selection prompt decides whether graph exploration should continue and which neighboring node should be explored next.It considers issue relevance, traversal depth, and the likelihood that a neighbor contains code relevant to the solution.
Loading 2507.23348v1…