Source-linked AI summary
XAgent: eXecution-guided Agentic AI for Effective Localization and Resolution of GitHub Issues
Hieu Huynh, Patanamon Thongtanunam, Michael Fu, Bach Le, Kla Tantithamthavorn
TL;DR
Existing agentic approaches rely heavily on static issue descriptions, leaving localization and validation incomplete for repository-level GitHub issues. XAgent uses dynamic execution behavior and broader program context to localize bugs and validate patches, achieving state-of-the-art SWE-bench-lite performance with lower cost. Its effectiveness depends on generating reproduction-script pairs, which succeeds for 93% of evaluated issues.
Problem
Static issue descriptions can cause incorrect localization and incomplete validation because they are noisy, incomplete, and narrower than the underlying program context.
Method
XAgent compares failing and passing executions to localize suspicious functions and augments validation tests using relevant program context.
Results
62.0% resolve rate and 72.8% function localization accuracy outperform existing approaches on SWE-bench-lite while reducing cost by 37%.
Takeaways & Limitations
XAgent supports a shift from static, description-oriented patch generation toward dynamic, execution-guided issue resolution for more robust software maintenance.
Takeaways & Limitations
XAgent relies on agent-generated reproduction scripts, which are produced for 93% of evaluated issues; the remaining 7% fall back to LLM localization.
Abstract
from arXiv · showhide
Agentic AI has enabled capabilities in leveraging Large Language Models (LLMs) to autonomously resolve repository-level GitHub issues. However, due to the reliance on limited static description of issues, existing agentic approaches suffer from incorrect localization and incomplete validation. Solely relying on this information can bias LLM reasoning toward the narrow scope of the issue description, leading to incomplete patches that fail to address the underlying issue. In this paper, we present XAgent, an execution-guided agentic framework that analyzes dynamic behavior and additional program context to localize and validate issues. The experimental results on the SWE-bench-lite dataset demonstrate that XAgent outperforms other existing approaches, achieving a resolve rate of 62.0% and a function localization accuracy of 72.8%, while maintaining cost efficiency. Our analysis further shows that XAgent successfully resolves 7 additional issues that the top existing baselines fail to address. This work highlights a shift from static, description-oriented patch generation toward dynamic execution-guided issue resolution, opening new opportunities for LLM-based coding agents to achieve more robust and generalizable software maintenance.
1 Introduction
Existing agentic approaches can mislocalize GitHub issues and incompletely validate patches when they rely mainly on static issue descriptions. XAgent addresses these limitations with execution-guided localization and context-aware test augmentation, achieving strong SWE-bench-lite results.
- Limitations of existing approaches: Static issue descriptions can bias localization toward irrelevant functions because issue descriptions are noisy or incomplete.Existing approaches commonly use keyword similarity between issue descriptions and repository files or functions.
- Limitations of existing approaches: Description-only validation can overfit generated tests to the issue's narrow scope, allowing patches to pass tests while missing the underlying issue.This incomplete validation contributes to overfitting patches and limits effectiveness in real-world scenarios.
- XAgent: XAgent compares failing and passing executions to localize suspicious functions and uses broader program context to generate edge-case and related-component tests.The framework shifts issue resolution from static description-oriented reasoning toward dynamic, execution-guided analysis.
- Empirical results: 62.0% resolve rate, 72.8% function-level localization accuracy, and 37% lower cost establish XAgent's leading SWE-bench-lite performance.XAgent also resolves 7 issues that top existing baselines fail to address.
- XAgent: The framework combines execution-guided localization with context-aware validation to address incorrect localization and incomplete validation.An ablation study reports up to 9% relative improvement in overall performance from these components together.
2 Background & Related Work
Repository-level issue resolution requires reproduction, localization, patch generation, and validation, but existing agentic methods rely heavily on limited issue descriptions and reproduction information. This can mislocalize bugs and produce patches that pass narrow tests without addressing underlying issues.
- Workflow: Existing repository-level workflows comprise reproduction, localization, patch generation, and validation of real-world bugs.LLM-based agents increasingly perform these stages through repository interaction and tool use.
- Localization: Issue descriptions and reproduction tests provide limited information that can bias LLMs toward failure scenarios and incorrect functions.Existing localization commonly extracts issue keywords and searches repository files and functions.
- Patch Generation & Validation: Validation tests derived from narrow issue descriptions can let patches pass while failing to address the underlying issue.This limitation motivates broader validation beyond the reported symptom.
3 Motivating Examples
Two motivating examples show how static issue-focused reasoning can mislocalize bugs and validate only surface fixes. Comparing execution behavior and testing related program components exposes deeper defects and broader correctness requirements.
- 3.1 Example 1: In Matplotlib-25433, existing methods localized functions named in the issue or reproduction, but the actual bug was deeper in delaxes.The issue concerned a RangeSlider callback that clears and redraws a figure, while existing approaches focused on on_changed or set_val.
- 3.1 Example 1: Comparing buggy and non-buggy traces identified 15 unique functions from traces containing 756 and 758 functions, including the developer-identified buggy functions.The differential comparison reduced the localization search space by focusing on functions appearing only in the buggy trace.
- Motivation: The examples motivate execution-guided localization and context-aware validation to address deeper defects and test whether fixes generalize beyond the reported symptom.The first idea contrasts failing and passing executions; the second broadens tests using program context.
- 3.2 Example 2: In Seaborn-2848, issue-focused validation concentrated on pairplot and missed related scatterplot behavior, causing patches to fail human-written tests.Shared logic among pairplot, scatterplot, and histplot motivates testing related plotting functions.
4 XAgent: eXecution-guided Agentic AI
XAgent combines failing-versus-passing execution analysis for localization with program-context-driven validation and iterative patch generation. Its workflow reproduces both buggy and non-buggy behavior, ranks suspicious functions, generates broader tests, and validates candidate patches.
- Overview: XAgent uses an execution-guided localizer and context-aware validation augmentor to overcome static localization and narrow validation limitations.The localizer compares failing and passing executions, while the augmentor generates tests from relevant program context.
- Reproduction: The reproduction phase generates and checks buggy and reference scripts before passing them to localization.The scripts represent unexpected and intended behavior, and are regenerated when execution results do not match the issue.
- Localization: Localization traces both executions as call trees and applies APTED to map their structural differences.Call-tree nodes represent functions and edges represent caller-callee relationships; APTED produces identical, deletion, and insertion mappings.
- Localization: XAgent prioritizes functions unique to the buggy execution, adds issue- or reproduction-highlighted functions, and ranks candidates by embedding similarity.The candidate list is ranked using UniXcoder similarity to the issue description and function representations.
- Patch Generation & Validation: The validation augmentor extracts relevant files and functions from the reproduction trajectory, then generates additional tests alongside reproduction and regression tests.The patch agent iteratively tests candidate patches until they pass the validation suite.
5 Experimental Setup
The evaluation uses SWE-bench-Lite to assess repository-level issue resolution, localization, component contributions, and performance across issue complexity. XAgent is compared with leading leaderboard methods under a common LLM backbone and constrained candidate set.
- Research Questions: The evaluation asks about localization and resolution effectiveness, component contributions, and how resolve rate changes with issue complexity.These correspond to RQ1, RQ2, and RQ3.
- Benchmark: The study evaluates XAgent on SWE-bench-Lite, a repository-level benchmark requiring understanding of code dependencies across projects.The benchmark differs from isolated-function code-generation datasets by targeting real-world software engineering issues.
- Experimental Configuration: XAgent uses Claude Sonnet 4, limits suspicious functions to K = 20, and compares against nine top-performing SWE-bench-Lite methods.The K = 20 threshold covers 80% of cases in the authors’ experiments while remaining manageable for the LLM.
6 Experimental Results
XAgent achieves strong SWE-bench-Lite localization and resolution performance while reducing cost, and its execution-guided components improve robustness on complex issues.
- 62% resolve rate and 72.8% function-level localization F1 make XAgent the strongest overall method on SWE-bench-Lite.It also achieves 86.6% file-level localization F1.
- XAgent reduces token usage by nearly 40% and overall cost by approximately $300 across all instances compared with prior approaches.Avoiding localization calls and multiple candidate patches contributes to the savings.
- RQ1: Localization and Resolution: XAgent uniquely identifies 11 perfectly localized instances, including 7 successes mainly attributed to differential analysis.It correctly localizes 181 instances, compared with 142 for Refact Agent.
- RQ2: Component Contributions: Execution-guided localization increases function-level F1 by 4.5% to 73.5% and reduces token use by 13% versus the vanilla agentic LLM.Adding the validation test augmentor raises resolution from 58.0% to 62.0%, resolving 12 additional issues.
- RQ2: Component Contributions: Semantic similarity provides the strongest ranking strategy, achieving the highest Top-10 and Top-20 localization accuracy among the evaluated rankers.Commit-history ranking often fails for older or long-stable code.
7 Discussion
The discussion examines XAgent’s search-space reduction, patch originality, and execution time, showing efficient localization alongside substantial agentic runtime.
- Search-Space Reduction: 52 functions on average and 11 at the median reduce the localization search space from 20,000 repository functions on average.The reduction uses functions uniquely executed in buggy rather than non-buggy runs.
- Patch Similarity: XAgent often generates functionally correct alternatives rather than exact copies of human patches, especially for complex multi-line or multi-function bugs.The analysis reports this as evidence against reliance on memorized ground-truth patches.
- Execution Time: 8.8 minutes is the average issue-resolution time, with reproduction and patch generation consuming 80.1% of total runtime.They take 181.8 seconds and 240.8 seconds, respectively.
- Execution Time: Execution-guided localization adds only 49 seconds, or 9.3% of total time, while test augmentation takes 56 seconds.The localization overhead is small relative to the two dominant agentic phases.
8 Limitations
XAgent’s effectiveness is evaluated primarily on SWE-bench-lite, despite dataset concerns and limited comparability with SWE-bench-verified. Its execution-guided approach depends on reproduction-script quality and includes a reproducibility trade-off from stochastic validation generation.
- Dataset scope: SWE-bench-lite may contain under-specified issues and flaky tests, but it is used for fair comparison because most baselines evaluate on it.The overlapping 93-instance subset with SWE-bench-verified provides evidence of comparable XAgent performance across the two datasets.
- Limitations: 93% of evaluated issues receive agent-generated buggy/non-buggy reproduction-script pairs, while 7% fall back to LLM localization.Under-specified issues or feature requests can prevent successful script generation.
- Reproducibility: Validation uses temperature 0.8 to generate diverse tests, introducing potential output randomness despite lower temperatures for reproduction and patch generation.The authors lower temperatures to 0 and 0.5 for the other stages to improve reproducibility.
9 Conclusion
XAgent combines dynamic program execution with context-aware test generation to improve repository-level GitHub issue resolution. On SWE-bench-lite, it reports higher resolution and localization performance than cited state-of-the-art approaches at lower cost, while TRAE cost remains unavailable for comparison.
- Conclusion: 62.0% resolve rate and 72.8% function localization accuracy are reported for XAgent on SWE-bench-lite, with 37% lower cost than cited state-of-the-art approaches.The reported results are framed as evidence that execution-guided reasoning supports more reliable automated software maintenance.
- Conclusion: XAgent integrates dynamic program execution with context-aware test generation, comparing failing and successful executions to localize buggy functions and reduce validation overfitting.The framework moves beyond static issue descriptions toward execution-guided issue resolution.
- Conclusion: TRAE’s generation cost is unavailable, so its cost cannot be compared with XAgent.This limits the scope of the cost comparison.
10 Data Availability
The authors state that source code and experimental details are publicly available through an anonymous GitHub repository. The manuscript reports no funding, relevant competing interests, or required ethics approval.
- Data Availability: Source code and experimental details are publicly available at an anonymous GitHub repository.The repository is identified as xag 2026-01-28.
- Data Availability: The manuscript reports no funding, no relevant competing interests, and no applicable ethics approval.These statements appear in the funding, competing-interests, and ethics-approval disclosures.