Source-linked AI summary

Know Before Fix: QA-Driven Repository Knowledge Acquisition for Software Issue Resolution

Haotian Lin, Silin Chen, Xiaodong Gu, Yuling Shi, Chengxi Pan, Jiaqi Ge, Mengfan Li, Jianghong Huang, Mengchieh Chuang, Beijun Shen, Haibing Guan

arXiv:2607.11111v1cs.SE

TL;DR

LLM coding agents often lack repository knowledge, while existing pre-repair methods do not explicitly identify those gaps. ACQUIRE acquires structured, evidence-grounded QA before patching and raises Pass@1 by up to 4.4 percentage points over the base repair agent on SWE-bench Verified.

  • Problem

    Existing pre-repair methods do not explicitly identify missing repository knowledge before patch synthesis, limiting evidence acquisition for issue resolution.

  • Method

    ACQUIRE identifies missing knowledge, generates targeted questions, obtains repository-grounded answers, and supplies structured QA to the Resolver before repair.

  • Results

    Pass@1 increases by up to 4.4 percentage points over the base repair agent, consistently outperforming representative pre-repair methods across two backbone LLMs.

  • Takeaways & Limitations

    ACQUIRE consistently improves issue resolution at modest additional cost and remains robust across model backbones, with QA injection accelerating knowledge-intensive repair stages.

  • Takeaways & Limitations

    Knowledge is injected as pre-generated QA before repair; dynamically refreshing it during debugging remains an open cost–freshness tradeoff.

Abstract

from arXiv · show

LLM-based coding agents have significantly advanced automated software issue resolution, yet they remain highly prone to factual errors caused by insufficient repository understanding. Recent methods attempt to mitigate this limitation through pre-repair repository exploration; however, their fix-driven strategies explore repositories without identifying the agent's knowledge gaps, often yielding imprecise context that fails to bridge the underlying understanding deficit. In this paper, we propose ACQUIRE, a QA-driven framework for software issue resolution. Mirroring how experienced developers first comprehend unfamiliar code before attempting a fix, ACQUIRE explicitly acquires repository knowledge prior to repair. The framework decouples knowledge acquisition from patch generation through two stages: in the first stage, a Questioner and an Answerer collaborate to acquire structured repository knowledge, where the Questioner poses targeted questions and the Answerer produces evidence-grounded answers through autonomous exploration; in the second stage, the Resolver leverages the resulting QA knowledge to generate informed patches. By transforming implicit knowledge gaps into explicit, factually reliable understanding, ACQUIRE accelerates knowledge-intensive repair stages and enables more accurate resolution. Experiments on SWE-bench Verified demonstrate that ACQUIRE consistently outperforms representative pre-repair methods, raising Pass@1 by up to 4.4 percentage points with modest additional cost and time.

I. INTRODUCTION … A. Overview

ACQUIRE addresses factual repair failures caused by insufficient repository understanding by acquiring structured, evidence-grounded repository knowledge before patch generation. Across oracle and benchmark evaluations, targeted QA improves resolution and ACQUIRE outperforms representative pre-repair methods with modest additional cost and time.

  • I. INTRODUCTION: LLM coding agents often produce factual errors because issue descriptions omit repository-internal dependencies, API contracts, and data-flow details needed for correct fixes.These gaps lead to shallow keyword localization and failures to trace faults across module boundaries.
  • II. MOTIVATION: Existing pre-repair methods remain fix-oriented, often producing incomplete or imprecise context because they rely primarily on issue keywords and structural pointers.Such methods depend on the agent to convert shallow repository pointers into the understanding required for repair.
  • I. INTRODUCTION: ACQUIRE is a QA-driven framework that decouples repository knowledge acquisition from patch generation into explicit acquisition and repair stages.Its contribution includes integrating repository-level QA into issue resolution and introducing a category-guided question taxonomy.
  • A. Overview: In the repair stage, the Resolver uses the structured QA knowledge to generate informed patches, transforming implicit knowledge gaps into explicit understanding before editing.This separation mirrors experienced developers’ practice of understanding an unfamiliar codebase before attempting a fix.
  • I. INTRODUCTION: Pass@1 increased by up to 4.4 percentage points over the base repair agent on SWE-bench Verified, with ACQUIRE outperforming representative pre-repair methods across two backbone LLMs.SWE-bench Verified contains 500 real-world GitHub issues, and the gains required modest additional cost and time.
  • I. INTRODUCTION: The acquired knowledge was factually reliable, reduced repair trajectory length, and directed agents toward causally relevant code regions on previously failed instances.These analyses clarify how pre-repair knowledge improves issue resolution.
  • II. MOTIVATION: 26 of 116 previously failed SWE-bench Lite instances were resolved after oracle QA injection, showing that targeted repository knowledge can unlock repair potential.The oracle experiment used Mini-SWE-Agent failures under DeepSeek-V3.2 and was intentionally constrained to one question-answer pair.
  • A. Overview: In the acquisition stage, a Questioner generates N targeted questions and independently instantiated Answerers explore the repository in parallel to produce the knowledge set K = {(q1, a1), . . . , (qN, aN)}.The Answerers produce evidence-grounded answers through autonomous repository exploration.

B. Stage 1: Question-Driven Knowledge Acquisition … A. Research Questions

ACQUIRE separates repository knowledge acquisition from repair: a Questioner generates targeted questions, parallel Answerers gather grounded evidence, and a Resolver uses the resulting QA knowledge before editing. The evaluation examines effectiveness, knowledge quality and influence, component contributions, and the effect of QA quantity on performance and cost.

  • B. Stage 1: Question-Driven Knowledge Acquisition: Stage 1 acquires structured repository knowledge through N question–answer interactions before any repair action begins.
  • 1) Question Generation:: A structured prompt template guides non-redundant, self-contained questions across four repair-relevant knowledge categories.The categories cover mechanism and behavior, design and usage, locating and structure, and ecosystem and standards.
  • 1) Question Generation:: The Questioner selects the most appropriate category per issue context, without external scheduling or round-robin constraints.A single category may recur when the most critical knowledge gaps fall within it.
  • 2) Evidence-Grounded Answering:: Each Answerer explores the repository in read-only mode and grounds answers in concrete artifacts such as file paths, function names, and code behaviors.When sufficient evidence is unavailable, the agent must acknowledge the gap rather than fabricate claims.
  • 3) Parallel Knowledge Acquisition:: The Questioner generates N questions, which N independently instantiated Answerers answer in parallel while exploring the repository.Each instance receives only the issue description and its assigned question, preventing cross-instance bias and reducing latency to the slowest instance.
  • C. Stage 2: Knowledge-Informed Repair: The Resolver receives the issue description and acquired knowledge, then iteratively navigates, edits, and tests until producing a candidate patch.The N QA pairs are statically prepended before repair instructions, establishing repository understanding before the first repair action and keeping it stable.
  • A. Research Questions: The evaluation asks whether ACQUIRE resolves issues effectively and whether generated QA knowledge supports repair behavior.
  • A. Research Questions: The study also tests contributions from question decomposition and category-guided generation, plus how the number of QA pairs affects repair performance and cost.

B. Datasets and Models … E. Implementation Details

The evaluation uses SWE-bench Verified with two complementary LLM backbones, measures effectiveness, cost, and time, compares ACQUIRE with representative exploration baselines, and standardizes repair-stage execution through Mini-SWE-Agent.

  • B. Datasets and Models: SWE-bench Verified contains 500 real GitHub issues involving functional bugs, evaluated by developer-written unit tests in isolated environments.Each instance provides only a natural-language problem description and its corresponding repository.
  • B. Datasets and Models: ACQUIRE is evaluated with DeepSeek-V3.2 and GPT-5-mini to cover open-source reasoning-specialized and proprietary efficiency-oriented model paradigms.DeepSeek-V3.2 uses a 671B-parameter MoE architecture with approximately 37B activated per token.
  • C. Evaluation Metrics: Pass@1 measures the proportion of instances successfully resolved in one attempt and serves as the primary effectiveness metric.The evaluation uses Pass@1 together with economic and runtime measures.
  • C. Evaluation Metrics: Average Cost measures monetary expenditure per instance across both pre-repair exploration and repair stages.This metric captures the economic cost of the entire pipeline rather than repair alone.
  • C. Evaluation Metrics: Average Time measures end-to-end wall-clock duration from pre-repair exploration through patch generation.Together with Pass@1 and Average Cost, it captures runtime efficiency.
  • D. Baseline Methods: ACQUIRE is compared with representative pre-repair exploration methods while Mini-SWE-Agent serves as the shared base repair agent.Compared methods include LocAgent, CoSIL, LingmaAgent, and SWE-Debate.
  • D. Baseline Methods: Mini-SWE-Agent iteratively proposes shell actions, observes results, updates its strategy, and produces a patch as both a standalone baseline and shared repair backbone.Its minimal design supports attribution of performance differences among methods.
  • E. Implementation Details: All methods use Mini-SWE-Agent repair settings in isolated Docker containers with 360-second step timeouts, 250-step trajectories, a $3.00 cost cap, and temperature 0.0.Other baselines follow the default settings reported in their original papers.

1) Baseline Method Configurations: … 2) Influence on Repair Behavior:

ACQUIRE combines explicitly configured QA-driven knowledge acquisition with repair, outperforming baselines across backbone models while improving factual reliability and reducing downstream repair effort. Its injected QA knowledge is used most heavily during locating and fixing, where it drives the observed acceleration.

  • 2) ACQUIRE Configurations:: ACQUIRE generates two targeted questions per issue, using a temperature of 0.7 for Questioner and 0.0 for Answerer under constrained read-only exploration.Answerer uses 120-second per-step timeouts, trajectories up to 150 steps, and a $2.00 per-instance cost cap; Resolver inherits the same scaffold and environment.
  • A. RQ1: Effectiveness of ACQUIRE: LocAgent and CoSIL gain moderately under DeepSeek-V3.2 but degrade under GPT-5-mini, illustrating the cross-model weakness of shallow localization signals.These approaches provide suspicious files or code fragments while relying on the backbone model to infer their relevance.
  • A. RQ1: Effectiveness of ACQUIRE: +3.8 Pass@1 on GPT-5-mini and +4.4 on DeepSeek-V3.2 are ACQUIRE’s largest gains across both backbone models, establishing cross-model improvement over all baselines.Unlike localization methods, its QA-driven approach captures behavioral context, design constraints, and structural knowledge.
  • B. RQ2: Quality and Influence of QA Knowledge: The RQ2 analyses use DeepSeek-V3.2 throughout to control for model variation while evaluating QA reliability and downstream repair behavior.The section examines both the quality of generated QA knowledge and its influence on repair.
  • 1) QA Factual Reliability:: 44 Fail→Pass recoveries and 22 Pass→Fail regressions yield a net gain of 22 resolved instances when comparing ACQUIRE with Mini-SWE-Agent.A human audit evaluates 232 QA pairs sampled across all four outcome-transition cells.
  • 1) QA Factual Reliability:: 99.1% of audited QA pairs are Supported, including 98 fully accurate pairs and 132 with minor local deviations; only 0.9% contain ungrounded central claims.The QA decomposition into narrowly scoped questions simplifies retrieval and reasoning, reducing opportunities for hallucination and enabling evidence-grounded answers.
  • 2) Influence on Repair Behavior:: QA injection reduces mean agent rounds by 7.1% across 500 instances and by 17.1% on the 44 Fail→Pass instances, bypassing expensive trial-and-error loops.The remaining trajectory analysis therefore focuses on the 44 Fail→Pass cases and labels steps according to explicit or evidence-following QA use.
  • 2) Influence on Repair Behavior:: 40.4% of all steps are QA-related, concentrated in Locating (51.5%) and Fixing (50.6%) rather than Reproducing (28.4%) and Verifying (27.4%).The most accelerated stages are those with the greatest QA use, directly linking injected knowledge to the speedup.

3) Pass-to-Fail Regression Analysis: … D. RQ4: Sensitivity to the Number of QA Pairs

ACQUIRE’s reliable QA knowledge generally improves repair, while regressions arise mainly from Resolver misuse rather than factual errors. Ablations show that decomposed, category-guided questioning is essential, and two QA pairs provide the best effectiveness–cost trade-off.

  • 3) Pass-to-Fail Regression Analysis:: Only 5 of 22 Pass→Fail regressions were misleading, indicating that most regressions were not caused by QA leading repairs away from the correct fix.Misleading cases involved QA emphasizing a related but incorrect location or mechanism and the Resolver repeatedly following that framing.
  • 3) Pass-to-Fail Regression Analysis:: Among the 10 QA pairs in the 5 misleading regressions, only 1 contained an ungrounded claim, while the other 9 were factually supported.This suggests misleading regressions primarily reflect plausible but incorrect repair framing rather than answer factuality.
  • 3) Pass-to-Fail Regression Analysis:: The regression bottleneck is Resolver utilization of reliable QA, making critical evaluation and selective application of injected knowledge a promising mitigation.The paper characterizes generated QA knowledge as highly reliable and links repair speedups to the knowledge-intensive Locating and Fixing stages.
  • C. RQ3: Ablation on Key Design Choices:: ACQUIRE-Proposal replaces the Questioner–Answerer pipeline with a single proposal-generation step, while ACQUIRE-FreeQ removes Questioner’s category-driven template.Both variants retain the rest of the pipeline unchanged except for their respective removed design component.
  • 1) QA Decomposition vs. Proposal:: 4.8 percentage points: ACQUIRE-Proposal reduces Pass@1 to 66.0%, below Mini-SWE-Agent’s 66.4%, showing that a single proposal actively degrades repair.The ablation produces the largest performance drop among the tested variants.
  • 2) Category-Guided vs. Free Question Generation:: 3.8 percentage points: ACQUIRE-FreeQ drops Pass@1 to 67.0%, while category-guided questions gain +0.38 in diagnostic utility, +0.38 in reasoning depth, and +0.78 in coverage.Answerability is virtually unchanged at −0.01, and the coverage gain indicates reduced clustering around one diagnostic angle.
  • D. RQ4: Sensitivity to the Number of QA Pairs:: N=1 raises Pass@1 by 2.6 percentage points from the Resolver-only baseline’s 66.4% to 69.0%, for only $0.005 additional cost.At N=0, the baseline costs $0.055 per instance.
  • D. RQ4: Sensitivity to the Number of QA Pairs:: Pass@1 peaks at N=2 with 70.8% and falls to 69.0% at N=3 as cost continues rising, making two complementary QA pairs the best trade-off.The paper attributes the decline after N=2 to overlapping knowledge and longer context diluting the Resolver’s attention.

E. Case Study … VII. THREATS TO VALIDITY

The case study shows ACQUIRE guiding targeted repository exploration, precise fault localization, and a minimal root-cause fix. The discussion highlights effectiveness, interpretability, extensibility limits, and validity threats concerning measurement, confounding factors, and language scope.

  • E. Case Study: In sphinx-doc__sphinx-9230, ACQUIRE targets the :param type-parsing mechanism, avoiding misleading surface-level keyword exploration.The issue stems from fieldarg.split(None, 1) splitting inside parenthesized types and misaligning the type–name boundary.
  • E. Case Study: The Resolver confirms the problematic split call, applies a bracket-aware replacement in one file, and adds comprehensive tests for compound-typed :param directives.The patch directly addresses the root cause identified in the QA answer.
  • A. Strengths: ACQUIRE’s decoupled question generation, repository-grounded answering, and patch synthesis make knowledge gaps explicit before editing and improve repair effectiveness across backbone models.The framework also reduces blind locating and trial-and-error while shifting effort toward reproduction and verification.
  • A. Strengths: Structured QA pairs provide an inspectable interface for requested knowledge, repair diagnosis, and clearer debugging signals than localization lists or repair descriptions.The category-guided question template improves controllability for future system improvement.
  • B. Limitations and Future Work: The four-category question template could be extended with hierarchical, issue-type-adaptive, or automatically learned category schemas for broader coverage.The current template is described as practical and effective despite this remaining refinement opportunity.
  • B. Limitations and Future Work: Dynamic knowledge acquisition could refresh QA context during debugging, but balancing knowledge freshness against its additional computational cost remains open.Current pre-generated QA injection is simple and stable in practice.
  • VII. THREATS TO VALIDITY: Construct validity is limited by whether Pass@1 fully captures repair quality, although time, cost, and trajectory-step analyses add an efficiency perspective.Pass@1 on SWE-bench Verified is the primary effectiveness metric.
  • VII. THREATS TO VALIDITY: Internal-validity risks from prompts, tools, or hyperparameters are mitigated through consistent protocols and ablations, while external validity remains limited by Python-focused evaluation.Ablations removing question decomposition and category-guided generation produced consistent performance drops; other languages remain future work.

VIII. RELATED WORK … Supplementary Material

Prior work advances repository-level issue resolution through structured workflows, repository exploration, reusable knowledge, and repository-scale QA, but often leaves knowledge gaps implicit. ACQUIRE addresses this limitation by making autonomous, evidence-grounded knowledge acquisition an internal step before repair, with experiments indicating improved and more interpretable resolution.

  • A. Software Issue Resolution: Repository-level issue resolution requires cross-file dependency reasoning, broad context understanding, and robust handling of ambiguous bug descriptions.SWE-bench and infrastructure such as Repo2Run support this setting, alongside agentic frameworks including SWE-agent.
  • A. Software Issue Resolution: Structured workflow decomposition improves repair controllability through localization, repair, validation, repository search, multi-step reasoning, and structured debate.Representative systems include Agentless, AutoCodeRover, LingmaAgent, and SWE-Debate.
  • A. Software Issue Resolution: Repository exploration and context management use graph-guided reasoning, structural traversal, efficiency control, pruning, and code-context compression to streamline repair-relevant search.LocAgent, RepoGraph, EET, SWE-Pruner, and later compression methods represent these directions.
  • A. Software Issue Resolution: Reusable knowledge and memory inform agentic software engineering through historical experience, dual-memory designs, shared knowledge bases, and reused execution trajectories.SWE-Exp, EXPEREPAIR, Agent KB, SAGE, and SE-Agent exemplify these approaches.
  • A. Software Issue Resolution: Empirical studies identify insufficient understanding of repository internals and behavioral contracts as primary sources of incorrect or incomplete patches.These findings motivate explicit pre-repair knowledge acquisition, which prior methods often neglect by assuming existing mechanisms surface all relevant evidence.
  • B. Repository QA and Knowledge Acquisition: Repository QA has progressed from semantic code search to full-repository comprehension requiring multi-file traversal and evidence aggregation across interdependent files.Benchmarks such as RepoQA, CodeRepoQA, CoReQA, and SWE-QA show that answers require synthesis across multiple abstraction layers, not retrieval precision alone.
  • B. Repository QA and Knowledge Acquisition: Existing repository QA typically assumes pre-provided questions, whereas ACQUIRE enables agents to identify knowledge gaps, formulate questions, and acquire contextual evidence before patch generation.This agent-to-repository model integrates QA as an internal capability rather than treating it solely as an external benchmark.
  • IX. CONCLUSION: ACQUIRE explicitly identifies missing knowledge, obtains repository-grounded answers, and repairs using structured evidence; its code and data are publicly available.Experiments report improved issue resolution at modest additional cost, robustness across model backbones, accelerated knowledge-intensive stages, greater verification effort, and knowledge encoded in most successful patches.

I. PROMPT TEMPLATES · A. Questioner Prompt · B. Answerer Instance Template

The prompt templates separate repository-question generation from read-only, shell-based answer construction. The Questioner produces prioritized, categorized JSON questions, while the Answerer explores repository evidence through constrained commands and submits a detailed final answer.

  • A. Questioner Prompt: The Questioner must generate exactly {N} repository-focused questions, sorted by importance and returned as a JSON object.Each question includes both a category and question field, uses the specified category names, and excludes text outside the JSON object.
  • A. Questioner Prompt: Questions are selected from four categories: Mechanism & Behavior, Design & Usage, Locating & Structure, and Ecosystem & Standards.Categories cover internal functionality, interfaces and contracts, repository structure, and external ecosystems or standards.
  • A. Questioner Prompt: Categories 1–3 target repository knowledge obtainable from the project, whereas Category 4 targets external knowledge independent of the repository.The prompt distinguishes repository-derived understanding from external libraries, language features, protocols, standards, and scientific concepts.
  • A. Questioner Prompt: The Questioner prioritizes current implementation understanding and forbids issue-only details, code changes, version history, commit references, and specific project names.Questions should be answerable by analyzing the repository as it exists now and use generic references such as “this repository” or “the codebase.”
  • B. Answerer Instance Template: The Answerer analyzes the codebase through a read-only computer shell to answer the supplied question accurately without modifying repository files.The template permits reading, searching, listing, and analysis commands while prohibiting file creation, editing, deletion, and unnecessary application execution.
  • B. Answerer Instance Template: Each response contains a THOUGHT section and exactly one bash code block containing exactly one command or commands connected with && or ||.Independent commands must be issued in later responses, and each action runs in a new subshell.
  • B. Answerer Instance Template: The Answerer follows a workflow of understanding the question, locating relevant code, reading files, searching implementations, synthesizing evidence, and submitting the answer.The template explicitly sequences repository exploration before final synthesis.
  • B. Answerer Instance Template: After gathering sufficient information, the Answerer submits a detailed answer with one heredoc command beginning with SUBMIT_ANSWER and referencing analyzed files and code.The submission format requires the heredoc syntax exactly and ends the task immediately after submission.

C. Question-Quality Evaluation Prompts · 1) Scoring – Individual Question:

The evaluation scores each repository-focused diagnostic question on five dimensions and separately flags external dependence, genericity, and issue restatement. It distinguishes whether an answer comes from repository artifacts from how much it reduces debugging uncertainty.

  • 1) Scoring – Individual Question:: Questions receive integer scores from 1 to 10 for relevance, Repository Answerability, Diagnostic Utility, Reasoning Depth, and Clarity.The evaluator assesses one question using the problem statement and question text.
  • 1) Scoring – Individual Question:: Relevance measures whether a question targets the issue’s failure mechanism, root cause, or key repair decision.Higher scores indicate stronger relevance to understanding or resolving the issue.
  • 1) Scoring – Individual Question:: Repository Answerability measures whether the question can be answered from repository code, tests, configuration, documentation, or architecture.Questions mainly requiring external knowledge should not receive high Repository Answerability scores.
  • 1) Scoring – Individual Question:: Diagnostic Utility measures how sharply answering the question narrows the debugging search space toward a subsystem, execution path, state transition, or repair decision.A question that only suggests where to look without reducing uncertainty should not receive a high score.
  • 1) Scoring – Individual Question:: Reasoning Depth measures the amount of reasoning required, from a superficial lookup to multi-hop analysis across components, state changes, or abstraction boundaries.Complexity alone should not be rewarded when a focused question is diagnostically strong.
  • 1) Scoring – Individual Question:: Clarity measures whether the question is concise, unambiguous, well-scoped, and technically precise.Ambiguous, confusing, overly broad, or multi-ask questions receive lower clarity scores.
  • 1) Scoring – Individual Question:: The evaluator sets three Boolean flags: requires_external_knowledge, too_generic, and rephrases_issue_without_added_value.These flags identify reliance on external knowledge, generic applicability, or restatement without diagnostic direction.

2) Scoring – Question Set (Coverage): · 3) Voting:

The framework scores repository-focused question sets by their coverage of complementary, nonredundant diagnostic angles and votes between candidate sets based on relevance, answerability, diagnostic utility, reasoning depth, and efficiency. Both procedures prioritize concise, repository-grounded questions that reduce debugging uncertainty over question count, verbosity, or complexity.

  • 2) Scoring – Question Set (Coverage):: Coverage evaluates whether a question set targets the issue’s key diagnostic angles for understanding, localizing, and resolving the defect.The set is judged as a whole using a single integer score from 1 to 10, with 10 as the best score.
  • 2) Scoring – Question Set (Coverage):: High coverage requires core relevance, complementary diagnostic perspectives, non-redundancy, and strong debugging usefulness.These criteria assess whether questions focus on the main failure mechanism, add distinct information, and reduce uncertainty about where and why the bug occurs.
  • 2) Scoring – Question Set (Coverage):: Scores of 1-2 indicate very poor coverage, 3-4 weak coverage, 5-6 moderate coverage, 7-8 strong coverage, and 9-10 excellent coverage.Excellent sets efficiently cover the most important diagnostic angles with targeted, complementary, information-dense questions.
  • 2) Scoring – Question Set (Coverage):: Coverage rewards efficient diagnostic value rather than breadth or question count, and reworded versions of one underlying question do not add coverage.A smaller set can score higher when it covers the issue’s key diagnostic needs more efficiently.
  • 3) Voting:: The examples favor sets combining subsystem localization, control-flow or state-transition behavior, and boundary conditions, validation paths, or missing tests.They contrast this with repetitive questions that ask only where the bug or responsible logic is located.
  • 3) Voting:: Voting selects the candidate set that is more useful for understanding, localizing, and resolving the issue, while ignoring presentation order, verbosity, and set size.The comparison favors concise, high-signal, repository-grounded questions and returns TIE when the sets are functionally equivalent.
  • 3) Voting:: Candidate sets are compared on relevance, repository answerability, diagnostic utility, reasoning depth, and coverage with efficiency.Repository answerability concerns whether repository artifacts provide the answer, whereas diagnostic utility concerns how much that answer helps narrow debugging uncertainty.
  • 3) Voting:: Voting prefers sharper, simpler question sets over verbose or complex alternatives when they provide greater practical debugging value.Complexity is not rewarded for its own sake, and more questions do not imply better coverage without complementary value.

II. SUPPLEMENTARY EXPERIMENTAL ANALYSIS … 1) Agent Round Distribution:

The supplementary analysis reports stage-wise time and cost on SWE-bench Verified and shows that QA injection reduces agent rounds, especially on instances changing from failure to success. ACQUIRE combines efficient pre-repair exploration with improved Pass@1 and modest end-to-end overhead.

  • A. Stage-wise Cost and Time Analysis: On 500 SWE-bench Verified instances, the analysis separates pre-repair exploration from downstream repair for wall-clock time and monetary cost.“Pre” denotes the pre-repair stage, while “Repair” denotes the downstream repair agent.
  • A. Stage-wise Cost and Time Analysis: Compared with LingmaAgent and SWE-Debate, ACQUIRE’s pre-repair stage is 2.5–11× faster and 5–26× cheaper while achieving the best Pass@1.The comparison concerns both pre-repair efficiency and the reported Pass@1 outcome.
  • A. Stage-wise Cost and Time Analysis: Compared with LocAgent and CoSIL, ACQUIRE has moderate pre-repair overhead and comparable total cost while delivering a clearly higher Pass@1.This finding contrasts ACQUIRE’s overhead and total cost with its stronger repair result.
  • A. Stage-wise Cost and Time Analysis: ACQUIRE adds only modest end-to-end overhead over bare Mini-SWE-Agent, with approximately 115 s / 227 s and $0.030 / $0.018 per instance reported.The passage presents these time and cost figures as the modest overhead comparison, though the supplied text truncates the associated labels.
  • 1) Agent Round Distribution:: Under DeepSeek-V3.2, QA injection shifts the agent-round distribution leftward across the full set of 500 instances.Figure 1 compares agent-round distributions with and without QA injection.
  • 1) Agent Round Distribution:: QA injection reduces mean agent rounds by 7.1% on all instances and by 17.1% on the 44 Fail→Pass instances.The larger reduction on Fail→Pass cases is presented as evidence that QA injection mitigates the cost of knowledge-deficient repair attempts by front-loading knowledge acquisition.

2) API-call Shift by Transition Type: … 2) Non-misleading Cases:

The analysis shows that QA reduces exploration effort in successful repairs, but can still misdirect some failures; most QA knowledge concerns mechanisms, while remaining errors largely reflect imperfect answer details or Resolver behavior. The audit identifies both grounded inaccuracies and rare central claims contradicted by repository evidence.

  • 2) API-call Shift by Transition Type:: Fail→Pass cases reduced API calls by 15.18 on average, while non-misleading Pass→Fail cases decreased by 3.64 and misleading cases increased by 11.40.The aggregate Pass→Fail change was +0.55, driven almost entirely by misleading cases.
  • C. Characterization of Generated QA: 74.4% of generated QA knowledge concerned Mechanism & Behavior, with Design & Usage, Locating & Structure, and Ecosystem & Standards comprising 13.9%, 9.5%, and 2.2%.Among Fail→Pass instances, 20 of 44 (45.5%) required at least one non-Mechanism question, indicating that cross-category knowledge sometimes supported successful fixes.
  • III. HUMAN AUDIT AND REGRESSION DETAILS: The supplementary analysis audits QA reliability and regression trajectories, including minor deviations, ungrounded claims, and Pass→Fail cases.These analyses provide additional detail for the human audit and regression analysis reported in the main paper.
  • A. Minor Deviation Breakdown: Among 132 Supported QA pairs with minor deviations, 67.9% had localized reference inaccuracies, 24.6% showed evidence-scope overreach, and 7.5% overgeneralized implementation boundaries.The categories cover imprecise code references or details, extrapolation beyond repository evidence, and overgeneralization of special cases or boundary conditions.
  • B. Cases Containing Ungrounded Claims: Two QA pairs contained useful repository-grounded information but made central mechanism claims unsupported by code.One wrongly asserted SimpleLazyObject inherited from Promise; another incorrectly claimed symbols() propagated cls through nested tuple recursion.
  • C. Pass-to-Fail Regression Details: Of 22 Pass→Fail trajectories, 5 were classified as misleading and 17 as non-misleading based on the dominant downstream effect of injected QA on repair behavior.The classification was determined through manual inspection of all analyzed regression trajectories.
  • 1) Misleading Cases:: In misleading cases, QA often contained grounded facts but supplied an incorrect repair framing that repeatedly steered the Resolver away from the gold fix.Among 10 QA pairs in the 5 misleading cases, only 1 was labeled Contains ungrounded claim and the remaining 9 were Supported.
  • 2) Non-misleading Cases:: In the 17 non-misleading cases, QA often helped locate files or mechanisms, while failures more commonly arose from Resolver overgeneralization, partial fixes, extra behavior, or other patch errors.The Resolver was instructed to cross-check QA against code, command outputs, and tests, so these regressions reflect limitations in using helpful QA rather than QA misleading the Resolver.
Loading 2607.11111v1…