SweRank: Software Issue Localization with Code Ranking

1 INTRODUCTION

• Agentic localization can require about 7–10 model interactions and cost about $0.66 per Claude-3.5 example, with high latency.The paper also describes agent traces as brittle and dependent on temperature sampling and complex tool orchestration.Source: §1 · p. 1
• Issue descriptions are verbose and describe observed failures, unlike the functionality requests or similar-code examples used in conventional retrieval training.This difference in query length and intent motivates training code ranking models specifically for issue localization.Source: §1 · p. 2
• SWERANKEMBED retrieves candidate code and SWERANKLLM reranks it, with training tailored to issue localization.SWELOC supplies real-world issue descriptions paired with code modifications; the retriever uses contrastive training and the reranker predicts a ranking permutation.Source: §1 · p. 2
• Figure 1 reports superior localization accuracy at significantly lower per-instance cost for SWERANKEMBED and SWERANKLLM than agent-based methods.The comparison is on SWE-Bench-Lite.Source: Figure 1 · p. 2
• SweRank achieves state-of-the-art localization across file, module, and function levels on SWE-Bench-Lite and LocBench.The paper also reports a better performance-to-cost ratio than agent-based approaches using closed-source LLMs.Source: §1 · p. 2

2.1 SOFTWARE ISSUE LOCALIZATION

• Spectrum-based and program-analysis fault localization rank suspicious code using test outcomes or program structure, but require precise program models and cannot use bug-report language context.SFL uses passing and failing test coverage; static and dynamic analyses include call-graph traversal, dependency analysis, and program slicing.Source: §2.1 · p. 2
• LLM agent approaches navigate repositories through multi-step reasoning and tool use, but incur substantial cost and latency and can be derailed by a failed step.SWERANK instead frames localization as a single-shot ranking problem.Source: §2.1 · p. 3

2.2 CODE RANKING

• Code-ranking models learn text-code representations and retrieval objectives, but prior models rarely train on error logs or verbose bug reports.SWERANK trains a bi-encoder and listwise LLM reranker on SWELOC’s issue reports paired with known buggy functions.Source: §2.2 · p. 3

3 SWELOC: ISSUE LOCALIZATION DATA

• SWELOC addresses the mismatch between functionality-focused retrieval datasets and detailed failure descriptions by pairing real-world issues with relevant code modifications.The dataset construction has three phases: selecting pull requests, extracting issue descriptions and modifications, then improving instance quality through filtering and negative mining.Source: §3 · pp. 3–4
• SWELOC issue queries average 382.56 tokens, and issues can modify multiple files, modules, or functions, especially at finer code granularity.Figure 3 shows distributions of query lengths and modified code-unit counts.Source: Figure 3 · p. 4

3.1 IDENTIFYING RELEVANT PRS

• SWELOC repositories come from the top 11,000 PyPI packages; filtering for at least 80% Python, benchmark exclusion, and deduplication leaves 3,387 repositories.Repositories appearing in SWE-Bench or LocBench are excluded.Source: §3.1 · p. 4
• Pull requests must resolve a linked GitHub issue and modify test files; this yields 67,341 initial pull-request/codebase pairs.For each pair, the issue description is collected alongside the codebase at the pull request’s base commit.Source: §3.1 · p. 4

3.2 LOCALIZATION PROCESSING

• Each modified function becomes a positive training instance for its issue query, with unmodified functions from the same codebase serving as negatives.A pull request modifying N functions yields N instances.Source: §3.2 · p. 4

3.3 CONSISTENCY FILTERING AND HARD NEGATIVES

• Issue descriptions may be vague, so consistency filtering retains an instance only when its positive function ranks within the top 20 repository functions by query similarity.The ranking uses CODERANKEMBED; the paper reports ablations of the threshold.Source: §3.3 · p. 4
• Hard-negative mining selects unmodified functions among the 15 most similar to the issue query, creating challenging contrasts for training.These negatives are selected using similarities previously computed for consistency filtering.Source: §3.3 · p. 5

4 SWERANK METHODOLOGY

• SWERANK first retrieves candidate code functions with a bi-encoder, then refines their ranking with an instruction-tuned listwise LLM.The retriever ranks functions by cosine similarity in a shared issue-code embedding space.Source: §4 · p. 5, §4.1 · p. 5

4.1 SWERANKEMBED

• The bi-encoder learns issue-code similarity with an InfoNCE contrastive loss, using corresponding modified functions as positives and mined hard negatives.At inference, functions are ranked by cosine similarity to the issue embedding.Source: §4.1 · p. 5

4.2 SWERANKLLM

• Because SWELOC lacks a ground-truth ordering among negative functions, SWERANKLLM is trained to predict the identifier of the single positive function as its top-ranked result.The training objective omits the end-of-sequence token, retaining the model’s ability to generate full ranked lists at inference.Source: §4.2 · pp. 5–6

5.1 SETUP

• Evaluation uses SWE-Bench-Lite and LocBench, measuring file-, module-, and function-level Acc@k; comparisons include agent-based methods and code-ranking baselines.The setup retains 274 SWE-Bench-Lite examples and uses a 560-example LocBench that also includes security and performance issues.Source: §5 · p. 6, §5.1 · pp. 6–7

5.2 LOCALIZATION RESULTS

• SWERANK sets state-of-the-art localization performance across granularities on SWE-Bench-Lite, while large variants set new state-of-the-art on LocBench and outperform Claude-3.5 agent methods across its categories.SWELOC-trained models also generalize across LocBench categories despite being trained primarily on bug reports.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3.1 DATA QUALITY AND SIZE

• Moderate consistency filtering (for example, K=20) balances SWELOC instance quality and size; no filtering makes finetuned SWERANKEMBED-SMALL perform worse than its pretrained model.With K fixed at 20, performance improves even using 5% of the filtered data, and generally rises with dataset size.Source: §5.3.1 · pp. 7–8, Figure 5 · p. 8

5.3.2 CHOICE OF RETRIEVER AND RERANKER

• Finetuning Arctic-Embed, CodeRankEmbed, and Arctic-Embed-v2.0 on SWELOC improves each retriever, with larger gains for initially weaker models.The analysis also evaluates SWELOC finetuning across listwise rerankers of different sizes, as identified by Table 4.Source: §5.3.2 · p. 8, Table 3 · p. 8, Table 4 · p. 9

5.3.3 INFERENCE COST ANALYSIS

• SWERANK provides up to 6X better performance-cost tradeoffs than LocAgent; precomputed embeddings and reranker outputs limited to candidate identifiers contribute to its lower inference cost.The comparison is against agent-based methods that are described as highly cost-intensive.Source: §5.3.3 · p. 9, Table 5 · p. 9

5.3.4 IMPACT ON DOWNSTREAM ISSUE RESOLUTION

• In SWE-Fixer, improved localization from SWERANK yields better issue resolution, while oracle results indicate that the code-editing model currently constrains repair performance.The oracle uses the ground-truth edited file as an upper bound for the repair framework.Source: §5.3.4 · p. 9, Table 6 · p. 9

5.3.5 PERFORMANCE ANALYSIS BY ISSUE COMPLEXITY

• Accuracy declines as more functions require modification, but SWERANKEMBED-LARGE retains 77.1% for issues with 2–3 modified functions, versus LocAgent’s decline from 82.0% to 47.1%.The reranker improves performance across complexity levels, capturing cross-function dependencies the bi-encoder may miss; issues requiring more than five functions have single-digit accuracy for all models.Source: Figure 6 · p. 9, §5.3.5 · p. 10

5.3.6 PERFORMANCE ANALYSIS BY LEXICAL AND SEMANTIC OVERLAP

• The analysis groups performance by lexical overlap (Rouge-1) and semantic overlap, with results broken down in Figures 7 and 8.The supplied figure descriptions identify the breakdowns but do not report their outcomes.Source: Figure 7 · p. 10, Figure 8 · p. 10

• SWERANK frames software issue localization as a specialized ranking task and uses a retrieve-and-rerank framework.SWELOC is a large-scale contrastive dataset derived from real-world GitHub issues, using consistency filtering and hard-negative mining.Source: §6 · p. 10
• Evaluations on SWE-Bench-Lite and LocBench show state-of-the-art localization performance and significantly better performance than costly closed-source agent-based systems.Source: §6 · p. 10
• SWELOC is presented as a valuable resource for advancing research on software issue localization.Source: §6 · p. 10

A.1 SWERANKEMBED

• SWERANKEMBED-SMALL uses a CODERANKEMBED encoder initialized with Arctic-Embed-M and is finetuned for two epochs with 15 hard negatives per example.Its query prefix is “Represent this query for searching relevant code: ”; training uses 8 GH200 GPUs, learning rate 2e-5, and batch size 64.Source: §A.1 · p. 16
• SWERANKEMBED-LARGE uses GTE-Qwen2-7B-Instruct with an issue-localization query prefix and is finetuned for one epoch with seven hard negatives per example.Training uses 8 GH200 GPUs, learning rate 8e-6, and batch size 64.Source: §A.1 · p. 16

A.2 SWERANKLLM

• For reranker training, each SWELOC tuple supplies one positive and nine randomly sampled negatives; candidates are shuffled, and long prompts are truncated to fit length limits.Rerankers are pretrained on text listwise reranking data; subsequent training uses one epoch, batch size 128, learning rate 5e-6, noisy embeddings, and 16 GH200 GPUs with ZeRO stage 3.Source: §A.2 · p. 16

• Llama-3.1 8B Instruct is pretrained on RankZephyr general text reranking data and then finetuned on SWELOC as a listwise reranker.Source: B EXPERIMENTS WITH MORE RERANKER MODELS · p. 16
• Finetuning Llama-3.1 8B Instruct on SWELOC yields significant performance gains on both SWE-Bench-Lite and LocBench.Source: B EXPERIMENTS WITH MORE RERANKER MODELS · p. 16
• The results support SWELOC’s usefulness for improving issue localization in LLM families beyond Qwen 2.5.Source: B EXPERIMENTS WITH MORE RERANKER MODELS · p. 16

C RETRIEVER CEILING ANALYSIS

• Table 8 reports extended retrieval metrics; Acc@K is the percentage of instances where all ground-truth functions are within the top-K retrieved candidates, and Acc@100 indicates a performance ceiling for the subsequent reranker.Source: Table 8 · p. 17
• The retrieval ceiling (Acc@100) is 93.43%, significantly higher than the best performance achieved by agentic methods such as LocAgent (approximately 78%), suggesting retrieval is not the primary bottleneck.Acc@100 indicates a performance ceiling for the subsequent reranker.Source: C RETRIEVER CEILING ANALYSIS · p. 17, Table 8 · p. 17
• The reported retrieval ceiling is 93.43% Acc@100, compared with SWERANKLLM-LARGE’s 88.7% Acc@10; this gap suggests that retrieval is not the primary bottleneck.Source: C RETRIEVER CEILING ANALYSIS · p. 17

D ABLATION STUDIES ON MODEL CAPACITY

• SWERANKEMBED-SMALL gains moderately over Qwen3-0.6B, while comparison with Qwen3-8B suggests diminishing returns from further increasing encoder depth.The authors attribute the gap between Qwen3-0.6B and SWERANKEMBED-LARGE as appearing driven by the larger embedding dimension.Source: D ABLATION STUDIES ON MODEL CAPACITY · p. 17
• SWELOC finetuning yields +21.9 points Acc@10 at D = 64 versus +12.5 points at D = 1024, while performance drops as embedding size decreases.The controlled ablation uses Qwen3-Embedding-0.6B with Matryoshka Representation Learning.Source: D ABLATION STUDIES ON MODEL CAPACITY · pp. 17–18, Table 10 · p. 17

E BENEFIT OF ITERATIVE NEGATIVE MINING

• Second-iteration hard negatives improve localization performance over first-iteration negatives, while random negatives yield smaller gains.The second iteration mines negatives using the small model finetuned in the first iteration.Source: E BENEFIT OF ITERATIVE NEGATIVE MINING · p. 18, Figure 9 · p. 18

F MULTILINGUAL GENERALIZATION

• 50.85% Acc@10 for SWERANKEMBED-LARGE exceeds Gemini-Embedding's 47.44% on SWE-Bench Multilingual despite finetuning on Python data.The evaluation covers 234 tasks across nine languages; finetuning on SWELOC also significantly improves both the retriever and reranker over their base models.Source: F MULTILINGUAL GENERALIZATION · p. 18, Table 11 · p. 18

G.1 INFERENCE LATENCY ANALYSIS

• 12.5 seconds per instance for local SWERANKLLM-SMALL is approximately 7× faster than LocAgent, and SWERANK remains nearly 3× faster when both use GPT-4o.Retrieval embeddings can be precomputed and indexed, leaving reranking as the primary latency bottleneck; the comparison averages 50 SWE-Bench-Lite instances.Source: §G.1 · p. 19, Table 12 · p. 19

G.2 TOKEN EFFICIENCY AND FOOTPRINT

• SWERANK uses approximately 3× fewer input tokens than the agentic baseline, and its output-token reduction is more pronounced.The authors connect the output-token reduction to lower latency and inference costs.Source: §G.2 · p. 19, Table 13 · p. 19

H QUALITATIVE EXAMPLES

• In two SWE-Bench-Lite examples, SWERANK localizes the function to edit correctly while LocAgent identifies functions where the problem likely manifests rather than originates.Figure 10 presents the qualitative examples.Source: H QUALITATIVE EXAMPLES · p. 19, Figure 10 · p. 20

I DIVERSITY OF ISSUE TOPICS IN SWELOC

• A topic analysis of 10k randomly sampled SWELOC instances uses Nomic Atlas to identify and display the frequencies of the top 15 issue topics.The topic identification uses cluster-based keywords and language-model-generated topics.Source: I DIVERSITY OF ISSUE TOPICS IN SWELOC · p. 19, Figure 11 · p. 20