SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks

• Frontier-model inference costs at least an order of magnitude more than competitive open-weight alternatives, while only a minority of tasks require frontier capability.Source: §1 · p. 1
• Prompt-only routers cannot distinguish some similarly described issues that require either a localized typo fix or a multi-module refactor.Agent observations such as failed tests, retrieved files, and stack traces supply structural signal unavailable from the prompt alone.Source: §1 · p. 1
• SWE-Router runs a weak model for exploratory turns, then uses a learned value head on the partial trajectory to choose continuation or escalation.Its Bayes-optimality result says this conditioning never harms routing and is strictly better when exploration is informative.Source: §1 · p. 1
• SWE-Router achieves substantial cost reductions at matched resolution on SWE-Bench Verified relative to strong prompt-only baselines.Source: §1 · p. 1

2.1. Multi-turn Trajectories in Agentic Tasks

• A complete SWE-agent trajectory contains the task description followed by per-turn thoughts, actions, and observations, and ends with a submitted patch checked against repository unit tests for a binary reward.Source: §2.1 · p. 2

2.2. LLM Routing

• An LLM router selects the minimum-cost model with sufficient capability by comparing cost-adjusted expected success rates for weak and strong models.Model inference costs are treated as task-independent, and λ converts costs into success-probability units to trace a cost–quality curve.Source: §2.2 · p. 2
• In agentic settings, task descriptions often leave difficulty unresolved, while early turns can reveal file structure, error patterns, and search depth.That gap motivates routers that condition on the partial trajectory rather than the prompt alone.Source: §2.2 · p. 2

• The router runs the weak model for K turns, then uses a pretrained LLM with a small classification head to predict its eventual success from the partial trajectory.The head is trained on binary trajectory rewards; the K exploratory turns are counted in escalated runs’ costs.Source: §3 · p. 2
• The decision rule continues with the weak model when its cost-adjusted predicted utility exceeds the strong model’s, otherwise switching to the strong model from the K-th step.On escalation, the strong model restarts from the original prompt rather than conditioning on the weak model’s reasoning.Source: §3 · p. 2
• Reported experiments use one-sided routing: continue with the weak model when its predicted solve probability clears a validation-tuned threshold.This rule treats the strong model’s success estimate as an unknown trajectory-independent constant.Source: §3 · p. 2

• The improvement is strict when the conditional cost-adjusted utility gap from the partial trajectory is not determined by the task description.Source: §4 · p. 3
• Conditioning on the partial trajectory cannot lower Bayes-optimal expected routing utility compared with conditioning on the task description alone.The result follows because the task description is determined by the partial trajectory.Source: §4 · p. 3
• In practical terms, exploration helps when it refines the model-choice comparison; the gain vanishes when the prompt already determines the optimal choice.The partial trajectory can retain fine-grained difficulty cues that prompt-only expectations average out.Source: §4 · p. 3

5.1. Evaluation Metrics

• Route-AUC is the normalized area under the cost–resolved-rate curve, while AUROC measures the value head’s discriminative quality.The cost–resolved axes normalize the all-weak and all-strong endpoints to (0, 0) and (1, 1).Source: §5.1 · p. 3
• After up to K exploratory turns, the weak model’s value score determines whether it continues or the task restarts with the strong model.The procedure returns the weak model’s patch immediately if it submits during exploration; otherwise, it compares the last-token value-head score with a tuned threshold.Source: §5.1 · p. 3

5.2. Experiment Setting

• Experiments use SWE-Smith and SWE-Bench training and test splits, SWE-Smith validation, and two weak–strong pairs with gemini-3-pro-preview as the strong model.The weak models are gpt-5-mini and deepseek-v3.2; each has lower per-instance cost and resolved rate than its paired strong model. The value head uses a LoRA-fine-tuned Qwen2.5-Coder-7B-Instruct classifier, and baselines include embedding-based logistic regression, k-NN, XGBoost, and the non-temporal router.Source: §5.2 · p. 3

5.3. Results

• Route-AUC improves for both model pairs; deepseek-v3.2 routing reaches 0.780, +15.3 pp over the non-temporal baseline, while gpt-5-mini trajectories improve +12 pp over the strongest baseline.The SWE-Router curve also rises above the random-assignment band for both pairs and briefly exceeds the all-strong endpoint.Source: Figure 2 · p. 4, §5.3 · p. 4

6. Related Work

• SWE-agent introduced an interface for browsing, editing, and executing code, while SWE-bench established repository-scale issue resolution against hidden tests.Earlier code benchmarks focused on self-contained problems.Source: §6 · p. 4
• FrugalGPT introduced cost-aware cascades, and RouteLLM formalized binary routing from preference data for cost–quality gains.Source: §6 · p. 4
• Existing routing methods condition on the prompt alone, unlike SWE-Router, which conditions on the partial agent trajectory.Source: §6 · p. 4

7. Conclusion

• SWE-Router conditions on a short partial trajectory from a cheap weak model rather than on the task description alone.Source: §7 · p. 4
• Theorem 4.1 shows that trajectory conditioning never harms routing and is strictly better when exploration is informative.Source: §7 · p. 4
• On SWE-Bench Verified, SWE-Router improves Route-AUC by at least +12 pp over the strongest baseline across two LLM pairs.Source: §7 · p. 4

Impact Statement

• The approach is reported to reduce frontier-model inference cost and energy use with minimal overall-performance tradeoff.Source: Impact Statement · p. 5
• Partial-trajectory routing could be misused to bypass safety-focused frontier models for malicious purposes.Source: Impact Statement · p. 5
• The authors identify routing that considers task safety alongside other objectives as future work.Source: Impact Statement · p. 5

A.1. Data Collection

• Weak-model trajectories are generated because SWE-smith supplies trajectories only for claude-3.7-sonnet, and the new data are released for training value functions.Resolved and unresolved labels are assigned by running SWE-smith unit tests against each submitted patch.Source: §A.1 · p. 8
• SWE-smith is partitioned by repository into about 1.7k training trajectories per weak LLM, about 210 validation tasks, and 170–346 test tasks.The test partition comes from a separate repository set; validation is used for threshold tuning.Source: §A.1 · p. 8
• Cross-distribution SWE-Bench Verified evaluation uses 500 issues across four LLMs, but mix-1 leaves only 100 issues genuinely held out.Four fifths of the instances enter training under mix-1.Source: §A.1 · p. 8

A.2. Experiment Designs

• Mix-1 combines SWE-smith training data with four fifths of a five-fold SWE-Bench Verified partition, uses SWE-smith tasks for validation, and has separate test splits for both sources.The split design is described as having separate SWE-smith and SWE-Bench Verified test splits.Source: §A.2 · p. 8
• The value head is trained for five epochs with a shared forward pass applying predictions at every user-turn boundary, up to Kmax = 4.Training uses paraphrased task questions and an 8192-token context limit.Source: §A.2 · p. 8
• The prompt-only baselines use text-embedding-3-large embeddings with logistic regression, k-NN, or XGBoost, alongside a non-temporal K=0 variant.The K=0 variant conditions only on the task description.Source: §A.2 · p. 8

B. Additional Experiment Results

• SWE-Router significantly outperforms all baselines in Route-AUC on SWE-Bench Verified test, while gpt-5-mini routing does not exceed the baselines on SWE-smith test.The caption reports these split-specific comparisons but provides no cell values in the supplied passage.Source: Table 2 · p. 9
• On SWE-smith, validation and test Route-AUC are not consistently correlated for the deepseek-v3.2 router.The authors attribute this pattern to repository-based distribution shift between the splits.Source: Table 2 · p. 9
• Table 2 reports AUROC and Route-AUC across SWE-smith validation, SWE-smith test, and SWE-Bench Verified test splits.Its rows group prompt-only embedding baselines, the K=0 variant, and SWE-Router at fixed budgets K=1–4.Source: B. Additional Experiment Results · p. 8, Table 2 · p. 9

C. Proof of Theorem 4.1

• The proof compares prompt- and trajectory-conditioned routing values by expressing the value difference through the expected absolute conditional utility gap.It uses that the prompt is a function of the partial trajectory, then applies conditional Jensen inequality to establish V(St) ≥ V(Q); equality holds when the trajectory adds no information about the utility gap.Source: C. Proof of Theorem 4.1 · p. 10

D. Additional Related Work

• The software-engineering related work spans agent platforms and methods, while SWE-smith supplies synthesized bug-and-fix data and SWE-RM supplies execution-free patch-scoring rewards.The value head is described as related to SWE-RM but operating on partial trajectories.Source: D. Additional Related Work · p. 10
• Repository-level benchmarks evaluate cross-file completion and tool use, and SWE-bench has multimodal, multilingual, higher-rigor, live, and performance extensions.The discussion notes contamination concerns for public-code tasks and identifies SWE-bench Pro as using private repositories.Source: D. Additional Related Work · p. 10
• Prior LLM routing work includes embedding, prompt-classification, graph, cost-controlled, reasoning, and thought-level routers, as well as RL-based policies and routing benchmarks.EmbedLLM and universal model routing embed prompts and models to target generalization to unseen models.Source: D. Additional Related Work · p. 10