Source-linked AI summary
SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations
Yu Wang, Yuchen Li, Rui Kong, Xinran Chen, Jiamin Chen, Hengyi Cai, Shuaiqiang Wang, Jiashu Zhao, Yulun Zhang, Zhonghao Lyu, Haoyi Xiong, Linghe Kong, Jimmy Xiangji Huang, Dawei Yin
TL;DR
Multi-turn routing requires handling historical context carefully because single-turn routers do not account for how history is segmented and retained, which can cause information loss and confusion. SWRouter combines similarity-based context construction with decoupled evaluation of construction and router performance. On ShareGPT and MTBench, it achieves 16.26% and 8.22% gains over the best single LLM and Conv-ID Context, respectively.
Problem
Single-turn routing methods do not directly transfer to multi-turn dialogue because routing depends on how historical context is segmented and retained, creating information loss and confusion.
Method
SWRouter uses similarity-based prompt construction and a decoupled evaluation framework to separate construction accuracy from router performance.
Results
SWRouter achieves 16.26% and 8.22% gains over the best single LLM and Conv-ID Context, respectively, on multi-turn benchmarks.
Takeaways & Limitations
Multi-turn LLM routing requires jointly designing context construction and evaluation rather than directly extending single-turn routing methods.
Takeaways & Limitations
Single-turn routing assumes the current query provides sufficient routing input, an assumption that does not hold in multi-turn dialogue.
Abstract
from arXiv · showhide
Large language models exhibit complementary strengths, motivating routing methods that dispatch each query to the most suitable model. Although existing routers are effective in single-turn settings, they do not directly transfer to multi-turn dialogue, where routing performance critically depends on how historical context is segmented, retained, and incorporated into the current prompt. This introduces two fundamental challenges: preventing information loss and information confusion during context construction, and evaluating routing quality without conflating model selection with prompt construction quality. In this paper, we propose SWRouter, a Similarity-Contractive Window Router for multi-turn large language model routing. SWRouter combines a similarity-based context segmentation mechanism for prompt construction with a dual-metric evaluation framework that decouples construction accuracy from router performance. Experiments on multi-turn dialogue benchmarks demonstrate that SWRouter consistently surpasses strong baselines, achieving a 16.26% improvement in evaluation accuracy over the best individual large language model and an additional 8.22% gain over the Conv-ID Context baseline. Our results highlight that multi-turn large language model routing requires a joint design of context construction and evaluation, rather than a direct extension of single-turn routing methods.
1 Introduction
Multi-turn routing must construct relevant context before selecting a model, because missing or obsolete history can distort responses and obscure whether failures arise from context construction or routing. SWRouter addresses these issues with similarity-based context segmentation, a pluggable router, and decoupled evaluation, outperforming individual LLMs and the Conv-ID Context baseline.
- 1 Introduction: Multi-turn routing depends on how dialogue history is segmented, retained, and incorporated, unlike single-turn routing based on an isolated prompt.Existing single-turn routers such as RouterDC and C2MAB-V do not directly address this context-construction requirement.
- 1 Introduction: Observed response accuracy conflates context-construction quality with model-selection quality because prompt quality affects all candidate models.The paper therefore distinguishes construction accuracy, evaluation accuracy, and router performance.
- 1 Introduction: SWRouter combines similarity-based context segmentation, contrastive router training on constructed prompts, and decoupled evaluation of construction and routing.Its context mechanism selectively incorporates semantically relevant historical turns before routing.
- 1 Introduction: Poor context segmentation creates information loss by omitting necessary history and information confusion by retaining obsolete context after topic shifts.These failures can produce under-specified or misleading prompts for the routed model.
- 1 Introduction: SWRouter improves average accuracy by 16.26% over the best individual LLM and gains 8.22% over the Conv-ID Context baseline on multi-turn benchmarks.The framework also reports a 2.68% improvement over Conv-ID Context on out-of-distribution tasks.
2 Background and Motivation
Multi-turn routing must select a model while also determining which historical context to preserve and how to incorporate it. SWRouter therefore combines similarity-based context construction with decoupled evaluation rather than directly extending single-turn routing.
- Multi-turn routing requires deciding both which model to select and which historical context to preserve and incorporate.
- Omitting relevant history causes information loss, whereas retaining irrelevant or outdated history causes information confusion before routing.
- Single-turn routers operate on the current prompt, but multi-turn prompts may be under-specified or misleading without proper history segmentation.
- Relative-ranking evaluation can conflate prompt-construction quality with router quality because prompts with identical rankings may yield different true scores.
- SWRouter addresses these challenges with similarity-based window partitioning for prompt construction and a decoupled evaluation framework.
3 Preliminaries and Problem Setup
The problem setup represents each routing instance as a current request paired with constructed historical context, then selects among a fixed pool of candidate LLMs. It separates relative supervision for model selection from absolute scores for evaluating prompt quality and routing outcomes.
- The context-construction function transforms raw dialogue history into a retained semantic window and context-enhanced prompt before routing.
- A multi-turn routing instance consists of a current user request and a constructed prompt containing selected historical information.
- The router outputs selection probabilities over a fixed candidate-model pool and aims to approach the highest-quality candidate for the constructed prompt.
- Relative candidate scores supervise router learning, while absolute response-quality scores support decoupled evaluation.
- The setup separately measures evaluation accuracy, construction accuracy, and router performance because ranking alone cannot show whether the constructed prompt is good.
- The study routes among already available LLMs without modifying them, with context construction occurring before generation.
4 Methodology
SWRouter jointly constructs semantically coherent context windows, routes context-enhanced prompts to candidate LLMs, and evaluates construction quality separately from model-selection quality.
- Framework overview: SWRouter uses similarity-based window partitioning, contrastive model routing, and decoupled evaluation to address multi-turn context construction and attribution.The framework distinguishes preserved context quality from routing effectiveness using absolute response scores.
- Similarity-based window partitioning: Adjacent user-turn similarity determines whether a turn extends the active window or starts a new semantic segment.Assistant turns within the retained window are preserved in the final prompt.
- Model routing: The router encodes the constructed prompt, compares it with learnable candidate-model embeddings, and selects a model from the resulting routing distribution.Contrastive objectives encourage separation between stronger and weaker candidate models under multi-turn prompts.
- Decoupled evaluation: Construction accuracy averages candidate-model scores, while evaluation accuracy measures the selected model and router performance normalizes it by the candidate average.These metrics enable separate analysis of prompt construction and routing effectiveness.
- Similarity-based window partitioning: Similarity windowing preserves relevant history while reducing obsolete or off-topic context, addressing information loss and information confusion.The threshold controls the trade-off between context retention and context purity.
5 Implementation
The implementation evaluates SWRouter on two multi-turn dialogue datasets with seven heterogeneous open-source LLM candidates and a standardized judge-based protocol.
- Candidate LLMs: Seven open-source candidates span Mistral-based and Llama-3-based general-purpose, instruction-tuned, domain-tuned, and language-adapted models.This heterogeneous pool tests routing across differing model capabilities.
- Datasets: Experiments use MTBench and filtered multi-turn ShareGPT data, split into 70% training and 30% testing.MTBench covers diverse task categories, while ShareGPT contains real user-AI interactions.
- Baselines: Baselines include single-model selection, Conv-ID Context with original conversation segmentation, and ZOOTER with similarity-window segmentation.The latter combination tests whether the windowing mechanism generalizes beyond SWRouter’s backbone.
- Evaluation protocol: Generated candidate responses are scored by judge models, and routing is evaluated using testing weighted accuracy.The evaluation generates M=10 responses using stochastic beam search with temperature 0.2.
- Configuration: The similarity threshold is set to τ=0.91, with 768-dimensional LLM embeddings and N=5 clusters.Router training uses AdamW for 1,000 steps with learning rate 5 × 10^-5, weight decay 0.01, and batch size 16.
6 Evaluation
The evaluation measures routed response quality with judge-model scores and examines multi-turn performance, OOD generalization, component contributions, hyperparameter sensitivity, and computational overhead.
- Evaluation goals: The evaluation addresses routing performance, OOD generalization, component contributions, hyperparameter sensitivity, and computational overhead.These questions define the study’s main empirical dimensions.
6.2 Overall Performance
SWRouter achieves the strongest reported in-distribution and average OOD results, with similarity-window construction contributing most to performance and threshold choice affecting both routing and response quality.
- Overall performance: 16.26%: SWRouter raises average evaluation accuracy from 24.63% for dolphin-2.9-llama3-8b to 40.89%, with gains on both datasets.The gains are 14.29% on ShareGPT and 18.22% on MTBench.
- Overall performance: 8.22%: SWRouter improves average evaluation accuracy over Conv-ID Context, indicating dynamically constructed semantic windows outperform exact ID-based partitioning.The comparison supports semantic context construction as a stronger routing input strategy than static conversation-ID segmentation.
- OOD generalization: 43.99%: SWRouter achieves the best average OOD performance, surpassing Conv-ID Context by 2.68% across PreAlgebra, MBPP, and C-EVAL.It wins on PreAlgebra and MBPP but not C-EVAL.
- Ablation study: 11.33% and 46.57%: removing similarity-window construction reduces evaluation accuracy on ShareGPT and MTBench, respectively.This is the largest reported ablation drop and identifies adaptive context construction as the dominant component.
- Ablation study: 25.70% and 38.17%: removing the sample-LLM loss reduces ShareGPT and MTBench performance, while sample-sample alignment also significantly hurts MTBench.The results support both contrastive objectives as contributors to the routing space.
- Threshold sensitivity: τ=0.91 is the best operating region because threshold selection changes both router performance P_router and absolute routed response quality s_i.The threshold therefore affects model-selection effectiveness and end-to-end response quality together.
6.5 Sensitivity Analysis
SWRouter’s sensitivity depends on the similarity threshold, with τ=0.91 providing the strongest or most stable results across construction and routing metrics.
- Similarity-window construction: 40.89% accuracy at τ=0.91 outperforms Conv-ID Context’s 32.67%, while other tested thresholds perform substantially worse.Accuracy falls to 25.6% at τ=0.85, 22.2% at τ=0.90, 20.2% at τ=0.92, and 18.3% at τ=0.95.
- Average Value of ¯w: 23.70% is the maximum MT-Bench average ¯w at τ=0.91, but overly strict τ=0.95 reduces it to 13.70%.
- Average Value of ¯w: 14.90% is ShareGPT’s highest reported average ¯w at τ=0.87, remaining comparable at 14.70% for τ=0.91 before declining at τ=0.95.
- Average Value of ¯w: τ=0.91 provides a stable trade-off between sample quality and coverage across MT-Bench and ShareGPT.
- Router Performance: 21.5% is the peak router performance at τ=0.91, compared with 15.4% at τ=0.87 and 12.1% at τ=0.95.
6.6 Robustness Analysis
Robustness analyses show that similarity-based retrieval recovers related turns despite intervening distractors, while long dialogues generally retain semantic coherence.
- Retrieval robustness: Similarity-based Top-K retrieval tests whether an earlier related query can be recalled after unrelated turns interrupt the dialogue.
- Retrieval robustness: Top-5 recall remains above 66% with 20 distractors, while Top-3 retrieval offers an effective recall–efficiency trade-off.
- Semantic drift: Over 70% of long-range ShareGPT turn pairs exceed cosine similarity 0.91, with average similarity above 0.92.
- Semantic drift: MT-Bench shows slightly lower average similarity than ShareGPT, but extremely low similarity cases below 0.80 remain rare.
6.7 Detailed Overhead Analysis
SWRouter adds little inference overhead and achieves favorable token-cost trade-offs, while the analyses indicate that prompt construction strongly influences routing outcomes.
- Inference overhead: Less than 1% of total inference time, approximately 2.2‰, is spent on SWRouter’s similarity calculation.
- Computational overhead: SWRouter’s training uses a lightweight encoder and is performed once before deployment, while candidate LLM inference remains dominant.
- Cost–performance comparison: 8.22% higher evaluation accuracy than Conv-ID Context costs approximately 1.23× its tokens, while a 1.69% gain over ZOOTER costs 1.02×.
- Additional analysis: τ=0.91 performs best among tested thresholds, and prompt construction quality is reported to outweigh router performance.
- Additional analysis: A weak negative correlation, Pearson R=-0.2301, links router performance with true scores, supporting decoupled evaluation metrics.
- Additional analysis: 40.89% evaluation accuracy at τ=0.91 exceeds Conv-ID Context’s 32.67%, while exact context partitioning is not necessarily optimal.
7 Related Work
Related work spans model combination, cascades, and routing, but prior routers generally assume complete prompts; SWRouter treats context segmentation and evaluation as routing components.
- Single-turn LLM routing: Prior routing methods generally assume each routing instance is already a complete prompt, an assumption that becomes fragile in multi-turn dialogue.
- Multi-turn context construction: Full-history concatenation can introduce irrelevant information, whereas current-turn-only prompts can omit essential context needed to track intent.
- Context construction: SWRouter integrates similarity-based window construction with router learning so the selected model receives a prompt designed for multi-turn relevance.
- Evaluation of routing under constructed prompts: Standard evaluation can conflate prompt information preservation with selecting the best model under that prompt.
- Evaluation of routing under constructed prompts: SWRouter reports evaluation accuracy, construction accuracy, and router performance, making context segmentation a first-class routing component.
8 Conclusion
SWRouter addresses information loss, information confusion, and attribution bias in multi-turn routing through similarity-based context construction and decoupled evaluation. It improves in-distribution and out-of-distribution performance, with gains attributed primarily to construction accuracy.
- 8 Conclusion: SWRouter combines similarity-based window partitioning with three complementary metrics to construct coherent prompts and distinguish evaluation accuracy, construction accuracy, and router performance.The framework jointly addresses context construction, router training, and evaluation in multi-turn routing.
- 8 Conclusion: 16.26% and 8.22% gains over the best individual LLM and Conv-ID Context, respectively, demonstrate improved performance on ShareGPT and MTBench.The reported gains are supported by extensive experiments on the two multi-turn dialogue benchmarks.
- 8 Conclusion: 2.68% average improvement over Conv-ID Context on PreAlgebra, MBPP, and C-EVAL indicates stronger out-of-distribution performance.These tasks form the reported out-of-distribution evaluation setting.
- 8 Conclusion: Decoupled metric analysis attributes the gains primarily to improved construction accuracy, while P_router > 1 validates effective model selection.The analysis separates prompt construction effects from router performance.
- 8 Conclusion: The framework is currently evaluated only on 7B/8B-scale models, leaving behavior with larger or proprietary LLMs for future study.Future work proposes heterogeneous model pools spanning different scales and proprietary models.