RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models

1 Introduction

• Assembling off-the-shelf LLMs can harness their complementary abilities and outperform reliance on a single LLM.The models can vary in strengths and weaknesses across tasks.Source: §1 · pp. 1–2
• Routing selects a suitable LLM per query and is more efficient at inference than ensembling, which queries all candidates before merging their outputs.Ensembling can merge outputs through majority voting [28] or pairwise ranking.Source: §1 · p. 2
• Existing routing can be suboptimal when multiple LLMs perform well: ZOOTER’s normalized scores can yield small, nearly uniform probabilities and weak supervision.The paper also notes that score differences between the top two LLMs are usually tiny in its experimental setting.Source: §1 · p. 2
• RouterDC uses sample-LLM contrastive learning to draw query embeddings toward top-performing LLMs and away from bottom-performing ones, and sample-sample contrastive learning to stabilize training.Training queries are clustered, with same-group queries pulled together and different-group queries pushed apart.Source: §1 · p. 2
• RouterDC is reported to have fewer than 100M parameters, avoid backpropagation through LLMs during training, and run 6× faster than Voting at inference.Inference computes only for the selected LLM, with negligible router cost.Source: §1 · p. 2
• +2.76% in-distribution and +1.90% out-of-distribution: RouterDC outperforms individual top-performing LLMs and existing routing methods.The paper reports these gains across the two task settings.Source: §1 · p. 3

Large Language Models (LLMs)

• Foundation models have been fine-tuned into specialized LLMs for tasks such as language understanding, code generation, and mathematical reasoning.This work studies assembling those models with a router to harness their strengths.Source: §2 · p. 3

LLM Ensembling

• Ensembling combines outputs from multiple LLMs, but methods that query all candidates require at least O(T) inference calls; RouterDC calls only the selected model.Voting [28], PairRanker and GenFuser, cascading [2], and Fusion of Experts are described as ensembling approaches.Source: §2 · p. 3

LLM Routing

• Existing routers include correctness classifiers [40], reward-model supervision in ZOOTER, task-wise routing [52], and classifier- or clustering-based approaches [41].The paper argues these approaches neglect queries for which multiple LLMs are well suited.Source: §2 · p. 3
• RouterDC uses contrastive losses to consider the strengths of several powerful LLMs for a query, addressing a limitation attributed to prior routers.The paper frames this as a distinction from routing approaches that neglect multiple suitable models.Source: §2 · p. 3

Contrastive Learning

• Contrastive learning distinguishes similar from dissimilar data pairs and has been used in visual, sentence, and vision-language representation learning.RouterDC applies two contrastive losses to learning a router for assembling LLMs.Source: §2 · p. 3

3.1 Problem Formulation

• Router learning selects a suitable LLM for each query, with the router expected to generalize across in- and out-of-distribution query distributions.The formulation assumes no single LLM is suitable for every training query and notes that differing architectures make merging all LLMs infeasible.Source: §3.1 · p. 3

3.2 Scoring

• For open-ended queries, the scoring procedure samples multiple LLM outputs and evaluates correctness; for multiple-choice queries, it scores option probabilities and assigns zero to an incorrect option.For multiple-choice questions, probabilities are normalized when the predicted option is correct to make scores comparable across LLMs.Source: §3.2 · p. 4

3.3 Sample-LLM Contrastive Loss

• The sample-LLM contrastive loss pulls a query embedding toward top-scoring LLM embeddings and away from bottom-scoring ones, allowing multiple positive LLMs for a query.The paper uses K+ = 3 in its experiments and argues that aligning router probabilities directly with scores can be unsuitable when multiple LLMs perform well.Source: §3.3 · pp. 4–5

3.4 Sample-Sample Contrastive Loss

• Because sample-LLM-only training can be unstable when similar queries receive dissimilar embeddings, the sample-sample loss encourages similar embeddings for queries in the same cluster.Training queries are clustered using embeddings from a pretrained encoder, dimensionality reduction with t-SNE, and k-means.Source: §3.4 · pp. 5–6

3.5 Training and Inference

• Training minimizes a combined objective of the sample-LLM and sample-sample contrastive losses, while inference selects the highest-probability LLM and generates its response.RouterDC has fewer than 100M parameters and does not backpropagate gradients through the LLMs; it calls the selected LLM once, unlike voting and cascade methods that require multiple calls.Source: §3.5 · p. 6

4.1 Experimental Setup

• Experiments use seven open-source LLMs across five in-distribution and three OOD tasks, pooling in-distribution training sets to train the router.GSM8K uses its default split; the other in-distribution datasets are randomly split 70/30.Source: §4.1 · pp. 6–7

4.2 Main Results

• RouterDC has the highest average accuracy on five in-distribution tasks, exceeding the best individual LLM by 3.98%.It improves on the top individual model on GSM8K (+0.51%), ARC-C (+0.57%), and HumanEval (+1.63%); it also beats Voting on four of five tasks and is about 6× faster in inference.Source: Table 1 · p. 7, §4.2 · pp. 7–8
• On three OOD tasks, RouterDC has the highest average accuracy, exceeding the best individual LLM by 1.9%.It is roughly comparable to the best individual model on each task, and is the only routing method reported to surpass that model.Source: Table 2 · p. 7, §4.2 · p. 8

4.3 Sensitivity Analysis

• Testing accuracy is better with both contrastive losses than with the sample-LLM loss alone, while remaining insensitive to λ ∈[0.5, 5] and to N ∈[4, 9].The average performance is also negligibly affected by H ≥1; H=0 makes the sample-sample loss constant.Source: §4.3 · p. 8, Figure 5 · p. 8, Figure 6 · p. 8
• Across tested K+ and K− settings, RouterDC outperforms the best individual LLM; selecting only the top LLM as positive performs worse.The weaker configuration uses K+ = 1 and K−= 6.Source: Figure 8 · p. 8, §4.3 · p. 8

4.4 Analysis

• Query assignments show task structure: most GSM8K and PreAlgebra queries go to the two math-capable LLMs, and embedding similarities roughly align with assignments.The cited examples are MetaMath-Mistral-7B and dolphin-2.9-llama3-8b.Source: Figure 9 · p. 8, Figure 10 · p. 8, §4.4 · p. 9
• On GSM8K, adding the sample-sample loss stabilizes training, separates task embeddings, raises an example query-pair similarity from −0.4589 to 0.9982, and improves testing accuracy.Without that loss, embeddings for different tasks are roughly mixed; the reported training curve also oscillates considerably.Source: §4.4 · p. 9, Figure 12 · p. 9, Figure 11 · p. 9
• Adding LLMs consistently raises average accuracy, and RouterDC withstands the loss of any single LLM during inference.The paper attributes this robustness to multiple top-scoring LLMs serving as backups for one another.Source: §4.4 · p. 9, Figure 13 · p. 9

5 Conclusions

• RouterDC effectively assembles LLMs and outperforms top individual LLMs and existing routing methods on in-distribution and out-of-distribution tasks.The conclusion describes the approach as using sample-LLM and sample-sample contrastive losses.Source: §5 · pp. 9–10
• The authors identify applying the two contrastive losses to other routing problems as future work.Source: §5 · p. 10

H MMLU GSM8K CMMLU ARC-C HumanEval Avg

• Adding LLMs consistently enhances average accuracy across the five in-distribution tasks.Table 7 reports testing accuracy by number of LLMs and task.Source: H MMLU GSM8K CMMLU ARC-C HumanEval Avg · p. 15, Table 7 · p. 15

• One example query asks how much of a $100 store credit remains unpaid after two partial payments.Source: B Detailed Results for Section 4.4 · p. 15
• A second example asks how much more money is needed for a $100 wallet after family contributions.Source: B Detailed Results for Section 4.4 · p. 15
• Figure 14 visualizes training-query embeddings extracted by the learned encoder using t-SNE.Source: Figure 14 · p. 15

• Testing accuracy saturates quickly as the number of training samples per task increases.Source: C Effectiveness of #training samples · p. 15
• About 100 samples per task are reported as sufficient for learning the router.Source: C Effectiveness of #training samples · p. 15
• With 30 samples per task, RouterDC achieves 57.21 overall versus 55.77 for the previous SOTA.Source: C Effectiveness of #training samples · p. 15

D Discussions on Availability of Task Identity

• The sample-sample contrastive training uses clustered samples, while available task identities can instead define the groups.The passage introduces the task-identity comparison as an alternative to clustering samples into N groups.Source: D Discussions on Availability of Task… · pp. 15–16
• On five in-distribution tasks, RouterDC is comparable to RouterDC with task identity, supporting the effectiveness of unsupervised clustering.The comparison replaces the unsupervised sample groups with groups defined by task identity.Source: D Discussions on Availability of Task… · p. 16, Table 8 · p. 16

E Discussions on Single Task Setting

• RouterDC largely outperforms the best candidate LLM, dolphin-2.9-llama3-8b, and existing routing methods on the HumanEval single-task test.The router is trained and tested on HumanEval splits, a setting where queries may share the same task information.Source: E Discussions on Single Task Setting · p. 16, Table 9 · p. 16

F Discussions on the Distant OOD Task

• RouterDC outperforms existing routing methods by a large margin on the distant OOD JavaScript task.JavaScript code generation is contrasted with HumanEval’s Python code generation.Source: F Discussions on the Distant OOD Task · p. 17, Table 10 · p. 17

G Effectiveness of Lsample-sample for ZOOTER

• +1.52% for ID and +0.81% for OOD are the reported improvements from adding Lsample-sample to ZOOTER.The comparisons use in-distribution and out-of-distribution testing accuracy, respectively.Source: G Effectiveness of Lsample-sample for… · p. 17, Table 11 · p. 17, Table 12 · p. 17

H Effectiveness of punishing s(t)

• Punishing s(t)_i performs better on average across five in-distribution tasks.The comparison evaluates RouterDC with and without this design.Source: H Effectiveness of punishing s(t) · p. 17, Table 13 · p. 18

I Disccusions on Incorporating Cost

• RouterDC is more cost-effective than CosineClassifier and ZOOTER on GSM8K and MBPP in RouterBench.Figure 16 presents the testing accuracy comparisons with different costs.Source: I Disccusions on Incorporating Cost · pp. 17–18, Figure 16 · p. 18

J Limitations

• Limited computational resources restrict evaluation to 8B LLaMA-based and 7B Mistral-based candidate models.The authors note that larger public models are available, but do not evaluate them here.Source: J Limitations · p. 18
• The framework’s extension to selecting LLMs from recent conversation context remains uninvestigated future work.The paper describes chat-context routing as a possible extension to its query-based design.Source: J Limitations · p. 18