JevSoup: System-One Routing for Training-Free LoRA Composition

• The paper frames LoRA libraries as modular specialized capabilities whose effective use requires choosing relevant experts and combining their contributions for each input.System One and System Two provide the paper’s analogy for expert selection and execution.Source: §1 · p. 1
• Prior approaches may require auxiliary data, router training, or autoregressive decoding, while adapter-based signals can tie routing to the backbone and leave task relevance implicit.Jev maps inputs and expert descriptions directly to probabilities over expert identifiers, but selection alone does not specify how to compose adapters.Source: §1 · p. 1
• JevSoup uses Jev to select an ordered expert pair, then retains the leading update, projects the second against its row space, and combines them equally.The framework requires neither expert training samples nor additional router or expert training.Source: §1 · pp. 1–2, Figure 1 · p. 2
• Across 14 tasks and three model scales, JevSoup achieves absolute gains of up to 1.19% in task-macro and 1.21% in sample-micro accuracy over the strongest evaluated external baselines.Source: §1 · p. 2

2.1. Problem Definition

• Given an input and an expert library, the method seeks an input-dependent update; in multiple-choice evaluation, routing sees prompts and candidate answers but not gold answers or task labels.Source: §2.1 · p. 2

2.2. JevSoup Framework

• Jev uses a typed choice interface to return probabilities over expert identifiers from the input and fixed capability cards, rather than generating an expert name.Source: §2.2 · p. 3
• The normalized probabilities select an ordered Top-2 pair; the higher-ranked update is retained while the other is projected to remove its component overlapping the anchor’s row space.The projection acts on frozen LoRA updates without gradients or training.Source: §2.2 · p. 3
• The anchor and projected update are combined with equal weights, without restoring the projected update’s norm or renormalizing branch weights.The two low-rank branches operate within one frozen backbone rather than averaging A and B factors or ensembling predictions.Source: §2.2 · p. 3
• The projection is implemented directly on the second expert’s low-rank factor, with the original scale unchanged and the anchor row-space basis obtained through a reduced QR decomposition and singular vectors.Source: §2.2 · p. 3

3.1. Experiment Setup

• Across 19,477 examples from 14 PorTAL tasks, evaluation uses task-macro and sample-micro accuracy on Qwen3-1.7B, 4B, and 8B with frozen experts.Baselines share the backbone, expert library, and scoring protocol; AdapterSoup uses 100 training examples per expert.Source: §3.1 · p. 3

3.2. Main Results

• JevSoup has the highest task-macro and sample-micro accuracy at all three scales, exceeding AM Top-2 in macro accuracy by 1.13, 1.19, and 0.21 points.Its micro accuracy is also higher than each external baseline at all three scales.Source: §3.2 · p. 3, Table 1 · p. 4

3.3. Ablation Study

• 75.30% macro and 73.81% micro accuracy make equal-weight projected Jev Top-2 the strongest 4B ablation, 0.66 and 0.32 points above unprojected Jev Top-2.Under equal weights without projection, Jev Top-2 exceeds AM Top-2 by 0.53 macro and 1.00 micro points; equal weights outperform probability weights.Source: §3.3 · p. 4, Table 2 · p. 4

3.4. Sensitivity Analysis

• With fixed routing and equal weights, tested projection strengths yield positive point-estimate gains over λ = 0; macro accuracy spans 75.01–75.34% and micro 73.72–73.95%.The reported variation is 0.33 and 0.23 percentage points, respectively, and λ = 1 is retained for the main results.Source: §3.4 · p. 4, Figure 2 · p. 4

4. CONCLUSION

• Across 14 PorTAL tasks and three Qwen3 model scales, JevSoup achieves the highest task-macro and sample-micro accuracy among evaluated methods.Jev selects an ordered pair from the input and expert descriptions; composition retains the first update, projects the second onto the orthogonal complement of its row space, and combines them equally.Source: §4 · p. 4

6. COMPLIANCE WITH ETHICAL STANDARDS

• No ethical approval was required for this study.Source: §6 · p. 5