PaperlayerView exact revision on arXiv

Source-linked AI summary

Query-aware routing for Cross-lingual performance gains in Encoders

Akshay Jain, Edward Kim

arXiv:2610.02875v1cs.CLcs.AIcs.IR

TL;DR

Cross-lingual retrieval can lag despite strong same-language performance, while changing the document encoder requires recomputing stored corpus vectors. The paper pairs a query-only adapter trained against frozen document embeddings with language-based routing; on sampled FIQA, SampoTron raises mean cross-language nDCG@10 from 0.241 to 0.291 across six directions, with same-language scores unchanged.

  • Problem

    Cross-language retrieval may be less effective than same-language retrieval, and changing a document encoder requires recomputing stored corpus vectors.

    Source: §1 · p. 1

  • Method

    A query-only low-rank adapter learns against frozen document embeddings and is selected for cross-language queries, while same-language queries use the original encoder.

    Source: §1 · p. 2

  • Results

    On sampled FIQA, SampoTron raises mean cross-language nDCG@10 from 0.241 to 0.291, with all six directions improving and same-language scores unchanged.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The routed approach supports cross-language specialization while retaining the original same-language path and reusable document vectors.

    Source: §6 · p. 8

    Evaluation covers English, Finnish, and Swedish; translated FIQA may contain systematic artifacts, and native Swedish lacks a dedicated benchmark.

    Source: §5.3 · p. 8

Abstract

from arXiv · show

Multilingual encoders can exhibit reduced retrieval effectiveness when queries and relevant documents differ in language, despite strong same-language performance. We investigate whether Finnish and Swedish cross-lingual retrieval can improve while preserving an encoder's existing same-language performance and document index. We combine a query-only low-rank adapter, trained against frozen document embeddings, with deterministic routing based on query and index languages. Cross-language queries use the adapter, while same-language queries use the original encoder. SampoTron, our fine-tuned low-rank (LoRA) adapter alongwith the Nemotron-3-Embed-1B model, improves average retrieval quality across six English, Finnish, and Swedish directions from 0.241 to 0.291 in normalized discounted cumulative gain (nDCG) at rank ten, a 20.9% relative gain on a sampled financial benchmark. All six cross-lingual directions improve, and routing preserves the original same-language performance, including two full-corpus Finnish evaluations. The approach enables selective cross-language specialization with reusable document embedding vectors.

1 Introduction

The paper seeks to improve cross-language retrieval without replacing the useful same-language retriever or recomputing its document index. It combines query-only adaptation with language-based routing and evaluates the approach across English, Finnish, and Swedish retrieval.

1 Introduction

  • The practical challenge is to improve cross-language retrieval without sacrificing a useful same-language retriever or requiring existing corpus vectors to be recomputed.Dense retrieval quality also constrains the candidate evidence available to downstream reranking and response generation.Source: §1 · p. 1
  • The method trains a low-rank query adapter against frozen document embeddings and activates it when query and index languages differ.One cross-language adapter serves all evaluated language pairs.Source: §1 · p. 2
  • The routed architecture reuses the original document index and preserves same-language scores by retaining the original encoder path.Same-language preservation follows from the inference path, not from the adapted model retaining monolingual performance.Source: §1 · p. 2
  • The paper analyzes how contrastive learning moves adapted queries toward relevant fixed document embeddings relative to distractors, and when routing preserves original rankings.The contribution concerns both query-to-document geometry and ranking-preservation conditions.Source: §1 · p. 2
  • SampoTron delivers a 20.9% mean cross-language gain on the sampled FIQA corpus, with all six language pairs improving and same-language scores preserved.SampoHarrier, with a separately trained query adapter, also improves cross-language retrieval on another base model.Source: §1 · p. 2

2 State of the art

Prior work spans multilingual representations, training methods for balancing monolingual and cross-lingual retrieval, and query or language adapters. This paper distinguishes its approach through explicit language routing and evaluates Finnish and Swedish alongside English.

2 State of the art

  • Multilingual E5 [15] and Qwen3 Embedding [19] provide multilingual representations for retrieval and other embedding or reranking tasks.Qwen3’s training pipeline also incorporates synthetic multilingual data.Source: §2 · p. 2
  • Earlier retrieval methods combine monolingual and cross-lingual training [1], distributional alignment with contrastive training [3], or English passages as a bridge.These approaches seek to preserve monolingual performance through training.Source: §2 · p. 2
  • Related adapter work includes LoRA [4], language and retrieval adapters [16], task-specific query and passage adapters [12], and fixed-index query adaptation [5].These works establish neighboring uses of adapters in multilingual or enterprise retrieval.Source: §2 · pp. 2–3
  • The paper combines a frozen-encoder query adapter with an explicit language rule: use the adapter across languages and the vanilla encoder otherwise.Same-language preservation therefore follows from inference-time selection.Source: §2 · p. 3
  • The evaluation covers English, Finnish, and Swedish, including both Finnish–Swedish directions, and reports gains on translated financial retrieval alongside maintained Finnish monolingual performance on Wikipedia.The findings are over the selected base models and datasets.Source: §2 · p. 3

3 Architecture and embedding alignment

The architecture adapts queries while keeping document embeddings fixed, then routes requests using query and index languages. Its contrastive objective aligns adapted queries with relevant documents relative to competitors, while routing preserves same-language rankings under stated conditions.

3.1 A fixed document encoder and an adaptable query path

  • Documents use the frozen encoder while a low-rank adapter updates query representations, allowing the original document index to be reused.For SampoTron, the encoder selects the first 1,024 coordinates of its 2,048-dimensional output; training uses positive and negative document vectors from the frozen encoder.Source: §3.1 · p. 3, Figure 1 · p. 3

3.2 Contrastive objective and query–document alignment

  • Reducing the contrastive objective favors query similarity to relevant documents over competing candidates, with softmax-weighted attraction and repulsion shaping the update.The document vectors remain stationary targets during training.Source: §3.2 · p. 4
  • The gradient lies in the candidate-document span, while normalization makes its correction angular; this geometry does not guarantee distinct language subspaces or improvement toward every positive.Subspace alignment is indicative of geometric agreement, not an absolute measure of convergence.Source: §3.2 · p. 4

3.3 Language routing and same-language preservation

  • Language labels select base versus adapted queries; same-language rankings persist under unchanged preprocessing, index, search, and tie-breaking, while mixed-language pools remain out of scope.The policy assumes one known language per index; if requests share adapter state, selection and encoding must be uninterrupted.Source: §3.3 · p. 5

4 Experimental setup

SampoTron trains a query-only LoRA adapter against frozen-base document vectors, using a multilingual financial-retrieval training mixture. Evaluation covers sampled cross-language and same-language FIQA retrieval plus full-corpus Finnish-only benchmarks.

4.1 SampoTron: Query - Routed LoRA Adapted Nemotron-3-1B-BF16 Embedder

  • The 113,043-row training mixture combines FIQA, MIRACL-derived, and Mr. TyDi-derived examples, with direction counts that are not balanced overall.Cross-language rows include machine-translated Finnish and Swedish text; FIQA hard negatives are mined with BM25, and matching test queries are excluded from training.Source: §4.1 · p. 5
  • Only LoRA parameters are trainable, and the document cache contains 90,490 unique texts encoded by the frozen base model.The implementation compares the original model, the adapter on every query, and the routed system using the same original document vectors.Source: §4.1 · p. 5, Table 1 · p. 6
  • Training settings report LoRA rank 64, but other ranks were also used and only the most optimal SampoTron settings are reported.Early stopping monitors trailing 20-step mean training loss from epoch 2.2 and stops after 60 optimizer steps without at least a 1% relative decrease.Source: §4.1 · p. 6

4.2 Evaluation methodology and metrics

  • The FIQA evaluation uses 194 questions in each of English, Finnish, and Swedish, testing three same-language and six cross-language directions against the same 14,330-passage sample.The sample is drawn from a 57,638-passage corpus and includes all 1,706 passages labeled relevant across the test questions plus 12,624 BM25-selected distractors.Source: §4.2 · p. 6
  • Finnish-only evaluation uses 1,271 MIRACL-fi and 1,738 Mr. TyDi-fi development questions against their full passage collections.The collections contain 1,883,509 and 1,908,757 Finnish passages, respectively.Source: §4.2 · p. 6
  • Retrieval ranks normalized-embedding dot products, reports nDCG@10, and averages scores across queries before calculating equal-weight direction means.Each passage is treated as relevant or not relevant to a query.Source: §4.2 · p. 6

5 Results and discussion

SampoTron improves cross-language retrieval on the sampled FIQA evaluation, while routing retains the original same-language scoring path. The results also report deployment characteristics and boundaries of the evaluation.

5.1 Cross-language gains in the final system

  • 0.2406 to 0.2908 nDCG@10 is the six-direction cross-language mean for SampoTron, a 20.9% relative gain, with all six directions improving.Figure 2 shows the routed language matrix and routed-minus-base differences for 194 query identities per direction and a 14,330-passage sample.Source: §5.1 · p. 7, Figure 2 · p. 7

5.2 Preserving multi-monolingual performance through routing

  • Applying the adapter to every query lowers Finnish MIRACL from 0.756 to 0.675 and Mr. TyDi from 0.728 to 0.665, despite a slight increase in mean same-language FIQA performance.Routing instead preserves every original diagonal score and both native-Finnish scores while retaining adapted-query gains for cross-language requests.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Deployment implications and limitations

  • The SampoTron adapter occupies approximately 84 MB and reuses the original document index, avoiding corpus re-encoding and duplicate storage for the adapted route.The evaluation covers Finnish and Swedish with English; it has no dedicated native-Swedish benchmark and uses translated FIQA queries and passages.Source: §5.3 · pp. 7–8
  • The study does not measure sensitivity to incorrect language labels, downstream answer quality, organizational productivity, or public-service outcomes.The authors note that translated FIQA material may not fully represent naturally authored queries.Source: §5.3 · p. 8

6 Conclusions

The conclusion presents query-aware routing as a way to improve cross-language retrieval while retaining same-language behavior and reusing the original document index. Reported gains are demonstrated on sampled FIQA financial retrieval.

6 Conclusions

  • 0.241 to 0.291 mean cross-language nDCG@10 is reported for SampoTron on sampled FIQA, with all six English–Finnish–Swedish directions improving and original same-language scores unchanged.SampoHarrier also obtains cross-language gains with a separately trained query adapter on another base model.Source: §6 · p. 8
  • The approach reuses the base model and document index; future evaluation is proposed for naturally authored Swedish queries, additional Nordic languages, broader domains, and mixed-language corpora.The paper also proposes testing whether combining routing with other cross-lingual training objectives adds retrieval gains while preserving same-language performance.Source: §6 · p. 8

Related papers

Loading 2610.02875v1…