Language Distances are Practical for Equitable Cross-Lingual Transfer

1 Introduction

• The disparity in transfer-selection utility across target resource levels remains understudied, alongside the task dependence of individual language distances.No single distance is best across all NLP tasks.Source: §1 · p. 1
• Across 10 tasks and two multilingual models, the study compares four source-selection paradigms by performance loss and task- and resource-level inequality.Performance loss is measured relative to an oracle source language.Source: §1 · p. 2
• Language-distance rankers mitigate the high- versus low-resource performance-loss gap, closing it entirely with LightGBM on XLM-R transfer.Source: §1 · p. 2
• Individual distances and English-always transfer show task-dependent variance; composite distances reduce it, and trained rankers reduce it further.Source: §1 · p. 2
• NNRank is competitive with composite distances but retains a large performance gap on low-resource languages.Source: §1 · p. 2
• Language distances are dataset-independent and need neither language models nor target-language corpora; the authors recommend trained rankers when task evaluations exist and composite distances otherwise.Source: §1 · p. 2

2.1 Language Distances for Cross-Lingual Transfer

• Language distances use typological, geographic, phylogenetic, script, or lexical features, and toolkits can cover thousands of languages at low computational cost.Source: §2.1 · p. 2

2.2 Transfer Language Selection

• Transfer language selection ranks candidate source languages for a target; approaches include individual or composite distances, trained rankers, and model-internal NNRank.Individual distances choose the closest source; composite distances average measures, trained rankers use evaluation data, and NNRank uses model representations.Source: §2.2 · pp. 2–3

2.3 Inequalities in Cross-Lingual Transfer

• No single distance is best across tasks, and whether selection utility also varies with target resource level remains underexplored.Source: §2.3 · p. 3

3 Experimental Setup

• For each task-model setting, a target language is a query and candidate source languages are ranked to identify high-performing transfer sources near the top.Source: §3 · p. 3

3.1 Task Setup

• The benchmark covers sentence classification, sequence labeling, question answering, and generation, using zero-shot XLM-R and mT5 transfer across source-target languages.The task and language coverage is summarized in Table 1.Source: §3.1 · pp. 3–4, Table 2 · p. 5
• Performance loss measures relative cost against an oracle-best source, with normalization reflecting that the same performance difference costs more when oracle performance is low.The study also reports unnormalized loss and uses conformal trial complexity as an auxiliary shortlist diagnostic.Source: §3.1 · p. 4

3.2 Rankers

• Six standalone distances and two composites are compared with LightGBM and MLP trained separately per task-model setting; trained rankers require task-specific performance evaluations.Composite-Equal averages distances, while Composite-RRF sums inverse rank positions; English-always transfer is also evaluated where applicable.Source: §3.2 · pp. 4–5

4.1 Task inequality

• Individual distances can be the strongest choice for a setup, but no single distance is systematically superior across tasks.Examples of task-specific strengths include genetic distance for SIB-200 with XLM-R, script distance for SIB-200 with mT5, and typological distance for WikiANN with mT5.Source: §4.1 · pp. 5–6
• English-always can perform best on some setups but is often among the worst, with task-dependence comparable to individual distances.Its ICC(1) is 0.66 on mT5 and 0.41 on XLM-R.Source: §4.1 · p. 6
• Averaging distances reduces task inequality without task-specific ranker training and can remain competitive: Composite-RRF has the lowest loss on UD-POS with XLM-R and is strongest on smaller-coverage TyDiQA.On TyDiQA, Composite-RRF performance loss is 6.1 for XLM-R and 8.6 for mT5.Source: §4.1 · p. 6
• Composite distances and trained rankers are more uniform across tasks than individual distances; LightGBM has ICC 0.06 on mT5 and the best performance in 8 of 18 Table 2 columns.Composite-Equal has the lowest XLM-R ICC at 0.10, with Composite-RRF close at 0.13.Source: §4.1 · p. 6

4.2 Resource level inequality

• LightGBM nearly closes the resource-level gap: LRL-HRL gaps are -3.1 on XLM-R and -1.7 on mT5, while training-free composite gaps remain positive at 15.9 and 13.6, respectively.Across both task and resource analyses, trained rankers show the strongest reduction in inequality.Source: Figure 2 · p. 7, §4.2 · pp. 7–8
• Individual distances and English-always show the largest resource-level inequality, with gaps from +12.5 to +36.6 for individual distances and +20.0 on XLM-R and +32.3 on mT5 for English-always.For individual distances, the gap reflects similarly high loss relative to the oracle for HRL and LRL targets while oracle performance is lower for LRL targets.Source: §4.2 · p. 7

5 Comparison with NNRank

• NNRank requires model access and parallel text, whereas composite distances require neither model internals nor task-specific evaluations or target-language corpora.Trained rankers still require model evaluations for related targets, but not access to hidden states or model-specific representations.Source: §5 · p. 8, §5.2 · pp. 8–9

5.1 Setup

• NNRank ranks sources by similarity between parallel Bible verses encoded by mBERT at layer 8, using 100 sampled eBible verses per target language.The evaluation is restricted to languages with available Bible translations.Source: §5.1 · p. 8

5.2 Results

• Distance-based methods often match or outperform NNRank, and LightGBM is superior in at least one model for every task except WikiANN and OPUS-100.Composite-Equal outperforms NNRank on almost half of task/model setups.Source: Table 4 · p. 8, §5.2 · p. 8
• NNRank has the lowest performance loss on at least one model for UD-POS, WikiANN, TyDiQA, XNLI, and OPUS-100.Its hidden representations may contain cross-lingual compatibility information not captured by language distances.Source: Table 4 · p. 8, §5.2 · p. 8
• For low-resource targets, LightGBM has the lowest median loss and tightest interquartile range, while NNRank has the highest median loss in both models.Composite-Equal also improves on NNRank; the same ordering appears for mT5, where low-resource losses are higher overall.Source: Figure 3 · p. 9, §5.2 · p. 9

6 Conclusion

• Across ten tasks and two multilingual models, composite distances substantially reduce task-dependent variance and narrow resource-level performance gaps, while trained rankers reduce variance further and effectively close the gaps.Trained rankers are feasible only when task-specific transfer evaluations are available.Source: §6 · p. 9
• Model-internal selection is competitive with composite distances, but language-distance methods excel for low-resource languages.The evaluation covers four transfer language selection paradigms.Source: §6 · p. 9
• Language distances support equitable transfer without model access or target-language corpora and individually cover thousands of languages.The conclusion presents them as a practical basis for transfer language selection.Source: §6 · p. 9

Task assumptions

• The evaluation restricts transfer selection to zero-shot, single-source settings and leaves multi-source and few-shot extensions to future work.The paper notes that multi-source transfer can improve performance and few-shot target-language fine-tuning can narrow the zero-shot gap.Source: Limitations · pp. 9–10

Language coverage

• Existing NLP datasets cover only a small subset of languages supported by the distance measures, and composite distances can be unavailable for up to 16% of languages in a task.The analysis finds an inherent bias toward languages with more resources despite robustness for low-resource languages.Source: Limitations · p. 10

Model coverage

• The evaluation covers only XLM-R and mT5, leaving comparisons with newer, especially decoder-only, multilingual models to future work.Source: Limitations · p. 10

• Source-language rankings are recommendations for downstream tasks, not linguistic or sociocultural judgments about languages or communities.The authors caution against interpreting these rankings as statements about intrinsic value or importance.Source: Ethical Considerations · p. 10
• Because the analysis uses public datasets and models, it may inherit implicit linguistic biases in those resources.The authors report collecting or presenting no new human-subject data.Source: Ethical Considerations · p. 10

A Conformal Trial Complexity for Calibrated Source Language Shortlists

• Performance loss measures the cost of a ranker’s first recommendation, while CTC measures a calibrated trial budget for finding a near-best source.The authors introduce CTC because top-1 performance does not fully characterize shortlist usefulness when practitioners can try several sources.Source: A Conformal Trial Complexity for… · p. 15, Table 5 · p. 16
• CTC estimates how many source languages must be evaluated for a calibrated shortlist to include an operationally near-best source with high probability.The calibration targets a 90% chance of including a source within 5% relative performance loss of the oracle best, under the stated statistical assumptions.Source: A Conformal Trial Complexity for… · p. 15
• CTC’s marginal coverage guarantee depends on exchangeability of target-language queries within a task-model setting and a ranking rule fixed before held-out evaluation.The authors report CTC only when a setting has at least 100 target-language queries because small calibration sets can yield an operationally uninformative budget.Source: A Conformal Trial Complexity for… · p. 15
• On Taxi1500, LightGBM is best on both performance loss and CTC for both models, whereas SIB-200 with XLM-R shows a trade-off between the two metrics.On SIB-200 with XLM-R, genetic distance has the lowest performance loss but a very large CTC, while MLP shows the opposite trend.Source: A Conformal Trial Complexity for… · p. 16
• The nominal 90% shortlist hit-rate target is met for every mT5 task up to standard error, but XLM-R entries fall below 90%, even when CTC is small.The authors suggest that target-query heterogeneity, not just shortlist length, contributes to the XLM-R results.Source: A Conformal Trial Complexity for… · p. 16, Table 5 · p. 16

B Resource Inequality on Unnormalised Performance Loss

• English-always is the only paradigm with significantly higher unnormalised loss on low-resource targets, while trained rankers significantly reduce that loss.The gaps for individual and composite distances are not significant.Source: B Resource Inequality on Unnormalised… · p. 17, Table 6 · p. 17

• The lowest-loss ranker is significantly better than always-English in almost all tasks where English is available.The comparisons use performance-loss estimates with 95% confidence intervals and paired tests.Source: C Testing and Uncertainty Quantification · p. 17
• Larger tasks generally have narrower confidence intervals for both the lowest-loss ranker and always-English, without a noticeable systematic advantage for either.The authors illustrate this pattern by contrasting Taxi1500, which has substantially more target languages, with TyDiQA.Source: C Testing and Uncertainty Quantification · p. 17
• Composite and trained methods significantly outperform individual distances in 12 settings, with no significant result in the opposite direction.Trained rankers significantly outperform composites in five settings, composites outperform trained rankers in one, and the paradigms are indistinguishable in 12.Source: C Testing and Uncertainty Quantification · p. 17

D Cross-Task Generalisation

• On unseen tasks, trained rankers have the lowest performance loss in four of 18 settings, versus six for composite and eight for individual distances.Using the better of LightGBM and MLP, cross-task performance is worse in 13 settings, better in four, and essentially unchanged in one.Source: D Cross-Task Generalisation · pp. 17, 20, Table 10 · p. 19

E Language Filtering

• Filtering languages with unavailable distances changes LightGBM’s low-resource performance loss on mT5 by at most 3.9 percentage points, and the authors find no impact on resource-level findings.At most 20.3% of low-resource languages, or 16% of all languages, are removed in any task.Source: E Language Filtering · p. 20, Table 11 · p. 20

F Source Selection for Low-Resource Targets

• For mT5 low-resource targets, LightGBM recovers part of the random-source-to-oracle gap on every task, showing that source selection remains meaningful.On Taxi1500, random-source performance is 12.1 versus 24.3 for the oracle; the oracle-to-median-source gap remains 19.2 points for XLM-R and 12.8 for mT5.Source: F Source Selection for Low-Resource… · p. 20, Table 12 · p. 20

G.1 Model evaluation

• XLM-R and mT5 training configurations are listed in Tables 13 and 14; UD uses 10,000 training samples to handle imbalance and a biaffine head for dependency parsing.Table 13 defines LR, batch size, training limit, and early-stopping patience; it also identifies task-specific configurations and references.Source: §G.1 · p. 20, Table 13 · p. 21, Table 14 · p. 22

G.2 Composite-RRF

• Composite-RRF uses k = 60, following the original RRF paper’s near-optimal setting, which was reported to be relatively insensitive to the exact value.The passage cites Cormack et al. (2009).Source: §G.2 · p. 21

G.3 Trained rankers

• Both trained rankers use six minmax-normalised distance features; LightGBM applies gradient-boosted trees with LambdaRank optimised for NDCG@3.LightGBM follows Lin et al. (2019), uses hyperparameter choices from Ng et al. (2026), and applies early stopping on validation loss.Source: §G.3 · p. 21
• The MLP has two hidden layers of 64 and 32 units, uses ReLU and dropout, and is trained with Adam and listwise ListNet loss.It outputs one scalar and uses early stopping on validation loss; ListNet minimises cross-entropy between softmaxed predicted and actual performance, averaged across target languages.Source: §G.3 · p. 21

H Computing Infrastructure

• Training XLM-RoBERTa Base and mT5-Base used one NVIDIA H100 80GB GPU and approximately 750 GPU hours; ranker experiments took an estimated six hours on an Apple M1 Pro.The models have 270M and 580M parameters, respectively.Source: H Computing Infrastructure · p. 21

I Language Coverage of NNRank Comparison

• The NNRank comparison with Composite-Equal and LightGBM uses a smaller language subset, requiring exact ISO 639 code matches in eBible; Table 15 reports source and target counts.The passage attributes the eBible corpus to PABNLP (2026).Source: I Language Coverage of NNRank Comparison · p. 21, Table 15 · p. 22

J Artifacts

• The study’s artifacts and licenses are listed in Table 16, and the artifacts and datasets were handled according to their licenses.The authors state these materials were used to study language representations and cross-lingual transfer.Source: J Artifacts · p. 21, Table 16 · p. 23

K Usage of AI Assistants

• Generative AI use was limited to manually verified code completions and proofreading.Source: K Usage of AI Assistants · p. 22