Source-linked AI summary
PSK at WMT 2026 MIST: Task-Specialized QLoRA Adapters for Multilingual Summarization and Question Answering
Srikar Kashyap Pulipaka
TL;DR
The paper addresses multilingual instruction following across context QA, summarization, and open QA under the WMT MIST setting. It uses a shared 3.35B Tiny Aya Global backbone with separate QLoRA adapters trained on task-specific multilingual data, including scientific paper–abstract pairs for summarization. On held-out data, context QA and summarization adapters outperform the multitask adapter, while open-QA results remain mixed and motivate three submissions with different open-QA adapters.
Problem
WMT MIST requires multilingual models to handle context QA, cross-lingual summarization, and open-ended generation across 24 languages under a 10B-parameter limit.
Method
The system uses one 3.35B Tiny Aya Global backbone with three task-specific QLoRA adapters trained on multilingual task data, including scientific papers paired with author-written abstracts.
Results
Context and summarization adapters perform better than the organizer-data multitask adapter on held-out data, while open-QA results vary with answer length and evaluation method.
Takeaways & Limitations
The submission keeps the context and summarization adapters fixed while varying the less stable open-QA adapter across three systems.
Takeaways & Limitations
Reference metrics do not directly measure factuality, are particularly limited for open QA, and the router assumes that the task label is available.
Abstract
from arXiv · showhide
We describe the PSK submission to the WMT 2026 Multilingual Instruction Shared Task. Our system uses the 3.35B-parameter Tiny Aya Global model with three QLoRA adapters, one for each task. The adapters are trained on multilingual document-summary pairs, passage-based question answering, and filtered standalone question answering. The summarization data also includes scientific papers with their author-written abstracts. On our held-out split, the context and summarization adapters perform better than our multitask adapter, which was trained only on data supplied by the organizers. Results for open QA are mixed and vary with answer length and evaluation method. We therefore submit three systems with the same context and summarization adapters but different open-QA adapters.
1 Introduction
MIST evaluates multilingual models on context-based question answering, cross-lingual summarization, and open-ended generation across 24 languages. PSK addresses these tasks with one multilingual backbone and three task-specific adapters.
- MIST evaluates context-based question answering, document summarization from language X to language Y, and open-ended generation under a 10B-parameter limit.The test set covers 24 languages and includes same-language and cross-lingual generation.
- PSK uses one multilingual backbone with three task-specific adapters whose supervision and decoding policies follow each task's output structure.The design retains a single 3.35B-parameter backbone while specializing task handling.
2 Task and System Overview
The system routes each labeled MIST task through a corresponding LoRA adapter and decoding configuration while keeping the backbone shared. It starts from Tiny Aya Global and initializes task-specific adapters from a multitask model trained on organizer-provided data.
- The inference-time task label selects one of three LoRA adapters and its corresponding decoding configuration, with no language-specific routing.All languages use the same task route once the task label is known.
- Tiny Aya Global is a 3.35B-parameter model covering 70 languages with an 8K context window.An initial multitask adapter is trained on 20,293 provided examples using an 85/15 split, producing 3,626 held-out examples for model selection.
- The task-specific adapters are initialized from the multitask adapter trained on the provided data.This initialization supports separate continuation for each task.
3 Training Data
The selected adapters use task-specific multilingual supervision assembled from organizer data and external resources. Summarization additionally includes full scientific papers paired with author-written abstracts, while open QA is filtered to standalone factual and explanatory questions.
- Validation examples and exact test prompts are excluded from the task-specific training data.The data summary describes supervised data used by the selected adapters, each initialized from the multitask adapter.
- The 12,000-example summarization mixture is split evenly between general and scientific text.Scientific examples pair full ACL Anthology papers with author-written abstracts, while non-English targets are filtered for language, length, repetition, and semantic consistency.
- The context-QA training set combines Belebele, answerable TyDi QA, MLQA, MCIF, UPDESH, and small Czech and Yoruba Aya subsets.MLQA contributes parallel and cross-lingual examples.
- Open QA combines provided examples, WMT25 MIST questions, and Aya data while retaining short factual and explanatory questions.Passage-dependent, multiple-choice, creative-writing, code, translation, and continuation tasks are excluded.
4 Training and Inference
Training uses QLoRA adapters with task-specific optimization settings, and inference uses deterministic greedy decoding. Repetition controls are added for open QA and summarization to address repetitive development-set outputs.
- QLoRA uses 4-bit NF4 quantization and BF16 computation, with LoRA applied to attention and feed-forward projections.Each adapter is trained for one epoch with effective batch size 16; rank is 16, alpha is 32, and dropout is 0.05.
- Learning rates are 1×10^-4 for summarization, 8×10^-5 for open QA, and 2 × 10^-4 for context QA.These settings are task-specific within the shared QLoRA training framework.
- Inference uses greedy decoding without sampling.Repetition penalties are 1.05 for open QA and 1.03 for summarization, with repeated 4-grams blocked in summarization outputs.
5 Development Results
Development results guide adapter selection for context QA, summarization, and open QA. The selected routes balance task-specific metrics and qualitative behavior, including scientific-summary performance and open-QA answer length.
- Context QA: The 8.5k context mixture is selected because it exceeds targeted continuation on EM and chrF, despite the latter being marginally higher on ROUGE-L and LaBSE.The comparison uses internal model-selection results on identical held-out rows.
- Scientific Summarization: The 12k summary mixture is strongest across all three metrics on a 336-example scientific validation set.The validation targets are author-written abstracts, with non-English targets translated and quality filtered.
- Scientific Summarization: Blocking repeated 4-grams removes the observed summary loops.
- Open QA and Submitted Systems: Best-score QA has the strongest aggregate automatic scores, while Long-form QA performs better on manually identified long-form prompts and reaches the generation limit less often.The main submission uses Long-form QA, and a second uses Best-score QA.
- Open QA and Submitted Systems: The submitted routes retain the selected context and summary mixtures while varying the open-QA route.Table 4 summarizes the three submitted routes.
6 Analysis
Separate task adapters improve context QA and summarization on development data, while open-QA results depend on metric choice and answer length. The language-level results reinforce strong summarization gains but mixed open-QA outcomes.
- Separate adapters improve context QA and summarization on the development set, whereas open QA has no clear winner.
- Best-score QA performs better on automatic metrics, while Long-form QA works better on manually checked longer questions and is less likely to hit the output limit.
- The shared summarization route improves chrF over the multitask adapter in all 28 represented languages.
- The shared context-QA route improves exact match in 19 of 25 languages, ties in four, and declines in two.
- Open-QA chrF improves in 9 of 19 languages for the primary route and 10 of 19 for variant A.
7 Conclusion
The submission presents a routed Tiny Aya Global system with one QLoRA adapter per MIST task. Development results fix the context and summarization routes while leaving open-QA routes varied across submissions.
- The system routes MIST tasks through one QLoRA adapter per task.
- Development results select the 8.5k context mixture and 12k summary mixture for submission.
- The three submissions vary the less stable open-QA route while sharing the selected context and summarization adapters.Official evaluation provides the shared-task comparison.
Limitations
The evaluation has important measurement and scope boundaries: reference metrics do not directly measure factuality, especially for open QA, and results come from one backbone and development split.
- Reference metrics do not directly measure factuality and are particularly limited for open QA.
- Translated scientific targets may retain artifacts, and the reported results use one backbone and one development split.
- The router assumes that the task label is available at inference time.
Ethics Statement
The system uses public and challenge-provided datasets under their respective terms and warns that generated outputs may reflect biases or contain harmful claims. Verification is required before treating the system as a factual authority.
- Public and challenge-provided datasets are used under their respective terms.
- Machine translation may reproduce source and model biases, while open-ended outputs may contain incorrect or harmful claims.
- The system should not be treated as a factual authority without verification.
A Language-Level Changes
Language-level held-out comparisons show broad gains for the shared summarization and context-QA routes, while open-QA changes remain mixed across systems and languages.
- All submitted systems share the summary and context routes, while the primary, variant A, and variant B use Long-form, Best-score, and multitask open-QA adapters, respectively.
- Context-QA changes are exact-match percentage points, whereas summary and open-QA changes are chrF points.