Source-linked AI summary
PlurVA-LLM-2026 Shared Task Track-1: Pluralistic Value Alignment in LLMs via Multilingual Fine-Tuning and Threshold Calibration
Vihindi Kotalawala, Nevidu Jayatilleke
TL;DR
Pluralistic value alignment must represent diverse preferences across populations, while multilingual capability alone does not ensure consistent alignment. The system fine-tunes a shared multilingual model with dataset-specific supervision and calibration strategies. It achieves a final macro-average accuracy of 0.805 across Chinese, Indonesian, and Sri Lankan data.
Problem
Conventional alignment may average diverse human preferences, and multilingual capability alone does not ensure consistent value alignment across languages and resource settings.
Source: §1 · p. 1Method
The system jointly fine-tunes a shared multilingual model with dataset-specific supervision, including vote expansion for Indonesian data and binary reformulation, augmentation, and threshold calibration for Sri Lankan data.
Source: §1 · p. 1, §3.2.2 · p. 4, §3.2.3.1 · p. 6Results
0.805 macro-average accuracy was achieved by the final selected model, with Sri Lankan accuracy reaching 0.916 after conditional threshold calibration.
Source: §3.4.1 · p. 7Takeaways & Limitations
The results suggest that shared multilingual learning combined with supervision tailored to each dataset's characteristics is a promising direction for pluralistic value alignment.
Source: §4 · p. 8The authors identify a need for larger, more diverse value-alignment datasets and improved methods for modeling annotator disagreement and multilingual transfer.
Source: §4 · p. 8
Abstract
from arXiv · showhide
We present our system for the PlurVA-LLM 2026 Shared Task Track-1, which focuses on pluralistic value alignment in the contexts of China, Indonesia, and Sri Lanka. For this resource-constrained track, we fine-tuned Llama 3.1 8B Instruct using 4-bit QLoRA. Our approach combines option-permutation augmentation for Chinese data, annotator vote expansion for Indonesian data, and binary reformulation with SinhalaMMLU augmentation for Sri Lankan data. We further applied conditional threshold calibration to the predictions for the Sri Lankan data. The final system achieved accuracies of 0.785 for Chinese, 0.715 for Indonesian, and 0.916 for Sri Lankan, resulting in an overall macro-average accuracy of 0.805.
1 Introduction
Pluralistic alignment seeks to represent culturally diverse values rather than assume one universal preference, since conventional methods may average preferences and multilingual capability alone does not ensure consistent alignment. The shared task evaluates locally grounded judgments across Chinese, Indonesian, and Sri Lankan contexts, and the system combines shared multilingual fine-tuning with dataset-specific supervision and Sri Lankan threshold calibration.
1 Introduction
- Pluralistic alignment aims to represent diverse population values because conventional alignment may learn averaged preferences, while multilingual capability alone does not guarantee consistent value alignment.The paper motivates this need by citing variation across populations and high- and low-resource languages.Source: §1 · p. 1
- The PlurVA-LLM 2026 Shared Task evaluates locally grounded value judgments in Chinese, Indonesian, and Sri Lankan contexts using datasets with distinct cultural foundations and annotation structures.The passage identifies D2VBench as the China daily-life-scenario dataset.Source: §1 · p. 1
- The system combines shared multilingual fine-tuning with dataset-specific supervision, using conditional threshold calibration for comparatively limited and imbalanced Sri Lankan data.Country-specific prompts identify the relevant value setting while one shared adapter learns across contexts.Source: §1 · p. 1
2 Related Work
Related work covers pluralistic preference modeling, culturally grounded multilingual alignment, and challenges from limited data, disagreement, and cross-lingual variation. This paper's setting requires one model to perform across three datasets with differing resources and supervision structures, while language-specific strategies reflect those resource differences.
2.1 Pluralistic Value Alignment
- Prior studies address heterogeneous values through pluralistic preference learning, culturally adapted fine-tuning, and evaluation against diverse human-value distributions.Examples include MaxMin-RLHF, Modular Pluralism, CultureSPA, and WorldValuesBench.Source: §2.1 · p. 2
2.2 Challenges in Value Alignment
- Value alignment across languages is constrained by scarce preference data, cross-lingual inconsistencies, annotation aggregation that can suppress disagreement, and answer-option order bias.Collapsing annotations to a majority label can remove minority judgments when disagreement reflects genuine value differences.Source: §2.2 · p. 2
- The study seeks strong performance from one model across three datasets with different resources, annotation structures, and data availability, without improving one dataset at another's expense.Its proposed direction combines shared multilingual learning with dataset-specific supervision and inference strategies.Source: §2.2 · p. 2
2.3 Language and Resource Context
- The paper frames Chinese as the highest-resource language among the three and Indonesian as a Category 4 language.The cited language-resource descriptions distinguish the contexts considered for multilingual training.Source: §2.3 · pp. 2–3
Sinhala:
- Sinhala is classified as a lower-resourced Category 2 language and is the most resource-constrained language considered in the study.Source: Sinhala · p. 3
3 Methodology
The methodology uses a shared Llama-3.1-8B-Instruct model, adapted with 4-bit QLoRA, while transforming inputs and targets for the three datasets' different task structures. Zero-shot comparison selected this model for its highest macro-average accuracy and comparatively balanced results across languages.
3 Methodology
- The three datasets use distinct output structures: Chinese has one gold answer among four options, Indonesian has five annotators, and Sri Lankan examples require one of four labels for two statements.Source: §3 · p. 3
- All examples were converted into chat-formatted supervised records, and a single shared adapter was jointly trained on all three datasets using 4-bit QLoRA.The final system used Meta Llama-3.1-8B-Instruct; all available labeled records were used because the dataset was extremely small, with no validation partition reserved.Source: §3 · p. 3
3.1 Base-model Selection
- 0.6394 macro-average accuracy made Llama-3.1-8B-Instruct the strongest zero-shot shared base model, although the best individual-language results varied by model.Llama was selected for its comparatively balanced performance across the three languages.Source: §3.1 · p. 3, Table 1 · p. 3
3.2 Country-specific Supervision
The section adapts supervision to the distinct label and annotation structures of the Chinese, Indonesian, and Sri Lankan datasets. It combines option permutation, annotator-vote expansion, binary Sinhala decisions, and SinhalaMMLU augmentation.
3.2.1 Chinese
- 0.785 vs 0.779: cyclic option permutation improved Chinese accuracy while remapping labels to preserve answer identity.Four cyclic orderings also produced 790 examples of each output label.Source: §3.2.1 · p. 4
- At inference, Chinese and Indonesian records were evaluated under four option permutations, with probabilities mapped back and averaged.This was intended to make predictions less sensitive to displayed option order.Source: §3.2.1 · p. 4, §3.2.2 · p. 5
3.2.2 Indonesian
- 0.689 vs 0.651: retaining all five Indonesian annotations outperformed majority-vote targets, which discarded disagreement.The five vote targets preserved minority judgments across questions with annotator disagreement.Source: §3.2.2 · p. 4
- 0.715 vs 0.689: adding four cyclic option permutations to Indonesian vote expansion further improved accuracy.Each question generated 20 training records from five votes and four permutations, for 7,320 SFT rows.Source: §3.2.2 · pp. 4–5
3.2.3 Sri Lankan
- 0.834 vs 0.832: binary Sinhala formulation performed slightly better than four-way training while preserving all four final outcomes.The reformulation turned 203 questions into 406 binary records and reduced the severity of the original class imbalance.Source: §3.2.3 · p. 5
3.2.3.1 SinhalaMMLU Augmentation
- The 85 manually selected SinhalaMMLU questions were converted into two-statement records using retained questions or manual rewriting.When direct conversion was unsuitable, native Sinhala-speaking authors amended questions and statements while preserving relevant knowledge and value judgment.Source: §3.2.3.1 · p. 5
- 0.890 vs 0.873: adding 85 SinhalaMMLU questions improved accuracy in the augmented Sinhala setup.The combined set comprised 288 questions and 576 binary SFT rows.Source: §3.2.3.1 · pp. 5–6
3.3 Prompt and Target Design
Prompts supplied the relevant value category and used language-specific instructions while keeping Sri Lankan content in Sinhala. The final targets were short categorical answers; rationale and value-summary supervision were excluded because they did not improve overall performance.
3.3 Prompt and Target Design
- Prompts included the supplied value category; Chinese and Indonesian instructions used their respective languages, while Sri Lankan prompts retained Sinhala content with concise English instructions.The Sri Lankan instructions described the broader context, including Sinhalese, Tamil, Muslim, and Burgher communities.Source: §3.3 · p. 6
- Rationale supervision and value-description summaries were excluded from the final model because neither improved overall performance.The authors suggest extra explanations may add unnecessary or noisy information for short categorical decisions.Source: §3.3 · p. 6
- Final targets used only short machine-readable answer markers: A/B/C/D for Chinese and Indonesian, and Yes/No for Sri Lankan.Source: §3.3 · p. 6
3.4 QLoRA Fine-tuning
The system uses shared multilingual QLoRA fine-tuning and dataset-specific methods, then calibrates conditional thresholds for Sri Lankan predictions. The calibration improves the final model’s reported accuracy and macro-average results.
3.4 QLoRA Fine-tuning
- QLoRA fine-tuning inserted trainable LoRA matrices into attention and feed-forward projections, with loss restricted to assistant answer markers.The targets contained only final answer markers.Source: §3.4 · p. 6
3.4.1 Conditional Threshold Calibration
- 0.916 accuracy, up from 0.8900 with default thresholds; the calibrated shared-adapter model achieved the highest macro-average accuracy of 0.805.The selected thresholds were τB = 0.92, τA|B=0 = 0.16, and τA|B=1 = 0.50.Source: Table 4 · p. 7, Table 5 · p. 7, §3.4.1 · p. 7
3.4 QLoRA Fine-tuning
- 0.796 macro-average accuracy for the shared adapter exceeded 0.770 for language-specific adapters, so the shared setup was selected.The shared setup reached 0.715 and 0.89 on Indonesian and Sri Lankan data, respectively.Source: §3.4 · p. 6, Table 4 · p. 7
3.4.1 Conditional Threshold Calibration
- Five-fold cross-validation guided three conditional thresholds: B is evaluated first, and A’s threshold depends on whether B is predicted valid.The four resulting classes are A, B, Both, and 0; the model tended to over-predict valid B cases.Source: §3.4.1 · p. 7
- Increasing τB while decreasing τA|B=0 improved performance; the high τB mainly filtered lower-confidence B decisions rather than confident ones.The results suggest asymmetric thresholds fit the model’s probability distribution better than a uniform 0.5 threshold.Source: §3.4.1 · pp. 7–8
4 Conclusion
The paper presents a shared-model system with dataset-specific supervision for pluralistic value alignment across Chinese, Indonesian, and Sri Lankan contexts. It identifies unresolved challenges in representing annotator diversity, limited data, answer-position bias, and adaptation across datasets.
4 Conclusion
- The system combines shared multilingual modeling with dataset-specific supervision strategies for the three cultural contexts.The approach includes cyclic option permutation for Chinese, annotator vote expansion for Indonesian, and binary reformulation, SinhalaMMLU augmentation, and conditional threshold calibration for Sri Lankan data.Source: §4 · p. 8
- Important challenges remain in preserving diverse annotator opinions, working with limited value-alignment data, reducing answer-position bias, and adapting across datasets.The conclusion identifies larger and more diverse datasets, improved disagreement modeling, and more robust multilingual training as future-work directions.Source: §4 · p. 8
- The authors describe shared multilingual learning combined with tailored supervision as a promising direction for pluralistic value alignment.Source: §4 · p. 8
Limitations
The authors identify data scarcity and imbalance, the track’s model-size cap, and possible statement-order sensitivity in Sinhala calibration as limitations.
Data Scarcity:
- Limited, uneven training data may affect how well the shared model learns each value context, and using all labeled data for training left no independent validation set.Source: Data Scarcity · p. 8
Model Size Limit:
- Experiments were capped at 8B parameters by the shared-task constraint, so effectiveness may differ for larger models or other model families.Source: Model Size Limit · p. 8
Asymmetric Calibration:
- Asymmetric Sinhala calibration may be sensitive to statement order because statement B is evaluated before selecting statement A’s conditional threshold.Source: Asymmetric Calibration · p. 8
A Prompt Templates
The appendix presents the language-specific prompt templates and describes their structure, dataset-field substitutions, and language choices.
A Prompt Templates
- Each prompt contains a system instruction, a user template, and a constrained output format, with placeholders filled by dataset fields such as questions or candidate statements.Source: A Prompt Templates · p. 10
- Figure 5 presents three templates for each dataset across the Chinese, Indonesian, and Sri Lankan experiments.Source: Figure 5 · p. 10, A Prompt Templates · p. 11
- Chinese and Indonesian prompts use their respective languages for instructions and input, while Sinhala input remains Sinhala and its instructions are in English.Source: A Prompt Templates · p. 11
B Negative Experiments
The experiments examine rationale supervision and value-level context, varying whether generated rationales and value summaries are included while holding architecture and training hyperparameters constant.
B Negative Experiments
- Across four experiments, the model architecture and training hyperparameters stayed fixed; only the inclusion of generated rationales and value summaries differed.Source: B Negative Experiments · p. 11
- Rationales explain why an answer fits the tested value; Llama-3.1-8B-Instruct generated them, and only those yielding the expected answer were retained.Source: B Negative Experiments · p. 11
- Value summaries aggregate generated rationales by value and summarize their common reasoning patterns; Table 6 presents the experiment performance.Source: B Negative Experiments · p. 11, Table 6 · p. 11
C Threshold Sensitivity Analysis
The analysis selects a threshold for statement B first, then selects statement A’s threshold conditionally on B’s prediction, and presents threshold sensitivity for Sinhala.
C Threshold Sensitivity Analysis
- τB was set to 0.92 for the highest statement-B accuracy, then τA|B=0 was set to 0.16 and τA|B=1 to 0.50.Source: C Threshold Sensitivity Analysis · p. 11
- The final threshold configuration was (τB, τA|B=0, τA|B=1) = (0.92, 0.16, 0.50).Source: C Threshold Sensitivity Analysis · p. 11
- Figure 6 plots the effect of conditional threshold values on out-of-fold Sinhala accuracy.Source: Figure 6 · p. 11