PaperlayerView exact revision on arXiv

Source-linked AI summary

Typhoon-S: Minimal Open Post-Training for Sovereign Large Language Models

Kunat Pipatanakul, Pittawat Taveekitworachai

arXiv:2601.18129v1cs.CLcs.AI

TL;DR

Sovereign LLM development lacks a validated post-training recipe that fits limited-resource settings while preserving regional capabilities and general assistant behavior. Typhoon S combines lightweight SFT, on-policy distillation, and small-scale InK-GRPO-based RFT, producing strong general performance and improved Thai-specific reasoning and knowledge in a Thai case study. The authors present this as a practical route to transparent sovereign LLMs under academic-scale resources.

  • Problem

    Sovereign developers lack a clear, validated post-training recipe that supports regional needs under limited compute, data, and engineering resources.

  • Method

    Typhoon S combines lightweight supervised fine-tuning, on-policy distillation, and small-scale RFT with InK-GRPO, which adds next-token prediction to GRPO.

  • Results

    SFT and OPD transform base models into competitive instruction-tuned assistants, while InK-GRPO improves Thai legal reasoning and agentic retrieval without degrading general capabilities.

  • Takeaways & Limitations

    Careful post-training design can support practical, transparent, sovereign LLMs with academic-scale compute rather than scale alone.

  • Takeaways & Limitations

    The study focuses on Thai and does not explore pre-training or mid-training, leaving generalization to other languages and cultural contexts for future work.

Abstract

from arXiv · show

Large language models (LLMs) have progressed rapidly; however, most state-of-the-art models are trained and evaluated primarily in high-resource languages such as English and Chinese, and are often developed by a small number of organizations with access to large-scale compute and data. This gatekeeping creates a practical barrier for sovereign settings in which a regional- or national-scale institution or domain owner must retain control and understanding of model weights, training data, and deployment while operating under limited resources and strict transparency constraints. To this end, we identify two core requirements: (1) adoptability, the ability to transform a base model into a general-purpose assistant, and (2) sovereign capability, the ability to perform high-stakes, region-specific tasks (e.g., legal reasoning in local languages and cultural knowledge). We investigate whether these requirements can be achieved without scaling massive instruction corpora or relying on complex preference tuning pipelines and large-scale reinforcement fine-tuning (RFT). We present Typhoon S, a minimal and open post-training recipe that combines supervised fine-tuning, on-policy distillation, and small-scale RFT. Using Thai as a representative case study, we demonstrate that our approach transforms both sovereign-adapted and general-purpose base models into instruction-tuned models with strong general performance. We further show that small-scale RFT with InK-GRPO -- an extension of GRPO that augments the GRPO loss with a next-word prediction loss -- improves Thai legal reasoning and Thai-specific knowledge while preserving general capabilities. Our results suggest that a carefully designed post-training strategy can reduce the required scale of instruction data and computation, providing a practical path toward high-quality sovereign LLMs under academic-scale resources.

1 Introduction

Typhoon S addresses the lack of a clear, validated sovereign post-training recipe under limited resources. It separates adoptability from sovereign capability and proposes lightweight post-training methods for each.

  • Motivation: Resource gatekeeping concentrates state-of-the-art LLM development among organizations with large compute, data, and complex post-training pipelines.Most models also rely primarily on English- and Chinese-centric data, limiting accessibility for smaller research groups and national initiatives.
  • Motivation: Sovereign settings require control and understanding of model weights, data, and training methods alongside regional, cultural, and in-domain alignment.These settings often operate with constrained compute budgets and limited human expertise.
  • Motivation: Sovereign-adapted base models may retain strong regional knowledge while lagging in general instruction following, tool use, and agentic behavior.This capability gap limits their practical adoption despite strong performance on Thai regional benchmarks.
  • Research framing: Typhoon S defines adoptability as transforming a base model into a competitive general-purpose assistant and sovereign capability as handling region-specific high-stakes tasks.Thai is used as the representative sovereign setting, with the two capabilities studied in separate experimental setups.
  • Approach: The proposed adoptability recipe combines lightweight SFT with on-policy distillation to transform sovereign-adapted base models into competitive instruction-tuned assistants.It uses open-source instruction data together with target-language data.
  • Approach: InK-GRPO extends GRPO with a cross-entropy next-token prediction loss to improve domain-specific performance while teaching new knowledge in parallel.The method is applied in standard and agentic RFT settings, with particularly strong improvements in Thai legal reasoning.

2 Adoptability

Typhoon S uses a lightweight two-stage post-training pipeline to turn base models into instruction-following assistants under limited-resource conditions. Experiments show that on-policy distillation improves robustness, target-language data supports Thai capabilities, and the recipe transfers effectively to a sovereignty-adapted base model.

  • 2 Adoptability: Under a limited budget, the pipeline combines supervised fine-tuning with on-policy distillation, using open English instructions augmented by a small Thai dataset.The assumed setting is less than one 8 × H100 for under a week; the target-language data supports Thai alignment.
  • 2 Adoptability: Thai constraint mixing improves performance across mathematics, code generation, and agentic behaviors while preserving data efficiency.The dataset randomly mixes English and Thai constraints with Thai user instructions.
  • 2 Adoptability: +6.49 points: SFT+OPD raises average performance from 37.45 to 43.94 over SFT alone.The largest differences include Thai code-switching and tool use, while knowledge-focused benchmarks remain similar.
  • 2 Adoptability: Full-logits OPD reaches 43.94 average versus 42.81 for Top-K distillation, with the clearest advantage on Thai code-switching.Full-logits OPD scores 93.4 versus 69.8 for Top-K on code-switching, while Top-K remains comparable on some constrained tasks.
  • 2 Adoptability: Removing Thai data causes large SFT regressions, whereas OPD remains more robust and uses Thai data mainly to refine Thai-native capabilities.For SFT, Thai code-switching falls from 65.4 to 34.4 without Thai data; OPD’s average changes from 43.94 to 42.02.
  • 2 Adoptability: Applied to ThaiLLM-8B, the recipe produces Typhoon-S-8B, which surpasses Qwen3-8B on the strict Thai suite and achieves a Thai average of 71.20 versus 66.66.The resulting model improves Thai chat, code-switching, Thai knowledge, retrieval-style QA, and agentic tasks, though gaps remain in hard knowledge, mathematics, and coding.

3 Sovereign Capability

General-purpose instruction tuning can leave models weak on region-specific knowledge and reasoning. Typhoon-S addresses this with InK-GRPO and agentic RFT extensions built on GRPO.

  • Evaluation: Typhoon-S-8B is stronger on Thai-centric benchmarks but trails Qwen3-8B on English scientific knowledge and mathematics.Table 7 reports overall averages of 51.88 for Typhoon-S-8B and 54.72 for Qwen3-8B.
  • Agentic RFT: Agentic RFT enables multi-step reasoning with external tools in a controlled retrieval-augmented environment.The model learns to interleave reasoning and tool calls during training and inference.
  • Motivation: Standard RFT often amplifies existing knowledge rather than introducing domain facts, limiting its use in sovereign settings.These settings commonly under-represent regional content in both base models and general-domain instruction data.
  • InK-GRPO: InK-GRPO augments GRPO with stochastic next-token prediction on a separate in-domain corpus.The design injects domain knowledge during RFT while retaining task-focused reinforcement learning.

2. In-domain text corpus: Used to compute the auxiliary next-token prediction loss (LCE)

The sovereign-capability pipeline combines in-domain knowledge injection with agentic tool use and evaluates whether these additions improve specialized performance without broad capability loss.

  • Training design: InK-GRPO combines task-specific optimization with stochastic in-domain next-token prediction during RFT.This design is intended to acquire domain knowledge while maintaining task-focused optimization.
  • Results: InK-GRPO improves accuracy over GRPO-only training by 4% on NitiBench and 1.6% on MIRAGE-Bench.NitiBench scores 19.30% versus 15.82%, while MIRAGE-Bench scores 22.63% versus 20.99%.
  • Results: NitiBench benefits more strongly than MIRAGE-Bench because it emphasizes legal and multi-step reasoning.The passage reports more than a 100% relative improvement over the base model on NitiBench, whereas MIRAGE-Bench shows smaller but consistent gains.
  • Agentic results: In agentic RFT, InK-GRPO outperforms GRPO and enables a 4B model to exceed GPT-5-level task-specific performance.The model operates in a multi-turn scenario with external tool use.
  • General capabilities: Across settings, average scores remain near the Qwen3 Instruct baseline, indicating no broad degradation of general capabilities.Reported averages range from 48.08% to 49.55%, compared with 48.07% for the baseline.

4 Summary: Typhoon-S Recipe

Typhoon-S provides two minimal recipes: SFT plus OPD for adoptable instruction models, and Agentic InK-GRPO for Thai legal RAG agents. The reported models use open data, transparent details, and academic-scale compute.

  • Recipe: The final recipes target both base-to-instruct transformation and sovereign legal RAG capability.The resulting releases are Typhoon-S-8B Instruct and Typhoon-S-4B Legal Agent.
  • Adoptability: The adoptability pipeline uses SFT followed by full-logits OPD with open English instructions and a small targeted Thai dataset.SFT trains on mixed instructions, tool use, and Thai alignment; OPD uses student-generated trajectories and token-level teacher distillation.
  • Adoptability results: Typhoon-S-8B improves Thai-native chat, code-switching, Thai knowledge, and Thai retrieval QA while remaining competitive on English benchmarks.The comparison is against Qwen3-8B, with detailed results reported in Tables 6 and 7.
  • Sovereign capability: The legal-agent recipe adapts Qwen3-4B-Instruct through a single Agentic InK-GRPO stage in a controlled legal RAG environment.The environment provides semantic search and full-document read tools, while training uses trajectory-level GRPO and in-domain legal text.
  • Sovereign capability results: Typhoon-S-4B Legal Agent achieves the strongest NitiBench agentic accuracy among the 4B variants and preserves general capabilities.It surpasses GRPO-only training and GPT-5 with comparable agentic setups without evidence of severe catastrophic forgetting.

5 Conclusion

Typhoon S combines minimal post-training components to support both general instruction tuning and sovereign capability under limited resources. In Thai experiments, small-scale InK-GRPO improves legal reasoning and agentic retrieval without degrading general capabilities.

  • SFT and OPD transform base models into competitive instruction-tuned assistants.
  • Small-scale RFT with InK-GRPO improves Thai legal reasoning and agentic retrieval without degrading general capabilities.InK-GRPO is used within the sovereign-capability stage.
  • Academic-scale compute supports the reported results, indicating that post-training design can support practical sovereign LLMs.

Limitations & Future Work

The report does not investigate pre-training or mid-training because those experiments require larger resources. It focuses on Thai and leaves broader cross-language and scaling evaluations for future work.

  • Pre-training and mid-training are not explored because they require resources beyond those currently available.
  • The study focuses on Thai because the authors have the necessary data, domain expertise, and cultural and linguistic knowledge for evaluation.
  • Generalization to other languages and cultural contexts remains a future research direction.
  • Most experiments use 4×H100 GPUs, while the final Typhoon-S-8B-Instruct run uses 8×H100 GPUs.Systematic study of scaling behavior remains an important future direction.

A Hyperparameters

This section records the hyperparameter configurations used in the post-training experiments. Unless otherwise noted, settings were preliminarily tuned and held fixed across benchmarks for comparability.

  • The section summarizes hyperparameter configurations used in post-training experiments.
  • Hyperparameters were selected through preliminary tuning unless otherwise noted.
  • Settings were held fixed across benchmarks to ensure comparability.

A.1 Hyperparameter for SFT & OPD

The SFT and OPD hyperparameters are shared across benchmarks unless explicitly stated. RQ2 uses top-k = 10 distillation, while other steps use the full distribution.

  • Table 12 reports the hyperparameters used for SFT and OPD experiments.
  • These settings are shared across all benchmarks unless explicitly stated.
  • RQ2 uses top-k = 10 distillation, while other steps use the full distribution.

A.2 Hyperparameter for RFT

The RFT experiments use LLM-based judges to generate structured rewards, while InK-GRPO combines on-policy GRPO optimization with stochastic in-domain cross-entropy augmentation. The appendix documents the associated prompt templates, algorithm inputs, and experiment configurations.

  • Reward evaluation: RFT rewards come from an LLM judge that scores responses on a 0–2 scale and returns both a numerical score and an explanation.Score 0 denotes incorrect or irrelevant responses, score 1 partial responses with significant gaps, and score 2 fully correct and complete responses.
  • Accuracy evaluation: A separate LLM judge evaluates answer correctness by comparing generated responses with reference solutions without partial credit or subjective interpretation.Figure 3 contains the prompt template for this ground-truth comparison.
  • Training procedure: InK-GRPO combines on-policy GRPO with stochastic in-domain language modeling to optimize task rewards while teaching domain knowledge.The auxiliary cross-entropy objective is mixed into training with probability ρ and weight λ.
  • Training procedure: At each training step, the procedure samples prompts and K responses from the current policy, computes rewards, and stores the resulting rollout tuples in a GRPO buffer.The algorithm then shuffles the buffer and processes it through GRPO epochs and minibatches, optionally using a reference policy.
  • Training procedure: For each minibatch, a Bernoulli(ρ) decision determines whether to add λ-weighted next-token cross-entropy on an in-domain batch to the GRPO objective.Otherwise, the update uses only the GRPO objective; the appendix lists these CE and GRPO settings across the reported benchmarks.
Loading 2601.18129v1…